Continual learning with Synaptic Intelligence — Campaign report

A single shared student policy, one MLP trunk with per-task heads, trained sequentially across 15 robot manipulation tasks on a Franka arm, resisting forgetting via Synaptic Intelligence (SI) alone — no new architectures, no progressive networks, no per-task experts.

What this is, and what was attempted

Synaptic Intelligence (SI) is a continual-learning regularizer: as a network trains on a new task, it tracks how much each parameter contributed to reducing loss (an online-computed "importance" score), then penalizes moving important parameters away from where earlier tasks left them. The importance score is monotonically accumulating — it only grows as more tasks are added, never resets — which becomes a central character in this report.

The setup: 15 manipulation tasks (push, pull, lift, throw, rotate, open/close, drag — see the demonstrations below), each with its own RL teacher policy already trained and certified to >95% standalone success. A single shared MLP student (one trunk, width 4096→2048→1024, with separate per-task output heads only for the final action layer) is trained sequentially, one task at a time, via behavior-cloning (Gaussian-KL divergence) against each frozen teacher — not via RL itself. SI is the only continual-learning mechanism used, by explicit design choice: no progressive networks, no expert routing, no per-task backbones. The goal set for this campaign was >95% average success across all 15 tasks, evaluated once at the very end of training, after every task's own turn.

This report picks up once a full pipeline (teacher extraction, distillation, SI, evaluation) was already working; it covers the campaign to push that pipeline's result from a first-pass 79.5% toward the 95% target — and the parallel, ultimately abandoned attempt to instead redesign the network into a fully-shared architecture with no per-task heads at all. The confirmed final result: 87.6% average success (3-seed mean), a real gain of nearly 10 points over the first validated result, but short of the 95% target.

Baseline, for reference

Every result in this report is measured the same way: 3 independent seeds, all 15 tasks trained in sequence, final average success evaluated once at the end (not a mid-training reading). The very first validated configuration — si-coeff=1.0, a value carried over un-tuned from earlier, much smaller (4- and 6-task) pilot runs, tasks trained in a fixed random order — is the fixed reference point every later number below is measured against.

How this campaign was run

The standing instructions for this phase: use SI as the only continual-learning mechanism (no progressive nets, no expert routing); prefer maximum parameter sharing (task-embedding style ideas over per-task backbones); skip intermediate validation scales and go straight from smoke tests to full 15-task runs; and, once a numeric target was set (>95% average success), keep iterating autonomously without pausing for per-experiment sign-off — launch a run, let it complete (each full 15-task/3-seed validation takes roughly 3–4 hours), read the result, decide the next experiment, and continue, checking in with brief status updates rather than after every step.

One discipline enforced throughout, learned the hard way partway through: never trust a single-seed result as a final answer. Several promising-looking single-seed hyperparameter screens were run cheaply first to shortlist candidates, but nothing was reported as an improvement until confirmed at the full 3-seed scale — the published si-coeff=24 result was first spotted as a single-seed screen at 82% (misleadingly close to a value that later confirmed lower, 87.6% at 3 seeds), and a later single-seed lead (si-coeff=20 at 88.8%) was intentionally left unvalidated and un-published when the campaign was called to a stop, rather than reported as a result.

Compute footprint

GPU-hours are wall-clock per training job (launch timestamp to its own completion), summed across all logged jobs — the same methodology as counting jobs run one-per-GPU concurrently on a shared cluster.

Every configuration tried, ranked

Every bar below is a 3-seed-validated mean (full 15-task sequence, final evaluation) — not a single-seed screen or a mid-training reading. Green bars beat the baseline; red bars regressed against whatever they were compared to at the time. Sorted by final average success.

The pattern that stands out: the two levers that actually moved the needle — si-coeff tuning and task reordering — are both structural/hyperparameter changes with zero new mechanisms. Every attempt to reweight the distillation loss itself (delta_action, 4 separate configurations) regressed relative to plain uniform-KL SI at the matching si-coeff, and the fully-shared architecture (explored as the "maximum sharing" design direction before this campaign, documented separately) landed far below even the untuned baseline.

The PushCuboid story

One task, PushCuboid, dominates the second half of this campaign. It is worth reading in detail because the investigation's own diagnosis of it turned out to be wrong — a useful cautionary tale in its own right.

Seven levers, seven failures

PushCuboid resisted every hyperparameter tried: si-coeff swept across a 128× range (0.5 to 64), four distinct delta_action loss-reweighting configurations, up to 4× its normal training-epoch budget, a 10× learning-rate range, and batch sizes up to 8× larger — stuck at 9–23% success through every single one.

A compelling, well-corroborated, wrong diagnosis

Reading PushCuboid's teacher training data directly found a striking anomaly: its target actions range from −30 to +47, an 18× wider spread than solved tasks (which sit in roughly ±2.5) and 4× wider than the next-hardest task. Cross-checking the original RL teacher's own training history found independent corroboration: PushCuboid was the only one of 24 RL-trained teachers whose training log records a numerical failure from exploding action variance, requiring a dedicated stability fix before it could be certified. Two independent signals, both pointing the same way — this looked like a genuine data/teacher-quality ceiling, not a tuning problem.

The fix that actually worked

Moving PushCuboid from position 10 to position 2 in the training order — so it faces importance accumulated from only one prior task instead of nine — fixed it almost entirely: success rose from the 9–23% range to 47–55%, and held there durably through all 13 subsequent tasks training afterward. No hyperparameter was changed. The same idea, applied to the next-hardest task (ThrowToBin, which has its own, milder version of the same wide-action-target property), produced a second clean win.

The lesson: when a task resists every hyperparameter lever tried under a continual-learning regularizer, test it standalone before concluding it is a data-quality problem. A well-corroborated diagnostic can still be measuring the wrong bottleneck if "can this survive its position in the training sequence" was never isolated from "can this be learned at all."

The full campaign journey

J1 — A parallel detour: the fully-shared architecture

Before this campaign's hyperparameter work began, a separate design direction was explored in parallel per an early "maximum sharing" instruction: a fully-shared network with a task-embedding and FiLM conditioning instead of separate per-task output heads. Six independent fix mechanisms were tried — FiLM conditioning at three different depths in the network, an si-coeff sweep from 0.1 to 10, extra capacity, and two flavors of per-task LoRA correction — and none closed the gap to the per-task-head design, landing at 34.2% best case. The diagnosed root cause: SI's monotonically-accumulating penalty, under 100% weight sharing, causes the shared backbone to drift across a 15-task sequence in a way no per-task correction mechanism (conditioning-based or additive) can compensate for, since the backbone itself is a moving target. This direction was set aside; every result in this report uses the per-task-head design.

J2 — Distillation-loss reweighting: four attempts, zero wins

A technique from earlier work on a different (smaller-scale) setup — up-weighting the per-sample distillation loss by how much the teacher's action changes step-to-step, on the theory that plain average-KL loss is blind to which few steps in an episode actually decide success — was retried here four times, at different scopes and si-coeff strengths: applied to all 15 tasks, applied to just the three hardest, applied to the two hardest on top of a much stronger si-coeff, and applied in isolation to PushCuboid alone. Every single configuration regressed relative to plain uniform-KL loss at the matching si-coeff. The technique that helped elsewhere didn't transfer to this setup, independent of how it was scoped.

J3 — The si-coeff sweep nobody had done

The si-coeff value in use (1.0) had never actually been tuned at the full 15-task scale — it was carried over from much smaller, earlier pilot runs (4 and 6 tasks). A sweep on a cheap 8-task subset found a clean trend that kept improving well past the range anyone had checked before, peaking at si-coeff=32 — 32× the value in use. Validated at full scale: 79.5% → 82.8%.

J4 — Epoch budget: the cheap fix that worked once

At si=32, two tasks (PushCuboid and ThrowToBin) finished their turn visibly under-converged (distillation loss an order of magnitude higher than solved tasks). Simply doubling their epoch budget: 82.8% → 83.8%, fixing ThrowToBin outright. PushCuboid did not move — the beginning of its own, longer story above.

J5 — The reordering discovery, and its limits

Following the PushCuboid isolation-test reversal (above), moving it to position 2 in the training order produced the campaign's biggest single jump: 83.8% → 85.6%. Extending the same idea to ThrowToBin (moved to position 3): 85.6% → 86.6%. Two follow-up attempts to extend the pattern to a third task, RotateValve, both looked promising for RotateValve itself but net-regressed the overall average (85.0% and 83.9%, both below 86.6%) — in one case because it displaced PushCuboid from its now load-bearing position 2 slot, in the other because RotateValve's own training dynamics (much larger-magnitude teacher returns) disrupted the shared trunk more than a small, stable task did when given the very first, completely unprotected training slot. Task reordering turned out to have real structure — only about two tasks can occupy the protected early positions before returns diminish or reverse, and which task sits first matters as much as how early it is — not a simple "earlier is always better" rule.

J6 — Re-tuning si-coeff for the new order

si-coeff=32 had been tuned for the original task order, and never revisited after reordering changed which tasks needed protection and for how long. A fresh screen found si-coeff=24 edging out 32 on the new order; validated at 3 seeds: 86.6% → 87.6%, with noticeably tighter run-to-run variance (±0.4 vs ±0.8 points) — a smaller gain than the reordering discovery itself, but confirmation that a hyperparameter tuned for one structural configuration should be rechecked after another one changes.

J7 — A late, unvalidated lead, and the stop

A finer screen around si-coeff=24 found si-coeff=20 scoring higher still on a single seed (88.8%) — a promising lead, but per the discipline of never reporting a single-seed reading as a result, it was left unvalidated when the campaign was called to a stop before a 3-seed confirmation run could be launched. It is recorded here, not published, as the natural next thing to try.

Key technical concepts

Task demonstrations

Videos below show the confirmed campaign-best run (seed 0, final average 88.1%) after training on all 15 tasks in the final sequence order. Per-task rate is that run's own final evaluation, after the whole sequence finished — a success example where the run achieved one, otherwise its best-return attempt. RotateValve and ThrowToBin remain the hardest tasks; see the journey above for why.

Final verdict

The remaining gap is no longer concentrated in one or two blocker tasks the way it was mid-campaign — RotateValve (67.2%) is the single largest gap, but closing it plus the broad set of tasks sitting in the 84–99% range would both be needed to approach 95%. Two concrete, unvalidated leads remain open: a single-seed screen suggesting si-coeff=20 may beat the published si-coeff=24 (never confirmed at 3 seeds), and a finer-grained reordering placement for RotateValve (position 3, between the two already-fixed tasks, rather than fully swapping into either of their slots) that was never tried. Beyond that, the levers explored within this campaign's constraints — hyperparameters and task ordering, SI as the only regularizer, one shared trunk with per-task heads — appear close to exhausted; closing the rest of the gap most likely needs either a lower-variance retrained teacher for the hardest remaining tasks, or the architecture-level changes (dedicated per-task capacity, a different conditioning mechanism) intentionally deferred to a later phase.