Two gaps in machine memory
A progress note. Ten weeks of small, pre-registered experiments on memory and continual learning, most of them negative, and the experiment we are running now.
Our first research project, latent-trunk, asked whether a language model could predict the next idea rather than the next word. This note is about what happened next. The short version: the latent pilot ran, it did not win, and the question underneath it turned out to matter more. Since then the work has been about memory, specifically how a small network can keep learning after it is deployed without overwriting what it already knows.
Where we left off
The latent-trunk pilot ran in early September on a single rented GPU for under a dollar of compute. Three models of about 138 million parameters each, trained from scratch on the same web text for the same 300 steps: a plain token transformer, a chunked recurrent core, and the same recurrent core with a latent-prediction objective and collapse controls. The transformer won on validation loss, 6.67 nats against 7.58 for the recurrent core and 7.53 with the latent objective. The latent objective helped its own architecture by under one percent and the recurrent models trained faster per step, but nothing in a run that short establishes the thesis, and we said so in the report.
We chose not to scale it. The reason was not the number. It was that a better language model was never the point. What we actually want is a system that remembers specific things it has seen, revises them, and keeps learning, and token loss on web text does not measure any of that. So the text pilot was filed as a diagnostic and the research moved to the thing it was supposed to serve.
How we work
Everything below was done with tiny models on a laptop CPU or a single rented GPU, for a total of roughly sixty dollars of cloud compute since September. Small on purpose: the questions are about mechanism, and a mechanism that only appears at scale is not one we can study.
Every experiment is written up as a protocol before it runs, with the pass and fail thresholds declared in advance, then frozen and hashed. If it fails, it fails, and the result is kept. We do not retune a failed gate until it passes. Each design and each result is reviewed by a frontier model in a separate context, a GPT model and a Claude model, which can reproduce the numbers and argue with the interpretation but cannot turn a fail into a pass. Several of the results below were corrected, or stopped, on that advice. This is not independent peer review and we do not pretend it is. It is the cheapest way we have found to keep ourselves honest.
What we found
We started with a known result, to make sure the testbed could see it. A small ResNet trained on five two-class slices of CIFAR-10 in sequence, with no rehearsal, ends at 20% accuracy overall and 0% on every class it saw before the last pair. Replaying stored examples lifts that to 57%, and a stronger replay method to 72%. This is catastrophic forgetting, exactly as the literature describes it. Nothing new here, but it is the problem in one sentence: a network trained on a stream forgets almost everything that is not in the current batch.
The main testbed since then has been a symbolic world of recorded agent histories, where a visual cue predicts an outcome within a ruleset, and a frame identifies which ruleset is in play. It lets us ask memory questions precisely: did the model retain this rule, across how many episodes, and what did it cost?
- Recurrent state alone does not bind rules across episodes. A small GRU with state carried between episodes helped prediction, but on rulesets it had never seen it either memorised per-ruleset tables or could not bind at all, even when handed the answer as input.
- An explicit record store with learned addressing works, in context. A content-addressed store of observed records, read by a learned two-hop address, scored 0.86 to 0.89 nats per query on sixteen never-trained rulesets against a 0.80 ceiling for a hand-supplied address, and recalled rule outcomes hundreds of steps and several episodes back. This is the one clean positive in the line. It is qualified: it depends on a supplied write trigger, a record-type flag and an observable episode boundary, and structurally it is a learned version of the supplied address.
- A learned write policy did not learn. A write gate trained only through the prediction loss was anti-selective and unstable, worse than writing at random. Deciding what to store needs its own signal.
- Weights retain something, but not a table. After training on three histories of a ruleset, prediction on a fourth history improved by 0.7 to 1.1 nats over a never-trained ruleset of matched difficulty. But on the same events the weights stayed 2 to 3 nats worse than an explicit per-ruleset table. What formed is a weight-mediated familiarity effect, not a lookup.
- It does not survive the next block. Trained on twelve rulesets in a block and then four more blocks without rehearsal, that effect deteriorated by 1.5 to 2.3 nats. Replaying one earlier batch in four cut the loss to about 0.6 nats. Standard forgetting, standard rehearsal benefit.
- Consolidating compact records into weights is real but weak. Replaying the store's own two-byte cue-to-outcome records, about a thousandth of the raw data, consolidated 0.69 nats into the weights, where the same number of raw bytes did nothing. But raw replay at matched updates protected 1.90 nats, so compressing experience to associations lost most of the benefit. A weight-penalty method protected strongly and paid for it in plasticity.
- Forcing the weights to answer did not make them learn. When we removed the store's help at training time and added an auxiliary loss so the weights had to hold the associations themselves, association learning was 0.53 nats with an interval that includes zero, and online performance got worse.
- A plain feed-forward predictor failed every frozen test. The simplest version, a cue-and-frame predictor trained block by block on a CPU, reached 51% exact accuracy with confident wrong answers, a single miss costing up to 27.6 nats, and lost about 20 nats of competence on an old block after the next one. Replaying records from the old block gave almost exactly that back.
Across every phase, the only thing that ever held a block was replay.
We did not set out to collect negative results, but we are glad to have them, because together they say something specific. Every single-mechanism attempt to write knowledge into shared weights failed on a different property: interference, calibration, or addressing versus prediction. And in every one of them, we chose what got written. The model never did.
Two gaps
That is the reading we have settled on. There are two problems here that the field usually treats as one.
The first is the write: adding knowledge to shared weights without destroying what is there. Replay, weight penalties, gradient projection and sparsity with task masks all partly work, and all of them are ways of limiting a global update. The second is selection: out of a stream of experience, what gets consolidated and when. This one is almost unworked. Pretraining consolidates everything. Fine-tuning consolidates what people curated. Retrieval consolidates nothing. Replay rehearses a buffer picked by recency or at random.
We think they are one problem, and that is the bet the rest of this line rests on. Selection needs a familiarity signal, a way for the system to know what it already contains. A dense network updated globally does not have one, which is why our feed-forward predictor was so confidently wrong. A local write with a familiarity measure gives you a selection rule almost for free: write what is new and useful, skip what is known or unreliable. Each half looks impossible from inside a dense network, which is why they get attacked separately and neither gets filled.
This is not an argument against gradient descent. It is an argument against global updates. The gradient learned every mapping we asked it to; the interference came from every example moving every parameter. The target is a sparse, high-dimensional geometry in which one experience's update touches almost nothing else, related experiences share, and familiarity falls out of the same geometry as the overlap between a query's active units and what has been written.
Running now
The current experiment is the designed, controlled version of that idea, and it is deliberately not the final one. We take the same predictor that failed above and replace its first hidden layer with a wide winner-takes-all layer, 8,192 units of which only k stay active, and sweep k from 8 to 512 against a capacity-matched dense control. The frozen gates ask whether acquisition, retention and generalisation to held-out rulesets hold at each k, measure interference directly as how far a new block's updates move the parameters an old block used, and test whether the fraction of a query's active units that received a verified write separates familiar rulesets from unfamiliar ones, against frequency and random-feature controls. Retention bought by not learning counts as a fail.
The protocol went through three design reviews and four freeze checks before it was locked. It runs on a single consumer GPU. There are no results yet, and the prediction is falsifiable: forgetting should fall as the active set shrinks, competence should fall too, and either there is a window where both hold or there is not. We will publish whichever it is.
After that
A hand-designed sparse layer can show that the mechanism exists. It cannot be the claim. The phase after this one has to learn the write rule and the familiarity signal from experience, and any claim of novelty is made on the learned version. Then real data: a class-incremental vision stream against published baselines on identical splits, and a long-term memory benchmark for a small language model with a store, retrieval alone against retrieval plus consolidation. And eventually a reward-driven environment with long-horizon memory dependencies, because what to remember is only really a question when remembering has consequences.
None of this is a capability claim. Results in a symbolic toy world are results about a mechanism on structure we supplied. The closest prior work, meta-learned sparse representations, random sparse coding and interference-aware replay selection, each gets locality from a frozen or random representation and does selection, if at all, on a rehearsal buffer. The distinction we are chasing is a representation that stays plastic, reads familiarity from its own geometry, and selects at write time with no buffer. Whether that exists is what the next few months are for.