Continual learning treats any drop in performance on previously seen data as forgetting, a convention inherited from settings where the prediction target is stationary. A world model's target is the environment, which changes, so knowledge that was accurate when learned can later become false, and revising it is correct behavior rather than a defect. World models also encode structural knowledge, such as physical laws and object permanence, that must never be degraded.
Retaining knowledge that has become false is a failure mode of continual world models, distinct from catastrophic forgetting, and current benchmarks exclude it by construction.
Organize world-model knowledge by how long it stays true, τ, and protect knowledge more strongly the longer it stays true.
An evaluation protocol that measures invariant retention and revision latency separately, instead of one forgetting score.
A household robot has learned where the furniture is. Then someone moves the chair. Current metrics cannot tell these three apart.
The loss of knowledge that is still true.
The removal of knowledge that is still true but unwanted, for instance for privacy.
The retention of knowledge that has become false.
Changing ground truth is well studied in concept drift and in the temporal factuality of language models, but it has not been formulated for world models. World models are different: alongside facts that change, they encode physical, geometric and causal structure, such as that unsupported objects fall and hidden objects continue to exist. That structure should survive every update, and no single label can check it.
The invariance timescale τ is the expected time until the environment makes a piece of knowledge false. It is a property of the environment, not of the model.
Refined, never degraded. A model's belief about an invariant may be wrong and should be refined as evidence accumulates, but it must never get worse. A Kalman filter makes this trade-off: its update rate falls as τ grows and, for τ = ∞, shrinks toward zero without reaching it.
Standard pipelines do not check whether invariants survived an update. EWC protects parameters by their importance for earlier tasks, but importance is not the same as persistence: a parameter can encode a fact the environment has since invalidated, and EWC then protects it most strongly.
A replay buffer that stores the old layout of a room keeps reasserting it after the room has changed. Methods that replace outdated exemplars or evict stale transitions address this; protecting invariants has no counterpart in either setting.
Three robots are tested on their original test data after the chair has moved. Switch between how standard metrics score them and what differential retention reports.
Every correct revision counts as forgetting, so the metric favors models that revise as little as possible, and the frozen model, the very failure continual adaptation is meant to remedy, scores best.
Refreshing the labels is not enough either: a model can fit the new room layout while degrading object permanence, because ordinary test data rarely exercises it. That is why invariants are probed directly.
Two quantities, reported separately, because a single score would hide which failure occurred.
A fixed set of probes, small tests that each check one invariant, drawn from the τ = ∞ stratum and held out from the adaptation data. They are re-run after every adaptation round, because a single evaluation at the end cannot show which update damaged an invariant. Unchanged lower-stratum facts, such as the table, are re-tested alongside.
Hide a ball behind the couch. Does the model still predict the ball is there?
For each fact the environment changes, how many observations the model needs before its predictions favor the new outcome over the old one on held-out queries. Facts that are never revised are counted separately. This generalizes the LoCA regret from a single local change to every changed fact.
How many times must the robot see the chair at the table?
Neither quantity is informative on its own. A frozen model loses no invariants but never revises. A model that adapts by overwriting revises quickly but damages its invariants.
Together, the two measurements place a model in one of four quadrants. Forgetting metrics alone reward high retention and slow revision, and so favor the frozen model.
To our knowledge, such probes have not been applied as a regression suite across an adaptation stream.
Each kind of invariant needs its own probe. For action-conditioned or latent world models, a probe can read out the model's own prediction error.
The strata are a starting proposal with domain-dependent boundaries. In benchmarks τ is set by design; estimating it reliably in deployment, especially for slowly changing regularities, remains open.
Mature probes exist mainly for physical invariants in generative video models; probes for geometric and causal structure still need to be developed and validated.
A door that is open on some days and closed on others should be suspended and later reinstated rather than overwritten.
If invariants are entangled with other knowledge inside a learned world model, protecting them separately may require architectural support.
@inproceedings{anand2026whatshould,
title = {What Should World Models Forget? Stratified Retention for Continual Adaptation},
author = {Anand, Nishit and Duraiswami, Ramani and Manocha, Dinesh},
booktitle = {NeurIPS 2026 Workshop on Continual World Models},
year = {2026},
eprint = {2610.03713},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.03713}
}