Abstract
TL;DR. The RSSM posterior encodes obstacles well (AUC 0.85), but its imagined rollout states progressively lose this signal (AUC 0.71 at 15 steps; 57% logit-separation decay). The planner's own ensemble-based epistemic term doesn't flag the loss (AUROC 0.52 — chance), making the failure an unknown unknown at planning time.
Model-based planners that roll out imagined futures assume the transition model faithfully propagates task-relevant signals — a property rarely measured directly. In a single-seed case study of an RSSM world model with active-inference planning in CARLA, we identify a posterior-prior gap: posterior (observation-filtered) states retain obstacle information that imagined rollout states progressively lose. A supervised obstacle probe achieves AUC 0.85 on posterior states, but the same probe applied to imagined prior states shows monotonic decay (AUC 0.85 → 0.71 over 15 steps; logit separation drops 57%). Because the probe is identical, the decay isolates the transition model — not the representation — as the information bottleneck. Obstacle avoidance drops to 0% without privileged runtime coordinates, and the planner's own ensemble-based epistemic term does not flag the loss — ensemble disagreement yields AUROC 0.52 as an obstacle classifier (chance level), making the failure an unknown unknown at planning time.
As a secondary contribution, we show that task-conditioned weighting of EFE components over the same world model yields lane keeping (81.2 ± 1.8% route completion) and obstacle avoidance (28/28 avoided, using privileged coordinates) in CARLA, with runtime weight modulation enabling post-evasion centering.
The Posterior-Prior Gap
Method at a Glance
An RSSM world model is trained on 96K frames of expert driving in CARLA Towns 04 and 06. A deep active inference agent plans by minimizing Expected Free Energy (EFE) over iCEM-sampled trajectories (500 samples, 50 elites, 5 iterations, horizon 12–15). The same world-model checkpoint serves lane keeping (Town04) and obstacle avoidance (Town06) — only the EFE component weights change between tasks.
The diagnostic is a single auxiliary head (MLP 320 → 128 → 1, BCE loss) trained to predict obstacle presence from the 320-D RSSM feature. The same frozen head is then applied to (a) the filtered posterior and (b) imagined prior states at depths k=1…15. The decay across depth isolates the transition model — not the representation — as the bottleneck.
Key Results
| Task / Probe | Metric | Result |
|---|---|---|
| Lane keeping (Town04, moderate curves) | Route completion | 81.2 ± 1.8% |
| Obstacle avoidance (Town06, privileged coords) | Obstacles avoided | 28 / 28 (100%) |
| Obstacle avoidance (no privileged coords) | Avoidance rate | 0% |
| Posterior obstacle probe | AUC | 0.85 |
| Prior obstacle probe, k = 15 | AUC | 0.71 |
| Ensemble disagreement as obstacle classifier | AUROC | 0.52 (chance) |
The result frames an unknown unknown: the planner's own uncertainty signal does not separate dangerous from safe imagined trajectories. From the planner's perspective, futures that lose track of an obstacle are indistinguishable from futures that never had one.