THE ROUTES HOLD

Run a Transformer on analog compute-in-memory and the stored weights wobble — the attention routes (who listens to whom) bend, worst in deep layers. Classic noise-aware training (NAT) injects noise and grades only the final answers — it can stay confident while its routes point at the wrong tokens. ABNAT grades the routes too: the noisy student must keep the clean teacher's headwise routes, not just its answers. (After Yao et al., arXiv:2610.22065 — labelled toy, invented noise, not their data. Paper: up to +11.28% noise-degraded accuracy, attention deviation cut up to 85.77%.)

analog noise σ 0.70
Probes ask layer-3 queries whose answer lives at the end of a route. Easy probes point nearby; hard probes point across the sentence — only an intact route reaches them. NAT's routes below are the raw noisy routes: answer-only training never touches them.
route deviation vs teacher (mean L1, lower = routes held) — L1 – · L2 – · L3 –
probes — easy – · hard – · confidence when wrong –
toy model · canvas 2d · invented CIM noise, fixed seed
gold = clean teacher routes · violet = student routes · ring = probe target