Every Test Changed the Scene and Kept the Chair Red

Four outdoor sites. Six recurrent architectures, trained on identical images through an identical convolutional front end. Eight separate experiments, logged in sequence, each asking whether a quadrotor that has learned to fly to an object in one place can still find it somewhere it has never flown. In every one of those eight experiments, the object is the same red folding camping chair. The chair changes location. It changes background six times over, from bare autumn ground to a foot of snow to a summer lawn to a brick patio ringed with other furniture. It never once changes color.

That chair is the entire empirical anchor beneath a striking published claim. Makram Chahine, Ramin Hasani, Patrick Kao, Aaron Ray, Ryan Shubert, Mathias Lechner, Alexander Amini, and Daniela Rus — all at MIT at the time — reported in Science Robotics that a class of brain-inspired continuous-time architectures, liquid neural networks, generalize to drastic real-world scenery shifts where five other state-of-the-art recurrent architectures degrade or fail outright, and that “this level of robustness in decision-making is exclusive to liquid networks, both in their differential equation and closed-form representations” ^[1]. The paper’s framing goes further than a robustness result. Liquid networks, it says, “are causal and adapt to changing conditions,” and the whole research program rests on one conditional stated plainly in its introduction: “if the causal structure of a given task is captured by a neural model from expert data, then the model can perform robustly even OOD” ^[1].

Read forward, that sentence describes what causal capture would produce. Read backward — from eight passed tests to a causal certificate — it needs something the sentence does not supply on its own: proof that the tests could have told the difference between a model that understood the task and a model that had simply found one very stable, very portable visual signature and ridden it through every shift the experimenters thought to apply. Across all eight entries in the paper’s own log, the chair’s paint is the one variable nobody manipulates. This article is about what that omission costs the causal claim, and what it does not cost the comparative result underneath it.

What Four Sites and Eight Log Entries Actually Show

Give the paper its full strength before touching the gap, because the strength is real and well-documented. The task is fly-to-target: a quadrotor is placed roughly 10 meters from a target and must center it in frame and hold station within 2 meters for 10 seconds, learning entirely from offline expert demonstrations with no bounding boxes, no object labels, and no hand-built state representation — pixels in, velocity commands out ^[1]. Six architectures share an identical convolutional head and an identical data-augmentation pipeline and differ only in the recurrent core that follows it: LSTM, GRU-ODE, ODE-RNN, a temporal convolutional network (TCN), a neural circuit policy (NCP, a sparsely wired liquid time-constant network), and a closed-form continuous-time network (CfC) ^{[1, 3, 4]}.

The paper’s own results list runs to eight items, worth stating precisely because its own text uses two different counts in adjacent sections: “1) Fly-to-target tasks… 2) Range test… 3) Stress test… 4) Attention profile of networks… 5) Target rotation and occlusion… 6) Hiking with adversaries… 7) Triangular (multistep) loop between objects… 8) Dynamic target tracking,” but the closed-loop-testing section that follows states “All seven tasks were evaluated in closed loop with environments by running the neural networks onboard the drone” ^[1]. The arithmetic resolves cleanly once item four is set apart: attention profiling is an offline saliency computation over recorded frames, not a flight. Seven of the eight entries are closed-loop control tests flown on real hardware; the eighth is a qualitative interpretability pass over the same convolutional features. Both kinds of evidence matter here, but they are not the same kind, and the paper’s own casual variation between “eight” and “seven” is worth naming rather than smoothing over.

On the seven flown tests, the comparative result is not close. Across four sites — Training Woods, an Alternative Woods location, an Urban Lawn, and a brick Urban Patio, at N=40 trials per architecture per site — liquid architectures win or tie at every site, and their margin grows exactly where the shift is hardest: on Urban Lawn, NCP reaches 57.5% and CfC reaches 90%, against 62.5% for LSTM, 32.5% for GRU-ODE, 17.5% for ODE-RNN, and 0% for TCN ^[1]. A range test pushed the same Training Woods policies out to distances never seen in training: at 20 meters CfC held 90% and NCP held 50% while, in the paper’s own words, “an LSTM network… lost the target in every single attempt at both these distances, leading to a 0% success rate. Only ODE-RNN managed to achieve a single success at 20 m among the nonliquid networks” ^[1]. Offline stress tests that perturbed brightness, contrast, saturation, and injected pixel noise into recorded flight frames found CfC’s output trajectories deviated least from their unperturbed originals across all four perturbation types, with NCP close behind and LSTM the best of the non-liquid group. Rotating and partially occluding the target cut CfC’s success by roughly 10% and 30% respectively while cutting GRU-ODE’s by closer to two-thirds. On an adversarial hiking course with intense glare and scattered distractor objects, CfC completed 14 of 20 runs, “more than twice” its closest non-liquid competitor’s rate, while TCN and GRU-ODE barely detected the first target at all. On a triangular multi-target loop, a CfC policy navigated four full laps and 12 checkpoints without external intervention; on a moving target, the same architecture averaged 8.8 checkpoints reached against an LSTM average of 5.8 steps before losing the mark ^[1].

The qualitative eighth entry adds a different kind of evidence to the same conclusion. VisualBackProp saliency maps computed over the shared convolutional head show liquid-network attention concentrated on the target from the first frame of a flight, while other architectures visibly wander: the paper’s own Figure 3 walks through one case where an LSTM policy “loses its target and gets confused by an adversary,” and a second where the same architecture “gets confused by the window reflections” of an unrelated building, in both cases while a liquid policy running the identical scene stays locked on the chair ^[1]. Rotation and occlusion tests push the point further apart by architecture: GRU-ODE’s success collapsed almost entirely once the target was rotated 90° or 180° from its training orientation, NCP and ODE-RNN each lost roughly forty percent of their nominal success rate at 90°, and CfC’s success stayed within about ten percentage points of its upright baseline at both rotation angles and lost only about thirty percent under partial occlusion ^[1].

None of this is a single lucky run. It is a comparative result assembled from real closed-loop deployment on a DJI M300 quadrotor with an onboard Jetson TX2 running GPU-accelerated inference, repeated across seven distinct test protocols and hundreds of individual trials, with training data, augmentation, and the convolutional backbone held fixed across every architecture compared ^[1]. Architecture choice, at fixed everything else, measurably changes how well an imitation-learned flight policy survives leaving the distribution it was trained on. That result does not depend on anything argued in the rest of this piece.

A Claim That Needs Its Converse to Be True

The paper does not stop at the comparative result, and it is careful, in one place, about exactly how far it means to go. In its discussion of what the results mean, it writes: “The primary conceptual motivation of our work was not causality in the abstract; it was instead task understanding, that is, to evaluate whether a neural model understands the task given from high-dimensional unlabeled offline data” ^[1]. That is a real and useful hedge, and it deserves to be quoted before anything is said against the paper — the authors themselves flag that “causal” is doing narrower work than the word usually implies, and that narrower reading is defensible on its own terms.

But the same section does not stay inside that hedge. It continues: “From the causality angle, liquid neural networks were DCMs: This was theoretically shown in Vorbach et al.” — referring to a 2021 NeurIPS paper by four of this paper’s own co-authors showing that the learning system described by a liquid time-constant network reduces to a Dynamic Causal Model, in Karl Friston’s formal sense, provided the network’s learned nonlinearity is monotonically increasing, bounded, and Lipschitz continuous ^{[2, 9]}. And the paper’s own abstract states, without the earlier hedge attached, that liquid networks “are causal and adapt to changing conditions,” while MIT’s own press description of the same work says the architectures “capture the causal structure of tasks from high-dimensional, unstructured data” and “extract crucial aspects of a task… and ignore irrelevant features” ^{[1, 11]}.

Two different claims are living inside one paper, and only one of them is fully licensed by what was actually tested. The Vorbach result is a property of a model class: an LTC network, described by the ordinary differential equation the paper reproduces as

reduces to a DCM whenever its learned function $f$ satisfies three stated analytic conditions ^{[1, 2]}. That is a real theorem about the architecture’s dynamical form, established independently of this drone task, and nothing in this article disputes it. What it does not supply, on its own, is a certificate that the specific $f$ this paper’s networks actually learned from confounded pixel data recovers the task’s true cause — the target’s identity and position — rather than a merely convenient, shift-stable correlate of it. Judea Pearl’s formal treatment of causal inference is explicit that a causal claim is licensed by an intervention: showing that changing one variable while holding others fixed changes the outcome in the way the causal story predicts, not by showing that an outcome remains stable under variables that were never the confound in question ^[8]. A theorem about the architecture’s capacity to represent interventions under training is not the same claim as a demonstration that this trained network used that capacity to separate the chair’s identity from the chair’s color, and the paper’s own evidence, read at the level of what was actually intervened on across all eight entries in its log, never performs the second demonstration.

The One Color No Test Ever Reassigns

Walk the paper’s own log again, this time asking one question of each entry: what, exactly, did this test change? Fly-to-target changes site and season. Range extends distance. The stress tests change brightness, contrast, saturation, and noise. Rotation and occlusion change the target’s orientation and visibility, not its identity. The hiking task changes the environment and adds “adversarial objects… including a blue bin, a yellow chair, and two red chairs” as extra clutter around the true target ^[1]. The triangular loop and the dynamic-tracking task change the target’s position and motion. In every one of these seven flown entries, the object the network is meant to reach is the same weathered red camping chair it was trained on, unrepainted.

The nearest the paper comes to testing the color hypothesis directly is its final and hardest site, the Urban Patio, and it is worth quoting exactly how that test was built: “we added an extra layer of complexity to this experiment by positioning a number of other chairs in the frame of different colors (including red) and sizes” ^[1]. This is a real, useful manipulation — it puts a same-colored decoy in the same frame as the true target — and it is exactly the site where both winning architectures post their lowest scores of the four: NCP falls to 52.5% and CfC falls to 67.5%, each its weakest showing anywhere in Table 1 ^[1]. That dip is consistent with a color-keyed detector under new pressure from a look-alike decoy. It is also consistent with an architecture correctly identifying the true target under harder visual clutter, exactly as the causal reading would predict. The Urban Patio result alone cannot adjudicate between these two readings, and it should not be over-read in either direction — which is precisely the point. The one experiment built to probe the color hypothesis stresses the hypothesis without resolving it, because it never performs the one manipulation that would resolve it: repainting the true target itself so that its position, not its hue, is the only thing a correct answer can still track.

No test anywhere in the released record — not the eight logged experiments, not the four supplementary stress-test perturbations, not the rotation, occlusion, hiking, loop, or tracking variants — recolors the true target while holding a same-colored decoy in frame. That is not a criticism of any single test in isolation; each one does what it was built to do, and does it honestly. It is an observation about what the set of eight tests, taken together, was never in a position to show, because the one variable a causal claim about “the task’s structure” would most need to vary — which object in the frame counts as the goal, independent of what color that object happens to be — sits fixed across every entry in the log.

A Controller That Was Already Riding Along in the Training Labels

A critique needs a rival account that fits the same numbers without needing the disputed assumption, and one is available directly from the paper’s own architecture and its own methods section — no new theory required.

Start with capacity. The paper describes NCP, its most consistent top performer, as “a sparse network configuration composed of fewer than two dozen LTC neurons,” feeding on a shared 128-dimensional convolutional feature vector produced by the same CNN backbone every architecture uses ^[1]. A recurrent core with that few internal states cannot afford to carry much of a busy forest or brick patio’s incidental detail forward through time; it has to compress the incoming feature stream down to whatever handful of dimensions best explains the training labels, at every site, in every season. The liquid equation above adds a second compressive pressure on top of raw capacity: the term $1/\tau$ acts as a per-neuron leak rate, so each unit’s state decays back toward a baseline unless a new input keeps refreshing it — a state-dependent low-pass filter over the incoming feature stream, in continuous time ^{[1, 4]}. A filter tuned to damp high-frequency nuisance variation — flickering foliage, shifting cloud cover, a differently textured wall — while passing whatever stays consistent frame to frame is exactly the kind of representation a small, sparse, continuous-time core would be pushed toward by gradient descent on this task, whether or not the underlying feature it locks onto happens to be causal in Pearl’s sense or merely durable. This is not a fringe possibility for vision systems generally: models trained end to end are well documented to default to whichever available signal is easiest and most stable to fit, including background and context cues that happen to be spuriously but reliably correlated with the label, rather than the object property a designer intended ^{[6, 7]}. A consistently red, consistently sized, consistently centered object near the middle of frame is about as easy and stable a signal as an outdoor vision task offers.

The paper’s own data-augmentation description then supplies something stronger than a plausible mechanism — a documented fact about where a large share of the “expert” supervision actually came from. To expand the hand-flown training set, the authors built synthetic sequences by repeatedly cropping a single still frame containing the target, sliding and rescaling the crop window over time to simulate approach, and assigning velocity labels by a fixed geometric rule: “we labeled yaw commands equal to the horizontal distance in pixels between the target and the center of the frame times 0.01 and likewise generated throttle (up-down) commands equal to the vertical offset in pixels times 0.01… pitch (forward-backward) commands proportional to the size of the cropped window” ^[1]. That is, in plain terms, a scripted proportional controller: center the known target in frame, scale forward speed to apparent size. The paper states this procedure, applied at scale, “increased the size of the training set by more than 2.5×” ^[1] — meaning a majority-or-near-majority share, by sequence count, of the commands every tested network is trained to imitate were never a human pilot’s stick inputs at all. They were generated by almost exactly the rule a scripted color-and-size tracker would also generate. A network that learns to reproduce those labels well is, on a large fraction of its own training signal, learning to reproduce a scripted centering controller — which makes the scripted controller not merely a plausible outside rival but a documented close relative of part of the supervision itself.

That motivates a concrete, minimal floor for any reanalysis: a scripted tracker with no learned parameters at all — threshold the frame for the target’s known hue, take the centroid, and convert its pixel offset and apparent size into yaw, throttle, and pitch commands by the same proportional rule the paper’s own synthetic-data generator already uses. If a controller with zero training and zero causal representation clears the paper’s eight-entry gauntlet at rates comparable to the liquid networks it is compared against, the gauntlet cannot be doing the certifying work the causal language assigns to it — regardless of how any individual trained network’s internal dynamics happen to be organized.

Defining the Test the Log Never Ran

A countermodel earns its keep only once it points at a specific, buildable measurement nobody has taken. Define, for any policy $\pi$, a Causal-Alignment Index under a cue-decorrelation intervention that repaints the true target a color other than red while placing an unmodified red decoy elsewhere in the same frame region class the paper’s own Urban Patio protocol already uses:

Pair it with a Recolor Robustness Ratio, comparing that score against the architecture’s own already-published performance on the paper’s hardest matched-appearance site:

where $s_{\mathrm{patio}}(\pi)$ is the architecture’s own Table 1 Urban Patio success rate ^[1]. An $\mathrm{RRR}(\pi)$ near 1 says the architecture tracks the target about as well when its color is no longer the give-away cue as when it was — the causal reading survives. An $\mathrm{RRR}(\pi)$ collapsing toward 0 says performance was riding on the color the whole time, and collapses precisely when that color is taken away and handed to a decoy — the shift-stable-correlate account is confirmed instead.

Nothing about specifying this test requires new hardware or new flights. The paper’s own released materials already contain everything needed to build it: trained NCP and CfC checkpoints, the full evaluation harness, and — critically — the exact script used to generate the paper’s own synthetic training sequences, preprocess/closed_loop_augmentation.py, sitting in the public repository alongside the original hand-collected flight data ^[10]. The same crop-and-relabel procedure that built the paper’s synthetic_chair training set could build a synthetic recolored-target evaluation set at essentially no marginal engineering cost, and the paper’s own offline stress-test methodology — perturb recorded frames, run them through a frozen network, integrate the output velocities, and measure where the resulting trajectory goes — is already the right tool for scoring frozen checkpoints against composited recolor-and-decoy imagery without a single new flight ^[1].

One discrepancy is worth flagging plainly before anyone spends time on this, since it costs nothing to state and could cost real time to discover mid-analysis: the paper’s own data-and-materials statement says its code and data are “openly available… under Apache 2.0 License,” but the license file actually committed to the live public repository is the GNU General Public License, version 3 ^{[1, 10]}. The code and data are live and reachable either way — this is a documentation mismatch to resolve before anyone builds a downstream tool on the assumption of the more permissive license, not a claim about anyone’s intent, and it does not bear on any result reported in the paper.

A second, cheaper diagnostic can ride alongside the flight-level test using a method the paper already trusts. The paper’s own saliency method, VisualBackProp, is what produced its Figure 3 evidence that liquid-network attention “stays locked to the target” while other architectures diffuse or drift onto adversaries ^{[1, 5]}. Run the identical saliency computation on the same recolored-target-plus-red-decoy frames: if the highest-saliency region continues to track the true target’s new color and shape rather than snapping onto the familiar red decoy, that is representation-level evidence pointing the same direction as a high Causal-Alignment Index. If saliency instead relocates onto the red decoy the moment it appears, that is representation-level evidence for the color-shortcut account, independent of whatever the drone’s trajectory ultimately does.

Two Numbers That Would Settle It

State in advance what each outcome would mean, so the test can be graded rather than argued with after the fact. To make the arithmetic concrete — using invented placeholder numbers only, to illustrate what the ratio means, not anything measured — suppose CfC’s Urban Patio baseline stayed at its published 67.5% while its measured $\mathrm{CAI}$ under the recolor intervention came back at 61%. That would give $\mathrm{RRR}(\mathrm{CfC}) \approx 0.90$: high, and hard to explain without the network actually tracking the target’s identity rather than its color. Suppose instead the same baseline of 67.5% paired with a measured $\mathrm{CAI}$ of 9%, most of those trials converging on the red decoy instead of the recolored true target. That would give $\mathrm{RRR}(\mathrm{CfC}) \approx 0.13$: a near-total collapse, exactly what a shift-stable color detector predicts once its one reliable cue is handed to the wrong object. Both numbers are fictional. They exist only to show what a high ratio and a low ratio would look like once real trials replace them; the actual test has not been run, by this article, by the paper’s own authors, or by anyone identified in a targeted search of the citing literature.

A threshold stated only in words is not yet a threshold. Matching the paper’s own design choice of $N=40$ trials per architecture per condition ^[1] keeps any comparison directly comparable to Table 1’s published rates and gives roughly ten percentage points of resolution on a proportion near the middle of the range — enough to separate the illustrative 0.90 and 0.13 ratios above by a wide margin, and enough, at that sample size, to distinguish either from a genuinely intermediate result using a standard two-proportion test against each architecture’s own matched-appearance baseline rather than an eyeballed gap.

The kill criterion follows directly from the same construction. If $\mathrm{RRR}(\pi)$ for NCP and CfC lands close to 1 — target convergence holding up once the color cue is reassigned to a decoy, at rates statistically indistinguishable from their own matched-appearance baselines — this critique fails outright, and the paper’s causal language should be read as exactly what its evidence, completed, would then support. If instead $\mathrm{RRR}(\pi)$ collapses toward 0 while a scripted, zero-training color tracker clears the same recolored-target trials at a rate no better than chance in the true target’s favor, the countermodel is confirmed and the causal claim needs the test it has been missing since publication. A third, intermediate outcome is honestly possible and worth naming rather than hidden: a partial ratio, say in the 0.3–0.6 range, would say the networks use color as one cue among several rather than the sole cue, and any drafted follow-up would need to say so plainly instead of forcing the result into either extreme.

What No Recount Changes

None of this touches the comparative engineering result this piece opened with. Architecture choice, holding data, augmentation, and the perception backbone fixed, materially changes how an imitation-learned flight policy survives scenery it has never seen — that result stands regardless of which way the recolor test eventually falls, because it was never about whether the surviving cue is causal, only about whether one architecture family finds and holds onto a shift-stable cue better than five others do. The real-hardware, closed-loop evaluation protocol across seven distinct test conditions is exemplary field robotics practice, and the released code, checkpoints, and both hand-collected and synthetic datasets — modulo the license label needing a fix — make this one of the more reproducible entries in its subfield ^{[1, 10]}. The Vorbach et al. theorem that liquid dynamics can represent a Dynamic Causal Model under stated conditions on the learned nonlinearity is a real, separate, unchallenged result about the architecture class, independent of whatever this specific drone task’s networks turned out to learn ^{[2, 9]}. And the authors’ own words — “the primary conceptual motivation of our work was not causality in the abstract; it was instead task understanding” — already concede more of the distinction this article presses than a skimmed abstract would suggest ^[1].

The people behind the paper have kept building in the same direction rather than resting on the 2023 result. Chahine remains an MIT CSAIL PhD student as of an unrelated April 2026 project on compressing state-space models during training, co-authored with several of the same collaborators ^[12]. Rus remains Director of CSAIL, honored in mid-2026 for contributions spanning soft robotics to brain-inspired AI ^[13]. And by Liquid AI’s own account of its history, four of this paper’s eight authors — Hasani, Lechner, Amini, and Rus — are the founders of the company now commercializing the liquid-network family this paper evaluates ^[14]. That fact is recorded here only as current-affiliation context, in the same spirit of transparency the original paper’s own competing-interest paragraph already modeled by naming outside affiliations for four of its eight authors at the time of publication ^[1] — never as an implication about the paper’s results, which were produced and reported before that company’s product line existed.

The Repaint Nobody Has Scheduled

Two things already exist that nobody has pointed at each other: a trained checkpoint that has flown circles around five rival architectures across a red camping chair in four seasons, and a can of paint. Repainting the target costs less than any single flight session the original team already ran, and the crop-and-relabel script needed to build the test frames is sitting, published, in the same repository as everything else. This is not a call to retract or apologize for anything — the seven-test comparative record survives a fully honest reading of everything this article has argued, and the theoretical grounding for calling liquid dynamics causal-capable survives untouched as a claim about the architecture class. It is a call to run the one test that would tell a reader whether “are causal” describes what a specific trained network actually did, or describes what any sufficiently shift-stable detector would have done to the same red chair regardless. The paint has been sitting unopened since 2023. Someone should finish the coat.