STRACE lifts agent optimization with causal tracing

A new tracing method boosts long-horizon agent success by 1.4x.

The paper introduces STRACE, short for Structural TRajectory Analysis and Causal Extraction, to modernize how reflection-based optimization uses traces. In long-horizon tasks, researchers have leaned on a large language model acting as an optimizer to diagnose failures and improve agent policies. But raw execution traces are noisy and sprawling, with many steps that do not matter for the failure at hand. The STRACE framework tackles this head on by building high signal-to-noise optimization contexts. At the batch level, it mines for failure patterns to filter out redundant traces and keep representative, informative ones. Within each selected trace, it performs causal localization over a textual dependency graph to remove non-causal steps and identify the true root-cause module for optimization. The result is a tighter, more focused signal for the LLM optimizer to work with.

The paper shows STRACE significantly outperforms standard context-filtering baselines. In a challenging formal verification task called VeruSAGE-Bench, the method optimizes human-expert designed agents and yields a 1.4x improvement in success rate, climbing from 42.5 percent to 58.5 percent. Benchmarks indicate these gains come from cleaner optimization signals rather than larger model sizes or brute-force data consumption. The team reports the approach nudges optimization toward causal moments in the trace, reducing the risk of overfitting to incidental or low-value steps that clutter traditional contexts.

For practitioners, the engineering takeaway is clear: the way you curate and interpret traces can matter as much as the optimizer itself. STRACE reframes trace management as a two-step signal process. First, prune the batch of traces to a representative subset that captures the core failure modes. Second, within each trace, sift the sequence through a textual dependency graph to isolate the causal spine that actually drives failure and recovery. This two-layer design can cut optimization noise and help an LLM converge faster to meaningful policy improvements.

Some practical constraints surface from the approach. Building the batch-level failure filters depends on patterns that generalize across tasks, so teams must calibrate what counts as a representative failure in their domain. The causal localization relies on a robust textual dependency graph, which means careful construction or validation of the graph is essential to avoid discarding relevant steps or keeping misleading ones. There is also an implicit processing overhead: STRACE requires analyzing traces and building graphs before optimization, which may add offline compute but can pay off in better sample efficiency and fewer failed iterations during policy refinement.

Looking ahead, STRACE offers a blueprint for making reflection-based optimization more reliable across domains. As teams push toward longer-horizon reasoning or different verification tasks, evaluating how the root-cause module shifts with trace quality will be important. The approach also invites exploration of alternate dependency representations and how different LLMs as optimizers interact with these curated contexts. In short, better signals, not bigger models, may be the lever that unlocks more robust agent learning in complex environments.

The STRACE approach underscores an ongoing engineering truth: improvements in AI systems come from smarter data hygiene as much as smarter models. By focusing optimization on the truly causal moments within traces, it is possible to achieve meaningful gains without chasing ever-larger networks or datasets.

The STRACE code is available on GitHub for teams looking to experiment with this signal-first trace strategy.