Zero Cost Attribution for AI Toolchains

BOHM reads routing weights in AI toolchains and attributes decisions at zero cost. The paper introduces a hierarchical attribution method that works with compound AI systems by tracing how tasks travel through a tree of specialized components, without peeking under the hood of each module or paying for coalitions to be evaluated.

In BOHM, leaf attribution is the path product of root-to-leaf routing weights, while level-k attribution is the induced distribution over depth-k nodes. That means you can see, at every layer of the decision network, which components actually steer a result, without requiring access to each component’s internals. The method is designed for deployed orchestrators and third-party APIs where the evaluator cannot run every possible subset of tools. Compared with traditional SHAP style attribution, BOHM has zero marginal cost and offers multi-resolution insights simultaneously; flat methods simply cannot deliver that kind of layered view within the same budget.

The paper demonstrates what the approach can do in practice. In a three-level hierarchy applied to 18 large language models across 880 LiveCodeBench problems, BOHM yields a Kendall tau of 0.928, while SHAP reaches 0.980 but at roughly 9000x more coalition evaluations per seed. In a five-driver, seven-benchmark agentic study spanning 35 cells with complete coverage, tools tend to be concentrated, with a median top-share around 0.65, and the cell-level agreement between BOHM and SHAP tracks this with a simple rule: when the empirically best tool is the driver’s top pick, the two methods align more closely; otherwise the gap widens. A US Census hierarchy with 475 leaves across four levels shows BOHM recovering ground-truth rankings at every level, with tau values up to 0.722. Taken together, the results position BOHM as an efficient, scalable lens into how routing decisions shape outcomes in layered AI systems.

The technical report states that BOHM satisfies efficiency, monotonicity, symmetry and weak suppression, but does not achieve Shapley additivity. In other words, it answers a different question than Shapley-based coalitional decomposition. The practical takeaway is clear: in deployed stacks where you cannot inspect every tool combination or where you must rely on a router to steer tasks, BOHM is a practical, zero-cost attribution source that scales with your system’s routing geometry. If you need the additive, theoretically exact breakdown that SHAP provides, you still have a role for SHAP, but you’ll pay in coalitions and compute, especially when APIs or agentic orchestrators limit what you can reconfigure for evaluation.

Analysts should view BOHM as a diagnostic dial you can tune without touching production endpoints. The vivid picture is a city traffic model: you do not have to reroute lanes to understand which streets actually influence a commute you experience; you simply follow the flows that your routing weights already produce. That clarity makes it easier to spot bottlenecks, dead weights, or unexpectedly dominant tools, and to justify changes to routing policies or toolkits before a quarterly release.

For products shipping this quarter, BOHM offers a fast, low-cost way to audit routing-driven behavior across complex toolchains, especially when API partners and orchestrators form the backbone of task flow. It is a complementary instrument to SHAP rather than a wholesale replacement, useful for rapid triage, monitoring, and incremental improvement in production systems.