AI agents are not coworkers

Labeling AI agents as employees lowers performance, study finds.

The paper shows that when workers are told outputs come from an agentic AI employee rather than a chatbot, they catch 18% fewer errors, a counterintuitive result that highlights how framing shapes human judgment and workflow. Emma Wiles, a Boston University business professor, led experiments with managers to see how the “co worker” label shifts expectations, trust, and scrutiny. The finding is subtle but devastating for teams chasing speed with quality, because the human in the loop may loosen vigilance when an AI is billed as a colleague rather than a tool.

In the broader industry arc, the trend toward agentic AI is not vague hype. Nvidia chief executive Jensen Huang has talked about workplaces inhabited by digital humans, and since April major players including Microsoft, OpenAI, Anthropic, and Google have rolled out tools aimed at coordinating teams of AI agents. The goal is to push AI from a single assistant to a networked workforce, with agents that can split tasks, renegotiate goals, and loop until a target is achieved. Yet the study’s key lesson is not about capability, but about governance and expectation management. With these tools, a large share of management teams already behave as if AI agents are co workers or even employees; about 23% of the 1,261 managers surveyed reported labeling AI agents on organization charts, and nearly a third are already framing AI work as part of teams rather than as standalone tools.

Benchmarks indicate that agentic AI can perform complex tasks more efficiently on paper, but the human side of the equation remains fragile. The paper notes that agents can effectively be thought of as AI tools programmed to operate in loops until they reach a goal, which is powerful in automation but fragile in practice if humans assume the tool carries the same accountability as a person. In real workstreams, managers who treat AI as a teammate may overestimate the AI’s reliability, while underestimating the need for human oversight. The result is a misalignment between what the AI can do in a controlled test and what humans expect it to do in production, especially when the outputs are framed as coming from an employee rather than a tool.

From a practitioner standpoint, two core constraints emerge. First, naming and framing matter a lot. The same AI output gains credibility when labeled as coming from an agent rather than a helper, but that credibility can dampen people’s vigilance and distort error discovery. The second constraint is governance. Do not put AI agents on an org chart or treat them as independent decision makers. Instead, assign clear ownership for the outputs, establish objective performance metrics, and keep critical decisions under human review. The study suggests you should also watch for automation bias, where teams defer too readily to AI judgments, and set explicit exit criteria for agent-driven loops to prevent runaway workflows.

What to watch next: as more teams pilot AI agents at scale, leaders should design evaluation plans that separate perceived usefulness from verifiable quality. Build governance rails that keep humans in the loop for sensitivity-heavy tasks, specify accountability for AI outputs, and calibrate incentives so teams reward accurate verification as much as speed. The takeaway is pragmatic and clear: treat AI agents as powerful tools, not as coworkers, and let governance and metrics reflect that boundary.