NVIDIA and AWS outline two infrastructure patterns for production AI agents

BlueField targets the data path inside AI factories, while Smartsheet’s AWS deployment shows how a remote MCP server can expose enterprise actions and data to internal and external agents.

NVIDIA and AWS published technical posts on July 16 and July 17 detailing different layers of infrastructure needed to run AI agents in production: NVIDIA focused on moving, securing, and reusing data within AI factories, while AWS described Smartsheet’s remote Model Context Protocol server for connecting agents to enterprise work-management systems.

The posts point to a practical shift for ML teams. Agent workloads do not stop at a single model inference request. They can invoke several models, tools, databases, policy checks, storage systems, and network services before returning an answer or completing an action. That makes infrastructure behavior, especially latency, context handling, access control, and data movement, part of the effective inference pipeline.

NVIDIA’s July 16 post positioned its BlueField infrastructure processors and DOCA software as a way to remove networking, storage, security, telemetry, and control-plane work from host CPUs. The company said agentic AI can generate many model calls, tool calls, memory lookups, policy checks, and network transfers from one user request, increasing pressure on systems that must keep GPUs supplied with data and context.

BlueField-4 DPUs are intended to offload and isolate those services in the AI factory data path, NVIDIA said. Its Vera BlueField-4 STX storage processors are aimed at data platforms that support context memory, high-performance storage infrastructure, and secure data services. DOCA provides the software layer for programming and operating those infrastructure services across networking, storage, security, telemetry, and lifecycle management.

NVIDIA’s central engineering argument is that long-context and agent workflows make infrastructure optimization inseparable from inference optimization. GPUs still generate tokens, but CPUs may coordinate tools, prepare prompts, process retrieval results, validate outputs, and manage the next step in an agent workflow. Meanwhile, the platform must preserve and retrieve the context that lets the system continue work across turns and sessions.

That context includes key-value, or KV, cache data generated during model prefill. Efficiently retaining and reusing that state can avoid repeated computation, but it also raises requirements for storage throughput, networking, isolation, and policy enforcement. NVIDIA said BlueField’s goal is to improve GPU utilization, latency predictability, isolation, cost per token, and tokens per watt. The company did not provide benchmark results, hardware throughput figures, power measurements, or comparative cost data in the post, so those production claims remain unquantified.

AWS’s July 17 post described the application-facing side of the same problem. Smartsheet built a remote MCP server on AWS to give AI clients structured access to its data and capabilities. MCP is an interface pattern that lets AI applications discover and invoke approved tools rather than relying on loosely structured prompt text or bespoke integrations for every client.

According to AWS, Smartsheet uses the same MCP layer for its Smart Assist product experience and externally connected clients such as Amazon Quick. The server allows assistants to analyze project data, update tasks, create sheets, and manage workspaces through natural-language requests. AWS said enterprises can also build autonomous agents that coordinate through Smartsheet for tasks such as capturing requirements, picking up work, attaching test results, and drafting documentation.

The architecture puts security and request handling ahead of the MCP server. Requests pass through AWS WAF, AWS Shield, an Application Load Balancer, and OAuth validation before reaching stateless MCP server containers running on AWS Fargate for Amazon ECS. The MCP layer then calls Smartsheet domain services through existing APIs for transactional work.

Behind that layer, Smartsheet uses Amazon Kinesis Data Streams and Amazon Managed Service for Apache Flink to ingest change events into Amazon S3. AWS said Amazon Bedrock supports LLM inference and Amazon Neptune supports a knowledge graph used for cross-project insights.

Smartsheet also built an AI-oriented interface on top of its existing APIs and intelligence layer to reduce token usage, improve reliability with enterprise data, and help limit hallucinations. AWS said internal telemetry shows the company has saved more than 3 billion tokens since launch through those optimizations. That figure has not been independently verified, and AWS did not disclose the baseline token volume, the period covered, the models involved, or the cost savings associated with the claimed reduction.

The two publications describe complementary, rather than identical, infrastructure priorities. NVIDIA is focused on the lower-level data path in GPU-heavy AI factories, where infrastructure processors can handle data movement and inline policy enforcement outside host CPUs. Smartsheet’s AWS design is focused on safely exposing enterprise capabilities to multiple AI clients through a common remote MCP interface.

For product leaders, the important design question is no longer only which model to deploy. It is whether the surrounding system can maintain context, control tool permissions, serve structured enterprise data, and execute actions reliably when agents take multiple steps. A highly capable model still becomes operationally expensive if it repeatedly rebuilds context, receives excessive data through tool calls, or waits on overloaded application services.

Neither company stated that the posts were part of a coordinated announcement, and the available material does not specify a shared geography or joint deployment. NVIDIA also did not disclose model parameter counts, since its post concerned infrastructure rather than a new model release. What is confirmed is a growing emphasis on systems that treat context storage, security policy, data access, and network efficiency as production constraints for agentic AI rather than secondary platform concerns.