Latency Profiling for Agent Workflows

An agent that's slow is an agent nobody uses. The fix isn't a faster model — it's profiling the workflow to find the one stage that's actually worth fixing.

Agent latency is rarely uniform. A workflow is a chain — model calls, retrievals, tool calls, handoffs — and one stage usually dominates. Profiling is how you find that stage instead of guessing.

The discipline: measure where the time goes before you change anything. A profile of the workflow — each stage, its duration, its share of total — is what turns "it's slow" into "stage X is 60% of the latency." Without the profile, every optimization is a guess.

• Measure each stage's duration.
• Compute each stage's share of total.
• Fix the dominant stage first.

• **Model calls.** Often the largest share — and the most optimizable (caching, batching, smaller model).
• **Retrieval.** Usually cheap, until it isn't (large corpus, complex filters).
• **Tool calls.** External latency you don't control, often the hidden dominant stage.
• **Handoffs.** Usually cheap, but serialization and network can dominate at scale.

Fix the dominant stage, not the easy one. A 5% optimization on the dominant stage beats a 50% optimization on a minor one. The profile is what tells you which is which — and most teams optimize the stage they're comfortable with, not the one that matters.

• Optimizing without a profile.
• Optimizing the model call when a tool call is the dominant stage.
• Optimizing for average latency when the tail is the problem.

Agent latency is a chain, and one stage usually dominates. Profile before you optimize, fix the dominant stage first, and refuse to optimize without measurement. The fastest win is almost always the one the profile points at.