Claude Managed Agents

Stop-the-presses corrections vs initial impressions

Beta status & limits

• Header managed-agents-2026-04-01 on every endpoint
• Outcomes + multiagent public beta as of 2026-05-06 — no access form needed
• Rate limits 300 req/min creates · 600 req/min reads
• Branding "Claude Agent" is the preferred surface label
• No published pricing in beta docs as of read

System tools disabled by default: network

Notable absence: gh CLI is not pre-installed. Add via packages.apt: ["gh"] on the environment — Anthropic pre-installs once and caches across sessions.

Specs

Authoring rules from best-practices

• YAML frontmatter required — name (≤64 chars, [a-z0-9-]+, no "anthropic"/"claude"); description (≤1024 chars)
• Description must be third-person"Processes Excel files and generates reports" ✓ — "You can help with..." ✗ (causes discovery problems)
• Body ≤500 lines for performance; split into bundled files referenced from SKILL.md
• One level deep for file references — Claude only head -100s nested references
• Gerund naming preferred — processing-pdfs, not pdf-helper
• Inside skill bodies, MCP tools must be fully qualified — ServerName:tool_name
• Bundled scripts are more reliable than asking the agent to regenerate equivalent code

Cross-surface gotcha

Custom skills do not sync between surfaces:

• Claude.ai uploads (per-user)
• Claude API uploads (workspace-wide via /v1/skills)
• Claude Code filesystem-based, per-project at .claude/skills/

Local kendo Claude Code skills (vue-vitest-testing, php-unit-test) cannot be copy-pasted. They use first/second person and reference Claude Code-only slash commands.

How it works

A single agent declares a roster of sub-agents. All sub-agents share the same container and filesystem. Each runs in its own session thread with isolated context — own conversation history, model, system prompt, tools, MCP servers, skills.

Threads are persistent — coordinator can send a follow-up to an earlier sub-agent and that sub-agent retains its full prior turns. Maps cleanly to multi-stage workflows: Planner → Implementer → Reviewer.

"multiagent": {
  "type": "coordinator",
  "agents": [
    { "type": "agent", "id": "agent_xxx" },
    { "type": "agent", "id": "agent_yyy", "version": 3 },
    { "type": "self" }
  ]
}

Constraints

Define what "done" looks like

The harness iterates the agent until the artifact passes a separate grader. Rubric is markdown with explicit per-criterion checks. Grader runs in a separate context window from the main agent — isn't biased by the agent's implementation choices.

{
  "type": "user.define_outcome",
  "description": "Build a DCF model for Costco in .xlsx",
  "rubric": { "type": "text", "content": "# DCF Model Rubric\n..." },
  "max_iterations": 5
}

Default max_iterations: 3, max 20. Rubric inline or via Files API. Span events span.outcome_evaluation_* make iteration loop observable. One outcome at a time per session, but outcomes can be chained sequentially.

Result types

Session events

Vault events

Delivery semantics

• Verification: SDK unwrap() checks X-Webhook-Signature, rejects payloads >5 min old
• Payloads carry event type + ID, not the full object — fetch via GET
• At-least-once retries with same event.id — dedupe accordingly

• Ordering not guaranteed
• 3xx counts as failure — redirects not followed
• Auto-disabled after ~20 consecutive failures or immediately on private-IP hostname

Sonnet · prompt 1 · clean comparison

Production ResearchAction system prompt against a story-gen input ("issue pinning" feature). Both paths ran to end_turn with no interruption.

Messages-API per-prompt cost was 3.5× higher than the first spike's average ($3.24) — production-shape ResearchAction prompts are far more open-ended than synthetic codebase questions, so the no-cache agent loop balloons accordingly. CMA's automatic prompt caching absorbs this — same prompt cost $0.56.

Verdict at the time: migration green-lit. Cost gate (≥ 2×) passes by 10× margin. Latency gate (CMA p95 ≤ 1.5× current) was scored against the spike's 483 s headline — trivially passable.

Post-implementation correction (2026-05-09): a query against prod-issue-tracker.ai_outbound_logs (n=36, last 30 days, Script tenant) showed real production p95 is 28 s, not 483 s. The spike's "production-shape" prompt was 17× the real production p95. Today's CMA smokes (Sonnet 191 s, Haiku 125 s) are 4–7× slower than production p95 — the latency gate fails against the real baseline. Cost case still holds. Full postmortem with methodology lessons: research/2026-05-09-production-telemetry-correction.md.

Harness reliability findings

Prompts 2-4 surfaced limitations of both the spike scaffold and Anthropic's session-control primitives. None block the migration decision; all flow into the ManagedAgentsService design when /plan-feature runs.

1. Production prompts run 5–10× longer than synthetic. First spike: 100–180s on CMA. Production-shape: 5–13 min. Patched session timeout 300s → 600s; future runs still need outlier tolerance.
2. user.interrupt doesn't reliably stop sessions mid-bash-tool-execution. One stuck session ignored 3 interrupts over 15s and kept running for ~13 min until its bash command naturally returned. Interrupts seem to take effect only at model-loop boundaries, not mid-tool-call. Production constraint: bound work via outcome rubrics, not client-side interrupts.
3. Reading session.usage too early returns 0/0/0/0. Patched to poll for idle status before reading. Without this fix, ~$1 of orphan-session spend went unattributed initially.
4. Full agent_toolset_20260401 is too broad to faithfully simulate ResearchAction. Production has 3 read-only tools; the spike agent had bash + read + write + edit + grep + glob + originally web_fetch + web_search. On prompt 2, the agent ignored "your ONLY job is to explore" and made 35 edit calls patching 16 files (a complete ValidationException-passthrough fix across all create/update MCP tools). No git push attempted — verified via bash log; the fix lived only inside the ephemeral container and was reaped on archive. Migration plan must use default_config.enabled: false + read-only allowlist.

Spend tally