Companion to the AI for Software Engineers field guide. A survey of the tools, infrastructure, and patterns for building agentic systems that stay fast, correct, and radically cheap in July 2026, with the trade-offs stated plainly so you can choose for yourself.
Every choice in this playbook is evaluated against one question: what does a correct outcome cost at 10,000× volume? Not tokens: outcomes. That reframe changes almost every default. A model that's 3× cheaper but wrong twice as often is more expensive. A vector database that's free but eats an engineer-week a month is not free. This is a survey: for each layer we lay out the serious options and their trade-offs, so the choice stays yours.
The recurring pattern you'll see in every section: cheap tier for volume, expensive tier for judgment, verification in between. Bulk steps (classify, extract, filter, summarize) go to models and infra priced near zero; the narrow set of decisions that actually need frontier intelligence get frontier prices; and deterministic checks (schemas, rules, tests, cross-references) sit at the boundary so errors from the cheap tier never silently propagate. Hold that shape in mind; §12 assembles it into one diagram.
The most under-appreciated fact of 2026: frontier-adjacent intelligence now costs pennies. DeepSeek V4-Flash lists at $0.14/$0.28 per million tokens with a 1M context, one to two orders of magnitude below flagship rates. At that price, whole categories of "too expensive to automate" workflows flip to trivially affordable. The engineering problem shifts from affording calls to containing errors.
| Model (open / cheap tier) | Price in / out ($/MTok) | Strengths | Watch out for |
|---|---|---|---|
| DeepSeek V4-Flash | 0.14 / 0.28 | 1M context; absurd price floor; fine for classification, extraction, summarization at volume | Highest hallucination propensity of the group; never let output through unverified |
| DeepSeek V4 / V4-Pro | sub-$1 promos | Frontier-class reasoning at budget rates; GA as of mid-July | Pricing churns: legacy endpoints retire July 24 and peak-hour surcharges arrive; provider throughput varies by host |
| MiniMax M3 | ~0.60 in | Cheapest open-weight frontier coder; strong SWE-bench Pro | Smaller ecosystem; test tool-calling reliability yourself |
| Kimi K2.x / K3 | low | K3 (July 17, 2.8T MoE, 1M ctx) tops frontend-code arenas; strong agentic + browsing behavior; "does whatever you ask" | Weights promised July 27, not yet delivered; compliance is a double-edge: it will also do the wrong thing confidently |
| GLM-5.2 · Qwen 3.5 · Inkling | low | Solid multilingual coders, MIT/Apache-licensed; Inkling (Thinking Machines, 975B/41B active) is the first US open-weight entrant at this tier, weights on day one | Quality varies more by task; benchmark on your own data |
| Haiku / Flash / mini tiers | ~0.25–1.50 | Proprietary small models: better instruction-following per dollar than raw benchmarks suggest | Costlier than open flash tiers; still not for final judgment calls |
Cheap models hallucinate more: fabricated facts, invented fields, plausible-but-wrong classifications. The mitigation is never "prompt harder"; it's structural:
Schema-first everything. Force output through a validated schema (Instructor/Pydantic). This kills format hallucination outright and turns content errors into checkable fields instead of prose you have to read.
Ground, don't recall. Never ask a cheap model a question from its own memory when you can hand it the source text and ask it to extract. Retrieval-grounded extraction with "answer only from the provided text, else return null" cuts fabrication dramatically, and nullable fields give the model an honest exit.
Verify with a different signal. Deterministic checks first (regexes, ranges, cross-field consistency, database lookups), then a second model as judge only where determinism can't reach. Self-consistency (sample twice, compare) is a cheap tiebreaker at these prices.
Cascade with confidence routing. The canonical pattern: cheap model attempts → validators score → high-confidence results pass, low-confidence escalate to a frontier model. You pay Opus prices only for the hard residue, typically a small fraction of volume.
Two operational notes. First, throughput is the real constraint for open models: quality is provider-dependent (the same weights served by different hosts differ in speed, context handling, even correctness), so pick hosts per model and treat TPM as your sizing unit. Second, run this through a gateway (LiteLLM, OpenRouter, Vercel AI Gateway) so routing rules, fallbacks, and budgets live in config, not code, and so a provider outage or price change is a one-line fix.
The field guide gave context engineering as a mental model; here it becomes a cost function. In an agent loop, the growing context is re-sent on every turn, so context discipline isn't just a quality lever, it's the largest line item you directly control. Four questions, asked at every step: what do we fetch, when do we fetch it, how do we compress it, when do we throw it away?
The cheapest token is the one never sent. In practice: truncate tool outputs at the tool layer (return the 20 rows that matter, not the 4,000-hit grep); cap what each retrieval call may contribute and drop chunks below a relevance threshold; and put a re-ranker between retrieval and context with a hard top-k: 50 candidates re-ranked to a precise 5 reliably beats 50 chunks dumped in and hoped over. This is also where "more context" actively hurts: past the budget, added material causes distraction (irrelevant bulk crowding the signal) and confusion (conflicting inputs pulling the model apart), the two canonical failure modes of naive pipelines.
Prompt caching discounts repeated prefixes by up to 90%, but only if your context is cache-friendly. The rules: keep the stable material (system prompt, tool definitions, skills, reference docs) at the front and byte-identical across calls; append new turns rather than editing history; and never interleave volatile content (timestamps, request IDs) into the stable prefix, because one changed byte invalidates everything after it. A practical corollary when routing through an aggregator: pin each model to a single upstream provider and disable automatic fallbacks for your cache-heavy paths, because a silent mid-conversation provider switch throws away the cache you engineered (and cache-read pricing differs by host). Inject genuinely dynamic material, like the current date, as a separate trailing block so the main prompt stays byte-stable. For agent loops this alone routinely halves effective input cost. Stack it with the batch API (−50% for anything async, which is most pipeline work) and the cascade from §02, and the same workload can differ in cost by an order of magnitude on architecture alone.
Long-running agents need active hygiene, in escalating order of intervention: tool-result clearing first (once a call deep in history has served its purpose, the raw result is dead weight; it's the safest, lightest compaction there is); compaction when the window nears its limit (summarize preserving decisions, open bugs, and unresolved threads; tune the summary prompt for recall first, precision second); structured notes outside the window for state that must survive resets; and sub-agents when exploration would pollute the main thread: an explorer burns 30K tokens in its own window and reports back 1–2K, which is simultaneously a quality pattern and a cost pattern. Position what remains deliberately: recall is strongest at the start and end of context, weakest in the middle, so the middle is where you put the material you'd least mind the model skimming.
Treat tokens as an SLO: track tokens and tool calls per task class, set budgets, and alert on regressions: an agent class that quietly doubles its tool-call count is a bug even when outputs stay correct. The metric that keeps everyone honest is cost per accepted outcome (including retries, escalations, and human review time), which is the number the reference architecture in §12 is designed to minimize.
Long context didn't kill RAG; it clarified it. Dumping a haystack into a 1M window measurably degrades recall versus giving the model only the relevant slices, so retrieval remains the discipline of finding those slices cheaply. The 2026 stack has three decisions: how you embed, how you search, and where vectors live.
| Option | Type | Trade-off summary |
|---|---|---|
| text-embedding-3-large (truncated ~1024d) | API | The value pick many teams land on: Matryoshka truncation to 1024d costs ~0.03 recall for 3× smaller vectors, ~$0.13/MTok. English-centric. |
| Voyage 4 family | API | Top retrieval quality, best-in-class for code and technical corpora. The v4 line (Jan 2026) shares one embedding space across its lite/standard/large tiers, so you can mix cheap and expensive embeddings without re-indexing; voyage-context-4 (June) embeds chunks with surrounding-document context: chunking's sharp edges, sanded down at the model layer. |
| Cohere Embed v4 + Rerank | API | The multilingual leader; embed+rerank as one coherent pipeline; strongest fit for European many-language corpora. |
| Gemini Embedding | API | The all-rounder: near-perfect cross-lingual retrieval scores in third-party benchmarks, with multimodal (text+image) coverage if your corpus will grow modalities. |
| BGE-M3 / Qwen3-Embedding | Self-host | MIT-class open workhorses; BGE-M3 does dense+sparse+multi-vector in one model (hybrid search from a single deployment); zero per-token cost, your GPU instead. |
Every serious embedding comparison converges on the same instruction, so take it as consensus: MTEB rank is not your corpus. Pick two or three candidates, run 50–200 real queries from your domain against each, score Recall@10, and let that decide: the leaderboard winner routinely loses to a cheaper model once your documents and your latency budget enter the picture.
Two findings that matter more than the model choice: a cheaper embedder plus a reranker usually beats an expensive embedder alone (embed for recall into top-50, rerank for precision on top-k), and most teams over-buy embedding quality while under-investing in chunking. Deterministic, structure-aware chunking with stable IDs is worth more than the last 0.02 of recall, and it's what makes incremental re-embedding possible without orphaned vectors.
Four patterns that recur in serious multi-source deployments, stated once so you recognize them. One namespace, not one per source: when the consumer is an LLM reasoning across sources, per-source indexes force you to merge K top-N lists whose scores aren't comparable; a single namespace with a source filter attribute returns one honestly-ranked global top-K, and taxonomy changes become query-time filter expansions instead of re-indexing jobs. Version by name, migrate by pointer: encode schema and embedding-model versions into the index name, build v2 in parallel, flip the read pointer, drop v1: no in-place migration ever. Key chunk IDs on character offsets, not text hashes: token-accurate windowing is good for the embedder, but IDs derived from structural position survive re-cleaning, so fixed text re-embeds in place instead of orphaning vectors. Make the asymmetric-embedding footgun impossible: most modern embedders take a document-vs-query mode flag, and mixing modes silently destroys recall with no error anywhere, so don't expose the parameter; expose embed_documents() and embed_query() and nothing else. Same spirit when swapping embedding backends (say, API to self-hosted): gate the cutover on a cosine-similarity parity check across both modes and record backend provenance per document, so numerics drift never silently mixes incompatible sub-spaces in one index.
| Store | Model | Trade-offs |
|---|---|---|
| pgvector (Postgres) | In your existing DB | Zero new infra, transactional joins with your data, fine to millions of vectors. You own tuning; scaling past that gets hands-on. |
| turbopuffer | Serverless, object-storage-first | The cost-structure outlier: cold data at ~$0.02/GB on S3-class storage, warm data cached to NVMe/RAM, so dormant namespaces cost near zero. Built for multi-tenant "one namespace per customer/company" shapes; hybrid vector+BM25 with rank fusion; used by Cursor, Notion, Linear. Trade-off: $64/mo usage minimum, no free tier, cold-start latency on rarely-touched namespaces. |
| Qdrant / Weaviate | Self-host or managed | Feature-rich engines with strong filtering; self-hosting is "free" until you price the ops time. |
| Pinecone | Managed serverless | Polished and hands-off; RAM-anchored pricing runs an order of magnitude above object-storage architectures at scale (~$0.33/GB vs ~$0.02). |
The structural insight for cheap-at-scale: match the store's cost anatomy to your access pattern. Mostly-cold, per-entity corpora (thousands of company dossiers, each queried rarely) fit object-storage architectures; a small hot corpus hammered constantly fits RAM-resident engines or pinned capacity. And hybrid search (dense vectors fused with BM25 keyword scores) is the 2026 default, because names, tickers, and legal identifiers are exactly what pure semantic search fumbles.
Garbage in is now the dominant failure mode: most RAG and extraction failures trace back to parsing, not models. The good news is the ingestion layer has specialized cleanly: pick by input type, not by fashion.
| Input | Options | Trade-offs |
|---|---|---|
| Web pages → clean text | trafilatura | The academic-grade main-content extractor: strips boilerplate/nav/ads, keeps metadata, fast and local. The default first step for turning HTML into LLM-ready text. Struggles only on heavily JS-rendered pages; pair with a fetcher. |
| Web at scale, hostile sites | Scrapling · Crawl4AI · Playwright | Scrapling's self-healing selectors survive redesigns and its stealth fetchers handle anti-bot walls (younger project, occasional rough edges). Crawl4AI crawls straight to LLM-ready Markdown/JSON. Playwright when you need a real browser. Always check for hidden JSON/XHR endpoints first: an API beats the DOM every time. |
| PDFs & Office docs, self-hosted | Docling · Marker · PyMuPDF | Docling (IBM, ~60k stars) is the open-source reference: a unified document model preserving layout, tables, reading order, exportable to Markdown/JSON, with a compact VLM (Granite-Docling) replacing the old OCR-cascade approach, and it ships an MCP server. Fully local, no per-page fees; weaker on forms/handwriting. PyMuPDF for fast raw text when structure doesn't matter. |
| PDFs, hosted / hardest cases | LlamaParse · Reducto · Mistral OCR | Vision-LLM parsers for dense tables, scans, brutal layouts. Reducto posts the strongest published extraction numbers (on its own benchmark, so verify on your documents); LlamaParse is the most RAG-integrated. You pay per page; reserve for documents that defeat the local tier. |
| Mixed corpora, broad formats | Unstructured · Apache Tika | Widest format coverage (email, HTML, Office, images) into typed elements; the pragmatic bulk-ingestion workhorses, with less precision on complex tables than the specialists. |
Whatever you pick, carry provenance through the pipeline: source path, page, position, parser + version, confidence. It's what makes extraction auditable, reprocessable, and debuggable (non-negotiable when your output feeds diligence decisions someone else will rely on). Version the extractor itself and bump that version on every behavior change: if your skip-existing guard keys only on the source file, an improved parser never reaches documents processed under the old one. Two cheap gates that save expensive downstream pain: validate magic bytes at ingestion (an HTML error page saved as .pdf poisons OCR output forever and no parser will warn you), and run a fast text-layer preflight before any GPU OCR pass so documents with good embedded text skip the expensive path entirely. And where a format is regular enough, prefer deterministic extraction outright: well-tested regex extractors hit 94–97% measured recall on structured references at zero marginal cost, which no LLM call at any price will beat on cost-per-correct-outcome.
Big context windows turned out not to be memory. Benchmarks keep showing the same shape: on one long-horizon memory suite, a small model scored 62.6% when handed the full session haystack and 89.2% when given only the relevant sessions: a 27-point gap from context bloat alone, and a good memory layer over the same model matched or beat the oracle. Hence a real product category: systems that reason over history in the background and serve back small, high-signal context.
| Option | Core idea | Trade-offs |
|---|---|---|
| Honcho (Plastic Labs) | Reasoning-first memory: ingest-time extraction plus background "dream-time" deduction; everything modeled as peers (users, agents, projects, ideas) in many-to-many sessions | State-of-the-art on memory benchmarks (90.4% LongMem-S, 89.9% LoCoMo) while using a median ~5% of available context: recall and token savings. The peer model fits multi-agent systems far better than a user/assistant pair. Open source, self-hostable (FastAPI) or managed; younger ecosystem, and background reasoning adds its own model costs. |
| Mem0 · Letta · Zep | Fact/profile memory stores with retrieval APIs; Letta descends from the MemGPT "memory as OS" lineage | Simpler mental models, broad integrations; mostly "store snippets and retrieve" rather than reasoning over relationships: fine for preference recall, thinner for evolving multi-entity state. |
| DIY: Postgres + summaries | Periodic summarization jobs writing structured state you control | No new dependency, fully inspectable; you re-implement forgetting, conflict resolution, and retrieval quality yourself, and it's deceptively deep. |
Decision rule: if your agents interact with the same entities repeatedly over time (users, companies, deals), a memory layer pays for itself in both quality and tokens. If each task is stateless, skip the category entirely and spend the effort on retrieval.
The framework question generates the most noise and matters the least, because the durable answer is layered: plain code for control flow, thin libraries for the LLM boundary, a framework only where its abstraction earns rent.
| Option | Shape | Choose it when / avoid it when |
|---|---|---|
| Plain Python + Instructor + LiteLLM | No framework | The default. Workflows (chain, route, parallelize, judge) are just functions; fully debuggable; nothing between you and the API. Avoid only when you genuinely need durable state machines or human-in-the-loop checkpoints. |
| PydanticAI | Typed agent runtime | Type-safe tools, DI, validated outputs, evals + Logfire built in. The natural step up for Python teams; not aiming to be a multi-agent orchestration graph. |
| Claude Agent SDK | Harness-as-library | The loop that powers Claude Code (context management, tools, permissions, sub-agents), programmable from Python/TS. Strongest when you want production-grade agent behavior without rebuilding a harness; couples you to the Claude ecosystem (and, since June, to explicit billing for headless use). |
| LangGraph | Graph state machine | Explicit stateful graphs, checkpoints, human-in-the-loop, biggest ecosystem. Earns its complexity on genuinely complex multi-actor workflows; overkill below that, and debugging through its abstractions is a skill of its own. |
| OpenAI Agents SDK · smolagents · CrewAI | Alternatives | OpenAI's SDK if you're Codex/GPT-centric; smolagents for code-executing agents (the model writes Python instead of chaining tool calls; surprisingly effective); CrewAI for role-based teams, more opinionated than most needs. |
| Mastra · Vercel AI SDK | TypeScript lane | The TS equivalents when the product surface is web-first. Vercel AI SDK owns streaming UI; Mastra is the rising TS agent framework. |
| Temporal · Restate / Inngest | Durable execution underneath | Not an agent framework: the layer that makes long pipelines survivable. Each agent step becomes a durable activity with its own typed retry policy, so a transient failure retries one slice instead of restarting the run, and state survives crashes via event-history replay. Deterministic control flow lives in the workflow; LLM judgment lives in the agents it calls. Earns its keep on multi-step pipelines with real money per run; overkill for request/response apps. |
One architectural note that generalizes: for evaluation-type work, a constrained judge + distiller (fixed steps, fixed schema) beats an open orchestrator (predictability and cost both), and open orchestration earns its keep only for breadth-first exploration where the steps genuinely can't be known in advance. Cognition's "Don't Build Multi-Agents" remains the right corrective whenever sub-agent enthusiasm strikes: shared context and simple loops first.
The quiet workflow revolution of 2026: your issue tracker is now an agent dispatch surface. A well-written ticket is a spec; a spec is a prompt; and both Linear and GitHub let you assign it directly to a coding agent that returns a draft PR.
Linear made agents first-class workspace members: assign issues to them, @mention them in comments, and, since June, delegate an issue to a secure "coding session" that runs Claude Code or Codex against your repo and attaches a reviewable diff to the issue (default model: Opus 4.8, drawing on workspace AI credits). The delegation semantics are worth copying in any system you build: the human stays primary assignee and the agent is added as a contributor, so accountability never transfers to software. Triage automations route incoming bugs to an agent automatically, with the agent pulling evidence from monitoring tools (Sentry, Datadog) over MCP before writing code; Linear reports resolving roughly 30% of its own incoming bug reports this way, mostly first-pass. The cadence tells you where this surface is going: agent chat in March, repo-level Code Intelligence in May, coding sessions in June, and as of July 20, "Loops": recurring agent jobs described in plain language and triggered on schedules or events. Third-party cloud agents (Cursor, Devin, Codex, Factory, and a dozen others) plug into the same delegation surface.
GitHub mirrors the pattern natively: assign an issue to Copilot's coding agent and it works in an ephemeral Actions environment (exploring code, running tests) and opens a draft PR; the Linear↔GitHub integration bridges the two so issues, branches, PRs, and statuses stay synced bi-directionally.
The trade-offs to hold onto: this workflow is only as good as your ticket hygiene (vague issues produce vague PRs; the spec discipline from the field guide's §05 is the actual enabler); agent sessions bill from credit pools you should budget like CI minutes; and the human review gate is the product, not a formality (the pattern works because a person approves every merge). Teams that measure it track cost-per-merged-PR and first-pass acceptance rate, which tells you which task categories to keep delegating and which to pull back.
The loudest open-source story of the year: two personal-agent harnesses (always-on assistants you self-host, reachable from your messaging apps, able to act on your machine and accounts) became the fastest-growing repos in GitHub history. You should understand them even if you never run one, because they preview where the harness layer is going.
OpenClaw started as Peter Steinberger's late-2025 side project (renamed twice after Anthropic trademark complaints: Clawdbot → Moltbot → OpenClaw), rocketed past 300k stars in months, and now lives in an independent foundation after Steinberger joined OpenAI. Its bet is the gateway: one agent answering across dozens of messaging channels, extended by ClawHub, a marketplace of tens of thousands of community "skills." Microsoft demoed its own enterprise agent running on OpenClaw at Build: a harness months old, on a keynote stage as infrastructure.
Hermes Agent (Nous Research, launched February 2026) is the challenger that overtook OpenClaw on OpenRouter's daily token rankings by May, passed 180K GitHub stars in under four months, and left the terminal-only era behind with a desktop app in June. Its bet is memory and self-improvement: an agent that carries your context across weeks and writes and refines its own skills over time. Same anatomy, opposite control point: OpenClaw starts from channels, Hermes starts from what the agent remembers and learns.
Why it matters beyond the spectacle: these projects are commoditizing the harness itself (loop, gateway, skills, memory as open MIT-licensed infrastructure), and they're a live security lesson. OpenClaw's early phase produced a cluster of serious CVEs (including a CVSS-9.1 skill-sandbox escape) and the ClawHavoc supply-chain campaign: over a thousand malicious marketplace packages across 23 compromised publisher accounts, with an estimated 15,000–25,000 installs; its originally permissive default permissions are exactly the anti-pattern §11 warns about. Hermes, container-first from day one with a hardline command blocklist, has kept a clean CVE record so far, which is the strongest available evidence that the security gap between these projects is architectural, not accidental. The honest posture: fascinating to run for yourself (sandboxed, least-privilege, on a throwaway account tier) and premature to wire into anything holding client data. Treat every community skill as an unaudited dependency with execution rights, because that's what it is.
Strip away the star counts and both harnesses share one architectural assumption: a single trusted operator. Neither has role-based access control. OpenClaw's access model is sender gating: DM policies (pairing codes, allowlists, open), per-platform allowlists, deny by default, and its multi-user story amounts to isolating DM sessions per sender. That's a bouncer at the door, not roles inside the room: once you're paired, you're the operator. Microsoft's February advisory called the default permission model "overly permissive for enterprise environments", and China restricting state enterprises and banks from running OpenClaw in March is the same judgment expressed as policy. Hermes is explicitly designed single-user; its security lead over OpenClaw is real, but it's single-tenant security. For a personal agent this scope is correct: you're the only principal, so roles would be ceremony. For a company it's disqualifying: the moment two employees share an agent, you need to answer who may ask it to do what, acting under whose credentials, gated by which approvals, with an audit trail that attributes each action to a person rather than to the bot's service account. Neither project models any of that, and bolting it on from outside means proxying every channel the gateway speaks, which means rebuilding the gateway itself.
That gap is the most interesting thing in this section. The harness layer commoditized in about a year (loop, gateway, skills, memory, all MIT-licensed), the model layer is contested, and the company layer above both (identity for agents, per-user scoped credentials, policy engines, human-attributable audit) has barely started. Linear's delegation semantics (§08: the human stays accountable, the agent is a contributor) is the closest shipped instance, and it's one product's feature, not infrastructure others can build on. So enterprise adoption is running ahead of its access-control substrate: Microsoft demos an enterprise agent on OpenClaw at Build while its own security team warns enterprises off OpenClaw's defaults. I'd expect the next concentration of capital here, the way identity vendors rode the cloud migration a decade ago: whoever ships the boring substrate (RBAC, SSO, policy, audit, packaged for agent fleets) converts half a million GitHub stars of personal adoption into an enterprise market. Until someone does, "we run OpenClaw for the team" stays a security finding rather than an architecture.
Teams quietly running personal-agent harnesses in production (yes, they exist) converge on a handful of structural rules worth stealing even for one agent on a VPS. Enforce capabilities with credentials, not prompts: if the agent must never send email, don't write "never send email" in its persona; give it an OAuth grant that lacks the send scope, so the attempt fails at the API with a 403. Persona rules are the second layer, tokens the first, and you red-team the boundary by trying the forbidden action. Zero inbound ports: connect to chat via outbound websockets, poll for mentions, sync files by polling deltas: trading seconds of latency for an attack surface of nothing. Separate what agents may propose from what they may grant themselves: letting an agent author and PR new skills for human review is healthy self-improvement; letting it install new capabilities (MCP servers, credentials) is privilege escalation, so the capability list lives in a repo the agent can't write to. Gate memory writes where agents are shared: on a multi-user agent, every long-term memory write, including the harness's own background self-improvement pass, stages for human approval, because a shared agent's memory is effectively a broadcast channel. And keep the whole machine disposable: identity and state on a mounted volume, everything else rebuildable from a script.
Agentic workloads have a distinctive shape: a small always-on control plane, bursty untrusted execution, and webhook-driven glue. No single host is best at all three; the cheap-at-scale answer is usually a composition.
The composition that keeps recurring for cheap-at-scale agent systems: a Hetzner-class box (or two) as the always-on control plane (queues, Postgres+pgvector, schedulers, dashboards), because nothing beats its price for steady-state compute and it keeps European data on European soil; Cloudflare at the edge for the event surface (webhooks, email-triggered pipelines, cron, object storage), because scale-to-zero pricing fits spiky agent traffic and the base cost is a rounding error; and microVM sandboxes (managed E2B/Modal, or self-hosted Firecracker on that same Hetzner metal once volume justifies the ops) wherever agent-generated or untrusted code executes. Self-hosting a sandbox runtime is precisely how you convert a vendor's per-second price floor into your price ceiling: the managed rate becomes the most you'd ever pay, not the least.
The counter-case is worth stating fairly: if your team is tiny and shipping speed dominates, a single managed platform (Fly, Railway, or all-in Cloudflare) is a legitimate choice: you're buying back engineering time, which is the one cost this playbook can't optimize for you.
Vendor snapshot for the sandbox tier, since it moved this quarter: Modal raised a $355M Series C at a $4.65B valuation in May with sandboxes now over a third of its revenue, which tells you where this category's demand is; E2B shipped pause/resume (filesystem and memory snapshotted, ~1-second restore), which makes long-lived agent sessions cheap to suspend instead of expensive to keep warm; and Daytona went closed-source in June, a reminder that "open-source sandbox" is a licensing state, not a guarantee: if self-hostability is why you picked a vendor, pin the last open release or build on the primitives (Firecracker, Kata, gVisor) directly. Also new this year: Fly's Sprites (Firecracker VMs with persistent disks and ~300ms checkpoint/restore) and Cloudflare's container sandboxes hitting GA, so the "managed microVM" column keeps gaining credible entrants.
Everything above assumes models arrive over an API. Past a certain volume, running open weights on your own GPUs becomes the cheaper lane, and 2026's tooling (vLLM with paged attention, prefix caching, speculative decoding) made it operationally realistic. The serious options, by shape:
| Option | Model | Trade-offs |
|---|---|---|
| Modal | Serverless GPUs, Python-first | Define a function, get an H100 (~$4/hr class), scale to zero when idle: the entire vLLM/load-balancing/failure-handling ops burden disappears. Best for bursty inference and batch jobs; the premium unit rate is what you pay for never touching Kubernetes. Weakest fit for steady 24/7 load, where you're paying serverless margins on utilization you could commit to. |
| Vast.ai | P2P GPU marketplace | The raw price floor: thousands of hosts bidding idle GPUs down: H100s from under $1/hr on the open market, ~$1.50–1.90 on verified datacenter hosts, interruptible instances at roughly half again. Per-host reliability scores, no central SLA. Excellent for fault-tolerant batch: embedding runs, bulk extraction, fine-tunes with checkpointing. Not recommended for customer-facing inference unless you build health checks and failover yourself. |
| RunPod · Lambda | Managed GPU cloud | The middle path: RunPod for range (community tier near marketplace prices, secure tier with real uptime, a serverless option); Lambda for the dedicated-box experience: SSH in, vLLM up, predictable access. Both fit sustained self-hosted endpoints better than a marketplace. |
| Hetzner GPU / dedicated | Owned-rate metal | The same logic as this section's control plane extended to inference: a dedicated GPU box at a flat monthly rate is unbeatable for high, steady utilization and keeps EU data on EU metal. You own drivers, monitoring, and capacity planning. |
The decision math is utilization: API prices embody someone else's idle time. But be honest about how contested the break-even is: published analyses disagree by an order of magnitude, from "low hundreds of millions of tokens per month crosses over" (a flat ~$450/mo GPU replacing $600–1,300 of API spend) to "hosted APIs win until a single model clears 30–50M tokens per day", because they assume different models, utilization rates, and whether your ops hours are free. The synthesis that survives both: at 2026's API prices, self-hosting below solid nine-figure monthly token volume is rarely a savings play; it's a sovereignty, latency, or throughput play (your own endpoint has no rate limits), and spiky or low-volume workloads never cross over, because scale-to-zero beats an idle card.
For batch GPU work specifically (embedding runs, OCR, bulk inference), the operators who do this at scale converge on rules that contradict the spec sheets. Score offers by dollars per unit of work, never by GPU model: batch pipelines often saturate CPU preprocessing and PCIe long before the tensor cores, so a card with 4× the benchmark FLOPS can deliver identical throughput. VRAM predicts throughput better than compute, because VRAM is batch size. The same GPU model varies ~50% in delivered performance across marketplace hosts, so measure per-offer, and expect 15–25% of the cheapest spot boxes to die mid-run: design the pipeline for reaping dead workers and redelivering their work (checkpointing plus idempotent chunks), at which point the marketplace price floor becomes genuinely usable. Price the reliability tax explicitly: counting restart overhead and lost compute, effective cost lands 20–55% above the listed rate on unverified marketplace hosts and 3–13% on verified ones, so budget 30–50% over the sticker and let long jobs favor verified hosts. A cheap trick that beats guessing: launch one self-destructing calibration box per GPU type, measure actual throughput, fit the price/performance curve, and let that pick your fleet: it costs a few dollars and routinely halves the bill. The honest full accounting includes your ops time and the capability gap: self-hosting buys you MiniMax/GLM/Qwen-class intelligence, not Opus-class, so it pairs naturally with the cascade: self-hosted weights as the cheap tier, API frontier for the residue. A sane progression: validate on APIs → move bulk steps to Modal or Vast batch → commit steady load to dedicated metal only once utilization is proven.
Agent security in 2026 is not a prompt problem; it's an architecture problem. Prompt injection remains unsolved, agents amplify every credential they hold, and the year's incident log (poisoned CI actions, malicious skills, shell-injection design flaws) reads like a preview of your threat model. The defenses that work are boring, structural, and layered.
Danger concentrates where one component combines private-data access, exposure to untrusted content, and the ability to act or exfiltrate. Design so no single agent holds all three: the agent that reads scraped pages doesn't hold database credentials; the agent that writes to client-facing systems never ingests raw external text. Everything fetched (web pages, PDFs, emails, tool results) is hostile input that may contain instructions; pipe it through extraction into typed fields before any privileged component sees it.
Never hand an agent your credentials. Mint short-lived, narrowly-scoped tokens per task; separate read from write paths; default-deny outbound network with explicit egress allowlists (both Cloudflare containers and E2B expose this; configure it, since exfiltration is the payoff of most injection attacks). Secrets live in a manager and get injected at the execution boundary, never pasted into context: anything in the context window can be echoed back out. The test of a real boundary is structural: if the persona says "read-only" but the token carries write scope, you have a suggestion, not a boundary. Grant scopes so the forbidden action fails at the API itself (an agent that must never send email gets a grant without the send scope, and the attempt 403s), then red-team the boundary by attempting the forbidden action and confirming it fails.
Containers (shared kernel) are adequate for your own trusted code; anything model-generated, third-party, or fully autonomous belongs in microVM-grade isolation (Firecracker/Kata/gVisor), where a kernel escape is contained to one guest. This is the concrete difference between the sandbox tiers in §10, and the reason "it runs in Docker" is not a security answer for untrusted execution.
MCP servers, agent skills, and marketplace extensions execute with your agent's authority, and the base rate is bad: a Snyk audit found roughly a third of tested marketplace skills carried prompt-injection vulnerabilities. Vet them like packages: pin versions, review before enabling, prefer an internal registry of approved servers, and re-review on update. The documented attacks hide malicious behavior in auxiliary files precisely because scanners check the obvious ones. The OpenClaw skills-marketplace campaigns and the poisoned-Action incident are the case studies. One asymmetry worth encoding in policy: skills (instructions) may be something agents propose and humans review in via PR; capabilities (servers, credentials) never are: the capability list lives where agents can't write.
Irreversible actions (merges, sends, payments, deletions) get a human approval step; that gate is what makes the whole delegation economy safe (§08). Log every tool call with full arguments into your tracing layer; agent-aware traces are simultaneously your debugging, your cost accounting, and your forensic record. Skim the OWASP LLM Top 10 quarterly; it tracks this threat landscape faster than any book will.
Everything above, composed. This is not a prescription; it's the shape that the cheap-at-scale constraints keep producing, drawn once so the whole playbook fits on one screen.
Beyond the field guide's syllabus: the Anthropic and Sourcegraph context-engineering guides plus the prompt-injection canon apply to every section here (§03 is their applied form); the Agent Skills spec underpins the cross-harness setup from the field guide; add turbopuffer's architecture posts (object-storage search economics), Docling's docs, Plastic Labs' blog (memory as reasoning), Linear's coding-sessions docs, and the OWASP LLM Top 10. For the personal-agent wave, follow the OpenClaw and Hermes repos directly: the READMEs move faster than any article about them.