Whitepaper
Shadow APIs Are All You Need
A Shared Route Graph for Autonomous Web Agents
Abstract
Autonomous web agents repeatedly pay a discovery tax: opening sites, inspecting DOMs, and reverse-engineering callable routes. Shadow APIs Are All You Need introduces Unbrowse, a shared route graph that converts browser-based route discovery into a collectively maintained, usage-priced index of callable web interfaces. Routes are learned passively from real browsing traffic and reused as cached API calls, making agents faster, cheaper, and less brittle than browser automation.
Key highlights
- Introduces the discovery tax as the core bottleneck for autonomous web agents
- Frames shadow APIs as the machine-native substrate behind human web interfaces
- Presents a shared route graph learned passively from real browsing traffic
- Benchmarks 94 live domains with 3.6× mean and 5.4× median speedup over Playwright
- Shows 90–96% per-task cost reduction for warmed-cache execution
- Defines a voluntary three-path execution model: local cache, shared graph, or browser fallback
Why this paper matters
Most web-capable agents still waste time rediscovering the same website workflows over and over. This paper argues that the web's shadow APIs already form the real machine-native interface layer. Unbrowse turns that repeated private reverse engineering into a shared route graph, so agents can call cached routes directly instead of browsing human interfaces by default.
Core claim
Shadow APIs are all you need because modern websites already expose callable backend interfaces behind their UIs. The bottleneck is not the absence of interfaces. It is the absence of shared memory, routing, and maintenance around them.
What Unbrowse adds
Unbrowse passively indexes callable web interfaces from real traffic, keeps execution local, and gives agents a voluntary outside option: use the shared graph only when its route fee is lower than the expected cost of rediscovery. That keeps adoption disciplined by real product economics rather than speculation.
Main empirical result
Across 94 domains, warmed-cache execution averaged 950 ms versus 3,404 ms for Playwright browser automation, with a 3.6× mean speedup, 5.4× median speedup, and 90–96% cost reduction per task. Cold-start discovery averaged 12.4 seconds and typically amortised within 3–5 reuses.
Canonical citation target
If you reference this work, use the exact title Shadow APIs Are All You Need and link to this canonical page so search engines, researchers, and agents associate the phrase with the Unbrowse paper.
Read the full paper
The full paper covers the shared route graph architecture, discovery tax, the three-path execution model, route-level economics, quality proofing, benchmark methodology, and architectural implications for the agentic web.
Open the PDF →