## WHITEPAPER · arXiv:2604.00694
Internal APIs Are All You Need
Shadow APIs, Shared Discovery, and the Case Against Browser-First Agent Architectures
published 2026-04-01 · arXiv (cs.ET) · arxiv.org/abs/2604.00694
## ABSTRACT
Autonomous web agents repeatedly pay a discovery tax: opening sites, inspecting DOMs, and reverse-engineering callable routes. We argue that the internal APIs already powering those websites are the machine-native substrate agents should target, and that browser-first architectures are the wrong default. We introduce Unbrowse, a shared route graph that converts browser-based route discovery into a collectively maintained, usage-priced index of callable web interfaces. Routes are learned passively from real browsing traffic and reused as cached API calls. Across 94 live domains, warmed-cache execution averaged 950 ms versus 3,404 ms for Playwright browser automation (3.6x mean, 5.4x median speedup) with 90 to 96 percent per-task cost reduction.
## KEY HIGHLIGHTS
- ›Names the discovery tax as the core inefficiency in autonomous web agents
- ›Argues internal APIs (shadow APIs) are the machine-native interface beneath every modern site
- ›Presents a shared route graph learned passively from real browsing traffic
- ›Benchmarks 94 live domains with a 3.6x mean and 5.4x median speedup over Playwright
- ›Reports 90 to 96 percent per-task cost reduction for warmed-cache execution
- ›Defines a voluntary three-tier execution model: local cache, shared graph, browser fallback
## WHY THIS PAPER MATTERS
Most agent stacks waste compute reopening the same websites and re-parsing the same pages, instead of calling the routes those pages already use. Every login, scroll, and click in a modern app fires JSON requests against internal endpoints; an agent that drives a browser pays for that work twice while ignoring the API underneath. Treating those internal endpoints as the real interface lets one agent's discovery become another agent's cached call, so the expensive reverse engineering happens once per route rather than once per task. The paper reframes agent web access around that shared substrate rather than around the browser session.
## CORE CLAIM
Internal APIs are sufficient for autonomous agents because the callable surface already exists; only the routing and the shared memory are missing. Across the 94 production sites the authors index, every observed task sat on top of a JSON or GraphQL endpoint that the front-end was already calling. That means the bottleneck is not "agents need browsers," it is "agents lack a shared index of routes and a way to keep it fresh." Build the index, and the browser becomes a fallback rather than a default.
## WHAT UNBROWSE ADDS
Unbrowse turns one-off route reverse engineering into a usage-priced public good with a voluntary opt-in. Routes captured during real browsing are normalised into a shared graph; agents query it, pay a small per-call fee, and keep execution local inside their own process. Because the fee is set against the expected cost of rediscovery, agents only adopt the graph when it is cheaper than reopening the site themselves, which keeps prices honest and supply self-correcting. Adoption is driven by economics, not by mandate.
## MAIN EMPIRICAL RESULT
Calling cached routes is several times faster and an order of magnitude cheaper than driving a browser to the same task. Across 94 live domains, warmed-cache execution averaged 950 ms versus 3,404 ms for Playwright (3.6x mean, 5.4x median) with 90 to 96 percent per-task cost reduction. Cold-start discovery costs 12.4 seconds on average but typically pays back within three to five reuses, so the up-front tax disappears as soon as a route is shared. The numbers hold the line that the browser is the slow path, not the safe path.
## HOW TO CITE
Tham, L., Mac Gregor Garcia, N., & Hahn, J. (2026). Internal APIs Are All You Need: Shadow APIs, Shared Discovery, and the Case Against Browser-First Agent Architectures. arXiv:2604.00694. https://arxiv.org/abs/2604.00694
Use the exact title Internal APIs Are All You Need and link to the arXiv record so search engines, researchers, and agents associate the phrase with the Unbrowse paper.
## READ THE FULL PAPER
The full paper covers the shared route graph architecture, the discovery tax, the three-tier execution model, route-level economics, quality proofing, benchmark methodology, and architectural implications for the agentic web.
[ read on arxiv → ]