Whitepaper
Internal APIs Are All You Need
Shadow APIs, Shared Discovery, and the Case Against Browser-First Agent Architectures
Abstract
Autonomous agents increasingly interact with the web, yet most websites remain designed for human browsers. Internal APIs Are All You Need presents Unbrowse, a shared route graph that transforms browser-based route discovery into a collectively maintained index of callable first-party interfaces, making direct API reuse faster and less brittle than browser-first automation.
Key highlights
- Argues that the web's first-party internal APIs, not browser UIs, are the real machine-native interface layer
- Presents a shared route graph learned passively from real browsing traffic and reused via direct API calls
- Benchmarks equivalent information-retrieval tasks across 94 domains with 3.6× mean and 5.4× median speedup over Playwright
- Reports fully warmed cached execution at 950 ms on average versus 3,404 ms for browser automation
- Shows well-cached routes completing in under 100 ms on the same host
- Defines a voluntary three-path model: local cache, shared graph, or browser fallback
Why this paper matters
Most web-capable agents still pay a discovery tax every time they touch a site: open the page, inspect the DOM, infer the request flow, retry when the UI changes. The paper argues that this is the wrong default. Modern sites already expose callable first-party interfaces behind their UIs; the missing layer is shared discovery and reuse.
Core claim
Internal APIs are all you need because the bottleneck is not interface existence. It is the repeated private cost of rediscovering those interfaces agent by agent. Unbrowse turns that redundant work into a shared route graph with a clean outside option when the graph is not good enough.
What Unbrowse adds
Unbrowse passively indexes callable web interfaces from real traffic, serves cached routes through direct API execution, and falls back to live browser capture only when needed. The paper also sketches the route-economy layer around this graph: search fees, one-time skill installation fees, optional site-owner execution fees, and delta-based contributor attribution via x402.
Main empirical result
Across 94 domains, fully warmed cached execution averaged 950 ms versus 3,404 ms for Playwright browser automation, with a 3.6× mean speedup and 5.4× median speedup. The paper also reports well-cached routes finishing in under 100 ms and frames cold-start discovery as an upfront cost that amortises across reuse.
Canonical citation target
If you reference this work, use the exact title Internal APIs Are All You Need and link to this canonical page so search engines, researchers, and agents associate the phrase with the Unbrowse paper.
Read the full paper
The full paper covers the discovery tax, shared route graph architecture, the three-path execution model, route-level economics, quality proofing, benchmark methodology, and the broader case against browser-first agent architectures.
Open the PDF →