Blog
Articles
Shadow APIs, browser automation, agent economics, and the agentic web.
Skip the Browser from Claude Desktop
Unbrowse is now a real MCP server. One JSON block in your config and your agent gets the web - without a browser.
Lewis Tham
3 Lines to Skip the Browser
Unbrowse v2.12.0 ships a first-party TypeScript SDK. If you're building on top of the local Unbrowse runtime, you no longer need to shell out to the CLI - just import, resolve, and execute.
Lewis Tham
v3: The Browser Is Now the Fallback
For two years the default answer to "how does an AI agent interact with a website?" has been "spin up a headless browser." That answer is expensive, fragile, and slow. With Unbrowse v3 we are making it official: the structured API is the primary path, and the browser is what you reach for when nothing else works.
Lewis Tham
Why We Wrote a Paper Against Browser Automation
Every browser automation agent does the same thing: spin up a headless browser, click through page elements, wait for renders, retry on failure, and burn tokens narrating what it sees. We spent the last year asking: what if most of that work is unnecessary?
Lewis Tham
Unbrowse 3.0: From Browser Tool to Agentic Infrastructure
Unbrowse 3.0 ships DAG workflow publishing, a publish review pipeline, MCP overhaul, and a full browser proxy mode. The wedge was speed. The cathedral is the shared route graph. This release builds the bridge.
Lewis Tham
Unbrowse vs Playwright: The Full Benchmark (94 Domains)
Detailed benchmark results from the arXiv paper comparing Unbrowse API calls vs Playwright browser automation across 94 domains with site-by-site breakdown.
Lewis Tham
Web Data Extraction Without Scraping: The API-First Approach
Why scraping is an unnecessary tax when websites already expose their data via internal APIs. The API-first paradigm for web data extraction.
Lewis Tham
Shadow API Discovery: A Step-by-Step Tutorial
Learn how to discover hidden shadow APIs behind any website. A practical step-by-step tutorial using Unbrowse to find, document, and call internal API endpoints.
Lewis Tham
MCP Browser Servers Compared: Speed, Tokens, and Reliability
Head-to-head benchmark of Playwright MCP, Unbrowse MCP, Browserbase MCP, and Firecrawl MCP across speed, token efficiency, and reliability.
Lewis Tham
AI Agent Web Access: The Complete Guide for 2026
The definitive guide to giving AI agents web access in 2026. Compare browser automation, web scraping, and API discovery with cost analysis and tool recommendations.
Lewis Tham
Top 10 Selenium Alternatives in 2026
Ten Selenium alternatives that solve the problems WebDriver cannot -- from AI-native agents to API-first approaches that skip the browser entirely.
Lewis Tham
Best Open Source Browser Automation in 2026
A comprehensive guide to every major open-source browser automation tool in 2026, from API discovery engines to LLM-driven browser agents.
Lewis Tham
5 Best Tools for Extracting API Data From Websites
Discover the top tools for reverse-engineering and extracting API data from websites, from automated shadow API discovery to manual traffic proxies.
Lewis Tham
6 Best Browser Automation Tools That Actually Work in 2026
A head-to-head comparison of the six browser automation tools that actually deliver in 2026, from AI-native agents to API-first approaches.
Lewis Tham
8 Best Web Scraping Tools for AI in 2026
Compare the top web scraping and data extraction tools for AI agents in 2026, from API discovery to managed proxies to open-source crawlers.
Lewis Tham
Replace Playwright in Your Agent Pipeline in 5 Minutes
Migrate from Playwright to Unbrowse in 5 minutes. Before/after code comparisons, performance benchmarks (3.6x faster), and a step-by-step migration guide for AI agent pipelines.
Lewis Tham
How to Access Product Hunt Data Without Scraping
Learn how to access Product Hunt launch data, upvotes, comments, and maker profiles through shadow APIs discovered from browsing traffic — no scraping or official API approval required.
Lewis Tham
How to Access Twitter Spaces Data Programmatically
Learn how to access Twitter/X Spaces data including live audio rooms, speaker lists, and listener counts through shadow APIs discovered from browsing traffic.
Lewis Tham
How to Access Craigslist Data Without Scraping
Learn how to access Craigslist listing data, search results, and classified ads through shadow APIs discovered from browsing traffic — no HTML parsing or scraping required.
Lewis Tham
Automate Job Applications Without Selenium
Use Unbrowse to interact with job boards like LinkedIn, Indeed, and Glassdoor through their shadow APIs. Skip the brittle Selenium scripts and call the real endpoints directly.
Lewis Tham
How to Access Stripe Dashboard Data Programmatically
Learn how to access Stripe dashboard analytics, charts, and internal reports through shadow APIs discovered from browsing traffic — data the official API does not expose.
Lewis Tham
Monitor Competitor Pricing With 3 Lines of Code
Track competitor prices across any e-commerce site with just 3 lines of code. Unbrowse discovers the internal pricing APIs automatically — no scrapers, no selectors, no maintenance.
Lewis Tham
5 Best Playwright Alternatives for AI Agents
Looking for Playwright alternatives that work better with AI agents? Compare Unbrowse, Stagehand, Browser Use, Steel, and Browserless for agent-first browser automation.
Lewis Tham
How to Access Notion Data Without the Official API
Learn how to access Notion pages, databases, and workspace data through shadow APIs discovered from browsing traffic — no official API integration or token setup required.
Lewis Tham
7 Best MCP Servers for Web Scraping in 2026
Compare the top MCP servers for web scraping in 2026. Detailed breakdown of Unbrowse MCP, Playwright MCP, Firecrawl MCP, Puppeteer MCP, and more for Claude and AI agent workflows.
Lewis Tham
Build an AI Research Assistant That Reads the Whole Web
Build an AI research assistant that collects data from multiple websites, extracts structured information via shadow APIs, and produces comprehensive summaries. No scraping frameworks needed.
Lewis Tham
How to Access Discord Data Without the Bot API
Learn how to access Discord server data, messages, and channels through shadow APIs discovered from browsing traffic — no bot tokens, no scraping required.
Lewis Tham
10 Best AI Browser Agents in 2026
A comprehensive comparison of the top AI browser agents in 2026, including Browser Use, Stagehand, Unbrowse, AgentQL, Skyvern, and more. Find the right tool for your AI automation needs.
Lewis Tham
How to Access Pinterest Data Without Scraping
Learn how to access Pinterest pin data, boards, and search results through shadow APIs discovered from browsing traffic — no scraping, no developer tokens required.
Lewis Tham
Build a Price Monitoring Agent in 10 Lines
Build an automated price monitoring agent that tracks Amazon and Walmart prices using Unbrowse's shadow API discovery. No scraping, no browser automation — just clean API calls discovered from real browsing traffic.
Lewis Tham
How to Access eBay Data Without Scraping
Learn how to access eBay listing data, pricing, and search results through shadow APIs discovered from browsing traffic — no scraping, no official API keys required.
Lewis Tham
The Agentic Browser Wars: 2026 Landscape Map
A comprehensive map of the 2026 agentic browser landscape. Categorizes and compares Browser Use, Stagehand, Browserbase, Steel, Browserless, Playwright, Puppeteer, Selenium, Firecrawl, Crawl4AI, Unbrowse, AgentQL, and Skyvern across five categories.
Lewis Tham
The $0.53 Tax: What Browser Automation Really Costs Per Action
A breakdown of the real cost of browser automation for AI agents: compute, LLM tokens for DOM processing, and latency opportunity cost. Compared to API-native alternatives.
Lewis Tham
The arXiv Paper Explained: Internal APIs Are All You Need
An accessible summary of the Unbrowse research paper (arXiv:2604.00694) for developers. Key findings: 3.6x mean speedup over browser automation, 5.4x median, 94 domains tested, sub-100ms cached routes.
Lewis Tham
What Is a Shadow API? The Complete Developer's Guide
A comprehensive guide to shadow APIs: the internal, undocumented endpoints that power every modern website. Learn what they are, how to find them, and why they matter for AI agents and browser automation.
Lewis Tham
How Unbrowse Works: Architecture of an API Discovery Engine
A complete technical walkthrough of Unbrowse's architecture, from passive traffic capture through endpoint extraction, authentication detection, graph construction, and cache-first resolution.
Lewis Tham
Your Agent Fleet Can Fund Itself: The Economics of API Mining
A detailed economic analysis of how running AI agent fleets with Unbrowse generates revenue through API route mining, with breakeven calculations and scaling projections.
Lewis Tham
x402 Micropayments: How AI Agents Pay for API Access
A technical explainer of the x402 micropayment protocol, how it enables AI agents to pay for API access in real time, and how Unbrowse uses it to create a self-sustaining marketplace for shadow API routes.
Lewis Tham
The 50 Most Valuable Domains to Mine With Unbrowse
A ranked guide to the highest-value websites for API mining with Unbrowse. Discover which domains yield the most endpoints, earn the most micropayments, and offer first-mover bonuses.
Lewis Tham
Mine the Internet: A Developer's Guide to Earning With Unbrowse
Learn how to earn USDC micropayments by using Unbrowse normally. Every API route you discover gets indexed to the marketplace, and every time another agent uses your route, you get paid.
Lewis Tham
CrewAI + Unbrowse: Build Agents That Skip the Browser
Tutorial for integrating Unbrowse with CrewAI to give your agent crews direct API access to websites. Skip browser automation entirely with cached API routes.
Lewis Tham
How to Give Your LangChain Agent Direct API Access
Tutorial showing how to use Unbrowse as a LangChain tool to replace browser-based web access with direct API calls. Includes code examples, setup, and performance comparison.
Lewis Tham
How to Set Up Unbrowse MCP Server for Claude Desktop
Step-by-step guide to install and configure Unbrowse MCP server for Claude Desktop. Covers installation, MCP configuration, testing with example resolves, and troubleshooting.
Lewis Tham
Best MCP Server for Web Browsing in 2026
A ranked comparison of the best MCP servers for web browsing in 2026, covering Unbrowse, Playwright, Browserbase, Firecrawl, Chrome DevTools, AgentQL, and Puppeteer MCP servers.
Lewis Tham
How to Access Airbnb Listing Data Without Scraping
Airbnb has no public API for listing data. Their website uses internal GraphQL and REST endpoints to serve search results, pricing, availability, and reviews. Unbrowse discovers these shadow APIs from real browsing sessions and makes them callable.
Lewis Tham
How to Access Netflix Catalog Data Programmatically
Netflix has no public API. Their catalog data -- titles, genres, ratings, cast, and availability by region -- is served through internal APIs that power the Netflix web interface. Unbrowse discovers these endpoints from real browsing sessions.
Lewis Tham
How to Access Instagram Data Without Scraping
Instagram's Graph API requires Facebook business verification and approved use cases. Shadow APIs offer direct access to public profiles, posts, reels, and hashtag data through the same internal endpoints Instagram's web app uses.
Lewis Tham
How to Access TikTok Data Without the Official API
TikTok's official API is restricted to approved partners with weeks-long review processes. Shadow APIs -- the internal endpoints TikTok's web app calls -- provide access to video metadata, user profiles, trending content, and search results without partner approval.
Lewis Tham
How to Access StackOverflow Data Without Rate Limits
StackOverflow's public API caps you at 300 requests per day without authentication. Shadow APIs -- the internal endpoints powering the StackOverflow frontend -- offer higher throughput and richer data than the official API ever provided.
Lewis Tham
How to Access Shopify Store Data via Shadow APIs
Shopify's official APIs require store owner approval and OAuth credentials. Shadow APIs -- the internal endpoints every Shopify storefront calls -- let you access product catalogs, pricing, inventory status, and collection data from any public store without authentication.
Lewis Tham
How to Access Zillow Real Estate Data Without Scraping
Zillow's official API was shut down in 2021, and scraping their site triggers aggressive anti-bot defenses. Shadow APIs offer a third path: the same internal endpoints Zillow's own frontend uses, discovered automatically from real browsing traffic.
Lewis Tham
How to Access Google Search Results Programmatically
Google Custom Search API costs $5 per 1,000 queries. Learn how to discover Google's internal search endpoints from browsing traffic using Unbrowse — structured SERP data without per-query costs.
Lewis Tham
How to Access HackerNews Data in Real-Time
HN's Firebase API is slow and limited. Learn how to discover Hacker News's internal endpoints for stories, comments, and search using Unbrowse — real-time data without polling delays.
Lewis Tham
How to Access GitHub Data Faster Than the REST API
GitHub's REST API is rate-limited to 5,000 requests per hour. Learn how to discover GitHub's internal APIs from browsing traffic using Unbrowse — faster data access without rate limit constraints.
Lewis Tham
How to Access YouTube Data Without the Official API
YouTube's official API has strict quotas that run out fast. Learn how to discover YouTube's internal APIs from browsing traffic using Unbrowse — video data, search results, and comments without quota limits.
Lewis Tham
How to Access Amazon Product Data via Shadow APIs
Amazon blocks scrapers and charges for its Product Advertising API. Learn how to discover Amazon's internal product APIs from browsing traffic using Unbrowse — pricing, reviews, and availability data without scraping.
Lewis Tham
How to Access LinkedIn Data Without a Browser
LinkedIn blocks scrapers aggressively. Learn how to discover LinkedIn's internal APIs from your authenticated browsing session using Unbrowse — structured profile and job data without automation detection.
Lewis Tham
How to Access Twitter/X Data Without Scraping
X API costs $100/month minimum. Learn how to discover Twitter's internal GraphQL APIs from browsing traffic using Unbrowse — structured data without paid API access or fragile scraping.
Lewis Tham
How to Access Reddit Data Without Scraping
Reddit killed free API access with $100/month minimum pricing. Learn how to discover Reddit's internal shadow APIs from browsing traffic using Unbrowse — no scraping, no paid API keys.
Lewis Tham
Playwright MCP vs Unbrowse MCP: Which Should Your Agent Use?
A detailed comparison of Playwright MCP and Unbrowse MCP for AI agent web browsing. Covers architecture, speed, token cost, authentication, and when to use each.
Lewis Tham
Unbrowse vs Bright Data: Cloud Proxy vs Native API Calls
Bright Data powers the web's presentation layer. Unbrowse bypasses it entirely. Compare proxy-based data collection vs API-native resolution for AI agents.
Lewis Tham
Unbrowse vs Apify: Actor-Based Scraping vs Route Caching
Apify industrialized web scraping. Unbrowse makes scraping obsolete for data retrieval. Compare actor-based scraping platforms vs API-native route caching.
Lewis Tham
Unbrowse vs Scrapy: Traditional Scraping vs API-First
Scrapy parses the web's presentation layer. Unbrowse taps the data layer directly. Compare Python scraping frameworks vs API-native resolution for modern web data.
Lewis Tham
Foundry: Turn Your AI Workflow History Into Reusable Skills
Foundry mines your local AI coding assistant history to discover repeated workflow patterns and turns them into portable, installable skills. Open source, works with Claude Code, Codex, and OpenClaw.
Lewis Tham
Unbrowse vs Browser Use: Which AI Browser Agent Wins?
Browser Use teaches AI to use browsers. Unbrowse teaches AI to skip them. Compare LLM-driven browser control vs API-native resolution — 3.6x faster, 40x fewer tokens.
Lewis Tham
Unbrowse vs Steel: Open Source Browser Infra Compared
Steel provides open-source cloud browser fleets. Unbrowse provides API-native intelligence. Compare open source browser infrastructure against API discovery for AI agents.
Lewis Tham
Unbrowse vs Firecrawl: Scraping vs API Discovery
Firecrawl scrapes HTML into markdown. Unbrowse discovers the APIs behind pages. Compare web scraping against API discovery for AI agent data retrieval.
Lewis Tham
Unbrowse vs Stagehand: AI SDK vs API Discovery
Stagehand adds AI to browser automation. Unbrowse removes the browser entirely. Compare AI-powered browser SDKs against API discovery for agent web interaction.
Lewis Tham
Unbrowse vs Browserbase: Infrastructure vs Intelligence
Browserbase makes browsers better. Unbrowse makes browsers unnecessary. Compare cloud browser infrastructure vs API-native resolution for AI agents.
Lewis Tham
Unbrowse vs Selenium: Why Browser Automation Lost
Selenium automates browsers. Unbrowse eliminates the need for them. Compare browser automation vs API-native resolution for AI agents — 3.6x faster, 40x fewer tokens.
Lewis Tham
Browser Automation in 2026: The Complete Guide
An evergreen guide covering every browser automation approach in 2026: Selenium, Playwright, Puppeteer, Browser Use, Stagehand, Firecrawl, and Unbrowse. Covers use cases, performance, and when to use each tool.
Lewis Tham
How to Access Walmart Product Data via Shadow APIs
Walmart's Affiliate API has limited product data and strict approval requirements. Shadow APIs -- the internal endpoints powering walmart.com -- provide complete product details, real-time pricing, inventory status, and reviews without affiliate approval.
Lewis Tham
test
test
Lewis Tham
Browser Automation Is Dead. Here's What Replaces It.
Every AI agent web action pays a hidden $0.53 tax -- the cost of launching a browser, rendering pixels, and converting structured data back into structured data. Across 94 domains, direct API calls achieved 3.6x mean speedup, 106x cost reduction, and eliminated 500 MB RAM per instance.
Lewis Tham
Every MCP Server Is a Mass Hallucination
There are 10,000+ MCP servers on GitHub. Each is a hand-written wrapper around one API. But most websites don't have official APIs, and the ones that do change constantly. The entire approach doesn't scale.
Lewis Tham
Shadow APIs: The Hidden API Layer Every Website Already Has
Every modern website calls internal API endpoints behind its UI. These shadow APIs are not documented, not public, but fully functional. Unbrowse discovers them by intercepting network traffic during normal browsing.
Lewis Tham
94 Domains, 100% Win Rate: The Full Benchmark
We tested Unbrowse against Playwright on every major website category. Browser automation lost every time.
Lewis Tham
Proof of Indexing: The Consensus Mechanism for the Agentic Web
Bitcoin burns electricity to secure transactions. Unbrowse burns browsing effort to secure routes. One built a ledger. The other is building the index.
Lewis Tham
Google Indexed the Web for Humans. Who Indexes It for Agents?
Google captured $2T in value by indexing HTML pages for human eyeballs. The agentic web needs its own index. Not an index of pages, but an index of machine-readable API routes. Unbrowse is building it, collectively, through proof of indexing.
Lewis Tham
Your Personal Agent Is 3.6x Slower Than It Should Be
Personal AI agents spend 80% of their time waiting for web pages to load — rendering pixels they will never see, parsing DOM they do not need, burning API credits on vision tokens. The Unbrowse plugin for OpenClaw gives your agent direct access to those APIs. 3.6x faster, 106x cheaper, zero browser overhead.
Lewis Tham
Your OpenClaw Agent Can Earn Money While It Works For You
You already use your OpenClaw agent to browse the web, search, book things, and research. With one plugin, every web interaction your agent makes starts earning you USDC.
Lewis Tham
Mine the Internet
The agentic web needs a new kind of index — not HTML pages, but machine-readable API routes. Unbrowse turns normal web browsing into mining: every site you visit contributes routes to a shared graph. When AI agents use those routes, you earn USDC micropayments.
Lewis Tham
The 50 Most Valuable Domains to Mine (And Why)
Not all domains are equal for mining. We benchmarked 94 domains in our paper — 61 had no bot detection at all, and even WAF-protected sites yielded a 2.1x speedup over headless browsers. This is the definitive list of the 50 most valuable domains to mine with Unbrowse.
Lewis Tham
Your Agent Fleet Can Fund Itself
If you're running 10, 50, or 100 AI agents that interact with websites, each one is a cost center burning $0.53 per browser action. With Unbrowse, every agent passively discovers API routes that get shared to a marketplace.
Lewis Tham
Internal APIs Are All You Need
Autonomous agents increasingly interact with the web, yet most websites remain designed for human browsers. Internal APIs Are All You Need presents Unbrowse, a shared route graph that transforms browser-based route discovery into a collectively maintained index of callable first-party interfaces.
Lewis Tham
Shadow APIs Are All You Need
Redirects to Internal APIs Are All You Need — the canonical paper page.
Lewis Tham