How to Access Craigslist Data Without Scraping

Craigslist is one of the oldest and most heavily trafficked classified ad platforms on the internet. With listings for housing, jobs, services, vehicles, and goods across hundreds of cities, it contains a massive volume of structured data. Real estate aggregators, job boards, market researchers, and price comparison tools all need Craigslist data.

But Craigslist has never offered a public API. The site is famously minimalist — plain HTML, no JavaScript framework, no official developer program. For decades, the only option was HTML scraping, and Craigslist has fought scrapers aggressively with legal action, IP blocking, CAPTCHAs, and structural changes designed to break parsers.

Here is what most people miss: even Craigslist has evolved. Modern Craigslist pages now make internal API calls for search results, gallery views, map data, and listing details. These shadow APIs return structured JSON and are far more reliable than parsing HTML. Unbrowse discovers them automatically from normal browsing.

What Are Shadow APIs?

Shadow APIs are the internal HTTP endpoints that web applications call from the browser to fetch and display data. They exist because modern web features like dynamic search, map views, and infinite scroll require structured data from the server.

When you search for apartments on Craigslist, the browser makes requests to:

• A search endpoint that returns listings matching your query as structured data with titles, prices, locations, and posting dates
• A map endpoint that returns geocoded listing data for the map view
• A gallery endpoint that returns image URLs and thumbnails for gallery browsing
• A listing detail endpoint that returns the full posting with description, attributes, and contact method
• A nearby endpoint that returns similar listings from adjacent regions

These endpoints predate Craigslist's visual redesign and have been gradually expanded. They return data that is much cleaner than what you get from parsing the HTML.

What Unbrowse Discovers on Craigslist

Unbrowse captures all HTTP traffic during browsing and indexes every API route. When you browse Craigslist through Unbrowse, it discovers:

• Search result endpoints — listings matching a query with price, location, date, and category metadata
• Listing detail endpoints — full posting content including description, images, attributes, and posting metadata
• Map data endpoints — geocoded listings for map-based search
• Gallery endpoints — image galleries with thumbnails and full-size URLs
• Category endpoints — the category tree for a specific city or region
• Nearby region endpoints — listings from adjacent Craigslist regions

Each endpoint is stored with its URL pattern, required parameters, and response format.

Getting Started

Install Unbrowse:

npm install -g unbrowse

Browse Craigslist to discover its internal APIs:

unbrowse browse "https://sfbay.craigslist.org/search/apa#search=1~gallery~0~0"

Unbrowse opens a real browser, captures the API traffic as you browse, and indexes the endpoints.

Resolve a discovered endpoint:

unbrowse resolve "craigslist search listings"

Execute the route:

unbrowse execute "craigslist search listings" --params '{"region": "sfbay", "category": "apa", "query": "2br", "maxPrice": 3000}'

Code Example: Craigslist Housing Monitor

Here is a Node.js script that monitors Craigslist for new apartment listings matching specific criteria:

import Unbrowse from 'unbrowse';

const unbrowse = new Unbrowse();

// Resolve Craigslist search endpoint
const searchSkill = await unbrowse.resolve('craigslist apartment search');

// Search for apartments with specific filters
const results = await unbrowse.execute(searchSkill, {
  region: 'sfbay',
  category: 'apa',
  minPrice: 1500,
  maxPrice: 3000,
  bedrooms: 2,
  hasPic: true,
  postedToday: true
});

// Process new listings
for (const listing of results.items) {
  console.log(`$${listing.price}/mo — ${listing.title}`);
  console.log(`  Location: ${listing.neighborhood || listing.city}`);
  console.log(`  Posted: ${listing.datetime}`);
  console.log(`  URL: ${listing.url}`);
  console.log(`  Images: ${listing.images?.length || 0}`);
  console.log();
}

console.log(`Found ${results.items.length} listings matching criteria`);

No HTML parsing. No CAPTCHA solving. No IP rotation. Just structured data from Craigslist's own internal API.

Performance Comparison

Method	Setup Time	Avg Response Time	Structured Data	CAPTCHA Risk
HTML Scraping (requests)	30-60 min	1-3 seconds	Requires parsing	High
HTML Scraping (Playwright)	30-60 min	3-8 seconds	Requires parsing	Medium
RSS Feeds	5 min	500ms-1s	Partial	None
Unbrowse (Shadow API)	2 minutes	100-500ms	Yes (native JSON)	Low

Craigslist's RSS feeds are an option for basic listing monitoring, but they lack the filtering capabilities, image data, and detailed attributes that the internal API provides. Unbrowse gives you the full dataset.

Why Craigslist Is Harder Than It Looks

Craigslist has a reputation as a simple HTML site, but scraping it reliably is surprisingly difficult:

• IP blocking — Craigslist aggressively blocks IPs that make too many requests
• CAPTCHAs — automated access triggers CAPTCHAs that require human intervention
• Regional fragmentation — each city is a separate subdomain with slightly different markup
• Layout changes — Craigslist periodically changes its HTML structure without notice
• Legal threats — Craigslist has sued multiple scraping companies

The shadow API approach avoids most of these issues. API calls from a real browser session with real cookies look identical to normal browsing. The JSON response format does not change when the HTML layout does.

FAQ

Is this legal? Unbrowse makes the same API calls your browser makes during normal browsing. It does not bypass CAPTCHAs, forge headers, or violate access controls. Review Craigslist's Terms of Use for your specific application.

Do I need a Craigslist account? No. Craigslist's search and listing data are publicly accessible. No account or login is needed.

Can I search across multiple regions? Yes. Each Craigslist region has the same API structure. You can call the search endpoint for different regions by changing the region parameter.

How do I handle rate limits? Craigslist's internal API is rate-limited per IP. Space your requests reasonably. Unbrowse caches route definitions, so you only need one browse session to discover the endpoints, and then individual API calls are lightweight.

Does this work for all Craigslist categories? Yes. The API endpoints support all categories — housing, jobs, for sale, services, community, and gigs. The parameters vary by category, and Unbrowse captures the specific schema for each one you browse.