Redfin Scraper: How to Scrape Redfin Listings with Python (2026)

A Redfin scraper turns search results and property pages into structured listing records. This Python example extracts embedded JSON from HTML, follows search pagination, enriches selected homes, and writes a CSV.

In our quick run on October 16, 2026, we collected 82 listings from two search pages and enriched five property records. This is a small snapshot, not a benchmark: it demonstrates one observed page structure and gives a rough traffic estimate. Review the access restrictions below before running automated requests.

What can a Redfin scraper collect?

Our parser collected prices, addresses, URLs, coordinates, room counts, and available floor areas from search pages. Property enrichment added bedrooms, bathrooms, construction year, category, posting date, and a photo count.

The structured data was already inside the downloaded HTML. Search listings used JSON-LD arrays containing a residence and a Product offer; property pages used an object typed Product and RealEstateListing. We did not parse visible listing-card text.

Do not rename rooms to bedrooms or infer status from a price. Keep unknown values empty. Our real estate data scraping guide discusses choosing fields before collection.

How do you build a Redfin scraper in Python?

Fetch HTML, extract its JSON-LD blocks, and map the observed objects into dictionaries. The script below is the version used in our quick run, including the exception retry added after a read timeout.

Use Python 3.12 and install requests with python3 -m pip install requests. Save the script as redfin_scraper.py. Its browser-like headers are ordinary request metadata, not a guarantee of access.

import csv, json, os, random, re, time
import requests

# Residential proxy from environment, e.g. http://USER:[email protected]:823
PROXY = os.environ.get("PROXY_URL")
PROXIES = {"http": PROXY, "https": PROXY} if PROXY else None
HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
                  "(KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    "Accept-Language": "en-US,en;q=0.9",
}
LD_JSON = re.compile(r'<script type="application/ld\+json"[^>]*>(.*?)</script>', re.S)


def fetch(url, retries=3):
    for attempt in range(1, retries + 1):
        try:
            r = requests.get(url, headers=HEADERS, proxies=PROXIES, timeout=30)
        except requests.RequestException as exc:  # timeouts, dropped connections
            print(f"{url} -> {type(exc).__name__}, retry {attempt}")
            time.sleep(5 * attempt)
            continue
        if r.status_code == 200 and "Human Verification" not in r.text[:3000]:
            return r.text
        print(f"{url} -> {r.status_code}, retry {attempt}")
        time.sleep(5 * attempt)
    return None


def parse_search(html):
    """Each listing is a JSON-LD array: [residence, product-with-offer]."""
    rows = []
    for block in LD_JSON.findall(html):
        data = json.loads(block)
        if not isinstance(data, list):
            continue
        home = next((d for d in data if "address" in d), {})
        offer = next((d.get("offers", {}) for d in data if d.get("@type") == "Product"), {})
        url = home.get("url", "")
        m = re.search(r"/home/(\d+)", url)
        rows.append({
            "home_id": m.group(1) if m else None,
            "address": home.get("name"),
            "zip": home.get("address", {}).get("postalCode"),
            "type": home.get("@type"),
            "rooms": home.get("numberOfRooms"),
            "sqft": home.get("floorSize", {}).get("value"),
            "price": int(offer["price"]) if offer.get("price") else None,
            "currency": offer.get("priceCurrency"),
            "lat": home.get("geo", {}).get("latitude"),
            "lng": home.get("geo", {}).get("longitude"),
            "url": url,
        })
    return rows


def parse_property(html):
    """The property page has one RealEstateListing JSON-LD block."""
    for block in LD_JSON.findall(html):
        data = json.loads(block)
        if isinstance(data, dict) and "RealEstateListing" in str(data.get("@type")):
            home = data.get("mainEntity", {})
            return {
                "beds": home.get("numberOfBedrooms"),
                "baths": home.get("numberOfBathroomsTotal"),
                "year_built": home.get("yearBuilt"),
                "category": home.get("accommodationCategory"),
                "date_posted": data.get("datePosted"),
                "photos": len(home.get("image", [])),
            }
    return {}


def scrape_city(city_path, max_pages=2):
    seen, results = set(), []
    for page in range(1, max_pages + 1):
        url = f"https://www.redfin.com{city_path}" + (f"/page-{page}" if page > 1 else "")
        html = fetch(url)
        if not html:
            break
        new = [r for r in parse_search(html) if r["home_id"] and r["home_id"] not in seen]
        if not new:
            break  # stop when a page adds nothing
        seen.update(r["home_id"] for r in new)
        results.extend(new)
        print(f"page {page}: {len(new)} listings")
        time.sleep(random.uniform(3, 6))
    return results


if __name__ == "__main__":
    rows = scrape_city("/city/30749/NY/New-York", max_pages=2)
    for row in rows[:5]:  # enrich a few listings from their property pages
        html = fetch(row["url"])
        row.update(parse_property(html) if html else {})
        time.sleep(random.uniform(3, 6))
    fields = ["home_id", "address", "zip", "type", "rooms", "sqft", "price", "currency",
              "lat", "lng", "url", "beds", "baths", "year_built", "category", "date_posted", "photos"]
    if not rows:
        raise SystemExit("No listings collected; check fetch() output first.")
    with open("redfin_listings.csv", "w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=fields, restval="")
        writer.writeheader()
        writer.writerows(rows)
    with open("redfin_listings.json", "w", encoding="utf-8") as f:
        json.dump(rows, f, ensure_ascii=False, indent=2)
    print(f"saved {len(rows)} listings")

Set PROXY_URL securely in your environment, then run python3 redfin_scraper.py. Without that variable, requests go directly from your machine. Never commit a real proxy URL containing credentials.

To inspect one property, pass its downloaded HTML to parse_property(); merge the resulting dictionary into its search record. Our Bayside example returned 3 beds, 2.5 baths, year 1970, posting date 2026-10-15, and 34 photo references.

The regex deliberately matches the markup we observed. Different attribute order, malformed JSON, or changed object types can break it.

How do you scrape Redfin search results and pagination?

Start with the search path, fetch successive pages, and stop when fetching fails, no unseen IDs appear, or the page budget is exhausted. Our run used the New York city path and its page-2 variant.

The parser extracts URLs from residence objects and takes the numeric suffix after /home/ as home_id. Treat this as a home identifier, not a guaranteed identifier for every listing event.

The seen set removes homes encountered on earlier pages. It does not remove duplicates within the same page, because the set updates after the list is built. Deduplicate the final export by home ID, and store a collection timestamp separately when tracking changes.

For a ZIP search, inspect that search’s actual pagination before adapting the path. Our two city pages returned 41 records each; that is an observation, not a universal page size. An empty parse can mean changed markup or a challenge page, not necessarily the end of inventory.

You may encounter references to Stingray endpoints. Redfin’s robots file names that path, but this does not establish an official API contract. We could not verify public listings API documentation or a published endpoint rate limit from Redfin, so this tutorial uses the HTML route and assumes no endpoint stability.

How do you export Redfin data to CSV or JSON?

Use a fixed column list, keep missing values empty, and deduplicate before export. The script above already does this: it writes the same 17 columns in a fixed order to both redfin_listings.csv and redfin_listings.json, deduplicates on home_id while paging, and stops with a message instead of writing an empty file. An earlier draft of our script built the CSV header from the first row, which duplicated the enrichment columns once that row had been enriched; a fixed list avoids that.

Keep ZIP codes as strings and fractional bathrooms as numbers. Do not replace absent square footage with zero: 15 of our 82 records lacked it. Only five records were enriched, so blank bedrooms on other rows mean enrichment was not attempted.

Our quick run recorded examples including 3818 223rd St at $1,050,000 and 731 Greenwich St #2 at $2,000,000. Validate a few rows against their source pages before importing the whole export.

Why does Redfin block scrapers, and how do proxies help?

A proxy changes the network path; it does not fix broken parsing or grant permission. Our direct datacenter request returned HTTP 405 with a roughly 2 KB Human Verification page, while the same URL through a DataImpulse rotating residential US proxy returned HTTP 200 and roughly 1.8 MB of HTML.

That comparison came from our quick run on October 16, 2026: a Linux server, Python 3.12, requests, public pages only, no login, and pauses of 3-6 seconds. The collection covered two search pages and five property pages. It cannot establish a success rate or isolate why access changed.

The first attempt stopped on a read timeout. The revised fetch function catches request exceptions and waits 5, 10, then 15 seconds between attempts. Those are linear backoff delays, not exponential backoff. It stops after three attempts, but its challenge check only examines the beginning of the response.

For recurring 403, 429, or CAPTCHA responses, pause and investigate rather than repeatedly cycling addresses. See our guides to 403 scraping errors and request rate limiting. Keep concurrency low and honor an explicit retry delay.

Choose rotation for independent requests and sticky sessions when a workflow needs continuity. Our real estate proxy guide covers selection. DataImpulse is proxy infrastructure, not a scraping API, managed unblocker, or VPN.

How much does it cost to scrape 1,000 Redfin pages?

Using our approximate HTML sizes and residential traffic from $1 per GB, 1,000 property pages work out to about $1.10 in proxy traffic. Collecting 1,000 search records is a different workload and was roughly $0.05 under our observed page density.

These decimal-unit estimates cover HTML only. Retries, headers, rendered assets, and enrichment add traffic; engineering and compute add separate costs. Page sizes were approximate observations from a small snapshot, not precise billed-byte measurements.

Search collection can be economical when its fields are sufficient. Fetching every property for bedrooms, bathrooms, or posting dates changes the cost substantially.

Should you use a ready-made Redfin scraper or API instead?

Consider a service when maintaining collection and parsing costs more than its fee. Compare pricing units and actual output schemas before treating two offers as equivalent.

The official pages below list these prices and allowances (as of October 2026). Formats refer to the named product or general scraping endpoint; a credit is not necessarily one completed listing.

Another option is Redfin’s Download All export: its cap is 350 homes (as of October 2026), availability depends on local MLS rules, and searches below 20 homes lack the option. For regional statistics, the Data Center offers market aggregates rather than individual listing records.

For alternatives to scraping Redfin, evaluate licensed real estate data APIs. Ask about geographic coverage, update cadence, history, and redistribution rights before choosing a source.

Is it legal to scrape Redfin?

Public visibility is not blanket permission to collect or reuse listings. Redfin’s Terms of Use prohibit automated access without prior express written permission, stating: “You may not automatedly crawl or query the Services for any purpose or by any means”.

The wildcard group in Redfin’s robots.txt disallows paths including /stingray/ and /myredfin/, with specific exceptions. Robots rules are crawling instructions, not legal permission; RFC 9309 explicitly distinguishes them from access authorization.

The Ninth Circuit’s 2022 hiQ v. LinkedIn opinion addressed public-page access under the US Computer Fraud and Abuse Act at the preliminary-injunction stage. It did not grant universal scraping permission or eliminate contract and copyright risk.

The US Copyright Office distinguishes facts from protected expression. Photos and descriptive text can raise different issues from factual prices; MLS restrictions and intended reuse also matter. For commercial collection, obtain advice about your specific workflow and jurisdiction. The decision is yours: check the law where you operate.

5-step Redfin scraping checklist

Use this 5-step Redfin scraping checklist to keep collection bounded and exports interpretable. Repeat validation whenever the response structure changes.

• Step 1: Fields. Define required columns and distinguish rooms, bedrooms, status, and missing values.
• Step 2: Pacing. Set a page budget, pause between requests, and stop persistent failures.
• Step 3: Proxies. Select an appropriate network path and keep credentials outside source code.
• Step 4: Validation. Check HTML, parsed IDs, duplicate records, and several source-page values.
• Step 5: Export. Use unique headers, preserve nulls, and record collection time separately from posting date.

Need a proxy connection for your Python collector?

If network access is the missing part of your authorized workflow, explore DataImpulse residential proxies, starting at $1 per GB. Begin with a small sample and measure traffic and field completeness before increasing volume.

Last updated: October 16, 2026.