Python property collection workflow showing listing extraction, validation and CSV export

A Rightmove scraper can extract listing fields from public-page HTML and export them with Python. The example below reproduces our small collection script, including the search JSON parser and property-page decoder that worked in our quick run on October 19, 2026.

Technical access is separate from permission. Rightmove’s terms of use prohibit automated collection and set out requirements for written consent. Establish authorization before running a collection project; a successful response does not establish that authorization.

Start with two search pages, inspect the output, and keep the limits visible. Our measurements describe a small snapshot, not a benchmark or a prediction of future access.


Key Facts

  • Our quick run on October 19, 2026 made 19 requests to rightmove.co.uk.
  • Each route’s script run returned 50 unique listing IDs.
  • All 50 listings in each route’s export had price and bedroom values.
  • The tested script exports listing summaries to CSV.
  • The tested script exports property details to JSON.
  • The six-response comparison is a snapshot, not a benchmark.

What can a Rightmove scraper collect?

A Rightmove scraper can turn listing content into records containing identifiers, asking prices, addresses and property characteristics. Define the fields you need before collecting extra pages.

In our snapshot, search records contained ID, numeric price, currency, displayed price, address, bedrooms, bathrooms, property subtype and listing URL. They also contained update labels and other metadata. The script collects a subset and adds a UTC capture timestamp.

Asking prices are marketing values, not completed sale prices. Preserve the display text because qualifications such as shared ownership change how a number should be interpreted. Do not compare a partial ownership price with an unrestricted asking price without accounting for that difference.

Missing bedrooms should remain missing rather than becoming zero. Likewise, an absent bathroom count does not mean the property has no bathroom. Keep missingness separate from parsing failure so downstream reports can distinguish incomplete listings from broken extraction.

Property details added tenure, postcode area and key features in our run. Search summaries were enough for basic price monitoring, which avoided unnecessary detail requests. The agent block existed in the search data, but our script did not collect it.

For the broader workflow, see our real estate data scraping guide. The Zillow scraper example is an adjacent implementation, not evidence that the same parser works here.


How do you build a Rightmove scraper in Python?

Use the tested Requests script below to fetch HTML, decode embedded JSON and write CSV and JSON outputs. It is a small smoke-test implementation, with limitations that matter before scheduling it.

Installation: Use Python 3.12 and install the measured dependency with python3 -m pip install requests==2.31.0. Save the script as rightmove_scraper.py, then run python3 rightmove_scraper.py from a directory where it can write outputs.

Inputs: The script fixes Bristol, two search pages and two detail pages in its final function call. It has no command-line argument parser. Its optional environment input is PROXY_URL; leave that unset for direct access, or inject your authorized proxy configuration privately. The comment contains illustrative placeholders, not working credentials.

import csv, json, os, random, re, time
from datetime import datetime, timezone
import requests

PROXY = os.environ.get("PROXY_URL")  # e.g. http://USER__cr.gb:[email protected]:823
PROXIES = {"http": PROXY, "https": PROXY} if PROXY else None
HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
                  "(KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36",
    "Accept-Language": "en-GB,en;q=0.9",
}
NEXT_DATA = re.compile(r'<script id="__NEXT_DATA__"[^>]*>(.*?)</script>', re.S)
BASE = "https://www.rightmove.co.uk"


def fetch(url):
    r = requests.get(url, headers=HEADERS, proxies=PROXIES, timeout=40)
    print(f"{url} -> {r.status_code}, {len(r.content) // 1024} KB")
    return r.text if r.status_code == 200 else None


def parse_search(html):
    results = json.loads(NEXT_DATA.search(html).group(1))["props"]["pageProps"]["searchResults"]
    rows = []
    for p in results["properties"]:
        rows.append({
            "id": p["id"],
            "price": p["price"]["amount"],
            "currency": p["price"]["currencyCode"],
            "price_text": p["price"]["displayPrices"][0]["displayPrice"],
            "address": p["displayAddress"],
            "bedrooms": p.get("bedrooms"),
            "bathrooms": p.get("bathrooms"),
            "property_type": p.get("propertySubType"),
            "added_or_reduced": p.get("addedOrReduced"),
            "url": BASE + p["propertyUrl"].split("#")[0],
        })
    return rows, results["pagination"]["total"]


def parse_property(html):
    """Property pages keep data in window.__PAGE_MODEL as a flat, index-linked list."""
    raw = html[html.index("window.__PAGE_MODEL"):].split("=", 1)[1].strip()
    model, _ = json.JSONDecoder().raw_decode(raw)
    flat = json.loads(model["data"])

    def resolve(i):  # every value inside objects/lists is an index into `flat`
        v = flat[i]
        if isinstance(v, dict):
            return {k: resolve(x) for k, x in v.items()}
        if isinstance(v, list):
            return [resolve(x) for x in v]
        return v

    p = resolve(0)["propertyData"]
    return {
        "id": p["id"],
        "price_text": p["prices"]["primaryPrice"],
        "price_qualifier": p["prices"]["displayPriceQualifier"],
        "address": p["address"]["displayAddress"],
        "postcode_area": p["address"]["outcode"],
        "bedrooms": p.get("bedrooms"),
        "bathrooms": p.get("bathrooms"),
        "property_type": p.get("propertySubType"),
        "tenure": (p.get("tenure") or {}).get("tenureType"),
        "listing_update": (p.get("listingHistory") or {}).get("listingUpdateReason"),
        "key_features": " | ".join(p.get("keyFeatures") or []),
    }


def scrape(location_page, pages=2, details=2):
    rows = []
    for page in range(pages):
        html = fetch(f"{BASE}/property-for-sale/{location_page}.html?index={page * 24}")
        if not html:
            break
        new, total_pages = parse_search(html)
        rows += new
        print(f"page {page + 1}/{total_pages}: {len(new)} listings")
        time.sleep(random.uniform(4, 8))
    captured = datetime.now(timezone.utc).isoformat(timespec="seconds")
    for row in rows:
        row["captured_at"] = captured
    detail_rows = []
    for row in rows[:details]:
        html = fetch(row["url"])
        if html:
            detail_rows.append(parse_property(html))
        time.sleep(random.uniform(4, 8))
    return rows, detail_rows


if __name__ == "__main__":
    listings, details = scrape("Bristol", pages=2, details=2)
    with open("rightmove_listings.csv", "w", newline="", encoding="utf-8") as f:
        w = csv.DictWriter(f, fieldnames=list(listings[0].keys()))
        w.writeheader()
        w.writerows(listings)
    with open("rightmove_details.json", "w", encoding="utf-8") as f:
        json.dump(details, f, indent=2, ensure_ascii=False)
    print(f"saved {len(listings)} listings and {len(details)} property pages")

Outputs: Inspect rightmove_listings.csv and rightmove_details.json. The script overwrites these names on each execution. Detail rows have no capture timestamp or URL, so join them to the listing export by ID when preserving a snapshot.

HTTPX, Parsel and Playwright were not dependencies of this script. For a future implementation, consult the official HTTPX proxy configuration documentation, Parsel usage guide and Playwright Python guide. Such a migration needs separate testing; our results do not validate it.


How do you parse listing pages and paginate safely?

Inspect the returned HTML before choosing a parser, and place a strict bound on pagination. Our two page types used different embedded data structures.

Search pages exposed __NEXT_DATA__, with listings under props.pageProps.searchResults.properties. We observed 25 entries per page, including one featured entry, and offsets of 24. Treat those as observations from this run, not permanent page-size guarantees.

The response reported 42 pages and a Bristol result count of 3,803. Those numbers do not establish that iterating the reported pages yields every matching property. We fetched only two pages per script run and checked the resulting IDs.

Property pages used window.__PAGE_MODEL. Its data string decoded into a flat list whose object and list values referenced other entries by index. The recursive resolver reconstructed propertyData. It assumes the observed structure and does not protect against cyclic references or changed encoding.

For scheduled collection, stop at the configured page budget, an empty result page, repeated IDs or a schema mismatch. Deduplicate by listing ID before analysis. The current script appends rows without deduplicating and displays the reported page total without enforcing it.

If the script tag disappears, inspect an authorized response before changing selectors. A parsing exception is not evidence that a browser or another IP is required. Do not treat an empty or challenged response as a successful zero-listing snapshot.


How should you handle rate limits, CAPTCHAs and IP blocks?

Keep concurrency low and stop on explicit denial or a CAPTCHA. Delays, browser-like headers and residential routing do not grant permission or guarantee access.

Our script uses sequential requests, 4-8 second pauses and a 40-second timeout argument. It has no retry loop, automatic challenge detector or global stop switch. A non-200 search response ends pagination, but detail requests can still follow if earlier rows exist. Supervise this example; it is not ready for unattended collection.

Before scheduling, add a global stop condition for denial, challenge content and unexpected redirects. For a transient network failure, a conservative design might allow one delayed retry, then end the job. Treat rate limiting as a reason to pause and review the collection agreement rather than rotate routes automatically.

Inspect the applicable robots.txt rules. Our review found the ordinary search and property paths outside the wildcard disallow list, while API, map and user paths were restricted. GPTBot has a separate site-wide disallow. Robots guidance and contractual permission are separate checks.

Our direct and residential requests both returned content. GB targeting was unnecessary in this snapshot. If an authorized workflow needs location selection, review UK proxy selection; for diagnosis, use our 403 troubleshooting guide. Keep session behavior deliberate and avoid changing several variables at once.


What did our small Linux run return and cost?

Our quick run on October 19, 2026 returned content through both routes. It was a small snapshot, not a benchmark.

We used Linux, Python 3.12, Requests 2.31.0 and curl_cffi 0.15.0. Across exploration and script execution, we made 19 requests to rightmove.co.uk, using public Bristol search pages and 2-3 property pages, no login and 4-8 second pauses. The residential route used a DataImpulse US exit.

The original six-response comparison follows. Sizes retain the run’s rounded units; exact byte counts, per-response UTC times and individual property IDs were not recorded in this comparison.

Request Route Status Decoded HTML Transferred body Outcome
Requests, search page 1 Direct 200 1.21 MB 154 KB Content
curl_cffi Chrome, search page 1 Direct 200 1.22 MB 155 KB Content
Requests, search page 1 Residential 200 1.22 MB 154 KB Content
curl_cffi Chrome, search page 1 Residential 200 1.21 MB 154 KB Content
Property page Direct 200 394 KB 106 KB Content
Property page Residential 200 382 KB 105 KB Content

Elapsed times ranged from 0.4-0.7 seconds directly and 0.6-2.4 seconds residentially across these six requests. No challenge appeared. These observations cannot establish comparative speed or reliability.

Each route’s script execution returned 50 listings from two search pages and two detail records. Each listing export had 50 unique IDs, all with price and bedrooms. Other fields had no aggregate completeness audit. The run artifacts were rightmove_listings.csv and rightmove_details.json, generated by rightmove_scraper.py; no code commit identifier was recorded.

At the first-party residential rate of $1 per GB, body-only cost per 1,000 pages is mean transferred bytes multiplied by 1,000, divided by 1,000,000,000. Approximately 154 KB per search response gives $0.15; 105 KB per detail response gives $0.11. Using decoded sizes instead would give approximately $1.22 and $0.39.

At the observed 25 entries per search page, 1,000 entries require about 40 pages, roughly 6 MB and under one cent. That is not a guarantee of 1,000 unique listings. Estimates exclude headers, retries, images and browser assets. There were no blocked responses to price separately. The script’s printed content length alone does not reproduce transferred-body measurements.


How can you track price reductions and store records securely?

Use built-in alerts when they meet your needs; use authorized timestamped snapshots when you need your own price-change rules. A reduction label alone is insufficient for a complete price history.

Rightmove’s property alert instructions describe creating an alert from a search and choosing an email frequency, including Instantly. Its reduction guidance sets a 2% threshold (as of October 2026) for reduction flags and associated alerts. That threshold is enough for many home-search workflows, but smaller drops need another authorized observation method.

Store each observation with listing ID, numeric price, currency, price qualification, source URL and capture time. Compare consecutive valid observations for the same ID. A change from GBP 300,000 to GBP 297,000 is a 1% reduction, regardless of whether a reduction label appears.

Suppress comparisons when currency or ownership qualification changes. A missing listing is not automatically sold, and a new ID should not automatically inherit another listing’s history. Preserve enough context for a human to resolve ambiguity.

The script assigns one capture time after search collection; it does not record a separate request time for each listing. Archive each completed export before another execution overwrites it.

For mobile access, publish a private summary behind authentication. For an n8n handoff, define a minimal record contract and validate it before connecting automation. Keep credentials outside exports, restrict storage access, encrypt backups and set a deletion period. Retain raw HTML only when necessary and authorized.


Does Rightmove offer an official API or export alternative?

We could not verify an unrestricted public listings API. Establish the permitted dataset and use case directly with Rightmove before choosing an integration.

The official Automated Datafeed documentation concerns loading and managing property data. Its scope is not a general permission to extract marketplace listings. The Property Data Services page is the relevant starting point for discussing a licensed data requirement.

The developer portal did not expose readable documentation during our review. We therefore make no claims about its keys, endpoints, authentication, quotas or export formats.

Ask for written coverage of required fields, historical depth, refresh frequency, retention and redistribution. Distinguish permission to receive your own branch’s records from permission to collect competing listings.

Our search-page JSON carried a notice restricting the internal API’s use to Rightmove unless its Legal Team grants express written permission. Embedded JSON is not evidence of an official public API contract. A third-party wrapper also cannot establish Rightmove’s permission merely by accepting a URL.


Which scraping approach should you choose?

Choose according to authorization, output requirements and who will maintain the workflow. Our small run supports no vendor ranking.

Approach Use X when… Avoid X when…
Python Use Python when you need custom fields and have maintenance capacity. Avoid Python when nobody owns parser updates.
Actor/extension Use an actor or extension when small exports fit verified capabilities. Avoid an actor or extension when permissions or data handling are unclear.
Licensed/API route Use a licensed/API route when the contract covers required data. Avoid a licensed/API route when assuming any feed permits marketplace extraction.
Residential proxy Use a residential proxy when an authorized workflow needs residential routing. Avoid a residential proxy when expecting guaranteed access or legal permission.

For a Rightmove scraper GitHub project, inspect its license, recent maintenance, fixtures and output schema. Run a small authorized sample before trusting examples or screenshots. A repository name does not establish compatibility with today’s HTML.

For an Apify Rightmove scraper actor, extension or hosted service, evaluate the exact publisher and version. Request evidence of supported fields, pagination behavior, export completeness and retention policy. Test missing prices and duplicate IDs, not just the successful example.

Compare the total work required: parser maintenance, validation, monitoring and storage as well as transfer charges. A residential proxy changes routing; it does not perform extraction. Our property-data proxy selection guide covers that narrower infrastructure decision.


Is scraping Rightmove legal?

There is no blanket legal answer: contractual restrictions, database rights, privacy, copyright and access authorization require separate assessment. Public visibility does not settle them.

Rightmove’s terms prohibit scraping and automated collection, with written-consent requirements for specified uses. Robots rules guide crawlers; they are not a contractual license.

The UK database regulations, also reproduced by WIPO, address substantial extraction and repeated, systematic extraction of insubstantial parts. Assess UK and applicable EU database rights before recurring collection.

The ICO joint statement explains that publicly accessible personal information remains subject to privacy law. Photographs can carry copyright protection. Avoid collecting people-related details or reproducing images without an appropriate basis.

CPS guidance addresses knowingly unauthorized access; contractual breach and criminal liability are distinct questions. The hiQ v. LinkedIn opinion concerns US law and creates no blanket UK permission.

Our Web scraping legal overview adds context. For the actual project, the reader decides, check local law.


How do you validate and maintain the scraper?

Apply the 4-step collection check: access, extract, validate, retain. Each step should pass before a snapshot reaches analysis.

Each step passes before a snapshot reaches analysis

  • Access: Confirm permission, applicable paths, request budget and stop conditions. Record response status and route without recording credentials.
  • Extract: Check the expected data container and schema before resolving records. Stop on a changed structure instead of silently returning blanks.
  • Validate: Check unique IDs, numeric prices, currency, URLs and missing-value rates. Preserve qualifiers and compare a small sample against its source.
  • Retain: Archive validated observations with timestamps and parser version. Apply access controls and delete data when its purpose or permission ends.

The tested script needs an empty-output guard: its CSV writer indexes the first listing and will fail if none exist. It also lacks explicit schema validation and automatic challenge detection. Resolve these issues before treating it as a scheduled collector.

Maintain authorized fixtures for a normal search response, property response, missing optional fields and changed structure. Check that unexpected input fails visibly. Record a code commit and dependency versions with each future run so a result can be traced to its parser.

Monitor missingness and unique-ID counts alongside HTTP status. A page can return 200 while its useful data changes. Alert on that change before it contaminates price history.


Frequently Asked Questions

Can you scrape data from Rightmove?

Our small run demonstrated that listing fields could be extracted from public-page HTML. Technical feasibility does not establish permission, so review the terms and obtain authorization for the intended collection.

Does Rightmove provide a public listings API?

We could not verify an unrestricted public listings API. Discuss the required dataset and permitted uses with Rightmove rather than treating embedded page data or a third-party wrapper as official access.

How do you extract Rightmove listings with Python?

The tested example fetches HTML with Requests and decodes the observed search and property data containers. It writes listing summaries to CSV and detail records to JSON, which need validation before analysis.

How can you track small Rightmove price reductions?

Compare numeric prices across authorized observations of the same listing ID. Preserve currency and price qualifications, and calculate changes from the values rather than relying solely on a reduction label.

When does a Rightmove scraper need residential proxies?

Consider residential routing only when an authorized workflow has a specific routing requirement. Direct requests worked in our snapshot, so the results do not demonstrate a need for residential proxies.

Does the script collect every Bristol listing?

No. It requests two search pages and two property pages, and its observed output was 50 unique listing IDs per route. Reported search totals do not establish complete collection coverage.

Why might the script fail after returning HTTP 200?

A successful HTTP status does not guarantee the expected data structure. Changed markup, missing fields or challenge content can break parsing, so inspect the response and stop before publishing incomplete records.


Match routing to an authorized collection workflow

If your project needs residential routing, DataImpulse offers residential proxies from $1 per GB. Start with a small request budget, validate the exported records and measure transferred bytes before scaling. Routing cannot replace access permission or parser maintenance.

Last updated: October 19, 2026.


Share article: