Gumtree Scraper: Build a Python Listing Collector

A Gumtree scraper can collect public UK listing titles, prices, locations and URLs into CSV using Python, after you establish permission for automated access. The tested collector below reads one London bicycles category page and opens three listing pages for their posted dates.

In our quick run on October 18, 2026, both complete runs returned 30 records. This was a small snapshot, not a benchmark: successful responses do not establish permission, future availability or coverage across categories.

What should a Gumtree scraper collect?

Collect the metadata needed for your analysis: listing ID, title, advertised price, currency, coarse location, URL and observation time. Exclude seller names, phone numbers, email addresses and account controls.

Our UK sample contained server-rendered cards selected by [data-q="search-result"]. Within them, we observed tile-title, tile-price, tile-location, tile-description and search-result-anchor attributes. These are inspected selectors from one snapshot, not a permanent interface contract.

The script saves price text rather than a numeric amount and separate currency. For analysis, retain that original text alongside any normalized value, and distinguish missing prices from zero. Add a UTC observation timestamp when building historical storage; the tested CSV does not contain one.

Search cards lacked posted dates. Detail pages contained relative text such as “1 day ago”, embedded age information and Product JSON-LD with GBP pricing. Preserve relative dates with the observation time instead of treating them as exact publication timestamps. Five of the 30 cards were marked Featured, so page position should not become a proxy for listing age.

How do you build a Gumtree scraper in Python?

After obtaining permission, install the dependencies, save the tested script as gumtree_scraper.py, and run it against its fixed category URL. It uses Requests and Beautiful Soup, a 40-second timeout and 5-8-second pauses before detail requests.

Install with pip install requests beautifulsoup4 lxml. Run directly with python3 gumtree_scraper.py while PROXY_URL is unset, or set that environment variable securely for an authorized proxy route. The example credential labels in the unchanged script are placeholders, not working credentials.

import csv, os, random, re, time
import requests
from bs4 import BeautifulSoup

PROXY = os.environ.get("PROXY_URL")  # e.g. http://USER__cr.gb:[email protected]:823 (UK exit)
PROXIES = {"http": PROXY, "https": PROXY} if PROXY else None
HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
                  "(KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36",
    "Accept-Language": "en-GB,en;q=0.9",
}
BASE = "https://www.gumtree.com"


def fetch(url):
    r = requests.get(url, headers=HEADERS, proxies=PROXIES, timeout=40)
    if r.status_code != 200:
        print(f"{url} -> HTTP {r.status_code}, stopping")
        return None
    return BeautifulSoup(r.text, "lxml")


def text(node, q):
    el = node.select_one(f'[data-q="{q}"]')
    return el.get_text(" ", strip=True) if el else None


def parse_results(soup):
    rows = []
    for card in soup.select('[data-q="search-result"]'):
        link = card.select_one('[data-q="search-result-anchor"]')
        price = text(card, "tile-price")
        rows.append({
            "id": link["href"].rstrip("/").split("/")[-1] if link else None,
            "title": text(card, "tile-title"),
            "price": price.replace(" ", "") if price else None,  # "£ 300" -> "£300"
            "location": text(card, "tile-location"),
            "featured": "Featured" in card.get_text(" "),
            "url": BASE + link["href"] if link else None,
        })
    return rows


def posted_date(soup):
    """Search cards have no date; the listing page shows 'Posted: 4 days ago'."""
    m = re.search(r"Posted:\s*(.+?)$", soup.get_text("\n"), re.M)
    return m.group(1).strip() if m else None


def scrape(category_path, details=3):
    # Page 1 of a category page. robots.txt disallows /search and most /pageN URLs,
    # so check robots.txt before you add pagination for your category.
    soup = fetch(f"{BASE}/{category_path}")
    if not soup:
        return []
    rows = parse_results(soup)
    print(f"{category_path}: {len(rows)} listings")
    for row in rows[:details]:  # one listing page per row, only for the date
        time.sleep(random.uniform(5, 8))
        page = fetch(row["url"])
        row["posted"] = posted_date(page) if page else None
        print(f"{row['id']}: posted {row['posted']}")
    return rows


if __name__ == "__main__":
    rows = scrape("for-sale/sports-leisure-travel/bicycles/uk/london", details=3)
    if not rows:
        raise SystemExit("No listings collected; check the HTTP status above.")
    fields = ["id", "title", "price", "location", "posted", "featured", "url"]
    with open("gumtree_listings.csv", "w", newline="", encoding="utf-8") as f:
        w = csv.DictWriter(f, fieldnames=fields)
        w.writeheader()
        w.writerows(rows)
    print(f"saved {len(rows)} listings")

Successful execution creates gumtree_listings.csv. Our two runs populated title, price, location and URL for every row, with posted dates for the three inspected details. Blank dates elsewhere mean those pages were not opened.

Consult the Beautiful Soup selector documentation when adapting the parser. If your application standardizes on another HTTP client, consult the HTTPX quickstart; this measurement did not test an HTTPX replacement. Our eBay Python collection guide offers a separate marketplace example.

Important limits remain: there is no automatic policy check, timestamp, pagination or SQLite writer. A failed detail request returns no page but does not terminate the remaining loop. Stop the process on denial; do not schedule this unchanged example unattended.

How do pagination and SQLite storage work?

The tested collector stops at page one and writes CSV. Pagination and SQLite history require a separate, validated extension.

Gumtree’s robots rules disallow /search and many numbered-page paths, with specific exceptions (as of October 2026). Inspect the exact category and applicable user-agent rules before following any next link. An exception for a broad category does not automatically cover its London bicycles subcategory.

For an authorized extension, follow an inspected next-page link rather than inventing page=N. Enforce a small page limit, track visited URLs, and stop on repeated pages, denial or invalid content. Keep detail requests inside the same request budget.

Design SQLite storage around two records: a listing keyed by ID, and an observation keyed by run ID plus listing ID. Store UTC observation time, raw price, normalized price, currency and run status. Deduplicate within each run without overwriting earlier observations. Commit a snapshot only after validating its completeness, then export its rows to CSV.

Zero parsed cards can mean an empty category, a changed selector or an unexpected response. Require recognizable empty-results content before labeling a run empty. Otherwise retain the last successful snapshot and record an extraction failure. A missing listing on page one is not proof that it sold or disappeared.

Why does Gumtree block requests?

A failed request needs diagnosis before another request. Our snapshot encountered no blocks, so it cannot identify Gumtree’s blocking triggers or establish universal datacenter restrictions.

Use the 4-step access check: policy, response, content, parser. Work through these checks in order, because changing selectors cannot fix a permission problem.

• Step 1: Policy. Confirm written authorization and evaluate the current robots rules for each URL. Treat a failed policy check as a stop condition.
• Step 2: Response. Record UTC time, final URL, status, headers, elapsed time and body size. Stop on denial before considering another route.
• Step 3: Content. Check whether the body contains listing cards, a challenge, an error or an empty-results message. HTTP 200 alone is insufficient.
• Step 4: Parser. Inspect selectors and required-field coverage against the actual response. Flag structural changes instead of silently saving empty data.

HTTP semantics defines 403 as refusal to fulfill a request. RFC 6585 defines 429 for excessive request rates and permits a Retry-After header. Honor that delay when retrying is authorized; do not automatically rotate addresses after either status.

Browser-looking headers and Chrome impersonation are not permission or guarantees. Both Requests and curl_cffi succeeded in our sample, so the evidence does not establish that impersonation was necessary. Inspect the HTML before adding browser rendering, and avoid regex as the primary parser for listing tiles. Retain the measured 5-8-second pacing rather than assuming 1-3 seconds is acceptable.

For implementation context, see our guides to 403 responses, Python retry handling and request planning and blocks.

When should you use proxies or another scraping tool?

Choose Python for control over fields and history, a managed scraper for a verified output contract, or native alerts for simple discovery. A proxy changes the network route; it does not grant permission or repair extraction logic.

For an apify Gumtree scraper, the community-maintained listing actor advertises pricing from $1.50 per 1,000 results (as of October 2026). Its documented Python workflow uses an Apify token, calls the actor and retrieves its dataset. That is a third-party actor API, not Gumtree’s API. Inspect current inputs and test required fields before purchasing; our run did not evaluate it.

Apply the same acceptance criteria when considering Octoparse or Bright Data: request a UK sample, confirm contact-field exclusion, inspect failed-run billing and establish authorized coverage. Product names alone do not support a ranking.

DataImpulse supplies proxy connections, not a scraping API, managed unblocker, VPN or data vendor. Our direct route also succeeded, so this snapshot does not show a proxy was necessary. The Craigslist proxy guide discusses another classifieds context, but its results should not be transferred to Gumtree.

How can you monitor listings and price changes?

Monitor by comparing validated, timestamped snapshots using listing IDs. Keep collection, comparison and notification delivery separate so a failed fetch cannot generate false price alerts.

Define keyword and coarse-location filters before scheduling. A bicycle search might include frame-size terms and a London area; furniture monitoring might use item type and a price ceiling. Applying filters locally to an authorized category snapshot avoids inventing search URLs, but only covers the cards collected.

Choose a schedule within the permission granted, prevent overlapping runs and retain the page and detail-request budgets. The example overwrites its CSV, so archive each successful output with its observation time before another run.

Compare normalized prices only when both observations are valid and currencies match. Label unseen IDs as newly observed, not necessarily newly posted. Treat missing IDs as outside the current snapshot until independently confirmed. Featured placement and page movement can otherwise create misleading changes.

Make notification destination, price-drop threshold and keyword rules configurable. An email, webhook or app message should contain the listing URL, old and new prices, and observation times. Deduplicate delivery separately from collection. For spreadsheet import, retain UTF-8 and neutralize formula-like text in a separate export layer without changing the archived source values.

Does Gumtree offer an official API or simpler alternatives?

Our official-source check did not establish a documented public Gumtree UK listing API. Do not build around guessed authentication, quotas, partner feeds, bulk exports or category RSS endpoints.

For discovery, consult the UK email and notification guide and check the controls in your account. Keep the intended outcome narrow: learning about matching bicycles or furniture is different from maintaining structured price history.

The Australian saved-search documentation describes immediate, daily or weekly emails, a keyword requirement for immediate delivery, and a 90-day duration (as of October 2026). Those details belong to Australia and should not be assumed for UK accounts.

If an API or partner arrangement is essential, ask Gumtree for documentation covering your market and intended use. A vendor endpoint containing “Gumtree” in its name does not establish first-party access. Native discovery tools can reduce collection work, but assess their actual output before treating them as a historical dataset.

What did our small Linux test return?

In our quick run on October 18, 2026, direct and residential requests returned full pages without observed challenges. This was a small snapshot, not a benchmark or an availability guarantee.

We used a Linux server with Python 3.12, public UK pages, no login or cookies, and 5-8-second pauses. About 20 requests covered search, the London bicycles category and listing details. The comparison window was 16:07-16:12 UTC; two complete script runs used UK residential and then direct access.

Source: our run notes and gumtree_scraper.py, reproduced above. The reproduction command is python3 gumtree_scraper.py. Dependency versions beyond Python 3.12 were not recorded, which limits exact environment reproduction. The notes contain aggregate sizes rather than a complete per-request timing log or publishable record archive.

Search responses are historical observations, not recommended targets. The written-permission status of this run is undocumented; successful access cannot resolve that gap. Any further live comparison should remain pending until permission is established.

At a $1/GB planning rate, payload cost per 1,000 pages is mean bytes divided by 1,000,000,000, multiplied by 1,000 and $1. The category measured 218 KB compressed versus 792 KB raw: about $0.22 or $0.79 per 1,000 pages. At 30 cards per page, 1,000 card observations require about 34 pages, under one cent of compressed payload, before deduplication.

Opening every detail page adds about $0.13 per 1,000 listings at 128 KB compressed each. These estimates cover successful payloads only. Failed responses were absent from this sample; protocol overhead, retries and browser assets can increase billed traffic.

What legal and contractual risks apply?

Public visibility does not settle permission, contract, copyright or privacy questions. Gumtree’s terms restrict automated collection without express written permission (as of October 2026), so establish that permission before running even a small collector.

Robots rules describe crawler access preferences; they are not a legal license. The robots exclusion standard explicitly separates those rules from access authorization. Evaluate both, and avoid collecting personal information simply because it appears publicly.

The 2022 hiQ v. LinkedIn opinion addressed a US CFAA issue involving public pages in preliminary-injunction proceedings. It did not grant blanket scraping permission or eliminate contractual and intellectual-property claims.

The EU Database Directive also addresses substantial extraction and certain repeated, systematic extraction. Applicability depends on the database, conduct and jurisdiction. Define a narrow purpose, minimize retained fields and assess personal-data obligations before sharing results. Our web scraping legality guide offers broader context, but cannot determine your circumstances: the reader decides, check local law.

Choose a proxy only when your collector needs one

If authorized collection needs another network route, consider DataImpulse after validating your parser and request budget. Start small, measure successful payload traffic and keep denial handling explicit. A proxy cannot guarantee access or replace Gumtree’s permission.

Last updated: October 18, 2026.