Home Depot Scraper: Python Tutorial for Product Prices and Store Context

A Home Depot scraper is software that extracts public product information into structured records for analysis. This tutorial concerns prices and product data, not paint, floor, ice, card or ceiling scraping tools.

Our quick run on October 18, 2026 collected 25 products from a home-page response through a residential route, while product pages remained blocked. Below is the tested Python script, its CSV and JSON outputs, and the context checks needed before treating an extracted price as a useful observation.

What does a Home Depot scraper collect?

A Home Depot scraper can collect product identifiers, names, prices, ratings and availability labels when those fields are accessible. The output represents what a particular response contained, not a complete catalog or a guarantee of inventory.

In our successful home-page response, product objects appeared inside window.__APOLLO_STATE__, an embedded cache. We observed identifiers, store-associated pricing, rating summaries and an availability type. The script exports item ID, name, model, price, rating, review count, availability and URL.

These were home-page carousel records. None of the three follow-up product pages supplied a successful detail response, so the exported prices must retain their home-page provenance.

Searches about paint removal, adhesive removal, concrete blades, rentals, corded versus cordless tools and physical-tool brands belong to a different buying task. They are outside this software tutorial.

The extraction pattern transfers between retailers, but the schema and context rules require separate validation. Compare our Walmart scraper tutorial and Costco scraper walkthrough before reusing assumptions.

How do you build a Home Depot scraper in Python?

Start with the tested document-fetching script below, which reads embedded product records and attempts three detail pages. It can export home-page records even when those detail requests fail.

Step 1: On Linux, create an isolated environment with python3 -m venv .venv, activate it with source .venv/bin/activate, and install the tested dependency with python3 -m pip install curl_cffi==0.15.0. Our interpreter was Python 3.12.3.

Step 2: Save the script as home_depot_scraper.py. It starts at the public home page and follows canonical URLs extracted from the response; it does not require guessed product slugs.

Step 3: Run python3 home_depot_scraper.py directly, or configure your own PROXY_URL environment variable before running it. The comment contains illustrative placeholders, not working account credentials. Keep real connection details out of source control and logs.

import csv, json, os, random, time
from curl_cffi import requests   # pip install curl_cffi: browser-like TLS fingerprint

PROXY = os.environ.get("PROXY_URL")  # e.g. http://USER__cr.us:[email protected]:823
PROXIES = {"http": PROXY, "https": PROXY} if PROXY else None
BASE = "https://www.homedepot.com"


def fetch(url):
    try:
        r = requests.get(url, impersonate="chrome", proxies=PROXIES, timeout=40)
    except Exception as exc:
        print(f"{url} -> {type(exc).__name__}")
        return None
    blocked = r.status_code == 403 or "Access Denied" in r.text[:2000] or "Oops!!" in r.text[:5000]
    print(f"{url[:80]} -> HTTP {r.status_code}, {len(r.content) // 1024} KB" + (", blocked" if blocked else ""))
    return None if blocked or r.status_code != 200 else r.text


def apollo_state(html):
    """Home Depot pages embed their GraphQL cache as window.__APOLLO_STATE__={...}"""
    start = html.find("__APOLLO_STATE__=")
    if start == -1:
        return {}
    data, _ = json.JSONDecoder().raw_decode(html[start + len("__APOLLO_STATE__="):])
    return data


def parse_products(state):
    rows = []
    for key, obj in state.items():
        if not isinstance(obj, dict) or obj.get("__typename") != "BaseProduct":
            continue
        ids = obj.get("identifiers") or {}
        # the pricing key carries the store id, e.g. pricing({"...","storeId":"8119"})
        pricing = next((v for k, v in obj.items() if k.startswith("pricing")), None) or {}
        ratings = (obj.get("reviews") or {}).get("ratingsReviews") or {}
        rows.append({
            "item_id": obj.get("itemId"),
            "name": f"{ids.get('brandName', '')} {ids.get('productLabel', '')}".strip(),
            "model": ids.get("modelNumber"),
            "price": pricing.get("value"),
            "rating": ratings.get("averageRating"),
            "reviews": ratings.get("totalReviews"),
            "availability": (obj.get("availabilityType") or {}).get("type"),
            "url": BASE + (ids.get("canonicalUrl") or ""),
        })
    return rows


if __name__ == "__main__":
    # 1) The home page ships a "top sellers" product carousel in __APOLLO_STATE__
    html = fetch(BASE + "/")
    rows = parse_products(apollo_state(html)) if html else []
    print(f"home page: {len(rows)} products")
    # 2) Try three product pages; same parser if they come back
    for row in rows[:3]:
        time.sleep(random.uniform(8, 12))
        page = fetch(row["url"])
        row["product_page"] = "ok" if page and parse_products(apollo_state(page)) else "blocked"
    if not rows:
        raise SystemExit("No products collected; check the fetch() output first.")
    with open("homedepot_products.csv", "w", newline="", encoding="utf-8") as f:
        w = csv.DictWriter(f, fieldnames=list(rows[0].keys()))
        w.writeheader()
        w.writerows(rows)
    with open("homedepot_products.json", "w", encoding="utf-8") as f:
        json.dump(rows, f, indent=2)
    print(f"saved {len(rows)} products")

Step 4: Inspect homedepot_products.csv and homedepot_products.json. Missing scalar values become empty CSV cells or JSON nulls. An absent availability value is unknown, not evidence that an item is unavailable.

Only the first three records receive a product_page result. The remaining records were not attempted. Also, blocked is a coarse label: it includes failed requests and responses without parseable products. An ok result only means some product objects parsed, not that the requested item’s identity matched.

The parser expects the exact cache marker and selects the first pricing key. It does not export store ID, ZIP, currency, fulfillment mode or observation time. Malformed embedded JSON can raise an exception. These limits matter before scheduling repeated collection.

Our broader diagnostics also used requests 2.31.0 and Playwright 1.58.0. For a separate browser implementation, consult the official Playwright installation guide and locator documentation. Those diagnostics did not unlock product pages; the script above needs neither dependency.

How do you track prices and availability by store?

Track prices by preserving product identity, store, ZIP, fulfillment mode and observation time together. Compare only records with matching context and successful extraction.

Use the 4-step price-context check before accepting a record into a price history:

• Identify product: retain item ID, model and source URL; confirm that variants and pack sizes match.
• Confirm store/ZIP: record the requested location and the location actually observed in the response.
• Distinguish fulfillment: keep shipping, delivery and pickup observations separate; leave unsupported modes unknown.
• Timestamp observation: record UTC retrieval time and source page type alongside the value.

Our run observed default store 8119 in pricing context, with no ZIP set. The exported Online availability label did not establish local stock, a quantity or pickup eligibility. A US proxy address alone cannot confirm a particular store.

Home Depot warns that local store prices may differ from displayed prices and inventory levels cannot be guaranteed (as of October 2026). See its price inquiries page.

Keep a companion observation log because the tested export omits context fields. Until those fields are captured reliably, treat the files as extraction evidence rather than a store-aware tracker. Trigger price alerts only between comparable observations; send missing-data alerts separately. Our guide to retail price snapshots and alerts explains the wider workflow.

Why do requests return 403 or incomplete data?

A 403 means the server understood the request but refused it; a missing field requires a different diagnosis. The status alone does not identify the rule responsible for refusal, as explained in MDN’s 403 reference.

• Access refusal: preserve status and a short nonpersonal error indicator, then stop persistent denials.
• Rendering or extraction failure: a successful response without the expected cache needs inspection before changing selectors.
• Missing field: retain null rather than substituting zero, an old price or a stock assumption.
• Context error: reject comparisons when the store, ZIP or fulfillment mode is unknown or mismatched.

Our residential product requests failed with requests, curl_cffi, a cookie session after loading the home page, and headless Chromium. That observation does not establish why they failed or predict another workflow’s outcome.

The tested script sets a 40-second request timeout and performs no automatic retries. Its three-item loop continues after individual failures, so supervise it and interrupt repeated refusals. For a future authorized job, set a total request budget and bounded retries for transient failures, not an endless 403 loop.

See our 403 diagnosis guide and bounded Python retry discussion. A successful home page is insufficient evidence that product collection works.

When should you use datacenter or residential proxies?

Choose a route from measured results for your authorized workflow. A proxy changes the network route; it does not establish permission, extract fields or guarantee access.

For continuity, preserve application state as well as the route. The tutorial makes separate requests and does not explicitly maintain a cookie session. Our separate cookie-session experiment still received product-page refusals.

DataImpulse supplies proxy infrastructure, not a scraping API, managed unblocker, VPN or data vendor. In this snapshot, its US residential route changed the home-page result but did not produce successful product-page extraction. That is not evidence of general proxy superiority.

What did our quick public-page run return?

Our quick run on October 18, 2026 returned 25 home-page product records through residential access and zero successful product-detail responses. This was a small snapshot, not a benchmark.

We used a Linux server, the dependency versions listed above, public pages only, no login and no CAPTCHA solving. Across exploratory diagnostics and the tutorial, we made about 30 requests to Home Depot domains, with pauses of 5-12 seconds; the tutorial sleeps 8-12 seconds between product attempts.

The same tutorial code and parsing rules were used on both routes. This was not a completed three-product paired comparison: the datacenter home page failed, preventing product URL discovery. One earlier datacenter product probe is listed separately.

Store 8119 came from the successful home-page context; blocked responses did not independently confirm it. Product IDs identify the attempted detail URLs without inventing missing slugs. Sources for the extraction are the tested home_depot_scraper.py and its homedepot_products.csv and homedepot_products.json outputs; timing and transfer observations are reported in the table.

Bandwidth estimate: using $1 per GB, mean transferred bytes x 1,000 / 1,000,000,000 x $1 gives approximately $0.17 per 1,000 home-page loads. Here, the mean has one successful observation: 174,593 bytes. Using decoded document size instead gives approximately $1.24, a hypothetical uncompressed estimate.

This is document bandwidth only, excluding overhead, retries and computing. It is not total browser traffic. Successful product-page cost remains unmeasured. The observations cannot establish a block rate, nationwide coverage or stable performance.

Which APIs, feeds or scraper services are alternatives?

Choose Python for extraction control, proxies for an authorized routing requirement, or a third-party service when you want to outsource extraction. Evaluate each against the same product, location and freshness requirements.

Does Home Depot offer an official public product API? We could not verify official documentation for a generally available public product API during this review. That finding does not prove that partner access, private integrations or eligible feeds do not exist.

For catalog integration, start with the official affiliate FAQ and customer support. Ask about eligibility, permitted uses, export formats and store-level coverage before designing around a feed.

Unwrangle’s Home Depot API documentation is third-party documentation, not Home Depot developer documentation. Its documented location requirement is a ZIP alongside the store number when requesting local details (as of October 2026). Validate actual responses before relying on them.

There is no defensible universal best Home Depot price scraper from this sample. Compare candidate services using identical products and contexts: required-field completeness, source timestamps, failure reporting, retention terms and cost per usable record. A formatted response is not enough if its price belongs to the wrong store or its availability meaning is unclear.

What legal and contractual limits should readers consider?

Public visibility alone does not settle whether collection or reuse is permitted. Review contractual restrictions, access controls and intended use before running a recurring job.

Check Home Depot’s current Terms of Use and robots.txt. Their text could not be reliably retrieved during this review, so no specific clause or directive is quoted here. Do not treat an inaccessible policy as permission, and stop rather than bypass login gates or CAPTCHA challenges.

The Ninth Circuit’s 2022 hiQ v. LinkedIn opinion affirmed a preliminary injunction involving public profiles and CFAA issues. It did not grant universal scraping permission or resolve every contractual claim.

Consider copyright before copying descriptions or images, minimize personal data, and assess database rights separately. Directive 96/9/EC addresses substantial extraction or reuse and certain repeated extraction of insubstantial parts of protected databases.

Document your purpose, scope, retention and authorization. For more context, read our web scraping legal overview. Commercial reuse can introduce additional risk; the reader decides, check local law

Test the route, then validate the records

If an authorized collection workflow needs a different network route, DataImpulse residential proxies start from $1 per GB. Begin with a small sample and measure usable records, context completeness and transferred bytes. This run shows why a successful home page alone is insufficient evidence for a production price tracker.

Last updated: October 18, 2026.