# Redfin search-page refresh (set up 2026-10-30)

**Why this way:** Redfin's robots.txt disallows the `/stingray/` JSON API but allows ordinary search pages. One
filtered search per city ("built 2015+", sold in the last year / for sale now) covers the whole recent market in
about 25 page loads, which is human-scale. The pages carry schema.org JSON-LD (address, coordinates, sqft, price,
Redfin URL) plus card text (sold date, beds/baths, days listed, description, photo).

**Volume (2026-10-30):** 803 homes built 2015+ across the 4 cities (559 sold in the last year, 244 for sale).
168 matched tracked middle-housing parcels: 126 sold and 42 for sale, mostly townhouses, condos and small lots.
The rest are detached SFR. For scale, about 130–140 for-sale-type middle-housing units are finished per year,
and about a third get sold. So expect roughly 4–7 new middle-housing listings a month.

## Routine (Claude does this in the built-in browser when Ted asks; ~10 min)
1. Open any redfin.com page in the built-in browser.
2. Store the extractor once:
   `sessionStorage.setItem('mh_fn', '(' + <function from pipeline/redfin_search.js> + ')()')`,
   then `sessionStorage.removeItem('mh_rf')`.
3. For each URL in `pipeline/redfin_urls.txt` and its `/page-2`, `/page-3`…:
   - navigate, wait ~1.5 s, then `await eval(sessionStorage.getItem('mh_fn'))`;
   - check that `noDate` is ~0. Cards only render the "SOLD <date>" badge once scrolled into view, and the extractor scrolls them.
4. Run `sessionStorage.getItem('mh_rf')` in the JS tool. The result is too big to show inline, so the harness saves it to a tool-results
   file. Parse that file (a JSON array whose `[0].text` is the JSON string) into `data/redfin_search/<YYYY-MM-DD>.json`.
5. `python3 pipeline/redfin_merge.py && python3 pipeline/build.py`.

## Dead ends (don't retry)
- `/stingray/` API: disallowed by robots.txt. `newest_listings.rss` is allowed but returns 404.
- Handing data from the redfin.com tab to localhost: fetch, iframe and clipboard are all blocked by the built-in browser.
- `include=sold-1yr` returns sold homes only; the plain filter returns active listings. Run both.
- The `property-type=townhouse` filter misses most townhouses, because Redfin labels many "Single Family". Filter by year built instead.

## Backups if Redfin changes
Zillow, Realtor.com, John L. Scott, Windermere and Compass all allow their listing and search pages in
robots.txt (checked 2026-10-29). Zillow blocks scripted fetches, though normal page views work.