The Great Divergence
A Decade of Australian House Prices

A data tale from AURIN, read through the Domain suburb performance statistics endpoint.

Property data platform · accessed by AURIN · Request access (EOI)

Double or stall: what drives a city’s housing market?

In 2016, Brisbane’s median house price was almost 50% lower than Melbourne’s. Ten years later, Brisbane’s median price had more than doubled, while Melbourne’s had seen only modest growth. By 2026, Brisbane had overtaken Melbourne.

For the buyer who owned in Brisbane in 2016, the last decade produced extraordinary wealth. For a 25-year-old trying to buy their first home there in 2026, the same numbers describe a barrier that did not exist when their parents or older siblings entered the market. The data tells both stories at once.

This is the first of two short data tales built on the property data platform accessed by AURIN. It answers a single question - how did Australia’s major cities diverge over the last decade? - using one endpoint, the suburb performance statistics: a handful of API calls returns a decade of quarterly data for every major city, enough to map the national divergence, compare houses against units, and track rental yield.

(The companion tale, “Inside a Flat Market”, uses the residential listings endpoint to look inside these city averages at the level of individual property data.)

A note on the data

Each city here is a sample of 30 suburbs, hand-picked to span that city’s price range - roughly ten at the dear end, ten in the middle and ten at the affordable end - so a city’s figure is not drawn only from its priciest or cheapest pockets. It is a sampling device rather than a formal price classification. “Melbourne” therefore means these 30 Melbourne suburbs, not the entire metropolitan area, and the same holds for every city.

Figures are for houses as Domain classifies them, a category broader than the word suggests: it also folds in townhouses, terraces, duplexes and semi-detached homes, while apartments, flats, studios and villas count as units and vacant land is excluded from both. Units appear only where a House / Unit toggle is shown. Each city line is the median of its 30 suburb medians in a given quarter - the middle sale rather than the average, a middle-of-the-middles that resists distortion from a handful of very expensive sales.

The sample covers the five mainland state capitals (Sydney, Melbourne, Brisbane, Perth, Adelaide) plus Canberra, and two south-east Queensland lifestyle markets - the Sunshine Coast and Gold Coast. Hobart and Darwin, the two smallest capital markets, are left out to keep the comparison on the largest markets, where the quarterly statistics run deepest.

Definitions: what counts as a “house”?
  • House - Domain’s category, and broader than the word suggests. Alongside detached houses it counts townhouses, terraces, duplexes and semi-detached homes.
  • Unit - apartments, flats, studios and villas. Vacant land is excluded from both.
  • Median sold price - the middle sale in a suburb for that quarter, not the average. Every bedroom count is pooled, so a two-bedroom and a five-bedroom sale count equally.
  • The city line - each suburb gives its own median for the quarter, and the city line is the middle value of those. A middle of middles.
  • % change since Q1 2018 - each city indexed to its own Q1 2018 price, which counts as 0%, so cities are compared on growth rather than price level.
  • Withheld prices - Domain returns some sold listings with no price. Through 2018-2024 this affected about 6% of Hawthorn sales and 1% of Frankston’s; by 2026 it reaches 29% in Hawthorn. Those sales are missing from every median here, and withholding is probably not random, since a disappointing result is the more likely one to be held back. Treat the most recent quarters as provisional.

In closing: what the statistics showed

Three things held across every chart in this tale.

The affordability shift since 2018 is broad but far from uniform. Houses in Brisbane, Perth, Adelaide and both Queensland coasts roughly doubled. Sydney and Melbourne did not: their houses rose 55% and 18%, and Sydney units managed just 8%. This is not one national market moving together. It is the cheaper cities closing hard on the expensive ones, and for a buyer the effect is the same either way, because the places that were once the accessible alternative are the ones that moved most.

The national picture hides as much as it shows. Brisbane’s surge and Melbourne’s apparent flatness look like two different stories, but the city-level median flattens each city into a single line. That is the strength of the statistics endpoint - breadth, a whole nation in a handful of calls - and also its limit: it reports what a suburb did on average, and the spread of individual sales underneath that average is invisible to it.

Rental yield is a quiet leading signal. Melbourne’s rising yield and Perth’s falling yield pointed to turning points well before they reached the headlines - the kind of signal that only shows up when you can track a decade of quarters cheaply.

One endpoint, one kind of question

The suburb performance statistics endpoint is built for breadth: map national divergence, compare growth trajectories, and track yield with minimal code and a tiny credit budget. When a city-level trend raises a question that an average cannot answer - is Melbourne’s flatness real, or two opposite movements cancelling out? - the companion tale, “Inside a Flat Market”, switches to the residential listings endpoint to look inside it, one property at a time.

Want to use this data in your own research?

The Domain data used throughout this tale is available to approved Australian researchers through AURIN. Access requires an expression of interest, an institutional affiliation (a university or research organisation login) and signing a data agreement, managed at data.aurin.org.au. Once signed, your account receives 5,000 API credits per month, resetting automatically. That allowance is enough for the kind of targeted analysis shown here without exhausting your budget, provided you design your queries before running them rather than exploring open-endedly.

Materials, reference and access

Training materials: step-by-step guides covering setup, credential management, credit budgeting, known API quirks, and hands-on Jupyter notebooks for the most common query patterns are available at github.com/AURIN-OFFICE/domain_public.

API endpoint reference and metadata: a complete listing of available endpoints with parameters, response fields, coverage notes, and known data limitations is at the domain_public repository.

Request access: Submit your expression of interest for the Domain API at aurin.org.au/domain and retrieve your credentials at data.aurin.org.au.

The code patterns in the training notebooks are written to be adapted directly to your own study area, time window, and research question. If your work involves Australian property markets, this is the fastest route from a research question to clean, queryable, and well-documented data.

This story as source: the complete source behind this analysis, code and narrative together, is on GitHub at datatale/. Every chart on this page is reproducible from it, and the “The Code” tab above lists the code for each figure.

Every chart in the Story tab is built from the code below, shown for reading only (these blocks are not re-executed). Order: the one-time data acquisition first, which the Story itself never runs, then setup, then the code behind each visual in Story order.

Everything needed to reproduce this tale is here, and the blocks run top to bottom on their own, from the datatale/ directory. The Quarto document behind this page, code and narrative together, lives in the datatale/ folder of the public repository. One thing to know before running it:

  • You need your own AURIN credentials. The Domain data is licensed, not open: the download blocks read AURIN_USERNAME and AURIN_PASSWORD from a .env at the repo root and call the AURIN proxy. Without them the code loads whatever is already cached and nothing else. The training materials walk through requesting access and setting those credentials up, step by step.

Setup and data acquisition

Manifest and download helpers

import sys, json
from pathlib import Path
from urllib.parse import quote
from datetime import datetime, timezone

# Import AURIN proxy helpers from the project root (domain_api/utils.py)
_ROOT = Path(".").resolve().parent
sys.path.insert(0, str(_ROOT))
from utils import PROXY_BASE, APICallTracker

_DATATALE = Path(".")          # renders from datatale/
_MANIFEST    = _DATATALE / "manifest.json"
_STATS_BASE  = f"{PROXY_BASE}/v2/suburbPerformanceStatistics"


# ── Manifest helpers (resume-safe: already-downloaded files are skipped) ────

def _load_manifest():
    return json.loads(_MANIFEST.read_text()) if _MANIFEST.exists() else {"captures": []}

def _save_manifest(m):
    _MANIFEST.write_text(json.dumps(m, indent=2))

def _already_captured(m, rel):
    return str(rel) in {e["file"] for e in m["captures"]}

def _record_capture(m, rel, endpoint, params, status, section, empty=False):
    m["captures"].append({
        "file":             str(rel),
        "endpoint":         endpoint,
        "params":           params,
        "captured_at":      datetime.now(timezone.utc).isoformat(),
        "http_status":      status,
        "empty_response":   empty,
        "datatale_section": section,
    })
    _save_manifest(m)

def _save_json(rel, data):
    dest = _DATATALE / rel
    dest.parent.mkdir(parents=True, exist_ok=True)
    dest.write_text(json.dumps(data, indent=2))

def _save_probe(rel, response):
    _save_json(rel, {
        "x_total_count": int(response.headers.get("X-Total-Count", 0)),
        "body":          response.json() if response.text.strip() else {},
    })

Download suburb performance statistics

Show code
# Download suburb performance statistics for every city used in the tales.
# Each suburb+category pair becomes one JSON file in statistics/{city}/.
# Files already recorded in the manifest are skipped automatically.

STATS_SUBURBS = [
    # ── Melbourne (VIC) ──────────────────────────────────────────────────────
    # Premium (10)
    {"city": "melbourne", "suburb": "Toorak",       "postcode": "3142", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Hawthorn",     "postcode": "3122", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Brighton",     "postcode": "3186", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Kew",          "postcode": "3101", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Camberwell",   "postcode": "3124", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Canterbury",   "postcode": "3126", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Malvern",      "postcode": "3144", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "South Yarra",  "postcode": "3141", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Balwyn",       "postcode": "3103", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Armadale",     "postcode": "3143", "state": "VIC", "categories": ["House", "Unit"]},
    # Mid-range (10)
    {"city": "melbourne", "suburb": "Carlton",      "postcode": "3053", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Fitzroy",      "postcode": "3065", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Preston",      "postcode": "3072", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Brunswick",    "postcode": "3056", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Richmond",     "postcode": "3121", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Box Hill",     "postcode": "3128", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Frankston",    "postcode": "3199", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Footscray",    "postcode": "3011", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Coburg",       "postcode": "3058", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Ringwood",     "postcode": "3134", "state": "VIC", "categories": ["House", "Unit"]},
    # Affordable (10)
    {"city": "melbourne", "suburb": "Reservoir",    "postcode": "3073", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Tarneit",      "postcode": "3029", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Point Cook",   "postcode": "3030", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Dandenong",    "postcode": "3175", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Broadmeadows", "postcode": "3047", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Melton",       "postcode": "3337", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Springvale",   "postcode": "3171", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Cranbourne",   "postcode": "3977", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Doveton",      "postcode": "3177", "state": "VIC", "categories": ["House", "Unit"]},
    {"city": "melbourne", "suburb": "Pakenham",     "postcode": "3810", "state": "VIC", "categories": ["House", "Unit"]},
    # ── Sydney (NSW) ─────────────────────────────────────────────────────────
    # Premium (10)
    {"city": "sydney", "suburb": "Vaucluse",     "postcode": "2030", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Mosman",       "postcode": "2088", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Neutral Bay",  "postcode": "2089", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Paddington",   "postcode": "2021", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Double Bay",   "postcode": "2028", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Rose Bay",     "postcode": "2029", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Balmain",      "postcode": "2041", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Pymble",       "postcode": "2073", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Killara",      "postcode": "2071", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Gordon",       "postcode": "2072", "state": "NSW", "categories": ["House", "Unit"]},
    # Mid-range (10)
    {"city": "sydney", "suburb": "Surry Hills",  "postcode": "2010", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Newtown",      "postcode": "2042", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Chatswood",    "postcode": "2067", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Parramatta",   "postcode": "2150", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Leichhardt",   "postcode": "2040", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Glebe",        "postcode": "2037", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Randwick",     "postcode": "2031", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Burwood",      "postcode": "2134", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Ryde",         "postcode": "2112", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Hornsby",      "postcode": "2077", "state": "NSW", "categories": ["House", "Unit"]},
    # Affordable (10)
    {"city": "sydney", "suburb": "Penrith",      "postcode": "2750", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Tregear",      "postcode": "2770", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Campbelltown", "postcode": "2560", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Fairfield",    "postcode": "2165", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Liverpool",    "postcode": "2170", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Blacktown",    "postcode": "2148", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Bankstown",    "postcode": "2200", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Auburn",       "postcode": "2144", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Cabramatta",   "postcode": "2166", "state": "NSW", "categories": ["House", "Unit"]},
    {"city": "sydney", "suburb": "Kingswood",    "postcode": "2747", "state": "NSW", "categories": ["House", "Unit"]},
    # ── Brisbane (QLD) ───────────────────────────────────────────────────────
    # Premium (10)
    {"city": "brisbane", "suburb": "New Farm",      "postcode": "4005", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Teneriffe",     "postcode": "4005", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Ascot",         "postcode": "4007", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Hamilton",      "postcode": "4007", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Paddington",    "postcode": "4064", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Bulimba",       "postcode": "4171", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Hawthorne",     "postcode": "4171", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "St Lucia",      "postcode": "4067", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Indooroopilly", "postcode": "4068", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Toowong",       "postcode": "4066", "state": "QLD", "categories": ["House", "Unit"]},
    # Mid-range (10)
    {"city": "brisbane", "suburb": "West End",      "postcode": "4101", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Chermside",     "postcode": "4032", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Wynnum",        "postcode": "4178", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Nundah",        "postcode": "4012", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Stafford",      "postcode": "4053", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Sunnybank",     "postcode": "4109", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Coorparoo",     "postcode": "4151", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Moorooka",      "postcode": "4105", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Zillmere",      "postcode": "4034", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Carina",        "postcode": "4152", "state": "QLD", "categories": ["House", "Unit"]},
    # Affordable (10)
    {"city": "brisbane", "suburb": "Logan Central", "postcode": "4114", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Woodridge",     "postcode": "4114", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Inala",         "postcode": "4077", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Deception Bay", "postcode": "4508", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Redcliffe",     "postcode": "4020", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Ipswich",       "postcode": "4305", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Springfield",   "postcode": "4300", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "North Lakes",   "postcode": "4509", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Caboolture",    "postcode": "4510", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "brisbane", "suburb": "Narangba",      "postcode": "4504", "state": "QLD", "categories": ["House", "Unit"]},
    # ── Perth (WA) ───────────────────────────────────────────────────────────
    # Premium (10)
    {"city": "perth", "suburb": "Dalkeith",         "postcode": "6009", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Peppermint Grove", "postcode": "6011", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Cottesloe",        "postcode": "6011", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Nedlands",         "postcode": "6009", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Claremont",        "postcode": "6010", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Subiaco",          "postcode": "6008", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Mount Lawley",     "postcode": "6050", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Bicton",           "postcode": "6157", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Floreat",          "postcode": "6014", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Mosman Park",      "postcode": "6012", "state": "WA", "categories": ["House", "Unit"]},
    # Mid-range (10)
    {"city": "perth", "suburb": "Fremantle",        "postcode": "6160", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Victoria Park",    "postcode": "6100", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Bayswater",        "postcode": "6053", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Bassendean",       "postcode": "6054", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Midland",          "postcode": "6056", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Belmont",          "postcode": "6104", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Maylands",         "postcode": "6051", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Morley",           "postcode": "6062", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Canning Vale",     "postcode": "6155", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "East Fremantle",   "postcode": "6158", "state": "WA", "categories": ["House", "Unit"]},
    # Affordable (10)
    {"city": "perth", "suburb": "Armadale",         "postcode": "6112", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Medina",           "postcode": "6167", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Gosnells",         "postcode": "6110", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Kelmscott",        "postcode": "6111", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Rockingham",       "postcode": "6168", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Mandurah",         "postcode": "6210", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Girrawheen",       "postcode": "6064", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Thornlie",         "postcode": "6108", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Maddington",       "postcode": "6109", "state": "WA", "categories": ["House", "Unit"]},
    {"city": "perth", "suburb": "Kenwick",          "postcode": "6107", "state": "WA", "categories": ["House", "Unit"]},
    # ── Adelaide (SA) ────────────────────────────────────────────────────────
    # Premium (10)
    {"city": "adelaide", "suburb": "Medindie",        "postcode": "5081", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Norwood",         "postcode": "5067", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Burnside",        "postcode": "5066", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Unley",           "postcode": "5061", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Glenelg",         "postcode": "5045", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Toorak Gardens",  "postcode": "5065", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Walkerville",     "postcode": "5081", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Springfield",     "postcode": "5062", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Glen Osmond",     "postcode": "5064", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Malvern",         "postcode": "5061", "state": "SA", "categories": ["House", "Unit"]},
    # Mid-range (10)
    {"city": "adelaide", "suburb": "Prospect",        "postcode": "5082", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Magill",          "postcode": "5072", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Tranmere",        "postcode": "5073", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Goodwood",        "postcode": "5034", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Henley Beach",    "postcode": "5022", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Glenunga",        "postcode": "5064", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Hawthorn",        "postcode": "5062", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Parkside",        "postcode": "5063", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Plympton",        "postcode": "5038", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Torrensville",    "postcode": "5031", "state": "SA", "categories": ["House", "Unit"]},
    # Affordable (10)
    {"city": "adelaide", "suburb": "Elizabeth",       "postcode": "5112", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Morphett Vale",   "postcode": "5162", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Davoren Park",    "postcode": "5113", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Salisbury",       "postcode": "5108", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Parafield Gardens","postcode": "5107", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Smithfield",      "postcode": "5114", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Christies Beach", "postcode": "5165", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Old Noarlunga",   "postcode": "5168", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Seaford",         "postcode": "5169", "state": "SA", "categories": ["House", "Unit"]},
    {"city": "adelaide", "suburb": "Andrews Farm",    "postcode": "5114", "state": "SA", "categories": ["House", "Unit"]},
    # ── Sunshine Coast (QLD) ─────────────────────────────────────────────────
    # Premium (10)
    {"city": "sunshine_coast", "suburb": "Noosa Heads",      "postcode": "4567", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Noosaville",       "postcode": "4566", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Peregian Beach",   "postcode": "4573", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Buderim",          "postcode": "4556", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Mooloolaba",       "postcode": "4557", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Bokarina",         "postcode": "4575", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Tewantin",         "postcode": "4565", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Coolum Beach",     "postcode": "4573", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Sunshine Beach",   "postcode": "4567", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Peregian Springs", "postcode": "4573", "state": "QLD", "categories": ["House", "Unit"]},
    # Mid-range (10)
    {"city": "sunshine_coast", "suburb": "Maroochydore",     "postcode": "4558", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Caloundra",        "postcode": "4551", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Alexandra Headland","postcode": "4572", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Nambour",          "postcode": "4560", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Sippy Downs",      "postcode": "4556", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Mountain Creek",   "postcode": "4557", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Bli Bli",          "postcode": "4560", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Palmview",         "postcode": "4553", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Wurtulla",         "postcode": "4575", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Birtinya",         "postcode": "4575", "state": "QLD", "categories": ["House", "Unit"]},
    # Affordable (10)
    {"city": "sunshine_coast", "suburb": "Landsborough",     "postcode": "4550", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Woombye",          "postcode": "4559", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Maleny",           "postcode": "4552", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Yandina",          "postcode": "4561", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Eumundi",          "postcode": "4562", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Pomona",           "postcode": "4568", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Cooroy",           "postcode": "4563", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Currimundi",       "postcode": "4551", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Beerwah",          "postcode": "4519", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "sunshine_coast", "suburb": "Glass House Mountains", "postcode": "4518", "state": "QLD", "categories": ["House", "Unit"]},
    # ── Gold Coast (QLD) ─────────────────────────────────────────────────────
    # Premium (10)
    {"city": "gold_coast", "suburb": "Mermaid Beach",       "postcode": "4218", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Broadbeach Waters",   "postcode": "4218", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Hope Island",         "postcode": "4212", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Mermaid Waters",      "postcode": "4218", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Burleigh Heads",      "postcode": "4220", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Palm Beach",          "postcode": "4221", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Currumbin",           "postcode": "4223", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Clear Island Waters", "postcode": "4226", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Broadbeach",          "postcode": "4218", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Tallebudgera",        "postcode": "4228", "state": "QLD", "categories": ["House", "Unit"]},
    # Mid-range (10)
    {"city": "gold_coast", "suburb": "Surfers Paradise",    "postcode": "4217", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Southport",           "postcode": "4215", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Varsity Lakes",       "postcode": "4227", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Robina",              "postcode": "4226", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Nerang",              "postcode": "4211", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Bundall",             "postcode": "4217", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Ashmore",             "postcode": "4214", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Runaway Bay",         "postcode": "4216", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Biggera Waters",      "postcode": "4216", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Benowa",              "postcode": "4217", "state": "QLD", "categories": ["House", "Unit"]},
    # Affordable (10)
    {"city": "gold_coast", "suburb": "Coomera",             "postcode": "4209", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Pimpama",             "postcode": "4209", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Ormeau",              "postcode": "4208", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Oxenford",            "postcode": "4210", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Helensvale",          "postcode": "4212", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Arundel",             "postcode": "4214", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Labrador",            "postcode": "4215", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Mudgeeraba",          "postcode": "4213", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Molendinar",          "postcode": "4214", "state": "QLD", "categories": ["House", "Unit"]},
    {"city": "gold_coast", "suburb": "Worongary",           "postcode": "4213", "state": "QLD", "categories": ["House", "Unit"]},
    # ── Canberra (ACT) ───────────────────────────────────────────────────────
    # Premium (10)
    {"city": "canberra", "suburb": "Forrest",     "postcode": "2603", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Griffith",    "postcode": "2603", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Red Hill",    "postcode": "2603", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Deakin",      "postcode": "2600", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Yarralumla",  "postcode": "2600", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Barton",      "postcode": "2600", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "O'Malley",   "postcode": "2606", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Kingston",    "postcode": "2604", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Narrabundah", "postcode": "2604", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Phillip",     "postcode": "2606", "state": "ACT", "categories": ["House", "Unit"]},
    # Mid-range (10)
    {"city": "canberra", "suburb": "Kambah",      "postcode": "2902", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Wanniassa",   "postcode": "2903", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Garran",      "postcode": "2605", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Curtin",      "postcode": "2605", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Hughes",      "postcode": "2605", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Weston",      "postcode": "2611", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Cook",        "postcode": "2614", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Chifley",     "postcode": "2606", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Torrens",     "postcode": "2607", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Pearce",      "postcode": "2607", "state": "ACT", "categories": ["House", "Unit"]},
    # Affordable (10)
    {"city": "canberra", "suburb": "Charnwood",   "postcode": "2615", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Banks",       "postcode": "2906", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Holt",        "postcode": "2615", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Macgregor",   "postcode": "2615", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Latham",      "postcode": "2615", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Fraser",      "postcode": "2615", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Florey",      "postcode": "2615", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Theodore",    "postcode": "2905", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Calwell",     "postcode": "2905", "state": "ACT", "categories": ["House", "Unit"]},
    {"city": "canberra", "suburb": "Gilmore",     "postcode": "2905", "state": "ACT", "categories": ["House", "Unit"]},
]

# One-time retrieval, off by default so this block can be read or run without
# spending credits. Files already recorded in the manifest are skipped, so a
# re-run on a populated cache is free. Left unguarded it fires live calls: a
# verification run on 2026-09-04 spent 2 credits and hit 401s mid-resume.
RUN_DOWNLOAD = False

if RUN_DOWNLOAD:
    _tracker_stats = APICallTracker()
    _manifest      = _load_manifest()
    _skipped       = 0

    print("Downloading suburb performance statistics...")
    print("(Files already on disk are skipped automatically)\n")

    for _entry in STATS_SUBURBS:
        _suburb   = _entry["suburb"]
        _postcode = _entry["postcode"]
        _state    = _entry["state"]
        _city     = _entry["city"]

        for _category in _entry["categories"]:

            _slug     = _suburb.lower().replace(" ", "_")
            _rel_path = f"statistics/{_city}/{_slug}_{_category.lower()}.json"

            if _already_captured(_manifest, _rel_path):
                _skipped += 1
                continue

            _url    = f"{_STATS_BASE}/{_state}/{quote(_suburb, safe='')}/{_postcode}"
            _params = {
                "propertyCategory":               _category,
                "periodSize":                     "quarters",
                "totalPeriods":                   45,
                "startingPeriodRelativeToCurrent": 0,
            }
            _r     = _tracker_stats.get(_url, params=_params)
            _empty = _r.status_code == 200 and not _r.text.strip()

            if _r.status_code == 200 and not _empty:
                _save_json(_rel_path, _r.json())
            elif _empty:
                _save_json(_rel_path, {})
                print(f"  WARNING: empty response for {_suburb} {_category}")
            else:
                print(f"  ERROR {_r.status_code}: {_suburb} {_category} -- {_r.text[:100]}")

            _record_capture(_manifest, _rel_path,
                            endpoint="GET /v2/suburbPerformanceStatistics",
                            params={"suburb": _suburb, "postcode": _postcode,
                                    "state": _state, "propertyCategory": _category,
                                    "totalPeriods": 45},
                            status=_r.status_code,
                            section="part1_statistics",
                            empty=_empty)
            print(f"  {'OK' if not _empty else 'EMPTY':6}  statistics/{_city}/{_slug}_{_category.lower()}.json"
                  f"  [{_tracker_stats.total} credits used]")

    _tracker_stats.checkpoint("Statistics download")
    print(f"\nStatistics done. {_skipped} files already on disk, {_tracker_stats.total} credits spent.")
    _tracker_stats.summary()

Imports and configuration

# Standard library modules for file I/O and suppressing noisy warnings
import json, pathlib, warnings
# Numerical computing and tabular data
import numpy as np
import pandas as pd
# Interactive charting (graph_objects for fine control, express for quick plots, subplots for multi-panel)
import plotly.graph_objects as go
import plotly.express as px
from plotly.subplots import make_subplots
# Interactive geographic map with clickable markers
import folium
from folium.plugins import MarkerCluster
import requests, time
import plotly.io as pio

# One shared chart template so every figure gets the same breathing room: a tall
# top margin gives the title + subtitle their own band (clear of the plot area and
# of band labels like "COVID" / "RBA hikes"), and trimmed side margins let the plot
# use the full panel width.
pio.templates["datatale"] = go.layout.Template(layout=dict(
    margin=dict(l=70, r=50, t=40, b=70),
    # One consistent legend everywhere: outside top-right, no box.
    legend=dict(x=1.02, y=1, xanchor="left", yanchor="top",
                bgcolor="rgba(255,255,255,0)", borderwidth=0),
))
pio.templates.default = "plotly+datatale"
# Plotly + Quarto/closeread render fix. Every stock notebook renderer injects a global
# init (requirejs + an ES-module import of plotly.js) into the page <head>; both stop the
# plain Plotly.newPlot(...) calls from finding window.Plotly, so every chart renders blank.
# This renderer emits one self-contained block per figure -- a plain <script> that loads
# plotly.js from the CDN and draws directly -- with no global init and no requirejs.
from plotly.io.base_renderers import MimetypeRenderer
class _QuartoCDNRenderer(MimetypeRenderer):
    def __init__(self, config=None):
        self.config = config or {}
    def to_mimebundle(self, fig_dict):
        from plotly.io import to_html
        # Per-figure config override: a figure may set layout.meta.crconfig to tweak the
        # Plotly config (e.g. enable scrollZoom on the map only).
        cfg = {"responsive": True, "scrollZoom": False}
        _meta = (fig_dict.get("layout") or {}).get("meta") or {}
        if isinstance(_meta, dict) and isinstance(_meta.get("crconfig"), dict):
            cfg.update(_meta["crconfig"])
        # include_plotlyjs=False: plotly.js is loaded once at body start by _amd-fix.html.
        # Emitting it per figure would make embed-resources inline the whole ~5MB bundle
        # once per chart.
        return {"text/html": to_html(fig_dict, include_plotlyjs=False, full_html=False,
                                     config=cfg, auto_play=False, validate=False)}
pio.renderers["quarto_cdn"] = _QuartoCDNRenderer()
pio.renderers.default = "quarto_cdn"


# Suppress pandas warnings about implicit sort order when concatenating DatetimeIndexes
warnings.filterwarnings("ignore", category=FutureWarning)
# Format floats with comma thousands-separators in DataFrame displays (e.g. 1,234,567)
pd.options.display.float_format = "{:,.0f}".format

# Root of the data tree; set to "." so paths resolve relative to this tale
DATATALE = pathlib.Path(".")
STATS_DIR   = DATATALE / "statistics"   # quarterly suburb performance aggregates
LIST_DIR    = DATATALE / "listings"     # individual property records

# Analysis window: 2018 to 2026; Q1 2018 is the index baseline for every indexed chart
STUDY_START = pd.Timestamp("2018-01-01")
STUDY_END   = pd.Timestamp("2026-12-31")

# Discover which cities have statistics data present on disk at runtime
cities_found = sorted(d.name for d in STATS_DIR.iterdir()
                      if d.is_dir() and d.name != "edge_cases")
print("Statistics available for:", cities_found)

Load statistics into tidy DataFrames (load_stats, all_stats)

def load_stats(path):
    """Read one suburb/category statistics JSON file and return a tidy DataFrame.

    The Domain API returns a nested 'series' > 'seriesInfo' structure; this
    function flattens it into one row per quarter.

    Returns None if the file is empty, missing the 'series' key, or contains
    no quarters with a medianSoldPrice (i.e. the suburb had too few sales to
    report in any quarter).

    Columns in the returned DataFrame (date-indexed, sorted ascending):
        medianSoldPrice, daysOnMarket, numberSold, p25, p75, meanRentYield
    """
    data = json.loads(path.read_text())
    if not data or "series" not in data:
        return None
    rows = data["series"]["seriesInfo"]
    records = [
        {
            "date":            pd.Timestamp(year=r["year"], month=r["month"], day=1),
            "medianSoldPrice": r["values"].get("medianSoldPrice"),
            "daysOnMarket":    r["values"].get("daysOnMarket"),
            "numberSold":      r["values"].get("numberSold"),
            "p25":             r["values"].get("25thPercentileSoldPrice"),
            "p75":             r["values"].get("75thPercentileSoldPrice"),
            "meanRentYield":   r["values"].get("meanRentYield"),
        }
        for r in rows
        if r["values"].get("medianSoldPrice")   # skip quarters with no price data
    ]
    if not records:
        return None
    return pd.DataFrame(records).set_index("date").sort_index()


def pct_change_from_q1_2018(df, col="medianSoldPrice"):
    """Add an 'indexed' column expressing price as % change from the Q1 2018 level.

    Q1 2018 is defined as any record where year==2018 and month in {1, 2, 3}.
    The baseline is the first available price in that window.

    Examples:
        indexed = +80  ->  price is 80% above its Q1 2018 level
        indexed =  -5  ->  price is 5% below its Q1 2018 level
        indexed =   0  ->  price equals its Q1 2018 level (the baseline quarter)

    Returns df unchanged (without an 'indexed' column) if no Q1 2018 data exists.
    """
    mask     = (df.index.year == 2018) & (df.index.month.isin([1, 2, 3]))
    baseline = df.loc[mask, col].dropna()
    if baseline.empty:
        return df
    df = df.copy()
    df["indexed"] = (df[col] / baseline.iloc[0] - 1) * 100   # 0 = Q1 2018 level
    return df


# Nested dict: all_stats[city][suburb_slug][category] = DataFrame
#   city         e.g. "melbourne", "brisbane"
#   suburb_slug  e.g. "hawthorn", "frankston"  (derived from the JSON filename stem)
#   category     "house" or "unit"
all_stats = {}
for city_dir in sorted(STATS_DIR.iterdir()):
    if not city_dir.is_dir() or city_dir.name == "edge_cases":
        continue
    city = city_dir.name
    all_stats[city] = {}
    for f in sorted(city_dir.glob("*.json")):
        # Filename convention: {suburb_slug}_{category}.json
        parts = f.stem.rsplit("_", 1)
        if len(parts) != 2:
            continue
        slug, cat = parts
        df = load_stats(f)
        if df is not None:
            df = pct_change_from_q1_2018(df)
            df = df[df.index >= STUDY_START]
        all_stats[city].setdefault(slug, {})[cat] = df

total = sum(len(s) for s in all_stats.values())
print(f"Loaded {total} suburb/category series across {len(all_stats)} cities")

City colours, labels, and centroids

# One consistent hex colour per city used across every chart in this tale
CITY_COLOURS = {
    "melbourne":      "#1f77b4",
    "sydney":         "#ff7f0e",
    "brisbane":       "#2ca02c",
    "perth":          "#d62728",
    "adelaide":       "#9467bd",
    "sunshine_coast": "#8c564b",
    "gold_coast":     "#e377c2",
    "canberra":       "#bcbd22",
}
# Display names used in chart titles, legend entries, and tooltips
CITY_LABELS = {
    "melbourne":      "Melbourne",
    "sydney":         "Sydney",
    "brisbane":       "Brisbane",
    "perth":          "Perth",
    "adelaide":       "Adelaide",
    "sunshine_coast": "Sunshine Coast",
    "gold_coast":     "Gold Coast",
    "canberra":       "Canberra",
}
# National divergence story: the six mainland capitals
MAJOR_CITIES   = ["melbourne", "sydney", "brisbane", "perth", "adelaide", "canberra"]
# Sea-changer section: Queensland lifestyle/coastal markets
COASTAL_CITIES = ["sunshine_coast", "gold_coast"]

# Approximate city centroids (lat, lon) for the geographic bubble map
CENTROIDS = {
    "melbourne":      (-37.8136, 144.9631),
    "sydney":         (-33.8688, 151.2093),
    "brisbane":       (-27.4698, 153.0251),
    "perth":          (-31.9505, 115.8605),
    "adelaide":       (-34.9285, 138.6007),
    "canberra":       (-35.2809, 149.1300),
    "sunshine_coast": (-26.6500, 153.0667),
    "gold_coast":     (-28.0167, 153.4000),
}

Chart code: the great divergence

Bubble map: house price % change since Q1 2018

# For each city, compute the cross-suburb median of the most recent 'indexed' value.
# This single number represents how much that city's median house price has moved
# since Q1 2018. It drives both bubble size and colour on the geographic map below.
city_index = {}
for city in MAJOR_CITIES + COASTAL_CITIES:
    if city not in all_stats:
        continue
    sl = [
        df["indexed"]
        for suburb_data in all_stats[city].values()
        for cat, df in suburb_data.items()
        if cat == "house" and df is not None and "indexed" in df.columns
    ]
    if sl:
        # Median across all suburbs in the city, then take the last (most recent) value
        med = pd.concat(sl, axis=1).median(axis=1).dropna()
        if not med.empty:
            city_index[city] = med.iloc[-1]

lats    = [CENTROIDS[c][0]          for c in city_index]
lons    = [CENTROIDS[c][1]          for c in city_index]
vals    = list(city_index.values())
labels  = [CITY_LABELS.get(c, c)    for c in city_index]
colours = [CITY_COLOURS.get(c, "#999") for c in city_index]
# Bubble size: shift by +15 so near-zero growth still renders a visible dot
sizes   = [max(15, (v + 15) * 0.28)  for v in vals]
# City names are drawn beside each bubble; the exact percentage shows on hover.

fig = go.Figure(go.Scattergeo(
    lat=lats, lon=lons,
    text=[f"<b>{l}</b>" for l in labels],
    customdata=[f"{v:+.1f}%" for v in vals],
    mode="markers+text",
    # East-coast labels point left so they don't run off the map edge / into the colourbar
    textposition=["middle left" if lo > 149 else "top center" for lo in lons],
    textfont=dict(size=13, color="#1a1a1a"),
    marker=dict(
        size=sizes,
        color=vals,
        # Diverging scale: blue (low/negative growth) -> orange -> red (high growth)
        colorscale=[[0, "#1f77b4"], [0.12, "#ff7f0e"], [1, "#d62728"]],
        cmin=-10, cmax=150,
        # Colourbar sits INSIDE the map, low and to the left, over open ocean:
        # horizontal so it reads as a caption, on a translucent white plate so the
        # ticks stay legible over the water, and short enough to clear the coastline.
        colorbar=dict(
            title=dict(text="% change since Q1 2018", side="top", font=dict(size=11)),
            orientation="h",
            # x and len are starting values only: _resize-plots.html re-derives both
            # from the map's drawn box after layout, because a geo subplot is centred at
            # a fixed aspect, so its left edge moves with the container width.
            x=0.18, xanchor="left",
            y=0.05, yanchor="bottom",
            len=0.44, thickness=16,
            tickfont=dict(size=10),
            outlinewidth=0,
            bgcolor="rgba(255,255,255,0.72)",
        ),
        line=dict(width=1.5, color="white"),
        opacity=0.88,
    ),
    hovertemplate="%{customdata}<extra></extra>",
))

fig.update_geos(
    lataxis_range=[-45, -10],
    lonaxis_range=[110, 155],
    showland=True,
    landcolor="#f0ede8",
    showocean=True,
    oceancolor="#d4e8f5",
    showcoastlines=True,
    coastlinecolor="#aaa",
    showsubunits=True,
    subunitcolor="#ccc",
    showframe=False,
)
fig.update_layout(
    # Sized to sit alongside tale 2's map, which is 88vh. The geo keeps its own
    # aspect and letterboxes inside this box, so the framing is unchanged - only the
    # drawn size grows. A px height cannot track vh, so this matches at a ~800px
    # viewport and is close either side of it.
    height=785,
    margin=dict(l=0, r=10, t=10, b=0),
    geo=dict(bgcolor="rgba(0,0,0,0)", domain=dict(x=[0.0, 1.0], y=[0.0, 1.0])),
    # Transparent paper so the map blends into the closeread sticky panel.
    paper_bgcolor="rgba(0,0,0,0)",
    # Scroll zooms this map (per-figure); zoom-OUT is capped at Australia by _resize-plots.html.
    meta=dict(crconfig=dict(scrollZoom=True)),
    modebar_remove=["zoomOutGeo"],
)

fig

Brisbane vs Melbourne crossover (actual dollars)

# For each quarter, collect suburb-level medianSoldPrice values for each city,
# then take the median of those suburb medians. This gives a city-wide price
# estimate that is more stable than any single suburb but reflects the whole market.
bris_abs, melb_abs = {}, {}

for city, store in [('brisbane', bris_abs), ('melbourne', melb_abs)]:
    for slug, cats in all_stats.get(city, {}).items():
        df = cats.get('house')
        if df is None or 'medianSoldPrice' not in df.columns:
            continue
        for dt, row in df.iterrows():
            p = row['medianSoldPrice']
            if pd.isna(p):
                continue
            store.setdefault(dt, []).append(p)

# Median of suburb medians for each quarter
bris_series = pd.Series(
    {dt: pd.Series(v).median() for dt, v in bris_abs.items()}
).sort_index()
melb_series = pd.Series(
    {dt: pd.Series(v).median() for dt, v in melb_abs.items()}
).sort_index()

bris_series = bris_series[bris_series.index >= STUDY_START]
melb_series = melb_series[melb_series.index >= STUDY_START]

# Find crossover: the first quarter where Brisbane's median >= Melbourne's median
crossover_date = None
for dt in bris_series.index:
    if dt in melb_series.index and bris_series[dt] >= melb_series[dt]:
        crossover_date = dt
        break

fig = go.Figure()

fig.add_trace(go.Scatter(
    x=bris_series.index, y=bris_series.values,
    name='Brisbane',
    mode='lines+markers',
    marker=dict(size=4),
    line=dict(color=CITY_COLOURS['brisbane'], width=2.5),
    hovertemplate='Brisbane: $%{y:,.0f}<extra></extra>',
))
fig.add_trace(go.Scatter(
    x=melb_series.index, y=melb_series.values,
    name='Melbourne',
    mode='lines+markers',
    marker=dict(size=4),
    line=dict(color=CITY_COLOURS['melbourne'], width=2.5),
    hovertemplate='Melbourne: $%{y:,.0f}<extra></extra>',
))

# Grey band: Australian COVID lockdown period
fig.add_vrect(
    x0='2020-03-01', x1='2020-12-01',
    fillcolor='grey', opacity=0.14, line_width=0,
    annotation_text='COVID', annotation_position='top left',
    annotation_font_size=12,
)

# Dotted line: first RBA rate hike (May 2022)
fig.add_vline(
    x='2022-05-01', line_dash='dot', line_color='#555', opacity=0.8,
    annotation_text='RBA hikes begin', annotation_position='bottom right',
    annotation_font_size=12,
)

if crossover_date is not None:
    fig.add_vline(
        x=crossover_date.isoformat()[:10],
        line_dash='dot', line_color='#555', opacity=0.8,
        annotation_text='Brisbane overtakes',
        annotation_position='top left',
        annotation_font_size=12,
    )

fig.update_layout(
    yaxis_title='Median price ($)',
    yaxis=dict(tickprefix='$', tickformat=',.0f', gridcolor='#eee'),
    xaxis=dict(gridcolor='#eee'),
    height=640,
    hovermode='x unified',
    plot_bgcolor='white',
    legend=dict(x=1.02, y=1, xanchor="left"),
)
fig

Multi-city index: % change since Q1 2018

# Time series: for each city, compute the cross-suburb median of the 'indexed' column
# (% change since Q1 2018) at each quarter, then plot all cities on one chart.
# Major capitals use solid lines; coastal lifestyle markets use dashed lines.
fig = go.Figure()

for city in MAJOR_CITIES + COASTAL_CITIES:
    if city not in all_stats:
        continue
    sl = [
        df["indexed"]
        for suburb_data in all_stats[city].values()
        for cat, df in suburb_data.items()
        if cat == "house" and df is not None and "indexed" in df.columns
    ]
    if not sl:
        continue
    med  = pd.concat(sl, axis=1).median(axis=1)
    medp = med[(med.index >= STUDY_START) & (med.index <= STUDY_END)]
    fig.add_trace(go.Scatter(
        x=medp.index, y=medp.values,
        name=CITY_LABELS.get(city, city),
        mode="lines",
        line=dict(
            color=CITY_COLOURS.get(city, "#999"),
            width=2.5 if city in MAJOR_CITIES else 1.8,
            dash="solid" if city in MAJOR_CITIES else "dash",
        ),
        hovertemplate="%{fullData.name}: %{y:.0f}<extra></extra>",
    ))

# Horizontal reference at 0% (= Q1 2018 price level for every city)
fig.add_hline(y=0, line_dash="dot", line_color="#888", opacity=0.7,
              annotation_text="Q1 2018 (0%)",
              annotation_position="bottom right")

# Grey band: Australian COVID lockdown period
fig.add_vrect(x0="2020-03-01", x1="2020-12-01",
              fillcolor="grey", opacity=0.14, line_width=0,
              annotation_text="COVID", annotation_position="top left",
              annotation_font_size=12)

# Dotted line: first RBA rate hike (May 2022), the key inflection point
fig.add_vline(x="2022-05-01", line_dash="dot", line_color="#555", opacity=0.8,
              annotation_text="RBA hikes begin", annotation_position="top right",
              annotation_font_size=12)

fig.update_layout(
    yaxis_title="% change since Q1 2018",
    xaxis_title="",
    height=640,
    hovermode="x unified",
    legend=dict(x=1.02, y=1, xanchor="left"),
    plot_bgcolor="white",
    yaxis=dict(gridcolor="#eee", zeroline=True, zerolinecolor="#bbb", zerolinewidth=1),
    xaxis=dict(gridcolor="#eee"),
)
fig

House vs Unit toggle

# Build two sets of traces: one for houses, one for units.
# Houses: solid lines. Units: dotted lines (same colour, visually distinct in Both view).
# The Plotly updatemenus buttons below toggle visibility between three states:
# House only, Unit only, or Both simultaneously.
house_traces, unit_traces = [], []

for city in MAJOR_CITIES:
    if city not in all_stats:
        continue
    colour = CITY_COLOURS.get(city, "#999")
    label  = CITY_LABELS.get(city, city)

    for cat, store in [("house", house_traces), ("unit", unit_traces)]:
        sl = [
            df["indexed"]
            for suburb_data in all_stats[city].values()
            for c2, df in suburb_data.items()
            if c2 == cat and df is not None and "indexed" in df.columns
        ]
        if not sl:
            continue
        med  = pd.concat(sl, axis=1).median(axis=1)
        medp = med[(med.index >= STUDY_START) & (med.index <= STUDY_END)]
        store.append(go.Scatter(
            x=medp.index, y=medp.values,
            mode="lines",
            name=f"{label}",
            legendgroup=city,
            line=dict(
                color=colour,
                width=2.2,
                dash="dot" if cat == "unit" else "solid",
            ),
            hovertemplate=f"{label} {cat}: %{{y:.0f}}<extra></extra>",
        ))

all_traces = house_traces + unit_traces
n_h = len(house_traces)
n_u = len(unit_traces)
# Boolean arrays that set trace visibility for each button state
vis_h    = [True]  * n_h + [False] * n_u   # House only
vis_u    = [False] * n_h + [True]  * n_u   # Unit only
vis_both = [True]  * (n_h + n_u)           # Both

# Fixed y-range spanning house + unit so all three toggle states share one scale
_ys = [float(v) for t in all_traces for v in t.y if v == v]
_pad = (max(_ys) - min(_ys)) * 0.05
_yrange = [min(_ys) - _pad, max(_ys) + _pad]

fig = go.Figure(all_traces)
fig.add_hline(y=0, line_dash="dot", line_color="#888", opacity=0.6,
              annotation_text="Q1 2018 (0%)", annotation_position="bottom right")
fig.update_layout(
    yaxis_title="% change since Q1 2018",
    height=640,
    hovermode="x unified",
    plot_bgcolor="white",
    yaxis=dict(gridcolor="#eee", zeroline=True, zerolinecolor="#bbb", zerolinewidth=1, range=_yrange, autorange=False),
    xaxis=dict(gridcolor="#eee"),
    updatemenus=[dict(
        type="buttons",
        direction="left",
        active=0,
        buttons=[
            dict(label="Both",  method="update",
                 args=[{"visible": vis_both}]),
            dict(label="House", method="update",
                 args=[{"visible": vis_h}]),
            dict(label="Unit",  method="update",
                 args=[{"visible": vis_u}]),
        ],
        pad={"r": 10, "t": 10},
        x=0.0, xanchor="left", y=1.14, yanchor="top",
        bgcolor="#f0f0f0",
        bordercolor="#ccc",
    )],
)
fig

Gross rental yield by city

# For each major city, compute the cross-suburb median of the meanRentYield column
# (annual rent as % of property value) and plot as a time series.
# Rising yield = rents are growing faster than prices (or prices are falling).
# Falling yield = prices are growing faster than rents.
fig = go.Figure()

for city in MAJOR_CITIES:
    if city not in all_stats:
        continue
    ys = [
        df["meanRentYield"].dropna()
        for suburb_data in all_stats[city].values()
        for cat, df in suburb_data.items()
        if cat == "house" and df is not None and "meanRentYield" in df.columns
    ]
    if not ys:
        continue
    y_med  = pd.concat(ys, axis=1).median(axis=1)
    y_medp = y_med[(y_med.index >= STUDY_START) & (y_med.index <= STUDY_END)]
    fig.add_trace(go.Scatter(
        x=y_medp.index, y=y_medp.values,
        name=CITY_LABELS.get(city, city),
        mode="lines",
        line=dict(color=CITY_COLOURS.get(city, "#999"), width=2.2),
        hovertemplate="%{fullData.name}: %{y:.2f}%<extra></extra>",
    ))

fig.update_layout(
    yaxis_title="Gross yield (%)",
    height=640,
    hovermode="x unified",
    plot_bgcolor="white",
    yaxis=dict(gridcolor="#eee"),
    xaxis=dict(gridcolor="#eee"),
    legend=dict(x=1.02, y=1, xanchor="left"),
)
fig

Coastal markets vs Brisbane and Sydney

# Coastal markets (Sunshine Coast, Gold Coast) vs Brisbane and Sydney as context.
# Coastal cities use solid lines and heavier weight; capital cities use dashed lines.
# The shaded band marks the peak COVID lifestyle-migration period (mid-2020 to mid-2022).
fig = go.Figure()

for city in COASTAL_CITIES + ["brisbane", "sydney"]:
    if city not in all_stats:
        continue
    sl = [
        df["indexed"]
        for suburb_data in all_stats[city].values()
        for cat, df in suburb_data.items()
        if cat == "house" and df is not None and "indexed" in df.columns
    ]
    if not sl:
        continue
    med  = pd.concat(sl, axis=1).median(axis=1)
    medp = med[(med.index >= STUDY_START) & (med.index <= STUDY_END)]
    is_coastal = city in COASTAL_CITIES
    fig.add_trace(go.Scatter(
        x=medp.index, y=medp.values,
        name=CITY_LABELS.get(city, city),
        mode="lines",
        line=dict(
            color=CITY_COLOURS.get(city, "#999"),
            width=2.5 if is_coastal else 1.8,
            dash="solid" if is_coastal else "dash",
        ),
        hovertemplate="%{fullData.name}: %{y:.0f}<extra></extra>",
    ))

fig.add_hline(y=0, line_dash="dot", line_color="#888", opacity=0.6,
              annotation_text="Q1 2018 (0%)", annotation_position="bottom right")
fig.add_vrect(
    x0="2020-03-01", x1="2022-06-01",
    fillcolor="#8c564b", opacity=0.06, line_width=0,
    annotation_text="Peak COVID migration", annotation_position="top left",
    annotation_font_size=12,
)
fig.update_layout(
    yaxis_title="% change since Q1 2018",
    height=640,
    hovermode="x unified",
    plot_bgcolor="white",
    yaxis=dict(gridcolor="#eee", zeroline=True, zerolinecolor="#bbb", zerolinewidth=1),
    xaxis=dict(gridcolor="#eee"),
)
fig