.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/tables_and_summaries/build_ablation_table.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_tables_and_summaries_build_ablation_table.py: Build ablation tables from sensitivity records ============================================== This example teaches you how to use GeoPrior's ``build-ablation-table`` utility. Unlike the figure-generation scripts, this command does not start from a plotting function. It starts from raw or semi-processed ablation records and turns them into tidy, reusable analysis tables. Why this matters ---------------- Ablation experiments are only useful if their outputs can be compared, filtered, exported, and reused downstream. This builder helps turn a folder of ``ablation_record*.jsonl`` files into: - one tidy ablation table, - one optional best-per-city table, - optional grouped S6 lambda grids, - optional grouped S7 toggle summaries. That makes it a strong first lesson for the ``tables_and_summaries`` section. .. GENERATED FROM PYTHON SOURCE LINES 31-36 Imports ------- We call the real production entrypoint from the project code. Then we read the generated tables back in and build one compact visual preview for the lesson page. .. GENERATED FROM PYTHON SOURCE LINES 36-51 .. code-block:: Python from __future__ import annotations import json import tempfile from pathlib import Path import matplotlib.pyplot as plt import numpy as np import pandas as pd from geoprior.scripts.build_ablation_table import ( build_ablation_table_main, ) .. GENERATED FROM PYTHON SOURCE LINES 52-76 Build a compact synthetic ablation study ---------------------------------------- The real script expects ablation records such as ``ablation_record*.jsonl``. Each record is a dictionary with run metadata and metrics. For the lesson, we create a small synthetic study with: - two cities, - two physics buckets, - a lambda_prior × lambda_cons grid, - overall metrics, - and per-horizon metrics. We also include the toggle-style fields used by the script's S7 grouped summary logic: - lambda_smooth - lambda_bounds - lambda_mv - lambda_q - use_effective_h - kappa_mode - hd_factor .. GENERATED FROM PYTHON SOURCE LINES 76-198 .. code-block:: Python lambda_prior_vals = [0.0, 0.1, 0.3, 1.0] lambda_cons_vals = [0.0, 0.1, 0.3, 1.0] rows: list[dict[str, object]] = [] counter = 0 for city in ["Nansha", "Zhongshan"]: city_shift = 0.0 if city == "Nansha" else 0.55 for pde_mode in ["both", "none"]: phys_penalty = 0.0 if pde_mode == "both" else 1.25 for lp in lambda_prior_vals: for lc in lambda_cons_vals: counter += 1 log_lp = np.log10(lp + 0.12) log_lc = np.log10(lc + 0.12) mae = ( 4.8 + city_shift + phys_penalty + 1.8 * (log_lp + 0.22) ** 2 + 1.1 * (log_lc + 0.18) ** 2 ) rmse = mae * 1.18 mse = rmse**2 epsilon_prior = ( 0.26 + 0.04 * city_shift + 0.08 * (log_lp + 0.22) ** 2 + 0.03 * (log_lc + 0.10) ** 2 + 0.08 * (pde_mode == "none") ) epsilon_cons = ( 0.22 + 0.04 * city_shift + 0.03 * (log_lp + 0.08) ** 2 + 0.09 * (log_lc + 0.25) ** 2 + 0.08 * (pde_mode == "none") ) coverage80 = np.clip( 0.95 - 0.08 * epsilon_prior - 0.06 * epsilon_cons, 0.65, 0.98, ) sharpness80 = ( 15.0 + 3.5 * (log_lp + 0.18) ** 2 + 5.5 * (log_lc + 0.25) ** 2 + 1.0 * (pde_mode == "none") ) r2 = np.clip( 0.95 - 0.050 * mae - 0.004 * sharpness80, 0.10, 0.96, ) use_effective_h = bool( city == "Zhongshan" or lp >= 0.3 ) hd_factor = 0.6 if use_effective_h else 1.0 kappa_mode = "bar" if city == "Nansha" else "kb" row = { "timestamp": ( f"2026-03-28T12:{counter:02d}:00" ), "city": city, "model": "GeoPriorSubsNet", "pde_mode": pde_mode, "use_effective_h": use_effective_h, "kappa_mode": kappa_mode, "hd_factor": float(hd_factor), "lambda_prior": float(lp), "lambda_cons": float(lc), "lambda_gw": ( 0.20 if pde_mode == "both" else 0.0 ), "lambda_smooth": 0.40 if lp >= 0.3 else 0.0, "lambda_mv": 0.08 if lp >= 0.1 else 0.0, "lambda_bounds": 0.05 if lc >= 0.1 else 0.0, "lambda_q": 0.03 if lc >= 0.3 else 0.0, "mae": float(mae), "rmse": float(rmse), "mse": float(mse), "r2": float(r2), "coverage80": float(coverage80), "sharpness80": float(sharpness80), "epsilon_prior": float(epsilon_prior), "epsilon_cons": float(epsilon_cons), "units": {"subs_metrics_unit": "mm"}, "per_horizon_mae": { "H1": float(mae * 0.86), "H2": float(mae), "H3": float(mae * 1.15), }, "per_horizon_r2": { "H1": float(min(0.99, r2 + 0.04)), "H2": float(r2), "H3": float(max(0.0, r2 - 0.05)), }, } rows.append(row) print(f"Number of synthetic records: {len(rows)}") print("") print("Example record") print(json.dumps(rows[0], indent=2)) .. rst-class:: sphx-glr-script-out .. code-block:: none Number of synthetic records: 64 Example record { "timestamp": "2026-03-28T12:01:00", "city": "Nansha", "model": "GeoPriorSubsNet", "pde_mode": "both", "use_effective_h": false, "kappa_mode": "bar", "hd_factor": 1.0, "lambda_prior": 0.0, "lambda_cons": 0.0, "lambda_gw": 0.2, "lambda_smooth": 0.0, "lambda_mv": 0.0, "lambda_bounds": 0.0, "lambda_q": 0.0, "mae": 6.287758135432752, "rmse": 7.419554599810647, "mse": 55.049790459571334, "r2": 0.5580287676470639, "coverage80": 0.9075371302077502, "sharpness80": 19.395831395324617, "epsilon_prior": 0.31950405687650674, "epsilon_cons": 0.2817090873688203, "units": { "subs_metrics_unit": "mm" }, "per_horizon_mae": { "H1": 5.407471996472167, "H2": 6.287758135432752, "H3": 7.230921855747664 }, "per_horizon_r2": { "H1": 0.5980287676470639, "H2": 0.5580287676470639, "H3": 0.5080287676470638 } } .. GENERATED FROM PYTHON SOURCE LINES 199-203 Write the synthetic JSONL file ------------------------------ The production command reads ablation records from files, so we follow that same workflow here. .. GENERATED FROM PYTHON SOURCE LINES 203-216 .. code-block:: Python tmp_dir = Path( tempfile.mkdtemp(prefix="gp_sg_ablation_table_") ) jsonl_path = tmp_dir / "ablation_record.synthetic.jsonl" with jsonl_path.open("w", encoding="utf-8") as f: for rec in rows: f.write(json.dumps(rec) + "\n") print("") print(f"Input JSONL written to: {jsonl_path}") .. rst-class:: sphx-glr-script-out .. code-block:: none Input JSONL written to: /tmp/gp_sg_ablation_table_gl6qbufn/ablation_record.synthetic.jsonl .. GENERATED FROM PYTHON SOURCE LINES 217-227 Run the real ablation-table builder ----------------------------------- We ask the script to produce: - the main tidy table, - a best-per-city table, - grouped S6 lambda grids, - and an S7 grouped toggle summary. We keep the output in CSV/JSON/TXT form for the lesson page. .. GENERATED FROM PYTHON SOURCE LINES 227-261 .. code-block:: Python out_stem = "ablation_table_gallery" build_ablation_table_main( [ "--input", str(jsonl_path), "--out-dir", str(tmp_dir), "--out", out_stem, "--formats", "csv,json,txt", "--sort-by", "mae", "--ascending", "auto", "--metric-unit", "mm", "--keep-per-horizon", "true", "--best-per-city", "--group-cols", "s6,s7", "--s6-metrics", "mae,coverage80,sharpness80", "--s7-metrics", "mae,coverage80,sharpness80", "--s7-agg", "mean", ], prog="build-ablation-table", ) .. rst-class:: sphx-glr-script-out .. code-block:: none [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery.txt [OK] best/city -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__best_per_city.csv [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__mae__Nansha__both.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__mae__Nansha__both.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__mae__Nansha__both.txt [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__coverage80__Nansha__both.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__coverage80__Nansha__both.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__coverage80__Nansha__both.txt [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__sharpness80__Nansha__both.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__sharpness80__Nansha__both.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__sharpness80__Nansha__both.txt [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__mae__Nansha__none.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__mae__Nansha__none.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__mae__Nansha__none.txt [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__coverage80__Nansha__none.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__coverage80__Nansha__none.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__coverage80__Nansha__none.txt [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__sharpness80__Nansha__none.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__sharpness80__Nansha__none.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__sharpness80__Nansha__none.txt [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__mae__Zhongshan__both.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__mae__Zhongshan__both.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__mae__Zhongshan__both.txt [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__coverage80__Zhongshan__both.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__coverage80__Zhongshan__both.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__coverage80__Zhongshan__both.txt [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__sharpness80__Zhongshan__both.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__sharpness80__Zhongshan__both.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__sharpness80__Zhongshan__both.txt [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__mae__Zhongshan__none.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__mae__Zhongshan__none.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__mae__Zhongshan__none.txt [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__coverage80__Zhongshan__none.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__coverage80__Zhongshan__none.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__coverage80__Zhongshan__none.txt [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__sharpness80__Zhongshan__none.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__sharpness80__Zhongshan__none.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S6__sharpness80__Zhongshan__none.txt [OK] csv -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S7.csv [OK] json -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S7.json [OK] txt -> /tmp/gp_sg_ablation_table_gl6qbufn/ablation_table_gallery__S7.txt .. GENERATED FROM PYTHON SOURCE LINES 262-266 Inspect the produced files -------------------------- The command writes multiple outputs. For the lesson, we list the main ones so the user sees the artifact family clearly. .. GENERATED FROM PYTHON SOURCE LINES 266-274 .. code-block:: Python written = sorted(tmp_dir.glob("ablation_table_gallery*")) print("") print("Written files") for p in written: print(" -", p.name) .. rst-class:: sphx-glr-script-out .. code-block:: none Written files - ablation_table_gallery.csv - ablation_table_gallery.json - ablation_table_gallery.txt - ablation_table_gallery__S6__coverage80__Nansha__both.csv - ablation_table_gallery__S6__coverage80__Nansha__both.json - ablation_table_gallery__S6__coverage80__Nansha__both.txt - ablation_table_gallery__S6__coverage80__Nansha__none.csv - ablation_table_gallery__S6__coverage80__Nansha__none.json - ablation_table_gallery__S6__coverage80__Nansha__none.txt - ablation_table_gallery__S6__coverage80__Zhongshan__both.csv - ablation_table_gallery__S6__coverage80__Zhongshan__both.json - ablation_table_gallery__S6__coverage80__Zhongshan__both.txt - ablation_table_gallery__S6__coverage80__Zhongshan__none.csv - ablation_table_gallery__S6__coverage80__Zhongshan__none.json - ablation_table_gallery__S6__coverage80__Zhongshan__none.txt - ablation_table_gallery__S6__mae__Nansha__both.csv - ablation_table_gallery__S6__mae__Nansha__both.json - ablation_table_gallery__S6__mae__Nansha__both.txt - ablation_table_gallery__S6__mae__Nansha__none.csv - ablation_table_gallery__S6__mae__Nansha__none.json - ablation_table_gallery__S6__mae__Nansha__none.txt - ablation_table_gallery__S6__mae__Zhongshan__both.csv - ablation_table_gallery__S6__mae__Zhongshan__both.json - ablation_table_gallery__S6__mae__Zhongshan__both.txt - ablation_table_gallery__S6__mae__Zhongshan__none.csv - ablation_table_gallery__S6__mae__Zhongshan__none.json - ablation_table_gallery__S6__mae__Zhongshan__none.txt - ablation_table_gallery__S6__sharpness80__Nansha__both.csv - ablation_table_gallery__S6__sharpness80__Nansha__both.json - ablation_table_gallery__S6__sharpness80__Nansha__both.txt - ablation_table_gallery__S6__sharpness80__Nansha__none.csv - ablation_table_gallery__S6__sharpness80__Nansha__none.json - ablation_table_gallery__S6__sharpness80__Nansha__none.txt - ablation_table_gallery__S6__sharpness80__Zhongshan__both.csv - ablation_table_gallery__S6__sharpness80__Zhongshan__both.json - ablation_table_gallery__S6__sharpness80__Zhongshan__both.txt - ablation_table_gallery__S6__sharpness80__Zhongshan__none.csv - ablation_table_gallery__S6__sharpness80__Zhongshan__none.json - ablation_table_gallery__S6__sharpness80__Zhongshan__none.txt - ablation_table_gallery__S7.csv - ablation_table_gallery__S7.json - ablation_table_gallery__S7.txt - ablation_table_gallery__best_per_city.csv .. GENERATED FROM PYTHON SOURCE LINES 275-281 Read the main table and the grouped outputs ------------------------------------------- The main table is the tidy run-level export. S6 is the lambda grid summary. S7 is the grouped toggle summary. The best-per-city table is a very useful quick summary. .. GENERATED FROM PYTHON SOURCE LINES 281-309 .. code-block:: Python main_csv = tmp_dir / "ablation_table_gallery.csv" best_csv = ( tmp_dir / "ablation_table_gallery__best_per_city.csv" ) s6_csv = ( tmp_dir / "ablation_table_gallery__S6__mae__Nansha__both.csv" ) s7_csv = tmp_dir / "ablation_table_gallery__S7.csv" tab = pd.read_csv(main_csv) best = pd.read_csv(best_csv) s6 = pd.read_csv(s6_csv) s7 = pd.read_csv(s7_csv) print("") print("Main tidy table") print(tab.head(8).to_string(index=False)) print("") print("Best per city") print(best.to_string(index=False)) print("") print("S7 grouped summary") print(s7.head(12).to_string(index=False)) .. rst-class:: sphx-glr-script-out .. code-block:: none Main tidy table timestamp city model pde_mode use_effective_h kappa_mode hd_factor lambda_cons lambda_gw lambda_prior lambda_smooth lambda_mv lambda_bounds lambda_q mae rmse mse r2 coverage80 sharpness80 epsilon_prior epsilon_cons per_horizon_mae.H1 per_horizon_mae.H2 per_horizon_mae.H3 per_horizon_r2.H1 per_horizon_r2.H2 per_horizon_r2.H3 2026-03-28T12:11:00 Nansha GeoPriorSubsNet both True bar 0.6000 0.3000 0.2000 0.3000 0.4000 0.0800 0.0500 0.0300 4.8868 5.7664 33.2518 0.6448 0.9154 15.2238 0.2643 0.2241 4.2027 4.8868 5.6198 0.6848 0.6448 0.5948 2026-03-28T12:12:00 Nansha GeoPriorSubsNet both True bar 0.6000 1.0000 0.2000 0.3000 0.4000 0.0800 0.0500 0.0300 4.9020 5.7844 33.4591 0.6424 0.9151 15.6279 0.2626 0.2307 4.2157 4.9020 5.6373 0.6824 0.6424 0.5924 2026-03-28T12:15:00 Nansha GeoPriorSubsNet both True bar 0.6000 0.3000 0.2000 1.0000 0.4000 0.0800 0.0500 0.0300 4.9730 5.8682 34.4357 0.6403 0.9152 15.2723 0.2681 0.2219 4.2768 4.9730 5.7190 0.6803 0.6403 0.5903 2026-03-28T12:16:00 Nansha GeoPriorSubsNet both True bar 0.6000 1.0000 0.2000 1.0000 0.4000 0.0800 0.0500 0.0300 4.9883 5.8861 34.6467 0.6379 0.9150 15.6763 0.2665 0.2286 4.2899 4.9883 5.7365 0.6779 0.6379 0.5879 2026-03-28T12:10:00 Nansha GeoPriorSubsNet both True bar 0.6000 0.1000 0.2000 0.3000 0.4000 0.0800 0.0500 0.0000 5.0951 6.0122 36.1470 0.6310 0.9140 16.0491 0.2713 0.2376 4.3818 5.0951 5.8594 0.6710 0.6310 0.5810 2026-03-28T12:14:00 Nansha GeoPriorSubsNet both True bar 0.6000 0.1000 0.2000 1.0000 0.4000 0.0800 0.0500 0.0000 5.1813 6.1140 37.3809 0.6265 0.9139 16.0975 0.2751 0.2355 4.4560 5.1813 5.9586 0.6665 0.6265 0.5765 2026-03-28T12:07:00 Nansha GeoPriorSubsNet both False bar 1.0000 0.3000 0.2000 0.1000 0.0000 0.0800 0.0500 0.0300 5.1872 6.1209 37.4659 0.6271 0.9139 15.8866 0.2776 0.2315 4.4610 5.1872 5.9653 0.6671 0.6271 0.5771 2026-03-28T12:08:00 Nansha GeoPriorSubsNet both False bar 1.0000 1.0000 0.2000 0.1000 0.0000 0.0800 0.0500 0.0300 5.2024 6.1389 37.6860 0.6247 0.9136 16.2907 0.2760 0.2381 4.4741 5.2024 5.9828 0.6647 0.6247 0.5747 Best per city timestamp city model pde_mode use_effective_h kappa_mode hd_factor lambda_cons lambda_gw lambda_prior lambda_smooth lambda_mv lambda_bounds lambda_q mae rmse mse r2 coverage80 sharpness80 epsilon_prior epsilon_cons per_horizon_mae.H1 per_horizon_mae.H2 per_horizon_mae.H3 per_horizon_r2.H1 per_horizon_r2.H2 per_horizon_r2.H3 2026-03-28T12:11:00 Nansha GeoPriorSubsNet both True bar 0.6000 0.3000 0.2000 0.3000 0.4000 0.0800 0.0500 0.0300 4.8868 5.7664 33.2518 0.6448 0.9154 15.2238 0.2643 0.2241 4.2027 4.8868 5.6198 0.6848 0.6448 0.5948 2026-03-28T12:43:00 Zhongshan GeoPriorSubsNet both True kb 0.6000 0.3000 0.2000 0.3000 0.4000 0.0800 0.0500 0.0300 5.4368 6.4154 41.1578 0.6173 0.9123 15.2238 0.2863 0.2461 4.6757 5.4368 6.2523 0.6573 0.6173 0.5673 S7 grouped summary city pde_bucket use_effective_h kappa_mode hd_factor smooth_on bounds_on mv_on q_on n mae coverage80 sharpness80 Nansha both False bar 1.0000 off off off off 1 6.2878 0.9075 19.3958 Nansha both False bar 1.0000 off off on off 1 5.7483 0.9101 18.2733 Nansha both False bar 1.0000 off on off off 1 5.9350 0.9099 17.8345 Nansha both False bar 1.0000 off on off on 2 5.7343 0.9112 17.2112 Nansha both False bar 1.0000 off on on off 1 5.3955 0.9125 16.7119 Nansha both False bar 1.0000 off on on on 2 5.1948 0.9138 16.0887 Nansha both True bar 0.6000 on off on off 2 5.4910 0.9115 17.6347 Nansha both True bar 0.6000 on on on off 2 5.1382 0.9140 16.0733 Nansha both True bar 0.6000 on on on on 4 4.9375 0.9152 15.4501 Nansha none False bar 1.0000 off off off off 1 7.5378 0.8963 20.3958 Nansha none False bar 1.0000 off off on off 1 6.9983 0.8989 19.2733 Nansha none False bar 1.0000 off on off off 1 7.1850 0.8987 18.8345 .. GENERATED FROM PYTHON SOURCE LINES 310-320 Build one compact visual preview from the generated tables ---------------------------------------------------------- This is not part of the production builder itself. It is a teaching aid for the gallery page. Left: an S6 heatmap for MAE in Nansha with physics on. Right: the best-per-city MAE summary. .. GENERATED FROM PYTHON SOURCE LINES 320-385 .. code-block:: Python # Rebuild the S6 matrix from the flat CSV. lcons = pd.to_numeric( s6.iloc[:, 0], errors="coerce" ).to_numpy() lpris = np.array([float(c) for c in s6.columns[1:]]) heat = s6.iloc[:, 1:].to_numpy(dtype=float) fig, axes = plt.subplots( 1, 2, figsize=(9.0, 3.8), constrained_layout=True, ) # S6 heatmap ax = axes[0] im = ax.imshow( heat, origin="lower", aspect="auto", ) ax.set_title("S6 preview: MAE grid\nNansha, physics=both") ax.set_xlabel(r"$\lambda_{\mathrm{prior}}$") ax.set_ylabel(r"$\lambda_{\mathrm{cons}}$") ax.set_xticks(np.arange(len(lpris))) ax.set_yticks(np.arange(len(lcons))) ax.set_xticklabels([f"{x:g}" for x in lpris]) ax.set_yticklabels([f"{x:g}" for x in lcons]) # Mark the best cell. i_best, j_best = np.unravel_index( np.nanargmin(heat), heat.shape, ) ax.scatter( [j_best], [i_best], marker="o", s=55, facecolors="none", edgecolors="white", linewidths=1.6, ) cbar = fig.colorbar(im, ax=ax, fraction=0.046, pad=0.04) cbar.set_label("MAE [mm]") # Best-per-city bar chart ax = axes[1] ax.bar(best["city"], best["mae"]) ax.set_title("Best-per-city summary") ax.set_ylabel("MAE [mm]") ax.set_xlabel("City") for i, v in enumerate(best["mae"].to_numpy(float)): ax.text( i, v + 0.06, f"{v:.2f}", ha="center", va="bottom", fontsize=9, ) .. image-sg:: /auto_examples/tables_and_summaries/images/sphx_glr_build_ablation_table_001.png :alt: S6 preview: MAE grid Nansha, physics=both, Best-per-city summary :srcset: /auto_examples/tables_and_summaries/images/sphx_glr_build_ablation_table_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 386-406 Learn how to read the main table -------------------------------- The main tidy table is the run-level table. A useful reading order is: 1. identify the configuration columns such as city, pde_mode, lambda weights, and identifiability settings; 2. read the main metrics such as MAE, RMSE, R2, coverage80, and sharpness80; 3. use the per-horizon columns only when you need to inspect how performance drifts across forecast steps. In other words: - the main table is the tidy archive, - the grouped tables are the paper-style summaries. .. GENERATED FROM PYTHON SOURCE LINES 408-427 Learn how to read the S6 table ------------------------------ The S6 output is the lambda-grid summary. For each city and each physics bucket, the script pivots one metric onto: - rows = lambda_cons - cols = lambda_prior This is extremely useful because it converts a long run table into a sensitivity surface that can later be: - plotted as a heatmap, - exported to TeX, - or inspected directly in CSV form. In the preview heatmap above, the white ring marks the best MAE cell in the Nansha + physics-on grid. .. GENERATED FROM PYTHON SOURCE LINES 429-455 Learn how to read the S7 table ------------------------------ The S7 output is the toggle summary. Instead of summarizing individual runs, it groups runs by higher-level settings such as: - city, - pde bucket, - use_effective_h, - kappa_mode, - smooth_on, - bounds_on, - mv_on, - q_on. Then it computes group means and group sizes. This is useful when the reader is not asking: "Which exact lambda pair won?" but instead: "Do runs with bounds or smoothness tend to behave better on average?" .. GENERATED FROM PYTHON SOURCE LINES 457-471 Why this builder is useful in practice -------------------------------------- This script is a strong bridge between raw experiment logs and later analysis pages. It helps with three different kinds of work: - quick filtering and ranking of runs, - paper-style grouped summaries, - and downstream figure generation such as S6 sensitivity maps. That is why it belongs naturally in the ``tables_and_summaries`` gallery rather than in ``figure_generation``. .. GENERATED FROM PYTHON SOURCE LINES 473-489 Practical takeaway ------------------ A good workflow for ablations is: 1. run the sensitivity or ablation jobs, 2. build the tidy ablation table, 3. inspect best-per-city rows, 4. inspect S6 lambda grids, 5. inspect S7 grouped toggle summaries, 6. only then move on to paper-ready plots. This order helps separate: - data collection, - tabulation, - and visualization. .. GENERATED FROM PYTHON SOURCE LINES 491-528 Command-line version -------------------- The same lesson can be reproduced from the CLI. Legacy dispatcher: .. code-block:: bash python -m scripts build-ablation-table \ --root results \ --out table_ablations \ --formats csv,json,txt \ --best-per-city \ --group-cols s6,s7 Paper-friendly TeX export: .. code-block:: bash python -m scripts build-ablation-table \ --root results \ --for-paper \ --err-metric rmse \ --keep-r2 \ --formats csv,tex \ --out table_ablations_paper Modern CLI: .. code-block:: bash geoprior build ablation-table \ --root results \ --out table_ablations \ --group-cols s6,s7 The gallery page teaches the builder. The command line reproduces it in a workflow. .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.376 seconds) .. _sphx_glr_download_auto_examples_tables_and_summaries_build_ablation_table.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: build_ablation_table.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: build_ablation_table.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: build_ablation_table.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_