.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/inspection/plot_physics_payload_meta_overview.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_inspection_plot_physics_payload_meta_overview.py: Inspect physics-payload metadata before opening the full payload ================================================================== This lesson explains how to inspect a physics-payload metadata sidecar, usually a file such as ``physics_payload.npz.meta.json``. This artifact is small, but it carries some of the most important semantic information attached to an exported physics payload: - which model and split produced the payload, - which PDE modes were active, - which groundwater convention was used, - which closure / tau-prior definition was assumed, - which units should be used when interpreting the arrays, - and which compact epsilon-style payload metrics were recorded. The goal of this page is therefore not only to call helper functions. It is to teach how to inspect the metadata first, so a user can decide whether the heavier NPZ payload is worth opening and whether its conventions match the intended analysis. .. GENERATED FROM PYTHON SOURCE LINES 28-113 .. code-block:: Python from __future__ import annotations import json import tempfile from pathlib import Path from pprint import pprint import matplotlib.pyplot as plt import pandas as pd from geoprior.utils.inspect import ( generate_physics_payload_meta, inspect_physics_payload_meta, load_physics_payload_meta, physics_payload_meta_closure_frame, physics_payload_meta_identity_frame, physics_payload_meta_metrics_frame, physics_payload_meta_units_frame, plot_physics_payload_meta_boolean_summary, plot_physics_payload_meta_core_scalars, plot_physics_payload_meta_payload_metrics, summarize_physics_payload_meta, ) pd.set_option("display.max_columns", 30) pd.set_option("display.width", 112) pd.set_option("display.max_colwidth", 88) PAYLOAD_META_PALETTE = { "core": "#0F766E", "metrics": "#7C3AED", "pass": "#15803D", "fail": "#B91C1C", "edge": "#334155", "panel": "#FCFCFE", } def _style_bar_panel( ax: plt.Axes, *, color: str, edge: str = PAYLOAD_META_PALETTE["edge"], ) -> None: """Polish a bar-based gallery panel.""" for patch in ax.patches: patch.set_facecolor(color) patch.set_edgecolor(edge) patch.set_linewidth(1.25) patch.set_alpha(0.94) ax.set_facecolor(PAYLOAD_META_PALETTE["panel"]) ax.tick_params(labelsize=9) ax.title.set_fontweight("bold") for side in ("top", "right"): ax.spines[side].set_visible(False) for side in ("left", "bottom"): ax.spines[side].set_color("#CBD5E1") def _style_boolean_panel(ax: plt.Axes) -> None: """Apply a distinct pass/fail palette to boolean panels.""" for patch in ax.patches: width = patch.get_width() height = patch.get_height() score = width if width != 0 else height color = ( PAYLOAD_META_PALETTE["pass"] if score >= 0.5 else PAYLOAD_META_PALETTE["fail"] ) patch.set_facecolor(color) patch.set_edgecolor(PAYLOAD_META_PALETTE["edge"]) patch.set_linewidth(1.15) patch.set_alpha(0.94) ax.set_facecolor(PAYLOAD_META_PALETTE["panel"]) ax.tick_params(labelsize=9) ax.title.set_fontweight("bold") for side in ("top", "right"): ax.spines[side].set_visible(False) for side in ("left", "bottom"): ax.spines[side].set_color("#CBD5E1") .. GENERATED FROM PYTHON SOURCE LINES 114-136 Why this artifact matters ------------------------- The full physics payload is often an NPZ archive that stores heavy arrays. Opening it directly can be unnecessary if the exported run already uses conventions or closures that do not match the question we want to study. The metadata sidecar is therefore the ideal first checkpoint. It lets us answer practical questions such as: 1. Was this payload produced by the model and split I expected? 2. Were both ``consolidation`` and ``gw_flow`` active, or only one? 3. Are groundwater values interpreted as depth below ground or something else? 4. Which tau-prior / closure formula was used to define the saved residual context? 5. Are the saved units compatible with the way I plan to report results or compare payloads across runs? In other words, this artifact is a semantic filter. It helps the user validate meaning before reading arrays. .. GENERATED FROM PYTHON SOURCE LINES 139-148 Create a realistic demo metadata sidecar ---------------------------------------- For documentation pages, we want a stable artifact that behaves like a real metadata sidecar without rebuilding a full physics payload. The generation helper was designed exactly for that use case. Here we create a metadata file with realistic identity fields, unit conventions, closure definitions, and compact payload metrics. .. GENERATED FROM PYTHON SOURCE LINES 148-185 .. code-block:: Python out_dir = Path(tempfile.mkdtemp(prefix="gp_payload_meta_")) meta_path = out_dir / "physics_payload_demo.npz.meta.json" generate_physics_payload_meta( meta_path, city="nansha", model_name="GeoPriorSubsNet", split="TestSet", created_utc="2026-03-30T10:18:55Z", pde_modes_active=["consolidation", "gw_flow"], time_units="year", time_coord_units="year", gwl_kind="depth_bgs", gwl_sign="down_positive", use_head_proxy=True, kappa_mode="kb", use_effective_h=True, hd_factor=0.6, lambda_offset=1.0, kappa_value=0.04639, tau_prior_definition="tau_closure_from_learned_fields", tau_prior_human_name="tau_closure", tau_prior_source="model.evaluate_physics() closure head", tau_closure_formula="Hd^2 * Ss / (pi^2 * kappa_b * K)", payload_metrics={ "eps_prior_rms": 3.57e-4, "eps_cons_scaled_rms": 2.61e-6, "closure_consistency_rms": None, "r2_logtau": 1.0, "kappa_used_for_closure": None, }, ) print("Written physics-payload metadata file") print(f" - {meta_path}") .. rst-class:: sphx-glr-script-out .. code-block:: none Written physics-payload metadata file - /tmp/gp_payload_meta_n8_xcj09/physics_payload_demo.npz.meta.json .. GENERATED FROM PYTHON SOURCE LINES 186-192 Load the artifact with the real reader -------------------------------------- Even in a lesson, it is worth using the same loading entry point a real workflow would use. That keeps the page close to a user's actual inspection path. .. GENERATED FROM PYTHON SOURCE LINES 192-206 .. code-block:: Python meta_record = load_physics_payload_meta(meta_path) print("\nArtifact header") pprint( { "kind": meta_record.kind, "stage": meta_record.stage, "city": meta_record.city, "model": meta_record.model, "path": str(meta_record.path), } ) .. rst-class:: sphx-glr-script-out .. code-block:: none Artifact header {'city': 'nansha', 'kind': 'physics_payload_meta', 'model': 'GeoPriorSubsNet', 'path': '/tmp/gp_payload_meta_n8_xcj09/physics_payload_demo.npz.meta.json', 'stage': None} .. GENERATED FROM PYTHON SOURCE LINES 207-223 Start with the compact semantic summary --------------------------------------- Before opening every table, start with the compact summary. This summary is useful because it reorganizes the raw metadata into a few inspection layers: - a short identity block, - conventions for groundwater and time, - closure / tau-prior semantics, - payload-level metrics, - and boolean checks for structural completeness. When you inspect many payload exports, this summary is often the fastest way to reject a mismatched sidecar before reading more. .. GENERATED FROM PYTHON SOURCE LINES 223-229 .. code-block:: Python summary = summarize_physics_payload_meta(meta_record) print("\nCompact summary") print(json.dumps(summary, indent=2)) .. rst-class:: sphx-glr-script-out .. code-block:: none Compact summary { "brief": { "kind": "physics_payload_meta", "created_utc": "2026-03-30T10:18:55Z", "saved_utc": "2026-03-30T10:18:55Z", "model_name": "GeoPriorSubsNet", "city": "nansha", "split": "TestSet", "pde_modes_active": [ "consolidation", "gw_flow" ], "n_pde_modes": 2 }, "conventions": { "time_units": "year", "time_coord_units": "year", "gwl_kind": "depth_bgs", "gwl_sign": "down_positive", "use_head_proxy": true, "kappa_mode": "kb" }, "closure": { "tau_prior_definition": "tau_closure_from_learned_fields", "tau_prior_human_name": "tau_closure", "tau_prior_source": "model.evaluate_physics() closure head", "tau_closure_formula": "Hd^2 * Ss / (pi^2 * kappa_b * K)", "use_effective_h": true, "use_effective_thickness": true, "hd_factor": 0.6, "Hd_factor": 0.6, "kappa_value": 0.04639 }, "payload_metrics": { "eps_prior_rms": 0.000357, "eps_cons_scaled_rms": 2.61e-06, "r2_logtau": 1.0 }, "units": { "n_units": 8, "quantities": [ "H", "Hd", "K", "Ss", "cons_res_vals", "kappa", "tau", "tau_prior" ], "tau_unit": "s", "tau_prior_unit": "s", "K_unit": "m/s", "Ss_unit": "1/m" }, "checks": { "has_model_name": true, "has_city": true, "has_split": true, "has_pde_modes": true, "has_units": true, "has_payload_metrics": true, "has_tau_prior_definition": true, "has_tau_closure_formula": true, "time_units_present": true, "time_coord_units_present": true, "kappa_value_present": true, "lambda_offset_present": true, "hd_factor_present": true, "payload_eps_prior_present": true, "payload_eps_cons_present": true, "uses_head_proxy": true }, "core_scalars": { "hd_factor": 0.6, "lambda_offset": 1.0, "Hd_factor": 0.6, "lambda_offsets": 1.0, "kappa_value": 0.04639 } } .. GENERATED FROM PYTHON SOURCE LINES 230-246 Read the identity frame first ----------------------------- The identity frame answers the first practical question: *What exactly does this metadata sidecar describe?* This is where we verify model name, city, split, PDE modes, time units, and groundwater conventions. In practice, this frame is often the quickest way to catch mistakes such as: - opening a payload from the wrong split, - mixing runs built with different PDE modes, - or confusing groundwater depth conventions. .. GENERATED FROM PYTHON SOURCE LINES 246-252 .. code-block:: Python identity = physics_payload_meta_identity_frame(meta_record) print("\nIdentity and convention view") print(identity) .. rst-class:: sphx-glr-script-out .. code-block:: none Identity and convention view field value 0 created_utc 2026-03-30T10:18:55Z 1 saved_utc 2026-03-30T10:18:55Z 2 model_name GeoPriorSubsNet 3 city nansha 4 split TestSet 5 time_units year 6 time_coord_units year 7 gwl_kind depth_bgs 8 gwl_sign down_positive 9 use_head_proxy True 10 kappa_mode kb 11 pde_modes_active consolidation, gw_flow .. GENERATED FROM PYTHON SOURCE LINES 253-267 Read the closure frame carefully -------------------------------- The closure frame is one of the most important parts of this artifact family. It tells us how the payload relates its physical fields to the tau prior and whether effective thickness was used. This matters because two payloads can look similar numerically while encoding different closure assumptions. When this frame changes across experiments, the comparison is not purely a metric comparison anymore. It becomes a modeling-choice comparison. .. GENERATED FROM PYTHON SOURCE LINES 267-273 .. code-block:: Python closure = physics_payload_meta_closure_frame(meta_record) print("\nClosure / tau-prior view") print(closure) .. rst-class:: sphx-glr-script-out .. code-block:: none Closure / tau-prior view field value 0 tau_prior_definition tau_closure_from_learned_fields 1 tau_prior_human_name tau_closure 2 tau_prior_source model.evaluate_physics() closure head 3 tau_closure_formula Hd^2 * Ss / (pi^2 * kappa_b * K) 4 use_effective_h True 5 use_effective_thickness True 6 hd_factor 0.600000 7 Hd_factor 0.600000 8 kappa_value 0.046390 .. GENERATED FROM PYTHON SOURCE LINES 274-287 Inspect the units before opening arrays --------------------------------------- The units block is the safeguard that prevents misreading the NPZ payload. A user should not interpret fields such as ``K``, ``Ss``, ``tau``, or residual values before checking the declared units. A healthy reading pattern is: 1. verify that the expected physical quantities are present, 2. check that ``tau`` and ``tau_prior`` share compatible units, 3. confirm that the residual values are expressed in the units you expect for later reporting. .. GENERATED FROM PYTHON SOURCE LINES 287-293 .. code-block:: Python units = physics_payload_meta_units_frame(meta_record) print("\nUnits block") print(units) .. rst-class:: sphx-glr-script-out .. code-block:: none Units block quantity unit 0 H m 1 Hd m 2 K m/s 3 Ss 1/m 4 cons_res_vals m/s 5 kappa dimensionless 6 tau s 7 tau_prior s .. GENERATED FROM PYTHON SOURCE LINES 294-314 Read the compact payload metrics -------------------------------- The payload metrics are not a replacement for the full evaluation artifact. They are a compact checkpoint attached directly to the exported payload. In this lesson, the most informative metrics are usually: - ``eps_prior_rms`` - ``eps_cons_scaled_rms`` - ``closure_consistency_rms`` - ``r2_logtau`` - ``kappa_used_for_closure`` Together they help answer a small but important question: *Does this payload still carry a physically interpretable closure context, or does it already look inconsistent before I read the heavier arrays?* .. GENERATED FROM PYTHON SOURCE LINES 314-320 .. code-block:: Python metrics = physics_payload_meta_metrics_frame(meta_record) print("\nPayload metrics") print(metrics) .. rst-class:: sphx-glr-script-out .. code-block:: none Payload metrics section metric value 0 payload_metrics eps_cons_scaled_rms 0.000003 1 payload_metrics eps_prior_rms 0.000357 2 payload_metrics r2_logtau 1.000000 .. GENERATED FROM PYTHON SOURCE LINES 321-336 Use the all-in-one inspector when you want the key tables together ------------------------------------------------------------------ ``inspect_physics_payload_meta(...)`` is useful when you want a compact bundle with: - the semantic summary, - the identity frame, - the closure frame, - the units frame, - and the metrics frame. This is especially useful for batch inspection tools that need to scan many sidecars quickly before deciding which payload archives deserve a deeper review. .. GENERATED FROM PYTHON SOURCE LINES 336-342 .. code-block:: Python bundle = inspect_physics_payload_meta(meta_record) print("\nInspector bundle keys") print(sorted(bundle)) .. rst-class:: sphx-glr-script-out .. code-block:: none Inspector bundle keys ['closure_frame', 'identity_frame', 'metrics_frame', 'summary', 'units_frame'] .. GENERATED FROM PYTHON SOURCE LINES 343-358 Plot the metadata scalars and payload metrics --------------------------------------------- The first figure answers two different questions. The *core scalars* plot shows top-level numeric metadata such as ``hd_factor``, ``Hd_factor``, ``lambda_offset``, and ``kappa_value``. This helps the user check whether the saved payload corresponds to the physical closure settings they intended. The *payload metrics* plot then summarizes the small diagnostic numbers attached directly to the payload. This is a quick way to see whether epsilon-like values are tiny, moderate, or obviously suspicious. .. GENERATED FROM PYTHON SOURCE LINES 358-379 .. code-block:: Python fig, axes = plt.subplots( 1, 2, figsize=(12.4, 4.8), constrained_layout=True, ) plot_physics_payload_meta_core_scalars( meta_record, ax=axes[0], title="Physics-payload meta: core scalars", ) _style_bar_panel(axes[0], color=PAYLOAD_META_PALETTE["core"]) plot_physics_payload_meta_payload_metrics( meta_record, ax=axes[1], title="Physics-payload meta: payload metrics", ) _style_bar_panel(axes[1], color=PAYLOAD_META_PALETTE["metrics"], edge="#4C1D95") .. image-sg:: /auto_examples/inspection/images/sphx_glr_plot_physics_payload_meta_overview_001.png :alt: Physics-payload meta: core scalars, Physics-payload meta: payload metrics :srcset: /auto_examples/inspection/images/sphx_glr_plot_physics_payload_meta_overview_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 380-399 How should you read these two plots? ------------------------------------ A good reading strategy is: - use the left plot to confirm that the saved metadata still reflects the closure regime you intended, - use the right plot to judge whether the payload-level epsilon values remain small enough to support a later deeper read. Not every project will use the same thresholds, but some common warning signs are: - missing or null critical scalars, - unexpectedly different ``hd_factor`` or ``kappa_value`` between runs you thought were comparable, - very large epsilon-style metrics compared with your usual runs, - or missing payload metrics when you expected the exporter to attach them. .. GENERATED FROM PYTHON SOURCE LINES 401-416 Plot the structural decision checks separately ---------------------------------------------- The boolean summary is useful because it turns the metadata into a compact pass/fail-style review: - identity present, - PDE modes present, - units present, - closure definition present, - payload metrics present, - key physical scalars present. This is the fastest plot for deciding whether a sidecar is ready for serious downstream inspection. .. GENERATED FROM PYTHON SOURCE LINES 416-428 .. code-block:: Python fig, ax = plt.subplots( figsize=(8.6, 4.8), constrained_layout=True, ) plot_physics_payload_meta_boolean_summary( meta_record, ax=ax, title="Physics-payload meta: decision checks", ) _style_boolean_panel(ax) .. image-sg:: /auto_examples/inspection/images/sphx_glr_plot_physics_payload_meta_overview_002.png :alt: Physics-payload meta: decision checks :srcset: /auto_examples/inspection/images/sphx_glr_plot_physics_payload_meta_overview_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 429-444 A practical reading rule ------------------------ For this artifact family, a useful first-pass decision rule can be: - model, city, and split are present, - PDE modes are recorded, - units are present, - tau-prior definition and closure formula are present, - payload metrics are present, - and the key scalar fields such as ``kappa_value`` and ``lambda_offset`` are not missing. If these checks pass, the metadata is usually healthy enough for the user to open the heavier NPZ payload with confidence. .. GENERATED FROM PYTHON SOURCE LINES 444-474 .. code-block:: Python checks = summary["checks"] must_pass = [ "has_model_name", "has_city", "has_split", "has_pde_modes", "has_units", "has_payload_metrics", "has_tau_prior_definition", "has_tau_closure_formula", "time_units_present", "time_coord_units_present", "kappa_value_present", "lambda_offset_present", "hd_factor_present", ] ready = all(bool(checks.get(name, False)) for name in must_pass) print("\nDecision note") if ready: print( "This metadata sidecar looks structurally ready for a " "deeper read of the full physics payload archive." ) else: print( "This metadata sidecar needs attention before you rely " "on the heavier payload it describes." ) .. rst-class:: sphx-glr-script-out .. code-block:: none Decision note This metadata sidecar looks structurally ready for a deeper read of the full physics payload archive. .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.348 seconds) .. _sphx_glr_download_auto_examples_inspection_plot_physics_payload_meta_overview.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_physics_payload_meta_overview.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_physics_payload_meta_overview.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_physics_payload_meta_overview.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_