Deborah.Rebekah.JLD2Loader

_recurse_scan!(
    bad::Vector{String}, 
    g::JLD2.Group, 
    basepath::AbstractString
) -> Nothing

Recursive depth-first traversal used internally by _scan_flat_vectors. Walks a JLD2.Group hierarchy and identifies suspicious values where a nested dictionary layout appears collapsed (i.e., a value is directly a Vector{Float64} instead of Dict{Symbol,Vector{Float64}}).

Arguments

• bad::Vector{String} : Mutable accumulator collecting detected offending dataset paths.
• g::JLD2.Group : Current group node being scanned.
• basepath::AbstractString : Current hierarchical prefix used for reporting full dataset paths.

Behavior

• For each dataset:
  1. Attempts safe deserialization (try/catch guarded).
  2. Detects common serialized forms:
     • Vector{Pair{Symbol,Any}} (SerializedDict-style)
     • Dict containers with shallow Vector{Float64} leaves.
  3. Appends human-readable paths such as "rw_data/solver1/w" when detected.

Returns

• Nothing (side-effect only; modifies bad).

Notes

• A failure to deserialize a leaf does not abort scanning.
• This function intentionally avoids mutation or rewriting of JLD2 nodes; it only reports.

_scan_flat_vectors(
    filepath::AbstractString
) -> Vector{String}

Diagnostic helper to inspect the internal schema of a .jld2 file and locate possible legacy or malformed entries introduced by older Miriam save formats.

Specifically scans leaf datasets for places where a nested structure expected to be of the form Dict{Symbol, Vector{Float64}} has instead been serialized one level too shallow as Vector{Float64}

This pattern frequently occurs when a SerializedDict round-trip produces a Vector{Pair{Symbol,Any}}, or when grouped solver metadata is written without explicit schema normalization.

Arguments

• filepath::AbstractString : Path to a .jld2 file opened read-only.

Returns

• Vector{String} : List of dataset paths (group-style, e.g., "summary/solver1/y") showing the locations where the suspicious flattening pattern was detected. If no issues are found, returns an empty vector.

Notes

• Does not modify the file; read-only inspection.
• Used to aid migration/repair of legacy bundles and provide context when JLD2.load throws schema mismatch errors.

load_jld2(
    filepath::String
) -> NamedTuple

Load JLD2 data file containing summary statistics, labels, and training identifiers.

This function reads a .jld2 file and returns a named tuple with keys :summary, :labels, and :trains. It is typically used in workflows that do not require reweighting data.

Arguments

• filepath::String: Path to the .jld2 file to load.

Returns

• A NamedTuple with fields summary, labels, and trains.

load_jld2_Miriam(
    filepath::String
) -> NamedTuple

Load a consolidated Miriam JLD2 bundle, including:

• results on multi-ensemble reweighting and transition point interpolation: summary (keyword-indexed),
• three measurement-style summaries from single-ensemble measurement points: summary_trace_meas, summary_moment_meas, summary_cumulant_meas,
• the kappa token list kappa_list (e.g., ["13570", "13575", "13580"]),
• the reweighting payload rw_data,
• the NLsolve.jl convergence payload nlsolve_status (optional; defaults to empty dict),
• and index metadata labels, trains.

This loader is tailored for overlap/error analysis and measurement-based plotting workflows.

Arguments

• filepath::String: Path to the .jld2 file previously written by save_miriam_results.

Returns

A NamedTuple with fields:

• summary :: Dict
• summary_trace_meas :: Dict
• summary_moment_meas :: Dict
• summary_cumulant_meas :: Dict
• kappa_list :: Vector{String}
• rw_data :: Dict
• nlsolve_status :: Dict (present if saved; otherwise empty)
• labels :: Vector{String}
• trains :: Vector{String}

Notes

• Measurement dictionaries are keyed as (field, :avg/:err, tag, kappa_str) where kappa_str $\in$ kappa_list. Many plotting utilities render κ as 0.<kappa_str> (e.g., "13580" → 0.13580).
• All summary matrices are indexed (label_index, train_index) and align with labels/trains.
• nlsolve_status enables convergence heatmaps; schema: Dict{String, Dict{String, Dict{String, NamedTuple{(:converged, :residual_norm), Tuple{Bool, Float64}}}}} with access nlsolve_status[label][train][solver].
• For backward compatibility, if nlsolve_status is not found in the file, an empty Dict() is returned instead of throwing.

Example

bundle = load_jld2_Miriam("out/miriam_results.jld2")
ks_tokens   = bundle.kappa_list
trace_meas  = bundle.summary_trace_meas
moment_meas = bundle.summary_moment_meas
cum_meas    = bundle.summary_cumulant_meas
rw          = bundle.rw_data
nls_status  = bundle.nlsolve_status  # {} if absent in older files