The `MetadataParser` Classes#

A MetadataParser pairs a source (where the metadata lives) with rules (what to pull out of it, and how to convert each matched value). matches() identifies the source, parse() extracts it, and each subclass owns all of its own file I/O.

parse() returns a Metadata: one record per BattINFO entity, plus BDF’s own bdf audit section, the verbatim raw source, and any extras a rule staged deliberately.

Three sources ship, one per kind of input file:

Parser	Source	Normalization
`TxtPreambleParser`	the head bytes of the data file itself	per rule
`JsonSidecarParser`	a vendor `.json` file beside the data file	per rule
`BdfSidecarParser`	BDF’s own `.metadata.json`, written by `save()`	none: the values are already canonical

The base MetadataParser never matches and extracts nothing, which is what a plugin declares when its files carry no metadata.

import json
import re
import tempfile
from pathlib import Path

import polars as pl

import bdf
from bdf.metadata_targets import METADATA
from bdf.metadata_parsers import (
    BdfSidecarParser,
    JsonRule,
    JsonSidecarParser,
    MetadataParser,
    RegexRule,
    TxtPreambleParser,
)
from bdf.normalization import AbsoluteTimeNormalization
from bdf.plugins import PLUGINS

work = Path(tempfile.mkdtemp())

# The base parser never matches and extracts nothing
probe = work / "probe.txt"
probe.write_text("nothing to see here\n")

base = MetadataParser()
print("matches():", base.matches(probe))
print("parse():  ", base.parse(probe).to_dict())

matches(): False
parse():   {}

`METADATA` — the target namespace#

A rule stages its value at a target, and targets are written as attribute paths on METADATA. The namespace reads the generated BattINFO models directly, so every path it accepts is a real field, and a misspelling raises AttributeError before any parser is built.

An open map (such as test.conditions) takes a subscript instead of a further attribute, and METADATA.extras[...] takes any key at all, for a vendor value no BattINFO field describes.

print(METADATA.battinfo_test.test.started_at)
print(METADATA.battinfo_test.test.conditions["temperature_c"])
print(METADATA.extras["firmware"])

try:
    METADATA.battinfo_test.test.startedat  # misspelled
except AttributeError as exc:
    print("AttributeError:", exc)

kind='metadata' path=('battinfo_test', 'test', 'started_at')
kind='metadata' path=('battinfo_test', 'test', 'conditions', 'temperature_c')
kind='extras' path=('firmware',)
AttributeError: 'battinfo_test.test.startedat' is not a declared BattINFO metadata path

`TxtPreambleParser` — metadata in the file’s own preamble#

Construct one with:

magic — tokens matches() looks for in the file’s head bytes
rules — a {target: RegexRule} mapping, where each pattern’s group(1) is the extracted value

A rule’s normalization converts the matched text. AbsoluteTimeNormalization turns a vendor timestamp into epoch seconds under the formats it declares; without one, the text is staged as it was read.

raw keeps the decoded preamble, so nothing a rule missed is lost. A bdf.read() cuts that source where the table begins, using the boundary its table parser locates; a caller who drives TxtPreambleParser directly, as this notebook does, states no boundary and keeps the whole decoded head. Where a file carries no preamble at all, raw is None.

preamble = work / "cell01.mpt"
preamble.write_text(
    "BT-Lab ASCII FILE\n"
    "Nb header lines : 4\n"
    "Acquisition started on : 05/13/2024 11:19:51.602\n"
    "Device : MPG-2\n"
    "time/s,Ewe/V,I/mA\n"
    "0,3.612,0\n",
    encoding="utf-8",
)

preamble_parser = TxtPreambleParser(
    magic=("BT-Lab ASCII FILE",),
    rules={
        METADATA.battinfo_test.test.started_at: RegexRule(
            pattern=re.compile(r"Acquisition started on\s*:\s*(.+)"),
            normalization=AbsoluteTimeNormalization(formats=("%m/%d/%Y %H:%M:%S%.f",)),
        ),
        METADATA.battinfo_equipment.equipment.name: RegexRule(
            pattern=re.compile(r"Device\s*:\s*(.+)"),
        ),
    },
)

print("matches():", preamble_parser.matches(preamble))

# The preamble states no zone, so tz says which one to read it in
meta = preamble_parser.parse(preamble, tz="Europe/Oslo")
meta.to_dict()

matches(): True

{'battinfo_test': {'test': {'started_at': 1715591991}},
 'battinfo_equipment': {'equipment': {'name': 'MPG-2'}},
 'raw': 'BT-Lab ASCII FILE\nNb header lines : 4\nAcquisition started on : 05/13/2024 11:19:51.602\nDevice : MPG-2\ntime/s,Ewe/V,I/mA\n0,3.612,0\n'}

Leave tz out and a naive timestamp is read as UTC, with a UserWarning saying so — the same rule the table path follows.

Every shipped plugin declares its parser this way. The BioLogic plugin’s is the one above, with the vendor’s own format list, so reading a real .mpt file goes through the identical machinery.

biologic = PLUGINS["biologic_mpt"].metadata_parser
print("magic: ", biologic.magic)
for target, rule in biologic.rules:
    print("target:", ".".join(target.path))
    print("regex: ", rule.pattern.pattern)
    print("formats:", rule.normalization.formats)

magic:  ('bt-lab ascii file', 'ec-lab ascii file')
target: battinfo_test.test.started_at
regex:  Acquisition started on\s*:\s*(.+)
formats: ('%m/%d/%Y %H:%M:%S%.f', '%m/%d/%Y %H:%M:%S')

`JsonSidecarParser` — metadata in a vendor JSON file#

JsonSidecarParser reads path.with_suffix(".json"). Each JsonRule states an ordered tuple of candidate paths through the document, and the first path present wins, so one rule covers a key a vendor renamed between versions.

raw holds the whole document, and a key no rule maps is preserved there. extras holds only what a rule staged under an explicit METADATA.extras[...] target.

data = work / "cell02.csv"
data.write_text("time/s,Ewe/V\n0,3.6\n")
(work / "cell02.json").write_text(
    json.dumps({"cell": {"name": "NaCR32140-04"}, "started": "2024-05-13T11:19:51", "fw": "2.1.4"})
)

sidecar_parser = JsonSidecarParser(
    rules={
        METADATA.battinfo_cell.cell_instance.name: JsonRule(candidates=(("cell", "name"),)),
        METADATA.battinfo_test.test.started_at: JsonRule(
            candidates=(("started",), ("start_time",)),  # first present wins
            normalization=AbsoluteTimeNormalization(formats=("%Y-%m-%dT%H:%M:%S",)),
        ),
        METADATA.extras["firmware"]: JsonRule(candidates=(("fw",),)),
    },
)

print("matches():", sidecar_parser.matches(data))
sidecar_meta = sidecar_parser.parse(data, tz="Europe/Oslo")
sidecar_meta.to_dict()

matches(): True

{'battinfo_test': {'test': {'started_at': 1715591991}},
 'battinfo_cell': {'cell_instance': {'name': 'NaCR32140-04'}},
 'raw': {'cell': {'name': 'NaCR32140-04'},
  'started': '2024-05-13T11:19:51',
  'fw': '2.1.4'},
 'extras': {'firmware': '2.1.4'}}

`BdfSidecarParser` — BDF’s own metadata file#

save(df, path, metadata=...) writes <stem>.metadata.json beside the artifact, holding the values that differ from their defaults. A Metadata carrying nothing writes no sidecar at all, and deletes one the target already had. A save() that states no metadata= beside an existing sidecar raises instead: the sidecar describes the data the previous save wrote, so it must not ride along with new data.

BdfSidecarParser restores that file verbatim: the values are already canonical, so no rule and no normalization run. A read takes its metadata from exactly one source, and this sidecar wins over the plugin’s own parser whenever it exists.

frame = pl.DataFrame(
    {
        "Test Time / s": [0.0, 1.0],
        "Voltage / V": [3.6, 3.61],
        "Current / A": [0.0, 0.5],
        "Unix Time / s": [1.7e9, 1.7e9 + 1],
    }
)

artifact = work / "cell02.bdf.csv"
bdf.save(frame, artifact, metadata=sidecar_meta, validate=False)
print("beside the artifact:", sorted(p.name for p in work.glob("cell02.bdf.*")))

# Restored verbatim, raw and extras included
BdfSidecarParser().parse(artifact).to_dict()

beside the artifact: ['cell02.bdf.csv', 'cell02.bdf.metadata.json']

{'battinfo_test': {'test': {'started_at': 1715591991}},
 'battinfo_cell': {'cell_instance': {'name': 'NaCR32140-04'}},
 'bdf': {'bdf_version': '0.2.0rc2',
  'ontology_version': '1.3.0',
  'battinfo_ref': 'eda462993ae43a64706e326253dddb44be100923'},
 'raw': {'cell': {'name': 'NaCR32140-04'},
  'started': '2024-05-13T11:19:51',
  'fw': '2.1.4'},
 'extras': {'firmware': '2.1.4'}}

# read() takes the sidecar over the plugin parser, and keeps every value it restores.
# read() then states bdf.source, and replaces bdf.time_reconciliation only when this read repairs.
_, restored = bdf.read(artifact, validate=False)
print("restored from the sidecar:", restored.battinfo_cell.cell_instance.name)
print("stated by this read:      ", restored.bdf.source)

# A save that states no metadata beside an existing sidecar raises, rather than let the
# previous save's metadata ride along with new data.
try:
    bdf.save(frame, artifact, validate=False)
except FileExistsError as exc:
    print(exc)

# A save that states an empty Metadata deletes the sidecar, so the artifact keeps no metadata.
bdf.save(frame, artifact, metadata=bdf.Metadata(), validate=False)
print("sidecar exists:", (work / "cell02.bdf.metadata.json").exists())

restored from the sidecar: NaCR32140-04
stated by this read:       bdf_csv
/tmp/tmp8k37u30p/cell02.bdf.metadata.json describes the data a previous save wrote, and this save states no metadata. Pass metadata= to keep or update it, or metadata=Metadata() to clear it. To keep both the sidecar and the data it describes, save to a different path. To discard it, delete the sidecar.
sidecar exists: False

The MetadataParser Classes#

METADATA — the target namespace#

TxtPreambleParser — metadata in the file’s own preamble#

JsonSidecarParser — metadata in a vendor JSON file#

BdfSidecarParser — BDF’s own metadata file#

The `MetadataParser` Classes#

`METADATA` — the target namespace#

`TxtPreambleParser` — metadata in the file’s own preamble#

`JsonSidecarParser` — metadata in a vendor JSON file#

`BdfSidecarParser` — BDF’s own metadata file#