The Plugin System#

A Plugin pairs a TableParser (reads tabular data) with a MetadataParser (reads file metadata). The PLUGINS dict maps plugin IDs to Plugin instances for all built-in formats.

Auto-detection runs in three stages:

detect_from_ext_or_magic_bytes – filters by file extension, falling back to magic bytes
detect_from_metadata – filters by magic token matching
detect_from_columns – picks the highest-scoring normalizer match

The top-level detect() orchestrates all three stages and returns early when unambiguous.

import re
from pathlib import Path

from bdf.plugins import (
    Plugin,
    PLUGINS,
    detect,
    detect_from_ext_or_magic_bytes,
    detect_from_metadata,
    detect_from_columns,
)
from bdf.table_parsers import DelimTxtParser
from bdf.table_normalizers import TableNormalizer, Syn
from bdf.metadata_parsers import TxtPreambleParser, MetadataSchema
from bdf.file_utils import resolve_source

BIOLOGIC_URL = (
    "https://zenodo.org/api/records/18986774/files/"
    "SINTEF__NaCR32140-MP10-04__2025-08-25__GITT_0p05C_25degC__BioLogic.mpt/content"
)

# Each key is a plugin ID; the value is the Plugin instance
list(PLUGINS.keys())

['arbin_csv',
 'arbin_res',
 'basytec_txt',
 'biologic_mpt',
 'biologic_mpr',
 'digatron_csv',
 'landt_csv',
 'landt_txt',
 'maccor_csv',
 'neware_csv',
 'neware_xlsx',
 'arbin_xlsx',
 'novonix_csv',
 'neware_nda',
 'bdf_csv',
 'bdf_parquet',
 'bdf_json',
 'bdf_ndjson',
 'bdf_ipc',
 'bdf_xlsx']

Constructing a custom `Plugin`#

A Plugin takes a table_parser and a metadata_parser. The structure mirrors the built-in BIOLOGIC_MPT plugin: a DelimTxtParser for the tabular data and a TxtPreambleParser for the header metadata.

custom_plugin = Plugin(
    table_parser=DelimTxtParser(
        normalizer=TableNormalizer(
            test_time_second=(Syn(hdr="time/{unit}"),),
            voltage_volt=(Syn(hdr="Ecell/{unit}"),),
            current_ampere=(Syn(hdr="I/{unit}"),),
        )
    ),
    metadata_parser=TxtPreambleParser(
        magic=("BT-Lab ASCII FILE",),
        regex_patterns=MetadataSchema(
            start_time=re.compile(r"Acquisition started on\s*:\s*(.+)"),
        ),
    ),
)
custom_plugin

Plugin(table_parser=DelimTxtParser(normalizer=TableNormalizer(test_time_second=(Syn(hdr='time/{unit}', assumed=False, source_unit=None, legacy=False, reverse_sign=False),), voltage_volt=(Syn(hdr='Ecell/{unit}', assumed=False, source_unit=None, legacy=False, reverse_sign=False),), current_ampere=(Syn(hdr='I/{unit}', assumed=False, source_unit=None, legacy=False, reverse_sign=False),), unix_time_second=None, cycle_count=None, step_count=None, step_id=None, step_type=None, ambient_temperature_celsius=None, step_record_index=None, record_index=None, step_time_second=None, charging_capacity_ah=None, step_charging_capacity_ah=None, cycle_charging_capacity_ah=None, schedule_charging_capacity_ah=None, discharging_capacity_ah=None, step_discharging_capacity_ah=None, cycle_discharging_capacity_ah=None, schedule_discharging_capacity_ah=None, net_capacity_ah=None, step_net_capacity_ah=None, cycle_net_capacity_ah=None, cumulative_capacity_ah=None, step_cumulative_capacity_ah=None, cycle_cumulative_capacity_ah=None, charging_energy_wh=None, step_charging_energy_wh=None, cycle_charging_energy_wh=None, schedule_charging_energy_wh=None, discharging_energy_wh=None, step_discharging_energy_wh=None, cycle_discharging_energy_wh=None, schedule_discharging_energy_wh=None, net_energy_wh=None, step_net_energy_wh=None, cycle_net_energy_wh=None, cumulative_energy_wh=None, step_cumulative_energy_wh=None, cycle_cumulative_energy_wh=None, power_watt=None, internal_resistance_ohm=None, dc_internal_resistance_ohm=None, ac_internal_resistance_ohm=None, real_impedance_ohm=None, imaginary_impedance_ohm=None, absolute_impedance_ohm=None, phase_degree=None, frequency_hertz=None, ambient_pressure_pa=None, applied_pressure_pa=None, surface_pressure_pa=None, temperature_t1_celsius=None, temperature_t2_celsius=None, temperature_t3_celsius=None, temperature_t4_celsius=None, temperature_t5_celsius=None, surface_temperature_celsius=None), unique_exts=frozenset(), kind='txt', separator=None, skip_rows=None, has_header=True, decimal_comma=None, truncate_ragged_lines=False, encoding='utf-8'), metadata_parser=TxtPreambleParser(kind='txt_preamble', magic=('BT-Lab ASCII FILE',), encoding='utf-8', regex_patterns=MetadataSchema[Pattern[str]](start_time=re.compile('Acquisition started on\\s*:\\s*(.+)'))))

Stage 1 — `detect_from_ext_or_magic_bytes()`#

Extension filtering is cheap but imprecise: .txt is used by multiple formats. detect_from_ext_or_magic_bytes() returns all plugins that accept the file’s extension, falling back to magic-byte sniffing when the extension is missing or unmatched.

biologic_file = resolve_source(BIOLOGIC_URL)

ext_candidates = detect_from_ext_or_magic_bytes(biologic_file)
print("Candidates from extension (.txt):", list(ext_candidates.keys()))

Candidates from extension (.txt): ['biologic_mpt']

Stage 2 — `detect_from_metadata()`#

Magic token matching reads only the file’s head bytes and is much more precise. detect_from_metadata() filters the candidates to those whose metadata_parser matches.

meta_candidates = detect_from_metadata(biologic_file, ext_candidates)
print("Candidates after metadata filter:", list(meta_candidates.keys()))

Candidates after metadata filter: ['biologic_mpt']

Stage 3 — `detect_from_columns()`#

detect_from_columns() sniffs the column headings and returns the plugin with the highest normalizer score. It returns a (plugin_id, Plugin) tuple.

plugin_id, plugin = detect_from_columns(biologic_file, meta_candidates)
print("Resolved plugin:", plugin_id)

Resolved plugin: biologic_mpt

Top-level `detect()` shortcut#

detect() orchestrates all three stages and returns early when the candidates are already unambiguous.

plugin_id, plugin = detect(biologic_file)
print("Detected:", plugin_id)

Detected: biologic_mpt

# Read the normalised data using the detected plugin
plugin.table_parser.read(biologic_file).collect()

shape: (157_985, 12)

Test Time / s	Voltage / V	Current / A	Cycle Count / 1	Step ID	Step Time / s	Net Capacity / Ah	Charging Energy / Wh	Discharging Energy / Wh	Net Energy / Wh	Power / W	Internal Resistance / ohm
f64	f64	f64	i64	i64	f64	f64	f64	f64	f64	f64	f64
0.0	1.7142304	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
10.0	1.7141516	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
20.000001	1.7141516	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
30.000001	1.7142304	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
40.000002	1.7141911	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
…	…	…	…	…	…	…	…	…	…	…	…
1.5676e6	1.6886346	0.0	0	6	0.0	0.01877	34.024512	33.11767	0.906842	0.0	0.0
1.5676e6	1.688674	0.0	0	6	0.0	0.01877	34.024512	33.11767	0.906842	0.0	0.0
1.5677e6	1.6886346	0.0	0	6	0.0	0.01877	34.024512	33.11767	0.906842	0.0	0.0
1.5677e6	1.6885953	0.0	0	6	0.0	0.01877	34.024512	33.11767	0.906842	0.0	0.0
1.5677e6	1.6887133	0.0	0	7	0.0	0.01877	34.024512	33.11767	0.906842	0.0	0.0

The Plugin System#

Constructing a custom Plugin#

Stage 1 — detect_from_ext_or_magic_bytes()#

Stage 2 — detect_from_metadata()#

Stage 3 — detect_from_columns()#

Top-level detect() shortcut#

Constructing a custom `Plugin`#

Stage 1 — `detect_from_ext_or_magic_bytes()`#

Stage 2 — `detect_from_metadata()`#

Stage 3 — `detect_from_columns()`#

Top-level `detect()` shortcut#