Reading datasets into BDF#

BDF supports a few ways to read source data:

a URL for a file hosted on the web
a path for a local file
an identifier from the BDF data registry

In this notebook, we will show some examples for how to read data from these different sources. We will also demonstrate some helpful features including:

viewing the raw data in the format supplied from the vendor / original source
keeping only the required columns or including all columns

# Import the package
import bdf

# Read from a local file path
filepath = "../../data/SINTEF__LiGrR2032__2024-04-30__25degC__Landt.csv"

# read() returns (frame, metadata)
df, meta = bdf.read(filepath)
df.head()

/home/runner/work/battery-data-format/battery-data-format/src/bdf/table_parsers.py:195: UserWarning: tz defaulted to UTC; pass tz=... if data was recorded in a different timezone
  result = self.normalizer.normalize(

shape: (5, 16)

Test Time / s	Voltage / V	Current / A	Unix Time / s	Cycle Count / 1	Step ID	Step Type	Record Index / 1	Step Time / s	Step Charging Capacity / Ah	Step Discharging Capacity / Ah	Step Charging Energy / Wh	Step Discharging Energy / Wh	Temperature T1 / degC	Temperature T2 / degC	Temperature T3 / degC
f64	f64	f64	f64	i64	i64	str	i64	f64	f64	f64	f64	f64	f64	f64	f64
0.02	2.9215	0.0	1.7145e9	1	1	"rest"	1	0.02	0.0	0.0	0.0	0.0	0.0	0.0	0.0
15.02	2.9215	0.0	1.7145e9	1	1	"rest"	2	15.02	0.0	0.0	0.0	0.0	0.0	0.0	0.0
30.02	2.9214	0.0	1.7145e9	1	1	"rest"	3	30.02	0.0	0.0	0.0	0.0	0.0	0.0	0.0
45.02	2.9214	0.0	1.7145e9	1	1	"rest"	4	45.02	0.0	0.0	0.0	0.0	0.0	0.0	0.0
60.02	2.9213	0.0	1.7145e9	1	1	"rest"	5	60.02	0.0	0.0	0.0	0.0	0.0	0.0	0.0

# Peek at the raw vendor file without normalization to BDF
vendor_df, _ = bdf.read(filepath, normalize=False, validate=False)
vendor_df.head()

shape: (5, 17)

channel_index	cycle_index	step_index	date_time_iso_string	test_time_s	step_time_s	current_A	voltage_V	discharge_capacity_Ah	charge_capacity_Ah	discharge_energy_Wh	charge_energy_Wh	Pressure_Psi	temperature_1_C	temperature_2_C	temperature_3_C	step_name
str	str	str	str	str	str	str	str	str	str	str	str	str	str	str	str	str
"1"	"1"	"1"	"04/30/2024 14:33:19"	"0.020"	"0.020"	"0.0000"	"2.9215"	"0"	"0"	"0"	"0"	"0"	"0"	"0"	"0"	"rest"
"2"	"1"	"1"	"04/30/2024 14:33:34"	"15.020"	"15.020"	"0.0000"	"2.9215"	"0"	"0"	"0"	"0"	"0"	"0"	"0"	"0"	"rest"
"3"	"1"	"1"	"04/30/2024 14:33:49"	"30.020"	"30.020"	"0.0000"	"2.9214"	"0"	"0"	"0"	"0"	"0"	"0"	"0"	"0"	"rest"
"4"	"1"	"1"	"04/30/2024 14:34:04"	"45.020"	"45.020"	"0.0000"	"2.9214"	"0"	"0"	"0"	"0"	"0"	"0"	"0"	"0"	"rest"
"5"	"1"	"1"	"04/30/2024 14:34:19"	"60.020"	"60.020"	"0.0000"	"2.9213"	"0"	"0"	"0"	"0"	"0"	"0"	"0"	"0"	"rest"

# Keep all columns unknown to BDF
df_req, _ = bdf.read(filepath, include_unknown=True)
print(df_req.columns)

['Test Time / s', 'Voltage / V', 'Current / A', 'Unix Time / s', 'Cycle Count / 1', 'Step ID', 'Step Type', 'Record Index / 1', 'Step Time / s', 'Step Charging Capacity / Ah', 'Step Discharging Capacity / Ah', 'Step Charging Energy / Wh', 'Step Discharging Energy / Wh', 'Temperature T1 / degC', 'Temperature T2 / degC', 'Temperature T3 / degC', 'Pressure_Psi']

/home/runner/work/battery-data-format/battery-data-format/src/bdf/_df_compat.py:90: UserWarning: Non-BDF columns present: ['Pressure_Psi']
  result = fn(self, _to_polars_lazy(df), *args, **kwargs)

# Read data on the Web from a URL
df, meta = bdf.read("https://zenodo.org/records/17289383/files/SINTEF__NaCR32140-MP10-04__2025-08-25__GITT_0p05C_25degC__BioLogic.mpt")
df.head()

shape: (5, 12)

Test Time / s	Voltage / V	Current / A	Cycle Count / 1	Step ID	Step Time / s	Net Capacity / Ah	Charging Energy / Wh	Discharging Energy / Wh	Net Energy / Wh	Power / W	Internal Resistance / ohm
f64	f64	f64	i64	i64	f64	f64	f64	f64	f64	f64	f64
0.0	1.7142304	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
10.0	1.7141516	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
20.000001	1.7141516	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
30.000001	1.7142304	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
40.000002	1.7141911	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0

Read from the dataset registry#

The dataset registry is a JSON file (datasets.json) that maps ids to URLs and optional plugin hints. This is separate from the metadata registry built by bdf.build_registry. Resolve an id to its entry with bdf.load_registry / bdf.get_entry, then pass the entry’s URL to bdf.read.

# Read from the BDF data registry
# This registry is a local JSON file (datasets.json), separate from the metadata registry.
import json
from pathlib import Path

registry_path = Path("out/datasets.json")
registry_path.parent.mkdir(parents=True, exist_ok=True)
registry_path.write_text(json.dumps({
    "schema_version": "0.3",
    "entries": [
        {
            "id": "sintef-biologic-demo",
            "url": "https://zenodo.org/records/17289383/files/SINTEF__NaCR32140-MP10-04__2025-08-25__GITT_0p05C_25degC__BioLogic.mpt",
            "plugin": "biologic_mpt",
        }
    ],
}, indent=2), encoding="utf-8")

reg = bdf.load_registry(registry_path)
entry = bdf.get_entry(reg, "sintef-biologic-demo")
df, meta = bdf.read(entry["url"])
df.head()

shape: (5, 12)

Test Time / s	Voltage / V	Current / A	Cycle Count / 1	Step ID	Step Time / s	Net Capacity / Ah	Charging Energy / Wh	Discharging Energy / Wh	Net Energy / Wh	Power / W	Internal Resistance / ohm
f64	f64	f64	i64	i64	f64	f64	f64	f64	f64	f64	f64
0.0	1.7142304	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
10.0	1.7141516	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
20.000001	1.7141516	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
30.000001	1.7142304	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
40.000002	1.7141911	0.0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0