{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "0",
      "metadata": {},
      "source": [
        "# The Plugin System\n",
        "\n",
        "A `Plugin` pairs a `TableParser` (reads tabular data) with a `MetadataParser` (reads file metadata).\n",
        "The `PLUGINS` dict maps plugin IDs to `Plugin` instances for all built-in formats.\n",
        "\n",
        "Auto-detection runs in three stages:\n",
        "1. **`detect_from_ext_or_magic_bytes`** \u2013 filters by file extension, falling back to magic bytes\n",
        "2. **`detect_from_metadata`** \u2013 filters by magic token matching\n",
        "3. **`detect_from_columns`** \u2013 picks the highest-scoring normalizer match\n",
        "\n",
        "The top-level `detect()` orchestrates all three stages and returns early when unambiguous."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1",
      "metadata": {},
      "outputs": [],
      "source": [
        "import re\n",
        "from pathlib import Path\n",
        "\n",
        "from bdf.plugins import (\n",
        "    Plugin,\n",
        "    PLUGINS,\n",
        "    detect,\n",
        "    detect_from_ext_or_magic_bytes,\n",
        "    detect_from_metadata,\n",
        "    detect_from_columns,\n",
        ")\n",
        "from bdf.table_parsers import DelimTxtParser\n",
        "from bdf.table_normalizers import TableNormalizer, Syn\n",
        "from bdf.metadata_parsers import TxtPreambleParser, MetadataSchema\n",
        "from bdf.file_utils import resolve_source\n",
        "\n",
        "BIOLOGIC_URL = (\n",
        "    \"https://zenodo.org/api/records/18986774/files/\"\n",
        "    \"SINTEF__NaCR32140-MP10-04__2025-08-25__GITT_0p05C_25degC__BioLogic.mpt/content\"\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Each key is a plugin ID; the value is the Plugin instance\n",
        "list(PLUGINS.keys())"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3",
      "metadata": {},
      "source": [
        "## Constructing a custom `Plugin`\n",
        "\n",
        "A `Plugin` takes a `table_parser` and a `metadata_parser`. The structure mirrors the built-in\n",
        "`BIOLOGIC_MPT` plugin: a `DelimTxtParser` for the tabular data and a `TxtPreambleParser` for the header metadata."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4",
      "metadata": {},
      "outputs": [],
      "source": [
        "custom_plugin = Plugin(\n",
        "    table_parser=DelimTxtParser(\n",
        "        normalizer=TableNormalizer(\n",
        "            test_time_second=(Syn(hdr=\"time/{unit}\"),),\n",
        "            voltage_volt=(Syn(hdr=\"Ecell/{unit}\"),),\n",
        "            current_ampere=(Syn(hdr=\"I/{unit}\"),),\n",
        "        )\n",
        "    ),\n",
        "    metadata_parser=TxtPreambleParser(\n",
        "        magic=(\"BT-Lab ASCII FILE\",),\n",
        "        regex_patterns=MetadataSchema(\n",
        "            start_time=re.compile(r\"Acquisition started on\\s*:\\s*(.+)\"),\n",
        "        ),\n",
        "    ),\n",
        ")\n",
        "custom_plugin"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5",
      "metadata": {},
      "source": [
        "## Stage 1 \u2014 `detect_from_ext_or_magic_bytes()`\n",
        "\n",
        "Extension filtering is cheap but imprecise: `.txt` is used by multiple formats.\n",
        "`detect_from_ext_or_magic_bytes()` returns all plugins that accept the file's extension,\n",
        "falling back to magic-byte sniffing when the extension is missing or unmatched."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6",
      "metadata": {},
      "outputs": [],
      "source": [
        "biologic_file = resolve_source(BIOLOGIC_URL)\n",
        "\n",
        "ext_candidates = detect_from_ext_or_magic_bytes(biologic_file)\n",
        "print(\"Candidates from extension (.txt):\", list(ext_candidates.keys()))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7",
      "metadata": {},
      "source": [
        "## Stage 2 \u2014 `detect_from_metadata()`\n",
        "\n",
        "Magic token matching reads only the file's head bytes and is much more precise.\n",
        "`detect_from_metadata()` filters the candidates to those whose `metadata_parser` matches."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8",
      "metadata": {},
      "outputs": [],
      "source": [
        "meta_candidates = detect_from_metadata(biologic_file, ext_candidates)\n",
        "print(\"Candidates after metadata filter:\", list(meta_candidates.keys()))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9",
      "metadata": {},
      "source": [
        "## Stage 3 \u2014 `detect_from_columns()`\n",
        "\n",
        "`detect_from_columns()` sniffs the column headings and returns the plugin with the highest normalizer score.\n",
        "It returns a `(plugin_id, Plugin)` tuple."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "10",
      "metadata": {},
      "outputs": [],
      "source": [
        "plugin_id, plugin = detect_from_columns(biologic_file, meta_candidates)\n",
        "print(\"Resolved plugin:\", plugin_id)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "11",
      "metadata": {},
      "source": [
        "## Top-level `detect()` shortcut\n",
        "\n",
        "`detect()` orchestrates all three stages and returns early when the candidates are already unambiguous."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "12",
      "metadata": {},
      "outputs": [],
      "source": [
        "plugin_id, plugin = detect(biologic_file)\n",
        "print(\"Detected:\", plugin_id)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "13",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Read the normalised data using the detected plugin\n",
        "plugin.table_parser.read(biologic_file).collect()"
      ]
    }
  ],
  "metadata": {
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12.8"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}