Entropy32-Recorder

Entropy32 Recorder

Firmware, hardware, and tooling for the Entropy32 Recorder — a standalone Geiger-counter-based board, built around an ATmega328P (Arduino Nano form factor), for recording raw entropy source data.

This is a separate board from Entropy32 / Entropy32 Plus, not a mode or alternate firmware of it. It does not generate seeds or any conditioned output. Its only job is to capture raw, unconditioned edge timing data from the entropy source (Geiger tube + LM393 comparator) so it can be run through statistical test suites (e.g. NIST STS) to validate the quality of the entropy source itself, before that source is trusted for use in Entropy32 Plus's seed generation.

Repository layout

• entropy32_evidence_protocol.md — the SP 800-90B reproducible evidence protocol these tools implement: capture requirements, the raw/derived evidence package layout, exact firmware reconstruction semantics, and the pinned NIST toolchain invocation.
• entropy32_recorder.ino — the recorder's firmware. It timestamps every rising edge on D2 (post-LM393 comparator output) and streams edge_index,timestamp_us CSV rows over serial. It does no filtering, interval pairing, or SHA-256 conditioning by design — it's for capturing raw edge timing data only, unmodified, for statistical testing.
• tools/capture_serial_to_csv.py — reads the serial stream produced by the sketch above and writes it to a CSV file matching the evidence protocol's raw/raw_edges.csv shape (edge_index, raw_timer_ticks, monotonic_timestamp_us). raw_timer_ticks is the board's micros() reading exactly as received (wraps every ~71.58 min); monotonic_timestamp_us is that value unwrapped into an ever-increasing 64-bit microsecond count, reconstructed on the host (via tools/edge_timing.py) so the firmware/ISR stays minimal. Every row is flushed and fsynced to disk as soon as it's written — negligible cost at this source's event rate, and each edge is an irreplaceable physical event, so nothing is ever buffered short of an actual power loss. One capture is one continuous, uninterrupted run from edge 0 — per the evidence protocol, there is no resume; the tool refuses to run at all if --out already has data, and an interrupted capture must be restarted under a new --out rather than continued. On a fully complete run, also writes a capture_status.json sibling (e.g. raw_edges.status.json) recording edge count, start/end UTC timestamps, and the zero overrun/drop counts implied by reaching that point at all — this feeds tools/package_evidence.py and the protocol's capture_status.json (§4). Requires pyserial.
• tools/edge_timing.py — the timer-wrap-unwrapping math used by the capture tool, in its own module mainly so it can be tested in isolation.
• tools/check_edges.py — pre-flight sanity check for a captured CSV: confirms edge_index/monotonic_timestamp_us are well-formed, and compares the inter-arrival distribution against a Poisson expectation to flag comparator bounce/chatter. Run this before feeding a capture into NIST STS / SP 800-90B tooling: python tools/check_edges.py tools/raw_edges.csv.
• tools/derive_evidence.py — reconstructs Entropy32's actual bit-extraction (200us reject filter, non-overlapping interval pairing, second-longer=1/second-shorter=0, ties discarded) per entropy32_evidence_protocol.md §5, producing the protocol's derived/ package — all_intervals_us.csv, firmware_accepted_intervals_us.csv, comparison_bits.bin, derivation_report.json. python tools/derive_evidence.py tools/raw_edges.csv --out-dir tools/derived --source-dir ../entropy32_plus. comparison_bits.bin is ready for ea_non_iid -i -v comparison_bits.bin 1 (bits_per_symbol=1, per the protocol's §8 — not a byte-per-symbol truncation). --source-dir (optional but recommended) points at a local checkout of the firmware these constants are claimed to match, and auto-fills derivation_report.json's source_repo/source_commit/ entropy32_firmware_blob_sha from its git metadata and .ino hash — left "unknown" rather than guessed if omitted.
• tools/package_evidence.py — assembles the protocol's immutable evidence directory (§4) from a capture, a derive_evidence.py run, and a separately-run ea_non_iid pass: copies raw/ and derived/ in, records the exact NIST command/toolchain/exit status under nist/, writes capture_status.json and a manifest.json (fields it can't know from the data — hardware, physical source, environment — are filled from setup.json via --setup, otherwise left null), and computes a SHA256SUMS covering every file in the package.
• tools/verify_evidence.py — independently checks a finished package: hashes, raw structure, a byte-for-byte re-derivation, and the saved NIST run. The one command a replicator needs.
• tools/make_evidence.py — runs everything from a finished capture to a verified package (check, derive, NIST via WSL, package, verify).
• setup.json — the recording rig's description (manifest fields) and local tool paths, read by make_evidence.py.
• KiCad/ — PCB design (schematic, layout, 3D models, and fabrication/production outputs for the Entropy32 Recorder board).

Hardware

The board pairs a Geiger tube's LM393 comparator output with an ATmega328P (D2, using the board's existing pull-down for bias). Design files are in KiCad/, including BOM, designators, netlist, and pick-and-place files under KiCad/production/.

Usage

All commands below run from the repo root (entropy32_recorder/).

Verifying a published package

If you're checking someone else's result, this is the only command you need (Python 3.8+, no dependencies):

python tools/verify_evidence.py entropy32_sp80090b_2026-10-03a

It checks every file against SHA256SUMS, checks the raw capture's structure (no missing edges, consistent timer unwrapping), re-derives the bit stream from raw/raw_edges.csv and requires it to match byte for byte, and checks the saved NIST output belongs to this package's comparison_bits.bin and reports all ten estimators. It ends with VERIFIED, VERIFIED as a PILOT package (consistent, but below the protocol's 1,000,000-sample gate), or FAILED, and exits non-zero on failure. Two optional, slower checks:

python tools/verify_evidence.py entropy32_sp80090b_2026-10-03a \
  --firmware-dir ../entropy32_plus \
  --ea-non-iid ~/SP800-90B_EntropyAssessment/cpp/ea_non_iid

--firmware-dir (any clone of Entropy32-Plus) confirms the firmware at the commit the derivation claims is the one whose hash is recorded. --ea-non-iid re-runs NIST (built as below) and requires identical output; on Windows a ~/... or /... path runs inside WSL automatically.

One-time setup for recording

1. Flash entropy32_recorder.ino to the board, and pip install pyserial.

2. Clone the Entropy32 firmware next to this repo, so the derivation can be pinned to it: git clone https://github.com/captainchapster/Entropy32-Plus.git ../entropy32_plus

3. Build the pinned NIST tool. ea_non_iid is a Linux build — on Windows use WSL (wsl --install, one-time) and run this inside it:

   sudo apt update && sudo apt install -y build-essential libbz2-dev
   git clone https://github.com/usnistgov/SP800-90B_EntropyAssessment.git
   cd SP800-90B_EntropyAssessment
   git checkout 68ed165fd7a3eeef26b87a546ba23f338e82a3f3  # pinned v1.1.8, see protocol §8
   make
   cd selftest && ./selftest && cd ..
   make non_iid

4. Edit setup.json: tools points at the firmware checkout and the built cpp/ea_non_iid; manifest describes your rig (detector, source, environment — null for anything unknown, never a guess); check_edges.dead_time_ns is your detector's dead time, if known. These are written into every package, so you describe the setup once, not per capture.

Recording a capture

1. Run the capture:

   python tools/capture_serial_to_csv.py --port COM5 --out tools/raw_edges.csv

   (COM5 on Windows, /dev/ttyUSB0 on Linux.) By default it runs until the protocol's stop gate — 1,000,000 comparison bits, counted live with the same reconstruction the derivation uses — so it can't stop one bit short the way a fixed edge count can (2,000,000 edges yields only 999,999 bits). For a quick smoke test use --count 2000 instead.

   If the board reports OVERRUN_DETECTED or the capture stops for any other reason, that run is invalid — one capture is one continuous run from edge 0, with no resume. Start over.

2. Build and verify the evidence package:

   python tools/make_evidence.py

   This runs the edge sanity check, the derivation, NIST (through WSL on Windows), packaging and verification, recording the NIST tool's commit and compiler from the build itself. It refuses to package an incomplete capture or an unpinned/modified NIST build. The package is named from the capture's start date, e.g. entropy32_sp80090b_2026-11-20a/.

3. Commit the package directory it prints. Once committed it's immutable — never edit it; a new capture gets a new package.

Running the steps by hand

make_evidence.py only strings together tools you can also run yourself, e.g. to inspect an intermediate result:

python tools/check_edges.py tools/raw_edges.csv --dead-time-ns 720000
python tools/derive_evidence.py tools/raw_edges.csv --out-dir tools/derived --source-dir ../entropy32_plus
# in WSL, from SP800-90B_EntropyAssessment/cpp, save stdout/stderr/exit status verbatim:
./ea_non_iid -i -v /mnt/d/Programming/entropy32_recorder/tools/derived/comparison_bits.bin 1
python tools/package_evidence.py --raw tools/raw_edges.csv --derived-dir tools/derived \
  --capture-id <id> --setup setup.json --nist-command "<exact command>" \
  --nist-stdout <file> --nist-stderr <file> --nist-exit-status <n> \
  --nist-compiler-version "<compiler>"
python tools/verify_evidence.py entropy32_sp80090b_<id>

Known limitations / best practices for a full campaign

• Sample size: a few thousand edges (a few hours at typical background rates) is only enough for a pipeline smoke test. SP 800-90B-style min-entropy estimation wants >= 1,000,000 samples before the result is meaningful — plan campaign length accordingly (at the ~16 CPM background rate seen in the pilot run, 1M edges is roughly 6 weeks of continuous capture).
• Timestamp resolution: micros() on the ATmega328P (16MHz, Timer0 prescaler 64) only increments in steps of 4us. That's negligible at the observed ~3.7s mean inter-arrival time (1 part in ~10^6), so it isn't a concern at current event rates. If a future hardware revision moves the comparator output to the Timer1 input-capture pin (ICP1 / D8 on a Nano) instead of D2, the ISR could latch TCNT1 directly for microsecond-or-better hardware-timestamped edges with no software ISR jitter — worth considering for a higher-throughput source, but out of scope for the current board (D2 is fixed on the fabricated PCB).
• Comparator bounce: check_edges.py compares the inter-arrival distribution against the Poisson process a radioactive source should produce; a pileup of anomalously short gaps indicates LM393 output chatter rather than genuine independent events. No bounce was observed in the pilot 2000-edge run.

License

• Firmware and software (entropy32_recorder.ino, tools/): MIT
• Hardware (KiCad/): CERN-OHL-S v2