Ecp5-fpc adapter

[CVPR 2026] SpiderCam: Low-Power Snapshot Depth from Differential Defocus

Marcos A. Ferreira^*,1, Tianao Li^*,1, John Mamish^*,2, Josiah Hester², Yaman Sangar², Qi Guo³, Emma Alexander^1
1Northwestern University, ²Georgia Institute of Technology, ³Purdue University, ^*Equal contribution.

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026

Official code for SpiderCam: Low-Power Snapshot Depth from Differential Defocus.

Repository layout

Running DfDD

Setup

To install required packages:

pip install -r requirements.txt

Data

Point data_path in a config under configs/data/ to your capture directory. Each group folder should contain paired frames:

• cam_1_500_480_{idx}.png — near-focus (+) image
• cam_0_500_480_{idx}.png — far-focus (-) image

Crop, frame range, depth step/offset, and image center are set in the same YAML (see e.g. configs/data/11-06.yaml).

Model parameters

Defaults live in configs/model/focal_split.yaml. Override any of them on the CLI with Hydra (model.<key>=...).

Optical constants (model.const). DfDD recovers depth from a linear combination of the image Laplacian and the differential image, with learnable maps A and B (one pair per scale):

• universal — single scalar (A, B) per scale (spatially constant; ablation / simplest FPGA mapping).
• rings — piecewise-constant (A, B) on concentric rings about data.center (paper default; matches on-device zone tables).
• radial — A(r), B(r) as low-order polynomials in image radius.
• polynomial — full 2D polynomial in (x, y).
• pixel-grid — independent (A, B) at every pixel (most flexible, heaviest).

Confidence (model.conf). With numerator/denominator terms built from V and W:

• VW — confidence ∝ V·W (paper default; aligns with the depth numerator).
• W2 — confidence ∝ W² (denominator energy).
• V / W — confidence ∝ |V| or |W|.

Higher confidence pixels are kept when evaluating at a given sparsity (see optim.sparsity).

Fusion (model.mode). Softmax weights ω combine channels across scales (and dx/dy if enabled):

• joint — fuse numerator and denominator first, then divide (more stable; default).
• separate — form a depth map per channel, then confidence-weighted average.

Paper setting (recommended start):

model.n_scales=2 model.dxdy=true model.mode=joint model.const=rings model.n_rings=16 model.conf=VW

Train

Calibrate / train optical constants with Hydra overrides:

python train_focal_split.py data=11-06 data.group=texture1 \
  model.n_scales=2 model.dxdy=true model.mode=joint model.const=rings model.n_rings=16 \
  model.conf=VW

Checkpoints and logs are written under ./results/. For the full ablation suite used in the paper, see train.sh.

Evaluate

After training, run inference with the same data/model/optim settings (must match the trained checkpoint path):

python run_focal_split.py data=11-06 data.group=texture1 \
  model.n_scales=2 model.dxdy=true model.mode=joint model.const=rings model.n_rings=16 \
  model.conf=VW

This loads checkpoint_{n_epochs}epochs.pth, reports MAE/MSE and working range at several sparsities, and saves heatmaps / depth maps under ./results/.

Live hardware (optional)

With the SpiderCam FPGA connected over FT232H:

python serialcam_ft232h/serialcam_stream_ft232h.py

Requires the serialcam stack (PyQt5, ftd2xx). Use Set DfDD Parameters in the viewer to push A/B/ω and confidence thresholds to the device.

Citation

@InProceedings{ferreira2026spidercam,
    author    = {Ferreira, Marcos A. and Li, Tianao and Mamish, John and Hester, Josiah and Sangar, Yaman and Guo, Qi and Alexander, Emma},
    title     = {SpiderCam: Low-Power Snapshot Depth from Differential Defocus},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2026},
    pages     = {41699-41709}
}

Acknowledgment

This research was partially supported by the National Science Foundation under award numbers CNS-2430327 and CCF-2431505. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation or other supporters. We would also like to thank Junjie Luo and Alan Fu for helpful discussions.