Thermal camera v2

Thermal camera

An open source thermal camera, including firmware, schematic and PCB design.

Principle of operation

Showing a frame on the thermal camera requires to perform three operations

• Reading the raw ADC data from the sensor via I2C
• Processing the data (compute temperatures, compute pixel colors)
• Writing the computed frame to the display via SPI

The sensor frame rate can be set anywhere from 1fps to 32fps in discrete increments (1,2,4,8,16,32). However, on the STM32F205RC processing one frame takes ~41ms of CPU time, so the maximum achievable frame rate is 16fps. TODO: to go faster we could use an MCU with a floating point unit such as an STM32F4.

Focusing on the current hardware, we have another problem: consider the 16fps rate, the required operations take the following time

• Reading sensor via I2C takes ~62.5ms (1/16s, mostly waiting for the sensor)
• Processing takes ~41ms
• Writing to display requires to send 128x128 16 bit pixels on a 16MHz SPI bus or ~18ms

Thus, the effective frame rate would be 1/(62.5ms+41ms+18ms)=8fps.

To solve this issue, we use pipelining. Instead of doing the operations sequentially

we use pipelining to perform each operation at the same time on three different data buffers

As always with pipelining, the latency (lag) is the same or higher than in the sequential case, but the throughput (frames per second) matches the longest pipeline stage, which in this case is waiting for the sensor.

But, I hear you say, how can you perform the three operations in parallel on a single-core CPU?

The answer is DMA. You may have noticed that two of the required operations are just receiving/sending data through a peripheral such as I2C/SPI. Thus with appropriately written device drivers which make use of DMA, the threads performing the read/write operations will be waiting on I/O letting the OS scheduler run the CPU-intensive computation thread.

Thread priorities are set so that the I/O threads have higher priority than the computation thread, making sure the computation thread is preempted as soon as I/O operations complete.

From an implementation perspective, the thermal camera uses the Miosix OS and the three threads performing the mentioned operations are in application.cpp:

• The sensor reading thread is Application::sensorThreadMain()
• The processing thread is Application::processThreadMain()
• The display writing thread is Application::renderThreadMain()

How to build

This guide assumes you already have a Miosix build environment installed, if not see here.

# Dependencies (should work both on Linux and WSL)
sudo apt install build-essential cmake libboost-all-dev libpng-dev

# Submodules
git submodule init
git submodule update

# Build mxgui code generators for embedding images and fonts
cd mxgui/_tools/code_generators
chmod +x compile_freetype.sh
mkdir build
cd build
cmake ..
make

# Build the thermal camera application
cd ../../../..
make -j`nproc`

How to flash

Install the dfu-util program on your pc.

Connect the USB cable between the PC an thermal camera, while the camera is off.

Use a jumper wire to connect BOOT0 (on the 4 pin header) to +3.3V (a convenient spot is a capacitor next to the voltage regulator). You can just hold the wire with your hands.

While the jumper is in place, press and hold the power on button of the camera.

You can disconnect the jumper but keep holding the power button. If you release the power button, you need to reconnect the jumper and start again.

On the PC, run the command while holding the power button.

dfu-util -a 0 -s 0x08000000:leave -D main.bin

You should see a progress bar on the PC, keep holding the power button till the progress bar reaches 100%.

Release the power button, and disconnect the USB cable.

Power on the camera and enjoy the new firmware. You cannot brick the camera, if something goes wrong you can repeat the operation.