ATARI CTRL PCB 1.25A UNTESTED BUT FIXED

Neuropong

This is the repo contains the NeuroPong project files. NeuroPong is an embedded, neuromorphic Atari (Pong) agent. It marries an event-based camera, TENNLab's Neuromorphic Starter Kit (https://arxiv.org/abs/2211.04526), and an Atari 2600 console. Spiking neural networks are trained on simulated events created from frames generated by the Arcade Learning Environment, and then, those spiking neural networks are put on hardware to examine how they perform in a live setting using real event camera inputs.

Prerequisites

Repositories needed to run existing spiking neural network agents:

• TENNLab Open-Source Framework: https://github.com/TENNLab-UTK/framework-open
• Inivation dv-processing Library: https://gitlab.com/inivation/dv/dv-processing
• TENNLab Embedded Neuromorphic repository(*):
  • Needed to compile code for Neuromorphic Starter Kit's Raspberry Pi Picos

(*) You will need to get access to this repo from Jim Plank (jplank@utk.edu) until it is open-sourced.

You will also need Python and a shell of some kind (bash/zsh) on Linux or Mac. It is recommended to set up a python virtual environment (Python <= 3.9) for compatability with Inivation event camera library.

Contents

• /models: STL models of 3D printed objects used in this project for the presented prototype

• /kit-code: Contains code used by embedded-neuromorphic/ repo to compile kit code.

• /pcb: Repo containing KiCad files for the PCB that contains the Atari Translator circuit

• /scripts: Scripts used to assist with flashing Picos and compiling kit code. calibration.py and event_server.py are used on the Raspberry Pi (or whatever is communicating with the Neuromorphic Starter Kit)

• /imgs: Contains various diagrams and images detailing parts of the NeuroPong system

• /networks: Contains the networks generated for the Neuropong paper.

Table of Hardware Materials/Components

• The Atari 2600 console and VideoOlympics cartridge together were ~$20.00.
• The DAVIS346 camera used for this project was purchased from [Inivation] (https://shop.inivation.com/).

Running the Hardware Setup

1. Turn on monitor, put VideoOlympics cartridge in Atari 2600 console, and turn on the Atari console.
   • You should see the Pong game appear on the screen.
2. Plug NeuroPong controller into the back of one of the Atari controller ports.
   • NOTE: you will need to make the NeuroPong cable. An detailed construction diagram can be found in the /imgs folder
3. Power on the Raspberry Pi, and plug DAVIS346 camera into one of the USB ports on the Raspberry Pi.
4. Compile the network using app_compile from the embedded_neuromorphic/ repo
   • Ex: ./bin/app_compile -j network_file -o kit-code/pico-pong_observations.c -a kit-code/pico-pong_actions[2,3].c -s
5. Flash the three Raspberry Pi Picos on the kit with the appropriate encoder, decoder, and network.
6. Power on the Neuromorphic Kit
7. On the Raspberry Pi, run scripts/calibration.py to assist with focusing the lens and cropping the camera's FOV to the screen.
   • The output produced by scripts/calibration.py is the command that is used to run scripts/event_server.py.
     • It figures out what the crop coordinates are and what the downsampling parameters need to be to match whatever the target downsampled resolution is. Target resolution is specified via global variables in the scripts/calibration.py program.
8. Run the command output by scripts/calibration.py
   • Ex: python3 event_server.py --crop ...
9. Wait 20-30 seconds for the system to initialize. If it's unreponsive, CTRL-C the event_server process and rerun it.
10. Begin the Atari game by toggling the ``Game Reset'' switch on the Atari system.

Running the networks in the framework-open processor_tool

You may want to run the networks using framework-open/bin/processor_tool in order to view spike rasters for a network at each timestep during an episode of that network playing Pong in the Arcade Learning Environment. In networks/processor_tool_commands are the commands for the processor tool, for each of the 32 networks, for one 2000 frame Pong episode each. Each file in networks/processor_tool_commands contains the Apply_Spike (AS) commands that communicate the event simulated and downsampled representation of each frame as they're generated by the ALE online. To run, do the following:

        ```
        UNIX> pwd
        .../neuropong
        UNIX> ./framework-open/bin/processor_tool_risp < networks/processor_tool_commands/pong0.txt
        0   INPUT  :    // No inputs are provided to the network during the first frame or timestep, so the spike raster is empty
        1   INPUT  : 
        2   INPUT  : 
        3   INPUT  : 
        4   INPUT  : 
        5   INPUT  : 
        6   INPUT  : 
        7   INPUT  : 
        8   INPUT  : 
        9   INPUT  : 
        10  INPUT  : 
        11  INPUT  : 
        12  INPUT  : 
        13  INPUT  : 
        14  INPUT  : 
        15  INPUT  : 
        16  INPUT  : 
        17  INPUT  : 
        18  INPUT  : 
        19  INPUT  : 

More information about processor_tool_risp can be found in the framework-open/markdown/ directory. This is provided as a method for you to see how the network acts when provided event-simulated and downsampled ALE frames.