Apollo3 module

Battery-Free Game Boy

This is the official public repository for ENGAGE: the battery-free gaming console.

Project Overview

Rationale

Battery-free devices can make phone calls, facilitate machine learning, or even track eyes, but immersive interactive experience without batteries has not been explored so far. In this project we focus on this ignored part of the battery-free device ecosystem. We explore how reactive and user-centric applications, mobile gaming in particular, can also be battery-free.

The Basic Problem of Gaming without Batteries: Power Failures

Powering small embedded computer only with energy harvesting would cause frequent power failures, resulting in a constant restart of a long-running application. Let us look at the above figure which symbolically represents an execution of a Tetris game on a battery-free platform powered by harvested (fluctuating) energy. Without any protection against power failures the game would play until energy is lost (i.e. at line 185) and then would restart from the loading screen. Intermittent computing techniques would protect from this unwanted behaviour: after a power failure, line 186 would be executed proceeding from the exact system state as before the power failure. This way a game state will resume at the exact frame where the play ended due to power failure (in other words, tetromino will be in the same place when the game resumes).

System Design: Game Boy Emulation and System State Checkpointing

We designed a Nintendo Game Boy emulator, which we call Energy Aware Gaming (ENGAGE) platform, as a proof of concept that interactive systems powered by only by (intermittently-available) energy harvesting is possible. ENGAGE is build around four key components (see also figure above):

1. Game emulation: Use processor emulation to play (Game Boy) retro games to resemble classic Game Boy platform feel;
2. Kernel: Efficiently checkpoint minimal game state to the dedicated non-volatile memory to protect from a restart after a power failure, enabling continuous game play;
3. Energy: Power battery-free gaming console from gaming actions (button presses) and from ambient solar energy and store this energy in a small capacitor;
4. Hardware: Exploit leading (but off-the-shelf and accessible to everyone) low-power 32-bit ARM Cortex M4 microcontroller and external FRAM to support checkpointing system.

Hardware

ENGAGE internal PCB is depicted on the above figure (left-hand side: front, right-hand side: back) with the main components (A) Ambiq Apollo3 Blue ARM Cortex-M4 MCU, (B) Fujitsu MB85RS4MT 512 KB FRAM, (C) ZF AFIG-0007 energy harvesting switch, (D) Semiconductor Components Industries NSR1030QMUTWG low forward voltage diode bridge, (E) micro USB debugging port, (F) display connector, (G) solar panels connector, (H) cartridge interface, (I) Texas Instruments BQ25570 harvester/power manager, and (J) Texas Instruments TPS61099 boost converter.

The specific internals of ENGAGE are as follows.

Microcontroller: ENGAGE is based on ARM MCU architecture, a departure from Texas Instruments MSP430 FRAM series microcontrollers typically used in intermittent computing systems. Specifically, we have used Ambiq Apollo3 Blue ARM Cortex-M4 (operating at a clock frequency of 96 MHz) chosen for its best-in-class energy efficiency. Apollo3 MCU (marked as (A) in the above figure) runs the Game Boy emulator and checkpointing software.

Memory: A fast, byte-addressable, non-volatile memory is crucial to enable saving and restoring ENGAGE state (in other words, game state) in spite of power failures. To enable this, we equipped ENGAGE with an external dedicated FRAM IC: Fujitsu MB85RS4MT 512 KB FRAM, marked as (B) in the above figure, connected through SPI to the MCU providing a fast (enough) and durable method of non-volatile storage for checkpoints.

Energy Harvesting: We extract energy from two sources: (i) button presses, using six mechanical off-the-shelf ZF AFIG-0007 energy harvesting switches - four switches at the D-pad and one switch per "A" and "B" button (see (C) in the above figure), and (ii) a set of Panasonic AM-1417CA amorphous silicon solar cells attached to the front of the Game Boy chassis. Solar panels are managed by Texas Instruments BQ25570 harvester/power manager. Harvested energy is stored in a 3.3 mF capacitor.

Cartridges: To be able to read Game Boy game cartridges, a cartridge connector is placed on the back of the ENGAGE platform PCB (marked as (H) in the above figure). The cartridge interfaces with the MCU using Semtech SX1503 I/O expanders. This extender translates the 3V system voltage to 5V logic required by the cartridge.

Display: It was difficult to source a large display that had reasonable power requirements, a decent refresh rate, and minimal external driving circuitry. We quickly discarded the use of electronic ink displays because of its slow refresh rate and complex driving circuitry. Instead we use a 26.02 × 27.82 mm Japan Display LPM013M126A LCD. The display is is smaller than the original Game Boy screen (47 × 43 mm) but has a better resolution of 176 × 176 pixels compared to original 160 × 144 pixel matrix.

Checkpointing

The core software component of ENGAGE is differential checkpointing library called MPatch located in software/libs/mpatch folder of this repository. Further description of MPatch is provided in a separate document.

Getting Started

This repository contains everything required to build your own Battery-Free Game Boy. Below we describe how to build your own ENGAGE hardware and software.

Building the Hardware

Hardware-related sources are stored in the following folders.

• /enclosure: Complete set of 3D STL files of the complete ENGAGE enclosure (front, back, buttons, D-pad, LCD holder)
• /hardware/main-board: Main ENGAGE board KiCad files. Specifically, the folder contains .gbr Gerber output files for manufacturing, .drl drill file for manufacturing, schematics of the PCB in .pdf format, .csv file with bill of materials and the KiCad project files .kicad_pcb, .pro and .sch.
• /hardware/apollo-3-module: As above, but for the de-attachable board with the MCU and FRAM (see above figure - component (A) and (B))

Enclosure can be printed using your favorite 3D printer or you can send all /enclosure files to one of online-available 3D printing services. As for the PCBs, you can either assemble everything yourself or send all /hardware files to many online-available PCB assembly companies for production.

Building the Software

Software-related sources are stored in the following folders.

• /software/apps: Applications' source code folder containing three applications:
  • emulator: main application of ENGAGE emulator, which can be also used to reproduce results of the paper;
  • checkpoint: application to test the performance of checkpointing mechanism;
  • memtracker: application to test the performance of memory use tracking mechanism;
• /software/config: Various configuration files.
• /software/external: External dependencies to the project, such as MCU SDK.
• /software/libs: Set of core libraries handling I/O (that is console buttons, game cartridge, console display, Game Boy emulator, FRAM access) and checkpointing (that is just-in-time checkpoint, memory state tracker, and MPatch checkpointing).
• /software/platforms: GPIO pin mappings.
• /software/scripts: Scripts pertaining to the build process.

Note on the Game Boy Emulator

We do not provide the copy of the Game Boy emulator in this repository. Our build system (using /software/scripts) automatically downloads and patches (applying optimizations and improvements) an external, not written by us, Game Boy emulator. Specifically, we have used Game Boy emulator for STM32F746 Discovery board written by Uwe Becker.

Build Process

After cloning this repository to your local computer the following instructions can be used to generate the ENGAGE binaries. From the project directory execute the following.

$ cd software/build
$ cmake ..
$ make install

Then using your favorite debugger (we have used one of SEGGER J-Link debug probes for this project) the ENGAGE binaries (stored in the /software/bin folder) can be uploaded to the MCU using a debugger whilst powering the platform using USB. The main binary is called emulator whilst the others provide a means to reproduce the results obtained in the paper. After this step a game cartridge can be loaded.

Last Tested Software Versions

This software was build successfully using the following software versions.

• CMake version 3.18.4
• arm-none-eabi-gcc version 10.2.0
• arm-none-eabi-newlib version 3.3.0
• arm-none-eabi-binutils version 2.35

How to Operate ENGAGE

There are two important operations when using ENGAGE that are not related to normal gaming: loading game cartridge and system reset, both described below.

Load Game Cartridge

After programming the emulator binary to ENGAGE, a Game Boy cartridge with your favorite game needs to be loaded to ENGAGE. To load a cartridge, first put the physical cartridge in the cartridge slot (at the back of ENGAGE) and then hold down Start and A buttons on ENGAGE while powering ENGAGE through the USB port. You can let go Start and A buttons after a few seconds. When loading of the cartridge has completed the game will start. Note that there is currently no progress indication of this operation on the screen of ENGAGE. Remember that the process can take five minutes (or more) for large cartridges! After a cartridge has been loaded it can be removed from the device.

System Reset

Due to the unique nature of ENGAGE's state resuming after power failure, resetting ENGAGE requires a unique key combination (unlike the original Game Boy that you could turn off and on again easily). This reset is performed by holding Start and Select buttons while powering ENGAGE through an USB port. Please note that this reset operation does not require you to reload the game cartridge.

Frequently Asked Questions

• When the energy source is changed from solar to button-mashing, how long is ENGAGE temporarily off?

We actually harvest from both energy sources at the same time! We developed a circuit that manages this internally. However, this still means that the user will experience interruptions of play, that can vary between sub-second (in extreme well outdoor light conditions) to many seconds (on a gloomy day, indoors). While the button presses generate extra energy to sustain game play, most of the energy is still generated by solar panels. More discussion on this can be found in our research paper.

• How do the buttons generate power?

The buttons generate power by moving a small but powerful magnet inside a tightly wound wire coil. The change in the magnetic field generates power. When you press the button (and when you release) it moves the magnet through the coil, this energy is then siphoned into a capacitor for immediate use by the ENGAGE hardware to support all activities. Because of advances in manufacturing over the past decade the magnet and coil are so small that they can fit inside a button that is acceptable to a user.

• Why is the screen so small?

Yes, the screen is smaller that in the original Game Boy (which is a source of complaints of many internauts), but the one we used in the initial ENGAGE design the best one we could find in terms of resolution and energy consumption. If you have a better screen to suggest, then make a new project issue and send us a pull request.

• Can ENGAGE play sound?

Unfortunately not. Sound generation costs extra energy and as we operate on a very thin energy budget we had to make choices which features of original Game Boy had to be excluded from ENGAGE.

• Can I buy one?

Unfortunately not. But you can build one! All hardware and software of ENGAGE, together with the process of how to make it run is simply in this repository. Please remember that ENGAGE was not (and is not meant to be) a product we can sell. This is a research project and a concept demonstration.

How to Contribute to this Project

We look forward to your contributions, improvements, additions and changes. Please follow the standard GitHub flow for code contributions. In macro terms this means the following.

1. Fork the master branch of this repository; make sure that your fork will be up to date with the latest master branch.
2. Create an issue here with a new feature or a bug report.
3. Perform changes on your local branch and push them to your forked clone.
4. Create a pull request referencing the issue it covers and wait for our response.

List of Known Issues

List of all known issues is listed in the Issues list of this project. If you found a bug or you would like to enhance ENGAGE: please contribute! We look forward to your additions. Let ENGAGE grow and become a better and more sustainable gaming experience.

How to Cite This Work

The results of this project have been published in a peer-reviewed academic publication (from which all technical figures in this file originate). Details of the publication are as follows.

• Authors and the project team: Jasper de Winkel, Vito Kortbeek, Josiah Hester, Przemysław Pawełczak
• Publication title: Battery-Free Game Boy
• Pulication venue: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 4, Issue 3, September 2020 and Proceedings of ACM UbiComp 2020
• Link to publication: https://doi.org/10.1145/3411839 (Open Access)
• Link to ACM UbiComp 2020 conference presentation video: https://www.youtube.com/watch?v=XxjrTHBFBSM

To cite this publication please use the following BiBTeX entry.

@article{dewinkel:imwut:2020:gameboy,
  title = {Battery-Free Game Boy},
  author = {Jasper {de Winkel} and Vito {Kortbeek} and Josiah {Hester} and Przemys{\l}aw {Pawe{\l}czak}},
  journal = {Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.},
  volume = {4},
  number = {3},
  pages = {111:1--111:34},
  year = {2020},
  publisher = {ACM}
}

Related Websites

• Project source code: https://github.com/tudssl/engage (this repository)
• Project announcement website: https://www.freethegameboy.info

The Press

The project has been covered extensively by international media. Here is the list of example posts and articles (as of October 30, 2020).

CNET, The Wall Street Journal, Mashable, Hackaday, The Verge, Gizmodo, Engadget, PCMag, The Register, Tech Times, Nintendo Life, Daily Mail, The Independent, r/gadgets

Acknowledgments

This research project was supported by Netherlands Organisation for Scientific Research, partly funded by the Dutch Ministry of Economic Affairs and Climate Policy, through TTW Perspective program ZERO (P15-06) within Project P1 and P4, and by the National Science Foundation through grants CNS-1850496 and CNS-2032408. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Copyright

Copyright (C) 2020 TU Delft Embedded and Networked Systems Group/Sustainable Systems Laboratory.

MIT License or otherwise specified. See license file for details.