PCB FINAL2.1

Termite Terminator

For a more in-depth dive into the working and development of this project, we refer to the paper also available in this repository. Below is brief summary on how to set up the game yourself.

The game

The player manages a coin budget and uses a spell 3 types of tangible towers to face off waves of ants attempting to pass by and eat your picknick. There is a heavy hitting tower with a long cooldown, a tower with a continuisly hitting but low damage laser and a tower that shoots as fast as the player presses its button. Additionally, there is a Freeze spell to hold enemies in place for a few seconds. The ants get increasingly stronger and come in larger groups, but the more enemies are held back, the more coins you earn for more defenses. 🎥 Watch a demo video on YouTube

Credits

Both repositories below have been used as a foundation for the python script used for the marker and gesture detection:

• OpenCV Marker Detection: https://github.com/KhairulIzwan/ArUco-markers-with-OpenCV-and-Python
• OpenCV Gesture Detection: https://github.com/kinivi/hand-gesture-recognition-mediapipe

Setup Instruction

The following instructions are open to adjustments, if the operator deems themselves worthy and knowleadgable enough to customize the setup. But for the others, these next few sections explain how to duplicate our specific setup rather than reconfigure everything.

Software requirements

Unity Version:

• 2022.3.49f1

python libraries: (Python 3.9 works)

• opencv-contrib-python==4.6.0.66
• imutils
• tensorflow
• mediapipe

Wireless Connection

Background Information

UDP is used to communicate between the main processing computer running the Unity and camera vision, and a set of ESP8266 microcontrollers processing and controlling everything concerning the individual towers.

During the design procedure, hardcoded IP's are used to easily indentify towers and communicate with them accordingly. To set up the wireless network we rely on a mobile hotspot from a modern phone, with a preset 255.255.255.0 subnetmask.

Configuration

Following table shows all IP-addresses used:

The easiest way to set this network up on new devices is duplicate the original network. Specifically, the network name should be "Galaxy S22A88A" and its password should be "uzjw7402". It is also essential to turn off roaming and internet sevices on the mobile phone, to avoid any external DHCP server to set IP's rendering them unable to be overwritten. Next, the IP of the main computer should be set to match the one in the table above.

Camera Vision & Towers

This part of the setup refers to the physical configuration. The fundamentals come down to a TV being laid flat on a table (projector would work, but the shadows mess with the gesture recognition) with a camera of any sort hanging above it. Ideally, the camera hangs straight above the center of the screen.

Within Unity, more specifically in scene "Wave1" in the "LevelManager" the setup can be calibrated. Under the TowerSpawner script skewFactorX, skewFactorY, skewFactorXX, skewFactorYY and heightLimit can be adjusted to conform to the height of the camera, causing a paralax error.

As for the towers, they work on 9V batteries. So, these batteries have to be connected, after which the pins from the connector have to be put in the correct pinheader. The program on the microcontrollers should still be programmed and continue sending and receiving data as normal. As for the recognition of the towers, original aRuCo markers are used with towers ranging from 1 to 7 and the corners of the TV being 21, 22 and 23.

Start the Game

After you managed to follow all the instruction above, you can now finally launch the game. Simply follow the next few steps to start everything up:

• launch the Unity game
• go to /TermiteTerminator/Python/Gestures/ and launch the python Camera Vision script combinedScripts.py
• make sure to place the three corner markers on your play area
• start the unity game

With this, the script should be sending all required information to the Unity game. Have fun!