A deep learning processor from Syntiant can play the first-person shooter Doom with just one milliwatt of power consumption. The recognition of opponents is not perfect, but good enough for a short round.
The first-person shooter Doom has been used for numerous experiments, for example running the game in the BIOS or on a calculator. Almost 30 years after the release of the original Doom, hobbyists are still not short of ideas. This is shown, for example, by an article published on IEEE Spectrum, according to which a processor can reasonably play the shooter with only one milliwatt of power consumption.
The experiment was conducted by David Garrett, who worked until the beginning of the year Senior Vice President Of Engineering at the Californian chip developer synthant was. With the NDP200, this in turn offers exactly the processor that could play Doom with such low power consumption. This is made possible by the design of the CPU: This is specially designed for applications in which a small deep learning network is to be operated with as little energy as possible. There is a clock of only 100 MHz maximum, and the entire chip is only connected via 40 pins.
Apparently that’s still enough for Doom: In order to control the game with the small processor, Garrett has reinforcement learning and the game version designed for it Vizdoom trained a deep learning network. Using around 600,000 parameters, this calculates the necessary inputs from the image displayed by the game. With the mentioned energy consumption of one milliwatt, the processor used can recognize opponents with 6 fps and make a decision as to which input it should transmit to the game next.
As the short demonstration video shows, this works quite well in a simple level where enemies are running towards the player from every direction. The processor only misses a few times and can reliably detect and shoot at the charging opponents until it runs out of ammunition. In view of the extremely low consumption that is necessary for this, this is quite impressive.
Just as exciting: Doom: Now also runs in the BIOS
It is precisely this particularly high level of efficiency that should be illustrated with the experiment: Garrett wanted to show that such a weak CPU is also suitable for recognizing people. Such an application should present the small chip with significantly greater challenges than Doom. After all, in a real photo, there are far more options for how characters can look and where they are positioned.