A prototype microchip design revealed today by IBM could pave the way for a world of much smarter devices that don’t rely on the cloud or the internet for their intelligence. That could help soldiers who operate drones, ground robots, or augmented-reality gear against adversaries who can target electronic emissions. But the new chip–modeled loosely on the human brain–also paves the way for a different sort of AI, one that doesn’t rely on big cloud and data companies like Amazon or Google.
Unlike traditional chips that separate memory from processing circuits, the NorthPole chip combines the two–like synapses in the brain that hold and process information based on their connection to other neurons. Writing in the journal Science, IBM researchers call it a “neural inference architecture that blurs this boundary by eliminating off-chip memory, intertwining compute with memory on-chip, and appearing externally as an active memory.”
Why is that important and what does it have to do with the future? Today’s computers have at least two characteristics that limit AI development.
First, they need a lot of power. Your brain, running on just 12 watts of power, can retain and retrieve the information you need have a detailed conversation while simultaneously absorbing, correctly interpreting, and making decisions about the enormous amount of sensory data required to drive a car. But a desktop computer requires 175 watts just to process the ones and zeros of an orderly spreadsheet. This is one reason why computer vision in cars and drones is so difficult, a huge limiting factor for autonomy. This energy inefficiency is one reason why many of today’s AI tools depend on enormous enterprise cloud farms that consume enough energy to power a small town.
The second problem is that we’re reaching the atomic limit of how many transistors we can fit on a chip.
The NorthPole prototype chip may be able to solve these two problems. “What we really set out to do is optimize every joule of energy, every capital cost of a transistor, and every opportunity for a single clock cycle, right? In an interview, IBM Senior Fellow Dharmendra Modha explained that the chip has been optimized in three dimensions: energy, time and space.
The NorthPole chip has 22 billion transistors and 256 cores, according to the paper. There are, of course, chips with more transistors and more cores. NorthPole is able to perform tasks such as processing moving images more efficiently due to its unique architecture. Against a comparable chip with “12nm silicon technology process node and with a comparable number of transistors, NorthPole delivers 25 higher frames/joule,” according to the paper. You could reduce the size of a cloud considerably if you were to use a large number in a cloud-based enterprise environment for a program that generates AI, such as ChatGPT. Cloud computing, which used to require a huge building full of servers now fits on the back of an airplane. But of course you also need fewer chips for things like small drones and robots.
Pentagon interest
IBM has been working on such neuromorphic chips for more than ten years, with funding from DARPA’s SyNAPSE program. The program spent more than $50 million between 2008 and 2014, and DOD has since 2019 invested another $90 million on the chips, a top Pentagon research official said.
“This is a prime example of what we would call patient capital,” Maynard Holliday, assistant defense secretary for critical technologies in the Office of the Under Secretary of Defense for Research and Engineering.
“For us, in the Department of Defense, we’ve always been looking for low…size, weight and power, and then increased speed for our processors. With the advent of generative AI we recognized that to do [computation] in a low-power fashion; we would need this kind of architecture…especially in a contested environment where our signals may be jammed. GPS may be denied, to be able to do compute at the tactical edge is an advantage.”
Smarter and network-independent chips could vastly improve the ability of various military systems–drones, ground robots, soldier headsets–to perceive and interpret the world around them. Holliday explained that they could help to ingest more data including infrared and sonar as well as audio. They also enable new sensors such as the “micro-power impulse radar”.
“It is able to segment, which means that it can distinguish people from a photo, and it can classify sound for you. Again, this can be done without using the internet. It could also revolutionize self-driving cars, not just for the military but in the commercial sector. You can look at this in terms of vehicles [or from a dismounted soldier standpoint.”
Various military labs, including the Air Force Research Lab and Sandia National Lab, are already looking into uses for the prototype chip, Holliday said.
NorthPole may even enable the military to do more with fewer and more domestically producible chips. That’s a rising concern as more and more officials warn of a potential Chinese invasion of Taiwan, one of the main suppliers of advanced microprocessors for phones, cars, etc.
Holliday said NorthPole already rivals the most advanced chips out of Asia, and future versions are expected to be even more efficient.
“NorthPole is at 14 nanometers. You know, the state-of-the-art stuff that’s in our iPhones and other commercial electronics is three nanometers. And that’s produced all in Asia at TSMC [in Taiwan], and Samsung. And so the fact that this chip is performing the way it does at 14 nanometers bodes very well. As we descend that technology node curve to single digit nanometers, it’s just going to be ever better performing.”
But the United States still has to heavily boost its ability to fabricate such chips in large quantities, he said, a process that’s barely begun.
The post Microchip breakthrough may reshape the future of AI appeared first on Defense One.
