The Washington Post says:
"IBM researchers on Aug. 18, 2011 unveiled a new generation of experimental computer chips designed to emulate the brain’s abilities for perception, action and cognition. The cognitive computing chips, informally referred to as the “brain chip,” could yield many orders of magnitude less power consumption and space than used in today’s computers."
December 2011 issue of Scientific American chronicles "10 World Changing Ideas" and amongst them is "A Computer Chip That Thinks - Neuron-based chips could solve unconventional problems" featuring IBM team's work on SyNAPSE / Cognitive Computing.
On December 6, 2011, The New York Times ran a series of articles on "Future of Computing" which included an in-depth profile of DARPA SyNAPSE project by Steve Lohr with quotes from Dr. Todd Hylton, Professor Rajit Manohar, Professor Giulio Tononi, Professor Chris Kello, and myself.
Here is a link.
Paul Merolla, John Arthur, Filipp Akopyan, Nabil Imam, Rajit Manohar, Dharmendra S. Modha,
"A Digital Neurosynaptic Core using Embedded Crossbar Memory with 45pJ per spike in 45nm,"
IEEE Custom Integrated Circuits Conference, September 2011.
ABSTRACT: The grand challenge of neuromorphic computation is to develop a flexible brain-like architecture capable of a wide array of real-time applications, while striving towards the ultra-low power consumption and compact size of the human brain—within the constraints of existing silicon and post-silicon technologies. To this end, we fabricated a key building block of a modular neuromorphic architecture, a neurosynaptic core, with 256 digital integrate-and-fire neurons and a 1024x256 bit SRAM crossbar memory for synapses using IBM’s 45nm SOI process. Our fully digital implementation is able to leverage favorable CMOS scaling trends, while ensuring one-to-one correspondence between hardware and software. In contrast to a conventional von Neumann architecture, our core tightly integrates computation (neurons) alongside memory (synapses), which allows us to implement efficient fan-out (communication) in a naturally parallel and event-driven manner, leading to ultra-low active power consumption of 45pJ/spike. The core is fully configurable in terms of neuron parameters, axon types, and synapse states and is thus amenable to a wide range of applications. As an example, we trained a restricted Boltzmann machine offline to perform a visual digit recognition task, and mapped the learned weights to our chip.
Jae-sun Seo, Bernard Brezzo, Yong Liu, Benjamin D. Parker, Steven K. Esser, Robert K. Montoye, Bipin Rajendran, Jose A. Tierno, Leland Chang, Dharmendra S. Modha, and Daniel J. Friedman,
"A 45nm CMOS Neuromorphic Chip with a Scalable Architecture for Learning in Networks of Spiking Neurons,"
IEEE Custom Integrated Circuits Conference, September 2011.
ABSTRACT: Efforts to achieve the long-standing dream of realizing scalable learning algorithms for networks of spiking neurons in silicon have been hampered by (a) the limited scalability of analog neuron circuits; (b) the enormous area overhead of learning circuits, which grows with the number of synapses; and (c) the need to implement all inter-neuron communication via off-chip address-events. In this work, a new architecture is proposed to overcome these challenges by combining innovations in computation, memory, and communication, respectively, to leverage (a) robust digital neuron circuits; (b) novel transposable SRAM arrays that share learning circuits, which grow only with the number of neurons; and (c) crossbar fan-out for efficient on-chip inter-neuron communication. Through tight integration of memory (synapses) and computation (neurons), a highly configurable chip comprising 256 neurons and 64K binary synapses with on-chip learning based on spike-timing dependent plasticity is demonstrated in 45nm SOI-CMOS. Near-threshold, event-driven operation at 0.53V is demonstrated to maximize power efficiency for real-time pattern classification, recognition, and associative memory tasks. Future scalable systems built from the foundation provided by this work will open up possibilities for ubiquitous ultra-dense, ultra-low power brain-like cognitive computers.
As part of IBM's centennial, IBM organized "THINK: A Forum on the Future of Leadership" on September 20-21, 2011 that I attended.
Our long-distance wiring diagram of the macaque monkey brain was merged with IBM’s signature word “THINK” to create a logo for this distinguished event.
Steve Hamm, co-author of Making the World Work Better: The Ideas That Shaped a Century and a Company, interviewed me on the challenge of bringing together a multi-year, multi-disciplinary, multi-institutional collaboration. The video is here.
The Economist published an in-depth and thought-provoking article a week ago:
"MOORE’S LAW—the prediction made in 1965 by Gordon Moore, that the number of transistors on a chip of given size would double every two years—has had a good innings." However, the transistors "have already shrunk to a size where every atom counts. Too few atoms can cause their insulation to break down, or allow current to leak to places it is not supposed to be because of a phenomenon called quantum tunnelling, in which electrons vanish spontaneously and reappear elsewhere. Too many atoms of the wrong sort, though, can be equally bad, interfering with a transistor’s conductivity. Engineers are therefore endeavouring to redesign transistors yet again, so that Dr Moore’s prediction can remain true a little longer."
2006:
Almaden Institute
2007:
“Mouse”-scale simulations
Talk Video: Cognitive Computing Talk at UC Berkeley
Talk Video: Cognitive Computing Talk at Decade of the Mind Symposium
“Rat”-scale simulations
2008:
DARPA SyNAPSE Phase 0
2009:
Talk Video: IEEE 125th Anniversary
“Cat”-scale simulations and ACM Gordon Bell Prize
DARPA SyNAPSE Phase 1
2010:
“Network architecture of the long-distance pathways in the macaque brain”
2011:
Cognitive Computing in Communications of the ACM
Talk Video: Cognitive Computing Keynote at DAC
Chips and DARPA SyNAPSE Phase 2
