Understanding the Competition

In About SyNAPSE I characterized neuromorphic devices as the opposite of conventional Von Neumann processors. This is somewhat of a oversimplification, however. Modern processors are actually quite evolved from pure Von Neumann devices. They are dramatically more capable on virtually every computational workload than their heritage would suggest is possible. For neuromorphic devices to find any success in the marketplace, they’ll need to offer a significant performance gain against existing solutions, but with comparable or lesser power consumption and cost.

Cliff Click did an excellent job covering many of the mechanisms used to increase commodity CPU performance in his recent talk at the 2009 JVM Languages Summit. The first half-hour or so is most relevant:

A Crash Course in Modern Hardware

As Click points out, a modern general-purpose processor includes a number of highly effective mechanisms for mitigating the Von Neumann bottleneck. Chief among these strategies is aggressive caching. Current processor generations have multiple levels of extremely high-speed memory integrated directly on-chip. The bandwidth to the lowest level of this memory is typically at least an order of magnitude higher than main memory. The latency is typically at least an order of magnitude shorter. As the processor requests information from main memory, the cache memory holds a local copy of the most recently-requested data. If the processor re-requests data in the cache, this is called a cache hit. The data is delivered to the processor directly from the high-speed cache memory. If the data isn’t available in the cache, a cache miss occurs. In a cache miss, the processor has to request the necessary data all the way from main memory.

The most important point of Click’s talk was the notion that performance of general-purpose processors is typically dominated by cache misses. Main memory is so much slower than cache that simply fetching data from it consumes the majority of processing time for many workloads. This is where more exotic processor architectures have been able to find market opportunities.

Caching is highly effective for computational workloads that require relatively low amounts of working memory and low parallelism. Graphics rendering is a canonical case of a computational workload where such assumptions are extremely undesirable. The rendering pipeline is highly parallel and requires exceptionally high memory bandwidth. Memory latency, however, is much less of an issue than with more conventional workloads. Hardware graphics accelerators offer a massive performance benefit because they make a different set of design trade-offs more suited to this workload. For graphics rendering and similar computational problems, a graphics accelerator can run up to a hundred times faster than a high-end conventional processor. For poorly-suited workloads, however, the conventional processor could be many times faster than the accelerator.

Graphics accelerators aren’t the only exotic architecture to have found a significant market niche. Digital signal processors can handle many audio and video tasks with far less power and cost than a standard CPU. Another example is the Cell processor, which is designed for certain types of video game and supercomputing workloads.

Neuromorphic computation is far beyond graphics in terms of parallelism and memory demands, as well as far less sensitive to numerical precision. Conventional graphics processors, however, are already a very fast and efficient means for simulating many neuromorphic algorithms. For neuromorphic hardware to carve out a market niche, it will have to offer dramatic benefits for a meaningful subset of computational workloads. This benefit will be measured against the (extremely impressive) computational prowess of modern commodity processors and hardware graphics accelerators, not a Von Neumann straw man.

(Image from Flickr user IsErik)