GPUs, TPUs, NPUs, and ASICs: Where Flexibility Becomes a Weakness

Roderick Gregory

For a long time, the central processing unit (CPU) was the true “brain” of all computers, and for good reason. It could do practically anything you threw at it. Any clever logic, any complex algorithm, any task—though not lightning fast, it could handle it. This “omnivorousness” made it king across powerful servers, home PCs, and even sophisticated embedded systems. But here’s the paradox: today, this very versatility, once its strength, is increasingly becoming its weakness.

Universality as a historical advantage – and as a limit

Modern tasks—machine learning, image processing, video streaming, cryptography, modeling—are characterized not by complex branching logic, but by huge volumes of similar operations. Matrix multiplication, parallel computing, signal processing—all of these tasks are poorly suited to an architecture designed for general-purpose instruction execution. The CPU is forced to spend a significant portion of its resources on control, branch prediction, caching, compatibility support, and abstractions that are simply unnecessary for these tasks.

The result is a paradox: the processor is becoming increasingly complex, with ever-increasing transistor count, yet the performance gains for specific tasks are minimal. Most of the silicon is devoted not to the useful result, but to maintaining generality. This is where the demand for specialized architectures arises, which sacrifice “do-it-all” capabilities for maximum efficiency within a single task class.

GPU: The First Massive Breakthrough in Versatility

Graphics processing units (GPUs) have become a shining example of how trading generality for specialized performance can yield significant gains. Originally designed for graphics processing, GPUs have proven to be exceptionally well-suited for parallel computing thanks to their architecture: multiple simple processing cores, minimal control logic, and high memory bandwidth. These characteristics are ideal for machine learning and scientific computing.

It’s important to understand that GPUs don’t outperform central processing units (CPUs) in every task. Their advantage manifests itself in areas where computation can be effectively parallelized. In scenarios with complex logic, unpredictable data access, or small data sets, CPUs can be more efficient. However, for mainstream tasks such as training and deploying neural networks and scientific simulations, the performance gains of GPUs are so significant that the generality of CPUs becomes secondary.

However, even GPUs have their limitations. They remain relatively general-purpose within their specialization, as their architecture must support a variety of computations, APIs, languages, and use cases. This inevitably leads to some loss of efficiency. Therefore, the next step in the development of computing technologies is even deeper specialization.

TPUs, NPUs, and ASICs: Efficiency at the Cost of Flexibility

TPUs, NPUs, and ASICs are a logical continuation of the rejection of generality. These chips are designed for specific operations, models, or algorithms. In Google’s TPUs, a significant portion of the logic is dedicated to tensor multiplication operations. In mobile NPUs, it’s dedicated to accelerating neural network inference with minimal power consumption. In ASICs, it’s dedicated to a strictly defined computational process that tolerates no deviations.

The result is a dramatic leap in efficiency. These chips consume less power, occupy less space, generate less heat, and deliver predictable performance. In data centers, this means lower operating costs. In mobile devices, it means increased battery life. In industrial applications, it means stability and scalability.

But the price for this is a loss of flexibility. ASICs cannot be “retrained” for a new task without physically changing the design. TPUs are optimized for specific model architectures and can lose efficiency when changing approaches. NPUs often depend on specific frameworks and compiler tools. Any paradigm shift requires either complex emulation or the release of a new generation of chips.

This is a key shift: while software previously dictated hardware requirements, hardware now increasingly dictates software constraints. Developers are forced to adapt to the architecture, accounting for data formats, tensor sizes, and memory requirements. Flexibility is shifting from runtime to design time.

Why flexibility becomes a weakness

It’s not that general-purpose processors (CPUs) are bad, but that they’re too expensive for most of the tasks we solve today. Modern computing has long since ceased to be chaotic: it’s predictable, repeatable, and we know what to expect from it. When a computer does the same thing for years—for example, running AI models, processing video, or analyzing data—its ability to do “anything” simply wastes energy and silicon and adds complexity to the system.

A general-purpose processor must always be ready to execute any code, even if it never actually does. This means more transistors, complex hardware, and a ton of caches and mechanisms to keep everything running. A specialized chip, on the other hand, strips away all unnecessary details and benefits precisely from its “rigidity.”

  • Power Consumption. General-purpose architectures waste energy supporting logic that isn’t involved in the target computation. At the scale of data centers or mobile devices, this translates into direct financial and thermal losses.
  • Computational Density. A CPU can’t pack computational units as densely as an ASIC or TPU because part of the die is always occupied by control logic. This limits performance per watt and per square millimeter.
  • Performance Predictability. Specialized chips deliver stable and predictable results. General-purpose processors are more dependent on the execution context, system load, and code specifics.
  • Scalability. As the system grows, general-purpose design no longer pays off. What’s convenient on a single server becomes ineffective on a thousand nodes, where every percent overhead is multiplied.
  • Development Economics. In conditions where key algorithms remain unchanged for years, it’s more cost-effective to invest in specialized hardware than to constantly compensate for the inefficiencies of general-purpose execution.

You know, it often turns out that the claimed flexibility of systems is a myth. In reality, most of them handle only a specific set of tasks. Take, for example, an inference server – it doesn’t need to be able to run anything. Or a surveillance camera – why would it need to be a full-fledged computer? Even smartphones use dedicated NPUs for neural networks, not regular CPUs. It turns out that versatility often remains just a nice idea that isn’t put into practice in real life.

But this doesn’t mean CPUs will soon disappear. General-purpose processors are still very important – they coordinate work, manage processes, and execute complex logic. They’re just increasingly becoming part of more complex, hybrid systems, where specialized chips take over the heavy lifting. The future of computing isn’t a single processor for everything, but rather a system where each element does exactly what it was designed for.