Technology
Science
Discovery
Cultures & People
Animals & Nature
Life & Self
Pop Culture
Food & Drinks
Travel
Smart Shopping
Home & Outdoors
Ask & Share
Neural Processing Units (NPUs) are specialized chips for handling AI tasks that can be found in smartphones and laptops. But what are they and why are they so important?
Since the dawn of computing, people have compared machines to the human brain. Two of the founders of computer science—John von Neumann, who wrote The Computer and the Brain, and Alan Turing, who in 1949 declared, “On balance, I see no reason why a computer should not be able to compete on an equal footing with human intelligence in most areas”—endorsed the analogy.
However, the traditional processor – the central processing unit (CPU) – is not at all like the brain. The CPU is too “mathematical” and logical. The neural processing unit (NPU), on the contrary, uses a different approach: it imitates the structure of the human brain in its circuits.
McCulloch's pioneering work inspired research in the 1950s and 1960s, but the idea fell out of favor, perhaps because of slow progress compared to the growing computing power of classical computers.
“There were a few individuals in Japan and Germany working on neural networks, but it wasn’t a field,” said Yann LeCun, a French-American scientist considered one of the “fathers” of AI, of his work with Geoffrey Hinton, another pioneer in the field, in the early 1980s.
“The field started to take shape in 1986.” But it took the success of speech recognition in the early 2000s to restore neural networks’ reputation in computer science. Even then, LeCun says, “we avoided the term neural networks because it had a bad reputation, and we switched to the term deep learning.”
These tech giants have invested billions of dollars into chip development, turning past work into processors that fit into our smartphones, laptops, and take inspiration from the structure of the human brain.
This means that instead of solving a problem sequentially, the NPU performs millions or even trillions of mini-calculations simultaneously. This is what is meant by the term "tera operations per second," or TOPS (a measure of computing power).
The difficulty, however, is that NPUs use deep learning instructions that have been pre-trained on huge amounts of data. For example, convolutional neural networks (CNNs, a type of neural network used for image processing) are often used to detect edges in photographs.
In a CNN, a convolutional layer applies a filter (called a “kernel”) to all areas of the image, looking for patterns that it has been trained to believe are likely to be edges. Each mathematical operation performed by the NPU is called a convolution, and it creates a feature map of the image. This process is repeated until the system is confident that edges have been found.
NPUs excel at performing convolutional operations quickly and with low power consumption, especially compared to CPUs. Graphics Processing Units (GPUs), although they do parallel computing, are less optimized for such tasks and therefore less efficient.
However, these NPUs were not very powerful - less than 1 TOPS, compared to 45 TOPS for modern chips like the Qualcomm Snapdragon X found in laptops. It also took several years before applications emerged that could take advantage of the unique structure of these chips.
Today, just 8 years later, AI applications are everywhere. For example, if your smartphone allows you to remove people from photos, it’s probably thanks to the NPU. Google features like “Circle to Search” or “Add Me” use augmented reality (AR) powered by the NPU to add you to the photo after the photo was taken.
NPUs are also appearing in laptops. Last year, Microsoft introduced “a new category of Windows PCs built for AI—Copilot+ PCs.” They required NPUs with at least 40 TOPS of processing power, eliminating AMD and Intel (whose early NPUs only managed 15 TOPS) from the race for leadership.
Qualcomm, whose Snapdragon X processors exceeded that threshold with 45 TOPS, had an advantage. Models with those chips include the Microsoft Surface Laptop and Snapdragon versions of the Acer Swift AI.
AMD and Intel have also released chips that meet Microsoft's requirements, and more laptops bearing the "Copilot+ PC" label have hit the market. However, there's a catch: More affordable laptops often use less powerful processors that don't meet the Copilot+ PC criteria.
It promises “photographic memory,” allowing you to rediscover what you’ve seen in Windows 11. Every photo Recall takes is analyzed by the NPU using context, optical character recognition (OCR), and sentiment analysis to create a searchable index. Recall then takes you back in time through a visual timeline.
Companies like Acer, HP, and Lenovo have also developed local AI tools that analyze documents on your PC, providing summaries and sentiment analysis. These tools are only going to get better.
As power increases, new capabilities will emerge, such as the ability to create realistic AI images locally on your computer without using services like Midjourney.
Over time, as software and hardware evolve and more developers become involved, we will see personal AI agents that understand us, “living” in our computers. They will not only remind us of the past, but will also perform actions on our behalf.
NPUs are likely to find their way into devices beyond smartphones and laptops. TVs will be able to create personalized news broadcasts with your favorite presenter avatar, fitness trackers will recommend workouts based on your mood and schedule.
Since the dawn of computing, people have compared machines to the human brain. Two of the founders of computer science—John von Neumann, who wrote The Computer and the Brain, and Alan Turing, who in 1949 declared, “On balance, I see no reason why a computer should not be able to compete on an equal footing with human intelligence in most areas”—endorsed the analogy.
However, the traditional processor – the central processing unit (CPU) – is not at all like the brain. The CPU is too “mathematical” and logical. The neural processing unit (NPU), on the contrary, uses a different approach: it imitates the structure of the human brain in its circuits.
The Birth of NPU
Electronic "brains" appeared back in the mid-1940s, when neurophysiologist Warren McCulloch and logician Walter Pitts created a "neural network" of electrical circuits.McCulloch's pioneering work inspired research in the 1950s and 1960s, but the idea fell out of favor, perhaps because of slow progress compared to the growing computing power of classical computers.
“There were a few individuals in Japan and Germany working on neural networks, but it wasn’t a field,” said Yann LeCun, a French-American scientist considered one of the “fathers” of AI, of his work with Geoffrey Hinton, another pioneer in the field, in the early 1980s.
“The field started to take shape in 1986.” But it took the success of speech recognition in the early 2000s to restore neural networks’ reputation in computer science. Even then, LeCun says, “we avoided the term neural networks because it had a bad reputation, and we switched to the term deep learning.”
These tech giants have invested billions of dollars into chip development, turning past work into processors that fit into our smartphones, laptops, and take inspiration from the structure of the human brain.
How NPUs Work
Modern NPUs are similar in many ways to those created by McCulloch and Pitts: their structure mimics the brain thanks to a parallel architecture.This means that instead of solving a problem sequentially, the NPU performs millions or even trillions of mini-calculations simultaneously. This is what is meant by the term "tera operations per second," or TOPS (a measure of computing power).
The difficulty, however, is that NPUs use deep learning instructions that have been pre-trained on huge amounts of data. For example, convolutional neural networks (CNNs, a type of neural network used for image processing) are often used to detect edges in photographs.
In a CNN, a convolutional layer applies a filter (called a “kernel”) to all areas of the image, looking for patterns that it has been trained to believe are likely to be edges. Each mathematical operation performed by the NPU is called a convolution, and it creates a feature map of the image. This process is repeated until the system is confident that edges have been found.
NPUs excel at performing convolutional operations quickly and with low power consumption, especially compared to CPUs. Graphics Processing Units (GPUs), although they do parallel computing, are less optimized for such tasks and therefore less efficient.
What are NPUs used for now?
Surprisingly, the first smartphones with NPU appeared back in 2017. Then Huawei introduced the Mate 10 , and Apple introduced the A11 Bionic chip in the iPhone X.However, these NPUs were not very powerful - less than 1 TOPS, compared to 45 TOPS for modern chips like the Qualcomm Snapdragon X found in laptops. It also took several years before applications emerged that could take advantage of the unique structure of these chips.
Today, just 8 years later, AI applications are everywhere. For example, if your smartphone allows you to remove people from photos, it’s probably thanks to the NPU. Google features like “Circle to Search” or “Add Me” use augmented reality (AR) powered by the NPU to add you to the photo after the photo was taken.
NPUs are also appearing in laptops. Last year, Microsoft introduced “a new category of Windows PCs built for AI—Copilot+ PCs.” They required NPUs with at least 40 TOPS of processing power, eliminating AMD and Intel (whose early NPUs only managed 15 TOPS) from the race for leadership.
Qualcomm, whose Snapdragon X processors exceeded that threshold with 45 TOPS, had an advantage. Models with those chips include the Microsoft Surface Laptop and Snapdragon versions of the Acer Swift AI.
AMD and Intel have also released chips that meet Microsoft's requirements, and more laptops bearing the "Copilot+ PC" label have hit the market. However, there's a catch: More affordable laptops often use less powerful processors that don't meet the Copilot+ PC criteria.
What are the most impressive features of Copilot+ PC?
Why pay extra for Copilot+ PC? Microsoft offers a number of exclusive features, and perhaps the most impressive, but also controversial, is Recall.It promises “photographic memory,” allowing you to rediscover what you’ve seen in Windows 11. Every photo Recall takes is analyzed by the NPU using context, optical character recognition (OCR), and sentiment analysis to create a searchable index. Recall then takes you back in time through a visual timeline.
Other features build on previous ideas. Image Creator uses the NPU to create images from text, an improved version of Windows Studio Effects adds creative filters for video calls, and Live Captions translates any video using the NPU.
Companies like Acer, HP, and Lenovo have also developed local AI tools that analyze documents on your PC, providing summaries and sentiment analysis. These tools are only going to get better.
What's next for NPUs?
In the coming years, some AI experts believe that NPUs will evolve like CPUs in their early stages—close to Moore's Law, doubling TOPS every year or two.As power increases, new capabilities will emerge, such as the ability to create realistic AI images locally on your computer without using services like Midjourney.
Over time, as software and hardware evolve and more developers become involved, we will see personal AI agents that understand us, “living” in our computers. They will not only remind us of the past, but will also perform actions on our behalf.
NPUs are likely to find their way into devices beyond smartphones and laptops. TVs will be able to create personalized news broadcasts with your favorite presenter avatar, fitness trackers will recommend workouts based on your mood and schedule.
