In this blog post, we’ll explore why GPGPUs are used in the deep learning training process, focusing on the differences in architecture and computation methods between CPUs and GPUs.
In the past, the match between Google’s Go AI, AlphaGo, and South Korean professional Go player Lee Sedol, a 9-dan, was a major topic of conversation for some time. AlphaGo is an AI system built using deep learning, a technology that is utilized not only in AI but also in various other fields such as speech recognition, recommendation systems, log analysis, natural language processing, and image recognition. Learning through deep learning requires a large number of matrix calculations. To compute matrices quickly, GPGPU (general-purpose computing on GPU)—which involves general-purpose programming using a GPU (graphics processing unit)—is employed. So why use GPGPU? Let’s explore the reasons by comparing it to computations performed by a CPU (central processing unit).
A CPU can perform operations such as integer and floating-point arithmetic, conditional branching, communication with peripheral devices, and reading from and writing to memory; through these operations, it performs general-purpose computations. Therefore, a CPU can handle almost everything we do when using a computer. For example, a wide range of tasks—from retrieving data from websites via a network card and displaying it on the screen to playing games and writing documents—are fundamentally processed using the CPU. Of course, deep learning can also be performed using a CPU. But why do we use a GPU for deep learning training instead of relying solely on the CPU? The reason lies in the speed difference between using only the CPU and using both the CPU and the GPU. GPUs are optimized for computer graphics processing, and compared to CPUs, the types of computations they can handle are limited. For example, they perform a small number of operations, handle limited branching, and have limited memory access. Additionally, unlike CPUs, GPUs differ in that they perform tasks under the CPU’s direction rather than executing calculations independently. Because of these differences, without GPGPU technology, GPUs would be used only for specific purposes, such as computer graphics processing.
Compared to GPU cores, CPU cores can perform a wider range of operations, and as a result, the individual CPU cores are larger. Because CPU cores are larger, a single CPU chip typically contains between 1 and 16 cores, whereas a single GPU chip contains hundreds to thousands of cores. The large number of cores in a GPU allows it to process a greater volume of data simultaneously than a CPU, enabling the GPU to perform calculations at a faster speed than the CPU. Intel’s i7-6700K has four cores and is capable of 2 trillion floating-point operations per second, while Nvidia’s GTX 1070 has 1,920 cores and is capable of 65 trillion floating-point operations per second.
The training data used in deep learning consists of massive multidimensional matrices; for example, 100,000 data points with 100 features each can be represented as a 100×100,000 matrix. During the training process in deep learning, these matrices are multiplied repeatedly. Due to the differences between CPUs and GPUs explained earlier, GPUs can process the multiplication of massive matrices more quickly, which is why GPUs are currently used for deep learning training. In practice, when performing deep learning training, tasks that take several days on a CPU are often completed in just a few hours when using a GPU.
A CPU can perform a variety of tasks on its own but is slow. A GPU can process large amounts of data in parallel compared to a CPU, allowing it to perform certain operations faster than a CPU; however, it must receive instructions from the CPU and cannot replace everything the CPU does. Due to the differences between CPUs and GPUs, the computations required for deep learning can be processed more quickly and efficiently by GPUs than by CPUs. Therefore, using a GPU significantly reduces the time required for training, and GPGPU has become an essential technology for deep learning.
