In this blog post, we’ll explore the principles behind how the human eye recognizes objects and how computer vision mimics this process to recognize and process the movements of objects and people.
Nintendo’s “Wii” game console, which sold over 70 million units worldwide and held a significant market share in the gaming console market, was a huge hit because it allowed users to conveniently control games simply by holding a special motion-sensing remote and moving their bodies. But would you believe there’s a game console that offers an even more convenient way to play than the Wii? “Kinect,” developed by Microsoft, is exactly that. While the Wii allowed users to play games freely simply by holding a motion-sensing remote, Kinect doesn’t even require a remote. In other words, you can play games simply by moving your body, without any separate control devices. When a user dances, the in-game character dances along; when a user runs, the in-game character runs along too. This is because the Kinect sensor, equipped with three cameras, identifies a person’s skeleton and joints from the images to recognize their movements. How is this technology possible?
To understand this, let’s first examine how humans perceive objects. To recognize an object, it’s important to know the distance from the eye to the object. If the distance from the eye to the object is short, the angle between the two lines of sight is large; if the distance is long, the angle between the two lines of sight is small. Humans use this principle to determine the distance to an object. The Kinect recognizes human movements using the same principle: it identifies movements by detecting not only the distance to the person but also the curvature of the body to distinguish the skeleton and joints. This technology is called stereo vision, which is a part of computer vision—the field of study that researches image processing in machines. Through computer vision, machines can see, recognize, and process people as if they had eyes of their own.
Computer vision is very similar to human vision not only in the examples mentioned above but also in the process of recognizing and processing objects. In humans, when an image of an object is focused on the retina through the lens, that image is transmitted to the brain via the optic nerve, where it is stored and processed. Similarly, in computer vision, when an image enters through a camera lens, the light signals detected by the CCD sensor are converted into electrical signals, stored in memory, and then processed by the CPU. In other words, the lens acts like the crystalline lens, the CCD sensor acts like the retina, memory acts like the brain’s storage function, and the CPU acts like the brain’s computational function.
As such, computer vision is very similar to humans in many ways. However, the specific methods used to process images are entirely different. For example, when a person sees a circle, they perceive the shape itself—as “round and not distorted”—and judge it to be a circle. A computer, however, determines that it is a circle by comparing whether the points match the graph of x² + y² = r². In other words, unlike humans, in computer vision, objects must be expressed as mathematical formulas—a format that computers can process.
Through this approach, computer vision can easily handle tasks that are difficult for humans. Consider the case of distinguishing between twins. Generally, people cannot clearly distinguish between twins. This is because when people look at a face, they judge it based on the overall impression rather than specific details. However, since computers process the subtle differences in facial features using mathematical equations, they can easily distinguish between twins.
On the other hand, there are cases where computer vision cannot efficiently handle tasks that humans find easy. When we meet someone, we look at their face and identify them effortlessly within a short time. However, in computer vision, this is not such a simple problem. This is because even the same person can look like a different person depending on their facial expressions and mood. Furthermore, comparing two identical images is not an easy task in computer vision either. This is because the same image can appear different depending on the viewing angle or the surrounding lighting. It is also a difficult challenge to implement a system that treats such variations as noise rather than mistaking them for different images.
Beyond Kinect, the potential applications of computer vision are endless. The parking guidance system at Yeongdeungpo Times Square is one such example. By entering a license plate number on the information board, users can view a photo of their vehicle, its parking location, and the route from their current location to the parking spot. The system reads the license plate with a camera and stores the vehicle’s location. Furthermore, the number of smartphone applications related to computer vision is steadily increasing. When you point your smartphone at the street while walking, the names of buildings or information about nearby stores appear. This is called augmented reality. As the name suggests, augmented reality “augments” the breadth and depth of our understanding and experience of reality by overlaying relevant information onto existing objects and phenomena in the real world. This augmented reality technology is also made possible by recognizing surrounding information through computer vision.
As the old saying goes, “If the body is worth a thousand nyang, the eyes are worth nine hundred nyang,” the eyes play a vital role in human life. Computer vision, the “eyes of machines,” is also playing an increasingly significant role in people’s lives. In a world becoming increasingly modernized through countless machines, computer vision will evolve from being merely the “eyes of machines” to becoming the “eyes of the world.”