Markets such as gaming and virtual reality have made use of gesture recognition for some time. CyberGlove Systems, for example, was founded back in 1990 and has been selling data gloves to companies, government agencies, and universities for many years. Consumers flocked to the Nintendo Wii when it came out in 2006, and then later embraced the Xbox 360, imperfect as it was, for hands-free gesture control. In the last few years controllers and gaming platforms have continued to improve.
These developments sold a lot of sensors, cameras, microcontrollers, and other components/subsystems to support gestural recognition and control software, but there wasn’t enough money in this business for many firms to see it as an opportunity. However, this situation is beginning to change and gestural recognition is emerging as a commercial possibility for a wide range of applications.
As a result, NanoMarkets believes that, for the first time, this is now creating significant revenue generation potential for components and subsystems suppliers in the gestural recognition space. We are encouraged in this belief by the following trends and possibilities.
The end of touch? Gesture recognition could, over the next decade, replace touch sensing as the leading edge computer input technology. Gesture recognition is emerging in 3D form and is likely to eventually replace the 2D gesturing seen in multifunctional touch screens. We suspect that this is one of the ways in which consumers will first encounter gestural recognition in a fully realized form.
Gestures moving beyond the nerd market. Gesture recognition is moving into applications beyond VR and gaming enabling the opportunity for manufacturers of sensors, cameras, etc. to grow. Gamers represent an enthusiastic but limited market. The real opportunities for components and subsystems makers will appear once gestural recognition moves beyond toys and specialized military markets
Optics comes first. We are already seeing gesture recognition creep into consumer electronics, especially in smart phones. Early versions have not been as impressive as they could be, however. NanoMarkets therefore sees a real opportunity for component suppliers to introduce and sell novel gesture-oriented devices. Most of the early successes are going to be found in primarily optical systems—those based on cameras and image sensors. But there are already gestural technologies that supposedly can provide high-performance gestural recognition and control using sub-systems other than optical ones
Gesture recognition entering new markets. NanoMarkets believes we will soon be seeing touchless gesturing in a much greater range of applications, from improving factory automation to providing a safer way for drivers to control the vast array of electronic devices in their cars. The “home of the future,” may very well contain multiple built-in devices activated by gesture. Touchless gesturing is especially compelling for wearable devices like smart watches, for which producing a usable touch screen in such a small form factor is a huge challenge.
We think that these trends will (1) pump up the volume of components required for the gesture recognition space and (2) require new kinds of components and subsystems with high value-added and protectable IP. Both (1) and (2) will add to the profit potential of firms active in the gesture recognition space and can be sure to drag more firms into it.
At present much of the activity that constitutes these trends is just R&D stage, but in some cases there are real next-generation gestural control products ready to sell either now or within the next year. NanoMarkets believes that there are three industry sectors that are going to make money as a result of the rise of gestural recognition:
- The sensors industry, especially those sectors that can specifically design products for gesture recognition
- Optical subsystems and components makers, especially companies that offers low-cost/lightweight cameras and appropriate lenses
- The semiconductor industry, both that part making discrete components and that making ICs
In order for consumers in any market segment to embrace gesturing it has to be easy to use, seamless, reliable, and offer significant advantages over existing user interfaces. Despite multiple attempts, the industry is not yet there. NanoMarkets believes that a lack of component integration and infrastructure is part of this problem. Systems also aren’t as intuitive as promised, leaving users in some cases disappointed and running back to familiar touch screens
Gestural Recognition and the Camera Industry
Optical systems are still the most common solution for gesture recognition, and likely to stay that way for some time to come, even though other types of gestural recognition and control systems are coming onto the market:
Gesture recognition is here now. Controllers that use multiple cameras have been around for several years. For example, Microsoft’s Kinect includes a color video camera and a depth sensor camera to detect 3D motion. The Leap Motion controller contains two infrared cameras with embedded image sensors.
Cameras are critical to the current generation of gesture recognition. There is typically something in the system to “see” the gesture. The roadmap here is very similar to the sensor space, and after all, cameras can be thought of as a special form of sensor.
Most existing gesture recognition systems boast software-only solutions that make use of hardware that is already integrated into devices. One obvious example is optical detection using the camera installed in a smart phone or laptop. Such systems do demonstrate gesture recognition ability, but they have several significant limitations.
Second generation cameras and system integration: The first wave of optical gesture recognition systems make use of standard 2D cameras, but systems will increasingly incorporate more advanced cameras for full 3D rendering. The most effective systems will combine cameras and multiple sensors for increased accuracy and a wider library of recognized gestures.
NanoMarkets believes that these more advanced trends are not going to develop into large money-making opportunities for several years. But the next steps which are beginning to happen will accelerate in coming years:
Multiple cameras in tablets and laptops. The integration of multiple cameras into devices such as tablets and laptops for seamless gesture recognition without the need for a separate controller could push up the volume of cameras.
Cameras for professional gesture recognition. However, consumer electronics will not be where we will see the highest performing cameras. Expensive solutions like time-of-flight cameras are probably not very compelling here. Lightning-fast response speeds and detailed accuracy are more important in industries where safety is paramount, including automotive, healthcare, and industrial. This is where more complex cameras and sensors may be successful.
Beyond optical systems. Optical systems are still the most common solution for gesture recognition, but other (non-optical) gesture recognition technologies promise (1) better accuracy, (2) lower power consumption and/or (3) the ability to work equally well under various lighting conditions; something that has plagued existing optical systems. Some of the most promising non-optical approaches detect disturbances in sound waves or electrical fields and use those data to interpret gestures.
The extent to which non-optical technologies take hold makes a great deal of difference to suppliers of components and components since the hardware requirements differ greatly from one technology to another. For example, ultrasonic systems rely on speakers and microphones, not on cameras. Sensor requirements vary depending on the type of gesture recognition technology.
Several companies are betting their futures on acceptance of approaches other than optical gesture recognition. To hear them tell it, their approach is superior to optical technology, but the market might not necessarily agree. And with companies like Intel and Google promoting camera-based systems, it might be hard for smaller firms with other approaches to make themselves heard.
Sensor Makers Responding to Gestural Recognition
Sensor makers and OEMs are just beginning to address enhanced gestural recognition, but are doing so primarily with off-the-shelf sensors. In fact, many sensor makers may be barely aware that their devices are being used in gestural systems.
While this off-the-shelf approach may be a good place to start, it is obviously not conducive to high margins. NanoMarkets believes that sensor makers will quickly adopt gestural recognition as a key opportunity within the context of emerging wearable and Internet-of-Things (IoT) businesses, especially once important OEMs focus publicly on gestural recognition. We think that sensor makers will also receive a push in the “right” direction from certain software makers.
Moving beyond the existing sensors: As we have already noted, the current generations of gestural recognition often make use of the existing sensors and other hardware that are currently out there. This is implicitly inadequate and some of the opportunities that seem to present themselves with regard to sensors include the following:
New photodetectors. A new class of photodetectors for gestural recognition may be required. Gesture recognition doesn’t always work well using existing sensors when the ambient lighting is too bright or too dim.
Improved power consumption. Better power consumption for sensors. While this is an opportunity that extends well beyond the needs of gestural recognition, gestural recognition can be especially power hungry and can drain the battery in mobile batteries
Additional functionality for sensors. A broader range of sensors and ones with more sensitivity will be required for advanced gestural recognition systems. The library of allowed gestures is limited, and detection accuracy leaves much to be desired.
At the present time, what we are seeing is the sensor makers taking their first tentative steps to addressing the needs of gestural control and related markets. These firms understand that the volumes of sensors sold in these markets could be huge, but they aren’t yet sufficiently convinced to aggressively go after these markets. It may take a few years before this happens. Sensors do add cost, of course, and require OEMs to change their manufacturing flow to incorporate them.
Other Opportunities
As the gestural recognition business emerges and becomes mainstream NanoMarkets believes that it will be increasingly seen as an opportunity for industry sectors beyond sensors and cameras.
Software and sensors. Gestural recognition software companies are already recognizing that that deficiencies in hardware are posing limitations on what their software can do.
Software relies on sensors that can accurately detect motion and distinguish between purposeful and accidental gestures, and we think that, as a result, software companies will partner with sensor suppliers to ensure that the sensors makers will deliver what’s needed.
Implications for the semiconductor industry: Semiconductor firms stand to benefit from the growth in gesture recognition as they come out with products that are specifically designed for such applications. Solutions need to fill the need for components that are small, light, and can be easily integrated into devices. For example there is definitely a market for components that combine microcontrollers with sensors in a single package.
Sensor fusion represents another opportunity for both the sensor makers and the semiconductor industry. With the proliferation of devices that have multiple sensors of different types built in comes the challenge of how to handle the data sensors are collecting and how to get the various sensors to communicate with each other. Microcontrollers, microprocessors, or the sensors themselves can accomplish this fusing of the incoming data.
Companies that can provide sensor fusion solutions that are seamless and balance the tradeoffs between cost and power consumption in gestural recognition systems have. We believe that this is an area that encourages the development of business ecosystems where sensor firms participate in tight alliances with software and semiconductor firms.
