How New Technology Could Let You Type in VR

Typing while wearing a virtual-reality headset could get easier thanks to a new invention that analyzes the vibrations in your bones. 

Using a keyboard in VR is tricky because you really can’t see where your fingers are landing. The new TapID gadget is a wristband that carries two motion sensors, with one worn on each wrist, according to researchers who recently published a paper on their invention. The device detects taps from each of the user’s fingers individually.

“Right now, most VR users use a handheld controller device that acts kind of like a laser pointer,” Scott Camball, a telecommunications and teleconferencing enthusiast, said in an email interview. “They need to point at each letter on the virtual keyboard, and it can take a very long time to type things out. This works for video games, but not for work-related tasks. TapID would allow users to have a virtual workspace with large screens without the bulk of a desktop computer.” 

Let Your Fingers Do the Talking

TapID works by analyzing the vibrations through your skeletal system using a machine-learning algorithm. In VR environments, users could type into a virtual keyboard, or interact with virtual objects on a surface, using their fingers. 

“We propose moving all direct interaction in VR to passive surfaces,” the authors wrote in their paper. “Compared to mid-air interaction, touch interaction on surfaces provides users with an opportunity to rest their arms between interactions while simultaneously offering physical support during prolonged interactions.”

Other Ways to Type in VR

Virtual reality companies are hoping to make users more productive by offering alternatives to the current pointer systems. Facebook Reality Labs is developing technology to use cameras on the Oculus headset to read your finger movements when you type on a virtual keyboard, Camball pointed out.

“The advantage of this is no bracelet is needed, but the Facebook Labs hand tracking does not currently work well enough for mainstream adoption,” he added. “Currently, the ergonomics of typing in VR is a major concern, as common input methods with a virtual keyboard are tiring due to the need to hold a controller or hands in the camera’s view.”

Manufacturers also are working on a ring-shaped gadget that virtual reality users can put on their finger, Ivan Pleshkov, a developer at ScienceSoft, a software development company, said in an email interview. “It is even more convenient to wear and more intuitive to use—the user can draw characters in the air and does not depend on the camera location,” he added. 

Another input method under development is a virtual keyboard that uses eye-tracking. “Thanks to the sensor that catches and processes eyes’ location, a user can type characters by resting their gaze on them to type them in,” Pleshkov said. “It promises to be a less tiring experience, although typing speed is under question—the method requires some accommodation to increase it.”

Using a real keyboard in VR is one option, although it’s unwieldy at the moment. The app Immersed for the Oculus Quest is a shared virtual workspace that offers a way to calibrate a physical keyboard so that you can type while in VR. In a demonstration posted on Reddit, Immersed founder Renji Bijoy demonstrated typing at 164 words per minute using the feature.

Some developers are hoping that voice control could be better than any keyboard. 

Clumsy hand controllers often put off new VR users, Ottomatias Peura, head of growth at voice control software company Speechly, said in an email interview. “This is especially problematic in certain VR environments that can be used only once or twice, such as environments for real estate or live music,” he added. 

Practice can improve how users control standard controllers, but manipulating VR through speech is even better, Peura said. 

“Voice is a great option for improving user experience in VR, too, because it's the most natural interaction modality for us humans,” he added. “While it can be hard to click B with the left-hand controller with your VR glasses on, it's very intuitive to say 'open door.'"