Mitsubishi Electric Develops Worlds First UI and App to Render Spoken Words Instantly as 3D Text in Live Video Recordings

Mitsubishi Electric announced today its newly developed SwipeTalk Air user interface (UI), which is believed to be the world’s first UI to use augmented reality (AR) technology to render spoken words instantly as three-dimensional text during live video recordings. Text positioning is implemented by simply swiping a finger over the tablet or smartphone screen. Mitsubishi Electric also announced its development of SwipeTalk Air app on iOS that integrates the UI, video recording and other functions for extra-expressive videos that are expected to enliven social media.

The SwipeTalk Air UI immediately renders spoken words as 3D text in a video as the cameraperson swipes a finger across the device screen. The device is pointed at the subject, a few words are spoken, a line is swiped on the screen and then the text instantly appears in the swiped area. Video recording and editing are seamlessly integrated, so the experience of capturing the moment live is never interrupted and users can enjoy inputting text intuitively.
The SwipeTalk Air UI will contribute to accomplish a new video expression by placing letters in anywhere of virtual midair in the screen. As the arranged text will look fixed in place, even when changing the camera angle, taking videos that combine such text and recorded objects enables new ways of expressing oneself and new uses based on users’ ideas. Due to swift text positioning, users can easily produce videos full of creativity.
Lines can be drawn in the video by tracing them on screen to supplement text with illustrations, such as arrows, pattern or cartoon-like decoration, to enhance the effect and clarity of the message. When the subject wears an AR marker (camera-recognizable images) on the subject’s finger, it will make text follow the path of the subject’s moving finger. The positioned text can be translated into a foreign language instantly. When the displayed text is changed to another language, it is read out loud in that language to enable communication and collaboration with people who speak other languages.A prototype of SwipeTalk Air will be displayed at the SXSW (South by Southwest) digital technology exhibition in Austin, Texas from March 8 to 17.

READ  Mitsubishi Electric Develops World's First Voice-activated Drawing Function that Displays Spoken Words Where Finger is traced on Screen