The Origins of Mixed Reality

In 1929 Edward Link created the “Link trainer” (patented 1931) probably the first example of a commercial flight simulator, which was entirely electromechanical. It was controlled by motors that linked to the rudder and steering column to modify the pitch and roll. A small motor-driven device mimicked turbulence and disturbances. Such was the need for safer ways to train pilots that the US military bought six of these devices for $3500. In 2015 money this was just shy of $50 000. During World War II over 10,000 “blue box” Link Trainers were used by over 500,000 pilots for initial training and improving their skills.

In the mid 1950s cinematographer Morton Heilig developed the Sensorama (patented 1962) which was an arcade-style theatre cabinet that would stimulate all the senses, not just sight and sound. It featured stereo speakers, a stereoscopic 3D display, fans, smell generators and a vibrating chair. The Sensorama was intended to fully immerse the individual in the film. He also created six short films for his invention all of which he shot, produced and edited himself. The Sensorama films were titled, Motorcycle, Belly Dancer, Dune Buggy, helicopter, A date with Sabina and I’m a coca cola bottle!

Morton Heilig’s next invention was the Telesphere Mask (patented 1960) and was the first example of a head-mounted display (HMD), albeit for the non-interactive film medium without any motion tracking. The headset provided stereoscopic 3D and wide vision with stereo sound. The idea of a mobile, hands-free device that alters the users reality is the basis for most modern-day VR and MR technologies.

In 1969 Myron Kruegere a virtual reality computer artist developed a series of experiences which he termed “artificial reality” in which he developed computer-generated environments that responded to the people in it. The projects named GLOWFLOW, METAPLAY, and PSYCHIC SPACE were progressions in his research which ultimately let to the development of VIDEOPLACE technology. This technology enabled people to communicate with each other in a responsive computer generated environment despite being miles apart.

Julie Martin creates first 'Augmented Reality Theater production', Dancing In Cyberspace, funded by the Australia Council for the Arts, features dancers and acrobats manipulating body–sized virtual object in real time, projected into the same physical space and performance plane. The acrobats appeared immersed within the virtual object and environments. The installation used Silicon Graphics computers and Polhemus sensing system. As people experimented with AR, the foundation for MR was built.

Naval researchers begin working on Battlefield Augmented Reality System(BARS), the robust, original model of early wearable units for soldiers. The complex, 3D battlefields are extremely demanding and introduce many challenges to the dismounted warfighter. These include limited visibility, lack of familiarity with the environment, sniper threats, concealment of enemy forces, ineffective communications, and a general problem of locating and identifying enemy and friendly forces.

Hirokazu Kato created the ARToolKit, an open-source software library that uses video tracking to overlay computer graphics on a video camera. In order to create strong augmented reality, it uses video tracking capabilities that calculate the real camera position and orientation relative to square physical markers or natural feature markers in real time. Once the real camera position is known a virtual camera can be positioned at the same point and 3D computer graphics models drawn exactly overlaid on the real marker. The ARToolKit is still used widely to compliment many augmented reality experiences.

Google announces an open beta test of its Google Glass augmented reality glasses. Google Glass is a type of wearable technology with an optical head-mounted display developed by Google, with the mission of producing a mass-market ubiquitous computer. The glasses reach the Internet through Bluetooth, which connects to the wireless service on a user’s cellphone. The glasses respond when a user speaks, touches the frame or moves the head. Users look forward while wearing the glasses and can see things that they would see on a computer screen, and even ‘search’ that computer by saying a few commands.

In 2016, Microsoft developed and manufactured pair of mixed reality smartglasses called the Microsoft Hololens, drawing on conepts of virtual and augmented reality experiences that had been developed before. The HoloLens is a head-mounted display unit connected to an adjustable, cushioned inner headband, which can tilt HoloLens up and down, as well as forward and backward. The Development Edition shipped on March 30, 2016 and was targeted to developers in the United States and Canada. Samsung and Asus extended an offer to Microsoft to help produce their own mixed-reality products, in collaboration with Microsoft, based around the concept and hardware on HoloLens, and on October 12, 2016, Microsoft announced global expansion of HoloLens and publicized that HoloLens would be available for preorder in Australia, Ireland, France, Germany, New Zealand and the United Kingdom.