VR, Augmented Reality and Mixed Reality


Photo credit: www.mortonheilig.com/InventorVR.html
Photo credit: http://www.mortonheilig.com/InventorVR.html

Noted virtual art practitioner and researcher Malcolm Le Grice describes the development of Virtual Reality out of a long-term impulse for increased reality in art throughout human history. He notes that through the ages, art has moved from flat symbols through variations of more illusionistic shapes, textures, colours and dimensionality. With the advent of photography and cinema, this impulse saw a dramatic increase with the development of colour, stereoscopy, movement and audio within the photographic arts. He also notes the cinematic attempts at complete immersion though widescreen and panoramic formats, SurroundSound, 3D visuals and “smell-o-vision” (228-229). Although all of these technologies served to further the goals of illusion, Le Grice demonstrates that it wasn’t until the development of real-time and interactive capabilities that art was able to come close to a Virtual Reality. In establishing an illusion that comes close to our reality, Virtual technologies are able to “suppress the contradiction between the representation and the represented” long enough to, if only briefly, allow us to fully immerse in a virtual environment (235).

Several authors, including Oliver Grau, have also noted the origins of Virtual Reality technology in military technology and industrial application. Flight and battle simulations, prototype development, data analysis and research visualizations were all early adopters of virtual reality technology.

Photo credit: http://en.wikipedia.org/wiki/Flight_simulator
Photo credit: http://en.wikipedia.org/wiki/Flight_simulator

In addition, the website of the Mixed and Augmented Reality Arts Research Organization (MARART) highlights the original terminology of Virtual Reality within a computer science concept of a mixed reality continuum, developed by Paul Milgram and Fumio Kishino. Although MARART disagrees with the linearity established by Migram and Kishino (preferring the model of a quantum network over a singular continuum), the original model still offers insight into the ways in which we underrstand virtual and augmented reality today. In this theorum, Milgram and Kishino illustrate a scale between VIRTUAL and REAL in order to develop an understanding of immersion that happens outside of the fully immersive experience stereotype of VR.

Photo credit: http://en.wikipedia.org/wiki/Mixed_reality
Photo credit: http://en.wikipedia.org/wiki/Mixed_reality

The spectrum between fully virtual and fully real is termed as “mixed reality” and within that grey-scale, immersion can lean towards Augmented Virtuality (where a physical object is incorporated into a virtual world ie. Streaming video of a person into a virtuality, or scanned 3D digitization) or Augmented Reality (the more common application, where reality is augmented by virtual elements ie. Information text overlay, simulated environments with physical interface etc). Milgram and Kishino describe a variety of possible interfaces for this activity, including monitors, head mounted displays, glasses, graphic displays with video reality and graphic immersive environments.

Although there is a variety of artistic practices which incorporate mixed reality, in projects ranging from virtual objects displayed on mobile devices in real-time-space environments, fully immersive computer-generated environments which isolate audience from reality with head mounted visual and audio devices, to computer-interface virtual environments like Second Life, the most significant link between many of the works is their interest in the blurred boundaries between science and technology. As new interfaces and devices are generated, artists appropriate them into the works to push the limits of their work even further. Intriguing new technological possibilities are currently available in the video-gaming industries, which have long engaged interactivity, and like other visual arts mediums, has worked to increase realism and immersion over time. In particular, two gaming technologies stand out as intriguing possibilities for artists. The first, a Google-sponsored interactive game entitled Ingress, engages participants in a mixed reality battle of “capture the flag”.

Photo credit: http://pedant.artoflivin.sk/2012/11/ingress-its-time-to-move-google/
Photo credit: http://pedant.artoflivin.sk/2012/11/ingress-its-time-to-move-google/

The participants use an app on their mobile devices to discover “portals” located in physical spaces (primarily places of public art, cultural monuments and architectural landmarks) around the world. Participants are collected into two teams, and are able to virtually and physically meet with other players to establish ownership over the portals. Similarly to Second Life, participants are both engaging with the gameplay, and taking ownership of it, on occasion establishing virtual monuments at the physical portals in memory of tragic occurrences, like a permanent truce and portal established in honour of Sean Collier, the MIT officer shot after the 2013 Boston Marathon bombing. Although it lacks the flexibility and inherent modifiability of Second Life, Ingress demonstrates how users are able to hack the game itself for artistic purposes.

Finally, several major gaming companies have recently helped to fund Oculus Rift, an independent 3D gaming hardware development.

Photo credit: http://www.businessinsider.com/oculus-rift-video-demo-2013-7
Photo credit: http://www.businessinsider.com/oculus-rift-video-demo-2013-7

Currently in development stage, the VR head-mounted display is intended to be an affordable consumer product for immersive gaming. Although it has not been released for general use at this time, artist Geoffrey Lillemon has already appropriated the technology to design “The Nailpolish Inferno” (2013), a surreal VR art exhibition designed for the Oculus Rift interface. This immediate adoption highlights the intriguing possibilities for the technology once it is available and affordable, especially considering the open-source development process the company is using to develop games and applications for the hardware. Google-glass is another example of a consumer product that offers possibilities for interaction and manipulation.

Although I haven’t dealt with it extensively, it is important to note the role that Massive Online Multiplayer Games (MMO)’s have played in Virtual Reality art. Although not immersive in the bodily sense, as many of these other examples have been, online experiences like Second Life offer a world of unlimited possibility, manipulation and play that many artists have taken advantage of. Second Life even offers a listing of a variety of art events, performances, installations and modifications that users have created as fully-functioning art works. This movement was shown a strong display of support by the Australian Centre of Virtual Art, which funded a Second Life Artist in Residency for several years. Chris Dodds was awarded the inaugural Residency for his work Babelswarm (2008).

Key Features
– Technology is a key component
– Physical interface and hardware.
– Often utilizes narrative, instantaneity (real-time), and interactivity
– Hybrid practices
– Creates a blurring between the boundaries of virtual and real
– Often engages histories of computer science, video gaming, the rituals and practices of internet, networking and virtual socialization.
– Engages issues of distanciation and mediation
– The role of the audience shifts from spectator to participant

Realms of Influence
VR and Augmented reality practices are influenced by the development of new technology within the realms of military, commercial industry, computer science, and more recently video gaming. Although initially interested in the ability to create immersion, interactivity and illusions of virtual space, augmented and virtual practices are now developing sophisticated thematics around spectatorship, surveillance, advertising, sociality, identity and more. More-so than most other visual arts mediums, Mixed Reality practices are interested in the intersections between art, science, technology and anthropology, which is reflected by the interests that many of the significant VR and AR organizations include in their mandates and diverse collaborations.

Babelswarm - Adam Nash, Christopher Dodds and Justin Clemens. Photo credit: http://www.businessinsider.com/oculus-rift-video-demo-2013-7
Babelswarm – Adam Nash, Christopher Dodds and Justin Clemens. Photo credit: http://irez.me/2010/09/28/story-enfield/

In addition, as these practices often require the development of physical technological structures as part of the virtual-real interface, many of the works are influenced by architecture and industrial design. Noted researcher and practitioner, George LeGrady, notes that his art attempts to extend “…the work’s meaning into its surrounding space…” through image projection, wall partitions, text, printed images, audio and other spatial designs that are extraneous to the artwork itself (226). Although LeGrady’s quote refers to the physical space around the virtual work, many completely immersive works (such as Charlotte Davie’s Osmose, 1995) design the architectural/ environmental elements into the virtual space, creating a fleshed out world for virtual exploration by the viewer through visuals, audio and movement. Combining the two realities, the Belgian collective CREW utilizes headphones and goggles to immerse the participants into a virtual world, while simultaneously engaging actors to interact with the participants. By blurring the lines between virtual and real, the collective draws attention to our negotiation of reality (Nordhjem).

Significant Practitioners
Jeffrey Shaw – http://www.jeffrey-shaw.net/
Lynn Hershman-Leeson – http://www.lynnhershman.com/
Monika Fleishmann & Wolfgang Strauss – http://fleischmann-strauss.de/
Ulrike Gabriel – http://v2.nl/archive/people/ulrike-gabriel
Edmond Couchot – http://www.banffcentre.ca/faculty/faculty-member/2855/edmond-couchot/
Geoffrey Lillemon – http://www.oculart.com‎
Dirk van Oosterbosch and Barbara Vos – http://www.kabk.nl/arlab/newsitem.php?newsid=0165&cat=01
CREW – http://www.crewonline.org
RE+PUBLIC: B.C. “Heavy” Biermann, Jordan Seiler & Ean Mering. – http://www.republiclab.com/about
Charlotte Davies – http://www.immersence.com/
Hiroo Iwata – http://www.iamas.ac.jp/interaction/i01/works/E/hiroo.html
Stephanie Rothenberg – http://www.pan-o-matic.com/
Sander Veenhof – http://www.sndrv.nl
Garrett Lynch – http://garrettlynch.tumblr.com/
Gleman Jun – http://glemanjun.blogspot.ca/
Second Front – http://www.secondfront.org/

Significant Researchers
Oliver Grau
Malcolm LeGrice
Jeffrey Shaw
Peter Weibel
Katherine Hayles
Roy Ascott
Bill Seaman
George Legrady
Dieter Daniels
Paul Sermon

Important Exhibitions and Projects
Ars Electronica Festival. Ars Electonica Centre. Annual festival of art, technology and society. Berlin, 1979-present. <http://www.aec.at/news&gt;.
Davies, Charlotte. “Osmose.” Musée d’art contemporain de Montreal, 1995. Artwork. <http://www.immersence.com/osmose&gt;.
Dodds, Christopher, and Justin Clemens. “Babelswarm.” Second Life, 2008. Artwork. <http://www.babelswarm.com&gt;.
Laurel, Brenda and Rachel Strickland. “Placeholder.” The Banff Centre for the Performing Arts, 1992. Artwork. <http://www.tauzero.com/Brenda_Laurel/Placeholder/Placeholder.html&gt;.
Lilleman, Geoffrey and Random Studio. “The Nail Polish Inferno.” Oculus Rift Virtual Reality Art Show, 2013. Artwork. <http://geoffreylillemon.com&gt;.
Veenhof, Sander and Mark Skwarek. “WeARinMoMA.” Site-specific intervention. Museum of Modern Art New York. New York, 9 October 2010. Exhibition. <http://www.sndrv.nl/&gt;.
Yuxweluptun, Lawrence Paul. Inherent Rights, Vision Rights. The Banff Centre for the Performing Arts, 1992. Artwork. <http://digitalarts.lmc.gatech.edu/unesco/vr/artists/vr_a_lyuxweluptun.html&gt;.

Important Texts
Fernie, Kate and Julian D. Richards. AHDS Guides to Good Practice: Creating and Using Virtual Reality: a Guide for the Arts and Humanities. Arts and Humanities Data Service, 2002. Web. 24 September 2013. <http://www.vads.ac.uk/guides/vr_guide&gt;.
Grau, Oliver. Virtual Art: From Illusion to Immersion. Cambridge: MIT P., 2003. Web. 24 September 2013. <http://issuu.com/piker77/docs/virtual_art_from_illusion_to_immers&gt;
Moser, Mary Anne, and Douglas MacLeod eds. Immersed in Technology: Art and Virtual Environments. Cambridge: MIT P., 1996. Print.
Morse, Margaret. Virtualities: Television, Media Art, and Cyberculture (Theories of Contemporary Culture). Bloomington: Indiana U. P., 1998.
Pollock, Griselda, ed. Digital and Other Virtualities: Renegotiating the Image. London: I. B. Tauris, 2010. Print.
Popper, Frank. From technological to virtual art. Cambridge: MIT P., 2007. Print.
Reiser, Martin and Andrea Zapp eds. New Screen Media: Cinema/Art/Narrative. London: BFI, 2002. Print.

Important Centres/ Organizations
ARlab; The Hague, Netherlands. <http://www.kabk.nl/arlab/activities.php?id=1&gt;.
ART+COM; Berlin, Germany. <http://www.artcom.de&gt;
Art Project: Google Cultural Institute; Online. <http://www.google.com/culturalinstitute/project/art- project>.
Australian Centre of Virtual Art; Melbourne, Australia. <http://www.acva.net.au/&gt;.
Banff Centre for the Arts; Banff, Canada. <http://www.banffcentre.ca/&gt;.
Carnegie Mellon University SIMLAB; Pittsburgh, USA. Defunct predecessor to Center/ Studio for Creative Inquiry.
Center for Advanced Visual Studies; Cambridge, USA. <http://cavs.mit.edu/&gt;.
Center (STUDIO) for Creative Inquiry; Pittsburgh, USA. <http://studioforcreativeinquiry.org&gt;.
Eyebeam; New York, USA. <http://eyebeam.org/&gt;.
Kunsthochschule für Grafik un Buchkunst; Leipzig, Germany. <http://www.hgb-leipzig.de/&gt;
Mixed Augmented Reality Research Organization; Online. <http://www.marart.org/&gt;.
OMI International Arts Center – Augmented Reality: Peeling Layers of Space Out of Thin Air; Ghent, USA. <http://artomi.org/page.php?Augmented-Reality-Peeling-Layers-of-Space-Out-of-Thin- Air-25>.
Second Life; Online. <http://secondlife.com/destinations/arts&gt;.
SOFTlab; New York, USA. <http://softlabnyc.com&gt;.
University of Geneva’s MIRALAB; Geneva, Switzerland. <http://www.miralab.ch&gt;.
York University Augmented Reality Lab; Toronto, Canada. <http://futurecinema.ca/arlab/&gt;.
Zentrum für Kunst und Medientechnologie Karlsruhe; Karlsruhe, Germany. <http://www.zkm.de&gt;.

Works Cited
Dodds, Christopher, and Justin Clemens. “Babelswarm.” Second Life, 2008. Artwork. <http://www.babelswarm.com&gt;.
“Ingress (game).” Wikipedia the Free Encyclopedia. Wikimedia Foundation, 24 September 2013. Web. 24 September 2013. <http://en.wikipedia.org/wiki/Ingress_(game)&gt;.
Legrady, George. “Intersecting the Virtual and the Real: Space in Interactive Media Installations.” Martin Resier and Andrea Zapp eds. New Screen Media: Cinema/Art/Narrative. London: BFI, 2002. 221-2226. Print.
Le Grice, Malcolm. “Virtual Reality – Tautological Oxymoron.” Martin Resier and Andrea Zapp eds. New Screen Media: Cinema/Art/Narrative. London: BFI, 2002. 227-236. Print.
Nordjem, Barbara. “Penetrating Deeper into the Temporal Lobe.” AR[t] Magazine #2. ARlab, November 2012. Web. 24 September, 2013. <http://www.kabk.nl/arlab/pageEN.php?id=0129&gt;.
Milgram, Paul and Fumio Kishino. “A Taxonomy of Mixed Reality Visual Displays.” IEICE Transactions of Information Systems. Vol. E77-D, 12 December 1994. Web. 24 September 2013. <http://etclab.mie.utoronto.ca/people/paul_dir/IEICE94/ieice.html&gt;.
Mixed Augmented Reality Art Research Organization. “About.” MARart.org. Mixed Augmented Reality Art Research Organization, 2013. Web. 24 September 2013. <http://marart.org/wordpress/?page_id=2&gt;.
“Oculus Rift.” Wikipedia the Free Encyclopedia. Wikimedia Foundation, 22 September 2013. Web. 24 September 2013. <http://en.wikipedia.org/wiki/Oculus_Rift&gt;.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s