Thesholds, by artist Mat Collishaw represents perhaps my biggest virtual/mixed reality project to date. Developed with the late Pete James who sadly passed away earlier this year, Thresholds is a multi-user physical virtual reality experience, where visitors get to explore one of the first exhibitions of photography: William Henry Fox Talbot’s 91 ‘photogenic drawings’ on display at King Edwards’ School in Birmingham, 1839. Visitors experience a lovingly recreated instance of the great hall (CGI work by VMI Studios), which has since been demolished, built with the guidance of architectural and social historians.

What makes thresholds unique however, is that unlike most VR, it doesn’t take place in an empty ‘real space’. Rather,  a physical representation of the room was created with the VR experience overlaid on that physical reality. This is a technique called substitutional reality, but done here to an unparalleled fidelity. Essentially, if you can reach a thing, you can touch it. The system uses leap motion cameras to also give you virtual representations of your hands, allowing you to interact both with the environment, and to manipulate the images on display. Thresholds stretches the boundaries of what we can do right now with consumer VR technology (literally in the case of the scale of the room we track with HTC vive base stations!).

Virtual exhibitions need not simply be recreations of existing museums – here in the virtual space, we can break the social rules, so with thresholds, you can pick up the images to take a closer look – but if you wanted to look at these images in real life – well, in many cases you simply couldn’t, as many of the images are now too light sensitive to be on display.

Thresholds has been on tour around the UK, premiering at Somerset House, before visiting Birmingham Museum and Art Gallery, Lacock Abbey, and The National Science and Media Museum, Bradford. It’s currently on display at Yapi Kredi Kültür Sanat Yayincilik A.S. Istanbul, and will be returning to the UK to go on display at the The Bodleian Library, Oxford later this year. To date more than eight thousand visitors have experienced it, and it has been a resounding critical success, with a number of TV, and national print media articles.

Thresholds is currently shortlisted for the 2018 South Bank Sky Arts Award for Visual Art

The physical space where you experience thresholds.

An etching and early photograph of the room where the exhibition took place

The experience is not just visual – for example the virtual fire is really hot!

Mat has created a viewing portal which allows for extraordinary images like these.


VR Playground

For over a year now, I’ve been working with Joe Marshall on the development of Brendan Walker‘s fabulous VR Playground. The spiritual successor to Oscillate, VR Playground, takes the idea of VR on swings and runs with it (or swings with it). A series of up to eight swings with each rider wearing a samsung gear VR, takes its riders on a journey through one of a series of beautiful abstract environments. In each world, the motion of the swing is re-mapped in a different way. High roller sees the rider zooming through a cityscape, as if trapped in a giant hamster-ball; Jellyfish sees them jetting upwards from the abyss through a coral reef and into the open water, encountering various creatures; shuttlecock sees you springing from building-roof to building-roof among the zeppelins; while walker has yo stomping though another city as if you were a giant robot. Brought to life with Brendan’s unique art style, and a stunning responsive soundtrack by Matthew Olden.

VR Playground has been exhibited at many locations, including contemporary art galleries, shopping centres, national museums, public parks and squares. VR Playground can be presented as visual artwork, performance, a technological marvel, or simply as an amusement ride. A unique visual-kinaestheic experience, part thrill-ride, part art-exhibit, part showmanship, VR playground represents a major outing for me and the Mixed Reality Lab, now on an international tour having been deployed all over the UK, as well as Munich, Seoul and as of this weekend: Philadelphia, it has thrilled some 13,500 riders to date.

VR Playground is commissioned by Horizon Digital Economy ResearchNorfolk & Norwich FestivalGreenwich + Docklands International Festival, and Without Walls. It is supported by Arts Council England and produced by Thrill Laboratory, Horizon Digital Economy Research, and Norfolk & Norwich Festival.

Integrated Immersive Inclusiveness

Working with Jo Robinson from English and Alex Mevel from Translation studies, I’ve recently been involved in a project with Red Earth Theatre called Integrated Immersive Inclusiveness. This is a project which seeks to explore tech interventions for deaf, deafened and hearing-impaired theatre goers. In conjunction with Red Earth theatre, who are already leaders in creating inclusive theatre, we have now run two of our three workshops exploring this area.

In particular we have been looking into captioning, and how to make it less rubbish than it currently tends to be. We’re addressing a few things – the process of actually generating captions, which is currently decidedly painful for producers, projection-mapping to present them in a more dynamic way, and device selection – delivering targeted captions wirelessly to known screens on stage.

This project is an ongoing concern and I will likely be writing more about it in the future, not least because the pictures are so fabulous!

Photocredit: Red Earth Theatre

Sentiment-Lite on Tour

Diane Wiltshire, has taken her work Sentiment, back on tour in a new slightly less bulky format (very sensible in my book!), originally designed to be taken to Jakarta, Indonesia, a trip for which the original installation would have been impractical, Sentiment Lite features the same soundscape as her original experience, but instead of using seven speakers that the visitor walks around, it instead uses noise-cancelling headphones and a gyroscope to allow the user to rotate in place to choose their audio-narrative. Biodata from the interviewees (who’s stories form the soundscape) is still presented to the user in the unique form of a back-caressing waistcoat.

Sentiment Lite was at Camden People’s Theatre this weekend, and Is heading to Liverpool next weekend.

Photocredit: Elly Clarke

Unity agents for ARIA platform

As part of the demonstrators build for ARIA-VALUSPA’s final review to the European Commission, I have been involved in developing the capability of running the ARIA-VALUSPA Platform in Unity3D. This allows you to create and place AVP characters (Virtual Humans) in any game environment, and use appearances and backgrounds from huge existing libraries. Previously ARIA agents lacked front-end usability, meaning that most work would necessarily focus on the agent rather than other aspects of the experience. Using Unity as the front end, offers designers the full scope of that mature experience development platform to create interesting applications for virtual humans.

To demonstrate this, I made a few simple VR and AR examples of the use of ARIA agents. Below are two different AR sample videos.

This is just one tiny part of the extremely impressive ARIA-VALUSPA project, directed by Michel Valstar which I urge to you check out in more detail.

Photo Realiser VR

I’ve been exploring using VR to reveal the context in which photgraphs were taken. Working with Maiken Umbach, Professor of Modern History and Research Director of the Faculty of Arts, we developed a simple proof-of-concept system, that allows users to walk through a photograph and look around, with a view to understanding more about the image. In this case we just simulate a photograph from an existing 3D model. The intention is to build the model from existing photographs, using historical knowledge to provide the context.

The experience, even of this simple proof-of-concept seems to be quite powerful.

Warning: video includes holocaust imagery that some people might find disturbing.

Corrupt Kitchen VR

I’ve been involved in developing a rather fun little game called Paradise Cafe, for a project called Corrupt Kitchen VR. Working with Martin Flintham of the Mixed Reality Lab, Richard Hyde from the school of Law and Jan Mayer-Sahling, Professor of Political Science in the School of Politics, this cross-disciplinary project aims to explore peoples’ attitudes to corruption in the workplace, as well as health and safety in commercial environments. You’d think this would be an opportunity to develop a a serious game, but since I was involved, players spend their time feeding burgers to a never-ending queue of hungry cartoon Elvisis, all the while slavishly helped out by a team of grumpy robots. While the game itself is anything but serious, and owes a debt to the humour of games like Job Simulator, the topic it is exploring and the research it is enabling genuinely are.

Unusually (and through a certain amount of coercion), I have written a separate set of development blog-posts about this one for the university’s digital research team. You can find them here, including a post from Richard about his experience of the collaboration.