Snorkelling with sharks: Is water the killer app for virtual reality?

Can virtual reality make dreams come true? Melvin wants to swim with one of the ocean's apex predators - the Great White. Hard to do when you live in the Netherlands. So we decided to make his dream come true - with the help of some pioneering VR technology.

Ever since watching the film Jaws, Melvin Smitter from Wormerveer in the Netherlands had dreamt of swimming with Great White sharks.

In October 2017, Melvin finally achieved his life’s ambition in a swimming pool in Amsterdam thanks to the world’s first underwater 4G connection.

Using a waterproof virtual reality (VR) headset and a 4G receiver mounted on a floating buoy, Melvin and other Dutch shark fanatics were able to enjoy a real-time underwater shark experience, as footage was streamed live from a 360-degree camera located in South Africa’s Shark Alley.

To make this unique experience possible, Vodafone’s creative agency turned to aquatic VR pioneer Stephen Greenwood, CEO and co-founder of San Francisco-based start-up Ballast Technologies.

Getting into the swim

Stephen is not a stereotypical tech entrepreneur. With a background as a documentary producer and then as creative director for Vox Media and Reddit, Stephen was invited to lead the creative development of VR projects at the Discovery Channel.

“They gave me a budget and said ‘go and become an expert in VR’,” says Stephen.

“So I learned the landscape and the trends, and spent a year and a half thinking about different types of applications for VR.”

Around that time, Stephen saw an episode of Netflix’s Stranger Things featuring a character being put into a sensory deprivation tank, wearing a helmet that allowed her to breathe underwater. A classic lightbulb moment, he says.

“There was something about that futuristic helmet and the body of water that triggered this idea in my head. I wanted to try going into a sensory deprivation tank with a VR headset on!”

“The same evening that I watched that show, I was out with my friend Allen Evans, who is a very talented engineer, and I ran this idea by him. He got very excited about it, and the next morning showed up at my apartment with a dive mask and some 3D printed parts,”

“I had a waterproof cell phone, so we were able to start experimenting with how this could be put together.”

Picture credit: Ballast

Swimming lessons

The pair showed the resulting prototype to Jeremy Bailenson, founding director of Stanford University’s Virtual Human Interaction Lab.

Once Jeremy had tested the headset in the pool, recalls Stephen, he“got very, very excited about it, and immediately introduced us to a journalist, who ended up writing an article about it. When the article came out, we had waterparks reaching out to contact us, asking if they could buy this!”

“Prior to this we hadn’t really considered water parks and theme parks. We thought that this could be good for military applications, or dive training – entertainment wasn’t the priority.”

“But we needed to understand all of the safety implications behind it and make sure that there’s lifeguards present. We needed to understand how people react to this when they are in the water.”

“But then we starting thinking that it could be a good way to grow a business, especially where these types of things could be conducted in a safe environment. Maybe this could be a consumer product?”

Picture credit: Ballast

Open water

In March 2018, Ballast unveiled the world’s first VR waterslide at Galaxy Erding, Europe’s largest waterpark. Every rider wears a lightweight, waterproof VR headset, which at its heart contains a Samsung Galaxy S8 smartphone capable of supporting NFC and wireless charging.

Stephen and the team had to overcome a series of technical challenges. To eliminate motion sickness and provide a convincing, high-quality VR experience, Ballast has to accurately measure how fast each rider is travelling, and correlate their physical position on the slide with what they are seeing through the VR display, in real-time.

But no two riders weigh the same. Everyone sits differently on the inflatable tubes. And even the tubes themselves are not uniformly inflated.

“We initially decided to place ultrasound beacons on the slide and measure how fast people were passing each beacon to get a rough calculation of overall speed,” says Stephen.

“As a rider passes, the beacons emit a high-frequency ultrasound tone which can be picked up by the microphone on the handset. But as we started doing this, we figured out another method – using the accelerometer data in the phones.”

By combining the ultrasound beacons with accelerometer signatures “we were able to create a pattern on the individual slide, and just by using the accelerometer we know exactly where someone is on the ride, see where they are, and can predict where they are going to be in the next second.”

“That, in combination with the ultrasound sensors we installed, gives each other a confidence check.”

The high-profile VRSlide at Galaxy Erding has not only generated much-needed buzz for Ballast, but has also proved to be a sound investment for the park owners. More than 130,000 guests have taken the plunge in just over a year.

“We think the owners will see a return on their investment within 18 months,” says Stephen, noting that throughput remains at a constant 100-120 riders per hour.

“Up to half of those experiencing the ride then return for a second go. Ridership figures suggest that VRSlide is running pretty much at capacity every day, irrespective of the park’s attendance.”

Galaxy Erding have already ordered new content for VRSlide – a photo-realistic jungle-themed experience complete with fog and lighting effects.

Diving deeper

Theme parks who have invested in VR are seeing the benefits, according to Stephen. “At Galaxy Erding, most of the people who get on the slide have never tried VR before, which is kind of crazy if you think about it – they are willing to then put on a headset and go down a slide as their first-time VR experience,” he says.

Ballast’s next launch is DIVR – virtual reality snorkelling. Described as a “full-body sensory experience”’, DIVR gives up to 10 people at a time a simulated zero-gravity journey through coral reefs, caves and shipwrecks.

Available as a relatively inexpensive off-the-shelf kit, Ballast are pitching the system at waterparks, hotels and resorts, encouraging them to repurpose existing pools. DIVR promises to deliver something truly unique: an oceanic VR experience, swimming alongside turtles, rays and humpback whales.

“We have developed a system where people wear a small floatation belt around their waist, connected to a weight at the bottom of the pool with an elastic band between the two, so they are able to swim in a 10-feet circular radius without hitting the walls or bumping into each other,” he says.

“Even if you are not swimming, with the VR headset on you still feel like you are moving. This is a snorkelling experience initially, so people are right at the surface of the water.”

Stephen is more cautious about underwater aquatic VR.

“We do have plans for an experience that fully submerges people, but I know this is going to take a lot of work – it is only going to be right for people who have some sort of experience or training as divers and know what they are doing,” he says.

Future fishing

During summer 2018, Stephen spent some time experimenting with a fully-submerged VR setup. This was, as he admits, “a whole different story. . . it made me realise how complicated using all the scuba equipment is and what a difference it makes being two or three metres under the water.”

He’s optimistic about the potential for future wireless technologies, including 5G, to define and improve the VR experience.

“Phones with depth-sensing capabilities are already shipping,” says Stephen. With high-bandwidth mobile connectivity, augmented and shared reality becomes real.

“We’ll be able to share the environments we’re in, recreate them digitally and have other people experience them in a way as if they are really there.”

“This situational awareness – having multiple people connected to the same virtual world at the same time – is going to be huge,” he predicts.

“But we will need to experiment to figure out just how much data we are able to transfer. It won’t just be a one-dimensional video stream: we’ll be able to send full 3D, depth-sensed environments, and share those in real time with each other.”

Stephen has a bold message for innovators – be willing share your idea with the world, even before you think it is commercially ready.

“If we had tried to isolate our idea and kept it close to our chests, it maybe would have never taken off,” he acknowledges.

“But because we took a risk, showing a prototype that really wasn’t ready to be shown, it captured the imagination of a few key people that were able to help us get it out to the world – and it allowed us to go on this journey. I want this to give encouragement to other people that also want to chase their dreams.”