Imagine taking a stroll around your local neighbourhood and encountering, not the familiar sights and sounds of an urban space choked in visual information but a new vibrant, virtual layer filled with contextual information and entertainment when you want it. Picture wondrous characters of all shapes and sizes going about their business. More than overlays, they inhabit the physical world, interacting with it and everyone who chooses to engage with them. Who knows, we may eventually come to think of them as fellow companions? This is just one entertaining vision of what an ever-present and evolving in real-time virtual world (or Metaverse) could bring. We at Nexus Studios, as storytellers, are imagining and exploring, having long ago realised the creative opportunities to blend realities and immerse audiences inside stories, by integrating the virtual with the physical spaces around us. We’ve summarised this approach rather neatly into something we refer to as The Three Cs.

Standing by our principles

---

As technology marches inexorably towards ever more impressive experiences, the ability to create virtual content for spaces will become easier, ubiquitous and inevitably more cluttered.

How will we cut through? At Nexus Studios we love all that technology affords us, but underneath the tech magic, we firmly believe in the power of story to bring character and intent to our creations. We believe in the need for context, to provide a legitimate reason for our virtual creations to be there and add value, whether that’s providing a whole new way to perceive and interact with the world around us, or simply providing a moment that makes us smile. How we regard the act of creating is fundamental.

An early AR project taught us a valuable lesson about the power of context. Armed only with AR markers, we literally took a story apart and reconstructed it in the forest! ’The Gruffalo Spotter’ takes the audience on a narrative journey through real woodlands where they can meet the story’s characters and step by step make their way towards an inevitable encounter with the Gruffalo himself!

Designed as walking trails the AR experience encourages families to visit the forest and enjoy nature. The outdoors setting and integral interplay between real and virtual created a fertile space for families to imagine themselves in the story world and play-act together.

This is a great example of the right story in the right place. The audience remains in the story world (the woods) whether actively engaging or walking between virtual moments. It also allows the experience to unfold at the participant’s own pace.

I like to think about virtual interaction as an opportunity to create an encounter. It’s an important distinction, because it will shape how I decide to engage with any characters, approach the copy (as dialogue and narratives) and shape the UX and UI to facilitate playful interaction, e.g, via a contextual prop instead of a button. It’s a precious moment to breathe life into ideas, before becoming bogged down in the necessary, but more functional processes of figuring out how the experience works.

And whilst technology will always set limits on my immersive ambitions e.g. permissions settings, scanning your space … every step, can still be a thoughtful, contextual and purposeful connection.

History brought back to life

---

In creating ‘Changdeok Arirang’ for UNESCO world heritage site Changdeok Palace in Seoul, South Korea, we provided a new and unique kind of AR tour experience not possible before.

As visitors roamed the palace grounds we literally brought history back to life, from interactive encounters with the king and his royal court, all accurately crafted and choreographed as volumetric capture performances and played out in the very buildings they would have once inhabited, to virtual recreations of historical artefacts and 3D scans of publicly inaccessible places, that gave visitors the opportunity to ‘look inside’ whilst protecting the old and fragile buildings. We pushed the best that AR has to offer to present an immersive vision of how the palace might have been over 240 years ago.

Leading this unique experience was the ‘Haechi’ an augmented reality guide and key part of the tour. A virtual beast that roams the grounds and feels deeply connected with the palace and its stories. We balanced traditional myths and legends against 21st century technology to create the ultimate tour guide. With real-time way-finding skills, the Haechi can plot a route from wherever you are in the grounds and walk you to your desired destination. He also provides the most accessible route if you require wheelchair access, And all of this whilst regaling visitors in stories about the Joseon dynasty, as one who was there and lived it. The Haechi embodies what we mean when we talk about the importance of good character.

To create a memorable virtual character we built on his legendary credentials. A mythical beast from Korean mythology and protector of ancient Joseon, he is righteous, possesses the power to discern between good and evil, can grab hold of the past in his mighty jaws and he’s been living at Changdeokgung for over 600 years. Who better to look after the palace and know a thing or two about its history!

Old paintings from the Joseon period inspired his vibrant and colourful appearance and his stone form was influenced by the many statues that inhabit Seoul. To bring him into the AR tour experience, we created a simple backstory. With the Joseon dynasty all but gone, the loyal Haechi fell into a deep slumber, eventually turning to stone. However, as new visitors appear at the palace, his spirit is reawakened and he assumes his old duties, checking over all who enter with an air of duty and self importance. But, behind his stoic sense of duty he is overjoyed to receive company. This in turn inspired his mannerisms. He trots gleefully, almost parading – so proud of the palace and his historic family. His tone of voice is wise and authoritative, but passionate and friendly. Chatting as if to an old friend, he talks about memories not historical trivia.

We went further, crafting interactions at every touch point to deliver contextual and in-character performances. From the moment you pass under the entrance gate, Haechi’s disembodied voice calls you to a halt while he assesses your good intentions. He then invites you to look around for his stone body (scan the environment to locate and generate AR content), As you wander you pass geolocated points of interest that trigger Haechi to make conversation, just as a tour guide shares anecdotes about the places you pass by. We really wanted Haechi to continuously guide visitors to their destinations, but like the Daleks, AR characters can’t climb stairs! (yet) so we invoked Haechi’s magic powers to create a workaround. Whenever he approaches stairs he jumps and disappears into thin air. Once we detect a new plane, he reappears and finishes his jump, landing on the newly detected floor where he continues to merrily trot along.

We completed him with characterful animated touches; he always looks to see if you are following and waits if you’ve stopped. He may even yawn or preen himself if you keep him waiting! And whilst guiding he sometimes takes a moment to sniff the ground, as if really searching for the ghosts of the past (the other AR activations situated throughout the palace).