What the future of AR looks like

The next two decades hold incredible promise, and danger, as our lives become more intertwined with extended reality (XR) technologies. Visions of exciting possibilities are tempered with privacy concerns, as major XR companies explore centralized visual positioning systems that store personal data.

Today’s pioneers of augmented and virtual realities (AR and VR) will determine the course of that future.

Nils Pihl, CEO of Auki Labs and Matterless Studios, is one such innovator building a decentralized protocol for collaborative spatial computing. His sights are set on a privacy-preserving future that he believes will inevitably be dominated by AR.

Here, we take a deeper dive with Nils on the current state of AR, what lies ahead, and the work he’s doing to influence it.

ADM: Tell us about yourself, how you were first introduced to the world of AR and what Auki Labs does.

Pihl: I’ve been fortunate enough to have my passion and career intersect in the field of behavioral, or memetic, engineering. Studying the behavioral mechanisms of how culture develops and spreads has kept me both mentally stimulated and put food on the table, and it was through the lens of meme theory that I ultimately saw what I believe has been missing from the field of augmented reality.

I remember the era around 2012 when Facebook, social media, and mobile were these incredibly dominant phenomena. The shift in paradigm away from desktop computing brought energy and optimism for the future, and concepts like augmented reality started occupying mind space.

Back then there was one Google search for augmented reality for every five searches for social media, and companies like Magic Leap and Niantic were born. Today, although we are no longer as deep in the conversation about the rise of social media, there is only one Google search for augmented reality for every hundred searches for social media.

Through the lens of memetics, and perhaps the power of hindsight, it is easy to see why:

Augmented reality experiences were not shareable, because devices had no good way of achieving the kind of positioning accuracy necessary for congruent rendering. And so, there was no good memetic vector for augmented reality to propagate into and through our culture, because the behavior was so difficult to observe and replicate.

When we started building augmented reality experiences in 2019, we quickly ran into this wall of poor positioning, and the memetic perspective helped us realize that augmented reality would never take off unless positioning was solved.

But we were not alone to have this realization, of course! Companies like Google, Microsoft, Meta, and Niantic have been hard at work creating new visual positioning systems that allow devices to glean their position in the world by comparing their camera feeds to a digital twin stored in the cloud. These kinds of visual positioning systems are based on a simple but terrifying transaction:

If you tell Google what you’re looking at, they will tell you where you are. To spell out the implication clearly: in a future where digital displays are replaced by augmented reality glasses, companies like Niantic will know what you’re looking at.

To try to thwart this particularly pernicious expression of surveillance capitalism, Auki Labs is building a decentralized peer-to-peer positioning system. It’s as much a labor of love and civic duty as it is an expression of our curiosity and ambition as engineers.

ADM: Can you briefly explain the biggest problem in AR right now and what the future of AR looks like?

Pihl: Augmented reality allows for a much larger canvas for information to be displayed on, and for more natural human-computer interaction. We find it likable, almost inevitable, that augmented reality will replace most digital displays in the next two decades. By the time I have grandchildren, I fully expect having a television in your home to be just as anachronistic as having a fax machine, or even a telegraph.

There is, however, a particularly big blocker to augmented reality gaining practical use and mainstream adoption, and it is the same blocker that hindered its memetic spread:

Digital devices have a very poor understanding of their position in the world, so it is exceedingly difficult to render a digital object in the same physical location on two devices. On the scale of finding directions to a restaurant the existing GPS satellite network does a decent job, because we only need to be brought within several meters of our destination - but shared augmented reality needs precision down to the centimeter, and that is something that satellite technology will never be able to accomplish.

The future of positioning will either be dominated by centralized visual positioning systems, as imagined by Google, Meta, Snap, and Niantic - or by decentralized, interoperable, and privacy-preserving peer-to-peer networks, as championed by Auki Labs and the parts of the Web3 community that see the rise of AR as inevitable.

ADM: When most people think of AR, their mind goes to gaming. But, what are other everyday, practical use cases for shared AR that people should know about?

Pihl: When information can be accurately placed in the world, it opens up about as many use cases as language itself. Ultimately, augmented reality is best understood as an extension of human language - or, as seen by the cyberdelic, language is the oldest form of augmented reality.

Language allows us to manifest our imagination in the minds of others, which in turn allows for communication, cooperation, and organization of labor. Anyone that has ever pointed at a cloud to make someone else see that it looks like a rabbit has used the AR power of language.

Today billions of PostIt notes are sold every year, because of the enormous practicality of putting reminders and communique in their right place, literally. PostIt notes are to augmented reality what lithographs and runes are to the internet. In shared augmented reality, we will be able to manifest our imaginations at a bandwidth currently unimaginable. Without exaggeration, I think it is reasonable to assume that the rise of shared augmented reality will herald a bigger transformation of human culture than the printing press or the internet.

Augmented reality is for gaming in the same way that iron is for spearheads.

ADM: You just launched ConjureKit, the SDK for collaborative spatial computing and shared augmented reality experiences. What are some of the benefits of collaborative spatial computing that ConjureKit unlocks?

Pihl: Collaborative spatial computing allows for shared augmented reality experiences that feel more lifelike, and that have more practical applications. When devices can reason about the world together they can even extend the abilities of what a single device can do in terms of drifting reduction. Everything from better light estimation, to drift reduction, to the battery and compute savings are unlocked by letting the devices collaborate on their understanding of the physical world.

Ultimately, when something is rendered in the same physical location for two people it genuinely feels like a matterless object that is only visible through the magical-looking glass of the device - rather than a dynamic video filter that is only manifested on your screen.

Augmented reality as championed by the likes of Snap and Niantic feels like effects in screen space, but shared augmented reality feels like your imagination conjured into the real world.

ADM: How exactly does the ConjureKit enhance privacy for end users, compared to other actors in the space?

Pihl: When you create a shared augmented reality experience using the tools provided by Meta, Snap, or Niantic you do so by having each participating device upload a wealth of visual data to their clouds. Essentially, you’re uploading your camera feed to these vendors and putting your trust and privacy in their hands. In one breath Niantic will tell you that privacy is important to them, in the next they’ll go on stage and tell the world that they’ve mapped thousands of locations using player data. The business model is based on monetizing your camera feed, and we should reject any euphemism that hides that simple truth.

Our patent-pending instant calibration technique requires absolutely no data about your environment to be shared with us or anyone else. It’s not only privacy-preserving but faster and more accurate to boot. Moving forward, we are developing decentralized protocols for environmental data so that app developers and their users can make more privacy-preserving choices regarding environmental data. It may well be the case that you’ll want a digital twin of your bedroom stored somewhere so that you can create augmented layers on top of it, but Auki Labs should never have a copy of your bedroom. Instead, our protocol will allow end users to self-host this data or choose a data custodian they trust.

ADM: What kinds of applications can developers create with ConjureKit?

Pihl: We are seeing everything from education to defense, from entertainment to health care. Once you embrace that the power of shared augmented reality is to put information in shared space, it truly becomes an extension of language. We’d never ask ourselves what the use cases for language are, and I suspect that just a few years from now no one will ask about the use cases of augmented reality.

We are seeing startups starting to make everything from real-time layers on top of live sporting events, to incredible virtual art installations, to tools for first responders to share life-saving information.

ADM: Who should AR be for? How does ConjureKit make it more accessible?

Pihl: The ConjureKit makes it easy for app developers to have real-time networking and collaborative spatial computing be a part of their product vision. You can put together a simple app in just a few hours, and once you’ve placed your first matterless object in the world I’m sure your imagination will run much faster than I could inspire you. Try it out, build something. You’ll be amazed at the ideas you’ll have.

Today we have an open beta of the SDK for Unity on iOS, but we have many more platforms coming out next year. We think shared and co-located augmented reality is an incredible tool in almost any scenario where people are interacting in real physical environments. 

ADM: How do you envision developers utilizing the ConjureKit across different industries/sectors and will AR play a role in connecting people?

Pihl: On a spiritual level we think that shared augmented reality has a very promising place in human culture. The last decade and the rise of social media have created shocking amounts of alienation, disconnectedness, and isolation. Allowing people to interact with each other in the real world, and taking the digital experience away from screen space into world space, is a path towards greater human connectedness and well-being.

We hope to see more innovative ways to share information in augmented reality, and we’re incredibly motivated by how educators, healthcare providers, and first responders are starting to embrace this medium. Of course, many fantastic entertainment and gaming projects will come too, but the applications that help further human thriving are closest to our hearts.

ADM: What's on the horizon for Auki Labs and Matterless?

Pihl: Matterless, which is a daughter company of Auki Labs, is on a mission to prove to the world that digital things are real. Transformative technology requires transformative narratives, and if we are to avoid the unforeseen but regrettable consequences of the rise of social media, we have to mindfully educate ourselves and others about the medium of shared augmented reality. To that end, Matterless is making toys and virtual companions - because few things are as inspiring as free play and giving affection.

Auki Labs’ mission is to replace the GPS as the default positioning protocol for electronic devices, and we’ll continue to build out our decentralized protocol for collaborative spatial computing because we think it is important. If augmented reality is an inevitable technology, as we believe it is, we must take great care that it does not become a tool of the surveillance capitalist machine. As we continue to expand, we invite anyone in the field of spatial computing to get in touch. It will take great collaboration and human effort to not hand the future over to the likes of Meta and Google.

ADM: Tim Cook, the CEO of Apple, said in a recent interview that AR will become the biggest tech of the future while devaluing the metaverse and VR tech. How do you respond to that?

Pihl: I think Tim Cook is exactly right. Augmented reality is the destiny of human-computer interaction, and one-day digital displays will be as quaint as the radio. Virtual reality will play a big role in the future of humanity, for sure - but it will never replace our need to interact with the real world.

The metaverse is more than a 3D interface on top of the internet. The metaverse is a state of mind, where we inhabit new modes of consciousness, where we accept that digital things are real, and we embrace our memetic identity. Most of the metaverse will be experienced in augmented reality.

It will bring new modes of communication, cooperation, and organization. It will help us work better, express ourselves more clearly, and love more intimately.

If done right, it might help us tackle the great crisis of alienation that we’re facing. The coming of shared augmented reality is an invitation to consciously build all the castles of the imagination, to leave the world more connected.

Nils Pihl

Nils Pihl is a behavioral engineer, inventor, and cyberdelic transhumanist specializing in the intersection of modern technology and human behavior. An avid proponent and popularizer of memetics, Nils uses the perspective of meme theory to unravel and decode human behavior and technological trends.

He is the founder and CEO of Auki Labs, building a decentralized protocol for collaborative spatial computing, on a mission to one day replace the GPS with a peer-to-peer positioning protocol.