Doing Business in the Metaverse: Leveraging Innovations in Immersive Technology

Doing Business in the Metaverse: Leveraging Innovations in Immersive Technology.

The metaverse is bringing businesses an expanding range of new efficiencies across their value chains, from how they visualize market opportunities to how they design and build factories and offices, and engage with customers and other stakeholders. The metaverse is a great leveler in that off-the-shelf technologies and devices make it accessible to businesses of any size. Business use cases within the metaverse are rapidly expanding. The $200 billion global gaming industry empowers its enthusiasts to save the world by egg-bombing enemies or create their own islands. Restaurant chains allow patrons to cook virtual meals or explore their locations on virtual reality (VR) headsets. In health care, the metaverse helps provide physicians composite views of a patient’s data on their headsets, or train them for delicate communications with the families of patients, such as parents of terminally ill children. Avatars help retail customers try out apparel or come close to testing a fragrance the way they would at an actual store. In more workaday and complex settings, companies drawing on the Industrial Metaverse can build digital twins of factories. This enables simulation of a range of real-life scenarios from product planning and sustainable raw material sourcing to accident prevention by anticipating maintenance issues. Yet another safety innovation is in training drivers with real-life simulations of road and weather conditions, or detecting distracted driving by tracking eye movements. And, in order to prepare emerging talent for that new world, leading universities, including Wharton, are creating business courses around the metaverse. In this white paper, faculty experts at Wharton and elsewhere, and industry practitioners, share insights into how businesses benefit from innovatively tapping into metaverse applications, even as they confront concerns and ethical issues that new technologies typically bring.

Gaining Industrial Foresight with Digital Twins Imagine operating a business where you get advance warning of every hurdle that lies ahead, and the ability to remove those hurdles before you get to them. That is just one aspect of the myriad benefits of the Industrial Metaverse, encompassing use cases across plants, machinery and supply chains. “The Industrial Metaverse is a virtual world, which is a representation of the real world,” said Annika Hauptvogel, head of technology & innovation management at Siemens based out of Munich in Germany. Hauptvogel leads the strategy department of Siemens Technology, the main driver of the company’s core technologies, and the Siemens Innovation Ecosystem. The Industrial Metaverse provides participants immersive experiences and simulations that are photorealistic, physics-based and in real time, she added. An example of such high-end simulation is the digital twin, which is at the heart of the Industrial Metaverse, according to Hauptvogel. Although the term digital twin was first coined in 2002, it has achieved maturity now, according to a Siemens blog post. Ian McGann, director of Siemens Innovation Office, defines a digital twin as “a precise virtual representation of a physical object, including its mechanical, electrical, and configuration management. This one digital twin evolves across its lifecycle with numerous models used to capture different aspects of the object’s physical behavior.” The digital twin has now evolved into an “executable digital twin,” or “xDT” that allows real-time simulations of say, a wind turbine,” to predict its performance and pre-empt maintenance issues, said McGann in an interview for the blog post. The digital twin can access a wide array of technology tools. Hauptvogel listed those tools: Artificial intelligence to help make predictions; machine learning with the data captured from the devices or machines it replicates;

edge technology or the ability to process data closer to where it is generated; advanced telecommunications networks like 5G or 6G; cloud platforms that enable collaboration; blockchain technology to provide a safe and secure environment for B2B applications; and AR and VR technologies to enable immersive experiences. In 2021, when Siemens wanted to build a new greenfield factory in Nanjing, China to overcome constraints it faced at two older plants making drives, motors and controllers, it first built a digital twin. The twin had all the features Siemens wanted in its real world factory, including expanded production capacity and sustainability features like energy savings and environmental protection. It proceeded to build the new factory only after it was successfully optimized in the digital world, eliminating the likelihood of surprises or snafus that could occur after commissioning. Siemens engineers check out a visual factory. Source: Siemens

“When you can make a digital twin of a factory or of a process, everybody involved can study it and try things out before actually commissioning it,” said Wharton management professor John Paul MacDuffie. He cited automaker BMW’s partnership with NVIDIA to build digital twins of a future factory, adding that it could potentially extend to cover all of BMW’s existing and future plants. He noted that while digital twins have been around for a while, the metaverse offers enhancements. “Anytime the physical engagement of more of the senses is valuable, there's a very clear business function for it.” MacDuffie is also director of Wharton’s Program on Vehicle and Mobility Innovation and one of the faculty experts designing Wharton’s planned metaverse course. Using Engagement to Fix Retailing Pain Points

That definition of the metaverse as it relates to the use of the senses comes clearly into play when the retailing industry inhabits the metaverse. One of the biggest benefits for retailing is that the metaverse boosts engagement with customers. “Any time that you can have a consumer engage more and over a longer period of time with your brand, you’re going to build loyalty,” said Wharton marketing professor Barbara E. Kahn, whose specialties include variety seeking, brand loyalty, retail assortment issues and patient decision-making. An important metric in retailing is the time that people spend in engagement with brands. “More engagement with the brand has been shown to correlate with loyalty and subsequent conversion,” she explained. “The metaverse as a 24/7, immersive, experiential and social environment.” Those features enable the metaverse to serve as “a virtual platform where you can look around, visit stores, play games, or attend a concert, she added. Building Trust and Transparency The metaverse is linked with blockchain, which adds new layers of trust, transparency, security and traceability that facilitate commerce in the virtual world. For instance, blockchain enables payments in the metaverse for transactions involving virtual assets. “If someone creates something in the metaverse, say an artistic endeavor, and if it’s associated with blockchain, you can trace their history give them unique credit for their work, and they can earn revenues off the lifetime of the piece,” said Kahn, who was formerly director of the Jay H. Baker Retailing Center at Wharton. Those creations could be paid for through NFTs, or nonfungible tokens, which are digital assets stored on the blockchain, representing proof of ownership and authenticity; they cannot be edited, copied or duplicated.

Exploring New Environments with Avatars The metaverse offers space for avatars; these are virtual characters that people can create to represent themselves. They can customize their avatars by skin type, gender, size or other features such as dress. People deploy them in virtual environments, where they can perform a variety of roles within the metaverse environment like explore a space, view a video, play a game; and the avatars can befriend others. “It becomes your second self,” Kahn noted. Avatars offer opportunities in retailing that were hitherto unimaginable. They can browse online stores, push different buttons to get deeper into a website, see product videos and even make purchases. Tapping the metaverse will of course call for bigger marketing budgets, and so the early birds may be brands and retailers with means and staying power. “You are seeing luxury [brands] put their toe in the water a lot, because what a luxury [brand] is building is a lifetime relationship with an elite group of customers,” said Kahn. A lot of these luxury brands are partly experiential, and the metaverse can help develop that relationship.” Building such a superstructure of consumer appeal in retail via the metaverse has its limitations, such as in the fragrance business. The metaverse does not lend itself to the sense of smell when buying a perfume, but it can conjure up the ambience. “You can have a video where you can do as much as you can to translate the scent into some kind of visual cue,” Kahn said. “If there are celebrity endorsements, you can engage with the lifestyle it’s associated with. You can do everything short of the smell to try to create what the fragrance stands for.”

Boosting Health Care, with Data and Empathy The operation theatre is where the metaverse is making a discreet appearance: Augmented Reality (AR) headsets allow surgeons performing sensitive procedures to access patient information without having to look at a separate screen. Magic Leap, a technology company based in Plantation, Florida, makes such AR devices that superimpose 3D imaging over physical objects, MacDuffie said. “Surgeons can just look at the patient the whole time, but also see [clinical data] in an appropriate part of their field of vision.” The enabling spectrum of the metaverse takes other forms, too. End-of-life conversations are now an inalienable part of medicine. Being able to tap AR and its immersive aspects has helped harried physicians navigate discussions about mortality with empathy. MacDuffie pointed to Strivr, a technology startup in Palo Alto, Calif., whose AR platform offers training applications across enterprise scenarios. But the one Strivr calls “The Art of Empathy” relates to an immersive learning program it developed for the Lucile Packard Children’s Hospital of Stanford to train physicians how to have better conversations with the parents of terminally ill children. Those are conversations which every physician dreads, as Strivr noted in a case example on its website. Strivr’s AR platform combined approaches, honed by neuroscience research, to design training sessions for the hospital’s physicians where they donned the headsets and repeatedly engaged in those conversations until they overcame their unease.

“Depending on what you say, parents of terminally ill will children react emotionally. “You could get anger, you could get tears, you could get denial. And you could be left with questions about whether what you said was too much of soft pedaling, or too blunt,” MacDuffie said. Finding ways to empathetically deliver sensitive information is where the AR training platform helped, he explained. “You actually have a heightened emotional and visceral experience while you're practicing – and it makes you more ready to do it later.”

Using Simulation to Make Driving Safer The metaverse is finding uses also in other settings where it helps overcome human frailties, such as decreased attention spans while driving. Helen Loeb, a robotics engineer and research scientist at the Children’s Hospital of Philadelphia, has created a product that allows people of all ages to test their driving skills through simulation. Jitsik’s software and suite of sensors are available off the shelf to complement a Meta Virtual Reality Headset. Jitsik’s Helen Loeb is targeting the market for driver training with her VR-driven immersive product.

First, users strap on sensors on the brake pedal, throttle and the steering in their own car. The car is stationary at all times. By wearing the headset, they get “immersed” in a simulated driving experience, complete with critical situations they may encounter in real-life such as a kid darting across the street or another driver making a wrong turn. With the AR and mixed reality features in the headset, drivers can see their car, their hands and feet, and the passenger sitting next to them. The software can offer a variety of road conditions such as highways or mountain roads. Jitsik, the Philadelphia, Pa.-based firm that Loeb owns and heads as CEO, is building prototypes of the product. It will target the three million young drivers who hit the roads each year. Her addressable market will leverage auto insurance companies eager to help drivers get safer on the road. Vehicle manufacturers will also be interested in the technology. Jitsik’s patent-pending product is in the prototyping stage now. Loeb said she expects the product will be attractive to driving students often deterred by the high cost of driving education, typically $100 an hour. At a $400 price tag, the Meta Quest 2, will provide unlimited hours of practice for young people.. Loeb anticipates her metadrive technology will transform how people learn to drive by making it safer in ways hitherto impossible. Features like eye control and eye tracking record where people are looking while they drive. “If somebody is not good enough as a driver, we are going to be able to measure that in a safe way,” Loeb said. “Also, driving education is a joke right now; you cannot expose people to dangerous scenarios, and there is no accountability in tracking how many hours of driving a teen student has done.” As one would expect, Jitsik’s product tracks and stores a large number of metrics.

“Driving education is out of reach for a lot of young people right now. It is expensive and driving schools are located in wealthy neighborhoods.. [What] we want to do is bring this education to where the young people are. Technology allows a lot of things to happen. We just integrated Google Maps 3D in our product. It will soon bd possible for people to virtually drive in their own neighborhood. As more people acquire VR headsets, so many things become possible. Smoothening the Way for Autonomous Driving Loeb has her sights also on the autonomous, self-driving market; since 2015, she has won nearly $2 million in research grants from leading automakers including General Motors, Ford, Toyota and Honda, and insurers like State Farm. The Pennsylvania chapter of the Tesla Owners Club is looking to use Jitsik’s driver training product for its members, while Audi and Toyota have shown interest in Jitsik’s self-driving product. MacDuffie visualized big opportunities for the autonomous driving space to benefit from the metaverse. He described the rough framework of how that might play out: Data captured from real-world autonomous driving would be fed into a mix also containing LiDAR information (Light Detection and Ranging), a remote-sensing technology that enhances the accuracy of self-driving cars, and other data from regular radar and cameras. All of that would be integrated in a process called “sensor fusion,” which combines data from these different sources. That integrated data would be processed by on a chip made by a company like NVIDIA, whose Omniverse platform is already driving many metaverse applications. That loaded chip would then deploy the “fused” data to power a digital twin of

the vehicle, which in turn would develop simulations to track driving conditions and other factors influencing autonomous driving. “What the metaverse can add is to give these companies a way to simulate a passenger’s actual experiences of riding in their autonomous vehicles, allowing ongoing adjustments to eliminate or significantly reduce the prospect of mishaps or discomfort,” MacDuffie said. Reimagining the Future of Work Well before metaverse entered the vocabulary of work environments, the Covid pandemic had prepared businesses and talent teams for virtual meetings. The enhancements the metaverse promises around the workplace are potentially transformative, although it is still an evolving space. A Zoom call is an advancement over a regular phone call in that it “gives us a bit of body language, but it's still two-dimensional,” MacDuffie said. “But if we were meeting in a metaverse room, we would have more of a sense of actually being in each other's physical presence. We would be able to see facial expressions much more fully and with more nuance. We would hear people spatially existing in a space that would conjure up being in a room together. The basic premises of the metaverse is that when more of our senses are engaged, our experience becomes closer to real life.” That higher level of engagement can lead to better communication, with emotions being more fully conveyed and more accurately perceived, MacDuffie added. “People can pick up on each other's excitement if they're having a great brainstorming or collaborating on a project.” Already companies like Meta and Microsoft are adding metaverse functionalities to their meeting rooms.

All communications technologies have an inherent limitation in that they cannot replace the power of face-to-face meetings. “We evolved over millennia to be extremely good at and dependent on reading social cues from face-to-face proximity,” MacDuffie said. “That is why face-to-face is still what we imagine to be the gold standard for really effective communication, and creativity and collaboration.” Another metaverse application in work-related communications is the ability for people to use avatars, MacDuffie continued. People could use one of several avatars for an informal meeting, and another for formal settings, he explained. “It potentially frees you from the constraints of just our one body or one personality, or things that people know of us.” To be sure, there are concerns over interactions with avatars, especially when the people behind them are unidentified. MacDuffie pointed to examples of people being harassed in the metaverse. “All of the bad behaviors that people can do face-to-face, they can certainly do in the metaverse, as they can do on social media and everywhere else. Maybe the greater immersiveness and the greater engagement of the senses can make some of that worse or cause a more negative impact.” Easing Payments with Digital Wallets and Tokenization Digital wallets are an important way the metaverse will play out in business settings, according to David Treat, senior managing director at Accenture, who leads its metaverse business, including its activities in blockchain and extended reality. Digital wallets will change marketing, branding, onboarding customers and managing those relationships, how we shop, collaborate, and make products, he remarked at a recent Mack Institute for Innovation Management conference on the metaverse.

Essentially, digital wallets are tokenized versions of multiple aspects of an individual, such as his or her government identity, employment and credit scores. “Every player in the global capital markets is working on tokenizing securities and how securities are traded in core capital markets,” Treat said. Treat saw digital wallets as critical for adoption by the financial services industry because of the efficiencies they will bring in the anti-money laundering and KYC (Know Your Customer) domains. The telecommunications industry could use digital wallets to look at ways to determine the uniqueness of each phone number. And the retailing industry could use metaverse features for ‘emotional tracking' of customers, and for eye tracking — capturing data as a shopper scans a product shelf, with readings on pupil dilation or changes in heart rate, Treat noted. Pitfalls around the metaverse For all the benefits the Metaverse promises, the technologies it spans have their share of shortcomings and risks of abuse. From a technological standpoint, the most commonly voiced concerns are over interoperability of devices and software that have different vintages and are on multiple platforms. Another common worry is about latency in communications systems, which for instance may affect how quickly an image refreshes, and variations in data upload and download speeds. Users of the metaverse must address those and other technology-related concerns to build robust business cases around the metaverse. “Most of the action is in the software and the hardware software integration is absolutely critical,” said MacDuffie. “Any of the experience that the software is meant to give you can be thrown off if you find the hardware is annoying, or if it interferes with immersion because it hurts your head or because it's too heavy or because you feel claustrophobic or whatever.”

“There are a lot of infrastructural advances that will be necessary for the visions of the metaverse that we hear most often to become a reality, and those are advancing on their own trajectories,” MacDuffie said at the Mack Institute conference. Those include advances in 5G and 6G telecommunications capabilities, widespread availability of broadband, video processing chips and capacities, he added. as well. “These things are not the metaverse, but how quickly they advance will have a lot to do with how quickly we can experience some of what the metaverse is promising us.” Experts have also pointed to the risks and dangers around avatars, which enable users to pretend to be another persona. Avatars can be great ways to play online games or do virtual shopping, but the anonymity they provide could potentially be misused to harass people online. “All of the bad behaviors that people can do face-to-face, they can certainly do in the metaverse, as they can do on social media and everywhere else,” said MacDuffie. “Maybe the greater immersiveness and the greater engagement of the senses can make some of that even worse or have more of a negative impact.” “There's so much capacity for mischief and bad stuff to happen with deep fakes and the like,” said MacDuffie. Those concerns could prompt regulators and other participants to put up “guardrails around the application of those agents,” he added. “It is possible today to create technology that will validate who you are,” Prasad Joshi, senior vice president of Infosys said at the Mack Institute conference on the metaverse. “That is still not trust; that is security. And the society has to make transition from security to trust. I don't think technology can ever make it.”

Metaverse in the Product Lifecycle Annika Hauptvogel, Head of Technology & Innovation Management at Siemens traced the applications of the metaverse across the lifecycle of a product: The Design Phase: In this phase, companies could experiment with different designs to visualize in the virtual world how customers might use a product. They could use the information they gain to refine their designs. The first advantage of virtual design is that it reduces the use of resources, compared to all that would be required to build a prototype and test it in the real world. Building prototypes in virtual environments is physics-based, which is why those products would work exactly the way they would in the real world. Secondly, the virtual design phase takes much less time to execute, resulting in a shorter time-to-market. Third, the metaverse facilitates collaboration; virtual environments can be designed to bring all those involved together, including users, design experts and members from maintenance teams. The design phase also incorporates inventory management of the bill of materials for raw materials and components or sub-assemblies. It offers a chance to do sustainability checks on materials or processes such as their environment-friendliness or cost optimization; those goals can now be factored in from the start.

The Construction Phase: The Industrial Metaverse will help increase the efficiency of production sites. Prior to commissioning, different scenarios can be tried out virtually. For example, if the production line has to be adapted or a new machine brought in, production can continue

uninterrupted while the commissioning is done virtually in the Industrial Metaverse. The new machine can be inducted into the production line after it has proved its utility in a virtual environment. This saves time and increases productivity. The Operations Phase: The Industrial Metaverse allows virtual replications of real-life operations in a building or production line. Virtual tracking of operations allows companies to proactively identify root causes of errors or malfunctions, and either fix them beforehand or be ready with remedies. Remote availability of industrial facilities also eliminates the costs involved in sending experts to physical locations where malfunctions occur. Another rich use case for the Industrial Metaverse is in training. Teams can be trained in how best to manage a production facility or any other system before it is built in the real world. This is preparedness well before the plant is commissioned. Such training is not limited to new facilities. For example, a photolithography machine costs about $200 million. In the industrial metaverse, companies could train people at any time on such machinery, without having to invest anything or take any risk with their facilities. Digital Twins continue tracking a product even after it is made and sold, all the way through to optimizing it for recycling or environment-friendly disposal. At Siemens, the Industrial Metaverse is still evolving, although some of its clients are using parts of it.

Watchwords for Tapping into the Metaverse Make it Relevant: “Companies will want to have their own environments, their own attributes, their own business models … [and not look to] ”join some universal virtual space,” Wharton professor of legal studies and business ethics Kevin Werbach said at the Mack Institute Conference on the Metaverse. Werbach is also an expert on the metaverse, and his research interests include blockchain and digital assets. According to Werbach, it is important for organizations to understand which dimensions of the metaverse are relevant to their business and customer environment. He spelled out the choices they must make to benefit from the metaverse: “There's a thread of the metaverse that is entirely about games – entirely about people having fun. There are pieces of the metaverse that are entirely about community. There are pieces that are entirely about work and business activity and shared spaces for business communication. Understand which dimensions of that are relevant to you and then understand always when dealing with emerging technologies, what's your value proposition?” From Physical to Virtual: Kahn offered some tips on how retailers are preparing themselves for the metaverse. As a first step, some companies are trying to translate their physical experience into a virtual one, she said. She pointed to the fast-casual food chain Chipotle, which invites its customers to put together a burrito virtually, and then Chipotle would connect that virtual creation with the real product and deliver it in real time. Fashion brand Dolce Gabbana, for instance, got its metaverse feet wet at the Decentraland Fashion Week, March 2022, where it had virtual catwalks, using cats as the models.

Leverage Innovation Labs: Businesses could set up facilities that are entrusted with the task of preparing their employees for new technologies such as those the metaverse encompasses. MacDuffie noted that many organizations already have created some kind of “innovation lab,” where they dedicate some people to learning new technologies, who could then give walkthroughs or demonstrations to their colleagues. Businesses could also invest in “local guides,” who could demonstrate features of the metaverse with pilots and training programs, he added. Build a Local Metaverse: Stanford communications professor Jeremy Bailenson built a local metaverse in the university’s Virtual Human Interaction Lab where he was founding director. In a YouTube interview he shared for this white paper, he discussed the psychology of virtual reality (VR) and how the medium will transform the way companies and schools "teach, train, operate and build culture.” According to Bailenson, training in VR bridges the gap between 2D training systems and one-on-one tutors, and offers the best of both worlds. He cited a case of how Strivr (where he is a co-founder) used VR to train 1.5 million Walmart associates across 4,700 stores to use a new device – a pickup tower– through 17 VR training stations. While the original training session was eight hours long, the VR version allowed it to be condensed into 15 minutes without any loss in the level of expertise Walmart associates acquired. Also they didn’t have to travel offsite for their training; remote learning through the VR program helped them save almost an entire workday. The training covered everything from how a deli store manager could handle multiple customers at once to ensuring that corn cobs are not stacked too high that they block ventilation or flout a code, Bailenson wrote in his book on virtual reality, “Experience on Demand.”

Walmart was so impressed with the Strivr program that it is building a library in which its written manual will come with “a set of experiences to train on,” Bailenson noted. “The beauty from their perspective is that VR is magnitudes of order cheaper than setting up a physical training store that is stocked with food and customers. But price aside, it is also more consistent – every trainee gets exactly the same experience, on demand.” Create Awareness Around a “System of Systems”: Almost any technology has “a hidden set of systems and subsystems behind it,” MacDuffie pointed out, citing Stanford technology management professor Steve Barley. “We tend to look at the most visible and most dramatic manifestation of a technology, looking for the killer app, and then we want to pin all consequences on that one breakthrough, whether it's opening up new markets or job loss and de-skilling,” he said. In much the same way, the metaverse is not one technology but “a system of systems” – a confluence of various technologies with different vintages and levels of performance, he added. “Sometimes the progress of what the most visible application can do is heavily constrained by the subsystems. And sometimes, when something miraculously appears as a new technological marvel, it's only because these various subsystems have advanced to the point that suddenly it’s possible.” Look out for Integration Issues: Some of the stumbling blocks could be difficulties in critical aspects like ensuring interoperability across different technologies, and integrating software with hardware. “Most of the action is in the software, and the hardware-software integration is absolutely critical,” MacDuffie said. That is because the experience that the software promises can be frustrated by shortcomings in the hardware or annoying features such as a headset that is too heavy or bulky, he explained. Other irritants could be in slow Wi-Fi speeds and high latency, i.e. time lags, even if tiny, that lead users to perceive unnatural delays in responsiveness of the virtual worlds they are in. Writer: Shankar Parameshwaran | Design: Quantent

About the Mack Institute for Innovation Management Emerging technologies and innovations can create and transform industries, while simultaneously introducing new risks and uncertainty to established organizations. The Mack Institute for Innovation Management is an exclusive network that connects business leaders, researchers, world-class Wharton faculty, and students. At the institute, thought leaders from across academic disciplines and industries come together to explore how companies survive, compete, and thrive through innovation management. The institute’s multidisciplinary faculty and researchers develop practical approaches to managing innovation and share this knowledge through thought-provoking conferences, workshops, and publications. Mack Institute for Innovation Management 3620 Locust Walk Steinberg Hall-Dietrich Hall 3400 Philadelphia, PA 19104-6371 Phone: +1.215.746.4831 Email: mackinstitute@wharton.upenn.edu