Event Follow-up: Geospatial + Game Engines

Patrick Cozzi (PJC): Thank you once again for having us. I'm a big fan of Geoawesomeness and this is a really amazing event. And thanks for everyone for attending, really looking forward to questions from the audience. So as for me, actually, I wrote the first line of code for CesiumJS a little over 10 years ago and it's a really special time for us in the world of Cesium now with the Cesium for Unreal release, we've just seen unbelievable traction and so much creativity. So we're really excited to just be helping software developers build amazing experiences right at the intersection of digital and physical worlds.

Aleks Buczkowski (AB): Great to have you here Patrick, we appreciate you being here. Marc, could you give us a short intro about yourself?

Marc Petit (MP): Sure, my name is Marc from Epic. I run at the Unreal Engine business here. It's been five years that we decided to take the engine beyond games. It's been an interesting journey. And my background, prior to that, for 11 years, I was running the media and entertainment division of Autodesk. In between Autodesk and Epic, I did a lot of sustainable mobility projects such as vehicles, school buses, and a lot of stuff about trying to do good with technology. But I'm happy to be at Epic where we even do better things with technology to help this planet.

AB: All right, Marc, really, really glad to have you here. I didn't know that you were from Autodesk. So actually you know the stuff, you know the drill. There is, of course, a big debate, always GIS versus CAD, but we'll save that for other discussions.

MP: Yes.

AB: Nadine, really glad to have you here today, could you tell us a couple of words about yourself?

Nadine Alameh: (NA): Yes. So thank you again, I echo everything that Patrick said about your efforts in bringing the community together, especially this last year. And thank you for, my goodness, the 500 people who are on the line. I am Nadine Alameh, the CEO of the Open Geospatial Consortium, OGC, and I've been the CEO for the last three years. But I've been in geo for the last 20 years. And my background is, I'm an MIT tech geek, so this is like music to my ears when we have computer graphics, and sorts of photogrammetry, and remote sensing, and gaming all coming together. And I seriously am standing here like a proud mom of Patrick and Marc, or of Cesium and Epic Games because they are both OGC members, and what they're doing here, to me, is what OGC is all about. I'm so happy right now and thankful.

PJC: Well said.

AB: Thanks Nadine. And I guess that for anyone that is involved with geospatial industry, we don't need to introduce OGC because, I mean, you guys do so much for the industry and the standardization that... I mean, this is great, great stuff. All right guys, so let's jump right into it. So let me begin with the first question to Marc.

Epic Games is making a huge investment in making the metaverse a reality. Can you explain Epic's vision for the metaverse and why it is worth pursuing?

MP: First of all, I would recommend to anybody to hear this from the horse's mouth and go to SoundCloud and listen to the metaverse presentation that Tim Sweeney gave a couple of years ago, I think it's very deep. And you would hear Tim talk in that presentation like he talks to us about his vision. And I don't think he has changed a lot over the past two years, except that after a year of pandemic, I think everybody kind of understands how much of our life is bound to be digital. And we've just come off of a year where all of our family interactions, social interaction, and work interaction has happened online, and we've all experienced Zoom fatigue and the fact that video is not the proper medium. So I think it's clear that immersive/interactive, potentially, when it's comfortable for consumers, is going to be the way to do so.

We look at the metaverse for us, it's a social platform, it’s an entertainment platform, it's a commerce platform, so we're trying to build that. We think Fortnite is a good beginning for the metaverse, but we look at the metaverse as something that has some important founding principles from our perspective because we come off with 20 years as a mobile platform when we became the product, when our data became the product. And we think that as we transition from the era of mobile to the era of immersive/interactive and the metaverse, it's a good opportunity to reset the practices in the model in terms of privacy and business model. So we look at the metaverse as something that's real-time 3D social media, it's about entertainment, it's about commerce, it's about everything.

Openness is core to everything we do. We love open source projects. We're not open source technically, the Unreal Engine, but the source code is published, and we have a URL that says that we want to be paid for games and just for games, so we're very, very clear. A strong social component, we think that the metaverse will evolve from what exists already like Minecraft, Roblox, and Fortnite, it's not something that's going to pop up out of nowhere, so it's going to be incremental development and connection of those platforms. And what really matters to us, and that's why next week we're going to be in a big battle with the giant, is it needs to be fair to creators, we want the economic model to happen on content that you've purchased when you like. It's just like in Fortnite, there is no advertising, you buy those things that you like, and you don't actually have to purchase anything to play the game.

We think that this is the kind of model that we would like to be very strong on data privacy, and very, very clear on the mobile, and very, very fair to the creator. So that's kind of our vision of the metaverse, and we're creating a platform and it's a business proposal, it's not a philanthropic endeavor, but we do business with an opinion and with a circumstance with values that we do not compromise on all the way to suing the biggest company in the world. So that's it for me.

AB: Thank you so much for sharing this answer. And by the way, Patrick and Nadine, if you've got something to add it, also Marc, to rectify their answers, just please feel free to do that. But let's jump to the next question, to Patrick, how does you Cesium fit into making metaverse a reality?

PJC: It's a great question. And I actually think we've been working on the metaverse for a while and we just recently realized it. And this has happened to us even in the past where we started CesiumJS for aerospace, and then when we made it open source, we had the community tell us that it has lots more use cases outside of aerospace, whether that be entertainment, CAD simulation, and so on. But when you think about the metaverse and having this two way bridge between the digital and physical world, you need to be able to make a digital twin of the real world. And if you think about modeling the globe all the way down to every pebble, and all the sensors that are available today that's capturing that data and capturing that data over time, it's an incredible amount of geometry, of textures, of material, and then you start throwing AI to generate semantics about that, and it's just a very massive world.

And where Cesium is really making our contribution is in creating a spatial index that allows that world to be efficiently disseminated and streamed. So you can take that massive database, and then you can eventually get something on your screen, that is your view of that world. And we're doing this in a very open way by working with groups like OGC and The Khronos Group to create open standards that help facilitate the transmission of these worlds. And then we're building open source visualization engines as well as commercial platforms for tiling data, and storing and disseminating data.

AB: Thank you for answering this one. And now a question to Nadine. Since OGC is a consortium focused on open standards and location information for developers, how does this fit into the metaverse? And do standards matter when it comes to metaverse?

NA: I have to start by actually thanking you for your question, but also if I can comment on Marc and Patrick's answers, which I love. I hope the people on the line appreciate that when you ask Marc about the vision of the metaverse, he did a draw through all the technical stuff, he actually started with the value. So thank you for actually starting with that because it means a lot from a consortium perspective that's representing the industry, and the government, and the academia, this means a lot. And same thing when I was listening to Patrick, he's really talking not about what he's done, but how he can make it easy for you, everybody here on the line. And again, I think this is why we're here together, there's this common theme, which is, I think, what we're trying to do, I hope that we're trying to do within the Open Geospatial Consortium.

So to answer the question, if you want to remember one thing, the answer is yes, standards matter. So at OGC, because we're a community of these global experts and we've always been, I say, obsessed with making just geospatial, now location information, findable, accessible, interoperable, reusable, the driver has always been, how do we help somebody solve their problem? And guess what, pick any problem, I'm not talking to geospatial, pick any problem faced by human people, it has to involve a lot of data, because nobody has all the data, and the data comes from all these heterogeneous data sources.

And yes, so traditionally I'm going to the, how does that fit? Traditionally, when you think about what we've been doing, all these, what, 25 plus years, some people say even 50 plus years, it's actually bringing this information together and making it easily integratable, and accessible, and just available right in what I call serious domains, defense and intelligence, science applications, emergency response. But isn't this also what the metaverse is all about?

Now in 2021, we can get to that representation of this world, and that, to me, is the metaverse, it's the two way street. And to keep it real, because we're talking about Unreal, even in the metaverse, or especially in the metaverse, you need to combine the data and have it synced up. And guess what makes it all come together, the glue? That's the standards. So yes, standards matter because if we want to build the metaverse in a scalable fashion, in an open fashion, and we want to support innovation, I mean, just the questions from the audience and the last two weeks since the release, we're going to have to rely and agree, actually together, on the glue so that we can all do what is really more important to us, either the impact, or honestly, just the impact. I'll stop at that, standards for impact.

PJC: Yeah, I think there's a nice theme between Marc talking on building a platform that's going to enable creators to be part of that ecosystem, and then Nadine, your work with standards. And that I always see just a hotbed of innovation form around the standards and when we're so early, I think, this is what you need because no one's going to do it all. You need standards and you need platforms to enable everyone to build.

MP: Yeah, we have a little bit of a complex relationship with standards, we love them, but at the same time, we have to acknowledge that the reason why people turn to a game engine to do like movies, or games, or any interactive application, is that the ability that the game engine has to represent all aspects of the fully-simulated world and the story built into it. And that's going to be very, very hard to get the world to agree on, standardizing those elements to make sure that you actually have an open representation of a fully simulated world. That's why our best answer to this is to move as fast as we can with our technology, make it available as source code, and then support all of those standards like Cesium 3D Tiles, glTF, or you may get that big investments consult to make sure that our platforms remains open.

The key value of the game engine is that integration of all the subsystems so that you get the real performance, and that's kind of an opposite concept to a neutral file format, which is like lowest common denominator construct most of the time, and this is more of a lagger than a driver of innovation. And so, we love standards, but we also want to make sure that we keep on innovating and building in real-time.

NA: So if Aleks can humor me. Marc, you're opened the door, the love-hate. To me, I think you're totally right. You both hit on what happens when we're doing things early on, and I think that's the message for all of us and what we're trying to do as a community, is how do we develop standards at the pace of innovation? Because I think even when we talk about standards, the standards process is changing just like what you guys are doing. And to me, I'm thankful for this because I can talk for hours about standards, but seeing is believing. And it's just the demo that Shehzan did earlier, seeing and believing. So appreciate it, it's not a done deal. I think what I take from the love-hate is actually we have a lot of work to do.

AB: Really great answers. I guess that even the fact that we've got OGC and you, Nadine, kind of sharing the points on the discussion today, this really matters for gaming industry. The standardization really drives the innovation rather than closing the environments because of maybe competitive thinking is not really going and driving anywhere, so really great to hear that.

All right, so Marc, now, again, over to you, game engines are famous for creating elaborate virtual worlds. How will the use of real world 3D content, in our case, we mean by this, geospatial data, transform world building in Unreal Engine, how do you see this expanding use cases for the platform?

MP: I think that, again, the core capability of game engines is that we kind of aggregate content and play back content in real-time from a lot of sources. And I think that means that the GIS data can be used, now, into a lot more context than before, because pick your favorite three, we have CAD, GIS, Reality Capture. I mean, the game engine is wide open to a lot of formats and a lot of standards. So I think that frees up everybody, it provides a context for everything. It starts with a game engine, doesn't end with the game engine, you have to do something with it. You have your scripting language, you have an application, you have something that you want to do. We're a platform and not just an end-user application. So what are people going to do when they can seamlessly integrate GIS, and games and stuff? I mean, we've seen some amazing work already, and I think we'll see more, so it's up to you guys to figure it out, we're just creating the possibility.

But the other thing that we care about, and Patrick touched on it, is as we think about cloud-based gaming, next-generation console, and the amount of content that we put in games now, we create entire new worlds. The promise of the game engine for transmedia production, in the same world you create the game, you create the movie and think about Avatar and Pandora, think about Star Wars and Tatooine, I mean, we want to be able to get those tools across the fence to people creating worlds that do not exist and have that ability to have visual quality. We want to drop a camera into Cesium and shoot a movie, that's the ultimate goal. We're not there, not quite there, but we will get there.

And then we need to add a layer of game stuff, add the trash can in the street and make it something that's life-like, well, then you have the buildings, so we have a lot of complementary technology, but we want to have that photorealistic quality. And then we're going to let people create stuff, but way outside of GIS, use that GIS construct to manage large-scale worlds and open worlds. I think that's going to drive a lot of convergence, to create opportunity for people to create games we haven't seen before or movies we haven't seen.

AB: So Patrick, so maybe following up on this question with you. I guess that you must have seen a lot of really interesting, and on the other hand, crazy use cases and you guys must have thought about a whole portfolio of different applications. Could you share some insights on like the most useful and the most crazy examples of the use cases?

MP: A question for Patrick, he is the guy with the crazy ideas.

NA: All the craziness!

PJC: So first, look, I mean, working really closely Unreal Engine is really exciting to me personally. So in high school, I did a lot of game programming on my own, way back with to BASIC and Turbo Pascal, and I thought that I wanted to go into game development. But then I realized, "Hey, I wanted to do graphics development in general." And now, I'm starting to see this 180, where am I like, "Well, geez, what is in graphics development? What is game development? This line is blurring here." In terms of the use cases that we're seeing, it really is this two-way connection in that we're bringing in 3D geospatial into Unreal Engine. And then, our core 3D geospatial community is now getting exposed to Unreal Engine. So we're seeing this kind of two-way collaboration and innovations on both sides.

Some of the fun and crazy stuff we see we've already seen on Twitter and LinkedIn, Full Sail University picking up Cesium for Unreal and using it in their course and having the characters flying around the photogrammetry models, which I think has been really cool. We've seen a ton of work in the simulation world, and I think we're in store for an amazing demo at the end of this presentation from Sami in the Flight Sim World. And then we see a lot in AEC, where you have a city, its photogrammetry, then you want to put in your future potentially built architecture model, or you kind of want to go backwards in history and, for example, recreate a flood that might've happened 50 years ago from a photograph. And I think we're very early and we're going to keep seeing this almost every day, definitely every week, but we constantly are seeing this. And for all the folks in the audience, please, if you're doing something cool or you just have an idea, drop it in the chat, send us a link to what you're posting on social media.

NA: Just I think when we talk about crazy, the examples that Patrick mentions and, seriously, the buzz on Twitter just from the release, this is two weeks, reminded me of when, remember when Google Earth came out? And we've been doing that, whatever that is, for years, but then Google Earth came out and all of a sudden it sparks because, again, seeing is believing, all these opportunities and that two way. I'm going to hit actually Marc on a use case that you mentioned because I think it's really great, it's the digital twins, it's the urban digital twins. And that's why I appreciated Shehzan in his demo, he kept going back to the physics model. This is not fake, this is not a game, this is actually based on models, and I think this is where our power is, because we've got the hydrology model. I mean, he had an ocean model, and we have an air model, and we have all of that and to bring it altogether, that's the crazy. I think we haven't seen the crazy.

MP: I agree. I think on May 5th, we have an online presentation on digital cities. I think it's going to be an interesting presentation around digital cities, we have the entirety of Wellington in the engine, the city of Wellington. Again, that's a starting point. Once we create, when you realize that, you still have to run the application to do something, do the analysis that you need to do, it's not just about creating some beautiful representation of the existing data, it's about enabling analysis, enabling simulation, enabling presentation.

AB: Yeah, thank you for these answers. And I mean, when I think about different crazy applications, Nadine you said that you have two boys. So I get that if you'd let them build, and then finding the different applications of use cases, we would all be very surprised by what they would create, because this is really giving a lot of space for unleashed creativity. All right, so Patrick, this is, I guess, a really very important question towards the core of the philosophy of Cesium. You've decided to release Cesium for Unreal free and open source, so why do you think this is the right move?

PJC: We put a lot of thought into this, so if you just think about the metaverse, the metaverse has the potential to have the same impact as the internet. If you go back 30 years ago and you think about the internet unfolding, I mean, you need to spark collaboration, you need to spark innovation. And we think doing open source, and making that source code available, and allowing communities to form around that, to submit bugs, to submit pull requests, even just to have access to the code, to help them debug, we think it's a great move. Now, at the same time, we have to run a business and our business is growing very quickly and we want to make sure that we can build something that's sustainable and ultimately scalable. So you look at, "Okay, well, how do you fund that open source?" So we didn't want to give away everything that we're doing as open source because it actually creates this kind of inverse incentive to not make your open source components as good as possible.

So we've done the whole visualization layer on top of Unreal as open source Apache 2.0 license, and then we use what's called an open core model, and we create separate optional value add software here in the form of cloud and on-premise services to tile your 3D data, and then to serve 3D content that we've curated such as terrain and buildings, that's ready to use. So it's a very fair model where we compete on the merit of our products, not on any type of vendor lock-in and we use, no surprise here, open standards to have our open source products or projects communicate with our commercial product.

MP: Good move!

PJC: Thank you.

NA: I think, Patrick, you described it so, so well because how do you balance being a profitable or even a sustainable business with being open? And I think what's happening here, that's why we're waiting on the crazy, because this group is the enabler for the innovation, and that's the value proposition in this case. What you guys are doing are essentially collaborating to compete or collaborating so that other people compete so that we all grow that pie. I think that's what I hear, just growing the opportunities based on the combination of open standards and open source.

PJC: Well said.

AB: And as I was telling Patrick, just before we even started, this is really also an inspiring approach. So I guess that this is really the move that will have a lot of followers in the industry. So this is really, really awesome. Nadine, another question is for you, from collaborator and innovator perspective, how does gaming fit into the bigger picture of location? Where have you seen this technology applicable and where is it going?

NA: So I'm smiling big time because this is reminding me of our prep call when Marc said I could spend the whole hour on this. So it's the perspective of collaborator and innovator, but it's also a perspective from a mom. You mentioned I have two boys, ages nine and 13, and I called them video game experts because I don't want to say addicts, so they're experts, but they generally spend more time in the metaverse than with me, I think it's a fact, and now their school is even virtual. So how does everything fit together, because exactly how Marc stated, everything fits together right now because we're living in a virtual world.

But on the serious end, because the previous question about the use case is this is the same thing, when we keep, I think, increasingly appreciating that everything happens in space and time, and that's geospatial, and that's how it's coming back all the time to location. And the cool thing that I can dream about is being able to take my son on college tours without actually going there, because my goodness, if we can fly around, how cool is that? And it's real, it's not fake. Or even real estate people showing houses during the pandemic, how cool if you can actually navigate from different perspectives and it's not pictures that you have right now?

And somebody on the chat mentioned underground, that's like a whole area. Can you imagine? I mean, that's why we're working so hard these days on an underground data model because the energy pipes, and the water pipes, and the phone pipes and everything underground, I'm not the expert, but it's still siloed and we have to bridge that gap so that we can actually navigate underground. And that's where, I think, when Patrick talks about the standards, there are several steps way before you get to the amazing demo that Shehzan did.

So I will end with the two examples, I think, where this is going, the impact. One, I mentioned the urban digital twins, so this is helping with urban planning. I mean, imagine, we see exchanges and events around if we put more green space, what happens to the community, but imagine seeing that because you can play with the time, imagine seeing how it improves the air quality, or reduces cost of childcare because people can go to the park, or they change their mobility patterns because they want to go to the park, just seeing that. So that's an amazing example, which is, I think, honestly, low-hanging fruit, we're not that far.

The one that's the dream is the climate understanding. So imagine doing this at a global scale. That's why, again, I appreciated the physics, the model, because if we have all these models, or sort of have the expertise, imagine starting to track and understand climate change from a local level. So if you make a decision in a county and how it affects us globally and vice versa, how something global goes all the way down to the local as to maybe influencing how much we pay for gas or how much we pay for any consumable. That's, I think, where eventually it fits altogether and it's all happening in space and time. And in some ways it's simple, but actually, it's very, very complex.

AB: Thank you, Nadine. So let me ask another question to you Marc, this is really important from the geospatial industry perspective. So what will game engine technology contribute to the geospatial industry?

MP: Okay, I mean, there's a few layers to that. I mean, the base answer is a high priority presentation with year-for-year data with photorealistic rendering. I mean, UE4 gives a lot of capabilities. You heard about UE5, it's kind of dealing with that. And then when you start adopting the engine as a platform, you can deliver your projects across a lot of other platforms like hardware platforms, like mobile, and the cloud, and Pixel Streaming, and Linux, and everything and a lot of other fancy systems like game consoles that you may not care about, but other people do care about.

The other thing is it's an aggregation platform, I said to ingest can combine a lot of data. And we do this in two ways. One, is we fully support standards like glTF and 3D Tiles, but we also have our architecture that allows us to make sure that we get the right data. And we're not the mercy of a vendor. If you want to import a BIM file, we don't want to be at the mercy of whatever people write in an open format. So we have connectors to a lot of CAD systems, and a lot of other systems to make sure that we have a metadata rich environment.

Then, it an interactive development application, I mean, whether you use Python or you use Blueprint, you can create interactive applications. You have UI, you have physics, you have a bunch of things available for free. C++ programming, you can integrate it to the library in your application, you can create mobile apps. I mean, once the data is in Unreal, it gives you a lot of platform. The other thing is around collaboration. I mean, the engine supports metauser editing. It's not something you usually find in a desktop software. Collaboratively, people around the planet, as we speak, are designing movies or games in the Unreal Engine. And the output application that you put out, like what Shehzan would demonstrate, can be a multi-user as well. This is a corporate game, the game is never a solitary experience.

And then we connect a lot of the technology like Free-Motion, it's a very easy, simple tool to use that allows you to... that's the easiest way you can have to tap into the part of the Unreal Engine to aggregate data, and then you can move that data across to the engine. We are essentially starting to work with the Reality Capture team from Capturing Reality in Slovakia because we are a strong believer that we can improve the photogrammetry pipeline. As you have probably seen, the quality of our Quixel Megascans, and we want basically any one of you to be able to produce that level of quality, the multiplicity of those formats, all those LODs, and really make it practical to go and scan everything in the planet and bring it into the Unreal Engine through Cesium or through other mechanisms.

So it's a complete ecosystem to create a digital twin of the planet and to create the application that's going to leverage this data. So it's number of things and it's free, there's no catch. I mean, our business model is based on the fact that for people like you where we don't market the tools. We sell support contracts, if you don't want to do it on your own, we're happy to support you but you would have to pay for that. We only collect a modest revenue share on games. That's our business model, it works fine source coding on the internet. It's very open. We are a very, very successful business, we don't need to lock people into our data, or overprice the software to make it great leading. So we were very, very happy to be able to run the business this way.

NA: Can I add to Marc? My God, you're very inspiring, Marc, I appreciate it. I can't stop myself, so you'll have to stop us. When I hear you talk about how will the game engine technology contribute to geospatial industry, all I'm hearing is it's almost like you're going to force us to work together more closely so that we can actually build the metaverse, or it will never get there because then you'll have to build all these connectors. And so, what I'm hearing is it's actually you have that environment where somebody can bring the pieces together. And now you're saying we have no excuse, I'm seeing the climate change also shirt on the chest like, "No, no, no, and you have no excuse, do it." And that's what I'm hearing that you can bring to the geospatial industry actually and beyond. It's all on you. He's like, "No, no, no."

MP: It's clear that the Unreal Engine is to create a pathway for the metaverse. I mean, we have this vision for the social, and entertainment, and commerce platform, it can be used at work as well. It's not going to be the only one, but of course, the engine is going to be the pathway into this, and we want to eliminate friction for all the digital cities and for the planet so that people can enjoy the planet in that platform. What they do with it, it's entirely up to them, the kind of obligation to do it for them. You would try to have fun and have games, or you do something very serious, like you mentioned, around explaining, doing visual communication about complex phenomenon that drives climate change. All that stuff is 100% up to the people, and that's why we want to be creator friendly and very open, it's the base of what we're trying to do, is build a momentum around that platform.

PJC: Yeah, then if I could just add a few comments of what Unreal Engine brings to the geospatial industry. So I've described it internally at Cesium and on social media that when we released Cesium in front of Unreal Engine, I really feel that we made 10 years progress overnight. When I look at anything from the biometric clouds to the water effects, I know and Cesium knows exactly how much work goes into building those type of presentation layer features and there's been, I guesstimate, billions of dollars of investment, again, in game engine technology and its ecosystem. And now, that's unlocked for us to build geospatial applications.

Game engines were historically for games, virtual made-up worlds. We've now added the precision layer, so global scale, WJS84, ellipsoid, massive scale streaming, accurate, the same accuracy that we put in CesiumJS, the same accuracy that we had in the aerospace software at AGI. So I really think it's a remarkable tool chain and platform for all broadly across geospatial.

AB: Thank you so much for answers. Let's maybe now jump more into specific technology. So let me maybe fast forward one question here and let's look into 3D Tiles. And this is a question to you Nadine, we understand that 3D Tiles, a standard built for streaming massive amounts of 3D data is a community standard originally created by Cesium. But why is interoperability important for creation platforms like Unreal Engine?

NA: So first of all, I have to give total full credit to Patrick and his team for 3D Tiles for actually not only seeing the future in 3D Tiles but seeing how much bigger of an impact, we were just talking about this, how much bigger of an impact it can have if it's an open community standard. And it's win-win, I think, on so many levels, and a great success story, and developing standards at the pace of innovation, and making the pie larger in creating a platform for innovation. So I can go on.

Again, I mean, I appreciate we started with the demo because it sort of grounds us. So when I go back to the demo and how we were talking about streaming, I mean, it looks so simple but it's so complex behind it. We were streaming, Shehzan was streaming massive heterogeneous datasets in the backend. That's what we're talking about here. How do you get what you need, just what you need, as tiles from the datasets, across datasets? That's the interoperability, so you don't have to do it one by one. Because we all know that the developers, that's not where they want to spend their time. So we get the basics for them, that's the streaming, that's the performance, so that they can get to the meat of the applications, of the decisions, and really help us with all the problems that we're talking about here. So the interoperability becomes just agreeing on how are we going to do the basics so that we can thrive on.

AB: Definitely. Totally agree. So let's just take one last question from my end, then we will move to the questions from the audience. And so, Patrick just wanted to ask you, now that we have 3D Tiles in the real world, inside game engines, what is the next for 3D Tiles?

PJC: Yeah, and I'll be brief. So we built 3D Tiles to be able to stream massive heterogeneous 3D geospatial datasets including the geometry and the metadata that gives you kind of the semantic meaning behind it. You're on the side of the building, what's a brick versus what's a window. And we're just seeing more and more semantic metadata being created for 3D models largely from AI and ML algorithms. And one of the things that we're doing to move forward the 3D Tiles specification is to be able to store that data at a variety of granularities. So we can very efficiently store that at a very fine grain of detail and I think semantically augmenting these 3D models is going to be a huge advancement for us.

AB: Thanks so much. So now the actual fun begins. So let's start taking questions from the audience.

Muthu Kumar (MK): Thank you. First of all, thanks a lot to all three of you for being focused on the questions here, and I'm not being distracted by all the questions and the nice conversations that are happening in the chat, but now is the time for you guys to answer some of the questions that the audience had. And let me start with a question that Leo sent us yesterday. And I found this question so interesting and also timely because we just had the Oscars, no idea who won, I don't want to know, don't spoil it, but what happens if we use the Cesium for Unreal Engine in movies, how do the licenses work? And if I win an Oscar, do I have to share it with Patrick and Marc?

NA: What about me?

MK: And Nadine too.

PJC: Yeah, I think the answer that last question is definitely yes, I know you'd love to be on stage. In terms of how Cesium for Unreal is kind of opening up our eyes, we are seeing a lot of interest here in being able to use the real-world models to make movies. And we're just now starting to understand kind of the downstream data licensing requirements for that. So I would really need to hear a little bit more, I would love for this person here to contact us because we want to make this use case work. We think it's a really amazing idea.

MK: How do you see this Marc? Do you share Patrick's opinion that you are fine with just getting the Oscar or do you also want the big money?

MP: Yeah, I mean, the data licensing is an open issue. I think, right now, when you shoot a movie in the city, you'll have to obtain an authorization, do some form of license. And I think all of those practices which are very diverse actually and not standardized at all, will apply in the digital world and I think we're all going to have to learn how to live with it, but certainly, it is not something I would overlook right now.

I mean, the OpenStreetMap data doesn't have a lot of consent into it. I don't think anybody will have trouble using OSM data, but as we get with photogrammetry and much more detail with presentation in buildings, and brands and stuff, it is something that we're going to have globally figure out. And the IP owner we're going to have to decide what they want to do with their IP. What do we do with your game at headquarters if somebody wants to fly a drone and scan it, can they put this in a game? I mean, it's kind of up to us to decide, and I think the legislation will have to adjust a little bit as well.

MK: I think Nadine, you have an answer as well, and I guess it doesn't involve you getting Oscars.

NA: I'm going to go to the party. No, but it's just occurred to me actually, it's because it's not a far-fetched idea. I know an Oscar winner, and used to be with a company called Pixie, an OGC member, and he was the editor of the WAIM stack, what is it, the Wide Area Imagery stack, and it actually won an Oscar. I mean, this is not crazy what we're talking about here. And this is so amazing what we do in geospatial for the serious stuff. Eventually, it translates into entertainment where you get the awards. So that was my contribution. Actually, we have Oscar winners amongst us already. And I'm looking again at the chat, it looks like next year I'll be getting some invites.

MK: Sounds like a plan. All right. So I have to admit that we have a ton of questions, but with an eye on the time and also not to overload you guys too much, I will go through a couple of them in random order. So for those of you whose questions are going to get left out, please don't worry, I will follow up with Patrick, Marc and Nadine to make sure that they answer it. So here goes for the first question, and I love this question actually. So we've been talking about game engines, we've been talking about what geospatial, and I'm going to start here with Marc, so which industry do you think has the more potential to evolve the other? Do you think it's going to be game engines changing the way we engage with geospatial data and systems, or do you think it's going to be the other way around?

MP: I think we are amazed to see every time an industry embraces the game engine, what they can do with it. So I actually don't know how to answer this question because I think we feed off of each other. I mean, gaming, in real-time 3D, it's the beginning. We're very used to digital video as a medium, it's the super early days of a real-time 3D as a medium. This is going to be such an integral part of all of our lives in the foreseeable future as soon as we get that hardware we've been waiting for. I don't know who's going to influence each other, but I know it's going to be part of everybody's life, at home and at work, guaranteed.

MK: What do you think, Nadine?

NA: I agree. I think we, and by we, in terms of geospatial, I think we are definitely opening a lot of worlds for gaming because it's not anymore restricted to gaming, just the simulation. Honestly, on the defense and intelligence side of things, it's just the whole world. And we see on the chat, the AEC stuff. But I also hear Patrick and I was nodding my head when he said, overnight, we make progress because of you as a geospatial industry of about 10 years, true. So I'm agreeing, it's both ways and it's both ways quick. So it's not going to take five years for us to see, it's actually really quick.

MK: Patrick, let me guess your bet is on the geospatial industry changing the game.

PJC: No, no, I agree with Marc and Nadine here, I think the line is blurry. Am I geospatial developer, am I a game developer or am I just a developer? I think that it's becoming blurry, and that's a good thing.

MK: Fantastic, speaking of developers, and I guess all three of you have some programming experience, so here comes a question that, for me, at least, is in that direction, with all of these 3D models and this wonderful integration of real-world into gaming engines and vice versa, do you see this as a bottleneck in terms of what we can handle with our GPUs?

MP: Well, this is where you get to have smart engineers on your team. And first of all, the acceleration of Moore's law has actually been beaten in this case many, many times, because the hardware gets more powerful. We're talking about high-speed networking everywhere. So I think the hardware platforms are going to be amazing. Some proper engineering, there's digital texturing and there's nanite, there's the virtual geometry system, I think, we can... We have line of sight on the moment when we can process in real-time and transfer on the geometries, but at the end of the day, that's simple. I don't want to be overly cocky about it. I've read a lot of people spending a lot of time to make it work, it's not going to work on the first try, but we think we know how we can process the kind of level of details that we need to do something that's photorealistic.

And there was a question in the chat about using procedural techniques 100%, I mean, captures only one thing that you need to add a procedural layer on top of it when you do reality capture. The machine knows what the image are, it's forest, it's the roads, it's the cars, it's the building. I don't have to use geometry on every single thing you can do. We have a bright future, we've mentioned learning by mixing 3D reconstruction, and then people are geometry, and synthesis and all the things we've been doing. We've been doing this for the past 25 years actually, so we understand those things and populate gigantic environments very, very easily in a way that's first plausible for sensors and may be plausible for humans.

NA: Yes, so I just want to add that it's a great question, it's a timely question because we are talking about this exponential growth. And to me, this is why I see within the OGC community, we have now players like HP, like Intel, NVIDIA is doing a lot. So I think this is why we're literally all coming together. This is the push. So we reach, and now we need the chip manufacturers, and the designers, the engineers, like Marc said, to take us to the next level.

MK: Patrick, do you have any thoughts?

PJC: Yes, I would say Cesium exists and my entire career exists because of this question. And I completely agree with Marc in that the GPU performance is unbelievable and adding procedural on top of the scanned world is very compelling. At the same time, the user expectation is growing very quickly where they expect these photorealistic scenes. And then the scale of the worlds are increasing and the detail that you model. So I think the core techniques that we're working on around streaming, level of detail, compression, all of the smart engineer stuff that Marc has said, there's still years and years ahead of us to keep pace with the amount of data being generated and then that kind of user demand.

MK: All right, fantastic. So that kind of leads me into the next question, you set it up rather well, so the demand for things are always increasing and people are already poor with your Unreal Engine 4, so we want to talk about Unreal Engine 5 already. I know it's only been a couple of weeks since you guys released it, so take a breather and we don't want you to do it tomorrow, but we definitely want you to do it soon. So the question was, if you can use nanotechnology to reconstruct 3D cities together with your location, can we do it in Unreal Engine 5?

MP: Eventually, you will. So we're still working on UE5. We announced preview for the spring. Spring has only started, if it's not finished, so look at the leaves and it's about to come. It's a very, very ambitious endeavor and actually providing that amount of complexity. But you'll see already a lot of it. Let's be honest, I mean, this is the kind of thing is you don't get 100% on the first release. And so, we'll make it work over the next months and the next couple of years, but yes, the short term, and we will see that come fairly soon, in my opinion.

MK: Okay, then let's maybe take one more question. I know that you guys are a bit tired, so humor me for one more question, I promise it's the last one and this is a bit of a philosophical one, I guess. So what are your views on the metaverse? Is it similar to a cryptoverse, de-centralized land, a sandbox? I have to be honest, I probably have to Google most of these terms after the event. So you are on your own for this ones. Nadine, do you want to go first?

MP: I can give you my answer, is whenever there is crypto in a sentence or NFTs, I just move to the next one. I think it's just sort of hugely undefined concepts. I mean, we know we're going to have VR and virtual world. But right now there are a lot of scammy concepts that revolve around those settings and I tend to stay away from them, to be honest.

MK: What about you guys? Do you agree with Marc? Do you see it differently? I mean, on a personal level, I don't understand it, so I don't invest in any of the NFTs and the cryptocurrencies. But how do you see this, and is the metaverse going to have a marriage with the-

PJC: Yeah, I'm probably not the right person to talk crypto, but I did like what Marc said much earlier here about the metaverse is going to enable creators to monetize their content. Doing that, as opposed to, for example, putting up a bunch of banner ads, and I think that's a really enriching and compelling place. And then there will be the technology underlying the distribution of those goods and the monetization of those goods, which, I guess, is maybe related to the question. Maybe Marc, that's something that you might want to tell us more about.

MP: No, I mean, and I was reading another comment on crypto, and decentralized finance, and general ledger technology, I mean, we don't even know if it's a weighed scale, so will those things play a role? Maybe. We know the principles, we don't quite know how we're going to implement that. Right now we use Google payment systems. We'll do whatever consumer wants at the end of the day, that's whatever it's going to be, the mainstream solution. What we care about, as you know, not to offer charge and grabbing undue piece of the pie because you're the guy providing the payment system. So we don't have any strong opinions on what is going to be the implementation of those, but we're very, very strong on our values and ideals, the things we want to see and don't want to see on the corner of the metaverse, I think there will be multiple of corners in metaverse. On ours, as you said, fair to creators and fair to consumers.

MK: Okay, maybe one parting question from my side. And so, I mean, all three of you, you've done incredible work in the last couple of years, and especially now with this Unreal Engine, but is there maybe one thing that you feel, and hope and wish that the industry understands better? Is there a specific aspect that you wish that the audience and the general public who are using your APIs and engines understood better? Is there one misconception, one engineering challenge that you wish people appreciated?

PJC: It's a really good question, I think you caught me off guard. I don't know, so one thing is we're big on education as the way that kind of we bring our products to market and we really love folks to jump in. We give tutorials and video tutorials so that you can try it out in 30 minutes. So we encourage folks, if you think there's potential and you're not sure, just dive in, it's fast and easy to get started.

MP: Yeah, give it a spin. It's all out there for the taking on the internet. And I think, be curious and experiment. We're hearing how 65-year-old cinematographers are downloading the engine and learning how to design their movies. And it was something we heard yesterday and it was an amazing thing that people can actually see all the value from it. So I think that's the thing, give it a spin, there is no catch.