By Exploring Virtual Worlds, AI Learns in New Ways

This could be an interesting approach! Unlimited experimentation and simulation. Note I did not have time to read this article, but I like the idea.

AI Makes Strides in Virtual Worlds More Like Our Own | Quanta Magazine Intelligent beings learn by interacting with the world. Artificial intelligence researchers have adopted a similar strategy to teach their virtual agents new skills.

Nvidia's transformation into a full-stack computing company

If it is not hype, then this is quite impressive! We will soon find out!

E.g. will the virtual restaurant waiters soon come to a table near you (stationary or mobile)?

"Nvidia is using its GTC conference, which kicked off today, to show how it is building out full-stack technologies to make access to AI and immersive experiences available to any enterprise with the ambition to take advantage of them. ...
Nvidia has created a “unified compute framework,” which treats AI models as microservices that can be run together or in a distributed, hybrid architecture ...
The Omniverse, Nvidia’s concept for interoperable “metaverse” virtual worlds, is built on the foundation of Universal Scene Description, a specification originally developed by Pixar. “We think of USD [Universal Scene Description] as the HTML of 3D,”  ... While it may not be governed by an organization like the W3C, a consortium of companies is working to advance USD ... For example, Universal Scene Description recently added a rigid body physics model Nvidia worked on with Apple and Pixar. ...

Jetson robotics platform built on the combination of its GPUs and Arm CPUs. The Jetson AGX Orin version scheduled for release in the first quarter of 2022 promises 6 times more processing power within the same form factor as the previous Xavier edition. Delivering 200 trillion operations per second, Jetson AGX Orin is like a GPU-enabled server you can fit in the palm of your hand, according to Nvidia. But even work on physical robots connects back to the Omniverse. Nvidia recently announced a toolkit for integrating the open source Robotics Operating System (ROS) with Isaac Sim, its simulation environment for robotics applications. Data replication with Isaac makes it possible to test virtual instantiations of robots in worlds populated with synthetic data ...
Nvidia’s technologies are finding their way into surgical robotics."

Nvidia's transformation into a full-stack computing company | VentureBeat

AI 2019: The Year Of Virtual Reality Environments

Posted: 6/17/2019

Perhaps, 2017 was the year when Generative Adversarial Networks (GANs) took off and became a serious and fast expanding research subject in AI.

Now in the middle of 2019, it appears that new and drastically improved, very realistic simulators or AV/VR environments might well be the highlight of AI for this year. This is a new class of interactive environments for AI research.

It appears that these new software platforms/environments/simulators are coming ever closer to being very realistic environments for AI. These new environments promise a great speed up in training of robots and other advances in AI.

Here are some relevant quotes (emphasis added):

1. “But researchers believe these systems will learn best if the simulated environments capture the subtle details — like mirror reflections and rug textures — necessary to make them virtually identical to the real thing.” (S1)
2. ““The Replica data set sets a new standard in the realism and quality of 3-D reconstructions of real spaces,”” (S1)
3. “Replica can be loaded up in AI Habitat, a new open platform for embodied AI research. Facebook AI created AI Habitat to be the most powerful and flexible way for researchers to train and test AI bots in simulated living and working spaces.” (S1)
4. “But it is also an important research tool for creating next-gen AR experiences that begin to merge the physical and digital worlds” (S1)
5. “While other simulation engines commonly run at 50 to 100 frames per second, AI Habitat runs at over 10,000 frames per second (multi-process on a single GPU). This enables researchers to test their bots much more quickly and effectively — an experiment that would take months on another simulator would take a few hours on Habitat.”
6. “For example, Facebook AI researchers will explore ways to build realistic physics modeling into AI Habitat so an AI bot can learn what happens when it knocks a virtual glass off a virtual table” (S1) [Too bad that physics modeling is not yet integrated]
7. “We present Habitat, a new platform for research in embodied artificial intelligence (AI). Habitat enables training embodied agents (virtual robots) in highly efficient photorealistic 3D simulation, before transferring the learned skills to reality.” (S2)
8. “Replica, a dataset of 18 highly photo-realistic 3D indoor scene reconstructions at room and building scale. Each scene consists of a dense mesh, high-resolution high-dynamic-range (HDR) textures, per-primitive semantic class and instance information, and planar mirror and glass reflectors.” (S3)
9. “On the other hand, this variety of [previous] simulation environments can cause fragmentation, replication of effort, and difficulty in reproduction and community-wide progress. Moreover, existing simulators exhibit several shortcomings … Most critically, work built on top of any of the [previous] existing platforms is hard to reproduce independently from the platform, and thus hard to evaluate against work based on a different platform, even in cases where the target tasks and datasets are the same.”
10. “By operating at 10,000 frames per second we shift the bottleneck from simulation to optimization for network training” (S3) [This sounds very desirable!]

Sources (S):

1. How Facebook researchers’ realistic simulations help advance AI and AR Replica and AI Habitat are key to developing experiences like social presence and intelligent assistants (published 6/14/19)
2. Habitat: A Platform for Embodied AI Research (published 4/2/19)
3. The Replica Dataset: A Digital Replica of Indoor Spaces (published 6/13/19)
4. EvalAI: Towards Better Evaluation Systems for AI Agents (published 2/10/19)
5. Evaluating state-of-the-art in AI