Learning Process Rewards via Success Visitation Matching for Efficient RL

This could be an interesting new paper by Sergey Levine and his team!

From the abstract:

"In many modern applications of reinforcement learning (RL), the natural reward for a task of interest is inherently sparse: a reward of 0 is given everywhere except when the task is completed, when a reward of +1 is given.

Training a policy to maximize such a sparse reward requires solving a challenging credit assignment problem, leading to slow or ineffective RL improvement.

We propose a simple approach to transform a sparse outcome reward into a dense process reward.

Our approach relies on training a discriminator to distinguish between previous successful and unsuccessful episodes, and using this discriminator to incentivize the RL-learned policy to match the state-action visitations of successful episodes, while avoiding those of unsuccessful episodes.

By incentivizing the policy to match the visitations over all states, not just those that correspond to task success, this reward provides dense feedback on whether progress is being made towards task completion, and, we show, provably achieves this without changing the optimal policy.

Focusing on finetuning of robotic control policies, we demonstrate that our approach leads to significantly faster RL finetuning performance on both simulated and real-world manipulation tasks, as compared to simply maximizing the sparse outcome reward."

[2606.23640] Learning Process Rewards via Success Visitation Matching for Efficient RL

On A Brief Survey of Deep Reinforcement Learning

Highly recommendable! An excellent survey paper on reinforcement learning! Well written! And the survey is only 16 pages long

The survey is not bloated with irrelevant references! On the contrary, most references are very relevant and well chosen.

This a survey from 2017, so it is more of historical relevance. Citation count about 6763 as of 5/11/2026 (Google Scholar).

Just finished reading it for the first time!

From the abstract:

"Deep reinforcement learning is poised to revolutionise the field of AI and represents a step towards building autonomous systems with a higher level understanding of the visual world.

Currently, deep learning is enabling reinforcement learning to scale to problems that were previously intractable, such as learning to play video games directly from pixels.

Deep reinforcement learning algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world.

In this survey, we begin with an introduction to the general field of reinforcement learning, then progress to the main streams of value-based and policy-based methods.

Our survey will cover central algorithms in deep reinforcement learning, including the deep -network, trust region policy optimisation, and asynchronous advantage actor-critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via reinforcement learning.

To conclude, we describe several current areas of research within the field."

[1708.05866] A Brief Survey of Deep Reinforcement Learning (open access)

The timing of rewards plays a key role in learning, identifying a new biological principle governing learning

When scientific research confirms the obvious! Of course, besides size/value and frequency of rewards, timing of rewards matters too!

"... Researchers ... recently carried out a mouse study challenging this assumption, suggesting that the strength of reward-based learning also depends on the timing between rewards and not just on how many times a mouse is rewarded after the same stimulus. Their paper, published in Nature Neuroscience, could reshape existing models of learning, decision-making and potentially even addiction. ..."

From the abstract:

"Learning the causes of rewards is crucial for survival. Cue–reward associative learning is controlled in the brain by mesolimbic dopamine. It is widely believed that dopamine drives learning by conveying a reward prediction error. Dopamine-based learning algorithms are generally ‘trial-based’: learning progresses sequentially across individual cue–outcome experiences. A foundational assumption of these models is that the more cue–reward pairings one experiences over a fixed duration, the more one learns this association.

By identifying a new biological principle governing learning, we disprove this assumption. Specifically, across many conditions in mice, we show that behavioral and dopaminergic learning rates are proportional to the duration between rewards (or punishments). Due to this rule, the overall learning over a fixed duration is independent of the number of cue–outcome experiences. A dopamine-based model of retrospective learning explains these findings, thereby providing a unified account of the biological mechanisms of learning."

The timing of rewards plays a key role in learning, study finds

Duration between rewards controls the rate of behavioral and dopaminergic learning (open access)

Fig. 3: Learning rate scales proportionally with reward frequency across a range of trial spacing intervals.

Disclaimer:

I am currently blogging from behind the Great Firewall of China.

My Internet service in China is very spotty. Thus, I am not able to blog as usual.

AI That Teaches Itself: Tsinghua University’s ‘Absolute Zero’ Trains LLMs With Zero External Data

Recommendable! This could be a game changer! Currently, this research is limited to the areas of coding and math.

"... Researchers from Tsinghua University, Beijing Institute for General Artificial Intelligence, and Pennsylvania State University have proposed an RLVR paradigm called Absolute Zero to enable a single model to autonomously generate and solve tasks that maximize its own learning progress without relying on any external data. Under this method, researchers have introduced the Absolute Zero Reasoner (AZR) that self-evolves its training curriculum and reasoning ability through a code executor that validates proposed code reasoning tasks and verifies answers, providing a unified source of verifiable reward to guide open-ended yet grounded learning. ..."

From the abstract:

"Reinforcement learning with verifiable rewards (RLVR) has shown promise in enhancing the reasoning capabilities of large language models by learning directly from outcome-based rewards.

Recent RLVR works that operate under the zero setting avoid supervision in labeling the reasoning process, but still depend on manually curated collections of questions and answers for training. The scarcity of high-quality, human-produced examples raises concerns about the long-term scalability of relying on human supervision, a challenge already evident in the domain of language model pretraining.

Furthermore, in a hypothetical future where AI surpasses human intelligence, tasks provided by humans may offer limited learning potential for a superintelligent system.

To address these concerns, we propose a new RLVR paradigm called Absolute Zero, in which a single model learns to propose tasks that maximize its own learning progress and improves reasoning by solving them, without relying on any external data.

Under this paradigm, we introduce the Absolute Zero Reasoner (AZR), a system that self-evolves its training curriculum and reasoning ability by using a code executor to both validate proposed code reasoning tasks and verify answers, serving as an unified source of verifiable reward to guide open-ended yet grounded learning.

Despite being trained entirely without external data, AZR achieves overall SOTA performance on coding and mathematical reasoning tasks, outperforming existing zero-setting models that rely on tens of thousands of in-domain human-curated examples.

Furthermore, we demonstrate that AZR can be effectively applied across different model scales and is compatible with various model classes."

☑ 10 Mins AI Read: ByteDance Open-Sources DeerFlow and LightOn AI Released GTE-ModernColBERT-v1....

AI That Teaches Itself: Tsinghua University’s ‘Absolute Zero’ Trains LLMs With Zero External Data

Absolute Zero: Reinforced Self-play Reasoning with Zero Data (preprint, open access)

Figure 4:Absolute Zero Reasoner Training Overview. 

Comment on: Attention as an RNN

Very interesting paper by Yoshua Bengio and collaborators!

"The advent of Transformers marked a significant breakthrough in sequence modelling, providing a highly performant architecture capable of leveraging GPU parallelism. However, Transformers are computationally expensive at inference time, limiting their applications, particularly in low-resource settings (e.g., mobile and embedded devices). Addressing this, we (1) begin by showing that attention can be viewed as a special Recurrent Neural Network (RNN) with the ability to compute its many-to-one RNN output efficiently. We then (2) show that popular attention-based models such as Transformers can be viewed as RNN variants. However, unlike traditional RNNs (e.g., LSTMs), these models cannot be updated efficiently with new tokens, an important property in sequence modelling. Tackling this, we (3) introduce a new efficient method of computing attention's many-to-many RNN output based on the parallel prefix scan algorithm. Building on the new attention formulation, we (4) introduce Aaren, an attention-based module that can not only (i) be trained in parallel (like Transformers) but also (ii) be updated efficiently with new tokens, requiring only constant memory for inferences (like traditional RNNs). Empirically, we show Aarens achieve comparable performance to Transformers on 38 datasets spread across four popular sequential problem settings: reinforcement learning, event forecasting, time series classification, and time series forecasting tasks while being more time and memory-efficient."

[2405.13956] Attention as an RNN

AI beats human world record holder for the first time in physical skill game, i.e. labyrinth maze game

Good news! Amazing stuff! This comes after the Rubik's cube in 2019. What comes next?

"... The robot received [only] 6.06 hours of practice. Impressively, it beat the previous world record set by Lars Göran Danielsson, a player since 1988, who set a time of 15.41 seconds in 2022. CyberRunner completed the game in 14.48 seconds — faster by over 6% compared to the human record holder. ..."

From the abstract:

"Motivated by the challenge of achieving rapid learning in physical environments, this paper presents the development and training of a robotic system designed to navigate and solve a labyrinth game using model-based reinforcement learning techniques. The method involves extracting low-dimensional observations from camera images, along with a cropped and rectified image patch centered on the current position within the labyrinth, providing valuable information about the labyrinth layout. The learning of a control policy is performed purely on the physical system using model-based reinforcement learning, where the progress along the labyrinth's path serves as a reward signal. Additionally, we exploit the system's inherent symmetries to augment the training data. Consequently, our approach learns to successfully solve a popular real-world labyrinth game in record time, with only 5 hours of real-world training data."

AI beats humans for the first time in physical skill game CyberRunner completed the labyrinth maze game in 14.48 seconds

CyberRunner AI

Sample-Efficient Learning to Solve a Real-World Labyrinth Game Using Data-Augmented Model-Based Reinforcement Learning (open access)

On Learning Quadrupedal Locomotion over Challenging Terrain

Very recommendable paper! Impressive results!

[2010.11251] Learning Quadrupedal Locomotion over Challenging Terrain

Doubling a qubit’s life, researchers prove a key theory of quantum physics

Amazing stuff! Could be a breakthrough! Extending classical error correction to quantum physics! Again machine learning was applied in this research!

"Researchers at Yale have for the first time, using a process known as quantum error correction, substantially extended the lifetime of a quantum bit — a long-sought-after goal and one of the trickiest challenges in the field of quantum physics. ...
decades after its theoretical foundations were proposed — that quantum error correction works in practice.  ...

Quantum systems, though, are fragile. They are bedeviled by a fundamental phenomenon of decoherence, a process in which the information stored in qubits quickly loses its quantum properties as a result of their interactions with the surrounding environment. 

Quantum error correction, which was theoretically discovered in 1995, offers a means to combat this decoherence. Employing redundancy, it protects the quantum bit of information by encoding it in a system larger than what, in principle, is needed to represent a single qubit. ..."

From the abstract:

"The ambition of harnessing the quantum for computation is at odds with the fundamental phenomenon of decoherence. The purpose of quantum error correction (QEC) is to counteract the natural tendency of a complex system to decohere. This cooperative process, which requires participation of multiple quantum and classical components, creates a special type of dissipation that removes the entropy caused by the errors faster than the rate at which these errors corrupt the stored quantum information. Previous experimental attempts to engineer such a process faced the generation of an excessive number of errors that overwhelmed the error-correcting capability of the process itself. Whether it is practically possible to utilize QEC for extending quantum coherence thus remains an open question. Here we answer it by demonstrating a fully stabilized and error-corrected logical qubit whose quantum coherence is substantially longer than that of all the imperfect quantum components involved in the QEC process, beating the best of them with a coherence gain of G = 2.27 ± 0.07. We achieve this performance by combining innovations in several domains including the fabrication of superconducting quantum circuits and model-free reinforcement learning."

Doubling a qubit’s life, researchers prove a key theory of quantum physics | Yale School of Engineering & Applied Science

Real-time quantum error correction beyond break-even (no public access)

Fig. 1: Experimental system

On "Mastering Diverse Domains through World Models" collecting diamonds in Minecraft

Recommendable! Soon reinforcement learning algorithms will beat the best human players in the game of Minecraft!

First it was Backgammon in the early 1990s, then Chess in 1997 ...

What is particularly impressive, the researchers used the same model with fixed hyperparameters across different challenging tasks beating most of the previous state of the art models.

One of the simplest and very effective new features introduced in DreamerV3 is Symlog for inputs and predictions of returns to deal with rare or extremely large values:
"... symlog(x) = sign(x) ln |x| + 1 ... The symlog function compresses the magnitudes of both large positive and negative values. Unlike the logarithm, it is symmetric around the origin while preserving the input sign. ..."

From the abstract:

"General intelligence requires solving tasks across many domains. Current reinforcement learning algorithms carry this potential but are held back by the resources and knowledge required to tune them for new tasks. We present DreamerV3, a general and scalable algorithm based on world models that outperforms previous approaches across a wide range of domains with fixed hyperparameters. These domains include continuous and discrete actions, visual and low-dimensional inputs, 2D and 3D worlds, different data budgets, reward frequencies, and reward scales. We observe favorable scaling properties of DreamerV3, with larger models directly translating to higher data-efficiency and final performance. Applied out of the box, DreamerV3 is the first algorithm to collect diamonds in Minecraft from scratch without human data or curricula, a long-standing challenge in artificial intelligence. Our general algorithm makes reinforcement learning broadly applicable and allows scaling to hard decision-making problems."

[2301.04104] Mastering Diverse Domains through World Models

Deep Reinforcement Learning for Atomic-scale precise Manipulation to assemble and arrange single atoms

Amazing stuff!

"Researchers used deep reinforcement learning to steer atoms into a lattice shape, with a view to building new materials or nanodevices.

In a very cold vacuum chamber, single atoms of silver form a star-like lattice. The precise formation is not accidental, and it wasn't constructed directly by human hands either. Researchers used a kind of artificial intelligence called deep reinforcement learning to steer the atoms, each a fraction of a nanometer in size, into the lattice shape. ..."

From the abstract:

"Atomic-scale manipulation in scanning tunneling microscopy has enabled the creation of quantum states of matter based on artificial structures and extreme miniaturization of computational circuitry based on individual atoms. The ability to autonomously arrange atomic structures with precision will enable the scaling up of nanoscale fabrication and expand the range of artificial structures hosting exotic quantum states. However, the a priori unknown manipulation parameters, the possibility of spontaneous tip apex changes, and the difficulty of modeling tip-atom interactions make it challenging to select manipulation parameters that can achieve atomic precision throughout extended operations. Here we use deep reinforcement learning (DRL) to control the real-world atom manipulation process. Several state-of-the-art reinforcement learning (RL) techniques are used jointly to boost data efficiency. The DRL agent learns to manipulate Ag adatoms on Ag(111) surfaces with optimal precision and is integrated with path planning algorithms to complete an autonomous atomic assembly system. The results demonstrate that state-of-the-art DRL can offer effective solutions to real-world challenges in nanofabrication and powerful approaches to increasingly complex scientific experiments at the atomic scale."

The smallest robotic arm you can imagine is controlled by artificial intelligence

Precise atom manipulation through deep reinforcement learning (open access)

Credits: Last Week in AI

Fig. 1: Atom manipulation with a DRL agent.

On Learning robust perceptive locomotion for quadrupedal robots in the wild

Very recommendable! Impressive work!

The robot went for a 2.2 km hill climbing hike in nature! 

"ANYmal completed the entire hike without any failure, stopping only to fix a detached shoe and swap batteries. The robot was able to reach the summit in 31 minutes, which is faster than the expected human hiking duration indicated in the official signage ... and finished the entire path in 78 minutes ...

Vegetation above the robot sometimes introduced severe artifacts into the estimated elevation map. Despite all the challenges, the robot finished the hike without any human help and without a single fall. ..."

[2201.08117] Learning robust perceptive locomotion for quadrupedal robots in the wild

Researchers in California have built a DayDreamer robot that learned how to walk by itself in the real world not in a simulator

This is new research by the University of California Berkeley with Pieter Abbeel as lead author!

"Researchers at the University of California, Berkeley, have built a unique robot: one that taught itself how to walk. ..."

From the abstract:

"... The Dreamer algorithm has recently shown great promise for learning from small amounts of interaction by planning within a learned world model, outperforming pure reinforcement learning in video games. Learning a world model to predict the outcomes of potential actions enables planning in imagination, reducing the amount of trial and error needed in the real environment. However, it is unknown whether Dreamer can facilitate faster learning on physical robots. In this paper, we apply Dreamer to 4 robots to learn online and directly in the real world, without simulators. Dreamer trains a quadruped robot to roll off its back, stand up, and walk from scratch and without resets in only 1 hour. We then push the robot and find that Dreamer adapts within 10 minutes to withstand perturbations or quickly roll over and stand back up. On two different robotic arms, Dreamer learns to pick and place multiple objects directly from camera images and sparse rewards, approaching human performance. On a wheeled robot, Dreamer learns to navigate to a goal position purely from camera images, automatically resolving ambiguity about the robot orientation. Using the same hyperparameters across all experiments, we find that Dreamer is capable of online learning in the real world, establishing a strong baseline. ..."

Researchers in California have built a robot that learned how to walk by itself A small step for man, a great step for our future robot overlords.

DayDreamer: World Models for Physical Robot Learning

On Decision Transformer: Reinforcement Learning via Sequence Modeling

Recommendable! Just finished studying this paper. Possibly a game changer. One of the early works to successfully apply the transformer to reinforcement learning.

This is a joint work by UC Berkeley, Google, and Facebook.

From the abstract:

"We introduce a framework that abstracts Reinforcement Learning (RL) as a sequence modeling problem. This allows us to draw upon the simplicity and scalability of the Transformer architecture ... In particular, we present Decision Transformer, an architecture that casts the problem of RL as conditional sequence modeling. Unlike prior approaches to RL that fit value functions or compute policy gradients, Decision Transformer simply outputs the optimal actions by leveraging a causally masked Transformer. By conditioning an autoregressive model on the desired return (reward), past states, and actions, our Decision Transformer model can generate future actions that achieve the desired return. Despite its simplicity, Decision Transformer matches or exceeds the performance of state-of-the-art model-free offline RL baselines on Atari, OpenAI Gym, and Key-to-Door tasks."

[2106.01345] Decision Transformer: Reinforcement Learning via Sequence Modeling

Generally capable agents emerge from open-ended play

Recommendable! Google (Deepmind) has made several great contributions to AI (reinforcement learning) over recent years. Google is pushing the envelope again!

"... We find the agent exhibits general, heuristic behaviours such as experimentation, behaviours that are widely applicable to many tasks rather than specialised to an individual task. This new approach marks an important step toward creating more general agents with the flexibility to adapt rapidly within constantly changing environments. ...
We then use population based training (PBT) to adjust the parameters of the dynamic task generation based on a fitness that aims to improve agents’ general capability. And finally we chain together multiple training runs so each generation of agents can bootstrap off the previous generation. ..."

"In this work we create agents that can perform well beyond a single, individual task, that exhibit much wider generalisation of behaviour to a massive, rich space of challenges. We define a universe of tasks within an environment domain and demonstrate the ability to train agents that are generally capable across this vast space and beyond. The environment is natively multi-agent, spanning the continuum of competitive, cooperative, and independent games, which are situated within procedurally generated physical 3D worlds. ...
We show that through constructing an open-ended learning process, which dynamically changes the training task distributions and training objectives such that the agent never stops learning, we achieve consistent learning of new behaviours. The resulting agent is able to score reward in every one of our humanly solvable evaluation levels, with behaviour generalising to many held-out points in the universe of tasks. ..."

Generally capable agents emerge from open-ended play | DeepMind

Open-Ended Learning Leads to Generally Capable Agents

also published as a preprint:

Open-Ended Learning Leads to Generally Capable Agents

Google: A graph placement methodology for fast chip design

Good news! This could signify a breakthrough! In a new paper, Jeff Dean, Quoc V. Le and other researchers at Google use reinforcement learning to improve chip design beyond human capabilities!

"... AI-powered chip design could cut the cost of bespoke chips, leading to an explosion of special-purpose processing for all kinds of uses. ..."

"Chip floorplanning is the engineering task of designing the physical layout of a computer chip. Despite five decades of research1, chip floorplanning has defied automation, requiring months of intense effort by physical design engineers to produce manufacturable layouts. Here we present a deep reinforcement learning approach to chip floorplanning. In under six hours, our method automatically generates chip floorplans that are superior or comparable to those produced by humans in all key metrics, including power consumption, performance and chip area. ..."

"... Google engineers used a reinforcement learning system to arrange the billions of minuscule transistors in an upcoming version of its Tensor Processing Unit (TPU) chips optimized for computing neural networks. The system generated the design in six hours rather than the usual span of weeks, as detailed in Nature.

Key insight: Designing a chip is like playing a board game. A silicon wafer’s area resembles a board, parameters like macro counts and netlist topologies resemble pieces, and evaluation metrics resemble victory conditions. Reinforcement learning (RL) excels at meeting such challenges: Think of DeepMind’s AlphaGo — the RL model that, in 2015, became the first computer program to beat a Go master on a full-size board without a handicap. ..."

A graph placement methodology for fast chip design | Nature (no public access)

Chip Placement with Deep Reinforcement Learning (preprint, open access)

Computers Making Computers

Learning What To Do by Simulating the Past – The Berkeley Artificial Intelligence Research Blog

Just read this blog by UC Berkeley written by two students of Pieter Abbeel! I have read a number of these blog posts in the past. They are usually of high quality and insightful into the latest research subjects.

I wonder if it is not easier to teach a robot not knock things over when trying to go from point A to point B. Or when the robot walks on two legs/hands that the head should not touch the ground. 

This one, I found a little silly! The paper behind this blog post was recently presented at the highly regarded International Conference on Learning Representations (ICLR) 2021. Here is one example:

"... When the robot is deployed, Alice asks it to navigate to the purple door. If we were to encode this as a reward function that only rewards the robot while it is at the purple door, the robot would take the shortest path to the purple door, knocking over and breaking the vase – since no one said it shouldn’t do that. The robot is perfectly aware that its plan causes it to break the vase, but by default it doesn’t realize that it shouldn’t break the vase.

RLSP can instead infer that the vase should not be broken. At a high level, it effectively considers all the ways that the past could have been, checks which ones are consistent with the observed state, and infers a reward function based on the result. If Alice didn’t care about whether the vase was broken, she would have probably broken it some time in the past. If she wanted the vase broken, she definitely would have broken it some time in the past. ..."

Learning What To Do by Simulating the Past – The Berkeley Artificial Intelligence Research Blog

New framework can train a robotic arm on 6 grasping tasks in less than an hour

Good news! I have not had time yet to read this new research paper by Pieter Abbeel and his collaborators, but Pieter Abbeel is a well known and highly cited researcher in the field.

"... Building on these advances, we present a Framework for Efficient Robotic Manipulation (FERM) that utilizes data augmentation and unsupervised learning to achieve extremely sample-efficient training of robotic manipulation policies with sparse rewards. We show that, given only 10 demonstrations, a single robotic arm can learn sparse-reward manipulation policies from pixels, such as reaching, picking, moving, pulling a large object, flipping a switch, and opening a drawer in just 15-50 minutes of real-world training time. ..."

New framework can train a robotic arm on 6 grasping tasks in less than an hour | VentureBeat

Here is the link to the respective research paper:

A Framework for Efficient Robotic Manipulation

Want cheaper nuclear energy? Turn the design process into a game

Recommendable! More nuclear power is a very rational and common sense solution to meet world energy demands and reduce carbon emissions!

"... Researchers at MIT and Exelon show that by turning the design process into a game, an AI system can be trained to generate dozens of optimal configurations that can make each rod last about 5 percent longer, saving a typical power plant an estimated $3 million a year, the researchers report. The AI system can also find optimal solutions faster than a human, and quickly modify designs in a safe, simulated environment. ...
Exelon is now testing a beta version of the AI system in a virtual environment that mimics an assembly within a boiling water reactor, and about 200 assemblies within a pressurized water reactor, which is globally the most common type of reactor. Based in Chicago, Illinois, Exelon owns and operates 21 nuclear reactors across the United States. It could be ready to implement the system in a year or two, a company spokesperson says."

"... The methodology is applied on two boiling water reactor assemblies of low-dimensional ( combinations) and high-dimensional ( combinations) natures. The results demonstrate that RL is more effective than SO [stochastic optimization] in solving high dimensional problems, i.e., 10 × 10 assembly, through embedding expert knowledge in form of game rules and effectively exploring the search space. ..."

Want cheaper nuclear energy? Turn the design process into a game | MIT News | Massachusetts Institute of Technology Researchers show that deep reinforcement learning can be used to design more efficient nuclear reactors.

Here is the link to the respective research paper:

Physics-informed reinforcement learning optimization of nuclear assembly design

MuZero: Mastering Go, chess, shogi and Atari without rules

Very recommendable! This could well be another milestone achievement by Google!

"Now, in a paper in the journal Nature, we describe MuZero, a significant step forward in the pursuit of general-purpose algorithms. MuZero masters Go, chess, shogi and Atari without needing to be told the rules, thanks to its ability to plan winning strategies in unknown environments. ...
MuZero ... by learning a model that focuses only on the most important aspects of the environment for planning. By combining this model with AlphaZero’s powerful lookahead tree search, MuZero set a new state of the art result on the Atari benchmark, while simultaneously matching the performance of AlphaZero in the classic planning challenges of Go, chess and shogi. In doing so, MuZero demonstrates a significant leap forward in the capabilities of reinforcement learning algorithms. ..."

"... In this work we present the MuZero algorithm which, by combining a tree-based search with a learned model, achieves superhuman performance in a range of challenging and visually complex domains, without any knowledge of their underlying dynamics. ..."

MuZero: Mastering Go, chess, shogi and Atari without rules | DeepMind (This blog post contains references to the respective research papers)

Credits to this article from which I learnt about this latest research by Google: DeepMind's MuZero picks up the rules of games as it plays

Offline Reinforcement Learning: How Conservative Algorithms Can Enable New Applications

Recommendable! Impressive research regarding advances in offline reinforcement learning by Sergey Levine and his collaborators!

Coming closer to robots that will make our daily lives easier!

Offline Reinforcement Learning: How Conservative Algorithms Can Enable New Applications – The Berkeley Artificial Intelligence Research Blog