91Ö±²¥

Many real-world decision-making problems rely on large amounts of data but only a small proportion of that data is fully labelled. This paper introduces a new reinforcement learning method that can learn effectively from both labelled (with rewards) and unlabelled data, making it better suited to real-world applications where collecting reward information is expensive or impractical, including healthcare.

Reinforcement learning (RL) is a powerful paradigm for policy optimisation in sequential decision making, it is used to train an intelligent agent (policy) to make optimal decisions by maximising some long-term reward. While online RL learns a policy by actively interacting with an environment and collecting new data through exploration and exploitation, offline RL learns a policy from a fixed, pre-collected dataset without further interaction with the environment. A typical challenge in offline RL is distributional shift, where the learned policy may encounter scenarios not covered by the offline data. This paper tackles distributional shift and integration of labelled (with rewards) data and unlabelled (without rewards) data in offline RL.

The proposed method, called semi-pessimistic pseudo labelling (SPL), makes it possible to take advantage of the unlabelled data to find a better policy than would be possible only using the labelled data. This improves the reliability of decision-making by improving estimation and accounting for uncertainty while remaining computationally efficient. The authors provide theoretical guarantees for the method and demonstrate its effectiveness through simulations, standard reinforcement learning benchmarks, and an application to adaptive deep brain stimulation for Parkinson's disease. By making better use of available data, the approach has the potential to improve AI systems in healthcare and other high-stakes domains.

You can find the full ‘Semi-pessimistic Reinforcement Learning’ paper here.

Jin Zhu^1,2, Xin Zhou³, Jiaang Yao⁴, Gholamali Aminian⁵, Omar Rivasplata⁶, Simon Little⁴, Lexin Li³âˆ—, Chengchun Shi¹âˆ—
¹London School of Economics and Political Science, ²University of Birmingham, ³University of California at Berkeley, ⁴University of California at San Francesco, ⁵The Alan Turing Institute, ⁶University of Manchester. *Corresponding authors.