Barto, Sutton Co-Author Second Edition of Ground-Breaking Textbook on Reinforcement Learning

A second edition of the widely used textbook, "Reinforcement Learning: An Introduction" by College of Information and Computer Sciences (CICS) professor emeritus Andrew Barto and alumnus Richard S. Sutton (PhD '84) was released this month by MIT Press, which describes reinforcement learning as "one of the most active research areas in artificial intelligence."

The book has been "significantly expanded and updated," presenting new topics and updating coverage of others. The book is also available online via a Creative Commons license.

CICS Dean Laura Haas says, "In 1998, Andy and Rich wrote THE book on reinforcement learning. We are so pleased that their work, through this second edition, will find a new audience in the next generation of artificial intelligence researchers. We are proud to count Andy and Rich among our college's storied and pioneering community members."

Many consider Barto "the father of reinforcement learning," who with Sutton, his Ph.D. student at the time, pioneered the new approach on campus several decades ago. The textbook's first edition was considered the bible of reinforcement learning and has been cited more than 28,000 times, according to Google Scholar.

Barto says, "In light of the first edition's success and subsequent advancements in reinforcement learning, an expanded second edition of our book is long overdue." The book includes new chapters on reinforcement learning's relationships to psychology and neuroscience, an updated case-studies chapter including AlphaGo, AlphaGo Zero, Atari game playing and the IBM Watson wagering strategy. The final chapter discusses the future societal impacts of reinforcement learning.

Reinforcement learning is a type of machine learning that relies on trial and error, allowing a system to learn from the consequences of its decisions instead of trying to replicate the decisions of human experts.It has strong connections to psychology and neuroscience, and is contributing to some of the most striking recent developments in artificial intelligence, such as DeepMind's AlphaGo program that has defeated human masters of the very challenging game of Go.

Barto received a B.S. with distinction in mathematics from the University of Michigan in 1970 and his Ph.D. in computer science there in 1975. He joined the computer science department in 1977 and served as chair for four years before his retirement in 2012.

Sutton received a B.S. in psychology from Stanford University in 1978 and, as an advisee of Barto, his Ph.D. in computer science in 1984. He is now a professor of computing science and Alberta Innovates Technology Futures Chair in Reinforcement Learning and Artificial Intelligence at the University of Alberta, Canada. He also serves as a distinguished research scientist at Google DeepMind.