 Название: Multi-Agent Reinforcement Learning: Foundations and Modern Approaches
Автор: Stefano V. Albrecht, Filippos Christianos, Lukas Schäfer
Издательство: The MIT Press
Год: December 17, 2024
Страниц: 396
Язык: английский
Формат: epub (true)
Размер: 14.3 MB
The first comprehensive introduction to Multi-Agent Reinforcement Learning (MARL), covering MARL’s models, solution concepts, algorithmic ideas, technical challenges, and modern approaches.
Multi-Agent Reinforcement Learning (MARL), an area of machine learning in which a collective of agents learn to optimally interact in a shared environment, boasts a growing array of applications in modern life, from autonomous driving and multi-robot factories to automated trading and energy network management. This text provides a lucid and rigorous introduction to the models, solution concepts, algorithmic ideas, technical challenges, and modern approaches in MARL. The book first introduces the field’s foundations, including basics of reinforcement learning theory and algorithms, interactive game models, different solution concepts for games, and the algorithmic ideas underpinning MARL research. It then details contemporary MARL algorithms which leverage Deep Learning techniques, covering ideas such as centralized training with decentralized execution, value decomposition, parameter sharing, and self-play. The book comes with its own MARL codebase written in Python, containing implementations of MARL algorithms that are self-contained and easy to read. Technical content is explained in easy-to-understand language and illustrated with extensive examples, illuminating MARL for newcomers while offering high-level insights for more advanced readers.
This book assumes that readers have an undergraduate-level background in basic mathematics, including statistics, probability theory, linear algebra, and calculus. A basic familiarity with programming concepts is required to understand and use the codebase. In general, we recommend reading the book chapters in the given sequence. For readers unfamiliar with reinforcement learning and deep learning, we provide the basics in Chapters 2, 7 and 8, respectively. Readers who are already familiar with reinforcement learning and deep (reinforcement) learning, and who want to quickly get going with deep learning-based MARL algorithms, may read Chapter 3 and then skip to Chapter 9 and onward. To aid lecturers in adopting this book, we have developed lecture slides (available from the book’s website) that can be modified as required to suit the course’s needs.
MARL has become a large field of research, and this book does not cover all aspects of MARL. For instance, there is a growing body of work on using communication in MARL, which is not covered in this book. This includes questions such as how agents can learn to communicate robustly when communication channels are noisy and unreliable; and how agents may use MARL to learn specialized communication protocols or languages for a given task. While this book does not focus on communication in MARL, the models introduced in this book are general enough to also represent communication actions. There has also been research on using evolutionary game theory for multi-agent learning, which is not covered in this book. Finally, with the steep rise of activity in MARL in recent years, it would be futile to write a book that tries to keep up with new algorithms. We instead focus on the foundational concepts and ideas in MARL, and refer to survey papers for a more complete list of algorithm developments.
One of the goals of this book is to provide a starting point for readers who want to use the MARL algorithms discussed in this book in practice, as well as develop their own algorithms. Thus, the book comes with its own MARL codebase (downloadable from the book’s website) that was developed in the Python programming language, providing implementations of many existing MARL algorithms that are self-contained and easy to read. Chapter 10 uses code snippets from the codebase to explain implementation details of the important concepts underlying the algorithms presented in the earlier chapters. We hope that the provided code will be useful to readers in understanding MARL algorithms as well as getting started with using them in practice.
First textbook to introduce the foundations and applications of MARL, written by experts in the field
Integrates reinforcement learning, Deep Learning, and game theory
Practical focus covers considerations for running experiments and describes environments for testing MARL algorithms
Explains complex concepts in clear and simple language
Classroom-tested, accessible approach suitable for graduate students and professionals across computer science, artificial intelligence, and robotics
Resources include code and slides
Contents:
Preface
1. Introduction
1.1. Multi-Agent Systems
1.2. Multi-Agent Reinforcement Learning
1.3. Application Examples
1.4. Challenges of MARL
1.5. Agendas of MARL
1.6. Book Contents and Structure
I. Foundations of Multi-Agent Reinforcement Learning
2. Reinforcement Learning
3. Games: Models of Multi-Agent Interaction
4. Solution Concepts for Games
5. Multi-Agent Reinforcement Learning in Games: First Steps and Challenges
6. Multi-Agent Reinforcement Learning: Foundational Algorithms
II. Multi-Agent Deep Reinforcement Learning: Algorithms and Practice
7. Deep Learning
8. Deep Reinforcement Learning
9. Multi-Agent Deep Reinforcement Learning
10. Multi-Agent Deep Reinforcement Learning in Practice
11. Multi-Agent Environments
A. Surveys on Multi-Agent Reinforcement Learning
References
Index
Скачать Multi-Agent Reinforcement Learning: Foundations and Modern Approaches
|
Навигация
Вход на сайт
Реклама
Multi-Agent Reinforcement Learning: Foundations and Modern Approaches 
