Read more
Informationen zum Autor Mykel J. Kochenderfer, Tim A. Wheeler, and Kyle H. Wray Klappentext A broad introduction to algorithms for decision making under uncertainty, introducing the underlying mathematical problem formulations and the algorithms for solving them. Automated decision-making systems or decision-support systems—used in applications that range from aircraft collision avoidance to breast cancer screening—must be designed to account for various sources of uncertainty while carefully balancing multiple objectives. This textbook provides a broad introduction to algorithms for decision making under uncertainty, covering the underlying mathematical problem formulations and the algorithms for solving them. The book first addresses the problem of reasoning about uncertainty and objectives in simple decisions at a single point in time, and then turns to sequential decision problems in stochastic environments where the outcomes of our actions are uncertain. It goes on to address model uncertainty, when we do not start with a known model and must learn how to act through interaction with the environment; state uncertainty, in which we do not know the current state of the environment due to imperfect perceptual information; and decision contexts involving multiple agents. The book focuses primarily on planning and reinforcement learning, although some of the techniques presented draw on elements of supervised learning and optimization. Algorithms are implemented in the Julia programming language. Figures, examples, and exercises convey the intuition behind the various approaches presented. Zusammenfassung A broad introduction to algorithms for decision making under uncertainty, introducing the underlying mathematical problem formulations and the algorithms for solving them. Automated decision-making systems or decision-support systems—used in applications that range from aircraft collision avoidance to breast cancer screening—must be designed to account for various sources of uncertainty while carefully balancing multiple objectives. This textbook provides a broad introduction to algorithms for decision making under uncertainty, covering the underlying mathematical problem formulations and the algorithms for solving them. The book first addresses the problem of reasoning about uncertainty and objectives in simple decisions at a single point in time, and then turns to sequential decision problems in stochastic environments where the outcomes of our actions are uncertain. It goes on to address model uncertainty, when we do not start with a known model and must learn how to act through interaction with the environment; state uncertainty, in which we do not know the current state of the environment due to imperfect perceptual information; and decision contexts involving multiple agents. The book focuses primarily on planning and reinforcement learning, although some of the techniques presented draw on elements of supervised learning and optimization. Algorithms are implemented in the Julia programming language. Figures, examples, and exercises convey the intuition behind the various approaches presented. Inhaltsverzeichnis Preface xix Acknowledgments xxi 1 Introduction 1 Part I Probabilistic Reasoning 2 Representation 19 3 Inference 43 4 Parameter Learning 71 5 Structure Learning 97 6 Simple Decisions 111 Part II Sequential Problems 7 Exact Solution Methods 133 8 Approximate Value Functions 161 9 Online Planning 181 10 Policy Search 213 11 Policy Gradient Estimation 231 12 Policy Gradient Optimization 249 13 Actor-Critic Methods 267 14 Policy Validation 281 Part III Model Uncertainty 15 Exploration and Exploitation 299 16 Model-Based Methods 317 17 Model-Free Methods 335 18 Imitation Learning 335 Part IV State Uncertainty 19 Beliefs 379 20 Exact Belief State Planning 407
