Plan, Activity, and Intent Recognition Theory and Practice pdf pdf
Plan, Activity, and Intent
Recognition This page is intentionally left blank
Contents
About the Editors ...................................................................................................................................xi
List of Contributors ............................................................................................................................. xiii
Preface .................................................................................................................................................xvii
Introduction ..........................................................................................................................................xix
PART 1 PLAN AND GOAL RECOGNITION CHAPTER 1 Hierarchical Goal Recognition .......................................................3
1.1 Introduction ..................................................................................................................3
1.2 Previous Work ..............................................................................................................5
1.3 Data for Plan Recognition ...........................................................................................6
1.4 Metrics for Plan Recognition .....................................................................................10
1.5 Hierarchical Goal Recognition ..................................................................................12
1.6 System Evaluation .....................................................................................................23
1.7 Conclusion .................................................................................................................30
Acknowledgments .....................................................................................................31
References ..................................................................................................................31
CHAPTER 2 Weighted Abduction for Discourse Processing Based on Integer Linear Programming ........................................................33
2.1 Introduction ................................................................................................................33
2.2 Related Work .............................................................................................................34
2.3 Weighted Abduction ..................................................................................................35
2.4 ILP-based Weighted Abduction .................................................................................36
2.5 Weighted Abduction for Plan Recognition ................................................................41
2.6 Weighted Abduction for Discourse Processing .........................................................43
2.7 Evaluation on Recognizing Textual Entailment.........................................................47
2.8 Conclusion .................................................................................................................51
Acknowledgments .....................................................................................................52
References ..................................................................................................................52
CHAPTER 3 Plan Recognition Using Statistical–Relational Models ..................57
3.1 Introduction ................................................................................................................57
3.2 Background ................................................................................................................59
3.3 Adapting Bayesian Logic Programs ..........................................................................61
3.4 Adapting Markov Logic .............................................................................................65
3.5 Experimental Evaluation ............................................................................................72
3.6 Future Work ...............................................................................................................81
3.7 Conclusion .................................................................................................................81
vi Contents
Acknowledgments .....................................................................................................82
References ..................................................................................................................82
CHAPTER 4 Keyhole Adversarial Plan Recognition for Recognition of Suspicious and Anomalous Behavior ........................................874.1 Introduction ................................................................................................................87
4.2 Background: Adversarial Plan Recognition ...............................................................88
4.3 An Efficient Hybrid System for Adversarial Plan Recognition .................................93
4.4 Experiments to Detect Anomalous and Suspicious Behavior ....................................99
4.5 Future Directions and Final Remarks ......................................................................115
Acknowledgments ...................................................................................................116
References ................................................................................................................116
PART 2 ACTIVITY DISCOVERY AND RECOGNITION CHAPTER 5 Stream Sequence Mining for Human Activity Discovery .....................................................................123
5.1 Introduction ..............................................................................................................123
5.2 Related Work ...........................................................................................................125
5.3 Proposed Model .......................................................................................................129
5.4 Experiments .............................................................................................................138
5.5 Conclusion ...............................................................................................................143
References ................................................................................................................144
CHAPTER 6 Learning Latent Activities from Social Signals with Hierarchical Dirichlet Processes ........................................149
6.1 Introduction ..............................................................................................................149
6.2 Related Work ...........................................................................................................150
6.3 Bayesian Nonparametric Approach to Inferring Latent Activities ..........................154
6.4 Experiments .............................................................................................................160
6.5 Conclusion ...............................................................................................................171
References ................................................................................................................172
PART 3 MODELING HUMAN COGNITION CHAPTER 7 Modeling Human Plan Recognition Using Bayesian Theory of Mind ...........................................................177
7.1 Introduction ..............................................................................................................177
7.2 Computational Framework ......................................................................................181
7.3 Comparing the Model to Human Judgments ...........................................................190
7.4 Discussion ................................................................................................................195
7.5 Conclusion ...............................................................................................................198
vii Contents
9.7 Conclusion ...............................................................................................................247
Acknowledgment .....................................................................................................248
References ................................................................................................................248
11.2 Proactive Assistant Agent ........................................................................................276
11.1 Introduction ..............................................................................................................275
PART 5 APPLICATIONS CHAPTER 11 Probabilistic Plan Recognition for Proactive Assistant Agents ........................................................275
10.7 Conclusion and Future Work ...................................................................................271
Acknowledgments ...................................................................................................272
References ................................................................................................................272
10.6 Model Evaluation .....................................................................................................263
10.5 Models for Choosing a Role ....................................................................................258
10.4 Importance of Role Recognition ..............................................................................257
10.3 Problem Definition ...................................................................................................255
10.2 Related Work ...........................................................................................................252
10.1 Introduction ..............................................................................................................251
CHAPTER 10 Role-Based Ad Hoc Teamwork ...................................................251
9.6 Related Work ...........................................................................................................246
CHAPTER 8 Decision-Theoretic Planning in Multiagent Settings with Application to Behavioral Modeling ....................................205
9.5 Experiment ...............................................................................................................241
9.4 Multiagent Plan Recognition with Action Models ..................................................235
9.3 Multiagent Plan Recognition with Plan Library ......................................................230
9.2 Preliminaries ............................................................................................................228
9.1 Introduction ..............................................................................................................227
PART 4 MULTIAGENT SYSTEMS CHAPTER 9 Multiagent Plan Recognition from Partially Observed Team Traces ..............................................................227
8.5 Conclusion ...............................................................................................................222
Acknowledgments ...................................................................................................222
References ................................................................................................................222
8.4 Discussion ................................................................................................................221
8.3 Modeling Deep, Strategic Reasoning by Humans Using I-POMDPs .....................210
8.2 The Interactive POMDP Framework .......................................................................206
8.1 Introduction ..............................................................................................................205
11.3 Probabilistic Plan Recognition ................................................................................277
viii Contents
11.5 Applications .............................................................................................................284
11.6 Conclusion ...............................................................................................................286
Acknowledgment .....................................................................................................287
References ................................................................................................................287
CHAPTER 12 Recognizing Player Goals in Open-Ended Digital Games with Markov Logic Networks ..........................................289
12.1 Introduction ..............................................................................................................289
12.2 Related Work ...........................................................................................................291
12.3 Observation Corpus .................................................................................................293
12.4 Markov Logic Networks ..........................................................................................298
12.5 Goal Recognition with Markov Logic Networks ....................................................300
12.6 Evaluation ................................................................................................................303
12.7 Discussion ................................................................................................................306
12.8 Conclusion and Future Work ...................................................................................309
Acknowledgments ...................................................................................................309
References ................................................................................................................309
CHAPTER 13 Using Opponent Modeling to Adapt Team Play in American Football .........................................................313
13.1 Introduction ..............................................................................................................313
13.2 Related Work ...........................................................................................................315
13.3 Rush Football ...........................................................................................................317
13.4 Play Recognition Using Support Vector Machines ..................................................319
13.5 Team Coordination ..................................................................................................321
13.6 Offline UCT for Learning Football Plays ................................................................326
13.7 Online UCT for Multiagent Action Selection ..........................................................330
13.8 Conclusion ...............................................................................................................339
Acknowledgment .....................................................................................................339
References ................................................................................................................339
CHAPTER 14 Intent Recognition for Human–Robot Interaction .........................343
14.1 Introduction ..............................................................................................................343
14.2 Previous Work in Intent Recognition .......................................................................344
14.3 Intent Recognition in Human–Robot Interaction ....................................................345
14.4 HMM-Based Intent Recognition .............................................................................348
14.5 Contextual Modeling and Intent Recognition ..........................................................349
14.6 Experiments on Physical Robots .............................................................................356
ix Contents
14.7 Discussion ................................................................................................................363
14.8 Conclusion ...............................................................................................................364
References ................................................................................................................364
Author Index .................................................................................................367
Subject Index ...............................................................................................379
This page is intentionally left blank
About the Editors
Dr. Gita Sukthankar is an Associate Professor and Charles N. Millican Faculty Fellow in the
Department of Electrical Engineering and Computer Science at the University of Central Florida, and
an affiliate faculty member at UCF’s Institute for Simulation and Training. She received her Ph.D.
from the Robotics Institute at Carnegie Mellon, where she researched multiagent plan recognition algo-
rithms. In 2009, Dr. Sukthankar was selected for an Air Force Young Investigator award, the DARPA
Computer Science Study Panel, and an NSF CAREER award. Gita Sukthankar’s research focuses on
multiagent systems and computational social models.
Robert P. Goldman is a Staff Scientist at SIFT, LLC, specializing in Artificial Intelligence. Dr.
Goldman received his Ph.D. in Computer Science from Brown University, where he worked on the first
Bayesian model for plan recognition. Prior to joining SIFT, he was an Assistant Professor of computer
science at Tulane University, and then Principal Research Scientist at Honeywell Labs. Dr. Goldman’s
research interests involve plan recognition; the intersection between planning, control theory, and
formal methods; computer security; and reasoning under uncertainty.
Christopher Geib is an Associate Professor in the College of Computing and Informatics at Drexel
University. Before joining Drexel, Professor Geib’s career has spanned a number of academic and
industrial posts including being a Research Fellow in the School of Informatics at the University of
Edinburgh, a Principal Research Scientist working at Honeywell Labs, and a Postdoctoral Fellow at
the University of British Columbia in the Laboratory for Computational Intelligence. He received his
Ph.D. in computer science from the University of Pennsylvania and has worked on plan recognition and
planning for more than 20 years.
Dr. David V. Pynadath is a Research Scientist at the University of Southern California’s Institute for
Creative Technologies. He received his Ph.D. in computer science from the University of Michigan
in Ann Arbor, where he studied probabilistic grammars for plan recognition. He was subsequently a
Research Scientist at the USC Information Sciences Institute and is currently a member of the Social
Simulation Lab at USC ICT, where he conducts research in multiagent decision–theoretic methods for
social reasoning.
Dr. Hung Hai Bui is a Principal Research Scientist at the Laboratory for Natural Language
Understanding, Nuance in Sunnyvale, CA. His main research interests include probabilistic reasoning
and machine learning and their application in plan and activity recognition. Before joining Nuance,
he spent nine years as a Senior Computer Scientist at SRI International, where he led several multi-
institutional research teams developing probabilistic inference technologies for understanding human
activities and building personal intelligent assistants. He received his Ph.D. in computer science in 1998 from Curtin University in Western Australia. This page is intentionally left blank
List of Contributors Noa Agmon Bar Ilan University, Ramat Gan, Israel James Allen Florida Institute for Human and Machine Cognition, Pensacola, FL, USA Amol Ambardekar University of Nevada, Reno, NV, USA Dorit Avrahami-Zilberbrand Bar Ilan University, Ramat Gan, Israel Chris L. Baker Massachusetts Institute of Technology, Cambridge, MA, USA Nate Blaylock Nuance Communications, Montreal, QC, Canada Prashant Doshi University of Georgia, Athens, GA, USA Katie Genter University of Texas at Austin, Austin, TX, USA Adam Goodie University of Georgia, Athens, GA, USA Sunil Gupta Deakin University, Waurn Ponds, VIC, Australia Eun Y. Ha North Carolina State University, Raleigh, NC, USA Jerry Hobbs USC/ISI, Marina del Rey, CA, USA Naoya Inoue Tohoku University, Sendai, Japan Kentaro Inui Tohoku University, Sendai, Japan Gal A. Kaminka Bar Ilan University, Ramat Gan, Israel Richard Kelley University of Nevada, Reno, NV, USA Christopher King University of Nevada, Reno, NV, USA
xiv List of Contributors
Kennard R. Laviers Air Force Institute of Technology, Wright Patterson AFB, OH, USA James C. Lester North Carolina State University, Raleigh, NC, USA Felipe Meneguzzi
Pontifical Catholic University of Rio Grande do Sul, Porto Alegre, Brazil
Raymond J. Mooney University of Texas at Austin, Austin, TX, USA Bradford W. Mott North Carolina State University, Raleigh, NC, USA Thuong Nguyen Deakin University, Waurn Ponds, VIC, Australia Mircea Nicolescu University of Nevada, Reno, NV, USA Monica Nicolescu University of Nevada, Reno, NV, USA Jean Oh Carnegie Mellon University, Pittsburgh, PA, USA Ekaterina Ovchinnikova USC/ISI, Marina del Rey, CA, USA Dinh Phung Deakin University, Waurn Ponds, VIC, Australia Xia Qu University of Georgia, Athens, GA, USA Sindhu Raghavan University of Texas at Austin, Austin, TX, USA Parisa Rashidi University of Florida, Gainesville, FL, USA Jonathan P. Rowe North Carolina State University, Raleigh, NC, USA Parag Singla Indian Institute of Technology Delhi, Hauz Khas, DL, India Peter Stone University of Texas at Austin, Austin, TX, USA Gita Sukthankar University of Central Florida, Orlando, FL, USAxv List of Contributors
Katia Sycara Carnegie Mellon University, Pittsburgh, PA, USA Alireza Tavakkoli University of Nevada, Reno, NV, USA Joshua B. Tenenbaum Massachusetts Institute of Technology, Cambridge, MA, USA Svetha Venkatesh Deakin University, Waurn Ponds, VIC, Australia Liesl Wigand University of Nevada, Reno, NV, USA Hankz Hankui Zhuo Sun Yat-sen University, Guangzhou, China This page is intentionally left blank
Preface
The diversity of applications and disciplines encompassed by the subfi eld of plan, intent, and activity
recognition, while producing a wealth of ideas and results, has unfortunately contributed to fragmen-
tation in the area because researchers present relevant results in a broad spectrum of journals and at
conferences. This book serves to provide a coherent snapshot of the exciting developments in the fi eld
enabled by improved sensors, increased computational power, and new application areas. While the
individual chapters are motivated by different applications and employ diverse technical approaches,
they are unifi ed by the ultimate task of understanding another agent’s behaviors.As there is not yet a single common conference for this growing fi eld, we hope that this book will
serve as a valuable resource for researchers interested in learning about work originating from other
communities. The editors have organized workshops in this topic area at the following artifi cial intel-
ligence conferences since 2004:- Modeling Other Agents From Observations (MOO 2004) at the International Conference on
Autonomous Agents and Multi-agent Systems, AAMAS-2004, organized by Gal Kaminka, Piotr
Gmytrasiewicz, David Pynadath, and Mathias Bauer• Modeling Other Agents From Observations (MOO 2005) at the International Joint Conference on
Artifi cial Intelligence, IJCAI-2005, organized by Gal Kaminka, David Pynadath, and Christopher Geib• Modeling Other Agents From Observations (MOO 2006) at the National Conference on Artifi cial
Intelligence, AAAI-2006, organized by Gal Kaminka, David Pynadath, and Christopher Geib
• Plan, Activity, and Intent Recognition (PAIR 2007) at the National Conference on Artifi cial Intelligence, AAAI-2007, organized by Christopher Geib and David Pynadath • Plan, Activity, and Intent Recognition (PAIR 2009) at the International Joint Conference onArtifi cial Intelligence, IJCAI-2009, organized by Christopher Geib, David Pynadath, Hung Bui,
and Gita Sukthankar
- Plan, Activity, and Intent Recognition (PAIR 2010) at the National Conference on Artifi cial
Intelligence, AAAI-2010, organized by Gita Sukthankar, Christopher Geib, David Pynadath, and
Hung Bui • Plan, Activity, and Intent Recognition (PAIR 2011) at the National Conference on Artifi cial Intelligence, AAAI-2011, organized by Gita Sukthankar, Hung Bui, Christopher Geib, and David Pynadath • Dagstuhl Seminar on Plan Recognition in Dagstuhl, Germany, organized by Tanim Asfour, Christopher Geib, Robert Goldman, and Henry Kautz • Plan, Activity, and Intent Recognition (PAIR 2013) at the National Conference on Artifi cial Intelligence, AAAI-2013, organized by Hung Bui, Gita Sukthankar, Christopher Geib, and David
Pynadath The editors and many of the authors gathered together at the 2013 PAIR workshop to put the fi n-
ishing touches on this book, which contains some of the best contributions from the community. We
thank all of the people who have participated in these events over the years for their interesting research
presentations, exciting intellectual discussions, and great workshop ). xviixviii Preface
FIGURE P.1
Tag cloud created from the titles of papers that have appeared at the workshops in this series.
Introduction Overview
The ability to recognize the plans and goals of other agents enables humans to reason about what other people are doing, why they are doing it, and what they will do next. This fundamental cognitive capability is also critical to interpersonal interactions because human communications presuppose an ability to understand the motivations of the participants and subjects of the discussion. As the complexity of human–machine interactions increases and automated systems become more intelligent, we strive to provide computers with comparable intent-recognition capabilities.
Research addressing this area is variously referred to as plan recognition, activity recognition, goal recognition, and intent recognition. This synergistic research area combines techniques from user modeling, computer vision, natural language understanding, probabilistic reasoning, and machine learning. Plan-recognition algorithms play a crucial role in a wide variety of applications including smart homes, intelligent user interfaces, personal agent assistants, human–robot interaction, and video surveillance.
Plan-recognition research in computer science dates back at least 35 years; it was initially defined in a paper by Schmidt, Sridharan, and Goodson
In the last ten years, significant advances have
been made on this subject by researchers in artificial intelligence (AI) and related areas. These advances have been driven by three primary factors: (1) the pressing need for sophisticated and efficient plan- recognition systems for a wide variety of applications; (2) the development of new algorithmic techniques in probabilistic modeling, machine learning, and optimization (combined with more pow- erful computers to use these techniques); and (3) our increased ability to gather data about human activities.
Recent research in the field is often divided into two subareas. Activity recognition focuses on the problem of dealing directly with noisy low-level data gathered by physical sensors such as cameras, wearable sensors, and instrumented user interfaces. The primary task in this space is to discover and extract interesting patterns in noisy sensory data that can be interpreted as meaningful activities. For example, an activity-recognition system processing a sequence of video frames might start by extracting a series of motions and then will attempt to verify that they are all part of the activity of filling a tea kettle. Plan and intent recognition concentrates on identifying high-level complex goals and intents by exploiting relationships between primitive action steps that are elements of the plan. Relationships that have been investigated include causality, temporal ordering, coordination among multiple subplans (possibly involving multiple actors), and social convention.
xix
xx Introduction A Brief History
The earliest work in plan recognition was rule based
following the dominant early paradigm
in artificial intelligence. Researchers attempted to create inference rules that would capture the nature of plan recognition. Over time, it became clear that without an underlying theory to give them structure and coherence, such rule sets are difficult to maintain and do not scale well.
In 1986, Kautz and Allen published an article, Generalized Plan Recognition
that has provided
the conceptual framework for much of the work in plan recognition to date. They defined the problem of plan recognition as identifying a minimal set of top-level actions sufficient to explain the set of observed actions. Plans were represented in a plan graph, with top-level actions as root nodes and expansions of these actions as unordered sets of child actions representing plan decomposition.
To a first approximation, the problem of plan recognition was then one of graph covering. Kautz and Allen formalized this view of plan recognition in terms of McCarthy’s circumscription. Kautz
presented an approximate implementation of this approach that recast the problem as one of computing vertex covers of the plan graph. These early techniques are not able to take into account differences in the a priori likelihood of different goals. Observing an agent going to the airport, this algorithm views “air travel” and “terrorist attack” as equally likely explanations because they explain (cover) the observations equally well.
To the best of our knowledge, Charniak was the first to argue that plan recognition was best understood as a specific case of the general problem of abduction
Abduction, a term originally defined by the
philosopher C. S. Peirce, is reasoning to the best explanation: the general pattern being “if A causes B and we observe B, we may postulate A as the explanation.” In the case of plan recognition, this pattern is specialized to “if an agent pursuing plan/goal P would perform the sequence of actions S and we observe
S , we may postulate that the agent is executing plan P.” Understanding plan recognition as a form of
abductive reasoning is important to the development of the field because it enables clear computational formulations and facilitates connections to areas such as diagnosis and probabilistic inference.
One of the earliest explicitly abductive approaches to plan recognition was that of Hobbs et al. In this work, they defined a method for abduction as a process of cost-limited theorem-proving They used this cost-based theorem-proving to find “proofs” for the elements of a narrative, where the assumptions underlying these proofs constitute the interpretation of the narrative—in much the same way a medical diagnosis system would “prove” the set of symptoms in the process identifying the underlying disease. Later developments would show that this kind of theorem-proving is equivalent to a form of probabilistic reasoning
Charniak and Goldman
argued that if plan recognition is a problem of abduction, it can best
be done as Bayesian (probabilistic) inference. Bayesian inference supports the preference for minimal explanations in the case of equally likely hypotheses, but it also correctly handles explanations of the same complexity but different likelihoods. For example, if a set of observations could be equally well explained by three hypotheses—going to the store to shop and to shoplift, being one, and going to the store only to shop or going to the store only to shoplift being the others—simple probability theory (with some minor assumptions) will tell us that the simpler hypotheses are more likely. On the other hand, if as in the preceding, the two hypotheses were “air travel” and “terrorist attack,” and each explained the
Introduction xxi
observations equally well, then the prior probabilities will dominate and air travel will be seen to be the most likely explanation.
As one example of the unifying force of the abductive paradigm, Charniak and Shimony showed that Hobbs and Stickel’s cost-based abductive approach could be given probabilistic semantics
and
be viewed as search for the most likely a posteriori explanation for the observed actions. While the Bayesian approach to plan recognition was initially quite controversial, probabilistic inference, in one form or another, has since become the mainstream approach to plan recognition.
Another broad area of attack to the problem of plan recognition has been to reformulate it as a parsing problem (e.g., Vilain
based on the observation that reasoning from actions to plans taken from a plan
hierarchy was analogous to reasoning from sentences to parse trees taken from a grammar. Early work on parsing-based approaches to plan recognition promised greater efficiency than other approaches, but at the cost of making strong assumptions about the ordering of plan steps. The major weakness of early work using parsing as a model of plan recognition is that it did not treat partially ordered plans or interleaved plans well. Recent approaches that use statistical parsing
combine parsing and
Bayesian approaches and are beginning to address the problems of partially ordered and interleaved plans.
Finally, substantial work has been done using extensions of Hidden Markov Models (HMMs)
techniques that came to prominence in signal-processing applications, including speech recognition.
They offer many of the efficiency advantages of parsing approaches, but with the additional advantages of incorporating likelihood information and of supporting machine learning to automatically acquire plan models. Standard HMMs are nevertheless not expressive enough to sufficiently capture goal-directed behavior. As a result, a number of researchers have extended them to hierarchical formulations that can capture more complicated hierarchical plans and intentions
Much of this latter work has been done under the rubric of activity recognition
The early
research in this area very carefully chose the term activity or behavior recognition to distinguish it from plan recognition. The distinction to be made between activity recognition and plan recognition is the difference between recognizing a single (possibly complex) activity and recognizing the relationships between a set of such activities that result in a complete plan.
Activity-recognition algorithms discretize a sequence of possibly noisy and intermittent low-level sensor readings into coherent actions that could be taken as input by a plan-recognition system. The steady decline in sensor costs has made placing instruments in smart spaces practical and brought activity recognition to the forefront of research in the computer vision and pervasive computing com- munities. In activity recognition, researchers have to work directly with sensor data extracted from video, accelerometers, motion capture data, RFID sensors, smart badges, and Bluetooth. Bridging the gap between noisy, low-level data and high-level activity models is a core challenge of research in this area.
As data becomes more readily available, the role of machine learning and data mining to filter out noise and abstract away from the low-level signals rises in importance. As in other machine learning tasks, activity recognition can be viewed as a supervised
learning task,
depending on the availability of labeled activity traces. Alternatively, it can be treated as a problem of hidden state estimation and tackled with techniques such as hierarchical hidden (semi)-Markov models
xxii Introduction
A specialized subfield of “action recognition” is dedicated to the problem of robustly recognizing short spatiotemporally localized actions or events in video with cluttered backgrounds (see Poppe
for a survey); generally, activity recognition carries the connotation that the activity recognized is a more complex sequence of behavior. For instance, “throwing a punch” is an example of an action that could be recognized by analyzing the pixels within a small area of an image and a short duration of time. In contrast, “having a fight” is a complex multiperson activity that could only be recognized by analyzing a large set of spatiotemporal volumes over a longer duration.
Several researchers have been interested in extending plan recognition to multiagent settings
and
using it to improve team coordination
If agents in a team can recognize what their teammates
are doing, then they can better cooperate and coordinate. They may also be able to learn something about their shared environment. For example, a member of a military squad who sees another soldier ducking for cover may infer that there is a threat and therefore take precautions.
In domains with explicit teamwork (e.g., military operations or sports), it can be assumed that all the agents have a joint, persistent commitment to execute a goal, share a utility function, and have access to a common plan library grounded in shared training experiences. This facilitates the recognition process such that in the easiest case it is possible to assume that all the actions are being driven by one centralized system with multiple “actuators.” For simpler formulations of the multiagent plan recognition (MAPR) problem, recognition can be performed in polynomial time
In the more complex case of dynamic FIGURE I.1 A mind map of research directions, methods, and applications in plan, activity, and intent recognition.
Introduction xxiii
teams, team membership changes over time and accurate plan recognition requires identifying groupings among agents, in addition to classifying behaviors
Grouping agents in the unconstrained case
becomes a set partition problem, and the number of potential allocations rises rapidly, even for a small number of agents. Prior work on MAPR has looked at both extending single-agent formalisms for the multiagent recognition process
and creating specialized models and recognition techniques
for agent teams
Thus, we see how far the field has evolved, from the genesis of plan recognition as a subproblem within classical AI to a vibrant field of research that stands on its own. Figure
illustrates the diversity of concepts, methods, and applications that now drive advances across plan, activity, and intent recognition.
This book provides a comprehensive introduction to these fields by offering representative examples across this diversity.
Chapter Map
The collection of chapters in this book is divided into four parts: (1) classic plan- and goal-recognition approaches; (2) activity discovery from sensory data; (3) modeling human cognitive processes; (4) multiagent systems; and (5) applications of plan, activity, and intent recognition. We discuss each of these areas and the chapters we have grouped under the part headings next.
Classic Plan and Goal Recognition
The book begins with chapters that address modern plan-recognition problems through the same abduc- tive perspective that characterized the seminal work in the field. The Chapter 1 addresses two important challenges in modern plan recognition. The questions are: How much recognition is actually needed to perform useful inference? Can we perform a more limited, but still useful, inference problem more effi- ciently? Blaylock and Allen, in “Hierarchical Goal Recognition” argue that in many cases we can, and propose to solve the simpler problem of goal recognition. They also address a second challenge: eval- uating plan-recognition techniques, proposing to use synthetic corpora of plans to avoid the problems of acquiring human goal-directed action sequences annotated with “ground truth” motivation.
Blaylock and Allen’s chapter provides a definition of goal recognition as a proper subset of plan recognition. In goal recognition all we are interested in is the top-level goal of the agent, while in plan
recognition we also ask the system to produce a hypothesis about the plan being followed by the agent,
and answer questions about the state of plan execution (e.g., “How much of the plan is completed?” and “What roles do particular actions play in the plan?”) Blaylock and Allen present an approach to goal recognition based on Cascading Hidden Markov Models.
As plan recognition is maturing, it is moving away from exploratory engineering of proof-of-concept plan-recognition algorithms. However, it is difficult to do “apples-to-apples” comparisons of different techniques without shared datasets. The Monroe Corpus of plans and observation traces created by Blaylock for his Ph.D. dissertation was one of the first publicly available corpora for training and testing plan-recognition systems. It has been a significant resource for the plan recognition community because it attempts to move from an exploratory to a more empirical foundation. This chapter introduces