* (research talk; 1 hour):
Title: Learning Hierarchical Task Networks from action traces
Abstract:
We describe HTN-MAKER, an algorithm for learning hierarchical
planning knowledge in the form of decomposition
methods for Hierarchical Task Networks (HTNs). HTNMAKER
takes as input the initial states from a set of classical
planning problems in a planning domain and solutions
to those problems, as well as a set of semantically-annotated
tasks to be accomplished. The algorithm analyzes this semantic
information in order to determine which portions of
the input plans accomplish a particular task and constructs
HTN methods based on those analyses. Our theoretical results show that  
HTN-MAKER is sound and complete. We show that the methods learned by
HTN-MAKER enable an HTN planner to solve problems faster than
planning from scratch.


* (seminar begins; 1 hour):
Title: An overview of Game AI
Abstract:
Over the past 20 years computer games have gone through a remarkable  
evolution in areas such as graphics and multi-player gaming. Another  
area where some evolution has been made is in the quality of the  
computer opponent. Despite this evolution, there is a big mismatch  
between what people in the game industry refer to as the "AI"  
(roughly, the program controlling the computer opponent) and what AI  
actually is ("...technology can be controlled especially if it is  
saturated with intelligence to watch over how it goes, to keep  
accounts, to prevent errors, and to provide wisdom to each decision"-  
Allen Newell, 1992). In this talk, we will look at contemporary  
computer games, explore techniques for implementing the AI (from the  
perspective of the game industry) and study opportunities to use AI  
(from the research perspective) to enhance the gaming experience.

* (seminar day 2; 2 hours):
Title: Using Hierarchical Task Network (HTN) planning to create Game AI
Abstract:
We will begin by introducing HTN planning, focusing on the most widely
used paradigm for HTN planning: Ordered Task Decomposition. This
paradigm has been used for a wide set of applications including
game playing, manufacturing of mechanical pieces, and web agent
navigation. We will present a case study of using HTN planning for
modeling effective team strategies for Unreal Tournament®
(UT) bots. HTNs were used to encode strategies that
coordinate teams of bots in FPS games and run them
effectively using standard FSMs to encode individual bots
behavior. As a result, a grand strategy is laid out by the
HTNs and event-driven programming allows the bots to
react in this highly dynamic environment while
contributing to the grand task.

* (seminar day 3; 2 hours):
Title: Using Reinforcement Learning (RL) to create Game AI
Abstract:
We will present Q-learning, a simple yet effective variant
of reinforcement learning (RL). Then we will present approaches
for using RL in games. Afterward, we will discuss in detail
a case study for online reinforcement learning developing winning policies
in team first-person shooter games. We will study three crucial
characteristics: (1) individual BOT behavior is fixed although not
known in advance, therefore individual BOTS work as "plugins",
(2) RETALIATE models the problem of learning team
tactics through a simple state formulation, (3) discount rates
commonly used in Q-learning are not used. We will contrast
the results of using RL with those of using HTN planning
from the day 2 of the seminar.


* (seminar day 4; 2 hours):
Title: Using  Case-based reasoning (CBR) to create Game AI
Abstract:
We will present an overview of case-based reasoning (CBR).
Afterwards, we will present an overview
of CBR applications to create game AI. Then we will focus on case
study of an agent that combines Case-Based Reasoning (CBR)
and Reinforcement Learning (RL) algorithms.
Unlike most previous work where RL is used to improve accuracy
in the action selection process, we demonstrate how to use CBR to allow RL
to respond more quickly to changing conditions. Our approach combines
two key features: it uses a time window to compute similarity and stores
and reuses complete Q-tables for continuous problem solving. We demonstrate
our approach on a team-based first-person shooter game, where our
combined CBR+RL approach adapts quicker to changing tactics by an
opponent than standalone RL. We will contrast the results of
using of CBR with those of using HTN planning (Seminar day 2)
and RL (seminar day 3).


* (seminar day 5; 2 hours):
Title: Games as a testbed for AI
Abstract:
In previous days we have studied how well-established
AI techniques can be used to create advanced game AI.
In this talk we will turn the problem around: rather
than using AI to create new game AI, we will
present case studies of new ideas advancing the state of
the art in AI and describe in detail how games can be
used as a solid testbed to demonstrate these ideas.
This lasts presentation will built on the presentations
of the previous days.