Adaptive AI Engine for RTS Games

Discussing the theory and practice

What’s done, what’s next?

Posted by Ogail on October 9, 2010

Good evening,
In this post we will give you a brief about our past research and future plans.

During the last year, We were concerned with the field of planning, learning and knowledge sharing in RTS (Real-Time Strategy) Games.

A- We started our work during our graduation project (Find it in this post) which was entitled “Adaptive Intelligent Agent for RTS Games ” (2010). We applied a novice planning technique (Online Case Based Planning) and a machine learning approach (Reinforcement Learning) in order to achieve an artificial intelligence that approaches the human behavior. We’ve made our best during this project from both research and development aspects. Below are research related aspects we’ve done:

1- Doing research using a number of papers, thesis & books as follows:

Papers encouraging research in this area:

RTS Games and Real–Time AI Research – 2003
Call for AI Research in RTS Games – 2004

Papers adopting Case Based Planning:

Case-Based Planning and Execution for RTS Games – 2007
On-Line Case-Based Plan Adaptation for RTS Games- 2008
Learning from Human Demonstrations for Real-Time Case-Based Planning – 2008
Situation Assessment for Plan Retrieval in RTS Games – 2009
On-Line Case based Planning – 2010

Papers adopting Evolutionary Algorithms & Dynamic Scripting:

Co-evolution in Hierarchical AI for Strategy Games – after 2004
Co-evolving Real-Time Strategy Game Playing Influence Map Trees with genetic algorithms
Improving Adaptive Game AI With Evolutionary Learning – 2004
Automatically Acquiring Domain Knowledge For Adaptive Game AI using Evolutionary Learning – 2005

Papers adopting Reinforcement Learning & Dynamic Scripting:

Concurrent Hierarchical Reinforcement Learning – 2005
Hierarchical Reinforcement Learning in Computer Games – After 2006
Goal-Directed Hierarchical Dynamic Scripting for RTS Games – 2006
Hierarchical Reinforcement Learning with Deictic repr. in a computer game- After 2006
Monte Carlo Planning in RTS Games – After 2004
Establishing an Evaluation Function for RTS games – After 2005
Learning Unit Values in Wargus Using Temporal Differences – 2005
Adaptive reinforcement learning agents in RTS games – 2008

Papers adopting Hybrid CBR/RL approaches :

Transfer Learning in Real-Time Strategy Games Using Hybrid CBR-RL – 2007
Learning continuous action models in a RTS Environment – 2008

Related Books:

AI Game Engine Programming
AI for Games

2- Developing an AI-Engine for RTS Games

We used an RTS Game Engine named “Stratagus” to develop our AI Game engine. The game that we used as a test-bed is a clone of the well-know Warcraft 2 game.

3- Maintaining the project blog

4- Maintaining the project repository:

5- Maintaining our own blogs:

  • (Omar Enayet)
  • (Abdelrahman Al-Ogail)

B- Our Next Step was publishing a paper entitled “Intelligent Online Case-Based Planning Agent Model in RTS Games” in ISDA 2010. Find it in this post.

Concerning our future plans, We are looking forward to achieve the following long-term goals:

1- Adding new theory in the area of “Simulation of Human Behavior”.
2- Developing a commercial AI Engine for RTS Games specifically and games in general. We already started and we have quite experience in game development.
3- Participate in related contests around the world for AI Engines in RTS Games (As Robocup, AAAI Starcraft Competition, ORTS Competition).
4- Initializing a major research group in Egypt in this field and become pioneers in it world wide.

However, our short term goal is enhancing the current engine , which will efficiently be able to plan and learn when playing against static AI, and use it as a test-bed to publish a number of papers , some of these papers are related to :

1- Introducing the whole Agent model and theory in AI related conference.
2- Introducing the whole AI Game Engine from a game industry point of view in a game-industry conference.
3- More Details & Testing concerning the hybridization of Online Case based Planning and Reinforcement Learning ( the topic of our last paper)
4- Knowledge representation for plans and experience in RTS games.
5- Enhancing agent’s situation assessment algorithm.
6- Comparing Case-Based Reasoning to Reinforcement Learning.

Other long-term papers’ topics include :
1- Include different planning algorithms/systems and let agent use them and make an intensive comparison between these panning systems.
2- Include different learning algorithms/systems and let agent use them and make an intensive comparison between these learning systems.
3- Multi-Agent AI : machine collaboration with other machines, or machine collaboration with human players.

4- Knowledge (Gaming Experience) Sharing.
5- Opponent Modeling.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: