Adaptive AI Engine for RTS Games

Discussing the theory and practice

Archive for the ‘Orientation’ Category

What’s done, what’s next?

Posted by Ogail on October 9, 2010

Good evening,
In this post we will give you a brief about our past research and future plans.

During the last year, We were concerned with the field of planning, learning and knowledge sharing in RTS (Real-Time Strategy) Games.

A- We started our work during our graduation project (Find it in this post) which was entitled “Adaptive Intelligent Agent for RTS Games ” (2010). We applied a novice planning technique (Online Case Based Planning) and a machine learning approach (Reinforcement Learning) in order to achieve an artificial intelligence that approaches the human behavior. We’ve made our best during this project from both research and development aspects. Below are research related aspects we’ve done:

1- Doing research using a number of papers, thesis & books as follows:

Papers encouraging research in this area:

RTS Games and Real–Time AI Research – 2003
Call for AI Research in RTS Games – 2004

Papers adopting Case Based Planning:

Case-Based Planning and Execution for RTS Games – 2007
On-Line Case-Based Plan Adaptation for RTS Games- 2008
Learning from Human Demonstrations for Real-Time Case-Based Planning – 2008
Situation Assessment for Plan Retrieval in RTS Games – 2009
On-Line Case based Planning – 2010

Papers adopting Evolutionary Algorithms & Dynamic Scripting:

Co-evolution in Hierarchical AI for Strategy Games – after 2004
Co-evolving Real-Time Strategy Game Playing Influence Map Trees with genetic algorithms
Improving Adaptive Game AI With Evolutionary Learning – 2004
Automatically Acquiring Domain Knowledge For Adaptive Game AI using Evolutionary Learning – 2005

Papers adopting Reinforcement Learning & Dynamic Scripting:

Concurrent Hierarchical Reinforcement Learning – 2005
Hierarchical Reinforcement Learning in Computer Games – After 2006
Goal-Directed Hierarchical Dynamic Scripting for RTS Games – 2006
Hierarchical Reinforcement Learning with Deictic repr. in a computer game- After 2006
Monte Carlo Planning in RTS Games – After 2004
Establishing an Evaluation Function for RTS games – After 2005
Learning Unit Values in Wargus Using Temporal Differences – 2005
Adaptive reinforcement learning agents in RTS games – 2008

Papers adopting Hybrid CBR/RL approaches :

Transfer Learning in Real-Time Strategy Games Using Hybrid CBR-RL – 2007
Learning continuous action models in a RTS Environment – 2008

Related Books:

AI Game Engine Programming
AI for Games

2- Developing an AI-Engine for RTS Games

We used an RTS Game Engine named “Stratagus” to develop our AI Game engine. The game that we used as a test-bed is a clone of the well-know Warcraft 2 game.

3- Maintaining the project blog

4- Maintaining the project repository:

5- Maintaining our own blogs:

  • (Omar Enayet)
  • (Abdelrahman Al-Ogail)

B- Our Next Step was publishing a paper entitled “Intelligent Online Case-Based Planning Agent Model in RTS Games” in ISDA 2010. Find it in this post.

Concerning our future plans, We are looking forward to achieve the following long-term goals:

1- Adding new theory in the area of “Simulation of Human Behavior”.
2- Developing a commercial AI Engine for RTS Games specifically and games in general. We already started and we have quite experience in game development.
3- Participate in related contests around the world for AI Engines in RTS Games (As Robocup, AAAI Starcraft Competition, ORTS Competition).
4- Initializing a major research group in Egypt in this field and become pioneers in it world wide.

However, our short term goal is enhancing the current engine , which will efficiently be able to plan and learn when playing against static AI, and use it as a test-bed to publish a number of papers , some of these papers are related to :

1- Introducing the whole Agent model and theory in AI related conference.
2- Introducing the whole AI Game Engine from a game industry point of view in a game-industry conference.
3- More Details & Testing concerning the hybridization of Online Case based Planning and Reinforcement Learning ( the topic of our last paper)
4- Knowledge representation for plans and experience in RTS games.
5- Enhancing agent’s situation assessment algorithm.
6- Comparing Case-Based Reasoning to Reinforcement Learning.

Other long-term papers’ topics include :
1- Include different planning algorithms/systems and let agent use them and make an intensive comparison between these panning systems.
2- Include different learning algorithms/systems and let agent use them and make an intensive comparison between these learning systems.
3- Multi-Agent AI : machine collaboration with other machines, or machine collaboration with human players.

4- Knowledge (Gaming Experience) Sharing.
5- Opponent Modeling.


Posted in Orientation | Leave a Comment »

I-Strategizer Project Documentation – Version 1.0

Posted by Ogail on October 9, 2010

We’ve wrote a documentation for the last year research in a document. This documents acts a reference manual for most of our research and development. It’s considered as general orientation document for any person would like to start researching in this area. Download it from link below:

I-Strategizer Documentation Version 1.0

Posted in Orientation | Leave a Comment »

Adaptive AI Engine for RTS Games

Posted by Ogail on July 15, 2009

  • Technical words that need explanation:
    • AI Opponent: we mean by AI opponent the computer that plays against human
    • Human Opponent: the human player playing against the computer
    • RTS Game:
      eal Time Strategy Games; that’s the technical name of strategy games (as Red-Alert)
  • Preface:
    • This project is in the field of Artificial Intelligence specially in Machine Learning area
    • The minor goal to the project is to make the machine (computer) able to predict human future decision
    • The project aims to provide this facility (prediction) to the AI opponent in Strategy Games
  • What are problems we want to solve:
    • AI opponent weaknesses easily detected
      • In most commercial complex major games, human players only need a couple of games to spot opponents’ weaknesses and to exploit them in upcoming games. This results in not effective boredom game
    • AI opponent get in the same trap repeatedly
    • Most AI opponents don’t know the answer of the questions:
      • Does the AI opponent know the safe map locations to get out from kill zones
      • Does AI opponent know if the human opponent rushes?
      • Does human opponent rely on units that require certain resources?
      • Does human opponent frequently build a number of critical structures in a poorly defensive place?
      • Does human player attacks are balanced or not?
  • Why this project is useful?
    • Currently there are automated machines (robots) that are used in wars instead of human. The army commander can use the AI Engine to:
      • Test his plans
      • Share other commanders experience in the plan
      • Discover flaws in the plan. And solves these flaws
    • The problems considered are not solved till now in commercial RTS Games (by Alex Champandard)
  • Other Issues:
  • What’s the expected output from the project?
    • AI Library for RTS Games:
      • A commercial library that is able to compete in the Game AI Industry
    • 2 Publications:
      • Publish 2 papers that summarizes what new we’ve add
  • How to test the AI Engine:
    • By solving previous problems and then play a game with the computer and ensure that its able to learn
  • Motivations:
    • We are interested in this project for the following reasons:
      • AI is our area of interest in computer science
      • In the future, we are planning to work in the industry of game development specially in AI Engines

      • We love playing strategy games, developing this project will be so interesting and fun!
      • Our wish to add something new to this world

Posted in Orientation | Leave a Comment »