Simple-Reflex Agents

Sometimes we do not need an extensive analysis of the game state to figure out our next move. At times, the game is simple enough that we can make a general strategy and represent it as a set of rules. A rule would specify that a if a certain condition is met, a certain action should be taken. An agent built using such rules is called a Simple Reflex Agent

A Simple Reflex Agent is typically employed when all the information of the current game state is directly observable, (eg: Chess, Checkers, Tic Tac Toe, Connect-Four) and the decision regarding the move only depends on the current state. That is, when the agent does not need to remember any information of the past state to make a decision.

For example, let's consider an agent for Connect-Four. This agent has two objectives: - Stop enemy from Completing four-in-a-row - Completing four-in-a-row

For the first aim, some simple rules can be made as follows:

• If any column is topped by two or more of your opponent's pieces, place your piece in that column.
• If you see two or more of opponents pieces placed adjacently in a row, with an empty place-able space on either side, place a piece there.
• If you see two or more of opponents pieces placed adjacently in a diagonal, with an empty place-able space on either side, place a piece there.

Here, place-able means that you can place a piece in that location by the rules of the game, that the place directly below it (if it exists) should be filled by a piece.

Each of these "two or more" rules can be broken down in to two rules, one fro "two" and one for "three", where the rules for "three" will take precedence.

Similarly we can choose strategies for the second aim, that is "Completing a Four-in-a-Row" and encode it in the form of such condition-action rules.

Most of our discussion during these lessons assume we want to build an AI agent that aims to be as smart as the player. When making AI enemies in video games, this is usually not the requirement. So even though not every information is directly observable in a game, a Simple Reflex agent might be a good choice because the AI enemy doesn't need to know everything. An enemy can simply function on what it can immediately see. This applies well to retro top-down games. If you're in the enemy's line of sight, it will attack you. Otherwise, it will patrol its predefined path or area. This can be achieved by making very simple AI enemies by defining a few simple rules of behavior.

Model-Based Reflex Agents

The problem with Simple Reflex Agents is that they can only operate on the current game state, and are unable to use information from older states, not even the directly previous one. This capability is achieved by Model-based Reflex Agents. These agents maintain an internal model of the game world, which they update at every turn using the observations from the current state. So this model is an aggregation of all states the agent observed.

Continuing the example from the previous section, consider the AI enemy that patrols until it sees the player. If we use a Simple reflex agent, it would only consider what it sees in that very instant. So the moment the player turns a corner, the enemy will stop chasing. So while the Simple Reflex Agent might be good for an automated turret, it's a bad model for enemy guards.

When we build a Model-Based Reflex Agent for an enemy guard, the guard can have an internal model of the world and himself. This allows him to use observations from the past. We can use this internal model to store the following information: - The state of the guard (eg: Patrolling, Chasing, etc) - Location where target was last scene - A cooldown timer (most games have a fixed time before the guard returns to his post) - The path he followed when chasing the enemy. (This is if you want the guard to return to his patrol)

Another use for Model-Based Reflex Agents is when initially your agent does not know the map, but you want it to explore and remember the places. So as it moves around and sees more of the level, it creates an internal map.