Goal-Based Agents

Previously we discussed Model-Based Reflex Agents as a way to design simple enemies. We considered a very simple behavior of the AI enemy which can be stated in the form of following condition-action rules:

• If patrolling and no enemy in sight then Patrol predefined path
• If patrolling and enemy in sight, switch mode from patrolling to chasing
• If chasing and enemy in sight , move towards enemy
• If chasing and enemy not in sight move towards enemy's last location
• If at enemy's last location and enemy not in sight and cooldown timer not started then start cooldown timer
• If at enemy's last location and enemy not in sight and cooldown timer expired then change mode to returning to post
• if returning to post and enemy in sight then set mode to chasing
• if returning to post and reached patrolling path and set mode to patrolling

Now as you may notice, we are still using condition-action rules like we did in the Simple Reflex Agent. The differentiating factor here is that we can maintain an internal state and model of the game environment and use it to take decisions. However, note that the decisions are still being taken using pre-defined condition action rules.

So what if we want to make our AI guards somewhat smarter? What if we want them to figure out what to do, themselves by understanding their situation and possible paths of action? This might be needed if our guard needs to choose between multiple possible responses like:

• Continue Patrolling
• Investigate Sounds
• Chase Intruder
• Stay in sight of other guards
• Go sound the Alarm

Now instead of having predefined Condition-Action rules for choosing the actions, our guard should choose the action that best aligns with his Goal. If we set the Goal to be Capture Intruder, then the guard will always give precedence to Chase Intruder and Investigate Sounds as these actions bring the guard to his goal of capturing the intruder. Similarly, if we set the Goal to Stay Safe, the guard would always choose to Continue Patrolling and Stay in Sight of Other Guards.

Ideally we want an Agent that considers all possibilities and choose the most useful one. This means that in case of multiple competing goals, the agent should choose the one with the highest Utility. Agents that use utility are also known as Utility-Based Agents.

The tree-traversing agents we discussed in previous lessons were actually forms of Utility-Based agents as they explored what actions would lead to what game states, and how much that state would be worth (score/utility), and then choose the action that leads the state with the highest utility.

As you might notice, Goal-Based agents need to understand the game and how the game state changes. Like the Model-Based Agents, Goal-Based agents also have an internal model of the game state. Where as Model-Based Agents only need to know how to update their internal model of the game state using new observations, Goal-based agents have the additional requirement of knowing how their actions will affect the game state. This is because, Goal-Based Agents use their internal model to test out strategies. When they know how their actions will affect the game state, they can find out the different states each move will lead to. Then based on the utility of these, they choose the best strategy.

While Goals can be defined in any way that suits your need and implementation of the game, they are usually defined in terms of an observable (or calculable) variable. For Example:

• Achieve distance to target = 0
• Maintain Health > 0

The Achieve type of Goal aims to reach a certain condition which isn't true initially, where as the Maintain type of Goal aims to keep a condition true (which is usually true initially). Now to be able to do this, the agent needs to understand how his actions affect these variables. The Agent will then judge if he is getting closer or further from fulfilling a goal.

Your implementation of the agent determines how the agent will behave. You can implement it to be very cautious and avoid even getting close to failing a Maintain goal; Or you can make it such that it sees Maintain goals as valid ranges in which to function, and thus free moves within the range, even if means coming very close to the border. You can also implement multiple modes, one for each such behaviour. This maybe needed as you would want the agent to be very conservative with Health, but you may need him to be flexible with metrics such as distance from target, as long as it leads to achieving other goals.

It also helps to provide a priority with goals, so the agents knows when it has to choose between two goals, whose utility should have a higher weightage. This usually requires you to normalize your utility values, so that you can compare "Come close to target by 100 units" and "Lose 10 Health" on an equal footing.