Index of /Kuliah2016-2017/KecerdasanBuatanUntukGame
Artificial Intelligence in
Game Design
Probabilistic Finite State Machines
and Markov State Machines
Randomness Inside State
• Randomness in actions taken by NPC
– Randomness inside update method
– Can depend on current state
Confident
Angry
Frightened
Attack Left 40%
60%
30%
Attack
Right
40%
35%
20%
Defend
20%
5%
50%
Randomness Inside State
• Randomness in Initial Setup
– Randomness in enter method
– Example: choice of weapon in Fight state
60%
35%
5%
Randomness in Transitions
• Same current state + same stimuli
= one of several possible next states
– Possibly including current state
• Performing different tasks at random
Guard Door and
Shout for Help
Player
visible
Patrol in
front of Door
60%
40%
Player
visible
Chase
Player
Random Behavior Timeouts
• Continue strong emotional behavior for random
number of steps
Predator
seen
Predator
seen
Wander
Flee
Predator
not seen
10%
Predator
not seen
90%
Unpredictability of World
• Small chance of “unexpected” occurrence
– Adds “newness” to game even after multiple plays
– Adds to “realism” of world
Target in sights
Reload
Finished
reloading
Aim
98%
Target in sights
Gun Jam
Gun
cleared
Unexpected
case
2%
Fire
Normal
case
Randomness in Emotional States
• Emotional transitions less predictable
• Effect of “delayed reaction”
Small hit by player
75%
Player HP < 10
Player HP < 10
40%
60%
Confident
Angry
Small hit by player
Heavy hit
by player
70%
Heavy hit by me
30%
25%
Heavy hit
by player
30%
Frightened
Heavy hit by me
70%
My HP < 10
My HP < 10
50%
50%
Probabilities and Personality
• NPCs with probabilities can give illusion of personalities
• Differences must be large enough for player to notice in
behavior
Small hit by player
10%
Player HP < 10
Player HP < 10
80%
20%
Confident
Angry
Small hit by player
Heavy hit
by player
70%
Heavy hit by me
30%
Heavy hit by me
70%
90%
Heavy hit
by player
30%
Frightened
My HP < 10
80%
My HP < 10
20%
Orc with
anger
management
issues
Dynamic Probabilities
• Likelihood of transition depends on something else
– More realistic (but not completely predictable)
– Can give player clues about state of NPC
% of bullets left
Firing
Patrol in
front of Door
Player not
firing
Player
visible
1- % of
bullets left
Reload
Guard Door and
1- Energy % Shout for Help
Energy %
Chase
Player
Emergent Group Behavior
• Each NPC in group can choose random behavior
– Can appear to “cooperate”
50 %
Patrol
Player
visible
Fire
Cover
reached
50 %
Take Cover
– Half of group fires immediately giving “cover” to rest
– If player shoots firing players, rest will have time to
reach cover
Emergent Group Behavior
• Potential problem:
Possibility all in group can choose same action
– All either shoot or take cover
– No longer looks intelligent
• Can base probabilities on actions others take
1 - % of other players firing
Patrol
Player
visible
Fire
Cover
reached
% of other players firing
Take Cover
“Markov” State Machines
• Tool for decision making about states
– Give states a “measure” describing how good state is
– Move to state with best measure
• Key: Measure changes as result of events
– Possibly returns to original values if no events occur
– Based (sort of) on Markov probabilistic process
(but not really probabilities)
“Markov” State Machines
• Example: Guard choosing cover
• Different cover has different “safety”
measures
• Firing from cover makes it less safe
(player will start shooting at that cover)
• Represent safety as vector of
values
1.0
trees
1.5
wall
0.5
brush
“Markov” State Machines
• Assign transition “matrix” to each action
• Defines how each state affected by action
– Multiplier < 1 = worse
– Multiplier > 1 = better
• Example: fire from trees
0.1
– Trees less safe
– Other positions marginally safer
(player not concentrating on them)
1.2
1.2
“Markov” State Machines
• “Multiply” current vector by
matrix to get new values
•
1.0
0.1
1.5
1.2
0.5
1.2
0.1
=
Note: real matrix multiplication
requires 2D transition matrix
1.8
0.6
0.1
0
0
0
1.2
0
0
0
1.2
“Markov” State Machines
• Further events modify values
• Example: Now fire from behind wall
0.1
1.2
1.8
0.2
0.6
1.2
0.12
=
0.9
0.72
“Markov” State Machines
• Note “total safety” (as sum of values) decreasing
3 2.5 1.74…
– May be plausible (all cover becoming less safe)
– Can normalize if necessary
• Can gradually increase
values over time
– Usually result of
time/turns without event
– Example: player leaves area
0.12
1.1
0.9
1.1
0.72
1.1
0.132
=
0.99
0.792
Game Design
Probabilistic Finite State Machines
and Markov State Machines
Randomness Inside State
• Randomness in actions taken by NPC
– Randomness inside update method
– Can depend on current state
Confident
Angry
Frightened
Attack Left 40%
60%
30%
Attack
Right
40%
35%
20%
Defend
20%
5%
50%
Randomness Inside State
• Randomness in Initial Setup
– Randomness in enter method
– Example: choice of weapon in Fight state
60%
35%
5%
Randomness in Transitions
• Same current state + same stimuli
= one of several possible next states
– Possibly including current state
• Performing different tasks at random
Guard Door and
Shout for Help
Player
visible
Patrol in
front of Door
60%
40%
Player
visible
Chase
Player
Random Behavior Timeouts
• Continue strong emotional behavior for random
number of steps
Predator
seen
Predator
seen
Wander
Flee
Predator
not seen
10%
Predator
not seen
90%
Unpredictability of World
• Small chance of “unexpected” occurrence
– Adds “newness” to game even after multiple plays
– Adds to “realism” of world
Target in sights
Reload
Finished
reloading
Aim
98%
Target in sights
Gun Jam
Gun
cleared
Unexpected
case
2%
Fire
Normal
case
Randomness in Emotional States
• Emotional transitions less predictable
• Effect of “delayed reaction”
Small hit by player
75%
Player HP < 10
Player HP < 10
40%
60%
Confident
Angry
Small hit by player
Heavy hit
by player
70%
Heavy hit by me
30%
25%
Heavy hit
by player
30%
Frightened
Heavy hit by me
70%
My HP < 10
My HP < 10
50%
50%
Probabilities and Personality
• NPCs with probabilities can give illusion of personalities
• Differences must be large enough for player to notice in
behavior
Small hit by player
10%
Player HP < 10
Player HP < 10
80%
20%
Confident
Angry
Small hit by player
Heavy hit
by player
70%
Heavy hit by me
30%
Heavy hit by me
70%
90%
Heavy hit
by player
30%
Frightened
My HP < 10
80%
My HP < 10
20%
Orc with
anger
management
issues
Dynamic Probabilities
• Likelihood of transition depends on something else
– More realistic (but not completely predictable)
– Can give player clues about state of NPC
% of bullets left
Firing
Patrol in
front of Door
Player not
firing
Player
visible
1- % of
bullets left
Reload
Guard Door and
1- Energy % Shout for Help
Energy %
Chase
Player
Emergent Group Behavior
• Each NPC in group can choose random behavior
– Can appear to “cooperate”
50 %
Patrol
Player
visible
Fire
Cover
reached
50 %
Take Cover
– Half of group fires immediately giving “cover” to rest
– If player shoots firing players, rest will have time to
reach cover
Emergent Group Behavior
• Potential problem:
Possibility all in group can choose same action
– All either shoot or take cover
– No longer looks intelligent
• Can base probabilities on actions others take
1 - % of other players firing
Patrol
Player
visible
Fire
Cover
reached
% of other players firing
Take Cover
“Markov” State Machines
• Tool for decision making about states
– Give states a “measure” describing how good state is
– Move to state with best measure
• Key: Measure changes as result of events
– Possibly returns to original values if no events occur
– Based (sort of) on Markov probabilistic process
(but not really probabilities)
“Markov” State Machines
• Example: Guard choosing cover
• Different cover has different “safety”
measures
• Firing from cover makes it less safe
(player will start shooting at that cover)
• Represent safety as vector of
values
1.0
trees
1.5
wall
0.5
brush
“Markov” State Machines
• Assign transition “matrix” to each action
• Defines how each state affected by action
– Multiplier < 1 = worse
– Multiplier > 1 = better
• Example: fire from trees
0.1
– Trees less safe
– Other positions marginally safer
(player not concentrating on them)
1.2
1.2
“Markov” State Machines
• “Multiply” current vector by
matrix to get new values
•
1.0
0.1
1.5
1.2
0.5
1.2
0.1
=
Note: real matrix multiplication
requires 2D transition matrix
1.8
0.6
0.1
0
0
0
1.2
0
0
0
1.2
“Markov” State Machines
• Further events modify values
• Example: Now fire from behind wall
0.1
1.2
1.8
0.2
0.6
1.2
0.12
=
0.9
0.72
“Markov” State Machines
• Note “total safety” (as sum of values) decreasing
3 2.5 1.74…
– May be plausible (all cover becoming less safe)
– Can normalize if necessary
• Can gradually increase
values over time
– Usually result of
time/turns without event
– Example: player leaves area
0.12
1.1
0.9
1.1
0.72
1.1
0.132
=
0.99
0.792