Markov Chain Model for Genetic Algorithm

B.2 Markov Chain Model for Genetic Algorithm

Discussing an idea of a genetic algorithm in Sect. 5.1 , we noticed that a state space (in the sense used in search methods) is created by consecutive populations (strictly speaking: by their unique representations) generated with the algorithm. Thus, processing in a genetic algorithm can be treated as a process of generating consec- utive populations corresponding to states of this space in successive genetic epochs

t ∈ T = {0, 1, . . .}. These populations: P 0 , P 1 ,... define a sequence {P t : t ∈ T}. Such a sequence can be treated formally as a stochastic process.

Definition B.10 {X t Let us denote a set of real numbers with R, and a set of natural numbers with N. If T = R, then we say {X t : t ∈ T} is a continuous-time process. If T = N, then we say {X t : t ∈ T} is a discrete-time process.

Operators of genetic algorithm (crossover, mutation, etc.) applied for a given population do not depend on previous populations. It means that a sequence of random variables {X t : t ∈ T}, being a discrete-time stochastic process fulfills the Markov condition, i.e., it is a Markov chain.

Definition B.11

A discrete-time process {X t : t ∈ N} having values in at most countable set {x 0 , x 1 ,... }, which fulfills the Markov condition, i.e.: for each n ∈ N, j ∈ N, m ∈ N, m < n, for each j 1 < j 2 < ···<j m < n , for each i 1 ,..., i m ∈N the following condition holds:

P(X n =x j |X j m =x i m ,..., X j 1 =x i 1 ) = P(X n =x j |X j m =x i m ) is called a Markov chain.

The set {x 0 , x 1 ,...

} is called a state space. 6

i to a state x j in a time step from n − 1 to n is denoted with p n

A probability P n =x j |X n −1 =x i

ij .

6 Each state x i , i = 0, 1, . . . represents a population P i .

Appendix B: Formal Models for Artificial Intelligence Methods … 255 Probabilities of transitions in at most a countable number of states form a transition

matrix defined as:

=p n

ij .

If a probability p n ij of transition from a state x i to a state x j is independent of time n, then a Markov chain is called homogeneous. We define this property in the following way.

Definition B.12

A Markov chain {X t : t ∈ N} is called a homogeneous Markov chain iff it fulfills the following condition:

P(X n =x j |X n −1 =x i ) = P(X m =x j |X m −1 =x i ),

for any n and m. Parameters of genetic operators used in a genetic algorithm are constant. 7 It means

that a transition from a population x i to a population x j in one time step is the same for any time step. Thus, a genetic algorithm can be modeled as a homogeneous Markov chain that allows us to define a transition matrix for a state space and to analyze many properties of this algorithm.

7 This condition does not hold for, for example, evolution strategies, cf. Sect. 5.2 .