Artificial Neural Networks Optimization Algorithms
23
FitnessObjective Functions
In order to handle both single nitrogen oxide and soot emission and multiple rate of heat release objective optimization tasks with identical algorithms, a single objec-
tive approximation function is used to describe the multi objective pareto optimal- ity
1
[90]. Given that the optimization task in the calibration part of the modelingoptimiza-
tion scheme is the search for the “best” set of model parameters, that is the set of model parameters which produces the smallest deviations from the measurements
e.g. least square errors, the single objective approximation function is defined as the weighted sum of the individual multiple objectives Table 3.3.
To account for the accuracy of the experimental data used in the model calibra- tion, a tolerance value is assigned to each objective output of the model prior to the
calculation of the objective function Figure 3.4.
Genetic Algorithm
The classic genetic algorithm GA [35] notwithstanding, the in-house developed GA uses real value parameter encoding, mutation, as well as a simulated annealing
2
mechanism in addition to the standard crossover mechanism for reproduction Table 3.4.
1.
Pareto optimality
- a.k.a. indifference curves, best solutions to a multi objective problem that could be achieved without disadvantaging at least one of the objectives
ROHR NO
X
SOOT
Tab. 3.3 Objective Functions used for the Model Calibration
a b
Fig. 3.4 Error Objective Function: a Standard Least Square Error LSE,
b LSE Including Tolerance Value
2.
Simulated Annealing
- probabilistic “neighbourhood” search method, inspired by the annealing technique in metallurgy heating up and controlled cooling of a material
f
obj
c
i
f
obj i
i
∑
c
1
Δ
2
ϕ
SOC
c
2
Δ
2
ϕ
10
c
3
Δ
2
ϕ
50
c
4
Δ
2
ϕ
90
… +
+ +
+ =
= c
5
Δ
2
m
prmx
c
6
Δ
2
Q
max
c
7
Δ
2
ϕ
Q
ma x
c
8
Δ
2
ε
Q
+ +
+ f
obj
Δ
2
NO
x
= f
obj
Δ
2
m
soot
=
x Obj ective
Funct ion
f
obj
Deviat ion
Δ
x-x x
f
obj
x Obj ective
Funct ion
f
obj
Deviat ion
Δ
x-x x
f
obj
x Obj ective
Funct ion
f
obj
Tolerance t
x = x ±
t
Deviation
Δ
x-x x
f
obj
x Obj ective
Funct ion
f
obj
Tolerance t
x = x ±
t
Deviation
Δ
x-x x
f
obj
24
After the stochastic initialization of the population 1
st
generation and the subse- quent evaluation of the objective function, reproduction mechanisms are applied to
the population, using statistical probabilities of crossover, transition and mutation. To select the individuals that are allowed to propagate to the next generation, a rou-
lette wheel algorithm
1
in combination with a guaranteed survival of the best individ- uals i.e. elitist survival is used.
Evolutionary Algorithm
Unlike classic GAs, the evolutionary algorithm EA used in this work employs crossover, intermediate and extended line recombination algorithms for the repro-
duction of the individuals Figure 3.5. Furthermore, strategic algorithm parameters are modified during the optimization, such as the range of the extended line recom-
bination
r
line
, which is reduced in order to increase the local optimization efficiency towards the end of an optimization run. Both tournament
2
and stochastic selection mechanisms are used to choose the individuals that propagate to the next generation
Table 3.5.
1.
Roulette Wheel Algorithm
- the probability of selection is proportional to the objective value, i.e. the higher the objective value, the higher the probability of selection
ENCODING
Real value encoding constrained
PARAMETERS n
pop
,
P
xover
,
P
trans
,
P
mut
,
n
elit
constant over generations
INITIALIZATION
Stochastic initialization of population 1
st
generation
REPRODUCTION
Crossover, transition simulated annealing, mutation
SELECTION
Roulette wheel selection, elitist survival Tab. 3.4
Genetic Algorithm Characteristics
2.
Tournament Selection
- the best individual amongst a group of individuals i.e. the tournament is selected; the size of the tournament determines the selection pressure ratio of the best individuals
selection probability to the average selection probability of all indiviuals
a b
Fig. 3.5 Recombination Mechanisms: a Intermediate, b Extended Line
Recombination According to [78]
Parent s Pot ent ial Offsprings
Parameter 1 [ -] P
a ra
m e
te r
2 [-
] Area of potent ial
Offsprings
Parent s Pot ent ial Offsprings
Parameter 1 [ -] P
a ra
m e
te r
2 [-
] Area of potent ial
Offsprings
Parents Pot ential Offsprings
Paramet er 1 [ -] P
a ra
m e
te r
2 [-
] Ext ended Line of
pot ent ial Offsprings
r
line
Parents Pot ential Offsprings
Paramet er 1 [ -] P
a ra
m e
te r
2 [-
] Ext ended Line of
pot ent ial Offsprings
r
line
25
“Covariance Matrix Adaptation” Evolutionary Strategy
The
μ
,
λ
Covariance Matrix Adaptation Evolutionary Strategy CMA-ES pre- sented by Hansen et al. [38] determines the k
th
offspring of the generation g+1, , given the center of mass of the selected individuals
, the global step size
and the random vectors according to Equation 3.1:
, 3.1
where
μ
is the number of parents,
λ
the number of offspring, and
σ
the initial step size. The symmetric positive definite
covariance matrix of the random
vectors is used to approximate the local search space. The global
step size used in the derandomized correlated mutation is deterministically adapted during the search process Table 3.6.
The assets of the CMA-ES are its adaptability to arbitrary optimization problems, small number of parameters, and computational efficiency, including a highly paral-
lel processing architecture.
“Genetic Algorithm Direct Search” MATLAB Toolbox
Apart from the direct search tools - which are not used in this investigation - the MATLAB Genetic Algorithm Direct Search GADS Toolbox 1.0.3 offers a set of
common GA mechanisms with numerous options for initialization, fitness scaling, selection, crossover and mutation.
ENCODING
Real value encoding constrained
PARAMETERS n
pop
,
n
parent
,
n
offspring
,
P
xover
,
P
line
, α
line
,
r
line
,
P
inter
, α
inter
,
P
mut
,
P
tour
adaptedmodified during search
INITIALIZATION
Stochastic initialization of population 1
st
generation
REPRODUCTION
Crossover, extended line intermediate recombination, mutation
SELECTION
Tournament, stochastic Tab. 3.5
Evolutionary Algorithm Characteristics
ENCODING
Normalized real value encoding
PARAMETERS n
offspring
= λ ,
n
parent
, = μ , σ
initial
adapted during search
INITIALIZATION
Fixed object variable start points
REPRODUCTION
Derandomized correlated mutation
SELECTION
No selection mechanism, individuals are adapted Tab. 3.6
CMA Evolutionary Strategy Characteristics
x
k g
1 +
x 〈 〉
μ g
σ
g
B
g
D
g
z
k g
1 +
x
k g
1 +
x 〈 〉
μ g
σ
g
B
g
D
g
z
k g
1 +
[ ]
+ =
k 1
… λ
, ,
= n
n ×
C
g
B
g
D
g
z
k g
1 +
26
This study employs normalized real value encoding, stochastic initialization, stan- dard crossover and mutation reproduction mechanisms, and uses truncation
1
as a selection criteria Table 3.7.