Evolutionary Design Of Process Controlle
EVOLUTIONARY DESIGN OF PROCESS CONTROLLERS
DP Searson, MJ Willis and GA Montague
University of Newcastle upon Tyne, England.
The Genetic Programming (GP) architecture offers a
potentially powerful framework for ‘intelligent’
controller design due to its inherent matching of
structure to utility. This article describes the use of GP
to design discrete time single loop controllers for two
different processes: a linear Auto-Regressive
eXogeneous (ARX) type system and a non-linear
simulated Continuous Stirred Tank Reactor (CSTR),
both with constraints on the actuator outputs. It is
demonstrated that the GP methodology is capable of
designing recursive controllers that, for a specific class
of control objectives, offer similar performance to PID
controllers on the nominal systems. The advantages and
disadvantages of controllers designed in this way are
also discussed.
INTRODUCTION
One approach in control systems design is to optimise
the parameters of a controller of known structure. This
can be done using numerical methods, adaptive
techniques or manual tuning methods. These systems
are then known as parameter optimised control systems
[Isermann (1)] and a common example of this type of
system is the widely known PID controller.
A contrasting approach is the use of structure optimal
control systems in which both the structure and the
parameters of the controller are matched in some way to
the parameters and structure of the process model
[Isermann (1)]. The deadbeat and Dahlin controllers are
examples of structure optimal systems. The applicability
of these types of controller is quite restricted, however,
because processes with non-linearities and constraints
generally do not lend themselves to the analytical
approach that the structure optimal control design
method requires. Another problem is that the designer
must know a priori what form the controller is to take.
Ideally, it would be desirable to have a method that,
given a model of the process, could generate a controller
without the designer needing to specify beforehand
what form it should take. It would also be desirable that
this method could be used with non-linear process
models of arbitrary structure as well as utilising any
additional information that is available about
constraints. The advantage of this over parameter
optimised controller design would be that no possibly
performance-restricting decisions on the controller
structure would need to be made, and better closed-loop
control could be achieved.
In reality, this is quite a demanding design problem: any
form of automated design procedure cannot easily
utilise qualitative knowledge about the process that the
control engineer normally has access to and must rely
on closed-loop simulations to extract ‘knowledge’ about
the problem. Additionally, any controllers produced will
have no guarantee of good robustness or stability
properties unless provision for this is specifically made
in the automated design procedure. This, however,
would not be easily achievable for controllers of
arbitrary structure.
This article does not seek to fully address the latter
problem but instead concentrates on the more tractable
task of simple step-change servo control for two
nominal processes. This is accomplished by utilising the
evolutionary computation method called Genetic
Programming (GP), which was developed by Koza (2),
to evolve the structure and parameters of simple
recursive single loop controllers.
The ideas behind GP are first explained and then the
control problem formulation, GP configuration and the
two example processes are described. Finally, the
results are presented and discussed.
INTRODUCTION TO GP
GP is a method of getting computers to learn how to
solve problems without being explicitly programmed to
do so. Koza (2) popularised the technique and applied it
to a number of problems e.g. symbolic regression, and
finite state automata. Although most applications of GP
are in the field of computing science recently there have
been a number of engineering applications.
In the field of control Alba et al. (3) used GP based
fuzzy logic controllers on the cart-centering control
problem first tackled using ordinary GP in Koza (2).
Howley (4,5) used GP to derive algorithms to perform
specific spacecraft attitude manoeuvres and also to
derive control laws for a two-link payload manipulator
problem. For further details of engineering applications
the survey given in Willis et al. (6) should be referred
to.
The study of GP in its own right, and the theory
underlying its mechanism, are also active areas of
research: there are many variants on the basic theme and
currently there is no generally accepted ‘best’ way to set
up a Genetic Programming procedure to solve a
problem.
Tree structure of GP
Given a problem, GP works by maintaining a
population of individuals, each of which is a computer
program that can be ‘executed’ to give a potential
solution to the problem. These candidate programs are
encoded as parse trees (rather than lines of computer
code). Figure (1) depicts a simple example of a treestructured program. The result of the program is always
returned at the top of the tree. In the case of Figure (1)
the program result is (3-b)+2.
Each node of the tree is either a terminal node or a
primitive functional node. Terminals are usually
specified to be variables (from the problem at hand) that
the designer has previously identified to be ‘inputs’ to
the problem. However, terminals can also be defined as
constants that are generated randomly when initially
creating the tree.
Primitive functionals are usually specified to be
‘operators’ that the designer believes will enable a
correct solution to be constructed from the terminals.
The selection of the appropriate primitive functionals
for a problem is, again, a choice that must be made by
the designer. A typical primitive functional set would
probably include the standard operators of addition,
multiplication, division and subtraction as well as any
other operators that the designer suspects to be of use in
the context of the problem. Koza (2) states that ability
of the sets of terminals and primitive functionals to
solve the selected problem is known as the sufficiency
property of the GP and that for certain classes of
problems, e.g. Boolean problems, the sufficient
primitive functional set is known exactly. As we move
towards more complex problems, however, the
minimally sufficient requirements for a problem
solution become less clear.
In the case of Figure (1) the terminals that have been
utilised to construct the tree are the constants 2 and 3
and the variable b. The primitive functionals used are
the scalar operators ‘+’ and ‘-‘. Generally, tree
programs do not need to use all the terminals or
primitive functionals available as the GP algorithm
assembles them in a probabilistic manner during a run.
‘fitness function’. The form of the fitness function is
dependent on the problem at hand and must be selected
by the designer.
The initial population of individual programs is
generated randomly from the sets of terminals and
primitive functionals. Each of the individual programs is
then executed and its fitness (at providing a solution to
the designated problem) determined. The most
successful individuals are then used as ‘parents’ for a
new generation of individuals. These offspring are
created using special reproductive operators (crossover,
mutation and direct reproduction) that combine parts of
the parent programs in such a way as to preserve valid
program syntax in the members of the new generation.
The probability of selecting each of the reproductive
operators is an algorithm control parameter of the GP
that must be decided upon by the designer.
The entire process is continued over a number of
generations until a pre-defined fitness criterion is
reached or until a set number of generations, e.g. 100,
has been completed. The final population of solutions
can then be inspected manually by the designer for
suitability as a problem solution. The method is known
as Genetic Programming because much of the process
can be explained in terms of an analogy to Darwinian
natural selection.
START
Initialise N random trees to
create Generation 0
Main generation loop
Calculate fitness values
Selection loop
Mutation
Crossover
Pick
Operator
Direct reproduction
Choose one member
from current generation
(fitness related selection)
Mutate selected member
and add to new generation
Choose one member
from current generation
(fitness related selection)
Choose two members from
current generation
(fitness related selection)
Copy unchanged to next generation
Cross over two selected
members and add offspring
to new generation
Loop until N new members
+
Add these to next generation
-
2
Loop until max. generations
or success criterion reached
STOP
3
b
Figure 1: A simple tree-structured program
Basic GP Algorithm
The measure of success of any individual tree program
is calculated by use of some performance criterion or
Figure 2: Flowchart of Basic GP procedure
In essence, the mechanism of GP is the random
juxtaposition of ‘genes’ from successful parents that
produce offspring that are, on average, fitter than their
parents. Its power also lies in its inherently parallel
nature, typically hundreds of points (i.e. candidate
programs) in the ‘solution space’ are evaluated at each
stage of the process and this information is
simultaneously utilised in a probabilistic manner when
producing the search points in the next stage of the
process. Another advantage of GP is that the
performance criterion need not be continuous and/or
differentiable, this means that complex and ‘real-life’
performance measures can be incorporated directly into
the GP algorithm. A flowchart of the basic GP
procedure is shown in Figure (2).
Reproductive Operators
The primary reproductive operator in the GP algorithm
is the crossover operator. The purpose of this is to create
two new trees that contain ‘genetic information’ about
the problem solution inherited from two ‘successful’
parents. A crossover point (node) is randomly selected
in each parent tree. The subtree below this point in the
first tree is then swapped with the subtree below the
crossover point in the other parent, thus creating two
new offspring.
The direct reproduction operator is the simplest of the
GP operators, it copies the selected ‘successful’ parent
unchanged into the new generation. The purpose of this
is to ensure that good genetic material is successfully
propagated from one generation to the next whilst
avoiding the possible disruptive effects of crossover and
mutation.
The mutation operator is used to enhance the diversity
of trees in the new generation thus opening up new
areas of ‘solution space’ to investigation by the GP. It
works by selecting a random crossover point in a single
parent and removing the subtree below it. A randomly
generated subtree then replaces the removed subtree.
CONTROLLERS AS TREE PROGRAMS
Although the purpose of this work was to generate
controllers whose structure was to be optimised to the
particular process/performance criteria in question, it
was decided that the terminal and primitive functional
sets should, as a minimal requirement, be able to easily
represent PID controllers. In addition it was decided to
force the GP to produce recursive controllers i.e.
controllers whose output, u(k), is equal to the controller
output from the previous time step, u(k-1) plus some
corrective term. The reason that this format was chosen
is due to the way of representing the positional form of
the discrete PID controller as a tree structured program.
T s k −1
e ( k ) + T i ∑ e (i ) +
i=0
+ u ss
u (k ) = K
Td
T s (e ( k ) − e ( k − 1 ) )
(1)
Equation (1) describes a discretised (by rectangular
integration) positional PID controller where uss is the
control signal at the steady state, K is the proportional
gain, e(k) is the setpoint error at time k, Ts is the sample
time (assumed small compared to the dominant time
constant of the process to be controlled), Ti is the
equivalent continuous controller integral time and Td is
the equivalent continuous controller derivative time. If,
however, we were to try to represent this as a tree
program using e(k), e(k-1) etc. as terminals then we
would need to define a general purpose integrator as a
member of the primitive functional set (one of the
restrictions of GP is that any primitive functional must
be able to accept any legal subtree, including those
containing nested integrators, as an argument). This
means that the value of any subtree that should happen
to be an argument of the general purpose integrator
functional must be stored for each time step in the
closed loop simulation. This, in itself, is not a problem
but it was felt that the recursive form of the PID
controller, see Equation (2), could be represented more
simply and elegantly as a tree program without recourse
to a specific integrator primitive. The recursive form
can be generated from Equation (1) by the use of the
backward differencing method.
u(k ) = u(k −1) + q0e(k ) + q1e(k −1) + q2e(k − 2) (2)
Where:
Td
k 0 = K 1 +
Ts
T d Ts
k1 = − K 1 + 2 −
T s Ti
Td
k2 = K
Ts
(3)
(4)
(5)
The GP can be forced to produce recursive programs by
fixing the u(k-1) term i.e.
u(k) = u(k-1) + [tree program]
(6)
In which case the tree program shown in Figure (3)
represents the recursive PID controller of Equation (2)
in a very straightforward manner. The recursive form
has the added of advantage of easy switching between
manual and automatic modes during closed loop
simulation of the controller/process system.
Of course, it is possible (and desirable) to construct
controller trees from richer sets of terminals and
functions than utilised in Figure (3). Past process
outputs and inputs can also be used as terminals and
sigmoidal processing elements (similar to those used in
Artificial Neural Networks) and ‘If-Then-Else’ type
constructs can also be used as primitive functionals. In
this way the GP has the potential and flexibility to
develop hybrid forms of control algorithms. In theory at
least, the GP should have the ability to pick out and
assemble the most appropriate controller structure for a
given process model/performance criterion.
A second order linear process of the form of Equation
(8) was used. The manipulated variable is u(k-1) and the
controlled output is y(k).
+
*
q0
*
q1
e(k)
DESCRIPTION OF ARX PROCESS
Ay (k ) = Bu (k − 1) + Cε (k )
*
e(k-1 )
q2
(8)
A, B and C are polynomials in the backshift operator z-1
and ε(k) is a stationary white noise sequence. In this
case the noise term was set to zero to give a
deterministic process.
The process input is constrained at high and low hard
limits and also rate-constrained so that it could not
change too rapidly from one step to the next. The
random setpoint changes presented to the GP were
constrained within a pre-defined operating region.
e(k-2)
Figure 3: Tree representation of recursive PID.
CONTROL PROBLEM FORMULATION
The GP was set up to construct a recursive algorithm of
the form described by Equation (6). The performance
criteria used were based on the transient time responses
of the closed loop systems to setpoint step changes. The
closed loop system is simulated over P randomly
generated setpoint changes ∆Sj (j=1,2,3…P). Each
response is run over n discrete time steps. The
performance criterion, I, used for both example
processes is described by Equation (7).
DESCRIPTION OF NON-LINEAR CSTR
A schematic of the CSTR system is shown in Figure (4).
Feed
R e a c to r
I =
∑
P
j =1
∑
n
k =0
k e ( k ) + r∆ u 2( k )
∆S j
(7)
This performance criterion seeks to minimise the
setpoint error, e(k), using an Integral of the Timeweighted Absolute Error (ITAE) term to give increased
weighting to steady state errors. It also includes a term,
∆u(k), which describes the control effort exerted since
the previous timestep and is weighted using some
constant r . The resulting value for the response is then
scaled according to the magnitude of the current
setpoint step change ∆Sj. This is done in order to prevent
contributions form the large step changes dominating
those form smaller steps when they are finally summed
to give I.
The P setpoint changes were generated every at
generation of the GP, rather than using a fixed set
throughout. This was done in order to produce
controllers that would work well over the entire
operating regions of the processes and not just on the
specific examples presented during the GP’s ‘training’.
Although a performance criteria that is non-stationary
from generation to generation is produced, this is not a
problem for the GP as the candidate programs selection
probabilities are calculated by ranking them relative to
each other rather than using absolute fitness values.
H eat exchanger
C o o lin g W a te r
P ro d u c t
Figure 4: Schematic of CSTR process.
In this system the objective is to control the reactor
temperature with the cooling water to the heat
exchanger as the manipulated input.
Reactant A enters the reactor in the feed stream and the
exothermic reaction AÆBÆC takes place with B and C
as products. The extent of the reaction is dependent on
the reactor temperature and so to control the proportions
of B and C in the product stream the temperature is
manipulated by means of a pump-around heat
exchanger.
As for the ARX process, the CSTR system was also
made subject to constraints on the manipulated input
and, again, the setpoints presented to the GP were fixed
to lie within a certain operating region.
The simulation model used consists of five coupled
differential equations that were derived in a mechanistic
manner based on mass and energy balances over the
reactor and heat exchanger. In order to simulate the
process the differential equations were integrated over a
fixed step-length using a fourth order Runge-Kutta
method.
in GP design of signal processing algorithms and
Esparcia-Alcazar and Sharman (8) have used it with a
GP in evolving the structures of recurrent neural
networks.
GP CONFIGURATION
RESULTS
Primitive Functionals:
200
100 generations elapsed
5 per generation
ARX: 100 CSTR: 140
Approximately 100 nodes
Current and past two values
of setpoint error, e(k), and
output, y(k).
+, -, *, /, exponential
function, square root
function , natural log
function, the backshift
operator z-1, the If-Then-Else
operator and a non-linear
transfer function.
Table 1: Some GP configuration details
The closed-loop simulation of the systems in response
to the setpoint changes is the most CPU intensive part of
the GP optimisation process and places a real limit on
the number of setpoint change scenarios that may be
used in ‘training’ the controllers. For the same reasons it
was found to be necessary to limit the size of the
controller trees (each of which is executed at every
discrete time step of each setpoint change response) in
order to keep computing time to a reasonable level.
Some of the primitive functionals cited in Table (1) are
worthy of further explanation:
The If-Then-Else function compares the values of its
first two arguments. If the first is smaller the third
argument is executed otherwise the fourth argument is
executed.
The backshift operator returns the value of its single
argument that was calculated one time-step previously.
The use of this operator allows values of errors and
outputs further back in time than explicitly presented in
the terminal set to be accessed.
The non-linear transfer function is a sigmoidal function
whose degree of non-linearity depends on the product of
the two arguments. Sharman et al. (7) used this function
60
Process Output
Population Size:
Termination Criterion:
Number of setpoint
changes:
Number of time steps per
setpoint change:
Max. allowed program
size:
Terminals:
For each process the GP was run approximately 25
times and, after each run, the best of run controllers
were extracted and then tested on a number of setpoint
changes across the operating regions that were defined
during the training process.
For both the CSTR and the ARX systems it was found
that approximately one third of the controllers gave
unacceptable performances i.e. they became unstable,
gave very large steady-state offsets or extremely
oscillatory responses to the step changes in setpoint.
Another third of the controllers could be simplified to
recursive form PID controllers. These gave varying
performances depending on the evolved parameter
values but they were generally equivalent to those
obtained through a trial and error PID tuning exercise.
The remaining third of the controllers were not of the
PID type and usually included many of the ‘non-linear’
primitive functionals including the non-linear transfer
function and the If-Then-Else operator.
Typical responses for the better GP designed controllers
to setpoint step changes are shown in Figure (5) for the
ARX process and Figure (6) for the CSTR system. In
addition, the manipulated input plots are shown beneath
the output responses. In each of these cases, for
comparison purposes, the response to the same setpoint
changes is shown for a recursive PID controller. These
comparison PID controllers were tuned by first using a
trial and error method to determine reasonable values of
the parameters and then using a Simplex optimisation
method to further fine-tune these parameters to the
performance criterion I.
50
40
30
20
0
50
100
150
0
50
100
150
200
250
200
250
300
350
400
60
Control Signal
In configuring the GP the population size, reproduction
operator selection probabilities, number of fitness cases
per generation (i.e. number of setpoint changes) and the
total number of generations were decided upon. This
was accomplished in an heuristic manner based, in part,
on some trial runs and also on values frequently cited in
the literature.
Table (1) shows the values of the most important GP
algorithm control parameters as well as a summary of
the terminal and primitive functional sets used.
40
20
0
300
350
T im e s te p
Figure (5): ARX system. GP and PID responses
Solid Line: GP controller
Dotted Line: PID controller
Dashed line: Setpoint
400
Reactor Temp. (K)
370
360
350
340
330
320
Cooling Water Flow (litres/sec)
0
100
200
100
200
300
400
500
300
400
500
600
10
8
6
4
2
because they are not easy to interpret upon inspection
and do not appear to have any physical interpretation.
Another problem is that at no part of the design process
is the stability of the closed-loop system explicitly
considered. For example, even with comprehensive
post-run testing, there is no guarantee that the controller
will work for all step changes presented to them. This
would obviously be a major issue in any practical
application.
Future work will consider the effect of using a transient
response performance envelope based fitness criterion
as well as improving the robustness properties of the GP
designed algorithms.
0
0
600
Time (sec)
Figure 6: CSTR system. GP and PID responses
Solid Line: GP controller
Dotted Line: PID controller
Dashed line: Setpoint
It can be seen that in both cases the GP controller offers
similar performance to the PID controllers. For the
ARX system the PID controlled output rises to the
setpoint more quickly but in the case of the CSTR
system the controlled output plots are virtually identical.
In addition to the testing against new step change
scenarios the GP and PID comparisons controller were
also tested for their performance on disturbance
rejection problems for the same systems with load
changes applied to the outputs. The PID comparison
controllers successfully managed to regulate the
processes at steady state but the GP controllers
frequently gave very poor responses or became
unstable.
The actual structure of the non-linear controller
algorithm varies greatly depending on the run and the
algorithms themselves are usually very lengthy and
difficult to follow. Hence, it does not seem likely that
the GP is converging to any particular optimal controller
structure. It seems that GP empirically determines a
number of solutions for a given problem and there is no
real way of analytically determining even simple
properties of these solutions without actually executing
them as programs.
CONCLUSIONS
It has been demonstrated successfully that Genetic
Programming is capable of constructing satisfactory
controllers for a specific class of control objectives. By
randomly generating the training cases at each
generation the controllers have ‘learnt’ to control
scenarios they were not explicitly presented with during
the training phase. This generalisation property does not
always extend to different control objectives, however,
when presented load regulation problems the GP
controllers tended to fail. It is difficult to analyse the GP
designed controller algorithms to see why they do this
BIBLIOGRAPHY
1.
Isermann R, 1989, “Digital Control Systems”, Vol.
1, 2nd Edition, Springer-Verlag.
2.
Koza J, 1992, “Genetic programming: On the
programming of computers by means of natural
selection”, The MIT press.
3.
Alba E, Cotta C and Troya J M, 1996, “Typeconstrained Genetic Programming for Rule-Base
Definition in Fuzzy Logic Controllers”, GP ‘96Proceedings of the 1st Annual Conference, July 2831, 1996.
4.
Howley B, 1996, “Genetic Programming of NearMinimum-Time Spacecraft Attitude Maneuvers”,
GP ‘96- Proceedings of the 1st Annual Conference,
July 28-31, 1996.
5.
Howley B, 1997, “Genetic Programming and
Parametric Sensitivity: a Case Study in Dynamic
Control of a Two-Link Manipulator’, GP ’97Proceedings of the Second Annual Conference, July
13-16 1997
6.
Willis M, Hiden H, Marenbach P and McKay B,
1997, “Genetic Programming: An Introduction and
Survey of Applications”, GALESIA 97 - 2nd
International Conference on Genetic Algorithms in
Engineering Systems, September 1-4 1997.
7.
Sharman K C, Esparcia-Alcazar A I and Li Y,
1995, “Evolving Signal Processing Algorithms
using Genetic Programming”, IEE conference on
Genetic Algorithms in Engineering Systems,
Conference Number 414, September 12-14 1995.
8.
Sharman K C and Esparcia-Alcazar AI, 1997,
“Evolving Recurrent Neural Network Architectures
by Genetic programming’, GP ’97- Proceedings of
the Second Annual Conference, July 13-16 1997.
DP Searson, MJ Willis and GA Montague
University of Newcastle upon Tyne, England.
The Genetic Programming (GP) architecture offers a
potentially powerful framework for ‘intelligent’
controller design due to its inherent matching of
structure to utility. This article describes the use of GP
to design discrete time single loop controllers for two
different processes: a linear Auto-Regressive
eXogeneous (ARX) type system and a non-linear
simulated Continuous Stirred Tank Reactor (CSTR),
both with constraints on the actuator outputs. It is
demonstrated that the GP methodology is capable of
designing recursive controllers that, for a specific class
of control objectives, offer similar performance to PID
controllers on the nominal systems. The advantages and
disadvantages of controllers designed in this way are
also discussed.
INTRODUCTION
One approach in control systems design is to optimise
the parameters of a controller of known structure. This
can be done using numerical methods, adaptive
techniques or manual tuning methods. These systems
are then known as parameter optimised control systems
[Isermann (1)] and a common example of this type of
system is the widely known PID controller.
A contrasting approach is the use of structure optimal
control systems in which both the structure and the
parameters of the controller are matched in some way to
the parameters and structure of the process model
[Isermann (1)]. The deadbeat and Dahlin controllers are
examples of structure optimal systems. The applicability
of these types of controller is quite restricted, however,
because processes with non-linearities and constraints
generally do not lend themselves to the analytical
approach that the structure optimal control design
method requires. Another problem is that the designer
must know a priori what form the controller is to take.
Ideally, it would be desirable to have a method that,
given a model of the process, could generate a controller
without the designer needing to specify beforehand
what form it should take. It would also be desirable that
this method could be used with non-linear process
models of arbitrary structure as well as utilising any
additional information that is available about
constraints. The advantage of this over parameter
optimised controller design would be that no possibly
performance-restricting decisions on the controller
structure would need to be made, and better closed-loop
control could be achieved.
In reality, this is quite a demanding design problem: any
form of automated design procedure cannot easily
utilise qualitative knowledge about the process that the
control engineer normally has access to and must rely
on closed-loop simulations to extract ‘knowledge’ about
the problem. Additionally, any controllers produced will
have no guarantee of good robustness or stability
properties unless provision for this is specifically made
in the automated design procedure. This, however,
would not be easily achievable for controllers of
arbitrary structure.
This article does not seek to fully address the latter
problem but instead concentrates on the more tractable
task of simple step-change servo control for two
nominal processes. This is accomplished by utilising the
evolutionary computation method called Genetic
Programming (GP), which was developed by Koza (2),
to evolve the structure and parameters of simple
recursive single loop controllers.
The ideas behind GP are first explained and then the
control problem formulation, GP configuration and the
two example processes are described. Finally, the
results are presented and discussed.
INTRODUCTION TO GP
GP is a method of getting computers to learn how to
solve problems without being explicitly programmed to
do so. Koza (2) popularised the technique and applied it
to a number of problems e.g. symbolic regression, and
finite state automata. Although most applications of GP
are in the field of computing science recently there have
been a number of engineering applications.
In the field of control Alba et al. (3) used GP based
fuzzy logic controllers on the cart-centering control
problem first tackled using ordinary GP in Koza (2).
Howley (4,5) used GP to derive algorithms to perform
specific spacecraft attitude manoeuvres and also to
derive control laws for a two-link payload manipulator
problem. For further details of engineering applications
the survey given in Willis et al. (6) should be referred
to.
The study of GP in its own right, and the theory
underlying its mechanism, are also active areas of
research: there are many variants on the basic theme and
currently there is no generally accepted ‘best’ way to set
up a Genetic Programming procedure to solve a
problem.
Tree structure of GP
Given a problem, GP works by maintaining a
population of individuals, each of which is a computer
program that can be ‘executed’ to give a potential
solution to the problem. These candidate programs are
encoded as parse trees (rather than lines of computer
code). Figure (1) depicts a simple example of a treestructured program. The result of the program is always
returned at the top of the tree. In the case of Figure (1)
the program result is (3-b)+2.
Each node of the tree is either a terminal node or a
primitive functional node. Terminals are usually
specified to be variables (from the problem at hand) that
the designer has previously identified to be ‘inputs’ to
the problem. However, terminals can also be defined as
constants that are generated randomly when initially
creating the tree.
Primitive functionals are usually specified to be
‘operators’ that the designer believes will enable a
correct solution to be constructed from the terminals.
The selection of the appropriate primitive functionals
for a problem is, again, a choice that must be made by
the designer. A typical primitive functional set would
probably include the standard operators of addition,
multiplication, division and subtraction as well as any
other operators that the designer suspects to be of use in
the context of the problem. Koza (2) states that ability
of the sets of terminals and primitive functionals to
solve the selected problem is known as the sufficiency
property of the GP and that for certain classes of
problems, e.g. Boolean problems, the sufficient
primitive functional set is known exactly. As we move
towards more complex problems, however, the
minimally sufficient requirements for a problem
solution become less clear.
In the case of Figure (1) the terminals that have been
utilised to construct the tree are the constants 2 and 3
and the variable b. The primitive functionals used are
the scalar operators ‘+’ and ‘-‘. Generally, tree
programs do not need to use all the terminals or
primitive functionals available as the GP algorithm
assembles them in a probabilistic manner during a run.
‘fitness function’. The form of the fitness function is
dependent on the problem at hand and must be selected
by the designer.
The initial population of individual programs is
generated randomly from the sets of terminals and
primitive functionals. Each of the individual programs is
then executed and its fitness (at providing a solution to
the designated problem) determined. The most
successful individuals are then used as ‘parents’ for a
new generation of individuals. These offspring are
created using special reproductive operators (crossover,
mutation and direct reproduction) that combine parts of
the parent programs in such a way as to preserve valid
program syntax in the members of the new generation.
The probability of selecting each of the reproductive
operators is an algorithm control parameter of the GP
that must be decided upon by the designer.
The entire process is continued over a number of
generations until a pre-defined fitness criterion is
reached or until a set number of generations, e.g. 100,
has been completed. The final population of solutions
can then be inspected manually by the designer for
suitability as a problem solution. The method is known
as Genetic Programming because much of the process
can be explained in terms of an analogy to Darwinian
natural selection.
START
Initialise N random trees to
create Generation 0
Main generation loop
Calculate fitness values
Selection loop
Mutation
Crossover
Pick
Operator
Direct reproduction
Choose one member
from current generation
(fitness related selection)
Mutate selected member
and add to new generation
Choose one member
from current generation
(fitness related selection)
Choose two members from
current generation
(fitness related selection)
Copy unchanged to next generation
Cross over two selected
members and add offspring
to new generation
Loop until N new members
+
Add these to next generation
-
2
Loop until max. generations
or success criterion reached
STOP
3
b
Figure 1: A simple tree-structured program
Basic GP Algorithm
The measure of success of any individual tree program
is calculated by use of some performance criterion or
Figure 2: Flowchart of Basic GP procedure
In essence, the mechanism of GP is the random
juxtaposition of ‘genes’ from successful parents that
produce offspring that are, on average, fitter than their
parents. Its power also lies in its inherently parallel
nature, typically hundreds of points (i.e. candidate
programs) in the ‘solution space’ are evaluated at each
stage of the process and this information is
simultaneously utilised in a probabilistic manner when
producing the search points in the next stage of the
process. Another advantage of GP is that the
performance criterion need not be continuous and/or
differentiable, this means that complex and ‘real-life’
performance measures can be incorporated directly into
the GP algorithm. A flowchart of the basic GP
procedure is shown in Figure (2).
Reproductive Operators
The primary reproductive operator in the GP algorithm
is the crossover operator. The purpose of this is to create
two new trees that contain ‘genetic information’ about
the problem solution inherited from two ‘successful’
parents. A crossover point (node) is randomly selected
in each parent tree. The subtree below this point in the
first tree is then swapped with the subtree below the
crossover point in the other parent, thus creating two
new offspring.
The direct reproduction operator is the simplest of the
GP operators, it copies the selected ‘successful’ parent
unchanged into the new generation. The purpose of this
is to ensure that good genetic material is successfully
propagated from one generation to the next whilst
avoiding the possible disruptive effects of crossover and
mutation.
The mutation operator is used to enhance the diversity
of trees in the new generation thus opening up new
areas of ‘solution space’ to investigation by the GP. It
works by selecting a random crossover point in a single
parent and removing the subtree below it. A randomly
generated subtree then replaces the removed subtree.
CONTROLLERS AS TREE PROGRAMS
Although the purpose of this work was to generate
controllers whose structure was to be optimised to the
particular process/performance criteria in question, it
was decided that the terminal and primitive functional
sets should, as a minimal requirement, be able to easily
represent PID controllers. In addition it was decided to
force the GP to produce recursive controllers i.e.
controllers whose output, u(k), is equal to the controller
output from the previous time step, u(k-1) plus some
corrective term. The reason that this format was chosen
is due to the way of representing the positional form of
the discrete PID controller as a tree structured program.
T s k −1
e ( k ) + T i ∑ e (i ) +
i=0
+ u ss
u (k ) = K
Td
T s (e ( k ) − e ( k − 1 ) )
(1)
Equation (1) describes a discretised (by rectangular
integration) positional PID controller where uss is the
control signal at the steady state, K is the proportional
gain, e(k) is the setpoint error at time k, Ts is the sample
time (assumed small compared to the dominant time
constant of the process to be controlled), Ti is the
equivalent continuous controller integral time and Td is
the equivalent continuous controller derivative time. If,
however, we were to try to represent this as a tree
program using e(k), e(k-1) etc. as terminals then we
would need to define a general purpose integrator as a
member of the primitive functional set (one of the
restrictions of GP is that any primitive functional must
be able to accept any legal subtree, including those
containing nested integrators, as an argument). This
means that the value of any subtree that should happen
to be an argument of the general purpose integrator
functional must be stored for each time step in the
closed loop simulation. This, in itself, is not a problem
but it was felt that the recursive form of the PID
controller, see Equation (2), could be represented more
simply and elegantly as a tree program without recourse
to a specific integrator primitive. The recursive form
can be generated from Equation (1) by the use of the
backward differencing method.
u(k ) = u(k −1) + q0e(k ) + q1e(k −1) + q2e(k − 2) (2)
Where:
Td
k 0 = K 1 +
Ts
T d Ts
k1 = − K 1 + 2 −
T s Ti
Td
k2 = K
Ts
(3)
(4)
(5)
The GP can be forced to produce recursive programs by
fixing the u(k-1) term i.e.
u(k) = u(k-1) + [tree program]
(6)
In which case the tree program shown in Figure (3)
represents the recursive PID controller of Equation (2)
in a very straightforward manner. The recursive form
has the added of advantage of easy switching between
manual and automatic modes during closed loop
simulation of the controller/process system.
Of course, it is possible (and desirable) to construct
controller trees from richer sets of terminals and
functions than utilised in Figure (3). Past process
outputs and inputs can also be used as terminals and
sigmoidal processing elements (similar to those used in
Artificial Neural Networks) and ‘If-Then-Else’ type
constructs can also be used as primitive functionals. In
this way the GP has the potential and flexibility to
develop hybrid forms of control algorithms. In theory at
least, the GP should have the ability to pick out and
assemble the most appropriate controller structure for a
given process model/performance criterion.
A second order linear process of the form of Equation
(8) was used. The manipulated variable is u(k-1) and the
controlled output is y(k).
+
*
q0
*
q1
e(k)
DESCRIPTION OF ARX PROCESS
Ay (k ) = Bu (k − 1) + Cε (k )
*
e(k-1 )
q2
(8)
A, B and C are polynomials in the backshift operator z-1
and ε(k) is a stationary white noise sequence. In this
case the noise term was set to zero to give a
deterministic process.
The process input is constrained at high and low hard
limits and also rate-constrained so that it could not
change too rapidly from one step to the next. The
random setpoint changes presented to the GP were
constrained within a pre-defined operating region.
e(k-2)
Figure 3: Tree representation of recursive PID.
CONTROL PROBLEM FORMULATION
The GP was set up to construct a recursive algorithm of
the form described by Equation (6). The performance
criteria used were based on the transient time responses
of the closed loop systems to setpoint step changes. The
closed loop system is simulated over P randomly
generated setpoint changes ∆Sj (j=1,2,3…P). Each
response is run over n discrete time steps. The
performance criterion, I, used for both example
processes is described by Equation (7).
DESCRIPTION OF NON-LINEAR CSTR
A schematic of the CSTR system is shown in Figure (4).
Feed
R e a c to r
I =
∑
P
j =1
∑
n
k =0
k e ( k ) + r∆ u 2( k )
∆S j
(7)
This performance criterion seeks to minimise the
setpoint error, e(k), using an Integral of the Timeweighted Absolute Error (ITAE) term to give increased
weighting to steady state errors. It also includes a term,
∆u(k), which describes the control effort exerted since
the previous timestep and is weighted using some
constant r . The resulting value for the response is then
scaled according to the magnitude of the current
setpoint step change ∆Sj. This is done in order to prevent
contributions form the large step changes dominating
those form smaller steps when they are finally summed
to give I.
The P setpoint changes were generated every at
generation of the GP, rather than using a fixed set
throughout. This was done in order to produce
controllers that would work well over the entire
operating regions of the processes and not just on the
specific examples presented during the GP’s ‘training’.
Although a performance criteria that is non-stationary
from generation to generation is produced, this is not a
problem for the GP as the candidate programs selection
probabilities are calculated by ranking them relative to
each other rather than using absolute fitness values.
H eat exchanger
C o o lin g W a te r
P ro d u c t
Figure 4: Schematic of CSTR process.
In this system the objective is to control the reactor
temperature with the cooling water to the heat
exchanger as the manipulated input.
Reactant A enters the reactor in the feed stream and the
exothermic reaction AÆBÆC takes place with B and C
as products. The extent of the reaction is dependent on
the reactor temperature and so to control the proportions
of B and C in the product stream the temperature is
manipulated by means of a pump-around heat
exchanger.
As for the ARX process, the CSTR system was also
made subject to constraints on the manipulated input
and, again, the setpoints presented to the GP were fixed
to lie within a certain operating region.
The simulation model used consists of five coupled
differential equations that were derived in a mechanistic
manner based on mass and energy balances over the
reactor and heat exchanger. In order to simulate the
process the differential equations were integrated over a
fixed step-length using a fourth order Runge-Kutta
method.
in GP design of signal processing algorithms and
Esparcia-Alcazar and Sharman (8) have used it with a
GP in evolving the structures of recurrent neural
networks.
GP CONFIGURATION
RESULTS
Primitive Functionals:
200
100 generations elapsed
5 per generation
ARX: 100 CSTR: 140
Approximately 100 nodes
Current and past two values
of setpoint error, e(k), and
output, y(k).
+, -, *, /, exponential
function, square root
function , natural log
function, the backshift
operator z-1, the If-Then-Else
operator and a non-linear
transfer function.
Table 1: Some GP configuration details
The closed-loop simulation of the systems in response
to the setpoint changes is the most CPU intensive part of
the GP optimisation process and places a real limit on
the number of setpoint change scenarios that may be
used in ‘training’ the controllers. For the same reasons it
was found to be necessary to limit the size of the
controller trees (each of which is executed at every
discrete time step of each setpoint change response) in
order to keep computing time to a reasonable level.
Some of the primitive functionals cited in Table (1) are
worthy of further explanation:
The If-Then-Else function compares the values of its
first two arguments. If the first is smaller the third
argument is executed otherwise the fourth argument is
executed.
The backshift operator returns the value of its single
argument that was calculated one time-step previously.
The use of this operator allows values of errors and
outputs further back in time than explicitly presented in
the terminal set to be accessed.
The non-linear transfer function is a sigmoidal function
whose degree of non-linearity depends on the product of
the two arguments. Sharman et al. (7) used this function
60
Process Output
Population Size:
Termination Criterion:
Number of setpoint
changes:
Number of time steps per
setpoint change:
Max. allowed program
size:
Terminals:
For each process the GP was run approximately 25
times and, after each run, the best of run controllers
were extracted and then tested on a number of setpoint
changes across the operating regions that were defined
during the training process.
For both the CSTR and the ARX systems it was found
that approximately one third of the controllers gave
unacceptable performances i.e. they became unstable,
gave very large steady-state offsets or extremely
oscillatory responses to the step changes in setpoint.
Another third of the controllers could be simplified to
recursive form PID controllers. These gave varying
performances depending on the evolved parameter
values but they were generally equivalent to those
obtained through a trial and error PID tuning exercise.
The remaining third of the controllers were not of the
PID type and usually included many of the ‘non-linear’
primitive functionals including the non-linear transfer
function and the If-Then-Else operator.
Typical responses for the better GP designed controllers
to setpoint step changes are shown in Figure (5) for the
ARX process and Figure (6) for the CSTR system. In
addition, the manipulated input plots are shown beneath
the output responses. In each of these cases, for
comparison purposes, the response to the same setpoint
changes is shown for a recursive PID controller. These
comparison PID controllers were tuned by first using a
trial and error method to determine reasonable values of
the parameters and then using a Simplex optimisation
method to further fine-tune these parameters to the
performance criterion I.
50
40
30
20
0
50
100
150
0
50
100
150
200
250
200
250
300
350
400
60
Control Signal
In configuring the GP the population size, reproduction
operator selection probabilities, number of fitness cases
per generation (i.e. number of setpoint changes) and the
total number of generations were decided upon. This
was accomplished in an heuristic manner based, in part,
on some trial runs and also on values frequently cited in
the literature.
Table (1) shows the values of the most important GP
algorithm control parameters as well as a summary of
the terminal and primitive functional sets used.
40
20
0
300
350
T im e s te p
Figure (5): ARX system. GP and PID responses
Solid Line: GP controller
Dotted Line: PID controller
Dashed line: Setpoint
400
Reactor Temp. (K)
370
360
350
340
330
320
Cooling Water Flow (litres/sec)
0
100
200
100
200
300
400
500
300
400
500
600
10
8
6
4
2
because they are not easy to interpret upon inspection
and do not appear to have any physical interpretation.
Another problem is that at no part of the design process
is the stability of the closed-loop system explicitly
considered. For example, even with comprehensive
post-run testing, there is no guarantee that the controller
will work for all step changes presented to them. This
would obviously be a major issue in any practical
application.
Future work will consider the effect of using a transient
response performance envelope based fitness criterion
as well as improving the robustness properties of the GP
designed algorithms.
0
0
600
Time (sec)
Figure 6: CSTR system. GP and PID responses
Solid Line: GP controller
Dotted Line: PID controller
Dashed line: Setpoint
It can be seen that in both cases the GP controller offers
similar performance to the PID controllers. For the
ARX system the PID controlled output rises to the
setpoint more quickly but in the case of the CSTR
system the controlled output plots are virtually identical.
In addition to the testing against new step change
scenarios the GP and PID comparisons controller were
also tested for their performance on disturbance
rejection problems for the same systems with load
changes applied to the outputs. The PID comparison
controllers successfully managed to regulate the
processes at steady state but the GP controllers
frequently gave very poor responses or became
unstable.
The actual structure of the non-linear controller
algorithm varies greatly depending on the run and the
algorithms themselves are usually very lengthy and
difficult to follow. Hence, it does not seem likely that
the GP is converging to any particular optimal controller
structure. It seems that GP empirically determines a
number of solutions for a given problem and there is no
real way of analytically determining even simple
properties of these solutions without actually executing
them as programs.
CONCLUSIONS
It has been demonstrated successfully that Genetic
Programming is capable of constructing satisfactory
controllers for a specific class of control objectives. By
randomly generating the training cases at each
generation the controllers have ‘learnt’ to control
scenarios they were not explicitly presented with during
the training phase. This generalisation property does not
always extend to different control objectives, however,
when presented load regulation problems the GP
controllers tended to fail. It is difficult to analyse the GP
designed controller algorithms to see why they do this
BIBLIOGRAPHY
1.
Isermann R, 1989, “Digital Control Systems”, Vol.
1, 2nd Edition, Springer-Verlag.
2.
Koza J, 1992, “Genetic programming: On the
programming of computers by means of natural
selection”, The MIT press.
3.
Alba E, Cotta C and Troya J M, 1996, “Typeconstrained Genetic Programming for Rule-Base
Definition in Fuzzy Logic Controllers”, GP ‘96Proceedings of the 1st Annual Conference, July 2831, 1996.
4.
Howley B, 1996, “Genetic Programming of NearMinimum-Time Spacecraft Attitude Maneuvers”,
GP ‘96- Proceedings of the 1st Annual Conference,
July 28-31, 1996.
5.
Howley B, 1997, “Genetic Programming and
Parametric Sensitivity: a Case Study in Dynamic
Control of a Two-Link Manipulator’, GP ’97Proceedings of the Second Annual Conference, July
13-16 1997
6.
Willis M, Hiden H, Marenbach P and McKay B,
1997, “Genetic Programming: An Introduction and
Survey of Applications”, GALESIA 97 - 2nd
International Conference on Genetic Algorithms in
Engineering Systems, September 1-4 1997.
7.
Sharman K C, Esparcia-Alcazar A I and Li Y,
1995, “Evolving Signal Processing Algorithms
using Genetic Programming”, IEE conference on
Genetic Algorithms in Engineering Systems,
Conference Number 414, September 12-14 1995.
8.
Sharman K C and Esparcia-Alcazar AI, 1997,
“Evolving Recurrent Neural Network Architectures
by Genetic programming’, GP ’97- Proceedings of
the Second Annual Conference, July 13-16 1997.