Criteria for a tournament the World Prof
2009
Journal of the Operational Research Society (2009) 60, 1670 --1673
Operational Research Society Ltd. All rights reserved. 0160-5682/09
www.palgrave-journals.com/jors/
Criteria for a tournament: the World Professional
Snooker Championship
SR Clarke1 , JM Norman2,∗ and CB Stride2
1 Swinburne
University, Hawthorn, Victoria, Australia; and 2 University of Sheffield, Sheffield, UK
Desirable qualities of a tournament are fairness (the better the player, the better his chance of success), balance
(few one-sided matches) and efficiency (long enough to benefit the more skillful yet being completed within
schedule). The World Professional Snooker Championship is examined to see how well it meets these criteria.
Journal of the Operational Research Society (2009) 60, 1670 – 1673. doi:10.1057/jors.2008.126
Published online 19 November 2008
A
U
TH
O
O
R
A well-designed tournament should be fair, balanced and efficient. A fair tournament is one in which the better the player,
the greater his chance of winning. In practice, it would be
unfair, for example, if the more skillful players in the tournament were made to play elimination matches among themselves before playing against the less skillful players. An
objective for a balanced tournament might be to reduce the
number of one-sided matches, which might be achieved if
players enter the tournament at different stages, according to
their ability. Thus, a balanced tournament might be one in
which each competitor has the same chance of winning his
first match. Finally, an efficient tournament might be one in
which the number of games played is just sufficient to establish with some probability the best player, having regard to
the expense of a long tournament and the need to provide
entertainment for spectators.
Glickman, 2007). Appleton (1995) has simulated various tournament designs to assess the likelihood of the winner being
the best of the entrants. In an analysis of a particular competition, Clarke (1996) shows how different tournament designs
can change the winning probabilities of competitors.
The issue of balance has given rise to several papers. Konig
(2000) discusses the competitive balance of Dutch soccer. He
draws attention to the uncertainty in the outcome of an individual match and the uncertainty of the outcome of the tournament as a whole, a distinction also made by Scarf and Bilbao
(2006). Soccer is in a sense, an unstable game in that because
few goals are scored, match results often do not reflect the
relative abilities of the two teams taking part (Lundh, 2006).
Soccer’s instability in this sense may be contrasted with the
stability of racquet sports, where a player with a little more
than 50% probability of winning a point may have a high
probability of winning the match (Clarke and Norman, 1979).
An unbalanced match is of little benefit to players or spectators and many tournament organizers arrange for competitors to enter at different stages according to their ability. In
the FA Cup, the teams in the top three divisions of English
football enter only at the third round proper. Perhaps seeding
is appropriate only in the later stages of a tournament, when
players are of high and near equal ability.
The issue of efficiency has attracted less attention, perhaps
because the measure of efficiency depends on the nature of
the tournament. Schutz (1970) develops a method for evaluating scoring systems in sport and compares scoring systems
in tennis. Percy (2007) evaluates rule changes in badminton,
introduced to make the game faster and more entertaining.
Scarf and Bilbao (2006) suggest several metrics to measure
the success of a sporting contest, but these metrics are
largely concerned with the issues of fairness and competitive
balance. Their paper deals with the contribution that operational research can make to the design of sporting contests.
C
Introduction
PY
Keywords: sports; statistics; snooker
Related work
There is a copious literature concerned with tournament design, much of it concerned with the issue of
fairness—ensuring that the best player is the player most
likely to win and that players progress in the tournament in
accordance with their skill (see, eg McGarry and Schutz,
1997; McGarry, 1998). Seeding players entering the tournament has attracted much attention (Hwang, 1982). Israel
(1982) is one of several authors to point out that seeding
may not ensure that the strongest players necessarily do well,
though adaptive seeding (modifying seeding after each round
of a tournament) may remedy this (Chen and Hwang, 1988;
∗ Correspondence: JM Norman, University of Sheffield, 9 Mappin Street,
Sheffield S1 4DT, UK.
E-mail: j.norman@sheffield.ac.uk
SR Clarke et al—Criteria for a tournament
A
U
TH
O
PY
R
The championship takes place in the Crucible Theatre in
Sheffield each year. A total of 32 players enter the first round
proper, half of them the first 16 in the world rankings and the
other half qualifiers from a preliminary tournament. The 16
top-ranking players are seeded according to their rankings so
that, for example, if the No. 1 player beats his (qualifying)
opponent in the first round he will meet in the second round
the No. 16 player if the No. 16 player has beaten his own
opponent. If the No. 1 player and the No. 2 player continue
to win their matches, they will meet at last in the final round.
The qualifying entrants are randomly allocated to matches
against the 16 top-ranked players. In 2007, there were 85
entrants to the qualifying rounds. Ten competed in the first
round, the five winners entering the second round, joining 27
other players. In each of the third, fourth and fifth rounds,
16 players joined the tournament, the winners proceeding to
the next round or to the first round of the tournament proper.
Players entered these preliminary rounds according to their
status in the rankings; the higher their ranking, the later they
entered the tournament.
A similar arrangement was followed in the years from 2004
to 2006, though in 2004 with a much larger than usual entry;
the first full round had 64 players, who competed among
each other to determine 32 entrants to the next round, these
winners playing each other to determine who should enter the
following round and play 16 entrants from the ranking lists.
In each of the preliminary rounds and in the first round of
the tournament proper, the winner is the first to win 10 frames.
In the televised rounds, the number of frames in each match
has varied over the years. In the 2007 Championship, in the
second round and the quarter finals the winner was the first
player to win 13 frames, in the semi-finals the first to win 17
frames and in the final the first to win 18 frames.
Players claim that a longer match gives them more chance
of showing their superiority and it is true that, other things
being equal, a weaker player has a greater chance of winning
in a shorter match.
In this section, we consider the last three qualifying rounds
and the first round of the championship proper, which takes
place at the Crucible theatre under television cameras. Thus
we consider four rounds in each of 4 years, with 16 matches
in each round. Of the 256 matches, however, two were
walkovers and one was unfinished, as one of the players
withdrew through illness. These three matches have been
excluded from the analysis.
For each match we recorded the score of the losing player,
the year in which it was played, the round, and whether the
match was won or lost by the player entering the tournament
in that round, his lower-ranked opponent having qualified by
winning his match in the previous round. We also recorded
the rank of both players in each match, taken from the ranking
lists published at the start of the season. For some players,
mostly entering the third round (the first round we consider)
as qualifiers, rankings were not available and we assigned
to them a rank of 100. All these factors might influence the
outcome of a match and because the outcome of a match
may be considered to have one of two values (whether it is
won or lost by the entering player), a conventional regression
approach is inappropriate. An approach is needed that takes
account of a categorical dependent variable.
A logistic regression was therefore performed to assess the
impact of a number of factors on the likelihood of a new
entrant winning his match. The model contained as independent variables: the year in which the match was played, the
round and the difference in rank between the player entering
the tournament and his opponent. The full model containing
all predictors was not statistically significant and explained
less than 1% of the variance of the dependent variable and as
shown in Table 1, none of the independent variables made a
statistically significant contribution to the model. It seems that
a player entering the tournament has about the same probability of winning his first match, whatever the round at which
he enters.
The scores of the 253 matches are shown in the bar chart
of Figure 1. The scores (with the score of the entering player
given first) are shown in the order 0–10, 1–10, 2–10, . . . ,
O
The World Professional Snooker Championship
Fairness and balance: the probability of winning a match
C
Szymanski (2003) is concerned with the contribution that
economics can make. He remarks that sporting contests
are of interest, particularly financial interest, to organizers,
competitors and fans and comments that much of the existing
literature fails to define the objective of the sporting contest.
How well this objective is realized is a measure of a contest’s
efficiency.
In the remainder of this paper, we analyse a particular
tournament, the annual World Professional Snooker Championship, and determine how well the criteria of fairness,
balance and efficiency are satisfied. The championship,
although watched by millions, has attracted little academic
attention, apart from Colwell and Gillett (1987) who consider
the expected length of the final match of the tournament.
1671
Table 1 Variables in the equation
round
round(1)
round(2)
round(3)
year
year(1)
year(2)
year(3)
rankdiff
Constant
B
SE
−0.020
0.510
0.322
0.363
0.374
0.367
0.136
0.560
0.371
−0.008
−0.225
0.363
0.374
0.369
0.007
0.386
Wald
d.f.
Sig.
2.867
0.003
1.859
0.770
2.661
0.140
2.245
1.009
1.186
0.339
3
1
1
1
3
1
1
1
1
1
0.413
0.956
0.173
0.380
0.447
0.708
0.134
0.315
0.276
0.560
1672
Journal of the Operational Research Society Vol. 60, No. 12
0.531, then his probability of winning a match playing up to
10 is 0.609 and vice versa.
An alternative estimate for p is given by dividing the
number of frames won by the entering player by the total
number of frames played, giving p = 2151/4008 = 0.537. We
note that the average number of frames played in a match is
4008/253 = 15.8.
30
25
15
Independence of frames within matches and
independence between matches
0
-9
-6
-3
0
3
6
9
points
Figure 1
Distribution of match scores.
A
U
TH
O
R
9–10, 10–9, 10–8, 10–7, . . . , 10–0, a ranking from the worst to
the best, from losing 10 frames and winning none to winning
10 frames without reply. For simplicity, the various results
are shown as ‘points’, a measure of the entering player’s
performance. The ‘points’ may be interpreted as net frames
(frames won minus frames lost) won by the entering player,
ranging from −10 to +10.
Of the 253 matches, 154 were won and 99 lost by players
entering the tournament and playing their first match. Thus we
may take 154/253 = 0.609 as an estimate of the probability
that a player entering the tournament wins the match. The
mean losing score in the 154 matches won by the entering
player was 5.63, while the mean losing score in the 99 matches
lost by the entering player was 6.17. The difference is not
surprising, as we would expect a losing player higher up in
the rankings to have a higher score than a losing player lower
in the rankings.
Players take turns taking the opening shot of each frame, so
the hypothesis that frames have independent outcomes is at
least plausible. Using a spreadsheet or another related simple
BASIC program, it is easy to calculate the probability that
a player entering the tournament will score 0, 1, 2, 3 . . . 9
frames if he loses the match, and the probability that his
opponent will score 0, 1, 2, 3 . . . 9 frames if he wins it. These
probabilities can be used to determine the expected number of
scores (out of 253) of the losing player of the 154 matches in
which the entering player wins and the 99 matches in which he
loses. The expected numbers of scores are shown in Table 2,
along with the actual numbers. The entering (higher-ranked)
player’s score comes first.
The 5% chi-square critical value on 13 degrees of freedom
is 22.362, so the total contribution to chi-square of 29.1
is clearly significant. The major contribution to chi-square
occurs at the low scores in matches won by the player entering
the tournament. Perhaps players faced with an opponent above
them in the rankings give up hope of winning the match too
quickly.
PY
5
O
10
C
Count
20
Fairness and balance: the probability of winning a frame
If frames are independent, if a player’s chance of winning a
frame is p and if a match is won by the first player to win n
frames, then the probability of the player winning the match
P can be calculated using a negative binomial or binomial
formulation, and is given by
P=
=
n−1
n−1+i
i=0
2n−1
i=n
i
2n − 1
i
pn (1 − p)i
pi (1 − p)2n−1−i
(1)
Given a value for P of 0.609, we can find the corresponding
value of p, using a spreadsheet or a simple computer program
written in BASIC: this turned out to be p=0.531. If a player’s
probability of winning a frame against a certain opponent is
Table 2
Score
10–0
10–1
10–2
10–3
10–4
10–5
10–6
10–7
10–8
10–9
9–10
8–10
7–10
6–10
5–10
4–10
3–10
2–10
1–10
0–10
Total
Actual
2
6
9
14
17
21
21
30
14
20
15
18
18
10
14
12
⎫
6⎪
⎬
5
0⎪
⎭
1
253
Actual and expected numbers of scores
17
12
Expected
0.5
2.1
5.5
10.3
15.6
20.5
24.0
25.8
25.7
24.1
21.2
20.0
17.7
14.6
11.0
7.4
⎫
4.3 ⎪
⎬
2.0
0.7 ⎪
⎭
0.1
253.0
Contribution to 2
8.1
10.0
1.4
0.1
0.0
0.4
0.7
5.3
0.7
1.8
0.2
0.0
1.4
0.8
2.9
7.1
3.4
29.1
SR Clarke et al—Criteria for a tournament
p
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
2
2
2
3
4
6
10
21
3
3
4
4
6
8
12
21
47
4
5
6
8
10
14
22
39
87
6
8
10
12
16
23
36
64
> 99
9
11
14
19
25
36
56
99
> 99
14
17
21
28
38
54
84
> 99
> 99
21
26
32
42
57
82
> 99
> 99
> 99
34
42
53
69
94
> 99
> 99
> 99
> 99
Appleton DR (1995). May the best man win? Statistician 44(4):
529–538.
Chen R and Hwang FK (1988). Stronger players win more balanced
knockout tournaments. Graphs and Combinatorics 4: 95–99.
Clarke SR (1996). Calculating premiership odds by computer—An
analysis of the AFL final eight playoff system. Asia-Pacific J Opl
Res 13: 89–104.
Clarke SR and Norman JM (1979). Comparison of North American
and international squash scoring systems—Analytical results. Res
Quart 50(4): 723–728.
Colwell DJ and Gillett JR (1987). The expected length of a tournament
final. Math Gazette 71: 129–131.
Glickman ME (2007). Bayesian locally-optimal design of knockout
tournaments. Presented at the International Conference on
Advances in Interdisciplinary Statistics and Combinatorics,
University of North Carolina, Greenboro.
Hwang FK (1982). New concepts for seeding knockout tournaments.
Am Math Month 89: 235–238.
Israel RB (1982). Stronger players need not win more knockout
tournaments. J Am Stat Assoc 76: 950–951.
Konig RH (2000). Competitive balance in Dutch soccer. J Roy Statist
Soc Ser D 49: 419–431.
Lundh T (2006). Which ball is the roundest—A suggested tournament
stability index. J Quant Anal Sports 2(3): 1–21.
McGarry T (1998). On the design of sports tournaments. In: Bennett
J (ed). Statistics in Sport. Edward Arnold: London, 199–217.
McGarry T and Schutz RW (1997). Efficacy of traditional sport
tournament structures. J Opl Res Soc 48: 65–74.
Percy DF (2007). A mathematical analysis of badminton scoring
systems. J Opl Res Soc, 7 November 2007. Advance online
publication doi:10.1057/palgrave.jors.2602528.
Scarf P and Bilbao M (2006). The optimal design of sporting contests.
Salford Business School Working Paper 320/06.
Schutz RW (1970). A mathematical model for evaluating scoring
systems with special reference to tennis. Res Quart 41: 552–561.
Szymanski S (2003). The economic design of sporting contests. J
Econ Lit 41: 1137–1187.
O
R
0.6
TH
0.60
0.59
0.58
0.57
0.56
0.55
0.54
0.53
0.52
Probability of winning the match P
References
PY
Table 3 Number of frames required to ensure certain
probabilities P of winning the match for given probabilities p of
winning the frame
O
As mentioned above, players may reasonably claim that a
longer match gives them more chance to show their superiority
over a weaker player. Equation (1) may be used to determine
how long a match needs to be to give a specified probability
of winning the match to a player with a specified probability
of winning a frame. The analysis reported in Table 2 shows
that the assumptions of independence between frames and
a constant probability of winning each frame do not hold.
Nevertheless, we may use Equation (1) to give useful, if only
approximate, results.
Where the match is won by the first player to win N frames,
Table 3 shows the minimum value of N to ensure a player
whose probability of winning a frame is p has a probability
of at least P of winning the match.
according to their progress in specific tournaments. The foregoing comments suggest that the rankings are a stable measure
of players’ ability.
Determining the number of frames in a match requires a
balance between the need to give the better player an appropriate chance of winning and the need to keep matches short.
In 2008 the qualifying rounds will take place in Prestatyn
over 8 days and later in Sheffield over 4 days. Large crowds
are not drawn to these qualifying rounds and the halls where
the matches are played have to be paid for. Perhaps playing
first-to-ten frames in a match, presumably arrived at through
experience, is a good compromise, as indeed an inspection of
the table above might suggest. Finally, particularly in the televised section of the championship, there needs to be a limit
to the advantage given to the better player: the audience, both
in the Crucible and watching at home, will want to give the
underdog a sporting chance.
C
Efficiency: how many frames are needed in a match?
1673
A
U
From this table, we may note that in a first-to-ten frames
match a player whose probability of winning a frame is 0.53
has a probability 0.60 of winning the match and a player
whose probability of winning a frame is 0.56 has a probability
0.70 of winning the match and so on.
Concluding remarks
In the championship, there are relatively few one-sided
matches, as can be seen in Figure 1 and because of the way
in which players enter the tournament at different stages
according to their position in the rankings, it may be argued
that the tournament gives the better players a better chance
of winning.
It is perhaps surprising that the probability of winning a
match is roughly constant for the player entering the tournament, whatever the round at which he enters. This suggests
that the gradation of ability in the rankings is gradual and
that there are not marked changes in the ability of players
close to each other on the rankings. The rankings are fixed
for a whole year, following the results of the World Championship, and are determined by points awarded to players
Received April 2008;
accepted September 2008 after one revision
Journal of the Operational Research Society (2009) 60, 1670 --1673
Operational Research Society Ltd. All rights reserved. 0160-5682/09
www.palgrave-journals.com/jors/
Criteria for a tournament: the World Professional
Snooker Championship
SR Clarke1 , JM Norman2,∗ and CB Stride2
1 Swinburne
University, Hawthorn, Victoria, Australia; and 2 University of Sheffield, Sheffield, UK
Desirable qualities of a tournament are fairness (the better the player, the better his chance of success), balance
(few one-sided matches) and efficiency (long enough to benefit the more skillful yet being completed within
schedule). The World Professional Snooker Championship is examined to see how well it meets these criteria.
Journal of the Operational Research Society (2009) 60, 1670 – 1673. doi:10.1057/jors.2008.126
Published online 19 November 2008
A
U
TH
O
O
R
A well-designed tournament should be fair, balanced and efficient. A fair tournament is one in which the better the player,
the greater his chance of winning. In practice, it would be
unfair, for example, if the more skillful players in the tournament were made to play elimination matches among themselves before playing against the less skillful players. An
objective for a balanced tournament might be to reduce the
number of one-sided matches, which might be achieved if
players enter the tournament at different stages, according to
their ability. Thus, a balanced tournament might be one in
which each competitor has the same chance of winning his
first match. Finally, an efficient tournament might be one in
which the number of games played is just sufficient to establish with some probability the best player, having regard to
the expense of a long tournament and the need to provide
entertainment for spectators.
Glickman, 2007). Appleton (1995) has simulated various tournament designs to assess the likelihood of the winner being
the best of the entrants. In an analysis of a particular competition, Clarke (1996) shows how different tournament designs
can change the winning probabilities of competitors.
The issue of balance has given rise to several papers. Konig
(2000) discusses the competitive balance of Dutch soccer. He
draws attention to the uncertainty in the outcome of an individual match and the uncertainty of the outcome of the tournament as a whole, a distinction also made by Scarf and Bilbao
(2006). Soccer is in a sense, an unstable game in that because
few goals are scored, match results often do not reflect the
relative abilities of the two teams taking part (Lundh, 2006).
Soccer’s instability in this sense may be contrasted with the
stability of racquet sports, where a player with a little more
than 50% probability of winning a point may have a high
probability of winning the match (Clarke and Norman, 1979).
An unbalanced match is of little benefit to players or spectators and many tournament organizers arrange for competitors to enter at different stages according to their ability. In
the FA Cup, the teams in the top three divisions of English
football enter only at the third round proper. Perhaps seeding
is appropriate only in the later stages of a tournament, when
players are of high and near equal ability.
The issue of efficiency has attracted less attention, perhaps
because the measure of efficiency depends on the nature of
the tournament. Schutz (1970) develops a method for evaluating scoring systems in sport and compares scoring systems
in tennis. Percy (2007) evaluates rule changes in badminton,
introduced to make the game faster and more entertaining.
Scarf and Bilbao (2006) suggest several metrics to measure
the success of a sporting contest, but these metrics are
largely concerned with the issues of fairness and competitive
balance. Their paper deals with the contribution that operational research can make to the design of sporting contests.
C
Introduction
PY
Keywords: sports; statistics; snooker
Related work
There is a copious literature concerned with tournament design, much of it concerned with the issue of
fairness—ensuring that the best player is the player most
likely to win and that players progress in the tournament in
accordance with their skill (see, eg McGarry and Schutz,
1997; McGarry, 1998). Seeding players entering the tournament has attracted much attention (Hwang, 1982). Israel
(1982) is one of several authors to point out that seeding
may not ensure that the strongest players necessarily do well,
though adaptive seeding (modifying seeding after each round
of a tournament) may remedy this (Chen and Hwang, 1988;
∗ Correspondence: JM Norman, University of Sheffield, 9 Mappin Street,
Sheffield S1 4DT, UK.
E-mail: j.norman@sheffield.ac.uk
SR Clarke et al—Criteria for a tournament
A
U
TH
O
PY
R
The championship takes place in the Crucible Theatre in
Sheffield each year. A total of 32 players enter the first round
proper, half of them the first 16 in the world rankings and the
other half qualifiers from a preliminary tournament. The 16
top-ranking players are seeded according to their rankings so
that, for example, if the No. 1 player beats his (qualifying)
opponent in the first round he will meet in the second round
the No. 16 player if the No. 16 player has beaten his own
opponent. If the No. 1 player and the No. 2 player continue
to win their matches, they will meet at last in the final round.
The qualifying entrants are randomly allocated to matches
against the 16 top-ranked players. In 2007, there were 85
entrants to the qualifying rounds. Ten competed in the first
round, the five winners entering the second round, joining 27
other players. In each of the third, fourth and fifth rounds,
16 players joined the tournament, the winners proceeding to
the next round or to the first round of the tournament proper.
Players entered these preliminary rounds according to their
status in the rankings; the higher their ranking, the later they
entered the tournament.
A similar arrangement was followed in the years from 2004
to 2006, though in 2004 with a much larger than usual entry;
the first full round had 64 players, who competed among
each other to determine 32 entrants to the next round, these
winners playing each other to determine who should enter the
following round and play 16 entrants from the ranking lists.
In each of the preliminary rounds and in the first round of
the tournament proper, the winner is the first to win 10 frames.
In the televised rounds, the number of frames in each match
has varied over the years. In the 2007 Championship, in the
second round and the quarter finals the winner was the first
player to win 13 frames, in the semi-finals the first to win 17
frames and in the final the first to win 18 frames.
Players claim that a longer match gives them more chance
of showing their superiority and it is true that, other things
being equal, a weaker player has a greater chance of winning
in a shorter match.
In this section, we consider the last three qualifying rounds
and the first round of the championship proper, which takes
place at the Crucible theatre under television cameras. Thus
we consider four rounds in each of 4 years, with 16 matches
in each round. Of the 256 matches, however, two were
walkovers and one was unfinished, as one of the players
withdrew through illness. These three matches have been
excluded from the analysis.
For each match we recorded the score of the losing player,
the year in which it was played, the round, and whether the
match was won or lost by the player entering the tournament
in that round, his lower-ranked opponent having qualified by
winning his match in the previous round. We also recorded
the rank of both players in each match, taken from the ranking
lists published at the start of the season. For some players,
mostly entering the third round (the first round we consider)
as qualifiers, rankings were not available and we assigned
to them a rank of 100. All these factors might influence the
outcome of a match and because the outcome of a match
may be considered to have one of two values (whether it is
won or lost by the entering player), a conventional regression
approach is inappropriate. An approach is needed that takes
account of a categorical dependent variable.
A logistic regression was therefore performed to assess the
impact of a number of factors on the likelihood of a new
entrant winning his match. The model contained as independent variables: the year in which the match was played, the
round and the difference in rank between the player entering
the tournament and his opponent. The full model containing
all predictors was not statistically significant and explained
less than 1% of the variance of the dependent variable and as
shown in Table 1, none of the independent variables made a
statistically significant contribution to the model. It seems that
a player entering the tournament has about the same probability of winning his first match, whatever the round at which
he enters.
The scores of the 253 matches are shown in the bar chart
of Figure 1. The scores (with the score of the entering player
given first) are shown in the order 0–10, 1–10, 2–10, . . . ,
O
The World Professional Snooker Championship
Fairness and balance: the probability of winning a match
C
Szymanski (2003) is concerned with the contribution that
economics can make. He remarks that sporting contests
are of interest, particularly financial interest, to organizers,
competitors and fans and comments that much of the existing
literature fails to define the objective of the sporting contest.
How well this objective is realized is a measure of a contest’s
efficiency.
In the remainder of this paper, we analyse a particular
tournament, the annual World Professional Snooker Championship, and determine how well the criteria of fairness,
balance and efficiency are satisfied. The championship,
although watched by millions, has attracted little academic
attention, apart from Colwell and Gillett (1987) who consider
the expected length of the final match of the tournament.
1671
Table 1 Variables in the equation
round
round(1)
round(2)
round(3)
year
year(1)
year(2)
year(3)
rankdiff
Constant
B
SE
−0.020
0.510
0.322
0.363
0.374
0.367
0.136
0.560
0.371
−0.008
−0.225
0.363
0.374
0.369
0.007
0.386
Wald
d.f.
Sig.
2.867
0.003
1.859
0.770
2.661
0.140
2.245
1.009
1.186
0.339
3
1
1
1
3
1
1
1
1
1
0.413
0.956
0.173
0.380
0.447
0.708
0.134
0.315
0.276
0.560
1672
Journal of the Operational Research Society Vol. 60, No. 12
0.531, then his probability of winning a match playing up to
10 is 0.609 and vice versa.
An alternative estimate for p is given by dividing the
number of frames won by the entering player by the total
number of frames played, giving p = 2151/4008 = 0.537. We
note that the average number of frames played in a match is
4008/253 = 15.8.
30
25
15
Independence of frames within matches and
independence between matches
0
-9
-6
-3
0
3
6
9
points
Figure 1
Distribution of match scores.
A
U
TH
O
R
9–10, 10–9, 10–8, 10–7, . . . , 10–0, a ranking from the worst to
the best, from losing 10 frames and winning none to winning
10 frames without reply. For simplicity, the various results
are shown as ‘points’, a measure of the entering player’s
performance. The ‘points’ may be interpreted as net frames
(frames won minus frames lost) won by the entering player,
ranging from −10 to +10.
Of the 253 matches, 154 were won and 99 lost by players
entering the tournament and playing their first match. Thus we
may take 154/253 = 0.609 as an estimate of the probability
that a player entering the tournament wins the match. The
mean losing score in the 154 matches won by the entering
player was 5.63, while the mean losing score in the 99 matches
lost by the entering player was 6.17. The difference is not
surprising, as we would expect a losing player higher up in
the rankings to have a higher score than a losing player lower
in the rankings.
Players take turns taking the opening shot of each frame, so
the hypothesis that frames have independent outcomes is at
least plausible. Using a spreadsheet or another related simple
BASIC program, it is easy to calculate the probability that
a player entering the tournament will score 0, 1, 2, 3 . . . 9
frames if he loses the match, and the probability that his
opponent will score 0, 1, 2, 3 . . . 9 frames if he wins it. These
probabilities can be used to determine the expected number of
scores (out of 253) of the losing player of the 154 matches in
which the entering player wins and the 99 matches in which he
loses. The expected numbers of scores are shown in Table 2,
along with the actual numbers. The entering (higher-ranked)
player’s score comes first.
The 5% chi-square critical value on 13 degrees of freedom
is 22.362, so the total contribution to chi-square of 29.1
is clearly significant. The major contribution to chi-square
occurs at the low scores in matches won by the player entering
the tournament. Perhaps players faced with an opponent above
them in the rankings give up hope of winning the match too
quickly.
PY
5
O
10
C
Count
20
Fairness and balance: the probability of winning a frame
If frames are independent, if a player’s chance of winning a
frame is p and if a match is won by the first player to win n
frames, then the probability of the player winning the match
P can be calculated using a negative binomial or binomial
formulation, and is given by
P=
=
n−1
n−1+i
i=0
2n−1
i=n
i
2n − 1
i
pn (1 − p)i
pi (1 − p)2n−1−i
(1)
Given a value for P of 0.609, we can find the corresponding
value of p, using a spreadsheet or a simple computer program
written in BASIC: this turned out to be p=0.531. If a player’s
probability of winning a frame against a certain opponent is
Table 2
Score
10–0
10–1
10–2
10–3
10–4
10–5
10–6
10–7
10–8
10–9
9–10
8–10
7–10
6–10
5–10
4–10
3–10
2–10
1–10
0–10
Total
Actual
2
6
9
14
17
21
21
30
14
20
15
18
18
10
14
12
⎫
6⎪
⎬
5
0⎪
⎭
1
253
Actual and expected numbers of scores
17
12
Expected
0.5
2.1
5.5
10.3
15.6
20.5
24.0
25.8
25.7
24.1
21.2
20.0
17.7
14.6
11.0
7.4
⎫
4.3 ⎪
⎬
2.0
0.7 ⎪
⎭
0.1
253.0
Contribution to 2
8.1
10.0
1.4
0.1
0.0
0.4
0.7
5.3
0.7
1.8
0.2
0.0
1.4
0.8
2.9
7.1
3.4
29.1
SR Clarke et al—Criteria for a tournament
p
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
2
2
2
3
4
6
10
21
3
3
4
4
6
8
12
21
47
4
5
6
8
10
14
22
39
87
6
8
10
12
16
23
36
64
> 99
9
11
14
19
25
36
56
99
> 99
14
17
21
28
38
54
84
> 99
> 99
21
26
32
42
57
82
> 99
> 99
> 99
34
42
53
69
94
> 99
> 99
> 99
> 99
Appleton DR (1995). May the best man win? Statistician 44(4):
529–538.
Chen R and Hwang FK (1988). Stronger players win more balanced
knockout tournaments. Graphs and Combinatorics 4: 95–99.
Clarke SR (1996). Calculating premiership odds by computer—An
analysis of the AFL final eight playoff system. Asia-Pacific J Opl
Res 13: 89–104.
Clarke SR and Norman JM (1979). Comparison of North American
and international squash scoring systems—Analytical results. Res
Quart 50(4): 723–728.
Colwell DJ and Gillett JR (1987). The expected length of a tournament
final. Math Gazette 71: 129–131.
Glickman ME (2007). Bayesian locally-optimal design of knockout
tournaments. Presented at the International Conference on
Advances in Interdisciplinary Statistics and Combinatorics,
University of North Carolina, Greenboro.
Hwang FK (1982). New concepts for seeding knockout tournaments.
Am Math Month 89: 235–238.
Israel RB (1982). Stronger players need not win more knockout
tournaments. J Am Stat Assoc 76: 950–951.
Konig RH (2000). Competitive balance in Dutch soccer. J Roy Statist
Soc Ser D 49: 419–431.
Lundh T (2006). Which ball is the roundest—A suggested tournament
stability index. J Quant Anal Sports 2(3): 1–21.
McGarry T (1998). On the design of sports tournaments. In: Bennett
J (ed). Statistics in Sport. Edward Arnold: London, 199–217.
McGarry T and Schutz RW (1997). Efficacy of traditional sport
tournament structures. J Opl Res Soc 48: 65–74.
Percy DF (2007). A mathematical analysis of badminton scoring
systems. J Opl Res Soc, 7 November 2007. Advance online
publication doi:10.1057/palgrave.jors.2602528.
Scarf P and Bilbao M (2006). The optimal design of sporting contests.
Salford Business School Working Paper 320/06.
Schutz RW (1970). A mathematical model for evaluating scoring
systems with special reference to tennis. Res Quart 41: 552–561.
Szymanski S (2003). The economic design of sporting contests. J
Econ Lit 41: 1137–1187.
O
R
0.6
TH
0.60
0.59
0.58
0.57
0.56
0.55
0.54
0.53
0.52
Probability of winning the match P
References
PY
Table 3 Number of frames required to ensure certain
probabilities P of winning the match for given probabilities p of
winning the frame
O
As mentioned above, players may reasonably claim that a
longer match gives them more chance to show their superiority
over a weaker player. Equation (1) may be used to determine
how long a match needs to be to give a specified probability
of winning the match to a player with a specified probability
of winning a frame. The analysis reported in Table 2 shows
that the assumptions of independence between frames and
a constant probability of winning each frame do not hold.
Nevertheless, we may use Equation (1) to give useful, if only
approximate, results.
Where the match is won by the first player to win N frames,
Table 3 shows the minimum value of N to ensure a player
whose probability of winning a frame is p has a probability
of at least P of winning the match.
according to their progress in specific tournaments. The foregoing comments suggest that the rankings are a stable measure
of players’ ability.
Determining the number of frames in a match requires a
balance between the need to give the better player an appropriate chance of winning and the need to keep matches short.
In 2008 the qualifying rounds will take place in Prestatyn
over 8 days and later in Sheffield over 4 days. Large crowds
are not drawn to these qualifying rounds and the halls where
the matches are played have to be paid for. Perhaps playing
first-to-ten frames in a match, presumably arrived at through
experience, is a good compromise, as indeed an inspection of
the table above might suggest. Finally, particularly in the televised section of the championship, there needs to be a limit
to the advantage given to the better player: the audience, both
in the Crucible and watching at home, will want to give the
underdog a sporting chance.
C
Efficiency: how many frames are needed in a match?
1673
A
U
From this table, we may note that in a first-to-ten frames
match a player whose probability of winning a frame is 0.53
has a probability 0.60 of winning the match and a player
whose probability of winning a frame is 0.56 has a probability
0.70 of winning the match and so on.
Concluding remarks
In the championship, there are relatively few one-sided
matches, as can be seen in Figure 1 and because of the way
in which players enter the tournament at different stages
according to their position in the rankings, it may be argued
that the tournament gives the better players a better chance
of winning.
It is perhaps surprising that the probability of winning a
match is roughly constant for the player entering the tournament, whatever the round at which he enters. This suggests
that the gradation of ability in the rankings is gradual and
that there are not marked changes in the ability of players
close to each other on the rankings. The rankings are fixed
for a whole year, following the results of the World Championship, and are determined by points awarded to players
Received April 2008;
accepted September 2008 after one revision