SMS SAFETY PERFORMANCE .1 ICAO SAFETY MANAGEMENT SARPs
Chapter 6. ICAO Safety Management SARPs 6-9
Figure 6-4. Transition from initial to mature ALoS related to an SSP
a finding out what is wrong hazard identification; b proposing and implementing a fix or fixes remedial action;
c making sure that the proposed fix or fixes work as intended continuous monitoring; and d constantly improving the management system to ensure efficacy and efficiency of the delivery of
services continuous improvement of the SMS. 6.5.3
An SMS is defined as a systematic approach to managing safety, including the necessary organizational structures, accountabilities, policies and procedures. The fundamentals of an SMS are discussed in Chapter 7. Just as
with the SSP, ICAO has developed an SMS framework to assist service providers in the implementation of an SMS. The framework is composed of four components and twelve elements, and is introduced in full in Chapters 8 and 9.
6.6 SMS SAFETY PERFORMANCE 6.6.1
Annexes 1, 6, 8, 11, 13 and 14 establish that a service provider’s SMS shall ensure remedial action to maintain safety performance and shall continuously monitor and shall regularly assess such safety performance.
6.6.2 The notion of safety performance is an essential ingredient in the effective operation of an SMS as well as
progressing towards a performance-based regulatory environment. It assists in monitoring actual performance of the SMS, and in avoiding just simply “ticking the appropriate boxes”. It is necessary for an SMS to define a set of
Timeline
Mature ALoS
Safety measurement and safety performance
measurement — Quantification of outcomes
of selected high-levelhigh- consequence events
— Quantification of selected high-level State functions
— Quantification of outcomes of selected low-levellow-
consequence events
State safety assurance
— Safety oversight — Safety data collection,
analysis and exchange — Safety-data-driven targeting
of oversight of areas of greater concern or need
Initial ALoS
Safety measurement — Quantification of outcomes
of selected high-levelhigh- consequence events
— Quantification of selected high-level State functions
6-10 Safety Management Manual SMM
measurable performance outcomes in order to determine whether the system is truly operating in accordance with design expectations — not simply meeting regulatory requirements — and to identify where action may be required to
bring the performance of the SMS to the level of design expectations. These measurable performance outcomes permit the actual performance of activities critical to safety to be assessed against existing organizational controls so that
necessary corrective action is taken and safety risks can be maintained ALARP. 6.6.3
A performance-based regulatory approach will assess the actual performance of activities critical to safety against existing organizational controls. Furthermore, only through assurance of the effective safety performance of the
SMS — through the establishment and measurement of specific safety performance outcomes — can the objective of continuous improvement of safety underlying safety management be achieved.
6.6.4
The safety performance of an SMS is not related to the quantification of high-consequence outcomes safety measurement but rather to the quantification of low-consequence processes safety performance measurement. The
safety performance of an SMS represents safety performance measurement exclusively. Safety performance expresses the safety objectives of a service provider, in the form of measurable safety outcomes of specific low-level processes of the
SMS. From the perspective of the relationship between the State and service providers, safety performance provides objective evidence for the State to measure the effectiveness and efficiency that the SMS of service providers should
achieve while the service providers conduct their core business functions. Such safety performance must be agreed between the State and service providers, as the minimum acceptable the service provider must achieve during the delivery
of services. The safety performance of an SMS is thus a reference against which the State can measure the safety performance of the SMS, that is, that the SMS works above and beyond regulatory compliance. In agreeing to the safety
performance of an SMS, it is necessary to consider such factors as the level of safety risk that applies, the costbenefits of improvements to the system, and public expectations about the safety of the aviation industry.
6.6.5
Within each State, the safety performance of each SMS will be agreed separately between the State and individual aviation organizations. Agreed safety performance should be commensurate with the complexity of an
individual aviation organization’s specific operational contexts, and the availability of an aviation organization’s resources to address them. In practice, the safety performance of an SMS is expressed by safety performance indicator values and
safety performance target values and is implemented through action plans. 6.6.6
The safety performance indicator values are short-term, measurable objectives reflecting the safety performance of an SMS. They are expressed in numerical terms; they should be obvious, measurable and linked to the
safety concerns of an SMS. Safety performance indicator values reflect safety performance measurement exclusively. The safety performance indicator values of an SMS should not reflect safety measurement. Since the safety
performance of each SMS will be agreed separately between the State and individual aviation organizations, the safety performance indicator values will therefore differ between segments of the aviation industry, such as aircraft operators,
certified aerodrome operators and ATS providers. An example is provided. 6.6.7
Through its SMS, a certified aerodrome operator has identified safety concerns regarding foreign object debris FOD in ramp operations. It has also identified safety concerns regarding traffic of unauthorized vehicles on
taxiways. It therefore defines the following safety performance indicator values, following agreement with the State’s civil aviation oversight authority: 15 FOD events in the apron per 10 000 operations, and 20 events of unauthorized vehicles on
the taxiways per 10 000 operations. These safety performance indicator values fulfil the conditions discussed in 6.6.6: they are expressed in numerical terms; they are obvious, measurable and linked to the safety concerns of the aerodrome SMS.
Furthermore, both safety performance indicators reflect safety performance measurement. 6.6.8
Safety performance target values are long-term, measurable objectives reflecting the safety performance of an SMS. Safety performance target values are expressed in numerical terms; they should be obvious, measurable,
acceptable to stakeholders and linked to the safety performance indicator short-term objective of an SMS.
Chapter 6. ICAO Safety Management SARPs 6-11
Figure 6-5. ALoS reflecting safety measurement
Figure 6-6. ALoS reflecting safety performance measurement
State
Will comply with all applicable international Standards.
Safety target
values
1. Constant descent arrival CDA procedures implemented. Arrival procedures charts designed for stabilized approaches.
2. Installation of ASDEX at 5 international airports.
3. ... [State]
Action plans
Safety indicator
values
1. non-conforming approaches NCA at 5 international airports per
operations. [Number]
[number] 2.
Cat B and C runway incursions at 5 international airports per
operations. 3. ...
[Number] [State] [number]
1. Reduce by the number of Maximum of
non-conforming approaches NCA at 5 international airports per
arrivals by .
2. Reduce by the number of Maximum of
Cat B and C runway incursions at 5 international
airports per operations by
. [number] [number]
[number] [date]
[number] [number]
[State] [number]
[date] 3. ...
1. Reduce by the number of Maximum of
CFIT and approach and landing accidents per
departures. [number]
[number] [number]
[number] 2. Minimum of
inspections of operators completed quarterly. 3. ...
Safety target
values
1. CFIT training package distributed to industry and supported by training courses. 2. Revision and, if necessary, update of hiring policy. Inspection manual updated.
3. ...
Action plans
Safety indicator
values
State
Will comply with all applicable international Standards. 1.
CFIT and approach and landing accidents per departures.
[Number] [number]
2. inspections of operators completed quarterly.
3. ... [Number]
6-12 Safety Management Manual SMM
6.6.9 Continuing with the example discussed in 6.6.7, the aerodrome defines the following safety performance
target values, following agreement with the State’s civil aviation oversight authority: by January 2009, reduce FOD events in the apron to 8 per 10 000 operations, and maintain 20 events of unauthorized vehicles on the taxiways per 10 000
operations. These safety performance target values fulfil the conditions discussed in 6.6.6: they are expressed in numerical terms; they are obvious, measurable and linked to the safety performance indicators of the aerodrome SMS. Furthermore,
both safety performance target values reflect safety performance measurement. 6.6.10
Action plans are the tools and means needed to achieve the safety performance indicator values and safety performance target values of an SMS. They include the operational procedures, technology, systems and
programmes to which measures of reliability, availability, performance andor accuracy can be specified. An example of an action plan to achieve the safety performance indicator values and safety performance target values of an SMS
discussed above would be as follows: implement a thrice-daily walk-in ramp inspection programme, develop and implement a training course for drivers and install aerodrome-specific taxiway signage.
6.6.11
The safety performance indicator values and safety performance target values of the safety performance of an SMS may be different, or they may be the same. Three aspects must be considered when assessing whether specific
safety performance indicator values and safety performance target values of the safety performance of an SMS are different or the same. First, consideration must be given to the availability of resources within the service provider to turn
the safety performance indicator value into a more demanding safety performance target value. Second, consideration must be given to how expensive the action plans deemed necessary to change the value of the safety performance
indicator into a more demanding value of the safety performance target are. Third, and most importantly, consideration must be given to whether the assessment of the safety risks of the consequences of the hazard addressed by the safety
performance indicator and safety performance target falls in the tolerable region of the safety risk management process discussed in Chapter 5, should the safety performance indicator value and the safety performance target value remain
the same. The safety performance indicator value may reflect a safety risk assessment that falls in the tolerable region under prevailing circumstances. However, changes in the system, growth and so forth may render such safety risk
assessment invalid. The safety performance indicator value must in this case be turned into a more demanding target value to be valid in the changed environment.
6.6.12
A range of different safety performance indicators and safety performance targets will provide a better insight into the safety performance of the SMS of an aviation organization than the use of a single indicator or target. In
other words, the safety performance of an SMS will always be expressed by a number of safety performance indicators and safety performance targets, never by a single one. Additional examples follow.
6.6.13
An aircraft operator has identified the approach and landing phases of flight operations as one major safety concern to be addressed by its SMS. It has also identified, though the safety risk management component of its SMS, a
safety concern regarding unstable or non-conforming approaches at those aerodromes of the network served by non- precision approaches. It therefore defines the following safety performance indicator value, following agreement with the
State’s civil aviation oversight authority: 10 unstable or non-conforming approaches per 1 000 landing operations at aerodromes of the network served by non-precision approaches. Subsequently, the aircraft operator defines the
following safety performance target value, following agreement with the State’s civil aviation oversight authority: within the next three years, reduce by fifty per cent the number of unstable or non-conforming approaches per 1 000 landing
operations at aerodromes of the network served by non-precision approaches. The action plan to achieve the safety performance indicator value and the safety performance target value discussed above would be as follows: development
of constant descent angle CDA GPS approaches at aerodromes of the network served by non-precision approaches. 6.6.14
An ATS provider has identified airport operations safety as one major safety concern to be addressed by its SMS. It has identified, though the safety risk management component of its SMS, a concern regarding runway
incursions and has defined the following safety performance indicator value: 0.8 Cat A and B most serious runway incursions per million operations through 2009. Subsequently, the ATS provider defines the following safety performance
target value: by 2010 reduce Cat A and B most serious runway incursions to a rate of not more than 0.5 per million operations.
Chapter 6. ICAO Safety Management SARPs 6-13
6.6.15 The safety performance of an SMS should be defined, to the extent possible, through quantitative safety
performance indicators and safety performance targets. It is recognized, however, that in many States the safety data collection and analysis capabilities of services providers may not be fully developed. Therefore, while such capabilities
are developed, the safety performance of an SMS can be defined through a combination of quantitative and qualitative safety performance indicators and safety performance targets. The objective should nevertheless remain the definition of
safety performance of an SMS through quantitative measures only. 6.6.16
The definition of the safety performance of an SMS is a requirement that goes above and beyond regulatory compliance with national and international requirements. Establishing safety performance for an SMS does
not replace legal, regulatory or other established requirements, nor does it relieve service providers from their obligations under relevant national regulations, and those arising from the Convention on International Civil Aviation
ICAO Doc 7300 and its related provisions contained in the Annexes to the Convention.
6.7 MANAGEMENT ACCOUNTABILITY 6.7.1