Presentation AUN QA UMI

1
1

What is Quality?
Govt.
Bodies

Employers

Students/
Graduates

University

QA in Higher Education

Faculty
Members

Quality


2

Society

Quality in Higher Education
Client
satisfaction: The
view of students
and
employers

Excellence:
The view of academia

Threshold:
The view of accreditors

Value added:
The view of students


Fitness for purpose:
The view of external
assessors aiming
for improvement

Value for money:
The view taxpayers and
governments

Source: Green, D. What is Quality in Higher Education? Concepts, Policy and Practice 1994

QA in Higher Education

3

Quality in Higher Education
Quality in higher education is a Multidimensional concept,
which should embrace all its functions, and activities:
teaching and academic programmes, research and
scholarship, staffing, students, buildings, facilities,

equipment, services to the community and the academic
environment. Internal self-evaluation and external review,
conducted openly by independent specialists, if possible
with international expertise, are vital for enhancing quality.

Source: The World Declaration on Higher Education for the Twenty First Century: Vision and Action (October
1998), Article 11, Quality Evaluation.

QA in Higher Education

4

What is Quality Assurance (QA)?

Quality assurance can be described as the systematic,
structured and continuous attention to quality in terms of
maintaining and improving quality.
Source: AUN-QA Manual for the Implementation of the Guidelines, P20

Quality assurance in higher education can be defined as

systematic management and assessment procedures to
monitor performance of higher education institutions.
Source: The Regional Report of Asia and the Pacific (UNESCO, 2003b)

QA in Higher Education

5

AUN-QA Models

6

Evolution of AUN-QA

7

AUN-QA Model

Programme QA
Assessment since

2007

Strategic
QA
(Institutional)

Systemic QA
(Internal QA System)

Functional QA
(Education, Research and
Service)
AUN-QA Model

8

Institutional QA
Assessment
commencing
Jan 2017


AUN-QA at Institutional Level
(2nd Version)

AUN-QA Model

9

AUN-QA at Programme Level

Version 0

AUN-QA Model

Version 1
2007 – 2010
18 Criteria

Version 2
2011 – 2016

15 Criteria

10

Version 3
2016/17
onwards
11 Criteria

1st Version
1. Goals and Objectives;
Expected Learning Outcomes
2. Programme Specification
3. Programme Content
4. Programme Organisation
5. Didactic Concept and
Teaching/Learning Strategy
6. Student Assessment
7. Staff Quality
8. Quality of Support Staff

9. Student Quality
10. Student Advice and Support
11. Facilities and Infrastructure
12. Quality Assurance of
Teaching/Learning Process
13. Student Evaluation
14. Curriculum Design
15. Staff Development Activities

2nd Version
1. Expected Learning
Outcomes
2. Programme Specification
3. Programme Structure and
Content
4. Teaching and Learning
Strategy
5. Student Assessment
6. Academic Staff Quality
7. Support Staff Quality

8. Student Quality
9. Student Advice and Support
10. Facilities and Infrastructure
11. Quality Assurance of
Teaching and Learning Process

3rd Version
1. Expected Learning
Outcomes
2. Programme Specification
3. Programme Structure and
Content
4. Teaching and Learning
Approach
5. Student Assessment
6. Academic Staff Quality
7. Support Staff Quality
8. Student Quality and
Support
9. Facilities and Infrastructure

10. Quality Enhancement

AUN-QA at Programme Level

16. Feedback Stakeholders
17. Output
18. Stakeholders
Satisfaction

AUN-QA Model

12. Staff Development Activities 6. Academic Staff Quality
7. Support Staff Quality
13. Stakeholders Feedback
10. Quality Enhancement
14. Output
15. Stakeholders Satisfaction
11

11. Output


AUN-QA at Programme Level
(3rd Version)
Stakeholders’ Needs

Expected
Learning

Programme
Specification

Programme
Structure &
Content

Teaching & Learning
Approach

Student
Assessment

Academic Staff
Quality

Support Staff
Quality

Student Quality &
Support

Facilities &
Infrastructure

Outcomes
Quality Enhancement

Output

Quality Assurance and (Inter)national benchmarking

AUN-QA Model

12

A
c
h
i
e
v
e
m
e
n
t
s

AUN-QA at Programme Level
(3rd Version)
Criterion

Sub-criterion

Checklist

1. Expected Learning Outcomes

4

3

2. Programme Specification

2

3

3. Programme Structure and Content

6

3

4. Teaching and Learning Approach

6

3

5. Student Assessment

8

5

6. Academic Staff Quality

10

7

7. Support Staff Quality

5

5

8. Student Quality and Support

5

5

9. Facilities and Infrastructure

7

5

10. Quality Enhancement

6

6

11. Output

3

5

Total

62

50

AUN-QA Model

13

Guide to AUN-QA Assessment at
Programme Level (3rd Version)
• Towards a principles-based
framework
• Reduced overlapping and ambiguity
• From 15 to 11 AUN-QA criteria
• Parallel implementation with 2nd
version in 2016.
• 3rd version will be effective from
January 2017.

AUN-QA Model

14

15

Accreditation, Assessment, Audit and
Evaluation
AUN-QA Assessment (for the purpose of
improving the effectiveness of QA system)




Non-prescriptive
Recommend areas for improvement rather
than mandating solutions
Contextualised rather than standardised
QA practices

AUN-QA Model

16

Assessment
(SAR)  AUN

Evaluation
(SER)  BAN

• Assessment provides feedback on
performance (strenghts, areas for
improvement and insights)
• Purpose is to improve the quality of
future performances
• Feedback based on observations of
strongest and weakest points
• SAR should cover what made the
quality of the performance strong;
and how might one improve future
performances
• SAR is used to improve performance
17

• Evaluation provides feedback
whether a standard was met
(success or failure)
• Purpose is to determine the
quality of the present
performances
• Feedback based on level of
quality based on a set standard
• SER should cover the quality of
the performance, often
compared to set standards
• SER is used to make judgments

PDCA Approach to
Self-assessment at Programme Level
Act
• Improve QA
• Finalise SAR
• Communicate SAR
• Get ready

Plan
• Communicate
intent
• Organise team
• Develop plan
• Understand AUN-QA
criteria & process

Change Management

Check
• Verify SAR
• Gather feedback

Plan

18

Do
• Self-assessment
•Collect data &
evidences
• Close gaps
• Write SAR
• Review SAR

Communicate Intent







Plan

Engage stakeholders
Objective and scope
Plan
Stakeholders roles and involvement
Set expectations and climate
Start of change management

19

Organise Team
• Structure and Roles
 System criteria
 Related criteria
 Random criteria

• Size
 Main and sub-groups

• Ownership
• Subject matter experts including English language
proficiency
• Capability and availability
• Support from sponsor, management and peers

Plan

20

Structure and Roles
System Criteria
Leader

Facilitator

Requirements

Input

(Criterion 1 to 5)

(Criterion 6 to 9)

Plan

21

Quality
Enhancement
(Criterion 10)

Output
(Criterion 11)

Structure and Roles
Related Criteria
Leader

Facilitator

Programme
(Criterion 1, 2, 3,
10)

Plan

Teaching & Learning
and Student
Assessment
(Criterion 4, 5, 10)

22

Resources
(Criterion 6, 7, 8, 9,
10)

Output
(Criterion 10,
11)

Structure and Roles
Random Criteria
Leader

Facilitator

Criterion 1, 2, 3

Plan

Criterion 4, 5

Criterion 6, 7, 8

23

Criterion 9, 10 ,
11

Structure and Roles
Key Roles & Responsibilities (example)
Leader

Responsibilities

Dean

• Link between faculty, QA Office and university administration

Head of Academic
Affair of the Faculty

• Link between department, Dean and QA office
• Link between faculty and data center

Head of Department

• Provide leadership
• Link between Dean, Academic Affair and Team

Head of Team

• Lead team
• Develop plan
• Assign and review task
• Divide team structure and roles
• Link between members with data center
• Collate and edit SAR

Plan

24

Structure and Roles
Define Key Roles & Responsibilities (example)
Key Role

Responsibilities

Facilitator (QA Office)

• Provide leadership
• Guide internal assessors
• Guide SAR team
• Train internal assessors
• Train SAR team
• Encourage and motivate SAR team
• Review SAR
• Link between faculty and university

Facilitator (Faculty)

• Guide SAR team
• Encourage and motivate SAR team
• Review SAR

Member (Department)

• Gather and organize information
• Write SAR

Plan

25

Develop Plan
Activity/Month
P
L
A
N

1

2

3

4

Communicate Intent
Organise Team
Develop Plan
Understand AUN QA criteria and
process

D
O

Self-assessment
Collect data & evidences
Close gaps
Write SAR
Review SAR

C
H
E
C
K

A
C
T

Verify SAR
Gather Feedback
Improve QA
Finalise SAR
Communicate SAR
Get Ready

PlanChange Management

26

5

6

7

8

9

1
0

1
1

1
2

Deadline

Assigned to

Status

Write SAR - Content of the SAR
Part 1: Introduction
• Content page
• Executive summary
• Organization of the self-assessment
• Brief description of the university, faculty and department
Part 2: AUN-QA Criteria Requirements
• Write-up on how the university, faculty or department addresses the
requirements of the AUN-QA criteria (use Appendix A - Checklist as a
reference)
Part 3: Strengths and Weaknesses Analysis
• Summary of strengths
• Summary of Weaknesses
• Completed checklist
• Improvement plan
Part 4: Appendices
• Glossary
• Supporting documents and evidences
Do

27

Understand AUN QA Criteria and
Process





Plan

Obtain copy of the AUN-QA manual
Educate stakeholders
Organise training for relevant stakeholders
Seek clarifications with internal and external
experts

28

Quality Assessment Process
Stage

Plan

Timeline

Agent
AUN
Invite assessors for assessment
Secretariat
Accept or decline invitation by
6 months
assessors. Submit Disclosure of Assessor
prior to
Conflict of Interest
assessment
 AUN
Appoint and confirm assessors
Secretariat
and assessment teams
 Assessor

Quality Assessment Process

Key Activity

29

Quality Assessment Process
Stage

Plan

Timeline

Agent
 AUN
Confirm assessment itinerary,
Secretariat
accommodation and logistics
 Applying
arrangements with applying university
university
 Assessor
AUN
2 - 3 months Send registration form to assessor
Secretariat
prior to
Submit registration form with flight
assessment
information, accommodation and diet Assessor
requirements
Submit progress report by applying
Applying
university
university
Monitor and track status of accredited AUN
university
Secretariat

Quality Assessment Process

Key Activity

30

Quality Assessment Process
Stage

Do

Timeline

Key Activity
Agent
Submit self-assessment report (SAR) to Applying
11/2 - 2
AUN Secretariat and assessors
university
months prior
Conduct desktop assessment and
to
clarify with applying university, if
Assessor
assessment
necessary
 AUN
Secretariat
1 day prior to Arrival of assessors and AUN
 Applying
assessment Secretariat
university
 Assessor
 AUN
Secretariat
Day 1 of
Opening session and site assessment  Applying
assessment
university
 Assessor

Quality Assessment Process

31

Quality Assessment Process
Stage

Timeline

Key Activity

Do/ Day 2 of
Site assessment and preparation of
Check assessment assessment report






Check

Day 3 of
Closing session and departure of
assessment assessors and AUN Secretariat





Quality Assessment Process

32

Agent
AUN
Secretariat
Applying
university
Assessor
AUN
Secretariat
Applying
university
Assessor

Quality Assessment Process
Stage

Act

Timeline

Key Activity
Agent
Submit assessment report to AUN
Within 4
Secretariat and provide feedback to AUN
weeks after Secretariat on the assessment, if
Assessor
assessment necessary. Request payment of
honorarium and submit claims, if any
Within 6
Send assessment report and
weeks after assessment feedback to applying
AUN Secretariat
assessment university
 Applying
Within 10
Submit assessment feedback and
university
weeks after
appeal, if necessary
 AUN
assessment
Secretariat
Submit progress report by applying
Applying
Within 9 – 12
university
university
months after
Monitor and track status of accredited
assessment
AUN Secretariat
university

Quality Assessment Process

33

Quality Assessment Process
Stage

Act

Timeline

Key Activity
Recommend the release of final
Within 14
assessment reports and accreditation to
weeks after the Board of Trustees for approval;
assessment Endorse the release of final assessment
reports and accreditation
Release of final assessment report and
Within 16
accreditation award to applying
weeks after
university and publish accreditation
assessment
status in AUN website
Within 30
Release of best practices of applying
weeks after university in the AUN-QA Benchmarking
assessment Database

Quality Assessment Process

34

Agent
AUN Secretariat
AUN-BOT

AUN Secretariat

AUN Secretariat

Quality Assessment Process
Stage

Act

Timeline

Key Activity
Request for progress report from
applying university on recommendations
Within 9 –
made by assessment teams
12 months
Submit progress report by applying
after
university
assessment
Monitor and track status of accredited
university

Quality Assessment Process

35

Agent
AUN Secretariat

Applying
university
AUN Secretariat

Site Assessment
• Document Review
• Evidence should be collected on all matters related to
the assessment objectives and scope
• Evidence should be collected through:
 Interviews
 Examination of documents/records
 Observation of activities and facilities
 Site tour





Working environment
Work practices (e.g. safety)
Condition of facility and equipment
Verify information and data with staff and students

Quality Assessment Process

36

Rating Scale
Score

Value Interpretation

Quality & Improvement Interpretation

1

Nothing (no documents, no plans, no
evidence) present

Absolutely inadequate; immediate improvements must
be made

2

This subject is in the planning stage

Inadequate, improvements necessary

3

Documents available, but no clear
evidence that they are used

Inadequate, but minor improvements will make it
adequate

4

Documents available and evidence that
they are used

Adequate as expected (meeting the AUN-QA
guidelines and criteria)

5

Clear evidence on the efficiency of the
aspect

Better than adequate (exceeding the AUN-QA
guidelines and criteria)

6

Example of best practices

Example of best practices (national-class)

7

Excellent (world-class or leading practices)

Excellent (world-class or leading practices)

As weight is not allocated to each criterion, the overall opinion should be based on the achievement or
fulfillment of the criterion as a whole. It should not be computed based on the average score of the statements
under each criterion. The overall score of the programme is computed based on the arithmetic average
of the 15 criteria.

Do

37

Fees

38

39

Thank You

40