H1S5 Strategi Asesmen Outcome - ASR

Strategi Asesmen Outcome
(Capstone Design Project as Major
Engineering Design Course)

Ir. Arief Syaichu Rohman, MEngSc, PhD, IPM
Workshop Nasional Strategi Meraih Akreditasi Internasional
Institut Teknologi Bandung, 18-19 April 2017

BACKGROUNDS

Comments, Complaints, etc
• Is it electrical literature or electrical engineering?
• Non standard quality of final project
• Most graduates do not have engineering design
experience
• (Long) delayed graduation due to (unfinished)
final project
• Some final projects are more like postgraduate
research
• Students do their supervisor’s project, often
disadvantage for the students


Consequences
• Culture change (the hardest)
• Faculty resistance
• Need solid ‘die hard’ team
• Need facts, measured result, &solid arguments
• Give enough room for gradual adaptation
• Presentation oral exam based on 6 pages paper
(NOT a book)
• Need extra financing & institutional supports

History of EE Program for ABET
accreditation
• 1997: intention to be internationally accredited
• 2000: invites overseas scholars to review the
readiness
• 2005: start for ABET preparation (PHKI funded)
• 2007: feasibility study of ABET accreditation
• 2008: curriculum revision
• 2010: ABET evaluator visit & get accredited

• 2013: EE program minor curriculum updates
• 2016: ABET evaluator visit for re‐‐accreditation

DEFINITIONS

PEO & SO
• Program Educational Objectives

• Program educational objectives are broad statements
that describe what graduates are expected to attain
within a few years after graduation. Program educational
objectives are based on the needs of the program’s
constituencies.

• Student Outcomes

• Student outcomes describe what students are expected
to know and be able to do by the time of graduation.
• These relate to the knowledge, skills, and behaviors that
students acquire as they progress through the program.


Assesment & Evaluation
• Assessment

• Assessment is one or more processes that identify, collect,
and prepare data to evaluate the attainment of student
outcomes.
• Effective assessment uses relevant direct, indirect,
quantitative and qualitative measures as appropriate to the
outcome being measured. Appropriate sampling methods
may be used as part of an assessment process.

• Evaluation

• Evaluation is one or more processes for interpreting the data
and evidence accumulated through assessment processes.
• Evaluation determines the extent to which student outcomes
are being attained. Evaluation results in decisions and actions
regarding program improvement.


PEO Program Educational
Objectives
• The objective of the EE Program is to produce
graduates who

• will have successful careers in their engineering
professions.
• are capable to be admitted in and successfully complete
their graduate studies.
• will have leadership and an active role in industry,
government or education sectors in Asia Pacific region
especially Indonesia.

Student Outcomes (1):
EAC ABET
a.
b.
c.
d.
e.

f.
g.
h.
i.
j.
k.

An ability to apply knowledge of mathematics, science, and engineering
An ability to design and conduct experiments, as well as to analyze and
interpret data
An ability to design a system, component, or process to meet desired
needs within realistic constraints such as economic, environmental, social,
political, ethical, health and safety, manufacturability, and sustainability
An ability to function on multi-‐disciplinary teams
An ability to identify, formulate, and solve engineering problems
An understanding of professional and ethical responsibility
An ability to communicate effectively
The broad education necessary to understand the impact of engineering
solutions in a global, economic, environmental, and societal context
A recognition of the need for, and an ability to engage in life-‐long learning

A knowledge of contemporary issues
An ability to use the techniques, skills, and modern engineering tools
necessary for engineering practice.

Student Outcomes (2): EE
Program of SEEI ITB
l.

breadth of knowledge over all areas within electrical
engineering (electromagnetics, power, electronics, signals
and systems, and computer engineering)
m. depth of knowledge in at least one area
n. knowledge of probability and statistics, including applications
to electrical and computer systems
o. knowledge of mathematics through differential and integral
calculus
p. knowledge of basic sciences, computer science, and
engineering sciences necessary to analyze and design
complex electrical and electronic devices, software, and
systems containing hardware and software components

q. knowledge of advanced mathematics, linear algebra, complex
variables, and discrete mathematics
r. background for graduate study

MAJOR
ENGINEERING
DESIGN

ABET EAC Engineering Design
• Engineering design is the process of devising a system,
component, or process to meet desired needs.
It is a decision‐making process (often iterative), in
which the basic sciences, mathematics, and the
engineering sciences are applied to convert
resources optimally to meet these stated needs.
• Students must be prepared for engineering practice
through a curriculum culminating in a major design
experience based on the knowledge and skills
acquired in earlier course work and incorporating
appropriate engineering standards and multiple

realistic constraints

Capstone Experience
• The Integration of Content (ABET)

• Baccalaureate degree programs must provide a capstone or
integrating experience that develops student competencies in
applying both technical and non-‐technical skills in solving problems.
• “ … the culminating or integrating experience, comprehensive
project, capstone or major design experience.” (ABET)

• Capstone Project

• Also called a capstone experience, culminating project, or senior
exhibition, among many other terms, a capstone project is a
multifaceted assignment that serves as a culminating academic and
intellectual experience for students, typically during their final year
of high school or middle school, or at the end of an academic
program or learning-‐pathway experience
• (http://edglossary.org/capstone-‐project/)


ABET EAC:
Criterion 5 Curriculum
• Major Engineering Design Experience

• Engineering design is the process of devising a system,
component, or process to meet desired needs. It is a
decision-making process (often iterative), in which the
basic sciences, mathematics, and the engineering
sciences are applied to convert resources optimally to
meet these stated needs.
• Multiple realistic constraint (e.g. economic,
environmental, social, political, ethical, health and safety,
manufacturability, and sustainability) [SO.(c)]
• Engineering standard
• Multi-discipline & Team work [SO.(d)]
• Significant design challenge [engineering practice]

Engineering Design Roadmap
• Freshmen year (ITB preparatory level):


• minor content KU1101 Introduction to Engineering and
Design KU1102 Introduction to Engineering and Design III

• Sophomore year:

• EL2102 Digital Systems Laboratory EL2205 Electronics
Laboratory

• Junior (third) year:

• EL3014 Microprocessor Systems EL3013 Instrumentation
Systems

• Senior (fourth) year:

• EL4090 Final Project (capstone design) I
• EL4091 Final Project (capstone design) II
• as a culminating major engineering design experience


Engineering Documents
• B100 Conceptual product to develop
• B200 Product specifications
• B300 Alternative designs
• B400 Implementation & realization
• B500 Evaluation, validation & testing

CONTINUOUS
IMPROVEMENT

2009 Assesment Results
Outcome

Courses

EL2095 EL2195
A

EL4091

3.24

(max 4) Priority

Action

Improvement

Maintain

2.62

x

2.74

2.92

x

3.63

3.09

x

1.7

1.10

2.14

2.5

2.81

2.89

x

3.43

D
E

EL4099

3.165

B
C

Result

x

F

2.152

2.5

3.23

2.86

x

G

2.496

2.3

3.42

2.91

x

H

2.3

1.42

2.02

I

2.4

3.90

3.15

1.65

1.88

3.94

3.08

x

L

3.71

2.75

x

M

2.42

2.46

x

N

1.23

2.00

x

J
K

2.62

2.336

x
x
x

O

2.67

x

P

2.84

x

Q

2.59

x

2.82

x

R

3.01

2009 Evaluation
• Direct Assesment (Rubrics)

• Our team think that these shortcomings are due to most of
students study only to pass the examinations. Most of them
do not have sufficient chances to use their knowledge to do
project or/and solve engineering problems.
• If they do a project, the students will learn that it is not
sufficient to learn only hard skills such as math, modeling, or
engineering science. They need also knowledge about
contemporary issues, teamwork skill, and other soft skills.

• Indirect Assesment (surveys)

• Some companies give feedbacks that some of our graduates
have weakness in communication and teamwork
• the exit survey, most students think that they do not have
enough cover on the experience of design under constraints

2009 Action
• A need of curriculum that:

• put/use together all necessary knowledge and skills to
solve problem/project
• give the student chance to exercise soft skills especially:
teamwork, communication, and presentation.

• The purpose of the capstone design course is to
provide students with a realistic design
experience that allows them to integrate and
apply the basic disciplinary knowledge they have
learned during their engineering program

Recommended Actions (1)
• Final project should be

• Capstone design project
• Not a research work
• No more simulation only
• Problem based
• Product prototype based
• Industry related (highly preferrable)

Recommended Actions (2)
• Final project should be






With standardized complexity level
Standarized process
With Standardized engineering documents
Team work of 2/3 students
With oral presentation

• With standardized academic reports
(publication paper & book)

EE Days 2012
• The first public event called EE Days
(Electrical Engineering Days)

• Logbook of activity records
• A group of 3 students must present their work before a panel of 2
examiners in public (they cannot choose group’s members)
• The group must also demonstrate their work. [responsibility &
professionalism]
• Best Engineering Design Award & Best Engineering Research Award
• Eligibility : 14 sks, passed 5 choosing breadth courses

• Evaluation

• The students were found difficult to do the capstone design due to
lack of breadth knowledge (3rd yr courses).
• Most of EE students were indicated not to have necessary skills &
knowledge to smoothly complete their process
• Coops (KP) may ruin team building & not passing 2nd yr courses is
sign for incompetenceies

EE Days 2013
• Second year experience








Introduce video for public expose
start to have industry involvement on project topics
involve multi discipline supervisor outside EE (e.g. aeronautics)
use social media (Facebook) for managing capstone design courses
Invited speakers from alumni and professionals (industry)
add new award for Best Video presentation
add requirements for eligibility: 1 month co-op min AND pass all
2nd year courses AND to pass comprehensive examination

• Evaluation:

• lack of broad view in the EE field is also seen in the implementation
of capstone design (final project course)
• Need more manhour to help run the event

EE Days 2014
• Third year experience

• 2013 EE Curriculum take effect on July 2014, i.e. Capstone Design
credits were redistributed from (2+4) credits into (3+3) credits;
• involve professional from industry as examiners
• publish book of abstracts of all projects for the event
• one strong industry involvement including industry internship
(1 semester)
• employ web based on‐line grading (oral, demo, and peer evaluation)
[SO.(f) profesionalism]
• involving 3rd yr students to help run the event & to introduce them
of what to expect in a capstone design

• Evaluation

• Date of the event is on recess, not many 3rd yr students contribute
significant help
• Problem in writing engineering documents, need simplified template

EE Days 2015 (1)
• Start to use weblog as logbook
[SO.(f) professionalism]
• use comprehensive test for eligibility on 9 (nine)
breadth courses;
• use YouTube for project’s communication media;
• embed rubrics in on-‐line grading
(oral, demo, and peer evaluation)
[SO.(g) communication]
• put attention to formulating design requirements
[SO. (e) problem formulation]
• Start to involve student union (HME)

• Evaluation

• Still problems in B200 documents (design specs)

EE Days 2015 (2)
• Web based on-‐line assesment

• Easy for examiners (faculty, alumni, industry)
• Easy for assesment
• Data may be presented in preferred format
• Embedded rubrics make it much simpler & easier for
examiners

Scheduled presentation seen by
a faculty member

Evaluation Form
for examiner & supervisor

Examiner Evaluation Form:
Identity Page

Supervisor Evaluation Form

Teamwork Evaluation
by Supervisor

EE Days 2016
• 1st graduate of 2013 EE Curriculum
• Involving Student Union (HME) to run the event

• Evaluation:

• Student diversity lower when group’s member are up to
students
• Not effective use of weblog due to no server’s admin
• Web-‐based TA management (LMS) has not been used
consistently

EE Days 2017
• Students can not choose members of the group
• EE Program will choose for them from three
groups of students’ pool
• Closely assesment of B100-‐B300 engineering
documents
• Back to use paper based logbook for group’s
activity records

IMPORTANT NOTES

Succesful Factors

Changing the culture
is the hardest
• Solid & consistency of EE Program teams (a must)
• Dedicated team members (a must)
• Supportive institution is highly preferable
• Having solid study based on measured performance,
facts, and sound arguments
• Series of socialization, communication, and good
defends/debates

Succesful Factors

Changing the culture
is the hardest
• Identify supportive faculties and their level of paticipations
(oppossing to committing)
• Gradual faculty participations and involvement
• Proving the effectiveness of the new ways or culture
• Financial commitments from the institution
• Alumni supports later for lab instruments & facilities
updates
• Continuous improvement cycle

EE DAYS UP TO NOW

Electrical Engineering Days
• http://eedays.stei.itb.ac.id/

• Video Tugas Akhir EE Days 2013 s/d 2016
• Abstrak Tugas Akhir EE Days 2016
• Poster Tugas Akhir EE Dyas 2015

• Facebook (closed) group: TA Elektro ITB

EE Days 2013

Ir.Yana S Rahardja, PT Quasar, Alumni EL’83, Advisory Board
member of EE Program:
“Tahap akhir produksi Fuel Saver Controller for coal mining
truck yg merupakan penyempurnaan dari proyek TA Real
Time throttle controller tahun lalu [2013]”

EE Days 2014 Layout

EE Days 2014 Awards & Prizes

EE Days 2016
Grup Presentation

EE Days 2016
Grup Demonstration

EE Days 2017
• Aula Timur ITB Ganesa
• 22-‐25 Mei 2017

• Sidang & Pameran Tugas Akhir
• Workshop
• Seminar
• Kompetisi
• http://eedaysitb.com/

Concluding Remarks (1)
• Capstone design project:

• Evolutive/continuous improvement
• Logbook: weblog vs paperbook
• Comprehensive test
• Kerja praktek min 1 bln
• Facebook use
• Web based assesment for outomes measurement
• RESULTS NOW?

Concluding Remarks (2)
• Deliverables of the Final Project (capstone
design) NOW
• Individual Final project report (max 40 pages)
• Engineering Documents (B100 to B500)
• Promotional poster (A1 size)
• Promotional Video (3 minutes)
• Individual paper in IEEE format (6 pages)
• Working prototype for demonstration

EE Days Event Activities (1)
• Open for public
• Demonstration of working prototype
• Oral presentation
• Program reports to public (constituents)
• Industry sponsorship
• Alumni/external examiner for quality control

EE Days Event Activities (2)
• Book of abstract
• Awards & Prizes
• Guest lecture from industry or alumni
(professionalism)
• Walk in interview
• Publication paper draft for students
• Sharing media for all (including constituents)

Thank you
You are invited to participate in the EE Days 2017
Aula Timur ITB on 22-‐24 May 2017