Implementing and Evaluating the Integrat

Abstract

The theorising of critical thinking has long been attempted in education. Problem Based Learning (PBL) has also been reviewed and utilised in higher education building and construction related subjects. However, how certain ‘key skills’, such as critical thinking, can

be integrated into PBL in order to promote deeper learning in these complex multi- disciplinary environments, warrants further exploration. This paper intends to address this knowledge gap by evaluating the integration of critical thinking into PBL, drawing on a recent Higher Education Academy funded research project. The research employed a multi- methodological research design, and was carried out on a large and innovative building ‘design and build’, multi-disciplinary group project, undertaken by 67 second-year undergraduate students in the environmental building discipline at a UK university. A process model developed in the previous research was adopted for integrating critical thinking into PBL and its integration and use indicated a number of impacts on student learning. The content and timing of the integration of critical thinking theory and practice requires modification to suit student learning more effectively. Greater emphasis on critical thinking in the project, particularly linked with assessment, is also needed. The holistic approach adopted provides a useful mechanism to maximise the benefits of integrating critical thinking into PBL, which supports students in achieving optimised design solutions and enhanced employability. The findings provide evidence of capitalising on the marriage of critical thinking with PBL in environmental building education, and should contribute to future debate on critical thinking and PBL in the wider knowledge community.

Keywords: Critical Thinking, Environmental Building, Problem Based Learning (PBL), Process, Student Learning

Introduction

Critical thinking theory has been entwined in education for over 100 years. As a concept it was first touched upon by Dewey in 1910, when he outlined what he believed constituted

‘reflective thoughts’ (1910, p.6). More recently, ‘it has become something of an educational buzz-word’ (Fisher, 2001, p.1), generating its own unique and sometimes conflicting discourse along the way. Mason (2008) suggested that contemporary education literature

now unquestionably links learning with critical thinking, and that ‘thinking skills’ have become

a ubiquitous ambition of education, perhaps even a ‘promised land’ (Papastephanou and Angeli, 2007, p. 604).

The current drive in Higher Education (HE) to foster such skills has in part been driven by employers and professional bodies becoming more vociferous regarding graduate abilities and attributes. At the same time, the role of education, particularly HE, has been seen as increasingly important to economic developments, which are having a greater influence on the curriculum (Barnett, 2005). One issue that has been repeatedly raised is that graduates should be able to ‘think smarter than was the case in the past’ (Pithers and Soden, 2000, p.237). More specifically, this point has been echoed in the field of engineering, where Mitchell et al. (2010) pointed out that both industrial bodies and professional institutions are expressing the need for graduates to be equipped with personal and transferable skills to better prepare them for their careers.

The approach of Problem Based Learning (PBL) has become increasingly attractive to a number of teaching contexts in HE, to address both the drive to improve graduate skills and give graduates more professional experience. As such, PBL in its differing forms has experienced a dramatic increase in its use (de Graaff and Kolmos, 2003). Kolmos and Holgaard more specifically identified PBL as being ‘quite successful in aligning engineering education with companies’ needs’ (2010, p.580).

Despite the attempted theorising of critical thinking and PBL, it remains unclear how skills such as critical thinking can be better integrated into the PBL process. This lack of

understanding risks the achievement of deeper learning in complex multi-disciplinary environments. Previous research explored the integration of critical thinking into a multi- disciplinary environmental building project-based learning (Pan and Allison, 2010), although

this fell short in evaluating the implementation and integration. This paper aims to contribute to addressing this knowledge gap by evaluating the integration of critical thinking into PBL, drawing on the recent Higher Education Academy funded research project. It investigates the

integration process, examines the evolution of students’ understanding of critical thinking, and refines the process of integrating critical thinking into PBL. The paper then develops a process model of integrating critical thinking into PBL for stimulating deeper learning in

environmental building.

PBL: concept and applications

Questions regarding the distinction between Project Based and Problem Based Learning often arise. Numerous PBL studies have been conducted utilising a project, and as de Graaff Questions regarding the distinction between Project Based and Problem Based Learning often arise. Numerous PBL studies have been conducted utilising a project, and as de Graaff

a project context. On this point, Perrenet et al. (2000) suggest that in isolation PBL has its limitations and risks, and that in order to better reflect some professions, project work is the best suited, interestingly adding that it should be integrated into the later stages of the curriculum.

The prevalence of PBL across an increasing range of curricula is understandable given the favourable reports that have been made regarding its enhancement on certain areas of student learning. Research indicates that students become more proficient at problem- solving, group working, critical analysis and communication (Overton, 2003; Mitchell et al., 2009; Martinez et al., 2011), all of which Sendag and Odabasi (2009) have expressed as desirable graduate key skills. However, much PBL research has tended to focus on assessing and analysing PBL, and its associated characteristics in specific subject areas, such as medical health (Barrows and Tamblyn, 1980), architecture (Maitland, 1997; Roberts, 2007), and civil engineering (Mgangira, 2003). Unfortunately little of this has been conducted in relation to multi-disciplinary programmes, and specifically in environmental building, an area that is attracting increasing interest in teaching and learning (de Wilde and Pilkington, 2006). Further to this, in response to the increasing attention to issues of the sustainability agenda, improving the criticality of students in environmental building will enhance their employability. Savin-Badin (2000) remarked that research into PBL should explore the more complex issues, such as the key skills that it has been linked to, and the interrelatedness of these with other aspects of teaching and learning. Therefore critical thinking and the complexity of fostering it in students through the PBL process warrants further exploration.

Theories of critical thinking

Critical thinking has been extensively reviewed in the context of HE from numerous perspectives, and a wide array of theoretical and conceptual models have been generated. However despite these developments, Winch (2006, p.5) pointed out that there still remains ‘disagreement on the nature of critical thinking’. Thinking itself as a skill has been discussed by Smith (2002), who questions that if it is a skill, is it general, domain, or content specific? Debate on the notion of ‘skills’ in education and moreover of teaching ‘thinking’ as a skill, will perhaps always remain. However, despite these different perspectives, Winch (2010) put forward the argument that the aim of education should be to develop people with the valuable attribute of thinking ably. Cottrell (2005, p.8) pointed out that it is an expectation of students to develop critical thinking skills, so that they can ‘dig deeper’ underneath the surface of their subject area, enabling them to engage critically with its theories and arguments.

Definitions regarding critical thinking vary with the theoretical and conceptual models they are based upon. Winch (2006) identified it as a balance between skills and virtues for evaluating evidence of authoritative statements that is textured by contextual variation. Norris and Ennis (1989, cited in Fisher, 2001, p.4) described it as ‘reasonable, reflective thinking that is focussed on deciding what to do’. Cottrell (2005, p.2) more generally defined it as a

‘complex process of deliberation which involves a wide range of skills and attitudes’, going on to say that it gives the tools for utilising scepticism and doubt in a constructive way to perform analysis. Similarly, Bowell and Kemp (2005) defined it as a tool, for argument analysis, and thinking clearly and rationally. It is no wonder then that it is seen as a crucial key skill for graduates to demonstrate, with some even suggesting that critical thinking abilities are needed in today’s workplace more than ever before (Sendag and Odabasi, 2009).

However, it could be argued that critical thinking is more than just a skill or a tool, and perhaps this in itself gives rise to the varying definitions and understanding that can be encountered. For if it is an increasingly desirable attribute that graduates need to demonstrate and carry over into their professional careers, then, as Hilsdon and Bitzer (2007) pointed out, it is as much about attitude and disposition. Perhaps, as Winch (2006) highlighted, the skill comes in determining when and how the attitude or disposition is exercised. Integrating a critical thinking aspect into the PBL process will give students the opportunity to practise this skill, helping them to clarify in their own minds what they believe it is, what it entails, and more importantly to develop, practise and evaluate its application for the future.

Hilsdon et al. (2010) developed a critical thinking model to help stimulate thinking, questioning and reflection. This model is grounded in a ‘functional-narrative’ approach, based on a structure of description, analysis and evaluation, to encourage a questioning approach for deconstructing and reconstructing problems, topics or claims (Hilsdon and Bitzer, 2007). Mason (2008, p.1) highlighted the need for students to question, suggesting ‘that if students are to learn to think then they should be encouraged to ask critical questions’. Pan and Allison (2010) adapted this model for use in the context of PBL in environmental building, and revealed that the introduction of the model, as well as the timing of other interventions on critical thinking, was influential on the students’ learning performance regarding criticality.

Methodology The PBL project

The research was carried out on a large and innovative building ‘design and build’ group project (referred to in this paper as ‘the project’), undertaken by 67 second-year undergraduate students in the environmental building discipline at a UK university.

The project is cross-module, multi-disciplinary, and role-playing. The approach adopted ensured that a suitable multi-disciplinary learning environment was provided. Its main features included:

 The inclusion of three modules: ‘Technology of Large and Innovative Buildings’;

‘Construction Management Processes and Principles’; and ‘Building Surveying Principles and Practice’.

 Students were from four inter-related courses: architecture; building surveying; construction management; and environmental construction surveying.

 Small groups were self-selected, with each group being required to cover at least five

roles throughout the project from the following: architect, structural engineer; construction manager; building surveyor; environmental surveyor; building services engineer; estimator; and buyer, thus covering the main roles involved in a ‘real-world’ building ‘design and build’ project team, and reflecting the complex and collaborative nature of the construction industry.

The nature of this project not only enables individual students to occupy a role that is closely associated to their own course, but also requires them to work closely with those from other disciplines. This helps them to make use of and develop their own specific knowledge base, as well as gain experience and appreciation of others’ roles whilst working in a large multi- disciplinary project context. This feature addresses the concerns of Fruchter and Lewis (2003), who felt that in order to become effective in multi-disciplinary teams when they graduate, architecture, engineering and construction students need to be exposed to cross- module, interdepartmental experiences.

Process of integrating critical thinking into PBL

The project required students to design a new building for the Faculty of Technology of the university, on the site of the current laboratories. This building should provide laboratory space and substantial teaching, research and administration areas as specified. It should be an architectural landmark, fitting in well within the contemporary campus and city centre. Furthermore, the design should be recognised as a flagship building of the university that boasts state-of-the-art technologies and facilities with sustainability, flexibility and wellbeing credentials. Detailed specifications and requirements were outlined in the design brief which was provided to the students at the outset of the project.

The project is designed as a continuous process of design development, feedback and improvement, leading towards final project presentation and exhibition. The process model developed in previous research (Pan and Allison, 2010) was adopted and adapted for integrating critical thinking into the PBL. This adapted process included 10 key stages (Table 1).

 The two hour workshop on critical thinking and its use for construction technology

appraisal was run with the students by the researchers (i.e. the academic and the Learning Development advisor) at the start of Term 1.

 Prior to the project, the students carried out a ‘construction technology appraisal’

assignment in Term 1. In self-selected groups students had to identify and analyse a specific aspect of construction technology from a real-life large and innovative building. The project followed this assignment and started in Term 2.

 The students were briefed on the importance of adopting a critical approach in their

project mobilisation lecture at the start of Term 2.  An interim assessment was undertaken six weeks into the project.

 The final assessment consisted of two parts: an assessment by a panel, including academic staff and industry experts and a peer assessment of student group  The final assessment consisted of two parts: an assessment by a panel, including academic staff and industry experts and a peer assessment of student group

Table 1 Key stages of integrating critical thinking into PBL No. Stage

Date

Detail

1 An introductory workshop of critical thinking

8 Oct 2009 (at the

Two hour workshop

start of Term 1)

- assignment (which required critical thinking) 2009)

2 Construction technology appraisal

Term 1 (Oct - Dec

3 A project mobilisation lecture

7 Jan 2010 (at the

Two hour lecture

start of Term 2)

4 First questionnaire survey

14 Jan 2010

5 Four interactive project advisory workshops, Jan - Mar 2010 Two hours for each and student group meetings

workshop

6 Group interviews

Mar 2010

Twenty minutes discussion with staff

7 An interim student presentation, with

Feb 2010

each student group

Two hours lecture project event

8 A review and mobilisation for the final

Apr 2010

One-day learning stakeholders

9 A final project event, involving wide-ranging

26 May 2010

- Note: In the UK, academic years start at the end of September, with Term 1 from Sept to Dec and

10 Second questionnaire survey

26 May 2010

Term 2 from Jan to May.

Research design and data collection and analysis

This study adopted an action research philosophy, and employed a multi-methodological research design, in order to enable triangulation of data analysis and evaluation of results (Bryman, 2004). The study included literature review, questionnaire surveys, group interviews, observations, and reflections as had been successfully employed by other studies on PBL.

The literature review, in the first instance, concentrated on the broader current practices in PBL and their relevant strengths and weaknesses (see Dochy et al., 2003; de Graaff and Kolmos, 2003). It then focussed on the application of PBL to subject areas closely allied to Environmental Building, such as Architecture (Fruchter and Lewis, 2003; Roberts, 2004, 2007) and Engineering (Wai Hung et al., 2003; Perrenet et al., 2000; Ahern, 2010). This literature was also considered in terms of the methods employed for the evaluation of PBL. This review highlighted that questionnaires have been successfully utilised as a method of evaluating the effectiveness of PBL and students’ experience of PBL by Douvlou (2006), Mantri et al. (2008) and Chung and Chow (2004), who also adopted a two-questionnaire approach. The questionnaire used by Douvlou was administered 11 weeks into the programme, and aimed to explore the students’ experience of the teaching methods The literature review, in the first instance, concentrated on the broader current practices in PBL and their relevant strengths and weaknesses (see Dochy et al., 2003; de Graaff and Kolmos, 2003). It then focussed on the application of PBL to subject areas closely allied to Environmental Building, such as Architecture (Fruchter and Lewis, 2003; Roberts, 2004, 2007) and Engineering (Wai Hung et al., 2003; Perrenet et al., 2000; Ahern, 2010). This literature was also considered in terms of the methods employed for the evaluation of PBL. This review highlighted that questionnaires have been successfully utilised as a method of evaluating the effectiveness of PBL and students’ experience of PBL by Douvlou (2006), Mantri et al. (2008) and Chung and Chow (2004), who also adopted a two-questionnaire approach. The questionnaire used by Douvlou was administered 11 weeks into the programme, and aimed to explore the students’ experience of the teaching methods

a number of statements that the students had to grade on a five-point scale, these then identified three levels of learning, deep, surface and achieving, whilst the ILP questionnaire asked students to choose from a list of statements those that best described their ideal learning environment. The ‘simple’ questionnaire employed by Mantri et al. (2008) was administered at the end of the programme, and was analysed in conjunction with observations that were carried out throughout the programme. Further to this, semi- structured interviews were the main method employed by Abrandt Dahlgren (2003) in evaluating PBL in three different programmes.

Data in this study were therefore collected through:  the first questionnaire survey carried out on 14 January 2010;

 group interviews on 25 March 2010;  the second questionnaire survey on 26 May 2010;  observations and reflections of the researchers during the process;  discussions with staff during the advisory workshops, interim and final presentations;  discussions with industry assessors on the project event day.

The use of two questionnaires, one at the start of the project and one at the completion of the project, highlighted any changes that had taken place in the students’ understanding, perceptions or application of critical thinking. The format of these questionnaires was partly quantitative with close-ended questions and Likert scales, and partly quantitative with open- ended questions.

The first questionnaire survey aimed to: investigate students’ understanding and performance in critical thinking and appraising construction technology; identify any problems and underlying issues; and develop strategies for improving student deep learning. Likert scales were then used in questions that broadly focussed on the different types of data that might inform judgements, levels and points of attention for analysis, the use of literature, and how to critique constructively. Whilst these questions were weighted towards the subject area and might be interpreted as being ‘leading’ students towards the ‘right’ answer, its purpose was to ensure they were heading in the right direction, and to gauge their strength of feeling. Out of 67, 44 responded to this questionnaire survey, representing a response rate of 66% which is considered justifiable drawing on the literature of response rates in questionnaire surveys (see Bryman, 2004).

The group interviews were conducted part-way through the project during the third of four advisory workshops. Six questions were used as prompts to enable the students to discuss their experiences of the integration of critical thinking into the PBL process. The interviews aimed to gain richer qualitative data and reveal students’ collective, or group, feeling on how The group interviews were conducted part-way through the project during the third of four advisory workshops. Six questions were used as prompts to enable the students to discuss their experiences of the integration of critical thinking into the PBL process. The interviews aimed to gain richer qualitative data and reveal students’ collective, or group, feeling on how

The second questionnaire survey was carried out on the project event day. This survey explored: if the students felt their critical thinking had improved; if it had helped their understanding of the subject matter; and, what could have been improved regarding its integration and delivery in the PBL process. From a total of 68, 31 students participated in this survey. However, the responses to the qualitative questions of this survey were limited both in number, with only ten out of the 31 providing any sort of response, and in depth, with many being answers of only a few words. The timing of this questionnaire should have been considered more carefully, as the students were significantly distracted by their presentations, and that they had already provided feedback through a couple of channels on the PBL aspect and the programme in general.

Throughout the PBL process observations were made on the development of students’ projects, particularly during the advisory workshops. In addition, the students’ interim and final presentations were observed, specifically looking at the students’ critical decisions and the rationale these were based upon, as the projects evolved.

Furthermore, discussions with the staff involved in the project and industry assessors on the project event day gave further insight into the criticality demonstrated by the students. The data generated from all these activities together enabled an effective and thorough evaluation of the integration of critical thinking in the PBL process and its impact on student learning.

The quantitative data was analysed using descriptive techniques. The qualitative data (e.g. responses to open-ended questions of the questionnaire surveys, transcripts of interviews, and notes of observations and discussions) was analysed using the ‘content analysis’ method which typically reflected the process of identifying the codes, themes and patterns of data/responses in relation to the research questions (Patton, 2002). This covered the areas including student perception and understanding of critical thinking, student evaluation of the integration of the critical thinking approach into PBL, issues associated and strategies for improvement. Where a broad range of (or conflicting) responses were received, the frequencies of the codes identified from such responses were counted to identify the primary themes and patterns. Such responses were also examined to enable interpretation of the complexity and variety of student perception and practice of utilising critical thinking in their PBL. The use of the combination of quantitative and qualitative analyses enabled the exploration of complex research questions and subsequently the achievement of effective interpretation of the results. The findings are examined in relation to the aim and objectives of the project, and discussed with relevant previous studies.

Results and Analysis Students’ perception and understanding of critical thinking

The results of the first questionnaire survey shows that only 2% of the students considered their initial understanding of critical thinking as ‘very good’, and 23% as ‘good’. Nearly two thirds (64%) held a ‘neutral’ attitude, while 11% considered their understanding ‘poor’ but no student felt it was ‘very poor’ (n=44) (Figure 1).

Figure 1 Students’ perception of their initial understanding of critical thinking

Most of the respondents (40 out of 44) provided a definition of critical thinking, 22 responses included analysis-based activities, none described it in terms of questioning, except one listing some of the key questions in the critical thinking model used, ‘how, what, where, when’ (see Pan and Allison, 2010). Many responses reflected key themes such as ‘thinking about’ (9), ‘justification’ (9), and ‘opinion-forming’ (11) (e.g. using information to make decisions on alternative options). These descriptions show a pattern beyond purely analysis-based descriptions, into the weighing-up and decision-making area. Such descriptions reflected the concept of evaluation, although no response included the exact word ‘evaluation’.

The cross analysis of students’ perception and description of critical thinking suggests a generic negative correlation: i.e. those who felt they had a poor understanding tended to give better descriptions, such as “assessing the positives and negatives of a topic/area” and “analysing a task/item to see what the drawbacks and strengths are”. However, those who felt they had a good or very good understanding, did not seem able to define it well, for instance, “Analysing and answering a question in a certain and particular way” and “it's important in decision making for work and projects”. This negative correlation implies that the more knowledge and awareness a student has regarding critical thinking, the less competent they may feel in their ability to exercise it. This in itself offers an interesting opportunity for further exploration. The broad range of responses reflects the complexity in defining criticality and the fact that this can be very individual. Additionally, time had been spent in the first The cross analysis of students’ perception and description of critical thinking suggests a generic negative correlation: i.e. those who felt they had a poor understanding tended to give better descriptions, such as “assessing the positives and negatives of a topic/area” and “analysing a task/item to see what the drawbacks and strengths are”. However, those who felt they had a good or very good understanding, did not seem able to define it well, for instance, “Analysing and answering a question in a certain and particular way” and “it's important in decision making for work and projects”. This negative correlation implies that the more knowledge and awareness a student has regarding critical thinking, the less competent they may feel in their ability to exercise it. This in itself offers an interesting opportunity for further exploration. The broad range of responses reflects the complexity in defining criticality and the fact that this can be very individual. Additionally, time had been spent in the first

Students’ perception and understanding of critical thinking was further measured in the first questionnaire (Table 2). The results suggest a generally good understanding of critical thinking in construction technology appraisal, although, apart from when ‘evaluate’ (32.6%) and ‘critique’ (34.9%) were mentioned, none of the statements generated particularly ‘strong’ responses either way. The students did seem to be less critical with regards to the use of references and justifying alternative construction technologies.

Table 2 Students’ perception of critical thinking in construction technology appraisal

Strongly

Strongly disagree Disagree Neutral Agree

agree

Both qualitative and quantitative

information are important to support critical

appraisals.

0 2.3% 18.6% 65.1% 14.0% Apart from describing the construction

technology, it is necessary to analyse and evaluate it.

00 2.3% 65.1% 32.6% Critical analysis means to acknowledge

the merits while pinpointing the shortcomings objectively.

0 2.4% 23.8% 57.1% 16.7% There is no need to critique references as

they are always reliable. 34.9% 48.8% 9.3% 4.7% 2.3% The more references included, the more

convincing your arguments will be.

0 30.2% 32.6% 32.6% 4.7% It is important to get feedback from people

who are involved in designing, building, developing, and/or using the technology.

0 2.3% 11.6% 62.8% 23.3% When proposing alternatives, providing

general information is enough. How alternatives would address issues with existing technology is unnecessary.

In the group interviews, students appeared to be divided regarding any change in their understanding of critical thinking over the course of the project, although it was generally considered to have improved. Three groups expressed definite improvement to their understanding of critical thinking, particularly in relation to structure and criticality. With responses such as “Yes definitely,… you [now] see things from different angles from different perspectives”, and, “Yes I think it is positive, it is necessary”. The other three groups held different views on this: “unconsciously thinking things over more than before” (which was positive); “not changed but more emphasised” (which was still positive); and “no”. One group In the group interviews, students appeared to be divided regarding any change in their understanding of critical thinking over the course of the project, although it was generally considered to have improved. Three groups expressed definite improvement to their understanding of critical thinking, particularly in relation to structure and criticality. With responses such as “Yes definitely,… you [now] see things from different angles from different perspectives”, and, “Yes I think it is positive, it is necessary”. The other three groups held different views on this: “unconsciously thinking things over more than before” (which was positive); “not changed but more emphasised” (which was still positive); and “no”. One group

In the second questionnaire survey, the number of respondents who felt that their critical thinking had improved over the course of the design project rose to 80% (n=30): 77% (n=30) of respondents to this survey also felt that their overall understanding of their subject area had improved. Additionally, the industry assessors’ comments on the students’ demonstration of critical thinking at the final presentation were positive. They highly regarded the overall achievement of the student projects, which demonstrated good research efforts and relevant employability skills.

These results suggest that the students gradually developed a deeper understanding of critical thinking as a process of describing, analysing, justifying and evaluating solutions. Whilst surface learning was observed in the PBL process, this was primarily at the early stages.

Students’ evaluation of critical thinking and the integration process Overall PBL process

The results of the first questionnaire survey show that the majority of the students considered critical thinking to be either important (65%) or very important (19%) to their project, particularly for selecting construction technology and identifying design solutions (n=40).

All of the six student groups interviewed agreed that the PBL process had encouraged a critical approach in their project. Their enthusiasm however, varied. Group 2 commented that “yes because you have to think about the aspects of a building critically”, that the critical thinking approach was very important for their project, and they had applied it to other modules. Group 1 liked the fact that they could apply critical thinking to real-life situations and saw it as beneficial on a programme that contains a great deal of theory, and that they “thought about processes more”. Groups 3 and 5 agreed that the PBL process had encouraged a critical approach, more so for this specific project than in other modules. However, Group 4 stated that the PBL process “sort of” encouraged a critical approach, although its impact was no more explicit than that found in other modules. Group 6 argued that the use of critical thinking in the project was not comparable to that in other modules, because of the complexity, large size and cross-disciplinary nature of the project.

All the student groups (five explicitly and one implicitly) agreed that critical thinking had been

a positive aspect of the environmental building discipline (programme), because it encouraged them to think about things (they might not otherwise have thought of), and helped them to provide reasoned opinions, instead of jumping to conclusions. However, Group 4 also expressed some frustration over not applying the critical thinking approach effectively, which did not help with their analysis in the previous coursework of appraising construction technology. Group 1 echoed this comment by claiming both positive and negative effects of using the critical thinking approach, albeit giving no elucidate explanation.

In the second questionnaire survey 100% (n=30) of respondents felt that the project had encouraged a critical approach.

Critical thinking workshop

Of the 44 respondents to the first survey, 40 commented on the effectiveness of the critical thinking workshop and its use for construction technology appraisal. The majority of the respondents found the workshop useful, albeit in varied degrees: very useful (3%), useful (25%), neutral (25%) and somewhat useful (35%). However, 13% of respondents found it ‘not useful’ (Figure 2).

Figure 2 Students’ evaluation of the critical thinking workshop

Forty three respondents to the first survey commented on the timing of the workshop in relation to the construction technology coursework assignment: 44% perceived it appropriate, 30% regarded it inappropriate and 26% were not sure. Forty two respondents commented on the timing of the workshop in relation to the project: 38% perceived it appropriate, 41% regarded it as inappropriate and 21% were not sure.

Impact of critical thinking on creation and innovation

The encouragement of a critical approach appeared to have helped students with creating their design solutions, but not to have had any specific influence on the innovation used in the design solutions.

Four student groups (1, 2, 4 and 5) confirmed that critical thinking helped them in forming opinions, reasoning and justifying design solutions more objectively and effectively. Group 4 commented “...especially discussing … about what services we are going to put in we were going to discuss…the sort of systems, the advantages and disadvantages of using different systems”. Group 6 expressed that critical thinking did not have much effect, while Group 3 did not make any clear position in this question. None of the groups explicitly referred to innovation, either in relation to their design or in relation to critical thinking. Group 4’s Four student groups (1, 2, 4 and 5) confirmed that critical thinking helped them in forming opinions, reasoning and justifying design solutions more objectively and effectively. Group 4 commented “...especially discussing … about what services we are going to put in we were going to discuss…the sort of systems, the advantages and disadvantages of using different systems”. Group 6 expressed that critical thinking did not have much effect, while Group 3 did not make any clear position in this question. None of the groups explicitly referred to innovation, either in relation to their design or in relation to critical thinking. Group 4’s

In responding to the second questionnaire survey 60% (n=19) of respondents commented that they had been able to address the issues affecting their critical thinking that they had raised in the first questionnaire survey and 57% (n=28) of respondents felt that the explicit nature of critical thinking in the project had helped the creativity of their design solution. However, this was not echoed in the group interviews, while 39% said it had no effect, none felt that it had hindered their creativity.

Transferability of critical thinking

All the groups (except Group 6 for which no response was recorded) agreed that they would

be able to transfer the critical approach to other learning areas, such as coursework and exams in other modules. Group 4 discussed how they struggled with the process of using the critical thinking approach. A level of uncertainty and lack of knowledge of using the approach were observed, with students indicating that more input was expected.

Issues with the use of critical thinking

Nearly three quarters (31 out of 44) of the responding students to the first questionnaire survey raised issues which affected their critical thinking in their coursework of appraising construction technology. Five broad themes were identified:

 Problematic time management. This appeared mainly attributed to a lack of general study skills (for coursework assignments and the project). The students also

expressed time pressures working on the design project (with its complexity) and dealing with other coursework at the same time.

 Insufficient skills at study and group management. There was a lack of

understanding of how to manage projects effectively and how to work in a team efficiently. Good practice, like “open discussions with team members” was suggested, while issues like “fragmentation of groups” were also raised.

 Perceived overwhelming magnitude of research and finding sources. Fifteen

(34%) of the participating students expressed a need for more training on research skills. Two respondents expressed specifically a need for more guidance on doing research in a thorough and effective way. Eleven students appeared to recognise the need for thorough and effective research as a backdrop for critical evaluation.

A lack of understanding of the critical thinking process and how to conduct

research critically. The lack of understanding appeared to stem from unfamiliarity with the actual process of critical thinking, in combination with a lack of appreciation as to how long the process of researching, thinking, evaluation, rejecting and re- assessing could take.

 The timing of organising the critical thinking workshop (i.e. in Term 1). Students suggested that the message be repeated during the process of learning.

In the second questionnaire survey, only a small number of students (four) responded to the question on the main issues that affected critical thinking in the design solution. The issues suggested were centred on student’s critical thinking technique, “backing up what we decided to use”, and sustainability as an essential requirement of the design.

Strategies for improving the integration of critical thinking into PBL

Over three quarters of the respondents (35 out of 44) to the first questionnaire survey provided recommendations for improving the workshop on critical thinking, which were centred on timing, content, and delivery style.

 The timing of the workshop. Half of respondents (51% (n=35)) specifically mentioned its timing or proximity to the assignments being set, although this varied

from it being too soon for some and too late for others. Four respondents preferred an earlier delivery, i.e. in Year 1 or at the very start of Year 2 (which was factual in the project). However, 10 responses indicated a preference for later delivery, i.e. immediately before or after the start of the Design Project. One respondent suggested providing both sessions.

 The content of the workshop. Eight students suggested that the workshop be more

tailored to their course, with more construction industry examples of utilising the critical thinking approach.

Again, more than three quarters (34 out of 44) of the respondents to the first survey also suggested personal measures for improving their critical thinking, by addressing the issues raised above. Three themes were identified:

1. To improve time management. Nine responses expressed that they should manage their time better. Two also commented that some students felt the need to assess more carefully their time management skills from the very beginning of the project. The nine responses were related to both the time the students allocated themselves and the time they actually applied to the process of research and analysis. The comments “give myself more time” and “spend more time on research and ensure enough carried out” suggest an awareness that they should be doing more research and the fact that they will need to learn how to work this increased requirement into their learning routine. Two responses suggested the use of a plan or planning.

2. To improve critical thinking, study and management skills. Seven responses expressed a desire for achieving a better understanding of critical thinking. These ranged from a complete lack of knowledge to a reasonable level of understanding, which suggests an absence of a well formed understanding of critical thinking and the critical thinking model. Ten responses indicated a desire to improve their study and management skills, suggesting that students perceive the improvement of personal skills as helping with the improvement of technical skills, i.e. critical thinking. These responses included an expressed desire for achieving greater focus (three) and motivation (two), trying harder (one) and being smarter (one), in addition to an increased use of effective study skills, such as having more practice (two).

3. To carry out more and better quality research. Eleven (out of 34) expressed that they should improve their level of research, including doing wider reading, re-reading lecture notes, using more sources, more use of books and journals, and less reliance on the internet.

Interestingly, only one response included “work as groups to critically appraise things”. This might be due to the fact that the question asked for personal measures and therefore students answered the question from their personal viewpoint. That groups were not mentioned in this context could also indicate: 1) that the students feel that critical thinking skills need to be acquired by everyone individually; and 2) that the critical thinking workshop was not recognised as a significant tool for the project.

Of the 44 respondents to the first survey 26 suggested measures for the University or lecturers to help improve students’ critical thinking for their design project. Three themes were identified: more structured and active teaching style for critical thinking, with the inclusion of more construction-related examples; earlier delivery and repeat of the critical thinking teaching; and, greater instruction and guidance on researching. Comments received from the group interviews centred on four aspects of the introduction of critical thinking: timing, content and delivery of the workshop, and transition from Year 1 to Year 2.

 The timing of the workshop. The critical thinking workshop was delivered on 8 Oct 2009, a week after the issue of the design project brief. Four out of six groups

preferred the workshop to be delivered at a later stage. Group 3 explained that students had not started to think about the project until March 2010, so the delivery of the critical thinking workshop would be better aligned with the process of student learning. Group 5, however, argued that the delivery should not be as late as that, but

be a week or fortnight after the start of the project (i.e. at the start of Term 2). There was one group which was happy with the timing and another group did not comment on the timing.

 More construction-related examples to be included. This was supported by

Groups 2 and 3. Group 4 added that the resources should be accessible to all.  Embedded, subliminal, multi-module delivery of the workshop. This was

suggested by Groups 1 and 3. Group 3 would also have preferred more practical delivery.

 Critical thinking as a step-change in the ways that students have to think. The

transition between the levels of learning, from Year 1 to Year 2, needs to be clearly signposted. Group 5 expressed that “...there was no transition period from the first stage to the second stage...” and “it needs to be right at the start so you are aware of the change in level”. These comments tie in with the feelings of Group 2 in Question

5, who said that their understanding of critical thinking has “definitely [changed] from the first year to the second year”. They also suggest in Question 6 in the second questionnaire survey, that “better warning of the critical thinking change from Year 1 to Year 2 would have been appreciated”.

Once again when asked how it could have been improved, the groups felt there could have been constant reminders throughout the process to make it more embedded, better use of examples, and better timing.

In the second questionnaire survey, there were far fewer respondents (seven) to the questions on strategies for improvement. The responses suggest two themes of improving the introduction of the critical thinking approach to the project. They were on the timing and content of such introduction. Three respondents showed a preference for earlier delivery of critical thinking, while two would have preferred a delivery at a later stage. One respondent requested more examples and “better transition of the critical thinking change from Year 1 to Year 2”; while another respondent indicated that the delivery of wider skills was necessary to aid the critical evaluation of ‘other’ issues (additional to the key focus of sustainability). One respondent, however, was happy with the implementation of the critical thinking strategy and suggested no improvements. Four respondents indicated that better training in critical analysis (e.g. more tutorials, brain storming sessions, and examples) would have helped students to better integrate critical thinking in the project. Three respondents indicated that a more detailed design project brief would have aided the incorporation of critical thinking. Better training on critical thinking might help students understand the project brief better and tackle project tasks (which are multi-disciplinary and complex) with more confidence.

Discussion

The results generated from the combined data collection activities provided effective evaluation of the integration of critical thinking into the PBL process in environmental building. A number of key results warrant evaluation within the context of previous research,

which leads to the development of a refined process that will be applicable in a wider context.

When and how to introduce critical thinking?

The introduction of the critical thinking approach appeared to play a significant role in student learning and engagement. The approach utilised in this study was adapted from the previous year (2008-09), when the workshop was scheduled at a later date, prior to the beginning of the PBL project. On the previous occasion it was felt that the students had not had sufficient time to really grasp the critical thinking concept, or apply the model or materials they had been introduced to before the project started. As a result, ‘students struggled to understand and implement the process model of critical thinking, and therefore did not really embrace the critical approach in their thinking’ (Pan and Allison, 2010, p.551), with the students only gaining momentum with regard to critical thinking towards the latter stages of the project.

In light of this issue, the critical thinking workshop was provided at the beginning of Term 1 in the academic year 2009-10, just after the students had received the construction technology appraisal assignment brief. This gave students an opportunity to use and become familiar with the materials, whilst also gaining summative feedback at the end of the coursework, prior to the project. In practice, the timing of the workshop adopted in this present study received a mixed response from the students, some feeling that it was sufficient, particularly for the construction technology appraisal assignment, whilst others felt it could have been In light of this issue, the critical thinking workshop was provided at the beginning of Term 1 in the academic year 2009-10, just after the students had received the construction technology appraisal assignment brief. This gave students an opportunity to use and become familiar with the materials, whilst also gaining summative feedback at the end of the coursework, prior to the project. In practice, the timing of the workshop adopted in this present study received a mixed response from the students, some feeling that it was sufficient, particularly for the construction technology appraisal assignment, whilst others felt it could have been