The Evaluation of Project-Based Learning at the Mechanical Engineering Department , Polytechnic Malaysia : AProduct Dimension Assessment

The purpose of the study was to evaluate the effectiveness of Project-Based Learning in the Mechanical Engineering Department at the Polytechnic in Malaysia. This study was designed with Context, Input, Process and Product (CIPP) model. Project-Based Learning is a different approach than the traditional learning in the sense that it is a student-centered learning approach based on the principles of constructivism. This study involved a random sample of 118 students and 43 supervisors in the Mechanical Engineering Department in Polytechnic Kota Bharu Malaysia. The instrument used in this study were a set of questionnaire. In terms of finding ideas, both students and supervisors agreed that students were encouraged to assess the real problem by their supervisors. However, the supervisors were uncertain whether the students went to the library to get the ideas for their project. In addition, the supervisors were barely agreed about the creativity of the students in finding the project ideas. However, students and their supervisors believed that critical thinking and creativity were needed in the ProjectBased Learning. The respondents also believed that e-SOLMS helped the students to work in group to complete the project.


Introduction
Higher education sector is at the forefront in driving the nation's progress and development.This sector plays a key role in shifting Malaysia's position to become a high income economy by 2020.The Ministry of Higher Education has deployed critical strategies and plans to strengthen and develop its human resources quality in terms to achieve the transformation's goals.The development of human capital is carried out by collaborative efforts using innovative methods, particularly in the context of teaching and learning [1].The Ministry of Higher Education through its dynamic efforts and collaboration with institutions of higher learning has laid a strong base foundation to supply graduates for Malaysian workforce [1].Economic competitiveness of a country is based on the skills of its workforce.The skills and competencies of the workforce are dependent on the quality of the country's education and training system.Technical and Vocational Education (TVE) is believed as the one of the various disciplines of education that can generate and increase economic growth of a country [2].Innovation of a student can be assessed based on his or her design, product, and solution.In vocational and engineering fields, the definition of quality graduates are able to design and produce the quality products.
Since the 16th century, Project-Based Learning method has been implemented in the architectural field [3].According to Knoll, Project-Based Learning grew out of the architecture which began in Italy during the late 16th century [3].Basicly, Project-Based Learning is considered as a non-traditional pedagogical model that emphasizes student-centered learning by embarking on complex, real-world projects through which the learners develop higher-order competencies.Different with the traditional teaching method, however, is still preferred by the majority of teachers in the exam-oriented system including in Malaysia.According to Diaz and Cartnal, substantial number of teachers thought that traditional teaching method was more suitable than the student-centered method when the focus is on the examination and the class size is large [4].Nevertheless, the major weakness of the traditional teaching method was the failure (of the students) to make connection between new information and what they had already known and between what they learnt and the real life situation.
An effective teacher should be able to apply varied teaching techniques to ensure his or her class is appealing and meaningful.Project-Based Learning approach has its root in the constructivist theory [5].Project-Based Learning aims to engage learners in realistic, and thought-provoking problems [6].Project-Based Learning involves mind and hands.In Project-Based Learning, students are facing a real problem or actual situation in which they are required to find the solutions by gathering various sources from books, journals, handbooks, manuals, brochures, Internet and so on.Projects tend to be more open-ended than problem-based learning, giving students more choices when it comes to demonstrating what they know [7].Unlike projects that are tacked on at the end real learning, the projects in Project-Based Learning are the centerpiece of the lesson.
The underlying theory of Project-Based Learning is constructivism.Constructivism theory views learning as the result of mental construction; that is, individuals learn by constructing new ideas or concepts based on their current and previous knowledge [8].Thus, projects provide learners with a real-world context or "authentic" task for learning, creating a strong "need to discover."By design, projects are open-ended.This means students need to consider and evaluate multiple options and solutions and, perhaps the creative ones.All these activities engage higher-order thinking skills.One of the goals of Project-Based Learning is to improve the creativity and innovative skills among the students.However, literature has shown that, generally, Asian students are less creative and innovative than their Western counterpart [9,10,11,12].
The industry is facing problem getting skilled workers due mainly to training mismatch.There are weak links between schools and industry [2,13,14].The skills mismatch and skills shortage have prompted the government to assess the suitability of the curriculum and training system that suggest to be reformed based on industry needs [15].Many higher education institution graduates have difficulty in finding the jobs and the employers faced challenges in searching the "right" workers.It has become the main challenge of TVET (Technical and Vocational Education and Training) in Malaysia.TVET model in Malaysia is rooted as institutional-based rather than industrial-based.The changes in work process due to rapid technological advances intensify demand for knowledge workers [16].Most of the workers do not possess adequate knowledge and skills to be recognized as K-workers [2,14,17,18].Ramlee stated that employers in the manufacturing industry in Malaysia believed that the technical graduates possess adequate technical skills but the employers feel less satisfied with regard to the employability skills of the graduates in terms of communication, interpersonal, critical thinking, problem-solving and entrepreneurial skills [2,13].The mismatch between the supply and demand for a skilled workforce should be focused in a more comprehensive manner through a structured academia-industry collaboration, especially in the critical areas such a curriculum development and industrial training [19].The issue of unemployed graduates has been in the news every year.According to Chew Cheng Sim, Malaysian Department of Statistics in 2010 shows that 65,500 graduates were unemployed in 2010 and they were predominantly in the 20 to 29 ages [20].There were several deficiencies happens to the graduates, such as lack of core knowledge and competency, lack of communication skills and language proficiency, and lack of general knowledge.
In the Malaysian polytechnics system, e-Students Learning Management System (e-SOLMS) is used in tandem with Project-Based Learning.The system was designed by Universiti Sains Malaysia [21].A networked project typically involves students in distant locations cooperating to research, exchange information, and learn from one another, although the distant partners may include experts.Students may conduct research, perform experiments in their own community, and report their findings.They may pose questions to experts or exchange information with their peers.

Literature Review
Constructivism has led to some learning models such as Project-Based Learning, Problem-Based Learning, Work-Based learning, or others models which are basically studentcentered.Blank stated that Project-Based Learning approach has its root in the constructivist theory [5].Basically, Project-Based Learning is about solving real-world problem.According to Schneider and Grant, Project-Based Learning is an instructional approach that emphasizes student-centered learning by assigning project [22,23].In addition, Project-Based learning is a pedagogical model that involves students in investigation of real-world problems that culminate in authentic products [24].
In the traditional learning approach, teachers usually stand as a source of knowledge.They usually assert the students as information recipient.Different with Project-Based Learning.It is a learning approach that transforms teaching from "teachers telling" to "students doing" [25].According to Andreas and Rogers, students will face some advantages such as, free choice in working, plan the project, participate in defining criteria, solve the problems, and able to present their project [26].In Project-Based Learning, supervisors are responsible to monitor the adequacy of resources, information, learning contexts, project time, and tasks [27].It is important to motivate and support the teachers to implement Project-Based Learning because they play a critical role in the teaching and learning process.
The Buck Institute for Education claimed that the important factors of Project-Based Learning are: critical thinking, problem-solving, collaboration, and communication [28].Specifically, Larmer and Mergendoller have identified seven essential elements of meaningful projects: a need to know, driving questions, students voice and choice, 21st century skills, inquiry and innovation, feedback and revision, and a tangible product [29].Students have ability to develop many competences -soft skills and hard skills.

METHODOLOGY
This research was a kind of descriptive study intended to assess the perspective of students and supervisors regarding the Project-Based Learning at one Polytechnic in Malaysia.Specifically, this study utilized a case study method which examined on one polytechnic.The research design was based on the research objectives and the conceptual framework.The demographic variables in this study were ethnic, gender, residential area, and program areas.Additional supervisors variables included academic qualification, and years of teaching experience.This study employed CIPP (Context, Input, Process, and Product) evaluation model to assess the effectiveness of Project-Based Learning.
The respondent for this study was separate into two groups.The first population was final-project course (J5012) supervisors in the Mechanical Engineering Department in the selected polytechnic.The second population was final-year students who took the final-project course (J5012).The project course started in June and ended in September 2012.Sample size was determined by using table of sample size by Krejcie and Morgan [30].According to Krejcie and Morgan, a population of 170 could be represented by a sample size of 118 respondents.However, there were only 43 supervisors that willing to participate in the study.

Result
The respondents of the study were divided into two groups, the students (n=118) and the supervisors (n=43).They consisted of 72.9% male and 27.1% female of the students and there were 79.1% males and 20.9% females of the supervisors.

The Perception of the Students and Supervisors Regarding the Process Dimension of the Project-Based Learning
The items were developed to examine the effectiveness of the Project-Based Learning.The items were separated into four categories: Finding Idea, Problem Solving, Time Management and Knowledge Improvement.

A. Finding Idea
Finding an idea of the project was the first main step in project.Regarding finding the project's idea, the students agreed (M=3.92;SD=0.79) that they had acquired the idea before project started (item 1).The students also agreed (M=3.92;SD=0.85) that they were encouraged by their supervisors/lecturers to assess the real problem in the industry (item 2).Nevertheless, the students seemed uncertain (M=3.48;SD=0.93) to find idea at the library (item 3).Thus, the students seemed to agree (M=3.87;SD=0.94) that they decided idea by themselves (item 4).However, the standard deviations for item 3 and 4 were relatively high suggested a high variation in responses.Finally, the students strongly agreed (M=4.26;SD=0.77) that they discussed about project idea with the supervisor before project begins (item 5).In terms of finding Idea from supervisors' perspective, they strongly agreed (M=4.33;SD=0.60) that they encouraged their students to examine the real problem in the industry (item S1).However, they just slightly agreed (M=3.51;SD=0.76) that their students went to the library to find project idea (item S2).Regarding to creativity, the supervisors seemed to agree (M=3.65;SD=0.72) that their students were creative in finding idea for project (item S3).The supervisors strongly agreed (M=4.23;SD=0.89) that their students discussed the project idea with them (item S4).However, the standard deviation for item S4 was relatively high which suggested a high variation in responses among the supervisors for the item.

B. Problem solving skills with Project-Based learning
Table 5 presents the problem solving skills from students' perspective.Items P6 -P11 were set to examine the students' problem solving skills.The students seemed to agree (M=3.75;SD=0.94) that their problem solving skills have increased through Project-Based Learning (item P6).They believed (M=4.09;SD=0.73) that they have discussed the project (item P7).They also strongly agreed (M=4.25;SD=0.63) that their critical and creative thinking were important in solving the problem (item P8).The students believed (M=3.90;SD=0.98) that conducting Project-Based Learning with e-SOLMS has improved their ability to work in group to solve the problem (item P9).The students strongly agreed (M=4.30;SD=0.73) that they shared ideas and topics needs on the project with their groups (item P10).The students concurred highly (M=4.20;SD=0.89) that they managed effectively the information with the group in solving the problem (item P11).In terms of Problem Solving Skills from supervisors perspective, they believed (M=3.86;SD=0.77) that Project-Based Learning, has increased student's problem solving skill (Table 6, item S15).They agreed (M=4.09;SD=0.86) that their students discussed with the supervisors if there were problems (item S16).The supervisors strongly agreed (M=4.47;SD=0.59) that critical and creative thinking were important in solving the problem (item S17).The supervisors agreed (M=4.00;SD=0.81) that Project-Based Learning has enchanced students' ability to work in group (item S18) and they also agreed (M=4.14;SD=0.67) that the students shared idea with their group (item S19).The supervisors agreed (M=4.16;SD=0.65) that the student has managed information with their group (item S20).

C. Knowledge Improvement
Regarding to the knowledge Improvement through Project-Based Learning at the Table 7, from the students' perception.The students believed (M=3.97;SD=1.05) that their supervisors assigned responsibility of project design to the students (Item P12).The students also agreed (M=3.67;SD=0.97) that Project-Based Learning has enhanced their self-confidence and motivation in project (item P13).To the items P14 and P15, the students believed (M=3.69;SD=1.03) that Project-Based Learning has increased the students' experience and knowledge about project (M=3.86;SD=1.00).The students agreed (M=3.91;SD=0.98) that they could applied the knowledge gained from the project for their future work.Then, the students believed (M=3.87;SD=1.04) that their lecturers/supervisors have helped them with the theoretical and technical aspects (item P17).8 presents the supervisors' perspetives of students' knowledge improvement.Item S21, the supervisors were uncertain (M=3.05;SD=1.21) whether the students assigned responsibility of project design.The supervisors agreed (M=4.02;SD=0.70) that Project-Based Learning has increased their students' self-confidence and motivation (item S22).The supervisor also agreed (M=4.12;SD=0.62) that their student had valuable experience in the project (item S23).Item S24, the supervisors strongly believed (M=4.53;SD=0.50) that medium e-solms has helped the students in the project.They also concured (M=4.09;SD=0.68) that the students could apply the knowledge and experience to meet the challenges of work (item S25 .70Note: In parenthesis is the margin of error at 95% confidence level.

Discussion
Regarding to Project-Based Learning knowledge, the students stated that they knew about Project-Based Learning.Some students believed that Project-Based Learning as e-SOLMS.Andreas and Rogers asserted that people when engaging in a project may have different perspectives regarding the project [26].Otherwise, the supervisors perceived Project-Based Learning as a theory.They also agreed that Project-Based Learning is an appropriate approach for the students' final project.In addition, the supervisors believed that e-SOLMS monitored the students' projects effectively.McAlpine, Reidsema and Allen asserted that online medium may facilitate Project-Based Learning implementation [31].
In terms of Project-Based Learning Process, the students believed that their supervisors have encouraged them to assess the real problem in the industry.However, the students defined that they did not get any information from the library.Mergondoller and Thomas asserted that the purpose of Project-Based Learning is to shift the responsibility from the teacher to the students [27].The students should make decision for their project by themselves.
Regarding of problem solving skill, the students and the supervisors believed that critical and creative thinking are important in the project.In addition, they agreed that students' problem solving skills have enhanced through the Project-Based Learning.From the data, if there was a problem during the project, the study found that the students met and discussed with the supervisors.There were several main problems in completing the project such as insufficient time, high project-cost, and lack of collaboration in the group.Kurzel and Rath stated that effective teamwork is critical in any project [32].
With respect to knowledge improvement, the students and the supervisors believed that the students have improved their knowledge after completing the project.However, the supervisors assigned the project to the students.The students and the supervisors also agreed that Project-Based Learning has enhanced the students' motivation and knowledge.Thus, several students stated that they did not get any improvement.This funding was same with the finding of Santana, Dias, Molinaro, and Abdalla Jr, that the students' involvement in project with Project-Based Learning approach has contributed to understanding the real and practical of engineering [33].Therefore, the students and the supervisors agreed that the students could apply their knowledge and experience into their future work.The supervisors agreed that the medium e-SOLMS helped them in monitoring the project.In addition, the supervisors could fill the mark, evaluation or discussion using e-SOLMS online system.By using medium e-SOLMS, the students could make consultation with their supervisors.

Conclussion
Several conclusions could be defined based on the empirical findings of the study.In terms of knowledge, most students claimed that they have knowledge about Project-Based Learning.Meanwhile, the supervisors claimed that they have knowledge on both Project-Based Learning and e-SOLMS.Regarding to the process dimension of the Project-Based Learning, some students have already had their project ideas since the beginning of their studies at the polytechnic.Thus, in the final semester, they just proposed their ideas to their supervisors to decide on the project's title.
Nevertheless, some of the students believed that they seldom go to the library to search for project ideas.This finding might lead to a conclusion that some students were overly relying on their supervisors' suggestion for the project.Thus, the supervisors, blamed the problem to the students' negative attitudes, lack of communication, facilities deficiency and lack of creative products.To solve these problems, the supervisors recommended more trainings and workshops to be given to the students in terms of enhancing their communication, critical and creative thinking skills.However, both the students and the supervisors agreed that Project-Based Learning has enhanced the students' motivation and knowledge of their project.Moreover, the students believed that the experience from the project would help them into their future real-work.Generally, based on the results of the study, Project-Based Learning is a systematic learning, started with finding idea, solving the "real" problem, creating project design, working together, and producing products.Project-Based Learning implementation needs a relevant module to achieve project goals.alleviate the negativity of the students toward Project-Based Learning.6.The polytechnic should nurture and incubate the students' higher order thinking, creativity and innovative thinking through various programs and training.7. The polytechnic library and resource center should be furnished with relevant and latest books, journals, materials and resources to be used by the students for their project.8. Employability of the polytechnic graduates is expected to be enhanced if their research experience and the new knowledge they gained from the project is optimized and could be applied in their future work.

Recommendation
Jurnal Ilmiah Pendidikan Teknik Elektro, Vol.1, No.1, Feb.2017 | 53 Essentially, the finding of this study may benefit the Ministry of Education in realizing Malaysian Goverment Transformation Program (GTP) and Economic Transformation Program (ETP).Based on the results of this study, several recommendations for practice and future research are offered: 1.The polytechnic should maintain and extend Project-Based Learning implementation into other departments.ISSN 2549-3698 Sadrina dan Ramlee Mustapha e-ISSN 2549-3701 CIRCUIT: Jurnal Ilmiah Pendidikan Teknik Elektro, Vol.1, No.1, Feb.2017 | 61 2. Project-Based Learning coordinator should seek input and feedback from numerous parties such as experts, lecturers, students, and industrial professionals.3. The polytechnic should offer a formal course in Project-Based Learning and e-SOLMS 4. Students' collaboration, communication, interpersonal and problem solving skills should be enhanced with proper training.5. Paradigm shifts in the mind and attitudes of the polytechnic students are required to

Table 1
shows the basic knowledge of Project-Based Learning from the students' perspective.From the 118 respondents, more than half (55.1%) of the students believe that they knew about Project-Based Learning.About one-fourth (27.1%) of them were uncertain and the Jurnal Ilmiah Pendidikan Teknik Elektro, Vol.1, No.1, Feb.2017 | 55 rest (17.8%) did not know about Project-Based Learning.However, majority of the students (74.6%) agreed that they were comfortable working with Project-Based Learning through medium e-SOLMS.Nevertheless, almost half (50.8%) of the respondents were rarely active (once in a week) in e-SOLMS to gain information regarding the final project.Table2presents basic knowledge of Project-Based Learning from the supervisors' perspective.Out of 43 supervisors, the majority of them (81.4%)claimed that they knew about Project-Based Learning, and only one supervisor (2.3%) did not know about Project-Based Learning.All the supervisors claimed that they were comfortable working with Project-Based Learning through e-SOLMS, although most (67.4%) of them only use e-SOLMS once a week.

Table 3
In parenthesis is the margin of error at 95% confidence level.