Assessment and Learner Performance in Problem Based Learning (PBL)

Mary Thomas
Temasek Polytechnic, Singapore

Abstract

Teaching presupposes testing and assessments in some manner. The assessments can be for self- improvement (self-awareness) or for measurement. Whatever the case may be testing in PBL is still an infant. Most PBL practitioners build authentic problems and facilitate learning with a degree of competence. However, a lot of assessments types used in the PBL technique are really a copy of traditional pen and paper assessments.

This paper will give an overview of some of the pertinent issues that face assessments in PBL. The idea of integrated subject problem design can complicate assessments. While it may be practical and relevant to engage in inter-disciplinary problem design and encourage the collapse of subject boundaries it may not be feasible to integrate assessments to such an extend that differentiation between and among subject specifics cannot be distinguished.

Introduction

This paper will give an overview of student preferences for PBL assessments. It will look at the types of assessments favoured by students, the place of the reflective journal as well as the feedback they can receive through the assessment modes in place for PBL specifically for the Language and Communication subjects.

This paper will try and offer some insights into Problem Based Learning (PBL) and how it can be regarded as the misunderstood teaching methodology. Also in relation to this point the place of assessments in this whole paradigm of learning will also be considered.

Problem based learning (PBL) is relatively new in the world of teaching and learning. The concept of learning brings to mind the need for an individual, who can in 21st century education decide, design, direct and demark the parameters of learning. This is comforting – at least for the learner. The learner knows with a measure of certainty that someone far older and most usually wiser will be responsible for his learning.

So how does this wiser individual manage learning? What resources are available to him? Is this process a touch and go event? Is learning erratic and thus open to this wise individual’s idiosyncratic nuances? These then become rather central to how we perceive learning. This responsibility of entrusting learning and teaching in the hands of this older and most usually wiser individual has to change in relation to education and change to meet the demands of the 21st century. Learning and thus teaching have always been supported by sound educational theories. These theories have then been supported by actual approaches to help maximize individual learning and the approaches have a scaffolding battery of techniques to make the learning stimulating. In this entire cycle the assessments have played a focal role in determining the success of the approach adopted by teachers. So what then are these theories? As far as we can gauge where language learning is concerned the two main theories are:

The Behaviourist – habit forming theory, and the Analytical – mentally engaging theory.

The first tells us that all learning is a matter of learning new habits. The learner has to be indoctrinated in a particular process such that learning will be the outcome given the exposure and the similarity of the process. Ultimately the learner may form the metacognitive concepts necessary for real learning. However the concept of engaged realization is not a prerequisite of this learning theory. The learner as proposed by Thorndike makes connections between the given stimulus and the appropriate response (Saettler, 1990). It is evident that within the frame of a learning theory there are aspects of variations. Thus even in the habit formation theory we have variations proposed by Pavlov, Skinner, Thorndike and others. But what we need to focus on here is that in essence they are proponents of the concept that all learning is the formation of recognized stimulus and learned response.

So habit, recognition and selected response are an aspect of learning. They may not have a very high-level engagement or significant worth but they are essential in building bonds and establishing connections especially in language learning.

Thus this learning theory has an important place in general learning and in specific learning at the operational level. We have seen many language courses, which have been designed with this learning theory in mind. Specifically courses that are short-lived, blitzy in nature have this strong inclination. This was especially so in the Second World War. The learning here is merely a recognized conditioned response to a stimuli which has already been internalized by the learner. Internalization in this respect need not amount to higher order cognition but could be cognition at merely the level of recognition or familiarity. In many instances we do practice this especially when we consider such phatic communication responses to a question on our state of health, example if someone were to ask us: “how are you?” Our conditioned response would normally be: “ Fine, thanks”. This response is so common and expected that any deviant response to the one we anticipate would cause us to enter the realm of actual cognition. Let us then move on the second learning theory.

The second theory is based on the premise that the learner has to be mentally engaged in a given process if real learning is to take place. The learners mind has to be captured if real engagement of the learning concepts is to happen. The learner is taught a rule, which he then uses to analyze situations, which may be different from the initial situation, used for teaching. Application and realization are essential ingredients in this type of learning.

The two learning theories actually make sense. We do learn through modelling and thus we learn sets of habits, which make our daily operations very effective. We also learn through rules example, once we learn the rules of subject-verb agreement, we will be able to apply this rule whenever this aspect of sentence construction comes into focus. We will be able to extend this rule to other sentences where this kind of a construction is required because we have learnt the concept governing the rule. The learning theory in this respect revolves around the idea of a mentally engaged event for the learner. Good & Brophy (1990), states that, “… cognitive theorists view learning as involving the acquisition or reorganization of the cognitive structures through which humans process and store information”.

Thus, Chomsky’s idea of the engaged learner, the analytical learner who is able to comprehend deep and surface structures (meanings) in a stretch of communication is relevant if we subscribe to cognitive learning theory. This idea of learning is dramatically different from that of the behaviourist in that learning is the creation and managing of meaning and not the mere manipulation and recognition of circumstances. We are not focusing on a bystander cum passive learner but on an involved cum engaged learner.

The third learning theory, the constructivist theory can be seen as an aspect of the cognitive theory. While cognitive theory looks at the mass of knowledge as an entity apart from the learner and thus carries with it an objective view of the mass of knowledge the constructivist see knowledge as personal and meaning as individual. The crux of the matter is that both views require the learner to be an active participant in the learning process. An individual who can make hypothesis based on what he has experienced such that a level of new realization occurs is the strength of the constructivist.

So a learner who is trained using the constructivist theory would be someone who will be independent, mentally engaged and self-directed. Realization is a personal event and learning will mean new realization on a continuum. How are these learning theories implemented in the actual design of a subject? Are these theories relevant and do they significantly mould the way we organize learning?

Approaches

The learning theories we have focused on are supported by two approaches to teaching and these are the inductive and the deductive approaches. In the inductive approach the teacher will present a whole range of examples. The reason why this is done is simply to present the learner with fertile learning situations. The learner through a process of trial and error will then unconsciously learn. It is believed that the learner will in time realize the concepts underlying his learnt behaviour. The deductive approach on the other hand begins with the teacher presenting a formula/a rule, which governs the learning. The learner is mentally engaged in the learning process and learning then becomes a rule-governed engagement.

Thus the approaches used in the classroom complement the learning theories discussed. The inductive approach is very much an aspect of the behaviourist-stimulus/response learning mode and the deductive approach is a result of the cognitive theory of learning. The variation of the deductive approach is the hypothetico-deductive approach. This approach where learners have to form hypothesis from the mental engagement they are involved in is the approach aim of the constructivist learning theory.

Thus when we design courses we need to take into consideration the learning theory we subscribe to and the approach we take in designing the actual classroom encounter for our learners. The whole realm of approach is then supported by techniques, the actual battery of tools we use to support the approach we have taken. In the inductive approach, on the other hand such techniques like repetition, reproduction, mimicry, drills are common. In the deductive approach any techniques that require mental engagement is the rule of the day. So such techniques like information-gap exercises, task exercises, critiquing, reviews are all tools in this approach.

Just as we need to be careful in our planning and implementation stages we need to also consider the aspect of assessments.

Assessments

Where then do assessments fit in? Do we need assessments or are they just routine activities we do as part of the rigour of teaching? Can learning occur without any assessments?

These are questions many of us may no longer consider because we too may have subscribed to certain conditioning. We all know we must test. We have to evaluate. Performance needs measurement and assessments are the very tools for this. In fact many a time the teachers’ performance is measured in terms of his learners performance in assessments especially nation wide examinations. Thus, often times the assessments drive the teaching and learning. We teach to the test and we test the learning. Generally assessments can be divided into 2 main categories – criterion referenced tests and norm referenced tests. The criterion-referenced tests begin with a set of criteria and the test is so designed to gauge how much of the given criteria have been assimilated by the learner. This kind of a test is non-competitive and often times authentic to what most likely takes place in the real world where one is evaluated through the tasks that have been assigned and the gauge for the success or failure of the task is measured in terms of how much of the set criteria have been met or not met. Norm referenced tests on the other hand pit one individual against another in the learning process. This would be useful to measure the success of cohorts in standardized national evaluation.

Problem-Based Learning (PBL)

What is the role of PBL in this whole process? Is this another fad? A vogue that is here today and gone tomorrow? There may be some pressing concerns that we may have with regard to PBL. Yes, many of us have been taught using the inductive or the deductive approaches and we have done well. We too have taught using these approaches with our students and have our won success tales to share as well as tragic failures, which we may wish to obliterate. So why this PBL and for what? PBL is not a teaching theory. It is a learning approach. It is an approach to get as close a fit as possible to the needs of the real world. It attempts to simulate the real world in order for realistic modeling and cognition to occur. Why do we need to have this? If we are training learners for a vocation then it only makes sense that the classroom should resemble the world of work as closely as possible. This is where PBL comes into the picture. It tries to simulate the situations and the sensitivities that are likely happenings in the real world. In doing this it requires the learner to also adopt a number of skills. They have to be resourceful, collaborative, investigative, decisive in making and taking decisions, sharers of information, self-directed and independent learners, delegators of tasks and solution finders.

The skills mentioned here are the plus factors why PBL is such an attractive methodology. Personally, PBL is a marriage of the inductive and the deductive approaches to learning. Often times it is know as the Hypothetico-deductive approach. The learner has to work on a hypothesis, which through a specific process can be confirmed or disconfirmed, and this then becomes the deductive rule that the learner will use when he operates in a given situation.

The learner is given a catalyst in the form of a problem. The learner then needs to seek out relevant information from an infinite repertoire of information available. Decisions have to be made on what is relevant information and how the information can be used. Sharing of the information to get a total picture as to how the problem can be solved requires teams to practice excellent group dynamic skills. They then have to analyse all facts presented and synthesise the information into a workable solution for the initial problem given. Thus we see all the skills one may need in the work place surfacing when we practice the PBL methodology.

So how then can we assess our students? Should we do this in a one off norm-referenced end of course assessment? Should we think of a new way to assess our students? We cannot do without assessments. Assessments allow for bonding between the learner and the facilitator. The relationship between the two participants in the learning process become firmer as feedback allows for questions and for clarification which, rightly leads to better understanding of the two key players in the teaching-learning process. Assessments through feedback actually tell a learner how he is getting on. Thus assessments give a succinct picture to individuals as to their learning curve and their learning status in the whole process of learning.

Learners And Learning Preferences

The whole of learner preferences especially in a PBL driven subject was the essential premise of this study. A survey questionnaire was administered to about 100 students pursuing a Business Communication subject in their third year semester 1 diploma in Business programme at Temasek Polytechnic, Singapore. The study examined three main issues, namely:

Learning styles
Assessment preferences
Assessments outcomes

This third year business communication subject prepares the learner to meet the communication challenges in the world of work. We focused on three main areas of business transaction and these were:

1) Business Correspondence
2) Job Securing Process
3) Meetings Management

The entire subject is taught using the PBL as well as the PBL on-line methodology. It was thus rather interesting to find out what were some of the preferred learning styles.

Generally learners preferred group work and group collaboration. The two questions on this aspect specifically, how they like to learn as well as what kind of a learning engagement they preferred showed clearly that more than 50% of the respondents liked to study in groups. We can see that PBL and group collaborative learning actually seem to be a preferred learning strategy.

Catalyst To Learning

On the question of what should be a catalyst that drives learning about 65% of them preferred the case and the project approach. The survey considered these two as problems that had to be tackled by the learner. Generally, in the business school students have a tendency of regarding projects and cases as problems that have to be tackled.

With regard to what learners prefer lecturers to do, close to 64% like to have two things happening – one, the lecturers directing the learner and the other the lecturer giving the learner a freehand on deciding what should be learnt. Only about 35% wanted the lecturer to do stand up teaching and classroom teaching. We can see that students actually like to have ownership in learning. Thus using PBL seems to be the right path to take if we wish to allow for independence and accountability in learning.

Assessments

86% of those surveyed felt that assessments should support leaning, learners also felt (54%) that PBL should be supported by a pen and paper assessment. When asked whether it would be better if PBL were assessed through projects alone about 53% agreed with this while 47% felt otherwise. Although there seems to be a slight majority who prefer projects alone as test instruments there is nearly an equal voice that would prefer other forms of assessments. When asked what kind of an assessment would be good to capture learning in a pen and paper test the majority about 92.8% felt a problem scenario question would best serve this purpose. Also 84.8% of the respondents were of the opinion that a problem-based assessment would evaluate higher order skills like analysis, synthesis and justification.

About 73.6% of the respondents felt that the structured journal is a good tool to help them reflect on learning through PBL. As reflection is a primary goal of learning in PBL students have used this tool effectively especially because reflection is a neglected skill and it would be good to direct the learner on the necessary parameters of reflection.

On the question of where an assessment should be done, 67.5% felt that a take home test would be a better tool then a classroom assessment which, won favour with only 32.5% of the respondents.

We can thus see that here is a need to think of new ways of challenging the learner especially in an assessment situation. We have a tendency to play safe and keep with the time-tested methods of a sit down controlled test. However we may now need to see whether an assessment to gauge the learners’ ability in more dynamic ways can be used to give us a true picture of the learners potential.

Evaluation Outcomes

Here we will consider how students like to be informed about their performance in a subject. In short, whether they like a grade as given currently or a competency profile description. 71.9% of the respondents felt that a competency profile description would make more sense for the learner in indicating how his performance can be interpreted. Thus it would be beneficial to the learner to understand the skills level of achievement he has attained in a subject. He can then decide whether he needs to do a further subject in the same area to build on his mastery level or whether he is happy with his achievement as indicated in his skills descriptor at this point in time.

Conclusion

PBL is new. It is good to take on new methodologies if learning can be enhanced. But if we change methodology but revert to traditional testing then we are liable to create confusion. The essence of assessment in PBL should be to authenticate testing as closely as possible to the process of learning which is the brand essence of PBL. Just as a battery of tools are used in real life to enable individuals to competently face the challenges of the world, so too must a battery of assessment tools be used in PBL testing. We begin assessments in business communication with group letter writing assignments, followed by the writing of individual application letters, resumes and interviews and end with a meeting role play with the necessary written documents like, notice, agenda and minutes. Thus a whole range of testing devices is used to help authenticate assessments just as a battery of problems is given to stimulate learning.

The essential drawback in PBL assessments is the fact that assessments in this mode of learning are still in its infancy. This is essentially true in the area of language and communication studies. We have many rudimentary issues to grapple with like that of problem design, the need for multiple issues in problem design, the question of facilitation and collaboration among learners, the need for effective group dynamics. When all theses issues congregate, the question of authentic assessments that model the reality of the classroom generally take a back seat. Also the level of confidence in structuring an assessment that replicates a PBL learning situation is generally wanting. In such situation it is often safer to use assessment types that have been proven reliable and valid like essay question, multiple-choice assessments, task exercises, reading comprehension question where the learner is asked to spill evidence from his stored knowledge base rather than study a problem and then tease out the kinds of knowledge necessary to offer a viable solution.

The switch to an assessment type that would mirror the PBL teaching methodology will take time. However there is strength in this type of assessments in that the learner has to be engaged and see probable solutions from a wider perspective. As progress in this methodology is made there will be the likely developments in assessments to complement the teaching and learning nexus.

About the Author

The Author lectures at Temasek Polytechnic, Singapore.

• References
• Chomsky, Noam. (1972). Language and Mind, Harcourt, Brace & Jovanovich, NY.
• Chomsky, Noam (1986). Knowledge of Language: Its Nature, Origin & Use, Praeger, NY.
• Constructivist theory (J.Bruner) [on-line]. Available: http://www.gwu.edu/tip/bruner.html
• Good, Thomas L. & Brophy, Jere E. (1990) Educational Psychology: A Realistic Approach. Longman,NY.
• Goodman, Nelson. (1979). Fact, Fiction & Forecast, Harvester Press, NY.
• Ivie, Stanley D. (1998). Ausubel’s learning theory: An approach to teaching higher order thinking skills. High School Journal, 82(1), 35-42.
• Saettler, P. (1990). The Evolution of American educational Technology. Libraries Unlimited Inc, Englewood.
• Skinner, Thorndike, Waton. [on-line]. Available: http://www.sfsu.edu/rsauzier/thorndike.html
• Stern, David G. (1995). Wittgenstein on Mind & Language, OUP, Oxford.