Online PSA programs aim to make PSA easier to deliver with straight forward project set ups, automatic grade monitoring that wave up anomalies, and student accounts that allow fast and therefore more effective feedback. Students are far more likely to react to peer feedback if it is accessible soon after a task is completed.
Understanding How Students Learn
It has been said that we learn…
10% of what we read
20% of what we hear
30% of what we see
50%of what we both see and hear
70% of what we discuss with others
80% of what we experience personally
90% of what we teach others or immediate use
(Topping 2001, Page 4)
Although this ‘learning pyramid’ is referenced to Topping its source is ambiguous and it appears in many forms and on countless websites. It is an extremely popular philosophy for learning as its simplicity allows a clear vision of how learning and retaining information can occur. On its surface it has a logical and believable structure.
PSA involves reading, hearing, seeing, discussing, experiencing and teaching others. However though all the levels of the pyramid are relevant to a PSA strategy there is only an assumption of evaluation lying within the ‘discuss with others’ level. The best result, achieved through applying knowledge by teaching others or immediate use, suggests there is more to learning beyond the pyramid’s explanation. In many respects the pyramid represents one method of learning that could lead to another, it’s not necessarily wrong but too simple.
So what happens, for example, after we teach someone something we know? Or when a new skill or information is used? There is normally a period of reflection and evaluation however brief, and this can sometimes be subconscious. It is accepted that by focussing on and developing this reflection learning can move onto a new level and assessment can be better understood.
Moon (n.d.) uses the Harry Potter novel The Goblet of Fire to cite an excellent description of reflection. Dumbledore uses a stone basin called the Pensieve, “One simply siphons the excess thoughts from one's mind, pours them into the basin, and examines them at one's leisure. It becomes easier to spot patterns and links, you understand, when they are in this form.' (Rowling 2000).
Reflection is essential in recognising ones own successes and learning needs and can help to nurture creativity. Success stimulates the desire to continue further and improve. Identifying patterns and links using PSA allows connections to be made beyond the act itself, connecting the act to a wider context and set of criteria beyond ones own perception. This in turn can identify learning needs and validate them within a larger context. The learning pyramid makes no mention of reflection and its effects on learning; a more in-depth educational model is required to contextualise PSA more effectively.
Bloom’s Taxonomy of Educational Objectives offers a theory beyond the learning pyramid. It is necessary to look at Bloom’s original theory before attempting to understand The New Taxonomy and its relationship with PSA.
Bloom’s Taxonomy - The Cognitive Domain
The outcome of the learning pyramid referenced to Topping addresses retention of information, or knowledge, and this conveniently appears as the first level outcome of Bloom’s Taxonomy. Bloom’s pyramid encompasses the first pyramid’s essence and provides further levels of learning, beyond retention of knowledge, that move towards higher order thinking skills. Each level of learning, Bloom believes, was a progression of difficulty to the next.
Figure 1 Based on Bloom (1956)
The table below explains the levels in more detail identifying the characteristic of each.
Figure 2 Information adapted from Bloom’s Taxonomy (1956)
Bloom’s theory suggests a progression of difficulty between the levels; comprehension is more difficult than knowledge and same for the further levels in order, application, analysis, synthesis and evaluation. This is the main criticism of the taxonomy; the hierarchical structure does not make sense from an empirical perspective.
“Educators who were trained in the structure of Bloom’s Taxonomy were consistently unable to recognise questions at higher levels as more difficult than questions at lower levels of the taxonomy”
(Marzano & Kendall, 2007, Page 8)
This clearly presents a problem with the taxonomy and Bloom et al partly recognises this when discussing the evaluation level. They did however stop short of fully realising its flaw and unsupportable structure.
“Although evaluation is placed last in the cognitive domain because it is regarded as requiring to some extent all the other categories of behaviour, it is not necessarily the last step in thinking or problem solving. It is quite possible that the evaluation process will in some cases be the prelude to the acquisition of new knowledge, a new attempt at comprehension or application, or a new analysis and synthesis”
Bloom et al (1956, Page 185)
This statement suggests that Bloom et al accepts that evaluation is part of the whole process and that it occurs within the layers of the taxonomy. Its incorporation into application, analysis and synthesis levels could even be considered a beneficial springboard to achieving these levels. This regular and ongoing evaluation throughout the process lends itself perfectly to PSA within the timeline of an assignment.
Furthermore it could be said that creativity would naturally occur at the point of application and through PSA this could be recognised and acted upon. Kolb’s Learning Cycle could be included as smaller cycles using PA as the focus points across the timeline of the project. Ideally formative assessment and evaluation would become a continuous process and would in the eyes of the student become somewhat transparent and considered a normal and natural part of the learning process.
Why though should effective evaluation be considered any more difficult to achieve than any of the other levels in Bloom’s taxonomy? The most complex of processes can be learned and performed with little effort when familiar (Marzano, 2007). The New Taxonomy states that any taxonomy based on levels of difficulty would never work for the following reason.
“The difficulty of a mental process is a function of at least two factors - the inherent complexity of the process in terms of steps involved and the level of familiarity one has with the process. The complexity of a mental process is invariant - the numbers of steps and their relationship do not change. However, familiarity with a process will change over time. The more familiar one is with a process, the more quickly one excecutes it, and the easier it becomes”
Marzano, J & Kendall, J, (2007, Page 10)
The New Taxonomy provides a solution with a different model for learning that introduces the metacognitive mental system and three domains of knowledge. This research takes the New Taxonomy model further by suggesting PSA as a metacognitive process and vehicle for its use.
The New Taxonomy of Educational Objectives - Where does Peer & Self Assessment sit within it?
The New Taxonomy’s model for individual’s behaviour to learning establishes that for effective learning to take place, a metacognitive process needs to be present. This would occur after an individual chooses to engage and prior to cognitive learning.
Figure 3 Taken from The New Taxonomy 2007 Page 11
The model (Marzano, J & Kendall, J, 2007) proposes three mental systems, the self-system, the metacognitive system and the cognitive system. The self-system addresses whether the individual decides to engage with a new task or not and if there is a positive response there is progression into the metacognitive system. Metacognition, simply put, is thinking about thinking or having knowledge about ones own knowledge. It is an important factor in engaging students where they need to engage however it does not tell them how to engage. Metacognition is a process that helps to recognise weaknesses and individual learning needs, the evaluation of ones own learning needs can then form a strategy towards achieving goals. This differs from cognitive processes that may only help towards meeting a goal rather than ensuring a goal is met. This could be taking Dumbledore’s Penseive of thoughts and using it within a process of learning, in this case peer and self-assessment. By applying personal thoughts onto their peers’ work students recognize and identify their own learning needs. If the students’ work is substandard to their own they can recognize this and provide peer assessment, if it is of higher quality they can use this to identify their own learning needs. The learner learns to take control over their cognitive learning.
Metacognition consists of two basic processes occurring simultaneously; monitoring your progress as you learn, and making changes and adapting your strategies if you perceive you are not doing so well (Winn, W and Snyder, D 1998). It is easy to see how PSA fits the requirements of metacognitive processes:
"Metacognitive skills include taking conscious control of learning, planning and selecting strategies, monitoring the progress of learning, correcting errors, analyzing the effectiveness of learning strategies, and changing learning behaviors and strategies when necessary."
(Ridley, D, Schutz, P, Glanz, R. 1992, Page 293)
Further to this the New Taxonomy (Marzano 2007) differentiates between three domains of knowledge accounting for the original taxonomies flaw in attempting levels of learning difficulty. The three types are as follows:
-
Information – understanding vocabulary, facts, time sequences, cause and effect and correlations between related factors
-
Mental Procedures or procedural knowledge - how to do things and the manner in which things are learned.
- Psychomotor Procedures - physical procedures carried out in every day life, can be complex procedures, initially learned then shaped before becoming automatic, for example driving a car.
PSA sits in the metacognitive system in the domain of mental procedures, it addresses many of the requirements for success. It can be of great benefit, particularly in creative and cultural studies where assessment and criteria can be subjective.
“Peer Assessment can increase learner’s understanding in the cognitive and metacognitive domains, and develop social and transferable skills”
(Topping, 1998, Page 249)
Up until this point this research has mostly grouped peer and self-assessment as a single task but they are of course different. The main aim of this issue is to raise awareness of both in the context in which the proposed PSA strategy is set, using online peer and self assessment software, so in this study they will continue mostly to be referred to together however where required they will be referred to as Peer Assessment (PA) and Self-Assessment (SA). Their use is frequently used within the same context and they benefit from being used simultaneously. PA can inform the individual as to whether a goal has been reached, and if not, what should be done to rectify it. When used in conjunction with PA, SA also becomes a very powerful tool when forming strategies and setting personal goals. Through giving and receiving PA, SA becomes more authentic within the context of a wider field and assessment skills are improved. This leads towards higher order thinking skills and a deeper understanding of what is required to achieve. The learning becomes student centred and a metacognitive process occurs.
Effective Peer & Self-Assessment
Effective PSA can produce good learning outcomes however to be successful it needs to be well prepared and organised with regular and ongoing monitoring. It is a form of collaborative learning and its aims should benefit both the assessor and assessed. It is essential that students should understand the benefits and be aware of any negative aspects before starting. It is then advisable to run some trial assessments that can be more informal in nature.
There are some negative aspects that need addressing, some of which are justified and others that are fears born of a lack of information. It is recommended that a class discussion should take place to dispel any fears; the following points should be discussed.
-
Students’ can hold the view that they want to be assessed by an expert, or the lecturer, rather than by their peers, as evidenced in the case study of the accompanying action enquiry It is difficult to answer this without using theory and case studies. Through PSA presentations, case studies and student trials it can be possible to dispel this as an unsubstantiated concern. See the student presentation webiste .
- Peer Pressure can be an issue as some students express concerns that they may have their work ridiculed or fear having other students aware of their failings or inability to complete tasks. When managed correctly the process should leave no opportunity for this to occur. By downgrading students that provide unsupported negative or offensive comments to their peers it is possible to manage these concerns. Asking students to grade according to pre-negotiated assessment criteria will also help and it must be made clear that criticism must be constructive.
- There is a perception that PSA is time consuming and time would be better spent working on assignments and addressing assessment criteria directly. This can only be addressed through trial projects and evaluating the outcomes. The CASPAR project used in this research demonstrated the students’ familiarity with the assessment criteria. Grades for this assignment, after Peer Assessment, were particularly high with the majority of students achieving distinctions for the criteria they addressed.
- Online issues arise when the assessment takes place where there is no access to the Internet. This can be a problem particularly with performances mostly taking place in performance areas and not IT suites. The performances can be video recorded and the peer assessment can take place at a later date. Is some respects this is good practice as the students see or hear the performance twice before assessing.
- There is a danger that friends might grade each other with high grades or may feel uncomfortable criticising friends work. Using CASPAR can help identify any unusually high grades and when used in anonymous mode it is possible to make comments without leaving a signature. 1
Falchikov and Magin (1997) have suggested that reliable and valid outcomes from peer assessment depend on the following:
1. Students must be committed to and fully understand the educational purpose of peer assessment.
2. Students must be involved in determining the criteria and agreeing on a grading scale and assessment procedure.
3. Students must receive feedback on peer assessment scores, both in relation to their own performance and to the overall pattern of scores.
Although it is suggested that criteria should be determined by the students’ involvement, ultimately the students have to achieve criteria that comes externally from the qualification. This is needs to be finely balanced and needs good guidance from the lecturer to enable the students to interpret the criteria in a way that is relevant to there thinking and language. By developing relevant criteria with the students, in their language, they begin to develop clear ideas about the standards they have to reach to achieve. Students are forever complaining that they do not understand assessment criteria, particularly National Diploma criteria. Through group discussion it is possible to relate criteria to students’ real life experience and reword in more accessible terms. The criteria can be qualitative in its nature and aim to understand meanings and enable the student to be able to describe and understand experience. Students are encouraged to ask ‘why’ and ‘how’ decisions have been made when attempting tasks, this helps them to understand intangible qualities such as beliefs and values.
By assessing their peers’ performance in this way the context and standards of their own work can be fully appreciated. This aims to resemble future professional practice and gives authenticity to tasks.
When PSA is used as formative assessment it helps develop metacognitive intelligence and can transform the learning experience of a student that may otherwise struggle with only cognitive learning processes.
Embedding PSA into the curricula is the biggest challenge for it will only become fully effective when it becomes a familiar process to the students. This means it must be used throughout the students learning and be a continual process.
Research Methodology - Has the Action Enquiry been successful?
This action enquiry has used a number of research methods that have largely been successful in helping to achieve the aims and objectives. Using a combination of qualitative and quantitative methods it has been possible to triangulate results and provide good evidence to support the identification of the issue.
Methods Used
1. Interviews
Audio recorded interviews provided qualitative research and were appropriate to use as there was no need to gauge visual responses to questions and they avoided the need to take notes. There was no visible signs of anyone being uncomfortable with the process. Having conducting a number of interviews it became apparent that there is more to it than just having a set of questions. Although I felt I had prepared well the conversations sometimes floundered and were a little awkward. This improved as familiarity with the subject grew over the period of the research and my interview technique improved after some web research on interview form. Firstly asking closed questions, name and job position for example, and then avoiding leading questions.
2. Surveys
The surveys have served their purpose and taken the research away from close colleagues and provided information from a wider field of lecturers. This was important to do to recognise the issue as certainly being faculty wide. It would make sense to now take the surveys to a college wide audience and not just Creative Industries. The information gained has helped to underpin and validate the interviews.
Survey 1 -The initial survey's goal was to provide additional evidence to underpin findings from informal and recorded interviews. From the limited responses, 13 in total, it was possible to take important information on the use of Peer & Self Assessment. I had hoped to gather a larger picture of actual use within the Academy and in response to this Survey 2 was designed with one question that invited a comment.
Survey 2 -This survey, with one question, was designed for maximum response. From 40 lecturers emailed with the survey link 28 replied and 10 left comments. This proved very successful in validating the identified issue.
Survey 3 - This student survey offered an insight into student perception. A captive audience of 20 students answered the questions.
Surveys 4 and 5 - These two surveys are designed to assess student perception after using CASPAR and have not yet been used.
Group Discussions
Group discussion with the students proved to be very useful in gauging students perception of PSA.
3. Observations
Observations began by being very specific but as the action inquiry progressed they took on a more continuous form. Informal conversations were a big part of the overall research to identify the issues and
4. Case Study
The case study served its purpose in identifying issues and shortfalls of PSA use within one specific area. A wider study would prove beneficial.
Using a mixture of qualitative and quantitative methods, as shown above, allowed theories to be backed up two or three times and helped to provide robust evidence for identifying the issue.
Summary
“The student is regarded as a source of knowledge, and collaboration between teacher and learner is central to the pedagogy. Interactive approaches include collaborative evaluations”
(Burke, 2002, Page 71)
The phrase students are empty vessels waiting to be filled is a well-used paradigm that is now generally accepted as not being true for the majority of diverse learners within education. Constructivists believe that effective learning is based on prior knowledge and experience and for learning to be meaningful an approach that encompasses this belief is essential.
Student-centred learning (SCL) aims to address this philosophy and it can be best explained by identifying the two main strategies for teaching: teacher-centred with emphasis on content delivered through lectures and student-centred whereby emphasis is on the learning process.
Lecturers should now focus on effective learning rather than performance measured by success values and assessment grading. Grading without formative assessment can lack any scope for students to reflect and improve.
Student Centred Learning (SCL) aims to encourage individual responses to stimuli and tasks and accepts that students learn in different ways. Learning is recognised as a dynamic process that is constantly changing and can only be effective when students can find meaning in their learning. This can be achieved through a variety of methods, (discussion, research and lectures) however it is reflection on content, particularly with peers, that takes their meaningful education to new levels.
The lecturer becomes an information manager and facilitates the students’ access and processing of information. This leads to students taking responsibility of their learning and they become active in seeking knowledge.
“from the student perspective, assessment defines the actual curriculum”
(Ramsden, 1992, 187)
Using PSA the student becomes part of the assessment process. The student moves away from being the assessed to being the assessor and is actively involved in creating assessment criteria for tasks and assignments. This ownership of the assessment criteria can often help the student answer the question why am I learning this? From this they can then ask how is it best to learn from this? Through PSA and formative assessment metacognitive intelligence is realised.
The barriers between the lecturer and the learner are removed and the students’ knowledge is validated creating an environment that encourages learning and higher order thinking skills.
Bibliography
Books
Bloom, B. S., Taxonomy of Educational Objectives, the classification of educational goals – Book I: Cognitive Domain. New York: McKay, 1956.
Burke, P. J., Accessing Education. London: Trentham Books Ltd, 2002.
Dewey, J., Democracy and Education. London: The Echo Library, 2007.
Evertson, C., Handbook of Classroom Management. London: Routledge, 2006.
Falchikov, Nancy. Learning Together: Peer Tutoring in Higher Education. London: Routledge, 2001.
Marzano, J & Kendall, J., The New Taxonomy of Educational Objectives. London: Cage Publications, 2007.
Ramsden, P., Learning to Teach in Higher Education. London: Routledge, 1992.
Rowling, J. K., Harry Potter and the Goblet of Fire. London: Bloomsbury Publishing, new addition 2001.
Winn, W. & Snyder D., Cognitive perspectives in psychology in Jonassen, D. H. Handbook of research for educational communications and technology, New York: Simon & Schuster Macmillan, 1996.
Websites and Web Pages
Moon, J., Guidance notes on Reflective Skills, n.d, accessed 8 April 2009
National Advisory Committee on Creative and Cultural Education. All Our Futures 1999, accessed 25 March 2009
National Foundation for Educational Research on behalf of the QCA, The Arts, Creativity and Cultural Education: An International Perspective 2000, accessed 25 March 2009
Journals
Ridley, D.S. Schutz, P.A., Glanz, R.S. & Weinstein,, C.E. Self-regulated learning: the interactive influence of metacognitive awareness and goal-setting. Journal of Experimental Education 60 (4), 1992.
Kolb’s Learing Cycle - Experience is followed by Reflection on the experience, and then followed by Application of known theories to the experience leading to Change or modification of the experience cycling around to the next experience. This could happen in an instance or over a longer period of time.
