I write to share and invite discussion of something I presented at a conference at Nottingham Trent University last week.
I have been thinking a lot about assessment methods and their advantages and limitations for a chapter I am writing for my PhD thesis. For example, I could set a paper test and mark it by hand, as indeed I set one last week and will be marking it when I finish this post, and this allows me to give a personal touch and assess students’ written work but one downside is that I can’t return marks to students very quickly. I could return marks immediately if I used automated assessment, but then setting the assessment would be more difficult and I may be limited in the range of what I could assess. And so on.
I have been trying to classify these advantages and their paired limitations. My thinking is that by viewing different assessment methods as balanced sets of advantages and limitations we can justify different approaches in different circumstances and, particularly for my PhD, explore the advantage/limitation space for any untapped opportunities, which I won’t go into now (but ask me).
Here is my current list of potential advantages that assessment could access. These advantages are each something that I think that some assessment method can offer. My question is: what am I missing? I would be pleased to receive your thoughts on this in the comments.
- Immediate feedback. This is linked to learning from mistakes, confidence and motivation. It can also prioritise procedural learning over conceptual understanding.
- Detailed, personalised feedback. Though there is much disagreement in what I have read whether a human, who can respond to individual student work, or a computer, which will tirelessly generate worked examples using the context of the question asked, will in practice provide this.
- Individualised assessment. This is achieved through randomisation of questions and is linked to repeated practice, deterring plagiarism, allowing students to discuss the method of a piece of work without the risk of copying or collusion.
- Assessing across the whole syllabus. For example, computers can’t mark every topic.
- Testing application of technique. Whether students can apply some procedure.
- Assessing deep or conceptual learning. For example, open-ended or project work may require a detailed manual review to mark. This is linked to graduate skills development, etc.
- Easy to write new questions. Assume it is easy for a lecturer to write questions that students can answer (it isn’t, but we’re talking principle here). Difficulty is introduced by having to second guess an automated system, or having to second guess students to program misconceptions.
- Quick to set assessments. Assume that writing a test manually takes time. By quickly, I really mean choosing items from a question bank.
- Quick to mark assessments. Assume that marking by hand is not quick, perhaps unless the assessment is very short and student answer format very prescribed, in which case the assessment is limited. This is perhaps linked to problems of consistency and fairness when using multiple markers.
- Easy to monitor students. Clearly marking individual work from every student by hand will give great insight, but here I refer to the ability to gain a snapshot of how individuals and the cohort are doing as a whole with a concept, perhaps very soon after a lecture that introduced that concept has taken place.
- Perception of anonymity. I’ve read that some students are happier to make their mistakes if only a computer knows. This can reduce stress.
- Testing mathematical writing. Clearly requires hand-written work.
- Testing computer skills. Clearly requires use of a computer.
For example, then it might be possible to offer ‘Easy to write new questions’, ‘Assessing deep or conceptual learning’ and ‘Testing mathematical writing’ through a traditional paper-based, hand-marked assessment, but this would preclude, for example, ‘Immediate feedback’.
Similarly, a multiple-choice question bank might offer ‘Quick to set assessments’ and ‘Quick to mark assessments’ at the expense of ‘Assessing across the whole syllabus’ and ‘Assessing deep or conceptual learning’.
And so on. I have loads of these for different assessment types.
My question really is, is there anything missing from my list that might be delivered by an assessment method?
The first MathsJam of the year was well attended. Despite not being on our usual table (there was no jazz band on this week, so we were allowed a bigger table further into the pub) everyone found us ok, and a few people brought baked goods – always a precursor to an excellent MathsJam.