A while ago I wrote an article based on my work in partially-automated assessment. The accepted manuscript I stored in my university’s repository has just lifted its embargo, meaning you can read what I wrote even if you don’t have access to the published version.
Thinking about assessment, it seems there are methods that are very good at determining a mark that is based on a student’s own work and not particularly dependent on who does the marking (call this ‘reliability’), like invigilated examinations and, to some extent, online tests/e-assessment (via randomised questions that are different for each student). These methods tend to assess short questions based on techniques with correct answers and perhaps therefore are more focused on what might be called procedural elements.
Then there are methods that are probably better at assessing conceptual depth and broader aspects that we might value in a professional mathematician, via setting complex and open-ended tasks with diverse submission formats (call this authenticity and relevance ‘validity’). People are often concerned about coursework because it is harder to establish whether the student really did the work they are submitting (not an unreasonable concern), which impacts reliability.
It is hard to ask students to complete high-validity coursework tasks (that might take weeks to complete) in exam conditions, and diverse submission formats do not suit automated marking, so two ways to improve reliability are not available. The idea with partially-automated assessment is that an e-assessment system can be used to set a coursework assignment with randomised elements which is then marked by hand, gaining the advantageous increase in reliability via individualised questions without triggering the disadvantage of having to ask for submission in a format a computer can mark. The payoff is that the marking is a bit more complex for the human who has to mark it, because each student is answering slightly different questions.
In the article I write about this method of assessment, use it in practice, and evaluate its use. It seems to go well, and I think partially-automated assessment is something useful to consider if you are assessing undergraduate mathematics.
Read the article: Partially-automated individualized assessment of higher education mathematics by Peter Rowlett. International Journal of Mathematical Education in Science and Technology, https://doi.org/10.1080/0020739X.2020.1822554 (published version; open access accepted version).
