Making Marks: assessment practices in art and design
Professor Susan Orr, Deputy Dean York St John University
Assessment, art and design, practice, connoisseurship
This article offers a brief overview of key
points emerging from my research into art and design lecturers’ assessment
practices. I have worked in this area for some years and have carried out a
range of interview-based and observation-based studies across more than eight
universities. In this article I
discuss the ways lecturers assess identities, artistic practices and artwork
holistically. My key point is that art and design assessment is best understood
as an artful practice - indeed it might be likened to a form of
I have researched assessment practices in art and design for some years. Most of my work has focused on lecturers’ experience of assessing student artwork in the studio (rather than exploring students’ experience of being assessed). Generally I have researched this area using interview-based methodologies but I have also employed studio-based observational approaches (Orr, 2005, Orr, 2007). I have observed or interviewed art and design lecturers in eight universities. In this short article I am going to draw on the breath of this research in an attempt to draw out the main characteristics of art and design assessment practice.
Group marking is central to assessment in art and design
We flood our assessment process with staff.
Art and design lecturers often work together to mark student artwork in the studio. The fact that group approaches to marking have remained a central tenet of art and design assessment in the face of massification and the intensification of lecturers’ work-loads underlines its importance. In some of the universities where I have researched assessment, artwork had been assessed by anything up to seven lecturers. This is a distinctive element of art and design assessment that contrasts sharply with assessment practices in text-based subjects where often only a small sample of texts will be double-marked or moderated (Price, 2005). It is my view that in text-based subjects double-marking and moderation are viewed as serving primarily quality assurance and regulatory purposes rather than being an approach that, in part, defines pedagogy. In contrast, for art and design lecturers, moderation conversations are a key site for judgement making. In the words of one lecturer I interviewed ‘assessment happens in that dialogue’.
Lecturers assess the students’
artistic practices and their artwork
But I think you can’t….distinguishing the work from the person…is….is quite difficult within our area of practice.
Lecturers in my studies shared the view that student identities, their artistic practices and their artworks are enmeshed. Art and design assessment practices are premised on this assumption. In the words of one lecturer ‘the work should carry the maker’. Art and design lecturers value their engagement with students’ artistic development over time. The following extract illustrates the ways that narratives about student identity coalesce into the narratives about the student artwork:
You see the students evolve the work and you really get into the, em, the concepts and the thinking behind it and the theories that they develop and the fact that you are also guiding and talking with them and, you know, other staff are doing the same, I think you can build up a closer understanding of that student's work so you also see the faults a lot [...] you know, there's a quality comes through the talking and the thinking and the.., the actual quality of the way they've produced the work, em, but yeh I mean you can.., it's not difficult to tell good pieces of work.
Lecturers’ dual interest in the student and
their practice leads to the next key element in art and design assessment.
Knowing the student is central to assessment
[When assessing art work with other lecturers] we would certainly want to take account of the views of the person who might know the student best…em…you know, who might be able to in one way or another, direct us to something we may have been missing.
When artwork is being assessed in the studio the lecturers in my studies privileged the assessment views of lecturers who had worked most closely with the students whose artwork was being marked. What this means is that if there was any kind of disagreement about the mark to be awarded the marking team would defer to the lecturer who knew the student best and had worked most closely with them. I noted that it was only occasionally that outsiders (who had not taught the students) were brought into the assessment process to offer a particular response to the artwork, divorced from its maker, but this was not an integral part of assessment. Arguably, this approach could be viewed as a resistance to anonymous marking and the commodification of assessment. This is significant given that anonymous marking is becoming a commonplace requirement in other disciplines in higher education. Anonymous marking represents an attempt to disentangle the student from the work. In contrast, for art and design lecturers, the work and the student are entangled. The assessment approaches adopted reflect their interest in the individual students and their particular learning trajectories (Orr, 2010).
Marking time: Assessing process and product
It’s hard because you're looking at this final project but you're going ‘Wow look where they came from!’
In art and design education there is continuing discussion concerning the emphasis that should be placed on the ‘process of development of the idea (making a work of art, design, etc) or in the end product (the work of art or design itself)’ (Cowdray and de Graaf, 2005, p.507). A focus on process is a ‘“truth” of the discipline’ in art and design that reflects the lecturers’ interest in the students’ developmental learning journeys (Barrow, 2006, p.365). The lecturers are interested in assessing the students’ artistic engagement. The concept of engagement ‘does not lend itself to oral explanation’ (Percy, 2004, p.146) but dictionary definitions stress engagement as being related to issues of commitment and a pledge to participate. Lecturers speak highly of students who engage fully in their emergent arts practice and for many lecturers engagement is viewed as a prerequisite for achievement. As one lecturer pointed out, students are expected to engage with ‘their own practice, with their peers and with their tutors and the wider art context’. Lecturers seek to reward the students who are ‘totally immersed [in their arts practice]’. Engagement is connected to issues of commitment and having a presence in the studio and it can only be demonstrated over time. Assessment approaches in art and design are designed to capture this engagement via reflective journals, documentation and the Crit.
Student and lecturer identities are fully implicated in art and design assessment practices
So it’s actually knowing the student over the course of the year [...]. So it’s kind of the notion that at the back of your mind you would have a sense of how the student had progressed, how they increasingly challenge themselves and address the challenge and through tutorial engagement. You’d log that, either literally or in your head’
Barrow argues that in design the ‘assessment regime is a technology of the self’, thus issues of identity are critical to understanding art and design assessment. Rowntree (1987) comments that lecturers never come to the act of assessment without preconceived ideas about how it is done, and one powerful source of knowledge is the lecturer’s own experience of being assessed. In my studies I noted that lecturers’ experiences of being art students informed their approaches to marking. Lecturers reproduce elements of the assessment practices they experienced as students. In addition their identities as creative practitioners relate to how they assess student work. Lecturers’ identities offer complex lenses through which student artwork is apprehended. In the quotation below the lecturer observes that issues of identity affects ‘how we see’.
I think there can be subtle differences in the way in which we approach things depending on our backgrounds, depending on how we see the ways in which the students have realised their ideas and presented them (my emphasis)’.
When lecturers position around a piece of student artwork in the studio, they relate to the work, the student and the team, drawing on a range of identity positions. In the list below I offer just a few examples that directly link to the ways that lecturers approached their assessment role:
- Lecturer as artist
- Lecturer as programme team member /leader
- Lecturer as part-timer
- Lecturer as sculptor /3D designer / painter
- Lecturer as positioned at end of career
- Lecturer as part of a particular university
- Lecturer as part of, or at odds with, university or departmental context
What the list illustrates is the ‘nexus of multi-memberships’ that span the local and the cultural (Wenger, 2004, p.159). To illustrate the ways that identity positions relate to assessment I offer the example of two lecturers who explore the ways that their identities as artists within very particular areas of artistic practice impact on their marking:
It’s quite rare to see somebody specialising in that area [his own area of practice] but when I see it, it’s quite nice, but actually it’s not necessarily an advantage to the students. I think from experience I’ve probably been quite tough [laughter].
One of the really interesting things that we’ve been discussing quite a lot recently is when.., when a student is working in an area that’s related to your own practice you can sometimes be harder….on that work because you recognise the weaknesses more clearly when.., when a student is working on an area that’s kind of alien to your own practice it’s easier to be…… impressed, exactly ‘cause you have that kind of lack of depth of understanding.
Both of these lecturers appear to be ‘harder’ markers with students who pursue their area of practice. Lecturers position student artwork in relation to their own practice and in relation to other artistic practices. In the extended extract below, one lecturer reveals the ways in which her sense of aesthetics informs how she classifies students’ work and how, in doing so, she classifies herself. This extract illustrates how her ‘values […] influence every step of the assessment process’ (Cresswell, 1996, p.57). As Bourdieu (1986, p.6) points out, ‘taste classifies, and it classifies the classifier’:
I definitely think that certain people have
pet hates as well, certain types of practices […] that they struggle with. […] Em
in terms of my, my own sort of pet hates I, em, struggle with stuff that looks
like modernist painting. I [..] I struggle with, em, that, em, oh I don’t know,
I think when I was at university I used to call it blobby splatty painting, I’m
not talking about abstract art broadly. I’m talking about a certain type of,
em, abstract painting, em, and, em, but I also have colleagues who I know, em, struggle
with, em, issues around performance-based work, em, and, em, and find that sort
of difficult in terms of what fine art is, what the parameters are, what the
boundaries are. So I think my issues come from within what I think fine art
practice is and what I think good and bad practice is, em, and what I think a
contemporary context is and what I think is relevant now.
Art and design lecturers seek to assess student artwork in relation to student intention
It is essential that you know something about who that person is and what they are trying to do, what they…what they think they’re doing in order to….to measure the quality of what they’ve done (my emphasis).
A specific manifestation of lecturers’ interest in process relates to the ways that lecturers talk about student intention in relation to assessment. Cannatella (2001) sees an interest in student intention as a ‘common factor in any art assessment process’ (p.320). In the extract above the lecturer explains that the student should clearly set out what they intend to achieve and then they should be marked in relation to that intention. Later in the same interview this lecturer acknowledges assessing in relation to intention is not a sanctioned view within his university context. This is contested territory that is explored more fully in Orr (2007) and Orr (2010).
Pre-determined learning outcomes and assessment criteria are sometimes viewed as problematic in art and design
Learning outcomes are useful, but the assessment process is looking holistically.
Yorke, Bridges and Woolf (2000, p.26) observe that the role of ‘professional judgement is particularly important in art and design’ because it is less ‘amenable to precise specification in advance’. Gordon (2004) builds on this view by reflecting upon the paradox of having to specify attainment in learning outcomes in advance whilst simultaneously allowing creative students to offer unanticipated creative solutions. She refers to this as the ‘wow’ factor (p.61) and argues that this is a necessary, but elusive, assessment criterion. Gordon suggests that the ‘wow’ factor includes ‘creativity, originality, inventiveness, inspiration, ingenuity, freshness and vision’ (p.61). In art and design, learning outcomes, to a certain extent, anchor lecturers’ assessment practice but their potential to make assessment overly prescriptive means that their role is contested.
Art and design assessment is a
form of connoisseurship
The subjective thing is being made by highly trained, educated kind of specialists in that subject, so there is a subjective decision being made, but by specialists.
My key point is that art and design
assessment practices collapse the binaries of process and product; artwork and
student identity. As a
consequence assessment in art and design might be best understood as an artful
social practice. Building on the work of Bourdieu (1977, 1986) and Shay (2003,
2005), my research conceptualises art and design assessment as a form of
connoisseurship. Within my particular conception of connoisseurship, lecturers’
assessment expertise is co-constituted and practiced within communities of
practice through participation and engagement (Orr, forthcoming).
Deputy Dean: Faculty of Arts
York St John University
Lord Mayor’s Walk
Professor Susan Orr is deputy dean at York St John University; prior to this she was the Teaching and Learning Coordinator at the London College of Fashion, University of the Arts London. In 2009 she was awarded a Chair in Pedagogy in Creative Practice. Susan’s research explores a range of creative practice pedagogies. In 2008 she worked with Dr Margo Blythman and Dr Bernadette Blair on an ADM-HEA funded project that explored the role of the Crit in Art and Design. Susan is on the editorial boards of the Journal of Writing in Creative Practice and Art, Design and Communication in Higher Education.
Barrow, M. (2006). Assessment and student transformation: linking character and intellect. Studies in Higher Education, 31(3), pp.357-372.
Bourdieu, P. (1977). Outline of a Theory of Practice. Cambridge, Cambridge University Press.
Bourdieu, P. (1986). Distinction: A Social Critique of the Judgement of Taste. London, Routledge.
Cannatella, H. (2001). Art Assessment. Assessment and Evaluation in Higher Education, 26(4), pp.319-326
Cowdroy, R., & deGraff, E. (2005). Assessing highly-creative ability. Assessment and Evaluation in Higher Education, 30(5), pp.507-518.
Cresswell, M. (1996). Defining, setting and maintaining standards in curriculum-embedded examinations: judgmental and statistical approaches. In H. Goldstein & T. Lewis (Eds.), Assessment: problems, developments and statistical issues (pp. 57-84). Chichester, Wiley.
Gordon, J. (2004). The 'wow' factors: the assessment of practical media and creative arts subjects. Art, Design and Communication in Higher Education, 3(1), pp.61-72.
Orr, S. (2005). ‘Justify 66 to me!’ An investigation into the social practice of agreeing marks in an HE art and design department. In C. Rust (Ed.), Improving Student Learning through Assessment: Proceedings of the 13th International Symposium (pp. 283-294). London, Oxford Brookes University.
Orr, S. (2007). Assessment Moderation: Constructing the Marks and Constructing the Students. Assessment and Evaluation in Higher Education. 32(6), pp.1-12.
Orr, S. (2010) The Role of Professional Judgement in Assessment: A Journey from Accuracy to Zing. Inaugural Professorial Lecture. York St John University. January 2010
Orr, S. (forthcoming) ‘We kind of try to merge our own experience with the objectivity of the criteria’: The role of connoisseurship and tacit practice in undergraduate fine art assessment, Art, Design and Communication in Higher Education.
Percy, C. (2004). Critical absence versus critical engagement. problematics of the crit in design learning and teaching. Art, Design and Communication in Higher Education, 2(3), pp.143-154.
Price, M. (2005). Assessment standards: the role of communities of practice and the scholarship of assessment. Assessment and Evaluation in Higher Education, 30(3), pp.215-230.
Rowntree, D. (1987). Assessing Students: How shall we know them? London, Kogan Page Limited.
Shay, S. (2003). The assessment of final year projects: a study in academic professional judgment. PhD Dissertation. Unpublished manuscript, University of Cape Town. South Africa.
Shay, S. (2005). The assessment of complex tasks: a double reading. Studies in Higher Education, 30(6), pp.663-679.
Wenger, E. (2004). Communities of Practice: Learning, Meaning and Identity. Cambridge, Cambridge University Press.
Yorke, M., Bridges, P., & Woolf, H. (2000). Mark distributions and marking practices in UK higher education. Active Learning in Higher Education, 1(1), pp.7-27.
- copy of email bulletin sent to members of ADM-HEA JISCmail list
- copy of email bulletin sent to members of ADM-HEA JISCmail list
- copy of email bulletin sent to members of ADM-HEA JISCmail list