Assessment in the Humanities

Assessment in the fields of the humanities and humanistic social sciences has been a durably challenging enterprise, regardless of whether the target of assessment is student learning, teacher or faculty performance, or the experience of public humanities audiences. Over the course of the last four years, our research team on Humanities and Liberal Arts Assessment (HULA) has developed a novel methodology for tackling this task. Our purposes in revisiting the question of whether it is possible to design assessment instruments that are appropriate to work in the humanities have been twofold. First, we have sought to equip humanistic practitioners—whether on campus or in public humanities settings—with tools to translate the impact of their work into the vocabulary of policy-makers and funders. Second, we have sought to equip these practitioners with tools that can provide meaningful insights into their work and support continuous improvement.

Scholars, teachers, and directors of humanities public programming are often told, explicitly or implicitly, that they can’t account for the impact or value of what they do. The suggestion is not merely that they haven’t found reasonable tools of assessment, but also that what they do may not be of much use at all. As someone who has labored in humanistic contexts and on behalf of the humanities for more than two decades, I know that neither of these suggestions is accurate. Humanists do have tools of assessment—for instance, the critical feedback written on student papers and the assignment of grades. And the work of humanists is of fundamental value to human development.

The challenge, I realized, is that those who practice a humanistic discipline have learned the tricks of our trade as members of craft guilds. We have apprenticed to master teachers and researchers and learned largely by modelling ourselves on them. Consequently, the standards and criteria for excellence around which our crafts are organized are often implicit.

As with any other craft, the multiple crafts of the various humanities disciplines all have “logics.” They pursue goals, deploy a sophisticated and relatively stable set of tools to do so, and even have causal accounts of why the relevant set of tools can be expected to achieve the sought after goals. In other words, in the language of social science, humanistic craft consists not only of goals and methods but also of identifiable mechanisms, cause-effect relationships that explain why the methods work. In the case of the humanities and the humanistic social sciences, because we depend on a model of master-apprentice relationships, those craft logics have been transferred from one generation to another without being explicitly formulated. This stands in contrast to modern laboratory science, in which big team collaborations force the principal investigators to make the craft of the lab explicit, at least to some degree, for every new member of the team.

Once one recognizes that the humanistic disciplines are a set of crafts the topic of assessment opens itself up. One has only to identify the craft logics structuring the practice of particular humanist scholars and teachers in order to develop assessment instruments that emerge organically from the content of the craft itself.

Importantly, one major element of any given humanistic craft practice is a “learning theory,” a rough and ready understanding of how certain kinds of cognitive, affective, and intersubjective engagements with texts, or art works, or music, or discussion and argument, lead to both short-term and longer-term developmental outcomes for participants. The striking thing about the craft practices of the humanities is the sheer diversity and variety of learning theories that can operate in this space.

Consider the following as a schema for thinking about the potential pathways to human development that pertain to the humanities. The process of human development begins with inputs that we take in through our senses or conjure up through our memory. In the humanities, those inputs may be verbal, visual, aural, kinesthetic, or behavorial. Everything, in other words, other than the quantitative (although number, too, is a form of language). Having taken in some sort of stimuli, we then process those stimuli with any or all of the following processing capacities: cognitive capacities, whether analytical or imaginative; metacognitive capacities (this is the capacity to think about how we think); affective or emotional capacities; intersubjective capacities (these are the ones that hone our relational or interpersonal skills); or the expressive component of our kinesthetic capacities (perhaps I go for a run to help me straighten out my thinking on a given subject; the rhythm of the running helps me bring order to my thoughts).

This processing work generates results. It generates some short-term results, namely, the development of skills or second-order capacities. It also generates long-term results by shaping the longer developmental arc of a life. A reasonably complete list of the short-term results might include: basic literacy; advanced literacy; critical thinking; understanding; appreciation; creativity; practical judgment; communicative skill; kinesthetic competence; and various personality factors, like resilience, grit, and self-confidence. As to the long-term results, we could divide those among the existential, the vocational, and the civic. Take the existential as identifying the long-term development of our experience as an individual, our sense of identity and subjectivity, and our experience of intimate relationships. Take the vocational as identifying the long-term development of our competence to fend for ourselves economically. And take the civic as identifying the arc of our development as people who participate in organizational and political communities—local, national, or global.

Here’s a chart of that schema.

Now a learning pathway is a process in which teachers and students focus on a subset of inputs, some particular modes of processing those inputs, and some short-term and long-term goals for that processing work. For instance, a teacher in a political philosophy class might engage students in a lot of close reading (a verbal input). She might emphasize the analytical work done by the students in the class by really asking them to focus on logical argumentation (a choice to emphasize the analytical aspect of cognition for the processing). She might do this seeking to cultivate “understanding” in the students of key political concepts and how they relate to living political practice, her short-term goal. She expects that equipping them with this sort of understanding will make them better civic agents. The long-term result she seeks is, in the first instance, civic, even if it also has some existential flavors. We could map her learning pathway this way:

On this schema, 1,080 distinct learning pathways are possible, if, that is, instructors focus on only one input, only one processing capacity, only one short-term result, and only one long-term result. For a partial representation of the explosion of possibilities, consider this:

But, of course, as we all know, no instructor focuses her efforts in such a thoroughly streamlined fashion and this is because students have different needs. One student in that political philosophy class may need a lot of work on the quality of her logical argumentation. Another, however, may be quite good at that but not very good at the sorts of intersubjective skills that facilitate discussions with other students. The good instructor will find herself switching learning pathways, accordingly as students’ needs become apparent to her. The number of combinatorial pathways, in other words, is vast. Here is where the efficiency of the humanities comes in. Teaching in the humanities can develop students in all of the ways sketched above, but it can also do many of these things at the same time.

Now how does all of this help with assessment? Consider simply the first pathway indicated above, where verbal inputs are engaged cognitively in the expectation that they will generate forms of understanding that support civic development. Happily, social and cognitive psychology offer up a variety of constructs that can be used in a developmental pathway that rests on this sort of causal chain.

In order to create a psychologically informed map of this learning theory, the HULA team broke it down into a causal sequence. First, it seems important to capture the extent to which a program engages participants in analysis, critical thinking, or reflection. Such active engagement may in turn lead to more general and stable cognitive changes. The cognitive changes may then influence more specific beliefs and attitudes related to the broader civic goals. For example, changes in tolerance for ambiguity or open-mindedness may lead to more complex perceptions of group identity (e.g., integrative rather than assimilative) and, consequently, a greater sense of community. Other mediators of the cognitive effects of programs on civic goals might include changes in participants’ confidence and willingness to become informed about current events or opportunities for civic action. This leads to the following model, in effect a hypothesis that could support a craft practitioner’s conviction that engaging students in analytical cognitive work fosters their growth as civic agents:

Experiences of Insight → Lower Need for Cognitive Closure (higher Need for Cognition) → More Complex Perceptions of Group Identity → Greater Sense of Community

Standard quantitative measures of these psychological constructs can then be deployed to design short, meaningful survey instruments to assess whether the actual classroom or event experience is indeed achieving the anticipated outcome and, if so, whether the achievement of the most aspirational outcome (the greater sense of community) is indeed linked to the medial steps.

This has been a quick summary of the strategy for assessment in the humanities and humanistic social sciences developed by the HULA team. Our methodological white paper and other materials may be accessed here: http://www.pz.harvard.edu/projects/humanities-liberal-arts-assessment-hula.