Chapter 11: Assessing Scientific Reasoning in a Liberal Learning Curriculum

Diane Ebert-May, Elena Bray Speth, and Jennifer L. Momsen

A well-rounded college education in the United States is synonymous with liberal education and includes science as one of the liberal arts (American Association for the Advancement of Science, 1990). The experience of learning science as a liberal art should contribute to students’ preparation as informed and responsible citizens who think independently, reason analytically, and communicate effectively. Science education is not about learning a litany of facts; rather, it is about developing the thinking skills to engage in twenty-first-century science. To help students achieve this, scientists should teach science as it is practiced and, in particular, teach scientifically by fostering scientific reasoning and quantitative literacy skills in all students (Bybee, 1997; Handelsman et al., 2004; National Leadership Council for Liberal Education and America’s Promise, 2007). All college graduates, regardless of their major, should know and be able to engage in the methods of scientific inquiry that lead to scientific knowledge. This capacity includes accessing and analyzing scientific information, using scientific evidence, and constructing reasoned arguments.

As the national interest in liberal learning in undergraduate education has accelerated, goals and outcomes, including the ability to use science as a way of knowing, have appeared on the institutional websites of nearly four in five institutions of higher education (Association of American Colleges and Universities, 2009). Although consensus about the need for scientific reasoning is widespread, few institutions have clearly outlined the curricular pathways and processes that should lead students to achieve scientific reasoning. Generally, we do not have evidence on whether faculty consider the liberal learning goals of their institution when designing their courses, or whether and how they translate these goals into instruction and assessment.

Designing specific course objectives and assessments that reflect a broad liberal learning goal (for example, scientific reasoning) and then implementing instruction that enables students to attain such a goal are challenging tasks for faculty. Faculty need to strive to guide students to reason scientifically through effective instructional design, teaching, and assessment strategies. Of course, faculty need to do so in the context of their course contents and objectives. Is this such a daunting task? We do not believe so. As we look into classrooms across campuses, however, we find that what faculty and students are actually doing to achieve scientific reasoning is not evident. Indeed, traditional lectures and classroom assessments—the metric with which we gauge student learning —fail to align with scientific reasoning goals. A passive, teacher-centered classroom does little to engage students in science as a way of knowing (Moore, 1993). An active, learner-centered classroom that models scientific teaching has a higher probability of helping students achieve scientific reasoning (Bransford & National Research Council Committee on Developments in the Science of Learning, 2000; Ebert-May & Hodder, 2008).

Currently, administrators and faculty committees at institutions of higher education are primarily occupied with articulating liberal learning outcomes for all students and pay less attention—except, perhaps, when preparing for accreditation by an external agency—to systematically assessing whether graduates achieve these goals. Furthermore, few, if any, efforts are made to critically evaluate actual classroom instruction intended to help students achieve the liberal learning goals. In this essay, we focus on the gap we perceive between such goals and assessments and what actually occurs in the classroom. We want to draw attention to the necessary alignment between what universities expect students to know and be able to do and how teachers teach. For instance, do science faculty actively promote students’ awareness, understanding, and achievement of scientific reasoning? If so, how? Passive lectures requiring minimal student engagement with the nature and process of science still reign in most large-enrollment introductory science courses. We cannot expect scientific reasoning to occur via diffusion.

We illustrate how a team of faculty incorporated scientific reasoning, a broad liberal learning goal of a large research university, into an introductory biology course and, within that course, a module on evolution (Figure 1). We used a process called backward instructional design to determine specific learning goals and objectives, develop assessments to measure students’ achievement, and implement instruction that aligns with the objectives and assessments (Wiggins & McTighe, 2005). Key to this process was the need for learner-centered instruction.

Figure 1: Hierarchy of Learning Goals at Michigan State University

Science courses at MSU are influenced by a hierarchy of learning goals designed to actively engage students in learning both in and out of the classroom. Figure created and provided by the authors.

TRANSLATING INSTITUTIONAL LIBERAL LEARNING GOALS INTO LEARNER-CENTERED SCIENCE COURSES

Recently, a campus-wide committee at Michigan State University (MSU) revised the liberal learning goals and outcomes that are intended to provide a framework for students’ active engagement in learning both in and out of the classroom. The liberal learning goals are expressed in terms of knowledge, attitudes, and skills that all students who complete an undergraduate degree program at MSU should achieve. The outcomes include analytical thinking, cultural understanding, effective citizenship, effective communication, and integrated reasoning (Michigan State University, 2009). These liberal learning outcomes are intended to apply to any and potentially all subjects and courses.

Concurrently, another initiative led by a team of science, statistics, and mathematics faculty produced a set of scientific reasoning and quantitative literacy (SRQL) goals (see Table 1; Michigan State University Committee on Liberal Learning, 2007). Faculty from six colleges on campus ranked the SRQL goals based on what they thought were important learning outcomes for all students at MSU. This process included the development of a psychometrically valid and reliable multiple-choice instrument to assess students’ SRQL skills at three points along their degree program: (a) during freshman orientation prior to the start of the academic year (baseline data); (b) midway through their degree; and (c) near the end of their degree program.

These two initiatives provide science faculty at MSU with two sets of liberal learning goals and outcomes—one set at the institutional level for all liberal arts and sciences and one set focusing on scientific reasoning and quantitative literacy. Both sets of goals are intended to make visible the dynamic relationships between the disciplinary goals defined by specific programs and courses and the knowledge, attitudes, and skills that characterize a liberal arts education.

Table 1: Analytical Thinking and SRQL Goals at Michigan State University

Analytical Thinking  (University Liberal Learning Goals and Outcomes)

Scientific Reasoning and  Quantitative Literacy (SRQL) Goals

The MSU graduate uses ways of knowing from mathematics,natural sciences, social sciences,  humanities, and arts to access information and critically analyzes complex material in order to evaluate evidence, construct reasoned arguments, and communicate inferences and conclusions: Aquires, analyzes, and evaluates information from multiple sources. Synthesizes and applies the information within and across disciplines. Identifies and applies, as appropriate, quantitative methods for defining and responding to problems. Identifies the credibility use and misuse of scientific, humanistic, and artistic methods.

Describes the methods of inquiry that lead to scientific knowledge and be able to distinguish science from pseudoscience and non-science. Make inferences and predictions and explain and justify conclusions based on data and other quantitative information. This includes discriminating between association and causation and identifying the types of evidences used to establish causation. Evaluate the credibility, use, and misuse of scientific and mathematical information in scientific developments and public policy issues. Use multiple representations to model real-world phenomena and, further, use model and theories as unifying principles that help us understand natural phenomena and make predictions. Use graphical, symbolic, and numerical methods to organize, analyze and interpret data and to effectively communicate findings. This includes: creating, reading, and interpreting representations of quantitative and scientific information such as tables, charts and graphs; and solving problems that require that application of geometric and algebraic properties, probability statistics, computer software or techniques, or critical thinking skills. Formulate hypotheses, identify relevant variables, and design and critique experiments, observational studies, and surveys to test hypotheses using appropriate research design relative to the research objectives. Recognize the interdependence and value of basic research, applied research, and technology development and how they interact with society. Illustrate the interdependence among social and ethical issues and scientific development.

For a complete list of all MSU Liberal Learning Goals and Outcomes, see Michigan State University. 2009. Liberal learning goals and outcomes. http://undergrad.msu.edu/outcomes.html.

The introductory biology course we developed targets students with a wide range of backgrounds and interests. Our course goals (Table 2) overlap almost entirely with the Analytical Thinking portion of the MSU Liberal Learning Outcomes. Within analytical thinking, the SRQL details explicit scientific reasoning and quantitative literacy goals. Specifically, students will make inferences and predictions and explain and justify conclusions based on data and other quantitative information. In addition, students will use graphical, symbolic, and numerical methods to organize, analyze, and interpret data and to effectively communicate the findings (Table 1). Our course goals specifically include SRQL Goals 2 through 6.

Table 2: Introductory Biology Course Goals

What you should be able to do with your knowledge of biology
Organize it . . . by identifying the complex relationships among biological concepts; by creating conceptual frameworks that you can use, expand, and modify with new information.	Communicate it . . . by constructing models that show your understanding of concepts and relations among them; by articulating scientific explanations targeted to different audiences.	Use it . . . to interpret and evaluate scientific claims in the popular media; to appropriately justify your own scientific claims; to inform your decisions as citizens.

The biological concepts in our course are integrated by the theory of evolution. Students learn and apply the principles of evolution in the context of several different case studies. In class activities prior to the case studies, students work with the key concepts of evolution: (1) phenotypic variation in populations; (2) the molecular origin of variation; (3) how variation is inherited;

(4) fitness of individuals within a population; and (5) populations evolve, not individuals. Following these activities, students apply their knowledge to a case study on antibiotic resistance. We developed several instructional modules, including the following on antibiotic resistance, to help students achieve several SRQL goals and discipline-based objectives for evolution. This module illustrates how backward design and learner-centered instruction were accomplished in a single class meeting within the course.

CASE STUDY ON THE EVOLUTION OF ANTIBIOTIC RESISTANCE IN BACTERIA

The evolution of antibiotic resistance in bacteria is an engaging topic for many students. Bacteria are common model organisms for evolution, in part because of their rapid rate of mutation and short generation time. Evidence of bacterial evolution is abundant and approachable for first-year students.

The objectives for this class meeting were:

1. Use data (evidence) to make claims about variation, fitness, selection, and evolution in populations.
2. Apply the general definition of natural selection and the concept of trade-offs to the evolution of antibiotic resistance in bacteria.

These objectives, along with classroom instruction and the assessments used to measure students’ achievement, align with three SRQL goals (Table 3):

Goal 2: Make inferences and predictions and explain and justify conclusions based on data and other quantitative information.

Goal 4: Use multiple representations to model real-world phenomena.

Goal 5: Use graphical, symbolic, and numerical methods to organize, analyze, and interpret data, and to effectively communicate findings.

Table 3: Selected Learning Objectives for Teaching and Learning Evolution in Introductory Biology for Majors and Non-Majors

Selected Evolution Learning Objectives	Aligned with SRQL Goal(s)	Activities and Assessments
Mechanisms of Evolution: 1. Natural Selection Use the key principles of evolution by natural selection (phenotypic variation in a population, genetic origin of variation, heredity, and differential fitness) to explain how populations change over time. Apply the general principles of natural selection to multiple specific cases of evolution. Use conceptual models to explain how selection leads to changes in populations.	2, 4, 5	Antibiotic resistance in bacteria as a case study. Pre-class JiTT (Novak, 1999); in-class activities; assessment on exam

Formative Assessment

To prepare for the class, students read a short review paper on antibiotic resistance (Genereux & Bergstrom, 2005) and electronically submit responses to two questions prior to the start of class—an example of a Just-in-Time-Teaching (JiTT) strategy (Novak, 1999):

1. “Some mutations change the bacterial proteins that are often the targets of antibiotic treatment. . . . Consider a random mutation that changes a bacterial protein” (Genereux & Bergstrom, 2005): (a) How does a random mutation cause a change in a protein? (b) How can such a change affect fitness in bacteria populations?
2. Bacteria carrying a mutation that confers resistance to a given antibiotic have increased fitness, whether the antibiotic is present or not. Is this statement correct or incorrect? Explain (that is, provide adequate warrants for your claim).

Alignment with SRQL Goal 2. Although instruction has not yet occurred, the teacher (in question 2) is already asking students to evaluate a claim using evidence from either the paper or other sources. As class begins, the teacher engages students and solicits their prior knowledge of antibiotic resistance with a brainstorming activity: “Do you use antibacterial soap? Why or why not? Discuss in your groups.” Students are given the opportunity to describe their experience with and opinions of antibacterial soap. As students report back, the teacher, using a tablet computer, categorizes student answers into “pro” and “con” and probes students to support their ideas with data and evidence. As in the JiTT homework, the teacher is focused on pushing students to use evidence as they evaluate their own claims regarding the use of antibacterial soap. The instructor is simultaneously probing: (a) students’ ability to construct and evaluate reasoned scientific arguments (an MSU liberal learning goal); and (b) students’ understanding of the bacterial evolution process (a course- and discipline-specific objective). Immediate feedback to the students during the classroom discussion represents the transition from “probing” to “teaching” (formative assessment).

Now the teacher turns to an experiment on antibiotic resistance (Figure 2a). Because students are unfamiliar with this experiment and inexperienced with interpreting this type of representation, the teacher scaffolds the inquiry. First, students consider whether there is variation among the bacteria in the image. Students report back and quickly agree that, yes, there is variation. The teacher probes further: on what evidence do you base your answer? Eventually, students identify the halo surrounding the antibiotic disc as a mix of dead and living bacteria that correspond to antibiotic-susceptible and antibiotic-resistant bacteria, respectively.

Figure 2a: Visual Evidence of Evolution

The heterogeneous bacterial lawn represents differing responses to various antibiotics administered by the white discs. At a basic level, students recognize the halo or ring surrounding the disc as evidence of phenotypic variation in bacteria: large rings indicate sensitivity to the drug, and small rings or halos indicate resistance. More advanced students quantify the evidence in this petri plate by measuring the rings or zones of inhibition that surround each antibiotic disc. Figure courtesy of Centers for Disease Control and Prevention.

The teacher then asks students to link this example to what they already know about evolution. She asks two simple questions: (a) What is the origin of this variation? and (b) Is the variation heritable? Students work quickly in groups and report their answers. Although many students recognize that mutation causes variation, the teacher asks students to link their knowledge of what mutations are and how they occur to this specific case. Students use their abstract, conceptual knowledge and apply it to this concrete example in order to explain why and how certain bacteria are resistant to antibiotics and how this resistance spreads in populations over time.

Alignment with SRQL Goals 4 and 5. In a second exercise, the teacher switches to a graphical representation of the problem (Figure 2b). This activity aligns with SRQL Goals 4 and 5 because students will work with another representation of the problem and will interpret data. Returning to what students already know, the teacher begins by asking them to describe the variation implicit in the graph. Students readily offer that some bacteria must be resistant and others susceptible to the antibiotic. The teacher asks students to identify and explain the error bars, which are quantitative evidence of variation in the graph. The teacher then challenges students to interpret the graph in terms of bacterial fitness and selection.

Figure 2b: Graphical Evidence of Evolution

This graph represents an experiment in which bacteria were repeatedly exposed to triclosan, a common biocide used in consumer products. Students must carefully interpret the graph. Although the y-axis begins at -2, a negatively sized zone of inhibition is not possible. Source: Welden, C., and R. Hossler. 2003. Evolution in the lab: Biocide resistance in E. coli. The American Biology Teacher 65:56–61. Figure used with permission from the National Association of Biology Teachers.

Alignment with SRQL Goal 4. A final activity of this class meeting asks students to create a model of how antibiotic-resistant bacteria might evolve (Figure 2c). Students must organize their ideas into a concept model with clearly labeled structures and processes. This activity is another example of SRQL Goal 4.

Figure 2c: Modeling Evidence of Evolution

Students build a box-and-arrow model to show the evolution of antibiotic resistance in bacteria. Source: Adapted from Genereux, D., and C. Bergstrom. 2005. Evolution in action: Understanding antibiotic resistance. In Evolutionary Science and Society: Educating a New Generation. Ed.J.Cracraft and R.W.Bybee. Colorado Springs: BSCS; Washington, D.C.: American Institute of Biological Sciences. http://bscs.org/curriculumdevelopment/highschool/evolution/pdf.html.

Summative Assessment

Throughout the unit on evolution, the teacher incorporates SRQL goals into a variety of case studies that drive instruction. As she creates the exam, she revisits the goals of each class meeting and the broader course goals. Because the students have worked with multiple representations of evolution—including bacteria and antibiotic resistance—the instructor decides that the exam must help students unify the principle of evolution across exemplars (SRQL Goal 4). On the exam, students are asked to complete a table (Figure 3) by explaining how each of the key concepts listed in the left-hand column applies in the context of two specific examples, one of which is the evolution of antibiotic resistance by bacteria. An ideal student answer may seem repetitive between the two examples, differing only in specific details, but demonstrates a solid understanding of the principles of evolution.

Figure 3: Summative Assessment Item from the Course Midterm

Students analyzed both evolution cases in class. Source: Antibiotic resistance figure adapted from Genereux, D., and C. Bergstrom. 2005. Evolution in action: Understanding antibiotic resistance. In Evolutionary Science and Society: Educating a New Generation. Ed. J. Cracraft and R.W. Bybee. Colorado Springs: BSCS; Washington, D.C.: American Institute of Biological Sciences. http://bscs.org/curriculumdevelopment/highschool/evolution/pdf.html; Wild Tobacco Populations figure developed by Amy Angert, Frances Kapczyk, Angela Roles, and Heather Sahli, students of PLB 802 at MSU.

LOOKING AHEAD

The national imperative for teaching science in the liberal arts curriculum is hardly new. In 1983, A Nation at Risk (National Commission on Excellence in Education, 1983) called for educational reform that included lifelong learning in science, focusing on concepts, methods of inquiry and reasoning, application, and the social and environmental implications of science. More than forty years prior to that report, John Dewey (1940) described the need for all students to achieve a proficient level of science education to meet the needs of the twentieth century. Now we want our students to achieve the scientific and quantitative reasoning goals so they have the tools to keep pace with science in the twenty-first century.

Much of the teaching we observe in university science classrooms is still primarily based on teacher-centered lectures and on the assessment of factual recall. Courses aimed exclusively at delivering knowledge of discipline-specific science facts provide students with only a limited, finite view of the world. The liberal learning goals for science are intended to prepare students to deal with the complexities of science as a way of knowing and with the intricate relationships of science and society.

To teach science as a liberal art, faculty must think more broadly about their courses and their teaching practices. Indeed, they may perceive that their students have accomplished some of these goals already, but their classroom assessments fail to demonstrate adequately the corresponding student outcomes.

As faculty begin to explicitly align their course objectives and assessments with department, college, and university learning goals, they may see this as a daunting task. The unit of instruction we described is an example of how to embed broader institutional outcomes into discipline-specific instruction and illustrates how learner-centered instructional designs help students achieve higher-level thinking, including analysis and synthesis.

In recent years, new program accreditation standards for engineering and pharmacy education stirred a great deal of discussion about how to equip students with the knowledge, skills, and attitudes represented by the new program outcomes (Abate et al., 2003; Felder & Brent, 2003). The practical issue of translating broad program outcomes into curricula has brought attention to the central role of the faculty and the need for guiding and supporting faculty throughout a process of change. Change at the program level can happen only if all faculty adopt a systematic, scientific approach to instruction by (a) identifying outcome-related course learning objectives; (b) teaching to address the outcomes; and (c) assessing learning appropriately. Key to this process is the implementation of backward instructional design and learner-centered pedagogies (Felder & Brent, 2003).

The liberal learning outcomes designed by universities for all their students could and should drive a similar effort in rethinking science education. Institutions need to invest in faculty, support their professional development, and ensure that they reflect the institution’s goals in their courses. Faculty need to invest in teaching strategies that allowall students in the liberal arts curriculum to achieve the scientific literacy and reasoning skills needed for effective citizenship. Nurturing the talents and abilities of all students in science is a necessary and long-overdue investment in our future.¹

ENDNOTES

1. Thanks to Donna Sundre, of James Madison University, for inviting a team from MSU to participate in her NSF-funded project (DUE-0618599) on assessing scientific and quantitative reasoning. We appreciate the input of our team members, Megan Donahue, Jennifer Kaplan, and Gabe Ording, in this endeavor. In addition, thanks to our students and to the instructors and graduate and undergraduate assistants with whom we teach.

REFERENCES

Abate, M., M. Stamatakis, and R. Haggett. 2003. Excellence in curriculum development and assessment. American Journal of Pharmaceutical Education 67:89.

American Association for the Advancement of Science. 1990. Science for All Americans. New York: Oxford University Press.

Association of American Colleges and Universities. 2009. Learning and Assessment: Trends in Undergraduate Education. Washington, D.C.: Hart Research Associates. http://aacu.org/membership/documents/2009MemberSurvey _Part1.pdf.

Bransford, J., and National Research Council Committee on Developments in the Science of Learning. 2000. How People Learn: Brain, Mind, Experience, and School. Expanded ed. Washington, D.C.: National Academy Press.

Bybee, R. W. 1997. Achieving Scientific Literacy: From Purposes to Practices. Portsmouth, N.H.: Heinemann Publishing.

Dewey, J. 1940. Education Today. New York: Putnam.

Ebert-May, D., and J. Hodder. 2008. Pathways to Scientific Teaching. Sunderland, Mass.: Sinauer Associates.

Felder, R., and R. Brent. 2003. Designing and teaching courses to satisfy the ABET engineering criteria. Journal of Engineering Education 92:7–25.

Genereux, D., and C. Bergstrom. 2005. Evolution in action: Understanding antibiotic resistance. In Evolutionary Science and Society: Educating a New Generation. Ed. J. Cracraft and R. W. Bybee. Colorado Springs: BSCS; Washington, D.C.: American Institute of Biological Sciences. http://bscs.org/curriculumdevelopment/highschool/evolution/pdf.html.

Handelsman, J., D. Ebert-May, R. Beichner, P. Bruns, A. Chang, R. DeHaan, J. Gentile, S. Lauffer, J. Stewart, S. M. Tilghman, and W. B. Wood. 2004. Scientific teaching. Science 304:521–522.

Michigan State University. 2009. Liberal learning goals and outcomes. http://undergrad.msu.edu/outcomes.html.

Michigan State University Committee on Liberal Learning. 2007. Quantitative and scientific reasoning. http://ucll.msu.edu/node/66.

Moore, J. A. 1993. Science as a Way of Knowing: The Foundations of Modern Biology. Cambridge, Mass.: Harvard University Press.

National Commission on Excellence in Education. 1983. A Nation at Risk: The Imperative for Education Reform. Washington, D.C.: The National Commission on Excellence in Education.

National Leadership Council for Liberal Education and America’s Promise. 2007. College Learning for the New Global Century. Washington, D.C.: Association of American Colleges and Universities.

Novak, G. M. 1999. Just-in-Time Teaching: Blending Active Learning with Web Technology. Upper Saddle River, N.J.: Prentice Hall.

Welden, C., and R. Hossler. 2003. Evolution in the lab: Biocide resistance in E. coli. The American Biology Teacher 65:56–61.

Wiggins, G. P., and J. McTighe. 2005. Understanding by Design. 2nd ed. Danvers, Mass.: Association for Supervision and Curriculum Development.