Fran Benne | Agronomy and Horticulture
Wortman applies a comprehensive approach to measure learning in the classroom
Several members of the teaching faculty in agronomy and horticulture have shared new approaches to teaching and how they measure learning impact.
Sam Wortman, assistant professor and environmental horticulturist in the Department of Agronomy and Horticulture, applied a more comprehensive approach to measure learning in his Horticulture 221 Plant Propagation class by assessing several different methods and activities.
Wortman partnered with Weaver Professor of Agronomy Martha Mamo to evaluate what the students reported on surveys, paid attention to the learning levels and created an education research article from this work.
“We tried a mix of different methods based on a workshop hosted by Marilyne Stains’ educational research team,” Wortman said.
This workshop is heavily grounded in research developed by Princeton University and funded by the National Science Foundation. Stains, a University of Nebraska–Lincoln associate professor of chemistry, comprehensively studies university science, technology, engineering and mathematics teaching to improve programs that train faculty to teach science better. Stains develops and conducts education workshops at the university to train faculty in new teaching methods. Wortman implemented some tactics from her workshop.
Wortman said he had excellent results using the Peer Instruction method the last two years in class. In the middle of a lecture, students were given a multiple-choice question. It wasn’t a recall question, but a problem-solving question. Students were asked to give an initial answer. The goal of this kind of problem was to get a fairly even split for possible answers among students. Students were asked to look around the room, find someone with a different solution, discuss and try to convince that person why their response was correct.
“You hope the class comes to a consensus with the right answer as you walk around the room and help facilitate and guide them in the right direction,” Wortman said. “Then we revote after that period and ask them to explain why they chose that answer. Sometimes there are varying degrees of correctness, and you try to play to common misconceptions.”
Another method he tried was Minute Papers which involved writing down answers to questions, at the beginning and end of class. Question examples included, “What’s the most important take-home message from the last hour?” “What’s the muddiest point from the previous hour?” “What questions do you still have?” Students submitted answers, and Wortman provided feedback.
Wortman used a Likert Scale, IRB-approved survey asking students whether the material helped them. When asked if the multiple-choice questions (Peer Instruction) were beneficial and helped students accomplish their learning objectives in the unit, Wortman said it was overwhelmingly positive and resulted in a 4.44 on the Likert Scale which was one of the highest of all the methods he tried.
But not all methods have a positive result. According to Wortman, the Minute Papers ranked low on the Likert Scale survey compared to the Peer Instruction method. He decided to discontinue this method based on the student survey results and the time it took for him to grade.
After discussing the results with Don Lee, professor and student advisor of agronomy and horticulture, Wortman decided to replace the Minute Papers with an Immediate Feedback Assessment Technique.
He had students watch a video before class, often a virtual reality tour of a greenhouse or similar situation, and then they worked in groups during class to take a multiple-choice quiz about the video using a scratch-off sheet. They kept scratching the answer choices until they chose the correct answer.
First-impressions from Wortman’s students were positive. The students did well on the quizzes and felt the quizzes were fun, low-stakes and they were able to work with their peers. He also had the students fill out a Likert Scale survey, and the results confirmed the students’ positive initial reactions to this learning method. One hundred percent of students in the class agreed or strongly agreed that the pre-class videos helped them learn about plant propagation — 4.47 on the Likert Scale — and 89 percent of students agreed or strongly agreed that the scratch-off quizzes about the pre-class videos helped them learn — 4.26 on the Likert Scale.
Rooted in what he learned, Wortman plans to use both the Peer Instruction and the Immediate Feedback Assessment Technique this fall when classes begin Aug. 20. He will be teaching Horticulture/Agronomy 100 Plants, Landscapes, and the Environment and Horticulture/Agronomy 375 Innovations for Agriculture.
Based on the survey results assessing student perceptions of learning outcomes, Wortman and Mamo compared these perceptions with other quantitative measures of student learning and looked at different ways to evaluate learning outcomes based not just on what students said, but how to measure beyond what students said. They wrote the research paper Developing a Holistic Framework for Learning Assessment Outcomes—A Case Study of Plant Propagation.
They found that asking students what they think they learned, is not very effective. “Generally, students feel more confident in what they learned than how they actually perform when you assess them on the material that should be learned,” he said. “In my evaluations, and this is pretty consistent, there’s not a big standard deviation of student responses to how they’re learning—it’s generally pretty high. But when you quiz them or hold an exam, you get greater variability in how students are performing.”
So, the next step for Wortman and Mamo was actually to tie these quizzes to a specific learning objective. “We had information about how well students believed these methods were working and we had information about how well students thought they accomplished learning objectives,” Wortman said.
Their whole objective of this research project was to see whether or not the students’ responses to the IRB survey were predictive of actual learning as measured by exams, quizzes and labs or vice-versa. They set out to measure 14 learning objectives.
“What was interesting was trying to assess this statement on three exams, quizzes and labs,” Wortman said. “They were not well correlated because they’re all assessing different things.”
They concluded that one should not substitute for the other. Each assessment tool has to be looked at individually as each gives unique information about how students are learning and what needs improvement. Looking at just one gives an over-inflated view, and may not tell the whole story. However, all used together can be great tools.
Wortman admits he tries to go to as many Research on Education workshops as he can.
“It takes a little bit of extra effort to document and to match some of these methods up with specific activities to figure out if it has meaningful value or if the students just think it’s fun,” he said. “Once you do that, then you can make informed decisions about which methods to carry forward and which to kick out over time.”
Wortman feels it’s important to revise each year. “Students’ preferences change over time,” he said. “Some of the methods are not necessarily new. It’s just being willing to try some of these methods and find what’s going to work to engage students today because it might be different in five years. It’s an adaptive teaching strategy.”
Teaching Impacts in Agronomy and Horticulture is a series this summer, highlighting teaching faculty and learning approaches in the Department of Agronomy and Horticulture, College of Agricultural Sciences and Natural Resources.