Student Success: Things That Make No Difference

Student “Success”

There has been great focus lately on that slippery, administratively-led, accountability-driven beast we call “student success”. Our college is creating entire plans designed to “help students succeed”, which means (depending on whom you talk to) passing their classes, transferring to university, obtaining a certificate, or just not dropping the hell out of college. (It does not tend to mean, interestingly, students succeeding in achieving their own personal goals like having a reason to get out of bed, or not getting kicked out of their parent’s house, or having somewhere they’re supposed to be, or hanging with their friends, or staying on their parent’s health insurance, but that’s the subject of another post.)

Sometimes the focus is on student services, making sure that students aren’t abandoned on campus, that they know where to go for help. But there’s also an instructional aspect. Recently, our Professional Development Committee received a statewide vision statement that included the sentence,  “All personnel will have ongoing opportunities to develop and expand the skills and practices that influence students’ ability to complete their educational goals”. I don’t think by “personnel” they mean janitors and those helpful people over in the cafeteria.

The implication in applying “student success” directives to instruction is that pedagogy has a strong influence on whether students succeed or not. This should be a no-brainer. It should be obvious that better teaching leads to more student success.

But it isn’t that simple, of course. If it were, teachers’ unions wouldn’t react so strongly about tying their pay to student grades. We wouldn’t have Hattie’s 2009 study showing that only 30% of student success variations are related to instructional quality. (or see here).  And we would have some agreement on what constitutes good teaching (a consensus on this rarely lasts long).

So here’s my anecdotal, non-scientific list, a short draft based on 25 years of teaching experience, of pedagogical choices that appear to make little or no difference to student success. None of them are based on research – yet. I have in mind a longitudinal study of my own pedagogical methods related to grades, but that would take a sabbatical.

What Doesn’t Seem to Matter

Giving all test questions in advance.

Multiple-choice or essay, it doesn’t matter. The top students use them to increase confidence as they get the A they would have gotten anyway.

Having open book (or open lecture, materials, web) exams.

Online or on-site, having access to materials may improve certain individual’s work (usually those who would get an A anyway), but not the overall grades. In online classes, the effect is negligible.

Community (see previous post).

Active interaction with peers increases happiness but not grades.

Using the web in class during a test.

Test scores vary little even when students have their devices and access to the web. This may be related to the kind of tests I design, which don’t test facts but force students to use them. If they don’t study and internalize the information, open access to it doesn’t help.

Methods other than lecture.

Class grades have been close to the same regardless of whether the primary mode of classroom activity was me lecturing, students working in groups, or student-led projects (for this last, the drop rate, however, is very high).

Peer grading or working publicly.

Despite what I’ve heard from others, I have seen no improvement in quality based on students writing in public or grading each others’ work – I do not see the peer pressure that is presumably at play here. Revision of exam works or essays in consultation with peers similarly causes little if any improvement.

So why bother?

Any difference, as I’ve been saying for years, is affective. Open book or open web exams make students less stressed about the test, because their fear is the failure of memorization. Same with giving questions in advance.

Some would say that since these pedagogical techniques make no difference, we shouldn’t bother doing them. Let’s have closed exams, no community, droning lectures. I see it the opposite way – there’s no reason not to have open access during exams or opportunities to form community. It does no harm and creates a friendlier learning environment.

It just doesn’t seem to improve “student success”.

2 comments to Student Success: Things That Make No Difference

  • Interesting post, Lisa. I had missed your community post, so glad you drew my attention back to it. Give me some time, and I expect I could pull up research studies that suggest each of these does matter…and that they fall in to best practices. These are all techniques…but what was not mentioned and does matter is that desire to make a difference in student lives (as I know you do). Is there a study that correlates student success to the degree to which faculty make known that they care … and mean it?

    I am struggling with your post precisely because it pushes some buttons that we tend to push. Our faculty development is framed around the notion that elearning is more than putting content online, and that community is part of engagement. In my 17 years of teaching online, I would hazard that part of my success (and my students’) lies in early efforts to form and maintain community.

    Maybe it is pre-Thanksgiving…but my online community is part of what I give thanks for. :-)

    • Hi Britt! That’s just it – these are all very important for student comfort and possibly their desire to work and engage. I have seen nothing saying it makes a difference in their grades, though.

      I had started some research into instructor presence and student success, and again found only affective results.