The Temptation of Publishers’ Products

I suppose the sign of an educated person is that they can learn from anyone and anything. This week I’ve learned from a publisher’s product, and the design it uses could solve some problems. The question is whether those problems should be solved, and whether this is the best way to do it. I’m tempted. They’re doing some very cool things, these publishers.

The product is an interactive textbook, with videos and little quizzes built into the page. They are taking the idea of proximity to its logical extent  – everything that relates to the topic is together. The design is intended to force the student to interact with the material several times while on the page, in an effort to reinforce the reading. The reading itself has been scaled down. Each chapter has five or six sections, each section is about four scroll screens, with a single column, lots of white space, and multimedia as well as text.  It is obviously designed to look good on a cell phone.

cc ryanne lai via Flickr

cc ryanne lai via Flickr

It has many elements of a textbook, but frankly it looks a lot more like my lectures. My lectures have media all over the place. What they don’t have is an assessment element, or if they do have that (my History of England lectures have a little Javascript self-quiz at the end of each lecture) the results don’t go to the grade book.

I confess to being impressed (I’ve seen this product demo’d now with two different textbooks), and tempted to adopt. I’ve asked our tech admin to find out how I can integrate this (and other) products into an LMS.

No, go back, don’t be tempted! But I am struggling with student retention and completion as issues the administration takes seriously, so I begin considering adopting this product. What it lacks in breadth it seems to make up for in depth. At the end of each unit, it has students write a reflection that connects the chapter to contemporary topics, and puts their posts into a discussion board. It’s a well-designed “learning system”. I do not buy all their crap about “engagement”, but it does force interaction with the material.

Structured as things are now, this product would replace the textbook. That’s what it’s intended to do. So what is the textbook for? If it’s to provide factual background information to my lectures, this is way bigger than that. It has its own pedagogy and its own interpretation of the material. It requires a different kind of analysis than a new textbook.

My existing course design

First, If I were to assign such a “text”, what would happen to the other elements of my class? These are:

1. My lectures – reported by students every semester as their favorite aspect of the class, my lectures are my interpretation of history and contain embedded primary sources, music, video, and my own voice and video.

2. Primary source research – the second-favorite with students, and my first favorite, I’ve written on using the discussion forum as a lab and I wouldn’t want to lose this.

3. Quizzes – My quizzes now include questions from lecture (including primary sources) and whatever I’m using as a textbook.

4. Writing assignments – I’m down to only five of these per semester, all based on the students’ primary sources in #2.

Since the self-declared reason students drop my class is “the class looked like too much work”, which of these is sacrificed for the more thorough online textual experience? The quizzes might not be an issue, except that they help make sure students are understanding the lecture.

Product location and service

Second, the product is located in a separate web location, in order to make sure everyone is paying for it. I’ve examined several publishers’ products now. Most force you to go outside into what’s becoming their own LMS. Only one lets you bring links in by chapter. I’ve checked out their LMSs, and they won’t work for the primary source forums – forum design is still the weakest area of ed tech, even after 15 years. Most products “link” or “connect” to Blackboard and Moodle, so a student has single sign-on, but the location of the material cannot be put “in” to the LMS in a way that’s seamless. This undermines the whole idea of proximity that is central to the effectiveness of the product. The lack of true integration means that these publishers aren’t yet in the 21st century (I still have to use a phone to call in for their webinars).

Also, because it is not my product, and not a supported LMS, it adds a third layer of possible technological problem and need for support. Publishers are famous for giving you the world until you adopt the product, then not being much help. And everything’s dependent on their servers.

Catering to bad habits

Third, what learning preferences are we catering to with such products? All of the webinars I’ve attended begin with the profs taking turns stating what their greatest challenge is in teaching the x survey class. The answers are totally predictable: underprepared students, getting students to read the text, getting students to use what they read. How do we diagnose these problems? Students aren’t doing the reading, or they’d do better in the class. We want them to do better. We want them to learn. At the same time, we don’t want to lower the standards of the discipline.

cc Chris Wejr via Flickr

cc Chris Wejr via Flickr

The solutions in this product, the depth-over-breadth approach, rely on the “current research” on learning. Well, not on learning, but on student success. Student reading attention span is short, so the solution is to “chunk” information and given them less content. Their reading level is low, so we dumb down the text and put in links to difficult terms. They like video (actually, the publishers claim they learn well with video – I have not seen that to be true in practice), so we add more (short!) videos. Their attention drifts from the text, so we force them to click to see this map, and take a little quiz, and click on the video, and rearrange these items, and do a bit of writing.

So the whole structure of the product is to cater to students who cannot create their own learning pathways, who are accustomed to having everything designed for them, who have difficulty reading and remembering, and who do not know how to study. We support all of these bad habits with this approach, but also use technology to reinforce some depth of understanding.

Weighing the considerations

I’m looking at three ways to go here:

1. Adopt: Foreground the retention concerns and adopt the product, jettisoning at a minimum my quizzes, and making lecture viewing optional. Figure out how to put it into Moodle so I can use the forums for primary sources. Or dump those too.

2. Redesign: Balance the retention concerns with my own pedagogy, by adopting the useful elements of the product using my own technology skills – putting mini-quizzes and pop-up definitions inside of lecture, and dumping the DIY textbooks I’ve been using. This would be, obviously, a huge amount of work.

3. Keep Calm and Carry On: Ignore the retention concerns and continue with my design, which requires extensive reading, weekly 25-question quizzes on lecture and text, weekly primary source posts, and five writing assignments based on these, a workload far less than what I did as a freshman, but which is increasingly becoming anachronistic in a world of weekly log-ins, minimal reading, low-stakes self-checks, and low grading standards.

I confess to being tempted by #1 for the first time in my career. Undertaking #2 is more like a sabbatical project, and could take all my time, but I’d like to explore the options in future posts. #3 is of course the default, encouraging my own bad habits.

The dark side does have cookies. They taste better now, even if they’re not good for you. And we seem to be in a world where everyone just wants dessert, higher grades for less work. Whither the artisanal prof who cares about her field?

Maybe free is bad – something else not to talk about

I have just spent the last few weeks doing as I meant to do for the last year – creating a book of Wikipedia text and my own edited primary source documents to create a free textbook for my students in Western Civ I.

Now that I’ve finished, and it’s all ready for my summer classes (both as a pdf they can download and print, and chapters inside my online class), I can go back and catch up on my reading about online teaching.

One of the things I’m supposed to be reading about is OEI, the California Online Education Initiative being run by a number of wonderful people. What they’re creating, however, will undermine artisan course design and bring in rubrics that already have several good online teachers in tears.

As part of this project, there are courses being offered by faculty at several institutions. A number of faculty have volunteered to have their classes be models for the new system (I declined when I saw the rubric). The word “model” has now been thrown around the administration as meaning they are great classes.

Some may be. Some of the most lauded, however, are taught with prepackaged course cartridges and full technology from a major publisher. I went and looked at that publisher’s offering for one “model” class, their costly package to students, and found what I expected – the cartridge is essentially teaching the class.

While it’s sickening that this kind of thing is the new “model” course for the future (I’ve ranted about that elsewhere), I was looking at the price. $177 new, with rentals varying from $80-133.

In all classes except one, now, I’ve given all the materials to my students. They don’t need to buy anything. My classes have students who go in and out, don’t do all their work, fail because they don’t follow instructions. In the “model” canned course, student success rates are high, as is retention. Extremely high. Only 10% seem to leave the class. Grades are high too. In History classes overall, it isn’t unusual to have 20% drop the class. We have always thought this is because our History classes are more demanding than what is being offered in other disciplines.

But there may be another aspect. If one pays $177, perhaps one is more dedicated to the class? Or could it be that the canned class makes it easier for students to pass without much stress (i.e. thinking) so they tend to stay? Or could the canned class be better? for whom? for learning? or just to make everything easier for everyone, student and instructor alike?

But wait! I know of another discipline (again, not mine) with high student success and retention also, where their online numbers equal their on-site numbers, but the classes are not canned, and in fact are outstanding artisan classes. The book? $95 new, $52-72 for rental.

My conclusion? I should not be creating free materials – it may be devaluing the classes I’m teaching. I know it’s not the quality of the materials – not only do I edit them all myself, but I have reviewed dozens of textbooks (see my name in many of them) and most are not very good. It’s the perception of the quality of the materials.

I had a student comment on an evaluation that he didn’t want to read the article I had linked from Wikipedia, because it made him feel like he wasn’t in a college class – if he wanted to read Wikipedia, he didn’t need to be paying college tuition. (Of course, he isn’t paying much tuition – the state has him covered – but that’s another post.) The quality of the article wasn’t the point – it was Wikipedia, so it must be useless.

If I’m right, the point that has gotten lost in the anger at high textbook prices, the insistence that community college remain open access, and the administrative concern about retention, is that students may want to pay high prices for textbooks. It may keep them dedicated to the class, even when they have to borrow money to buy them. I don’t think anyone really wants to talk about that possibility.

Wacky online class stuff, part 47

I say “part 47″ because I’m sure I’ve posted at least 46 wacky things so far – I just haven’t called them that.

This wacky thing is about my US History classes. I offer two online sections of the class each semester, and have done for years. Thus, on the registration page, it lists two sections, each with a different section number (say, #1561 and #1562).

Invariably, both sections fill (not because everyone needs US History, but because everyone thinks they do – but difficulties in academic advising are a topic for a different post).

Also invariably, the first section numerically (here #1561) fills first, and that shows in red lettering when one tries to enroll. A logical person would see that it’s full and register for the other section.

Instead, every semester, a handful of students add themselves to the wait list for the first section, even when the other section has plenty of seats.

Now here’s the other wacky part. Every semester, the second section (here #1562) will contain a larger number of students who throughout the semester fail to follow instructions, and whose work is of significantly lower quality.

I am not sure what this means. Is it simply that students who register first for any particular class are naturally drawn to the first section, and they’re first because they plan ahead and can follow instructions? Since I encourage the wait-listed students to instead register for the second section, this might increase the number of students who don’t quite get it, and they combine with the students who added late, either for lack of planning or lack of interest in the course.

Whatever the reason, I can pretty much predict the grade distribution at the end of the semester, depending on the section number. Which seems kinda wacky.

Changes for spring

Yes, I change stuff every semester (Jen Dalby likes to watch me prep, cuz it makes me tweet crazy things). Here’s what’s on for spring semester:

Expectation zone: percentage changes 

crocus

Crocus, harbingers of spring
(cc Julie ann Johnson via Flickr)

I believe students expect the quizzes (based on reading, and factual retention) to count for more. This semester, quizzes are only 30% of the grade. I’m raising it to 50%. Primary sources, which they like and which require some research, have been only 20% – I’m raising it to 30%.

Back to the future : shorter and more frequent writing

Having gone the other way, from weekly writing, to every other week, to six assignments, to five, I am going back to the way I had it at first – weekly writing. The big change is that it won’t be as formal, even though it still scaffolds up to the final essay. The other big change is it won’t be “graded”. Rather, writing will be part of the “A bit of writing and conversation” forum. Students will be encouraged to post a short piece of writing based on their choice of primary sources, and converse with each other.

But I did it!: contribution grading will be based on completion

Instead of the Contribution Assessment, which I have used successfully for awhile (I admit it, I get bored), I am counting 20% of the grade (10% for each half of the semester) for the writing and conversation, all together. I intend to base this primarily on completion, but want flexibility for quality issues and to touch base with students in the middle of the class, which is what I liked best about the assessments. If I change my mind, I’ll do self-assessments.

Autobots roll out: automation for completion

I am turning on the Activity Completion feature in Moodle, and using it for the first time. The “bit of writing and conversation” is a checkbox on the main page – a single post marks it complete. Same for the primary source. For the quiz, completing it and getting an (automated) grade completes the task.  Very few of my students fail the class because they can’t do the work – most fail because they don’t complete the tasks. Since I believe the tasks are important elements of practice, each one is low stakes but all need to be done. The green check (or lack thereof) on the main page will be obvious.

What I’m not changing

In a survey earlier this semester, students indicated that what they liked best were (1) my lectures, and (2) posting their own primary sources. So those aren’t changing at all. They didn’t like discussion, so that’s now optional and part of the writing forums.

But why change anything? (apart from, like, the boredom thing)

magician

cc JD Hancock via Flickr.

I am trying to answer a concern I have about this semester – the drop rate has been very high in all but one section. The drops mostly came at the beginning of the class. When I surveyed students who dropped, they told me the class was too much work. Interestingly, some wanted a class where they only had to log in once a week. I have not reduced the work at all, which I may regret later, but I am going to dare to use my experience to interpret what they’ve said.

I think part of the problem is that the class looks like too much work on the main page. Using Activity Completion makes the work more instantly visible, so they can see their progress and what’s lacking immediately. I am also removing the “labels”, or titles, for each section, so there is less text (a shorter list) to see for each week, while keeping the interactive syllabus format.  For those who think they can do the class only logging in  once a week, they can if they’re smart, by taking the quiz, posting their source, and posting their writing all on the same day. Since I don’t plan to “grade” conversation, if someone really doesn’t want to participate or talk to anyone, they don’t have to.

As with everything, we’ll see what happens!

Their favorite part of the class

I had a high drop rate the first week of class in several of the online sections, so I asked those who dropped to do a quick survey for me about why they dropped:

surveydrops

Because overload seemed to be an issue, I wondered what part of the class I might be able to do away with.

So I polled my students in the fourth week of class, asking them “What’s your favorite part of the class so far?” The result from those who responded out of five sections of History classes:

31 – the lectures
21 – the primary source readings linked from lecture
33 –  posting my own primary sources
3 – the writing assignments
3  – the discussion forums
1  – connection with other students outside class
5  – the textbook (only two classes have a textbook – 4 of these were from the England class, which actually has only an atlas)

What’s interesting about this is that so few like discussion. Discussion is, actually, my most recent addition to the classes. I added it only because it was hard to discuss in either the primary sources boards or the writing forums, and there were some cool issues that we could do only in discussion, such as ways history connects to what’s going on today. So I set it up to be student-led, and not every week. We don’t do discussion on the six weeks we do writing. And after the first two discussions, I rarely participate. The activity level has been high, even though I have not set a specific number of responses or replies.

But they don’t like it, or at least don’t love it. I expected that they would like posting their own primary sources, but I didn’t realize they were even reading the sources linked from lecture, so that’s cool. The lectures – well, those are mine. I spent a long time constructing and revising them. They have video clips and lots of graphics and lots of, well, ME. So I’m glad they like them.

I wonder whether removing discussion would help the class seem less overwhelming?

Why students drop

Each semester, most of my online classes fill and have a wait list. I usually email the top several people on the wait list, and invite them to join before the class starts. Then I let in about 5 over the limit (which is 40 – too many, but that’s another post). This way the enrollment will balance down to about 40 by the time our official census occurs, at three weeks in.

classef14But in two classes this semester, I’m down to 33. Usually only one section gets that low, and not until much later in the semester.

Why is this a problem, when 40 is too many for them to get to know each other, and 32 is about right? Because online classes are under the microscope everywhere. There are still people who don’t believe college classes can, or should, be taught online. I just last week heard about a college department all ready to go online, with skilled teachers, and then changing their mind upon hearing from an administrator that the subject just can’t be taught online. At our place, there is concern about the gap in “student success” (defined as those who finish with a grade of C or higher) between online and on-site classes.

Those of us doing this awhile know there are many reasons students might not be as successful online as they are on-site. We know that many students take an online class assuming it will be easier, or require fewer hours. We know they confuse flexibility with total workload. We know some don’t have the technical skills to know where to click or how to submit work. We know that some who aren’t prepared for college are even less prepared online.

But all this knowledge is based on experience, while admins like to focus on the less messy numbers. We do have some studies (like this one from 2005) that indicate some predictors of failure in online classes – students with a low GPA, young age, not taking an orientation, and a habit of dropping classes tend to fail. Another (from 2006) indicates that success online depends on time spent on the class, student initiative and student competence.  This one from 2007 concluded that “self-regulated learning strategies” are essential. A 2011 study concluded that individual student attributes (including persistence, academic achievement, and time management) plus life factors (including resources, skills and time) predict both student satisfaction and success. By 2013 we see a study claiming that self-efficacy and “task value” are the biggest predictors. Some studies note which types of students fail more often: males, younger students, AfricanAmerican students, and low GPA students (here). And “students with higher levels of technology self-efficacy and course satisfaction” earn higher grades (here).  Specifically referring to students who dropped out, another recent study showed that those who stayed in the class “had higher levels of academic locus of control and metacognitive self-regulation skills than dropout students” (here). An even more recent study decided that in addition to employment and academic preparation for the class, just the fact that the class was online was a predictor of final exam performance.

All of these suggest that the greatest predictors of student success reside in the student. And yet the pressure is on faculty, and the focus on instructional design. The implication is that if students succeed at high levels, it’s because we’re doing something right. If they don’t succeed, we’re doing something wrong. This view creates more pressure for cookie-cutter, idiot-proof, publisher-developed, team-created courses with no academic freedom or creativity for instructors.

CC Flickr Matthew Langham

It was too much work on board
CC Flickr Matthew Langham

So with such a high drop rate and such high stakes for me, I am surveying (anonymously, at a colleague’s suggestion) the students who dropped on or after the first day of the semester. So far, results are interesting. Most of them are saying that the class looked liked too much work, and that they wanted a class where they could log in just once per week. One said s/he dropped because the class wasn’t in Blackboard.

After years of designing my classes to provide quick, low-stakes assignments for the purpose of immediate feedback, so students can track their progress, this is a blow. I have no mid-term exam and no research paper – rather the writing is scaffolded, the quizzes are short and weekly, and the primary source posts take about half an hour of student time. The final exam is an essay they work on for several weeks, with feedback from each other and me.

Does this bring us to the last thing we know, but only anecdotally: that not all online instructors are requiring a similar workload for similar number of units? That my class is now too hard because other online classes are easier? And (a horrible thought) to what extent has my encouragement of online classes led to this?

Or does this bring us back to the studies, where student success clearly resides in student self-efficacy?

As they say, watch this space…