Grades: the low-down on the drop-downs

In addition to submitting a grade for each student, and a last date of attendance of they failed, we are now asked to assess the level of learning outcomes for each student for two elements: critical thinking and global awareness.

Our grade sheet is starting to look like a data entry form.

I have heard faculty complain that this is ridiculous and impossible – it would take far too much time to reassess each student’s class performance in outcome areas (last year it was just one) as well as their final grade.

I don’t think so.

I remember many, many years ago, we had a full faculty meeting about developing and tracking our first Student Learning Outcomes. It was the third or fourth iteration of this idea, and we were all sick of it – sick of hearing about this stuff that had clearly come in from the outside, through administrative fiat. And one of my favorite colleagues stood up and said, “Don’t we already have this? It’s called GRADES.”

I’ve never forgotten that. The grade I give means something. I spend a lot of time determining what percentage of the final grade counts for each assignment and skill. So does my grade now mean nothing when set up against outcomes? Do I really have to reassess each student for their demonstration of critical thinking and global awareness?

No, because these are built into the Course of Study, the class design, and my pedagogy. When I give that final grade, it says something already about the student’s achievement in critical thinking and global awareness.

The drop-downs have levels of achievement on these:

gradesurf
My default for a real passing grade (A, B or C) is “Practitioner – Met”. If they hadn’t met my standard for critical thinking and global awareness, they wouldn’t have passed the course.

My default for a D or F is “Apprentice- Not Met” if the student finished the class. If they stopped attending, it’s “Novice-Not Met”.

If I recall their work as being excellent, Critical Thinking jumps to “Expert – Exceeded”. Few get this designation – I am the expert, and few excel in either critical thinking or global awareness. But if they did, I remember it – I don’t have to look anything up.

Similarly, I recall other details leading to exceptions: the brilliant expert student who got a D for not turning stuff in, the B student who didn’t know where China was, etc. Again, no need to look those up.

So even though it seems burdensome, the process goes pretty quickly. Because I trust my grades.

Co-posted at MiraCosta’s Reflections on Practice blog

Rhizo15: Toppling the teacher

Historically, when a dictator is removed from power, all the factions being oppressed by that dictator fight each other for power. This happens regardless of the peaceful or democratic or socialistic ambitions of those who topple the dictator. In ancient Greece, the pattern was from monarchy, to oligarchy, to tyranny (in this case a ruler brought to power by the people), to democracy, and then often back again to monarchy.

This isn’t because monarchy is a natural state, necessarily. It is because having been under a monarchy or dictatorship, people have had little opportunity to acquire the skills necessary to build something better, particularly with other people.

If we look at our educational system, many have described it as a hold-over from 19th century industrialization, the need to teach people to be good factory workers. This isn’t really true, as many great and creative thinkers came out of those schools, and made improvements in education as well as work.

But it’s a handy narrative for those who want to create a “freer”, inquiry-based experience for students, instead of emphasizing rote learning and one-size-fits-all curriculum.

ComierToppledSo this week we ask whether we could get rid of our glorious leader, Dave. We don’t want to focus on getting rid of Dave as Dave – Dave is so inherently likable, and he gave us all this opportunity to get rid of him. But if he were a dictator, and we were to topple him as a symbol of industrial education, with the goal of creating our own inquiry-based class, would it work?

Yes, because this class is full of people who have, usually through their own efforts and sometimes exclusively so, aquired the skills to be able to do that. If we do it with our students, it can certainly work for some of them (there are many examples of successful inquiry-based classes), but only if enough students acquire the skills necessary to function in that environment.

It is unlikely that students suddenly without a teacher would fight among themselves for control, however. Instead, they would likely seek another leader. I’ve seen this happen repeatedly, in committee meetings, classrooms, and local government. Many people do not want to inquire – they want to be told what to learn, what to think. When we open up the curriculum, they are lost and frustrated without enough guidance.

How do we get past this, help students (and ourselves) acquire the skills necessary to direct their own learning? Won’t we just be leaving different people behind when we topple the dictator?

Rhizo15: But I like content!

As Rhizo15 leaves the week about content (obviously I was not paying attention), I feel obligated to be the voice at the back of the (now empty) room saying, “But I like content!”

oliver-twist-007I love it. I’m the kid who sat on the floor reading the encyclopedia. I’m the student who got thrown out of the library when it closed. I’m the one looking up studies on the internet. I love content. All content. The expression of human knowledge, going back for centuries. Give it to me. In books, online, in text, on video. I want it all.

Why do we diss content in favor of connections? I like connections, I learn from them, but only when I bring something to the table. What do I bring? What do my students bring? Understanding of, or questions about, content. Content is what we’ve read, seen, heard.

Let’s not remove content – please don’t take it away. If we do that, we’ll all be connecting and communicating, but about what? About connecting and communicating? I like information – it gives me something to argue about.

Say, all these Rhizo15 tweets and posts I’m reading – they’re content! The product of other people’s minds, set out for me to absorb/enjoy/dispute/misunderstand. We create content, we share it, we respond and the response is more content.

I’ve MOOCed and rhizomed and connected and I still love content. The content we’ve inherited, the content we’re given, the content we discover, and the content we make.

Rhizo15: Symbolic measurement

I can’t measure learning, only the symbolic artifacts of learning.

That’s not so strange. We measure civic responsibility by how many people vote, but we can’t measure how “good” those votes are, the extent to which they are backed by intelligent thought or research into the issues. We can only measure outcomes.

As a college instructor with over 200 students and no assistants, I’m in an impossible situation to assess learning. I can only assess outcome achievement. I pretend that I can create assignments that will produce symbolic artifacts of learning. Then I grade the artifacts.

But it’s all a ruse. A student comes in with certain skills. Perhaps they already know how to learn, or have already learned the subject. They get As and Bs because they are engaged and eager to learn. When I give them an A for producing excellent outcomes, I have no idea whether I am grading their learning. What if they already knew or had examined the material before my class? What if they did all the work, but it didn’t change their mind or approach in any way? The “A” is a measurement of outcome achievement, regardless of background.

Similarly, the student who turns in no work at all may have learned something, something amazing, something that may or may not have related to what I taught, but was connected to my class. I’ve had military wives who learned, not history, but how important it was for them to have somewhere to be each day. I had a surfer guy who learned that if he synthesized information and then created his own interpretation, his conclusions were valid and could be important to others. I have students who learn that if they are polite to me and treat me with respect, they will in turn be treated with respect, and students who learn that faceless institutions don’t have to be impersonal.

If my measurement for that were individual, it wouldn’t relate to their grade in History. If my measurement were societal, I’d need to look to society. When I look to society, I see an awful lot of people behaving as if they’ve learned nothing from history. So instead I hope that they learned what they needed, whether or not I was able to assess it.

(this post related to the Rhizo15 class)

The Temptation of Publishers’ Products

I suppose the sign of an educated person is that they can learn from anyone and anything. This week I’ve learned from a publisher’s product, and the design it uses could solve some problems. The question is whether those problems should be solved, and whether this is the best way to do it. I’m tempted. They’re doing some very cool things, these publishers.

The product is an interactive textbook, with videos and little quizzes built into the page. They are taking the idea of proximity to its logical extent  – everything that relates to the topic is together. The design is intended to force the student to interact with the material several times while on the page, in an effort to reinforce the reading. The reading itself has been scaled down. Each chapter has five or six sections, each section is about four scroll screens, with a single column, lots of white space, and multimedia as well as text.  It is obviously designed to look good on a cell phone.

cc ryanne lai via Flickr

cc ryanne lai via Flickr

It has many elements of a textbook, but frankly it looks a lot more like my lectures. My lectures have media all over the place. What they don’t have is an assessment element, or if they do have that (my History of England lectures have a little Javascript self-quiz at the end of each lecture) the results don’t go to the grade book.

I confess to being impressed (I’ve seen this product demo’d now with two different textbooks), and tempted to adopt. I’ve asked our tech admin to find out how I can integrate this (and other) products into an LMS.

No, go back, don’t be tempted! But I am struggling with student retention and completion as issues the administration takes seriously, so I begin considering adopting this product. What it lacks in breadth it seems to make up for in depth. At the end of each unit, it has students write a reflection that connects the chapter to contemporary topics, and puts their posts into a discussion board. It’s a well-designed “learning system”. I do not buy all their crap about “engagement”, but it does force interaction with the material.

Structured as things are now, this product would replace the textbook. That’s what it’s intended to do. So what is the textbook for? If it’s to provide factual background information to my lectures, this is way bigger than that. It has its own pedagogy and its own interpretation of the material. It requires a different kind of analysis than a new textbook.

My existing course design

First, If I were to assign such a “text”, what would happen to the other elements of my class? These are:

1. My lectures – reported by students every semester as their favorite aspect of the class, my lectures are my interpretation of history and contain embedded primary sources, music, video, and my own voice and video.

2. Primary source research – the second-favorite with students, and my first favorite, I’ve written on using the discussion forum as a lab and I wouldn’t want to lose this.

3. Quizzes – My quizzes now include questions from lecture (including primary sources) and whatever I’m using as a textbook.

4. Writing assignments – I’m down to only five of these per semester, all based on the students’ primary sources in #2.

Since the self-declared reason students drop my class is “the class looked like too much work”, which of these is sacrificed for the more thorough online textual experience? The quizzes might not be an issue, except that they help make sure students are understanding the lecture.

Product location and service

Second, the product is located in a separate web location, in order to make sure everyone is paying for it. I’ve examined several publishers’ products now. Most force you to go outside into what’s becoming their own LMS. Only one lets you bring links in by chapter. I’ve checked out their LMSs, and they won’t work for the primary source forums – forum design is still the weakest area of ed tech, even after 15 years. Most products “link” or “connect” to Blackboard and Moodle, so a student has single sign-on, but the location of the material cannot be put “in” to the LMS in a way that’s seamless. This undermines the whole idea of proximity that is central to the effectiveness of the product. The lack of true integration means that these publishers aren’t yet in the 21st century (I still have to use a phone to call in for their webinars).

Also, because it is not my product, and not a supported LMS, it adds a third layer of possible technological problem and need for support. Publishers are famous for giving you the world until you adopt the product, then not being much help. And everything’s dependent on their servers.

Catering to bad habits

Third, what learning preferences are we catering to with such products? All of the webinars I’ve attended begin with the profs taking turns stating what their greatest challenge is in teaching the x survey class. The answers are totally predictable: underprepared students, getting students to read the text, getting students to use what they read. How do we diagnose these problems? Students aren’t doing the reading, or they’d do better in the class. We want them to do better. We want them to learn. At the same time, we don’t want to lower the standards of the discipline.

cc Chris Wejr via Flickr

cc Chris Wejr via Flickr

The solutions in this product, the depth-over-breadth approach, rely on the “current research” on learning. Well, not on learning, but on student success. Student reading attention span is short, so the solution is to “chunk” information and given them less content. Their reading level is low, so we dumb down the text and put in links to difficult terms. They like video (actually, the publishers claim they learn well with video – I have not seen that to be true in practice), so we add more (short!) videos. Their attention drifts from the text, so we force them to click to see this map, and take a little quiz, and click on the video, and rearrange these items, and do a bit of writing.

So the whole structure of the product is to cater to students who cannot create their own learning pathways, who are accustomed to having everything designed for them, who have difficulty reading and remembering, and who do not know how to study. We support all of these bad habits with this approach, but also use technology to reinforce some depth of understanding.

Weighing the considerations

I’m looking at three ways to go here:

1. Adopt: Foreground the retention concerns and adopt the product, jettisoning at a minimum my quizzes, and making lecture viewing optional. Figure out how to put it into Moodle so I can use the forums for primary sources. Or dump those too.

2. Redesign: Balance the retention concerns with my own pedagogy, by adopting the useful elements of the product using my own technology skills – putting mini-quizzes and pop-up definitions inside of lecture, and dumping the DIY textbooks I’ve been using. This would be, obviously, a huge amount of work.

3. Keep Calm and Carry On: Ignore the retention concerns and continue with my design, which requires extensive reading, weekly 25-question quizzes on lecture and text, weekly primary source posts, and five writing assignments based on these, a workload far less than what I did as a freshman, but which is increasingly becoming anachronistic in a world of weekly log-ins, minimal reading, low-stakes self-checks, and low grading standards.

I confess to being tempted by #1 for the first time in my career. Undertaking #2 is more like a sabbatical project, and could take all my time, but I’d like to explore the options in future posts. #3 is of course the default, encouraging my own bad habits.

The dark side does have cookies. They taste better now, even if they’re not good for you. And we seem to be in a world where everyone just wants dessert, higher grades for less work. Whither the artisanal prof who cares about her field?

Maybe free is bad – something else not to talk about

I have just spent the last few weeks doing as I meant to do for the last year – creating a book of Wikipedia text and my own edited primary source documents to create a free textbook for my students in Western Civ I.

Now that I’ve finished, and it’s all ready for my summer classes (both as a pdf they can download and print, and chapters inside my online class), I can go back and catch up on my reading about online teaching.

One of the things I’m supposed to be reading about is OEI, the California Online Education Initiative being run by a number of wonderful people. What they’re creating, however, will undermine artisan course design and bring in rubrics that already have several good online teachers in tears.

As part of this project, there are courses being offered by faculty at several institutions. A number of faculty have volunteered to have their classes be models for the new system (I declined when I saw the rubric). The word “model” has now been thrown around the administration as meaning they are great classes.

Some may be. Some of the most lauded, however, are taught with prepackaged course cartridges and full technology from a major publisher. I went and looked at that publisher’s offering for one “model” class, their costly package to students, and found what I expected – the cartridge is essentially teaching the class.

While it’s sickening that this kind of thing is the new “model” course for the future (I’ve ranted about that elsewhere), I was looking at the price. $177 new, with rentals varying from $80-133.

In all classes except one, now, I’ve given all the materials to my students. They don’t need to buy anything. My classes have students who go in and out, don’t do all their work, fail because they don’t follow instructions. In the “model” canned course, student success rates are high, as is retention. Extremely high. Only 10% seem to leave the class. Grades are high too. In History classes overall, it isn’t unusual to have 20% drop the class. We have always thought this is because our History classes are more demanding than what is being offered in other disciplines.

But there may be another aspect. If one pays $177, perhaps one is more dedicated to the class? Or could it be that the canned class makes it easier for students to pass without much stress (i.e. thinking) so they tend to stay? Or could the canned class be better? for whom? for learning? or just to make everything easier for everyone, student and instructor alike?

But wait! I know of another discipline (again, not mine) with high student success and retention also, where their online numbers equal their on-site numbers, but the classes are not canned, and in fact are outstanding artisan classes. The book? $95 new, $52-72 for rental.

My conclusion? I should not be creating free materials – it may be devaluing the classes I’m teaching. I know it’s not the quality of the materials – not only do I edit them all myself, but I have reviewed dozens of textbooks (see my name in many of them) and most are not very good. It’s the perception of the quality of the materials.

I had a student comment on an evaluation that he didn’t want to read the article I had linked from Wikipedia, because it made him feel like he wasn’t in a college class – if he wanted to read Wikipedia, he didn’t need to be paying college tuition. (Of course, he isn’t paying much tuition – the state has him covered – but that’s another post.) The quality of the article wasn’t the point – it was Wikipedia, so it must be useless.

If I’m right, the point that has gotten lost in the anger at high textbook prices, the insistence that community college remain open access, and the administrative concern about retention, is that students may want to pay high prices for textbooks. It may keep them dedicated to the class, even when they have to borrow money to buy them. I don’t think anyone really wants to talk about that possibility.