By Lisa, on May 13th, 2013% Every semester I ask my students about my online classes. After doing this for many years, I have come to the following conclusions regarding those negative comments, the ones that may be few but that keep us up at night. So if you are dealing with some negative feedback, and blowing it out of proportion, consider:
If you use text, video and audio to explain the navigation of the class at the beginning, they complain that there is too much to do. If you don’t use media to explain the navigation of the class, they complain because they get lost.
If you use a linear form of navigation throughout the class, they complain because they don’t understand what to do. If you use a non-linear form of navigation, they complain that they’re lost.
If you use nested forums where all posts are visible, they complain because the page is too long to scroll. If you use threaded forums where each post must be clicked, they complain that they can’t follow the conversation.
If you use the default buttons in the LMS, they complain because the course is just as boring as their last class. If you don’t use the default buttons in the LMS, they complain because the class doesn’t look the same as their last class.
If you grade things slowly because you’re putting lots of comments on assignments, they complain because they aren’t getting their work back fast enough. If you grade quickly, they complain because they aren’t getting enough detailed feedback.
If you post instructions in one place, they complain that they didn’t see them and so didn’t know about the assignment. If you post them in many places, they get confused that there was so much material they couldn’t find it all.
If you require quizzes provided by the publisher, they complain because either the publisher’s system didn’t work, or they didn’t want a different password, or the wording of the questions was too difficult. If you provide custom quizzes, they complain that the questions were not asking exactly what was written in the readings.
If you are nice and give a student a break on one assignment, they assume that all assignment deadlines are negotiable. If you don’t give them a break, you’re being cruel because it wasn’t their fault.
If you have them submit all assignments privately, they complain because they weren’t given any examples. If you have them work publicly in a forum, they complain because their work is seen by others.
If you have only a few types of tasks, they complain that they weren’t given enough chances to show their knowledge. If you have many different types of tasks, they complain that there was too much to do.
If you provide a rubric, they complain that they didn’t know about it or that their circumstances don’t apply to it. If you don’t provide a rubric, they claim grading was arbitrary.
If you provide only text-based lectures and assignments, they complain because there is so much reading. If you augment with audio or video, they complain because they couldn’t get the technology to work or didn’t think those parts were assigned.
If you do not require context reading aside from lectures, they complain that the course is subjective and they needed the facts. If you provide context reading from Wikipedia, they complain that Wikipedia shouldn’t be assigned because it’s not a good source.
If you do not allow outside readings as a source for writing, they complain because they were limited to only what was provided. If you allow outside readings, they complain because they didn’t know how to choose them or weren’t allowed to use them instead of the assigned readings.
If you create similar interactive activities for each unit, they complain because they’re doing the same thing every week. If you create varied activities for each unit, they complain that it isn’t consistent so they don’t know what to do.
If you do not ask them to complete an anonymous survey, they complain because they weren’t allowed to give feedback about the class. If you do ask them to complete an anonymous survey, they complain about things they would not mention in front of other students.
No, I won’t stop asking them for feedback, and I’ll bet you can tell it’s that grading / marking / begging / exceptions time of year, and many of my students would consider this list a mischaracterization because they love my class, and how it was constructed, and they tell me so, and they are right. But when it comes to considering that negative feedback….pass the grains of salt, please.
By Lisa, on April 23rd, 2013%  This semester I did a bad thing – I took something that was working perfectly well (ok, maybe it had one little weakness) and tried to improve it.
By “improve”, I mean I changed it to make it more detailed instead of leaving it broad.
It’s that Contrbution Assessment, the one I’ve been so happy with. I revised it because there were one or two things being misconstrued. The criteria all seemed roughly equivalent, so some thought they could not do any context readings, or not help others. I thought I could fix that.
So I changed it to make the assignment more specific, by asking the students to comment on each of the 12 criteria in the new rubric I made:
————————
Forum Posts, Essays and Final Exams
- Use of class materials and activities is expected.
Essays have made full use (A), good use (B), some use (C), little use (D), or no use (F) of class materials and activities.
- Essay theses must be interpretive.
Essay theses are highly interpretive (A), solidly interpretive (B), primarily factual with some interpretation (C), factual (D) or not a thesis (F).
- Essays must use the required number of primary sources from the forums.
The required number of sources used in the essay are all primary (A or B), mostly primary (C), mostly secondary (D), all secondary or not used(F).
- Sources must be fully cited.
Sources in the essay are fully cited (B), almost all fully cited (B), mostly cited (C), not all cited (D) or not cited (F).
- Writing must be at the college level.
Writing in the essay is at the college level or higher (A), at the college freshman English 100 level (B), at the high school level (C), below the high school level (D or F).
Contribution to the Class
- Students should log in at least three times a week.
Logs in more than three times a week (A), 2-3 times a week (B), once a week (C), less than weekly (D or F).
- Students should post/reply at least three times a week in the forums.
Posts in forums three times a week (A), 2-3 times a week (B or C), once a week (C or D), less than weekly (F).
- Students should respond to guidance from instructor, learn from group (rather than individual) feedback, and get help from the FAQ and college resources as needed.
Responds to instructor guidance as provided through examples, replies, and messages – always (A), almost always (B), mostly (C), occasionally (D), never (F).
- Students should be helpful to others through commenting, suggesting, or providing good examples in the forums.
Is helpful to other students weekly (A), regularly (B), occasionally (C), rarely (D), not at all (F)
- Work in the forum should be connected to class lectures and readings.
Work in the forum is clearly connected to class lectures and readings in every post (A), in many posts (B), in some posts (C), in few posts (D), in no posts (F).Lectures and readings should be completed weekly.
- Lectures and readings have been completed every week on schedule (A), weekly (B), most weeks (C), few weeks (D), rarely (F).
- Student work should reflect the student’s own interests.
Writing for theses and essays has been obviously related to the student’s own interests every week (A), most of the time (B), occasionally (C), rarely (D), never (F).
————————
A couple of things happened, and I noticed it first on the mid-term assessment. Some students copied the whole criteria list, and I couldn’t tell which level they were indicating as the one they achieved (text formatting got lost, I suspect, between their writing program and Moodle). Some just listed them (1,2,3) and just put a grade for each (1.A, 2.B, 3. B+) and I kept having to reference the list to tell what they were assessing. Some ignored it completely and just told me what their grade should be.
Now typically, the grades they tell me are really close to what they’ve contributed in terms of their writing and forum posts, and helping each other. Their evaluations of their own contribution have been honest, and the assignment interesting to grade. I’ve been able to check what they say against a downloaded list of their total posts, and their activity as tracked by the LMS.
But I noticed on the mid-term assessment that the grades were a little more off. More students (not a lot) were saying they should get a higher grade than was indicated. They would even discuss all 12 items, tell me they were earning C’s in three or four of them, then say they should get an A anyway.
And now I’m seeing it again, only more marked, on the end-of-term assessments. Some students didn’t read the comments on the first assessment, so didn’t improve, but listed the same grades and said they should get the same as last time. Many more are asking for higher grades than indicated by the criteria. I’m feeling like a meanie, instead of affirming their view. Ugh.
A couple of possibilities:
The 12 points were overwhelming rather than encouraging of reflection.
The 12 points made it appear overly formulaic, so they felt they didn’t have as much flexibility in determing their grade. It came off like a game to trap them, rather than an iterative process designed to engender self-analysis. People respond to games by deciding not to play – or, if the game seems deceptive, they cheat.
Either way, I constructed this and now need to deconstruct it for the summer, or certainly for the fall, and go back to the way I had it before – reference the rubric, then say what you’ve earned and why, and that’s it. In this case, flexibility was better than precision.
By Lisa, on April 8th, 2013% Instructor presence, though it can be defined more technically, is a perception that the instructor is there and available to the student during a class. Lately, in addition to my weekly summaries and guidance in the discussion forums, plus announcements and messages with individual students, I’ve also been using two other elements: an introductory video at the beginning of class, and a Voki introduction to each week.
Here’s the introductory video for this coming summer:
Here’s video on how I create my weekly Voki:
While both are clearly presentations rather than interactivity, I think the spontaneity is important in encouraging students to see me as a real person at the “other end” of the class. That’s why I prefer cheap and easy methods like these (webcam and iMovie for the video, free Voki for the animation) rather than more “professional” means.
By Lisa, on March 27th, 2013% It happened all of a sudden. The feed from one POT Cert Class participant just wasn’t coming into the Pedagogy First aggregated blog. I spent hours trying to figure out why not – the feed finder screen would just go blank on only her feed. I Googled, I pounded, I went through what there is of FeedWordpress documentation. Mostly I wished I were Alan Levine or Tim Owens.
I have mentioned before that technologies known for doing some really cool things are becoming unreasonably complicated. This particular technological problem rests on a self-hosted installation of the software WordPress (built and maintained by a wonderful community) and the FeedWordpress plugin (built and maintained by a wonderful coding person). When one gets updated, it often doesn’t play nice with the other. And I can’t fix it. I say again unto you, I am not a coder. I find code, I steal code, I envy code, but I do not code.
I finally asked that a new blog be created for this participant, and it seems to be feeding. For now. Of course, the other one had fed too, all of the first semester. Given my own significant limitations, we will not be able to do this again this way next year.
The recipe at the moment is this. Start with recent adventures with self-hosted Moodle, add this new self-hosted WordPress crisis, mix with a dash of cloud failure (Google abandoning Reader, Posterous closing shop, and SeesmicWeb being bought and killed by the inferior HootSuite ). Stir and cook with a big dollop of my recent participation in reviewing a publisher-created program for grading student essays, and you have the kind of disillusionment you get by realizing you have already been devoured by the whale but didn’t know it.
 The monsters (big proprietary systems, cloud-based sites, self-hosting) appeared to be separate, but were actually all parts of the same beast.
Self-hosting, a domain of ones own, the path of ds106 and the noble D’Arcy Norman – this has been the antidote to the bullying tactics of the LMS and publisher-created content. I have held it up as the way to avoid both big proprietary monsters and the vagaries of the disappearing web apps and fly-by-night cloud offerings. I have scoffed (quietly) at those who said they could not run their own blog, it was too hard. While I have not been guilty of encouraging anyone to run their own Moodle installation, I have persisted in doing it myself as a bulwark against Moodlling ignorance and exterally-run systems.
All this begins to seem like folly, a folly based on desire. An example: I want nested discussion forums where students can post multimedia, so I have Moodle. I find out today that (cloud-based) Schoology has nested forums! Yay! No! Wait! They are touted around the web as a “start up” of four years or so who use proprietary code (cue John Williams’ Empire Strikes Back music). I will have a free class, but never be able to access it otherwise, years down the line.
Fact is, none of these options are perfect, or even sufficient. The big LMS systems (including Moodle) upgrade and you can’t restore old courses and actually view student work – they say you can, but in fact it doesn’t work. I have all my courses backed up as Moodle .zip files, but now they’ve changed to .mbz. Out in the cloud, I can export my Posterous as they close down, but when I import it into WordPress a bunch of stuff is wrong or missing or ugly. These things weren’t built to be transferrable, or to cater to the archiving tendencies of the mere customer. Whether proprietary and exorbitantly priced, or open source and impossible to run without an IT degreee, none of the options have a sense of history, only a blindered vision of a future fulfilled by profits, market share, or geeky street cred.
Perhaps I am dissembling now to be running a class encouraging faculty to plunge into explorations of web tools and new technologies. I cannot in good conscience suggest anyone build a course around any of them. My colleague Todd Conaway says that it’s better to learn from creating, to meet the challenge of the occasional failure, to engage the technologies and learn from them even if they’re transient. I know that is true. But if you spend too much time in the belly of the beast (whether self-hosted, cloud-based, or LMSed) , things start to smell fishy.
By Lisa, on January 7th, 2013% I have three days left to prepare six classes, and I keep staring at the grading load for each and wondering whether there isn’t a better way.
As organized, each course has a 100-point system, with items like assessments and assignments each earning a particular number of points depending on their quality.
Traditional. Also dull. Also a trap, for me as much as for them.
 Back in 2010 I was whining about rubrics , and in that post I mentioned that my (now retired) colleagues David and Don Megill once said in a teaching meeting (we don’t have those anymore) that there is no such thing as an objective exam – all grading is subjective. They were right. So why the numbers, percentages and other trappings of objectivity? I should just decide their grade. The fact is, for about 85% of my students I can tell in the first two weeks what grade they’ll get at the end of the course.
So the traditional schema traps me into assigning points for things, and reorganizing the grading scheme depending on what I think should be worth more or less, and at what point it should be evaluated. Which means I have to evaluate right then, even if it really isn’t a good time for them or for me. That’s the exam date, so that’s it, even if half the class doesn’t have the essay idea down yet.
That sounds like a fairly progressive way to think about it, trying to avoid pseudo-objectivity. Now on to the really cynical part.
Last semester I learned that students don’t see grading as “feedback”. Only direct, personal, extensive commentary on everything they submit is considered “feedback”. All else is “grading”, and it doesn’t count somehow as communication. Correspondingly, any kind of group commentary (for example, posting a sample essay for comparison or commenting on the particularly good posts and explaining why they’re good) also doesn’t count as feedback, as I’ve noted in a previous post. This is despite the fact that when individual feedback is given on, say, an essay exam, they often don’t read it or don’t respond to it by changing their work in any way. My most positive results last semester, where students really changed their work in response to my commentary, happened as a result of them asking for personal feedback by chat during my online office hours. Without that very personal touch (with 240 students?) they have trouble seeing that any other form of feedback applies to them.
I’ve considered more individualistic grading schemes (letting students decide what percentage of their grade is assigned to each item, for example) but they all look like big bookkeeping nightmares at 40 students per class. I have forced them to read the rubric and evaluate their own work in the Contribution Assessment, which has been extremely valuable but still doesn’t prevent complaints about “not enough feedback”. The burden over the last few years has shifted from them trying to improve their work in response to my commentary to me trying to individually improve their willingness and ability to improve their work. Yuck.
So how about this – One Rubric to Rule Them All.
Course grading rubric
A = all or almost all of the time
B – most of the time
C = about 70% of the time
D = not often enough to succeed
F = rarely or never
__ Being in class on time every day (unless sick)
__ Taking notes during class or actively engaging the topic being presented or discussed
__ Contributing to discussion in a way that increases inquiry or understanding for the class
__ Completing all reading and homework at a level indicating significant comprehension of materials
__ Working at a high collegiate level on any assignments or writing
__ Applying creative thought to historical problems
__ Researching independently any vocabulary or events that are difficult to understand
__ Actively seeking help from the instructor or college services when needed
For self-assessment, they can mark on the lines what letter grade they’re getting in each, and average them out. Or I can do it. Or both. They can do this many times during the semester, to see how they’re doing. Then at the end, I assign a course grade. And that’s it.
I wonder whether I should give it a try.
By Lisa, on December 11th, 2012% The brilliant and knowledgeable Alec Couros is trying to organize etmooc, a MOOC about educational technology. He’s got a Google Doc, a team of people helping, a Google Community group , an #etmooc hashtag, and a WordPress website. Wonderful, experienced, exciting people have been making a huge number of suggestions on approach, and resources, and speakers, and so much more. During a meeting of some of the helpers, some choices arose about how to organize the course:
- Should it follow a syllabus (one was laid out originally, with topics and experts to invite to facilitate or speak or something)? or should participants organize it and/or determine content?
- Should there be guest speakers and webinars, or a move away from that model?
- Should it be like ds106, with a big database of assignments to choose from and lots of freedom and artifact creation?
- Should the course be geared toward beginners, people new to online learning and the open web? or toward experienced MOOCers and avid ed techies?
- Should there be any assessment? Badges? Self-assessment? No assessment? Formative? Summative?
- Should the topics continue over time, instead of stopping when the next topic comes up? (Alan Levine had some great ideas about this and reasons for why getting away from the whole “course” idea would be good).
- To what extent can a more organized approach be combined with a lot of freedom? Is there a spectrum between a typical university course and an open educational community?
Well….
If I were designing the course, my combination would be a main course with set topics that launch on a particular date (Alan’s idea), and after some basic introduction to the topic, development of a mini-community that continues with that topic.
For the main course, I would have a syllabus, start and end dates, and the list of topics, two weeks apart. Maybe 6 topics total so no one loses interest – a 12-week class. The strands happening now in the Google Community might work as topics: Connected Learning (this could include set-up), Digital Citizenship, Digital Storytelling, Open Learning, Tools in Context 1, Tools in Context 2.
The goal for each topic would be to develop a mini-community on that topic. The first task for each topic would be a collaborative document where participants put what they want to learn about (inquiry) and start listing resources (content). These resources could include materials from the “experts” who were normally be guest speakers, including their videos. If there are many participants in the topic, they could vote on the top 12 resources to focus on. The time for posting the list of questions and resources would be limited to a week or so, then reflection could begin on blogs and/or elsewhere (synchronous sessions, creation of artifacts, etc). For the sake of alignment, the facilitator for that topic would suggest contributions in a format appropriate to the topic (Digital Storytelling might look ds106-ish, Tools in Context 1 might suggest the use of Prezi or Diigo).
The mini-community for that topic established over the two weeks, we’d go on to the next topic. Again, brief introduction by the main instructor, then collaborative inquiry/question collection and resource gathering, then reflection and communication. And so on.
These topical mini-communities would each have their own space somehow, either as a Google Community topic or a WordPress tag or something, and at least one leader or facilitator (yes, someone would be in charge). That mini-community could continue long past the course or not, continue for as long as it stayed alive. Participants could come and go from the main course, participating in all of the topics like a regular class, or in just one or two.
So you’d need one central instructor (Alec, or me if it were my course), then at least 6 facilitators (teams might be better), plus as many participants as want to join.
I’d have no assessments, no awards, no assumption of the acquisition of mad tech skilz by participants. There would be structure for the main course and at the top level of each topic, but freedom in reflection, creation and community. There could be guest speakers for individual topics if the mini-community wanted them, but none for the main course.
So, after thinking about it for a few days, that’s what I would do.
|
|
Recent Comments