In addition to submitting a grade for each student, and a last date of attendance of they failed, we are now asked to assess the level of learning outcomes for each student for two elements: critical thinking and global awareness.
Our grade sheet is starting to look like a data entry form.
I have heard faculty complain that this is ridiculous and impossible – it would take far too much time to reassess each student’s class performance in outcome areas (last year it was just one) as well as their final grade.
I don’t think so.
I remember many, many years ago, we had a full faculty meeting about developing and tracking our first Student Learning Outcomes. It was the third or fourth iteration of this idea, and we were all sick of it – sick of hearing about this stuff that had clearly come in from the outside, through administrative fiat. And one of my favorite colleagues stood up and said, “Don’t we already have this? It’s called GRADES.”
I’ve never forgotten that. The grade I give means something. I spend a lot of time determining what percentage of the final grade counts for each assignment and skill. So does my grade now mean nothing when set up against outcomes? Do I really have to reassess each student for their demonstration of critical thinking and global awareness?
No, because these are built into the Course of Study, the class design, and my pedagogy. When I give that final grade, it says something already about the student’s achievement in critical thinking and global awareness.
The drop-downs have levels of achievement on these:
My default for a real passing grade (A, B or C) is “Practitioner – Met”. If they hadn’t met my standard for critical thinking and global awareness, they wouldn’t have passed the course.
My default for a D or F is “Apprentice- Not Met” if the student finished the class. If they stopped attending, it’s “Novice-Not Met”.
If I recall their work as being excellent, Critical Thinking jumps to “Expert – Exceeded”. Few get this designation – I am the expert, and few excel in either critical thinking or global awareness. But if they did, I remember it – I don’t have to look anything up.
Similarly, I recall other details leading to exceptions: the brilliant expert student who got a D for not turning stuff in, the B student who didn’t know where China was, etc. Again, no need to look those up.
So even though it seems burdensome, the process goes pretty quickly. Because I trust my grades.
I suppose the sign of an educated person is that they can learn from anyone and anything. This week I’ve learned from a publisher’s product, and the design it uses could solve some problems. The question is whether those problems should be solved, and whether this is the best way to do it. I’m tempted. They’re doing some very cool things, these publishers.
The product is an interactive textbook, with videos and little quizzes built into the page. They are taking the idea of proximity to its logical extent – everything that relates to the topic is together. The design is intended to force the student to interact with the material several times while on the page, in an effort to reinforce the reading. The reading itself has been scaled down. Each chapter has five or six sections, each section is about four scroll screens, with a single column, lots of white space, and multimedia as well as text. It is obviously designed to look good on a cell phone.
I confess to being impressed (I’ve seen this product demo’d now with two different textbooks), and tempted to adopt. I’ve asked our tech admin to find out how I can integrate this (and other) products into an LMS.
No, go back, don’t be tempted! But I am struggling with student retention and completion as issues the administration takes seriously, so I begin considering adopting this product. What it lacks in breadth it seems to make up for in depth. At the end of each unit, it has students write a reflection that connects the chapter to contemporary topics, and puts their posts into a discussion board. It’s a well-designed “learning system”. I do not buy all their crap about “engagement”, but it does force interaction with the material.
Structured as things are now, this product would replace the textbook. That’s what it’s intended to do. So what is the textbook for? If it’s to provide factual background information to my lectures, this is way bigger than that. It has its own pedagogy and its own interpretation of the material. It requires a different kind of analysis than a new textbook.
My existing course design
First, If I were to assign such a “text”, what would happen to the other elements of my class? These are:
1. My lectures – reported by students every semester as their favorite aspect of the class, my lectures are my interpretation of history and contain embedded primary sources, music, video, and my own voice and video.
2. Primary source research – the second-favorite with students, and my first favorite, I’ve written on using the discussion forum as a lab and I wouldn’t want to lose this.
3. Quizzes – My quizzes now include questions from lecture (including primary sources) and whatever I’m using as a textbook.
4. Writing assignments – I’m down to only five of these per semester, all based on the students’ primary sources in #2.
Since the self-declared reason students drop my class is “the class looked like too much work”, which of these is sacrificed for the more thorough online textual experience? The quizzes might not be an issue, except that they help make sure students are understanding the lecture.
Product location and service
Second, the product is located in a separate web location, in order to make sure everyone is paying for it. I’ve examined several publishers’ products now. Most force you to go outside into what’s becoming their own LMS. Only one lets you bring links in by chapter. I’ve checked out their LMSs, and they won’t work for the primary source forums – forum design is still the weakest area of ed tech, even after 15 years. Most products “link” or “connect” to Blackboard and Moodle, so a student has single sign-on, but the location of the material cannot be put “in” to the LMS in a way that’s seamless. This undermines the whole idea of proximity that is central to the effectiveness of the product. The lack of true integration means that these publishers aren’t yet in the 21st century (I still have to use a phone to call in for their webinars).
Also, because it is not my product, and not a supported LMS, it adds a third layer of possible technological problem and need for support. Publishers are famous for giving you the world until you adopt the product, then not being much help. And everything’s dependent on their servers.
Catering to bad habits
Third, what learning preferences are we catering to with such products? All of the webinars I’ve attended begin with the profs taking turns stating what their greatest challenge is in teaching the x survey class. The answers are totally predictable: underprepared students, getting students to read the text, getting students to use what they read. How do we diagnose these problems? Students aren’t doing the reading, or they’d do better in the class. We want them to do better. We want them to learn. At the same time, we don’t want to lower the standards of the discipline.
The solutions in this product, the depth-over-breadth approach, rely on the “current research” on learning. Well, not on learning, but on student success. Student reading attention span is short, so the solution is to “chunk” information and given them less content. Their reading level is low, so we dumb down the text and put in links to difficult terms. They like video (actually, the publishers claim they learn well with video – I have not seen that to be true in practice), so we add more (short!) videos. Their attention drifts from the text, so we force them to click to see this map, and take a little quiz, and click on the video, and rearrange these items, and do a bit of writing.
So the whole structure of the product is to cater to students who cannot create their own learning pathways, who are accustomed to having everything designed for them, who have difficulty reading and remembering, and who do not know how to study. We support all of these bad habits with this approach, but also use technology to reinforce some depth of understanding.
Weighing the considerations
I’m looking at three ways to go here:
1. Adopt: Foreground the retention concerns and adopt the product, jettisoning at a minimum my quizzes, and making lecture viewing optional. Figure out how to put it into Moodle so I can use the forums for primary sources. Or dump those too.
2. Redesign: Balance the retention concerns with my own pedagogy, by adopting the useful elements of the product using my own technology skills – putting mini-quizzes and pop-up definitions inside of lecture, and dumping the DIY textbooks I’ve been using. This would be, obviously, a huge amount of work.
3. Keep Calm and Carry On: Ignore the retention concerns and continue with my design, which requires extensive reading, weekly 25-question quizzes on lecture and text, weekly primary source posts, and five writing assignments based on these, a workload far less than what I did as a freshman, but which is increasingly becoming anachronistic in a world of weekly log-ins, minimal reading, low-stakes self-checks, and low grading standards.
I confess to being tempted by #1 for the first time in my career. Undertaking #2 is more like a sabbatical project, and could take all my time, but I’d like to explore the options in future posts. #3 is of course the default, encouraging my own bad habits.
The dark side does have cookies. They taste better now, even if they’re not good for you. And we seem to be in a world where everyone just wants dessert, higher grades for less work. Whither the artisanal prof who cares about her field?
Yes, I change stuff every semester (Jen Dalby likes to watch me prep, cuz it makes me tweet crazy things). Here’s what’s on for spring semester:
Expectation zone: percentage changes
I believe students expect the quizzes (based on reading, and factual retention) to count for more. This semester, quizzes are only 30% of the grade. I’m raising it to 50%. Primary sources, which they like and which require some research, have been only 20% – I’m raising it to 30%.
Back to the future : shorter and more frequent writing
Having gone the other way, from weekly writing, to every other week, to six assignments, to five, I am going back to the way I had it at first – weekly writing. The big change is that it won’t be as formal, even though it still scaffolds up to the final essay. The other big change is it won’t be “graded”. Rather, writing will be part of the “A bit of writing and conversation” forum. Students will be encouraged to post a short piece of writing based on their choice of primary sources, and converse with each other.
But I did it!: contribution grading will be based on completion
Instead of the Contribution Assessment, which I have used successfully for awhile (I admit it, I get bored), I am counting 20% of the grade (10% for each half of the semester) for the writing and conversation, all together. I intend to base this primarily on completion, but want flexibility for quality issues and to touch base with students in the middle of the class, which is what I liked best about the assessments. If I change my mind, I’ll do self-assessments.
Autobots roll out: automation for completion
I am turning on the Activity Completion feature in Moodle, and using it for the first time. The “bit of writing and conversation” is a checkbox on the main page – a single post marks it complete. Same for the primary source. For the quiz, completing it and getting an (automated) grade completes the task. Very few of my students fail the class because they can’t do the work – most fail because they don’t complete the tasks. Since I believe the tasks are important elements of practice, each one is low stakes but all need to be done. The green check (or lack thereof) on the main page will be obvious.
What I’m not changing
In a survey earlier this semester, students indicated that what they liked best were (1) my lectures, and (2) posting their own primary sources. So those aren’t changing at all. They didn’t like discussion, so that’s now optional and part of the writing forums.
But why change anything? (apart from, like, the boredom thing)
I am trying to answer a concern I have about this semester – the drop rate has been very high in all but one section. The drops mostly came at the beginning of the class. When I surveyed students who dropped, they told me the class was too much work. Interestingly, some wanted a class where they only had to log in once a week. I have not reduced the work at all, which I may regret later, but I am going to dare to use my experience to interpret what they’ve said.
I think part of the problem is that the class looks like too much work on the main page. Using Activity Completion makes the work more instantly visible, so they can see their progress and what’s lacking immediately. I am also removing the “labels”, or titles, for each section, so there is less text (a shorter list) to see for each week, while keeping the interactive syllabus format. For those who think they can do the class only logging in once a week, they can if they’re smart, by taking the quiz, posting their source, and posting their writing all on the same day. Since I don’t plan to “grade” conversation, if someone really doesn’t want to participate or talk to anyone, they don’t have to.
As with everything, we’ll see what happens!
I recently acquired a MacBook Air, running 10.9 (Mavericks), because my old MacBook (running 10.6 Snow Leopard) was struggling with ordinary tasks. (I think I skipped the leonine versions, 10.8 Mountain Lion and 10.7 Lion).
After clicking on all the “skip” boxes to prevent linking the new laptop with my Apple ID, I proceeded to add it to my household network, which is wired. It allowed me to access my local hard drives and my desktop Mac. But it will not let me print unless I add the IP address of my locally networked printer to iCloud. Um, no thanks. I guess I’ll walk that flash drive across the room.
I tried using TextEdit, and my new laptop warned me it could not save back to the networked computer. I also noticed there was no “Save As” command at all. It took much searching to find out I now have to press Command, as if “Save As” is an unusual thing to do.
It became apparent that Apple wants me to use iCloud. For everything.
Russell Lee, Highway scene, Idaho County, Idaho,
July 1941, from Photogrammar
And so the fog rolled in. I could not install my version of Microsoft Office from my old computer. Or my old versions of anything else. This is the first Mac not to let me drag, drop and use those wonderful old apps that always worked – GraphicConverter, VisualHub, iMovieHD, etc. – regardless of felinity.
As I’ve continued to work with this sleek sans-CD-drive lightweight little miracle, it’s become clear to me what’s going on. There is pressure on my workflow to rise upward and become cloud-like. And it isn’t just an Apple thing. Microsoft is doing the same with its Live stuff – my college’s email is in the cloud now. I can’t buy a boxed version of Office, or Dreamweaver, for the new laptop. Googly Chromebook wants to always be cloudy – it takes special apps to work offline.
I think it’s clear (or rather, foggy) that future consumer computer systems will consist of only a few components: the Processing Unit, an input device (gestural or mouse-like or keyboard-ish) and a networked printer. There will be no wires (only power cords) and no ports on the PU (I don’t say CPU because it won’t be central to anything – it will be a screen with a motherboard).
All your work will be saved to the cloud. All communication with your printer and phone and other devices will be done in the cloud or by Bluetooth-like wireless communication. Your USB hubs and LaCie drives will end up in the garage with the SCSI peripherals and floppies. There will be no need for local files or storage, except for temporary files waiting for the wireless connection to come back on because service was disrupted (kind of like battery backup). There will be no on/off button for the wireless on any device – printer, modem, and PU will all have wireless connections that are always on.
All your family photos will be uploaded into the cloud, conveniently sorted with facial recognition. Your banking transactions will float above the neighborhood with those of everyone who lives near you. Sex chats and Snapchats will drift in the air like confetti. If you are a scientist or someone working on something that needs to remain secret until it’s published, you will find it challenging to keep updating your passwords and praying that cloud services won’t have a security breach like Target or Home Depot.
I, for one, do not welcome our wireless, cloud-based overlords. If I want a machine to which I can privately save my work, neither the hardware nor software will be there to support me. All my recent advice about saving a copy of your stuff is about to become impossible to follow.
I will be very glad of one thing – my manual typewriter. It may be the only way to ensure privacy, and bring in a little sunshine, five years from now.
Lecture is not about discovery, unless it is the discovery of how the professor processes and uses information. It can be excellent modelling. When I talk through a historical subject, and its significance, I can model how historians think.
But most professors lecture in order to relay information. This always seemed silly to me, since the “information” was in the book. Now the “information” is on the web. But if they don’t read the information, and understand it, they can get it from the lecture.
In my online lectures, there is indeed information. It is told from my perspective, and everything about it (including what I choose to discuss) represents my interpretation. Because it contains events and dates, and explains them chronologically, it constitutes “content” in the class. My online lectures include my own writing (recorded in audio), links to websites, embedded video, and specially marked links to primary sources.
These sources were originally collected in a paper workbook. Now each unit’s collection is on a web page. Some of them have audio reading the documents (as I’d like to do with Edmund Burke). Questions on these documents are included in the lecture quiz each week.
In “discussion”, my students do not discuss, but rather post their own primary sources, then write about several of the ones posted and the way they tell us something about that era. Through this process I teach historical writing (thesis and evidence).
This week something interesting happened.
In my Depression lecture for US History, I feature a section on Orson Welles’ War of the Worlds, his radio broadcast from Halloween 1938:
But not until a student posted it as a YouTube video did any of them, as far as I can tell, actually listen to it.
Then this happened:
So I’m thinking. No one went to find the audio before, but they watched a video (that’s really just audio) posted in the forum. Someone was interested because a fellow student posted it. Sure, she posted it because it was mentioned in lecture (maybe), but a student actually experienced it because it was posted by another student rather than by me.
So….back to those other primary sources, the ones I actually assign. Those written primary sources are posted by me as part of the lecture. I suspect few actual read them except to answer quiz questions.
Perhaps if they discovered them instead of me providing the sources? We already do that – they find and post sources every week. Maybe I shouldn’t select sources at all. Perhaps the collection they make is fine – especiallly if they actually look at or listen to what the others post.
I must think on the implications of instructor-provided content. We have this idea that instructors need to curate content. I could do that in a different, more engaging way. But first I need to be sure that, at least when it comes to primary sources, I should be doing it at all.
Yes, it’s supposed to be to share pinboards, like a edu-Pinterest. But that’s not what I need. I need a presentation tool.
Trouble is, I’m not very linear anymore when it comes to lecturing. I used to be. I used to have outlines, both on the overhead projector (my specialty is medieval technology) and on my notes (notes? what are notes?).
But over the years I’ve become less linear. But I’ve also become more visual. Powerpoints don’t do the trick – they’re linear too. I don’t like going back five slides, or keeping to multi-slide view open so I can see where to go next.
I tried CoolIris for this a year or two back. It was kind of OK, but clunky. Had to be installed on my classroom computer, which has the RAM of a small treadle sewing machine. I tried installing it on my own server. But then I had to download and upload folders and, well, those little tiles were kinda hard to see unless you moved the mouse just right to see down the…line.
So, pinboards! This is great, I thought, when I first saw Pinterest. So I tried it, but it compressed the images, and when you clicked on them it didn’t make them that big, not in good enough resolution for that big screen at the front of the classroom.
Along comes Educlipper, and that’s working better. I can add not only images from wherever on the web, but pdf files (the homework and documents) and video clips (don’t want to miss that cool Darmok episode where Picard does Gilgamesh).