We aren’t assessing learning

In a presentation yesterday for the Learning Analytics and Knowledge open online class, Gardner Campbell argued that we are engaging in a factory model of student education, rather than one that reflects the complexity inherent in actual learning. In my favorite section, he uses analogies such as a daily pill box and cows feeding at a trough to convey his points.

Campbell went on to discuss the ways in which the tools of learning analytics we use (rubrics, student learning outcomes, objectives) reflect and sustain this industrial model, by establishing pre-set goals and tracking “achievement”. These methods cannot track learning, which is much more complex.

In addition to being, as Gardner is, just wonderful, I think this is good logical ammunition against the belief that we are actually assessing learning with our “learning management systems” and other tools designed to organize, manage and measure. It’s important to understand that our measurement systems reflect a linear, simplified model when we engage in the assessment of student work. It also goes to the heart of measuring our own effectiveness as teachers.

If learning is complex, messy, riddled with failures that lead to successes, then to assess it using methods that are simple, linear, and concrete is rather bizarre.

[Gardner's full talk as a Collaborate recording is here]

Middle ground: web 1.5, sets and more

This week I have been attending to the various discussions around the recent work of Jon Dron and Terry Anderson, including the Hot Seat forum related to the 8th International Conference on Networked Learning (no, I’m not going to Maastricht) and Jon’s presentation at the Change MOOC (the web space for this is here). Sources also include Dron and Anderson’s 2009  article Lost in Information Space: Information retrieval issues in Web 1.5.

I am intrigued and delighted with an approach I see as middle ground between the Web 1.0/closed classes/LMS/hard tech/group focus and the Web 2.0/open education/network/connectivism focus.

For the former, the model seems to be formal groups of people, such as a class, using closed but comfortable ways to learn online in a manner prescribed by an instructor, with the result being a grade.

For the latter, we have the open ideas of networks, learning taking place in weak and strong connections, with softer technologies that provide for more creative work. Looser forms of assessment (such as badges and feedback) follow this model as well.

In the middle of this dichotomy is what Dron and Anderson  call Web 1.5, and what they refer to as “sets”, unintentional collectives of people who learn within a shared interest. Thus there is a middle ground between the horribly limited, walled garden system and the wild, diffuse, scary openness of networks.

These discussions are especially timely in light of my own efforts to find such a balance between closed/open, safe/scary, hard/soft in designing my online classes for next semester, where the unscary stuff (my own work, such as lectures, information, readings) is completely open, but the scary stuff (anything graded, such as student forums, quizzes, and essays) is closed in the Moodle LMS.

It is not, as I’ve noted, an ideal balance or compromise, because unlike the Elgg VLE used by these authors, I do not have granularity of permissions where students may choose exactly what elements of their work are open or closed.

I had thought that creating a broader “group” in Facebook (of all my students, not just those enrolled in one class) would design some openness for them, but of course I have been reminded that Facebook is closed too. It’s just an LMS with advertising, and a flat social space. On the other hand, at least there isn’t any hierarchy, and perhaps there is comfort there within a more open way of communicating, if not a more open role.

As much as I love the far out, wild west, final frontier elements of open web-based learning for my own use, I do not think my students are ready to the extent that I could handle helping 240 of them manage it. At the same time, the limitations of LMS-based education drive me crazy. So it’s productive to envision Web 1.5 pedagogies that could, in Jon’s Goldilocks analogy, get to a point of “just right”.

 

 

Comment feed bundle to get around the GFWC

We’ve had a problem in the POT Cert Class at Pedagogy First! One of our mentors,  Brandon Davis-Shannon, made a Google bundle for all the feeds coming in from everyone’s blogs, which is great. Plus he posted for everyone on how to use them.

It turns out that edublogs is blocked in China, so our participants there can only read the posts either through Pedagogy First! which is aggregated, or Brandon’s bundle. But they can’t read the comments (or post any) so they’re missing the conversation.

This morning I took Brandon’s idea and began making a bundle for comments, by going to everyone’s blog to grab the comments feed. This proved slow to do individually, and difficult because some don’t have an RSS feed for posts, much less for comments. So I began doing this (change to 480p if you actually want to see it):

The bundle is here. But there were a couple of problems (listed to both warn people and ask for help!):

  • Posterous — I couldn’t find a comment feed for Posterous blogs (thus two blogs are missing from the list)
  • Tags and categories — while it’s not too bad to find the comments feed for all WordPress and Blogger blogs, and not too hard to find feeds for tags or categories, I could not figure out how to do both — get a feed for comments that are responses to posts only in a specific tag or category

Flipping research

“Flipping” a class (putting lectures and presentations online, then using class time for collaboration and discussion) is all the rage at the moment.  Everyone’s in on it now, from The Economist to The Daily Riff to Daniel Pink. I don’t see the big deal, since basically using a textbook is the same thing (read at home, then come to class). Maybe in the 16th century people were wowed by this idea. I’m glad it’s come back around – I don’t like using class time for relaying information you could find elsewhere. You want to know when the Battle of Waterloo was? Go look it up. It’s easy now to look it up, so it’s easier to justify not lecturing on the facts unless I’m presenting a different paradigm. I think I started doing that, what, fifteen years ago?

But I want to be more innovative, so I’m doing flipped research.

Here’s the idea. I’m referred to in education research circles as “just a practitioner”. I do stuff. I teach history at community college, and try web technologies to help me do that. I use learning management systems, video, slidecasts, whatever it takes. I’ve done everything from Webboard to WordPress to Google + hangouts.

But I’m not an expert in educational theory, nor in the entire edtech field that I’ve watched emerge (and achieve frightening proportions) over the last decade.

So now I’m doing something really cool: offering an open, online class to prepare faculty to teach online. It’s designed like a MOOC, but it’s not a MOOC because it’s not massive (90 enrolled, 81 blogs aggregated). I’m taking ideas from other people, including Alec Couros’s EC&I831 for the structure and the mentoring, Jim Groom’s ds106 for the aggregated blog and the element of insanity, George Siemens and Stephen Downes’ whatever-MOOC-they’re-doing-today for the large topic and distributed conversation. But I haven’t done much “reasearch”. I do read articles on learning management systems and MOOCs, but basically I’m … just a practitioner.

So now I’m writing a research paper based on what I’m doing, and it all feels like reverse engineering. I’m looking up studies on open classes and professional development for online faculty, and finding a few people doing what I’m doing. Some of the articles show findings that what I’m doing might be good; most end with “further research is required”. I’m doing that research, survey participating faculty, planning to note how approaches change on their blogs.

For a historian, this is so not what you do. You create a provisional thesis based on the reading of primary sources. I guess in this case, according to the educational research folks, the primary sources are the studies. They should have guided what I do, and I should have planned my class based on the findings of others. I should have been consciously aware of what everyone else was studying, not just doing. Instead of pulling cool ideas from Alec, Jim, Stephen and George, I should have read a bunch of studies on the ways of doing this and built my class with a solid theoretical and practical foundation.

But no, I just did it. I created the class based on my experience rather than my research. I’ll probably be kicked out of the historians’ club if they find out. And that’s what makes me…just a practitioner.

A funny thing happened on the way to the…

We don’t have a forum at our POT Certificate Class SMOOC (Small-to-Medium Open Online Class). Nor a Google Group (though that’s been suggested and may appear), nor a Moodle forum, nor a Ning.

When I took EC&I 831 from Alec Couros last fall, I felt we needed a forum, and he and I looked for one we liked, but nothing we found had the particular combination we wanted (no ads, free, nested comments and the ability to post multimedia), so we gave it up. On his current page for this semester, he’s listed many types of communication, but not a forum.

And now I notice that the Change MOOC, co-hosted by George Siemens, Stephen Downes, and Dave Cormier, has a forum, but it’s kinda awkward. Not like a full-featured thing at all. On the main page, there are three Participating links for blogs, two for the newsletter, one for the backchannel, and one for threads. The class starts tomorrow and there are three posts for a class of hundreds.

And yet, all my college history classes feature forums as the heart of the action. I couldn’t do without them.

So why no forums for these classes? I think because the idea of distributed communication has become something we’re all taking seriously.

If we’re going to focus on the wonders of learning on the web as our topic, it makes sense that our participants should be communicating there rather than in a central location. We not only want them communicating via blogs, wikis and Facebook, but we also want them creating their own places and spaces (which is why the potcert11 class may yet have a Google Group, because the participants may be creating it — and they’re starting the Twitter thing, not me).

chart
From Rick Schwier’s Pursuing the elusive metaphor
of community in e-learning environments

I also think there’s more staying power to a distributed community. If you have one space, as we all know from using learning management systems, the course seems finite. Even if the space is available after the class is over, the feeling is still that the class is over. But if the conversation is distributed (our class has a very active Facebook group, which surprised the heck out me), there’s a feeling of perpetuity. The river continues and you step in and out when you wish.

As Rick Schwier has noted, informal communities have higher levels of engagement over time. I wonder whether de-emphasizing a formal location for conversation makes these classes lean more toward informal learning, even though they are organized as classes, with set timeframes.

So come on, hit me with the research. Do distributed communities last longer? Are we right to “force” them instead of offering a rich place for a single discussion?

Freaking out? a story of baking bread

See this? It’s bread, just good home-baked bread.

So what’s the big deal? It’s that I made it.

You may not know me well, but if you did, you’d know I’m no Dora Domestic. In fact, I used to be somewhat famous for not knowing my way around a kitchen.

But I decided I wanted to make bread. Might have something to do with medieval technology, or just wanting to do something self-sufficient.

It took awhile. I made some doorstops. At first I thought I couldn’t make bread, that I had no talent for it. Other people didn’t help much — they told me to keep at it, because bread was hard to make. Oh no, I thought, it’s too hard. I stopped for awhile.

I kept looking for a good recipe, better yeast, how to get the water exactly 115 degrees. I found a simpler recipe, just water, flour, yeast, salt, sugar, oil. It came out good. I started to tinker with it. The bread got better each time. Now I make it a lot. And it’s easy. I don’t use measuring spoons. It’s peaceful to make it now, instead of stressful.

It wasn’t hard. It was new. That made it seem hard. But I kept doing it, looking for ways to make it better and easier.

Are blogs, web pages, and teaching online new to you? Keep doing it. You don’t need talent, a feel for it, or strong hands. Just keep making the bread. Soon it will be good.