The centrality of the textbook

It is an axiom frequently ignored that any technology has to have a reason to exist in a class. The textbook is a technology. If one were to actually read it, that would be a huge investment of time in an era where attention is continually diverted. A chosen technology is either central to learning or it shouldn’t be there at all.

Students, understandably, won’t read the textbook without stakes (a quiz, a paper). Increasingly, many students cannot sustain the attention and access the vocabulary skills required to read one.

As a result, publishers and professors have developed ways to force students to read the textbook. Publisher’s courses read the book aloud to the student, provide embedded quizzes and pop-up vocabulary as they go, and assess performance before pushing the grades into the Learning Management System. Annotation systems like Perusall make it possible for students to annotate a textbook together. These approaches are far too much if we only want the textbook as background.

I confess that I’ve edited several textbooks of my own that I use in my classes. These are classes where I have substantial lecture material, so the book is context. For the first two years of the pandemic, I made reading them optional and eliminated book quizzes, but now that students are more accustomed to online learning, I’m bringing them back, with some regret.

             my new OER textbook

This regret is coloring my view as I design two “new” World History classes (I’ve only taught them in the classroom and that was many years ago). For the first, I’ve spend the last several months with the Cengage textbook I’ve chosen, and made it central to the short recorded lectures. In these lectures I explain the chapter, note its strengths and weaknesses, clarify points. The lecture itself is quizzed internally, using the quiz function in Canvas Studio. Class starts in a little over a week. I hope it works, but either way I’ve made the book central to the class.

Now I think about the other half of the course. For this one, world history to 1500, I have found an Open Educational Resource, a free textbook. Of course, it’s only free in its electronic form. So I’m thinking how to use it, since it must be central. I have no lectures prepared for the class, and am not sure I want them.

So, a new idea. Since it’s free and electronic, I could put it inside Perusall, the social annotation program. But instead of expecting students to annotate, or requiring that they do it (as I do with primary sources), perhaps I’ll put in the annotations little videos of me, glossing the text myself that way. Perhaps I’ll ask questions, invite participation, and grade it in Perusall.

In the old days we turned up our noses at “teaching from the textbook”, ridiculing those who tied their lectures to it. Perhaps we felt that we could leave students alone with the textbook, and they’d read and understand it. I doubt this was ever true, but in a world where we can chose to eschew the textbook entirely, create ungrading schemes, and have at our fingertips more resources to share than ever before, we should consider the textbook differently.

The increasingly possible: online labs

Ever since I discovered the back pages in H. G. Wells’s Text-book of Biology, (1893),  I’ve known a bit about scientific work being done at home. Doing “practical work” at home was important for correspondence education at the end of the 19th century, so that students could study for examinations even if they didn’t have access to a laboratory.

When online teaching started at our college, back in 1998, and began to grow, a number of science instructors were concerned. You could do a lecture online, fine, but you couldn’t do a lab. Simulations weren’t enough, they said. You need real materials. Wells’s students, of course, had real materials. They ordered them by mail or, in the case of frogs, went and caught them.

So here we are in 2020, with online labs foisted onto unsuspecting faculty, and they’ve done brilliantly. I attended this session, where four professors, from auto shop to biotechnology, showed how they do labs online. For inspiration alone it’s worth the 50+ minutes.

MiraCosta College: Hands-On Labs in an Online World on Vimeo.

It’s an odd feeling for me, a promoter and practitioner of online education since the 1990s, to see that the materials (lab kits, go-pro cameras, etc) have come so far. But it’s even more thrilling to see the new attitudes, confidence, and willingness to serve students this way. It feels (finally!) like the new world we were hoping for.

As I watched, I thought like a student. In-person education isn’t always the best way to learn. When I was 19, if I’d had a way to learn about cars without having  to show my ignorance to the guys in auto shop, I might know how to fix my car today. I was shy, and had already been subject to sexism in art class — I certainly wouldn’t walk into a guy-dominated shop. I was also clumsy, but if I’d been able to make mistakes with those test tubes at home, I might have given it a try in high school or college. (I actually had a chemistry set as a child, and created something so horrid the chemistry prof at the local university had to be called so we knew how to dispose of it.)

At any rate, I think H. G. Wells would be proud. I’m delighted.

Modified Ungrading

I have written a lot on this blog about grading, about my longing for “ungrading”, my qualms about heading that direction, and the things I’ve decided to do instead. This is a more specific post about the latter, and my gradual shift in emphasis I’m calling “Modified Ungrading”.

We know the arguments against A-F grades. They encourage a focus on marks rather than learning, and they can be discouraging. We know there are also arguments in favor. They rank performance against a standard, they inform the student of their level of success, and they mark improvement. (Many years ago I heard a speaker say he did not want to drive across a bridge built by someone who got a C in engineering, or be operating on by someone who got Cs in medical school. I get that.)

This is an area where faculty workload and student desires can profitably intersect. Most of us teach large groups of students, and at community college we do it without teaching assistants. The big argument in favor of ungrading is that students benefit more from individualized feedback than from grades. But in this environment, tutorial-style feedback and individualized learning isn’t possible. Auto-grading, with current technology, is possible, and helps fill the gap between grading and ungrading.

In most learning management systems, when a student submits something, a certain number of points can be automatically assigned. The arguments for and against doing this are the same as those for grading in general. But in addition, auto-grading provides immediate marking and a sense of completion, two factors which are often ignored in discussions of grades vs feedback.

So far, I’ve implemented auto-grading for two types of assignments, both of which are submitted only to me: lecture notes, and document paragraphs. What this enables me to do is have 2 points automatically assigned, then go back and look at the work itself. Sure, I could assign the points when I look, but auto-grading instead changes things, for me and for the students.

For the students, they get their two points immediately, so it’s confirming they did it. For me, I then have ample time to go back and peruse their work. And with the points already given, I can pay more attention to my Assignment Comments (feedback). The tendency is to talk to them first, then think about the grade, instead of the reverse.

I’ve had to change the grade sometimes, but I’ve had the chance to explain individually before doing that, and to give an opportunity for corrections.

This shift to Modified Ungrading may seem ridiculously subtle, but it’s quite real. Mentally I can shift from “processing” the work of 40 students per section, to responding to it personally.

In fall, I intend to apply this to discussion forums, which we use for posting primary sources. Ultimately, I would like everything auto-scored except writing assignments, on which I can spend more individual time.

This way, I can focus my work more on communication than grading. And when students see their points immediately, and rarely changed, I’m hoping they will do the same.

Perusing pictures

I almost forgot, in all the madness, that I am trying an experiment in my History of Technology class. I detailed the idea in my post from July. But I hadn’t implemented it till this semester.

First a review: in every class I teach online, instead of a traditional discussion board, I assign a set of primary source documents. I put these in Perusall (using the LTI in Canvas), uploaded as a pdf. Students then can select parts of the text, and annotate it. They can respond to each others annotations, and add images or video to help each other read the material.

This semester I’ve tried it with images. I put together two sets of images in Microsoft Word, one for images from the Book of Hours of the Duc de Berry (for the Middle Ages), and one for a collection of postcards from the early 20th century imagining the world in 2000 (for the turn of the 20th century). I then saved each collection as a pdf, and uploading them into Perusall.

I’ve just been reviewing the latter. The History of Technology class is always difficult to get talking. The class attracts a wonderful assortment of students, particularly those in computer science or who already work in information technology. They don’t tend to be much for chit-chat, and some of the written articles on technological history don’t interest them. The pictures, however, have created much more participation.

You can see here that not only did they comment on the images, but that they also replied a lot to each other (the bubble with the number is replies to the comment showing).

So I’m calling this one a success, and plan to do more!

Disaggregated knowledge and the LMS

The advantage of teaching so many classes online is that I see patterns in student messages that lead me into larger issues. This one is HUGE. It’s not just about Canvas. It’s about the decline of Western education as we know it.

I stopped using Modules last term, because they “flattened” the elements of my class, making it appear as though each were of equal worth. Modules also forced students along a linear path of that week’s work.

I instead chose to keep my weekly pages, which list the things we do each week and when they’re due. I use bold for the higher-stakes assignments. Canvas automatically puts my due dates on the Calendar, and thus populates the students’ To-Do and Upcoming lists, which appear on the main (Home) page.

Over the years, more and more classes have switched to Canvas, so the average full-time student at MiraCosta would have four classes in a term. What the Canvas Calendar does is acts like any other calendar — it lists the tasks for each day or each week or each month. On the student Canvas app, it shows the To-Do list for each week from all their classes.

Sound convenient? It is convenient in the same way that bottled water is convenient, and that credit cards are convenient. It undermines traditional relationships globally, and creates a sea change.

Yes, I probably sound crazy saying that the Canvas Calendar represents the decline of Western education as we know it. But bear with me.

This week, the first week of class, I have had an unusual number of students message me saying they missed the assignment because they didn’t “see” it. By probing this, I’ve discovered that they mean it isn’t appearing in the To-Do list. This is regardless of the fact that I did check to the box to add these items to the To-Do list (I”ll check that technical issue later). I quickly responded with the yellow highlighted note on the Home page you see below, but I still was getting apology messages for missing work they couldn’t tell they needed to do.

This morrning a student wrote me saying she was sorry she missed it, but the primary source assignment wasn’t on the To-Do list. I sent a student my screenshot in Student View, showing that the assignment was indeed appearing on the list.

 

She replied with two screenshots where it wasn’t there. Here’s the one she sent from her phone:

And it suddenly hit me. The process she’s accessing, the To-Do List, lists all the tasks for all the classes a student takes. It thus disaggregates the courses entirely. She’s no longer taking my History class, or a Sociology class. She’s just doing work, clicking links, crossing things off a list.

By showing the student the tasks for the day, for all three of her classes, Canvas has not only reasserted its contention that all learning tasks are equivalent, but that they are tasks unrelated to anything else. They are just stuff the student needs to complete.

Most scholars think in terms of their field, then teachers think in terms of wrapping elements together to encourage understanding. On my weekly page, you can see that the tasks for the week relate to each other. They are all part of that week’s topic. They follow sequentially: first post the primary source (forum), then check it for points (quiz). My design has instantly become irrelevant.

My practical response today has been to go through all my classes, adding the weekly page to the To-Do list, as the first thing that week. It will be tricker to do this for my lectures and other non-graded or linked items, since Canvas doesn’t “see” those at all. I will have to link each on a Page and put the Page on the To-Do list, forcing students to click twice to get to it. This will take all weekend.

But my holistic response is much more important. The units we teach are no longer units — they contain no flow or contiguity when seen as disparate tasks. If students access all academic work as a flat list of tasks, there is no connection between assignments. There is no connection, for example, between Reading 3 and Quiz 3. Assign the Reading for Monday. Assign the Reading Quiz for Wednesday, and it isn’t clear they relate to each other.

This explains the other messages I’m receiving. “I see we have a Lecture quiz due, but what is that on?” At first I smirked and thought, “The Lecture, of course!” But now I realize they don’t see the Lecture unless they’re on the weekly page. “The Calendar says the second post is due – where do I post?” You can’t put two due dates for the same discussion forum. They don’t know where to return to in order to post.

In an age when we worry that students don’t read whole books, we have something here that is much worse. How can they do sequential and scaffolded learning when the system won’t let you scaffold?

It changes the rules utterly. Here are the “new” rules (some have been good practice for awhile):

1. Assessment and responses must appear with the content.

Quiz 3, in other words, must contain Reading 3 within it. You can’t have a link for Reading 3 on Monday and Quiz 3 on Wednesday.

Note here that group text annotation, of the kind I’m using in Perusall, is ideal. The content and the activity are inextricably linked.

2. Double-level discussion may not work.

In Canvas, to have students return to a discussion, it will be necessary to link to that same discussion in the Calendar later in the week.

One alternative will be to have the entire class inside a discussion forum. This won’t work in Canvas because it doesn’t allow real threads, but might work in other systems.

3. Navigation schemes are useless.

Obviously, my own weekly page navigation, even if it’s on the To-Do list, is worked against forcefully by Canvas.

Some would say return to Modules. But Canvas’ own Modules are irrelevant, except for adaptive release, or to force task order. Students won’t use the Modules page either, even if it’s the main page. They may never see it.

This also applies to the Home page itself, especially a nice one. It is now obsolete. All we’ve learned about making the Home page welcoming is irrelevant.

Again, the new rules (and I’m sure there will be more as we all think about it) are the result of the disaggregation of content and tasks. This is both an effect of the technology, and a cause of the disaggregation of knowledge. We’d better plan accordingly.

Ending the reading/quiz cycle

I certainly didn’t mean for the relationship to end suddenly. It has been tenuous for awhile, various arguments and complaints, but I always thought it was a communication problem. But finally I had to walk away.

In my History of England class, the textbook has been an issue for a long time. The most suitable and available academic text, by the Messrs Roberts, is two volumes, so that won’t work. The class is, unfortunately, only one semester (we have two for the history of the U.S., with a much shorter history). And realistically, even one volume is asking for trouble.

All the books I’ve used, including works by Roy Strong, Asa Briggs, and F.E. Halliday are outdated, out of print, or both. These have been replaced by volumes that sell well in the U.K., but require a previous knowledge lacking in American students: works by Jeremy Black, Simon Schama, and Simon Jenkins. I’ve been using the Penguin Illustrated History, which I love. It’s visual, I’ve written a bunch of quizzes for it, and it’s beautifully written. But students have become less enthusiastic, and it’s outdated now anyway.

Besides, it costs money. All my other classes require only my own pdf textbook, freely downloadable and printable. We are now required to indicate in the class registration system if we have a low-cost or no-cost class, so that automatically creates competition among classes as students realize they can take a class for less money. This was my last class with a textbook that had to be purchased, and it wasn’t easy for the bookstore, or students online, to find an inexpensive copy. So I unceremoniously dumped the textbook, and spend a couple of weeks this summer downloading, editing, and reformatting appropriate pages from the web. Then I wrote matching quizzes.

Canvas is not user friendly when it comes to importing, editing, and reusing quizzes, in any format. It’s test banks are obscure and hard to use. The result of my machinations was a set of single-question, matching quizzes for the new readings, and my old (now five-question) multiple-choice quizzes for my lecture. So I put one due Wednesday and the other Sunday.

Well, now the weekly course page was becoming really cluttered:

I don’t want to go all Copernican on this, but if I were a student it’s starting to look like hoops to jump through, instead of ways to explore material.

So I thought, what do I want them to get out of the reading anyway? My lectures already have a good outline of the main events of English political, social, and cultural history. And the depth is already provided by the documents we “read and discuss” (i.e. annotate in Perusall) each week. These readings are pretty intense for today’s community college students (Magna Carta, Archbishop Cranmer). So instead of adding new readings, it may be better to have them deal with these documents more.

I spent yesterday deleting all the quizzes, both lecture and reading. Instead, I’m having each student submit a “Document analysis paragraph” which uses all the names of the documents to support any single idea they have about the era we’re studying. I made a silly sample paragraph to model what I want them to do:

It’s in the form of a single-question graded survey, which will automatically apply points if they turn it in, so I can read them at my leisure and communicate individually with students who are struggling. And now students are doing, not reading and quizzing.

And thus I’ve broken the entire reading/quiz cycle in one swoop. I didn’t set out to do this — it just happened. But I’m pretty sure I’ll have no regrets.

 

Class annotation of images

This is another post where I share how I did something, solely so I don’t forget how to do it.

Perusall is a wonderful program for annotating documents with a whole class, and I’m currently using it for all my online classes, which are located in the horror of an LMS they call Canvas. I upload a PDF, and students and I can highlight the document, with a panel popping up for discussion. When anyone clicks on the question mark, it indicates a request for responses. When anyone uses @Someone, it notifies them someone has responded. I have used it to solve the “what if they don’t do the reading?” problem, since we all kind of do the reading together.

All this is great. The system “auto-grades” (though I have to set it then check it very carefully), and pushes the grades to Canvas gradebook on my command, so I can focus on the discussion itself instead of evaluating it.

But you can’t do this with images — just upload and everyone talk about it.

Except…you can. Perusall won’t upload images natively, nor link to images directly on the web. So I downloaded an image, and saved it as a pdf in Preview, then uploaded it. Then I clicked on a section of the picture. Instead of highlighting text, Perusall put a pin. I can then ask a question or make a comment about just that portion of the image. Click the pin, and the conversation panel opens.

But the interface itself takes up a lot of the screen, which we don’t want for images. So I’m going to show students what to do about that:

If they do it, then it will look like this:

More room for the image, less clutter. I’m thinking it would be possible to put several images on a page to be discussed for that week.

What it’s doing is similar to ThingLink, which I learned about from our wonderful art historians over a decade ago. But ThingLink and similar programs, although they can be embedded into Canvas with iframes, cannot track a student’s comments, nor auto-grade them. Perusall can, which shortens my workflow so I can focus on the discussion, just as I do with annotated text.

So, annotations for images when I teach a European history course that focuses on the Humanities, and a History of Technology class that can get bogged down in text? I’m in!

 

The hour before the deadline

We all have stories, usually told while shaking ones head, of how students do things at the last minute.

Long ago, the deadlines for my online classes were set at 5 pm on the day something was due. But “everyone else” (the dozen or so others teaching online) set theirs at Sunday midnight. I would receive panicked emails between 5 pm and midnight, or after an assignment was graded late. So my nicely planned evenings of sitting and marking papers didn’t work. I changed my deadlines to Sunday midnight.

So when I began using Perusall for annotations, I asked, repeatedly, for students to participate beginning on Tuesday and ending Sunday midnight. Some do so. But many try to read five documents and cram in all the annotations on Sunday. This prevents the close reading I had intended by assigning annotations in the first place, an issue for another post. But it also provides a bizarre opportunity.

Perusall emails me when a student answers a question I’ve posted.  It also emails when a student tags me with an @. Throughout each Sunday, these emails increase, with a flood of them in the last hour: 11 pm to midnight.

If I go into Perusall on Sunday night between 11 pm and midnight, I can participate in the discussion, adding questions and using @ to reply to individual students, and I’ll get a response. It becomes almost synchronous.

I’m not saying I do this every week and every class, but if I’ve assigned a particularly difficult document (last week’s Henry Miller: Tropic of Cancer comes to mind), I can engage a great deal of the class at that hour.

No, I’m not recommending this. But it does seem to open potential for the last hour before the deadline. I have a colleague who uses Google Docs and notices that same activity shortly before the deadline — he sees the number of participant bubbles at the top increase as he watches.

So I wonder two things. First, is it safe now that we have many online classes to change deadlines to a more reasonable hour? And is the last hour before the deadline an opportunity to teach that we should be using?

Continual feedback on my work

Recently, I added a new assignment to my online classes, called Lecture Notes.

It’s just a one-screen write-in-the-box assignment, with the following instructions:

 

It automatically scores credit when they submit it. This is for two reasons: immediate feedback/acknowledgement of the submission, and so I don’t have to rush to grade anything. I can stop by and enjoy them all, reading through, at my leisure.

My original intention was just to make sure they were somehow responsible for the lectures, into which I’ve put a huge amount of work. These lectures are written out, with audio of me reading the lecture, illustrations, iframes with the assigned readings, embedded video, music snippets, and quite a bit of my own personality and interpretation. I’ve created them for all six of my classes, 15-16 lectures each (one per week). I don’t want them ignored!

But what’s happened is even better, thanks to the first and last bits of my instructions.

The “implied thesis” has told me a lot about whether my interpretation is getting through. Fact is, most students don’t get this idea, and just tell me what the lecture was about: “You talked about immigrants coming over from other countries and the impact of industrialization”. But some students comprehend that this is interpretation: “You implied that the capitalist system made it difficult for labor unions to succeed”.

Even better is the “brief response” I’ve requested. This has given me a chance to talk with students directly.

Those who don’t understand that I’m an interpreter of history write things like, “I don’t agree that the Romans should have invaded Gaul”, so I get that they’re responding to certain events. But often they tell me things like, “I was surprised that the new household technologies made more work for women” or “I didn’t know that Augustus Caesar was a title instead of a name” or “it made me really sad to learn how immigrants were treated in the factories” or “this seems just like today”.  Sometimes they ask questions like, “why did Cleopatra kill herself instead of just sending a big army against Octavian?”

I can respond to each student privately, which often opens an ongoing dialogue. And the cool thing is: it’s always about the material, never about the grades/assessment/navigation. They’re real conversations, about history.

It’s like ongoing feedback for me – it tells me what’s getting through, how I’m doing, and what they think.

It’s entirely possible that it’s all there, and could replace stultifying assessments like quizzes, or (shock!) papers. We’ll see. For now, it’s just great.

New Title V and substantive interaction

In its wisdom, the state is proposing the following changes for online classes:

Originally, the issue was teacher-student contact, a response to the fear that teachers would just upload stuff and not do any work. Now they’ve decided to legislate pedagogy, and insist on student-student contact.

I know several faculty who aren’t happy about this, because their pedagogy doesn’t rely on student-student interaction. Rather their course is set up to maximize student interaction with the course materials. Several instructors I know engage students in highly individualized feedback. One responds to each student’s posted analysis of each reading each week. Another evaluates submitted assignments in great detail. They both have in-depth, ongoing, substantive contact with every single student. Neither relies on students interacting with each other.

So this is bogus from the start, pedagogically speaking. The state has decided to intervene in methodology, based on research that is questionable at best. Studies using physical classroom research, for example, compare straight lecture to group activities (active learning). They tend to ignore the many other activities that can be assigned in an online class but aren’t reliant on other students (for example, instant feedback on a quiz question). Other studies of “community” in online classes focus on the affective domain rather than the quality of work. While it is an option, it is not necessary to have student-to-student interaction for a student to feel like part of the class; that can also be achieved through instructor presence, student-instructor interaction, and student-content interaction. Focusing on students interacting also ignores the teacher-student tutorial model,  which provides individualized responses. Many students want individual attention far more than they want “community”.

But we don’t make the rules.

Since the state is so dedicated to the word “substantive”, I looked it up:

Substantive- “Having a firm basis in reality and so important, meaningful, or considerable.” (Oxforddictionaries.com)

1. denoting substance, 2. expressing existence, 3. meaningful or important (OED)

So if it’s there and it’s real, it’s fine. An instructor, without being pressured, wouldn’t have such interactions if they didn’t believe them to be important or meaningful. In the case of “considerable”, a class that has a lot of meaningless discussion would also be covered.

This brings us to Lisa’s Rule on Rules: when confronted with legislative stupidity, it is usually best to look at what you’re already doing and see if it can be used to demonstrate that your work already complies with the new rule.

Tonight I attended a wonderfully balanced session on the new pending rules. It was led by Sean Davis, our wonderful sociologist at MiraCosta. He noted the controversy over the rule, but focused on options for how to create such interactions. These included discussion/question forums, emails between students, peer feedback, student groups, interactive video (we have a pilot for Arc), video conferencing, chat, and collaborative projects. I would add to this group annotation, which is a major source of interaction in my own classes.

For those instructors who do any sort of student-student interaction, it should not be hard to justify what they do within the new context. It is indeed these instructors who are most concerned about the new law.

Most instructors have a discussion forum. Some of these are problematic or meaningless, but if they do it every week, they’re good. I can justify much of my students’ work, because in addition to annotating written primary sources, they post their visual sources in a forum, and “like” the sources they enjoy. “Like” is interaction, and if you ask students to “like” for a purpose (they plan to use the idea, or vote up ideas the class will use), it’s meaningful.

This sort of “passive interaction” or “accidental interaction” can satisfy the rule. Passive interaction is like a cocktail party — everyone is in the same place but not necessarily at the same time or for the same purpose. But they see what the others are doing, and adapt their behavior according. Open forums where students post assignments can serve this purpose. Particularly if they’re graded only individually, and privately, posted student work can help other students see where they stand, demonstrate alternative ways of doing the work, and serve as examples.

So the professor who has students submit their assignments directly to her could instead have them submitted to a forum (if necessary, the forum could be set not to show other work until ones own work is submitted). The prof who responds to his students individually after they post in a forum could add a summary post noting the contributions of particular students, asking that they be used as examples.

I admit that I am completely uncomfortable with asking students to email or message each other privately, and I have never been a fan of peer feedback although I know people who do it well. And I remain appalled (I am often appalled) that the rules never seem to care about the interaction between the student and the content. Where’s the rule that says courses should include deep, sustained, meaningful interaction between the individual student and the materials being studied? Where’s the rule that says courses should integrate important and meaningful contact between the student and the norms of the discipline?

It’s bad enough that so many courses contain discussion forums because the instructors think they should. That has been going on for years, even before this rule was enacted. The inevitable response to the new rule will be more one-post-two-reply “discussions” that frustrate online teachers and become rote and meaningless for students. Surely the new rule must be a misguided attempt to replicate the classroom, except perhaps the people who wrote it haven’t been in one for awhile. Where’s the rule requiring students in a non-distance education class to be forced to interact with each other?

In the meantime, such attempts should be dealt with by showing that what we already do satisfies the rule. It’s just a shame that we have to.