Wrapping Things Up

Wrapped by morebyless on Flickr, used under a CC-BY 2.0 license

All this reading and rereading resulted in resetting the effort expectations table. I’m essentially using the same chart as in the past, though the assignments and the feedback system are changed slightly. Here’s an excerpt from the chart as it is now:

The numbers now follow the grade level numbers: A=95%, B=85%, C=75%, D=65%, and F=64% or less. The number of major projects is also restored to the previous version: A=5, B=4, C=3, D=2, F=less than 2.

Dealing with Ghosting Students

The one nagging problem with the setup has been the ability of student to ghost their group and still get credit. That scenario could potentially give a student a C or better even though they have done little to no work. That bugs me. I want to trust students and believe that no one would do this. As much as I want to, I just can’t do it. This phenomenon is frequently called “the free-rider effect” in the articles I have been reading (e.g., Chang & Kang, 2016; Cherney et al., 2018).

It’s occurred to me in the last hour that the solution isn’t in changing the effort expectations chart. Rather it’s to change the way I manage the groups. A number of the articles and chapters that I reviewed focused on group sizes from three to five students. Some folks say more; some say less. I’ve done some work on how to form groups online (in a project that I was never able to maintain, which is a long story for another day).

I’m thinking that I should aim for four to five students in a group. With that number, even if a student does disappear, there should still be enough folks leftover to complete the projects. Groups of three would be problematic in that scenario. I’m also thinking of using the method of forming groups that I tried in the past: groups are composed of students who like to work in similar ways and (as possible) at similar times. For instance, students who like to work ahead of time and be finished in advance are grouped together. Students who like to wait until the last minute are in groups together, and so forth.

I’m grading for completion, not quality, so it doesn’t worry me that (in my experience anyway) the students who like to work ahead usually do better on their work. The other groups aren’t at any disadvantage. They just have to complete their work. They don’t have to knock my socks off.

What I’m adding is a practice that I found mentioned online and among some of my colleagues on Facebook. To put it bluntly, a free-riding student who disappears from their group or does not contribute can be kicked out of the group by the other group members. That student can then either work on the group projects independently or join a group of other students who have been removed from their groups. If I adopt this system, no student should pass the course while doing none of the work.

Assessing the Project Management Work

With all that sorted out, I just need to figure out how to focus assessment on the process of managing the project, not so solidly on the project itself. I’m worried about the workload, so I don’t want to add onerous reflections.

I’m also not a fan of asking students to use on of the strategies where they grade each other by assigning points or percentages based on the work each person does. See the “Student Assessment of Group Product” section of the Grading Methods for Group Work page from Carnegie Mellon’s Eberly Center for examples. Such systems work, but they can put students at odds and can be unfair to students who do not fit the demographics of other group members (whether because of race, class, gender, LGBTQ status, and so forth.

I think the answer is to add project wrappers to the major projects. Somehow as I was reading yesterday, I happened upon exam wrappers. I’ve heard the term many times before, but had always thought they were a way of adding information when you turned in an exam. I assumed that they were similar to the draft letters and memos that I have used frequently, where students tell me about their project and what they want me to comment on as I add feedback.

In other words, I missed their full potential. Exam wrappers are also used after an exam, when students receive the feedback on their work. They typically ask students to look at the errors, figure out the correct answers, and reflect on what went awry. That process is similar to the Revision Plan assignment that I have used in the past. In that assignment, I ask students to return to a project from earlier in the term and write a description or instructions for revising the project. Depending upon the course, I might also ask for storyboards, outlines, or other kinds of mockups. I liked that assignment as a final exam because it allowed students to explain what they’d do if they had more time without asking them to do all that work while they were swamped with other finals.

Somehow, despite knowing about exam wrappers, I didn’t see the connection of using them both before AND after the exam. You’d think that the work wrapper would have tipped me off—it is an active learning strategy that wraps around another assessment. Yes, I’m feeling a bit daft right now.

I realized that adding something when a major project is submitted and then after the feedback is returned allowed me to structure reflection in a meaningful way that avoided asking students to write long reflections and to come up with deep thoughts about their work. I can ask some basic survey questions about the time spent on the project, which portions they focused on, how group members coordinated, and so on in the wrapper they complete when turning in their work. After I return the work with feedback, I will have similar survey style questions but also some short answer questions that will ask them to reflect on what they did and how them might change in the future to improve their writing and collaboration.

Asking multiple-choice and short-answer questions, I can limit the amount of effort that students have to expend. Coordinating the pre- and post-wrappers should trigger reflections on cause and effect (or perhaps impact) of the strategies they have used. Asking for a revision or revision plan focuses the work on the project (the product). Asking students to think about why their work habits resulted in the performance and how they might change it in the future focuses the task on the process. It feels like the perfect solution for paying attention to the project management aspect of the work that they will do in the course.

FINALLY, I think I have things figured out. None of it is written up or published in Canvas, mind you, but I think I know what I’m doing. I see a lot of long work days between now and Monday, when classes start. On the chance that you aren’t aware of the practice of exam wrappers, I’m including three links to basic information on the strategy:

Temporary Signs of Progress

Another beautifully random temporary sign from TFL by WithAssociates on Flickr, used under a CC-BY-SA license

I’ve been worried that I’m asking for too much work to earn the various grade levels. Admittedly, I’m at a point I typically reach when I’m working on assessment and assignments: Everything feels completely random. Why 95% for a B? Why all five projects for a C or higher? Why any of this? The only thing I can say is that I know why there are five projects–because that is set (more or less) in the department’s new course template. Everything else is just a well-intentioned guess.

I mentioned last week that from my perspective, some of the measures of labor in Inoue’s system have the potential for ableism. Carillo amplifies my concern, bringing in discussion of the inequality of using the number of hours of labor to measure assessment. I’ve been worried about students who have less time to commit to coursework because of other obligations (such as work, parenting responsibilities, and requirements of scholarships). Carillo addresses those concerns and further argues that different people need varying amounts of time to complete a task. What takes me one hour may take someone else two hours. She points out how this system is particularly troubling for students with disabilities, who may face “access fatigue” (Konrad 2021) and, therefore, may need more time to complete their tasks. See the concept of “crip time” (Price 2011) for more information.

Where does all that leave me? The randomness of assessment systems and the inequality of focusing on the amount of work students are able to complete land me back where I began. I’m no longer feeling all that confident about the grade expectations I posted yesterday. Asking students to do everything for a B feels wrong now. Asking them to do even more work for an A feels even worse.

Now I think these are the plans and questions for the Fall:

A Tentative Effort Expectations Contract Chart

Effort by secretlondon123 on Flickr, used under a CC-BY-SA 2.0 license

I stand by the short list of goals that I shared in my last post. If anything, I feel stronger about them. I am set with focusing on Completion meaning that “you do all that is asked of you in the manner and spirit it is asked” (Inoue, 2023, p. 327). Less of a specifications grading system (Nilson, 2015), the system checks to ensure that the basic things that are asked for are there. I’m leaning on the explanations from Inoue’s work here. There’s much more explanation in the book (which is available free from the WAC Clearinghouse), but I particularly liked the three questions he included for assessing the work students did during a week in FYC:

Each labor assignment is complete and counts if it meets in the affirmative the following questions:

• Is the labor product(s) posted on time and in the correct place?
• Does the labor product(s) include everything I asked for and meet the minimum word count?
• Is there a labor tweet/Slack(s) posted as instructed (if applicable)?

(Inoue, 2023, p. 196)

My class will have different framing questions, but the idea will be the same:

• Did the student turn in the work before the end of the grace period?
• Does the work include everything listed in the criteria for the activity?
  • Is it the right assignment? (e.g., Is it a memo?)
  • Is it on the right topic? (e.g., Is it a good news memo about employee holidays?)
  • Is it a full draft? (e.g., Does it include memo headings and a memo body?)

I may be addressing some specific issues that arose in the past with my questions 😉 You know, like that one time a student responded to the assignment for a pitch presentation on their proposed report topic with a presentation that argued Lionel Messi is the GOAT. Um, no. That is not a Complete. Nor were the Recommendation Reports that were only outlines. Yes, that was a submission, but not a Complete.

I feel reasonably comfortable that this kind of framing questions will address to check student work to ensure that students are on track and doing what they are supposed to. Further, I see what I’ve come up with as more rigorous than what I was doing in the past. Students have to do all of the work with attention to its expectations to do well in the course. It wasn’t really a goal to increase the rigor, but I’m happy that it has turned out that way.

I’m also going with the essential idea of Inoue’s grade levels, though I am phrasing my system in terms of work completed rather than work that is missing to avoid the deficit focus. Like Inoue’s system, my tentative plan allows students to earn a B by doing everything in a way that meets basic criteria. To earn more than a B, I will follow Inoue again and provide a list of additional tasks that students can complete. Each task earns 1/3 of a grade. For instance, if a student with a B does one task on the list, their grade will increase to a B+. This extra work for grades higher than a B also increases the rigor in the course.

The table below shows the tentative contract for effort in the course. Naturally I will need to add a lot more information. In particular I need to include an explanation of what it means to earn a Complete as well as sketch out all of the tasks that are available for grades above a B. I want to adopt Inoue’s practice of allowing students at any level to improve grades by completing these extra tasks as well (2023, p. 330–331). So a student with a C+ could complete a couple of extra tasks and move up to a B. That gives students more options so that one bad week doesn’t ruin their chances.

Here’s that setup so far:

Weighing Student Work

Scales of Justice by Michelle Grewe on Flickr, used under public domain

Fooling Myself

To start, what are my beliefs about assessment? I would be easier to point to the people whose pedagogy I try to work within: Jesse Stommel, Alfie Kohn, and Susan Blum.

Grades don’t work. They cause students to focus on the wrong things. In my contract, students may still be focused on the wrong thing. They are counting (and obsessing) over writing tasks. I’m fooling myself to think that the counting of Completes is immune from the curse of focusing on the elusive grade.

My great confusion on how to count (or not) the group work should have tipped me off sooner. Trying to figure out what would count as effort in a group is the wrong thing. I’m trying to invent a system where I decide how the quality and quantity of work determines the ultimate grade.

I’m still acting as the arbiter of course knowledge and assessment since I am the judge of Complete or Incomplete. I decide whether the effort that students put in counts. If I were truly faithful to the ideals of ungrading, I would be powerless. Students would determine what and when they have learned.

So what if I scrapped it all? What if I counted only submissions? Did the student turn the task in? Yes? Then it counts toward the expectations.

A Model Contract

So what if I go back to Inoue’s Labor-Based Grading Contracts system? I don’t like it for reasons I’ll get into, but it is a starting place. Here are his contract expectations (p. 127):

First, let me get my problems with these guidelines out of the way. I don’t like counting attendance, which is what the # of Non-Participation Days essentially does. I believe it’s not only ableist, but also inhumane to students stretched thin with other obligations. The system does have exceptions for excused absences, which could reduce some of the ableism. Still in a time when COVID still exists, I do not want to encourage any system that would ask a potentially sick student to come to class to avoid slipping down the expectations chart. Since my courses are all asynchronous online classes, it’s also irrelevant. There is not such thing as an attendance day to count at all.

I’m not a big fan of punishing students for late assignments either, so that column of the chart is problematic for me too. I believe in a more humane system that trusts students to put in their best effort. If they need an extra day, they should have it. I do recognize that Inoue’s course is highly interactive, and if students do not bring their work to class, they miss getting feedback from others. Peer review is a tough problem to negotiate in a due date system. I may not like this portion of the chart, but I understand why it’s there and critical to the way his classes work.

What counts in this contract model bugs me the most though. This is a deficit model. It counts failures and devalues the idea of failing as a critical part of learning. I’ve read enough of Failing Sideways to know that it’s more valuable (to me anyway) for students to be allowed to fail, to try again, to take risks without being punished.

Toward My Own Contract Model

I need to articulate my own model, and this time, I want something that is more in line with the spirit of ungrading. I haven’t worked out the details yet. Honestly, I haven’t even begun the work of determining what I expect students to do to demonstrate their learning. I do know that I have these goals in mind: