Rubrics - what are they really for?

[downer]

I'm told that my syllabus for next semester needs to include rubrics, because of Middle States requirements. Odd, because I don't think other schools are making me do this. But, whatever.

I will have to guess how many rubrics I need to include in my syllabus. One for each kind of assignment would make sense, but then what counts as a kind? Does a long paper have the same rubric as a short paper? What if I am requiring students to revise their first drafts and produce second drafts -- do I have the same rubic for the final paper then?

Many of my syllabi already have some paragraphs on what counts as adequate, good and excellent work. According to
https://www.cmu.edu/teaching/assessment/assesslearning/rubrics.html
"A rubric is a scoring tool that explicitly describes the instructor's performance expectations for an assignment or piece of work."

But in fact the very examples they supply suggest that this is not a good definition at all. The examples are much more rough and ready, spelling various desiderata along different dimensions, but giving no clue whatsoever about how those different dimensions are weighted.

For my syllabi, I will of course copy and paste whatever i can find from the web, and I doubt that anyone from Middle States will actually look at my syllabi. I suspect that if I started taking rubrics seriously, the grades I assign would go down at this school.

My guess is that rubrics serve two real functions. First, to get faculty to think about how they assess student work in a consistent fashion. Second, to provide a post-hoc justification of a grade that a student received.

I have heard that implementing a rubric in the LMS can really speed up grading. That would be useful.

[Durchlässigkeitsbeiwert]

Published rubric resolves quite a few typical students complaints:
- points for a partial solution of a problem
- points for right calculations with wrong inputs
- points for multi-step problems (e.g. measurement => calculation 1 => calculation 2) with mistakes in the early steps
- points for "fishing" answers (e.g. student writing two paragraphs in response to question requiring two-word answer)

There is quite a variation in how these are treated in different courses

[Puget]

I put the rubrics into the LMS and make them visible to students, and really encourage them to look at them (in my seminar I actually require that they do a self-evaluation of their term paper using the rubric) -- this produces better assignments, or failing that, fewer questions about why they missed points. And it does indeed speed up grading, because it nearly eliminates the need to type any comments (because each point value has a specific description already), and calculates the grade just by clicking on the appropriate box for each criteria.

[mleok]

Setting up a rubric on Gradescope increases the level of consistency in grading a large number of scripts, and allows you to tweak the point deduction across the board. It's not necessary to have a complete rubric before you start grading in order to use it this way.

[AvidReader]

Two of my recent schools have had departmental rubrics that apply to all essays. They give about a paragraph for each letter grade that says something like "An "A" paper will exhibit the following characteristics" with a list of qualities. Repeat for each successive grade. Not every criterion on the rubric applies equally to all essays; for instance there is a line that says something like "research reflects understanding of the sources and attributes material accurately" and I just skip it if the work doesn't require sources.

One of my undergraduate instructors had something like "Requirements for an A in the course" (and other grades) on every syllabus. I can't remember whether there were also descriptions of papers at each level.

I do find that grading goes more quickly with a rubric, though it goes much better with a rubric of my own invention. Lots of times students will have A-level organization and C-level grammar (hard to negotiate with the one-size-fits-all rubric). But I think you could certainly write paragraph descriptions of A-level work (and so on) that would apply to most assignments, and provide more detailed rubrics separately if individual assignments warranted. I actually spend a class walking my students through the university rubrics when they exist, because many of their elements are incredibly vague.

(I use a rubric entirely of my own invention for revisions; my goal is to help students evaluate and apply feedback, and using the standard essay rubric doesn't reflect this).

AR.

[marshwiggle]

I put the rubrics into the LMS and make them visible to students, and really encourage them to look at them (in my seminar I actually require that they do a self-evaluation of their term paper using the rubric) -- this produces better assignments, or failing that, fewer questions about why they missed points. And it does indeed speed up grading, because it nearly eliminates the need to type any comments (because each point value has a specific description already), and calculates the grade just by clicking on the appropriate box for each criteria.

I use checklists like this, and require students to attach the checklist to the assignment which the TA will fill out. That makes it impossible for them to claim they've never seen it. (And I go over it in class, with examples, etc.)

The TAs love them.

[polly_mer]

I'm told that my syllabus for next semester needs to include rubrics, because of Middle States requirements.

The institution needs to demonstrate that it meets Middle States Standard III.2.a:


III. Design and Delivery of the Student Learning Experience

2. student learning experiences that are designed, delivered, and assessed by faculty (full-time or part-time) and/or other appropriate professionals who are:
a. rigorous and effective in teaching, assessment of student learning, scholarly inquiry, and service, as appropriate to the institution's mission, goals, and policies;


Source: https://www.msche.org/standards/

Note that assessment of student learning must occur.  A standard way to assess student learning is through the use of rubrics.  A logistically simple way to collect all the necessary rubrics for each course is to have it be part of the syllabus that is already being collected for other reasons.

Thus, while rubrics themselves are not a requirement for Middle States, a straightforward way to demonstrate meeting the assessment of student learning is to require rubrics from faculty.

I doubt that anyone from Middle States will actually look at my syllabi.

As part of the review, Middle States will ask for specific syllabi to examine.  If yours is selected and you don't have all the features that the institution claimed, then that goes into the report.  If most of the syllabi selected are not in compliance, then that's a "fun" conversation that the institutional representatives get to have with the Middle States folks and "fun" always rolls downhill internally.

I suspect that if I started taking rubrics seriously, the grades I assign would go down at this school.

Middle States wants institutions to keep college education rigorous.  Whether your institution wants grades to go down to reflect student performance (i.e., stop ripping off the students with grades students did not earn) or wants faculty to step up to be more effectively supportive so that students rise to the occasion and learn (i.e., stop ripping off the students on skills/content they aren't mastering) is unclear.

My bet is the institution hasn't thought through that part.  The institution is focused on having enough evidence to demonstrate III.2.a when clearly they are employing many adjuncts who won't meet the standard as it stands for rigorous, effective, qualified (III.2.b), sufficient in number (III.2.c) and adequately supported (III.2.d).

I have heard that implementing a rubric in the LMS can really speed up grading. That would be useful.

Years ago, LarryC had a great post about how rubrics make grading easier.  The gist was:

* Follow standard rubric assembly with categories and rankings (excellent, good, acceptable, poor, failing) for whatever is relevant for the assignment.

* Put a comment box at the bottom and have a separate list of standard comments from which you can copy and paste into the box.

* Assign no weightings to individual categories to give yourself flexibility.

* Instead, have a key like:
                    *At most, one good and the rest excellent is an A
                    *A mix of excellent and good with at most one acceptable is a B
                    * A mix of excellent, good, and acceptable with nothing below acceptable is a C
                    * A mix with no failing is a D
                    * Any failing is an overall F

*Then, check the relevant rankings in each category, copy and paste three comments into the box, double check that the key indicates a grade lower than the one you plan to assign, and then assign the grade you know is acceptable at your institution for this quality work.

Don't do anything foolish like check a lot of failing and then award a B, but few students will argue that the paper that should be a C per the rubric got a B and it should have been an A.  If students do argue, then you can lower their grade to the C they earned per the rubric and sigh about the original error.

That way, your grading was faster, students get the feedback they need to improve without all that much effort on your part, and you have a way to demonstrate to auditors that you are applying reasonable standards to student work.

[the_geneticist]

Yes, you can add rubrics easily in most LMS, even the old version of Blackboard has a rubric feature for assignments.  And it really easy to copy & customize for different assignments.

I second Polly's idea of having a generic rubric for each major assignment type where a rubric is appropriate (papers, lab reports, presentations, discussion posts, etc.).  You don't necessarily need one for all assignments (automatically scored quizzes). 

[downer]

I can see how a rubric is useful for the students and maybe it also reduces that number of students who question grades or want an explanation of their grade.

But maybe not -- it may also give them more ammunition with which to question a grade. I'm not too worried. I very rarely get students questioning grades.

My main concern about taking rubrics seriously is that it could be time consuming and then boxes me in assigning grades.

Has there actually been any research on the pedagogical effectiveness of rubrics? Comparisons of effectiveness of different kinds of rubrics? Or is it all anecdote and speculation?

[Aster]

So the answer is no, you do not need to supply rubrics to meet regional accreditation requirements. Your institution is feeding your faculty a full shovel load of BS.

Mandatory use of rubrics imposed onto faculty is severe micromanagement and a violation of academic freedom. Sure, lots of people use rubrics. Lots of departments and faculty even agree to standardize and use them communally. But *require* rubrics wholesale of faculty, and also require that those rubrics be preloaded onto course syllabi?? That's just... dimwitted and inflexible.

If you believe that you must comply with a false directive, my advice is to keep your rubric as basic and flexible as possible to suit your individual assessment strategy. Since your a$$hat institution says that you must stick this thing onto your official course syllabus, you are now effectively self-constrained by whatever you write onto that document. So don't lock yourself out of assessment flexibility.

[Cheerful]

+1 for Aster's post.

[Caracal]

 I find rubrics most helpful when I'm grading things that are particularly artificial. In class essays, for example, are rarely actually "good." Students have a limited amount of time to answer a question putting together various things from class. Occasionally, I have an exceptional essay that is actually interesting and insightful, but that isn't actually a requirement to get a good grade. I'm really just trying to assess whether students have understood themes in the course well enough to put things together.

I find my rubrics really useful for grading these, because it provides an easy way to assess whether students are doing the things I ask them to do. On the other hand when I'm grading an out of class paper, I don't find rubrics very helpful. I have a more organic sense of what makes a paper good or bad and having to deal with a bunch of points on a rubric just slows me down.

[Hibush]

My main concern about taking rubrics seriously is that it could be time consuming and then boxes me in assigning grades.

I really had this concern, and am now both a user (albeit not in the classroom) and skeptic.

The intent is to make sure evaluation is fair and consistent, and they are good for achieving that. I find that using a rubric is a lot more time consuming because I have to evaluate each important aspect, rather than give my gut judgement based on the first read through. So it is working as intended. Using it should get faster with practice.

A good rubric should reflect the elements and criteria you think are important. If you find that your initial attempt is boxing you in, or weighting things wrong, then you need to revise it until it fits. That is challenging, but it does make you think about what you want to see, and helps communicate those things to the people submitting things. 

[the_geneticist]

My main concern about taking rubrics seriously is that it could be time consuming and then boxes me in assigning grades.

I really had this concern, and am now both a user (albeit not in the classroom) and skeptic.

The intent is to make sure evaluation is fair and consistent, and they are good for achieving that. I find that using a rubric is a lot more time consuming because I have to evaluate each important aspect, rather than give my gut judgement based on the first read through. So it is working as intended. Using it should get faster with practice.

A good rubric should reflect the elements and criteria you think are important. If you find that your initial attempt is boxing you in, or weighting things wrong, then you need to revise it until it fits. That is challenging, but it does make you think about what you want to see, and helps communicate those things to the people submitting things.

What makes grading hard isn't the answers that are incomplete or not-so-good, it's the ones that are just plain weird.  That's why it's important to include a category that means "answered question being asked & includes relevant supporting evidence".

[downer]

My main concern about taking rubrics seriously is that it could be time consuming and then boxes me in assigning grades.

I really had this concern, and am now both a user (albeit not in the classroom) and skeptic.

The intent is to make sure evaluation is fair and consistent, and they are good for achieving that. I find that using a rubric is a lot more time consuming because I have to evaluate each important aspect, rather than give my gut judgement based on the first read through. So it is working as intended. Using it should get faster with practice.

A good rubric should reflect the elements and criteria you think are important. If you find that your initial attempt is boxing you in, or weighting things wrong, then you need to revise it until it fits. That is challenging, but it does make you think about what you want to see, and helps communicate those things to the people submitting things.

So it sounds like taking rubrics seriously means that a course that was previously working well will be destabilized and it will take a few semesters of trial and error to get it working as well as it did previously, and then maybe after a few years the course will be improved. Good plan!

Actually it seems to me that the real way to take rubrics seriously is to make them one-dimensional. For each assignment, you only test for one skill. For a 15 week semester, you could have 14 assigments, testing 7 skills twice each, plus a final exam. I'd love to try that as an experiment and see how it works. That would be teaching to the rubric.

So the answer is no, you do not need to supply rubrics to meet regional accreditation requirements. Your institution is feeding your faculty a full shovel load of BS.

Mandatory use of rubrics imposed onto faculty is severe micromanagement and a violation of academic freedom. Sure, lots of people use rubrics. Lots of departments and faculty even agree to standardize and use them communally. But *require* rubrics wholesale of faculty, and also require that those rubrics be preloaded onto course syllabi?? That's just... dimwitted and inflexible.

If you believe that you must comply with a false directive, my advice is to keep your rubric as basic and flexible as possible to suit your individual assessment strategy. Since your a$$hat institution says that you must stick this thing onto your official course syllabus, you are now effectively self-constrained by whatever you write onto that document. So don't lock yourself out of assessment flexibility.

I am entirely inclined to believe you. This is a school with a lot of probems, with senior administrators coming and going frequently. There seem to have been about 4 registrars within the last 3 years, and maybe as many presidents or acting presidents. It may well be that the real issues that Middle States will focus on are financial insecurity.

I'm definitely not investing any energy into some long term plan for improving rubrics with this school given its problems. They don't have the energy to police any policies they have for all courses.