Setting Up IDEA Evaluations

[Count Orlock]

We are supposed to set out IDEA goals for our student course evaluations. I am trying to figure out if there is some particular way to set it up to yield the highest score?

Context:
For example, I put my "writing skills" setting to "extremely important" (or whatever the term is), but, since some disgruntled and/or lazy students hate to have to do the paper, they use that opportunity to trash my score, even though I work so hard to help them.

[jerseyjay]

First, try to figure out how you administration will use the scores. Mine really have no idea how to read the data, and just look at comments.

Second, pick two or three things that you class could be expected to focus on. Then make everything else not relevant.

This has worked for me, although, as I've said, nobody at my school seems to know or care how to parse the data.

As with all evaluations, there are always students who seem clueless, or will base their evaluations on the course and not the instructor. (I get that many students do not like history and find reading tiresome--but since I am a history professor, there is not much I can do about that.)

[teach_write_research]

If your goal is to get high scores that your department and institution will see as "success", then find out what the faculty who will be reviewing you put on theirs. Follow their lead just enough for it to look complimentary but still leave yourself a bit to show your individuality and teaching style, but not too different. Prioritize the items that students will find easy to recognize and rate highly. Use those buzzwords when talking to them about it.

If your goal is to get some sort of authentic feedback and you're at a place that supports that, then align the items with your course objectives or departmental assessment. Then you are able to put even low scores into a meaningful context and discuss how you will address them, informed by the comments etc.

The IDEA system asks students to respond to items that are in educator-speak. I've been curious about what students think the prompts are asking.

[jerseyjay]

Some years ago my school moved from pencil and paper evaluations that the students filled out in class to IDEA evaluations that require their doing so on a computer. One of the biggest results of this is that completion rates plummeted, usually below what is considered statistically significant. Before the pandemic, there was a ritual shaming of departments that had low completion rates (only 15% of students in basket weaving filled out the form, but 75% of math students did!)

So now there is a big push to have professors get students to do the evaluations. Before the pandemic, I would dedicate part of a class to this, telling the students to take out their phones or their computers and do the survey right then. I also offered extra credit if 75 per cent or more of the students filled out the evaluation.

Then when I had my annual review with the dean, he would look at the percentage of students who filled them out and cherry pick positive comments. I suppose that if he didn't like me, he could cherry pick negative comments. He has never actually looked at the numeric data.

When the IDEA evaluations began, one of the administrators announced that all adjuncts had to be "above average" in order to be retained. Of course, that meant that half the adjuncts would be dismissed each semester. But I have never known the data to be used against or for somebody unless there is some other reason that the administration likes or dislikes that person.

I have found that the comments are more or less as useful as the comments on the old paper form. They may serve as a rough guide but need to be taken with a grain of salt. The numbers tend to confirm what I already know (e.g., my 11am class is going much better than my 8am class of the same course). Of course consistently really bad comments might be a red flag if a professor is showing up drunk or something.

When I get tenure, I will probably care less about this.

So to repeat, find out what your administration wants to use these for, and tweak your objectives accordingly.