Hive Brain: Post your assessment challenges here

[San Joaquin]

Given the tremendous talents and cumulative experience on these boards, if anyone has assessment questions that might benefit from crowd wisdom, please feel free to post them here.  Or you can rant a little: that's okay, too.

[spork]

Rant:

When you put faculty members who don't understand basic concepts like the need to operationalize variables in charge of designing and assessing general education requirements, isn't it obvious that the end result is going to be a totally meaningless waste of everyone's time?

[downer]

I'd be interested to get an example of an assessment process that revealed useful information, that was not obvious to those involved before the assessment, that led to real improvements of the educational process being assessed. Just one.

TBH, that's a rant in disguise.

[Hegemony]

I'm with Downer.

[ergative]

I'd be interested to get an example of an assessment process that revealed useful information, that was not obvious to those involved before the assessment, that led to real improvements of the educational process being assessed. Just one.

TBH, that's a rant in disguise.

I have one that I ran into last year. I gave a test that involved both mathematical computation---easy, blind plug-and-chug as long as you memorize the formulas---and conceptual understanding. I was debating whether to put the plug-and-chug on the test, since I cared more about the conceptual understanding and I didn't want people to lose points from arithmetic errors. Also, I'm in a decidedly not-mathy field, so a lot of people were scared by the numbers. Maybe they would do better and feel more comfortable if I kept assessments computation-light? But I did want them to know where the numbers came from.

In the end, they did far, far better on the plug-and-chug than on the conceptual bits, and that was good because they bombed conceptual understanding. So I have learned the following:

1. Computational plug-and-chug may be a good thing to include, because people find memorizing and applying formulas easier than studying concepts and knowing how to apply them, and maybe that stone-cold formula memorization will give them a better sense of preparedness and control on exam day.

2. I desperately need to improve my instruction of the conceptual stuff.

I can't yet say it led to real improvements, since the next go-round of this class doesn't start until this coming semester, but I like to think that it will be better in part because of what I saw on last year's final exam.

[San Joaquin]

For Spork:  yes.

For downer and Hegemony:  we ran an evaluation of our capstone based on the outcomes we chose, and discovered that our students were doing very well on the practical aspects of research, but that they systematically didn't know how to use theory to frame a question or interpret findings.  This was our deficit: upon examining the curriculum, there was no place we had been asking them to acquire and practice those skills prior to demonstrating mastery at capstone level.

We chewed on that one for a while, several options were floated, and then we achieved consensus over where and when that could be fit into the learning stream in ways that didn't compromise anyone's instructional discretion, but provided ample support for the students.

We tried it the following year, and it did result in about a 40% improvement in the next cohort. 

But, and this is a BIG however, we were at a place where the faculty owned instruction, curriculum, and assessment, and we took those collective responsibilities seriously, since most of us came there from other kinds of places