University forcing professors to adopt standardized testing?

[xerprofrn]

I'm guessing the aversion to this is somewhat discipline specific.

In chemistry, at least, common final exams across sections isn't an uncommon strategy. Not done everywhere, but certainly not something I'd be surprised to find. Even places that don't give common exams, faculty work to make sure students are being assessed by similar metrics and at similar times across sections of the same course.

Similarly, it's not uncommon to give final exams that are nationally standardized (from the American Chemical Society), where everyone in the country gives the same final exam for the same level course.

It obviously works better in some disciplines than others, but it's not like the concept is foreign- just maybe not a good fit for your department.

I work in a department that does that, too.  For example, Exam 1 for all sections of a particular course will be the same for a particular term.  Then, we (meaning the faculty teaching team) will get together to analyze the results and make changes to those questions.  Fifty percent of the questions are changed every term. The difference is that we, the faculty, are in charge of this process of exam writing.  The whole team.  Not one person's exam that admin-types have deemed appropriate and forcing all others to use.

If they want to standardize across sections of a single course, that would be workable.  However, faculty must make the decision on the actual exam writing.  This does not seem to be the case here.  In addition, the weighting of each exam is up to the teaching team.

[ciao_yall]

Sure, so are we. Most of us publish regularly on pedagogy.

That still doesn't mean that by the end of the semester, students in different sections of the same course should not be able to complete the same assessment. In fact, it gives some fascinating insight into strengths and weaknesses of those different styles and how they effect the end goal of student learning. Surely you have shared learning objectives? If so, I'm having a hard time visualizing how one or more of those objectives could not be assessed uniformly.

Myself and a peer taught Intro to Advertising side by side for years. We had the same official college course outline. At the end of the semester, broadly speaking, our students achieved the outcomes we set out, and we always enjoyed having one another's students in follow-on classes because they were well-prepared in our discipline.

Like I said, sounds like some field specific differences. In my field, there's a pretty consistent expectation of what a student should be able to do by the end of the course based on our defined learning objectives. Even if we all teach differently and assess differently, adding a new 5% asessement at the end of the semester shouldn't be a big deal.

Still, giving the same multiple choice final exams would have been a disaster. We used different textbooks, which used slightly different models and vocabularies for basic phenomena. So whoever got the "right" exam, keyed to the "right" book would have been golden.

So giving students, even a 5% assessment, would have to have been general enough that anyone could probably have passed it whether or not they took the class. And it wouldn't be fair to the student who was on the border but slipped below because they got the "wrong" textbook and didn't recognize the terms in the exam.

[Caracal]

I'm guessing the aversion to this is somewhat discipline specific.

In chemistry, at least, common final exams across sections isn't an uncommon strategy. Not done everywhere, but certainly not something I'd be surprised to find. Even places that don't give common exams, faculty work to make sure students are being assessed by similar metrics and at similar times across sections of the same course.

Similarly, it's not uncommon to give final exams that are nationally standardized (from the American Chemical Society), where everyone in the country gives the same final exam for the same level course.

It obviously works better in some disciplines than others, but it's not like the concept is foreign- just maybe not a good fit for your department.

I work in a department that does that, too.  For example, Exam 1 for all sections of a particular course will be the same for a particular term.  Then, we (meaning the faculty teaching team) will get together to analyze the results and make changes to those questions.  Fifty percent of the questions are changed every term. The difference is that we, the faculty, are in charge of this process of exam writing.  The whole team.  Not one person's exam that admin-types have deemed appropriate and forcing all others to use.

If they want to standardize across sections of a single course, that would be workable.  However, faculty must make the decision on the actual exam writing.  This does not seem to be the case here.  In addition, the weighting of each exam is up to the teaching team.

I think the big differences are between disciplines where the content and the skills learned through that content are seen as identical, and ones where there is a looser connection between the two. It can make sense to have standardized courses in Chemistry, because the idea is that there are certain things students should know how to do to get to more advanced courses, and they can't know how to do them unless they were taught them. In English, you want students to come out of an intro course with certain skills, but it isn't as closely tied to the actual books they read. You could standardize it and make everyone teach the same five books, but there's no real rationale for it. You'll get better classes if you let professors pick their own books and have clear ideas of what students should be learning how to do through reading them and writing about them.

[polly_mer]

I just came from the "My Students are Getting Worse" thread.  While aster's institution has many problems with the implementation details, having a few terms of the same assessment weighted heavily enough that students will take it seriously means one has evidence to show accreditors that students are indeed meeting the learning outcomes or at least information is being used to modify classes to have a larger percentage of students meeting the outcomes.

In the past 20 years, the requests for proving that a college education does indeed inculcate the stated skills (or at least filters so that everyone who graduates has those skills) have become very, very loud.

The academic freedom for faculty is supposed to be the ability to choose which effective teaching technique will be used to achieve the stated goals, not give blanket permission for individual faculty members to do anything with no oversight.  While a standardized test is not appropriate for many classes, a standardized assessment with some ease of data crunching for comparison across sections and terms is pretty common.  While faculty may not like it, a pattern across several sections in different terms of one faculty member's courses always scoring significantly worse probably does warrant a discussion in the relevant department.  Anyone can have one dud section; every section being duds and only duds for one faculty member indicates a problem.

With the nickname "Big Urban", I'm betting we're not talking a hundred papers from the senior capstone taken by all students that a faculty committee can score and then discuss or similar one-shot-for-everyone-with-faculty-spending-two-days-discussing-what-the-results-mean-for-this-cohort-of-students.  Getting everyone to send in useful assessment data instead of random crap at some point in the next term or two is difficult and gets worse the more people involved. 

Failing to have an effective student learning assessment program that includes using the data to make changes in courses is increasingly a way to go on probation with the accreditor.  Being on probation or then later "show cause" is worse than figuring out how to make the assessment program work for your department so you can use the data for your departmental needs.

[ciao_yall]

At our CC, there was a standard course outline for each course, so everyone had the same student learning objectives*. The objectives were then mapped to program (certificate, degree) outcomes, which then mapped to the institutional outcomes.

At the end of each semester, faculty decided, on their own, which objective to measure and how to measure that. Maybe it was a question on an exam, maybe it was an element of their class project, whatever. Then they reported this into the magic system.

Then data would be rolled up and Reports were written and Spiffy Charts and Graphs were made..

Surprise, across the college, about 70% of students achieved the objective. These were 99.999999% correlated with the students who passed the class. The 30% who did not achieve the objective were 99.999999% correlated with the students who failed the class.

Disaggregating across ethnic, etc groups, the data was consistent. Groups of students less likely to pass the class were also less likely to achieve the objective. Woohoo.

Benefit - it gave faculty teaching the same, and similar courses, a space in which to talk about teaching, learning and best practices.

Cost - some software licenses, time to set it up. A few extra minutes on top of grading to report SLOs. Some faculty took it very seriously and then realized that with 99.99999% correlation there wasn't that much value-added in fine-tuning "Meets" vs "Partially Meets" vs "Did not Meet" the objective.


* Or outcomes? One buzzword is considered dated and passee but I can't remember which.

[Caracal]

While a standardized test is not appropriate for many classes, a standardized assessment with some ease of data crunching for comparison across sections and terms is pretty common.  While faculty may not like it, a pattern across several sections in different terms of one faculty member's courses always scoring significantly worse probably does warrant a discussion in the relevant department.  Anyone can have one dud section; every section being duds and only duds for one faculty member indicates a problem.

In principle, there's nothing wrong with that as long as
1. Departments actually get to determine what the assessments are, they can be based on things that most faculty teaching a course are already doing, and there is enough flexibility to allow them to be tailored for particular classes.
2. Various factors that make classes different are actually taken into account. Some courses have more non majors in them, some courses are more challenging for students for various reasons. As we've seen with assessment in secondary education, a lot can go wrong here.

[pepsi_alum]

We went through drama about this exact issue at my last job. The problem was not the assessment itself as much as it was the sense that people leading the assessment push had a hidden agenda of their own and/or weren't willing to take responsibility for problems or shortcomings in the process.

Case in point #1: my former department was pressured by the central assessment office to write an assessment plan and report several years before I got there. It was difficult to get faculty support for it, but they did eventually get a majority of faculty on board and spent considerable time and effort producing a detailed assessment plan and report, which than sat unread on a bookcase for several years. When the department was asked to revisit assessment several years later and replied "We'll start with our 2008 plan," the same head assessment person replied "Oh, that plan won't be acceptable anymore." This led to the feeling that the assessment person was not respecting faculty time or being transparent about their true purpose.

Case-in-Point #2: one of my colleagues who was a low-performing instructor by most other measures  got funded to go to a fancy assessment workshop of some sort and came back assessment crazy. But colleague's teaching became even more incoherent: I remember vividly that they created a 6-page rubric for grading a 4-page student essay that measured bizarre, irrelevant things. (I can't go into specifics, but earning an "A" on the essay depended more on reading an intricate rubric than it did reading the article on which the essay prompt was based). They also became indignant that other instructors and I refused to use a rubric that was longer than the assessment we were grading. My colleague also believed that ever single assignment in a class—down to individual reading quiz questions and participation grades—needed to be linked to specific course SLOs or otherwise wasn't a "valid" grading tool. I wouldn't say that this improved their teaching at all. If anything, it made then more rigid and more prone to fuss about how terrible students were. This colleague sort of jumped the shark when they starting advocating a bizarre grading scheme that linked course grades to SLO mastery in addition to percentages. Think along the lines of "to earn an 'A' in this class, you must have a 90% or higher in SLO 1, a 90% in SLO 2, and so on." They never actually implemented this system, but needless to say, they didn't endear themselves to either students or other faculty.

Believe it or not, I don't think all assessment is bad. But there needs to be transparency about purpose and faculty buy-in, and it can't be farmed out to rogue agents who hijack the process for their own benefit.

[ciao_yall]

Benefit - it gave faculty teaching the same, and similar courses, a space in which to talk about teaching, learning and best practices.

One more benefit - Because outcomes for courses had to map to programs, it made everyone be more intentional about why a course was required or recommended for a particular program. Many courses were included as required because the instructor whined about having enrollment for their pet class, not because it was relevant or important for that particular degree or certificate.

[polly_mer]

Benefit - it gave faculty teaching the same, and similar courses, a space in which to talk about teaching, learning and best practices.

One more benefit - Because outcomes for courses had to map to programs, it made everyone be more intentional about why a course was required or recommended for a particular program. Many courses were included as required because the instructor whined about having enrollment for their pet class, not because it was relevant or important for that particular degree or certificate.

That's an immensely "fun" discussion to have as people find themselves unable to map anything in their course to even the general education goals.

One of my "favorites" was right after we went through the pain of revamping the general education program and then had mandatory review of every course to ensure it met the new goals.  One particular course went through the committee several times because the arguments went:

"The title of the course is <Nouns> Around the World.  Therefore, it clearly meets the global diversity goal.  Students are required to write 3 papers, so it clearly also meets the writing goal."

"Each of those goals have three components.  Clearly, it will be easy for a faculty member to write one sentence indicating how each component is addressed.  This can be an overview or tying a particular activity/assignment to each component.  The assessment block should then have a sentence or two regarding how each goal will be assessed."

"But it's so obvious how the attached syllabus meets all those components and assessments"

"Then it will take almost no time at all to write the relevant sentences on the form."

"No, just read the syllabus and it's clear."

"No, if you want to teach this class, then write the sentences to ensure we're all thinking the same thing regarding the goals."

Eventually, I ended up sitting with the faculty member and asking them to tell me about the course so I could fill out the form.  No, the syllabus wasn't as clear and obvious as the faculty member claimed because the faculty member apparently hadn't read all the components for each goal and so this course as designed didn't meet several of the components for any of the goals.

I found this hilarious because the faculty member sat on the renovated committee, which meant they were in the room while we discussed what each goal meant and argued for months on particular wording for each component.  This particular faculty member also was part of the committee reading the paperwork for every course being reviewed and was in the room as we talked about whether each nspecific course met the components or needed to be returned for revision. 

Don't even get me started on the faculty member who argued the equivalent of his course goal of learning to water ski including specific acrobatics was absolutely going to be met by the units on how to climb a tree, how to drive the boat, and how to downhill ski.  I won't tell you how to teach your course so far outside my expertise, but I will question how students could possibly meet the objectives of the course when nothing in the planned syllabus addresses any of those skills or knowledge areas.

[mythbuster]

I am one who would love to have a common final exam (or at least a test bank), four our large intro course. This course is taught by at least six different faculty members on a regular basis. However, I am smart enough NOT of offer to spearhead said effort, because I know from past experience how well these ideas go over.
     I learned my lesson the time I suggested in a faculty meeting that we as a department should agree one on bibliographic format for all student papers in the major! I wasn't even advocating for any particular format, just that we all pick one. Some people have strange choices in terms of which hill they are willing to die on.