Open Book Test

[HigherEd7]

Has anyone here given open book exams or let students use their notes on the exam? What is the learning benefit to doing this? I was in school this was unheard of.

[Parasaurolophus]

If they have to make their own cheat sheets, they have to find and curate the info for themselves, and still have to understand how to apply it. All that's lost is the memorization requirement.

[marshwiggle]

For open book exams, if they've been keeping up on the material, they may only need to look up an occasional detail. If they haven't, the book won't help much* because, as I told my students, "You can't learn the course in 2 hours."



*An exam that consists of mainly regurgitation of stuff from the textbook and/or lecture notes isn't terribly useful.

[Aster]

If you make an exam open book or open notes, be prepared to have a lot of students who will desire much longer test taking times.

I'd recommend what Parasaurolphus has stated. Not open notes or open book, but something that students have to specifically craft just for the text. And you as the professor directly controls the parameters of that resource. For example, a maximum size for a review sheet. Or that the notes be handwritten. Or that the review sheet is single sided only.

And then (as Marshwiggle said), your exams must be calibrated to prevent false evidence of learning. What that means is that you should not assess directly on topics where answers are able to be merely copied from notes/books to exams without any critical thinking taking place on the student's part.

Exams that allow students to use learning aids are going to be necessarily require more careful thought from professors.

[downer]

Students may plead for an open book test, but they do even worse than usual if you allow it.

Sometimes I allow them to write notes on one 4"x6" index card. Then they have to work out what's important to decide what to put on the card, and then they don't really need the card.

[HigherEd7]

I tried it once and my students did horrible most of them did not study and just depended on their notes! I have been thinking about trying it again and looking for a better way to do it.

[mamselle]

There's another thread on this, with more info.

The search function here is better than the old Fora's was, you just have to be in the Home tab for it to search all threads (i.e., by observation, I happened to type in a word I knew I'd just used while having a different thread open; the results didn't show that entry, but did when I went to the "Home" tab and searched there.)

Does that make it context-sensitive?

In any case that seems to be what works.

I know I typed in a detailed reply to a question very much like that of the OP's; don't really want to do that again now....

M.

[bibliothecula]

I always let them do open-notes, open-book, open-internet, call a friend, work with another person on the exams. It helps teach them how to find information, rather than memorize it, and how to find reliable sources for that information.

[Aster]

I always let them do open-notes, open-book, open-internet, calla friend, as another person exams. It helps teach them how to find information, rather than memorize it, and how to find reliable sources for that information.

I would be very interested in seeing the grade distributions for your course. And seeing those assessments.

The form of assessment described above would not be appropriate for any lower-level learning questions (e.g. anything knowledge-based from Bloom's Taxonomy). Assessment questions would probably need to be at least application-based or higher.

I am very interested in knowing what steps you are taking to ensure that none of your assessment content is being uploaded or in any way distributed online.

And I am not at all sure why you are endorsing cheating. Unless your major assessments are specifically group-based, a student does his/her own work. A friend does not do the student's work for him/her. Nor does anyone else do a student's work for him/her.

[bibliothecula]

It's not cheating. I teach students how to find information, assess it, analyse it, and explain it to others. For questions involving facts, we discuss what kinds of facts are important to have memorized and what can be looked up when the situation demands it. Students all contribute questions that they think should be on the exams, and I always include several of these. Part of this is training them to think like the teachers they will be (and in some cases, already are) and to examine why we study what we do, how we value aspects of that learning and information, and how we weigh its importance.

[ciao_yall]

Has anyone here given open book exams or let students use their notes on the exam? What is the learning benefit to doing this? I was in school this was unheard of.

I did it all the time. Mainly because it was more about applying the concepts they had learned and less about memorizing factoids. They would be asked to analyze a complex case using very specific terms.

[pepsi_alum]

I think Bibliothecula's point is precisely to move students beyond lower-level Bloom's Taxonomy recall/define/summarize questions and into analysis/synthesis/explanation. Assuming that students are writing their own answers, it's just as rigorous (if not more so) than closed-book, closed-note testing.

When I taught I taught online classes at my last university, I did open-note, open-book exams because the university didn't subscribe to a proctoring service. Averages were slightly higher than my in-person exams, but not by much. (The classes still had the same overall grade distribution because more students just plain quit the online class midway through).

[downer]

Tests for online courses are, necessarily (except for proctored exams), open book. But then the assignments are either basic and the tests are just highlighting important info and giving students ways to add some sorely needed points, or else they are more like mini-papers. The grade distribution tends to be pretty much the same as normal, though it is harder for students to completely fail on them. It's more common for students to just fail to submit anything.

[polly_mer]

I always let them do open-notes, open-book, open-internet, calla friend, as another person exams. It helps teach them how to find information, rather than memorize it, and how to find reliable sources for that information.

I would be very interested in seeing the grade distributions for your course. And seeing those assessments.

The grades come out about the same because, as Marshwiggle wrote, no one can learn the whole course in an hour or two.  The good students who know a lot will not spend much of their time helping someone who has nothing to contribute, but a couple good students who put their heads together can solve much harder problems than they can solve individually.

The form of assessment described above would not be appropriate for any lower-level learning questions (e.g. anything knowledge-based from Bloom's Taxonomy). Assessment questions would probably need to be at least application-based or higher.

Yes, most of the questions were multi-part computations applying several key ideas from the lecture.  I would always put in a few true/false and multiple guess, but they were not just definitions that one could look up.

I am very interested in knowing what steps you are taking to ensure that none of your assessment content is being uploaded or in any way distributed online.

I went the other way and ensured that the whole class had last time's exam as a practice exam and posted the answers myself.  As bibliotheca wrote, the point is to learn the material.  Having more practice available helps the students who truly want to learn.  Merely knowing the answers to a similar test is not all that helpful to the students who are trying to avoid learning.

And I am not at all sure why you are endorsing cheating. Unless your major assessments are specifically group-based, a student does his/her own work. A friend does not do the student's work for him/her. Nor does anyone else do a student's work for him/her.

What kind of jobs have you had that are outside of formal education?  Nearly everything I do in my current job as professional scientist is as part of a team.  We're currently setting up our formal goals for the year at my institution and we have multiple mandatory, prepopulated, cannot-edit-for-individual-plans goals related to functioning effectively in a team.  Yes, I have to do good work as my part of being a team and I'm on my specific teams as the subject matter expert, but the whole endeavor will fail if all we have are excellent SMEs who all work alone.

I would be very surprised if students could find a friend who would do the whole test for them in the time allotted.  Even buying an expert would not be easy  in the time allotted for the advanced engineering classes where I allowed open everything, but you only get two hours, and I purposefully set a test that takes me longer than two hours and then scale based on a variety of factors.

[Aster]

Thank you for the responses. I am pleased to see that some people actually put careful thought into their assessments. And not just the assessment itself, but the delivery of the assessment, and who actually may be completing the assessment.

I used to think that professors were more professional with their assessment strategies to ensure fairness, appropriate rigor, and security.

But it grated on me to keep going to department meetings and having people constantly complain about cheating, other professors' lax grading standards, being stuck with terrible students because the pre-requisite professors weren't assessing properly, blaming adjuncts for everything, blaming the administration for everything, blah blah blah. Plus, the complaints never seemed to go anywhere. Most complaints were based on anecdotal hearsay. Complainers rarely offered solutions.

And then I stopped taking my colleagues' words for it, and began directly checking into professors' assessments and grade distributions. The resources at Big Urban College allow me to do this very easily.

Curiously, the majority of the courses with severe problems (within my discipline) are the fully online courses. With these specific issues.
- online assessments are *not* high-level Bloom's Taxonomy (they are regurgitation knowledge questions that are easily googled)
- online assessments are *not* secured from repeat viewing or copying (and yes, I've seen some of my colleague's exams pirated onto the internet. Answer keys also)
- online assessments are *not* group-work based, but there is no way to prevent students from completing as groups or with anybody else (professors are not implementing the available security tools)
- online assessments are *not* original nor frequently changed out (to offset successful cheating)
- online assessments are few and high-stakes (this enhances pressure to cheat)
- there are online advertisements in the area where students can pay somebody else to take their online assessments for them (and then they list example courses which happen to be courses in my department)

These courses are our department's "problem" courses that we get no support from administration to correct. The grade distributions are among the best in the college. The classes instantly fill. That's "success" to any outside of the Academy. The TT and adjunct faculty that engage in these shenanigans are extremely difficult to remove from their courses. They just leapfrog the department heads, deans, and provosts and talk directly to the president about their sky-high pass rates, sky-high attendance figures, and sky-high Rate-My-Professor ratings.

So while I will apologize if I'm coming off sounding rough when people describe fully open-book, non-secured assessments, know that I have seen these practices heavily abused, and I continue to see them heavily abused. I would like to see professors take their assessment duties both seriously and competently, and I would like professors to understand the special challenges of operating fully online assessments.

I know very few professors that don't recycle their same exams over and over again. This is not so terrible if exams are heavily secured and never released to students. But a professor is an idiot if he/she releases exams (with or without keys) and then reuses the exact same exam within consecutive terms.

I know far too few professors that even know what Bloom's Taxonomy is (or any other standardized learning strategy hierarchies). Not that this is essential knowledge in my discipline, but it certainly nips the next problem in the bud.

I know far too many professors that only assess on regurgitation, memorization-only knowledge. This is fine if assessments are heavily secured, and the completion of those assessment heavily proctored. But a professor is an idiot if he/she crafts purely knowledge-based exams online, doesn't secure them online, and doesn't proctor online assessments or activate any of the standard preventative tools (e.g. randomized questions, randomized responses, single question viewing, timed responses, single attempts).