A whole new ballgame in cheating. Introducing ChatGPT

[apl68]

In real estate: https://www.usatoday.com/story/money/personalfinance/real-estate/2023/01/31/real-esatate-agents-chatgpt-ai/11155895002/

That businesses are using this thing to write routine notices and scripts just shows how little actual thought--as opposed to stringing together expected cliches--goes into writing those pieces. 

This doesn't bode well for writers of genre fiction. 

[Caracal]

I've not read through the entire thread, but it seems to me to strengthen the case for relying on f2f, proctored paper exams with no electronics (other than possibly a calculator) allowed. In that sense, ChatGPT isn't that revolutionary as a cheating tool - plagiarism, ghost writers, etc have been around a long time.

Yes, exactly. It's just a search engine that returns results in language form instead of lists. It can't think and it can't make arguments on its own. It can reproduce arguments that already exist. If an assignment is already vulnerable to plagiarism, ChatGPT probably exacerbates the issue by allowing a student to cheat more easily and in a way that is less detectable. If an assignment is designed in a way that makes it hard to plagiarize, ChatGPT isn't going to make any difference.

[jerseyjay]

Over the weekend I've played around with ChatGPT. On one hand, it is impressive. I have asked it questions in three languages, and it has responded with competently written responses in each. Some of the answers are pretty good. It does a really good job of imitating a competent but not really good essay. However, some are really weird.

When I asked it to compare and contrast Colombia and Venezuela, it gave some decent answers, but then said that Venezuela is mainly Protestant while Colombia is mainly Catholic. (This is not true.) 

When I asked it to compare French and Spanish grammar, it gave some good responses, but said that Spanish has many neuter nouns while French has gendered nouns. (I then asked it for examples of neutral Spanish nouns and all of them were gendered, e.g., la mesa.)

When I asked it to compare the Spanish in Mexico City to the Spanish in Monterrey, one of the differences was that in Mexico City the slang is more formal and in Monterrey the slang is more informal. I am not sure what that means.

When I asked if the Bolshevik Revolution would have happened without Lenin, it argued that it would have (which is an okay thesis I guess) and argued in part:
"Finally, the Bolshevik Revolution was also helped by the chaos of the Russian Civil War, which followed the February Revolution of 1917. The civil war was a chaotic and violent period in Russian history, and it created the perfect conditions for a revolutionary movement to arise."
This is just a jumble that mixes cause and effect.

Finally asked who I was and its response was mainly correct (it summarized my specialty pretty well) but it said I was a professor emeritus at a school I've never taught. I asked again who I was (using my middle initial), and it got my specialty more or less accurate again, again listed me as an emeritus professor (at a slightly different university) and gave a list of books that I didn't publish but which are related to the ones I have. I then asked it to write a five-paragraph essay about the historiographical significance of my research. It gave a slightly different university that it said I taught at, identified my general specialty, attributed a book I didn't write to me, and made the (unfortunately wrong) assertion that my "work has been highly influential in [my] field.... and [my] contributions have been integral to the development of the field in recent decades."


So at this moment, I am not particularly afraid of it becoming too useful. It seems best used to write first drafts of simple topics that the author, having knowledge of the field, can edit to make more accurate and more specific. In other words, it seems likely that many students who do not know anything will generate essays that are off base.

[Wahoo Redux]

The thing is (and several peeps have posted this idea) that this technology is going to advance very quickly, particularly now that it is getting so much social media and news media attention.  Right now it would be only marginally useful for academic purposes----although businesses are apparently using it already----but that is going to change, very soon.

[Caracal]

The thing is (and several peeps have posted this idea) that this technology is going to advance very quickly, particularly now that it is getting so much social media and news media attention.  Right now it would be only marginally useful for academic purposes----although businesses are apparently using it already----but that is going to change, very soon.

That might be true, but it's going to get better at the things it is designed to do. The comparison, again, is search engines. Using search engines used to be much more like using JSTOR. To get the right results for anything, you usually had to zero in on your search terms and set the parameters correctly. Now, you don't usually need to do any of that, google is really good at using natural language to figure out what you're looking for.

I'd be interested to hear from people in relevant fields about this, but it seems like the hard part for CHATGPT is that it is trying to give you a single answer, which gives it less margin for error. If google returns a results for jerseyjay's emeritus colleague when you search for him, that isn't a big problem as long as most of the results are right. Anybody looking at the results should be able to easily discard the results that are obviously not what they are looking for. However, CHATGPT is trying to give an answer and if it scrubs the wrong things from the internet, it's just wrong. Obviously, it can get better, but I'm not sure we should expect the progress to be that rapid.

As for the inability to make new arguments-as someone wrote way upthread, it is not a general artificial intelligence bot. It isn't reasoning, which is why when it can't find an argument, it creates word soup. It isn't going to start reasoning and making arguments that don't exist elsewhere because it isn't made to do that.

[AmLitHist]

As a crocheter (and a composition teacher), I absolutely loved this article:  "ChatGPT Keeps Imploding Because of Crochet. (Seriously.)."

I'm off to ask it to write me a pattern for a stuffed elephant.

[Kron3007]

The thing is (and several peeps have posted this idea) that this technology is going to advance very quickly, particularly now that it is getting so much social media and news media attention.  Right now it would be only marginally useful for academic purposes----although businesses are apparently using it already----but that is going to change, very soon.

That might be true, but it's going to get better at the things it is designed to do. The comparison, again, is search engines. Using search engines used to be much more like using JSTOR. To get the right results for anything, you usually had to zero in on your search terms and set the parameters correctly. Now, you don't usually need to do any of that, google is really good at using natural language to figure out what you're looking for.

I'd be interested to hear from people in relevant fields about this, but it seems like the hard part for CHATGPT is that it is trying to give you a single answer, which gives it less margin for error. If google returns a results for jerseyjay's emeritus colleague when you search for him, that isn't a big problem as long as most of the results are right. Anybody looking at the results should be able to easily discard the results that are obviously not what they are looking for. However, CHATGPT is trying to give an answer and if it scrubs the wrong things from the internet, it's just wrong. Obviously, it can get better, but I'm not sure we should expect the progress to be that rapid.

As for the inability to make new arguments-as someone wrote way upthread, it is not a general artificial intelligence bot. It isn't reasoning, which is why when it can't find an argument, it creates word soup. It isn't going to start reasoning and making arguments that don't exist elsewhere because it isn't made to do that.

Perhaps, but ChatGTP is just one of may different programs.  Each one will have different strengths and weaknesses, and overall the ability of AI will continue to improve quickly.  I only got my first smart phone about 10 years ago...

[MarathonRunner]

As a crocheter (and a composition teacher), I absolutely loved this article:  "ChatGPT Keeps Imploding Because of Crochet. (Seriously.)."

I'm off to ask it to write me a pattern for a stuffed elephant.

Thanks for sharing! I'm a crochet designer and a crocheter, as well as a PhD candidate. Crochet, unlike knitting, can't be made by machines, so not a huge surprise!

[Langue_doc]

From the Washington Post https://www.washingtonpost.com/opinions/2023/02/06/college-students-professor-concerns-chatgpt/

Why I'm not worried about my students using ChatGPT
By Lawrence Shapiro
February 6, 2023 at 8:00 a.m. EST

ChatGPT has many of my university colleagues shaking in their Birkenstocks. This artificial-intelligence tool excels at producing grammatical and even insightful essays — just what we're hoping to see from our undergraduates. How good is it, really? A friend asked ChatGPT to write an essay about "multiple realization." This is an important topic in the course I teach on the philosophy of mind, having to do with the possibility that minds might be constructed in ways other than our own brains. The essay ran shorter than the assigned word count, but I would have given it an A grade. Apparently ChatGPT is good enough to create an A-level paper on a topic that's hardly mainstream.

Universities are treating the threat as more dire than an epidemic or even a budget reduction. The most obvious response, and one that I suspect many professors will pursue, involves replacing the standard five-page paper assignment with an in-class exam. Others expect to continue with the papers but have suggested that the assigned topics should be revised to focus on lesser-known works or ideas about which a chatbot might not "know" too much.

Good luck with that. If ChatGPT can pen a solid essay on multiple realization, an issue on which I happen to be a world authority in good part thanks to lack of company, I doubt it would have difficulty constructing essays about lesser-known Shakespearean sonnets or unremarkable soldiers who fought for the Union Army. Besides, if we're going to demand deep thought from our students, shouldn't it be about the more important stuff?

Here's what I plan to do about chatbots in my classes: pretty much nothing. Let me say first that as much as I value the substance of what I teach, realistically my students will not spend more than a semester thinking about it. It's unlikely that Goldman Sachs or Leakey's Plumbing or wherever my students end up will expect their employees to have a solid background in philosophy of mind. Far more likely is that the employees will be required to write a letter or an analysis or a white paper, and to do this they will need to know how to write effectively in the first place. This is the skill that I most hope to cultivate in my students, and I spend a lot of time reading their essays and providing them with comments that really do lead to improvements on subsequent assignments. In-class exams — the ChatGPT-induced alternative to writing assignments — are worthless when it comes to learning how to write, because no professor expects to see polished prose in such time-limited contexts.

I should emphasize just how desperately my students need formal instruction in writing. My wife confirms that I'm noticeably crankier than when I first started teaching 30 years ago. Everything today seems worse than it was back then: traffic, TV news, macaroni and cheese. But I don't believe that the deterioration in writing quality that I see is a consequence of age-tinted glasses. I read too many papers from upperclassmen, from students who have taken other writing-intensive courses, in which only one sentence out of five is not grammatically or stylistically defective. I would be failing these students if I let ChatGPT discourage me from teaching them what might be the most essential competence they can gain from me.

But what about the cheaters, the students who let a chatbot do their writing for them? I say, who cares? In my normal class of about 28 students, I encounter one every few semesters whom I suspect of plagiarism. Let's now say that the temptation to use chatbots for nefarious ends increases the number of cheaters to an (unrealistic) 20 percent. It makes no sense to me that I should deprive 22 students who can richly benefit from having to write papers only to prevent the other six from cheating (some of whom might have cheated even without the help of a chatbot).

Here's an idea for extracting something positive from the inevitable prominence that chatbots will achieve in coming years. My students and I can spend some class time critically appraising a chatbot-generated essay, revealing its shortcomings and deconstructing its strengths. This exercise would bring a couple of rewards. First, analytical writing, like any skill, benefits from seeing examples of what works and what does not. While students might reasonably object to having their own essays made a target of public inspection, chatbots couldn't possibly care. Second, given that chatbots are not going to fade away, my students might as well learn how to refine their products for whatever uses the future holds.

I urge my colleagues not to abandon writing assignments for fear that some students will let artificial intelligence do their work for them. Instead, let's devise ways to make chatbots work for all of us. Truly, the cheaters are only hurting themselves — unless we respond to them by removing writing assignments from the syllabus.

[Parasaurolophus]

Philosophers discuss Shapiro's column here (Shapiro is a philosopher).

I'm with the bulk of the commenters. There's a lot more cheating than that at my institution. A lot a lot more.

I had a 'who cares' attitude to cheating when I only found a plagiarist once in a blue moon. I mean, I'd deal with it, but I didn't go out of my way to find cheaters. But since coming here, I routinely find at least 30% of each class plagiarizing or cheating in some other way. Since I teach eight sections of 35 students a year, that's a lot of cheating. Worse, I suspect a larger number of also cheating, but am unable to substantiate it (contract cheating was particularly big here before ChatGPT). Shoving my head in the sand would be more pleasant, but it also means seriously compromising (my already heavily compromised) standards.

[Kron3007]

Philosophers discuss Shapiro's column here (Shapiro is a philosopher).

I'm with the bulk of the commenters. There's a lot more cheating than that at my institution. A lot a lot more.

I had a 'who cares' attitude to cheating when I only found a plagiarist once in a blue moon. I mean, I'd deal with it, but I didn't go out of my way to find cheaters. But since coming here, I routinely find at least 30% of each class plagiarizing or cheating in some other way. Since I teach eight sections of 35 students a year, that's a lot of cheating. Worse, I suspect a larger number of also cheating, but am unable to substantiate it (contract cheating was particularly big here before ChatGPT). Shoving my head in the sand would be more pleasant, but it also means seriously compromising (my already heavily compromised) standards.

Never fear, there's a bi for that too!

https://www.npr.org/2023/01/09/1147549845/gptzero-ai-chatgpt-edward-tian-plagiarism

https://www.bigrapidsnews.com/news/article/cheaters-beware-chatgpt-maker-releases-ai-17754244.php

[Caracal]

Philosophers discuss Shapiro's column here (Shapiro is a philosopher).

I'm with the bulk of the commenters. There's a lot more cheating than that at my institution. A lot a lot more.

I had a 'who cares' attitude to cheating when I only found a plagiarist once in a blue moon. I mean, I'd deal with it, but I didn't go out of my way to find cheaters. But since coming here, I routinely find at least 30% of each class plagiarizing or cheating in some other way. Since I teach eight sections of 35 students a year, that's a lot of cheating. Worse, I suspect a larger number of also cheating, but am unable to substantiate it (contract cheating was particularly big here before ChatGPT). Shoving my head in the sand would be more pleasant, but it also means seriously compromising (my already heavily compromised) standards.

You mean that you are bringing in 30 percent of your class to talk to you about plagiarism in their papers? Having to check their papers to see how extensive it is? If I was doing that, I would never get any of my grading done, actually do any teaching or live the rest of my life in any way. That just isn't a sustainable or reasonable approach. Disciplines can be really different in terms of the sort of things we assign and how easy it is for students to cheat on these things, so I don't want to make unfair assumptions, but if I thought that many students were cheating in consequential and successful ways, I'd change the assignment.

The way to avoid compromising  academic standards is not to wage a time consuming war on cheating, it's to manage and control it.

Sometimes that means just not caring too much about academic dishonesty when it isn't consequential. I assign a lot of reading responses. These are really easy for students to cheat on in all kinds of ways-as are the CMS reading quizzes I sometimes assign instead. That's fine, the assignments don't count for a large percentage of the grade and the point is just to get students to remember they are supposed to do the reading. It works. When I don't have quizzes nobody does the reading and we can't have discussions in class. The reading responses and quizzes get enough students to actually do the reading. Could I do things to make it more difficult for students to cheat on these assignments? Sure, but then I would be spending a lot more time dealing with them. Ultimately, students who don't do the reading are just shooting themselves in the foot on the exams.

When the assignments matter more, my goal is just to structure them in a way so that the benefits of cheating are likely to be pretty limited. In lower level classes that usually means very directed questions about some particular database or source. (Go to x page, find one of these, answer these 4 questions about it.) In upper level classes, students have more latitude, but I'm usually asking them to write on sources from some particular collection in a particular way. Do these things make cheating impossible? No, of course not, but it removes some of the temptation, makes it easier to spot, and also means that if students do get away with it, they are rarely getting that much benefit from it.

Do some lazy students save themselves time and effort by cheating to get mediocre grades without having to do as much work? I'm sure they do sometimes. Rampant cheating isn't some new problem caused by the internet. We live in an imperfect world. That doesn't mean I should ignore obvious cheating when I see it, but I'm a teacher, not an academic crimes investigator.

[marshwiggle]

The way to avoid compromising  academic standards is not to wage a time consuming war on cheating, it's to manage and control it.

Sometimes that means just not caring too much about academic dishonesty when it isn't consequential. I assign a lot of reading responses. These are really easy for students to cheat on in all kinds of ways-as are the CMS reading quizzes I sometimes assign instead. That's fine, the assignments don't count for a large percentage of the grade and the point is just to get students to remember they are supposed to do the reading. It works. When I don't have quizzes nobody does the reading and we can't have discussions in class. The reading responses and quizzes get enough students to actually do the reading. Could I do things to make it more difficult for students to cheat on these assignments? Sure, but then I would be spending a lot more time dealing with them. Ultimately, students who don't do the reading are just shooting themselves in the foot on the exams.

When the assignments matter more, my goal is just to structure them in a way so that the benefits of cheating are likely to be pretty limited. In lower level classes that usually means very directed questions about some particular database or source. (Go to x page, find one of these, answer these 4 questions about it.) In upper level classes, students have more latitude, but I'm usually asking them to write on sources from some particular collection in a particular way. Do these things make cheating impossible? No, of course not, but it removes some of the temptation, makes it easier to spot, and also means that if students do get away with it, they are rarely getting that much benefit from it.

Do some lazy students save themselves time and effort by cheating to get mediocre grades without having to do as much work? I'm sure they do sometimes. Rampant cheating isn't some new problem caused by the internet. We live in an imperfect world. That doesn't mean I should ignore obvious cheating when I see it, but I'm a teacher, not an academic crimes investigator.

This seems like it should be part of an FAQ for new instructors.

[Parasaurolophus]

Caracal: I have, in fact, been constantly changing my assessments to circumvent the cheating I encounter. That, too, is time-consuming, and has not proven very effective. Cheating is endemic at this institution and, IMO, the university's unscrupulous recruitment practices are a big part of the problem (i.e. they're basically running a scam for international tuition, and taking in a lot of desperate students who simply do not have the language skills to undertake a university-level education in English. And I really mean it--a sizeable number of my students are functionally illiterate in English. This is not their fault, but it makes things difficult.). The culture of cheating here reinforces itself, as these things do, and is not helped by the culture of cheating in the regions we recruit from.

Before the pandemic, I'd moved most of my assessments to in-class work to circumvent the cheating. But now I teach mostly online. And even for timed evaluations like exams, I find the questions posted to Chegg and other sites in real time. Staying on top of all that is a chump's game, especially when you teach the same course 4+ times every year. There's no way to rejig every assignment every time. But ignoring it is not a viable solution, either.

I don't go out of my way to find the cheating. The 30% or so I catch is obvious. I also don't strictly follow the reporting policies, because it's not worth the time, and because the forms just go to the integrity office to die (this is widely known among faculty here, and we've not yet succeeded in changing things).

So, I cheat-proof assignments as much as I can, and I only bother with obvious cases. But yeah, that still leaves me with 30%+. And that's not a number I can comfortably ignore. If I had as few students cheating as Shapiro does, it would be a different matter.

[Caracal]

Caracal: I have, in fact, been constantly changing my assessments to circumvent the cheating I encounter. That, too, is time-consuming, and has not proven very effective. Cheating is endemic at this institution and, IMO, the university's unscrupulous recruitment practices are a big part of the problem (i.e. they're basically running a scam for international tuition, and taking in a lot of desperate students who simply do not have the language skills to undertake a university-level education in English. And I really mean it--a sizeable number of my students are functionally illiterate in English. This is not their fault, but it makes things difficult.). The culture of cheating here reinforces itself, as these things do, and is not helped by the culture of cheating in the regions we recruit from.

Before the pandemic, I'd moved most of my assessments to in-class work to circumvent the cheating. But now I teach mostly online. And even for timed evaluations like exams, I find the questions posted to Chegg and other sites in real time. Staying on top of all that is a chump's game, especially when you teach the same course 4+ times every year. There's no way to rejig every assignment every time. But ignoring it is not a viable solution, either.

I don't go out of my way to find the cheating. The 30% or so I catch is obvious. I also don't strictly follow the reporting policies, because it's not worth the time, and because the forms just go to the integrity office to die (this is widely known among faculty here, and we've not yet succeeded in changing things).

So, I cheat-proof assignments as much as I can, and I only bother with obvious cases. But yeah, that still leaves me with 30%+. And that's not a number I can comfortably ignore. If I had as few students cheating as Shapiro does, it would be a different matter.

Huh, fair enough, but your institution sounds like its an extreme case. I think there's a fair amount of cheating at my school, but it doesn't approach those kind of levels.