A whole new ballgame in cheating. Introducing ChatGPT

[Caracal]

I wonder if multiple students using ChatGPT used the same prompt -- say, the prompt from the assignment sheet -- would the responses be similar enough to flag Turnitin or other plagiarism detection? Even if the students subsequently attempted to personalize the AI-generated response, would the AI, given similar prompts provide significantly similar answers?

My guess is that it would-but those answers would also be pretty similar to a lot of the standard bad student answers-vague statements, no evidence etc. One of the things that makes grading so depressing is that bad student answers really cluster together and blend in to each other.

[RatGuy]

I wonder if multiple students using ChatGPT used the same prompt -- say, the prompt from the assignment sheet -- would the responses be similar enough to flag Turnitin or other plagiarism detection? Even if the students subsequently attempted to personalize the AI-generated response, would the AI, given similar prompts provide significantly similar answers?

My guess is that it would-but those answers would also be pretty similar to a lot of the standard bad student answers-vague statements, no evidence etc. One of the things that makes grading so depressing is that bad student answers really cluster together and blend in to each other.

Sure, but there's a difference between two vague and superficial analyses which have similar content, and two vague analyses which contain such substantial similarities that they trigger Turnitin's sensors. I've had plenty of C- essays which are vague and superficial, but they have zero text matches. The ones which trigger Turnitin are those that significantly match internet sources or other students.

[Kron3007]

I wonder if multiple students using ChatGPT used the same prompt -- say, the prompt from the assignment sheet -- would the responses be similar enough to flag Turnitin or other plagiarism detection? Even if the students subsequently attempted to personalize the AI-generated response, would the AI, given similar prompts provide significantly similar answers?

My guess is that it would-but those answers would also be pretty similar to a lot of the standard bad student answers-vague statements, no evidence etc. One of the things that makes grading so depressing is that bad student answers really cluster together and blend in to each other.

Sure, but there's a difference between two vague and superficial analyses which have similar content, and two vague analyses which contain such substantial similarities that they trigger Turnitin's sensors. I've had plenty of C- essays which are vague and superficial, but they have zero text matches. The ones which trigger Turnitin are those that significantly match internet sources or other students.

I tried entering the same very simple  prompt twice in a row.  The output was very similar (not identical) and would likely trigger Turnitin.  However, it is unlikely students would use the exact same phrasing, so I doubt it would catch it.  Further, I used a very simple prompt and they were not identical.  As the input gets more complex, I suspect the differences would grow.  So, I doubt Turnitin woukd be very effective and students could probably get away with using this.  They probably already are...

[bio-nonymous]

I was inspired by this entire thread and had some experiments I wanted to try, this is what I got:
" ChatGPT is at capacity right now
Get notified when we're back

Write a sonnet about the status of ChatGPT.

Amidst the rush of curious minds,
The chatbot ChatGPT stands strong and true,
Withstanding the influx of requests,
Working hard to fulfill each one's due.

But alas, the server cannot cope,
And the error message rings loud and clear,
"Please check back soon," it gently hopes,
As it begs for a moment's reprieve, to reappear.

As the chatbot works to restore its might,
We wait in anticipation, with hopeful hearts,
For the chance to chat and learn, with delight,
And see what knowledge and wisdom it imparts.

So hold on tight, and wait for its return,
For ChatGPT will soon be back, and your patience will be well-earned."

It may be a revolution...

[Wahoo Redux]

I was inspired by this entire thread and had some experiments I wanted to try, this is what I got:
" ChatGPT is at capacity right now
Get notified when we're back

Write a sonnet about the status of ChatGPT.

Amidst the rush of curious minds,
The chatbot ChatGPT stands strong and true,
Withstanding the influx of requests,
Working hard to fulfill each one's due.

But alas, the server cannot cope,
And the error message rings loud and clear,
"Please check back soon," it gently hopes,
As it begs for a moment's reprieve, to reappear.

As the chatbot works to restore its might,
We wait in anticipation, with hopeful hearts,
For the chance to chat and learn, with delight,
And see what knowledge and wisdom it imparts.

So hold on tight, and wait for its return,
For ChatGPT will soon be back, and your patience will be well-earned."

It may be a revolution...

Brilliant.

[marshwiggle]

Would it be possible to give some sort of IQ test to ChatGPT? I'd guess there must be some that would have purely verbal questions.

[Puget]

Would it be possible to give some sort of IQ test to ChatGPT? I'd guess there must be some that would have purely verbal questions.

Can't remember where I read this now, but someone apparently did and it scored about an 85. I'm not sure what IQ test that was though-- yes, it would need purely verbal questions so probably something off the internet that may not be particularly valid. Actual validated IQ tests have moved much more toward non-verbal tests that are less subject to cultural and socioeconomic bias.

[marshwiggle]

Would it be possible to give some sort of IQ test to ChatGPT? I'd guess there must be some that would have purely verbal questions.

Can't remember where I read this now, but someone apparently did and it scored about an 85. I'm not sure what IQ test that was though-- yes, it would need purely verbal questions so probably something off the internet that may not be particularly valid. Actual validated IQ tests have moved much more toward non-verbal tests that are less subject to cultural and socioeconomic bias.

Given the responses to the prompts that people have shown, and the sort of "C student" level of output, it seemed to me that it wouldn't be surprising if it had a "measurable" IQ. (And it occurred to me that blind people, among others, may be an audience for purely verbal tests.)

[RatGuy]

One "bonus" question I like to ask my class is their ideal cast for one of our novels. When I asked the AI, it said "I cannot do that since I am not a major motion picture director" then it just gave me a summary of the novel. At least it was upfront about its limitations.

[quasihumanist]

Honestly, I don't care.  If someone wants to cheat their way to a C, they can.  It's not like the students who honestly earn a C have learned anything of value - they've just managed to memorize and regurgitate a bunch of trivia without really understanding it or being able to apply it to anything.  (And if we had higher standards for a C, we would have to fail enough students - many of whom are simply unable to do the work (we can think of them as disabled) - to put our university in danger of closing.)

[marshwiggle]

Honestly, I don't care.  If someone wants to cheat their way to a C, they can.  It's not like the students who honestly earn a C have learned anything of value - they've just managed to memorize and regurgitate a bunch of trivia without really understanding it or being able to apply it to anything.  (And if we had higher standards for a C, we would have to fail enough students - many of whom are simply unable to do the work (we can think of them as disabled) - to put our university in danger of closing.)

This is what prompted my IQ question. It would not be surprising if the lowest "acceptable" standard for our students is within the capabilities of an AI. That says more about *our standards than it does about the AI.

*(and by "our", I mean society's- not individual instructors'. As long as we have this idea that basically everyone should be able to "succeed" at post-secondary education, our standards for "success" will have to be extremely low.)

[Kron3007]

Honestly, I don't care.  If someone wants to cheat their way to a C, they can.  It's not like the students who honestly earn a C have learned anything of value - they've just managed to memorize and regurgitate a bunch of trivia without really understanding it or being able to apply it to anything.  (And if we had higher standards for a C, we would have to fail enough students - many of whom are simply unable to do the work (we can think of them as disabled) - to put our university in danger of closing.)

This is what prompted my IQ question. It would not be surprising if the lowest "acceptable" standard for our students is within the capabilities of an AI. That says more about *our standards than it does about the AI.

*(and by "our", I mean society's- not individual instructors'. As long as we have this idea that basically everyone should be able to "succeed" at post-secondary education, our standards for "success" will have to be extremely low.)

This is true today, but ina. Few years, AI may very well perform at a B or A level.  2 years ago, AI generated text would have failed.  In just a couple years it has developed into a fairly proficient writer.  It seems inevitable that ina short period it may outperform many of our students.

[marshwiggle]

Honestly, I don't care.  If someone wants to cheat their way to a C, they can.  It's not like the students who honestly earn a C have learned anything of value - they've just managed to memorize and regurgitate a bunch of trivia without really understanding it or being able to apply it to anything.  (And if we had higher standards for a C, we would have to fail enough students - many of whom are simply unable to do the work (we can think of them as disabled) - to put our university in danger of closing.)

This is what prompted my IQ question. It would not be surprising if the lowest "acceptable" standard for our students is within the capabilities of an AI. That says more about *our standards than it does about the AI.

*(and by "our", I mean society's- not individual instructors'. As long as we have this idea that basically everyone should be able to "succeed" at post-secondary education, our standards for "success" will have to be extremely low.)

This is true today, but ina. Few years, AI may very well perform at a B or A level.  2 years ago, AI generated text would have failed.  In just a couple years it has developed into a fairly proficient writer.  It seems inevitable that ina short period it may outperform many of our students.

I mostly do labs. So I'm safe until the robots take over.

[Kron3007]

Honestly, I don't care.  If someone wants to cheat their way to a C, they can.  It's not like the students who honestly earn a C have learned anything of value - they've just managed to memorize and regurgitate a bunch of trivia without really understanding it or being able to apply it to anything.  (And if we had higher standards for a C, we would have to fail enough students - many of whom are simply unable to do the work (we can think of them as disabled) - to put our university in danger of closing.)

This is what prompted my IQ question. It would not be surprising if the lowest "acceptable" standard for our students is within the capabilities of an AI. That says more about *our standards than it does about the AI.

*(and by "our", I mean society's- not individual instructors'. As long as we have this idea that basically everyone should be able to "succeed" at post-secondary education, our standards for "success" will have to be extremely low.)

This is true today, but ina. Few years, AI may very well perform at a B or A level.  2 years ago, AI generated text would have failed.  In just a couple years it has developed into a fairly proficient writer.  It seems inevitable that ina short period it may outperform many of our students.

I mostly do labs. So I'm safe until the robots take over.

I have a mix, so am also somewhat safe.  I also prefer oral presentations over written reports, which will help stave off the bots.  However, even Power point is integrating AI to help optimize slide layout etc.  No one is safe!

[marshwiggle]

Honestly, I don't care.  If someone wants to cheat their way to a C, they can.  It's not like the students who honestly earn a C have learned anything of value - they've just managed to memorize and regurgitate a bunch of trivia without really understanding it or being able to apply it to anything.  (And if we had higher standards for a C, we would have to fail enough students - many of whom are simply unable to do the work (we can think of them as disabled) - to put our university in danger of closing.)

This is what prompted my IQ question. It would not be surprising if the lowest "acceptable" standard for our students is within the capabilities of an AI. That says more about *our standards than it does about the AI.

*(and by "our", I mean society's- not individual instructors'. As long as we have this idea that basically everyone should be able to "succeed" at post-secondary education, our standards for "success" will have to be extremely low.)

This is true today, but ina. Few years, AI may very well perform at a B or A level.  2 years ago, AI generated text would have failed.  In just a couple years it has developed into a fairly proficient writer.  It seems inevitable that ina short period it may outperform many of our students.

I mostly do labs. So I'm safe until the robots take over.

I have a mix, so am also somewhat safe.  I also prefer oral presentations over written reports, which will help stave off the bots.  However, even Power point is integrating AI to help optimize slide layout etc.  No one is safe!

So when are the "cheat-detecting" bots coming online? Let the arms race begin!