A whole new ballgame in cheating. Introducing ChatGPT

[marshwiggle]

I am a little surprised by these discussions of what ChatGPT can and cannot do.  Whatever the program cannot do right now it will be able to do in the near future.

NBC News: ChatGPT banned from New York City public schools' devices and networks

Lower Deck:

A spokesperson for OpenAI, which developed ChatGPT, said it is "already developing mitigations to help anyone identify text generated by that system."

That's ridiculous. The whole point of the AI is to produce speech indistinguishable from that written by a human. Making it possible to recognize as not being written by a human defeats the purpose. Anything they would do to make it identifiable would be counter-productive, so at the very least it would be easy to circumvent.

[Puget]

I am a little surprised by these discussions of what ChatGPT can and cannot do.  Whatever the program cannot do right now it will be able to do in the near future.

NBC News: ChatGPT banned from New York City public schools' devices and networks

Lower Deck:

A spokesperson for OpenAI, which developed ChatGPT, said it is "already developing mitigations to help anyone identify text generated by that system."

Huh? Why would we think that? I don't know much about AI or programming, but it isn't magic. They've created something that is very good at scrubbing the internet and paraphrasing what they find there fluently. It can't really make arguments or develop a thesis. This isn't a small thing. It is much of what you're trying to teach students in an intro writing course. Students do the same thing that Chat GPT does-just restate a thesis over and over in different words because they don't know how to develop the thesis and make it persuasive.

It's like watching a Roomba vacuum a room and concluding that the people developing it are on the verge of releasing a robot capable of cleaning your own entire house without your assistance. You just need to add some arms so it can climb up on surfaces and grab trash, and mops so it can wash floors-and oh, yeah, it would need to be able to know how to climb up on your table without damaging it and distinguish trash from non trash. Maybe somebody is trying to build this this robot but it isn't a Roomba with a few tweaks.

Right, I think it is easy to look at ChatGPT generated texts and mistake it for a general AI, but it isn't-- it's a large language model. That means it is very good at generating certain kinds of text, but completely incapable of anything involving actual reasoning (including elementary math and logic problems). It will certainly get better at the language tasks it is designed for, but it isn't going to become a general AI in the process-- that's a very different type of model, and a much harder one to develop (despite a LOT of trying).

[Wahoo Redux]

Comment by a Reddit poster on the "r/professors" thread.

ChatGPT is an excellent writer for letters of recommendation

I've been using it last few weeks.

Me: Need a letter of recommendation for a university student

ChatGPT: Sure, I'd be happy to help you write a letter of recommendation for a student. To get started, can you provide me with some information about the student and your relationship with them? This will help me to personalize the letter and include specific details that will highlight their skills and achievements.

and then go from there. Took me 2 minutes to get a good, personalized letter.

[jimbogumbo]

A look at how it does with geometry "help"  (and a little phonics): https://www.educationnext.org/chatgpt-is-not-ready-to-teach-geometry-yet/

[Caracal]

Comment by a Reddit poster on the "r/professors" thread.

ChatGPT is an excellent writer for letters of recommendation

I've been using it last few weeks.

Me: Need a letter of recommendation for a university student

ChatGPT: Sure, I'd be happy to help you write a letter of recommendation for a student. To get started, can you provide me with some information about the student and your relationship with them? This will help me to personalize the letter and include specific details that will highlight their skills and achievements.

and then go from there. Took me 2 minutes to get a good, personalized letter.

Letters of rec are highly formulaic, or at least they can be. When I write one for something generic, I open up the last one I wrote and modify the details as needed. That's just what ChatGPT is doing, using templates from somewhere. That seems like the kind of thing it can do. I can't imagine it would do a great job with something more specialized, like a grad school or academic job recommendation.

[Wahoo Redux]

I just post the things that I find on the program.

But I do want to point out that already ChatGPT is doing things, as someone else pointed out, that no program did 5 years ago.

Imagine another 5 years of programming. 

Deep Blue lost to Kasparov 4 to 2 on their first meeting and then beat him 3 times and played to a draw only one year later.  And yes, chess is prescribed, formulaic and mathematical and not the same thing as writing a research report or a novel...but imagine this ChatGPT 5 years from now.  Human abilities may not be as precious and unmatchable as we imagine, no matter how threatening the alternative is.

[Sun_Worshiper]

Any undergraduate student that turns in a paper written entirely by ChatGPT in the spring semester will be getting a good grade. For now, none of us can tell the difference - certainly not well enough to be sure that the student didn't write it (and certainly not to prove it). There will be AI-detection tools soon enough, but in the meantime students will cheat freely. And, as several people have already noted in this thread, the technology will become much more advanced very quickly.

The real questions people should be asking (and some are) is how will you adjust your classes.

[Liquidambar]

Any undergraduate student that turns in a paper written entirely by ChatGPT in the spring semester will be getting a good grade. For now, none of us can tell the difference - certainly not well enough to be sure that the student didn't write it (and certainly not to prove it).

Nope.  I received a paper written mostly by ChatGPT.  It was easy to spot, and I could also tell which part was written by the student.  The key elements were 1) the assignment was pretty specific and 2) it involved writing about a source that probably wasn't in the training set (journal article behind a paywall).

My school hasn't tested this with an honor court case yet, but our office that handles such things seems to view the detector linked upthread as pretty solid proof.

[Caracal]

Any undergraduate student that turns in a paper written entirely by ChatGPT in the spring semester will be getting a good grade. For now, none of us can tell the difference - certainly not well enough to be sure that the student didn't write it (and certainly not to prove it). There will be AI-detection tools soon enough, but in the meantime students will cheat freely. And, as several people have already noted in this thread, the technology will become much more advanced very quickly.

The real questions people should be asking (and some are) is how will you adjust your classes.

I'm quite genuinely confused about these takes. This is a ludicrous claim for all the reasons several of us have pointed out. It seems to be based on both a misunderstanding of what CHATGPT is and what papers are actually asking students to do in most fields. As far as I can tell, most of the people making these claims are in STEM or the Social Sciences. Maybe its coming from people who don't really assign papers? Or assign different sort of papers? There just seem to be some people who are oddly invested in the idea that this particular technology is going to upend everything in academia or make papers obsolete. It doesn't seem to matter that those of us who actually assign papers have tried it and found that this thing isn't capable of writing argumentative essays.

To add some nuance, which is lacking here, this has made me thinking of changing a few things. There's no need for me to adjust anything in essays I assign for upper level classes based on this. In lower level classes, it does have me thinking of changing one assignment, I already had some concerns with. The assignment is based on a very well known source which shows up in lots of primary source collections and I ask students a series of very basic questions about the source for them to answer in individual paragraphs. I already had some worries about the possibility for plagiarism, both from a million online sources and from students who have taken the course in a previous semester, but this is the sort of thing where CHATGPT might be able to write something that would get a decent grade, partly because my standards for these sorts of assignments are quite low.

But that's the point. This isn't "a new whole ballgame" it's just a different variation.

[mbelvadi]

Any undergraduate student that turns in a paper written entirely by ChatGPT in the spring semester will be getting a good grade. For now, none of us can tell the difference - certainly not well enough to be sure that the student didn't write it (and certainly not to prove it).

Nope.  I received a paper written mostly by ChatGPT.  It was easy to spot, and I could also tell which part was written by the student.  The key elements were 1) the assignment was pretty specific and 2) it involved writing about a source that probably wasn't in the training set (journal article behind a paywall).

My school hasn't tested this with an honor court case yet, but our office that handles such things seems to view the detector linked upthread as pretty solid proof.

The comment about the article being behind a paywall makes me laugh just a bit at the irony of how hard we in academe have been fighting to remove those paywalls, mostly via the "open access" movement, and the unintended consequence implied here.
Open Access has been exploding with "read and publish" deals in the last couple of years. Don't expect that paywall to still protect the integrity of your assignment in 5 years, anymore than you can expect ChatGPT output to be as grade-poor (eg C level rather than A or B) as it is now, in 5 years.

[Sun_Worshiper]

Any undergraduate student that turns in a paper written entirely by ChatGPT in the spring semester will be getting a good grade. For now, none of us can tell the difference - certainly not well enough to be sure that the student didn't write it (and certainly not to prove it). There will be AI-detection tools soon enough, but in the meantime students will cheat freely. And, as several people have already noted in this thread, the technology will become much more advanced very quickly.

The real questions people should be asking (and some are) is how will you adjust your classes.

I'm quite genuinely confused about these takes. This is a ludicrous claim for all the reasons several of us have pointed out. It seems to be based on both a misunderstanding of what CHATGPT is and what papers are actually asking students to do in most fields. As far as I can tell, most of the people making these claims are in STEM or the Social Sciences. Maybe its coming from people who don't really assign papers? Or assign different sort of papers? There just seem to be some people who are oddly invested in the idea that this particular technology is going to upend everything in academia or make papers obsolete. It doesn't seem to matter that those of us who actually assign papers have tried it and found that this thing isn't capable of writing argumentative essays.

To add some nuance, which is lacking here, this has made me thinking of changing a few things. There's no need for me to adjust anything in essays I assign for upper level classes based on this. In lower level classes, it does have me thinking of changing one assignment, I already had some concerns with. The assignment is based on a very well known source which shows up in lots of primary source collections and I ask students a series of very basic questions about the source for them to answer in individual paragraphs. I already had some worries about the possibility for plagiarism, both from a million online sources and from students who have taken the course in a previous semester, but this is the sort of thing where CHATGPT might be able to write something that would get a decent grade, partly because my standards for these sorts of assignments are quite low.

But that's the point. This isn't "a new whole ballgame" it's just a different variation.

I'm not saying the bolded - and I don't think anyone really is.  What people are saying is that students are going to use it to cheat (true), most of this cheating will go undetected (true, for now, although I admit to being a bit hyperbolic in my last post), and the AI-generative text is going to evolve very quickly (true), becoming harder to detect and more sophisticated in its production.

Anyone not taking this seriously is going to be overlooking a ton of cheating and unethical behavior beyond the classroom as well. We should not panic and we should instead accept that this technology is going to revolutionize many things, probably with a net positive effect, but we also should not ignore its short-term negative effects.

[Wahoo Redux]

found that this thing isn't capable of writing argumentative essays...

...yet.

[Caracal]

Any undergraduate student that turns in a paper written entirely by ChatGPT in the spring semester will be getting a good grade. For now, none of us can tell the difference - certainly not well enough to be sure that the student didn't write it (and certainly not to prove it). There will be AI-detection tools soon enough, but in the meantime students will cheat freely. And, as several people have already noted in this thread, the technology will become much more advanced very quickly.

The real questions people should be asking (and some are) is how will you adjust your classes.

I'm quite genuinely confused about these takes. This is a ludicrous claim for all the reasons several of us have pointed out. It seems to be based on both a misunderstanding of what CHATGPT is and what papers are actually asking students to do in most fields. As far as I can tell, most of the people making these claims are in STEM or the Social Sciences. Maybe its coming from people who don't really assign papers? Or assign different sort of papers? There just seem to be some people who are oddly invested in the idea that this particular technology is going to upend everything in academia or make papers obsolete. It doesn't seem to matter that those of us who actually assign papers have tried it and found that this thing isn't capable of writing argumentative essays.

To add some nuance, which is lacking here, this has made me thinking of changing a few things. There's no need for me to adjust anything in essays I assign for upper level classes based on this. In lower level classes, it does have me thinking of changing one assignment, I already had some concerns with. The assignment is based on a very well known source which shows up in lots of primary source collections and I ask students a series of very basic questions about the source for them to answer in individual paragraphs. I already had some worries about the possibility for plagiarism, both from a million online sources and from students who have taken the course in a previous semester, but this is the sort of thing where CHATGPT might be able to write something that would get a decent grade, partly because my standards for these sorts of assignments are quite low.

But that's the point. This isn't "a new whole ballgame" it's just a different variation.

I'm not saying the bolded - and I don't think anyone really is.  What people are saying is that students are going to use it to cheat (true), most of this cheating will go undetected (true, for now, although I admit to being a bit hyperbolic in my last post), and the AI-generative text is going to evolve very quickly (true), becoming harder to detect and more sophisticated in its production.

Anyone not taking this seriously is going to be overlooking a ton of cheating and unethical behavior beyond the classroom as well. We should not panic and we should instead accept that this technology is going to revolutionize many things, probably with a net positive effect, but we also should not ignore its short-term negative effects.

My impression from people who actually know this stuff (experts please correct me as needed) is that the problems with the largest issues with the AI generated text for academic work are features of what CHATGPT is. It can't make arguments because that isn't what it is designed to do. It might get better at identifying when arguments are being asked for and scrubbing the internet for arguments to combine and paraphrase, but it isn't going to start coming up with them on its own.

There are basically two possible ways to manage plagiarism. One is catching people when they've done it and the other is making it hard for them to do it effectively in the first place. If we aren't thinking about plagiarism when we craft our assignment, and relying on plagiarism checkers to detect it, ChatGPT does mess up that strategy, but it wasn't a good strategy before CHATGPT anyway. If there isn't anything on the internet that is really going to effectively work for an assignment, CHATGTP isn't going to be able to make it up out of thin air. I might actually just start using it when I write assignments as a quick and easy way to assess whether students tempted to cheat might find a lot of material, even if they are just plagiarizing the old fashioned way.

[Wahoo Redux]

I began with the prompt "Bach is the greatest composer of all time.  Agree or disagree." then hit "Regenerate response" or restarted the program with a new prompt.  The only editing I did was to eliminate any verbatim repeats in the language, and I arranged some of the text so it had at least some global structure.  Took a little less than 20 minutes while chatting and petting the dog.

It is difficult to definitively say who the greatest composer of all time is, as opinions on this matter vary greatly. However, Johann Sebastian Bach is certainly one of the most highly regarded and influential composers in the history of classical music. His contributions to the development of Western music are significant and enduring, and he is considered by many to be a master of counterpoint, harmony, and structure. Bach's music continues to be widely performed and studied today, and he is considered one of the greatest composers in the Western tradition.

He is known for his instrumental and choral works, such as the Brandenburg Concert, the Mass in B Minor, and the keyboard works known as the "Well-Tempered Clavier." Bach's music is characterized by its technical precision, complex musical structure, and use of counterpoint. These and many of his other works are considered masterpieces of Western music and are still widely performed and admired today. Bach was also an accomplished organist, and his music for the organ is particularly renowned. He was a prolific composer, writing music for a wide variety of instruments and ensembles, including the organ, harpsichord, violin, and choir. Bach's music is known for its technical complexity, emotional depth, and spiritual expression.  He made significant contributions to the development of the Baroque style and helped to establish many of the forms and techniques that are still in use in classical music today.  He is widely considered one of the greatest composers in the history of Western classical music.

Baroque music is a style of Western art music composed from approximately 1600 to 1750. Characterized by elaborate ornamentation and intricate counterpoint, it was the dominant style of music in Europe during the Baroque period. Some of the most famous composers of Baroque music include Johann Sebastian Bach, George Frideric Handel, Antonio Vivaldi, and Claudio Monteverdi. Baroque music is known for its use of complex tonal counterpoint, elaborate ornamentation, and strong, dramatic contrasts. It was a major influence on the development of classical music, and many of the techniques and forms used in Baroque music are still in use today.

Bach's contributions to music were numerous and far-reaching. He is credited with the development of the modern tonal system, which forms the basis of Western classical music. He also pioneered the use of counterpoint, in which multiple melodies are combined in a way that is harmonically interesting and pleasing to the ear. Additionally, Bach's music had a strong influence on the development of classical music and continues to be an important reference point for composers to this day.

There are many reasons why Bach is considered great, including the following: Technical Mastery: Bach was a highly skilled musician and composer, and his music is known for its complex counterpoint and intricate musical structures.  Creativity: Bach was able to take existing musical forms and conventions and use them in innovative ways, creating new and original compositions.
Range: Bach composed music in a variety of different styles and forms, including choral works, solo instrumental pieces, and concertos.  Influence: Bach's music has had a lasting influence on classical music and has inspired many other composers.  Beauty: Many people consider Bach's music to be beautiful and emotionally powerful.  Overall, Bach's technical skill, creativity, range, influence, and beauty have all contributed to his reputation as a great composer.

It is difficult to say what his "best" work is, as it is a matter of personal opinion and many of his works are highly regarded for their technical proficiency, artistic beauty, and historical significance. Some of his most famous works include the Brandenburg Concert, the Mass in B Minor, the Well-Tempered Clavier, and the St. Matthew Passion. Other notable works include the Cello Suites, the Partitas for solo violin, and the Toccata and Fugue in D Minor.

The Toccata and Fugue in D minor is a famous organ piece composed by Bach in the early 18th century. It is one of Bach's most famous works and is a staple of the organ repertoire. The toccata is an virtuosic, fast-paced piece that showcases the performer's technical skill, while the fugue is a contrapuntal composition in which a single musical theme is repeated and imitated by voices entering one after another. The Toccata and Fugue in D minor is known for its dramatic, intense sound and its use of chromaticism (the use of notes outside of the diatonic scale). It is often used in films and TV shows to create a sense of mystery or suspense.

The Brandenburg Concert are a set of six instrumental works composed by Bach in the early 18th century. They are some of Bach's most famous works, and are considered to be some of the greatest examples of Baroque music. Each concerto features a different combination of instruments, and Bach's use of counterpoint and harmony in these works is particularly noteworthy. The Brandenburg Concerto No. 3, which features three violin parts, is one of the most popular of the set.

The Well-Tempered Clavier (also known as the "48 Preludes and Fugues") is a collection of solo keyboard music composed by Bach in the early 18th century. It consists of two volumes, each of which contains 24 preludes and fugues. The preludes are short, free-form pieces that serve as introductions to the fugues, which are longer, more structured compositions. The Well-Tempered Clavier is notable for its use of all major and minor keys, and it is considered one of the most important works in the history of Western classical music. It is often studied by musicians and is frequently performed in concerts.

In short, Bach was important because he composed some of the most beautiful and enduring music in the Western classical repertoire, and his music continues to be an important influence on musicians and composers today. Bach's music continues to be widely performed and enjoyed by audiences around the world.

It is a matter of personal opinion whether Bach was the greatest composer of all time. There have been many talented and accomplished composers throughout history, and it is difficult to make a definitive statement about who the greatest one is. Some people might consider Bach to be the greatest composer because of the complexity and beauty of his compositions, while others might have different criteria for what makes a composer great. Ultimately, whether or not Bach is considered the greatest composer is a subjective judgment that depends on one's personal tastes and values.

[marshwiggle]

I began with the prompt "Bach is the greatest composer of all time.  Agree or disagree." then hit "Regenerate response" or restarted the program with a new prompt.  The only editing I did was to eliminate any verbatim repeats in the language, and I arranged some of the text so it had at least some global structure.  Took a little less than 20 minutes while chatting and petting the dog.

Not a bad response, but the prompt is a question that all kinds of people have discussed for centuries. So there will be lots of stuff online discussing that exact question.

How about asking "How did Bach influence *Kanye?" and seeing how insightful it is.

*(Or the Beatles, etc.)