CS240 AI Cheating Retrospective (turkeyland.net)

93 pointsby ArchAndStarch6 hours ago68 comments

aenigmaclamo 3 hours ago

  * Do not use variable length array in this class
  * Some of are you are copying code and concepts from things outside of course material
     * `fgetc()`, `sizeof()`, `malloc()`, etc
     * You're probably cheating.
     * You're also making your life more difficult

I always found these sorts of instructions quite frustrating as a student. If I'm a student enthusiastic about learning a language, I would have definitely been one of the kids trying to figure out how to use malloc and creating novel approaches to the projects that are assigned to me. It's a university class, why the hell not? Besides, isn't that going to be an advantage towards me considering the code is going to be evaluated by a anti cheat engine?

The AI stuff aside, this comes across as a professor who feels threatened when his course isn't taken as doctrine. Being called a cheater for using external resources is ridiculous to me.

tyre 3 hours ago

Maybe you would have. Maybe you wouldn't have. But they have to design these things for the 90% case. People using things that are far outside what's being taught—in the age of AI—are maybe more likely to be cheating than everyone going above and beyond.

Students are human. Many have been taught to focus on one end goal: a grade. They see a mythical program that knows 6000x more than is taught in this 200 level course; of course they want to use it!

ryandrake 3 hours ago

When I took my Freshman year undergrad programming course decades ago, I had already been programming in that language for at least three years. It would have been quite painful to have to deliberately not do things I already know how to do because "I didn't learn it yet."

This was definitely a problem in classes before AI. I had a number of infuriating teachers who insisted on failing me for doing X, because "you're going too fast and we haven't gotten to X technique yet."

icedchai 3 hours ago

Same. I taught myself C when I was in high school, back in the early 90's. Fortunately, in college, I was able to test out of the intro classes for credit.

sothatsit 3 hours ago

I remember being marked down for using a ternary in Java for an introductory programming course at uni because the teaching assistant didn’t know what it was. Pretty minor loss of marks but it always annoyed me.

That said, having to do the introduction to programming class at uni despite programming for like a decade in high school was the bigger annoyance.

OkayPhysicist 2 hours ago

Decent professors distinguish between "using stuff we haven't taught yet" and "trivializing the assignment by using stuff that wasn't intended". If the assignment in an Algorithms course is to implement a square root function, calling "sqrt(x)" is obviously inappropriate. Structuring their solution using a do while() loop rather than while() loop is not.

OkayPhysicist 2 hours ago

This isn't the 90% case. At least half of CS students come in with at least a basic proficiency in programming. It's a big enough portion of the class that not boring them into ignoring the actual Computer Science parts while getting the other half up to speed is the defining issue of lower-division CS courses.

smelendez 3 hours ago

Yeah, and sizeof and malloc are very easily things students could have been exposed to in high school or their own studies. I’ve seen C books that use them in the first chapter.

It’s fine to say solve this problem using the following tools, but not great to say “if you find another approach, I assume cheating.”

broodbucket 3 hours ago

My professor in a C course said anyone that uses `goto` gets failed. It was quite liberating for me to then end up writing `goto`s for a living

OkayPhysicist 2 hours ago

Are you so graybeard that structured programming was interesting enough to be part of the focus of a C course? Because I can see in that context requiring the use of appropriate control flow structures. Nowadays, I can't imagine in what context students would be so eager to reach for GOTO that they don't preferentially use other control structures.

broodbucket 2 hours ago

This was in...2013? I think the main point was just to stop students from footgunning themselves

vintagedave 4 hours ago

> Do not look at other students’ homework files and do not work with other students. At all.

This is a horrific policy. We work better with other people.

When I studied computer science, other students came to me for help. I helped them. We discussed problems together. I saw their code. They saw mine. No one copied: we shared approaches and style and thinking. I shared how I solved things. They learned my thinking. By explaining, I understood what I knew better. Sometimes I learned from them: an idea I wouldn’t have had alone, a new direction. All of us were better off. This was across multiple people.

It was in no way discouraged and so long as you wrote your work — while asking others for help was fine — you were okay.

Humans learn through teaching. We learn through discussion. We learn through interaction. Solitude — ‘do not work with other students. At all’ — is the exact opposite of a healthy learning environment.

I don’t care what their artificial intelligence policy is. I cannot state strongly enough how much I revolt against their human intelligence policy.

As for the lecturer’s warnings in slides: that reads like a trap.

ianjbutler 3 hours ago

naive to think people would follow this policy in the spirit in which it’s intended, but even then it clearly selects for the most social animals amongst your students instead of the most talented. And discussion alone is all it takes to describe the right approach, so if that’s the test, it’s meaningless to protect the mechanical syntax of implementation.

Clearly in this world the frats and the sorority members get the highest scores, and after that the best looking, the most amusing or charming etc.. so unless that’s the intent the whole approach is perhaps worth reconsidering.

If you want to teach to improve your understanding the normal thing is you become a TA so that access is open.. not open every assignment to ad hoc collaboration so you disadvantage half of the students on every assignment

UqWBcuFx6NV4r 3 hours ago

This really reads like some sort of incel manifesto. The odds aren’t stacked against you nearly as much as you make out lol.

ianjbutler 2 hours ago

It may be surprising to you but, caring about fairness and democratizing access to helpful resources doesn’t actually require any kind of disadvantage or a complex. What’s your deal though, since you want to attack that?

charamis 3 hours ago

The course Programming in C, and other courses like Compilers, have this policy across many CS universities for many years now. It s a strict policy but it has its reasons. Assignments on early semesters are simple and short, and if students cooperate on the implementation, the result would be almost identical for everyone, while very few would have truly thought it out by themselves. Most students would not go easily through the process to struggle in order to think about the solution, if they could just copy the answer from someone else.

godelski 3 hours ago

  >> Do not look at other students’ homework files and do not work with other students. At all.
  > This is a horrific policy. We work better with other people.

I think that line was in connect with quizzes, which are presumably online. Near the top of the article it says

  > You may discuss assignments in a general way with other students, but you may not consult anyone else's work.

I'd be with you if they made everyone work in "clean rooms" but that is appropriate for quizzes and tests.

As someone who's graded lots of undergrads I can tell you that cheating is prolific. It's easy to identify who is copying code, but it is hard to prove. My wife teaches in another department and says the same thing. AI use has grown exponentially and it's really destructive to the education process. Though I have more complex thoughts on this and that it is just exacerbating larger problems that no one wants to actually solve because they'd be very hard to solve in a way you can have clear metrics for the bureaucrats (including industry). Proper evaluation is unfortunately always going to be fuzzy. Metrics are in contention with making a meritocracy, especially if you don't have a deep understanding of said metrics

dolebirchwood 3 hours ago

Some of us are perfectly happy to learn in silence and solitude, and we thoroughly excel at doing so. Sorry that you don't. The broad sweeping "we humans" language gets a groan out of me. It's the sort of mentality that leads to unproductive gatherings in the working world where people want to force "learning through collisions/synergy".

UqWBcuFx6NV4r 3 hours ago

I sat on a university student disciplinary panel. I assure you that your experiences are biased. There are plenty of people that flat-out copy each other.

OkayPhysicist 2 hours ago

GP was describing the platonic ideal of why we have universities instead of just textbooks, so I'd venture a guess that you're the one with a biased perspective here. Sure, some students cheat, and the appropriate penalty is throwing them out of the program. But pursuing that end to the point that you forbid students from engaging in normal collaborative learning is nuts.

As you point out, there are plenty of students who flat-out copy each other. Filtering out those students does plenty to enrich the remaining students, you don't need to go on witch hunts to find ever-more subtle cheating.

alcidesfonseca 3 hours ago

I agree. Here at U. Lisbon, we are following the recommendation by the recent MIT report: Evaluate students by talking with them, making it a social interaction, which they are lacking.

We need students to work together and share their experiences. Then we need each one to achieve the learning goals (and we should not assess work they did far from our eyes. We should assess their understanding and skills in front of us).

calvinmorrison 3 hours ago

learning to think is orthogonal to american universities, who's goal is to get government money via fed backed loans to fund professors real hobby: doin cool science

standardized and written tests, online school, homework, etc is just a way for professors (and increasingly, just TA's) who have no interest in teaching, to get the job done with less time and more efficiently.

Oral exams solve this problem entirely

OkayPhysicist 2 hours ago

I had a CS professor who worked like this. We had assignments and tests, and they did have grades, but your overall grade in the course ultimately boiled down to convincing him that you understood the course material. As someone who excelled at the material, but could be a bit hit-or-miss on bothering to complete assignments, I loved it. Students who were used to optimizing their efforts and obsessing over each point hated it. It didn't come up in that course, but the issue with such a grading scheme is that sometimes students and professors really don't see eye-to-eye. Most of the time it's the professor's fault, holding some sort of prejudice. A grading scheme like I described would be disastrous for such students.

golly_ned 2 hours ago

Given the professor's professional interest in cheating-catching tools, I'm suspecting wanting to get good, interesting data motivated a lot of their behavior here, even in ways he might not consciously realize.

summarity 4 hours ago

If you remove the mention of AI from this post, the authors process and setup of the notice to students is absolutely mental. The intentions might be good, but damn if this isn’t a comically inept way to set up a kafkaesque nightmare rather than a place for actual learning. The defensive attitude doesn’t help either. Just the sheer energy that went into all the steps mentioned here, not invested in any even superficial attempt to perhaps adapt. That isn’t to “give into AI” or cheating, but if 50% of students break your rules, more rules and litigation about the rules are not the next step to take.

altruios 4 hours ago

Using LLMs for the classroom assignment is like using a forklift to lift weights in a gym: completely inappropriate and atrophic.

That some 50% are using the LLMs rather than spend effort learning is not a cause to remove the ban of LLMs in the classroom.

90% of the power of an LLM is being yourself competent enough to judge its outputs. Besides, do we really need more humans coding like LLMs?

summarity 4 hours ago

My comment explicitly did not focus in the use of LLMs and in fact much of the post doesn’t. Leading with the depressing “work alone” slide, it focused on similarity, not AI use. Between students, ruling even “inadvertent” exposure as the same punishable (and literally literally undeniable!) offence. There’s millions of ways to structure a class for learning, this myopic focus is not one of them. Ironically the setup ignores the very basics of human collaboration. AI merely amplifies extremes, it’s a factor but neither the cause nor culprit here.

The title includes “retrospective” but the learning and changes following such a process are absent.

altruios 3 hours ago

> Leading with the depressing “work alone” slide

working in a team is valuable. but that value is proportional to the teammates chosen. teammates who have worked alone and can think through their decisions are more valuable, thus it is this way.

>AI merely amplifies extremes, it’s a factor but neither the cause nor culprit here.

you may have missed this part: ~As discussed in the paper, we have also run the tool against earlier semesters with very compelling and interesting results. In particular, the indicators that we use are virtually nonexistent prior to 2024 and their presence increases rapidly over the following years.~

So it stands to reason that LLM's are in fact, the culprit here.

MattGaiser 3 hours ago

All these analogies are kind of tortured as they ignore the elephant in the room. The students want to be there to get their job getting paper. They aren’t there to learn.

If society offered some reward for lifting the weight while not enforcing how, forklifts would absolutely be used to lift weights.

altruios 3 hours ago

... Thus the enforcement?

AndrewDucker 3 hours ago

The job-getting paper ceases to be meaningful if it no longer verifies competence. If it merely verifies the ability to prompt an LLM then nobody will pay a premium for people have it, and then nobody will pay to obtain one.

babelfish 4 hours ago

This is the professor who had beef with a /middle schooler/ for copying some content on how to use a calculator from his website. His defensive attitude is clearly indicative of the massive, massive chip on his shoulder. Just reading this triggered me to my experience with similar professors in college about a decade ago. What a jackass

thewakalix 3 hours ago

> Additionally, the kid that I "stuck it to" wasn't even in high school - I'd guess middle school or possibly even grade school.

"Okay, but that's worse. You do get how that's worse, right?"

OkayPhysicist 2 hours ago

To be clear to anyone reading this, that is a verbatim quote from the man's own FAQ. Wild.

Gracana 4 hours ago

The insistence that you must "work alone" and that similarity between submissions will get you flagged would make me really uncomfortable. I think I would be afraid to talk to the TAs lest I get poisoned by their knowledge.

ryandrake 3 hours ago

Yea, this letter could have been written about garden variety "suspected cheating" 20 years ago. And the professor's attitude is the same as a couple of professors I remember back then:

Prof: "I know you cheated."

Student: "How do you know it, can you prove it?"

Prof: "I have ways, trust me I know."

Student: "I know the material, I'll re-take the test right in front of you if you want."

Prof: "Either admit that you cheated and get a F, or don't admit and I'll send you to the school's academic integrity department."

RomanPushkin 3 hours ago

Professors will be the last to admit their skill set is outdated. Those folks and academia in general aren't outcome-driven. They don't care if students will be ever able to find a job or not. They just want to educate like they got used to, and want their money. Or is it me being jealous? I had to change. Change a lot, learn a lot in last couple of years. But what about those folks who still use Borland compiler? I can imagine they're comfortable in academia, and can be in the next 10 or even 20 years - depends on how aggressive they're defending their old-world principles.

personjerry 4 hours ago

Like other comments, I can't help feel uncomfortable with the way everything was written defensively, where every example is used to justify the author's actions.

The author concludes near the end:

> That said, I have no judgment to pass on you here.

And follows with this as the very next sentence:

> We are all human beings, we all make poor choices and mistakes in our lives.

Which seems to be clearly judging the students for their "poor choices and mistakes".

I feel the author is out of touch with the students and the times; If so many of them are leveraging AI perhaps the better move is to find ways to align that usage with learning. He quotes:

> "it is better 100 guilty Persons should escape than that one innocent Person should suffer"

But if there is such a scenario in a classroom rather than a courtroom, then perhaps the methodologies should be revisited?

It feels like the author is the ACKCHYUALLY meme, striving to prove themselves technically right, rather than empathizing with the students, which I feel shows a lack of maturity for an educator more than 2 decades older than his students.

I understand that education and evaluation is difficult in this AI era, but I can't come to agree with the author's actions or conclusions in this case.

ndriscoll 3 hours ago

Assuming that the students were using generative AI when told not to...

The way you leverage AI for learning is to do the work yourself and ask it to review and provide feedback, or to ask it to explain some concept, ask it questions to check your understanding, ask it to generate questions for you to answer, etc. If the entire point of the exercise is the exercise, then generating the answer... misses the point.

Like I can easily find online how to encode if statements in the lambda calculus, or how to make a half-adder out of logic gates, or how to prove that bijections are invertible. It's always been cheating to not do it yourself. School intentionally gives you easy problems with known solutions, not open research problems where you should use any and all tools and information you can find.

Jabrov 3 hours ago

> If so many of them are leveraging AI perhaps the better move is to find ways to align that usage with learning

Should the course be updated to better leverage AI? Sure.

But as a student, if you're explicitly instructed not to use AI for a particular course/assignment and you do it anyway, that's academic dishonesty. No matter how much you disagree with the principle of the thing, you still lied and cheated.

2001zhaozhao 2 hours ago

I'm pretty convinced now that grading take-home assignments is pointless with the prevalence of AI cheating. (There's surely already "innovation" going on targeted at letting people cheat on assignments while staying undetectable.)

It's probably best going forward to grade only in-person proctored exams using paper, or offline air gapped computers in the case of programming courses.

crispr245 22 minutes ago

It's beyond me how binary their thinking is when dealing with an issue as nuanced as AI. For instance euros have oral exams often and they are much more helpful on seeing if the kid has any idea on what they are doing. Furthermore it forces them to think through the problem more clearly since they have to explain it to someone else at some point (rather than some autograder).

woopwoop 4 hours ago

> Second, while the percentage of students identified was substantial, it was still less than 50%.

No doubt the student body will learn this lesson, and in the future the number of students identified as having cheated will be near zero.

EduardLev an hour ago

I recently completed my masters in CS. I agree with the sentiment of wanting to avoid LLM use in general, but that ship has sailed. No amount of policies and slides is going to be enough for this systemic issue. I remember all the policies and volumes of reading I would have to do to keep up with what was expected to be done in a very rigorous and usually pedantic way. Change to oral exams or something similar

djoldman 4 hours ago

> This form must be completed .... Failure to complete the form will result in an unfavorable letter being sent to the Dean of Students along with further potential disciplinary action.

> Through careful analysis and manual review, we have identified what we consider to be clear and concrete indicators in one or more of your homework assignment solutions that it was partially or entirely generated by an AI/LLM tool like ChatGPT, Claude, Copilot, etc....

> You are required to submit a response to this form....Failure to respond will result in a grade of 'F' for the course.

> Honest responses that align with our analysis will result in a score of O for the identified assignment(s).

> You also have the option to meet and dispute our findings. Should you choose to do so and we determine that our original findings of AI/LLM usage stand, we will instead apply the standard course policy from the syllabus and assign a grade of F for the course.

The above letter sent to a bunch of students.

> It is an open question whether this is reflective of their actual behavior or instead something that underscores the purported, and unintentional, potentially coercive nature of the form.

Sounds like the professor discovered (or disputes the possibility of?) the concept of forced confessions.

"Honest responses that align with our analysis": did the professor consider the possibility of dishonest responses that align with the analysis?

"Should you choose to do so and we determine that our original findings of AI/LLM usage stand" Not much needs to be said about that.

If the professor was confident enough about the analysis to send the letter, then all of the students getting the letter should have gotten an F and no letter at all.

argee 4 hours ago

It's just a textbook, garden-variety power trip. It's neither the first nor last time this professor will go on one.

https://www.ratemyprofessors.com/professor/2231495

MeetingsBrowser 3 hours ago

My “power trip professor” alarm went off reading:

> If we find reason to believe that a student or team has cheated on any assignment, we may inform the student or team promptly, or we may decide to silently accumulate evidence against the student or team on later assignments.

Why role play what you may or may not do if you encounter cheating.

The university has rules about how academic integrity is handled independent of the syllabus.

Including this line in the syllabus, and again at the top of the blog post is weird

OkayPhysicist 3 hours ago

That's not roleplaying, that's the basics of how you handle stuff like this. For lower-division level technical assignments, there's only so much free expression to extract signal from with regards to copying per assignment. Any individual assignment could likely be appealed as coincidence. If the same students turn in 2 more assignments that are suspiciously similar to the same other students, then you can have a pretty iron-clad argument for cheating.

Students cheating at lower-division university level courses is almost entirely a symptom of the students being poorly suited for the course material. Students who are interested and capable, but overwhelmed with the amount of work tend to just turn in incomplete assignments, or not turn in work at all. Encouraging the cheaters to drop while they can still get "dropped" rather than an "incomplete" or "F" is in everyone's best interest, as it provides an opportunity to consider a pivot to something that interests them more.

IMO, the overreaction by the professor was seeking further penalty for students who already dropped the course, or who would do so when confronted with their misbehavior. A student who cheats in CS is not necessarily a student who will cheat at chemistry, or business, or journalism. If they've already dropped out of CS, there's little benefit to be gained by pursuing the matter further.

MeetingsBrowser 2 hours ago

> that's the basics of how you handle stuff like this

Anecdotal but not in my experience. The professor can inform students, collect evidence, etc without detailing each scenario in the syllabus.

Every CS class I’ve been a TA for had the same generic university provided statement on cheating.

The professors did not want to spend time thinking about or dealing with people cheating.

Every time MOSS flagged assignments they basically sat with me and the student and said “You are likely cheating. Please don’t do it again.”. No one ever admitted to cheating and no one ever got flagged twice.

jubilanti 2 hours ago

edit: HN seems to loves people who steal, lie, and cheat. my bad

MeetingsBrowser 2 hours ago

That’s an entirely different situation.

It would be more like a low level manager telling new employees what he would do if he ever catches them stealing office supplies.

The university has a policy on how to handle cheating. The professor doesn’t need to imagine potential scenarios in the syllabus

dbalatero 3 hours ago

I'm surprised by the negative response towards this professor. The syllabus was extremely clear up front, and students did not respect it and decided to see what they could get away with instead. Yeah there's some weird rules to follow, but the homework seems fairly scoped to smaller problems and it's just one class out of a few dozen.

OptionOfT 2 hours ago

> Every student is also responsible for protecting his/her own work. If you inadvertently allow your work to be accessed, you may still be held responsible for facilitating an academic integrity violation.

This is insane. Absolute power tripping. And I say that being in favor of banning AI throughout the course.

fwipsy an hour ago

I suppose it's to close a loophole where students can collaborate/cheat and then, if caught, claim that someone else copied off of them.

ThrowawayR2 3 hours ago

Anyone who is lame enough to cheat on an intro to C class is doomed anyway; the AI they cheat with will be the AI that replaces them as a worker because they didn't learn anything. Give 'em hell, Prof. Turkstra!

dhon_ an hour ago

Any policy that asks students not to use AI and trusts that they will comply is giving a massive advantage to dishonest students and punishing honest ones.

This article sounds like the professor was reprimanded by higher powers and tried to dig themselves out of a hole for having a badly structured grading system.

Students should be encouraged to use AI in their learning, because they will anyway. The understanding should be that they will be assessed in supervised environments so they need to learn the work either way. This gives the students incentive to learn without fear of using the best tools available to do so, and provides evidence to the powers that they actually did so.

0xbadcafebee an hour ago

Can't they just do in-person testing? Build a cluster of cheap, crappy dumb terminal from 40 years ago. Have students type out the source code live. Grade the work. Hell, let them use reference books! Just don't allow re-typing AI-generated code on notebook paper. They can cheat all they want in the interim, it'll all become clear on the midterm and final exam.

gred 3 hours ago

I'm a fan of zero tolerance policies for academic dishonesty. The dean getting involved and mitigating consequences for the ~~customers~~ students is classic.

fapjacks 3 hours ago

Haha calling the fire marshal ahead of time "to ensure we remained within the capacity constraints for the facility" is an amazing touch!

doctorpangloss 3 hours ago

it's hard to rein in cheating with LLMs in the classroom. this guy is trying at great cost to his personal life - i mean, if not in academia, where else are mean people supposed to achieve great things?

imo watermarking will do little to reduce this awkward, personal, adversarial thing. as always, technology cannot 100% solve, or even 50% solve, a political problem.

> the decision that was reached was to reverse the retroactive use of the tool on earlier homework submissions... Students that had dropped the course would be permitted to re-enroll.

it's too bad the professor completely capitulated. like people have so much angst against H1Bs, so much excitement about unionizing, agitation about layoffs or whatever... look, 40% of your classmates are cheating. 40% of your colleagues are cheats. the call is coming from inside the house!