AI did all my homework, so I redesigned my entire course

Think of education as a wishing well. There is a wishing well for math, law, dentistry, baking, and so on.

Once you have a wishing well - i.e. you own an eduction in baking or dentistry or law or math or whatever - you can ask it anything and it will grant you your wish. If others want something from your well, they have to come and ask you. What you give them has your name attached to it. So, your reputation rides on the quality of what your wishing well produces.

Prior to AI, universities chose which wishing well they would specialize in building and offered to share that knowledge with students.

In the age of AI, the creation of personal wishing wells has become trivial but universities still think they are in the business of teaching people how to create wishing wells.

Ultimately, the only truth left is for owners of the wishing well to start offering the product of their wishing well to the market and attach their name to it. If there is a market for their product, great. If not, go make another wishing well.

Education used to be pets. Now, it is livestock. Universities are not willing to accept this shift in mindset.

I am teaching some math courses, and I see how LLMs disrupted all standard approaches, and I don't know what to do.

I rely on written exams that are open book, but forbid any use of computers and smartphones in class.

The university insists that homework can't be optional, but it lost its meaning. I've tried to explain to my students that it's in their interest to think about home assignments on their own, but the majority will obviously use gen AI, and it's such a waste of time to check and grade LLM output.

I just had an experience where all students were given "sample problems" to try at home and prepare for the written test. Many of them just dumped the document into an LLM, asked it to produce some "exam guide", and showed up with that thing printed out, asking me during the test to explain what LLM output meant.

It seems like the university also has many people pushing for AI use for everything, but I teach basic stuff where the goal is to make students think on their own and digest some fundamental ideas, LLMs can produce perfect solutions, but relying on them is pointless.

Maybe the Germans were right? One in-person test, preferably an interview, at the end of the semester should be enough to motivate the smarter part of the bunch to make sure they understand the issues at hand. The rest is chaff anyway. Don't optimize for chaff.

I have a fairly long experience of both teaching and grading, even though I stopped a couple of years ago. After a couple of years of trial-and-error, I think I found the "right" way:

* Alternate between theory and practice in the course of a single 2-hours slot. Theory is necessary, but the attention span of people is very limited.

* Grade either a homework assignment in the form of a real project with specifications. You give the assignment half-semester, so that students who start early can ask questions. Otherwise, grade a in-session exam, with every resource available, including the course and the whole Internet. However, you don't assess the knowledge, but if the student is able to apply their knowledge to the different tasks at hand.

These approaches are now completely moot in the age of AI.

I was surprised the first time a colleague asked me to do an oral exam. I thought it was a burden on people with poor social interaction skills. It went surprisingly well: you could see in a couple of minutes if the student had integrated the concepts of the course, even if they were shy. Now I wonder if it's the way to go. There's one caveat, though: it doesn't scale.

Seems the core problem is that many/most students aren't there to learn the material, just to get grades. The class is a complex mechanism to force them to learn material against their will in a validation arms race. The teacher's job becomes something like a military commander. Seems depressing.

> I now shifted to in-person interactions with a TA. After every assignment, each student needs to schedule a 15-minute meeting with a TA to answer a couple of questions in a live conversation

Isn't the answer the 'flipped classroom' - students are assigned reading/studying to do before a class and the class becomes more interactive, answering questions/discussing topics/solving problems etc depending on subject. Of course, this is more expensive than a hall with 400 people listening to a lecture with a couple of tests/essays.

https://fltmag.com/the-flipped-classroom/

https://en.wikipedia.org/wiki/Flipped_classroom

I was at EduLearn (a major edtech conference) this year and talked to many professors exactly about this. A lot of them are pulling work back into the classroom, but keep running into the same problems:

- some skills only develop through the actual writing (similar to how you can't learn to code just by watching YouTube videos)

- in-class assignments are not always scalable (and students end up using AI anyway)

- students aren't motivated enough to participate actively

I think looking at the writing process is a more realistic approach, so I built Turingo (https://www.turingo.net). It replays how a Google Doc was written, highlights unusual edits (large pastes) and tracks how that text changed afterwards. Surprisingly, several professors reported that students often underestimate how much LLM-generated text is left in their final drafts, so just seeing the replay changes the conversation.

It's not meant to be a verdict machine like the existing AI detectors (another big problem in education), more of a starting point for the kind of conversations the author's TAs are having. In fact, we're also beta testing a feature that suggests a few questions for the professor to ask based on the replay.

And yes, it's not perfect and autotypers do exist, but they're dumber than you'd expect and nowhere close to mimicking a real human (I hope to address that soon).