In this blog, we are going to continue our discussion on ChatGPT as a learning tool, how often it is used nefariously (e.g. student cheating), as well as it’s usefulness for teachers. The tutors in this discussion, all of whom tutor with Warp Drive Tutors, are all full time mathematics and physics teachers at very good, academically challenging schools – Cameron is in the US and teaches mathematics at one of the top 5 STEM schools in the country. He earned a degree in Materials Science Engineering from Carnegie Mellon University and then a PhD in Mathematics from UCLA, focusing on the geometry of curvature. Amadeo is a physics and mathematics teacher at a terrific STEM school in Italy; he majored in Physics at the University of Pisa, Italy and completed his PhD in Theoretical Particle Physics at the International School of Advanced Studies in Trieste, Italy. He has 8 publications with more than 200 total citations in peer-reviewed journals and more than 25 official communications in scientific meetings and invited seminars. Geoff is a physics, mathematics, astronomy and computer science teacher at a renowned boys’ school in the UK. He received his PhD at the University of Cambridge, where he studied relic radiation from the big bang while working in the same department as the late Stephen Hawking. And then Josh runs the company.

Josh: OK so today we want to discuss AI and chat GPT; I think version 03 is now out, and so I wanted to talk about how that affects mathematics learning as well as teaching, from the student’s and teacher’s perspective. So to start, Cameron teaches AP and then Geoff and Amadeo I know teach both IB and A-levels.

Geoff: So I know that for A-levels, for example, that it’s really a non-question about whether ChatGPT, whatever version, can help or not help because it all boils down to the grade that you get on the exam at the end of the year.

So that, you know, whatever your homework problems were or whatever your test scores in the middle of these, that’s all apparently meaningless and moot because you can get a very, whatever grade you get on your three-hour exam at the end of the year is the grade you get in the course. So I don’t know really whether, in terms of A-levels, if it is even applicable to talk about chat GPT and how it helps you, or helps you cheat…whereas for the IB, I’m not so sure if it can or cannot. I would suspect though with APs it is definitely an important topic of discussion because the AP exam grade course is not your grade.

Cameron: The AP exam is a large portion of your grade. We teach AP classes here, which just means that we follow the curriculum set by the college board and the students take an AP test at the end of the year, which is written and given by the AP. That test score has no impact on their grade for the course, in part because the scores don’t come out until the school year has already ended.

Josh: Right. So in this, so in the situation with APs, certainly then GPT can be used for, shall we say, less than the most academically beneficial reasons. Chat GPT can be used to cheat. Because you can have chat GPT write your lab report; you can have chat GPT do your homework.

On the other hand, chat GPT can be very helpful in certain situations. So let’s start with Cameron and the APs. What is your opinion about how chat GPT can either help, or hurt, a student’s performance?

Cameron: Okay. Start with the biggest question of all. And I think that it’s like a lot of other tools out there. If misused, it would hurt the student’s performance. But if they’d learn, you know, how to use it and when it’s appropriate to use, it could be another tool to help improve performance. Like a lazy student who uses chat GPT to do their homework will not get the practice that they need and won’t do as well on tests on the way.

I want to compare something like the chatbots to already available tools like Khan Academy or the College Board which does have courses, practice problem sets, and other resources to go with their AP classes. I think that ChatGPT can serve to replace some of those, but not necessarily as an improvement on what I’ve seen.

Josh: In terms of the latest version 3.0, which has just been rolled out, do you think that changes that situation at all? Does that make it an improvement in any way in terms of helping students? Because many of the previous versions were often very erroneous.

Geoff: Well, I’ve done a couple of experiments with it, and I don’t know.

I don’t know if I’m using the best versions out there or when updates were made. But if you ask chat GPT to do something like write some practice problems on related rates, some calculus practice problems, it’ll do a good job of creating problems that are fairly routine, not really having any elements that you wouldn’t find in a textbook problem. But it’ll write the problems, and then if you ask it to, it’ll give you a walkthrough solution, which is, as far as I can tell, usually very accurate, detailed, and clear. And so these are what I call the good uses of the chatbots. A student who knows what they’re having trouble with and wants to see some more examples can easily find it. On the other hand, it’s not that much different from what already exists in something like Khan Academy or the official sources. On that point, Josh, you mentioned about the newest version. So I pay for the premium subscription for GPT, I’ve got it open now, it’s 4.4.0, I think, and it is the version you get with the premium one.

And the newest version of that was just released about three weeks ago (early December 2024). And one of the big stories that I saw when the new release came out was how much better it is for, particularly, maths problems. Before that, it was pretty good, but like you said, it would often make some silly mistakes.

But some US colleges put some of their undergraduate and graduate level maths questions into the newest version of 4.0, and it did amazingly better than the previous version. Now, I only use it for physics work and sometimes astrophysics as well; I am not a huge amount of using it for mathematics. But the newest version for all the physics problems I give it is remarkably powerful.

One particular thing it can now do, which it couldn’t do before, is you can take a screenshot or copy and paste a page from an exam, which has a diagram in it. And for physics, many of the questions in physics have those, like here’s a picture of a seesaw with a mass on the end and it’s labelled with the distances. And ChatGPT, the newest version, can work out what’s in the image. It can extract the distances, the measurements, the components that have been used. It knows that it’s a seesaw and it explains what’s in the image remarkably well, which the previous version couldn’t do. I think the newest version is incredibly good for what I’ve tested it for in physics.

I’ve still found a few things that it can’t do. So I was meeting with the head of maths just before the Christmas break about this exact conversation. And he said he was still skeptical because often he would give it harder Olympiad level problems and it wouldn’t get them right a lot of the time.

I’ve not tried the newest version with the Olympiad problems, but I bet it would do a lot better than the last version. But again, that would be something that would be quite interesting to investigate.

Amadeo: At times though I actually call it artificial stupidity with my students rather than artificial intelligence, because it’s not reasoning. It’s just collecting bits of information and putting them together, Although yes, in a smart way –

but that’s precisely I think why it doesn’t work with Olympiad level questions, because it cannot find the solutions anywhere. So it cannot work out the solutions by itself; it has to find something similar and then it joins the pieces.

And that’s why sometimes it makes mistakes. And yeah, it’s amazing because it can read the text and try to process and solve the exercise. But it does well sometimes and sometimes it doesn’t.

But one thing it can be very useful for, if you’re a non-native speaker, is for example in the IB when you have to write the IAs or the extended essay. If you’re a non-native speaker, it’s useful to let a student rephrase some sentences here and there. So sometimes when I have to write an email, I write it first in my “Italian English”, and then I ask it to just rewrite it in better English. And of course it does a good job at that.

And I know that the IB has a policy that not more than, if they find out more than 60% of these IAs or extended essay are written by an IA, it’s considered cheating. I don’t know how they measure that. So how can they figure out the 60%? I have no idea. And 60% anyway, it’s a large percentage, right? So basically they’re allowing it to rewrite parts of your work.

Geoff: We, at our school, we have a relatively new AI policy, which is factored into our teaching and learning contracts that we have all the students sign.

It’s got some name like guidelines and standing orders that the boys will have to sign at the start of the year, confirming they won’t cheat, confirming they won’t blah, blah, blah. And so we have a section in this, which we’ve now added to with the new plagiarism detection software that we use.

So there are loads of tools out there that claim to detect the use of GPT with varying efficacy. And we have tried a few. The one that we’ve just bought a license for is called Turnitin. Now Turnitin is a US company. Many schools use them and some schools in Europe use them for IB checking as well. And Turnitin, it’s not a foolproof plan; we use it more as a threat to the students. They know that we can use it. And having that threat, that sort of Democles hanging over them, I think is a useful deterrence.

It won’t, of course, be perfect. But we have a, when you run an essay through this and what we can do is, you can batch upload a few hundred documents at the same time or a few hundred PDFs or Word documents. And it does them all, runs them through its engine.

And then a few minutes later, you get a list of a percentage. You can look at the percentage and the breakdown of where that’s come from. It also finds plagiarism as well as AI, so it can see if they’ve copied and pasted a paragraph from Wikipedia or from a book or something. And it gives you a breakdown of where they’ve nicked all the work from. And it gives you a percentage AI score, which again should be taken with a pinch of salt.

But Turnitin say that if a report comes back more than about 25% AI, then it was probably written with AI. Now they’re saying that their system is not perfect, but if you get a high percentage, that might be where the 60% has come from, Amadeo, because maybe if it’s detected as being 60%, then that is almost certainly produced by an AI engine, maybe less than that. If it was only say 25%, they can’t be certain and they can’t make the accusation against the student because they’re not certain that it is indeed plagiarized. I would guess maybe that’s it.

Cameron: And there’s a fine line between you don’t want the AI to create the response, for sure, but isn’t it okay to use it for things like grammar checking? If it’s taking my ideas and just rearranging them a bit to be more readable, I’m not sure where to draw the line for something like that.

Geoff: Yeah and would that be, if you gave it your ideas and then it reworded them, would that then get picked up by Turnitin as AI content, I wonder?

Josh: How often, so that’s getting to my next question, are you guys able to tell if a student turns in a lab report or other work written by ChatGPT? So I’m guessing then this software is used to potentially detect that, but lacking that software, if you were just, Colin, can you tell if math homework, for example, has been completed by ChatGPT?

Cameron: So math homework is harder to tell because there’s so many other sources out there. And usually the way that you start to be suspicious is if a student has perfectly correct answers on their homework.

That alone is not suspicious because most homework problems are formulaic enough that a good student should be able to do it. But then when their tests and in-class work just doesn’t add up, then you can get suspicious. But whether it came from a ChatBot, whether it came from PhotoMath, which is a sort of proto ChatBot, or whether it came from their parents or a tutor or a classmate, these things are hard to tell.

Geoff: Yeah. With physics reports and the IA reports that I’ve seen, I think once you’ve been with a class for a while, you learn the style of each of the students. So sometimes it’s painfully obvious when they’ve plagiarized the whole thing because, you know, you just read the first sentence.

You’re like, this kid has not written this report. It’s almost a comedic sentence that they’ve just written word for word, or they’ve gone about the derivation in some weird way that they would never have done themselves. When we use Turnitin, I mean, some teachers will just batch upload everything just to make it easier for them.

But what I use it for is I can tell once I am I’m suspicious. I know that it’s been made by AI. I don’t need Turnitin to tell me.

I know 100 percent. And if I challenge the student on it, they will lie to me. I’ll say, look, you used ChatGPT, and they’re like, yes, I have a little, and sorry, I won’t do it again. But I will also put it through Turnitin, which will then come back like 45 percent AI generated. So I’ve got that digital kind of receipt, if you like. So having that extra digital confirmation of guilt is useful.

But I think many teachers, at least where I teach, they know in science and math, they know pretty easily from the report that it’s been plagiarized without needing to put it through one of these tools. And I guess that’s something that you get once you’ve seen a class for a year or two.You get to know the kids. And that’s something that AI doesn’t do, I suppose.

Josh: How often do you think that happens, Chris? Like, what percentage of kids in a class of, you know, if you teach 100 kids in a year, what percentage of those kids would then be…

Geoff: It’s very few and far between, actually. So the number that I’ve caught, as in the last term, so from September to December, I’ve caught about five or six. But a lot of the homework that I set is not writing reports. You know, they’ll have to make a video or write a handwritten experiment report or produce some graphs.

So a lot of the stuff they physically can’t use, ChatGPT for anyway. And a lot of the homeworks that I set, I’ve kind of changed the way that I set homeworks out. I deliberately don’t set homeworks that can be done with ChatGPT, because I know that that’s just tempting. It’s just tempting fate. So I’ve changed the way that I do my assessment. All the assessment is a lesson, some paper-you know, it’s almost foolproof. It’s almost gone full circle, the revolution of education. We’ve gone back to the most efficient and most prudent and secure way of testing kids is give them a piece of paper and a pencil, put them in a hall for two hours. They can’t possibly cheat. And then you find out what they know. It’s almost, I find it almost ironic that it’s gone back to that level now, that we’ve been through this phase of using tech for everything, but for exams to be foolproof and to stop them from using AI, you know, put them in a big hall with an exam paper and give them a pen.

Josh: Schools in general are trending in that direction?

Cameron: Yeah. It’s easier to do that in some subjects than others. The humanities classes, English classes, history classes, where the major works of the class might be a long essay or a research paper. This is where I think they have a much harder job to adapt to these new technologies than math and science classes might. And here, we don’t know what to do. We’re still trying to figure it out.

Amadeo: Yeah, I agree. And also, I think this is going to change quite quickly because, for example, I also noticed when a kid uses the AI to produce a lab report, it can be easily noticed. But soon enough it will become harder to tell what fraction of the report has been written by the AI, because they will realise that they can ask the chatbot to rewrite that sentence or paragraph in such a way that it looks more similar to the language a teenager may use.

Refining the prompt (the input you give to an AI) is also going to be a skill to learn because the “prompt maker” is going to be a job soon: the better the prompt, the better the AI outcome. For example, we have all seen those amazing images produced by the AI: to obtain such a high quality image, you need to write a long and detailed request to the AI. Writing such requests will soon a be a job.

And to be good at that job, you have to be able to understand how the machine works.

Another example: if you ask an AI to write an email and that emails looks too polished, you can ask the chatbot to write it in as a teenager would write it, and it will do it. But our students haven’t understood that yet, I think.

As I said before, this will become harder because, of course, the ninth graders, the tenth graders have never written a lab report, and they write bad lab reports in general. The first ones they write are bad also in terms of the language. And so, if they use the chatbots, the language is going to be much better than what you know they are able to produce and you can easily spot if the have made use of the AI.

But if they understand that they can ask the chatbot to be less good, then it’s going to be harder for us. And I really wonder how Turnitin measures that percentage of AI production.

Cameron: Yeah, it’s like finding ways to figure out whether or not students are using it is certainly one challenge.

But the more important challenge is figuring out how to teach students to harness the power of these tools accurately. And in math, I compare it to the calculator revolution, where what was mathematics education like before 1950, right? And so, I teach almost 80% without using calculator at all, but I’m always referencing to what we could do if we had one, right? And what we can’t do without one. And how many tools your calculator has, you have to know what they are and how to use them, right?

Amadeo: This is a comment I’ve heard more than once comparing this to the calculator. Actually, I was thinking about this and I remember that when I was in the primary or secondary school, we had like five years primary school and three years we call middle school. And there, I was taught how to extract roots by hand with some sort of long procedure. Actually, I cannot do that now. And I find it’s kind of stupid. Why would I lose a lot of time with this complicated algorithm? Which, okay, if you think about it, then it has something interesting inside, of course. But just to extract the root, when you don’t understand why you’re doing exactly that, it’s just an algorithm that’s just meaningless, right? There’s a calculator you can use. Or I found an old book of mine when I was in high school, which had the logs table. And you probably remember those. And of course, my textbook had at the end, it has the logarithm tables. And of course, now they don’t have it anymore because it doesn’t make sense.

Cameron: Do they teach statistics still with the full-size sheet of Z-scores? We don’t need that.

Amadeo: Also, also, yeah. So maybe in 10 years, we would have a completely different conversation about this. And also because there will be jobs that, this is always the case when technological revolutions happen, right? There will be jobs in 10 years connected to this AI business that we can’t even imagine now. And of course, this will have some influence also in school. We cannot avoid that, I think.

Cameron: We’re thinking about jobs. To what extent are we worried about our own jobs?

Josh: Good question. That leads to my next question, which is how much more helpful is ChatGPT relative to, for example, subject tutors like math tutors?

Geoff: That’s an interesting question. And for the first time, in about November this year, I transitioned to using ChatGPT more often than Google. And that was a big change. So we all Google, all of us are Googling maybe 20 or 30 times a day, right? But I find ChatGPT is a much more coherent and direct source to what I want to find out. When you’re Googling something, it’ll take you a few minutes to find a page and look through it. And then 10 minutes later, you’ve got the information you wanted. And you can, obviously, the better you can use a search engine, the faster you can find that information. But I now use ChatGPT for almost every web search that I do. And it gives me a better quality answer and gives me the links to the things I want to find compared to Google. And that was a big change. And that’s why I think Microsoft was terrified about this. And Google was terrified because they realized that this was going to succeed. And that’s why they pumped all this money into Gemini, which is the Google version of ChatGPT and trying to advertise it to make people think that Google’s not dead in the water yet. So that links to your question about how much more useful is it than online chat. So for many things, I try and learn new topics. And I use ChatGPT for my personal tutor all the time. I use it as a 24-7 tutor. I’ll ask it questions. Sometimes it doesn’t quite get them. When I look into it, it gives me references. It doesn’t quite give me the whole story.

But I find it really helpful for learning. And that’s what most of our boys in our school have said.

So I did a big questionnaire for all 800 boys about ChatGPT a few weeks ago and asked them questions about how do they use it? Which subject do they use it in? Have they used it for cheating? How much of their coursework can it write? Which subjects can it be most useful for? How do you use it? And we found about 80 to 90 percent of our students were using it, which is expected.

A load of them said, about a third of them said they use it as a personal tutor. It’s like a 24-7 personal tutor that they can ask questions about any subject, any time of the day, and they get an immediate response. And they said it was really helpful for them because they didn’t need to wait until the next lesson to answer that question.

Amadeo: I’m worried about a couple of things with this use, because I’ve tried it myself last summer.

I was studying for a completely different thing. I was studying for a sommelier exam, and I was asking ChatGPT to build a resume of wine areas and certification and these things. And after a couple of days, I realized it was making huge mistakes. Really huge. And of course, I had no control on that because at that time it wasn’t providing  or I wasn’t paying attention enough to check which websites it was referring to. But anyway, even if so, I’m sure after a while, I went back and I checked the answers, and it was such huge mistakes that it cannot possibly have found it on any website because it was just completely wrong.

And so it was probably picking pieces here and there and putting them together according to the algorithm it has. And it just wasn’t working. And so what I did, I uploaded a PDF of the book I was studying, and THEN it was good.

And it was not making a mistake with the classifications of wines and so on, because it could read them out from the PDF I had uploaded. And that’s why I paid last summer for a couple of months while I was having for this exam to have the pro version so I could upload the PDF. But if you just ask questions and use it as a personal tutor, if I do a Google search and then go on a website, somehow I can understand whether that website is reliable or not, also by the way things are written sometimes on it.

But ChatGPT takes that and then rephrases the whole sentences in a better way, even if the source material is not so good. And so it looks reliable, but then you don’t really know unless you go back to check the website and then what is the advantage of it. So my worry is how much can I trust it as a personal tutor, as you say.

This is evolving very quickly. So six months ago, it wasn’t reliable for that specific topic I was studying. It was not reliable at all.

It was a small thing, but then who knows? What if I ask something else? I had a question. I was in the car with my daughter. She’s seven years old. She asked me, how many languages are there in the world? And yeah, let’s ask ChatGPT. He said 7,000. Well, it’s good.

Where did he take it from? Okay, we could go and check. Then we went on Wikipedia and it was okay. But somehow I still, I don’t know, maybe I’m not Gen Z or whateever enough, but I still trust Wikipedia more than ChatGPT.

Josh So that gets to the question then. Do you think that, I think it’s theoretically possible that ChatGPT might ultimately replace personal tutoring if, as Geoff says, a lot of kids are using it and relying on it heavily. On the other hand, as Amadeo is saying, it is often inaccurate.

I think it is still a realistic possibility that there might not be any tutoring firms or personal tutors, say 50 years from now or even 20 years from now. Although I have a tendency to believe that the human interaction portion of a student and a tutor or a student and a teacher is always going to prevail. And these different types of tools have been around in varying forms for some time now in Khan Academy, College Board resources.

12 years ago, students were plugging questions into Google and coming up with answers. And yet tutoring has certainly grown since then. So what do you guys think? I mean, do you guys think that it’s ever going to just completely replace personal tutoring or diminish it or have no effect on it? What’s your opinion on that?

Geoff I think it’s pretty safe.

I mean, I agree with you entirely that the personal interaction you get, you just can’t get that from ChatGPT. Even the new conversation feature on the app, on the phone app is better. It’s more like you’re having a conversation with a friend, but it’s still not quite the same.

And all of our students confirmed that in their questionnaire when we said, do you find it more useful than a teacher or more helpful than coming to class? And they all said, no, they all prefer to go into class. Also, meeting with the teachers, because you get that personal interaction that you just don’t get with ChatGPT. So I think tutoring will be alive and well for many years to come, personally.

Cameron: I agree. We’re a social creature, are we not? And there are some things, yeah, maybe these will be overcome within five, 10 years, but I don’t know how good ChatGTP is right now at recognizing student handwriting, especially when it’s sloppily written and poorly organized and riddled with errors. And I don’t know how good, I don’t know how long it will take for it to be able to understand the weaknesses and detect the root cause of misunderstandings.

Sure, it’s good at providing clear explanations of facts that you ask it, but you have to ask it the right questions. And so, yeah, we’re mature adult learners and we can ask questions. And if the answers don’t pass the smell test, then we can double check the sources and all of that. But yeah, I feel relatively safe for now, but things are changing so fast.

Josh: Interesting. And I also think that to come back to your question from a little while ago, will teachers’ jobs be safe? Well, I think it’s a safe thing to say that yes, teachers’ jobs will in fact always be safe.

Like when I was at Edufest at Rosey this past May in Switzerland, someone put up a picture from the earliest days of photography of a classroom from like 1860 or whatever. And the basic model of the classroom then is the same as the basic model of the classroom now. And it’s, you know, throughout all the various changes that have come to fruition, I mean, with calculators and with Microsoft Word and Google and everything else, the basic model of the classroom has not changed. So I don’t think teachers’ jobs are going to be in trouble anytime soon, is my opinion. I want to ask just one more question, and then we got to, maybe hopefully, then I think we should wrap this up. And that question is, how much, in your opinion, you guys’ opinion, how much more helpful is ChatGPT relative to what has already existed? For example, relative to Khan Academy, relative to College Board Resources, relative to, you know, Googling your answers.

What do you guys think in terms of how much of its usefulness relative to those tools that already have been around for some time?

Cameron: I think I kind of answered this earlier, that I don’t think it, I don’t think I’ve seen any evidence that it can provide any unique resources that are unavailable from any of those other sources so far. However, it can be more efficient. And if you develop some skill with prompt engineering, it can probably be more flexible and fine-tunable in what it can provide.

Josh: Prompt engineering?

Cameron: In how you ask the questions.

Josh: Yeah, right. Yes.

Amadeo: It’s easier, probably, you can ask it to write an exercise, as we were saying at the beginning. And in that, it’s much faster and easier than looking through Khan Academy or other resources, because it’s immediate.

Just ask the question, it produces an answer. And whether or not the answer itself is better than the one you may find, spending some more time on other websites or textbooks, that’s another story. But certainly, it’s faster.

Geoff: I think certainly more efficient. And the buzzword in the UK is called “prompt craft”, which I quite like as a word. It’s the word that’s been floated around the edX channels and the tech fairs in the European education market.

It’s more efficient. I still try and convince students to think of it like a search engine, because you’re searching stuff and it’s providing information from websites that it’s got in its library. Now, it can distill those into a slightly more condensed form.

And that’s what it’s useful for. It’s more efficient for students to find answers. But there is still a lot of value in the resources that have always been used.

For now, it can’t make video resources or animations, which are complementary to the information that it gives. And particularly learning maths and particularly physics, which I teach a lot of, and astronomy and computer science, a lot of visual cues are really key in the learning process. So text-based learning, whilst it can be good and whilst it can be sufficient, it’s still not enough on its own to provide the same quality that we’ve been providing for the last 15 or 20 years, I think.

Amadeo: But for computer science, it’s probably a whole different story, right? Because I’ve heard a lot of friends of mine who work in programming and coding, and they use it heavily. And it saves a lot of time. So in providing some basic code from just describing in words what you need, it’s very efficient.

And whether this code is good or efficient or not, the code itself, it’s another story. But for example, I’ve used it myself also to do some things on spreadsheets, some complex things. I didn’t figure out exactly how to do it in a good way, and it was extremely helpful.

Cameron: I also know programmers who’ve used it. Because I was impressed with your sommelier studying example, I’ll tell you one in my personal life that I’ve used it. And I was talking with my wife and I was trying to explain why I liked Dr. Zhivago more than Anna Karenina. And I couldn’t really come up with a good way to explain why. So I asked ChatGTP to compare these two novels. And then I read their answer, and then I understood why. Because they’re…

Josh Don’t tell me the ending of Anna Karenina because I’m reading it now.

Cameron: Well, it’s good. But if you want my opinion just a half notch better, get the Pasternak next.

Josh: Okay!

Cameron: But hey, read the ChatGPT summary to see if you agree or not.

Josh: Will do.

Amadeo: With those things, summaries and comparing literature work, I think it’s really good because it’s a large language model. So this is what it’s built for.

And then also, as I said, when you give it the document and to write the resume or to write down a table with the data, it can extract. This is fantastic. Basically, when you can control the source, I think that’s the point.

When you have control over the source or you are providing the source or a novel, it’s a standard source, there’s no discussion about it, then it does an amazing job for sure. I’ve also read a comment about it that at some point, there are some news websites or also blogs, whatever, that write a certain percentage of their production directly with chatbots. And there was a discussion that this is going to constantly lower the quality of the information you can find on the internet. Because the chatbot, whatever bot they use, they extract the information from the original source, which is a good quality, and then there is a quality degradation. Because then the next ones will be trained on material that will be bot created.

Cameron: I am pretty suspicious of what I see online, because much is apparently bot created. What percentage of bot output is based on bot created training?

Amadeo: Yeah, exactly. And so the new AIs will train on AI produced material. And then does this produce a degradation of information or maybe it’s getting better? I don’t know. I think this is an open question.

Josh: We’ll find out. Okay, let’s probably wrap it up. Anybody have any other closing comments, again, on how much ChatGPT is going to help students versus hurt them?

Amadeo: Yeah, there was one thing I wanted to say related to what we said about it replacing, ChatGPT replacing, or whatever chatbot replacing teachers. And I don’t know if you share the same opinion, but I’ve seen also degradation in the mental health of our students in the past 10 years.

And I’ve conducted some research about it. I know I’ve read this book from Jonathan Haidt, which probably you know, especially in the US, I think he’s quite often on TV and so on. And that’s a real thing.

And this technology mediated interaction is not doing well to our kids. And so the human factor has to be restored as more central in their life, and especially in education, which is a large part of their life. And so, yeah, I always think about some old friend of mine who was also a teacher told me, you know, if the kids don’t love you, they won’t study what you ask them to study.

Of course, there will be super motivated kids that do it anyway, and they will loop from Khan Academy, ChatGPT, whatever. But you know, the average kid who is struggling with maths or physics or whatever other subject, if you don’t find it, manage to have a good relationship with them, it’s not going to work.

Josh: So ChatGPT is not going to be where they’re going to find that relationship, is it?

Amadeo: Not yet. Not yet. Later we don’t know.

Geoff: Yeah, I think teaching and tutoring is safe for now.

I personally love ChatGPT and other tools that are better for certain things. ChatGPT is only one of many that do particularly maths. There are some dedicated math bots, which are like they’re based on the GPT engine, but they’re tailored specifically for maths questions.

Equally, there’s some physics bots, which are tailored specifically to physics questions. So ChatGPT is one of many, and they will only get more powerful over the next few years. So it’ll be an interesting thing to keep an eye on and see how they use.

I think they can enhance students’ work if used carefully, and if we spend time training them in how to use them effectively, and make sure that they use them responsibly. And that’s part of, I suppose, our job as teachers is to embrace this new tech, don’t shy away from it, but make sure that we train them in the correct and responsible usage of how to use these tools, I think. And that will change over the next 10 years as these systems get more powerful, which they will.

I think things might change, but it’ll be an exciting journey.

Josh: I’m glad we have this opportunity to sit and talk! Take care everyone, see you all soon.