Hi Everyone! So over the course of the next year, we will be posting a series of conversations on how ChatGPT affects learning, in a variety of different STEM subjects. In this blog we focus on ChatGPT and Chemistry, and we discuss the subject with 4  chemistry teachers: Eddie and Sam who teach chemistry in the US, and Roger and Mike, who teach chemistry in the UK. Generally in the UK schools will follow either the IB® or A Levels, or GCSE for younger students; in the US schools will follow the AP® curriculum or have their own independently designed courses.

Josh (moderator): Okay, here we go. Yeah. So I wanted to talk about ChatGPT and chemistry. And I guess one of the main questions is what do you guys think in terms of how is ChatGPT going to affect chemistry teaching, going forward?

Roger (Chemistry teacher in the UK): I’ll start because I think I probably precipitated this conversation. I was quite frustrated with it a while ago; it’s a language model and so it’s really good at writing really convincing stuff and it writes it in a really coherent way. And I was just finding, we put numerous things up in our department meetings where we we look at things and it was just absolutely solid gold…rubbish. It writes the wrong thing really, really convincingly and it would 100% convince all my students, even the very best ones, they would buy absolutely everything because it sounds really good. And so that was my real concern. The danger is that if you read something that looks like it’s been written by a student, and there’s obviously Student Room and Quora and all sorts of websites that have people write it, you can tell that it’s just a student giving their opinion and my students take that with a pinch of salt. But this just sounds like a teacher. It sounds so well written.

But – just this afternoon, I was just sort of fiddling around with it, just sort of reminding myself what has irritated me. And I actually have changed my mind! It’s actually so much better. I asked it exactly the same questions that we’d asked before and it’s almost, you know, it was almost impossible to make a mistake and it’s actually now seemingly really really good.

Mike (Chemistry teacher in the UK): Which one was that? Was that ChatGPT 4?

Roger: Well I was using the free account on ChatGPT and it gives you a certain amount of free hits on on the GPT-4 little zero and then I asked it I don’t know ten questions and it said right “you’ve used your free plan until 6:22” so in another two hours I get another question. But yeah that was certainly much better. And the questions I was asking it were things like explain what aromatic means, discuss the stability of the sort of cyclohexadienes you know the 1,3 and the 1,4 and the chelate effect. Those were things that we had found in the past that it was awful at. The chelate effect, which is just something that, you know, in our syllabus, where if you’ve got ligands, you’ll always substitute bi-dentate ligands. Monodentate ligands will be substituted by bi-dentate ligands. But the reasoning of that is something that our students would have to explain. And it gave them exactly the wrong answer, the sort of mistake that students would make. But I think part of the problem, I’ll shut up in a minute, part of the problem is actually the UK system. And I’ll be interested to see what the guys in America say. Because we are so beholden to exams, and that there is a right answer and there is a wrong answer. And it’s not about, we’d love if it was, it’s not about a general understanding of chemistry, it’s about can you nail these answers. And the danger is they do this in ChatGPT, they find out something else that is sort of right, but it’s not what our exam boards want them to write, and therefore it worries me that they’ll go down the wrong path. But I think that’s a problem with the UK as much as anything.

Josh: So do you think that this is a… when you’re saying that it now has become better, do you think that the problem been fixed at this point, do you think? I mean, or is it still…

Roger: It’s a needle, isn’t it? And it’s… at one side, it was sort of active. Whatever it was writing was almost universally rubbish. It’s now… you know, I was asking it about this chelate effect. And it was, I would want it to give an answer that’s just based on entropy. And it was giving me an answer saying, well, it is entropy, but it was also the enthalpy of the things. And it was not great, but it was sort of, I suppose it was like a good, another good A-level student, if you like. You know, it’s at that level. And I suspect it will just improve to the fact that it becomes to the level of a poor teacher and then it’ll get to the level of a really good teacher at some point but I think if we wait until it’s perfect before we encourage people to use it well we’ll never get there. But is it doing more good than bad? if you’d asked me two months ago I’d have said it’s doing more bad than good. I think now I’d say it’s actually more good than bad.

Mike: I think it depends what you’re going to use it for really. I think one of the one of the dangers going forward is going to be individual website creators using it to create exam style content, but in a lazy manner. And getting it to either generate questions or generate mark schemes because they a) can’t be bothered to do it themselves or b) obviously it’s a high throughput thing and everything else is copyright protected. So I think there is a potential of of getting some rubbish questions out there on question banks and even worse answers and we’re starting to see this in places where you’re getting these websites being created and hosted. We’ve been having issues with people being able to get hold of exams on these websites when they shouldn’t be able to. And I wonder whether what’s going to happen is there’s this going to be this sudden, you know, pay 9.99 a month and get all the exam questions you want and kids will start learning the wrong things. Because I have used it in the past as Roger was saying. I haven’t used it since probably March or April when we were revising with my iGCSE group, so my 16 year olds. We used it deliberately as an activity to correct what ChatGPT was getting wrong. And it was quite fun and actually quite eye-opening for them at that stage to go, look, if you just looked at this, the equations look great. Everything I actually asked it, you know, can you explain – one of my favorites was, please could you explain the formation of the products at the electrodes in the electrolysis of aqueous copper sulfate solution. And you looked at it and it looked great. It looked brilliant, it had, you know, all the superscripts and electrons here and there. When you, when you then went into it, it was utter rubbish. It was talking about reduction at the anode and the reduction process was Cu 2+ going to Cu+, or Cu 2+ plus electrons going to whatever. It was horrendous. It had Cu 2+ losing two more electrons and going back to copper metal. So it had all the right, it was Morcombe and Wise, it had all the right bits but not necessarily in the right order. Which obviously would be really convincing for a teenager. (Editor’s note for non Brits: Morcombe and Wise were a British comedy duo).

Roger: We did something with some pH calculations and the students were, six students, and they were working away at it. And it was actually the best student in the room was kind of laughing and said, oh hang on, it’s made another mistake. And then he/she would say to it, isn’t the answer this? And then ChatGPT would go, oh, I’m so sorry, yes, you are correct. So that student was actually having fun and picking up, saying okay, it got the right answer, ah, it got that wrong, and it was really good. But the weakest student was completely flabby, he was just like, I just don’t understand this, I don’t get this, I can’t do any of this, you know, I think it’s this, and I was like, “you’re right!”. And he told me “but it says it’s this”, and I replied, “well, it’s wrong”. And that student that didn’t have the confidence to fight with it, was actually just being really sort of demoralized. And who are the students that are going to go to those websites and go to those places? It’s going to be the students at the lower end of the ability spectrum, I think, who are looking for a quick solution and looking for a quick practice. I’m sure, Mike, you’d find the same. The ones that jump to the exam papers in about the first week of the course and you’re like “oh God”, it’s like it’s like trying to take a driving test after your first lesson so “I’ll just give myself some practice”. No, don’t! Just learn chemistry and then we’ll worry about the exams later on but you know when it’s just on a plate and it’s so easy I think it has been very bad. But as I said it feels it feels better now; it feels like it’s got a lot better.

Mike: Yeah, I think IB® as well. IB® has a has a major coursework component. It’s counts for one-fifth of the of the work and then you also have to do a compulsory 4000 word essay on a subject which for most students is not chemistry. It’s going to be something actually where ChatGPT probably is very very tempting to use for history or economics. But we’re starting to have worries and concerns about how students might use ChatGPT to create quite significant chunks of text because they’re, you know, these are multi thousand word scientific reports. They have to discuss experiments, they have to write introductions. So it is a piece of writing. And I think I think the IB and possibly other exam boards are taking on the fact that its out there and they have got to deal with it, and it’s about educating students and teachers on the best way of doing that. So actually we created an in-house kind of declaration that students are now having to put at the bottom of every single submitted piece of coursework where they pick one of five statements statements which best represents their use of AI – starting with “I had nothing to do with it I didn’t even touch it” to “I used it to generate some prompts”, “I used it to create a structure for me”, “it wrote the essay for me”… I don’t know anyone would self-declare that but it’s there as an option kind of to preempt this possibility that you can use it for good, you can use it to get some ideas, maybe help with some revision once it gets good enough. But also then there’s the nefarious side of downright plagiarism.

Josh: Eddie and Sam, can we hear from you guys? You wanna weigh in? Yeah, Eddie you can go.

Eddie (chemistry teacher in the US): I think in the chemistry department, what we’ve seen in our experiments is that, similar to what Roger and Mike were saying, that it can get about 75% of the way there. The content is pretty good, but it can’t fully answer the questions. And you can tell as a teacher if students have been using AI for lab reports. I mean, really, we see it. We don’t have them write papers. We see it mostly in the lab report setting. I think that’s put a premium on questions that are intentionally designed such that you can’t really use AI. It’s also moved a number of our questions into the classroom. But I think it’s a big problem. We have half of my department, we have a department of 10 chemistry teachers, half of them don’t want to give lab reports anymore because they think their kids are going to cheat. I’m on the other side. I think we need to educate our kids and make sure that they understand the dangers and the uses of AI. I’m kind of moved by Mike’s comment about declaring their use of AI. I think we would say that many teachers don’t want them interacting at all with that platform. We’ve also seen it on exams. The other thing that I read about, I don’t know if you saw it – it turns out OpenAI has a platform that can detect when material has been generated using ChatGPT. They’ve had this for a year but they haven’t released I; it just came out in the New York Times. They haven’t released it because they’re worried about their usage rates going down by about 30%. I think it’d be interesting if they did release that tool.

I think one good use of AI that I could see, I have a friend who works for a company called Got It AI, and what they do is design, they’ve started with math, but they design a chat box that will lead the students through a problem. It’s designed mostly for community college students who might not have the resources to really thrive in calculus and in chemistry. So maybe you have an integral, or a series, you don’t know how to solve it, you can type in “I’m struggling with this” and they’ll lead you through step-by-step how to do that problem. They have a tool for chemistry that I’ve been working on with them a little bit. Things that they struggle with are Lewis structures and chemical equations that might not be easily transcribed from math to chemistry. But that I could see being a helpful tool for students who need help and might not have somebody readily available. But I think overall, I would say that most of my colleagues see very limited upside to AI.

Sam (Chemistry teacher in the US): I think I am kind of feeling the same as Eddie. I think a lot of my colleagues don’t know what to do; I think one of our major admins called it kind of like a fad that was going to go away. And so I think it’s just like such a big problem that they don’t know how to deal with it. So they’re choosing not to at the moment, and as we have a lot of independence as teachers at our school as well, I think that it’s kind of just up to each teacher to figure out what to do with it, which is tricky. I think what we’ve seen over the past like at least decade is that the college arms race, especially in the US, is just getting so intense that a lot of students and their parents feel like they have to take as many classes as possible, do as many extracurriculars as possible, to max out how they’re using their time. And ChatGPT and AI is a really lovely way to, you know, any take-home homework, just have ChatGPT do it for you, so then you don’t have to waste your time on it, and then you can kind of do other stuff with your time. That obviously catches up with them on in-class work, quizzes, tests, and stuff like that, but then the students and the parents get upset and will say “My children showed that they could do this work in the lab report, and you’re saying that they couldn’t do it on the quiz or the test”, and then you have to have all these conversations. You have to be really careful, obviously, about accusing kids of, you know, cheating or using things that we ask them not to use.

Josh: As a separate note, the level of cheating that goes on – now, you know, for many years, I did this, I did tutoring in students’ homes. The level, and you guys I’m sure all know this, but it’s just, the level of cheating is amazing. And it’s not just the kids, it’s the parents. It’s the parents. The parents will say, okay, so this kid was home from school sick for one day, often faking it with the parents’ approval – I don’t know. But the mother will – right in front of me – will say to her son, “call up Freddy, he took the test today, so you can go over it with Josh”. And I had another parent, this kid had a take-home test and the father said “oh you have my permission to help him on the take-home test”. It’s over the top.

Sam: Yeah, I keep track of when students miss the day of a test. We have a block schedule, so some kids take the test on one day versus another. There’s obviously different versions. And the number of kids who are just conveniently missing school on test days has ramped up tremendously. And then you obviously call them out on it, but there’s only so much you can do. I was going to say, so I like the idea of the AI statement at the bottom of the assignment. I would love my students to be honest about it. I don’t know how many of them would be. I think it kind of sounds like our, at the beginning of every quiz or test or whatever at the top of the page, like we require our students to write some kind of honor statement that says like, I didn’t cheat or whatever, and like put that on every assignment. So I kind of like that thing at the bottom of each assignment of yours, if only to constantly remind them. And if you know, you have to have a conversation with them or their parents, you can say “you stated clearly here that you didn’t use AI. Like, you had the choice to be honest, and you chose not to”.

Mike: I think the one benefit of the series of statements is actually educating them that there might be times when using it is actually a really good thing for them to do, because they might learn something. They might get through a blockage. So especially when you’re asking them to write a report in a particular way, you know maybe in the style of a peer-reviewed publication and you know, we do our best to try and educate them, but they want that extra bit you know, they can use ChatGPT as a prompter and you know, I had a student writing a 4,000 word essay on how is bone density affected by sources of calcium in older women or something like this. And then another example, they had to have another example and we were like, crikey, you know, so we just typed it in to ChatGPT and you find some examples of bone density that looked at something different from calcium and then it goes “these are all of the options” and you go through them and half of them have used something else and a few of them didn’t so you go “ah we’ll use that” so they’ll declare that at the bottom that they used it to help find some examples. It’s just an electronic librarian ultimately in that case. It didn’t write anything for them, it just guided them to then some journal articles that they could then read and incorporate. So I think that’s a really healthy way of doing it.

My worry is about teacher use as well. We would love to think that all our colleagues around the world are as dedicated and as educated as we are and our other esteemed colleagues are, but that simply isn’t the case. There will be teachers out there who take the shortcut or simply don’t recognize or are naive about it and don’t recognize when errors are made by ChatGPT and are pushing it to students as the gospel truth and we then might end up with a kind of an epidemic of you know having to – you know I mean good for the tuition business – you know but having to undo quite a lot of wrongs and that does worry me as well. And obviously tutoring is an issue because obviously it’s unregulated and you don’t have to have any qualifications to be a tutor and there is that temptation of creating lessons or content or whatever I can imagine but you know someone who is doing it under pressure…

Roger: The thing I think we’ll find with lots of that though is that there’s a very sort of natural review system that’s in there. You know you talked at the beginning about the service that sells past papers. Well if actually our students bought one of those and they were doing it with their mates and they were like, oh that question’s rubbish and they took it to their teacher and their teacher said no no no that question’s wrong. I think those sites will only stand, they’re so easy to make them that they will only be successful if they get good reviews. And I think there’s almost nowadays a very sort of organic process that allows these things to be reviewed and refined. But if they’re rubbish, then I think they won’t survive. And I think that would be the same with teachers and the same with tutors. If you get turned out to be telling students the wrong thing, then I think you’re very quickly off Josh’s list of preferred tutors. I think it’s been really interesting listening to the difference between the American systems which are largely not examined and ours which largely are because your lab reports count. Your tests that you do in class they count. They count for their GPA presumably whereas in ours they count for nothing. At the end of the day they know that we cannot help them. Whatever we tell them, if they cheat all the way through the course they’re going to sit in an exam room at the end of the day and if they fall flat on their face they’re just gonna look stupid. You know Josh you said that this cheating is really widespread. I actually don’t think it is particularly; I think especially Mike and I work for schools where they have to pay quite a lot of money to come there and I think they invest in that, they know they’re paying for a service and if they… of course they’ll maybe cheat on the odd homework when they run out of time to do it but largely they are engaged with that education process. They want to get better and they know that if they’re just going to cheat but the example that Mike said with the IB® – of course you get that coursework that does count and it does actually translate directly into an IB® Baccalaureate, and that’s when the temptation is to cheat. But actually in this situation (A Levels) I sort of can be a bit more relaxed. If they cheat on their homework, it’s going to cost them in the long run.

Eddie: Roger, that’s an interesting point about the difference and I think that’s one of the reasons why we might have a different lens that we view the AI through. I mean, grade inflation in the United States, it’s so high and so there there are these micro differences in the students’ GPA and anything the kids can do to get ahead they’re going to use it. Josh, you know I should be clear AI and chemistry I think is a minor issue relative to the humanities, in terms of the cheating aspect.

Josh: Yes, I agree.

Eddie: But I don’t even think we would get to the point that Mike described where we use it as a resource. I mean, I think we should ultimately move in that direction. I think many teachers would be scared that if we introduce ChatGPT to the kids, they’re gonna use it for nefarious reasons.

Josh: Agree, okay, I have to wrap up in like five minutes. I just have one final question which is that let’s say that they come out with the quote-unquote perfect version as Jules was alluding to earlier because eventually they will come out with a close to perfect version or a very, very good version. How about then? Let’s say that they do that in a year from now. How helpful or harmful do you think it will be in terms of overall assistance as an aid to learning chemistry?

Roger: I think it won’t affect schools because I think we’ve all realized that schools are far more than just the dissemination of information and I think kids still like to be taught by a human being and have fun and jokes and enjoy with their classmates. I do wonder where and how it will affect the the tutoring business? So I think not on schools, but I think yes on tutoring, I think it’ll have an effect.

Mike: I think it will depend on the assessment system as well, because obviously the answer you get from ChatGPT is only really as good as the question you ask it, especially if you’re getting something that’s near to perfect, you’re going to have to ask it some quite nuanced questions to get the answer that you really are seeking. So, I think it will depend on the assessment systemin terms of will tutoring become irrelevant? Because one of the benefits of having a conversation with someone who has the experience is that you can ask and you can get those subtleties out of a student and out of the content I think much more easily than a student sitting and asking a fairly mundanely worded question and getting an answer but actually it doesn’t quite hit the mark. You know what it is, I mean I’ve seen students try to do Google searches and I never quite understand you know “I want you to find out something about the history of benzene” and at no point in Google do they type in the words history or benzene and you kind of wonder what’s going through their head.

Roger: So you wonder whether the better ChatGPT will already know what year group they’re in, the history, what subjects they’re studying, and actually when they ask a fairly clumsy question, if it’s actually perfect or it’s much better at building and it knows everything about you, whether that will actually, it will know what you wanted to ask. There’s all sorts of ways it could, I totally agree.

Josh: I personally am not terribly worried about it because of the, I mean obviously I have a vested interest in not worrying about it or maybe I’m being overly optimistic, but I believe in what you guys are saying, which is that the subtleties of the interactions where a person can detect where a student needs to go to get to that next level of understanding is never going to be superseded in ability by an artificial intelligence program.

Sam: Yeah I might be a little cynical so apologies but especially since COVID the number of students who, like I give them just like a paragraph to read or even like half a page to read and they’ll just be like “can you just explain it to me instead?”. So like even if ChatGPT makes the perfect answer I don’t know if they’re going to be willing to actually sit down and read the whole thing and understand it, rather than just like, they’ll go to a tutor and say, pardon, just explain it to me. So our lab reports used to be much longer, we’ve had to cut them down in terms of how much content we give them to read, because otherwise they just won’t read it because unfortunately, ChatGPT cannot read for them. Which is kind of like the barrier that they’re having to face.

Josh: Eddie, any last minute thoughts?

Eddie: Yeah, I guess my final thought is I think chemistry is hard. With AI, it’s a lot harder than math. It’s a lot harder than history and English. You know, just thinking about the next step beyond general chemistry, you give it a simple, simple organic chemistry problem, it can’t do it. So I think it’ll get to the point where it’s good at the rudimentary stuff. I don’t see it being as robust, and again, maybe I’m a little bit cynical too, but to the point where it is a powerful tool for chemistry, the chemistry that we teach.

Josh: Excellent. Okay, well, thanks everybody and have an awesome day!

Warp Drive Tutors, Inc. has been developed independently from and is not endorsed by the International Baccalaureate Organization. International Baccalaureate®, Baccalaureat International®, Bachillerato Internacional® and IB® are registered trademarks owned by the International Baccalaureate Organization. AP®, Advanced Placement®, College Board® and SAT® are trademarks registered by the College Board, which is not affiliated with, and does not endorse, this product. ACT® is a registered trademark of ACT, Inc. which was not involved in the production of, and does not endorse, Warp Drive Tutors. Inc. Warp Drive Tutors, Inc. has been developed independently from and is not endorsed by Cambridge Assessment International Education. A-Level, GCSE and iGCSE are registered trademarks owned by Cambridge Assessment International Education. None of these trademark holders are in any way associated with Warp Drive Tutors, Inc or this website.