Department of Methodology, LSE
20 Nov 2026

GENIAL: When do Generative AI tools act as a catalyst for learning?
We asked students to share their chat logs and in the case of my data science courses, I also collected the git histories of their assignments.

Looking at the data, it was very clear just how much of a double-edged sword GenAI could be: two second-year undergraduates doing the same assignment, both using ChatGPT but with very different approaches to their independent study practices.
Student A Second year, International Social and Public Policy. No prior programming experience.
The task involved writing code to:
Student A copy-pasted links and HTML by hand, because ChatGPTās example output looked like the finished task and never asked for the scraping code.
Student B Second year, Economics. Had coded in Python before.
ChatGPT often produced overcomplicated code. The student went back to the teaching material and told the AI:
āyour stuff is not in line with what I learnedā

Sallai, Cardoso-Silva, Barreto, Panero, Berrada & Luxmoore (2024). Approach Generative AI Tools Proactively or Risk Bypassing the Learning Process in Higher Education. LSE Public Policy Review.
Cardoso-Silva, Sallai, Kearney, Panero & Barreto (2025), the GENIAL framework, SSRN.


Brave new words: how AI will revolutionise education, with Salman Khan Ā· DSI Event Ā· Wed 26 Jun 2024
š Read my book review.

Matt Barnum, Chalkbeat, 9 Apr 2026 Ā· Why Sal Khanās AI revolution hasnāt happened yet, according to Sal Khan

Khosravi, H., Gasevic, D., Sadiq, S., et al. (2026). Building AI Companions that Prioritise Learning over Performance. Preprint, arXiv, pp. 13ā14. doi.org/10.48550/arXiv.2605.04816
In an interview April 2026, though, Sal Khan was saying that for many students Khanmigo had been a non-event (so far?).
It started out as a āSocraticā bot, but seeing the frustration of students, Khanmigo developers adjusted it: the tutor now offers worked examples and hints that fit what each student requires, alongside its questions.
Default āLearning modeā / Socratic mode, in use by all major AI companies, is NOT a good pedagogical feature.


Claude responding to a studentās question used as a study tool, with an ever-sprawling number of sub-questions.
It gives me five really great conceptual questions that I should think about, but then it gets so lost in the tiny little sub-questions that I donāt even get to talk about the five big ones.
Lena (pseudonym), postgraduate student, interview after the course
Cardoso-Silva, J., & Ruf, J. (2026). Coexistence of mathematical reasoning and cognitive offloading: A longitudinal case study of one studentās chatbot use across a term of proof-based mathematics. Preprint, SSRN. doi.org/10.2139/ssrn.7608220
ā¦while trying to overcome the limitations mentioned before
(simplified high-level view) The goal is to give the LLM enough context on the why we do things in our courses and a particular order and how, that is the way we expect students to learn in our disciplines.
I built and deployed this as a Claude Project. Letās have a look āļø
While this software is not ready, I was able to replicate this behaviour using only prompts and text files.

CURRENT WEEK and CURRENT PHASE manually. Claudeās clock inference was unreliable.weekly/ and assignments/ encode what the course has covered so far, up until the current moment in the course.ā ļø This is an old template, shown here just so you have a more concrete idea of how it works! We are going to publish a preprint and a tutorial, along with some of our newer prompt templates you could adapt to Claude, ChatGPT or Google Gemini for your own students.

DS105W (Winter 2025/26): formative practice W01 to W04, then Mini Projects across the term.
Daily message volume across the teaching term. Counts include only the 40 students who shared at least one chat log from our AI tutor. Background bands mark assessment periods; dashed lines mark Thursdays (lecture day).
Every time a student asked about a concept, the AI tutor pointed them to where and when in the course they had already met it, before asking what approach they would take.
Questions about platform, set up, or more operational in nature didnāt go āSocraticā but were delivered as a straight answer to the student.
Questions about off topic content, in relation to where we were in the course, were dealt with correctly.

The AI tutor correctly indicate to the student in Week 04 that they should not (yet) use the pandas library at this stage in the course. The tutor reinforced the fact I wanted people to explore different tools at different points in the course.

Later in the term, answering to another student about suitability of libraries, the AI tutor correctly indicated that at that point in the course, the pandas library was the most adequate choice, as indicated in the course.
The AI tutor (and the students) sometimes referred to me by name.

I found 60 occurrences of my name in the dataset of chat logs š
Thankfully, every one of those replies was accurate to what I actually instructed the chatbot to say. It didnāt fabricate anything.
Note though that fabricated attributions are indeed a risk of this type of technology. We canāt forgo AI literacy training.
Students themselves mentioned my name in 16 messages, across 22 chat logs.

Each square represents 20 messages exchanged with the AI tutor.
We could not control programmatically how Claude was responding to students: the longer a conversation ran, the more likely the LLM was to drift from the instructions.

The student should be the one justifying its choices. Here, the AI tutor offers a justification when the student didnāt even ask for it!
In a later implementation, we added a protocol (always on) to make sure conversations donāt change topics or go longer than 15 to 20 messages on the same topic.
| 1 | Write model questions and answers, not āwhen a student asks X, do Y.ā An LLM given the second form parrots the instruction back to the student, word for word. |
| 2 | Keep each protocolās trigger distinct from every other protocolās. If two protocols could both plausibly match the same message, the LLM has to guess which one you meant. |
| 3 | Start with one system prompt, then test it by playing a student, before splitting it into separate files. |
DS105A and DS105W (data science and programming courses)MY202A (machine learning)MG317 (organisational learning)AC102 and AC105 (introductory accounting courses), amongst othersWhat I am testing now: give the chatbot the skills we expect each student to demonstrate awareness of or proficiency in each week.

MY202A, Week 01.
You would fill out a form describing your course and how you want the tutor to behave.

Course description
Teaching approach


Cardoso-Silva, J., Sallai, D., Kearney, C., Panero, F., & Barreto, M. E. (2025). Mapping Student-GenAI Interactions onto Experiential Learning: The GENIAL Framework. SSRN Electronic Journal. (Under Review)
Sallai, D., Cardoso-Silva, J., et al. (2024). Approach Generative AI Tools Proactively or Risk Bypassing the Learning Process in Higher Education. LSE Public Policy Review, 3(3), 7.
Cardoso-Silva, J., & Ruf, J. (2026). Coexistence of mathematical reasoning and cognitive offloading: A longitudinal case study of one studentās chatbot use across a term of proof-based mathematics. Preprint, SSRN.

Cardoso-Silva, J., & Williams, S. (2026). Beyond Socratic modes: designing a chatbot tutor around the courseās sequence and pedagogy. Preprint coming soon.
Contact me if youād like to exchange ideas, or to test these ideas and this analysis on your own courses or research.
The whole āhuman in the loopā concept, borrowed from software engineering, doesnāt make sense in education. We should be thinking of āAI in the loopā instead.
Cardoso-Silva (2026) | 20 Nov 2026 Ā· AI in Teaching Working Group, CIEL, Kingās College London