Redesigning first use for an AI tutor
Iris is an AI tutor built into Artemis, an open-source university learning platform for programming courses at the Technical University of Munich (TUM). It answers with hints and leading questions rather than solutions. The approach is pedagogically sound, but nothing in the interface prepared students for it.
For my Master’s thesis, I treated the model and the pedagogy as fixed and redesigned everything around them: where students find Iris, what they meet when they open it, and what they expect before they type. I shipped the changes into the Artemis codebase and tested them there.
Role
Timeline
Focus
Context
Some introductory programming courses at TUM enroll more than 1,000 students. Tutoring groups are too large for one-on-one help, so a student who gets stuck either waits for the next tutorial slot or works through it alone.
Iris was built for that gap. On paper it beat the other AI tools students were already using. It sat inside the platform, it could see their work, and it cost nothing.
Reads the current exercise, submitted code, and test results straight from Artemis. No pasting, no explaining.
Holds back complete solutions at every support level, which no general-purpose AI assistant does.
No subscription required. ChatGPT’s better models cost money that not every student has.
Iris had been part of Artemis before I started. Published work on the platform had already documented low adoption without establishing the cause. One earlier study traced the dissatisfaction to the interface rather than the model, but did not specify which part of it.
Results
SUS mean score
4. Brooke, J. (2013). SUS: A retrospective. Journal of Usability Studies, 8(2), 29–40.
All students rated the redesign higher. The ratings clustered where the baseline ones had been scattered. 6 of the 10 baseline scores sat below the industry average of 68. All ten redesigned scores cleared it.
Onboarding comprehension
Baseline
8 passed
Redesign
10 passed
After each version, participants answered 5 questions about what Iris does, what it refuses to do, and what it can already see. 4 correct was the passing mark. The students who failed at baseline both missed what Iris is for, which is the misunderstanding the onboarding was built to fix.
Problem
Students who had formed an accurate picture of Iris trusted it more than general-purpose tools for exercise-specific questions. That was the finding that shaped everything after it.
Students were not rejecting Iris. Most never found it, and the ones who heard about it saw no reason to try. The few who opened it were left to work out what it was, from a help page they would not read.
I ran thirteen sessions with students in introductory programming courses: nine interviews, and four watching them use the existing interface.
The entry point sat below everything around it, changed position between page types, and was missing from the Artemis sidebar where students orient themselves. The main interface was hidden behind a tab labeled "Dashboard," a word unrelated to Iris.
5 out of 9 students
never tried Iris
They knew it existed but could not name a reason to choose it over what they already used. One described Iris as an old-fashioned chatbot that provides technical support in Artemis, rather than a coding tutor.
Iris opened by asking how the student planned to approach the exercise. It did not say what Iris could do, and it suggested nothing to type. The widget on exercise pages opened with an empty input field and no suggestions.
An About Iris page existed. It was long and text-heavy, and those who opened it skimmed once and said they would not read it again. Every student who knew about Iris had learned about it from an instructor or an in-course introduction, never from the interface itself.
Designing First Use
Iris replaced Dashboard in the Artemis sidebar. Adding an entry alongside it would have been safer, but it would have left the confusing label in place.
The onboarding had one job that most onboarding does not: it had to explain a refusal before a student ran into it. Iris does not hand over answers, and a student who learns that by asking and being refused reads it as a broken tool.
For this, I used Google Gemini and Figma Make to build the first two versions quickly and tested each one informally. Both are shown below as screen recordings from those tools. The third is the Figma design I then built into Artemis.
Three chips under the input answer that by example: Learn, Quiz, Tips. Each fills the field with a prompt built from the exercise the student is on, so a student can send one without composing anything. All three are phrased as inquiry, since a chip reading "fix my code" would demonstrate the one thing Iris refuses to do.
The widget had no room for chips, so it got a cycling placeholder and ghost-text completion instead. Same job, less space.
Students could not tell what Iris was doing while it worked. The thinking state existed, but it used technical labels beginners did not recognize. Rewriting them in plain language made them readable and still did not say much.
It now moves through the stages as they happen: reading the exercise, checking the submitted code, writing a reply.
Resuming a past conversation meant leaving the exercise page and going back to the Dashboard. One student had given up and started a new chat every time. A dropdown in the widget header now lists chats for the current exercise, sorted by last activity.
Testing
10 students
Each used both interfaces in one session
3 tasks
Understand Iris, send a first message, find a past chat
2 interfaces
Old → new, back to back in one session
Each participant worked through the old interface first and the redesign second, thinking aloud throughout, and rated each one immediately after finishing its tasks. All ten had used Artemis and Iris before, so nobody in the study was encountering Iris for the first time.
Participants hovered over the chips, read them, then typed their own question. Almost nobody clicked one.
I built them to save students the effort of composing a first message. What they did instead was show what kind of question Iris handles, which students used to write a better question themselves.
Low click rate is not a reason to remove them. Showing students what to ask turned out to be worth more than saving them the typing.
After the redesign, participants described Iris as a tutor. At baseline, they had called it a chatbot and a search tool.
Both participants who failed the comprehension check at baseline gained more than the group average, 65 and 35 points against a mean of 29.5.
S8
+65.0
S7
+52.5
S5
+40.0
S4
+35.0
S6
+30.0
S3
+20.0
S2
+17.5
S10
+17.5
S9
+10.0
S1
+7.5
Implementation
20 pull requests
Reviewed by Artemis developers and merged into main
1,000+ students
Iris reaches this many students per semester on the platform
I used Claude Code to get the designs into the Artemis repository . Building on an open-source platform meant working inside its component library, design tokens, and conventions.
Iris was not doing that. It used ad-hoc color and spacing values instead of the Artemis tokens, so before implementing anything I consolidated the colors Iris uses onto the existing token set. It now looks like part of the platform. Every new component conforms to WCAG 2.2 AA.
Limitations
Every participant used the old interface first. Some of the gain is familiarity with the tasks rather than the redesign, and counterbalancing the order would have separated the two. I also designed the interventions and ran every session, so participants were rating my work with me in the room. Both are reasons to read the numbers as a ceiling rather than a settled figure.
All ten participants had used Artemis and Iris before. The measures describe how returning students respond to a new first-use experience, not how students meeting Iris for the first time respond to it. That is a real gap for a project about first use, and closing it would mean recruiting students at the start of a semester before any instructor has mentioned Iris.
The sidebar entry, the onboarding, and the prompt scaffolding were tested together. The improvement belongs to the three of them as a set, and nothing in the data says how much any one contributed. Isolating them would have required separate conditions and more participants than the study had.