SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems

Rima Hazra and Mykola Pechenizkiy (Eindhoven University of Technology), Bikram Ghuku and Sayan Layek (IIT Kharagpur), Somnath Banerjee (Cisco Research), Ilona Marchenko (Taras Shevchenko National University of Kyiv), Yaroslava Tokarieva (National Technical University of Ukraine) and Julia Stoyanovich (New York University). arXiv 2603.17373, 19 March 2026. Preprint; tiered T1 as a primary study reporting its own method and data.

Held here for its Appendix D, not for its benchmark. The paper's subject — whether an LLM tutor erodes learning by handing over answers — sits with ../engineering-education-wiki and ../ai-governance-wiki, and the adjacency is noted rather than duplicated. What this spoke wanted is the thing who-annotates-in-nlp showed almost nobody publishes: one labelling operation written down end to end. Appendix D is that document. Ingested via the research pass against growth edge 1a.

The operation, stage by stage

Three stages, three workforces, three pay rates, all stated:

Stage	Who	n	Pay	Task
1 — domain validity	undergraduates, ≥2 years of coursework in the subject	6 (2 each: maths, physics, chemistry)	$5/hr	is the student question well-formed, realistic, in scope?
2 — risk alignment	doctoral students, ≥2 years teaching or tutoring	3 (1 per subject)	$15/hr	does the interaction really instantiate its assigned risk category?
3 — crowd check	Prolific workers, STEM degree, ≥98% approval rate	24 (8 per subject)	$10/hr	the Stage 2 judgement, in plain language

Timing is reported too: roughly 2.5 minutes per single-turn instance and 6 minutes per multi-turn conversation at Stage 1. Stage 2 annotators are also paid for calibration and norming time, which most papers do not mention at all.

The paper never states a total spend. From its own rates and times, Stage 1 alone comes to about 195 annotator-hours, near $975 — our arithmetic from their numbers, not a figure they report.

Training, and the gate at the end of it

Every annotator in every stage goes through the same four steps before touching real data:

Guideline review — a document defining 11 risk categories and 48 sub-risks with safe/unsafe examples.
Worked examples — three per category: one clear-safe, one clear-unsafe, one borderline, each with the rationale for its label.
Pilot round — 30 instances (Stages 1 and 2) or 15 (Stage 3), drawn evenly across categories and subjects.
Norming discussion — the whole stage reviews its disagreements together. For the Prolific workers this runs asynchronously in a shared document with a moderator answering.

Then the gate: ≥80% agreement with gold-standard labels on the pilot before the main task, plus a separate 10-item qualification test requiring ≥8/10. Pass rates are published, which is rarer than the test itself: Stage 2 is 3/3; Stage 1 loses one physics undergraduate, who is replaced; Stage 3 passes 24 of 33 recruits (72.7%). So the workforce that reached the data is a filtered one, and the paper says by how much.

Adjudication, with rates

Stages 1 and 2 use two annotators per instance; disagreements go to discussion, and to a senior learning-sciences researcher if that fails. Stage 3 is three-way majority, with no-majority cases sent to the same reviewer — 4.2% of instances (152/3,600). Cross-stage conflicts (valid at Stage 1, misaligned at Stage 2) run to 3.8% (57/1,500), of which 21 are removed from the benchmark and 36 retained with a "cross-category" flag in the metadata. Both the rate and the disposition are stated.

Agreement, and what it is confounded with

Fleiss' κ falls monotonically across the stages — 0.81 → 0.73 → 0.68 on average. The authors read this as an expertise effect. It is not cleanly that: stage, task and workforce change together, so the undergraduate/PhD/crowd gradient is entangled with a factual/interpretive/simplified-label gradient. The other two patterns are cleaner. Single-turn beats multi-turn by Δκ ≈ +0.05 everywhere, because a dialogue makes it ambiguous when the harm occurred. And maths (κ 0.76) beats chemistry (0.74) and physics (0.72), which matches measuring-annotator-agreement's finding that agreement holds up where the label space is constrained.

Note what the pay gradient does not show: the $5/hr undergraduates agree with each other most, and the $15/hr doctoral annotators least. Rate tracks the difficulty of recruiting for the task, not the quality that came back.

The rest of the paper trail

Guidelines quoted verbatim (D.4), separately for each stage, including the plain-language rewrite and the added Unsure label the crowd stage gets.
The interface is named: Label Studio for Stages 1 and 2, a Qualtrics survey inside the Prolific flow for Stage 3, with a mandatory free-text justification on every negative label.
Label distribution after adjudication: 93.4% valid at Stage 1, 87.1% aligned at Stage 2, 84.7% at Stage 3.
Volume: 3,000 + 1,500 (plus ~450 from a 30% cross-validation sample) + 10,800 = 15,300 labels.
Ethics: informed consent at every stage, protocol reviewed, pay benchmarked against local minimum wage and Prolific's fair-pay guidance.

What it does not settle

This is an academic operation: 33 recruits, 15,300 labels, over a benchmark whose items were generated rather than sampled from the world — Stage 1 is validating a generation pipeline, so "sampling design" here means benchmark construction, not drawing from a population. It is not the industrial pipeline that data-cascades describes failing, and no total cost is published. For the industrial half of the same edge, see annotation-requirements-autonomous-driving, which reaches the practice by interview instead.

The benchmark result, recorded because it is the paper's actual claim: across the models tested, pedagogical failures rise from 17.7% single-turn to 77.8% multi-turn, and scale does not reliably help.

who-annotates-in-nlp · measuring-annotator-agreement · training-data-quality · annotation-requirements-autonomous-driving · pervasive-label-errors · rima-hazra