Tailoring Diagnostic Modeling to Individual Learners: Personalized Distractor Generation via MCTS-Guided Reasoning Reconstruction

1 Zhejiang University
2 Hong Kong University of Science and Technology
3 East China Normal University

*Corresponding Author

Abstract

Distractors—incorrect yet plausible answer choices in multiple-choice questions (MCQs)—are vital in educational assessments, as they help identify student misconceptions by presenting potential reasoning errors. Current distractor generation methods typically produce shared distractors for all students, ignoring the individual variations in reasoning, which limits their diagnostic effectiveness. To tackle this challenge, we introduce the task of Personalized Distractor Generation, which tailors distractors to each student's specific cognitive flaws, inferred from their past question-answering (QA) history. While promising, this task is particularly demanding due to the limited number of QA records available for each student, which are insufficient for training, as well as the absence of their underlying reasoning process. To overcome this, we propose a novel, training-free two-stage framework. In the first stage, Monte Carlo Tree Search (MCTS) is used to reconstruct the student's reasoning process from past errors, creating a student-specific misconception prototype. In the second stage, this prototype guides the simulation of the student's reasoning on new questions, generating personalized distractors that resonate with their individual misconceptions. Our experiments, conducted on 1,361 students across 6 subjects, demonstrate that this approach outperforms existing methods in generating plausible, personalized distractors, and also effectively adapts to group-level settings, highlighting its robustness and versatility.

Introduction Figure

Group-level distractors often fail to reveal diverse student misconceptions, while personalized distractors better align with individual reasoning errors.

Dataset Curation

To support our personalized distractor generation task, we construct Student_1361, a dataset of sequential student QA records collected from PTA and Eedi.

Method

Method Figure

Overview of our two-stage personalized distractor generation framework. In the first stage, given a student's multiple-choice question-answering (MCQA) records, we construct a personalized misconception prototype. This involves: (1) extracting relevant knowledge concepts for each record; (2) applying Monte Carlo Tree Search (MCTS) to reconstruct plausible reasoning trajectories that lead to the student's selected distractors; and (3) summarizing these error trajectories and related concepts into a personalized misconception prototype. In the second stage, given a new question, we retrieve relevant misconceptions from the prototype, feed them into the model to simulate a plausible reasoning, and generate a personalized distractor.

User Study

Case Figure

The control-group student chooses None of the above, because none of the default group-level distractors reflects their actual misconception. In contrast, the treatment-group student selects the personalized distractor that accurately reflects the misconception.