An AI Tutor for Oral Exams: Setting Difficulty and Assessing Answers

A student recognizes a correct explanation in notes but struggles to formulate an answer and handle follow-ups. Preparation needs recall, argumentation, and knowledge transfer into conversation. AI can assist when questions, help, and feedback fit the objective. Otherwise, a pleasant dialogue lets the assistant finish thoughts while students mistake supported responses for readiness.

Start with a short subject scenario a teacher can check. The tutor asks questions, allows time for attempts, and analyzes actual answers. It should not be the sole source of criteria or final competence judge. The procedure below concerns practice, separate from formal exams and readiness demonstrated independently under agreed conditions without the same hints.

Describe the Answer Students Must Construct

Define the subject action: explain causation, compare approaches, justify a choice, or apply a rule. ‘Speak confidently’ is an insufficient knowledge criterion. Fluent speech can contain foundational errors, while hesitant answers can have accurate logic. Distinguish these rather than reward external ease instead of understanding.

Record signs of a sufficient answer: identify a decisive condition, explain its effect, and give a relevant example. Do not demand verbatim notes if understanding is the goal, but identify terms whose meanings must remain precise. Students then know where personal wording is acceptable and where substitution changes the subject meaning.

Check whether questions actually elicit these features. Definition questions cannot readily assess application; an immediately complex case may conceal the missing foundation. CMU's learning principles help connect component skills, integration, and application conditions. Oral practice needs a sequence revealing each part in the answer.

Prepare Verified Subject Material

Provide a short foundation of definitions, rules, limits, and checked examples, with instructions for absent answers. The assistant should identify gaps rather than invent explanations. Begin with one small module the teacher knows well instead of a course with uncertain sources and complex file contradictions.

Save material dates and versions. Changed rules can leave old answers influencing later questions. After updates, begin a new controlled scenario or explicitly check context use. Uploading a new file does not necessarily remove old wording. The mechanism depends on the service and settings.

Check several questions independently before class. Do they exceed the source, and is feedback adequately supported? UNESCO's guidance on generative AI centers responsible educational use. Here, it begins with human verification of subject foundations and clear teacher, student, and supporting-tool roles.

Create Several Question Types

The first tests concept understanding in the student's words and distinction from a related concept. The second applies it to a new case. The third checks boundaries: when the usual method fails and what is missing. This reveals more than repeated definition paraphrases answerable from memorized text without substantive distinctions.

For each question, prepare decisive answer elements and possible mistakes rather than one long model answer to reproduce. Content conditions allow different correct formulations and expose plausible errors. If an answer depends on an assumption, supply it or allow students to identify uncertainty honestly.

Follow-ups should test thinking without quietly revealing the answer. ‘Did you consider the exception?’ already signals its importance. In checking mode, request application boundaries without naming the particular element. Targeted hints can help practice, but distinguish them from independent condition recognition.

Define Difficulty Through the Mental Action

A lower level can use known concepts and explicit conditions; a middle level a new situation with the same foundation; a higher level relevant-feature selection or explanation of a decision boundary. Do not increase difficulty only through question length or terminology. Explain the additional action and its connection to practice goals.

Change one decisive factor at a time when analyzing mistakes. Simultaneously adding a topic, unfamiliar context, and shorter time hides failure causes. Combine conditions later if useful. Initial feedback should reveal whether the issue is foundations, rule selection, reasoning, or oral-answer organization.

Allow simpler practice without labeling a weak student. After an application error, request the decisive condition and then another case. This checks the component skill and permits continuation without lowering everything. The foundation may be known while recognition of its application still needs practice.

Separate Practice and Checking Modes

During practice, the assistant can pause, ask a targeted question, and help revise a passage. During checking, students answer uninterrupted and receive analysis afterward. State the mode beforehand. Strongly prompted and independent answers should not appear equivalent: they reveal different independence levels and practice needs.

Define how students request help. Offer a sequence: goal reminder, relevant-feature guidance, partial example. Reserve complete solutions for separate analysis. This provides sufficient support without transferring all mental work to the tool. Record assistance levels where relevant to progress interpretation.

Do not reward only rapid completion. Students may need pauses to construct explanations. Unless speed is a subject objective, do not make it hidden. Time limits can serve organization, but explain their role. Match exam preparation to actual requirements instead of adding arbitrary pressure unrelated to knowledge.

Build a Rubric with Observable Features

CMU describes a rubric through criteria, levels, and performance characteristics. Choose a few substantive dimensions, such as accuracy, reasoning, and application. Consider presentation clarity separately if it is an objective. Do not merge everything into an impression of voice, confidence, or answer length.

Describe level differences specifically. Strong reasoning connects a feature with a decision; partial reasoning names it without explanation; insufficient reasoning relies on irrelevant grounds. These show students what to revise. Test the rubric on real examples, refining it if correct approaches receive different ratings solely because of form.

Do not request percentage-level grading precision you cannot justify. Practice often needs only a level per criterion with a subject explanation. Make supporting evidence visible. If a score appears, connect it to a specific answer passage. ‘Good, but could improve’ provides no usable next action.

Check the Evaluator's Performance

Prepare short answers of different quality already assessed by a teacher. Include a correct answer in unusual form, a polished erroneous answer, and a partial answer honestly identifying a boundary. Compare AI feedback with subject criteria. Test consistent recognition of decisive features rather than obedience to one score, ensuring style does not hide substantive errors.

If judgments differ, check both foundations: the human reference can also be unclear. Refine a faulty rubric and repeat the control case. If the assistant misreads subject content, limit that scenario and retain human review. NIST GenAI Profile provides context for generative-output risks; local checks must concern your configuration and learning task.

Repeat decisive examples after material, instruction, or service changes. One successful run does not establish overall reliability. A pilot can conclude narrowly which answers can be practiced, where extra checks are needed, and what should not yet be assessed automatically without subject oversight.

Separate Speech from Its Transcription

For voice scenarios, check whether decisive words were heard correctly. Misrecognized terms can look like student errors. Compare audio and available text or repeat a disputed passage. Do not infer knowledge from an inaccurate transcript. Recording and text availability depend on the tool and need pre-class checks.

Test subject terminology, speech rates, and typical audio conditions. You need not study every voice, but identify barriers affecting this exercise's fairness. If recognition is unstable, offer an agreed text route or human analysis. Preserve the objective and explain differences from oral practice if speech itself is part of the outcome.

Do not use voice-based assumptions about emotions, personality, or intelligence as subject-answer criteria. Assess demonstrated content and explicitly defined communication features. Pauses, accents, and pronunciation must not quietly replace knowledge checks. Students need criteria they can understand, meet, and discuss through specific answer passages.

Set Up the Feedback Cycle

After an attempt, identify one decisive strength and one targeted problem. Suggest a short next action: explain a condition, compare cases, or fix a causal connection. Avoid a long improvement list. Students must revise rather than merely read an assessment. If the problem persists, change the explanation or return to a prerequisite.

Use a new example with the same subject requirement. Repeating a prompted text verbatim shows something different. Compare whether students independently identified the decisive feature and linked it to the decision. That supports progress claims; conversation counts and praise do not replace independent-application checks.

Keep only needed summaries: topic, criterion, assistance level, and next practice. Full voice histories are not necessary for every objective. Agree on materials, access, and retention according to the process. Fictional or specially prepared answers suffice for a pilot; private stories are unnecessary for testing question logic, rubric quality, or typical subject errors.

A Hypothetical Rule-Explanation Exercise

Imagine a module explaining why calculating a mean from known values should not automatically replace missing entries with zeros. First students distinguish missingness and zero, then analyze a fictional table, then identify information missing for a whole-group conclusion. This is a proposed scenario rather than an experiment or proven assistant effect.

The tutor listens to the complete attempt first. If states are confused, it asks about an empty cell. After explanation, it provides another table without the same hint. The teacher checks preserved logic. Independent distinction matters rather than repetition of assistant wording. Persistent errors call for data-foundation practice instead of faster or harder conversation.

The rubric separates correctness, explanation, and application. A student may calculate correctly without explaining the denominator and need reasoning practice. Another explains the foundation but mixes numbers and needs a short controlled calculation check. The next practice follows actual answers rather than a general preparation-level label.

Check Progress Under Comparable Conditions

Save a short independent baseline answer. At the end, use another question on the same skill at similar difficulty, with the same rubric and assistance. Comparing an independent beginning with an extensively revised final answer exaggerates progress. Identify what students now do themselves and where support remains needed.

Add a delayed short check where possible to distinguish familiarity with recent text from more durable understanding. Small personal improvements do not justify broad causal claims because many learning components contribute. A practical plan can identify the next need and describe conditions honestly; researching tool effects requires an appropriate comparison design.

Ask separately about convenience, clarity, and barriers. These improve experience but do not replace demonstrated knowledge. Students can enjoy assistance while remaining hint-dependent, or find practice difficult while independent application improves. Consider both without substituting an easily surveyed outcome for another.

Agree on an Accessible Practice Route

Provide an approach for participants poorly served by a particular voice interface. WCAG 2.2 helps check relevant web-interaction properties without replacing analysis of the objective and actual tool. Another demonstration format may suit a subject objective. If oral action itself is trained, agree on a route supporting its practice with appropriate assistance and oversight.

Students should know how to stop, report technical problems, and obtain human explanations of disputed feedback. Do not make the assistant the only route to the teacher. Pilot error reports reveal where instructions, materials, or recognition fail the intended action rather than necessarily indicating poor effort.

If the WeStudy LMS platform organizes the program, connect preparation, exercise, and analysis through clear assignments. Check external voice-assistant availability and result-transfer methods separately. Initial practice may need only a short criteria-based summary. Students should see the next step and obtain subject feedback, while teachers understand attempt conditions.

Add Self-Assessment Before External Feedback

After answering, students identify what they consider sufficient and where they are uncertain before opening tutor feedback. This compares their understanding of criteria with external comments. Exact score prediction is unnecessary; explaining why a passage meets or misses a requirement is more useful. It practices self-monitoring needed without constant assistant access.

For differing self-assessment and feedback, return to rubric and source. Students can ask substantive questions and explain disagreement. The tutor should not end discussion by repeating its score. Provide human review for difficult cases. AI remains a supporting partner and criteria a shared analytical foundation applicable beyond one interface.

End with the student's own rule for the next attempt: check a condition, explain causation, or give an example. ‘Speak better’ lacks that actionable basis and makes progress checking harder.

End the Pilot with a Decision About Next Practice

Record working question types, oversight needs, and unresolved difficulties. Do not immediately make the tutor a universal examiner. A limited scenario with verified foundations, clear criteria, and an independent final attempt can already support a course. Its value is a specific learning action and an understandable improvement route.

When the WeStudy LMS platform organizes the path, keep instruction dates, subject materials, and rubrics beside practice. Repeat decisive control examples after changes. Well-designed practice helps students build more accurate answers, handle new follow-ups, and explain decision boundaries. Those justify the next learning step; smooth dialogue and approving AI scores remain only aspects of tool experience.

This will make the next attempt more focused.

Sources

• Carnegie Mellon University: Creating and Using Rubrics.

• Carnegie Mellon University: Learning Principles.

• UNESCO: Guidance for generative AI in education and research.

• NIST: Generative Artificial Intelligence Profile.

• W3C: Web Content Accessibility Guidelines 2.2.