There are two ways to automatically update skill levels based on question results: SOP questions and confirmation tests in Training (Courses). In both cases, records show who answered which question incorrectly. Because each method has strengths and weaknesses depending on what you want to verify, this article explains when to use each method and how to combine them.

Available in Core / Pro / Enterprise plans. For information about the automatic update mechanism and settings, see Automatically update skill levels when courses complete, certifications are earned, or SOP records improve.

Differences between the two methods

Guidelines for choosing which to use

When SOP questions are suitable

• When you want to determine if a worker is ready for solo completion
• When you want to return to the right way at the point of error, ensuring work does not proceed while incorrect
• When you want to know which step in the work has weak areas
• When your operation involves learning work while viewing SOPs on site, such as OJT

When confirmation tests are suitable

• When you want to verify knowledge that should be remembered, such as work reasons, quality standards, and safety rules
• When you want respondents to answer through to the end even if wrong, and decide pass/fail by score
• When you want to keep records of taking the course and pass/fail as a wrap-up to new employee training or periodic training
• When you want to know weak areas by which knowledge questions are weak

Combining the two methods

You can set both methods for the same skill at the same time. Because the Target level is set separately for each, you can use them in stages.

For example, you can set it so that passing the confirmation test brings the level up to the "understanding" stage, and completing related SOPs as solo completion brings the level up to the "can perform alone" stage. Weak knowledge areas can be identified from incorrect answers on the confirmation test, and weak work areas can be identified from Stumbles in SOP questions.

Automatic updates only work to increase levels, so whichever condition is met first, the higher level remains.

Configuration flow

Evaluate with SOP questions

1. Set questions for each step in the SOP and configure the correct answer (Step-level questions settings).
2. In the skill definition edit [Assignment / progress] tab, link the SOP as a Related procedures and select [Target level on completing all].
3. [Training] → [Common settings] → [Automatic evaluation] → in "Raise from related SOP records", select the condition to mark as completed. The default is "Only when completed as Solo completion". In the same section, [Times needed for solo completion] lets you choose how many times solo completion counts (default is Once is enough).

Evaluate with confirmation tests

1. Add SOPs to a course, attach a confirmation test, and specify the Pass threshold.
2. In the course [Skills gained], select the skill and Target level.
3. Assign the course to the target audience.

For information on how to create courses, see Training feature: how to create and assign courses, how to view progress status.

Check where mistakes were made

For SOP questions

To check one person at a time, follow these steps.

1. [Training] → open the target person's skill page and click the target skill.
2. In the [Development records] tab, under "Related procedures viewing records and Completion report status", click the display in the [Solo completion] column.
3. The Stumble map opens, showing which Checkpoint the person stumbled on and which choice was selected incorrectly.
4. Pressing [Viewing records] on the same row opens that person's filtered SOP viewing records (Work records and Stumble map).

When checking by SOP with the Entire team, use the [Stumble map] in that SOP's viewing records (open from the […] menu in the Step list with [Viewing records], or from the SOP name in the Usage Report). You can see which Checkpoint, how many people, and which choice they stumbled on, and identify who stumbled (Check viewing records by SOP).

For confirmation tests

The Attempt history records each attempt's responses, so you can see who answered which problem and which choice was selected. Attempt history is accessible from the course's "Assignment / progress" and "Course analytics", and from the [Training] tab on the Personal page.

Frequently asked questions

With SOP questions, can I let people continue to the end even if they answer incorrectly?

Questions with correct answers set cannot proceed to the next step until correct. If you turn on [Continue when unexpected], it will not stop and will continue, but the record becomes Normal / Abnormal rather than correct / incorrect. Even if you answer "Abnormal", completion status does not change, so whether there are errors is not reflected in the skill level. If you want respondents to answer through to the end even if incorrect, use a confirmation test.

With SOP questions, can I ask about knowledge?

Yes. When having AI create questions, set the Question type to "Comprehension test" to create knowledge-based questions. However, the behavior is the same as other questions—cannot proceed to the next step until correct, and there is no score or pass/fail. If you want to decide pass/fail by score, use a confirmation test.

Does a person's skill level go down if they make a mistake?

No. Automatic updates only work to increase levels. After someone reaches solo completion, if they make a mistake later, a "Needs re-check" mark appears (Automatically update skill levels when courses complete, certifications are earned, or SOP records improve "Needs re-check mark").

Are changes reflected for people who worked through the SOP before settings were configured, or who already passed the confirmation test?

Yes. Changes are reflected at the moment you save the link or Target level—the level is updated for people who already meet the condition. However, for people who set the skill level after their record (added, changed, or reset to unevaluated; Excel imports are excluded), the level is not changed and only the development record remains.

Related

• Automatically update skill levels when courses complete, certifications are earned, or SOP records improve
• Training feature: how to create and assign courses, how to view progress status
• Step-level questions settings
• Check implementation results in the Usage Report