Description
Guess correction (or: guess correction/ guess correction) is correcting the score of candidates for the possibility that candidates have guessed answers.
Guess correction can be set at test level in the settings of the blueprint. Select 'yes' under 'Apply guess correction'. Remindo will calculate the board score for the entire test based on the closed questions in the test. This is because the board score can only be calculated for closed question types; questions where the correct answer can be guessed. The correct answer is always located in one of the answers or answer combinations. For open questions, such as the open-ended question, worksheet question or fill-in-the-blank question, no guessing is possible, therefore no guessing correction is applied for these questions.
Attention!
The guess correction cannot be calculated in advance in the administration environment. When setting up the test matrix, it is not yet known which questions will be included in the test, so Remindo does not 'know' at that moment what the guessing chance correction will be.
Tip!
It is possible in the test matrix settings to generate a sample score table where you can enter the number of points and the expected guess probability to estimate the scores and the effect of the guess correction.
Guess probability and guess score
Guess probability
The guessing probability indicates the probability that the candidate will answer a question correctly by answering randomly. When a question has 4 answer options of which one is correct, there are 4 possibilities to answer the question and the probability of guessing the correct answer is 1/4, or 0.25. When two answer options out of the four are correct (multiple correct), there are 10 possibilities to answer the question, but only 1 answer combination is correct. The guess probability is then 1/10, or 0.10.
Guess score
The guess score indicates the score subtracted from the total score of the question when the guess correction is applied in a test. For all questions, the guess score consists of the sum (points to be gained per answer option * the guess probability). The points to be gained per answer option depend on the score type and the total number of points to be gained on the question. When the point total of an answer (combination) is negative (this is possible with score type 'Score distribution' where negative points can be scored) it is truncated to 0, so a guess score does not become negative.
Impact of applying the guess correction
The total guess score of the test (sum of all the guess scores of the questions in the test) is factored into the norming. Applying the gues correction shifts the caesura point and requires more points to achieve a passing score.
The graph below shows the norming of a test consisting of ten 4-choice questions of 1 point each. The original caesura was set at 55% for the grade of 5.5. Each question has a guess score of 0.25 (chance) x 1 (point). The total guess score is 2.5 points out of a total of 10 points to be obtained (25%).
The location of each point on the line is corrected using the formula: (total guess probability test) + ((100% - total guess probability test) × [percentage]) = 25% + (75% × [percentage]). That means the caesura becomes 66.25%: 25% + ((100%-25%) x 55% = 66.25%. This formula can be found on the Results page > in the left menu, choose the button 'Test' > Result calculation > Show detailed explanation of the calculation.
Let op!
When the board probability correction is applied, it affects the caesura point. This is only visible after the test and is explained in the Result calculation. Go through the details of the result to the menu on the left, click on Test and click on 'Calculation of result' under the result. Then you can see an extended calculation of the result under 'Show extended explanation of calculation'.
Application by question type
Multiple choice questions
One correct answer / Dropdown question
With a single correct answer, the probability depends only on the number of answer options. With four answer options and one correct option, the guessing probability is ¼, or 0.25. When the scoring type correct answer or score distribution is chosen with a total score on the question of 1 point, the guess score is the sum of the points obtainable per answer option x the guess probability 0.25.
Multiple response question
This refers to the multiple response question. With multiple correct answers, the guessing probability and score is calculated by taking into account all combinations that can be made. The probability of guessing with this question type is a lot lower than when only one answer can be entered. For a question with 4 answer options of which two are correct and the choice is limited to two, there are 10 possible answer combinations.
Matching/Matrix questions
One association
This is effectively a multiple choice question with one answer, but with distractors distributed over a number of rows. Therefore, the guess probability and guess score are calculated in the same way as for a single multiple choice question. For three rows with four answer options each, the guess probability is ¼ x ¼ x ¼ = 0.015625.
One or more associations per row
This is effectively multiple multiple choice questions (possibly with one answer), in one. The guess probability and guess score per row are therefore calculated in the same way. The guess probability across all rows is multiplied; the guess score summed.
Order questions
Order questions
Sorting questions take into account the number of different sequences a candidate can complete. For a sorting question with 3 elements to be put in the correct order, there are 6 different possibilities:
- ABC (correct) / ACB / BCA / BAC / CAB / CBA
Two types of scoring can be used here: correct answer and the quotient rule (for more information on the quotient rule, see the article "Score Types").
With score type 'correct answer', points are only awarded for a completely correct sequence. When there is a total score of, say, 3 points and a guess probability of 1/6 (0.1667), the guess score is 0.5.
When the scoring type "quotient rule" is applied, Remindo takes into account how well the elements are in relation to each other. In the image below, the green color shows that the location is completely correct and orange indicates that the location is partially correct relative to the other two elements. In the answer combination BAC, C is completely correct; B and A are not counted completely wrong because they are partially correct relative to C.
For each combination chosen, calculate what the score would be. In this case, you could calculate using a distribution of 6 points for each answer combination where green is worth 2 points and orange is worth 1. With BAC's chosen order, the ratio of right to wrong is 4 to 2. The score based on the quotient rule is 1.5 points (for a total score of 3 points). The guess score will be 1.5 * (1/6) = 0.25. Calculate the guess score of the remaining answer combinations and add them together for the total guess score of the question.
Attention!
If a question requires four answers in the correct order, there are 24 possible answer combinations. The guessing probability then becomes: 1/24 = 0.0416.
Point question types
Hotspot questions
In the hotspot question, the answer area is as large as the image. Remindo divides the image into 1000 points that can be chosen. When it is set that a candidate may click more than one hotspot, the candidate has the same chance each time. A candidate gets points only once for a well-placed hotspot. When two areas are selected that overlap, Remindo will first see the area that is "on top" in the editor.
Attention!
The actual guess probability (and therefore the guess score) is always higher than the guess probability calculated by Remindo because a candidate can extract information from the image that gives direction on where the candidate will/will not place their hotspot.
Drag & Drop questions
Again, one large background image is used where the area of correct answer areas relative to the total area determines the guessing probability. When 25% of the background image is selected as the correct answer area, the guessing probability is 0.25. When there are multiple areas to choose from, the guess probability is "probability area X" x "probability area Y. Only the scoring type 'score distribution' can be chosen for this question type, so minus points can also be used. A candidate gets all the points from the selected area when at least 55% of the dragged image lies on the selected area.
Association questions
Graphic associate question
With a graphic associate question, the number of combinations of lines that can be made is the number of answer options. This question type has only the scoring type "score mapping". The guess probability and guess score are otherwise calculated in the same way as for a multiple response question.
Combined question
For combined questions, which consist of multiple interactions, the guess probability is calculated from all possible answer combinations of the sub-questions by multiplying the total score of each answer combination by the guess probability (the probability of choosing exactly the correct answer combination across all sub-questions). Then all the guess scores of the different answer combinations are summed.
When there are interactions within the combined question that can produce a negative score, the total score of the combination of answer options chosen is calculated first. When the total guess score of a combination of answer options is negative, it is truncated to 0. The score is then multiplied by the guess probability and thus the guess score of each answer combination is calculated.
I don't know option
When the "I don't know option" is used in the questions of the test, we recommend not applying the guess correction to the test because it is assumed that in case the candidate does not know the answer, the candidate won't guess, but use the "I don't know" answer option. When the guess score is applied anyway, the guess score will be calculated based on the remaining options. For a multiple-choice question with 4 answer options, this will result in a guess score of 0.25.
Disclaimer: This text was automatically translated from the Dutch version.