Advanced method for question scoring

  • Updated

This method is only for Automatically Marked Questions.

Inspera Assessment provides advanced methods for authors to achieve highly granular control over automatically marked questions. 

While using the same core scoring tools, this article outlines the advanced scoring rules available, clarifies how settings apply to question elements versus the whole question, details specific limitations, and explains lower and upper mark limits per question element.

Understanding advanced scoring rules

Advanced scoring rules support various methods to calculate marks for automatically marked questions. These methods provide fine-tuned control over how scores are awarded, enabling nuanced calculations beyond basic correct/incorrect criteria.

The advanced scoring rules support:

  • Marks per correct answer (introduced in the Basic method)
  • Marks per wrong answer (introduced in the Basic method)
  • Marks for unanswered
  • Marks for all correct
  • Minimum marks

All settings for Marks per correct answer, Marks per wrong answer and Marks for unanswered must have a value. Conversely, Marks for all correct and Minimum marks do not require a value, and should be left empty if you don't intend to use them in the scoring rules.

Scoring settings: Element or question

When several question elements are used within a single question (like in a Composite question type), it's important to understand which scoring settings apply to individual elements and which apply to the complete question.

Applies to each question element: 

  • Marks per correct answer
  • Marks per wrong answer

Applies to the question as a whole: 

  • Marks for unanswered
  • Marks for all correct
  • Minimum marks

In the following image, you see an example of several question elements in the same question: 

Example of several question elements in the same question.

Core advanced scoring methods

Let's look at the core advanced scoring methods in detail:

  •  
    • These marks are awarded if all question elements within a question are left unanswered.
    • If one question element is answered, the remaining unanswered ones will be marked as wrong answers. with a specific exception for multiple response questions (see Known limitations below).
    • This setting must have a value, and the default is 0.
  •  
    • These are the marks awarded if the candidate gets all answers correct.
    • These are not bonus points - they do not stack with the max achievable marks.
    • Not reccomended: If this is set to a lower value than maximum achievable marks, the result is that the maximum achievable marks is limited to the same amount of marks set in the Marks for all correct field.
    • This setting does not require a value and is empty by default. When it is left empty, the marks for all correct will be defined by maximum achievable marks as set in marks per correct answer.
  •  
    • If this is not set, the lowest achievable score will be defined by the settings on the options above.
    • If this is set to 0, the question will never award negative marks.
    • This setting doesn't require a value and is empty by default.

Lower and upper mark limit per question element

Individual question elements can have their achievable marks further limited. This can be set up in Options per question element. Using this will override the marks per correct answer and marks per wrong answer set at the general question level. If you don't want to use this, please leave the setting empty; any other value will be part of the scoring rules.

To access these options: Click on the question element to get the Option menu available on the right.

  • You can use this to override marks per correct answer. If this is set to a lower value than marks per correct answer, the upper mark limit will override marks per correct answer.
    • Example: A question consists of four question elements, each with one correct answer. Marks per correct answer is set to 2, and the question can award a maximum of 8 marks. Setting the upper mark limit on one of these elements to 1.5 changes the maximum to 7.5 achievable marks.
    • Caution: Setting the upper mark limit to 0 means that the question element cannot award positive marks at all.
  • You can use this to override marks per wrong answer. If this is set to a value that is higher than marks per wrong answer, the lower mark limit will override marks per wrong answer.
    • If the question in the general scoring rule is set to award negative marks per wrong answer and the lower mark limit is set to 0, the element will not award negative marks.
    • Setting a lower mark limit is useful in those cases where you want to limit the impact of one wrong answer compared to the other wrong answers.
    • Note: If the value of minimum marks set on the question is higher than the sum of all lower mark limits, the minimum marks setting will override the combined lower mark limits.

Complex scoring scenarios

Consider a multiple response question with two question elements that both have two correct and two wrong answers:

  • Marks per correct answer: 2
  • Marks per wrong answer: -2
  • Lower mark limit on each question element: -3

Let's look at a scenario: The candidate gets all correct on question element 1 and all wrong on element 2.

  • The raw score would be 4 marks for element 1 (all correct) and -4 for element 2 (all wrong).
  • The result without a lower mark limit would be 0 of 8 marks.
  • However, because we have a lower mark limit of -3 on both elements, the final result will be 1 of 8 marks (element 1 contributes 4, element 2 is capped at -3. 4 - 3 = 1).
  • If the candidate answers all wrong on both elements, the result will be -6 of 8 because of the lower mark limit (capped at -3 per element).
  • If you remove the lower mark limit, answering all wrong would result in -8 of 8.

Any minimum marks set on the question (at the question as a whole level) will override all of the above element-specific limits. For example, setting minimum marks to 7 for the whole question will award 7 of 8 marks on every possible combination of answers, except for all correct, which will still award 8 of 8.

General tips for setting up advanced scoring

  • Always answer the question in preview mode to verify that marks are awarded as intended.
  • Important Note: It is not possible to change the scoring after the test has started. Please make sure that marks are awarded as intended before the test begins.
  • Upper and lower mark limit per question element is easy to overlook. Make sure that they are empty if they are not to be used, as any other value will be part of the scoring rules.

Known limitations

The following limitations apply only to Multiple Choice and Multiple Response questions when they are used as elements within a Composite question, as these two question types have different scoring logic when left unanswered in a composite.

Example Scenario: If you create a composite question where Marks per wrong answer is set to -1, and Marks for unanswered is set to 0, the following will occur:

  • If the composite question is attempted and a Multiple Choice question element is left unanswered, the scoring gives -1 per unanswered question element.
  • If the composite question is attempted but a Multiple Response question element is left unanswered, the scoring gives 0 per unanswered question element.

Was this article helpful?

10 out of 35 found this helpful