Introduction

Numerical Simulation is a programmatic, automatically-marked question type where questions are answered as numeric values (for equation responses, see Symbolic Math). By setting variables, their parameters, and equations within the Program Model of the question, values within the question and their answers will be randomised each time the question is generated for a candidate. The allocation of values to the Program Variables happens when the test is started and in this way, each test can have unique Program Variables.

Numerical Simulation utilizes a Program Model that enables simulations using an expressive programming language of Maxima which is based on Lisp.

The “Pump Water” example used in this document is shared with permission from the creator - the UNSW School of Banking and Finance, University of New South Wales, Sydney, Australia.

Click to Download the QTI package of the question shown in the video

Content

• How to Author
  • Create New Question
  • The Question Text
  • The Program Model
    • Error Carry Forward
  • The Question Text: Assign Program Variables
  • The Response Outcome
    • Evaluation Options
  • Response-Specific Feedback
  • Save and Preview
  • Other Options
• How to Preview a Numerical Simulation
• How to Mark a Numerical Simulation
• Override an automatically marked question

How to Author

There are 3 core elements when authoring a Numeric Simulation question: the Question Text; the Program Model; and the Response Outcomes.

• The Question Text are the fixed elements of the question. The only parts that change are the values of the program variables you insert within the question text.
• The Program Model: this is where all the variables, their parameters, and calculations are set.
• The Response Outcome: this is where you decide how the candidate’s response will be evaluated.

Create New Question

1. Select the Author module > Questions.
2. Click Create new.
3. Under Automatically marked, click Numerical Simulation.

The Question Text

1. Author the question text in the main authoring area and wherever you’d like a variable to appear, click + Insert followed by Program Variable. This will insert a variable placeholder.
2. Where you’d like a candidate input field for the candidate to respond, click + Insert followed by Candidate Response. This will insert a response placeholder.
   

The Program Model

To set all the variables and their parameters, you must edit the Program Model. The Program Model uses MAXIMA for its operations. In MAXIMA, variables are declared by first writing the variable name, followed by a colon. After the colon, you specify the value or parameters for the variable. Remember, each line of code must end with a semicolon to be correctly interpreted.

For a step-by-step example of a question being authored, see the video at the top of this article.

For further detail on the Program Model syntax, see The Program Model.

1. From the right menu, click Program Model > Edit Program Model
2. Write the Program Model and when finished, click ‘Compile’ to verify your syntax and ensure the example values are being generated as expected. Click Save.

Error Carry Forward

Symbolic Math has Error Carry Forward functionality. This works the same way as it does for Numerical Simulation. See Error Carry Forward for further information.

The Question Text: Assign Program Variables

With the program model complete, return to the question text and click on each Program Variable placeholder you inserted. From the Program Variables dropdown, select which variable you’d like to display.

The Response Outcome

To set the type of input you expect from the candidate, as well as the different ways you’d like to validate them, you must set the Response Outcome for each Candidate Response.

1. Click on each Candidate Response field and select Edit response outcome
2. Select how you'd like it to be evaluated (see below for detail).
   
   • Equal
   • Abs. Tolerance
   • Rel. Tolerance
   • Unequal
   • Greater than
   • Less than
3. Select the variable to be evaluated against
4. Select the outcome
   • Correct
   • Partially Correct
   • Incorrect

Evaluation Options

Equal

The candidate input is exactly the value of the variable as set-out in the programme model.

Abs Tolerance

The candidate input can be between a set value lower and higher than the variable as set-out in the programme model.

See Absolute and Relative tolerance for more information.

Rel Tolerance

The candidate’s input can be between a higher and lower range relative to a percentage of variable as set-out in the program model

See Absolute and Relative tolerance for more information.

Unequal Than

The candidate input is any value other than the value of the variable as set-out in the programme model

Greater Than

The candidate input is greater than the value of the variable as set-out in the programme model

Less Than

The candidate input is less than the value of the variable as set-out in the programme model

Response-Specific Feedback

You can set response-specific feedback for the candidate. See Response Feedback for further information.

Save and Preview

Once you've finished setting the Question Text, Program Model, the Response Outcomes, simply click Save. If you'd like to preview the question, click the Preview eye icon in the top right.

As an author, each time you refresh your browser the variables will be recalculated. This is not functionality available to candidates. For candidates, once variables have been calculated, they remain fixed for that attempt.

Other Options

You can further customise the response outcome by visiting the Options menu.

1. Select the Options drop-down.
2. From here, you have the following options for input validation which restricts what the candidate can type:
   • Expected length
   • Expand input field automatically
   • Restrict number of characters to input width

Note: The formatting of the Program Variable is not affected by the input field setting. It must for now be defined in the Program Model.

How to Preview a Numerical Simulation

You can preview your question during the authoring process to see what it looks like for the candidate.

1. Within the Author editor, in the upper right-hand corner, click Preview.
   • Program variables are displayed with their numeric value in the preview pane.
2. Enter an answer within the Candidate input field and click Check answer. To enter a different Candidate input, click Try again. 
   • Wrong answers are given the message Wrong.
   • Correct answers are given the message Correct.

While you are previewing the question, please make sure that the decimal separator is behaving as you want. 

The default behaviour of decimal separators is that both comma and point are allowed, but comma will be transformed to a point when the response is stored. Candidates will see the decimal separator as a point when they navigate back to a question they have already answered, or when they view their submission after the test. 

If your organisation wants to enforce decimal point instead of comma as a decimal separator in this question type, please contact the Service Desk for activation.

Please be aware of the following when enforcing decimal point as decimal separator:

• Enabling this will also add specific information about decimal points to both Authors and Candidates

• This will also apply to the Numeric Entry question type

How to Mark a Numerical Simulation

Numerical Simulation is an automatically marked question. Automatically marked questions are automatically marked by Inspera Assessment once the candidate has submitted the test. As the Grader, you do not need to mark an Automatically marked question, unless you intend to override the mark. Automatically marked questions can be found within the Marks dropdown, and are identified with Automatically calculated. 

Numerical simulation question types that contain Error carry forward are rewarded with partial marks.

Override an automatically marked question

As a Planner/Grader, you can override the automatically calculated marks. 

1. Within the Marks dropdown, Click Override.
2. Within the text field, enter your marks (up to 2 decimal places).
3. Click Apply.

Once you have overridden automatically calculated marks, the status of the question changes to Overridden. 

Known limitations:

• Feedback per response outcome is not displayed in Candidate report
• Correct answer not included in downloaded question set PDF