1. Inspera Help Center
  2. Deliver
  3. Assessment types and settings

Multiple attempts

Updated
Inspera Assessment
Important: This feature is currently in Open Beta.

Multiple attempts can only be activated on test accounts, not production. If you do not have a test account, please contact your account manager to find out more.

This is available for Planners and Graders.

Multiple Attempts is a feature that allows candidates to retake the same assessment multiple times. This feature helps candidates to learn and understand the topics better by giving them an opportunity to practice until they get it right. As a result, candidates can improve their understanding, confidence, and overall academic performance.

This feature is also useful for educators and institutions looking to provide students with a more dynamic and effective learning experience.

In nutshell, with Multiple Attempts, candidates can:

  • Improve their understanding of the topic by practicing multiple times
  • Increase their confidence by identifying and correcting mistakes
  • Enhance their academic performance by getting better grades
  • Prepare more effectively for exams by identifying areas that require more focus

Content

Scope

The multiple attempts closed beta would allow the candidates to practice, learn and master a topic at their own pace. They can take the time to understand the subject matter, identify areas that require more attention, and practice until they get it right.

  1. Content for test: The test would have only fully automatically marked questions/ question sets, including Question Type - Numerical Simulation, with or without Predefined feedback per question and per alternative. 
  2. Configuration of test: The planner decides the max number of attempts that can be taken on the test, and the grading logic for the overall evaluation. 
  3. Candidate experience: The candidates have full flexibility to give as many attempts as they like - up to the max configured - within the time window of the test.

Note: The test settings for each attempt taken on the test are be the same. 

Planner

The planner is able to set up a test with multiple attempts and specify: 

  • Maximum number of attempts allowed.
  • The final grading logic for overall evaluation.

The planner is able to view the list of attempts on the test summary page with the following:

  • attempt name with hyperlink to test summary page of that attempt. 
  • status
  • start date
  • end date
  • duration
  • “more” with call to action “Go to Grade”

Note: 

  • The planner should only use fully automatically marked questions/ question sets, with or without predefined feedback.  
  • The admin can change max # of attempts on test until the test end time (TET) by deactivating the test and increasing the max # of attempts, (or) decreasing the max # of attempts provided it is higher than the # of attempts taken by any candidate by then. The deactivation cannot happen after the first submission/ attempt on the test by any candidate has been made. 
  • The 1st attempt is generated by default but ongoing attempts are generated on the go (and not upfront) as and when requested by any candidate. 

Template

The planner should ideally use the following test settings (mandatory ones are in bold) to ensure best candidate experience: 

  • Test name
  • Question Set
  • Unit (optional)
  • Template (optional)
  • Test start time
  • Test end time
  • Duration (optional)
  • Enable Multiple Attempts
    • Maximum number of Attempts
    • Setting final result
  • Add candidates
  • General settings > Marking 2.0 
  • After test > Post-submission review, Final Results (optional)
  • Grading workflow (optional*)
  • Explanation of grades (optional)

*If needed, you can configure the grading workflow to suit your preferences, but keep in mind that the multiple-attempt test can still function without grading. The final score will be calculated by aggregating scores from each attempt, utilizing the chosen grading logic. 

Configuring Multiple Attempts

  1. Within the Assessment Setup, click Enable multiple attempts.
  2. Set a maximum number of attempts in the text field.
  3. Under Setting final result, choose between the following options:
    • Highest: The highest score achieved among all attempts will be considered the final result.
    • Average: The average score obtained across all attempts will be calculated and used as the final result.
    • Latest: The most recent score from the candidate's attempts will be considered the final result.
  4. Add Candidates
  5. Under General settings, enable Marking 2.0.
    • Note: Candidate report can be enabled, but the report will not contain information about the individual attempts of the test. 
  6. Under After test settings, enable Post submission review, Final results, or Results page according to your specific needs for the test. These settings will apply to the individual attempts the same way as they do for regular tests.
    • Note: Results page cannot be changed once the test is activated.
  7. Under Assessment settings, you can use a Grading scale if you want to provide a grade for the overall test based on overall score. This works the same way as for regular tests, and Thresholds will be used as the basis for calculating a grade.

pasted image 0.png

 

 

Candidate Dashboard

Candidate experience

In this section, we will cover the candidate's experience in the following situations:

Before Attempt #1

 

 

After Attempt #1

The candidate can see the upcoming attempt in Upcoming test with these details: 

  1. The parent test name
  2. The view more details that shows the attempt details for attempts taken so far
  3. The new attempt #
  4. The start time and end time of the attempt
  5. The final score logic that is being used to calculate final grade
  6. The number of attempts submitted, and # of attempts remaining
  7. The preliminary final score so far based on the number of attempts taken and final score logic

 

After All Attempts or Test End Time 

The candidate can see the previous attempts (grouped) with the following information in Previous tests:

  1. The parent test name
  2. The view more details that shows the attempt details for attempts taken so far
  3. The start time and end time of the attempt
  4. The number of attempts submitted, and # of attempts remaining
  5. The final score logic that is being used to calculate final grade
  6. The final score is based on the number of attempts taken and the final score logic

On clicking View more details, the candidate can access the following attempt details for the attempts taken so far: 

  1. The attempt #
  2. The time of submission
  3. The score 
  4. The post-submission review with predefined feedback

Candidate dashboard with Marks only

 

Candidate dashboard with grade assigned

 

Grader

You can grade Multiple attempts by assigning a grade, or with marks only. When using a test with Multiple attempts, a Planner may prefer a total mark over a grade because total marks take into account all attempts and provides a more granular assessment of the candidate’s performance, allowing for a better understanding of their strengths and weaknesses. Graders can gain an overall insight into the candidate’s total marks by downloading the following information:

Grade Multiple attempts

  1. Within the Marking workspace, navigate to Your grade.
  2. Assign a grade.
  3. Click Confirm.

Download all data for multiple attempts as CSV

  1. Options > Download all data for multiple attempts as CSV
  2. Open Excel file from downloads.

The download of CSV is not limited to after grading, and it can be downloaded at any time with the latest information available at that point in time. 

The Excel file has the following columns:

  • Candidate ID
  • Test name
  • Attempt number
  • Attempt start time
  • Attempt end time
  • Total score
  • Grade

Download final data for multiple attempts as CSV

  1. Options > Download final data for multiple attempts as CSV
  2. Open Excel file from downloads.

The download of CSV is not limited to after grading, and it can be downloaded at any time with the latest information available at that point in time. 

The Excel file has the following columns:

  • Candidate ID
  • Test name
  • First/last attempt start times
  • First/last attempt end times
  • Total number of attempts taken by the candidate
  • Final score
  • Final grade

How MA works with Numerical simulation 

In this example, we have the following variables:

  • Group_1: Generates a random number between 1 and 9, with a step size of 1.
  • Group_2: The candidate input

Randomizing the variable can be useful in tandem with Multiple attempts because it can help to reduce the impact of guessing on the final score. Multiple attempts allow us to combine randomization with the benefit of automatic marks. 

Candidate experience

Attempt 1, where ta1 randomly generates a value of 9.0

 

Attempt 2, where ta1 randomly generates a value of 3.0

 

Attempt 3, where ta1 randomly generates a value of 5.0

 

A question is marked with Correct in View submissions

How MA works with Duration

Multiple Attempts now support timed attempts, which means that every attempt in the test will have a duration limit. This can be enabled using the existing Duration feature in test settings.

Known limitations

  1. The test is solving the use case to drive test driven learning for fully automatically marked questions, where candidates can receive predefined feedback and scores immediately after each submission/attempt without the intervention of a grader. Therefore, this feature therefore does not work with manually marked questions. If a question set with one or more manually marked questions is used, the user will get the following error - “Please make sure the question set has only fully automatically marked questions for this Multiple Attempt test”.



  2. MA does not work with the Assessment path, meaning we cannot use a Multiple Attempt test within an Assessment path. 
  3. The Multiple Attempt test works with Dynamic Maths question type, and we will improve handling of a large number of simultaneous candidates. Currently, if many candidates start the test at once, they can experience issues starting the test or slowness during the test.
  4. An MA test has a maximum number of attempts that a candidate can take. The settings for the test, including that for all its attempts, are kept the same by default, for the feature to work as intended. Any changes we make to the MA test (not just start/end times, but any other general test settings) will have to be manually changed in the "already created" attempts. If a new attempt is created after some changes to the MA test, the new attempt will have the updated settings. In any other scenario, we advise not to modify the settings for any of the individual attempts for the functionality to work as expected. We advise making sure you have the correct settings on an MA test before you activate it.

  5. A Multiple Attempts test will have the individual attempts listed as standalone tests in Deliver and Grading UI. It is advised to ignore these references to attempt level tests. 

Snapshot:

  1. In some instances when the candidate starts an attempt from the dashboard, where they are directed to this error page instead of the test player. If this happens, we advise the candidate to go navigate back and try taking the attempt again by clicking Start Test after a few seconds (ref snapshots below). We are fixing this issue before the next release.




  2. There are no validations for test start/end times as of now. Our assumption is that validations are not required, and we can keep it flexible till the time we discover otherwise based on customer feedback. 

Articles in this section

See more
Was this article helpful?
1 out of 1 found this helpful

Comments

0 comments

Article is closed for comments.