Introduction
Multiple Attempts is a test that can be taken multiple times by the student on automatically or manually marked questions.
Additionally, it supports performance-based progression, where the grader can decide if a given candidate should be given the next attempt based on their performance on the previous one.
Note:
- This feature is not compatible with any Lockdown Browsers such as Inspera Integrity Browser (IIB) and Safe Exam Browser (SEB).
- The multiple attempts feature is not supported when using LTI integration/LMS.
How to set up a Multiple Attempts test
Note: This feature is now generally available for all customers to use. It is available on demand and you can reach out to the service desk to get it activated.
Key Decisions
On enabling MA, there are 3 core decisions you have to take when configuring a Multiple Attempts Test.
-
Choose method of attempt creation: this is how the administrative user would like to schedule the attempts.
- System automatically creates attempts - The system generates attempts automatically with the same time window as that of the overall attempt. This is common workflow in case of assessments or quizzes with automatically marked questions only, where a grader is not required, the attempts could all have the same time window as that of the overall assessment.
- I want to specify attempt time windows - The planner has the flexibility to schedule attempts as they like. This is common workflow in case of assessments with manually marked questions, where grader’s manual intervention is required, the attempts are scheduled sequentially, allowing graders to manage their workflow effectively.
-
Choose how attempts are available to candidates- This decides who has control in the decision of taking the next attempt.
- Candidate Control - It means that the candidate can decide when to give any attempt they have been enrolled on, within the time window of each attempt.
- Grader Control - It is a grader that decides if a given candidate should get the following attempt based on their performance on the previous attempt.
-
Select the final results criteria - The way you would like the overall score to be calculated based on the scores for each attempt taken. The options you have are,
- Highest - The overall score is the highest score from all attempts
- Average - The overall score is the average score from all attempts.
- Latest - The latest score is the latest score from all attempts.
Common Workflows
Before we move on to each of these settings, it would be worth highlighting the most common workflows used by our customers.
Workflows |
Question Type |
The method of attempt creation | Availability of attempts | |
1 | Automatically Marked Multiple Attempts |
Automatically Marked ~ Multiple Choice |
Automatically create attempts.
Recommended way of scheduling: Attempts in the same overall time window (parallel) |
Candidate control |
2 | Manually Marked Multiple Attempts |
Manually Marked ~ Essay |
I want to specify time windows. Recommended way of scheduling: Attempts in sequential time windows (sequential) |
Candidate control |
3 |
Manually Marked Multiple Attempts + Performance based Progression |
Manually Marked ~ Essay |
I want to specify time windows. Recommended way of scheduling: Attempts in sequential time windows (sequential) |
Grader control |
Default Setup
The default setup when you enable Multiple Attempts is shown below.
Choose Method of Creation of Attempts
- If “system automatically creates attempts” is selected (default), you need to specify the maximum number of attempts as,
- Unlimited (default) or
- Limited
- If “I want to specify attempt time windows” is selected, the planner has the flexibility to create the attempts with their respective time windows using the “add attempts” widget.
- Click on “Add Attempt” to open the scheduling modal.
- Specify the Attempt start and end time on the scheduling modal.
- Click on “Create Attempt” on modal to create the attempt
You can add as many attempts as required,
Note:
- The attempts can be added before or after the activation of the test.
- The attempt time windows have to be within the overall assessment’s time window.
- The start and end times of an attempt can be modified from the scheduling widget itself.
Choose Method of Creation of Attempts
- If “Candidate control” is selected (default), the candidates can decide if/ when they want to appear for an attempt. The candidates will also be aware of the maximum number of attempts available to them on the assessment.
- If “Grader control” is selected, the grader has to decide if the candidate gets the following attempt based on their performance on the previous attempt. The candidates will have an attempt available at a time, and will be made aware if they have a subsequent attempt available or not based on grader’s evaluation. Note, the Grader control works
Note:
- Grader control is supported only for "I want to specify attempt time windows" method of attempt creation. If you want "system automatically create attempts", please choose Candidate control.
Choose Final Results Criteria
- Highest - The overall score is the highest score from all attempts.
- Average - The overall score is the average score from all attempts.
- Latest - The latest score is the latest score from all attempts.
Note:
- The above information is also displayed to candidates in their dashboard.
- The logic applies on marks, and not on grade. The overall score is calculated based on this logic, and if required, an overall grade can be provided based on the overall score. In case you don't use marks in your institution, then grader could grade each of the attempts, and grade the overall assessment as well without using marks.
The General test settings
There are specific settings supported with MA.
- General settings
- Marking 2.0 (Mandatory)
- Candidate Report (Sep Release)
- Exam Day
- Lost connection alert
- After test
- Post-submission review
- Feedback
- Final results
- Grading workflow
Grading: How is a Multiple Attempts test graded
The grader can evaluate each attempt, and the overall assessment based on performance across attempts. The overall assessment and each of the attempts have their own respective grading workspaces.
- For each attempt, the grader can do marking and/or grading along with the feedback using annotations and explanations. This encourages and allows the candidate to perform better on the following attempt.
- For overall assessment, the overall marks are calculated based on the attempt level marks and the final grading logic which could be highest, average or latest. If the attempts are not marked and only graded, the overall assessment can be graded at grader’s discretion.
Grading at Attempt
The grader can access the grading workspace for a given attempt by going to attempt level test and opening it in grade by clicking on "Open in Grade".
The grader can assign marks.
The grader can provide feedback.
If Performance based progression is enabled: If “Grader control” is enabled, the grader has the possibility to flag a candidate for the next attempt. All the candidates flagged for the next attempt will be enrolled on the next attempt at the time of confirmation by the grader. The grader can take this decision by checking the "Permit new attempt" for the candidates.
Grading at Parent
Once the attempts are marked, the grader can choose to provide an overall grade to the candidates. If they go to the overall multiple test's grading workspace, they can see the total marks for the overall test calculated based on the final results criteria in the test settings.
The grader can also view the details for each attempt for a given candidate by expanding the view. They can see marks and grades if available for each attempt taken by candidate.
In case performance based progression ("Grader control") is enabled in test settings they can also see the decision taken at each attempt.
They can provide an overall grade to the candidates.
What kind of information is available to administrative users?
Download Multiple Attempts as CSV
Planners can gain an overall insight into the candidate’s total marks by downloading the following information:
Download all data for multiple attempts as CSV
- Options > Download all data for multiple attempts as CSV
- Open Excel file from downloads.
The download of CSV is not limited to after grading, and it can be downloaded at any time with the latest information available at that point in time.
The Excel file has the following columns:
- Candidate ID
- Test name
- Attempt number
- Attempt start time
- Attempt end time
- Total score
- Grade
Download final data for multiple attempts as CSV
- Options > Download final data for multiple attempts as CSV
- Open Excel file from downloads.
The download of CSV is not limited to after grading, and it can be downloaded at any time with the latest information available at that point in time.
The Excel file has the following columns:
-
Candidate ID
-
Test name
-
First/last attempt start times
-
First/last attempt end times
-
Total number of attempts taken by the candidate
-
Final score
-
Final grade
Note: The first attempt is generated by default, and ongoing attempts are generated as requested by candidates.
Candidate Dashboard: How is a Multiple Attempts test taken by candidate?
In this section, we will cover the candidate's experience in the following situations:
The candidate can start giving their first attempt from their candidate dashboard. They would see the overall test window before they try to attempt the test, and the attempt's time window when they have decided to attempt the test.
If Performance based progression is enabled: If "Grader control" is enabled, they are notified that they have made the submission and are being evaluated.
The candidates will see when a new attempt is available or if they dont have any more attempts to take.
Known Limitations
1. Limitation related to the start or end times of a multiple attempts test.
a. When extending the end time for a multiple attempt test, any existing test attempts that were created prior to the end time change will not automatically reflect the new end time. These existing attempts will retain the original end time and may appear as expired or unavailable to students, even if the overall test period has been extended.
Recommendation:
To resolve this issue, you need to manually update the end time for each existing attempt. Any new attempts created after the end time extension will automatically inherit the new end time.
- Navigate to the Admin Panel.
- Locate the test and select the student’s existing attempt.
- Manually adjust the end time to match the new extended end time.
Note: This logic applies to any test setting on the overall multiple attempts test.
b. When creating a parent test and adding a child test, be aware of this limitation:
No warning or error will appear if you change the parent test's start or end time to fall outside the child test's time. This happens if:
- The parent test's start time is set after the child test's start time.
- The parent test's end time is set before the child test's end time.
Recommendation: Always ensure that the parent test's time frame fully covers the child test's start and end times before saving.