Note
This feature is not compatible with any Lockdown Browsers such as Inspera Integrity Browser (IIB) and Safe Exam Browser (SEB).
This is available for Planners and Graders.
Multiple Attempts is a feature that allows candidates to retake the same assessment multiple times. This feature helps candidates to learn and understand the topics better by allowing them to practice until they get it right. As a result, candidates can improve their understanding, confidence, and overall academic performance.
This feature is also useful for educators and institutions looking to provide candidates with a more dynamic and effective learning experience.
In a nutshell, with Multiple Attempts, candidates can:
- Improve their understanding of the topic by practicing multiple times
- Increase their confidence by identifying and correcting mistakes
- Enhance their academic performance by getting better grades
- Prepare more effectively for exams by identifying improvement areas
Content
- Best Practices
- Planner
- Candidate Dashboard
- Grader
- How MA works with Numerical simulation
- Post Submission Review Settings with MA
Best Practices
-
Question Set Considerations:
- The planner should only use fully automatically marked questions/ question sets, with or without predefined feedback. With predefined feedback enabled, candidates see the feedback after the test submission.
-
Numerical Simulation:
- Is supported with Multiple Attempts, allowing unique randomized Numerical Questions on every attempt.
-
Randomized Question Order:
- Is supported with Multiple Attempts to make it more dynamic and engaging by preventing reliance on memorization and promoting independent engagement with the content.
-
Setting up the test through templates:
- The planner can set up a test with multiple attempts either from scratch using “create test” or through a template. We recommend using a template as it allows you to save and manage settings specific to your workflow more effectively.
-
Maximum # of Attempts
- The admin can change max # of attempts on test until the test end time (TET) by deactivating the test and increasing the max # of attempts, (or) decreasing the max # of attempts provided it is higher than the # of attempts taken by any candidate by then. The deactivation cannot happen after the first submission/ attempt on the test by any candidate has been made.
-
Post-Submission Review (PSR):
- The PSR is optional but recommended for candidates to access submissions with or without feedback. Visibility settings can be configured for instant availability, after test end time, after a specific time, or for a specific duration.
-
Final Results and Assessment Settings
- The MA test allows for the calculation of a final score by normalizing individual attempt scores, with the option to also assign an overall grade based on the cumulative result.
-
Modifications to Multiple Attempts:
- Any modifications to the settings of an MA test, including start/end times, require manual updates in existing attempt level settings. New attempts created post-changes will adopt the updated settings. It's recommended to ensure correct MA test settings before activation.
Planner
The planner can set up a test with multiple attempts either from scratch using “create test” or through a template.
The planner can specify the following to ensure the best candidate experience:
- Test name
- Question Set
- Organisation Unit (optional)
- Template (optional)
- Test start time
- Test end time
- Duration (optional)
- Test Information (optional but recommended)
- Enable Multiple Attempts
- Maximum number of Attempts
- Setting final result
- Invitation Mode
- Add candidates
- General settings > Marking 2.0
- Exam Day > Lost connection alert (optional but recommended)
- After test
- Post-submission review (optional but recommended)
- Final Results (optional but recommended)
- Feedback (optional but recommended)
- Grading workflow > General workflow
- Assessment Settings (optional)
Note: Please note that when a user enables Multiple Attempts, all the incompatible settings will get grayed out. Also there are error validations in place to make sure activation of MA is possible only when required settings are enabled/ specified. The user will be shown information messages to make them aware of the impact of enabling Multiple Attempts.
How to Enable Multiple Attempts
- Within the Assessment Setup, click Enable Multiple Attempts.
- Set a maximum number of attempts in the text field.
- Under Setting final result, choose between the following options:
- Highest: The highest score achieved among all attempts will be considered the final result.
- Average: The average score obtained across all attempts will be calculated and used as the final result.
- Latest: The most recent score from the candidate's attempts will be considered the final result.
- To allow candidates an unlimited number of attempts, click Unlimited Attempts.
- Under General settings, click Enable Marking 2.0 on this test.
- Under “Assessment settings”, you can use a Grading scale if you want to provide a grade for the overall test based on overall score. This works the same way as for regular tests, and Thresholds will be used as the basis for calculating a grade.
Note: The MA feature cannot be disabled after an MA test has been activated/ deactivated.
Multiple Attempts enabled
Marking 2.0 enabled
Post Activation
After activating the test, the Planner can access a summary page containing the following information:
- Attempt name with hyperlink to test summary page of that attempt.
- Status
- Start date
- End date
- Duration
- “More” with call to action “Go to Grade”
Summary page after test activation
Attempts within the summary page
Download Multiple Attempts as CSV
Planners can gain an overall insight into the candidate’s total marks by downloading the following information:
Download all data for multiple attempts as CSV
- Options > Download all data for multiple attempts as CSV
- Open Excel file from downloads.
The download of CSV is not limited to after grading, and it can be downloaded at any time with the latest information available at that point in time.
The Excel file has the following columns:
- Candidate ID
- Test name
- Attempt number
- Attempt start time
- Attempt end time
- Total score
- Grade
Download final data for multiple attempts as CSV
- Options > Download final data for multiple attempts as CSV
- Open Excel file from downloads.
The download of CSV is not limited to after grading, and it can be downloaded at any time with the latest information available at that point in time.
The Excel file has the following columns:
- Candidate ID
- Test name
- First/last attempt start times
- First/last attempt end times
- Total number of attempts taken by the candidate
- Final score
- Final grade
Note: The first attempt is generated by default, and ongoing attempts are generated as requested by candidates.
Candidate Dashboard
Candidate experience
In this section, we will cover the candidate's experience in the following situations:
Before Attempt #1
After Attempt #1
The candidate can see the upcoming attempt in the “Upcoming tests” tab with the following details:
- The parent test name
- The view more details that shows the attempt details for attempts taken so far
- The new attempt #
- The start time and end time of the attempt
- The final score logic that is being used to calculate final grade
- The number of attempts submitted, and # of attempts remaining
- The preliminary final score so far based on the number of attempts taken and final score logic
After All Attempts or Test End Time
The candidate can see the previous attempts (grouped) with the following information in the “Previous tests” tab:
- The parent test name
- The view more details that shows the attempt details for attempts taken so far
- The start time and end time of the attempt
- The number of attempts submitted, and # of attempts remaining
- The final score logic that is being used to calculate final grade
- The final score is based on the number of attempts taken and the final score logic
On clicking View more details, the candidate can access the following attempt details for the attempts taken so far:
- The attempt #
- The time of submission
- The score
- The post-submission review with predefined feedback
Candidate dashboard with Marks only
Candidate dashboard with grade assigned
Grader
You can evaluate Multiple attempts by assigning a grade, or by using achieved marks as the final result. When using a test with Multiple attempts, a Planner may prefer a total mark over a grade because total marks take into account all attempts and provide a more granular assessment of the candidate’s performance, allowing for a better understanding of their strengths and weaknesses.
Grade Multiple attempts
- Within the Marking workspace, navigate to Your grade.
- Assign a grade.
- Click Confirm.
How MA works with Numerical simulation
In this example, we have the following variables:
- Group_1: Generates a random number between 1 and 9, with a step size of 1.
- Group_2: The candidate input
Randomizing the variable can be useful in tandem with Multiple attempts because it can help to reduce the impact of guessing on the final score. Multiple attempts allow us to combine randomization with the benefit of automatic marks.
Candidate experience
Attempt 1, where ta1 randomly generates a value of 9.0
Attempt 2, where ta1 randomly generates a value of 3.0
Attempt 3, where ta1 randomly generates a value of 5.0
A question is marked with Correct in View submissions
Updating PSR Visibility:
The user has the flexibility to modify the visibility of the Post Submission Review (PSR) for all attempts by candidates on the MA test at any given moment. This enables the user to manage access and comprehension of candidates' performance according to the testing process requirements.
Making PSR Available:
When administering an MA test, the user aims to make the Post Submission Review (PSR) accessible to all candidates based on scheduling preferences. This can be provided instantly, scheduled for a specific date/time, or for a defined time duration. This adaptive approach ensures responsiveness to both user’s and candidates' needs.
Scenarios for PSR Availability:
Scenario 1a: Availability Period: Instantly after the candidate’s submission to forever
- Instant access not enabled
- Availability from [greyed out]
- Availability to [blank]
Scenario 1b: Availability Period: Instantly after candidate’s submission to 30 July
- Instant access enabled
- Availability from [greyed out]
- Availability to [30 July]
Scenario 2: Availability Period: 3 July to 30 July
- Instant access not enabled
- Availability from [3 July]
- Availability to [30 July]
Scenario 3: Availability Period: 10 July to Forever
- Instant access not enabled
- Availability from [10 July]
- Availability to [blank]
Note for Availability Date: Achieve any scenario using the provided settings; avoid using "always" from the calendar.