Known limitations
General Information
- The help tooltips or links inside the product do not open or lead to the correct help documentation. Kindly use the help documentation shared with you as the single source of information for open beta.
- The only language available in the administrator interface for configuring the assessment and the new candidate dashboard is English. We plan to add translations for other languages in the future.
Test Setup
-
A test cannot be added to multiple assessments.
-
Assessments cannot be assigned to organizational units.
-
Contributors cannot be added to assessments through CSV.
-
Contributors added to an assessment are automatically assigned to the linked test. Modifying contributors for tests is possible if needed.
-
To control candidate allocation to committees, use "edit grading committee" in Monitor. Candidates can be assigned to committees automatically or manually.
-
Activation of assessments requires:
- Selection of a grading scale
- Specification of assessment start and end times
- Addition of at least one test to the assessment
-
When adding tests with start times after end times using create test, a validation error may not show in the assessment interface but will appear when activating the component individually.
Grading
-
When enabling grading for assessments, use the "Grade" type as the main evaluation, and selecting a grading scale is mandatory.
-
For assessments with multiple graders:
- The planner enables marks alignment and thresholds for child tests.
- Graders align marks for each component.
-
Ensure marks are aligned at the child level to calculate the aggregate mark correctly. Failure to align may result in a miscalculated parent score (known bug).
-
Equal test weights can't be set automatically but can be adjusted manually. Note limitations when dealing with odd numbers of components.
-
Test weights can be changed from the marking tool before any submissions are created, with limitations on changes once the assessment path is graded.
-
Threshold values for an assessment can be changed anytime before grading, while the grading scale can be changed until assessment activation.
-
When using Test weight for point calculation, the planner sees threshold values as percentages, while the grader sees them as absolute values in the grading workspace.
-
If confirmation methods differ between tests and assessments, ignore the validation error when ending a test for confirmation, and grading can proceed as expected.
-
Use the same grading scale for assessments and tests to enable automatic grading.
-
Collaborative marking on assessments may lead to incorrect grade calculations. Private grading is recommended for the assessment as a whole.
Candidate Dashboard
Note: The new candidate dashboard will be in beta throughout 2023, and we don’t recommend using it in live summative assessments before its general release. However, it can be used on low-stake formative tests from Q3.
The new candidate dashboard will coexist with the old dashboard during the beta period and candidates will be directed to the new dashboard for Assessment path until we make the full transition to it.
- The only language fully supported is English. Any other languages are not fully supported and there may be missing captions.
- The marks for manually marked questions in the question set are not shown consistently. The view results page and existing candidate report in the dashboard show marks for automatically marked questions but not for manually marked questions in the question set of a test.
- The explanations given by the graders for an assessment and its tests are not available for candidates to view.
- The assessment and its tests appear in both the "upcoming” and “previous” sections of the dashboard, regardless of their status. The status of components is currently updated.