Known limitations

• Generic
• Test Setup
• Grading
• Candidate Dashboard

Generic

1. The help tooltips or links inside the product do not open or lead to the correct help documentation. Kindly use the help documentation shared with you as the single source of information for open beta. 
2. The only language available in the administrator interface for configuring the assessment and the new candidate dashboard is English. We plan to add translations for other languages in the future.  

Test Setup

1. The same test cannot be added to multiple assessments. 
2. The assessment cannot be assigned to an org unit. 
   1. The contributors cannot be added to the assessment through CSV. 
3. The contributors added to the assessment are also assigned to the test linked to it by default. However, it is possible to modify the contributors for tests if required. 
4. If you want to control the allocation of specific candidates to specific committees, we recommend using “edit grading committee” for the selected candidate(s) in Monitor. 

The assignment of candidates to committees can be done in the following ways:

• Automatic distribution of candidates (Add unassigned candidates)
• Manual distribution of candidates (View in Monitor > Edit grading committee). 

7. The error messages related to the activation of the assessment may not always have a clear description. Please note that to activate the Assessment path, it is mandatory for: 
   • The grading scale to be selected
   • The assessment start and end times are to be specified
   • At least one test is to be added to the assessment
8. Adding the test using create test, with their start time after the end time, doesn’t show a validation error in the assessment interface. However, the error would be shown when trying to activate the component individually. 

Grading

1. Once the grading of assessment is enabled, “main evaluation” type that is supported is Grade, and selecting a “grading scale” is mandatory to activate the assessment. It is not recommended to use Points as the “main evaluation” type. 
2. If there are multiple graders on the assessment, then we recommend using the following workflow: 

• The planner enables the marks alignment and thresholds for child tests. 
• The graders align marks for each component (instead of grades)

Note: The above is suggested as

• It ensures an effective calculation of the aggregate mark for the whole assessment with information about the result for each component preserved at the aggregate level. 
• It allows for the evaluation to happen at the right stage, where the graders with the required subject matter expertise to evaluate a specific component, are available.
• If marks are not aligned at the child level, then the parent score is calculated wrongly (known bug). 

3. The ability to set equal test weights automatically is not working as expected but will be available in the future. The workaround is to change the test weights manually, but since test weights cannot be a fraction or have decimal, it may not be possible to manually set equal weights if the number of components is an odd number.  
4. The test weights can currently be changed anytime from the marking tool, but from the parent, it is limited to before any submissions are created. Changing thresholds and weights is disabled when the status of assessment path is graded.
5. The threshold values for an assessment can be changed anytime before the assessment is graded. 
6. The grading scale can be changed until an assessment is active, but is locked from editing once it is activated.  
7. When the calculation of points for the assessment path is set to use Test weight, the threshold values are available as percentages (Percentage correct) for the Planner, but not yet available for the Grader in the grading workspace. The grader can only see the threshold values as absolute values (Minimum points) in the grading workspace.
8. If the method of confirmation used for test is “Confirm each candidate individually” and for the assessment is “Confirm all candidates at once”, there is a validation error displayed on ending the test (Summary > End now and start Marking). This message can be ignored, and the grader would be able to start grading the assessment as expected. 
9. You need to use the same grading scale for assessment and tests for automatic grading to work on the assessment. 
10. The grades for the assessment as a whole are not calculated correctly when collaborative marking is used on the assessment. We recommend using private grading for the assessment as a whole.

Candidate Dashboard

Note: The new candidate dashboard will be in beta throughout 2023, and we don’t recommend using it in live summative assessments before its general release. However, it can be used on low-stake formative tests from Q3.

The new candidate dashboard will coexist with the old dashboard during the beta period and candidates will be directed to the new dashboard for Assessment path until we make the full transition to it. 

1. The only language fully supported is English. Any other languages are not fully supported and there may be missing captions. 
2. The marks for manually marked questions in the question set are not shown consistently. The view results page and existing candidate report in the dashboard show marks for automatically marked questions but not for manually marked questions in the question set of a test. 
3. The explanations given by the graders for an assessment and its tests are not available for candidates to view. 
4. The assessment and its tests appear in both the "upcoming” and “previous” sections of the dashboard, regardless of their status. The status of components is currently updated.