Differences and Similarities with Legacy Grading Workflows

Although Grading Workflows predominately relates to administrative functionality, there are some changes and new additions to the Grade UI to cater for this new functionality.

What’s the same?

• The number of screens a Grader sees
• The way Graders award marks
• The way Graders leave annotations and page notes
• The way Graders and/or Planners leave Explanations
• The way Graders navigate between candidates and submissions
• The way Graders confirm their marking
• The way Graders download and print submissions

What’s different and new?

• The information that is displayed in the header
• The introduction of Approve/ Do Not Approve functionality for quality assurance phases
• The ability to see what marks other graders have awarded in the same or previous phases (if enabled)

Grade Header

Captions

Once a phase has been opened for grading, the status will show as Marking in progress along with a progress tracker showing n of x candidates graded. This tracks the number of candidate submissions that have been completed in full.

Once the Grader has marked all work allocated to them in the phase, the status will read as ‘All candidates/questions marked. Click Confirm to finish’.

Once the Grader has confirmed all their marking, the status will show as Confirmed on hh:mm DD MMM YY.

Moving between Groups

Groups (formerly committees) work in the same way at the moment. This means if a user is a Planner and Grader in the same test, or if a Grader is in multiple groups across multiple phases, they still need to toggle between groups via the header.

Visibility of other Grading

Depending on the settings configuration of Visibility of other Grading in the workflow, Graders may see a table under the marking space which will display information such as:

• What phases the submissions has currently been through
• Which graders marked them
• What marks were given per question
• What the total marks for that grader were
• What calculated total mark (or conflict) was resolved

Any information that is not selected to be displayed will be hidden.

When multiple graders are making the same submissions in the same phase and Visibility of other Grading is set to display marks to each other, if one grader has not yet marked the question and/or submission then they will not show up in the table yet.

Question audit trail

When Visibility of other Grading has been enabled on a phase, Graders will be able to see two tables in the marking area of the Grade Module. The first table is titled Question audit trail.  This will display what mark each grader awarded in the current and/or previous phase(s) along with the name of the grader.

If View grader name has not been enabled for that phase, ‘Anonymous’ will be displayed in place of the grader’s name. 

Test audit trail

The second table is titled Test audit trail and displays the cumulative number of marks awarded to the candidate’s entire submission. 

It’s important to remember this is an active total so if two graders are marking at the exact same time, the total mark for the test will change as they mark more questions.

How Calculations are displayed

The above image displays the marking trail of the first phase during the first phase. There is no calculation information because the phase has not closed yet. 

However, the moment the phase is closed and calculations are processed, the calculated result will be displayed to any grader in subsequent phases with visibility permissions enabled. The example below is where the calculation was set to be an average.

Graders in Approved / Not Approved phases

If a phase is not set for marking but rather just Approved/Not Approved, once the phase opens, the Grader will see Review in Progress as well as Approve and Do Not Approve buttons.

Upon clicking one of the buttons, they will be presented with a confirmation screen which also contains space for notes. 

These notes will show up in the Progress Dashboard for the Planner to read.