Enabling the feature
The Psychometrics API data is a feature that needs to be enabled. Please follow the normal process of requesting through our Service Desk.
| Note: The feature requires use of Item Banks with improved permissions and Custom User Roles. These items will be enabled together with the Psychometrics API data, they are not optional. |
Overview of the feature
1. Item banking pre-requisites
Psychometrics data will not be provided in an API by default for your tenant. You will need to raise a service request to enable this functionality.
Once your tenant is set up, you will need to create a new item bank so that it inherits the required rules. Metrics will automatically be calculated from a test created using a question set from this item bank.
2. Supported test deliveries
For the initial release, you will be able to fetch data for submissions of the following types:
- Manual delivery by test taker
- Offline submission uploaded by Invigilator
- Invigilator submission on behalf of the test taker via Resubmission action
- Test taker submitting after their Test was opened for resubmission
- Submissions made when Auto submission is enabled
- Withdrawn and blank submissions made by test takers
The CTT metrics calculated will be based on these types of submissions. Additionally, to ensure that we can provide data early, these metrics (apart from exposure) will only be calculated for automatically marked questions.
3. Early availability of data
The submission data of a test created using an item bank is available as soon as students submit their test. The submissions are then used to calculate metrics even before grading is completed for that test!
This allows graders to detect flaws in a test and take marking decisions early on for a hassle-free grading process.
The calculations will start about 2 hours after the last candidate has submitted - this delay is to account for any extra time added during the test to ensure that as many submissions as possible are included in the calculations.
Psychometrics API data: Calculation event logs
4. Controlled access to sources
Based on the data requirements of a user, administrators can selectively restrict access to data sources by assigning them permissions. Additionally, a user can only view the test data from item banks that they have access to.
This ensures that a user can only view data they genuinely need for their work, and not have unrestricted access to test data.
Psychometrics API data: Custom User Role (CUR)
5. Off-the-shelf CTT metrics
Apart from providing raw submissions data Psychometrics API data: Response data source, the API also provides precalculated test-level psychometrics (such as facility and discrimination). You can fetch these metrics and create visualisations without the need to process the data!
Script to fetch API data
The fileExport/order API is an existing endpoint which can be used to extract submission data and psychometrics data for a test. The API also provides options to filter the data based on query parameters (such as test ID or item bank ID), and specify the file format (JSON or parquet).
The job ID returned needs to be polled using the fileExport/status endpoint to track the progress of the request and to download the file.