.png)
Overview
The Cirrus digital assessment platform will be implemented for High-Stakes Assessments in NWU, enhancing the assessment ecosystem by seamlessly incorporating Cirrus into the assessment framework.
The purpose of this page is to provide you with supporting material, training opportunities and the standard operating process to adopt and implement Cirrus within your module.
Creating a new digital assessment on Cirrus
Introduction to Cirrus
This section will give you a brief introduction to Cirrus and all the functionalities to the platform.
Admin:
This page provides you with the steps to follow to edit your profile on Cirrus.
The following steps will assist you in the creation of your Cirrus assessment.
Step 1: Adding Cirrus as an external tool on your module site
1. Adding Cirrus to an eFundi site:
This video demonstrates how to add Cirrus as an external tool on your eFundi site
2. Adding participants to Cirrus:
This video provides you with the steps to follow when manually having to add participants on your Cirrus site, adding these participants on the eFundi site, enables the site instructor to be able to add them on a Cirrus assessment as a student or invigilator
Step 2: Creation of your question collection in the library
Library:
The library is the home for creating and storing all the questions you plan to use in assessments. You organise your questions into Collections (also referred to as 'question banks')
This page provides an overview of collections
Exporting question pools from eFundi
This page provides an overview of how to export question pools from eFundi
Converting eFundi question pool to a Cirrus Collection
This page provides the steps to follow when converting your question pools on eFundi to a Cirrus collection
1. How to add and use the library
This page provides an overview of collections, how to create and use collections.
2. Adding learning objectives to Cirrus
This page provides you with the steps to follow when creating learning objectives in Cirrus.
3. Create welcoming and closing pages
This page provides you with the steps to follow when creating an open and close page for an assessment, every assessment needs open (This includes assessment instructions with the time allocation etc) and closing page (This may include a survey of just a concluding page to an assessment).
4. Adding different question types
The pages below provide you with the different question types you can add to a collection (question pool)
It presents respondents or test-takers with a question or statement and provides a set of predefined answer options, among which they must choose the correct or most appropriate answer.
The Multiple Response (MR) question type is similar to Multiple Choice, only it allows you to set more than one correct answer to be selected by the candidate.
the Either/Or for questions with only two alternative answer options. You create the question in the same way as Multiple Choice, but you will not be able to add or remove answer alternatives. Also, you can only tick one answer alternative to be correct. This question type is also sometimes known as yes/no or true/false questions.
• Numeric
The Numeric question type allows the candidate to enter a numeric response in order to answer the question. This can include alphanumeric characters as well if you wish, although you can set the question up so that they are ignored.
The Fill in the Blanks question type requires the candidate to fill in a blank. This may be a missing word from a sentence, or the answer to an acronym as in the example below.
• Order
The Order question type requires candidates to put the available answer options into the correct order.
• Hotspot
The Hotspot question type allows you to set a question that requires clicking on the correct answer. Think of clicking the correct city on an image or map. Or clicking the correct part in a mechanical flow chart.
The Select from List (SFL) question type requires the candidate to select their answer from a predefined list of possible correct answers. For example, the candidate may be required to select the correct word within a given sentence.
The Short Answer question type allows you to set a question that requires the candidate to provide a short written answer in response. This question type is suitable for questions that require a couple of words or one line answers
The Drag and Drop question type was developed together with a customer. The assignment aims to move pictures over a background so they are positioned on the corresponding placeholders.
Use the Match question type to allow your candidates to match options using drag and drop.
Extended Match allows you to create questions where an answer options can be matched with multiple others or none (so called distractors). Use the Extended Match question type to allow your candidates to match options using drag and drop or clicking (if you have a long list and scrolling is not an option).
• Comprehensive integrated puzzle
A Comprehensive Integrated Puzzle is a matrix that a candidate needs to complete. It aims to test (clinical) reasoning.
• Financial statement questions
The Financial Statement item type allows creating versatile questions specifically geared toward Balance Sheets, Income Statements, Account statements, etc. The Financial Statement question type requires the candidate to fill out a table with numeric values (integers) based on the question or case study data presented.
The Programming question type requires the candidate to write or debug code. Candidates can be offered the option to run and evaluate their code. This evaluation uses testcase(s) defined by the question author. The question is auto-scored based on the number of test cases (written by the author) that are successfully passed.
question type for the exact sciences is an open ended item that allows the answers to be in the form of formulas. The answers can be scored and checked automatically and can be accompanied with targeted feedback. Moreover, questions can be created with random parameters, that make every question unique. Read more about 'What types of questions Sowiso items can be used for?"
The Essay question is an open question type that allows the candidate to provide a longer, free text answer to a question. You can visualise the limit on the number of words the candidate can write. And you can upload a marking scheme for the marker. (This question type cannot be computer marked).
With a 'section page' you can present a case-study or relevant information for a question or set of questions. You can also use a section to create an assessment within the assessment, including its own duration time and pass mark.
How to add and use media elements
This page provides you with an overview on how to add media items to your Library and how to use them in your items in your assessments.
Using polytomous scoring rules:
When questions in Cirrus allow for multiple answers, you can apply Dichotomous and Polytomous scoring rules:
- Dichotomous: The candidate must answer everything correctly to get the max score. When answered partially correct, the candidate will score 0.
- Polytomous: Gives a partial score when the candidate answers the question partially correct. Sharing collections This page provides an overview on the settings that determine who can work together on Items (questions) and who can use them in assessments.
This page provides an overview on the settings that determine who can work together on Items (questions) and who can use them in assessments.
Step 3: Creating assessments
Quick Start: add a new assessment
This quick guide will help assessment authors to understand to process of creating exams fast.
This page will give you an introduction to the Assessments.
This page provides details about the assessment and the steps you need to take to be able to publish it.
Adding an assessment
- Step 1: Create new assessment
- Step 2: Assessment options
- Step 3: Question selection
Simplified blueprint (Using topics)
Extended Blueprint (Using Learning Objectives)
- Step 4: Forms
Assessment form: Creating Introduction and Finish pages
This tab will assist you in making changes to your assessment. However, changes can only be made to an assessment if the assessment is not yet published.
Use the Share button to work on assessments together or to share your assessment via an Anonymous link for demo and preview purposes.
Cirrus allows you to create different versions of your assessment. See the benefits and how to version your assessment in this tab.
This tab contains information for previewing your assessment in a candidate simulation before scheduling. This is useful to ensure you detect errors in the questions and the functionality of the question type for your envisioned design.
Setting an active time frame in which your Assessment can be scheduled
This allows the scheduling of an assessment in a specific time frame.
A notepad during the exam is a useful tool for candidates. It can be used simply as a notepad for internal notes, but it also happens for certain questions that the candidate answered wrong, however the reasoning behind it was perfect.
This allows you to grant additional assessment time to your students. Learn the two ways of going about it in this tab. If you will be printing your assessment This option is useful if you plan on printing your assessment.
Learn how to enable candidates to report feedback on questions during their exam
Step 4: How to schedule an assessment
This section provides you an overview of how to schedule an assessment, monitor ongoing assessments (Invigilation) and schedule review sessions in Cirrus.
This page explains the role of a scheduler as well as an invigilator Quick start on creating a new schedule This page provides a guide to help instructors understand the process of scheduling assessments. Scheduling
This page explains how a schedule can be created and some of the important functions.
This page provides you with the options to make changes on a created schedule.
The first tab (Overview) contains the Assessment Dashboard. This page provides details about the assessment and the steps you need to take to be able to publish it.
On the 'Candidates'-tab you can add candidates to your schedule:
On the 'Invigilators'-tab you can select the invigilator(s) you wish to assign to the assessment. This is optional and can be done right up to the start of the assessment window as long as the schedule is deactivated:
After all steps before 'Activate schedule' are completed, the 'Activate' button on the top right will become active. Only once a schedule has been activated will the assessment be available / visible to candidates.
The assessment options (to be found via the 3 vertical dots in the top right side) is visible to the person scheduling the assessment are if the author has allowed this.
This page provide an overview to the difference between the duration of an assessment and the timeframe of the schedule
Invigilator:
This quick start guide will help invigilators/proctors to understand to process of monitoring an ungoing test in Cirrus. ·
This page provides an overview of the invigilation overview page.
This page provides an overview on how to invigilate an assessment.
This page provides you with the steps to follow when pausing and than resuming, giving extra time, voiding, sending messages and generating a PIN code for an assessment for a participant once they are completing their assessment .
This page provides invigilators access to an assessment to access candidate logs, this is great when wanting to see the actions / progress of the candidate during an assessment.
Once the assessment is taken and its results have been published, you can optionally schedule a review session with the candidate. During this session the completed assessment will be made visible to the candidate. Both teacher and candidate can sit down together and discuss the outcome.
This page provides an overview on how create a review session for a candidate.
This page provides an overview for options panel for candidate reviews.
The pages provide`s an overview on how to update review sessions here.
The page It is possible to archive review sessions so candidates cannot access them anymore.
Step 5: How to pull assessment reports
Marking your Assessments
This quick start guide will help assessors to understand to process of marking under the Assess tab.
Decide if your assessment will be completely auto-scored, have the scheduler be the assessor or have the coordinator assign assessors. This tab allows you to set up different marking workflows between three option that allows for seamless marking experience.
An extensive marking workflow requires a marking coordinator to set up this workflow.
Users with access to the Coordinator tab, and who are assigned to certain hierarchies, will be able to set up marking workflows. This starts here, on the coordinator overview page.
The Coordinator Dashboard provides marking coordinators with the tools they need to effectively manage the marking process. This tab will outline the features of the dashboard, aiming to help you understand how to make the most of it for tracking assessor/moderator progress and ensuring timelines are met.
Conducting the marking:
The Assess tab gives access to all the assessments that you can mark.
When an assessor opens a schedule to mark, it will lead to the Submissions tab. Here they'll find all the scripts they need to assess. This page explains the Submissions tab and Progress report tab.
Once you've selected a user to start scoring the ‘Candidate summary screen’ will open showing a summary of the candidate's responses to the questions in the assessment.
The scoring page for manual and autoscored questions are generally the same, but essay question types have more options since they are manual scored.
The Progress Report provides you with a visual presentation of the workload. It includes the list of all candidates and an outline of all questions
Once your team of assessors and moderators has submitted their scores of the candidates answers, the results are known internally so to speak. The next step is publication of the results.
The Rescoring-tab will show all schedules for which all results have been submitted and the schedule time has ended. This allows you to re-score auto-scored results before publishing.
Re-marking will enable your organization to handle candidate appeals for re-evaluation. This tab explains how this is done.
There are various reasons for voiding a candidate’s assessment such as cheating or errors on specific candidate’s assessment.
This tab shows results of all individual candidates who took an exam on your Cirrus platform. These results come from exams that have already been assessed, (auto)scored, submitted and published.
The Assessment Performance report offers the opportunity to review the overall psychometrics for each of your assessments. Using this you can evaluate your assessments and how well the questions in them perform.
This tab shows the assessment statistics. The Rit‑value indicates correlation of question score to the total examination score. It tells us to what extent an item contributes to isolating the good candidates from the entire pool of test takers. In other words: it demonstrates the discriminating properties of the item and tells something about the performance of the item versus the test as a whole. The Rir‑value is similar to Rit but more precise because the score of the item itself is not taken into account.
A score report is a breakdown of the results from the assessment taken by the candidates. The scores are divided into sections/attributes for each individual candidate.
The Angoff method is a statistical technique used to set cut scores, the minimum scores required to pass a test or evaluation. The Angoff method is often used to set cut scores for professional licensing exams, such as the bar exam for lawyers or the medical licensing exam. It is also used in education to set passing scores for high stakes tests, such as state-mandated assessments.
At the completion of marking your assessment, you will need to export the results to eFundi gradebook, this tab will show you how. In the platform candidate results can be exported at various places.
Testimonial
.png)
This past semester was the second semester that I have been conducting all my assessments in Cirrus. I have been receiving a lot of help and guidance from the DTLE team. I found the screenshares that we did very helpful, especially in dividing the marking for assessors which was something new that I had to do this semester. By means of a Group Chat on Teams, I received very timely feedback when I encountered challenges. My students will be writing their high-stakes exam in Cirrus on 30 October and with the current support I think we will be able to execute this. I hope to continue to use Cirrus in 2024.