Cirrus Lecturer Support

Overview

The Cirrus digital assessment platform will be implemented for High-Stakes Assessments in NWU, enhancing the assessment ecosystem by seamlessly incorporating Cirrus into the assessment framework.
The purpose of this page is to provide you with supporting material, training opportunities and the standard operating process to adopt and implement Cirrus within your module.

 

            Creating a new digital assessment on Cirrus

       Technical support for already created digital assessments

 

 

Introduction to Cirrus

This section will give you a brief introduction to Cirrus and all the functionalities to the platform.

Admin:

Introduction to Cirrus

Editing your Cirrus profile

This page provides you with the steps to follow to edit your profile on Cirrus.

Creating assessment group

The following steps will assist you in the creation of your Cirrus assessment.

Step 1: Adding Cirrus as an external tool on your module site

1. Adding Cirrus to an eFundi site:

This video demonstrates how to add Cirrus as an external tool on your eFundi site

2. Adding participants to Cirrus:

This video provides you with the steps to follow when manually having to add participants on your Cirrus site, adding these participants on the eFundi site, enables the site instructor to be able to add them on a Cirrus assessment as a student or invigilator

Step 2: Creation of your question collection in the library

Library:

The library is the home for creating and storing all the questions you plan to use in assessments. You organise your questions into Collections (also referred to as 'question banks')

Overview of the library

This page provides an overview of collections

Exporting question pools from eFundi

This page provides an overview of how to export question pools from eFundi

Converting eFundi question pool to a Cirrus Collection

This page provides the steps to follow when converting your question pools on eFundi to a Cirrus collection

1. How to add and use the library

This page provides an overview of collections, how to create and use collections.

2. Adding learning objectives to Cirrus

This page provides you with the steps to follow when creating learning objectives in Cirrus.

3. Create welcoming and closing pages

This page provides you with the steps to follow when creating an open and close page for an assessment, every assessment needs open (This includes assessment instructions with the time allocation etc) and closing page (This may include a survey of just a concluding page to an assessment).

4. Adding different question types

The pages below provide you with the different question types you can add to a collection (question pool)

Multiple choice questions

It presents respondents or test-takers with a question or statement and provides a set of predefined answer options, among which they must choose the correct or most appropriate answer.

Extended multiple choice

The Multiple Response (MR) question type is similar to Multiple Choice, only it allows you to set more than one correct answer to be selected by the candidate.

Either/or

the Either/Or for questions with only two alternative answer options. You create the question in the same way as Multiple Choice, but you will not be able to add or remove answer alternatives. Also, you can only tick one answer alternative to be correct. This question type is also sometimes known as yes/no or true/false questions.

Numeric

The Numeric question type allows the candidate to enter a numeric response in order to answer the question. This can include alphanumeric characters as well if you wish, although you can set the question up so that they are ignored.

Fill in the blank

The Fill in the Blanks question type requires the candidate to fill in a blank. This may be a missing word from a sentence, or the answer to an acronym as in the example below.

Order

The Order question type requires candidates to put the available answer options into the correct order.

Hotspot

The Hotspot question type allows you to set a question that requires clicking on the correct answer. Think of clicking the correct city on an image or map. Or clicking the correct part in a mechanical flow chart.

Select from list

The Select from List (SFL) question type requires the candidate to select their answer from a predefined list of possible correct answers. For example, the candidate may be required to select the correct word within a given sentence.

Short answer

The Short Answer question type allows you to set a question that requires the candidate to provide a short written answer in response. This question type is suitable for questions that require a couple of words or one line answers

Drag and drop

The Drag and Drop question type was developed together with a customer. The assignment aims to move pictures over a background so they are positioned on the corresponding placeholders.

Matching question

Use the Match question type to allow your candidates to match options using drag and drop.

Extended matching question

Extended Match allows you to create questions where an answer options can be matched with multiple others or none (so called distractors). Use the Extended Match question type to allow your candidates to match options using drag and drop or clicking (if you have a long list and scrolling is not an option).

Comprehensive integrated puzzle

A Comprehensive Integrated Puzzle is a matrix that a candidate needs to complete. It aims to test (clinical) reasoning.

Financial statement questions

The Financial Statement item type allows creating versatile questions specifically geared toward Balance Sheets, Income Statements, Account statements, etc. The Financial Statement question type requires the candidate to fill out a table with numeric values (integers) based on the question or case study data presented.

Programming question type

The Programming question type requires the candidate to write or debug code. Candidates can be offered the option to run and evaluate their code. This evaluation uses testcase(s) defined by the question author. The question is auto-scored based on the number of test cases (written by the author) that are successfully passed.

Mathematical question type

question type for the exact sciences is an open ended item that allows the answers to be in the form of formulas. The answers can be scored and checked automatically and can be accompanied with targeted feedback. Moreover, questions can be created with random parameters, that make every question unique. Read more about 'What types of questions Sowiso items can be used for?"

Essay question type

The Essay question is an open question type that allows the candidate to provide a longer, free text answer to a question. You can visualise the limit on the number of words the candidate can write. And you can upload a marking scheme for the marker. (This question type cannot be computer marked).

Section question type

With a 'section page' you can present a case-study or relevant information for a question or set of questions. You can also use a section to create an assessment within the assessment, including its own duration time and pass mark.

How to add and use media elements

This page provides you with an overview on how to add media items to your Library and how to use them in your items in your assessments.

Using polytomous scoring rules:

When questions in Cirrus allow for multiple answers, you can apply Dichotomous and Polytomous scoring rules:

  • Dichotomous: The candidate must answer everything correctly to get the max score. When answered partially correct, the candidate will score 0.
  • Polytomous: Gives a partial score when the candidate answers the question partially correct. Sharing collections This page provides an overview on the settings that determine who can work together on Items (questions) and who can use them in assessments.

Sharing collections

This page provides an overview on the settings that determine who can work together on Items (questions) and who can use them in assessments.

Step 3: Creating assessments

 

Overview of Assessment:

Quick Start: add a new assessment

This quick guide will help assessment authors to understand to process of creating exams fast.

Assessment Overview

This page will give you an introduction to the Assessments.

Assessment Dashboard

This page provides details about the assessment and the steps you need to take to be able to publish it.

Adding an assessment

Manual question selection 

Simplified blueprint (Using topics)

Extended Blueprint (Using Learning Objectives)

  • Step 4: Forms

Creating an assessment form

Assessment form: Creating Introduction and Finish pages

Assessment form: Questions

Multiple forms

Assessment Form: Options

This tab will assist you in making changes to your assessment. However, changes can only be made to an assessment if the assessment is not yet published.

Sharing your assessment

Use the Share button to work on assessments together or to share your assessment via an Anonymous link for demo and preview purposes.

Versioning your assessment

Cirrus allows you to create different versions of your assessment. See the benefits and how to version your assessment in this tab.

Previewing your assessment

This tab contains information for previewing your assessment in a candidate simulation before scheduling. This is useful to ensure you detect errors in the questions and the functionality of the question type for your envisioned design.

Setting an active time frame in which your Assessment can be scheduled

This allows the scheduling of an assessment in a specific time frame.

Enabling Notepad

A notepad during the exam is a useful tool for candidates. It can be used simply as a notepad for internal notes, but it also happens for certain questions that the candidate answered wrong, however the reasoning behind it was perfect.

Allocating extra time

This allows you to grant additional assessment time to your students. Learn the two ways of going about it in this tab. If you will be printing your assessment This option is useful if you plan on printing your assessment.

Candidate feedback

Learn how to enable candidates to report feedback on questions during their exam

Step 4: How to schedule an assessment

This section provides you an overview of how to schedule an assessment, monitor ongoing assessments (Invigilation) and schedule review sessions in Cirrus.

Introduction to delivering

This page explains the role of a scheduler as well as an invigilator Quick start on creating a new schedule This page provides a guide to help instructors understand the process of scheduling assessments. Scheduling

This page explains how a schedule can be created and some of the important functions.

This page provides you with the options to make changes on a created schedule.

The first tab (Overview) contains the Assessment Dashboard. This page provides details about the assessment and the steps you need to take to be able to publish it.

On the 'Candidates'-tab you can add candidates to your schedule:

On the 'Invigilators'-tab you can select the invigilator(s) you wish to assign to the assessment. This is optional and can be done right up to the start of the assessment window as long as the schedule is deactivated:

After all steps before 'Activate schedule' are completed, the 'Activate' button on the top right will become active. Only once a schedule has been activated will the assessment be available / visible to candidates.

The assessment options (to be found via the 3 vertical dots in the top right side) is visible to the person scheduling the assessment are if the author has allowed this.

This page provide an overview to the difference between the duration of an assessment and the timeframe of the schedule

Invigilator:

This quick start guide will help invigilators/proctors to understand to process of monitoring an ungoing test in Cirrus. ·

This page provides an overview of the invigilation overview page.

This page provides an overview on how to invigilate an assessment. 

This page provides you with the steps to follow when pausing and than resuming, giving extra time, voiding, sending messages and generating a PIN code for an assessment for a participant once they are completing their assessment . 

This page provides invigilators access to an assessment to access candidate logs, this is great when wanting to see the actions / progress of the candidate during an assessment.

Candidate review

Once the assessment is taken and its results have been published, you can optionally schedule a review session with the candidate. During this session the completed assessment will be made visible to the candidate. Both teacher and candidate can sit down together and discuss the outcome. 

       This page provides an overview on how create a review session for a candidate.  

       This page provides an overview for options panel for candidate reviews.

       The pages provide`s an overview on how to update review sessions here.  

       The page It is possible to archive review sessions so candidates cannot access them anymore.

Step 5: How to pull assessment reports

Marking your Assessments

Quick start: Marking

This quick start guide will help assessors to understand to process of marking under the Assess tab.

Choosing the right workflow

Decide if your assessment will be completely auto-scored, have the scheduler be the assessor or have the coordinator assign assessors. This tab allows you to set up different marking workflows between three option that allows for seamless marking experience.

Coordinator-Extended flow

An extensive marking workflow requires a marking coordinator to set up this workflow.

Users with access to the Coordinator tab, and who are assigned to certain hierarchies, will be able to set up marking workflows. This starts here, on the coordinator overview page.

The Coordinator Dashboard provides marking coordinators with the tools they need to effectively manage the marking process. This tab will outline the features of the dashboard, aiming to help you understand how to make the most of it for tracking assessor/moderator progress and ensuring timelines are met.

Step 1: Settings

Step 2: Adding assessors

Step 3: work allocation

Conducting the marking:

The Assess tab gives access to all the assessments that you can mark.

When an assessor opens a schedule to mark, it will lead to the Submissions tab. Here they'll find all the scripts they need to assess. This page explains the Submissions tab and Progress report tab.

Once you've selected a user to start scoring the ‘Candidate summary screen’ will open showing a summary of the candidate's responses to the questions in the assessment.

The scoring page for manual and autoscored questions are generally the same, but essay question types have more options since they are manual scored.

The Progress Report provides you with a visual presentation of the workload. It includes the list of all candidates and an outline of all questions

Publishing results

Once your team of assessors and moderators has submitted their scores of the candidates answers, the results are known internally so to speak. The next step is publication of the results.

Rescoring

The Rescoring-tab will show all schedules for which all results have been submitted and the schedule time has ended. This allows you to re-score auto-scored results before publishing.

Remarking

Re-marking will enable your organization to handle candidate appeals for re-evaluation. This tab explains how this is done.

Voiding attempts

There are various reasons for voiding a candidate’s assessment such as cheating or errors on specific candidate’s assessment.

Overview of the Reports:

Candidate Results

This tab shows results of all individual candidates who took an exam on your Cirrus platform. These results come from exams that have already been assessed, (auto)scored, submitted and published.

Assessment Performance

The Assessment Performance report offers the opportunity to review the overall psychometrics for each of your assessments. Using this you can evaluate your assessments and how well the questions in them perform.

Statistics Rit and Rir values

This tab shows the assessment statistics. The Rit‑value indicates correlation of question score to the total examination score. It tells us to what extent an item contributes to isolating the good candidates from the entire pool of test takers. In other words: it demonstrates the discriminating properties of the item and tells something about the performance of the item versus the test as a whole. The Rir‑value is similar to Rit but more precise because the score of the item itself is not taken into account.

Score Reports

A score report is a breakdown of the results from the assessment taken by the candidates. The scores are divided into sections/attributes for each individual candidate.

The Angof Method

The Angoff method is a statistical technique used to set cut scores, the minimum scores required to pass a test or evaluation. The Angoff method is often used to set cut scores for professional licensing exams, such as the bar exam for lawyers or the medical licensing exam. It is also used in education to set passing scores for high stakes tests, such as state-mandated assessments.

Exporting Candidate Reports

At the completion of marking your assessment, you will need to export the results to eFundi gradebook, this tab will show you how. In the platform candidate results can be exported at various places.

Interactive Tutorial Guide

As an innovative platform with ongoing updates to enhance your assessment experience, this tutorial may become outdated over time. Therefore, we strongly recommend that you first engage with the five steps outlined above to ensure you have the most current and relevant information

Interactive Tutorial Guide

 

 Testimonial

 

This past semester was the second semester that I have been conducting all my assessments in Cirrus. I have been receiving a lot of help and guidance from the DTLE team. I found the screenshares that we did very helpful, especially in dividing the marking for assessors which was something new that I had to do this semester. By means of a Group Chat on Teams, I received very timely feedback when I encountered challenges. My students will be writing their high-stakes exam in Cirrus on 30 October and with the current support I think we will be able to execute this. I hope to continue to use Cirrus in 2024.