Metrics and reports
The World Control Panel separates metrics by the decision they support:
- Overview shows readiness and the current state of the world.
- Build describes challenge configuration and content coverage.
- Run shows live operational activity and issues needing attention.
- Review shows final outcomes, standings and challenge performance.
- Reports contains downloadable results. Recruitment worlds expose the same outcomes under Recruitment → Analytics and Recruitment → Reports.
Metric catalogue
| Metric | Definition | Applies to | Primary location | Included in reports |
|---|---|---|---|---|
| Participating teams | Participant teams registered in the world. This is registration, not proof of activity. | Team events | Review overview | Event PDF, standings CSV |
| Engaged participants | Distinct participants who opened a challenge instance. | Worlds with challenge presence tracking | Review overview | Event PDF |
| Engaged time | Time recorded while participants were active in challenge instances. It is different from elapsed wall-clock time. | Worlds with challenge presence tracking | Review overview; Recruitment analytics | Event and recruitment reports |
| Completed challenge / solve | A non-pending challenge completion, or a successful flag submission when the challenge has exactly one flag. A flag on a multi-flag challenge is partial progress, not a solve. | Scored challenge worlds | Review overview, standings and performance | Event PDF, team PDF, standings CSV |
| Completion rate | Completed team–challenge pairs divided by possible team–challenge pairs. Only challenges available by the reporting cutoff are in the denominator. | Team events | Review overview | Event PDF |
| Available challenge | A visible challenge whose appearance time is at or before the reporting cutoff. | Challenge worlds | Used by completion rate | — |
| Reporting cutoff | The scoreboard freeze time when configured; otherwise the current complete dataset. Results after the cutoff do not change final reporting metrics. | Scored worlds | Review | All final reports |
| Average rating | Mean post-event world rating. No rating is shown when no responses exist. | Worlds with ratings | Review overview | Event PDF |
| Challenge ratings | Number of participant challenge feedback responses. | Worlds with ratings | Review overview and performance | Event PDF |
| Applicant status | Current application state: applied, in progress, passed or failed. | Recruitment worlds | Recruitment overview and Applicants | Applicant PDF and CSV |
| Assessment elapsed time | Wall-clock time between an applicant starting and finishing, or the current time for an active assessment. | Recruitment worlds | Recruitment Analytics | Applicant PDF and CSV |
| Applicant engaged time | Time the applicant was actively present in assigned challenge instances. | Recruitment worlds | Recruitment Analytics | Applicant PDF and CSV |
| Challenge completion rate | Applicants who completed the assigned challenge divided by applicants assigned that challenge. | Recruitment worlds | Recruitment Analytics | Challenge PDF and CSV |
| Challenge stall rate | Assigned applicants who started but did not complete the challenge, divided by applicants who started it. | Recruitment worlds | Recruitment Analytics | Challenge PDF and CSV |
Reading completion correctly
Multi-flag challenges award progress as each flag is submitted. Those submissions can affect score, but the challenge is only counted as completed when its completion record exists. Pending score submissions never count as completions.
Final standings and review reports honour the scoreboard freeze. Live operational screens may continue to show activity after that point, so use Review for final outcomes and Run for what is happening now.
Download reports
For a standard event, open Review → Reports. The Event review report card downloads the event-wide PDF and standings CSV. The Team report card lets you select a participant team and download a PDF containing its challenge outcomes, partial flag progress, solve and engaged time, hints, incorrect attempts, and member contributions. The Challenge report card downloads either an all-challenge PDF or a report for one selected challenge. University results are presented as aggregate solver counts, separately from the individual solver list, which does not include university affiliation.
For a recruitment assessment, open Recruitment → Reports. Select an applicant in the Applicant report card to download their PDF, or use the Challenge report card for an assessment-wide challenge PDF. CSV exports remain available for further analysis. Applicant attribution identifies whose account recorded the activity; it should not be interpreted as proof that the person worked alone.