CAA Case Study

Civil Aviation Authority

UK CAA & UK CAA International (CAAi)

The UK Civil Aviation Authority (CAA) is the government appointed regulator of aviation safety in the UK. The CAA's responsibilities include ensuring that the aviation industry meets the highest safety standards and manages security risks effectively. One of the key ways in which this is achieved is through pilot licensing. This licensing includes requirements to pass specified written exams. These exams are harmonised across the world to ensure consistent levels of safety, with standards and requirements set by the International Civil Aviation Organisation and within Europe by the European Aviation Safety Agency.

CAA International is a technical business arm of CAA, providing advisory, training, examination and licensing services to agencies and aviation authorities in other countries. This support has been provided commercially across more than 140 countries.

Background

World Class Arena Limited (WCAL) began working for the CAA in 2008. Our role has been to support an organisational-wide change programme, focusing specifically on the migration of paper-based exams to computer-based exams in the UK and internationally. CAA licenses pilots and aircraft engineers seeking to operate under a UK licence; CAA International provides those examination services to other countries' aviation regulators on a commercial basis.

Initially, WCAL's work focused on procurement support, assisting the CAA in selecting an item-banking solution and test delivery vendor. This work subsequently included migrating the paper-based item-bank to computer, as well as managing the first CBT delivery in Malaysia.

The Requirements

As part of an organisation-wide modernisation programme, CAA was looking to migrate its paper-based pilot and engineer examinations to computer. There was a need to find an item-banking solution to support exam items generated and maintained on a Europe-wide basis. This item banking solution would need to support mandated quality processes. There was also a requirement for an exam delivery and channel solution that could meet needs particular to the aviation industry - large and very small venues, often remote venues, infrequent testing and global partners in multiple time-zones.

A further need identified by CAA International was that international clients were increasingly expecting pilot licensing exams to be available on computer, meaning that a CBT solution had become a pre-requisite for CAA International to continue winning business globally.

Finding a solution to meet these requirements also involved:

QA of the legacy item bank prior to migration to the chosen computer platform.

Project management of implementation.

Creation of ‘As Is’ business process maps and accompanying documentation for all test- related functions to support change management at CAA.

Review and refinement of CAA International’s Quality Systems, including documentation to achieve ISO9001 certification for all business functions.

Designing a model for the strategic management of testing services across CAA, including in its licensing and commercial functions.

Our Solution

WCAL has supported the CAA and CAA International since 2008. Our support has reflected a commitment to work with clients over the long-term, as well as reflecting an understanding that for many organisations, choosing and implementing a CBT solution involves substantial organisational change and this takes time to manage effectively. Such change can be highly risky and can require patient and confident steps, designed to deliver a successful journey. The steps that WCAL has put in place with CAA include the following.

Selection of an Item Banking and Exam Delivery Solution

WCAL supported CAA's procurement by undertaking a 'cold-eyes' expert review of CBT proposals that had been submitted by vendors to CAA. This review subsequently led to involvement in the interview, evaluation and selection process of vendor's proposed item-banking solutions, including the extent to which proposed solutions could support the CAA's item-management and form creation processes. WCAL's review also covered financial, risk and proposed implementation plans. Based on this, WCAL provided a written evaluation and recommendation to CAA.

QA review of the Item Banking and Implementation Project Management

Following selection of a vendor, it was decided that CAA International would undertake the first implementation of the system for a government client in Malaysia. Achieving this involved project management of a QA process and teams of SMEs to review and approve all legacy paper-based items as they were migrated to the chosen item bank solution. WCAL also provided project management between CAA International, their vendor and the agreed Implementation Plan for Malaysia.

Change Management

As part of the wider organisation-wide changes at CAA, WCAL created a comprehensive suite of Visio Process Diagrams, showing the then current paper-based exam business processes. The process of interviewing staff to create those process maps in itself led to a number of identified improvements. Longer-term, the process mapping was used to identify and control changes that would be required within the exams and licensing systems as part of the migration to CBT.

Alongside the 'As Is' documents, WCAL created a Target Operating Model (TOM) for CAA and CAA International, providing governance arrangements and control across all exams-related functions.

ISO9001

CAA International sought further assistance and support from WCAL to achieve organisation-wide accreditation against the ISO9001 standard. CAA International had already achieved this accreditation for their training functions, but now wished to extend it to a comprehensive Quality Plan covering all exams and business related functions. WCAL reviewed CAA International's Quality Plan and all process documentation, reviewed the document library and business processes against the ISO9001 standard. This work extended through accreditation audits undertaken by BSI, helping CAA International achieve full accreditation.

The Result

CAA and CAA International have now successfully navigated their transition to CBT. Their partnership with ASPEQ, the chosen CBT vendor, has enabled CAA to open a large, bespoke CBT Test Centre at their main office building in Gatwick. CAA International is now competing effectively for government and pilot licensing clients internationally on the basis of the CBT solution they are able to offer.