Research and analysis

The CASLO Approach

Published 18 November 2024

Applies to England

Authors

Paul E. Newton, Milja Curcin, Latoya Clarke, and Astera Brylka, from Ofqual’s Standards, Research, and Analysis Directorate.

The CASLO Research Programme

This report is part of a series that arose from Ofqual’s 2020 to 2024 programme of research into the CASLO approach:

1. The CASLO Research Programme: Overview of research projects conducted between 2020 and 2024.
2. The CASLO Approach: A design template for many vocational and technical qualifications in England.
3. How ‘CASLO’ Qualifications Work. (This was published in February 2022.)
4. Origins and Evolution of the CASLO Approach in England: The importance of outcomes and mastery when designing vocational and technical qualifications.
5. Responding to Criticisms of the CASLO Approach (Report A): A taxonomy of potential problems.
6. Responding to Criticisms of the CASLO Approach (Report B): Views from awarding organisations.
7. Responding to Criticisms of the CASLO Approach (Report C): Views from qualification stakeholders.
8. Responding to Criticisms of the CASLO Approach (Report D): Properties of qualifications from the CASLO research programme.
9. Understanding Qualification Design: Insights from the 2020 to 2024 CASLO qualification research programme.

The CASLO approach

This report answers questions like these:

• what characterises the CASLO approach to qualification design?
• how does it contrast with other approaches to qualification design?
• why might a qualification designer decide to adopt the CASLO approach?

It draws together insights from a series of reports that present findings from Ofqual’s 2020 to 2024 CASLO research programme, focusing on those insights that seemed most pertinent to characterising the approach.^{[footnote 1]}

We can think of the CASLO approach as a high-level template for designing qualifications. It is not restricted to Vocational and Technical Qualifications (VTQs), but it became increasingly popular as a basis for designing VTQs in England during the 1980s and especially into the 1990s. We contrast it with the classical approach to qualification design, which lies at the heart of most traditional tests, exams, and qualifications in England (and overseas), which we will describe later. We refer to qualifications that have been designed on the foundation provided by the CASLO template as CASLO qualifications. We will explain the origin of the acronym shortly.

An example of a CASLO qualification

The easiest way to introduce the CASLO approach is by example, and we have chosen a Level 2 NVQ in plastering for this purpose.^{[footnote 2]} Table 1 presents an extract from a unit specification from this NVQ in plastering, which comprises 6 mandatory units and 4 optional ones (from which 1 must be chosen). The example unit is titled ‘Applying finishing plaster to prepared surfaces in the workplace’ and it is specified as requiring 60 Guided Learning Hours.

The core characteristics illustrated in Table 1 relate to how the content of the unit is specified in terms of learning outcomes (in the first column) and how standards for the unit are specified in terms of assessment criteria (in the second column). All of these learning outcomes need to be acquired (to the specified standards) in order for the learner to pass the unit.

Learning Outcomes	Assessment Criteria
The learner will:	The learner can:
1. Interpret the given information relating to the work and resources when applying finishing plaster to prepared surfaces.	[not reproduced here]
2. Know how to comply with relevant legislation and official guidance when applying finishing plaster to prepared surfaces.	2.1. Describe their responsibilities regarding potential accidents, health hazards and the environment whilst working: in the workplace, below ground level, in confined spaces, at height, with tools and equipment, with materials and substances, with movement/storage of materials and by manual handling and mechanical lifting. 2.2. Describe the organisational security procedures for tools, equipment and personal belongings in relation to site, workplace, company and operative. 2.3. Explain what the accident reporting procedures are and who is responsible for making reports.
3. Maintain safe and healthy working practices when applying finishing plaster to background prepared surfaces.	[not reproduced here]
4. Select the required quantity and quality of resources for the methods of work to apply finishing plaster to prepared surfaces.	[not reproduced here]
5. Minimise the risk of damage to the work and surrounding area when applying finishing plaster to prepared surfaces.	5.1. Protect the work and its surrounding area from damage in accordance with safe working practices and organisational procedures. 5.2. Minimise damage and maintain a clean work space. 5.3. Dispose of waste in accordance with current legislation. 5.4. Describe how to protect work from damage and the purpose of protection in relation to general workplace activities, other occupations and adverse weather conditions. 5.5. Explain why the disposal of waste should be carried out safely in accordance with environmental responsibilities, organisational procedures, manufacturers’ information, statutory regulations and official guidance.
6. Complete the work within the allocated time when applying finishing plaster to prepared surfaces.	[not reproduced here]
7. Comply with the given contract information to apply finishing plaster to prepared surfaces to the required specification.	[not reproduced here]

Table 1.         Example of how units tend to be specified within CASLO qualifications

The set of learning outcomes specified for any particular CASLO unit indicates what a learner who has passed the unit will (therefore) know, understand, or be able to do. For qualifications like this NVQ in plastering, the units, and the learning outcomes within them, describe distinguishable elements of an overarching competence. So, part of being able to plaster is being able to ‘apply finishing plaster to prepared surfaces in the workplace’ (which is the title of the unit from Table 1). And the set of learning outcomes that is specified for this unit explains what we mean by being competent in this aspect of plastering, while the assessment criteria explain what having acquired the learning outcomes (to a satisfactory standard) actually looks like in practice.

By way of example, if a learner provides us with credible evidence that they can ‘explain what the accident reporting procedures are and who is responsible for making reports’ (assessment criterion 2.3) then we will have gathered part of the evidence that we need in order to conclude that the learner ‘knows how to comply with relevant legislation and official guidance when applying finishing plaster to prepared surfaces’ (learning outcome 2).

This approach to specifying learning outcomes and assessment criteria is common to all CASLO qualifications, and CASLO units are typically specified using exactly the same list-like format as is illustrated in the table.

Core characteristics

In order to capture the nature and scope of our research programme (circa 2020) we realised that we needed a label to describe qualifications of this sort. The literature offered various alternatives, but none captured exactly how these qualifications presented in England at that point in time. For instance, we could have described them as:

• outcome-based qualifications (but we wanted to indicate that they comprised a distinct subset of outcome-based qualifications)
• mastery-based qualifications (but we wanted to indicate that they comprised a distinct subset of mastery-based qualifications)
• competence-based qualifications (but many did not actually assess competence in the sense that this term is often used in relation to VTQs in England)

Considering the range of qualifications that we wished to capture, we concluded that they all shared 3 core (and therefore, for our purposes, defining) characteristics:

1. unit content is specified in terms of learning outcomes

2. the unit standard is specified via assessment criteria for each learning outcome

3. to pass each unit, a learner must acquire all of the specified learning outcomes, which we refer to as the mastery requirement ^{[footnote 3]}

Although the approach is best characterised at the unit level, designers typically adopt the approach wholesale, that is, across all units, hence the idea of a CASLO qualification.^{[footnote 4]}. Because qualifications of this sort are designed to Confirm the Acquisition of Specified Learning Outcomes, we decided to call them CASLO qualifications.

We will return to this point later, but it is worth noting at the outset that CASLO qualifications do not have to be limited to a single passing grade. There are all sorts of ways in which CASLO qualifications can be adapted to award higher grades, and it is not uncommon for CASLO units to award merit and distinction grades, which are then aggregated to an overall qualification grade.

The CASLO approach

It is important to emphasise that the designation ‘CASLO’ does not describe a type of qualification, in the sense that ‘type’ tends to be used in relation to regulated qualifications in England. So, it is not like the designation ‘A level’ or ‘Functional Skills Qualification’ for which Ofqual has type-specific rules and guidance.

The designation ‘CASLO’ captures a far broader family of qualifications, which includes NVQs, many Applied General qualifications – including many BTECs, which played an important role in the history of the CASLO approach – and many of the qualifications that used to be regulated under the Qualifications and Credit Framework (QCF). This is why we describe it as an approach to qualification design, which has underpinned some very differently conceived qualifications since it was introduced during the 1980s.

We have already described the CASLO approach as a high-level template for designing qualifications. And, later on, we will describe it more formally as a qualification model. The important point is that the approach transcends particular qualification types, embracing a very diverse array of qualifications. Having said that, CASLO qualifications often have certain ancillary features in common, and we will explore these later on too.

Roots of the CASLO approach

To appreciate why CASLO qualifications share the common characteristics they do, it is important to understand the historical roots of the approach. These roots are both outcome-based (associated with the idea that clearly stated educational objectives underpin effective curriculum planning) and mastery-based (associated with the idea that effective teaching focuses upon mastery learning). Neither of these ideas is inherently more relevant to vocational and technical courses than to general (academic) ones. Yet, they both resonated with the intentions of VTQ designers (from the 1970s onwards) who were keen for qualifications to certificate practical competence. It is not incidental that the idea of mastery certification resonates with the idea of certificating full occupational competence.

Roots

The roots of the CASLO approach can be traced back to the first half of the twentieth century, and to the work of North American curriculum theorists. Their scholarship underpinned what came to be known as the Objectives Movement. This movement emphasised the critical importance of clarity when defining and articulating educational objectives. This clarity is reflected, within current CASLO qualifications, in the level of detail that learning outcomes and assessment criteria are specified in. As such, outcome-based qualifications (of which CASLO qualifications are a subset) are conceptually rooted in objectives-based curriculum theory.

Outcomes (objectives)

Ralph Tyler is often described as the father of the Objectives Movement, and the classic statement of this movement is contained within the pages of his relatively short book entitled ‘Basic Principles of Curriculum and Instruction’ (Tyler, 1949). Tyler began this book by identifying 4 fundamental questions that have to be answered when developing a curriculum and plan of instruction:

1. what educational purposes should the school seek to attain?

2. what educational experiences can be provided that are likely to attain these purposes?

3. how can these educational experiences be effectively organised?

4. how can we determine whether these purposes are being attained?

His starting point was an observation that many educational programmes did not have clearly defined purposes, and the focus of his first chapter was how to define purposes clearly. Hence the idea that curriculum planning must begin by specifying educational objectives, which then become the single point of reference for planning, delivering, and evaluating teaching and learning. Assessment (that is, evaluation) was a fundamental consideration for Tyler. Educational objectives needed to be specified in a manner that was amenable to assessment, to make it possible to determine whether or not they had actually been achieved.

Tyler paid particular attention to the form in which educational objectives ought to be written, to provide a suitable point of reference. He took issue with the traditional format – the list of syllabus content – which simply stated the areas or topics that teachers and assessors were expected to cover. This left entirely unclear what learners were supposed to ‘do’ with that content – recall it? understand it? apply it? or what? In other words, it left the learning objectives underdetermined. Worse still, human nature being what it is, this format tempted teachers and assessors to default to the lowest common denominator – typically recall – which reduced the likelihood of higher-level skills actually being taught or assessed. He therefore recommended that each educational objective ought to be specified along 2 dimensions: first, content area (for example, nutrition, digestion, and so on) and, second, cognitive behaviour (for example, understanding of facts and principles, ability to apply principles, and so on).

Mastery

Benjamin Bloom was mentored by Ralph Tyler at the University of Chicago. His famous ‘Taxonomy of Educational Objectives’ (Bloom, et al, 1956) can be understood as an extension of Tyler’s work. During the 1960s and into the 1970s, he extended the logic of the Objectives Movement in a slightly different direction – that of mastery learning – which became an educational movement in its own right.

His starting point was the observation that even when school teachers went to considerable lengths to define and clearly articulate educational objectives, they still often failed to ensure that those objectives were actually achieved. So, why was this? He lay much of the blame for this failure on the tradition of educational testing in the USA, which had dominated assessment practices since the early decades of the twentieth century. The guiding principle of educational testing, from a psychometric perspective, was to differentiate effectively between students, and this had led to a situation in which ‘grading on the curve’ was widespread. In other words, teachers began each school year with the expectation that around a third of students would perform really well, another third would perform satisfactorily, and the remainder would fail or just scrape by. Bloom took exception to this presumption. He believed that, with satisfactory instruction and sufficient time, the vast majority of students could be brought to the point of having mastered all intended learning outcomes.

Once again, clarity in defining and articulating intended learning outcomes was critical to the success of mastery learning. So, too, was rigorous evaluation of student progress in their journey toward domain mastery. Indeed, Bloom was also a father of formative assessment, although he described his approach as ‘formative evaluation’ during the 1970s (Bloom, et al, 1971).

It is important to appreciate that the conception of mastery at the heart of the CASLO approach – and therefore at the heart of our research programme – is heavily influenced by Bloom: not just philosophically (the belief that most students are capable of achieving mastery) but also pragmatically (in terms of how mastery is operationalised). Operationalising mastery, from this perspective, involves:

• characterising the domain of learning (that is to be mastered) in terms of its principal learning objectives, that it, its intended learning outcomes ^{[footnote 5]}
• treating mastery of the domain of learning as a matter of mastering all of the intended learning outcomes that comprise it ^{[footnote 6]}
• explicating mastery of each individual learning outcome via explicit criteria that can be used by both teacher and student to gauge progress towards mastery

Since Bloom popularised the idea of mastery learning, during the 1970s, it has been appropriated in many different ways. So, it is worth noting that this particular conception does not necessarily foreground features or processes that might be central to other formulations.

Resonance

The ideas of outcome-based curriculum design and mastery-based teaching and learning began to resonate with educationalists in England during the 1970s. This included organisations with responsibility for curriculum and examinations, in the wake of frequently voiced concerns over the narrowing effect of the exam format.

England has had an enduring love-hate relationship with the exam format ever since it began to work its way into mainstream education during the nineteenth century. One of the most frequently voiced concerns is that exams have a tendency to target lower-level skills (like recall) at the expense of higher-level ones (like application and evaluation). This concern was raised in the Norwood report on ‘Curriculum and Examinations in Secondary Schools’ (Norwood, 1943), and subsequently in the Haslegrave report of the ‘Committee on Technician Courses and Examinations’ (Haslegrave, 1969). Both reports recommended greater reliance on internal (school-based or college-based) assessment approaches, to help assess the skills that exams often failed to reach.

Related to this issue, it is important to recognise that exam syllabuses in England during this period (1940s to 1960s) were minimally specified. They tended to embody the traditional syllabus content list format, which Tyler had warned against, owing to its likely negative backwash impact on teaching, learning, and assessment. Insights from the North American Objectives Movement would therefore have resonated with qualification designers in England, from the 1970s to the 1990s, as they embarked upon successive qualification reforms.

One of the motivating factors behind successive qualification reforms of this period was a legacy of wastage (dropping out of courses), retardation (delay in completing courses), and failure (failing exams and not receiving a qualification). This was particularly true in the context of traditional, college-based VTQs. Compounding this, the economic recession of the 1970s had led to a situation during the 1980s in which technical and vocational education and training provision needed to flex to accommodate the needs of a new wave of learners with low levels of attainment. As such, insights from the North American Mastery Movement would no doubt also have resonated with qualification designers in England at that time.

Origins and evolution

Our analysis concluded that NVQs, which were introduced during the late-1980s, were the first CASLO qualifications of national prominence in England. However, there were many important precursors in the landscape of technical and vocational education and training during the 1960s, and particularly during the 1970s, epitomised by the introduction of TEC and BEC awards.

TEC and BEC awards

The Haslegrave report recommended a radical overhaul of technician-level qualification systems, both for industry and for commerce. This included recommending that 2 new bodies be established to coordinate provision: the Technician Education Council (TEC) and the Business Education Council (BEC). They reconceptualised the provision of technician-level qualifications, operating as ‘validating’ rather than examining bodies. This involved specifying qualification structures for TEC and BEC awards, and to some extent qualification content too, while handing over primary responsibility for programme development, delivery, and assessment to centres. Both the TEC and the BEC required that qualification content be specified in accordance with insights from the Objectives Movement.

The following example illustrates how the TEC achieved this, by specifying unit content in terms of ‘general objectives’ which functioned like learning outcomes (see 7 below) and ‘specific objectives’ which functioned more like assessment criteria (see 7.1 to 7.4 below). The example outlines the second topic area of a modified standard unit in Electronics:

B CATHODE RAY TUBE

7 Knows the principles of operation of a cathode ray tube.

7.1 Labels a diagram of a C.R.T.

7.2 Explains the functions of the following:

(a) electron gun

(b) focus control

(c) intensity control

(d) blanking pulses

7.3 States that deflection can be produced by electric and/or magnetic fields.

7.4 Demonstrates the use of timebases and of vertical and horizontal deflection controls.

(Hunter, 1985, page 285)

Notice how this goes beyond a traditional syllabus content list by identifying what learners are expected to be able to ‘do’ with each item of content: label, explain, state, demonstrate, and so on. Although this approach is clearly a precursor to the CASLO approach, with prototypical outcomes and criteria, both the TEC and the BEC operated the mastery principle fairly loosely, in contrast to the stringent application of mastery that we associate with the CASLO approach.

National Vocational Qualifications

The introduction of TEC and BEC awards, under the control of these 2 new coordinating bodies, was an attempt to ‘rationalise’ what policy makers often described as a ‘jungle’ of vocational and technical courses and qualifications. The introduction of NVQs, under the control of a new coordinating body, the National Council for Vocational Qualifications (NCVQ), was an even bolder attempt.

The NCVQ created a new qualification framework, the NVQ framework, to which it anticipated accrediting the full range of VTQs in England. Critically, it specified accreditation requirements in considerable detail. This included the idea that all NVQs ought to be based upon standards of occupational competence, and that a statement of occupational competence ought to comprise 3 levels of detail:

1. the NVQ title

2. units of competence

3. elements of competence (learning outcomes) with associated performance criteria (assessment criteria)

NCVQ accreditation criteria effectively specified that all NVQs had to adopt the CASLO approach. Importantly, detailed guidance on NVQ development, which was based upon a methodology known as functional analysis, insisted that NVQ learning outcomes – known as elements of competence – had to be specified in a very particular manner. Rather than framing outcomes in terms of the elements of knowledge, skill, and understanding that underpinned competent performance, the outcomes were specified directly in terms of elements of competent performance. This meant specifying outcomes purely in terms of what performing competently actually looked like, that is, in terms of the competencies that collectively comprised competent performance in the workplace. This is an example of an element of competence (EoC) and associated performance criteria (PC) taken from one of the original NVQ guidance documents (TA, 1988):

EoC  The candidate should be able to originate formats for a data base file.

PC1  The file and record structure is appropriate for the sorting and retrieval specifications and requirements.

PC2  The file and record structure maximises the use of permanent and temporary memories within the parameters of the specification.

PC3  The data file is capable of implementation within the limitations of the software and hardware available.

This approach proved to be highly controversial from the outset. In theory, it merely left implicit that which a traditional ‘book knowledge’ syllabus would traditionally have foregrounded, that is, the knowledge and understanding that underpinned competent performance. In practice, however, many interpreted it as a rejection of the importance of underpinning knowledge and understanding. The NCVQ defended its approach for a long time, miring NVQ rollout in controversy for years. Ultimately, though, NVQ outcomes came to be specified more conventionally, in terms of what learners need to know and understand, as well as what they need to be able to do.

Subsequent CASLO qualifications

Although the NCVQ anticipated that all VTQs would eventually migrate to the NVQ framework, this did not actually happen. This was basically because of how prescriptive its accreditation criteria were, which would have turned all VTQs into competence-based qualifications along the lines just discussed. However, while the idea of specifying outcomes purely in terms of elements of competence had limited appeal, the more general idea of specifying qualifications in terms of learning outcomes and assessment criteria – and the associated idea of expecting learners to achieve all specified learning outcomes – proved to be more attractive. It influenced the design of a variety of qualifications, from General National Vocational Qualifications (GNVQs) to Open College Network (OCN) awards. Notably, when the TEC and the BEC merged to become the BTEC, the new organisation continued to specify its awards in terms of objectives (outcomes). During the early 1990s, BTEC awards adopted the CASLO template wholesale, and BTECs were to become the most prominent brand of CASLO qualification in the regulated VTQ landscape.

During the mid-2000s, the Qualifications and Curriculum Authority, which had replaced the NCVQ in 1997, introduced a new qualification framework, the Qualifications and Credit Framework (QCF), to which it anticipated accrediting the full range of VTQs in England. The QCF also specified accreditation requirements in considerable detail, including the requirement that all qualifications needed to adopt the CASLO approach.

By the mid-2010s, the vast majority of VTQs in England had been accredited to the QCF, which meant that the vast majority of regulated qualifications came to adopt the CASLO approach. Incidentally, but importantly, because such a variety of qualifications eventually migrated to the QCF, the core characteristics of the CASLO approach were now embedded within some very differently conceived qualifications. So, the CASLO family became an extremely diverse one.

The QCF was withdrawn in 2015, and Ofqual no longer requires any regulated qualification to adopt the CASLO approach. There are fewer CASLO qualifications nowadays than a decade or so ago, although it is unclear exactly how many currently adopt the approach (as this information is not centrally collated).

Classical versus CASLO

The classical approach to qualification design still lies at the heart of most tests, exams, and qualifications in England (and overseas). It is strongly associated with academic disciplines, such as history, biology, or English literature. And it is the approach that England’s university-derived examining boards have adopted since they began to certificate school learning back in the mid- to late-nineteenth century.

Historically, the clearest guide to what these academic exams assessed – and therefore what students would need to learn in order to do well in them – was the set of ‘past papers’ for the exam in question. This is nicely illustrated within documents from 1858 reproduced by the Cambridge Assessment Archives Service. In 1858, the University of Cambridge provided exams for non-university students at Junior (age 16) and Senior (age 18) level. The Junior scheme was divided into 3 parts, with different disciplines examined (in different ways) across those parts. For Part 2, students chose from 10 discipline-based exam options (choosing at least 3 but no more than 6). The last of these options was Zoology and Botany, and the 1958 regulations specified the content of these exams – in full – as follows:

Elementary questions will be set on the description and classification of Animals, their habits and geographical distribution; and on the mercantile and industrial uses of animal products:

Also on the description and classification of Plants, their uses and geographical distribution:

British plants and parts of plants will be given for description.

(Cambridge Assessment Archives Service, 2008, page 8)

The exam was administered on Wednesday 15 December 1958, from 6pm to 8pm. It comprised 8 mandatory questions on Zoology, similar in structure to question 2:

Which Vertebrates are oviparous; which are abranchiate; and which have gills during a period only of their existence? What mammals have the simplest kind of teeth?

(Cambridge Assessment Archives Service, 2008, page 55)

In addition were 7 mandatory questions on Botany, similar in structure to question 4:

Describe a perfect flower of an Exogenous plant, and shew by a diagram the position of its several parts.

(Cambridge Assessment Archives Service, 2008, page 56)

A century later, past exam papers were still the clearest guide to what students needed to learn in order to do well in them, although accompanying syllabus documents also helped to clarify this. For example, the University of Cambridge Local Examinations Syndicate 1957 syllabus for O level biology spanned 6 pages, covering 8 topics: mammalian physiology, the features, habits, movements, and life history of 4 animals, insects, amoeba and hydra, flowering plants, plant physiology, microscopic plants, ecology. These were specified – again, in full – like this:

7 MICROSCOPIC PLANTS. Structure, nutrition, and life-history of Spirogyra and Mucor (or other mould fungus).

8 ECOLOGY. The relation of plants and animals to their environment based on studies of

(a) aquatic habitats (seashore between the tide marks or ponds or streams or marshes);

(b) terrestrial habitats (grassland or lawns or gardens or hedgerows or woods).

The point of this extended example is to illustrate how the classical approach to qualification design – epitomised by examining board practices from the mid-nineteenth century through to the mid-twentieth century – revolved around the exam itself, supplemented by information concerning the topics that exam questions might be set on.^{[footnote 7]} In other words, intended learning outcomes were specified only by implication, and even then only fairly sketchily. This approach risked teachers not teaching, and even examiners not examining, all of the important dimensions of knowledge, skill, and understanding that comprised the intended domain of learning.

The CASLO approach is fundamentally an attempt to mitigate risks associated with the classical approach to qualification design. Intended learning outcomes are made explicit, and are described in a comprehensive and authentic fashion, to provide a single point of reference for planning teaching, learning and assessment. The idea of comprehensiveness indicates a desire to characterise the intended breadth of the targeted learning domain as thoroughly as possible, so that important learning outcomes do not get overlooked. The idea of authenticity indicates a desire to characterise the intended depth of the targeted learning domain, outcome by outcome, to eliminate ambiguity concerning how learners are expected to engage with the subject matter.

Core characteristics

We can now compare the core characteristics of CASLO qualifications with the details of how classical qualifications work:

1. CASLO unit content is specified primarily in terms of outcomes that need to be learnt, whereas classical qualification content is specified primarily in terms of topics (themes, concepts, facts, or suchlike) that need to be taught

2. CASLO unit standards are explicit and atomistic – specified in terms of discrete assessment criteria for each learning outcome – whereas classical qualification standards are more implicit and more holistic

3. to pass a CASLO unit, a learner must acquire all of its specified learning outcomes, which we refer to as the mastery requirement, whereas classical qualifications do not make requirements concerning specific learning outcomes

In terms of learning, CASLO qualifications are more exacting than classical ones. To pass a CASLO qualification, learners have to achieve all of the specified learning outcomes, period. CASLO certification therefore requires consistently satisfactory achievement across the entire domain of learning. Classical certification, on the other hand, makes no such requirement. Students who just manage to pass a classical qualification will tend to have what is often referred to as a ‘spiky’ profile of achievement across the domain of learning: they are likely to perform less than satisfactorily in certain areas, and more than satisfactorily in others, which averages out to a just satisfactory level of performance overall.

Students who are studying for a CASLO qualification will be focused on explicit performance targets throughout their course of learning.^{[footnote 8]} Conversely, students who are studying for a classical qualification are, in effect, invited to perform as well as they feel capable of performing throughout the course. And the fact that many students end up with a spiky profile of achievement across the domain of learning is anticipated (if not necessarily desired).

In terms of assessment, these contrasting learning expectations translate into different assessment approaches. First, CASLO qualification standards are defined analytically, outcome by outcome. In other words, the qualification standard ultimately resides at the level of the individual learning outcome. Classical qualification standards are defined less explicitly, and more holistically, in terms of overall attainment levels. So, the classical qualification standard ultimately resides at the level of the overall mark total.

Second, but relatedly, CASLO qualifications apply a conjunctive aggregation principle, which means that students need to reach the standard for each and every learning outcome independently. In other words, failing to achieve even just one learning outcome means failing to achieve the qualification. Conversely, classical qualification standards apply a compensatory aggregation principle, which means that what counts is how well the student performs on average. In other words, a really exceptional achievement in one area of the domain can compensate for a really poor achievement in another.

Third, the fact that CASLO qualifications require all learning outcomes to have been achieved means that assessors need to assess each and every learning outcome. In other words, the domain of learning has to be assessed exhaustively, that is, in full. Conversely, the fact that classical qualification standards are holistic and compensatory reduces the perceived need to assess absolutely everything. Assessment within classical qualifications typically involves sampling from across the domain of learning. This assumes that it is legitimate to generalise from how well learners perform on a subset of learning outcomes to how well they have achieved overall.^{[footnote 9]} Sampling enables the assessment process to be much less burdensome.

Cross-fertilisation and hybridisation

We noted that the CASLO approach was fundamentally an attempt to mitigate risks associated with the classical approach to qualification design, which tended to revolve around meagre syllabus content lists and past papers. Nowadays, classical qualification syllabuses (now known as specifications) tend to include much more detail, and their aims and objectives are far more clearly specified. Indeed, current specifications even include assessment objectives, which perform a similar function to learning outcomes in the sense of clarifying expectations concerning the nature and depth of the learning that needs to be demonstrated. That said, assessment objectives lack the detail associated with learning outcomes, functioning more as a tool for calibrating assessment standards than as a tool for planning teaching, learning, and assessment. In short, although the gulf between classical and CASLO qualifications is less extreme nowadays (than it would have been back in the 1970s) there is still an important distinction to be drawn. Annex 1 provides an example of how documents developed for the current suite of A levels provide far greater insight into intended learning outcomes than syllabus documents from the 1970s or earlier.

In addition, it is important to appreciate that the 2 approaches are not mutually exclusive. It is possible to incorporate CASLO units within otherwise classically designed qualifications, and vice versa. For instance, A level practical science assessments essentially function as CASLO components within classical qualifications. Similarly, BTEC awards that were redesigned to comply with Department for Education (DfE) requirements for Performance Table Qualifications typically incorporated classical units alongside CASLO ones.

Hybridisation of this sort is not entirely unproblematic. It raises philosophical questions for CASLO qualifications that only apply the concept of mastery learning to certain units. It also raises pragmatic questions, including the viability of running entirely different quality assurance models within the same qualification, especially as effective quality assurance for CASLO units is resource intensive.

Features of CASLO qualifications

The following subsections extend our characterisation by noting some of the features and processes that are often, but not always, associated with the CASLO approach. After considering the general idea of increased potential for flexible delivery, we will explore assessment, teaching and learning, and quality assurance, before considering whether it is possible to characterise the archetypal CASLO qualification.

Flexibility

Despite the fact that the CASLO approach incorporates tightly specified learning outcomes and assessment criteria, one of the characteristics that is often associated with the approach is its potential for flexible delivery, in relation to both assessment and teaching and learning.

Flexible assessment

In relation to assessment, the potential for flexibility mainly derives from the clarity, or transparency, that is achieved by specifying learning outcomes and assessment criteria in great detail. This helps to scaffold the assessment process by making explicit – for teachers, trainers, learners, and assessors alike – both the nature of the evidence that will need to be elicited and the basis upon which it will need to be judged.

The potential for flexible assessment was intended to be a major selling point for both NVQs (which were designed with workplace delivery in mind) and for early BTECs (which were designed with college delivery in mind). Rather than having to satisfy assessment requirements at a fixed point in time, typically at the end of a course of learning, the transparency provided by explicit learning outcomes and assessment criteria meant that learning could be assessed at any point. At one extreme, an NVQ candidate might offer themselves for assessment without having followed any formal course of learning at all. At the other extreme, a BTEC candidate might be assessed continuously throughout their college course.

Flexibility of this sort means that some learners may be able to satisfy qualification requirements earlier than others. More importantly, it means that learners who need additional time to satisfy qualification requirements may straightforwardly be given it (rather than being forced to sit an end-of-course exam at a fixed point in time, which they are very likely to fail).

Another major selling point for NVQs, in particular, was the process of Recognition of Prior Learning (RPL), which enabled previously achieved competencies to count towards a formal qualification. This was intended to be attractive for existing employees who had gained considerable knowledge, skills, and understanding on-the-job, yet who lacked a formal qualification. Having to complete a course of learning from scratch under these circumstances would not be very efficient. Instead, by providing evidence of prior learning – against clearly articulated sets of learning outcomes – they were able to shortcut the qualification process and certificate much earlier.

More generally, assessment against clearly articulated sets of learning outcomes has the potential to decouple certification from mode of learning. In other words, whether or not learning occurs formally in college or informally at work, if it can be demonstrated that the specified learning outcomes have been acquired to requisite standards then this learning can be certificated.

The potential for flexibility also arises in relation to the format that is used to elicit assessment evidence, for example, direct observation of performance in the workplace versus a written exam. Indeed, for certain CASLO qualifications, there are no constraints over how learning outcomes are assessed, just as long as they are assessed according to the appropriate assessment criteria.

Flexible teaching and learning

Some of the flexibility just discussed in relation to assessment applies in exactly the same way to teaching and learning. Most obviously, not having to conduct assessments at fixed points during the year enables the learning programme to flex accordingly. This helps to accommodate learners who need to commence and complete courses at different points in the year – hence the idea of Roll-On-Roll-Off (RORO) delivery, which can be facilitated by adopting the CASLO approach, as well as by qualification unitisation more generally. The idea of achieving a qualification in steps also helps to accommodate the fact that sometimes workplace demands require learning to be put on pause for a period of months, or perhaps even years. Once again, flexibility of this sort was seen as a major selling point for qualifications like NVQs and early BTECs.

In relation to teaching and learning, however, the potential for flexibility also derives from the level of specificity in which learning outcomes and assessment criteria are articulated. For instance, presented below are learning outcomes 2 and 3 from unit 4 of the GQA Qualifications Level 2 NVQ in Fenestration Installation (which was one of the ‘exemplar’ qualifications that we studied as part of our research programme). The title of this unit is ‘Identify and confirm installation requirements in glass and related work’ and it comprises 5 learning outcomes in total:

LO 2     Know how to establish the type, location, characteristics and features of the installation.

AC 2.1  Explain how to establish the type and location of the installation work.

AC 2.2  Explain how to identify the characteristics and features of the site.

AC 2.3  Explain how characteristics, features and other conditions can affect the way the installation is carried out.

LO 3     Be able to examine the materials and components to ensure they meet the specification of the installation.

AC 3.1  Examine the materials and components and confirm that they meet the specification of the installation.

AC 3.2  Inspect the materials and components for damage prior to installation.

The point is simply that these learning outcomes and assessment criteria are written at a level of specificity that can accommodate a range of different work environments, allowing for their adaptation to different contexts. So, although installation requirements might differ from one conservatory to the next in terms of structural details and materials used – aluminium, timber, plastic, and so on – the assessment criteria are sufficiently general to accommodate variation of this sort.

The potential to accommodate this kind of variation is especially useful for situations in which learners tend to specialise in different areas, for example, if workers in certain circumstances only dealt with aluminium installations, or only dealt with plastic ones. However, although the CASLO approach potentially supports personalisation of this sort, it does not necessarily presume it, and can straightforwardly reduce or eliminate it. For instance, sometimes learning outcomes are accompanied by a specification of the full range of contexts across which competence needs to be demonstrated, for example, across aluminium, timber, and plastic installations (specifying that a candidate needs to be competent across all of the specified contexts).

Assessment

The only assessment requirement to follow directly from the 3 core characteristics is the expectation of exhaustive (all-encompassing) assessment. That is, if a CASLO qualification is to certify mastery of all specified learning outcomes, then it stands to reason that all students will need to be assessed in relation to each individual outcome. Or, in other words, no sampling of learning outcomes. This contrasts with the classical approach, where the idea of sampling across topic areas lies at the heart of the assessment process.

Beyond this stipulation, there are features and processes that CASLO qualifications tend to be disposed towards, and features and processes that CASLO qualifications tend to be disposed against.

Disposed towards

Internal assessment

The fact that CASLO qualifications need to be assessed exhaustively is one of the main reasons why they tend to rely heavily upon internal (also known as centre-based) assessment. This might involve a teacher or trainer if the assessment takes place in college, or it might involve the student’s supervisor or manager if the assessment takes place at work. Reliance upon internal assessment is essentially a pragmatic issue. The larger the number of specified learning outcomes, the larger the assessment burden, and the less viable it will be to assess all of them externally. The burden of assessing all specified learning outcomes tends to be compounded, in vocational and technical settings, by the additional burden associated with assessing practically oriented learning outcomes in real-life or simulated scenarios.

There is, however, another important pragmatic justification for relying heavily upon internal assessment. Even if a CASLO qualification were to be assessed entirely externally (in whatever format or variety of formats) centres would still want to assess all candidates exhaustively prior to this, to be certain that they were ready to pass. Remember that the stakes of external assessment are very high for CASLO qualifications – fail on just one learning outcome and you fail the entire qualification. The fact that (exhaustive) external assessment would essentially duplicate (exhaustive) internal assessment raises questions concerning its cost-effectiveness, particularly if we can be confident in the validity and integrity of internal assessment processes. This idea is developed further in the section, below, on quality assurance.

Phased or continuous assessment

The justification for spreading the burden of assessment across a course of learning – rather than concentrating it all at the end – makes sense pragmatically, but it also makes sense in terms of keeping students and teachers or trainers engaged.

It has often been said that certain learners – particularly adults who are returning to education after having experienced limited success in the past – benefit from being assessed on a step-by-step basis, whether at the unit level (phased assessment) or at the learning outcome level (continuous assessment). The idea, here, is that each small successful step can be formally recognised by the assessment process, which is both rewarding and confidence building.

In fact, phased or continuous assessment has the potential to benefit all learners, particularly when qualifications are premised upon mastery certification. Mastery certification presumes mastery teaching and learning, which requires close attention to the progress that each learner makes towards achieving each of their learning outcomes from each of the units within the qualification they are studying. Because learners have to master all specified learning outcomes to pass the qualification, it makes sense for any lack of learning to be exposed and rectified in real time, as they are progressing along their learning journey, through ongoing formative assessment.

Phased or continuous assessment also facilitates effective external quality assurance, which can be scheduled to occur during the course of learning rather than exclusively at the end. This enables false positive judgements to be identified before the end of the course, which provides an opportunity for remedial teaching and learning to take place. In other words, where an external quality assurer spots that an internal assessor has erroneously judged a learner to be competent, when they have not yet met the relevant criteria, it is helpful if this does not occur right at the end of the course when the learner is ready to progress to another destination. In situations like this, a student might simply decide to progress without satisfying the necessary criteria, thereby sacrificing the qualification. If, instead, at least a significant chunk of the external quality assurance occurs prior to the end of the course, then this risk of non-completion is significantly mitigated.

Centre-devised assessment

As already noted, the CASLO approach helps to scaffold the assessment process by detailing the nature of the assessment evidence that needs to be elicited and the basis upon which it needs to be judged. This makes it easier for assessors to devise their own assessment tasks, selecting from a variety of assessment formats:

• a direct observation of performance in a real-world context

• a direct observation of performance in a simulated context

• an oral exam or discussion

• a practical exam

• a written assignment

• a multiple-choice test

• another assessment format, or

• a combination of assessment formats

Whereas classical qualifications tend to utilise a small number of formats, and all students tend to be assessed in the same format, it is easier to loosen these strictures for CASLO qualifications. This offers the potential to adopt a much wider range of formats across the learning outcomes that comprise the qualification. This even offers the potential for learners to be assessed on the same learning outcome using different formats, to bolster confidence in the assessment judgement. For instance, if a learner appeared to demonstrate limited understanding during a direct observation of real-world performance, this might be followed up by a professional discussion to establish whether or not they genuinely understood the situation.^{[footnote 10]}

Exceptions

Although the CASLO approach disposes a qualification toward internal assessment, either phased or continuous, and although its design helps to facilitate centre-devised assessment, none of these characteristics is entirely necessary. It is quite possible for a CASLO qualification (particularly a small one with relatively few learning outcomes) to be assessed entirely externally and terminally. Indeed, some are. It is even more common for CASLO qualifications to be based on externally devised, but internally assessed, end-of-unit assignments.

Disposed against

External written tests

The CASLO approach is not theoretically opposed to external written testing. However, it:

• is opposed to assessing only a sample of learning outcomes, and it

• does require learning outcomes to be assessed (as having been achieved) independently of each other (and assessed reliably at this level)

Both of these requirements militate against relatively short written tests. In addition, the likelihood of failing to demonstrate competence on even just a single learning outcome – which would mean failing the qualification – militates against terminal external written testing. The risks and impacts of failure are too high, including the risk of erroneous failure attributable to assessment error. Where terminal external written tests are incorporated within otherwise CASLO qualifications, they therefore tend to be based on the classical approach.

To design a written test strictly according to the CASLO approach, each learning outcome would require its own mini test – with its own pass-fail boundary – and each mini test would need to be long enough to ensure that its results were sufficiently reliable. A test of this sort might well involve mark aggregation, with each student’s test result depending on their overall mark total. The pass-fail grade boundary for each mini test would need to be set fairly high – for example, at around 70% of the total available marks, if not higher – to warrant a claim that the targeted learning outcome had genuinely been mastered.

Sampling

Although the CASLO approach is inherently opposed to sampling learning outcomes for assessment purposes – because it requires evidence of the acquisition of all specified learning outcomes – it is not necessarily opposed to sampling at lower levels. For instance, any particular learning outcome is likely to present somewhat different demands across different contexts. Although we would expect a learner to be able to demonstrate their competence across a variety of contexts – and we might even stipulate mastery across a specified range of contexts for the purpose of assessment – it would never be possible to assess a learning outcome exhaustively across each and every conceivable context. So, there is always likely to be an element of sampling within any CASLO assessment.

Possible features

Grading

Because of the association with mastery learning (and the idea of having to master each and every specified learning outcome) it might seem that the CASLO approach is inherently disposed against grading. Yet, a recent investigation into regulated VTQs in England suggested that grading is fairly common within CASLO qualifications (see Newton, 2018). Having said that, it is fair to say that the approach is most closely associated with pass-fail qualifications, particularly qualifications that are designed to certify occupational competence. In this instance, the assessment criteria tend to be designed to indicate minimum competence to practice safely and effectively. 

In fact, learning outcomes are almost never binary, in the sense of being totally absent or fully formed. They almost always come in degrees. As such, there is no logical barrier to specifying criteria at multiple levels for the same learning outcome – pass criteria, merit criteria, and distinction criteria, for instance. This opens the way to awarding unit and qualification grades.

Some CASLO qualifications do exactly this. They specify three sets of criteria for each learning outcome – one set for pass, another set for merit, and another for distinction – so candidates are graded directly at the learning outcome level and may only be awarded a higher grade if they have satisfied all of the criteria for lower ones (in keeping with the spirit of the CASLO approach).

Other CASLO qualifications adopt a different approach to awarding higher grades, while still adopting the CASLO approach to determine the pass-fail outcome. For instance, some qualifications award merit and distinction grades holistically, on the basis of all of the evidence presented (permitting an element of compensation across the learning outcomes).

Synopticity

The idea that qualifications ought to incorporate synoptic assessment tasks became prominent in England during the early-1990s, as a mitigation against the risk of fragmented learning within modular A level schemes. Modular A level syllabuses were required to incorporate final year synoptic units that were designed to confirm that students had acquired an holistic grasp of the subject, grasping connections between different elements of the subject.

More recently, in response to the 2011 Wolf Report on vocational education, the DfE introduced design requirements for VTQs that were to be counted in performance tables. These requirements included synopticity: to confirm that students were able to apply, in an integrated way, an appropriate selection of skills, techniques, concepts, theories and knowledge from across the whole vocational area. This was intended to mitigate the risk of fragmented learning, particularly within CASLO qualifications where the domain of learning is deconstructed not simply into units but also into discrete learning outcomes. Where CASLO qualifications were reformed to comply with these requirements, this often meant introducing a classically designed, terminal synoptic unit alongside existing (or new) CASLO units.

Although the risk of fragmented learning is real – particularly in relation to the CASLO approach – it is entirely possible to build synopticity into CASLO qualifications without having to adopt a classical approach. This can be achieved using end-of-unit assignments that require learners to demonstrate their ability to integrate learning outcomes from across the whole unit (unit-level synopticity). Similarly, the terminal units of some CASLO qualifications require leaners to demonstrate their ability to integrate learning outcomes from across the entire qualification (qualification-level synopticity).

Teaching and learning

It is important to note that our definition of a CASLO qualification is independent of any model of teaching, learning, or curriculum organisation. In other words, there is nothing in the definition that presumes a particular model, and the CASLO approach is compatible with a variety of different models. The only obvious exception to this is that it would be extremely unwise to adopt the CASLO approach to assessment, which involves mastery certification, without also simultaneously adopting a mastery approach to teaching and learning. 

In fact, the CASLO approach has been associated with a wide variety of approaches to teaching and learning over the decades. During the 1980s and 1990s, BTECs became associated with a particular philosophy of teaching and learning, which made heavy use of projects and assignments, team working, work-related problem solving, active involvement in own learning, as well as stimulating and personalised teaching and learning strategies. This contrasted starkly with NVQs, for example, which were marketed as being independent of any particular mode (let alone philosophy) of teaching and learning.

Programmes and support materials

The distinction between classical qualifications and CASLO ones has sometimes been described as a distinction between ‘input-based’ and ‘outcome-based’ approaches. The classical approach revolves around ‘inputs’ to teaching, epitomised by the syllabus content list, whereas the CASLO approach revolves around ‘outcomes’ from learning, that is, learning outcomes. This distinction is unfortunate, however, as it suggests that CASLO qualifications are somehow opposed to ‘inputs’ like syllabuses, teaching and learning programmes, support materials, or suchlike. This is not true. The CASLO approach simply insists that the proper starting point for planning curriculum, pedagogy, and assessment is a comprehensive and authentic statement of learning outcomes. It is on this foundation that a sequenced and coherent programme of teaching and learning will need to be devised. CASLO qualification awarding organisations sometimes (although not always) provide materials to support teachers and trainers with their instructional planning, as we noted in relation to certain of the ‘exemplar’ CASLO qualifications that we studied.

Similarly, all regulated qualifications (including CASLO qualifications) are required to take into account to the likely prior learning of learners from their targeted cohort, which is a different kind of input-related consideration. This includes specifying Guided Learning Hours and Total Qualification Time, and many CASLO students enrol on sessional courses that are designed to accommodate these specifications.

Integrated system

Because CASLO qualifications are based upon a very stringent conception of mastery – fail to achieve a single learning outcome from a single unit and you fail to achieve the qualification – it becomes critical to ensure that learners remain on track. This is why we emphasise that mastery certification necessitates mastery teaching and learning. It is also why CASLO qualifications need to be understood as integrated teaching-learning-assessment systems, that is, the mastery certification requirement necessitates a mastery approach to teaching and learning.

It is not incidental that the father of mastery learning, Benjamin Bloom, was also a father of formative assessment. Ultimately, the structure of assessment within the classical approach means that even students with significant gaps in their learning can still pass the qualification. Because the CASLO approach is designed to prevent this from happening, gaps in learning need to be exposed, and closed, if learners are to pass. Hence the centrality of formative assessment to the CASLO approach, particularly for learners who struggle to monitor their own learning.

Quality assurance

Our first published report on the CASLO approach investigated how CASLO qualifications work, through the lens of quality assurance (Newton & Lockyer, 2022). This recognised that:

• many CASLO qualifications were entirely internally assessed, and

• the approach to quality assuring internally assessed components of classical qualifications (mark-based moderation adjustment) would not work for internally assessed units of CASLO qualifications (where evidence is judged directly against assessment criteria)

This raised the question of how CASLO qualification awarding organisations managed to remain fully accountable for the awards they make, despite devolving a substantial amount of responsibility for assessment processes to centres. In other words, how do they manage effective external quality assurance when there is potentially so much assessment evidence to quality assure, and without being able to implement mark-based moderation adjustment? What became apparent is that CASLO qualification external quality assurance needs to be far more intensive, extensive, and ‘hands on’ than tends to be the case for classical qualifications, which means that awarding organisations need to work closely in partnership with their centres.

This partnership model suggests that quality assurance is a process, and even a relationship, that is built and honed over time. It involves supporting centres to become self-regulating systems, by internalising the awarding organisation’s quality standards and quality assurance practices. The expertise and integrity of the (lead) internal quality assurer is key to this process, as the quality assurance lynchpin. Whereas moderation, for classical qualifications, tends to focus squarely upon the accuracy and consistency of assessment judgements, CASLO quality assurance is likely to have a broader purview, encompassing the effective delivery of a qualification in the round – teaching, assessment, internal quality assurance, and management too – consistent with the idea of sustaining an integrated teaching-learning-assessment system.

The archetypal qualification

In our report on how CASLO qualifications work, we wondered whether it was possible to describe the ‘archetypal’ qualification, bearing in mind how much diversity the CASLO family embraces. In fact, beyond the 3 core characteristics discussed earlier, we struggled to identify additional characteristics that could paint a useful picture of the archetypal CASLO qualification. The most plausible candidates seemed to be a continuous internal assessment approach, alongside a flexible delivery model, both of which have been described in detail above. We suggested that the archetypal qualification would have been designed with flexibility in mind to accommodate learners studying under a variety of circumstances, for instance:

• learners following different pathways within the same qualification

• learners developing alternative (customised) skillsets within the same qualification

• learners studying in different locations (college-based, work-based, and so on)

• learners demonstrating their competence in different workplace settings

• learners starting and finishing their learning on different timelines

This raises an interesting question concerning the degree to which the CASLO approach is especially attractive for accommodating the needs of somewhat divergent learner cohorts on somewhat diverging learning journeys. This is not unrelated to the highly pragmatic issue of qualification delivery viability, for relatively small cohorts of learners with somewhat idiosyncratic needs and aspirations.

Reflecting on our attempt to describe the archetypal CASLO qualification (with a couple more years of research under our belts) we note that there are dangers in overstating the similarity of CASLO qualifications. It has become clear to us that there are some very different approaches to qualification design within the overarching CASLO family – including some that incorporate a particular philosophy of teaching and learning and others that are explicitly agnostic – and blurring differences of this sort risks obscuring their differing strengths, weaknesses, opportunities, and threats.

Qualification models

We will end this characterisation of the CASLO approach by noting that it is best understood as a qualification model. The idea of a ‘model’ is a useful way of describing the CASLO approach for a number of reasons, but particularly because:

1. its core characteristics constitute a high-level template, according to which all CASLO qualifications are built

2. what is distinctive about this approach is that it attempts to provide a comprehensive and authentic model of the proficiency that the award of the qualification is intended to certificate ^{[footnote 11]}

This second point contrasts sharply with the classical approach to qualification design – especially as it was implemented prior to the 1980s – where the target proficiency is modelled only implicitly, through syllabus content lists, exam paper mark schemes, and suchlike.^{[footnote 12]}

Just as with any modelling process, the model provided by a CASLO specification is not a perfect fit to reality. It would be naïve to think that any human proficiency could be fully characterised by a list of learning outcomes and associated assessment criteria. In the academic literature, this inevitable misfit between CASLO specifications and the constructs that they are presumed to represent has been a source of considerable controversy. Yet, debate of this sort risks overlooking the fact that the fundamental purpose of a model is to simplify the world, which means that models are at least somewhat inaccurate by design. Instead, the critical questions concern whether the model is sufficiently accurate to be fit for the purposes for which it is used, and whether the benefits that accrue from its use outweigh disbenefits. As such, debate ought really to focus upon pragmatic utility rather than philosophical purity.

The CASLO approach embodies a very particular way of modelling the target proficiency that corresponds to mastering a domain of learning. This involves:

1. deconstructing the overarching proficiency into component elements (units)

2. deconstructing those component elements a step further (into learning outcomes)

3. typically making considerable use of ‘command verbs’ to help explain the nature and depth of the required learning (these are terms like ‘state’ or ‘describe’ or ‘explain’ that often feature within assessment criteria)

4. treating proficiency as a conjunctive concept, which assumes that being fully proficient means having acquired the full range of component proficiencies (the mastery principle)

This approach is not uncommon, internationally speaking, and it has been the dominant approach to designing regulated VTQs in England since the 1990s. However, it is important to remember that this is only one way of modelling the proficiency that we wish learners to acquire, and that it has weaknesses as well as its strengths. Its main strength lies in its representation of breadth and depth of learning.^{[footnote 13]} Its main weakness lies in leaving implicit the structure of proficiency within a domain of learning, that is, in not explicitly representing how the various elements of competence integrate with each other when an individual is practicing proficiently within the domain.

What that means is the CASLO approach is just one approach to qualification design alongside other approaches (including, but not limited to, the classical approach). In particular, there are other ways of operationalising an outcome-based approach, and there are other ways of operationalising a mastery-based approach. So, although the CASLO approach has come to dominate the landscape of regulated VTQs in England over the past 3 or 4 decades, we should not assume that it is the only way of designing an outcome-based or mastery-based qualification, nor even necessarily the best one. We return to this issue in the final report of our research programme.

References

Bloom, B.S., Englehart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of Educational Objectives: The classification of educational goals. Handbook I: Cognitive domain. London: Longmans, Green and Co Ltd.

Bloom, B.S., Hastings, J.T., & Madaus, G.F. (1971). Handbook on Formative and Summative Evaluation of Student Learning. New York: McGraw-Hill Book Company.

Cambridge Assessment Archives Service (2008). 1858 Question Papers: The University of Cambridge Local Examinations Syndicate. Cambridge: Cambridge Assessment.

Haslegrave, H.L. (1969). Report of the Committee on Technician Courses and Examinations. London: HMSO.

Hunter, D.W. (1985). The Teaching of Electronics in Schools and Further Education: A case study in curriculum change. PhD thesis. Durham: University of Durham.

Newton, P.E. & Lockyer, C. (2022). How ‘CASLO’ Qualifications Work. Ofqual/22/6895. Coventry: Office of Qualifications and Examinations Regulation.

Newton, P.E. (2018). Grading Vocational & Technical Qualifications: Recent policies and current practices. Ofqual/18/6441/3. Coventry: Office of Qualifications and Examinations Regulation.

Newton, P.E. (2022). Demythologising A level exam standards. Research Papers in Education, 37(6), 875–906.

Norwood, C. (1943). Curriculum and Examinations in Secondary Schools. London: HMSO.

TA (Training Agency) (1988). Development of Assessable Standards for National Certification. Guidance note 2: The Definition of Competences and Performance Criteria. Sheffield: The Training Agency.

Tyler, R.W. (1949). Basic Principles of Curriculum and Instruction. London: The University of Chicago Press Ltd. London.

Wolf, A. (2011). Review of Vocational Education – The Wolf Report. London: Department for Education.

Annex 1: A level learning outcomes

Classically, that is, prior to the 1970s, qualification syllabuses in England typically took the form of lists of topics that students were likely to be examined on (themes, concepts, facts, and suchlike). These syllabuses provided no information concerning the nature of the learning that students were expected to acquire: neither its depth, nor its structure, nor its application.

The introduction of outcome-based qualification design was a response to this perceived shortcoming. It was based on the assumption that effective planning for curriculum, pedagogy, and assessment necessitates the prior specification of an explicit and complete model of the learning that a qualification is supposed to certificate – which we refer to as a proficiency model – that can then be translated directly (and consistently) into implications for teaching, learning, and assessment. For CASLO qualifications, these models were specified in terms of learning outcomes, assessment criteria, and the mastery principle.

Although classically designed qualifications have continued to dominate the general education landscape, they too have evolved in response to this perceived shortcoming. They now explicate learning outcomes in far greater detail than they used to. This explication occurs across a variety of official documents, including:

• Department for Education (DfE) subject content

• Ofqual subject-level conditions and requirements (and associated guidance)

• awarding organisation specifications (syllabuses)

We will discuss each of these in turn – in the context of A level business – before discussing the exemplification of standards at A level.

Subject content

Subject content for the most recent suite of A level subjects was published by the DfE from 2014 onwards. These subject content documents set out, at a national level, the ‘aims and objectives’ for each A level subject, as well as core ‘subject content’ (that any specification in the subject area would need to incorporate). Although all DfE subject content documents include both of these sections, they are not all written in exactly the same style, and this includes how they refer to learning outcomes. The following example from A level business is fairly typical though.

In its ‘aims and objectives’ section, the DfE states that A level specifications in business “must encourage students” to:

• develop an enthusiasm for studying business
• gain an holistic understanding of business in a range of contexts
• develop a critical understanding of organisations and their ability to meet society’s needs and wants
• understand that business behaviour can be studied from a range of perspectives
• generate enterprising and creative approaches to business opportunities, problems and issues
• be aware of the ethical dilemmas and responsibilities faced by organisations and individuals
• acquire a range of relevant business and generic skills, including decision making, problem solving, the challenging of assumptions and critical analysis
• apply numerical skills in a range of business contexts

These aspirations include outcomes that will not be assessed directly (such as enthusiasm) and outcomes that will (such as applying numerical skills). Importantly, these outcomes are not certification requirements, but design principles. Thus, specifications are intended to “encourage” their acquisition.

The ‘subject content’ section of this document goes on to specify that the “knowledge, understanding and skills” set out in A level business specifications must:

• enable students to investigate different types and sizes of organisations in various business sectors and environments, with an awareness of local, national and global contexts, recognising that they face varying degrees of competition
• enable students to:
  • identify business opportunities and problems
  • investigate, analyse and evaluate business opportunities and problems
  • apply numerical skills, including those indicated in the Annex
  • make justifiable decisions using both qualitative methods and quantitative methods, including those indicated in the Annex
  • identify and understand how businesses adapt to operate successfully in a dynamic business environment
• promote an holistic understanding of business and enterprise
• apply basic business concepts to a wider range of contexts
• understand and apply more sophisticated concepts and techniques
• analyse, interpret and evaluate more complex business information
• take a more strategic view of business opportunities, problems and issues

It then goes on to specify 8 “areas of study” to be applied across a range of types and sizes of business organisations in different contexts (each of the following 8 areas is elaborated with a number of bullet points):

1. Business objectives and strategy

2. Marketing

3. Accounting and finance

4. People in organisations

5. Operations management

6. External influences

7. Business analysis

8. Change

Finally, its annex provides a list of “quantitative skills” that are deemed necessary for acquiring the specified knowledge and understanding in business:

• calculate, use and understand ratios, averages and fractions
• calculate, use and understand percentages and percentage changes
• construct and interpret a range of standard graphical forms
• interpret index numbers
• calculate cost, revenue, profit and break-even
• calculate investment appraisal outcomes and interpret results
• interpret values of price and income elasticity of demand
• use and interpret quantitative and non-quantitative information in order to make decisions
• interpret, apply and analyse information in written, graphical and numerical forms

Conditions, requirements, and guidance

Ofqual’s conditions and requirements for A level business incorporate the DfE’s subject content. Critically, from the perspective of proficiency modelling, they also include a statement of assessment objectives for A level business specifications:

AO1	Demonstrate knowledge of terms, concepts, theories, methods and models to show an understanding of how individuals and organisations are affected by and respond to business issues	20 to 30%
AO2	Apply knowledge and understanding to various business contexts to show how individuals and organisations are affected by and respond to issues	20 to 30%
AO3	Analyse issues within business, showing an understanding of the impact on individuals and organisations of external and internal influences	20 to 30%
AO4	Evaluate quantitative and qualitative information to make informed judgements and propose evidence-based solutions to business issues	20 to 30%

These objectives indicate the intended weighting of outcomes of different kinds within the assessment (in particular, the relative weighting of cognitive behaviour categories like knowledge, analysis, and evaluation). Ofqual’s subject-level guidance documents include further insight into the anticipated breakdown of the assessment objective requirements for each subject.

Specifications

In addition to fleshing out the DfE subject content in greater detail – and presenting additional content bespoke to the qualification in question – awarding organisation specifications often provide additional information of relevance to modelling intended learning outcomes. For instance, they often provide further information concerning the distribution of assessment objectives across assessment components, like this table, which has been adapted from the OCR A level business specification:

Component	AO1	AO2	AO3	AO4	Total
Operating in a local business environment	6 to 8	7 to 9	8 to 10	8 to 10	33.3%
The UK business environment	9 to 11	7 to 9	6 to 8	7 to 9	33.3%
The global business environment	6 to 8	8 to 10	9 to 11	7 to 9	33.3%
Total	25%	25%	25%	25%	100%

This specification is useful from a comparative perspective because it breaks the qualification content specification into detailed topic lists (nested within broader subject content areas) and then breaks down each topic list in terms of “what the learners should be able to do as a result of following the OCR A Level in Business”. This makes it look much like a CASLO qualification specification. For instance, 16 topics are listed within the “Business objectives and strategy” content area, and the following extract details the outcomes (or perhaps criteria) associated with 2 of these 16 topics:

Business plan (learners should be able to):

• explain the purpose of a business plan
• describe the main contents of a business plan and explain why these are included in the plan
• evaluate the advantages and disadvantages to a business of having a business plan
• evaluate the impact and importance of a business plan to the stakeholders of a business
• explain what is meant by the ‘Plan-Do-Review’ cycle
• analyse how the Plan-Do-Review cycle can improve a business’ performance
• evaluate the impact and importance of a strategic review to a business and its stakeholders.

Risk and uncertainty (learners should be able to):

• explain what is meant by risk and reward
• explain the relationship between risk and reward
• evaluate the relationship between risk and reward in a business
• distinguish between quantifiable and unquantifiable risk
• evaluate quantifiable and unquantifiable risks to a business
• evaluate ways in which a business can reduce its level of risk
• evaluate the consequences to a business and its stakeholders of poor risk management
• evaluate the specific risks faced by an entrepreneur
• explain what is meant by uncertainty
• distinguish between internal and external causes of uncertainty in a business
• evaluate the impact of internal and external causes of uncertainty on a business and its stakeholders
• explain how uncertainty can affect a business including effects on aims and objectives, planning, decision making, business analysis and forecasting

The OCR specification lists 105 content areas in total for A level business, each linked to a sizeable set of bullet-point outcome-criteria (minimum 1, maximum 28). These outcome-criteria are preceded by “learners should be able to” although this is presumably not intended to imply that, having studied the relevant content, all A level business students should necessarily be able to demonstrate each and every one of the outcomes listed for each of the 105 topic areas. Note, importantly, that this would be the implication if this were a CASLO qualification. Moreover, there would be an expectation that each learner should be assessed on, and should pass the assessment for, each of these outcome-criteria.

Exemplification

CASLO standards reside at the level of each individual learning outcome, and are stated explicitly in terms of associated assessment criteria. A level standards, conversely, reside at the level of the overall mark total that a student achieves across all of the assessment components for a qualification. This means that A level standards are holistic rather than atomistic. Furthermore, they tend to remain implicit rather than explicit, as A level standards are neither described nor exemplified at the national level.

Technically speaking, A level standards are ‘carried forward’ from one year to the next (and by extension from one decade to the next) via complex statistical and judgemental procedures (see Newton, 2022). These are employed to provide some guarantee that students who achieve the same overall level of attainment in a subject area would be awarded the same subject grade regardless of which year they took their exams.

Although, currently, these subject grade standards are not formally explicated, they have been formally explicated in previous years. For instance, the document that preceded the current Ofqual conditions and requirements for A level business included a table that provided “grade boundary performance descriptions” for the A/B threshold and the E/U threshold, broken down for each assessment objective. Thus, the A/B boundary for the first objective (‘demonstrate knowledge and understanding of the specified content’) indicated that students on the A/B grade boundary characteristically:

a. demonstrate precise knowledge and understanding of key business concepts drawn from the [year 2] specification, with few omissions

b. show knowledge and understanding of the links between business concepts

Clearly, these grade descriptions are quite vague and, in practice, they played only a nominal role in the grade awarding process. This helps to explain why they were discontinued.

Overview

The design of classical qualifications evolved over time, particularly during the 1980s, to provide greater insight into their intended learning outcomes. This was true for both GCSE and A level qualifications. Yet, in this General Qualification context, the principal rationale for explication was to improve the comparability of demands across awarding organisations when offering essentially the same qualification (for example, A level business). So, at least during the early years, this enhanced modelling was more for the benefit of assessment planning, by awarding organisations, than for the benefit of planning curriculum and pedagogy. Perhaps, though, the more detailed explication of learning outcomes within recent General Qualification specifications – as illustrated above for OCR A level business – may suggest a trend towards wider usage in certain contexts nowadays.^{[footnote 14]}

Ironically, the modelling of outcome-criteria within the OCR specification appears to be even more extensive than might be the case for a typical CASLO qualification. Yet, it is important to emphasise that it does not serve the same purpose. In the CASLO context, the core proficiency model translates directly into implications for teaching, learning, and assessment. In the OCR A level business context, the same would not be true, and there is no presumption that all of the outcome-criteria would need to be mastered in the sense that is central to the CASLO approach.

In conclusion, although qualifications like GCSEs and A levels model learning outcomes more explicitly and completely than used to be the case, it is still fair to say that they continue to follow the classical tradition, as their modelling serves different purposes and is less fundamental to qualification delivery. This raises an interesting question for further research concerning the nature and uses of proficiency modelling within classical qualifications.

1.  As such, further details, including the evidence and analysis that backs up this account, can be found in the other reports (which are listed prior to the contents page of the present one). ↩

2.  The details of this example were taken from an actual regulated qualification (circa 2024): the NOCN_Cskills Awards Level 2 NVQ Diploma in Plastering (Construction) – Solid (603/2368/1). Its Total Qualification Time was specified as 870 hours (which means that a learner might reasonably be expected to complete the qualification in this much time from a plausible baseline of prior attainment). ↩

3.  For many CASLO qualifications, the mastery requirement also operates at the level of assessment criteria. That is, a learner must satisfy all of the criteria for each learning outcome in order to be judged to have achieved it. ↩

4.  This also suggests that CASLO qualifications tend to be segmented into units, which is true, although the idea of a single-unit CASLO qualification is entirely legitimate. ↩

5.  Bloom described this as a “model of outcomes” (Bloom, et al, 1971, page13). Following Tyler, he recommended representing it as a table of specifications, which recorded ‘content’ on one axis and ‘cognitive behaviour’ on the other. ↩

6.  This seems to be consistent with his belief that: “The art of teaching is the analysis of a complex final product into the components which must be attained separately and in some sequence.” (Bloom, et al, 1971, page 13). ↩

7.  We use the term ‘classical’ to indicate that it is the ‘traditional’ or ‘standard’ approach. We do not mean to imply that it is the ‘definitive’ or ‘highest quality’ approach. ↩

8.  Where CASLO units award grades, faster students may end up focusing on higher-level targets, while slower students end up focusing on lower-level ones. But they will all be focused on explicit performance targets throughout, and they will all be expected to perform consistently well throughout to be awarded a pass or a higher grade (Newton, 2018). ↩

9. To have confidence in the generalisability of results, the sampling needs to be sufficiently representative and sufficiently large. ↩

10.  There are pros and cons associated with flexibility of this sort, so its validity and utility cannot be taken for granted, its risks need to be mitigated, and it needs to be employed skilfully.  ↩

11.  We use the term ‘proficiency’ here, although we could equally have used the term ‘competence’ or the more technical term ‘construct’ to describe the body of learning that is acquired when mastering a domain of learning. ↩

12.  Nowadays classical qualification specifications describe topic lists in far greater detail than in previous decades, and they also make some attempt to model their target proficiency, for example, via assessment objectives. Yet, this modelling still tends to be partial, and it serves different purposes from those served by CASLO qualification learning outcomes and assessment criteria (see Annex 1). ↩

13.  It is perhaps better at representing breadth than depth because, although command verbs are useful for indicating depth, they are fairly blunt tools for this job. ↩

14.  Having said that, it is important to appreciate that this is not a universal feature of current A level specifications, for instance, the AQA A level business specification does not identify learning outcomes in the same way as the OCR specification. Indeed, most of the current suite of OCR A level specifications are not presented in the same level of detail. ↩