Evaluation of the Community Organisations Cost of Living Fund: Full Report

Question 1

Executive summary

Accepted Answer

Programme overview

The Department for Culture, Media and Sport (DCMS) launched the Community Organisations Cost of Living Fund (CCLF or ‘the Fund’) in July 2023 as part of the Voluntary, Community and Social Enterprise organisation (VCSE) Cost of Living (CoL) Programme. The CCLF was part of a wider package of support under the programme, alongside the VCSE Energy Efficiency Scheme.

The CCLF was a £76 million package of funding for frontline VCSE sector organisations in England facing increased demand for their services and increased delivery costs. The CCLF was administered by an intermediary grant maker (IGM), The National Lottery Community Fund, which provided grants of between £10k and £75k to eligible organisations. The main aim of the CCLF was to ensure that organisations delivering critical frontline services were able to maintain delivery or increase in volume and/or expand the scope of their activity to meet increased demand.

Approach to the evaluation

Ecorys and Dr Diarmuid McDonnell of Braw Data Ltd were commissioned in October 2023 to undertake an evaluation of the CCLF. The evaluation comprised a mixed-method approach, and included a process, impact and economic evaluation. The overarching objectives of the evaluation were to understand: (1) how the CCLF funding was delivered; (2) what the outcomes and impacts were for beneficiaries, grant holders, and the wider VCSE sector; and (3) what the economic costs and benefits of the CCLF were. In addition, the evaluation aimed to support wider DCMS and His Majesty’s Government (HMG) objectives by building the evidence base in key areas of interest, alongside improving understanding of organisational resilience and VCSE finances, the wider sector ecosystem, and the impact of interventions on end beneficiaries.

Evaluation activities involved several key methods– interviews with programme stakeholders, case studies with VCSEs, and 2 waves of surveys with awarded and non-awarded VCSE applicants. These methods were complemented by Bayesian Process Tracing, v-f-m analysis drawing on an assessment of programme costs and potential benefits, and analysis of programme and analysis of programme management information (MI) data.

Findings: process evaluation

Set-up of the Fund: During the set-up phase of the CCLF, The National Lottery Community Fund was appointed via a direct award approach as the intermediate grant maker (IGM). DCMS also convened a VCSE Cost of Living Advisory Board (comprised of experts from different VCSE organisations / membership bodies) to inform the design and set-up of the programme. Views on the design and set-up phase were generally positive, with stakeholders highlighting the collaborative approach, the partnership structure, the availability and expertise of the IGM, and the use of existing tools and mechanisms as key factors for success.

Design of the Fund: In terms of design, stakeholders and grant holders appreciated the inclusion of the retrospective funding element of the Fund, and 91% of the 316 VCSE survey respondents found the inclusion of retrospective funding costs to be effective. Retrospective funding was considered by many to be an uncommon approach to grant funding but a very useful mechanism for helping organisations to avoid dipping into their reserves. Having the flexibility to use the grant for core costs was also considered as a positive, and almost all VCSE grant holders responding to the survey (98%) felt that how they could use the funding was effective.

Challenges: The main challenges identified related to condensed timescales, some uncertainty amongst some programme stakeholders about the focus of the CCLF (in terms of the types of organisation and services it was intended to be funding), the complexity of designing the retrospective claim element, and the need to design and implement grant management processes in parallel to the design phase. Timescales particularly impacted the design phase, with only around four-months lead in time for designing the Fund; this meant that some design work took place concurrently to the delivery.

Reach of the Fund: The CCLF opened for applications on 24th July 2023 and closed on 18th October 2023. A short (2-week) lead-in time meant that public communications of the scheme only happened once the CCLF had launched. The main advertisement of the scheme was via the organisations involved (The National Lottery Community Fund, DCMS and the Advisory Board members), which placed announcements on their respective websites, alerted organisations on mailing lists (e.g. newsletters), and shared information via social media. The Fund received 3,736 applications from a wide range of VCSEs across different regions, sectors and sizes. However, there were mixed views on the targeting of funding to different organisations and some questions about whether the CCLF should have had quotas set for VCSEs of different characteristics. Additionally, survey data suggested that organisations not linked into the networks listed above may not have heard about the Fund.

Feedback on the application process: Applicants were generally satisfied with different aspects of the application process; only around a tenth of applicants surveyed were dissatisfied with the ease of completing the application form. The application process was not viewed as particularly burdensome given the type (covering a range of core and delivery costs) and amount of funding (up to £75,000) available. The majority of survey respondents from organisations awarded CCLF funding (81%), and over half (58%) of unsuccessful applicants, reported that the amount of time they spent applying to the CCLF was as expected in relation to the amount of funding requested. Some specific challenges with the application process were noted by VCSE strategic staff, including confusion with the budget template, the timescales for the retrospective funding budget, and accessibility and technical issues. However, whilst not unique, these challenges were not found to be widespread.

Assessment of applications: Once applications were received, The National Lottery Community Fund followed a multi-step process to assess and decide on applications, with DCMS involved as an observer. The process was generally viewed as efficient, although it was noted that delays occurred due to the large number of applications submitted just before the deadline. Compared to those awarded funding, VCSEs not receiving grants were more likely to face longer delays in hearing their decision and were more likely to be dissatisfied with the timescales for notification. The National Lottery Community Fund and DCMS decided to prioritise notifying the successful applicants first, given the compressed programme timeline for delivery and spend – this meant that The National Lottery Community Fund could meet their aim of responding to people within 12 weeks. 76% of grant holders responding to the Wave 1 survey were satisfied with the time they had to wait between submitting the application and hearing the decision about the funding; however, qualitative feedback suggested some did encounter challenges, particularly around their ability to plan and effectively budget, especially in relation to retrospective funding.

Funding awarded: Overall, 1,500 organisations were awarded CCLF funding, with a median award of £48,000. This was in line with the number of awards that the CCLF team was aiming for (as highlighted in the Theory of Change developed for the programme and its evaluation). Overall, grant allocation totalled to the planned £70.9 million.

Grant management and delivery: Throughout the programme grant monitoring requirements were kept to a minimum, with most organisations only required to provide an end-of-grant report and a retrospective payment claim form. Most grant holders were satisfied with these requirements and the support from their The National Lottery Community Fund funding officer. However, many grant holders expressed concerns about the short timescales and the lack of flexibility to spend the grant over a longer period.

Findings: impact evaluation

Short, medium and long-term outcomes were experienced by grantee organisations in relation to several key areas: ability to maintain or adapt service delivery to meet demand, financial resilience, and outcomes for staff and volunteers. Whilst presented as separate areas, it is important to note that across the case study research it became apparent that often these outcomes were interrelated for organisations.

Service delivery outcomes: In the short-term, the CCLF funding helped organisations sustain and adapt their services to meet higher demand and increased costs over the winter period of 2023-2024. The survey data suggests that the CCLF positively impacted organisations’ ability to maintain and increase the number and range of services they provided, with a large proportion increasing the number and range of services between March 2023 and March 2024:

42% of survey respondents reported that the number of services they provide had ‘gone up a lot’, with an additional 39% of respondents reporting that the number had ‘gone up a little’.
28% of respondents also reported that the range of their services had ‘gone up a lot’, with 47% reporting it had ‘gone up a little’.

Bayesian process tracing (BPT) drawing on survey results also found that CCLF contributed to organisations maintaining or increasing both the number and range of services they provide. These findings were echoed in the case study research. Staff and volunteers from grantee organisations reported that they provided more services and activities due to the funding. In some instances, using CCLF to hire new staff and volunteers, and to buy additional food and emergency supplies, enabled them to extend their services to new areas. Additionally, some VCSE strategic staff observed that covering staff and volunteer expenses with the CCLF allowed them to remain open for longer hours, or on previously closed days, and to deliver existing services more frequently. Some VCSEs also adapted their services by changing their approaches and ways of working; for example, by introducing ‘drop-in’ sessions, home delivery of food parcels, or gradually moving from a ’food bank’ to a ’community supermarket’.

Through adapting and maintaining service delivery, organisations were able to meet demand for their services in the short-term. Around half (51%) of survey respondents stated that the CCLF funding had a ‘substantial impact’ on their ability to meet demand. Similarly, 53% reported that the CCLF funding had a ‘substantial impact’ on their ability to maintain the number of beneficiaries supported. These findings were supported by the BPT, which reinforced the evidence that CCLF supported organisations to manage increased demand and increase their beneficiary reach, with there being a high chance that CCLF contributed to organisations maintaining and/or increasing the number of service users reached.

Following the funding period, the majority of organisations were able to continue maintaining or adapting their service delivery to meet continued rising demand. Organisations that completed the wave 2 survey reported that both the range of services they offer, as well as the number of services they offer, either increased or stayed the same, with very few seeing a decrease in service number or range. This was true for both successful grant holder organisations and unsuccessful applicant organisations, which also increased the number and range of services during the same period. This suggests that across the sector more widely organisations sought to expand and increase their service delivery.

Financial Resilience: The CCLF funding helped improve the financial position of organisations by enabling them to cover high delivery costs and avoid unplanned use of reserves in the short-term. Some organisations’ strategic and delivery staff agreed that, without the CCLF funding, they would have had to continue investing time and resources into applying for grants and fundraising to help them meet their delivery costs for the winter period. While they would have hoped to secure alternative funding, they acknowledged that there was no guarantee their efforts would materialise and, in that eventuality, some reported they would have been unable to continue delivering all of their services.

The counterfactual impact evaluation (CIE) found that CCLF resulted in a statistically significant increase in total gross income for grant holders with an income below £10 million prior to CCLF. There was also an increase in expenditure for these organisations. CCLF had a statistically significant positive impact on net income for organisations receiving funding. This increase in gross income, being on average close to the average grant award, indicates that CCLF funding was unlikely to have displaced other sources of funding for these organisations (i.e. funded organisations appear to have continued to access other funding at a similar rate to their non-funded counterparts).

However, despite overall positive financial trends observed with CCLF funding – both from survey data and the counterfactual impact evaluation – the process tracing presents only moderate evidence that CCLF has contributed to financial resilience of VCSEs. This is because the BPT focused on the perceived impact of the funding on the risk of insolvency or shrinkage, which might not directly align with the short-term financial improvements captured by the CIE. The BPT may also indicate lingering concerns among these organisations about their long-term financial stability, even with the CCLF funding.

Overall, the interviewed VCSE strategic staff reported that the CCLF improved their short-term financial position, with many highlighting the role of retrospective funding and the unrestricted nature of CCLF in providing a ‘financial cushion’ for the organisations concerned. Some of the VCSE strategic leads also highlighted that the CCLF funding made up a sizeable amount of their income and, in some cases, it was the largest grant they had received.

The CCLF funding also supported organisations to avoid having to make unplanned use of their reserves. In order to meet rising costs, organisations reported having to make unplanned use of reserves to meet delivery costs prior to receiving CCLF funding, with over a third of organisations needing to do this (35%). Furthermore, over half of respondents (52%) reported that, had they not received the CCLF funding, then they might have had to make unplanned use of their reserves. In reality, as a result of receiving funding from the CCLF, 55% of respondents to the survey reported that they did not need to take any cost management decisions, such as unplanned use of reserves, during the funded period.

Although the funding provided short-term relief, many organisations anticipated that they would continue to face financial challenges in the medium to long term and need to make cost-management decisions. The wave 2 survey provided an insight into the wider context facing VCSE organisations following the funding period, with the majority reporting that along with demand continuing to increase, costs had also continued to rise; consequently, organisations largely still remained concerned for their financial situation.

In the face of continuing increased demand and increasing costs, some organisations reported that their organisation’s financial situation was challenging. Around 70% of respondents to the wave 2 CCLF grant holder survey reported that their organisation had to take cost-management decisions between the funding period ending and October 2024. The most common cost management decisions were making unplanned use of reserves (42%), reducing staff hours (22%) and increasing the price of services (14%).

Outcomes for Staff and Volunteers: The funding provided critical support to alleviate pressure on staff, enhance their confidence and motivation, and prevent short-term job losses during the funded period. Numerous organisations succeeded in retaining their personnel and volunteers, with some even expanding their workforce or increasing volunteer recruitment during the funded period. However, following the funded period, in the medium to long-term, case study VCSE organisations reported that financial constraints posed ongoing challenges to staff retention. Nevertheless, the funding enabled organisations to sustain their workforce and strengthen staff capacity.

Outcomes for Beneficiaries: In the short term the funded services addressed the immediate needs of beneficiaries, including but not limited to:

Provision of food and emergency supplies – many grantee organisations provided food in various forms, whilst others provided essential items such as nappies or sanitary products. Beneficiaries who used these services reported that the services helped them feed themselves and their families and meet wider needs.
Safe spaces – Including safe spaces for women, children and young people, the elderly, and migrant communities, as well as emergency shelters for homeless people. These spaces also sometimes provided other facilities such as laundry or cooking facilities to meet other needs beyond shelter.
Financial and housing advice, for example to those who need help with managing debt and finances, or those who have been made homeless. Interviewed beneficiaries highlighted that the financial support they received was instrumental in stabilising their financial situation.

Many grantee organisations provided holistic support to beneficiaries accessing CCLF-funded services by providing further mental and physical health support, employment and education advice, digital inclusion support, or wider family support. Additionally, interviewed staff and volunteers from grantee organisations often reported that they regularly signpost their beneficiaries to other organisations and services to access relevant support. As a result of this support, in the medium to long term, beneficiaries noted enhancements in their mental and physical health, financial stability, and overall well-being. Consequently, some beneficiaries observed longer-term outcomes, such as reduced effects of poverty, improved mental and physical health, and increased access to education and employment opportunities.

Findings: Economic Evaluation

The CCLF cost £75,542,252 over two years (2023/24 and 2024/25). This was an underspend of £427,748 compared with its original estimated cost of £75,970,000. Most of the underspend was due to efficiencies by The National Lottery Community Fund in grant management (i.e. not the grants themselves). The majority of the costs across the Fund related to grant funding, with grants awarded totalling £70,900,000.

Building on the results from the CIE, the economic evaluation showed that the CCLF generated substantial benefits to funded organisations in the form of net income and gross expenditure and income. Using the increases in gross expenditure as an approximation for additional/sustained service delivery, the overall benefits to individuals were high and far exceeded the investment made in the CCLF.

In particular, while not able to be fully quantified and monetised, there are likely to have been notable economic benefits accruing to CCLF end-beneficiaries. To provide an indication of these likely benefits, estimates were calculated using figures derived from the counterfactual impact evaluation to estimate the additional numbers of beneficiaries supported by the Fund for each CCLF ‘service type’. The precise services provided by CCLF funded organisations varied, but were broadly grouped as providing: food and supplies, advice, warmth, safe space, and shelter. Based on a calculation drawing on number of additional beneficiaries receiving this support, the total aggregate value of potential benefits to beneficiaries across the service types is estimated to be £158,987,181. This figure should be treated with some caution as it relies on several assumptions (due to precise data on the numbers of beneficiaries supported with specific services – e.g. food support – not being available and there being a potential for double counting within the estimates produced; hence potentially overestimating the amount of benefits likely to accrue).

Question 2

1.0 Introduction

Accepted Answer

This section provides an overview of the background and context of the CCLF, before providing details on the evaluation aims and objectives, methodology, and limitations. It then sets out the structure of the rest of the report.

1.1 Background and context

As part of the Voluntary, Community and Social Enterprise organisation (VCSE) Cost of Living (CoL) Programme, the Department for Culture, Media and Sport (DCMS) launched the Community Organisations Cost of Living Fund (CCLF or ‘the Fund’) in July 2023. The CCLF was a £76 million package of funding for frontline VCSE sector organisations in England facing increased demand for their services and increased delivery costs.

The CCLF was administered by an intermediary grant maker (IGM), The National Lottery Community Fund (The National Lottery Community Fund), which provided grants of between £10k and £75k to eligible organisations, for existing services that supported low-income households and individuals facing severe challenges due to the increased cost of living. To be eligible, organisations needed to be delivering at least one of the following types of service: food and emergency supplies, emergency shelter, safe spaces, warm spaces, or financial and housing advice. Grants could be used to continue, expand or adapt the delivery of existing services, and could cover a wide range of costs, such as staff salaries, direct costs of service delivery, improvements to systems and infrastructure, small-scale refurbishment and overheads. Funding could be provided for:

‘future costs’ of delivering, expanding or adapting existing critical service(s), from the date funded through to 31st March 2024; and
‘retrospective costs’ of funding the existing critical service(s) delivered between 24^th July 2023 (i.e. the application open date) and the date organisations were funded

The main aim of the Fund was to ensure that organisations delivering critical frontline services were able to maintain delivery or increase in volume and/or expand the scope of their activity to meet increased demand. All funding was to be spent by the end of March 2024.

1.2 Overview of the evaluation

The overarching objectives of the evaluation were to understand: (1) how the CCLF funding was delivered; (2) what the outcomes and impacts were for beneficiaries, grant holders, and the wider VCSE sector; and (3) what the economic costs and benefits of the CCLF were. In addition, the evaluation aimed to support wider DCMS and HMG objectives by building the evidence base in key areas of interest, alongside improving understanding of organisational resilience and VCSE finances, the wider sector ecosystem, and the impact of interventions on end beneficiaries.

At the outset of the evaluation, a programme Theory of Change (ToC) and detailed evaluation plan was developed and agreed with DCMS. The evaluation framework developed to guide the study is included for reference in Annex 1, alongside the ToC in Annex 2. The ToC provides a basis through which to assess the CCLF’s implementation and outcomes. As can be seen in the diagram in Annex 2, the ToC for the CCLF includes three different pathways – one at VCSE organisation/sector level, one at beneficiary level, and one at DCMS level. These different pathways reflect the different levels at which outcomes and impacts were anticipated to occur across the programme.

1.3 Methodology

Evaluation activities comprised several key methods – interviews with programme stakeholders, case studies with VCSEs, surveys of grant holder and unsuccessful VCSE applicants, and analysis of programme management information (MI) data. Details of the implementation of these different strands is provided below.

Figure 1.1: Overview of approach

1.3.1 Programme stakeholder and funding manager interviews

High-level stakeholder interviews were carried out in February and March 2024 with key representatives involved with the CCLF’s design and development. The evaluation team spoke to 19 individuals through a combination of 1:1 and small group interviews. Participants included DCMS staff (6) and members of the Advisory Board ^{[footnote 1]} (representatives from umbrella bodies across the VCSE sector – (7) and the leads from The National Lottery Community Fund (6)). Interviews explored the application process and its implementation, including perceived effectiveness of the assessment process and delivery, and any learning that could be transferred to other funds. The sample of interviewees was agreed in discussion with DCMS.

In addition to stakeholder interviews, two focus groups were held with funding managers from The National Lottery Community Fund. Seven funding managers were consulted in total across the two groups. Participants included those involved in the initial set up and delivery of the CCLF, the sifting of applications, members of assessment panels and those involved with grant management. The focus groups explored the experiences of funding managers working on the CCLF (and, as relevant, any other similar funds), covering their reflections on the application process and the implementation of the CCLF as well as any lessons learned that may be applicable to future funds.

1.3.1 Case studies

To cover implementation and shorter-term outcomes, as well as longer term outcomes and impacts, 25 case studies were planned to be undertaken with grantee organisations in a first wave of data collection, and a further 12 in a second wave of data collection. The first wave of case studies was initially planned to take place at the end of the delivery period for the grant in March 2024. However, due to unforeseen delays, case study research for wave 1 took place between April - June 2024. The second wave of case study research then took place between October 2024 - March 2025. In addition, due to an unforeseen cancellation, one case study in wave 1 was not able to take place and so was carried over to the second wave. As a result, 24 case studies were conducted in wave 1, and 13 in wave 2.

For the first wave of data collection, organisations were sampled based on a framework developed to closely mirror the characteristics of all VCSEs receiving grant funding. A similar approach was taken for the second wave, with additional priorities identified to fill in gaps in knowledge and strengthen the evidence base. A full breakdown of the sample and how this compares to the overall grantee population, can be seen in Annex 3. Within this sample, the evaluation team also ensured that the organisations varied in terms of other characteristics such as how they spent the funding, whether they received retrospective funding, and the target demographic groups they worked with. For the second wave of case study data collection, 6 of the organisations who participated were those that had already participated in wave 1. The focus of these repeat visits was to allow for a longitudinal approach to analysis, and understand in more detail whether and how things had changed for the organisation in the medium-longer term following the end of the grant period. For the remaining 6 new case studies, the focus was on understanding their experiences of the funding, as well as capturing some of the medium-longer term outcomes.

Sampled organisations were largely receptive to participating; however, some felt they lacked the capacity to engage at the time of the research. In these instances, the organisation concerned was replaced with an alternative with similar characteristics. For example, at wave 1 a total of 48 organisations were contacted in order to meet the target number of 25. This meant that identifying organisations and scheduling fieldwork took longer than anticipated. Similar challenges were encountered during the second wave of the fieldwork, likely exacerbated by the longer gap between the fieldwork and the end of the grant period.

Case studies generally included up to 10 interviews as follows: strategic staff member (x1); delivery staff and/or volunteers (x2-3); local stakeholders (e.g. local community organisation representatives, potentially drawn from Boards/trustees; LA leads for adult social care; other statutory service representatives etc. (x2-3); and end beneficiaries (x2-3). Where possible, researchers visiting case study sites also took structured observational notes (e.g. about the premises, staff and volunteers’ interactions with beneficiaries, how busy sites were, etc.). Interviewee numbers and the exact make up of participants at each case study site varied according to what was most appropriate for that organisation. Research was conducted through a combination of online/telephone interviews, as well as face to face research. Again, this was decided based on what was most appropriate for the organisation and individuals involved. For repeat case study organisations in wave 2 of fieldwork, where possible we spoke with the same individuals who took part in the first wave of fieldwork.

Table 1.1 summarises the number of interviews, by participant groups, that have informed the development of this report. Overall, at the time of reporting, 229 interviews have been conducted with a wide range of individuals across different stakeholder groups.

Table 1.1: Case study interview numbers, by participant group

Wave	Number of case studies	Interviews with staff and volunteers	Interviews with local stakeholders	Interviews with beneficiaries	Total
One	24	104	18	77	199
Two	13	50	9	49	108

Source: CCLF evaluation team.

1.3.3 Surveys

Two online surveys were administered at two waves in the fieldwork period. One targeted organisations that received awards, and one targeted those who had applied for the funding but had been unsuccessful.

The first wave of the survey was open for a month between May 2024 and June 2024 and gathered data on organisations’ context, use of funding (for grant holder organisations), financial situation before and after the CCLF, and provision of services. In addition, the grant holder survey gathered data on implementation costs and outcomes for service users. To encourage participation, respondents were entered into a prize draw to win a donation of £1000 to their organisation. The survey for organisations receiving grants achieved 853 responses (a 57% response rate), while the survey for organisations not receiving grants achieved 400 responses (an 18% response rate).

The second wave of the survey was open for a month between November 2024 and December 2024 and questions sought to understand what difference the funding made to their organisation, and the people it provides services to, in the longer-term, and what aspects of CCLF were most important for the organisation. The survey for those who didn’t receive grants focussed on understanding changes to demand for services, their organisation’s financial situation, and reflections on funding needs. Respondents were entered into a prize draw to win one of two donations of £500 to their organisation. The survey for organisations receiving grants achieved 316 responses (a 21% response rate), while the survey for organisations not receiving grants achieved 193 responses (a 9% response rate).

Drawing on the survey data, this report includes relevant descriptive statistics, cross-tabulations, and comparisons between grant holders and unsuccessful applicants where appropriate. In addition, survey evidence was also utilised as part of the impact evaluation through Bayesian Process Tracing analysis (BPT), a statistical technique that provides a systematic and transparent method for evaluating impact ^{[footnote 2]}. Bayesian process tracing assesses how plausible different explanations are for an event by evaluating evidence and updating beliefs using probabilistic reasoning. The approach combines qualitative analysis with Bayesian probability to assess how new evidence strengthens or weakens competing hypotheses, ensuring a transparent and rigorous approach to causal inference. Using the survey results, the process-tracing methodology was employed to assess the overall contribution of CCLF across four contribution claims ^{[footnote 3]}. VCSE survey data were also used to derive key metrics—such as cost-to-income ratios and gross profit—that help define CCLF’s impact. As the focus of the survey data is unique organisations, each organisation can be given a likelihood value that can be aggregated to present findings as relative proportions of different likelihood ranges (see Box 1 for a summary of how to interpret BPT results). More information about this approach can be seen in Annex 4.

Box 1: Interpreting bayesian process tracing results

In this analysis, Bayesian Process Tracing (BPT) assigns probabilities to four competing hypotheses for each organization based on available evidence. The results are then aggregated to show the proportion of organizations where each hypothesis falls within specific probability ranges (in 10% increments).

High confidence (≥70%): A hypothesis strongly supported by the evidence, indicating a clear explanation for the majority of organizations in this range
Moderate confidence (30–69%): Some support for a hypothesis, but alternative explanations remain plausible
Minimal support (<30%): The hypothesis is unlikely given the evidence, and alternative explanations should be prioritized

By examining the distribution of organizations across these probability ranges, we can assess which hypotheses are most supported across cases and where uncertainty remains. In the case of CCLF, this would translate to a strong contribution of the CCLF programme where there is high confidence, and to a weak contribution where there is minimal support.

1.3.4 Analysis of management information data

Management Information (MI) data collected for the CCLF programme was analysed to help understand the characteristics of grant-recipient organisations, including what services, activities and outputs the funding enabled, and whether the funding reached the intended organisations and their beneficiaries. Data was also analysed to support the economic and impact evaluations. The MI data was predominantly drawn from the following data sources:

Applications, which included information about the organisations that applied (e.g. size of organisation and type/sector of organisation) and information about the grants provided (e.g. funding amount, purpose)
Project budgets, which included information about how the funding was intended to be spent
End of grant reports, which included information about how the funding was spent
MI data was also matched with information from the Charity Commission Annual Returns as part of the Counterfactual Impact Evaluation, enabling a comparison of organisational financial health before and after the introduction of CCLF. More details about this approach can be found in Annex 5

1.3.5 Limitations

Every effort has been made to ensure that the evaluation methodology and design is robust, ensuring the accuracy of findings. However, there are a small number of limitations and contextual factors that are important to acknowledge.

As noted, of the initial organisations contacted to participate in the case studies, many reported not having capacity to take part at that point in time. A wide range of reasons were given, including being short staffed, facing a busier period than usual, or only having a small team. This may have led to a small limitation in that all organisations involved were those with the capacity to participate. This limitation likely also affects the survey to some extent
Programme stakeholder interviews and focus groups were conducted early in the evaluation, between February and March 2024. The data from these interviews was used to shape the case study research design, and also fed into the development of the impact evaluation approach (outlined in Annex 4 and 5). However, due to the early point at which they took place, stakeholders were unable to comment on outcomes from the Fund, other than speculatively
The response rate to the second wave of the survey was lower than the first wave, most likely due to the length of time between the survey and the funded period. Limited survey responses and consequently a smaller sample size may mean that results do not reflect the broader population. For process tracing analysis this can lead to bias in the posteriors potentially skewing results ^{[footnote 4]}. However, despite being lower at wave 2, the overall response rates were broadly positive, especially when data could be combined with the first wave meaning the potential for any skewed bias is minimal

1.4 Report structure

The report is structured as follows:

Chapter 2.0 covers the findings from the process evaluation, including a summary of key findings and reflections on what worked well and less well, and any lessons learned
Chapter 3.0 examines the outcomes and impact of the CCLF on VCSE organisations, staff and volunteers, and service users or beneficiaries. This section includes a summary of data relating to different expected outcomes, as well as the findings from the Counterfactual Impact Evaluation (CIE) and Bayesian Process Tracing (BPT)
Chapter 4.0 focuses on the findings from the economic evaluation, including costs to the CCLF, benefits, and an overall assessment of value-for-money
Chapter 5.0 provides final conclusions and reflections on learning to date

Question 3

2.0 Process evaluation

Accepted Answer

The CCLF process evaluation explored how the CCLF was designed, implemented and delivered, reflecting on what worked well, what the challenges were, and what the learning was, to help inform the development of future similar funds. This chapter first discusses the design of the CCLF, then moves onto implementation, examining the perceived effectiveness of the communications of the CCLF, application processes, decision making, and overall reach. It then discusses project delivery, in terms of the types of activity grant holders delivered, as well as ongoing grant management, and what worked well and less well in this.

Chapter summary

Design and Set-up Phase: The CCLF was designed to be launched quickly, with The National Lottery Community Fund appointed as the intermediate grant maker due to their experience and reach. The design phase included workshops and input from a VCSE Cost of Living Advisory Board. Stakeholders appreciated the inclusion of the retrospective funding, the flexibility to use the grant for core costs, and the partnership structure (with the inclusion of the VCSE Advisory Board). However, there were mixed views on the targeting of funding to different organisations and some questions about whether the CCLF should have had quotas set for VCSEs of different characteristics.

What worked well: Views on the design and set-up phase were generally positive, with stakeholders highlighting the collaborative approach, the availability and expertise of the IGM, and the use of existing tools and mechanisms as key factors for success. The decision-making process was efficient, with The National Lottery Community Fund using their existing systems and processes. The average time between application and decision was 88 days. Grant holders were also positive about the design elements, including retrospective funding and the flexibility to use grants for core costs.

Challenges: The main challenges identified related to condensed timescales, some uncertainty amongst programme stakeholders about the focus of the CCLF (in terms of the types of organisation and services it was intended to be funding), the complexity of designing the retrospective claim element, and the need to design and implement grant management processes in parallel with the design phase.

Lessons learned: Stakeholders suggested that future similar funds should be made available as soon as rises in the cost of living occur (some stakeholders felt CCLF could have been launched in 2022/23, when the cost of living in England started to rise), more clarity on the target audience, and closer collaboration with other government departments.

2.1. Design and set-up phase

Following the announcement of the VCSE Cost of Living package at the Spring Budget in March 2023, DCMS began work on developing the financial business case for the CCLF, with the aim of launching the Fund in mid-July 2023. Following an extensive options appraisal, the DCMS Policy Team agreed early on that The National Lottery Community Fund should be appointed as the intermediate grant maker (IGM) via a direct award approach. This was due to The National Lottery Community Fund’s experience and expertise in delivering similar, large-scale funding programmes at pace, its extensive reach across the VCSE sector in England, and its availability to administer the CCLF in the timescales required. DCMS also engaged a VCSE Cost of Living Advisory Board (comprised of experts from different VCSE organisations / membership bodies) to inform the design and set-up of the programme. This phase of activity included several workshops to work through key aspects of the CCLF design (e.g. on eligibility, timelines, grant / award sizes, target services) and continued through to mid-July.

2.1.1. Reflections on overall CCLF design

Figure 2.1 below shows CCLF grant holders’ views on the effectiveness of the design features of CCLF, which grant holders were asked about in Wave 2 of the survey. Overall, grant holders were very positive about all design elements. The survey and qualitative research provided further evidence on stakeholders’ perceptions of the effectiveness of the design features of the CCLF. The key elements discussed were:

Retrospective funding: As highlighted in Figure 2.1, 91% of the 316 VCSE survey respondents found the inclusion of retrospective funding costs to be effective (68% of whom said it was ‘very effective’). Qualitative survey responses and interviews with VCSEs highlighted that it was an uncommon, but very useful mechanism to help them avoid dipping into reserves to cover costs. In addition, the majority of programme stakeholders felt that including retrospective funding alongside the ‘future costs’ grant was positive, in terms of enabling a longer period to be covered by the grant given the need to spend all funding by March 2024

This retrospective funding gave us a buffer and a moment to breathe as we continued with frontline services.

– Grant holder survey respondent

Having the option to use grant funding for core costs: Almost all VCSE grant holders responding to the survey (98%) found how they could use the funding to be effective. Some programme stakeholders, case study VCSE representatives and many survey respondents noted that a key design advantage was that funding could be used for core costs (e.g. staff salaries, overheads) as well as for delivery of specific project activities. They noted that this type of funding was often difficult to secure and many VCSE representatives in the survey and the case studies welcomed this flexibility, particularly in the challenging context where they had to deliver at pace to meet the demand of their beneficiaries

It’s for core funding as well, because that’s the thing we all struggle with

– VCSE organisation strategic staff / volunteer

Targeting of funding: across programme stakeholder interviews, there were mixed views on how far funding should have been targeted. Beyond the eligibility criteria, the CCLF’s design did not have any prioritisation criteria or quotas for the types of organisations receiving funding. Some stakeholders suggested that a proportion of funding should be ringfenced for organisations working with particular beneficiary groups, although others noted concerns about potential positive discrimination, preferring to limit the use of quotas and instead improve targeting and the reach of Fund advertising

Figure 2.1: CCLF grant-holders’ views on the effectiveness of each CCLF design feature

Figure 2.1: CCLF grant-holders' views on the effectiveness of each CCLF design feature

Source: Wave 1 grant holder survey (n=316 for each element).

2.1.2. Effectiveness of the design and set-up phase

Overall, programme stakeholders were positive about the CCLF design and set-up. There were four key factors seen as contributing to this:

Collaborative approach: Most stakeholders involved in CCLF design highlighted that it had been collaborative, benefitting from the significant combined experience and expertise of DCMS, The National Lottery Community Fund and the Advisory Board. Several stakeholders noted that opting for the ‘direct award’ approach for the IGM enabled this collaboration to start as soon as possible. Stakeholders also felt that the Advisory Board was engaged effectively and had the opportunity to input and provide (constructive) challenge to the design process

It was a really good thing for DCMS to bring in a steering [advisory] group from the sector and such a broad steering group as well…so it was a really positive thing that existed in the first place.

– Programme stakeholder

The availability and ability of the IGM: several stakeholders commented that the availability of The National Lottery Community Fund, and its ability to adapt its existing plans and accommodate CCLF, was key to successfully launching the CCLF in the short timescale available. Competitive procurement for an IGM would have increased timescales, likely delaying the fund launch
Benefits of team members’ expertise: stakeholders noted that many of those involved in the design had experience in developing similar scale schemes (e.g. the Coronavirus Community Support Fund), a deep understanding of the VCSE sector, and tacit knowledge of what works well and less well, which they were able to apply. This was important in facilitating a speedy set-up (given the short timescales for designing the Fund
Drawing on existing tools and mechanisms: several stakeholders commented on the benefits of being able to incorporate many of The National Lottery Community Fund’s existing tools, processes and mechanisms into the design of the CCLF. This meant that they did not have to ‘start from scratch’ and could be confident in the design given they were using tried and tested methods

While the general experience of the design and set-up phase was positive, stakeholders noted several challenges:

Condensed timescales: all stakeholders reflected on the challenging timescales (around four months) for designing the Fund. The key implications included: having less time for researching lessons learned from previous funds; stretching the capacity/availability of the teams involved; having to test and iterate ideas in a ‘live’ environment and make design decisions very quickly; and developing programme materials (e.g. application forms) alongside writing the financial business case. The skills and experience of team members across DCMS and The National Lottery Community Fund, as mentioned above, were key to successfully navigating these challenges
Designing the retrospective claim element: use of retrospective claims was fairly innovative for short-term and rapid funding, meaning there was limited experience across the teams on how best to implement it. Stakeholders noted that designing this element was complex, in terms of agreeing the practicalities (the cut-off date for retrospective funding, what activities were in scope, how much could be used) and processes for implementation (e.g. application form questions/guidance, grant monitoring)
Designing whilst implementing the work: stakeholders noted that they needed to design and implement grant management processes in tandem with launching the CCLF. This created additional pressures, particularly for The National Lottery Community Fund, requiring the engagement of temporary staff to support the application process and ensure that the team had the capacity to design, implement and deliver aspects of the Fund simultaneously
Limited lead-in time for organisations: several stakeholders noted that approval of the financial business case occurred relatively late in the process, meaning there was little time between gaining approval, sending out an advance notice (i.e. an announcement to the Advisory Board that the Fund would be launching), and the Fund launching. Stakeholders commented that, ideally, this would have happened earlier than the two weeks before the CCLF was launched, thereby ensuring that Advisory Board members had time to alert their members, and that VCSE organisations had the time to prepare and gather the required documents (e.g. financial accounts) ahead of applications opening

2.2. Engagement, application processes and decision-making

2.2.1 Effectiveness of communications and outreach

The short (2-week) lead-in time (see section 2.1.2) from the informal advance notice to Advisory Board members to the CCLF launching (24^th July 2023), meant that public communications of the scheme only happened once the CCLF had launched. The main advertisement of the scheme was via the organisations involved (The National Lottery Community Fund, DCMS and the Advisory Board members), which placed announcements on their respective websites, alerted organisations on mailing lists (e.g. newsletters), and shared information via social media. The National Lottery Community Fund also delivered a series of 15 webinars (in partnership with sector organisations such as the National Council for Voluntary Organisations (NCVO) over a ten-week period (July-September 2023). These were watched by approximately 3000 individuals across the VCSE sector. They aimed to introduce the CCLF, explain the eligibility criteria and the funding opportunity, and highlight requirements and timescales. However, some programme stakeholders interviewed questioned the effectiveness of these webinars in terms of expanding awareness of the Fund; rather, they were seen as being more helpful for explaining how the application process worked.

Survey findings indicate that organisations not linked into these networks might not have heard about the scheme. Figure 2.2 shows that around two-fifths (43%) of organisations (both grant holder and unsuccessful applicants) first heard about the CCLF from The National Lottery Community Fund, while just under a third (30%) heard from a membership body or network, other VCSEs, or other organisations that they work with. Around 15% were alerted to the CCLF by colleagues. Overall, very few organisations surveyed found out about the opportunity via the internet (e.g. social media or search engine), although both were slightly more common for unsuccessful applicants, compared to grant holders. Other sources (2%) were mainly via funding newsletters and hearing from local authorities.

Figure 2.2: How organisations first heard about the CCLF

Source: Wave 1 survey (grant holder survey (n=853) and unsuccessful applicant survey (n=400))

2.2.2. Application process

Applicants were required to complete an online application form and upload supporting documents.^{[footnote 5]} Support provided from The National Lottery Community Fund included application guidance, the aforementioned webinars (section 2.2.1), and an advice line, which supported applicants with any queries on the process. The CCLF closed to applications on 18th October 2023 and received 3,736 applications overall.

Overall, evidence from the Wave 1 survey, VCSE case studies and programme stakeholder interviews suggested that VCSEs had a positive experience of the application process and found it relatively straightforward. This was particularly the case for those that were familiar with applying for previous funds via The National Lottery Community Fund, as the systems and processes used were very similar. Furthermore, the application process was not viewed as particularly burdensome given the type (covering a range of core and delivery costs) and amount of funding (up to £75,000) available. The majority of survey respondents awarded CCLF funding (81%), and over half (58%) of unsuccessful applicants, reported that the amount of time they spent applying to the CCLF was as expected in relation to the amount of funding requested.^{[footnote 6]} Figure 2.3 shows the aspects of the application process that grant holders and unsuccessful applicants were most satisfied with.

Figure 2.3: Percentage of survey respondents (grant holders and unsuccessful applicants) satisfied with various aspects of the application process

Source: Wave 1 grant holder survey (n=853), wave 1 unsuccessful applicant survey (n=400)). Note: for presentation purposes, the response categories were simplified (i.e. ‘Satisfied’ includes those responding ‘Very satisfied’ or ‘Somewhat satisfied’).

Overall, as shown in Figure 2.3, while levels of satisfaction were high overall, grant holders were more likely than unsuccessful applicants to be satisfied across all elements of the application process, particularly with the information about the aims and objectives of the CCLF (97% of grant holders, 76% of unsuccessful), clarity of the eligibility criteria (96% and 76%) and the application guidance (90% and 72%). Where survey respondents had support from The National Lottery Community Fund, the majority of grant holder survey respondents (85% of 710), and around half of unsuccessful applicants (51% of 265) were satisfied with the support received. The VCSE case studies and the interviews with programme stakeholders highlighted that the support from the IGM was particularly useful in relation to clarifications on the retrospective funding application questions. Case study research also indicated that VCSE organisation leads found the system to be user-friendly and, when completing the application, welcomed having space to express in their own words how the funding would make a difference to their organisation and the people they support.

Specific challenges experienced by VCSE strategic staff with the application process included: confusion with the budget template (especially for the retrospective costs), in terms of requiring a high level of granularity for how costs would be allocated to different budget lines, and splitting the budget between retrospective and future costs; the retrospective funding budget timescales not aligning with organisations’ financial accounts timelines (but rather the date from which the fund opened); accessibility and technical issues, such as the submission portal crashing ^{[footnote 7]}, and not being able to submit the form in another format (e.g. Word document or video application), which could have been more accessible for applicants with additional needs; ^{[footnote 8]} and issues with evidencing increased demand and costs due to cost-of-living challenges, where a common reason for declining applications was because applicants did not adequately explain the impact of increasing costs (despite further guidance introduced by The National Lottery Community Fund).

2.2.3. Decision-making processes

Once VCSEs submitted their applications, a funding officer at The National Lottery Community Fund undertook basic checks (e.g. for missing documents) and an initial sift (i.e. declining those that did not meet the basic eligibility criteria). Funding officers then conducted full assessments, culminating in a recommendation for funding (or not). These were sent to weekly panels run by The National Lottery Community Fund ^{[footnote 9]} for final review and decision-making. As part of the IGM agreement, The National Lottery Community Fund was the ‘designated principal’ for decision-making, which meant that it did not need sign-off from DCMS on decisions to award funding or not. A DCMS representative joined the weekly panels in Autumn 2023 in an observer / quality assurance role.

As shown in Figure 2.3 above, 76% of grant holders responding to the Wave 1 survey were satisfied with the time they had to wait between submitting the application and hearing the decision from The National Lottery Community Fund. Around half (49%) of unsuccessful applicants were satisfied. Further analysis (see Figure 2.4 below) shows that the average (median) length of time between applying for funding and being notified of the award decision was 88 calendar days, which was just a few days over the 12 weeks that The National Lottery Community Fund advised applicants it would take to receive a decision. This was lower for those awarded funding (71 days), likely due to some being screened out earlier if they did not meet the criteria for example – as can be seen in the outliers in the graph below. The National Lottery Community Fund and DCMS decided to prioritise notifying the successful applicants first, given the compressed programme timeline for delivery and spend. Again this may have contributed to some outliers seen in the graph below, as those who did not receive funding were more likely to be notified later.

Figure 2.4: Time between submitting the application and being notified of the decision

Source: The National Lottery Community Fund CCLF application data (n=3,736)

While, as noted, the majority of applicants were satisfied with the timescales, qualitative feedback via the surveys and the case study research illustrated some of the challenges faced. Funded survey respondents providing feedback (n=265) at Wave 1 often noted that they would have welcomed earlier decisions, but this was mainly so that they would have had a longer time to spend the funds, given the end of March (2024) finish point for CCLF. Some also noted that the uncertainty over whether an award would be made or not had implications for their organisational contingency planning, as they were already experiencing higher demand for services whilst waiting to hear the decision. Several VCSE leads also highlighted that their notification of award was delayed, which meant that they had to recalculate the amount of retrospective funding in their budget. This led to it needing to be signed off again before they could start delivery. According to a programme stakeholder, this was a frequent occurrence because applicants did not factor in the estimated 12-week period between submitting an application and being awarded funding. Future funds including a retrospective spend element should make it clearer that applicants need to consider this time period in their applications.

We were not quite sure how much of our money we could spend because you know, they already expected us to start spending money, you know, like we would normally do. But then because we were not quite sure if we were going to get the grant, we were very cautious of how much we invested in the things that we really wanted to do.

- VCSE organisation strategic staff member / volunteer

Reflecting on the decision-making process, programme stakeholders felt it generally worked well, because it used The National Lottery Community Fund’s existing approach to decision-making (i.e. the multi-step process), which meant systems and processes were already in place; it had experienced senior managers making the decisions; and there were weekly panels (rather than less regular, such as fortnightly or monthly) to enable The National Lottery Community Fund to work through and consider the high volume of applications in an efficient and staggered way. In addition, stakeholders welcomed having DCMS involved as an observer on the panels, with the representative able to clarify points on policy. They also felt that the panel had done well at monitoring the characteristics of organisations awarded to try and ensure a good spread of organisations across geographies and service types.

2.3. Programme reach

Analysis of application data indicates that, overall, the 3,376 organisations applying for CCLF funding varied substantially, in terms of geographical reach, organisational type, service focus and organisation size:

Over two-fifths (41%) of applicants were located in London, the South East and East of England with a fairly even distribution of applications from the other regions, which is broadly reflective of the proportion of the population sizes in these regions.^{[footnote 10]} The likelihood of VCSEs being awarded funding was similar across every region (i.e. around 40% of applications in each region respectively were awarded funding) ^{[footnote 11]}
The most common organisational type for applicants was not-for-profit company (31%), although there was also good representation of other organisational types, such as charitable incorporated organisations (20%), registered charities (unincorporated) (18%), and organisations that were both registered charities and not-for-profit companies (13%). Where they made applications, charitable organisations were most likely to be awarded CCLF funding (e.g. 44% of charitable incorporated organisations applying were awarded funding compared to 30% of community interest companies). Overall, there are statistically significant differences (Cramer’s V=.12. p=.000) between organisation types in the likelihood of being awarded funding. However, there is no obvious reason for this to be the case, as the organisation type field was self-selected by applicants and there are limited meaningful differences in legal form or operations between the categories
The five main service types targeted by the Fund were food and emergency supplies, financial / housing advice, warm spaces, safe spaces / refuge, and emergency shelter. Many organisations applying for funding stated that their CCLF grant would cover multiple service types. Food and emergency supplies were most common (69% of applicants), followed by advice (49%), warmth (37%) and safe spaces (32%). Only 6% of applicants had projects delivering emergency shelter. Organisations working across multiple service areas were statistically significantly more likely (Cramer’s V=.15. p=.000) to receive funding than not (e.g. 60% of organisations covering five service types were funded; compared with 34% of organisations only offering one service type)
Just over half (53%) of applicants stated that their project would be targeted. ^{[footnote 12]} There was no statistically significant difference in the success rate of targeted projects, relative to non-targeted projects. Of the targeted projects, nearly half (44%) said they would be targeting people who were educationally or economically disadvantaged. Just under a third (33%) were targeting faith communities. When looking at the grant holders compared with the applicants not awarded funding, the proportion of projects targeting educationally or economically disadvantaged groups and faith communities was slightly higher for the grant holders, compared with the proportion of applicants not awarded CCLF that were targeting these groups. The proportion of projects targeting older or young people, communities facing racial inequalities, women and girls, disabled people, migrants, additional categories/other groups and LGBTQ+ was higher for non-awarded applicants, compared with successful applicants
85% of funded organisations were classed as Micro (3%), Small (23%), or Medium (59%). ^{[footnote 13]} Medium-sized and larger organisations (i.e. large and major, but not super-major) applying were more likely to be awarded funding. For example, of the large organisations applying to the Fund, nearly half (46%) were awarded funding. In comparison, 37% of small and 36% of micro-organisations applying for funding were successful. However these differences between organisation sizes overall were not statistically significant (Cramer’s V=.05. p=.187)

Analysis also identified that organisations awarded funding tended to be supporting a larger number of beneficiaries, have a higher number of volunteers, and a higher number of staff overall, across their organisation (see Figure 2.5). These findings were most likely explained by the National Lottery Community Fund’s evaluation criteria.

Figure 2.5: Comparison of VCSE organisation characteristics, by award status {:#figure-2.5}

Figure 2.5: Comparison of VCSE organisation characteristics, by award status

Source: The National Lottery Community Fund CCLF application data (n=3,376)

A total of 1,500 organisations were awarded CCLF funding (around a 40% success rate). This was in line with the original targets where, as highlighted in the ToC, it was intended that around 1,300 to 1,600 VCSEs would be awarded grant funding. 83% of grant holder organisations received funding for retrospective costs. For organisations that received both types of grants (83%), future costs funding typically accounted for 70% of total funding received.

Figure 2.6 shows the distribution of grants by the awarded amounts. It shows that the average (median) grant amount was £48,000, with the distribution slightly skewed toward the upper limit of the grants, with 248 organisations being awarded between £72,900 and £75,000.^{[footnote 14]} Overall, 113 organisations receiving the maximum grant amount of £75,000. The overall expected costs – and the requested grant amounts – of the projects awarded CCLF funding were, on average, higher than the expected costs for VCSEs not awarded funding: funded organisations had project costs that were typically £15,000 more than unfunded organisations, and requested typically £10,000 more in funding (see Figure 6.4 Annex 6). Projects typically applied for less funding than what they expected their total project to cost, indicating that they had existing resource or funding that they intended to use to supplement the CCLF grant.

Figure 2.6: Grant amounts awarded by number of organisations

Source: The National Lottery Community Fund CCLF application data, n=1,506

2.4. Project delivery

2.4.1. Overview of services delivered

In accordance with the CCLF’s aims and objectives, all organisations receiving a grant delivered critical, subsistence-level and/or advice services. Nearly two-thirds of grant holders (65%) delivered two or more types of service, with many offering a holistic service (e.g. providing groceries, warm meals, clothing, advice, and opportunities for socialising as part of a single support offer). Figure 2.7 shows that the majority (77%) of grant holders provided food and supplies, over half (58%) delivered advice, 41% provided warm spaces, just under a third (31%) safe spaces, and 8% offered shelter.

Figure 2.7: Service types delivered by grant holder organisations

Source: The National Lottery Community Fund Application data

Nearly all organisations responding to the Wave 1 grant holder survey reported being able to reach low-income individuals or households, either fully (86%) or partly (14%), as the CCLF intended. Around half (51%) of grant holders responding to the survey reported that they delivered ‘targeted’ services; i.e. where the majority of their beneficiaries were from a specific population/demographic group. Figure 2.8 shows that over half of survey respondents (57% of 455) whose organisations delivered targeted services said they reached educationally or economically disadvantaged people. Around a third of organisations reached specific age groups (e.g. people aged 65+ or people aged under 25), communities experiencing ethnic or racial inequity, women and girls, and disabled people.

Figure 2.8: Groups reached by ‘targeted’ services

Figure 2.8: Groups reached by 'targeted' services

Source: CCLF wave 1 grant holder survey (n=455). Note: respondents could select more than one group.

Case study research highlighted that those not targeting services tended to be open to anyone within the community in need of support, who may be in vulnerable circumstances. However, even those that did provide targeted support often said that their services were open to all and that they would not turn people away.

It’s really important to note that [organisation name] is an amazing place and the team regularly go above and beyond to support all of the local community, irrespective of who they are.

- Local stakeholder

2.4.2. Need for funding

The survey and case study research highlighted why organisations needed the CCLF funding. Over half of respondents to the wave 1 survey (59%) reported that, prior to receiving funding from the CCLF, their organisations’ costs had risen. In addition, strategic and delivery staff/volunteers interviewed spoke about how, prior to CCLF funding, their organisation had been struggling to deal with the increasing costs of running their services, along with a significant increase in their utility bills. It was evident that this particularly impacted services providing warm and safe spaces to their service users, such as community and youth centres, shelters for homeless people, and other similar services. Furthermore, staff and volunteers from organisations providing food and emergency supplies highlighted that their services were impacted by the increased cost of food ^{[footnote 15]} and other emergency supplies (e.g. hygiene and period products, baby formula, sleeping bags). At the same time, the staff and volunteers from those organisations also shared that they have seen a general decline in the number of donations, including from the general public as well as supermarkets and commercial partners as a result of cost of living challenges (see Box 2 for an example).

We’ve also seen a decline in donations, and I think that is due to the [increased] cost-of-living […] which means we’re having to buy more items.

– VCSE organisation delivery staff member/volunteer

Box 2: Example of food bank services dealing with decline in donations and increased demand for their services

A delivery staff member from one of the grant holder organisations providing emergency food services highlighted that they experience decline in donations seasonally, after the Christmas period. Combined with the drop in the number of donations due to increased cost of living, they needed the CCLF funding to buy more food, so they could continue delivering their services.

We get donations from all the supermarkets in [the area], but it’s different at different times of the year. January is really low in donations, just after the festive period and everything.

– VCSE organisation delivery staff member/volunteer

Additionally, the strategic staff explained that the food bank has been experiencing increased demand for their services. According to their data, they saw a substantial 51% increase in the volume of food items distributed between January and August 2023, compared to the same period in 2022. This increase in demand for food assistance placed a considerable financial and capacity burden on the organisation. The organisation applied for the CCLF funding to be able to continue delivery of their existing services for the 2023/2024 winter period, whilst also expanding their ‘Top-up Shop’ service into 3 new locations, allowing more people to access the service closer to their homes.

Coupled with the increase in costs were challenges in how organisations could finance them. Some strategic staff interviewed said that they struggled to cover the overhead costs as an organisation, as many VCSE sector grants did not cover these costs (instead paying for the delivery of a specific project). Therefore, being unable to cover the core costs of services was another reason why organisations needed funding from the CCLF.

Alongside increases in costs, CCLF funded organisations reported that they were experiencing increased demand for services as more people were affected by cost of living challenges and were turning to their local organisations for support. In response to the Wave 1 survey, organisational respondents reported that:

Prior to receiving the funding, demand for their services had gone up between March 2023 and the point at which the CCLF funding was awarded. The majority reported that ‘it went up a lot’ (78%), or ‘it went up a little’ (21%) (See Table 6.13 in Annex 6)
Over half of respondents (60%) reported that their ability to meet the demand for their services had increased in March 2024 compared to March 2023. A quarter of respondents (25%) reported that their ability to meet demand had ‘gone up a lot’ and over a third (35%) that this had ‘gone up a little’. Conversely, under a fifth of respondents reported that their ability to meet the demand for their services had ‘gone down a little’ (15%), or ‘gone down a lot’ (5%) (See Table 6.12 in Annex 6)
The number of beneficiaries supported in March 2024 compared to March 2023 had increased. Over two-thirds (68%) of respondents reported that the number of beneficiaries supported had ‘gone up a lot’, with an additional 27% reporting that the number of beneficiaries had ‘gone up a little’(See Table 6.12 in Annex 6). This suggests that organisations were facing an increased demand for their services, and this was translating into the number of beneficiaries they were supporting

Case study organisations reported similar trends and highlighted areas where new demand had emerged. For example, in one case study – offering essential baby items, food and clothing – staff and volunteers said they were seeing much higher demand for support (than in previous years) from families where the parent(s) were in work. In other cases, interviewees spoke about how they were seeing increased demand for their organisation because other local services were closing down, and people had fewer options for support.

It is a busy time [for the food bank]. People definitely have more outgoing costs, especially in the last couple of years with their costs going up.

– Local stakeholder

Representatives from other types of organisations, such as community centres and shelters, noted that they needed the CCLF funding to diversify/adapt their offer to meet the increasingly more complex needs of their service users. One delivery staff interviewee, for example, reported that they had an increased number of people coming in with mental health conditions, with this having led to other issues, such as financial difficulties. To support their clients and provide more holistic support, it was noted that the organisation concerned aimed to diversify their provision to provide a therapy service for mental health support alongside other advisory services.

2.4.3. What worked well in delivery

Whilst grant holder organisations delivered a wide variety of services, to different target groups, the case study research highlighted commonalities in what worked well in delivering critical frontline services. These are outlined below:

Delivering multiple and holistic services in one setting: As highlighted earlier in this chapter, most organisations delivered more than one of the eligible service types, which allowed beneficiaries to use multiple services at a single setting. During the case study visits, a number of beneficiaries, often from community centres, emphasised that what worked really well for them was having a range of different services and activities available under ‘one roof’

You get a bit of everything. It’s really helpful having that, so you don’t have to go to one place with this and to another for that.

– Beneficiary

Several staff and volunteers also explained how they took a holistic approach to providing support to their beneficiaries. Alongside offering a range of services, this also included working closely with other local organisations (i.e. statutory services or other VCSE sector providers) so that the complexity of their beneficiaries’ needs could be effectively addressed via signposting and providing them with multiple forms of support for different challenges they face
Service design informed by beneficiaries and person-centred approaches: Case study participants also appreciated the different ways in which the awarded organisations involved the beneficiaries in the design and delivery of services. In a number of case study organisations, volunteers and staff noted that initially they were accessing the services themselves, and were encouraged by staff to volunteer, with some volunteers also moving to paid roles

It really helps that a lot of volunteers have been in the position of service users at [name of the organisation] beforehand, it makes the service more relatable.

– VCSE organisation delivery staff member/volunteer

Stakeholders across multiple VCSE organisations highlighted that involving beneficiaries in the design and development of activities worked well for several reasons, including: creating an inclusive environment and making sure people feel welcomed, so they feel comfortable and can open up; helping to better meet people’s needs by developing services they need; and increasing engagement with activities. Beneficiary involvement approaches highlighted by interviewees included using surveys or informal conversations to get feedback; taking a beneficiary-led approach, where beneficiaries would highlight the need for the service and then staff/volunteers would address this by securing funding/developing the activity; and developing peer-support models. For example, in one case, in the context of English as a Second Language (ESOL) provision, beneficiaries supported others with informal language learning. More broadly, peer support and befriending was an element that worked well across the case studies, particularly to help people experiencing loneliness and isolation
Building trusted relationships: Case study participants also emphasised the importance of developing trusting relationships, including relationships between delivery staff and beneficiaries as well as between the VCSE sector organisation and local statutory services. One of the VCSE organisation leads interviewed likewise explained that they often act as a ‘bridge’ between the community and statutory services, as there can be a certain level of mistrust and suspicion towards services such as police or social services

The community… can be quite suspicious. They’re worried about getting the children taken off them. They could be worried about being evicted. They could be worried about losing benefits, and so they tend to not access some of the things.

- VCSE organisation strategic staff member

Another organisation’s staff member described the importance of building trusting relationships with their beneficiaries, often people at risk of or experiencing homelessness, in order to understand their story and provide appropriate support. Several service users across the case studies highlighted their positive interactions and relationships with staff and volunteers. Some beneficiaries further explained that not being turned away, and having staff willing to support them regardless of what the problem is, was crucial for building their trust with the organisation concerned

It’s the only charity like this that give ongoing support to people. Their door is never closed […] They’re always here with help.

– Beneficiary

Building sustainable support, leading to independence: Some staff and volunteers mentioned that advisory support, as well as mentoring and peer support, are effective in providing beneficiaries with solutions and coping mechanisms, leading to greater independence. Additionally, the grantees providing food supply services expressed that moving away from long-term use of emergency food parcels towards other more suitable options, such as ‘food pantries’, is crucial for building people’s independence. It was noted that this gives them a greater variety of foods to choose from, as well as an opportunity to financially contribute towards the food

And I’ve still got a bit of dignity, because it’s not that you feel embarrassed when you go to other food banks, but these make you feel that you want to come. You don’t feel that because you are struggling that you are on handouts, even though you are only paying a couple of quid

- Beneficiary

Across the case studies, other aspects of delivery that often were viewed by staff, volunteers and beneficiaries as working well included: having an ‘open door’ policy to welcome anyone who needs support; offering ‘drop in’ services to provide as much flexibility to beneficiaries as possible; and having staff with lived experiences who can relate to and know how to support people.

2.4.4. Challenges and area for improvements in delivery

Timescales for delivery: a common challenge noted in the case study research, by some survey respondents (see Figure 2.9 below), and through qualitative feedback in the survey was the tight timescales for delivering projects. People often noted that the window to spend the money was quite difficult to deliver within, especially in cases where funding decisions came towards the end or just after the end of the decision-making process. Many interviewees and survey respondents suggested that longer timescales for delivering projects would have been helpful, although many within these groups did recognise the constraints that The National Lottery Community Fund/DCMS had in terms of the timescales
Need for increased, specialised mental health support: Although some strategic and delivery staff highlighted their holistic approach as a crucial success factor for effective delivery, other delivery staff and volunteers noted that addressing the complex needs of their beneficiaries is challenging. Several strategic and delivery staff highlighted the importance of improving access to mental health support, alongside other critical services

It’s a vicious cycle, really. People have mental health problems that lead them into difficulty with finances, which then causes more mental health problems and then they may end up being homeless.

– VCSE organisation delivery staff member/volunteer

Needing more resources to meet the level of demand: whilst staff and volunteers very much welcomed the CCLF funding, some mentioned that their organisation was still struggling to meet the demand for services, and that they needed further funding to help them to retain and recruit staff, volunteers and resources (e.g. food and personal hygiene products) to address increasing levels of need
Increased awareness raising: Case study interviewees also mentioned that more emphasis could be placed on further raising awareness of the services they offer. Several beneficiaries interviewed reported that they learned about the services only through word of mouth, as other family members or friends used the services previously. Others learnt about the services from other providers. During the case study interviews, some staff members and volunteers likewise suggested that their service could be better promoted via social media or other means

2.4.5. Grant monitoring and reporting

Overall, the majority of grant holders responding to the Wave 1 survey were satisfied with CCLF grant management – as shown in Figure 2.9 below. Grant holders were most satisfied with the grant monitoring requirements (85% ‘somewhat’ or ‘very’ satisfied) and the information and support from their funding officer (84%). While most grant holders responding to the survey were satisfied with the evidence required for the retrospective claim form (78%), several programme stakeholders noted that midway through the funding period The National Lottery Community Fund (in agreement with DCMS) increased the timescales for evidencing spend, based on feedback from grant holders that providing this evidence in time was challenging. This change was reflected in the qualitative survey responses, where some respondents noted challenges with gathering the required evidence initially but were pleased when the changes were introduced.

Figure 2.9: Grant holders’ level of satisfaction with various aspects of grant management

Figure 2.9: Grant holders' level of satisfaction with various aspects of grant management

Source: Wave 1 CCLF grant holder survey (n=853, except for response category “Producing the evidence needed for the retrospective claim form”, which was n=693 (as this question was only available to those organisations that received retrospective grant funding)). To simplify the graphic, the chart does not include the percentage of respondents selecting “Don’t know” or “Prefer not to say / not applicable”. This is why “Information and support from funding officer at The National Lottery Community Fund’ is a shorter bar than the others, as 6.1% said ‘not applicable’.

2.5 Lessons learned from the CCLF process evaluation

Programme stakeholders, case study VCSE interviewees and survey respondents identified a range of key lessons or elements of the CCLF that could have been done differently, or that they would suggest are considered in the context of improving the design of future challenge response funds.

Earlier funding: While stakeholders acknowledged the CCLF timescales were immovable, several highlighted that the need for funding in response to the increased cost of living emerged in early 2022. As such, they felt that had funding been made available sooner, they could have spent more time on design and would have been in a position to launch the Fund earlier to alleviate some of the pressures on VCSEs that occurred in winter 2022/23. Stakeholders indicated that they would welcome earlier collaboration between Treasury, government departments and sector representatives for future, similar funds (although the extent to which this would have been possible for CCLF was limited, as discussions started in 2022)
Providing longer lead-in time for funding: following inclusion in the 2023 Spring Budget an advance notice for CCLF was then only sent informally, to the VCSE Advisory Board members, two weeks before Fund launch. Future funds should – as far as timescales allow – provide advance notice earlier than this and publicly announce the funding so that VCSE applicants can prepare the documents they might need for applications. More time would also enable organisations to plan their resourcing (which might be stretched in a crisis situation) and provide sufficient time to write an application
Ensuring funding for core costs: As noted in section 2.1.1, almost all grant holder survey respondents welcomed the flexibility of funding to cover core costs, and case study interviewees often highlighted the limited availability of grants for these cost types. Future response funds should consider the option of providing funding to core costs / unrestricted funding (if such options aren’t provided elsewhere already), to maximise organisations’ flexibility and to help keep services going

Having core costs covered was extremely helpful, as not many funders provide this. Yet we are the front line and we need core costs to be covered to ensure the stability of the service we are providing.

– Grant holder survey respondent

Longer-term funding pots: case study research and qualitative feedback from the surveys suggest that organisations would welcome longer-term funding opportunities over shorter-term funding pots, especially in contexts such as rising inflation / increased cost-of-living, where impacts are likely to be felt for a long time. We return to this point in section 3.1.2
Ensuring suitably experienced and qualified design team members: DCMS and The National Lottery Community Fund successfully navigated the challenging circumstances surrounding setting up the scheme (e.g. short timescales for design) because both organisations had team members involved with significant experience of delivering similar schemes. Development of future schemes should ensure that the right people are on board, e.g. using secondments or temporary changes in teams to help facilitate this
Closer collaboration with other Government departments: DCMS consulted with the then Department of Levelling Up, Housing and Communities (DLUHC) ^{[footnote 16]} in designing the Fund. However, several stakeholders suggested that, on reflection, they would have worked more closely with other departments, especially to support with promoting the Fund to its target sectors

Question 4

3.0 Outcomes and impacts

Accepted Answer

This chapter sets out evidence gathered relating to desired short-, medium- and longer-term outcomes from the Fund, as well as presenting findings from the impact evaluation. The outcomes relate to those at VCSE organisation level, for staff and volunteers, and for end beneficiaries. The impact evaluation focuses on desired medium to long-term outcomes at organisation level.

Chapter summary

Service Delivery Outcomes: In the short-term, the CCLF funding enabled organisations to sustain and adapt their service delivery to meet increased demand. Many organisations increased the number and range of services they provided, with a significant portion reporting a substantial impact on their ability to maintain and expand services. In the medium to long-term, organisations continued to maintain or adapt their services beyond the funding period, although some faced challenges in sustaining certain activities due to ongoing financial pressures.

Financial Resilience: The CCLF funding improved the financial position of organisations, allowing them to meet high delivery costs and avoid unplanned use of reserves. The counterfactual impact evaluation found that CCLF led to a statistically significant increase in total gross income for grant holders with less than a £10 million income prior to CCLF. There was also an increase in expenditure for these organisations. CCLF had a statistically significant positive impact on net income for organisations receiving funding. However, while the funding provided short-term relief, many organisations continued to face financial challenges in the medium-long term and needed to take cost-management decisions. However, the funding helped some organisations build financial resilience and secure additional funding.

Outcomes for Staff and Volunteers: The funding helped reduce pressure on staff, improve their confidence and motivation, and avoid job losses over the short-term during the funded period. Many organisations were able to retain staff and volunteers, while some hired new staff or increased volunteer recruitment. In the medium to long-term, organisations faced ongoing challenges in retaining staff due to financial pressures, but the funding helped some organisations maintain their workforce and improve staff capacity.

Outcomes for Beneficiaries: The funding helped meet the immediate needs of beneficiaries, such as providing food, emergency supplies, and safe spaces. Beneficiaries reported improvements in their mental and physical health, financial stability, and overall well-being. As a result, some beneficiaries experienced longer-term outcomes, such as reduced impact of poverty, improved mental and physical health, and better access to education and employment opportunities.

3.1. Impact of CCLF on VCSE organisations

To understand whether CCLF led to any improved outcomes for VCSE organisations, the survey and case studies explored the extent to which CCLF affected organisations’ finances, ability to maintain service delivery, and capacity and resourcing. This section describes short, medium and long-term outcomes experienced by grantee organisations in relation to several key areas: ability to maintain or adapt service delivery to meet demand, financial resilience, and outcomes for staff and volunteers. Whilst presented as separate areas, it is important to note that across the case study research it became apparent that often these outcomes were interrelated for organisations.

3.1.1. Service delivery outcomes

3.1.1.1. Organisations sustain and adapt service delivery in the short-term

Organisations being able to sustain or adapt their delivery of target services in the short-term to support demand was a desired CCLF outcome reflected in the programme ToC. Overall, there was evidence to suggest that, in the face of increasing costs and demand, organisations were able to maintain and adapt their service delivery as needed over the 2023-2024 winter period as a result of receiving CCLF funding.

The survey data suggests that the CCLF positively impacted organisations’ ability to maintain and increase the number ^{[footnote 17]} and range ^{[footnote 18]} of services they provided, with a large proportion increasing the number and range of services. According to the Wave 1 survey data, grantee organisations reported that they had increased both the number and range of services delivered between March 2023 and March 2024:

42% of survey respondents reported that the number of services they provide had ‘gone up a lot’, with an additional 39% of respondents reporting that the number had ‘gone up a little’.
17% of respondents reported that the number of services they deliver had stayed the same.
28% of respondents also reported that the range of their services had ‘gone up a lot’, with 47% reporting it had ‘gone up a little’.
for a smaller portion, the range had stayed the same (23%)(See Table 6.12 in Annex 6)

Respondents also reported that the CCLF funding had supported them to do this. Respondents to the wave 1 survey were most likely to report that the funding had a ‘substantial impact’ (49%) or ‘quite a bit of impact’ (38%) on sustaining the number of their services. Only a small number (2%) reported that the Fund had no impact and, of these, respondents largely reported that their organisation had not needed the CCLF funding for expanding or adapting their existing services. Responses were similar for the impact that CCLF funding had on organisations’ ability to maintain the range of services they provide, with 45% of Wave 1 respondents reporting a ‘substantial impact’, and 38% reporting ‘quite a bit of impact. (See Table 6.12 in Annex 6) Process tracing of survey results also found that CCLF contributed to organisations maintaining or increasing both the number and range of services they provide. Details of the process tracing results can be seen in Box 3 ^{[footnote 19]}.

Box 3: Process tracing results for likelihood of maintaining and / or increasing the number and range of services provided

The results of the process tracing analysis indicate that it is likely, for most VCSE organisations, that the CCLF contributed to their maintaining and /or increasing the number and range of services they provided. Specifically:

490 grant holders (72%) had a greater than 60% chance that CCLF contributed to them maintaining and / or increasing the number and range of services they provided.
667 grant holders (98%) had a greater than 40% chance that CCLF contributed to them maintaining and / or increasing the number and range of services they provided.
678 VCSE grant holders (99%) had a greater than 20% chance that CCLF contributed to them maintaining and / or increasing the number and range of services they provided.

A full breakdown of these results can be seen in Annex 6, Table 6.21 and Figure 6.5.

The qualitative evidence from the case studies further echoed that the grantee organisations were able to increase both the number and range of their service and activities. Staff and volunteers from grantee organisations shared that they delivered a higher number of services and activities as a result of the funding. In some cases, using CCLF to pay for new staff and volunteers, and to purchase more food and emergency supplies, allowed them to expand their services to new locations. Furthermore, some VCSE strategic staff noted that covering the staff and volunteer costs via the CCLF allowed them to be open for longer, or on days they were previously closed, and to deliver existing services more often. This allowed some services to be open for all 5 working days and, in some cases, every day of the week.

So, like welfare advice, we went from having like one session, and eventually, we sort of ended up with two every other week.

- VCSE organisation strategic staff member, wave 1

The evidence from the case studies also showed that some VCSEs adapted their services by changing their approaches and ways of working; for example, by introducing ‘drop-in’ sessions, home delivery of food parcels for the most vulnerable, or gradually moving from a ’food bank’ to a ’community supermarket’ model. From their perspective, this led to improving the quality of their services. Some VCSEs providing food and emergency supplies reported that the funding allowed them to include a greater variety of products and purchase items that might be more expensive. Purchasing equipment such as freezers, fridges and cookers allowed them to store and process fresh foods, avoiding high volumes of waste. Staff and volunteers from one of the organisations consulted also reported that the CCLF allowed them to cater for people with different dietary requirements by purchasing gluten-free or halal foods that tend to be more expensive.

We were able to source more food because what we get in donations is just never enough to make versatile food parcels. We could buy what we needed to supplement what’s not available. So, we had food parcels that were actually dignified with nice things for them.

-VCSE organisation strategic staff member, wave 2

3.1.1.2. Organisations meet demand in the short-term

Through maintaining or adapting service delivery as outlined above, the CCLF aimed to support organisations to meet the increased demand they were facing from service users. When asked about the impact they felt that CCLF funding had on their ability to meet demand, the majority of Wave 1 survey respondents described this having an impact. Around half (51%) of respondents stated that the CCLF funding had a ‘substantial impact’ on their ability to meet demand. Similarly, 53% reported that the CCLF funding had a ‘substantial impact’ on their ability to maintain the number of beneficiaries supported.

Survey data from individual organisations were used in the process tracing analysis. As can be seen in findings presented in the box ^{[footnote 20]} below, this reinforced the evidence that CCLF supported organisations to manage increased demand and increase their beneficiary reach, with there being a high chance that CCLF contributed to organisations maintaining and/or increasing the number of service users reached.

Box 4: Process tracing results for likelihood of CCLF contributing to organisations maintaining and/or increasing the number of service users reached

The process tracing results indicated that the CCLF funding played a significant role in enabling VCSE organisations to manage increasing demand and expand their beneficiary reach. Aggregated findings from the BPT are presented as follows:

350 grant holders (97%) had a greater than 60% chance that CCLF contributed to them maintaining and / or increasing the number of service users supported
350 grant holders (97%) had a greater than 40% chance that CCLF contributed to them maintaining and / or increasing the number of service users supported
357 grant holders (99%) had a greater than 20% chance that CCLF contributed to them maintaining and / or increasing the number of service users supported

Full findings from this analysis can be seen in Annex 6, Table 6.22 and Figure 6.5

Interviewed staff and volunteers from grantee organisations reported that by using the CCLF to adapt or expand their services, they could meet increased demand. Stockpiling food and emergency supplies allowed them to remove any existing limits on the number of products they could provide,^{[footnote 21]} subsequently leading to quicker support for more beneficiaries and reducing their waiting lists.

We already had a waiting list of people who we couldn’t accommodate, and their needs became more urgent during the winter period. So, we were able to respond to more people in need.”*

- VCSE organisation strategic staff member, wave 1

3.1.1.3. Organisations maintain delivery of services in the medium to long-term

Following the funding period, the majority of organisations were able to continue maintaining or adapting their service delivery to meet continued rising demand. Organisations that completed the wave 2 survey reported that both the range of services they offer, as well as the number of services they offer, either increased or stayed the same, with very few seeing a decrease in service number or range.

As can be seen in the figures below, this was true for both successful grant holder organisations and unsuccessful applicant organisations, which also increased the number and range of services during the same period. This suggests that across the sector more widely organisations sought to expand and increase their service delivery.

Figure 3.1: Changes in range of services provided between March 2024 and October 2024

Source: Wave 2 grant holder survey (n=316), wave 2 unsuccessful applicant survey (n=193)

Figure 3.2: Changes in number of services provided between March 2024 and October 2024

Source: Wave 2 grant holder survey (n=316), wave 2 unsuccessful applicant survey (n=193)

While, on the whole, the number and range of services had stayed the same or increased for Wave 2 survey respondents, evidence from case study organisations was more nuanced than this. Across a small number of case studies in the second wave of fieldwork, interviewees described having to reduce certain aspects of their delivery, stopping certain activities or strands of work (see Box 5). In some instances, this was CCLF-funded expansion, in others, this related to existing service delivery prior to CCLF funding. For example, one case study organisation representative described how they had to close half-term provision as they were no longer able to meet the high costs of delivery. Others similarly reported closing specific activities such as home visits and cooking sessions due to a lack of funding or reducing the number of rooms for people experiencing destitution they had on offer.

Box 5: Reduction in CCLF funded service

Interviewees in one organisation involved described how after the CCLF funding period they were unable to maintain all of the aspects of expanded delivery that they had funded through CCLF. Some of their CCLF funding was used to pay for a staff member to identify other services and organisations they could signpost to in relation to meeting the increased cost of living challenges. However, at the end of the funding period, they did not have sufficient funding to continue with this work, so they had to think about how to support beneficiaries through that time and see which aspects of the work could be picked up by other team members.

I think the, the only difficulty that we had in terms of sustainability was that obviously the funding came to an end on the 31st of March and there were still those clients that we were working with. So we had to almost like look at exit strategies and where we could carry on supporting them within our organisation.

- VCSE Organisation Strategic Staff Member

When they then did secure some additional funding to bring in support this was a welcome relief, as the strategic staff member interviewed reported they were concerned their staff were going to burnout.

To be perfectly honest because we couldn’t carry on [the activities after CCLF funding], you know, like I said that we were grateful when we got that money because demand started to creep up and if we weren’t careful, we would have burnt the staff out that we’ve already got.

– VCSE Organisation Strategic Staff Member

Despite the specific role no longer being continued, they were able to embed the work that had been established and the organisation described having a much stronger list of referral partnerships and organisations to signpost to as a result.

Many organisations reported that they are still facing a high demand for their services after the funding period, which may have influenced the need for additional services. This was reflected in the wave 2 survey results, with 88% of grant holders reporting that between March 2024 and the time of completing the survey, demand for their services went up a lot (55%) or a little (33%). Similar responses were seen for unsuccessful organisations as well, suggesting that – at the time of reporting - the sector more widely continued to face increasing demand (see Table 6.25, Annex 6). Case study organisations also mentioned increased demand and some reported waitlists for their services. Several factors were noted by case study organisations as contributing towards this, including:

Continued high need as a result of the wider context and continued cost of living pressures
Demand not being seasonal and seeing consistently high demand across the year or peaks at other times (such as school holidays)
Once people are introduced to the service they may become more aware of the support that is available and their needs are highlighted, driving up demand for services
As organisations sought to promote their services in the local area this also drove up demand

I’d say demand is still [high], there’s more demand than we can provide service for and sometimes we do have to turn people away and say like can you come back another day, come back another time

– VCSE organisation delivery staff member/volunteer, wave 2

Some case study organisations noted how in the months following the funding period they had continued seeing increases in demand in relation to specific areas, which was affecting their delivery. For example, one reported that people were increasingly seeking support for benefits advice. Another also reported this and highlighted how the recent changes to the winter fuel allowance entitlement had put a lot of pressure on their services with older people, as people were no longer eligible for the allowance and needed help.

I think in general we are much busier than last year, given all the recent changes that have been announced…we’ve been supporting people with making sure they are getting pension credit…the right sort of benefits they are entitled to.

– VCSE organisation delivery staff member/volunteer, wave 2

To meet the ongoing demand, some organisations continued to make small adaptations or expansions to their services following the funding period. Examples provided mostly included streamlining services and better management of resources. Some staff and volunteers reported further expanding their services, introducing new activities, or seeking larger premises to meet growing demand. For those that purchased equipment or stockpiled food and emergency supplies during the funding period, this supported the management of resources longer term and helped them to maintain delivery beyond the funded period. Additionally, small refurbishments, which allowed the expansion or improvement of organisations’ facilities, also created a lasting impact.

Despite these actions, it was still noted by some that demand was exceeding their capacity to deliver, with some reporting waitlists for their services or having to turn people away. This was also noted by an external stakeholder talking about a CCLF-funded organisation:

There are more and more of our clients coming every day and the funding is getting more and more restricted, especially since we’ve left the EU and lost that option for funding. So, you know, through no fault of [CCLF grantee organisation], sometimes their waiting lists are long. It’s at their capacity.

- Local organisation representative, wave 2

3.1.2. Financial resilience

VCSE organisations being able to meet high costs related to service delivery was another key desired outcome for the CCLF. As described in section 2.4, organisations reported that prior to receiving CCLF funding they were facing increasing costs. However, there is evidence to suggest that as a result of receiving funding from the CCLF, VCSE organisations were better able to meet these high costs that they were facing in the short-term. Case study organisations highlighted that without the CCLF funding, they would have had to take cost-saving measures to maintain the services during the 2023/2024 winter period, potentially reducing the number and/or quality of their services. For example, according to several VCSE staff, it was likely that their organisation would have reduced their opening hours, the number of sessions, or the support they could receive by introducing caps to provision. Others suggested they would have had to introduce or increase existing service fees where relevant (e.g. a community food pantry).

Some organisations’ strategic and delivery staff agreed that, without the CCLF funding, they would have had to continue investing time and resources into applying for grants and fundraising to help them meet their delivery costs for the winter period. While they would have hoped to secure alternative funding, they acknowledged that there was no guarantee their efforts would materialise into additional funding, and in that eventuality, some reported they would have been unable to continue delivering all of their services. The wave 1 survey suggested this may also have been the case for unsuccessful applicants, as only 32% reported that they had received alternative funding to support the delivery of the service(s) that their CCLF funding application was for (See Table 6.26 in Annex 6), and 60% reported that whilst they were able to continue delivering the service they had to reduce what they delivered or who they delivered to. (See Table 6.27 in Annex 6)

I would have been having to do way more work on fundraising, which takes me away from doing other work…. Last year, I had to do a lot of development for one of the outreaches, so I wouldn’t have been able to do that.

- VCSE organisation strategic staff member, wave 1

Similar trends were seen in the Wave 1 survey findings. Over half (59%) of survey respondents reported that, without the funding, they would have had to reduce the number of services they provide and reduce the range or level of services (64%) (see Figure 3.3). In reality, between receiving the CCLF funding and March 2024, only 4% had reduced the number of services they provide, and 5% had reduced the range or level of services they provide. (See Table 6.10 in Annex 6)

Figure 3.3: Impact of CCLF funding on cost management decisions relating to service delivery (See Table 6.10 in Annex 6)

Figure 3.3: Impact of CCLF funding on cost management decisions relating to service delivery

Source: Wave 1 grant holder survey (n=853)

3.1.2.2. Increased income and expenditure for CCLF organisations

The counterfactual impact evaluation also indicates that the CCLF funding has had a positive impact on the financial well-being of funded organisations (further details of this approach can be found in Annex 5). The analysis examined the impact of CCLF funding on total gross income, total gross expenditure, and net income, comparing the outcomes of organisations that received CCLF funding to a comparator group of organisations that did not receive CCLF funding. This comparison allows us to estimate the additional income, expenditure, and net income attributable to the CCLF, beyond what these organisations would have been likely to achieve without the funding. Recognising the influence of very large organisations (38 with income greater than £10 million), analysis was focused on two samples:

Organisations with income less than £10m (n = 1816)
Organisations with income less than £1m (n = 1460)

For completeness, results including all organisations (n = 1854) are included in the annex but should be interpreted with caution.

Results are reported alongside their p-values and confidence intervals (Table 6.28 in Annex 6). A p-value below 0.05 is commonly considered statistically significant, indicating that there is less than a 5% chance that the observed effect occurred due to random variation. However, rather than focusing solely on this yes/no threshold of significance, it is more informative to consider the confidence intervals. These intervals provide a range within which the true effect of the CCLF is likely to fall.

Key findings were:

Total gross income: CCLF funding led to a statistically significant increase in total gross income for organisations with less than £10 million in income prior to CCLF, with an estimated treatment effect of £44,514 (95% CI: £10,908 to £78,120, p = 0.009). For organisations with less than £1m in income the estimated treatment effect was £31,058 (95% CI: £16,576 to £45,540, p < 0.001). This suggests that the funding helped these organisations to secure additional income, beyond what similar organisations were able to obtain through other means
This increase in gross income, being on average close to the average grant award, indicates that CCLF funding was unlikely to have displaced other sources of funding for these organisations. In other words, funded organisations appear to have continued to access other funding at a similar rate to their non-funded counterparts. Conversely, the fact that the effect on gross income does not significantly exceed the grant size suggests that, on average, CCLF funding did not lead to a substantial ‘crowding in’ of further funding beyond the initial grant amount relative to the comparator group. However, it is important to note that these are average effects, and there will naturally be variation in outcomes across individual organisations
Total gross expenditure: For organisations with less than £10 million in income prior to CCLF, the estimated treatment effect on total gross expenditure was £23,340 (95% CI: -£7,551 to £54,231, p = 0.14), and for organisations with less than £1 million in income, the estimated treatment effect was £9,943 (95% CI: -£103 to £19,990, p = 0.052). While these results were not statistically significant at the 95% level, the confidence interval suggests that the true effect could be positive. This suggests that the funding enabled these organisations to increase their spending relative to the comparator group, potentially on expanding services or meeting increased demand
Net income: CCLF funding had a statistically significant positive impact on net income for organisations with less than £1 million in income prior to CCLF. The estimated treatment effect was £21,114 (95% CI: £10,786 to £31,443, p < 0.001). For organisations with less than £10 million in income prior to CCLF, the estimated treatment effect was £21,174 (95% CI: -£245 to £42,592, p = 0.053). The positive direction of these effects is encouraging, suggesting that the funding improved the overall financial balance/health of these organisations compared to the comparator group

The findings suggest that CCLF funding has had a positive impact on the financial health of VCSE organisations. An estimated treatment effect provides an indication of the causal effect of an activity on a particular outcome. In this case, the positive impact on income, expenditure, and net income for these organisations, compared to similar organisations without funding, indicates that the funding has helped them to maintain or even expand their services in the face of rising costs and increased demand. The direction of the treatment effects was positive across all income levels; however, the variability increased when very large charities were included in the sample, leading to a loss of statistical significance for this group. This suggests that the funding may have been less impactful for larger organisations, potentially due to their larger scale of operations and pre-existing funding sources.

Overall, aligned to the analysis presented above, the interviewed VCSE strategic staff reported that the CCLF improved their short-term financial position, with many highlighting the role of retrospective funding and the unrestricted nature of CCLF in providing a ‘financial cushion’ for the organisations concerned. Some of the VCSE strategic leads also highlighted that the CCLF funding made up a sizeable amount of their income and, in some cases, it was the largest grant they had received. Staff from other VCSEs reported that the CCLF contributed to improved cash flow for their organisation, with one of the organisations highlighting that they were able to pay rent for their facilities and storage unit 6 months up-front.

The amount of money we got was just over a quarter of all of our income. I can’t overstate that enough. It was a huge proportion of our income […] I was so happy, and it’s lifted the pressure off massively.

- VCSE organisation strategic staff member, wave 1

Box 6: Process tracing results for likelihood of CCLF contributing to organisations avoiding shrinkage and insolvency

When examining the impact on financial resilience—specifically the ability to avoid shrinkage and insolvency—the Process Tracing approach presents a more nuanced picture. For the majority of CCLF funded VCSE organisations, there is only a small chance that CCLF made a strong contribution to their financial resilience by avoiding shrinkage and / or insolvency:

273 VCSE organisations (30%) had a greater than 60% chance that CCLF contributed to maintaining financial resilience by avoiding shrinkage and / or insolvency
360 VCSE organisations (39%) had a greater than 40% chance that CCLF contributed to maintaining financial resilience by avoiding shrinkage and / or insolvency
733 VCSE organisations (80%) had a greater than 20% chance that CCLF contributed to maintaining financial resilience by avoiding shrinkage and / or insolvency
A full breakdown of these results can be seen in Annex 6, Table 6.22, Table 6.23 and Figure 6.5.

It would be expected that a strong contribution of CCLF would be reflected in relatively high likelihood values in respect of the process tracing analysis. As described in the methodology section and Box 1, process tracing evaluates the plausibility of different explanations are for an event by systematically assessing the available evidence. However, despite overall positive financial trends observed with CCLF funding – both from survey data and the counterfactual impact evaluation – the process tracing presents only moderate evidence that CCLF has contributed to financial resilience of VCSEs, as can be seen below.

While the Counterfactual Impact Evaluation (CIE) shows a positive impact of CCLF funding on the financial health of organisations, based on increases in objective measures of income and expenditure, the BPT results, based on survey responses, may present a less clear-cut picture. This is because the BPT focused on the perceived impact of the funding on the risk of insolvency or shrinkage, which might not directly align with the short-term financial improvements captured by the CIE. The BPT may likewise indicate lingering concerns among these organisations about their long-term financial stability, even with the CCLF funding. This difference highlights the importance of considering multiple perspectives and data sources when evaluating the impact of such interventions ^{[footnote 22]}.

3.1.2.3. Use of reserves

Another desired outcome for the CCLF was that, as a result of receiving funding, organisations would be less likely to need to spend their reserves to maintain their delivery in the face of increasing costs and demand. In order to meet rising costs, organisations reported having to make unplanned use of reserves to meet delivery costs prior to receiving CCLF funding, with over a third of organisations needing to do this (35%). Furthermore, over half of respondents (52%) reported that, had they not received the CCLF funding, then they might have had to make unplanned use of their reserves.

In reality, as a result of receiving funding from the CCLF, 55% of respondents to the survey reported that they did not need to take any cost management decisions during the funded period (see Figure 3.4). The most common action that organisations still had to take despite receiving funding, however, was indeed the unplanned use of reserves (reported by 17% of Wave 1 survey respondents) see Table 6.10 in Annex 6. These results suggest that, while for many organisations the CCLF funding did prevent unplanned use of reserves, for a small portion reserve use was still necessary.

Figure 3.4: Impact of CCLF funding on cost management decisions relating to unplanned use of reserves (see Table 6.10 in Annex 6)

Figure 3.4: Impact of CCLF funding on cost management decisions relating to unplanned use of reserves

Source: Wave 1 grant holder survey (n=853)

These findings were generally echoed in the case study research, as organisations reported that being able to meet core delivery costs for services meant that their organisations were under less financial strain and, therefore, less likely to need to spend their reserves. Furthermore, some VCSE staff also reported that the retrospective aspect of the funding allowed their organisations to replenish their reserves, which had been depleted in meeting the rising costs of service delivery in recent times.

I wouldn’t say we were in trouble, but we were beginning to go into a reserve to cover some of our costs. So that was really helpful to be able to put that sum back and then have enough for some months to operate without stress.

- VCSE organisation strategic staff member, wave 2

3.1.2.4. Medium to longer-term effects on financial resilience

In the medium- to longer-term the CCLF ToC postulated that as a result of receiving CCLF funding organisations would be at a reduced risk of shrinkage, inactivity or insolvency. In order to explore this outcome, the following section sets out the effects of CCLF on organisations’ overall financial resilience.

The wave 2 survey provided an insight into the wider context facing VCSE organisations following the funding period, with the majority reporting that along with demand continuing to increase, costs had also continued to rise; consequently, organisations largely still remained concerned for their financial situation. As can be seen in Figure 3.5, overheads (e.g. cost of rent/mortgage, energy costs, insurance) were the most common area in which organisations saw costs rise between the end of the funding period in March 2024 and the time of the survey in October 2024. Only 4% of respondents reported no change to their overhead costs during that period ^{[footnote 23]}. Not only was this the area organisations most commonly saw a rise, but costs also rose by a higher proportion in this case than other areas examined. Nearly a quarter of survey respondents (24%) reported that their overheads had risen by over 15%. The costs of equipment and materials also rose for many organisations, alongside staff wages.

Figure 3.5: Breakdown of extent to which organisational running costs rose between March 2024 and October 2024

Source: Wave 2 grant holder survey (n=316)

In the face of continuing increased demand and increasing costs, some organisations reported that their organisation’s financial situation was challenging. Around 70% of respondents to the wave 2 CCLF grant holder survey reported that their organisation had to take cost-management decisions between the funding period ending and October 2024. The most common cost management decisions were unplanned use of reserves (42%), reduced staff hours (22%) and increased price of services (14%). A large proportion of organisations also reported that they would have to take one or more cost management decisions over the coming winter period up to March 2025. Most commonly reported was a likely need to use reserves, of which 52% of grant holders who responded to the wave 2 survey reported.

Ongoing financial challenges were similarly echoed in the case study research, with grantee organisation representatives reporting ongoing challenges and financial risks, alongside being reliant upon unpredictable grants and donations. Some organisational representatives also highlighted that their costs remain higher than income, which presents an ongoing challenge. This may indicate that while the CCLF provided some short-term relief, in at least some cases challenges across the sector remain and longer-term funding is required. Organisational interviewees likewise reported that they need to continue applying for funding and some described still feeling in a ‘financially risky’ situation. Some similarly anticipated either growing costs or reductions in income which were beyond their control, such as the need to cover upcoming national insurance rises and changes to the minimum wage, or funding from other agencies like local councils being cut and affecting their finances.

However, at the wave 2 survey, unsuccessful applicants were more likely to report having taken cost-management decisions, as well as to anticipate having to make future cost management decisions. For example, 51% of unsuccessful organisations reported having to make unplanned use of their reserves following the funding period, compared to 42.4% of grant holders. Similarly, 60% of unsuccessful applicants anticipated that they would have to make unplanned use of reserves in the coming winter months, compared to 52% of grant holders. As noted earlier, grantee organisations often reported the positive effects of CCLF on their finances in the short-term, which may have somewhat protected them from the challenging environment.

[It helped] for the three months it covers, but really it’s a drop in the ocean.

- VCSE organisation strategic staff member, wave 1

Despite these challenges, over a quarter of CCLF grantees (28%) reported that they did not have to take any cost-management decisions between the end of the funding period and October 2024 when completing the survey. The picture was less positive for the coming winter period, with only 15% of respondents reporting that their organisation would not be likely to need to take any cost-management decisions between completing the survey and March 2025 (however, an additional 9% reported that they ‘don’t know’ or preferred not to say).

In addition, some VCSE strategic leads involved in the second wave of the case study research highlighted that receiving the CCLF put them in a better position financially. Getting through the winter period without having to use their reserves was also beneficial in the medium to longer term, and some were able to increase their reserves. The CCLF meant they did not have to dip into reserves and, despite facing additional financial challenges, case studies broadly reported that their reserves remain in a healthy place. Having this financial stability had further knock-on effects, as organisational representatives described feeling able to use that security or their reserves to be more strategic and think about longer-term sustainability of their services.

One is that we built up reserves, we strengthened our reserves in that time because in that period we didn’t spend any of the money from our membership fees, so it meant our reserves have been built up

- VCSE organisation strategic staff member, wave 1

Moreover, the representatives of organisations involved in the case study research reported that the retrospective funding and the ability to cover the costs of delivery staff allowed their organisations to free up some of their other resources and staff capacity. This was reported as helping them to plan strategically and focus on bid-writing and fundraising, particularly in respect of more senior staff who reported that the funding allowed them to free up their capacity or delegate some of their work. Doing so allowed them to focus more on management and strategic planning for the organisation, such as creating business plans or improving their monitoring. They were also able to spend more time fundraising, bidding for new grants, or looking into diversifying their income streams.

It meant that I could then focus on doing other fundraising and developing things. So, I was able to make an application to another trust, for example, because it gives you that breathing space.

- VCSE organisation strategic staff member, wave 1

Finally, many of the case study organisations also reported that CCLF funding had supported them to secure additional funding and partnerships. Interviewees described how the funding enabled them to increase their impact and consequently demonstrate their value to funders, show financial resilience which they felt presented them as more dependable to funders, and strengthen their track record in delivery, as well as in grant management and reporting. Box 7 below provides one example of this from the case study research. In addition, some VCSE leads also reported that they were able to invest more time into building new partnerships and networks with other organisations, as well as improving their marketing and community outreach, with staff from some of the organisations highlighting improved social-media presence.

If you’ve got financial stability, other grant funders will see that which can then provide the growth. So, there’s that future knock-on of just having stability with other people looking at you going forwards that can give you the growth or additional stability.

- VCSE organisation strategic staff member, wave 1

Box 7: Securing additional funding

One organisation visited as part of the case study research reported that they applied for the CCLF as the organisation was expected to be in a financial loss position after their financial review in 2024. It was crucial, therefore, that they were able to find additional resources to continue delivery of their services (described as ‘food, warmth and community’) on the same scale. Most of the funding was used to cover staff costs and utilities. Funding was also used for purchasing food and hygiene supplies, including FareShare orders, two freezers, and a cooker. Furthermore, the organisation lead reported that the CCLF provided financial security for the organisation for the upcoming year; hence it enabled them to think more strategically about their longer-term financial situation and secure more funding for future delivery. The retrospective funding also allowed them to free more financial resources and staff capacity, which enabled them to apply for more funding and consider other ways of diversifying their income.

Knowing that we’ve got some sort of buffer for us is really helpful. It gives us breathing space to look for more money and it allows us to plan.”*

- VCSE organisation strategic staff member

During the follow-up visit, the organisation lead further reported that the organisation’s income for 2024/2025 had increased as a result of successful grant applications. This included a successful bid for the Energy Efficiency Scheme (also funded by DCMS as part of the VCSE Cost of Living programme), which they expected would help them to reduce their high energy bills. Additionally, they also secured funding from the other funders which they used to provide food and other essential supplies between October 2024 and March 2025, as well as to hire a volunteer coordinator.

3.1.3. Outcomes for staff and volunteers

As well as supporting organisations on a financial level, the CCLF also aimed to have positive outcomes for staff and volunteers, particularly around retention and averted job losses and the ability to meet pressures over the winter period.

3.1.3.1 Ability to meet short-term pressures

There was evidence to suggest that CCLF funding helped to reduce pressure on staff and ease their workload over the 2023/2024 winter period. This was often a result of the funding being used to secure new staff and volunteers and freeing up the capacity of strategic staff, or pay for resources or equipment that helped staff do their work more efficiently. For example, larger or more kitchen equipment allowed larger amounts of food to be cooked at one time.

Some of the VCSE staff and volunteers participating in the case study research also reported feeling more confident and motivated in their roles, often due to taking on more or new responsibilities or attending training. This led to increased confidence amongst the delivery staff and volunteers in delivering their daily tasks. For some of the strategic staff, securing CCLF was a crucial ‘win’ that motivated them to continue applying for grants of bigger value.

It’s also been a bit of a psychological boost for me as a fundraiser because that was a big achievement, and it made me feel like I can do these bigger applications. So no, that was unexpected and really positive. It really gave me a massive lift when I suddenly had a lot of money I didn’t have to fundraise for.

– VCSE organisation strategic staff member, wave 1

The interviewed VCSE staff and volunteers also reported wider staff/volunteer benefits as a result of CCLF funding received, for example volunteers’ travel or food being covered or the introduction of an employee assistance programme. In addition, the VCSEs that used the funding for staff and volunteer training (e.g. safeguarding, first aid, mental health, trauma-informed approach) reported that it improved staff effectiveness in providing more holistic support. For example, one of the VCSE strategic leads shared that they were able to improve their internal training programme and induction process for volunteers. This allowed volunteers to identify their preferences and strengths. This approach was reported as enhancing volunteer satisfaction.

Now anyone who volunteers gets a thorough induction and every volunteer spends time in each section of the pantry so they understand what they want to do and then they can express a preference for what they like.

- VCSE organisation strategic staff member, wave 1

However, some of the VCSE staff participating in the case studies shared that they did not see the positive impact of the CCLF on their workload. In fact, some of those staff reported that they experienced increased pressure and workload in the short term. This was due to a couple of factors. Firstly, some VCSE staff reported an increased burden as they had to spend money within a short time frame. Secondly, other VCSE staff shared that the improvements made via the CCLF funding contributed to an even steeper increase in demand, leading to an initial increase in workload, which was later alleviated by increasing staff capacity.

When we had this CCLF funding, it was kind of a blessing in disguise for us in that it helped us get through. Well, the staff really felt motivated for that time. Particularly our senior staff because they were feeling a bit helpless, as you know. What can they do? Where can they get the money from? All of a sudden, the funding pressure was so enormous on us that, you know, it’s difficult to describe in words.

- VCSE organisation strategic staff member, wave 2

3.1.3.2. Staff retention and averted job losses

The survey results and case study data indicate that the CCLF funding meant organisations were able to avoid having to make staffing cuts, a desired outcome set out in the CCLF ToC. Prior to receiving CCLF support, 22% of wave 1 survey respondents reported already having to reduce staff hours due to financial constraints, and 53% anticipated they would have had to without the funding. However, after receiving CCLF funding, only 7% reported needing to make such cuts (see Table 6.10, Annex 6).

Similarly, while 8% of wave 1 survey respondents had made redundancies before receiving CCLF funding and 24% believed they would have had to without it, only 3% reported making redundancies during the funded period. Moreover, nearly half (46%) of respondents stated that, without CCLF support, their organisation’s workforce would have decreased during the autumn/winter of 2023-2024. These insights highlight the role of CCLF funding in safeguarding jobs and volunteer engagement, helping VCSE organisations maintain their workforce despite the sector’s financial challenges (see Table 6.10, Annex 6)

The process tracing confirmed that for many organisations it was likely that CCLF contributed to them being able to retain staff and volunteers. As can be seen below, in or over a quarter of organisations there was a high chance that this was true (see Box 8).

Box 8: Likelihood of CCLF contributing to VCSE organisations retaining staff / volunteers

The results of the process tracing analysis indicate that it is likely for some VCSE organisations that CCLF contributed to them retaining staff and volunteers, as can be seen below:

244 VCSE organisations (27%) had a greater than 60% chance that CCLF contributed to them retaining staff and / or volunteers
248 VCSE organisations (27%) had a greater than 40% chance that CCLF contributed to them retaining staff and / or volunteers
417 VCSE organisations (46%) had a greater than 20% chance that CCLF contributed to them retaining staff and / or volunteers

Full details can be seen in Annex 6, Table 6.23, Table 6.24 and Figure 6.5

Many staff interviewees engaged through the case studies also reported that their organisations had been able to retain staff and avoid job losses over the winter period due to the funding received. One organisational representative noted they might have faced a situation where staff thought about leaving because of financial uncertainty, as some had started looking at other jobs before the funding award. Having the funding gave staff the reassurance and stability to stay at the organisation, at least in the short term. Another VCSE strategic lead highlighted that the CCLF allowed their organisation to improve staff wages and pay them a ‘fair salary’. In some cases, VCSEs organisation representatives reported being able to increase staff hours for those who worked part-time and provide them with full-time positions or move some of their volunteers into paid roles.

For us to be able to buy that capacity and turn staff into a full-time position for that time period, get another member of staff in, it’s been ideal.

- VCSE organisation strategic staff member, wave 1

Case study data also suggested that the CCLF allowed a number of grantee organisations to hire new staff and recruit volunteers, as the funding covered costs related to the recruitment and the salaries of the new staff for the funding period. In some cases, the organisation concerned invested in advertising their volunteering roles across different platforms. More generally, a number of staff participating in case study interviews reported that they saw a significant increase in interest in volunteering within their organisation, as the CCLF allowed them to invest more resources into promoting volunteering opportunities. Some interviewed VCSE leads highlighted that increasing numbers of volunteers was crucial during the winter period, as staff are more likely to be off sick and they previously struggled to cover the sick staff members and had to close their service temporarily. Volunteers were seen as helping to alleviate this by providing cover.

That really reduced how much we had to close because over the winter months we get a lot of staff sickness. [CCLF] enabled us to take on more mentors to work with more children. So, it meant that we were able to support more young people.

– VCSE strategic staff member, wave 1

Having the opportunity to hire new staff and volunteers also meant that organisations were able to fill gaps in their workforce and recruit people with specific skill sets. For example, some organisations were able to hire administrative staff, which allowed strategic staff to focus on management and planning and front-line staff on service delivery (see Box 9 for an example). Another VCSE representative emphasised that the CCLF allowed them to hire people with language skills and cultural awareness of the minority communities they support. This was also appreciated by their beneficiaries.

Box 9: Pathways to volunteering and paid opportunities

One of the grant holder organisations applied for funding to increase their staff and volunteer capacity, to help them expand their delivery abilities, and provide opportunities for service users. The grant was used to fund two volunteers into full time roles, which meant the management team could invest more time in delivering training and establishing more partnerships.

Research during Wave 1 of the case study highlighted that providing volunteers with paid employment had benefits for the new staff members, such as a regular and reliable income, as well as greater commitment to the role and opportunities for progression. Interviewees that experienced this transition emphasised an increased sense of pride in their role. Similarly, some beneficiaries were offered volunteering opportunities too, which they were then able to put on their CVs.

The wave 2 visit revealed that more volunteers had been given opportunities through the organisation, as well as some work experience opportunities. Some of the new volunteers interviewed had previously been service users at the organisation, indicating a service user to volunteering opportunity pathway had been established. Beyond the funding period, the two staff members who were given paid work were able to continue their roles and continue working for the organisation. This shows the longer term impact the grant had for this organisation, as they were able to establish and embed paid employment opportunities, which helped to increase the organisation’s overall delivery capacity.

Other organisations were able to attract more beneficiaries into volunteering roles. Some of the interviewed volunteers shared that they decided to volunteer with the organisation to give back to the community and support people with similar life experiences. This further contributed to developing peer support in some of the CCLF-funded services. In another case, an organisation paid their volunteers during the funding period to accommodate increased demand. While following the funding period the individuals returned to voluntary roles, the organisation concerned is able to still pay these individuals for small tasks to thank them for their time.

The funding helped to take the pressure in terms of paying the volunteers. They got something out of it, and for us, as an organisation, it was really important that we reward the volunteers.

- VCSE organisation strategic staff member, wave 1

Some VCSE staff reported that increased hours or changing roles of staff and volunteers allowed them to provide dedicated volunteer coordination and oversight. However, one of the VCSE strategic leads shared that, although they appreciated that CCLF allowed them to recruit new volunteers, they found it challenging to manage a higher number of new volunteers as some of them had complex needs themselves. Representatives from some of the organisations taking part in the case study research, however, pointed out that recruiting and retaining volunteers remains a challenge, and the ability to recruit the right people into posts had been a barrier to the potential impact the funding could have. For example, staff at one case study organisation reported they had not been able to expand their service as planned as they couldn’t find suitable volunteers.

3.1.3.3. Sustaining positive effects on resourcing and capacity beyond the funding period

There is evidence to suggest that, broadly, organisations were able to retain their staff and volunteers following the funding period, although the wider challenging environment for the sector meant this was not always possible. In the period between March 2024 and October 2024, some of the surveyed organisations in the Wave 2 survey reported having to make cost-management decisions in relation to their staff. As can be seen in Figure 3.6 below, 22% of successful grant holder survey respondents reported having to reduce staff hours in that period, 11% reported having to make staff redundancies, and 6% reported having to ask staff to take a pay cut or forgo a pay rise.

Organisations that had not received CCLF funding, however, were more likely to have to take these cost saving measures, with 37% reporting that they had to reduce staff hours, 18% having to make staff redundancies, and 16% reported having to ask staff to take a pay cut or forgo a pay rise.

Figure 3.6: Staffing related cost-management decisions taken between March 2024-October 2024

Source: Wave 2 grant holder survey (n=316), wave 2 unsuccessful applicant survey (n=193)

It was also evident that some organisations face ongoing difficulties with retaining staff. For example, staff from one case study organisation noted employees facing challenges in meeting the increased cost of living in their own lives and increasingly looking for higher-paid roles elsewhere. It was also noted by some case study organisations that upcoming changes to employer national insurance and the minimum wage would have an impact on their financial situation and consequently their ability to retain staff. Figure 3.7 shows that nearly a third of respondents in the Wave 2 survey (28% of grant holders and 34% of unsuccessful applicants) anticipated that they would need to reduce staff hours between the time they completed the survey (November / December 2024) and March 2025.

Figure 3.7: Staffing related cost-management decisions anticipated between survey completion and March 2025

Source: Wave 2 grant holder survey (n=316), wave 2 unsuccessful applicant survey (n=193)

Similar findings were seen in the case study research, with the majority reporting that they had been able to retain staff and volunteers following the funding period. The ability to retain staff beyond the funding period was often down to additional funding being secured, or changes to their role. One organisational representative also described how they felt bringing in additional roles using CCLF funding had made other roles feel more sustainable and thus prevented staff from leaving the charity. However, a small number reported not being able to retain roles following the funding period or having to let staff go. In one case, short-term capacity support brought in through CCLF had not continued fully but some staff and volunteers stayed on and absorbed the work of the individual concerned, putting increasing pressure on those remaining.

[A] couple of people left so, the workforce reduced, but we stepped up. We knew that we didn’t have so much hands on ground, but in a way we’ve been able to pull through.

– VCSE organisation delivery staff member/volunteer, wave 2

3.2. Outcomes for beneficiaries

3.2.1. Short-term outcomes

3.2.1.1. Immediate needs met

As a result of the services delivered by funded organisations, people’s short-term basic needs were generally reported as being met. Most grantee organisations participating in the case study research provided food in various forms, whether as a café providing free (or subsidised) hot meals, a community pantry providing food for a reduced fee, or a foodbank supplying free emergency food parcels and food vouchers. Beneficiaries who used these services reported that the services helped them feed themselves and their families, as they would have struggled to cover their food costs otherwise.

Some food services were able to diversify their offer and provide more fresh and whole food, often alongside cooking lessons, provision of recipe ideas, and giving tips on reducing food waste or meal preparation. Staff and volunteers from these organisations suggested that these factors helped beneficiaries not only ‘stretch their money further’ but also eat healthier.

I think it helped us increase the healthy choices that we provided to service users – so, more fruits and vegetables. We also bought the air fryer to help us out with using less oil. We didn’t used to do that before.

- VCSE organisation strategic staff member, wave 1

According to qualitative evidence from the case studies, services providing emergency supplies, such as toiletries and period products, baby supplies, or bedding and SIM cards for people facing homelessness, also reduced the immediate financial pressures on individuals and families in challenging financial situations. Some interviewed beneficiaries said they repurposed the money they would have used for food and other emergency supplies to pay their bills or purchase clothes for their children. One of the beneficiaries described being forced to decide between buying food or paying their bills.

It’s really helping to save some money so you can buy whatever you need extra.

–VCSE organisation beneficiary, wave 2

3.2.1.2. Accessing suitable safe spaces

Services supported by CCLF funding also created warm and safe spaces for their service users. The VCSE staff and volunteers participating in case study research highlighted that ‘safe spaces’ varied in their nature, including safe spaces for women, children and young people, the elderly, and migrant communities, as well as emergency shelters for homeless people. Alongside warmth and safety, emergency shelters for homeless people also provided users with facilities to wash their laundry, take showers, charge their phones, or use equipment such as computers and phones to access the internet.

Observations and interviews showed that the hubs, kitchens, cafes and centres visited were highly valued in the community by beneficiaries, while the wave 1 survey found that 73% of grantee respondents felt that the majority of their service users experienced their organisation as a safe space (see Table 6.15. in Annex 6 Organisations often focussed on delivering a range of wellbeing, sports and social activities to bring people in whilst creating warm and safe spaces for the most vulnerable groups to stay. It was evident that regular visitors to the organisations engaged with groups and networks, such as women’s and men’s groups, youth activities, language groups, computer classes and day trips that the services organised. The social and well-being benefits of these were widely reported. Specifically, beneficiaries noted that they met new people and created friendships with other service users, which had a positive effect on their mental health.

My depression would have been really bad if I had not had all these different things where I could talk to people and participate in activities.

- VCSE organisation beneficiary, wave 1

Interviewed beneficiaries, especially the elderly and those who faced loneliness or isolation, reported that having space where they could build relationships and receive targeted support improved their well-being. For example, beneficiaries at one case study site reported that they found the structure and stability that visiting the centre gave them highly valuable, as they could better manage their time day-to-day and feel less like they are stuck in a rut.

I live on my own, so it’s good for me to come down here. I make so many friends […] it’s a structure in your life.

- VCSE organisation beneficiary, wave 1

Interviewed beneficiaries who were also parents reported the benefits of having a safe space to spend time with their children, particularly where low-cost activities were offered. Some organisations ran youth clubs that young people could visit, giving them something to do after school. Other examples included £1 movie nights, free after-school clubs, and hot meals for families who struggle to provide this. Parents appreciated the low or no-cost services for their children, especially as these provided warm spaces in the winter months for parents struggling to heat their homes. Furthermore, some parents emphasised the positive impact these services have on their wider family lives as well as positive behavioural outcomes for their children.

3.2.1.3. Accessing financial and housing advice

Many grantee organisations provide financial and housing advice, for example to those who need help with managing debt and finances, navigating difficult relationships with landlords, or those who have been made homeless. Several staff and beneficiaries from grantee organisations highlighted during the interviews that they conducted benefits eligibility checks and helped beneficiaries apply for benefits to which they were entitled, such as Universal Credit or Personal Independence Payment (PIP). Some grantee organisations provided additional support to their beneficiaries by securing direct financial support via other available funds, such as the Cost of Living Crisis Fund or Household Support Fund, to help them pay bills or secure rental deposits. This support was largely appreciated by the interviewed beneficiaries, highlighting that the financial support they received via benefits or other funds was instrumental in stabilising their financial situation.

It takes quite a few weeks from start to finish until they actually get the benefits – but people I supported were saying, ‘Well now I can use this money to have my heating on more’.

– VCSE organisation delivery staff member/volunteer, wave 1

However, there were mixed views on the impact and effectiveness of courses or workshops aimed at helping people manage their finances via CCLF-funded organisations. During interviews, staff and volunteers from organisations delivering these services cited that the activities positively impacted people’s money management. However, interviewed beneficiaries had mixed views on the impact and effectiveness of this approach. While some felt that the advice enabled them to manage their limited budgets better or, in some cases, save money to repay debts, other beneficiaries reported minimal impact on their finances.

Some CCLF-funded organisations also provided housing advice, often linked to wider homelessness support. The support ranged from helping people apply for social housing, signposting them into temporary accommodation, or liaising with landlords to help them sustain their private accommodation if they struggled with payments. The delivery staff from those organisations noted that people at risk of homelessness often included people from migrant communities, including people who were recently granted asylum. For those beneficiaries, the support was often extended to helping them secure identification documents, setting up bank accounts, applying for visas or other essential documents needed before accommodation applications could be made.

I am leaving here without stress. I am getting my paperwork cleared, and I am not scared. I got accommodation from their advice.

– VCSE organisation beneficiary, wave 2

3.2.1.4. Signposting and use of other services by beneficiaries

Many grantee organisations provided holistic support to beneficiaries accessing CCLF-funded services by providing further mental and physical health support, employment and education advice, digital inclusion support, or wider family support. Additionally, interviewed staff and volunteers from grantee organisations often reported that they regularly signpost their beneficiaries to other organisations and services to access relevant support. They emphasised the importance of signposting their beneficiaries to access wider support to meet their complex needs, often mentioning their mental health and wellbeing needs. However, qualitative evidence from the beneficiary interviews provided limited examples of signposting to external services.

We signposted around benefits and a lot of signposting around mental health. That’s just a big and, unfortunately, growing area.

- VCSE strategic staff member, wave 1

Organisations already delivering a range of activities were also signposting their beneficiaries to other in-house services. The qualitative evidence from interviews and observations across many case study organisations showed that once a service user had engaged with one part of the organisation, they began engaging with other activities and groups provided by the organisation. Often, this included in-house employment support (writing CVs, job searches, mock interviews), digital skills courses, or well-being activities. In some cases, the grantee organisations also provided one-to-one mental health support via counselling or mentoring.

Other grantee organisations offered a ‘hub’ model, working with partners and commissioning other suppliers to bring specialised services into their organisations. Often, this included financial or legal advice services (e.g. Trent Citizens Advice, ‘Severn Trent Water Big Difference scheme ‘Green Dr’ or health and social services (e.g. NHS outreach teams, Family Hubs). On various days representatives from these services would come in and be there for a morning or afternoon, holding drop-in sessions. Interviewed staff and volunteers from grantee organisations noted that operating these services in spaces where beneficiaries already feel safe meant they were more likely to open up and discuss their needs. Likewise they noted that this was less intimidating for beneficiaries than going somewhere new and that it helps build a sense of trust.

If the worker is here, people are much more likely to engage, whereas some would be afraid to go up to the council, for instance.

– VCSE organisation delivery staff member/volunteer

Over a half (54%) of the wave 1 grant holder survey respondents reported that at least some beneficiaries were signposted to other services, whilst over a third (35%) reported that they signposted ‘the majority of their beneficiaries’ to external services (see Table 5.15. This was further echoed by interviewed staff and volunteers from grantee organisations who reported they signposted their beneficiaries to external services so they could get the support that the grantee organisations could not provide themselves. This included signposting to a range of statutory services (e.g. housing teams, job centres, social care, children services, local college courses, GPs, mental health and addiction support, and other NHS services) and services provided by other voluntary organisations (e.g. food banks, Citizens Advice, or VCSEs specialising in financial and debt advice). Interviewed staff and volunteers also reported that they often go beyond just signposting beneficiaries in that they help them to physically access the service or accompany them if they do not have the confidence to visit alone.

We support with everything and anything, from doctors’ appointments to court appointments. We discuss with the clients if they might benefit from a [signposted] service. We’d look into it and recommend various organisations or services and ask if they’d like us to introduce them. We often physically attend appointments to support them with accessing that as well.

– VCSE organisation delivery staff member/volunteer, wave 1

Several strategic staff from the grantee organisations also emphasised the importance of signposting in terms of receiving referrals, typically from the same organisations they also signpost people to, or from other statutory services such as local schools. Some of the interviewed staff likewise noted that they work with social prescribing services often linked to the NHS, working with GP surgeries, social workers, and youth justice services and receiving referrals that way.

3.2.2. Medium to longer-term outcomes

Through taking a two-wave approach to the case study research and the survey, the evaluation sought to gather early evidence of medium to longer-term outcomes for service users. It is important to note that, whilst most individuals were felt to go on to experience medium to longer-term outcomes, some organisational representatives were more explicit about these than others. For example, some services focussed explicitly on skills building, education support, or longer-term changes to a person’s situation. Others were more short-term focussed in their activities, although some medium to longer term outcomes were still reported. In addition, some strategic staff highlighted the preventative aspect of their services and emphasised that, as a result of their services, issues faced by their beneficiaries are under control and that support prevents further worsening of their situation.

Across the case study data gathered, the following medium-longer term outcomes were reported:

Reduced impact of poverty – participants described the positive effects of accessing advice and support on their finances and, as a result, experiencing reduced impacts of poverty. In some examples, beneficiaries had been able to claim benefits they were entitled to or find out more about different housing options available to them, to which had a longer-term financial impact. Others reported being able to save money or meet other costs due to being able to access a foodbank or pantry, or receiving assistance with utility bills which helped to mitigate the impacts of poverty on their day to day lives. The provision of services that provided safe spaces and activities also contributed to this, as they offered places for people to go for little to no cost. For example, youth services were reported as supporting families to keep children entertained during school holidays without high costs.

We had a couple of people who we have supported who we helped get into the social housing with the council, who have managed to get housing benefit, universal credit, rent deposit to enable them to kind of stay or find new accommodation

– VCSE organisation delivery staff member/volunteer, wave 1

Mental and health and wellbeing – beneficiaries reported the positive effects of accessing services on their wellbeing and mental health. Often it was the social aspect of accessing services which had this effect, or the positive approach and support from staff and volunteers. In other cases, support with this had been more targeted, for example supporting someone to access counselling or speaking with a trained mental health first aider. Respondents to the wave 2 survey also reported that as a result of the CCLF funded service, either the majority of beneficiaries (47%) or some of their beneficiaries (48%) experienced an improvement in their mental health and wellbeing.

Before I came here, I struggled with so many health problems, mental health, physical and then ever since I came here speaking to the people here it’s just helped.

– VCSE organisation beneficiary, wave 1

Physical health - within some case studies beneficiaries also experienced an improvement in their physical health as a result of accessing services. This was often due to activities relating to food, for example providing healthy food through a pantry or foodbank or offering cooking classes. Where this was relevant to the services being provided, survey respondents also reported that their beneficiaries experienced an improvement in their physical health. 22% of respondents to the wave 2 survey reported that the majority of their beneficiaries experienced this, and 58% reported that some of their beneficiaries experienced this. The majority of the remaining respondents reported that this outcome area is not relevant to their CCLF funded service (17%).
Educational and employment outcomes – although not the focus of all organisations and services, for those where this aspect was relevant positive outcomes around education and employment were reported. This was as a result of a range of activities, such as access to tutoring, ESOL support, volunteering opportunities or on site employment support. This outcome area was least common amongst survey respondents, with 35% reporting that education and employment were not relevant to their CCLF funded service. However the majority for whom it was relevant reported that beneficiaries accessing the CCLF funded service were supported with accessing education and employment opportunities, with 11% reporting that the majority experienced this, and 48% reporting that some beneficiaries experienced this.
Improved skills and confidence – some participants also described how accessing services had supported beneficiaries to improve their skills and confidence. For example, some reported feeling more confident, or empowered, to access information and support that they need and therefore likely to be less reliant on services.

The little things like they would learn how to access a GP appointment, which they probably couldn’t do to start with, but by learning better English they overcome quite basic things like that.

– VCSE organisation delivery staff member/volunteer, wave 2

As well as accessing information and support through CCLF funded services, as described in the previous section, many beneficiaries were also supported through other organisations and other internal services that they were signposted to by the CCLF funded service. As a result, outcomes in some instances are not only a result of the individual service, but other services signposted to. By being able to signpost to other services, some beneficiaries reported that they were in a stronger position longer term to seek out the support that they needed. Staff likewise reflected on this aspect:

[A large portion] have now got the knowledge and feel empowered and got the capacity to deal with those issues themselves because there was a raft of information that was provided about where you could get support from as well.

-VCSE organisation strategic staff, wave 2

Despite short, medium and longer-term outcomes being identified, there was still unmet need evident for beneficiaries/service users. Some beneficiaries reported how they had remaining needs which the service had not been able to meet. Others described how service users were still coming back to the organisation as, despite the support they had received, there had been no material change to their personal circumstances.

We have the same people coming in the door now that we did a year ago […] because things haven’t gotten better. So it means that we’re still supporting a lot of the same regular families.

– VCSE organisation delivery staff member/volunteer, wave 1

3.3. Wider outcomes

During the case study interviews, staff and volunteers from participating organisations also reported unintended outcomes of participating in the CCLF. These included:

Organisations’ reputation: Staff and volunteers reported a positive impact on their organisation’s reputation amongst other local services, the community, and other potential funders, as they were able to manage a large grant and implement improvements to their services. This also led to increased interest from their partner organisations; for example, by using their space or further networking.

I don’t think we anticipated how many doors it [CCLF] would open.

– VCSE organisation delivery staff member/volunteer, wave 1

Improvements led to even higher demand: The improvements that the CCLF enabled continued to increase the ‘popularity’ and demand for their service.

I think when you’ve had funding, and people know you’ve got a service, it naturally attracts more people. So, I would say that knowing that that service is available has probably given them more people to serve, which is good. But [it is] bad if you haven’t got the resources to [meet the demand].

- Local organisation stakeholder, wave 2

Increased trust from the community and beneficiaries: Several staff and volunteers emphasised that the continuation of their services meant that the beneficiaries and community could rely on the funded organisations to deliver the support they needed.

It enables us to be consistent, […] rather than being in a situation where we have to close and say we don’t do Fridays and then someone turns up and loses faith in a service […] It helped us to build our community and trust with people and diversify the areas of support that we provide as well.

- VCSE organisation strategic staff member, wave 1

Question 5

4.0 Economic evaluation

Accepted Answer

This chapter presents the economic evaluation of the CCLF, assessing its costs and benefits, and concluding with an overall assessment of the value for money of the CCLF.

Chapter summary

Costs and Underspend: The CCLF cost £75,542,252 over two years (2023/24 and 2024/25), which was an underspend of £427,748 compared to the original estimated cost. This underspend was mainly due to efficiencies in grant management by The National Lottery Community Fund.

Benefits and Impact: The counterfactual impact evaluation (CIE) assessed the causal impact of CCLF funding on the financial resilience of VCSE (Voluntary, Community, and Social Enterprise) organisations. The findings suggested that CCLF funding had a positive impact on the financial health of VCSE organisations. Wider evidence also suggested that there were benefits of the funding to end beneficiaries of VCSE organisations, as well as staff and volunteers.

Additionality – Benefits to VCSEs: The total estimated benefits (additional to the comparator group) on funded organisations’ finances was £31.7 million in net income (an approximation of overall financial health) and £35 million in net expenditure (supporting service delivery).

Additionality – Benefits to VCSE beneficiaries: The total value of potential benefits to beneficiaries is estimated to be £158,987,181. With costs of the CCLF being estimated at £75,542,252, the benefits of the CCLF are more than double its costs considering the benefits to individuals from the CCLF alone, according to the analysis undertaken. However, this finding should be treated with some caution as it relies on several assumptions (due to precise data on the numbers of beneficiaries supported with specific services – e.g. food support – not being available and there being a potential for double counting within the estimates produced; hence potentially overestimating the amount of benefits likely to accrue).

Overall Value for Money: The overall assessment concluded that the CCLF generated substantial benefits in net income, gross expenditure, and gross income (which was approximately equal to the overall value of grants awarded). In addition, the CCLF generated a positive return through the benefits realised by beneficiaries. The CCLF’s positive impact on financial health, service provision, and organisational resilience should be considered when assessing its overall value and making decisions about any future funding.

Long-term Impact: The evaluation focused on a relatively short timeframe and the long-term impact of the CCLF on organisational sustainability and financial health may be greater than that captured in the current analysis.

Indirect Economic Benefits: Based on the evidence presented in preceding chapters, the CCLF is likely to have generated various indirect economic benefits, stemming from increased service provision, improved beneficiary outcomes, and enhanced organisational resilience

4.1. Costs of the CCLF

The CCLF cost £75,542,252 over two years (2023/24 and 2024/25). This was an underspend of £427,748 compared with its original estimated cost of £75,970,000. Most of the underspend (£384,258 – i.e. 90% of the £427,748 total underspend) was due to efficiencies by The National Lottery Community Fund in grant management (i.e. not the grants themselves). This was due to reduced staff resources, and associated oncosts, travel and expenses required to deliver the programme. The National Lottery Community Fund’s underspend (£384,258) represented a 9% saving on the planned grant management administration budget, spending £3,815,742 against a budget of £4.200,000.

The £75,542,252 costs of the CCLF comprised:

£74,715,742 for The National Lottery Community Fund, comprising:
- £70,900,000 grants awarded
- £3,815,742 grants administration – this included an estimated £150,000 spent expanding internal capacity to deal with an increase in Awards for All applications because of the signposting from CCLF.
£826,510 for DCMS, comprising:

Other costs may include overheads that are part of the organisations’ concerned operations and support functions (i.e. so were not specific to CCLF) such as data, communications, evaluation, service design, IT and Human Resources. The evaluation did not identify any other costs or indirect costs for The National Lottery Community Fund and DCMS to be considered such as match funding, referrals to the programme or other in-kind usage of buildings, facilities or other programmes.

4.2. Economic assessment of the CCLF – benefits to VCSEs

This section presents a focused economic assessment of the CCLF. Building upon the results of the counterfactual impact evaluation (CIE) presented in the previous chapter, this analysis focuses specifically on organisations with an annual income of less than £10 million. This subset captures the vast majority of funded organisations (only 13 were removed) without being overly influenced by the financial dynamics of very large charities. The primary aim of this assessment is to explore the additionality of CCLF funding on the financial position of VCSEs.

The CCLF funding was designed to be spent within a specific timeframe with the primary aim of supporting VCSE organisations to deliver services to meet the needs of beneficiaries in the face of increased cost of living pressures. However, there was also an underlying intent to support VCSE organisations to navigate cost of living challenges which in turn would bolster the financial health of VCSE organisations, enabling them to emerge stronger in the long run. This assessment delves into the sustainability of the observed financial improvements, recognising that the benefits of the CCLF may extend beyond the immediate period covered by the CIE data (Charity Commission Annual Returns for the 2023 to 2024 financial year).

The economic assessment centres on the concept of “financial health” and utilises the CIE results as a foundation. Specifically, it examines the following outcomes:

Net income: Improved net income, calculated as gross income minus gross expenditure, signifies enhanced financial sustainability and resilience, allowing organisations to better withstand financial shocks and plan for the future. This may also include better reserves, which can be important for long-term financial stability
Total gross expenditure: Increased expenditure suggests that the funding enabled organisations to increase their spending on service delivery or expansion, potentially leading to greater (longer-term) impact and reach, and staff retention
Total gross income: The analysis includes gross income as an outcome, although the estimated increase (from the CIE) is very similar to the average funding amount. This suggests that the primary impact of the CCLF on gross income is the direct infusion of the grant itself, rather than enabling organisations to attract additional funding. While this direct benefit is valuable, the focus of the VfM assessment will be on the other outcomes, net income and expenditure, which provide insights into the broader financial impact of the CCLF

Table 4.1 presents the average treatment effects for the organisations with less than £10 million income that were included in the CIE. Whilst not all results were statistically significant at the 5 per cent level, the results in context (including the range of organisation funded) were considered strong enough evidence of likely CCLF benefits. For completeness, lower and upper confidence intervals are provided, which provide a range within which the true estimate (or benefit) is likely to fall.

Table 4.1: Average treatment effects for organisations with less than £10 million income

	Average treatment effects	Average treatment effects	Average treatment effects
Outcome	Estimated treatment effect (£)	Lower confidence interval (£)	Upper confidence interval (£)
Net income	21,174	-245	42,592
Gross expenditure	23,340	-7,551	54,231
Gross income	44,514	10,908	78,120

Table 4.2 provides the estimated total treatment effects, which are calculated by multiplying the average treatment effects (the ‘benefits’) by the total number of funded organisations (1,500). Whilst this extends beyond the 816 funded organisations (where there was Charity Commission data covering a set period) included in the CIE analysis, including all funded organisations provides an indication of the overall additionality of CCLF funding.

Table 4.2: Estimated total treatment effects (1,500 funded organisations)

	Total treatment effects / additionality	Total treatment effects / additionality	Total treatment effects / additionality
Outcome	Central estimate	Lower estimate	Upper estimate
Net income	£31,761,000	£0	£63,888,000
Gross expenditure	£35,010,000	£0	£81,346,500
Gross income	£66,771,000	£16,362,000	£117,180,000

Note: Negative treatment effects limited to £0.

The key observations from Table 4.2 include:

The overall additionality of CCLF funding, relative to the comparator group that potentially had to use more of their reserves, on all financial outcome measures was substantial
For net income, which can be considered an approximation of overall financial health, £31.7 million additionality to the sector was estimated
For gross expenditure, which most directly relates to increased/sustained service delivery (see next section for further exploration of this), additionality was estimated to be around £35 million
For gross income, the estimated £66.7 million additionality was close to £70.9 million in grants awarded, which suggests the CCLF funding did not crowd out other funding

4.3. Economic assessment of the CCLF – benefits to beneficiaries

In addition to the economic assessment of the CCLF to organisations, the assessment was extended to incorporate potential benefits to individuals supported by CCLF organisations. It should be noted that this analysis should be treated with some caution as it relies on several assumptions as explained below (due to precise data on the numbers of beneficiaries supported with specific services – e.g. food support – not being available and there being a potential for double counting within the estimates produced; hence potentially overestimating the amount of benefits likely to accrue).

In estimating potential economic benefits relating to end beneficiaries of CCLF support, the increased expenditure estimated through the counterfactual impact evaluation was used to approximate potential impacts on the number of service users maintained or increased as a result of the funding. The steps taken to calculate the number of potential beneficiaries of the CCLF involved the following:

Taking the estimated increased expenditure, of £23,340 for organisations with less than £10 million income, which represents around 3 per cent of the average annual expenditure of these organisations (£781,337, based on Charity Commission data)
Following this, calculating the average number of individuals supported by each CCLF funded organisation (prior to funding), this figure being 6,497 (source: CCLF application data)
Subsequently, assuming a relationship between changes in expenditure and the number of individuals supported, estimating that on average 194 individuals maintained their existing support or started to receive support due of the CCLF per organisation (i.e. 3 per cent of the 6,497)
Using the 816 organisations that were included in counterfactual impact evaluation (where Charity Commission annual returns were submitted between December 2023 and March 2024, and where their overall income was less than £10 million), the total number of estimated beneficiaries would thus be 158,367
If all 1,500 CCLF funded organisations are considered, the total estimated number of individuals where support was maintained or newly received would thus be 291,116

Potential benefits were then identified and calculated based on the types of service delivered by CCLF organisations. The precise services provided by CCLF funded organisations varies, but were broadly grouped as providing:

Food and supplies
Advice
Warmth
Safe space
Shelter

Most organisations provided multiple services, leading to some potential for double counting and overestimation as noted above, but the available data can still be used to identify potential appropriate benefits and the number/proportion of individuals these might apply to. This is illustrated in Table 4.3.

Table 4.3: Benefits to CCLF beneficiaries

Benefit	% of beneficiaries	No. of beneficiaries	Unit cost from full Business Case (FBC)	Value of benefits (£)
Food and supplies	77%	224,159	£243.50	£54,582,794
Advice	58%	168,847	£198.50	£33,516,185
Warmth	41%	119,358	-	-
Safe space	31%	90,246	£785.50	£70,888,202
Shelter	8%	23,289	-	-
TOTAL	-	-	-	£158,987,181

The total value of potential benefits to beneficiaries is therefore estimated to be £158,987,181. This comprises:

£54,582,794 for food and supplies. Table 4.3 above illustrates that 77% of grant holders delivered food and supplies, which equates to 224,159 out of a total of 291,116 beneficiaries. Applying a unit cost of £243.50 from the Full Business Case ^{[footnote 25]}, provides total benefits of £54,582,794
£33,516,185 for advice. Table 4.3 above illustrates that 58% grant holders delivered advice, which equates to 168,847 out of a total of 291,116 beneficiaries. Applying a unit cost of £198.50 from the Full Business Case ^{[footnote 26]}, provides total benefits of £33,516,185
£70,888,202 for providing a safe space. Table 4.3 above illustrates that 31% of grant holders provided a safe space, which equates to 90,246 out of a total of 291,116 beneficiaries. Applying a unit cost of £785.50 from the Full Business Case ^{[footnote 27]}, provides total benefits of £70,888,202

Shelter (Table 4.3 above illustrates that 8% of grant holders provided shelter) was not included in the calculation to avoid further potential double-counting due to its similarity to providing a safe space. Warmth (Table 4.3 above illustrates that 41% of grant holders provided warmth) was also not included due to its similarity in the benefits realised from a safe space and supplies.

With costs of the CCLF being estimated at £75,542,252, the benefits of the CCLF are more than double its costs considering the benefits to individuals from the CCLF alone, according to the above analysis. The counterfactual impact evaluation accounts for economic ‘deadweight’, so no further adjustment for deadweight (as in the Full Business Case) is required.

4.4. Overall assessment of value for money

Building on the results from the CIE, the CCLF showed the potential to generate substantial benefits to funded organisations in the form of net income and gross expenditure and income. Using the increases in gross expenditure as an approximation for additional/sustained service delivery, the overall benefits to individuals were high and far exceeded the investment made in the CCLF.

In terms of the National Audit Office’s value for money framework national audit office, as well as these findings on effectiveness (maximising outcomes such as financial resilience while minimising inputs – ‘spending wisely’) and equity (the extent to which services are available to and reach all people that they are intended to – ‘spending fairly’), the CCLF also showed evidence of economy (minimising the cost of inputs or resources used or required – ‘spending less’) and efficiency (maximising outputs such as staff costs from services while minimising inputs – ‘spending well’) by delivering within the £75,970,000 budget, and realising efficiencies in grant administration to achieve an overall saving of £427,748. From the case study research and MI returns there were examples of efficiencies through bulk buying (mostly food), sourcing discounted rates, or investing (sometimes in combination with other grants or funding) to save money overall – for example, e.g. buying equipment or transport instead of contracting out. These efficiencies were secured despite the tough economic situation, increasing demand on services and high utility bills. Any efficiency savings were reinvested in delivery.

Moreover, the evaluation focused on a relatively short timeframe. The long-term impact of the CCLF on organisational sustainability and financial health may indeed be greater than what is captured in the current analysis. In addition, the economic evaluation has been based only on the CIE and direct financial benefits. The CCLF may have generated various indirect economic benefits captured in the impact evaluation (see section 3.0 of this report), such as increased service provision, improved beneficiary outcomes, and enhanced organisational resilience. These benefits are difficult to quantify in monetary terms but should be considered when assessing the overall value of the CCLF.

Question 6

5.0 Conclusions and recommendations

Accepted Answer

This chapter sets out the key conclusions from the process, impact and economic evaluation, before highlighting some recommendations for future, similar response schemes.

5.1. Conclusions

This report has presented the findings from the evaluation of the CCLF, outlining evidence gathered in relation to the process evaluation, outcomes and impact, and value-for-money. Across the evaluation, several key themes have emerged:

Process evaluation

At a programme-level, the CCLF was delivered in line with the intended activities and outputs. While it met its targets in terms of the number of organisations receiving a grant, it was also over-subscribed, with around 40% of applicants awarded funding. This signals the strong demand for this type of crisis funding
Key elements of the CCLF’s design that programme stakeholders and VCSEs felt worked well included having the retrospective funding, the flexibility to use the grant for core costs, and the partnership structure (with the inclusion of the VCSE Advisory Board). VCSE organisation representatives awarded funding shared their positive reflections on using funding flexibly in practice and being able to reclaim the costs they already incurred during the early winter period. Indeed, analysis of how projects used the funding indicates the strong use of funding for costs such as staff salaries, volunteer costs, overheads (e.g. utilities) and capacity-building
The experiences of the design and set-up phase of the CCLF were generally positive, with stakeholders highlighting the collaborative approach, the availability and expertise of the IGM, and the use of existing tools and mechanisms as key factors for success. Challenges during the design and set-up phase included condensed timescales, some Advisory Board members’ uncertainty about the focus of the CCLF, complexity of designing the retrospective claim element, and the need to design and implement grant management processes in parallel
Most organisations heard about the CCLF via The National Lottery Community Fund, VCSE umbrella bodies, and word-of-mouth in their organisations. This underlines the key role and added value that well-established organisations, with pre-existing links to the sector, play in being able to advertise a large-scale programme to a range of organisations, in the short timescales that a programme responding to cost of living challenges requires
A range of organisations applied for funding, from different geographies, and of different organisational types and sizes. Although some programme stakeholders had concerns about whether there were enough applications from organisations supporting minority groups, as we highlight below, there was evidence of reach to a range of minority groups amongst CCLF grant holders (although exact numbers of people supported from different demographics is not known). Analysis also identified that organisations offering multiple services were more likely to be awarded funding than those with single, specialist target services. The number of services provided may be a factor that future funds consider in monitoring the spread of those organisations awarded funding
Applicants had high levels of satisfaction with different aspects of the application process, although were more likely to be dissatisfied with the ease of completing the application form than other aspects. In terms of the decision-making processes, The National Lottery Community Fund followed its tried and tested multi-step process to assess and decide on the applications. The process was generally efficient but faced some delays due to the large number of applications that were submitted just before the deadline. VCSEs not awarded funding were more likely to face longer delays in hearing their decision, and were more likely to be dissatisfied with the timescales for waiting, compared with those awarded funding
In terms of projects’ reach, the evidence indicates that almost all CCLF projects have largely reached the main intended beneficiary group (low income individuals and households), with evidence of reach to communities experiencing ethnic or racial discrimination and inequality, older people, women and girls, children and young people, disabled people, and other vulnerable groups

Outcomes for VCSE organisations

In line with the key aim of the CCLF, the evidence indicates that CCLF funding contributed to organisations’ ability to maintain their service delivery in the short-term, especially over the winter period of 2023-2024. Most adaptations were relatively small and focused on expanding an existing service, improving quality, or increasing the offer or depth of support
In the more medium to longer term, CCLF funded services as well as those across the sector have continued to increase the services they offer, likely in response to continued increases in demand. Findings from the Wave 2 surveys with grant holders and unsuccessful applicants suggest that the sector as a whole faces increasing demand, the need for more services, and a challenging financial environment
Organisations were better able to weather the cost-of-living storm in the short-term and meet high costs relating to service delivery. Organisations reported a better financial situation as a result of receiving the funding, and being able to sustain or adapt their services. It was also reported that without the funding many would have had to reduce the services they offered
Organisations were less likely to use their reserves to meet the costs of delivery than they would have been had they not received the CCLF funding. Although some organisations still reported needing to use their reserves, others reported that their reserves were in a healthier position as a result of receiving the funding
The impact evaluation found that funding helped organisations to increase their total gross income, allowing them to access resources that might otherwise have been out of reach. There was wider evidence to suggest that the CCLF funding contributed to this by providing a buffer or freeing up capacity over the winter period 2023-2024 to concentrate on applying for funding
The increase in income experienced by grantees has, in turn, facilitated greater expenditure, potentially on expanding services, meeting increased demand, or investing in their future sustainability. Furthermore, the funding has positively impacted net income, a key indicator of financial health and long-term sustainability. While the impact on larger organisations is less clear, the positive effects on smaller organisations, which often have fewer resources and less financial security, are particularly noteworthy.
Measuring the effects of the CCLF on shrinkage, inactivity or insolvency proved challenging in the timeframes of this evaluation. However, it seems reasonable to assume that having greater income places CCLF funded organisations at a reduced risk

Outcomes for staff and volunteers

Staff and volunteers reported being better able to meet the short-term pressures of demand over the winter period, as they benefited from increased capacity and support across teams, changes to delivery which streamlined their work, or accessing training
As a result of an improved financial situation and reduced pressures on staff, organisations were able to avoid staffing reductions. The process tracing evaluation indicated that CCLF played a contributing role in this. As a result, organisations are likely to retain the networks, relationships, knowledge and skills of staff and volunteers. However, wider challenges in the sector in staff retention were reported

Outcomes for service users

The evaluation has also highlighted the positive outcomes for beneficiaries, with evidence of short-term basic needs being met, access to information and support being facilitated, and access to safe and warm spaces being enabled. Many organisations in our case studies were strongly linked with their local community and other local organisations, and therefore regularly signposted people to the relevant support. It should be noted that these organisations are often longstanding and it is not possible to attribute all positive outcomes to the CCLF, although funding did support service delivery and reaching new beneficiaries
In the medium to long term, outcomes experienced by beneficiaries were more varied, as this depended on the type of service accessed. Some organisations focus more on supporting longer term outcomes, whereas others focus on providing immediate support. However, across the diverse range of services funded through CCLF the positive outcomes experienced by beneficiaries were noted

Remaining needs

Overall, while VCSE representatives were very appreciative of the funding, there remains a strong demand for sustained funding that can cover core costs as organisations continue to struggle with the ongoing impacts of increased cost of living. In addition, while the research underlined that there was a need for funding in the winter period, it also highlighted that increased demand arose at different time points, such as during school holidays and at the start of the financial year following changes to taxes and benefits, which can have significant consequences on people’s financial situation. This highlights that there could be other time periods, alongside the winter period, in which to offer similar funding opportunities

Value for money assessment

There is evidence to suggest the effectiveness of the Fund, especially in relation to the CIE findings around increased income for organisations. Outcomes such as financial resilience have been maximised, whilst minimising inputs – ‘spending wisely’
There is also evidence of equity, as services are available to and reach all people that they are intended to – ‘spending fairly’
The CCLF also showed evidence of economy in that the cost of inputs or resources used or required was minimised – ‘spending less’
Finally, the CCLF demonstrated efficiency as outputs such as staff costs from services were maximised while inputs were minimised – ‘spending well’. The Fund was delivered within the £75.97million budget, and efficiencies in grant administration to achieve an overall saving of £427,748

5.2. Recommendations

Based on the findings of this evaluation, some key considerations for the design and delivery of future, similar response programmes are presented below:

Support for smaller organisations: The impact evaluation indicated that the funding was particularly impactful for small and medium-sized organisations in terms of improving their net income. Future schemes, that offer similar grant sizes, and aim to support financial resilience, should consider ensuring that VCSE size is monitored when awarding grants, and that there is good representation of micro, small and medium-sized VCSEs in the pool of grant recipients. TNCLF and DCMS did this effectively in the CCLF, also considering other factors, such as service type and target beneficiaries
Ensuring funding for core costs: Almost all grant holder survey respondents welcomed the flexibility of funding to cover core costs, and case study interviewees often highlighted the limited availability of grants for these cost types. Future response funds should consider the option of providing funding to core costs / unrestricted funding (if such options aren’t provided elsewhere already), to maximise organisations’ flexibility and to help keep services going
Longer-term funding pots: case study research and qualitative feedback from the surveys suggest that organisations would welcome longer-term funding opportunities over shorter-term funding pots, especially in contexts such as rising inflation / increased cost-of-living, where impacts are likely to be felt for a long time. Our analysis showed that, while CCLF had a positive impact on income and net income for grant holders, some concerns remained with regard to other aspects of financial resilience such as ability to cover ongoing costs, level of reserves and financial runway (i.e. how long VCSEs have before they run out of money). Furthermore, survey analysis shows that after the CCLF funding period, VCSE organisations continued to experience increased demand for services and increased costs for delivery. Longer-term funding may help ensure that VCSEs can maintain delivery for a longer period of time, potentially buying more time to source other funding (as we saw happen in some cases in CCLF)
Provision of retrospective funding: many grant holders and programme stakeholders welcomed the choice of offering funding for retrospective costs. It allowed for a longer period of time to be funded, whilst enabling time for The National Lottery Community Fund to review a high number of applications and for each application to receive sufficient due diligence. While technically difficult to design and implement as the mechanism was new to DCMS and The National Lottery Community Fund, the CCLF now provides a blueprint for how this could be done in future

Question 7

6.0 Annexes

Accepted Answer

Annex 1: Evaluation framework

Evaluation question - Process evaluation	Sub-question	Data source(s)
How was the application and decision-making process implemented?	- How effective was the application process and why/why not? - How effective was the decision-making process and why/why not? - What was the learning from the application and decision-making process?	- Interviews with DCMS - Interviews with The National Lottery Community Fund delivery leads - Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies
What were the characteristics of grant recipients?	- What types of services did grant recipients provide? - What were the sub-sectors of grant recipients? - What size organisations received grants? - What types of organisations (e.g. legal structure/form) received grants?	- VCSE case studies - Programme MI data - Survey - Administrative data (where available)
What services, activities and outputs did the funding enable?	- What has the funding been used for? Was there anything grantees felt was needed that weren’t able to be funded by the grant? - How did these differ from those services, activities and outputs that the grant holder was already delivering? - How did VCSEs adapt their services?	- Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Programme MI data - Survey
How was the funding experienced by staff and volunteers?	To what extent and in what ways did the funding influence perceptions of short-term pressures on staff and volunteers?	- VCSE case studies - Survey
Did the funding reach the intended beneficiaries (low-income individuals and households)?	- How many beneficiaries did the funding reach? - Which geographic locations were beneficiaries from?	- Programme MI data - Survey
What was beneficiaries’ experience of the support they received?	- What worked well with the support beneficiaries received? - What worked less well? What could be improved?	- VCSE case studies (including beneficiary interviews) - Survey
What were the characteristics of beneficiaries and/or communities who were supported through this fund?	Which specific groups/communities were beneficiaries from?	- Programme MI data - Survey
What have we learned about what works in this sector, both in terms of grant-making and service delivery in relation to emergency funding to address cost of living pressures?		- Interviews with DCMS - Interviews with The National Lottery Community Fund delivery leads - Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Survey
What are the transferable lessons for other funds?		- Interviews with DCMS - Interviews with The National Lottery Community Fund delivery leads - Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Wider literature

Evaluation question - Impact evaluation	Sub-question	Data source(s)
What impact did the Fund have on grantholders in supporting continued service delivery?	To what extent and how did the Fund affect organisations’: - ability to maintain service delivery? - use of spending reserves for delivery costs - use of spending funds that were for other activities - capacity to focus on fund-raising activity - levels of staffing	- Interviews with The National Lottery Community Fund delivery leads - Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Survey - Administrative data (publicly available datasets)
What impact did the Fund have on the overall volume of services delivered?	To what extent did the Fund help VCSE organisations to avoid service reduction?	- Interviews with The National Lottery Community Fund delivery leads - Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Survey
What evidence is there of greater reach and depth of target services (relative to what would have happened in the absence of the Fund)?	- How many people were supported through CCLF-funded services? To what extent did the Fund enable VCSEs to support more beneficiaries? - To what extent and how did the Fund enable VCSEs to provide more services to their existing beneficiaries? - To what extent did the Fund avoid service reduction in specific sub-sectors?	- Interviews with The National Lottery Community Fund delivery leads - Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Survey
What was the impact of the Fund on the retention, recruitment and resources of staff and volunteers?	- To what extent and in what ways did the funding help address the short-term pressures facing staff and volunteers? - To what extent did the funding help to free up VCSEs’ resources?	- Interviews with The National Lottery Community Fund delivery leads - Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Survey
To what extent did the Fund contribute to intended outcomes among beneficiaries?	- To what extent and in what ways did the Fund meet immediate critical support needs - for example, for food and emergency supplies, emergency shelter, safe spaces, warmth, and financial/housing advice? - To what extent and in what ways did the Fund enable people to be signposted to relevant support and services as appropriate?	- Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Programme MI data - Survey
To what extent did the Fund contribute to potential medium-term outcomes among beneficiaries?	To what extent and in what ways did the Fund contribute to (as relevant): - Improved mental and physical health - Reduced impacts of poverty - Any educational or employment outcomes?	- Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Survey
To what extent did the Fund impact beneficiaries’ use of other services, particularly statutory services?	To what extent and in what ways did people access relevant support services following support from funded activities?	- Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Survey
What were the impacts on the sector as a whole, including around resilience and financial sustainability?	To what extent and in what ways have VCSE sector organisations funded through the CCLF maintained their capacity to deliver frontline critical services?	- Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Survey
What were the impacts on the sector as a whole, including around resilience and financial sustainability?	To what extent and in what ways has the Fund contributed to the sector retaining staff and volunteers’ knowledge and skills?	- Interviews with DCMS - Interviews with The National Lottery Community Fund delivery leads - Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Programme MI data - Survey
What would have happened in the absence of the funding? [links to economic evaluation – What was the additionality of the Fund?]		- Interviews with DCMS - Interviews with The National Lottery Community Fund delivery leads - Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Programme MI data - Survey - Administrative data (publicly available datasets)
What, if anything, were the unintended or unanticipated impacts and why did they occur?		- Interviews with DCMS - Interviews with The National Lottery Community Fund delivery leads - Interviews/focus groups with The National Lottery Community Fund funding managers - VCSE case studies (including beneficiary interviews) - Survey

Evaluation question – Economic evaluation	Sub-question	Data source(s)
What were the costs and benefits of the Fund, and to whom?	- What were the costs of the Fund? - Who incurred these costs?	- Interviews with DCMS - Interviews with The National Lottery Community Fund delivery leads - Interviews/focus groups with The National Lottery Community Fund funding managers - Programme MI data - Survey
What were the costs and benefits of the Fund, and to whom?	What were the benefits of the Fund? - What is the value of monetisable benefits? - What other benefits were there?	- Outcomes data [see ‘impact evaluation’] - Programme MI - Survey - Wider literature
What were the costs and benefits of the Fund, and to whom?	Who experienced these benefits?	- Outcomes data [see ‘impact evaluation’] - Programme MI - Survey
Has the Fund been cost effective?	To what extent were the outcomes achieved at the lowest possible cost?	- Interviews with DCMS - Interviews with The National Lottery Community Fund delivery leads

Annex 2: Theory of Change

Annex 3: Case study sampling

Wave 1

The 25 case study organisations were sampled from the full list of organisations which received funding. Using the following criteria, organisations were sampled to where possible closely mirror the portion of organisations in each category across the overall grantee population:

Geographical location: Organisations were chosen to reflect the number of grantees per region; for example 40% of grantees were in London, South East and East of England, so the target number of case studies within this area was 10.
Service category/focus: During sampling organisations delivering a broad range of services were included, across the funded service types: advice, warmth, food and supplies, safe space, shelter, and financial and housing advice as potential services. Many organisations provided more than one of these service types.
Organisation size: Organisations were classified by The National Lottery Community Fund using NCVO classifications which divides organisations into micro, small, medium, large, major and super major based on income ^{[footnote 28]}. Due to only two super-major organisations being given funding, we chose not to sample them as it would have placed them at risk of being identifiable.
Funding amount: Organisations received a range of funding amounts, and sampled to ensure a spread of funding amounts. Organisations were divided into Awarded Amount groups of £10,000-£24,999, £25,000-£39,999, £40,000-£54,999 and £55,000 -£75,000 and again the percentage of case studies spoken to reflected the percentage of grantees who received funding in each amount group.

Wave 2

Sampling criteria for organisations being revisited

For the 6 organisations to be revisited from wave 1, the following were used as the primary sampling criteria:

Experiences of the funding, outcomes and impact: given we had more information about these organisations, we purposively sampled to identify a range of organisations which reflected a variety of experiences of the funding, and any perceived longer-term outcomes or impacts.
Engagement with the first wave of case studies: we identified organisations based on the first wave who we believed would have sufficient capacity to participate in and support the data collection activities. We also prioritised organisations that were likely to provide valuable and actionable insights, for example where we were previously able to recruit an adequate number of participants across different stakeholder groups.

Secondary sampling criteria used included ensuring that the organisations visited where possible were diverse across the following characteristics:

Region
Organisation Size
Funding Amount
Service type

However, given that the population was limited to the 24 visited in wave 1, and potentially further limited if organisations are not able to facilitate a second wave of fieldwork, there were limitations to how varied the sample could be.

Sampling criteria for new organisations

For the 6 new organisations visited for wave 2, the following were used as the primary sampling criteria:

Services delivered: we reviewed those contacted in the first wave and where possible sought to ensure that the second wave filled in any gaps in relation to services delivered – this included activities or services offered, as well as demographic groups targeted by the services.
Retrospective funding: wave 1 of the case studies highlighted the value of the retrospective funding offered to organisations. In the second wave of case studies, we sought to ensure that at least 3 of the 6 new organisations visited received retrospective funding.

Similar to those being revisited, secondary sampling criteria used will include ensuring that the organisations we visit, where possible, are diverse across a range of characteristics outlined in wave 1.

However, again due to only contacting 6 organisations we were not able to ensure full coverage across all groups but instead sought to ensure a diverse sample across these categories where possible.

As noted, there was also an additional organisation contacted with whom fieldwork was not possible in wave 1.

Exclusion criteria

For new organisations in wave 2, the following exclusion criteria were applied:

Those on a list provided by The National Lottery Community Fund of organisations who may have ongoing issues/investigations relating to the funding and therefore whom it would be inappropriate to contact.
Those invited to participate in wave 1 but who declined due to capacity.

Summary of full sample

Table 6.1: Breakdown of sample criteria and targets for case study organisations across Waves One and Two

Criteria	Number of grantees	% of grantees	Case studies in wave 1	New case studies in wave 2	Total case study organisations	% in the sample
Region	-	-	-	-	-	-
East and West Midlands	293	19%	6	1	7	22.6%
London, SE and East of England	610	40%	9	3	12	38.7%
North East & Cumbria	119	8%	2	1	3	9.7%
North West	230	15%	4	0	4	12.9%
South West	124	8%	2	0	2	6.5%
Yorkshire and Humber	134	9%	2	1	3	9.7%
Organisation size	-	-	-	-	-	-
Micro	41	3%	1	0	1	3.2%
Small	352	23%	6	1	7	22.6%
Medium	888	59%	15	2	17	54.8%
Large	209	14%	3	2	5	16.1%
Major	18	1%	0	1	1	3.2%
Super-Major	2	0%	0	0	0	0%
Grant amount	-	-	-	-	-	-
£10,000 - £24,999	305	20%	5	0	5	16.1%
£25,000 - £39,999	316	21%	5	3	8	25.8%
£40,000 - £54,999	247	16%	4	1	5	16.1%
£55,000 - £75,000	638	42%	11	2	13	41.9%

Source: The National Lottery Community Fund CCLF application data (N= 1,506), CCLF evaluation team

Annex 4: Process tracing

The evaluation required an assessment of the impact of the CCLF programme. Due to the lack of secondary data on some of the key outcomes (i.e., service delivery, staff outcomes, some financial outcomes) as well as uncertainties in retrieving data on financial measures for treatment and control groups using primary data collection (i.e., the survey) a full QED methodology assessing the full set of alternative impacts from CCLF was considered unfeasible. A process tracing approach provides an alternative that is a systematic and transparent approach to assessing the impact of the funding. The use of process tracing is increasingly accepted as a robust alternative to experimental approaches in evaluation where the use of comparator or control groups is not feasible. ^{[footnote 29]}

This approach enabled us to arrive at an overall assessment (in terms of a percentage probability) of the impact of CCLF across the four areas detailed in the hypotheses we set out (see Table 6.2 below). In terms of the overall analysis of impact, this approach allowed us to arrive at a conclusion on the proportion of VCSE organisations in different probability ranges relating to how likely it is that those hypotheses would be true in their case. The recommended steps by Befani (2020)^{[footnote 30]}, a leading academic in the methodology, were followed, and are set out below.

Box 10. Summary of steps in applying process tracing

The evidence pieces to test the claims around VCSEs were chosen based on the literature, and discussions within the evaluation team including input and review from experts. In total, 15 pieces of evidence were selected across the four hypotheses, derived from questions in the survey of VCSE organisations.
Probability values were then assigned to each piece of evidence – relating to the relevant hypothesis - (using scales), indicating:
- Sensitivity: The likelihood the evidence would be observed if the contribution claim (CC) was true
- Type 1 error: The likelihood the evidence would be observed if CC is false
A prior probability had to be chosen, representing our existing belief about whether the claim was true. As is recommended in the literature, this was chosen to be 50% indicating there was no prior information to suggest whether the claim was likely or unlikely to be true.
Using the Sensitivity and Type 1 error values, Bayesian formula was applied for each piece of evidence and VCSE organisation, giving a single posterior value that represents an updated belief, or probability, that CC is true for each hypothesis.

Develop the hypothesis: A hypothesis, and its complement (the null hypothesis), had to be chosen that were mutually exclusive and allowed causality in the theory of change to be concluded. The theory of change set out short-term outcomes - largely related to during and immediately after the funding period – and medium to longer term outcomes helping to determine expected precedence in causality

Based on the research questions, the ToC and consultations with the expert on the project, we produced the following hypotheses:

Table 6.2: Process Tracing Hypotheses 1 - 4

Hypotheses	Null Hypotheses
- H1: During the period from March 2023 to March 2024, the CCLF contributed to VCSE organisations maintaining and / or increasing number and range of services provided.	- Null Hypothesis (H0): During the period from March 2023 to March 2024, solely by means other than CCLF VCSE organisations maintained and / or increased number and range of services provided.
- H2: During the period from March 2023 to March 2024, the CCLF contributed to VCSE organisations maintaining and / or increasing number of service users supported.	- Null Hypothesis (H0): During the period from March 2023 to March 2024, solely by means other than CCLF VCSE organisations maintained and / or increased number of service users supported.
- H3: During the period from March 2023 to March 2024, the CCLF contributed to VCSE organisations’ financial resilience by helping organisations to avoid shrinkage and insolvency.**	- Null Hypothesis (H0): During the period from March 2023 to March 2024, means other than CCLF solely contributed to VCSE organisations’ financial resilience by helping organisations to avoid shrinkage and insolvency.
- H4: During the period from March 2023 to March 2024, the CCLF contributed to VCSE organisations retaining staff and volunteers.	- Null Hypothesis (H0): During the period from March 2023 to March 2024, means other than CCLF solely contributed to VCSE organisations retaining staff and volunteers.

These hypotheses are mutually exclusive and testable, i.e., it should be possible to find evidence that proves or disproves the hypothesis claim.

Designing the data collection: We considered the probative value of responses to specific questions and prioritised responses with a higher probative value (i.e., evidence that is likely to be present if H is true, or evidence that is very likely to be seen if the H is false). It is suggested in the literature that evidence can be described by the different process tests and a variety should be considered; with a particular focus on hoop evidence and smoking gun (see Table 9). With this in mind, the main data collection for process tracing in CCLF is the survey of grant holders, i.e., successful applicants. Key questions on outcomes were informed by the specified hypotheses, to tailor data collection to the outcomes of interest.
Assigning the probabilities: Based on evidence from the literature, discussions within the team and input from our experts, ‘sensitivity’ and ‘type 1 error’ values were determined for specific pieces of evidence, as follows:
- Sensitivity: The probability of finding evidence if H is true. The ‘sensitivity’ value can be quantified, as a subjective probability between 0 and 1.
- Type 1 error: The probability of finding evidence if H is false.

The probability values associated with finding particular evidence (regarding whether H is true or false) are based on the following scales:

Very likely – 90%
Likely – 70%
Uncertain – 50%
Unlikely – 30%
Very unlikely – 10%

The value of the process tracing approach rests on the validity of the subjective probability estimates for the sensitivity and type 1 errors assigned to each piece of evidence. We used a combination of survey responses (and derived metrics) providing mainly ‘hoop’ evidence as it was difficult to identify ‘smoking gun’ and ‘doubly decisive’ evidence based on any one piece of evidence that we had. The probability estimates were agreed after extensive and constructive conversations amongst the members of the evaluation team and after a thorough review of case study evidence. This allowed the evaluation team to reach consensus on the probative value of evidence. While Bayesian inference itself is mathematical in nature, the application of Bayesian reasoning in process tracing relies on qualitative judgments that are confirmed through a collaborative process of validation and layered with other evidence to present a contextual interpretation of evidence. Bayesian process tracing should be understood as a systematic approach to qualitative inference rather than a purely quantitative method.

The list of probative values assigned can be found in Table 6.4.

Table 6.3. Process tracing test

Test	Interpretation	Sensitivity	Type 1 error
Expect to see/ hoop evidence	If the hypothesis is true we would see it; the absence of it does not mean the hypothesis is not true.	Likely	Likely
Smoking gun	If observed it means the hypothesis is almost certainly true, however in its absence the hypothesis could still be true.	Very Likely/ Likely	Uncertain/ Likely
Doubly Decisive	If observed it means the hypothesis is almost certainly true, the absence of observing it suggests the hypothesis is almost certainly not true.	Very Likely	Very Unlikely/ Unlikely
Straw in the Wind	If observed it does not increase the likelihood of the hypothesis being true significantly, the absence of observing does not increase the likelihood of the hypothesis not being true significantly.	Uncertain/ Unlikely/ Likely	Uncertain/ Unlikely/ Likely

Conduct Data Collection and Update Confidence About the Claim: After collecting the results from the survey, the sensitivity and type 1 error values were inputted for each evidence piece from each organisation. So, for example below in Table 10 we have a VCSE organisation that had an increase in gross profit but an increase in costs so using the thresholds in table 9, a sensitivity value of 0.7 (70%) and a type 1 error of 0.1 (10%) would be allocated.

Table 6.4: Example evidence of assigning Sensitivity and Type 1 error probabilities.

Evidence Number	Variable	Question number(s)	Variable description	Rationale	Thresholds	Sensitivity	Type-1
E2	Change in gross profit margin	Context: Q23 - Rising Costs Q16, Q17, Q18, Q19	Change in gross profit in 2023 and 2024	Indication of financial resilience, by comparing the gross profit margin before and after the CCLF, taking into account raising costs	- Increase in gross profit & No rising costs - Increase in gross profit & Rising costs - Decrease in gross profit & No rising costs	75% 70% 5%	20% 10% 50%

After assigning these values, the Bayes’ formula (see below) is applied to the evidence, in order to calculate the posterior probability of H being true.

Choosing the prior probability: A prior had to be chosen for the likelihood that a hypothesis is true without any of our new observed evidence. As this was an unprecedented situation and no evidence was available on what an appropriate starting point should be, a 50% prior was chosen, indicating it was uncertain.

Continuing with the example above, if we find evidence E2 and apply Bayes’ formula this results in a posterior probability of 0.58 (58%)– in other words, this has increased our confidence that H is true.

1 PHE=PEH×PHPEH×PH+PE~H×P~H

= 70%×50%70%×50%+10%×50%

=0.350.35+0.05

=87.5%

However, as we need to apply multiple pieces of evidence, not just one, we used the formulas 2 and 3 to get a posterior probability based on all the pieces of evidence. The % value would go up or down and robustness increases with the number of pieces of evidence.

2 PEH= PEnHE1-n PEn-1HE1-n-1 PEn-2HE1-n-2 PEn-3HE1-n-3…

3 PE~H= PEn~HE1-n PEn-1~HE1-n-1 PEn-2~HE1-n-2 PEn-3~HE1-n-3…

So, to continue the example above, if there was the following sensitivities and type 1 errors for additional evidence:

Table 6.5: Example allocating sensitivities and Type 1 errors

Evidence	Sensitivities	Type 1 error
E1	0.8	0.5
E2	0.7	0.1
E3	0.5	0.3

P(EH=0.8×0.7×0.5=0.28 (28%)

PE~H=0.5×0.1×0.3= 0.015 (1.5%)

PHE=PEH×PHPEH×PH+PE~H×P~H

= 28%×50%28%×50%+1.5%×50%

=0.140.14+0.0075

=94.9%

After including additional evidence, it has increased the likelihood of H being true; however the calculation needs to be rerun with all pieces of evidence to arrive at a final probability value. This approach was applied to every organisation that answered the survey. This then allowed us to make the statements based on groupings of organisation made in the main report e.g. x% of VCSE organisations had more than an 80% likelihood that CCLF contributed to maintaining and / or supporting services.

Evidence	Unit	Variable	Sensitivity (P(E/H)	Type 1 error (P(E/~H)
E2	% Change in gross profit	Increase in gross profit Rising costs	0.7	0.1

Any organisation with no evidence – and therefore a 50:50 chance of a true hypothesis – was excluded from our analysis.

Box 11: Process Tracing Worked Example - Hypothesis 1

Limitations

The process tracing method does have several limitations, including assumptions of independence that may not hold, complexities in aggregating data across diverse organisations, and challenges in handling confounding factors. As a result, while the method provides valuable insights, its estimates of CCLF’s impact should be interpreted cautiously and supplemented with other methodologies. These limitations include:

Assumption of independence – The method assumes that the individual pieces of evidence are independent. However, evidence from VCSE organisations – especially when comparing very large charities with much smaller ones – may be correlated, which could distort the aggregated likelihood estimates.
Complexity in Aggregation Across Diverse Organisations - Variability in organisational characteristics can complicate the aggregation process, potentially underestimating the role of other factors in measured impact. An organisation’s sector or geographical location may affect impact in ways that the survey does not capture, and process tracing does not measure.
Limited Handling of Confounding Factors – While process tracing is adept at tracing causal mechanisms, it may not fully account for alternative explanations or confounding variables, especially in heterogeneous contexts

These limitations suggest that while the aggregated evidence provides useful insights, the estimated contribution of CCLF should be interpreted with insight from other methodologies to complement. The influence of CCLF may therefore be more ambiguous than is presented here.

Evidence Pieces

The table below identifies the pieces of evidence (E1, E2 etc) which will be based on analysis of the ‘impact’ questions from the survey and the hypotheses to which they relate.

Hypothesis	Evidence Piece	Questions	Metric	Rationale
H3: financial resilience through avoiding shrinkage and insolvency	E1	Q16, Q17, Q18, Q19, Q23	Change in cost to income ratio in 2023 and 2024 (operating expenses / operating income)	Indication of financial resilience. This measure would allow us to compare how the cost to income ratio changed before and after the CCLF, taking into account the wider context of raising costs.
H3: financial resilience through avoiding shrinkage and insolvency	E2	Q16, Q17, Q18, Q19, Q23	Change in gross profit margin [(income - expenses)/income] in 2023 and 2024	Indication of financial resilience, by comparing the gross profit margin before and after the CCLF, taking into account raising costs.
H3: financial resilience through avoiding shrinkage and insolvency	E3	Q20, Q21, Q23, Q28, Q29	Change in levels of reserves in 2023 and 2024	Indication of financial resilience. This measure would allow us to compare whether the CCLF had any effects on levels of reserves before and after, taking into account any cost cutting decisions (particularly around reserves).
H3: financial resilience through avoiding shrinkage and insolvency	E4	Q16, Q17, Q18, Q19, Q20, Q21, Q23	Change in runway financial measure expressing a change in years to insolvency (2023 in comparison to 2024)	Indication of financial resilience. This measure is the change in years to insolvency before and after the CCLF. It will provide context to what was the risk of insolvency before the CCLF and what is the risk afterwards. Also here we include the questions on rising costs right before they received the CCLF to assess the context in which these changes are happening.
H3: financial resilience through avoiding shrinkage and insolvency	E5	Q16, Q17, Q18, Q19, Q23	Change in yearly burn rate	Indication of financial resilience. The change in yearly burn rate indicates the difference in yearly burn rate before and after the CCLF was implemented.
H3: financial resilience through avoiding shrinkage and insolvency	E6	Q23, Q28, Q29	Cost-cutting decisions before and after the CCLF, in the context of rising costs	Financial resilience. We can compare cost-cutting decisions before and after receiving CCLF, in the context of rising costs.
H4: Retaining staff and / or volunteers	E7	Q23, Q30	Proportion of hours cut	Retaining staff, proportion of hours reduced in the context of rising costs.
H4: Retaining staff and / or volunteers	E8	Q29, Q31, Q32, Q35	Percentage of redundant staff between receiving the funding and March 2024	Retaining staff. We can calculate the percentage of staff that were made redundant between receiving the CCLF and March 2024, using the total staff from March 2023.
H4: Retaining staff and / or volunteers	E9	Q29, Q35	Percentage of staff who took a pay cut / forgo a pay rise between receiving the funding and March 2024	Retaining staff. We can calculate the percentage of staff that took a pay cut / forgo a payrise between receiving the CCLF and March 2024, using the total staff from March 2023.
H4: Retaining staff and / or volunteers	E10	Q35	Number of staff before and after the CCLF	Retaining staff. We can compare the number of staff before and after the CCLF (staff in March 2023 - staff in March 2024), in the context of questions around cost management, what would have happened without CCLF and rising costs.
H4: Retaining staff and / or volunteers	E11	Q25, Q26	Number of volunteers before and after CCLF	Retaining volunteers. We can compare the number of volunteers before and after CCLF, in the context of rising costs
H2: Maintaining and / or increasing number of service users	E12	Q25, Q26	Increase / decrease of beneficiaries	Maintaining / increasing number of service users, by understanding change before and after CCLF
H1: Maintaining and / or increasing number and range of services	E13	Q25, Q26	Increase / decrease of number of services	Maintaining / increasing number of services provided, by understanding changes before and after CCLF
H1: Maintaining and / or increasing number and range of services	E14	Q25, Q26	Increase / decrease in range of services	Maintaining / increasing range of services provided, by understanding changes before and after CCLF
H1: Maintaining and / or increasing number and range of services	E15	Q24, Q25, Q26	Increase / decrease of ability to meet demand	Maintaining / increasing ability to meet demand before and after CCLF

Annex 5: Counterfactual Impact Evaluation

Methodology

The CIE aimed to estimate the impact of CCLF funding on recipient organisations financial resilience and health relative to a statistically matched comparison group of organisations that applied but were unsuccessful.

The evaluation utilised data from two secondary data sources:

CCLF Application Data: This provided information on organisations that applied for CCLF grants, including details on funding amounts, beneficiary numbers, and staffing levels.
Charity Commission Annual Returns: This data source provided financial information (income and expenditure) for registered charities, enabling a comparison of financial health before and after the introduction of CCLF.

Data cleaning and matching involved a complex process of linking organisations across these data sources using charity registration numbers. This involved identifying the most recent annual return for each organisation with an end date between December 2023 and March 2024, and then linking this with the corresponding annual return for the previous year to establish a baseline. This ensured that the analysis captured any impact of CCLF on income and expenditure and allowed for a stronger attribution claim. The process also ensured that financial data covered a full 12 months to avoid bias. The final analysis dataset included 1,854 organisations, representing 63% of all applicants with a registered charity number.

The CIE used propensity scores with inverse probability weighting to create a statistically matched comparator group. Propensity scores are a composite measure of the likelihood of an organisation being funded based on multiple key characteristics prior to CCLF. These propensity scores are then used as weights in the analysis. This technique helps ensure that the organisations that received CCLF (i.e. the treatment group) are being compared to an as similar group of organisations that did not receive CCLF (i.e. the comparator group). The key characteristics (all prior to CCLF) used for the propensity score weightings included:

Income
Expenditure
Number of beneficiaries supported
Staffing levels.

Combined, these characteristics provided a nuanced measure of each organisation’s size and operations. Other characteristics, for example, region, were considered and tested but did not appear to strengthen the analysis. This approach ensures that any observed differences in outcomes between the funded and non-funded groups can be more confidently attributed to the impact of CCLF.

Strengths and limitations

This CIE has several strengths that enhance the reliability and validity of its findings:

High-quality data sources: The use of Charity Commission Annual Returns ensures that the financial data used in the analysis is accurate and reliable.
Focus on financial resilience: The outcome measures used in the evaluation directly address the key aim of the CCLF, which is to support the financial resilience of VCSE organisations.
Robust matching process: Propensity scores with inverse probability weighting helps to create comparable groups, reducing the risk of bias and increasing the confidence in attributing observed differences to the CCLF intervention.

Limitations that are important to recognise include:

Limited CIC representation: The analysis primarily focused on registered charities due to limitations in the available financial data for Community Interest Companies (CICs) from Companies House. This data was accessed but only covered balance sheets, which did not cover income or expenditure. However, this data would have only covered around 50 additional CCLF funded organisations after data cleaning and matching.
Unable to explore insolvency: The evaluation could not quantitatively explore the impact of CCLF on organisation insolvency or dissolution due to the short timeframe since the fund’s launch. Only two organisations (out of the 1854 in the sample) were identified as insolvent, dissolved, or in administration in the Charity Commission data.
Exclusion of survey data: Primary data from surveys was not included in the CIE due to smaller sample size, focus on validated financial data, and the need for qualitative analysis of more nuanced outcomes such as the impact on service delivery. The latter is covered through more appropriate approaches in section 3.0.

Descriptive statistics

This section briefly discusses the key characteristics of funded and not funded organisation groups, prior to any matching. The purpose is to understand any key differences between the groups in the CIE sample.

In terms of finances, funded and not funded organisations were, on average, similar:

Funded organisations had a slightly higher average income (£1,643,428) compared to not funded organisations (£1,630,322) prior to CCLF. However, the standard deviations were large, indicating substantial variation within each group. The trimmed means, which are less sensitive to outliers,^{[footnote 31]} were very similar for both groups (£660,765 for funded and £ 655,367 for not funded).
Funded organisations had a higher average expenditure (£1,802,918) compared to not funded organisations (£1,638,631). Again, there was substantial variation and trimmed means revealed smaller differences between the funded and not funded organisations (£648,534 and £637,805, respectively).
The median income for funded organisations was £428,172, compared to £356,998 for not funded organisations. The median expenditure was £417,695 for funded and £351,247 for not funded.

These figures suggest that while there were some differences in the distributions of income and expenditure, the two groups were broadly similar in terms of their financial characteristics prior to CCLF. Importantly, these figures highlight the influence of very large organisations.

Table 6.6 shows the distribution of (prior to CCLF) income across organisations. Overall, there were 38 organisations with income greater than £10m. Whilst this is only a small percentage of organisations, some had incomes more than £100m.

Table 6.6: Income groupings for organisations included in CIE

Income group	CCLF funded	CCLF funded	Not funded	Not funded
Income group	Count	Percent	Count	Percent
1. £10m+	14	1.7	24	2.3
2. £1m to £10m	172	20.7	184	18
3. £500k to £1m	177	21.3	190	18.6
4. £250k to £500k	183	22	220	21.5
5. Less than £250k	284	34.2	406	39.6
Total	830	100	1024	100

Differences on other characteristics, including number of beneficiaries supported, staff/volunteer numbers, and organisation type and region, were similar across groups (after adjustment for outliers). Summary tables are provided below.

Data tables

variable	mean_funded	mean_not funded	sd_funded	sd_not funded	mean_trim_funded	mean_trim_not funded
app_awarded_amount	49172	-	21234		49789	-
app_future_costs_payment_amount	32814	-	16411		32137	-
app_income_for_the_year	1490846	1626537	9914727	8599570	600560	613973
app_initial_requested_amount	49155	40062	21314	19671	49755	39733
app_number_of_beneficiaries_supporting	24861	13518	338848	77362	4219	3891
app_number_of_staff	32	30	249	160	13	12
app_number_of_volunteers	178	88	1583	403	53	41
app_retrospective_costs_payment_amount	16288	-	13592	-	15513	-
app_total_project_cost	133310	101025	298432	212033	90333	67511
cc_total_gross_expenditure_base	1802918	1638631	14661726	8626897	648534	637805
cc_total_gross_income_base	1643428	1630322	10919000	8358128	660765	655367

variable	median_funded	median_not funded	min_funded	min_not funded	max_funded	max_not funded
app_awarded_amount	50888	0	10071	0	75000	0
app_future_costs_payment_amount	30643	-	3537	-	75000	-
app_income_for_the_year	374706	324998	10	0	2.47E+08	1.66E+08
app_initial_requested_amount	51024	36890	10071	10001	86910	75000
app_number_of_beneficiaries_supporting	2200	1710	17	8	8925000	1611000
app_number_of_staff	8	7	0	0	6521	3721
app_number_of_volunteers	35	25	0	0	35000	8599
app_retrospective_costs_payment_amount	12940	-	0	-	54691	-
app_total_project_cost	69909	50000	10071	10001	3880619	3059493
cc_total_gross_expenditure_base	417695	351247	0	0	3.86E+08	1.76E+08
cc_total_gross_income_base	428172	356998	0	0	2.69E+08	1.67E+08

app_organisation_type	n_funded	n_not funded	percent_funded	percent_not funded
Charitable incorporated organisation (CIO)	163	218	19.6	21.3
Community Interest Company (CIC)	1	2	0.1	0.2
Faith Organisation (Church Mosque Synagogue etc.)	8	13	1	1.3
Not-for-profit company	282	389	34	38
Registered Charity and Not-for-profit Company	195	167	23.5	16.3
Registered charity (unincorporated)	181	235	21.8	22.9

app_region	n_funded	n_not funded	percent_funded	percent_not funded
East and West Midlands	157	168	18.9	16.4
London, SE and East of England	359	445	43.3	43.5
North East & Cumbria	57	89	6.9	8.7
North West	114	145	13.7	14.2
South West	66	77	8	7.5
Yorkshire and Humber	77	100	9.3	9.8

Annex 6: Supporting tables and figures

Figure 6.1: Geographic distribution of applicants

Source: The National Lottery Community Fund CCLF application data (n=3,676). Correlation = 0.03. Note: ‘Wide’ refers to ‘England-wide’ project – only one organisation intended to deliver provision England-wide)

Figure 6.2: Beneficiary type of applicants, by award status

Source: The National Lottery Community Fund CCLF application data (n=3,736)

Figure 6.3: Organisation size of applicants, by award status

Source: The National Lottery Community Fund CCLF application data (n=3,736). Correlation = 0.06.

Table 6.7: Rate of satisfaction of different elements of the application process.

Elements of application process	Very satisfied	Very satisfied	Somewhat satisfied	Somewhat satisfied	Neither satisfied nor dissatisfied	Neither satisfied nor dissatisfied	Somewhat dissatisfied	Somewhat dissatisfied	Very dissatisfied	Very dissatisfied	Don’t know	Don’t know	Prefer not to say / not applicable	Prefer not to say / not applicable
-	Awarded	Not awarded	Awarded	Not awarded	Awarded	Not awarded	Awarded	Not awarded	Awarded	Not awarded	Awarded	Not awarded	Awarded	Not awarded
Information about the aims and objectives of the CCLF	608 (71.3%)	124 (31%)	215 (25.2%)	181 (45.3%)	18 (2.1%)	65 (16.3%)	5 (0.6%)	22 (5.5%)	5 (0.6%)	4 (1%)	1 (0.1%)	4 (1%)	1 (0.1%)	0 (0%)
How clear the eligibility criteria for the fund were	630 (73.9%)	137 (34.3%)	191 (22.4%)	167 (41.8%)	19 (2.2%)	56 (14%)	6 (0.7%)	32 (8%)	5 (0.6%)	7 (1.8%)	1 (0.1%)	1 (0.3%)	1 (0.1%)	0 (0%)
The ease of completing the application form	394 (46.2%)	85 (21.3%)	341 (40%)	173 (43.3%)	58 (6.8%)	68 (17%)	51 (6%)	59 (14.8%)	6 (0.7%)	14 (3.5%)	2 (0.2%)	1 (0.3%)	1 (0.2%)	0 (0%)
The application form guidance	453 (53.1%)	114 (28.5%)	315 (36.9%)	171 (42.8%)	60 (7%)	70 (17.5%)	16 (1.9%)	32 (8%)	5 (0.6%)	9 (2.3%)	3 (0.4%)	2 (0.5%)	1 (0.1%)	2 (0.5%)
The level of information our organisation needed to provide for the application	448 (52.5%)	90 (22.5%)	300 (35.2%)	179 (44.8%)	61 (7.2%)	61 (15.3%)	32 (3.8%)	54 (13.5%)	8 (0.9%)	14 (3.5%)	2 (0.2%)	0 (0%)	2 (0.2%)	2 (0.5%)
Any support provided by The National Lottery Community Fund as we were making our application	449 (52.5%)	58 (14.5%)	156 (18.3%)	79 (19.8%)	93 (10.9%)	78 (19.5%)	12 (1.4%)	38 (9.5%)	1 (0.1%)	16 (4%)	17 (2%)	31 (7.8%)	126 (14.8%)	100 (25%)
Timescales from submitting the application form to hearing the decision	360 (42.2%)	60 (15%)	290 (34%)	126 (31.5%)	66 (7.7%)	66 (16.5%)	105 (12.3%)	95 (23.8%)	29 (3.4%)	47 (11.8%)	2 (0.2%)	0 (0%)	1 (0.1%)	6 (1.5%)

Source: Wave 1 grant holder (awarded) survey, Q2 (sample size = 853). Wave 1 unsuccessful applicant survey (not awarded) Q2 (n = 400).

Figure 6.4: Project costs and request amounts, by award status

Source: The National Lottery Community Fund CCLF application data (n=3,736)

Table 6.8: Grant holders rate of satisfaction with different aspects of the funding process.

Awarded	Very satisfied	Somewhat satisfied	Neither satisfied nor dissatisfied	Somewhat dissatisfied	Very dissatisfied	Don’t know	Prefer not to say / not applicable
The length of time from hearing the decision to receiving the funding	448 (52.5 %)	230 (27%)	44 (5.2 %)	100 (11.7 %)	28 (3.3 %)	2 (0.2 %)	1 (0.1 %)
The grant monitoring requirements	502 (58.9%)	224 (26.3%)	57 (6.7%)	55 (6.4%)	13 (1.5%)	1 (0.1%)	1 (0.1%)
Producing the evidence needed (e.g. bank statements, payroll information) for how we spent the funding for the retrospective claim form	321 (46.3%)	219 (31.6%)	62 (8.9%)	72 (10.4%)	18 (2.6%)	0 (0%)	1 (0.1%)
Information and support from our Funding Officer at The National Lottery Community Fund	562 (65.9%)	158 (18.5%)	60 (7%)	11 (1.3%)	2 (0.2%)	8 (0.9%)	52 (6.1%)
The amount of time we had to spend the funding	338 (39.6%)	233 (27.3%)	89 (10.4%)	138 (16.2%)	52 (6.1%)	2 (0.2%)	1 (0.1%)
Total	-	-	-	-	-	-	853 (100%)

Source: Wave 1 grant holder survey Q9 (n = 853)

Table 6.9: Cost rises between March 2023 and award of funding from the CCLF

Awarded	They didn’t rise	Less than 5%	Between 5 and 10%	Between 10 and 15%	Above 15%	Don’t know
Staff wages	118 (13.8%)	190 (22.3%)	332 (38.9%)	101 (11.8%)	68 (8.0%)	44 (5.2%)
Volunteer costs	280 (32.8%)	241 (28.3%)	174 (20.4%)	73 (8.6%)	36 (4.2%)	49 (5.7%)
Equipment and materials	47 (5.5%)	108 (12.7%)	299 (35.1%)	184 (21.6%)	166 (19.5%)	49 (5.7%)
Overhead costs	46 (5.4%)	63 (7.4%)	248 (29.1%)	203 (23.8%)	242 (28.4%)	51 (6.0%)
Other costs	101 (11.8%)	64 (7.5%)	181 (21.2%)	134 (15.7%)	121 (14.2%)	252 (29.5%)
Total	-	-	-	-	-	853 (100%)

Source: Wave 1 grant holder survey, Q23 (n = 853).

Table 6.10: Cost management decisions made had the organisation not received CCLF funding

Awarded	Cost management decisions made had the organisation not received CCLF funding	Cost management decisions made between receiving CCLF funding and March 2024
Reduce staff hours	455 (53.3%)	57 (6.7%)
Make staff redundancies	205 (24.0%)	28 (3.3%)
Ask staff to take a pay cut or forgo a pay rise	197 (23.1%)	28 (3.3%)
Reduce the number of services you provide	502 (58.9%)	32 (3.8%)
Reduce the range / level of services you provide	547 (64.1%)	46 (5.4%)
Unplanned use of reserves	446 (52.3%)	146 (17.1%)
Increase the price of services	146 (17.1%)	76 (8.9%)
Take on debt	54 (6.3%)	10 (1.2%)
Sell assets	42 (4.9%)	8 (0.9%)
Renegotiate grants/ commissioned contracts	109 (12.8%)	68 (8.0%)
Cancel existing grants/commissions contracts	33 (3.9%)	12 (1.4%)
Other [please specify]:	28 (3.3%)	17 (2.0%)
Don’t know	40 (4.7%)	54 (6.3%)
No cost management decisions would have been needed or were needed	68 (8.0%)	469 (55.0%)
Total		853 (100%)

Source: Wave 1 grant holder survey Q29a&b (n = 853).

Table 6.11: Impact on services, March 2024 in comparison to March 2023

Awarded	No impact	A little impact	Quite a bit of impact	Substantial impact
Maintain the number of services you provide	14 (1.7%)	103 (12.2%)	319 (37.7%)	410 (48.5%)
Maintain the range of services you provide	29 (3.4%)	116 (13.7%)	319 (37.7%)	382 (45.2%)
Maintain the number of beneficiaries supported	10 (1.2%)	82 (9.7%)	308 (36.4%)	446 (52.7%)
Meet the demand for your service(s)	8 (0.9%)	91 (10.8%)	313 (37.0%)	434 (51.3%)
Total	-	-	-	846 (100%)

Source: Wave 1 grant holder survey, Q26 (n = 846).

Table 6.12: Changes in services, March 2024 in comparison to March 2023

Awarded	Gone up a lot	Gone up a little	Stayed the same	Gone down a little	Gone down a lot	Don’t know	Prefer not to say / n/a
The number of services you provide	353 (41.4%)	331 (38.8%)	141 (16.5%)	19 (2.2%)	1 (0.1%)	1 (0.1%)	7 (0.8%)
The range of services you provide	235 (27.5%)	401 (47.0%)	193 (22.6%)	15 (1.8%)	2 (0.2%)	1 (0.1%)	6 (0.7%)
The number of beneficiaries supported	582 (68.2%)	228 (26.7%)	26 (3.0%)	13 (1.5%)	2 (0.2%)	1 (0.1%)	1 (0.1%)
Ability to meet the demand for your service(s)	209 (24.5%)	302 (35.4%)	160 (18.8%)	128 (15.0%)	46 (5.4%)	5 (0.6%)	3 (0.4%)
Total	-	-	-	-	-	-	853 (100%)

Source: Wave 1 grant holder survey, Q25 (n = 853).

Table 6.13: Demand for services between March 2023 and award of funding from the CCLF

Demand	Awarded
It went up a lot	666 (78.1%)
It went up a little	175 (20.5%)
It stayed the same	10 (1.2%)
It went down a little	0 (0.0%)
It went down a lot	0 (0.0%)
Don’t know	2 (0.2%)
Total	853 (100%)

Source: Wave 1 grant holder survey, Q24 (n = 853).

Table 6.14: If staff numbers would have fallen during the autumn/winter 2023/24 without the CCLF

Awarded
Yes	391 (45.8%)
No	337 (39.5%)
Don’t know	125 (14.7%)
Total	853 (100%)

Source: Wave 1 grant holder survey, Q34 (n = 853).

Table 6.15: Extent to which beneficiaries accessing CCLF-funded services have experienced the outcomes

Awarded	The majority of beneficiaries experienced this outcome	Some of our beneficiaries experienced this outcome	None of our beneficiaries experienced this outcome	This outcome area is not relevant to our CCLF-funded service
People’s immediate needs were met	792 (92.8 %)	52 (6.1%)	0 (0%)	9 (1.1%)
People were signposted to other relevant support and services	306 (35.9 %)	461 (54.0%)	18 (2.1%)	68 (8.0%)
People were able to access the advice and information they needed	632 (74.1%)	162 (19.0%)	1 (0.1%)	58 (6.8%)
People were able to access a suitable safe space for their needs	622 (72.9%)	115 (13.5%)	5 (0.6%)	111 (13.0%)
People experienced an improvement in their mental health and wellbeing	531 (62.3%)	283 (33.2%)	0 (0.0%)	39 (4.6%)
People experienced an improvement in their physical health	265 (31.1%)	393 (46.1%)	14 (1.6%)	181 (21.2%)
Total	-	-	-	853 (100%)

Source: Wave 1 grant holder survey, Q15 (n = 853).

Table 6.16: How the ‘range’ of services provided by organisations has changed, when thinking about October 2024 compared to March 2024

Change	Successful	Successful	Unsuccessful	Unsuccessful
Gone up a lot	75	23.7 %	59	30.6 %
Gone up a little	124	39.2 %	78	40.4 %
Stayed the same	101	32.0 %	39	20.2 %
Gone down a little	12	3.8 %	14	7.3 %
Gone down a lot	4	1.3 %	3	1.6 %
Don’t know	0	0.0 %	0	0.0 %
Prefer not to say / not applicable	0	0.0 %	0	0.0 %
Total	316	100.0 %	193	100.0 %

Source: Wave 2 grant holder survey, Q2 (n = 316). wave 2 unsuccessful applicant survey, Q2 (n = 193).

Table 6.17: How the ‘number’ of services provided by organisations has changed, when thinking about October 2024 compared to March 2024

Change	Successful	Successful	Unsuccessful	Unsuccessful
Gone up a lot	61	19.3 %	45	23.3 %
Gone up a little	150	47.5 %	84	43.5 %
Stayed the same	90	28.5 %	48	24.9 %
Gone down a little	12	3.8 %	12	6.2 %
Gone down a lot	3	0.9 %	4	2.1 %
Don’t know	0	0.0 %	0	0.0 %
Prefer not to say / not applicable	0	0.0 %	0	0.0 %
Total	316	100.0 %	193	100.0 %

Source: Wave 2 grant holder survey, Q2 (n = 316). Wave 2 unsuccessful applicant survey, Q2 (n = 193).

Table 6.18: Whether organisations took cost-management decisions between March 2024 and October 2024

Cost-management decisions	Successful	Successful	Unsuccessful	Unsuccessful
Reduced staff hours	70	22.2 %	71	36.8 %
Made staff redundancies	34	10.8 %	34	17.6 %
Asked staff to take a pay cut or forgo a pay rise	19	6.0 %	30	15.5 %
Unplanned use of reserves	134	42.4 %	99	51.3 %
Increased the price of services	43	13.6 %	47	24.4 %
Took on debt	5	1.6 %	15	7.8 %
Sold assets	5	1.6 %	11	5.7 %
Renegotiated grants/ commissioned contracts	39	12.3 %	33	17.1 %
Cancelled existing grants/commissions contracts	5	1.6 %	3	1.6 %
Other [please specify]	30	9.5 %	30	15.5 %
Don’t know	9	2.8 %	2	1.0 %
Our organisation did not have to take any cost-management decisions	87	27.5 %	26	13.5 %
Total	316	100.0 %	193	100.0 %

Source: Wave 2 grant holder survey, Q6 (n = 316). Wave 2 unsuccessful applicant survey Q7 (n = 193).

Table 6.19: Anticipated cost management decisions over the coming winter period (between time of completing the survey and March 2025)

Cost-management decisions	Successful	Successful	Unsuccessful	Unsuccessful
Reduce staff hours	89	28.2 %	65	33.7 %
Make staff redundancies	44	13.9 %	26	13.5 %
Ask staff to take a pay cut or forgo a pay rise	38	12.0 %	40	20.7 %
Needing to use reserves	164	51.9 %	116	60.1 %
Increase the price of services	57	18.0 %	54	28.0 %
Take on debt	10	3.2 %	14	7.3 %
Sell assets	13	4.1 %	8	4.1 %
Renegotiate grants/ commissioned contracts	57	18.0 %	34	17.6 %
Cancel existing grants/commissions contracts	9	2.8 %	6	3.1 %
Other [please specify]	31	9.8 %	35	18.1 %
Don’t know / prefer not to say	29	9.2 %	12	6.2 %
Our organisation will not need to take any cost-management decisions	48	15.2 %	21	10.9 %
Total	316	100.0 %	193	100.0 %

Source: Wave 2 grant holder survey, Q7 (sample size = 316). Wave 2 unsuccessful applicant survey, Q8 (sample size = 193).

Table 6.20: Portion delivery costs rose by between March 2024 and October 2024

Portion rise	Staff wages	Volunteer costs	Equipment and materials	Overheads	Other costs
They didn’t rise	13.6 %	26.6 %	7.3 %	4.4 %	7.7 %
Less than 5%	26.3 %	25.9 %	16.1 %	13.0 %	4.9 %
Between 5 and 10%	34.5 %	21.8 %	33.5 %	31.0 %	18.5 %
Between 10 and 15%	10.4 %	8.2 %	22.2 %	19.3 %	12.2 %
Above 15%	5.1 %	4.7 %	13.9 %	24.1 %	9.8 %
Don’t know / not applicable	10.1 %	12.7 %	7.0 %	8.2 %	46.9 %
Total	100.0 %	100.0 %	100.0 %	100.0 %	100.0 %

Source: Wave 2 grant holder survey, Q6 (sample size = 316).

Figure 6.5: Proportion of VCSEs by Process Tracing Probability Ranges per Hypothesis

Source: Wave 2 grant holder survey, Q16 – Q35; interpreted using Bayesian Process Tracing

Table 6.21: Likelihood of maintaining and / or increasing the number and range of services provided^{[footnote 32]}

Percentage likelihood of maintaining and / or increasing the number and range of services provided	VCSE organisations	Percentage of VCSE respondents
0% up to 10%	5	0.7%
11% up to 20%	0	0%
21% up to 30%	11	1.6%
31% up to 40%	0	0%
41% up to 50%	0	0%
51% up to 60%	177	26%
61% up to 70%	0	0%
71% up to 80%	2	0.3%
81% up to 90%	103	15%
91% to 100%	385	56%

Table 6.22: Likelihood of maintaining and / or increasing number of service users^{[footnote 33]}

Percentage likelihood of maintaining or increasing the number of service users supported	VCSE organisations	Percentage of VCSE respondents
0% up to 10%	2	0.6%
11% up to 20%	0	0%
21% up to 30%	7	2%
31% up to 40%	0	0%
41% up to 50%	0	0%
51% up to 60%	0	0%
61% up to 70%	0	0%
71% up to 80%	3	0.9%
81% up to 90%	347	97%
91% to 100%	0	0%

Table 6.23: likelihood of maintaining financial resilience by avoiding shrinkage and / or insolvency

Percentage likelihood of avoiding shrinkage and / or insolvency	VCSE organisations	Percentage of VCSE respondents
0% up to 10%	115	13%
11% up to 20%	67	7%
21% up to 30%	44	5%
31% up to 40%	329	36%
41% up to 50%	67	7%
51% up to 60%	20	2%
61% up to 70%	143	16%
71% up to 80%	65	7%
81% up to 90%	15	2%
91% to 100%	50	5%

Table 6.24: Likelihood of VCSE organisations retaining staff / volunteers

Percentage likelihood of retaining staff and / or volunteers	VCSE organisations	Percentage of VCSE respondents
0% up to 10%	225	25%
11% up to 20%	273	30%
21% up to 30%	19	2.1%
31% up to 40%	150	16%
41% up to 50%	0	0%
51% up to 60%	4	0.4%
61% up to 70%	244	27%
71% up to 80%	0	0%
81% up to 90%	0	0%
91% to 100%	0	0%

Table 6.25: Demand for services between March 2024 and October 2024

Demand	Grant holder	Grant holder	Unsuccessful	Unsuccessful
It went up a lot	172	54.4 %	139	72.0 %
It went up a little	105	33.2 %	44	22.8 %
It stayed the same	26	8.2 %	7	3.6 %
It went down a little	10	3.2 %	2	1.0 %
It went down a lot	2	0.6 %	1	0.5 %
Don’t know	1	0.3 %	0	0.0 %
Total	316	100.0 %	193	100.0 %

Source: Wave 2 grant holder survey, Q1 (n=316). Wave 2 unsuccessful applicant survey, Q1 (n=193).

Table 6.26: Whether unsuccessful organisations received other funding to support delivery of services CCLF application was for

Received other funding	VCSE organisations	Percentage of VCSE organisations
Yes	126	31.5 %
No	261	65.3 %
Don’t know	13	3.3 %
Total	400	100.0 %

Source: Wave 1 unsuccessful applicant survey, Q7 (n=400)

Table 6.27: Extent to which unsuccessful organisations were able to continue delivering services that CCLF application was for

Extent to which unsuccessful organisations were able to continue delivering services	VCSE organisations	Percentage of VCSE organisations
Completely – we have continued delivering the services(s) as we had been before and have expanded our service(s)	24	6.0 %
Completely – we have continued delivering the service(s) as we had been before we applied for CCLF funding	110	27.5 %
A bit – we have continued delivering the service(s) but we have reduced what we deliver / who we deliver to	238	59.5 %
Not at all – we had to stop delivering the service(s)	23	5.8 %
Don’t know	5	1.3 %
Total	400	100.0 %

Source: Wave 1 unsuccessful applicant survey, Q9 (n=400)

Table 6.28: Treatment effects by sample and outcome

Sample	Outcome	Estimated treatment effect	Std. Error	Lower CI	Upper CI	P value
All organisations (n = 1854)	Total gross income	£412,631	1364398	-2261588	3086851	0.762
All organisations (n = 1854)	Total gross expenditure	£373,887	1977285	-3501592	4249367	0.85
All organisations (n = 1854)	Net income	£38,744	27428	-15015	92504	0.158
Organisations < £10m income (n = 1816)	Total gross income	£44,514	17146	10908	78120	0.009
Organisations < £10m income (n = 1816)	Total gross expenditure	£23,340	15761	-7551	54231	0.139
Organisations < £10m income (n = 1816)	Net income	£21,174	10928	-245	42592	0.053
Organisations < £1m income (n = 1460)	Total gross income	£31,058	7389	16576	45540	0
Organisations < £1m income (n = 1460)	Total gross expenditure	£9,943	5126	-103	19990	0.052
Organisations < £1m income (n = 1460)	Net income	£21,114	5270	10786	31443	0

Source: Charity Commission data on income and expenditure

The VCSE Cost of Living Advisory Board was comprised of experts from different VCSE organisations / membership bodies, and was set up to advise on the design of the VCSE Cost of Living scheme, and support its implementation, by advertising the programme through its networks. The Board was updated on a monthly basis on the progress of CCLF delivery. ↩
For example: Rothgang, M., & Lageman, B. (2021). The unused potential of process tracing as evaluation approach: The case of cluster policy evaluation. Evaluation, 27(4), 527–543. https://journals.sagepub.com/doi/full/10.1177/13563890211041676; Wadeson, A., Monzani, B. and Aston, T. (2020) Process Tracing as a Practical Evaluation Method: Comparative Learning from Six Evaluations, Befani, B., 2020. Diagnostic evaluation and Bayesian Updating: Practical solutions to common problems. Evaluation, 26(4), pp.499-515. ↩
The full detail of how this methodology was carried out and further detail on outputs is given in Annex 4 with a visualisation of results as a stacked bar chart in annex 6. ↩
Posteriors refer to the updated probabilities of hypotheses being true after considering new evidence, and reflect the strength of the evidence supporting or contradicting a hypothesis. ↩
Including latest financial accounts, financial position and the proposed project budget ↩
The National Lottery Community Fund have noted that customer satisfaction rates for applicants that have not been awarded funding tend to be lower than for funded applicants ↩
This happened only once, but, according to the research, created a challenging situation for those using the portal when it happened. ↩
The National Lottery Community Fund facilitated an offline application process via its Advice Line, for those that requested support. ↩
This panel was comprised of experienced Senior Managers at The National Lottery Community Fund. ↩
Distribution of funding across the regions generally reflected the England population distribution, according to 2022 ONS data. For example, the London, South East and East of England population comprised 43% of England’s population in 2022, and 41% of CCLF organisations were based in this region. Data available from the ONS; ↩
Differences between regions in terms of funding status were not statistically significant. ↩
At the application stage, applicants were asked to state if 75% or more of the people supported by their project would be from specific groups. Those that said yes are considered to be ‘targeted’ in this analysis. ↩
Categories derived from NCVO’s UK Civil Society Almanac. Categories are defined in relation to annual income, as follows: ‘Micro’ (less than £10,000); ‘Small’ (£10,000 to £100,000); ‘Medium’ (£100,000 to £1mil)’ ‘Large’ (£1mil to £10mil); ‘Major’ (£10mil to £100mil); and ‘Super Major’ (more than £100mil). See:https://www.ncvo.org.uk/news-and-insights/news-index/uk-civil-society-almanac-2023/ ↩
As the figure is a histogram, each bar on the chart represents a c. £2100 increment of funding from £10,000 up to £75,000. For example, this means that the first bar represents the number of awards made where the award was between £10,000 and £12,100. ↩
According to Consumer Prices Index Consumer Prices Index, in March 2023, food inflation was 19.2%; the highest annual rate in the last 45 years. Although the inflation rate had been decreasing since, in January 2024 the inflation rate for food and non-alcoholic beverage was at 7%, which was considerably higher than the general inflation rate of 4%. ↩
Now Ministry of Housing, Communities and Local Government (MHCLG) ↩
Number’ refers to the number of services that an organisation might have provided (e.g. number of advice sessions of a particular service, number of classes of a particular service). ↩
‘Range’ refers to different types of services that an organisation might have provided (e.g. warm space, safe space, food bank, advice). ↩
For 232 organisations non-responses to some survey questions meant the evidence feeding into BPT for hypothesis 1 was not available. As there was insufficient evidence to draw conclusions around likelihoods for H1 for these organisations, they have been excluded from the analysis. ↩
For 556 organisations non-responses to some survey questions meant the evidence feeding into BPT for hypothesis 2 was not available. As there was insufficient evidence to draw conclusions around likelihoods for H2 for these organisations, they have been excluded from the analysis. ↩
Prior to CCLF, some organisations providing essential food and hygiene products had to place limits or caps on the number of products they could provide per beneficiary, to manage the demand and ensure more people would be able to access products. CCLF enabled some organisations to remove this cap or limit, temporarily. ↩
It is important to note that while this methodology provides a structured approach to updating our confidence in our set of causal claims, it has several limitations which may explain this discrepancy, which are explored further in Annex 4. ↩
8% also reported ‘don’t know / not applicable’ ↩
The VCSE Energy Efficiency Scheme (EES) – also part of the VCSE Cost of Living programme with the CCLF. ↩
This figure was derived from a range of unit value estimates provided by the HACT Social Value Bank, although corrected to reflect the likely overall impact of accessing provision, as the original unit value estimates would be a vast over-estimation. The correction provided a range representing lower and upper bound estimates, and DCMS selected the mean value to present in its FBC. ↩
This figure was derived from data provided in the Citizens’ Advice Bureau’s 2021/22 impact reporting, which estimated the financial gain for users of its financial hardship phoneline. The total financial gain was divided by the total number of phoneline users to provide the estimated value used in the FBC. ↩
Similarly, this figure was derived from unit value estimates provided by the New Economics Foundation on protection from harm, violence or abuse, but adjusted downwards to reflect likely impact of accessing a safe space. The mean value was selected in the FBC. ↩
NCVO How many voluntary organisations are there? ↩
For example: Rothgang, M., & Lageman, B. (2021). The unused potential of process tracing as evaluation approach: The case of cluster policy evaluation. Evaluation, 27(4), 527–543] Wadeson, A., Monzani, B. and Aston, T. (2020) Process Tracing as a Practical Evaluation Method: Comparative Learning from Six Evaluations, Befani, B., 2020. Diagnostic evaluation and Bayesian Updating: Practical solutions to common problems. Evaluation, 26(4), pp.499-515. ↩
Befani (2020) UKES / CECAN Online Masterclass Bayesian Updating (Diagnostic Theory-Based Evaluation) ↩
Trimmed means involve removing a small percentage of the largest and smallest values (to remove outliers) before calculating the means. ↩
For 232 organisations non-responses to some survey questions meant the evidence feeding into BPT for hypothesis 1 was not available. As there was insufficient evidence to draw conclusions around likelihoods for H1 for these organisations, they have been excluded from the analysis. ↩
For 556 organisations non-responses to some survey questions meant the evidence feeding into BPT for hypothesis 2 was not available. As there was insufficient evidence to draw conclusions around likelihoods for H2 for these organisations, they have been excluded from the analysis. ↩

Cookies on GOV.UK

Executive summary

Programme overview

Approach to the evaluation

Findings: process evaluation

Findings: impact evaluation

Findings: Economic Evaluation

1.0 Introduction

1.1 Background and context

1.2 Overview of the evaluation

1.3 Methodology

1.3.1 Programme stakeholder and funding manager interviews

1.3.1 Case studies

1.3.3 Surveys

Box 1: Interpreting bayesian process tracing results

1.3.4 Analysis of management information data

1.3.5 Limitations

1.4 Report structure

2.0 Process evaluation

Chapter summary

2.1. Design and set-up phase

2.1.1. Reflections on overall CCLF design

2.1.2. Effectiveness of the design and set-up phase

2.2. Engagement, application processes and decision-making

2.2.1 Effectiveness of communications and outreach

2.2.2. Application process

2.2.3. Decision-making processes

2.3. Programme reach

2.4. Project delivery

2.4.1. Overview of services delivered

2.4.2. Need for funding

Box 2: Example of food bank services dealing with decline in donations and increased demand for their services

2.4.3. What worked well in delivery

2.4.4. Challenges and area for improvements in delivery

2.4.5. Grant monitoring and reporting

2.5 Lessons learned from the CCLF process evaluation

3.0 Outcomes and impacts

Chapter summary

3.1. Impact of CCLF on VCSE organisations

3.1.1. Service delivery outcomes

3.1.1.1. Organisations sustain and adapt service delivery in the short-term

Box 3: Process tracing results for likelihood of maintaining and / or increasing the number and range of services provided

3.1.1.2. Organisations meet demand in the short-term

Box 4: Process tracing results for likelihood of CCLF contributing to organisations maintaining and/or increasing the number of service users reached

3.1.1.3. Organisations maintain delivery of services in the medium to long-term

Box 5: Reduction in CCLF funded service

3.1.2. Financial resilience

3.1.2.1. Ability to meet high costs related to service delivery

3.1.2.2. Increased income and expenditure for CCLF organisations

Box 6: Process tracing results for likelihood of CCLF contributing to organisations avoiding shrinkage and insolvency

3.1.2.3. Use of reserves

3.1.2.4. Medium to longer-term effects on financial resilience

Box 7: Securing additional funding

3.1.3. Outcomes for staff and volunteers

3.1.3.1 Ability to meet short-term pressures

3.1.3.2. Staff retention and averted job losses

Box 8: Likelihood of CCLF contributing to VCSE organisations retaining staff / volunteers

Box 9: Pathways to volunteering and paid opportunities

3.1.3.3. Sustaining positive effects on resourcing and capacity beyond the funding period

3.2. Outcomes for beneficiaries

3.2.1. Short-term outcomes

3.2.1.1. Immediate needs met

3.2.1.2. Accessing suitable safe spaces

3.2.1.3. Accessing financial and housing advice

3.2.1.4. Signposting and use of other services by beneficiaries

3.2.2. Medium to longer-term outcomes

3.3. Wider outcomes

4.0 Economic evaluation

Chapter summary

4.1. Costs of the CCLF

4.2. Economic assessment of the CCLF – benefits to VCSEs

4.3. Economic assessment of the CCLF – benefits to beneficiaries

4.4. Overall assessment of value for money

5.0 Conclusions and recommendations

5.1. Conclusions

5.2. Recommendations

6.0 Annexes

Annex 1: Evaluation framework

Annex 2: Theory of Change

Annex 3: Case study sampling