Research and analysis

​Rough sleeping and complex needs process evaluation: Understanding service user experiences and progress

Published 11 December 2025

Applies to England

Foreword

The Ministry of Housing Communities and Local Government is committed to following an evidence-informed approach to reducing homelessness and rough sleeping. The Rough Sleeping and Complex Needs Process Evaluation is a further step to understanding how services can best support people who sleep rough.

This report builds on the evidence on how interventions work or services are experienced. It has been published alongside a number of other reports focused on rough sleeping including the Rough Sleeping Initiative process research, the Housing First Regional Pilots process research, qualitative research exploring access and use of support services by people with experience of rough sleeping, and the initial findings of a large-scale survey using the Rough Sleeping Questionnaire. In addition, the department has published reports addressing wider homelessness: a review of the Homelessness Reduction Act and a report on the costs of Temporary Accommodation.

This report focuses on services, funded as part of the Homelessness Prevention Programme, which provided support for people with experience of rough sleeping with complex needs, such as co-occurring mental health and substance misuse needs.    

The report would not have been possible without support from a number of people who invested considerable time and energy.  We are grateful to Steve Green and Sarah Pearson and the wider team at Centre for Regional Economic and Social Research at Sheffield Hallam University for delivering this research.

We are grateful for the advice of all those who fed into the design of the questionnaire, including Rob Edgar at Groundswell, and Nick Maguire of Southampton University.

I would also like to thank Jenny Jackman and Lucy Spurling, as well as the wider Homelessness and Troubled Families Analytical Team in MHCLG.

And, most importantly, we would like to thank all the service users and all the local authorities and service providers who gave their time to participate in this research, without whom the research would not have been possible.

Stephen Aldridge

Chief Economist & Director For Analysis and Data

Ministry of Housing Communities and Local Government

Key findings

This report presents data collected from 12 case study sites funded through the Rough Sleeping Grant (RSG) and Rough Sleeping Social Impact Bonds (SIB). The data outlines the experiences and needs of service users, the progress they have made through the case study projects and an estimation of the costs of service delivery.

The case study projects were selected for inclusion in the evaluation because they aimed to support people with complex needs, defined for the purposes of this evaluation as projects that were supporting people who were experiencing mental ill-health and/or substance misuse. The data provides rich insights into the needs and experiences of people with complex needs who are currently, have experienced, or are at risk of sleeping rough.

The majority of service users who participated in this research were male (78%), and White (84%). Almost half (47%) were under 40 years of age.

The experiences of people with complex needs who are sleeping rough

To summarise the experiences and needs of service users:

• The service users had experienced insecure housing, both immediately prior to their engagement with the projects and often for many years before that. One fifth of service users had not had access to secure accommodation for over ten years and it was common for individuals to have been in and out of different forms of housing and to have experienced different forms of homelessness.
• The majority of service users self-reported that they had at least one of a mental health issue, addiction issue or a long-standing physical illness or disability, and there were high levels of co-morbidity. They also had levels of mental well-being that were much lower than the national average and were engaging frequently with health services, including GPs, A&E and ambulance services. Many reported that they had first experienced health and well-being needs in childhood and young adulthood; over two-fifths of respondents reported that they had first experienced issues related to drug addiction under 18 years of age, and over a third of respondents reported that they had first experienced issues related to alcohol addiction under 18 years of age. 
• Service users had also often experienced other negative life events. Almost three quarters had spent time in prison, and almost half had been in care. Two fifths had experienced permanent exclusion from school.

Progress of service users

Data on the progress of service users looked at change for individual service users between the baseline (close to the point of initial engagement with the projects) and six, nine or 12 months. Follow-up sample sizes were low, and this has limited the analysis of outcomes. However, statistically significant reductions were identified in rough sleeping and increases in access to permanent accommodation across the projects. There was also a significant increase in the numbers of SIB service users accessing drug and alcohol support services at the six-month stage. The analysis also identified a statistically significant reduction in A&E attendance amongst service users. There were no statistically significant improvements in the mental health and wellbeing of service users, and access to mental health services was low, despite high levels of self-reported mental health needs at the baseline stage. There were also no statistically significant improvements in training or work outcomes, and service providers were of the view that these were longer-term goals for most service users, which were unlikely to be achieved in the timeframe for this evaluation.

Differences between RSG and SIB case studies

A higher proportion of service users in the RSG case study projects had stayed in long-term accommodation compared to those in the SIB case studies. This reflects that three of the RSG case study projects were delivering housing first style initiatives which prioritises access to long-term accommodation.

SIB service users were more likely to have stayed in homelessness or temporary accommodation or experienced sleeping rough than RSG service users. This reflects the difficulties experienced across all the case study projects in accessing appropriate long-term accommodation

Income, costs and staffing

Analysis of income, costs and staffing in the case study projects revealed that costs were higher in RSG projects which were delivering housing first style initiatives, but overall the average costs of delivery per beneficiary were lower than the benchmarks considered.^{[footnote 1]} Non-operational costs were higher in the SIB projects, reflecting greater monitoring and management costs required to oversee the SIB. The evidence available suggests that the projects were staffed with people at appropriate grades for the respective task or activity.

Conclusions

The data on the experiences and needs of service users present a number of implications for policy and practice:

• The needs and experiences of people who experience rough sleeping should not necessarily be seen as separate to those of other groups who experience homelessness. Many of the people being supported by the case study projects had been in and out of many different forms of accommodation and had experienced periods in temporary accommodation, rough sleeping and sofa surfing. Early access to suitable permanent accommodation needs to be a priority for this group.
• The high levels of complex health and wellbeing needs amongst this group require specific service approaches. The projects had been able to improve access to drug and alcohol treatment services and as a result there had been significant reductions in the numbers of service users reporting needs in relation to drug and alcohol misuse. This was not the case for mental health and wellbeing outcomes, where no significant improvements were found, and as discussed in case study learning reports, lack of timely access to mental health services remains a major challenge in meeting the needs of those who are experiencing multiple vulnerabilities.
• There were no significant changes in the education or employment status of service users and service providers prioritised their attention on other outcomes such as access to health and support services. There may be scope for these outcomes to improve in the longer-term for the SIB projects.
• Common features of the delivery models and implementation of the RSG and SIB case study projects included person-centred and tailored support, focusing on targeted outreach, improving and sustaining access to services (including housing), reductions in service duplication and improved cross-referrals through collaboration. The analysis presented here suggests that these are effective approaches, but that their success depends crucially on capacity within mainstream services to accommodate the needs of this group.

1. Introduction

1.1 Background

In December 2016 the government committed £51m to homelessness prevention schemes across England. This included targeted support for people sleeping rough and those at risk of rough sleeping across several funding streams.  This evaluation was concerned with two of these streams:

• £20 million for Rough Sleeping Grants (RSGs) to provide targeted support for people at imminent risk of sleeping rough or those who are new to the streets.
• £11m to support locally commissioned Social Impact Bonds (SIBs) to help those sleeping rough with the most complex needs.

In addition, £20 million was released for Homelessness Prevention Trailblazers to pilot new initiatives to tackle homelessness in local authority areas.

1.2 The rough sleeping and complex needs process evaluation

The Rough Sleeping and Complex Needs Process Evaluation was funded by MHCLG and carried out by the Centre for Regional Economic and Social Research (CRESR) at Sheffield Hallam University.  The evaluation examined how two funding initiatives – Rough Sleeping Grant (RSG) and Rough Sleeping Social Impact Bond (SIB) – provided support for people who had slept rough and had complex needs^{[footnote 2]} (co-occurring mental health and substance misuse needs). The evaluation focuses only on projects supporting this particular cohort (referred to as ‘projects in scope’), rather than on all projects funded by the RSG and Rough Sleeping SIB.

The aim of the process evaluation was to provide evidence on different approaches taken to working with people who are sleeping rough who have co-occurring mental health and substance misuse needs, and to identify the lessons learned in designing, setting up and delivering these across the RSG and Rough Sleeping SIB funding programmes.

This includes specific objectives:

• To understand what the different projects working with this service user group are delivering, considering the needs of the service users, any multi-agency partnerships and any constraints in the local context.
• To understand the influence of the funding mechanism on design, set up and delivery.
• To identify the process factors that stakeholders identify as working well across the design, set up and delivery of the interventions, where the challenges lie, and how these have been or may be addressed.
• Where services are near to the end of their funding period, explore any plans for sustaining the intervention in original or redesigned form and the reasons behind this.

These objectives were met through research focused on 12 projects.  Six were Rough Sleeping SIBs, and six were initiatives funded by the Rough Sleeping Grant.  Table 1.1 provides an overview of the 12 case studies.

Table 1.1: Case Studies

Area	Project	Fund	Funding Total (£)	Launch Date
A	Rough sleeping outreach	RSG	263,700	April 2017
B	Rough sleeping outreach	RSG	390,736	July 2017
C	Rough sleeping outreach	RSG	166,209	September 2017
D	Housing first style initiative	RSG	393,000	September 2017
E	Housing first style initiative	RSG	211,629	January 2018
F	Housing first style initiative	RSG	376,652	March 2017
G	Rough Sleeping SIB	SIB	1,000,000	January 2018
H	Rough Sleeping SIB	SIB	1,251,000	November 2017
I	Rough Sleeping SIB	SIB	1,261,980	November 2017
J	Rough Sleeping SIB	SIB	3,000,000[footnote 3]	November 2017
K	Rough Sleeping SIB	SIB	1,449,020	September 2017
L	Rough Sleeping SIB	SIB	1,540,000	January 2018

Qualitative and quantitative data collection was carried out.  Interviews were undertaken with a range of stakeholders, including local authority commissioners, delivery teams, partner services and service users who had used the projects in scope and had experienced rough sleeping and complex needs.  A questionnaire survey of service users was analysed, project outcome logs were analysed, and an assessment of the costs and resourcing of the projects was made.

The evaluation has produced the following reports:

• Report 1: Key Findings from the Evaluation
• Report 2: Understanding Service User Experiences and Progress (this report)
• Report 3: Learning from Six Rough Sleeping Grant Case Studies
• Report 4: Learning from Six Rough Sleeping Social Impact Bond Case Studies

Also, three short briefing notes, aimed at practitioners and commissioners, are available on the following subjects: Workforce Development, Access to Services, and Meeting Accommodation Needs.

1.3 This report

This report relates to the first two evaluation objectives: understanding the needs of service users and the influence of different funding mechanisms. It presents data collected from the case study sites to outline the experiences and service needs of people with complex needs who are sleeping rough, the progress they have made through the case study projects and estimate the costs of service delivery associated with rough sleeping.

The remainder of this report is structured as follows.  Chapter Two explores the characteristics and experiences of people with complex needs who were sleeping rough in the case study areas.  Chapter Three presents analysis of progression and outcomes for service users.  Chapter Four looks at the costs of service provision, and Chapter Five presents a brief conclusion.  There are two appendices.  The first contains analysis of cross-sectional outcome data, and the second is a technical note on the key data sources for the report.

Data collection

This report draws on a number of data sources:^{[footnote 4]}

• Data on the characteristics and experiences of people with complex needs have been gathered through a service user questionnaire, developed by MHCLG, and administered by project staff in the case study areas. This questionnaire was developed in an iterative process with input from: a psychologist with expertise in the homelessness sector; people with lived experience; and experts in the homelessness sector. The baseline questionnaire was designed to understand the extent and the range of clients’ support needs and to establish patterns of public service use, with a longer-term goal of understanding the costs of rough sleeping and people’s journeys into homelessness and rough sleeping. MHCLG also produced a follow-up questionnaire designed to measure the progress of service users against baseline, as a means to explore how well homelessness services were working to inform a potential impact evaluation. The questionnaires received ethical approval from Southampton University. Baseline questionnaires were administered as soon as practical after a service user joined the projects and follow-up questionnaires were administered at a later date. Although the target for the administration of follow-up questionnaires was after six months of engagement with the project it was not always feasible to implement this in practice. The time lapse between baseline and follow-up surveys for individuals ranged from two months to 11 months, with a mean of eight months and a median of nine months.
• Data on accommodation, mental and substance use treatment and employment outcomes were collected via a quarterly outcome log submitted by service providers. This collected consenting clients’ status on these outcomes for their first year in the service.
• Data on the costs of services were collected through a template which was filled in by service managers.

This data is a rich source of evidence in relation to the characteristics and needs of individuals who are being supported by the case study projects involved in this research but there are three important caveats to note, particularly in relation to the service user questionnaire:

• Survey respondents were identified and recruited by the service providers as clients appropriate to take part in the research (based on the trust and rapport built up between client and service provider and the vulnerability of the client). There is therefore potential for bias in the sample and the evaluation does not have data on all of the service users being supported through the projects. However, the data have been discussed with stakeholders across the RSG and SIB case studies who have confirmed that they generally typified the people who had used their services.  This provides some validation, therefore, that the sample was broadly representative of those being supported by the case study projects.
• Although financial support was available to assist the projects in data collection, completion of the questionnaire was not a condition of funding for the projects, or for receipt of support on the part of service users. The sample is dependent therefore, on the willingness and capacity of the projects to administer the survey successfully. As a result, there are differences in the numbers of responses from different case study areas, and overall follow-up sample sizes were small, limiting the opportunity to identify statistically significant outcome changes.
• Almost three quarters of the sample are service users from SIB projects which have an explicit objective to work with those who are sleeping rough and have the most complex needs.

Nevertheless, the questionnaire provides a dataset for a cohort of people who have experienced rough sleeping and have lived with complex needs, including poor mental health and alcohol and substance misuse. The data greatly enriches our understanding of the needs and experiences of this cohort and provides the context necessary for understanding the outcomes achieved by projects over the timeframe of the evaluation.

2. The experiences of people with complex needs who are sleeping rough

Summary

This chapter is based on data collected through a survey of people with complex needs who had experienced rough sleeping and were being supported by the case study projects. It shows that for the majority of respondents, insecurity had been the defining feature of their housing experience, both immediately prior to engagement with the projects and often for many years before that. One fifth of respondents had not had access to secure accommodation for over ten years. The data also confirms that it was common for respondents to have been in and out of different forms of housing and to have experienced different forms of homelessness.

Around half of the respondents self-reported their health status to be ‘fair’. Nevertheless, the large majority of respondents self-reported that they had at least one of a mental health issue, an addiction issue or a long-standing physical illness or disability, and there were high levels of co-morbidity. The respondents also had levels of mental well-being that were much lower than the national average. As a result of these needs, the respondents were engaging frequently with mainstream services, including GPs, A&E and ambulance services.

Many respondents reported that they had first experienced health and well-being needs in young adulthood. Not all respondents were able to identify when they had first experienced health and well-being needs but of those that did, the average ages for first experiencing issues relating to drug or alcohol addiction were 18 years and 20 years respectively. The average age for engagement with mental health services was 25 years.

Virtually all the respondents were unemployed when they engaged with the projects. They had also often experienced other negative life events. Almost three quarters had spent time in prison, and almost half had been in care. Two fifths had experienced permanent exclusion from school.

2.1 Introduction

This chapter draws on data collected through the baseline service user questionnaire to highlight the characteristics and experiences of people with complex needs who were sleeping rough. This provides important context in which to locate the subsequent data on progression and outcomes and the costs of services. 

A service user questionnaire was developed by MHCLG and administered by project workers in 11 of the 12 case study areas. The analysis presented in this chapter is based on data collected via this questionnaire which service users were asked to complete at, or close to, the start of their engagement with the service. 197 responses were received in total by November 2018. Responses from individual case study areas ranged from 2 to 47, and a total of 52 responses were submitted from clients of RSG projects and 145 from clients of SIB projects.

Demographically, the cohort is not diverse. The majority of those responding to the service user questionnaire identified their gender as male (78%), and 91% identified as heterosexual.  Eighty-four per cent of respondents were White and 7% were Black British/African/Caribbean. The majority considered themselves to be English (61%). In terms of age, two fifths were over 30 years of age and over half (52%) were aged between 30 and 49 years. This chimes with other data on rough sleeping which confirms that the majority of people who sleep rough are single men aged over 26 years, and from the UK.^{[footnote 5]}  Despite the homogeneity of the group however, it is important to note that all of the case study projects emphasised the need for personalised services which are specifically tailored to individual needs.^{[footnote 6]}

The findings are discussed under the following headings:

• Housing and homelessness
• Health and wellbeing
• Education and employment
• Mainstream service use
• Life experiences

2.2 Housing and homelessness

This section explores the housing and homelessness experiences of respondents to the survey, both immediately prior to their completion of the first questionnaire, and over the months and years preceding it. This is important as the projects aimed to support both people who were new to the streets or at risk of rough sleeping (particularly through the RSG case studies) and those who had experienced longer spells of sleeping rough (particularly through the SIB projects). The data is presented under two sub-headings: recent experience and lifetime experience.

Recent experience

The service user questionnaire provides data on the most recent accommodation experiences of the respondents by asking them where they had stayed the night before completing the questionnaire. These figures confirm that the case study projects were indeed working with a group of people who had lacked recent access to secure accommodation. 63% of respondents were not staying in ‘long-term’ accommodation (measured by where they were staying the night before completing the questionnaire). A majority were staying in temporary or insecure accommodation or sleeping on the streets, in tents or other people’s homes (Figure 2.1).

Figure 2.1: Non-long-term accommodation stayed in last night

Base: 125

The questionnaire also asked about where clients had stayed for the majority of the past month (Figure 2.2). Thirty-five per cent had stayed mainly in temporary accommodation and another 34% had been mainly in long-term accommodation (including supported accommodation).^{[footnote 7]} Just over one fifth had mainly slept rough in the month prior to joining the case study projects.

Where people lived seemed to make a difference to their experiences of housing and homelessness. A greater proportion of respondents outside London reported that they had mainly slept rough in the last month (26%), compared to this within London (15%).

Figure 2.2: Where stayed most of the time for the last month

Base: 197

Lifetime experience

Where respondents did not report having a long-term place to stay last night, they were asked when they last had somewhere secure and long-term to live (Figure 2.3).

Twenty-six per cent of these respondents either could not remember or did not want to provide an answer. There was also a high proportion of missing responses to this question^{[footnote 8]} so caution should be taken when interpreting results. However, the data confirms that for the majority of respondents, insecurity had been the defining feature of their accommodation experience for a long time: almost 60% of respondents had not had a secure place to live for two years or longer, and 20% had not had secure accommodation for over 10 years.

Figure 2.3: When service users last had somewhere secure and long-term to live

Base: 94

Figure 2.4 below explores the lifetime housing experiences of respondents. As would be anticipated give the focus of the interventions, 89% of respondents indicated they had slept rough at some point in their lives. However, the responses also revealed that people with complex needs who are sleeping rough experience precarious and unstable housing circumstances, with frequent periods in informal accommodation being common experiences for the majority. Eighty-nine per cent of respondents indicated they had stayed in short-term or temporary accommodation. Fifty-eight per cent had stayed in supported accommodation^{[footnote 9]} in the past and 54% had sofa surfed (stayed at a friend or family member’s house on an informal and temporary basis).

It was common for respondents to have been in and out of different forms of housing and to have experienced different forms of homelessness (Figure 2.5). Seventy per cent of respondents indicated that they had experienced at least three of the types of housing outlined in Figure 2.4 with 29% of these stating they had experienced all four.

Figure 2.4: Housing experiences (lifetime)

Base:197

Figure 2.5: Number of co-occurring housing experiences (lifetime)

Base: 197

Figure 2.6 below shows the last time service users had experienced specific housing situations. The majority of those indicating that they had experienced a housing situation had done so within the past year, with 76% spending time in temporary/homeless accommodation in that period.  This data also demonstrates that most respondents had also experienced multiple housing situations in the last year.

Figure 2.6: When service users last experienced housing situations

Base: Spent time sofa surfing (59), spent time in supported housing (114), asked the local authority for help with housing (140), slept rough (144), spent time in temporary/homeless accommodation (156).

For those who had experienced living in secure accommodation, there were a variety of reasons for leaving. For 23% of respondents their tenancy had ended, or they had been given notice by their landlord for other reasons (see Figure 2.7).^{[footnote 10]} Other reasons included personal choice and the breakdown of partner and family relationships, and in reality, the reasons for leaving and being asked to leave long-term accommodation are complex and nuanced. 

A greater proportion of Non-London based respondents reported that they had left their last settled long-term accommodation due to their contract ending than London-based clients (26% compared to 15%).

Figure 2.7: Reasons for leaving last settled long-term accommodation

Base: 157

There are a range of temporary accommodation options in most areas of the UK, so it was pertinent for the questionnaire to ask respondents who indicated they had slept rough at some point what had led them to sleep rough the last time this happened (Figure 2.8).  Almost half (48%) stated they had slept rough as there was no accommodation available, suggesting that suitable accommodation was in short supply in the case study areas. This finding was confirmed by qualitative work with stakeholders in the case study areas, who cited a lack of appropriate and available accommodation as a key challenge in supporting people with complex needs who were sleeping rough.

Figure 2.8 also demonstrates that almost one in four respondents were offered housing but turned it down and that many respondents lacked advice and support, or the motivation (or wherewithal) to actively seek accommodation. There may well be, therefore, very logical reasons for refusing an offer of housing, and in the qualitative work carried out for this evaluation, service users identified a lack of choice as a key barrier to moving into sustainable accommodation, particularly where options failed to offer the opportunity to move away from potentially harmful situations or peer groups.

Figure 2.8: Reasons for last sleeping rough

Base: 145

This data demonstrates that although there are differences in individual experiences of housing and homelessness there are common patterns for this cohort, with periods of rough sleeping punctuated by movement in and out of different forms of accommodation. It may be beneficial therefore for policy and practice to consider people who experience rough sleeping as a part of a wider group who experience different types of homelessness and for provision to meet the needs of this group to be integral to a broad service offer for people experiencing all forms of homelessness.   

2.3 Health and wellbeing

People who sleep rough are vulnerable to a wide range of health and wellbeing issues. These include mental and physical health problems, substance misuse and addiction, and isolation from family and friends. The definition of complex needs adopted for this evaluation reflects the complex interrelationships between homelessness and the wider health and wellbeing needs of people who experience rough sleeping. As such, this section of the report outlines the respondents’ self-reported health and wellbeing needs, looking at health, connectivity (engagement with family and friends) and mental well-being.

The service user questionnaire asked respondents to rate their health in general (Figure 2.9) and revealed that whilst around half of respondents (46%) felt that their health was fair, around a third rated (34%) their health as bad (either ‘very bad’ or ‘bad’). This is compared to just 5% of adults in England and Wales who rate their health as ‘very bad’ or ‘bad’ and 13% ‘fair’. In contrast, 81% of adults in England and Wales rate their health ‘very good’ or ‘good’.

Figure 2.9: Health in general

Base: 196 (service users), England and Wales figures from 2011 Census.

The service user questionnaires also asked respondents, on average, how often they met up in person with family and/or friends (Figure 2.10). Half of respondents indicated that they meet up with family and/or friends at least once a week or more, however, 29% stated they only meet family and/or friends between once a fortnight or less often than once a month. Just over a fifth indicated that they never meet up with family and/or friends.

Figure 2.10: Frequency that service users meet up in person with family and/or friends

Base: 153

Perhaps unsurprisingly, and particularly given that the majority of respondents were being supported by the Rough Sleeping SIBs which had a remit to support people with the most complex needs, the questionnaire revealed high levels of health needs amongst the respondents (Figure 2.11). The majority of respondents self-reported that they had at least one of a mental health issue, an addiction issue or a long-standing physical illness or disability.^{[footnote 11]}  Almost half (47%) also self-identified that they needed support in relation to alcohol misuse.

Figure 2.11: Health Needs

Base: 184-196

The data confirms that the respondents had high levels of co-morbidity in addition to current or recent experience of homelessness. Eighty-seven per cent of service users indicated that they had experienced two or more of the four health needs outlined in Figure 2.11, with 15% stating they had experienced all four. Just 15% had experienced only one or none (Figure 2.12).

Figure 2.12: Number of co-occurring health support needs

Base: 197

Figure 2.13 below shows when clients first experienced addiction issues and first had contact with mental health services, including the average age of that first experience. Caution should be taken when interpreting results due to the high proportion of clients who could not remember or did not want to say on these questions.^{[footnote 12]} Respondents often experienced addiction or mental health problems early in life. Forty-three per cent of respondents indicated that they had first experienced issues related to drug addiction in childhood or young adulthood (under 18 years of age), and the largest single grouping for this response was under 15 years of age. Similarly, 32% of respondents indicated that they had first experienced issues with alcohol addiction under the age of 18 years. Responses indicated that generally, issues with addiction emerged before contact with mental health services. Only 17% had contact with mental health services in childhood or young adulthood.

Figure 2.13: Age when clients first experienced addiction issues or contact with mental health services

Base: Issues related to drug addiction (122); Issues with alcohol addiction (90); Contact with a mental health service (118)

In terms of mental wellbeing, the questionnaire asked respondents to indicate their overall wellbeing using the Short Warwick Edinburgh scale (SWEMWBS).^{[footnote 13]}  Figure 2.14 shows the results and indicates that overall, wellbeing scores were relatively low for respondents. The mean score for the full sample was 19.4 (out of a possible 35), compared with 25.2 for England.^{[footnote 14]}  More than three-quarters of those surveyed had a mental wellbeing score which was below the national average, and a significant minority of respondents (27%) scored very low – below 17. Only 7% of respondents scored higher than the national mean average.

Figure 2.14: Mental Wellbeing scores (Short Warwick Edinburgh metric)

Base: 106

2.4 Employment and education

Whether service users were in employment, education or training was recorded via the baseline outcome logs. Virtually all service users were unemployed (97%) at the baseline stage.

2.5 Recent contact with mainstream services

Due to the high levels of complex health and care needs, the respondents were frequently engaging with mainstream services. Sixty-nine per cent had experienced an in-patient stay in hospital. Figure 2.16 shows how respondents had accessed GPs, A&E, ambulance and prison services prior to their completion of the first questionnaire. Ninety-six per cent of respondents indicated they were registered with a GP and 74% had visited a GP in the last three months.^{[footnote 15]} Thirty per cent had visited Accident and Emergency (A&E) in the last three months and 24% had used an ambulance in the same period. Just 4% had spent time in prison in the last three months, although 23% had spent time in prison in the last year, and 71% had spent time in prison during their lifetime (Figure 2.16).

Figure 2.16: Use of mainstream services by type

Base: 189-193

2.6 Life experiences

The survey explored the degree to which respondents had other life experiences which might have contributed to, or happened as a result of, their status as people with complex needs who were sleeping rough (Figure 2.17). A high percentage (71%) of respondents indicated they had spent time in prison at some point in their lives and 43% had spent time in care as a child or young person. Thirty-eight per cent reported that they had been permanently excluded from school, while 7% had been in the armed forces.

Figure 2.17: Life Experiences

Base: 190-197

Finally, this data provides evidence on the degree to which the respondents to this survey were experiencing needs across a range of aspects of their lives. Eighty-one per cent of respondents indicated that they had needs across all three areas examined in the survey (housing, health and life experiences).

This chapter has presented data from baseline surveys to understand the experiences of people with complex needs who were rough sleeping prior to their engagement with the projects. It has presented a picture which characterises their experiences of longstanding housing vulnerability involving stays in different types of accommodation, punctuated by periods spent sleeping on the streets or sofa surfing. Service users typically have multiple, longstanding health and well-being needs and are frequent users of mainstream services, including non-planned and emergency care services. 

The next chapter looks at data from the case study projects to assess the progression and outcomes achieved by services users when they engaged with the case study projects.

3. Progress of service users and the differences between the RSG and SIB case study projects

Summary

This chapter looks at progress and outcomes for service users in the case study projects, and the differences between the RSG and SIB projects. Analysis used outcome logs for service users, and baseline and follow-up service user questionnaires.  Findings are limited by low numbers of follow-up data, although a number of statistically significant findings were identified.

There was significant outcome change for individuals:

• Increases in access to homeless or temporary accommodation.
• A reduction in cocaine use.
• Reductions in mainstream service use (hospital appointments for physical health problems, A&E and Ambulance services).

There were no significant improvements in education or employment outcomes or in the mental health and wellbeing of service users over the period of data collection.

There were significant differences between RSG and SIB projects since service users stated engaging with their projects:

• A higher proportion of service users in the RSG case study projects had stayed in long-term accommodation compared to those in the SIB case studies. This reflects that three of the RSG case study projects were delivering housing first style initiatives which prioritises access to long-term accommodation.
• SIB service users were more likely to have stayed in homelessness or temporary accommodation or experienced sleeping rough than RSG service users. This reflects the difficulties experienced across all the case study projects in accessing appropriate long-term accommodation.

3.1 Introduction

This chapter looks at progress made by service users – the progression made, therefore, by a cohort of people with multiple complex needs with experience of sleeping rough^{[footnote 16]} and the differences between the RSG and SIB case study projects.

It presents two sets of analyses:

• The progress of service users: statistically significant changes for individuals between baseline and follow-up stages (regardless of service delivery). This tells us what outcomes service users across projects have experienced over time.
• The differences between delivery models: statistically significant differences between RSG and SIBs in achieving certain outcomes for their service users. This is a useful assessment of the focus and successes of different delivery models.

Analysis

Two different data sources have been used for this analysis: outcome logs for service users, and the baseline and follow-up service user questionnaires. Both of these data sources were collected by the service providers. In order to maximise the potential to use this data to present a comprehensive picture of change for the service users the data sources have been used both individually and in combination for different outcome and progression indicators. For each section of analysis, the data sources which have been used are detailed in the text.

The analysis also uses data which presents change over different time periods: some of the analysis focuses on change between the baseline and follow-up service user surveys^{[footnote 17]}; at other points change between the baseline and six-month outcome logs or change between the baseline and twelve-month outcome logs is examined.^{[footnote 18]} Tables 3.1 and 3.2 below provide a breakdown of the questionnaires and outcome log responses received by funding stream.

Table 3.1: Questionnaires by funding stream

Funding Stream	First questionnaire	Follow-up
RSG	52	13
SIB	145	53
Total	197	66

Table 3.2: Outcome log responses by stage and funding stream

Funding stream	Baseline	6-months	12-month
RSG	42	21	20
SIB	85	60	61
Total	127	81	81

In addition, some of the analysis presented focuses just on clients who had been participating in SIB projects for a minimum of nine months and for whom follow-up outcome log data was available for at least one quarter following nine months of engagement. Where follow-up outcome data was collected at more than one quarter, the data collected from the quarter closest to the nine-month stage has been used. Nine months was used, rather than six or twelve months in order to maximise the responses available. There were 75 SIB service users in total for whom nine-month data was available. 

Significance testing

Differences in outcomes between RSG and SIB service users have been tested for statistical significance^{[footnote 19]} and where significant differences have been identified these are presented. Statistical testing is important because it is only in instances where the difference is statistically significant that there is sufficient evidence to indicate that the observed difference has not occurred due to chance. Non-statistically significant differences have not been reported.

Changes over time (e.g. change in accommodation status between baseline and follow-up) have also been tested for statistical significance.^{[footnote 20]} Only where significant differences have been identified have these been reported. Analysis has focused on within-person change (i.e. examining changes in service user responses for those with both baseline and follow-up responses) as the most robust approach to identifying change for individuals. Cross-sectional change (i.e. comparing all baseline responses to all follow-up responses) was also examined and the cross-sectional analysis can be found in Appendix One.

In line with good practice, only results for questions receiving at least 30 responses (e.g. 30 at baseline and 30 at follow-up) and with at least 10 responses for each response option have been presented. In addition, only where a change in the number of responses within a category represents more than five people, have results been reported. 

It should also be noted that any statistically significant changes identified in this report cannot necessarily be attributed to the projects. The evaluation was not set up to establish a comparator or counterfactual (to identify what might have happened in the absence of the case study projects) and in the absence of a rigorous impact evaluation it has not been possible to attribute change to the projects with any degree of certainty. The lives of people with complex needs are likely to be highly fluid (for example, the data presented in Chapter Two demonstrates that housing situations may change for a range of reasons) and as such we cannot be sure that any progress made is directly, or solely, as a result of the support that they have received. However, the data in Chapter Two and qualitative evidence outlined in other reports from this evaluation also demonstrate that many people in this group have long experience of needs which do not appear to have been met successfully by mainstream services. A recurring theme of the process evaluation has been the view of service users and providers that the emphasis in interventions on providing holistic, person-centred support has differentiated these projects from other services. It is likely therefore that the projects have made a substantial positive contribution to any progress or change identified. 

Limitations of the data

Due to the limitations in the follow up data collected during the period of this evaluation (both in terms of sample size and data quality), it has not been possible to fully assess or substantiate the progress reported in interviews with service providers and service users.

The analysis presented in this chapter focuses on within-person change (i.e. examining changes in service user responses for those with both baseline and follow-up responses). The initial analytical approach taken was to examine cross-sectional change (i.e. comparing all baseline responses to all follow-up responses). This would have allowed a greater number of responses from individuals to be included in the analysis, however, given the limitations of the follow-up data, it was not possible to be confident that any changes identified were not due to bias in the sample.  

A key lesson from this evaluation is, therefore, that data collection needs to be embedded within future projects, and the importance of data collection emphasised to project staff, to enable the collection of robust enough data able to demonstrate progress and outcomes over time.

3.2 Progress of service users

The analysis presented in this section examines the change in individual circumstances and behaviour of service users between baseline and follow-up stages. Analysis has focused on within-person change (i.e. examining changes in service user responses for those with both baseline and follow-up responses) and only statistically significant changes are reported. This analysis tells us what outcomes service users across projects have experienced over time.

Housing and homelessness

The individual accommodation status of all clients (both RSG and SIB combined) was recorded via the outcome logs at baseline and both six and 12 months after initial engagement with the programme.

Two positive statistically significant differences have been identified between the baseline^{[footnote 21]} and follow-up outcome logs in terms of proportions accessing more secure forms of accommodation at different stages.

• At six months: the proportion of service users in homeless or temporary accommodation increased from 28% to 49%.
• At nine months: the proportion of SIB service users in homeless or temporary accommodation increased from 42% to 67%.

Substance misuse

The questionnaires also sought to understand the frequency and scale of substance misuse needs in the service users, and the numbers who had accessed treatment support services. Male respondents to both service user questionnaires were asked to indicate how frequently they had consumed eight or more units of alcohol on a single occasion in the three months prior to being surveyed, and female respondents were asked to indicate how frequently they had consumed six or more units. Respondents were also asked if they had used various drugs in the three months prior to completing the survey.

Just one positive statistically significant difference was identified between the first and follow-up questionnaires in terms of reductions in substance usage (those who did not know, could not remember or did not want to say have been excluded from the analysis):

• The proportion of service users who had used cocaine at least monthly in the last three months decreased from 61% to 39%.

Whether SIB service users were receiving treatment at baseline and then following nine months since first engagement with the service was also examined. Figure 3.4 below shows that the proportion of SIB service users receiving treatment increased from 17% to 36% and this increase was identified as statistically significant.

Figure 3.4: Whether SIB service users were receiving substance misuse treatment at baseline and nine months or above

Base: 70

Education and employment

Whether service users were in employment, education or training was recorded via the baseline and follow-up outcome logs. The majority of service users (both RSG and SIB combined) at both baseline and at six and 12 months, where their status was known, were unemployed. Consequently, there were no statistically significant improvements in economic status that could be identified.

Mainstream service use

Finally, as outlined in Chapter Two, service users were frequently accessing mainstream services prior to their engagement with the case study projects. The service user questionnaires therefore asked respondents if they had engaged with different types of health services in the three months prior to being surveyed. Three significant decreases in service usage over time were identified (Figure 3.5):

• The proportion of service users who had attended a hospital appointment for a physical health problem decreased from 39% to 23%.
• The proportion of service users using A&E decreased from 38% to 18%.
• The proportion using ambulance services decreased from 35% to 18%.

Figure 3.5: Whether service users had used mainstream services in the last three months

Base: 62 (Hospital appointment for a physical health problem); 61 (A&E); 60 (Ambulance)

3.3 Differences between service delivery models

The analysis presented in this section examines the differences between RSG and SIB projects in achieving certain outcomes for their service users. Differences in the change in individual circumstances and behaviour of service users recorded between baseline and follow-up stages has been examined by the two different delivery models. Only statistically significant differences are reported, providing a useful assessment of the focus and successes of the different delivery models.

Housing and homelessness

Figure 3.1 below shows – by funding stream – the types of accommodation entered into by service users since they first engaged with the service. This analysis draws on both outcome log and questionnaire data.^{[footnote 22]} Seventy-eight per cent of RSG service users and 38% of those supported by SIBs had moved into long-term accommodation. This is a reflection of the fact that three of the RSG case study projects were delivering housing first style initiatives^{[footnote 23]} which prioritised access to long-term accommodation. SIB service users were significantly more likely to have moved into homelessness or temporary accommodation or experienced sleeping rough since their engagement with the project. This reflects the difficulties experienced across all the case study projects in accessing appropriate long-term accommodation.

Figure 3.1: Accommodation experienced since start of intervention

Base: 49 (RSG); 145 (SIB)

Health and wellbeing

Figure 3.2 below shows the proportions of all service users who had received mental health support since they first engaged with the service by funding stream. This analysis draws on both outcome log and questionnaire data.^{[footnote 24]} As outlined in Chapter Two (Figure 2.11), 85% of service users self-reported in the first questionnaire that they had a mental health issue. However, there were no statistically significant improvements in mental health and wellbeing that could be identified and only 47% of RSG service users and 25% of service users in SIBs that had reported a mental health issue had accessed mental health support since the start of the intervention.^{[footnote 25]}

Figure 3.2: Received mental health support since start of intervention

Base: 43 (RSG); 108 (SIB)

One of the RSG projects was an outreach service specifically designed to increase access to services for people with co-occurring of mental ill-heath and drug or alcohol addiction, but across all the projects access to mental health services was problematic, and stakeholders in delivery organisations reported frequently that there were high levels of demand for overstretched services. This suggests that there remain high levels of unmet need in relation to the mental health and wellbeing of this group of service users.

3.4 Summary

This chapter has looked at progression and outcomes for service users in the RSG and SIB case study projects. There were significant differences between RSG and SIB projects in the proportions reporting staying in long-term accommodation, reflecting the housing first style delivery model of three RSG case studies.

It has also indicated that there have been improvements in accommodation outcomes across all projects and reductions in the use of cocaine for some service users. There have also been improvements in access to drug and alcohol treatment services for some service users in SIB projects and reductions in the use of some mainstream services. However, it is important to note that the analysis has been restricted by the low sample sizes for the follow up questionnaire and outcome logs, meaning that it has not been possible, for example, to carry out analysis which looks at relationships between different types of intervention and service user outcomes. A key lesson to emerge from this evaluation is the importance of good quality data through which to identify outcome change for service users and triangulate the findings from the qualitative data. Further data collection, providing a larger sample of service user outcomes would be beneficial going forward.  The next chapter presents data on the costs of delivering services to meet the needs of this group.

4. Analysis of income, costs and staffing

Summary

This chapter analyses income, costs and staffing in the case study projects.

The average cost per beneficiary into long-term accommodation was calculated to be £15,065 after 12 months.  This was lower than the benchmarks which these figures were set against, although delivery costs need to be put into context of the outcomes achieved.  Accepting limitations in these comparisons (for example regarding the nature of the beneficiary population, the outcomes considered and potentially the quality of outcomes achieved) it appears that overall the projects have provided good value for money.

The average cost per beneficiary, composition of expenditure, level of staffing and composition of staffing all varied by the nature of the intervention and the funding mechanism applied in the projects. On average the RSG housing-led projects (housing first style initiatives) had the highest average cost per beneficiary and level of staffing per month per 100 beneficiaries. This is to be expected given they delivered an intensive intervention to relatively few beneficiaries. On average non-operational costs comprised a greater proportion of the SIB project expenditure compared to the RSG case study projects. This relates to the increased monitoring and management costs required to oversee the SIB.

The evidence suggests that the projects were staffed with people at appropriate grades for the respective task or activity.

4.1 Introduction

This chapter provides evidence on income, costs and staffing in the case study projects.

The evaluation provides evidence on the design, set-up and delivery of the interventions funded through the Rough Sleeping Grant (RSG) and Social Impact Bonds (SIBs). Within this broad objective is a requirement to assess the costs and resourcing of the projects, to:

• Quantify the average ‘unit’ cost of providing each project to a beneficiary
• Explore how these are affected by factors such as the project type and funding mechanism used
• Inform the future funding of interventions to support people with multiple, complex needs who are sleeping rough.

The 12 case study areas participating in the evaluation were asked to complete a ‘Project Costs Tool’ for their intervention(s) which were delivered through either the RSG or SIB. The Costs Tool built up a detailed and robust evidence base on the costs and staffing of the projects. The information gathered covered actual financial (excluding VAT)^{[footnote 26]} and staffing information for each of the last two full financial years: 1st April 2017 to 31st March 2019. The projects were also asked to provide the number of beneficiaries that they had worked with over the same two-year period (to 31st March 2019). The projects themselves commenced on a staggered basis so were not up and running for the whole two-year period. Some of the projects were also ongoing post March 2019.

The responses received were completed by representatives in local authorities and provider organisations (it was left up to the case study areas to decide who was best placed to provide information for their project). To support consistency, an online guidance video was provided which gave further instruction on completing the Costing Tool.

Completed returns were received from 10 of the 12 case study areas. These included six SIB funded projects and four RSG projects (two of which were housing first style initiatives). The data provided by the projects is self-reported within the guidelines provided. It has not been possible to verify the accuracy of the responses. 

The remainder of this chapter provides analysis of these responses.

4.2 Project income

This section provides detail on the income of the projects. It is important to note that several of the SIB projects received the loan for their project up-front, even though the funds are used to cover expenses over subsequent years.

Table 4.1 shows that the total income received by the projects in the two financial years 2017/18 and 2018/19 was £4,236,000. Of this amount:

• 72% (£3,046,000) was funding from MHCLG
• Loan payments made to run three of the SIB projects accounted for 15% (£630,000); these were paid to support delivery costs and will be paid back over a staggered timeframe, when outcomes have been achieved
• 9% (£368,000) comprised outcome payments made to SIB projects for payment by results; this will increase as further outcome payments are made against those delivered  
• 4% (£184,00) was from financial commitments made by the projects themselves; half (five) of the projects recorded an own commitment in either of the two financial years
• A residual income of £8,000 came from other sources.

Table 4.1: Sources of income

Source	Income (£)	Percentage of total (%)
MHCLG	£3,045,535	72
SIB loan payment	£630,000	15
SIB outcome payments	£368,160	9
Own commitments	£184,092	4
Other	£7,914	0
Total income	£4,235,701	100

Table 4.2 demonstrates the breakdown of sources of income by different project types.

Table 4.2: Sources of income by project type

Project type	SIB	RSG housing led	RSG non-housing led
Source	Income (£)	Percentage of total (%)	Income (£)	Percentage of total (%)	Income (£)	Percentage of total (%)
MHCLG	£2,060,430	64	£580,905	97	£404,200	100
SIB loan payment	£630,000	19	-	-	-	-
SIB outcome payments	£368,160	11	-	-	-	-
Own commitments	£165,801	5	£18,291	3	-	-
Other	£7,914	0	-	-	-	-
Total income	£3,232,305	100	£599,196	100	£404,200	100

This means that for every £1 of MHCLG funding an additional 39 pence had been levered in from other sources. This latter amount included 21 pence from loans, 12 pence from outcome payments and six pence from own commitments.

The SIB projects collectively levered the highest amount of additional funding over the two-year period: 57 pence for every £1 of MHCLG grant funding. This compares to just three pence for the RSG housing led projects and nothing levered in for the RSG non-housing led projects.

Table 4.3: Income per beneficiary

Project type	Income per beneficiary (£)
RSG housing led	£11,343
SIB	£4,493
RSG non-housing led	£2,247

Across the 10 projects the average income per beneficiary was £5,414 (Table 4.3). Though, this value varied greatly by project from £12,600 to £600 per beneficiary.  Analysis by project type reveals the two RSG housing led projects had the highest income level per beneficiary (an average of £11,343 per beneficiary); whereas the two RSG non-housing led projects had the lowest level of income per beneficiary, an average of £2,247 per beneficiary. On average the six SIB projects had an income of £4,493 per beneficiary. This pattern reflects the respective resource demands for the three different project types, a point which is developed further below.   

4.3 The overall costs of providing the projects

This section considers the overall costs for the 10 projects that completed the Costs Tool in the two years covered by the data collection. These are presented as a total and per beneficiary amount. The latter is important given the variation in the numbers of beneficiaries that each project has worked with.

The total cost of the 10 projects was £3,863,000 over the period 1st April 2017 to 31st March 2019. At an individual project level, costs ranged from £74,000 to £737,000 over the two years, reflecting the varied nature of the interventions, the number of beneficiaries supported and local costs. Costs were higher in the second financial year, with £1,228,000 costs in the financial year 2017/18 and £2,601,000 in the year 2018/19. This trend, for higher second year costs, was repeated across all 10 projects. In the main this reflected project start dates, with only two of the 10 projects starting in the first half of the 2017/18 financial year.

Estimating the average cost of working with a beneficiary

The average project cost of working with a beneficiary was £5,041. This is the average unit cost and is calculated by dividing the total cost reported by the projects over the two years by the number of beneficiaries that they had supported. At a project level unit cost varied from a high of £12,641 per beneficiary to a low of £645 per beneficiary. The RSG housing led projects had the highest average project cost per beneficiary: £11,335 per beneficiary (Table 4.3). This reflects the level of input required to deliver housing led projects which by their nature involve working intensively with a smaller number of beneficiaries. The average cost per beneficiary was far lower across SIB and RSG non-housing led projects: £3,890 and £2,198 per beneficiary respectively. These projects worked with greater numbers of beneficiaries providing less intense intervention over a shorter timeframe. The lowest costs per beneficiary were in the projects focused on outreach work. When interpreting these differences, it is important to avoid conclusions about one approach being better than another purely on the basis that its average cost per beneficiary is lower. There is also a need to consider the scale and average cost of outcomes that are achieved. Also, there are likely to be differences in the groups that different projects work with and the nature of the outcomes achieved.

Table 4.3: Average total cost per beneficiary

Project type	Cost per beneficiary (£)
RSG housing led	£11,335
SIB	£3,890
RSG non-housing led	£2,198

4.4 Estimating cost effectiveness and comparison to the costs of rough sleeping

It was also possible to estimate an average project cost per additional beneficiary into long-term accommodation (in evaluation terms this is the average cost effectiveness of the programme). The average cost was calculated to be £15,065 per beneficiary into long-term accommodation after 12 months. This has been achieved by dividing the total cost by the estimated additional number into long-term accommodation after 12 months. Where the additional number into long-term accommodation after 12 months has been calculated by applying the percentage change identified in Section 3.2 to the number of beneficiaries who had been supported by the 10 projects that provided costing information. Note due to low base levels to the survey it has not been possible to compare the cost effectiveness of the different types of project.

The following provide two benchmark comparisons to other interventions and general costs of rough sleeping, where lower average costs are preferred:

• The Ministry for Housing, Communities and Local Government’s Rough Sleeping Strategy (2018)^{[footnote 27]} estimates the costs of rough sleeping to the public purse to be between £14,300 and £21,200 per person, with higher costs being if rough sleeping occurs alongside substance misuse and offending. Excluding benefits, the range is £7,100 to £15,200 per person per year. Note these figures are not comparable costs of interventions to address rough sleeping, rather they are examples of the ‘do nothing costs’ to the public purse of rough sleepers.
• Crisis, as part of their 2018 report ‘Everybody in: How to end homelessness in Great Britain’^{[footnote 28]} suggest the (weighted) average cost per person supported by the recommended mix of solutions across their five ‘End Homelessness’ objectives between 2018 and 2041 is £34,460. This ranges from £53,900 per person for a mix of solutions recommended to achieve Objective 3 (no one living in emergency accommodation) – to £6,282 per person supported for a mix of solutions recommended to achieve Objective 5 (everyone at immediate risk of homelessness gets the help that prevents it from happening).

It is important to note that this second benchmark relates to cost per respective outcome achieved rather than the cost of working with a beneficiary. The cost per beneficiary will be higher since it is highly unlikely that all participants will achieve an outcome. Accepting limitations in these comparisons (for example regarding the nature of the beneficiary population, the outcomes considered and potentially the quality of outcomes achieved) it appears that overall the projects have provided good value for money. 

The next section considers in more detail the costs of delivering the complex need projects and why there are differences in costs across the projects.

4.5 Composition of project costs

This section considers the composition of project costs. It starts by providing a breakdown of overall costs, and then explores differences by funding source and primary delivery method.

The Costs Tool asked projects to break their costs down according to the nature of their expenditure. A list of more detailed categories under nine broad expenditure types was provided to assist projects in completing this information. The analysis presented here focuses on the broad categories of expenditure due to the wide variation in the more detailed categories at an individual project level.

Just over two thirds of expenditure over the period was accounted for by staffing costs (£2,628,000; 68% of total expenditure). This compares to non-staffing costs of £1,235,000; 32% of total expenditure.

On average non-staff costs accounted for 38% of overall costs for the SIB projects (£1,475 per beneficiary). This proportion was higher than the respective proportion for RSG non-housing led (23% or £418 per beneficiary) and RSG housing led projects (17% or £1,821 per beneficiary). Higher non-staff setup, and operative costs in the SIB projects are the main contributing factors to this difference.   

As is expected operative costs^{[footnote 29]} account for the vast majority of costs: £2,831,141 or 73% of overall costs. The average operative cost across the 10 projects was £3,799 per beneficiary. This ranged from £9,664 per beneficiary to £638 per beneficiary at a project level. The two RSG housing led projects had the highest average value operational cost per beneficiary: £8,945 (Table 4.4). This was over three times the average operational cost across the six SIB projects (£2,710 per beneficiary) and the two RSG non-housing led projects (£1,916 per beneficiary).

Proportionally, operative costs accounted for different levels of overall costs across the 10 projects (Table 4.5). The percentage was highest for the two RSG non-housing led projects (an average of 87% across the two projects), reflecting the nature of the interventions that are being delivered in these projects. In comparison, operational costs accounted for 70% of costs on average across the six SIB projects. This lower level is attributable to costs for the set-up and monitoring of the SIB. 

Breaking operative costs down: operative staffing costs comprised the largest component of expenditure, accounting for 57% of expenditure (£2,212,000) in the two financial years 2017/18 and 2018/19. In addition, a further 16% (£619,000) was accounted for by non-staffing operative costs.

Monitoring costs accounted for the next largest component of expenditure in the two financial years (12%). Nine per cent (£332,000) of total expenditure was staffing monitoring costs (to monitor the operation and success of the projects) and 3% (£128,000) was non-staffing monitoring costs. On average monitoring costs accounted for 14% of total costs across the SIB projects; this is equivalent to £532 per beneficiary. This proportion is at least double that for the RSG non-housing led projects (on average 7% of total costs or £259 per beneficiary) and RSG housing led projects (on average 5% of total costs or £688 per beneficiary).

Of the remainder:

• 4% (£172,000) was set-up costs, including £158,000 in non-staffing set-up costs
• £12,000 (less than 1%) was capital costs
• 10% were other costs, including repayments on loans. 

Table 4.4: Average expenditure by type of activity (per beneficiary)

Type of activity	SIB (£)	RSG housing led (£)	RSG non-housing led (£)
Setup	£222	£403	£22
Monitoring	£532	£688	£259
Operational	£2,710	£8,945	£1,916
Capital	£7	£164	£0
Other	£418	£1,135	£0
Total	£3,890	£11,335	£2,198

Table 4.5: Composition of expenditure by type of activity

Type of activity	SIB (%)	RSG housing led (%)	RSG non-housing led (%)
Setup	6	4	1
Monitoring	14	6	12
Operational	70	79	87
Capital	0	1	0
Other	11	10	0
Total	100	100	100

4.6 Project staffing and staff costs

The Costs Tool also asked projects to provide their average full-time equivalent (FTE) staffing inputs in each year (2017/18 and 2018/19), broken down by type of activity (set-up, operative, monitoring or other activity) and role (managerial/leader, supervisor/middle manager, frontline delivery staff, administration/support/other staff and temporary/agency staff). This section provides analysis of the responses received.

The 10 projects collectively drew on an average of 44.1 full-time equivalents (FTEs) of staff time per month. This included an average of 35.1 FTEs per month in 2017/18 and 53.1 FTEs per month in 2018/19. The latter is higher because the projects were fully operational in this year.

Across the 10 projects, on average there was 6.0 FTEs staff per month per 100 beneficiaries. This number was highest for two RSG housing led projects who on average had 14.6 FTE staff per month per 100 beneficiaries. Staff numbers were less than a third of this number in the SIB and RSG non-housing led projects: on average 4.1 FTEs and 3.0 FTEs per month per 100 beneficiaries respectively. This difference results from the more staff intensive intervention being delivered through the RSG housing led projects.

Analysis of FTE staff by their role reveals frontline delivery staff accounted for 70% of staff time. While managerial and leader staff accounted for 15% of staff time and 10% was accounted for by supervisors and middle managers. The remaining 5% was accounted for by administration, support or other staff.

RSG non-housing projects had the lowest proportions of managerial/leader staff (9%) and supervisory/middle manager staff (0%) and the highest proportions of frontline delivery staff (77%) and administration, support or other staff (14%). This can be explained by the nature of the projects and their funding mechanism requiring less management and oversight. 

The reverse staffing structure is true across the six SIB projects, which had:

• the highest proportions of managerial/leader staff (18%) and supervisory/middle manager staff (11%)
• the lowest proportions of frontline delivery staff (67%) and administration, support or other staff (3%).

The average costs per FTE staff member by grade are broadly in line with the local government pay scale.^{[footnote 30]} These are calculated to be as follows:

• managerial and leader £40,000 per annum
• supervisors and middle managers £34,000 per annum
• frontline delivery staff £27,000 per annum
• administration, support or other staff £29,000 per annum

In terms of the staff breakdown by activity, on average across the projects 88% was viewed to be operational, 11% was monitoring, 1% as other staffing costs (for example on costs and other costs of employment) and less than 1% was set up staffing costs.  

• 100% of FTE staff in the RSG non-housing led projects were involved in operative roles
• Staff in the SIB projects were split between operative (on average 85% of FTEs) and monitoring functions (on average 14% of FTEs)
• 94% of FTE staff were in operative roles, 4% were in monitoring roles, 1% were involved in set up and 1% were involved in other roles.

5. Conclusion

Utilising data collected by the case study projects, this report has presented data on the levels of need among the service users, the progress of service users and the differences between the RSG and SIB projects. It also reported on the income, costs and staffing of the RSG and SIB case study projects.

A number of projects faced challenges in collecting data through which to identify outcomes for service users. Whilst the analysis outlined in this report presents a valuable step forward in building the evidence base on the provision of support for people with complex needs who are experiencing rough sleeping, one immediate conclusion is that there is a need for further systematic data collection going forward. This would support robust impact evaluation of interventions which was not possible here due to limitations in the follow up data collected during the period of this evaluation.

It is clear from the analysis that the needs of people who are rough sleeping should not be seen as separate to those of other groups who experience homelessness. Many of the people being supported by the case study projects had been in and out of many different forms of accommodation and had experienced periods in temporary accommodation, rough sleeping and sofa surfing.

The case study projects had made progress in delivering improved access to permanent accommodation and reductions in rough sleeping, particularly in the RSG housing first style projects, suggesting that early access to suitable permanent accommodation is an important prerequisite for outcomes in other areas.

However, it is also clear that the high levels of complex health and wellbeing needs amongst this group, and the fact that many have experience of the criminal justice and care systems, require specific service approaches. The SIB projects had been able to facilitate service users’ access to drug and alcohol treatment services. This was not the case for mental health and wellbeing outcomes however, where no significant improvements were found in either RSG or SIB projects on the basis of the data available, and lack of timely access to mental health services was reported in interviews with service providers and service users as a major challenge in meeting the needs of those who are experiencing multiple vulnerabilities.

There were, perhaps unsurprisingly, no significant changes in the education or employment status of service users. In qualitative work conducted for this evaluation, service providers were strongly of the view that the majority of service users were so far from the labour market and had such complex health and care needs that any changes in these outcomes were not achievable in the short-term. As such they prioritised attention to other outcomes such as access to health and support services. However, almost all service users had worked at some point, and there may be scope for these outcomes to improve in the longer-term for the SIB projects and for further testing of different approaches to supporting service users to achieve outcomes in this area.

The delivery models and implementation of the RSG and SIB case study projects are reported on elsewhere^{[footnote 31]} but common features included person-centred and tailored support, focusing on targeted outreach, improved and sustained access to services (including housing), reductions in service duplication and improved cross-referrals through collaboration. The analysis presented here suggests that these are effective approaches, however, as explained in the other reports their success depends crucially on capacity within mainstream services to accommodate the needs of this group.

Appendix 1: Technical note

The research objectives

MHCLG developed the Complex Needs evaluation as a multi-method research project to answer three key objectives (see section 1.2).  

Quantitative data was collected from services and directly from service users to help to understand a cohort of people who sleep rough with multiple complex needs, to understand their different support needs and experiences. Improved information and data on service users and their different support needs is particularly important in relation to understanding what is required from an effective workforce, the associated challenges and the outcomes that can be achieved.

The secondary aim of the quantitative data collection was to inform an impact evaluation. The initial planned design had been to compare the Rough Sleeping Grant users against the Social Impact Bond users; or to compare those receiving housing first style services or housing led services to those receiving other types of services focused on mental health and substance misuse. The researchers at Sheffield Hallam assessed the feasibility of conducting an impact evaluation with the quantitative data collected and concluded that this would not be feasible with the small sample sizes.

An additional aim of the data collection was to collect data to further improve MHCLG’s understanding of how much rough sleeping and homelessness costs to the public purse, taking into account public service use.  The data on the 197 participants in this research will be used as part of a larger-scale project run by MHCLG.

The design of the research

Identification of the areas

The 13 projects involved in the evaluation were identified as working with cohorts with multiple complex needs with experience of sleeping rough, although the projects adopted different approaches. 

Taking part in the evaluation was not a condition of the funding and the case study projects were identified by MHCLG as areas with capacity to engage with the evaluation. 12 of the 13 projects completed the questionnaire collection; and11 provided additional follow up/outcome data. Services received a small grant to reflect the time commitment of this work.

This evaluation is not suggesting the projects are representative of all local authorities in England. While it does include a mix of urban, rural and coastal areas, it samples only areas receiving MHCLG funding via the Rough Sleeping Grant or the Rough Sleeping Social Impact Bond, working with a cohort with complex needs.

Research tools

The three objectives listed above shaped the design of the research materials used in the project: two questionnaires and an outcome log.

The baseline questionnaire

The questionnaire has been developed by the Ministry of Housing Communities and Local Government to improve the evidence base on people who currently or previously have slept rough. It received ethical approval from Southampton University Ethics Board. The questionnaire was designed to be completed by service users as soon as possible from when they first engaged with the funded SIB or RSG project. However, there is considerable variation across the different projects reflecting the range of start dates of the projects across the different funded programmes. See Table 1.1 for start dates.

The questionnaire covered the following topics:

• Demographic information
• History of housing and homelessness, including recent experiences of rough sleeping, temporary accommodation, sofa surfing and supported housing
• Health and wellbeing (including both physical and mental health questions) and health service use
• Substance misuse (drugs and alcohol)
• Criminal justice experience
• Time in care and schooling
• Employment and income, including time in the armed forces
• Public and third sector services they may have used (including Housing Options and local authority services)

It had been designed with input from external survey experts, psychologists, analytical and policy colleagues within MHCLG and other government departments. Invaluable input from people with lived experience of rough sleeping, and from frontline staff improved the questionnaire as well.  Whilst it includes, wherever possible, standardised questions to make it comparable with other data, the priority was on making this an accessible questionnaire for service users and where necessary this was prioritised above using standardised questions.

How was this rolled out?

Frontline staff in the services engaged their clients and talked them through the research materials, explaining the objectives of the research project. All frontline staff were given guidance on the research and how to explain this to clients. In the RSG areas all service users would be eligible to take part in the research, and in the SIB areas the staff aimed to complete the research with new clients starting in their intervention. This different approach was a reflection of the size of the RSG and SIB projects and a recognition that areas were more likely to complete the questionnaires if the number had been agreed with them as being manageable. As some services started earlier than others and due to staff capacity constraints the fieldwork period for this baseline questionnaire was extended to eight months. This means that there were, in some instances, lags of some months between clients starting on the project and completing the baseline questionnaire. The average time between starting on the project and baseline was five months, but this ranges from zero to fourteen months. 

The SIB programmes collected the wellbeing measure, Warwick Edinburgh Mental Wellbeing Score (WEMWEBs), as part of their outcomes. To avoid over-burdening respondents, where participants had completed a wellbeing measure recently, the SIB programmes agreed to send this information to MHCLG separately. This was then incorporated into the analysis.

Figure A1 provides an overview of when baseline questionnaires were completed.

Figure A1: Completion of baseline questionnaires by month

Base: 197

The follow up questionnaire

Similarly developed by MHCLG, this was produced as a follow-up to the original baseline questionnaire and designed to feed into a potential impact evaluation. Whilst the impact evaluation was ultimately ruled out due to low sample retention, this information has informed the quantitative report and information about progress and clients’ change.

The follow up questionnaire covered the following topics:

• Wellbeing
• Physical and mental health and health service use
• Substance misuse (drugs and alcohol)
• Recent homeless and housing situation

It was designed to be significantly shorter than the baseline questionnaire and followed up information that might not otherwise be possible to collect via the outcome logs (see section below). 

How was this rolled out?

The follow up questionnaire was originally designed to be completed twice, once 6 months after the baseline questionnaire and another time at 12 months after the questionnaire. Capacity issues within the participating homelessness services and the extension offered to the baseline questionnaire meant that the follow up questionnaire was completed only once by respondents to reduce burden on services. Service staff were asked to help participants complete one follow up questionnaire at least six months after the baseline questionnaire was completed. 

Some 66 follow up questionnaires were completed, representing a retention rate of 34%.  On average a mean of eight months and a median of nine months passed between participants completing the baseline and follow up questionnaire and on average a mean of 13 and median of 14 months had passed since starting on the project.

A number of those who did not complete the follow up questionnaire may not have been in contact with the service anymore. This may be from having been found accommodation and moved on from the service (reported by a few services in the outcome logs), or from having disengaged with the service and moved back onto the streets or other unstable accommodation.

The quarterly outcome log

Consenting respondents had their outcomes across a set of measures shared with MHCLG for up to a year after starting with the RSG or SIB service. The aim of this data collection was to understand service user progress and the change throughout their engagement on the project. MHCLG designed an outcome log for service staff to complete every quarter, providing snapshot high-level information about consenting participants’:

• Accommodation status – where the client was were staying that week
• Mental health support – whether the client had received or was receiving mental health treatment or had had a mental health assessment
• Substance misuse support – whether the client had received or was receiving drug and/or alcohol treatment or had had an assessment
• Economic status – whether the client was in employment, training, education or volunteering

As many services do not hold case level information that would provide retrospective data, the outcome log was designed to collect current information to avoid placing any additional burden on services.

The information collected resembled the rate card outcomes monitored as part of the SIB. 

How was this rolled out?

This relied on services completing the outcome logs during the same week in each quarter for all consenting participants.

Only one of the 12 areas provided at least some outcome log data for all five necessary quarterly follow-up waves (June 2018, September 2018, December 2018, March 2019 and June 2019). Seven areas provided at least some data for three or four waves, three provided at least some data for one or two waves and one area provided no follow-up quarterly data. The attrition rate between the first quarterly wave (June 2018) and the last (June 2019) was 82%.  This can be explained by a number of factors, including staff capacity within the services to accurately monitor data or complete and share these logs, service user disengagement meaning monitoring their situation was no longer possible (from either positive or negative reasons), or in some cases service user death.

As a result, analysis looking at the changes between baseline and follow up periods focuses on the following key timepoints: six months, nine months and one year. The majority of comparisons between baseline and follow up data focus on the change by nine months, this allows for us to include the largest sample size for the comparison of outcomes and baseline data.

---

1. Unit cost and effectiveness figures taken from (formerly) DCLG and Crisis work. ↩

2. The term ‘complex needs’ was used throughout the evaluation and in this report to refer to people who were sleeping rough and also experiencing one or more other needs which included mental and physical ill-health, substance addiction and offending behaviour. In many cases these needs are severe and long-standing. ↩

3. This includes a contribution of £1m from the host authority. ↩

4. Further detail on data collection and methods is contained in the Technical Note at Appendix Two. ↩

5. See Rough sleeping snapshot in England, Autumn 2019, Ministry of Housing, Communities and Local Government, 27/02/2020. ↩

6. The approaches taken by the projects are discussed further in case study learning reports. ↩

7. It should be noted that service users did not always complete the questionnaire immediately on joining the project. The average time between engagement and completion of the baseline questionnaire was five months. Therefore, the relatively high proportion of service users indicating that they had stayed mainly in long-term accommodation in the month before completing the questionnaire includes service users who have been supported by the projects to move into sustainable accommodation (particularly in the housing first style case study sites).  ↩

8. Only 94 out of an expected 164 provided a response to this question including ‘do not know’/’cannot remember’ or ‘do not want to say’. ↩

9. It is noted that there is a risk that respondents may have selected supported housing and hostels interchangeably, where the distinction is unclear in the service.   ↩

10. Fitzpatrick, S. et al (2019) The Homeless Monitor: England 2019. London: Crisis, pp 67. ↩

11. Physical health needs were commonly identified as an issue by service users and providers and, as discussed in the SIB case study report, providers were strongly of the view that the physical health needs of service users needed to be addressed before or alongside other issues but that this was not reflected in outcomes on the Rough Sleeping SIB rate card. ↩

12. The service user questionnaire had been amended to address difficulties with recall ahead of further roll-out. ↩

13. Short Warwick Edinburgh Mental Well-Being Scale (SWEMWBS) © NHS Health Scotland, University of Warwick and University of Edinburgh, 2008, all rights reserved.  ↩

14. SWEMWBS figures for England from 2015-2016: ONS (2018) Measuring National Wellbeing, September 2018 release.  ↩

15. These figures are likely to reflect efforts on the part of the projects to prioritise registering service users with a GP so that they could address existing health needs. ↩

16. There was an intention to examine the links between background factors reported via the baseline questionnaire and subsequent outcomes achieved, however, the number of responses proved insufficient to undertake this. ↩

17. The intervals between baseline and follow-up surveys varied by individual. Follow-up surveys were completed on average eight months after the baseline survey and 14 months after first engagement with the service. ↩

18. Outcome log data was collected on a quarterly basis. The six- and 12-month time points post first engagement were calculated for each individual and if an outcome log was completed around these stages it was assigned to either the six- or 12-month point. Three-month windows were used. For example, if an individual’s six-month point fell between May ’18 and July ’18 then their June outcome log was the response used for six-month analysis if available.   ↩

19. The statistical significance of the difference in outcomes between RSG and SIB users was estimated using the z-test for proportions. 95% confidence intervals were applied.   ↩

20. The statistical significance of change over time was also estimated using the z-test for proportions. 95% confidence intervals were applied. ↩

21. The outcome log baseline data is different to that from the first questionnaire and there are some differences in responses related to ‘recent experience’. ↩

22. Accommodation indicated either via the baseline questionnaire (if completed more than one month since the date they first engaged with the project – recorded in the quarterly outcomes log); the follow-up questionnaire; or the quarterly outcome logs. ↩

23. See Bellis, A. and Wilson, W. (2018), Housing First: tackling homelessness for those with complex needs. Briefing Paper Number 08368, House of Commons Library. ↩

24. Mental health support indicated either in the first questionnaire (if consent date greater than three months since start date); in the follow-up questionnaire; or in the outcome logs. ↩

25. The difference in service access is statistically significant and reflects the specific remit of at least one of the RSG case study projects to improve access to mental health services. ↩

26. As per Treasury Green Book Guidance. ↩

27. Ministry for Housing Communities and Local Government (2018) Rough Sleeping Strategy. ↩

28. Everybody In: How to end homelessness in Great Britain. ↩

29. Operative costs are costs associated with the day-to-day delivery of the project and support to beneficiaries. ↩

30. https://www.healthcareers.nhs.uk/working-health/working-public-health/employers-public-health-staff/local-government/pay-rates-local-government. ↩

31. See Rough Sleeping and Complex Needs Process Evaluation: Learning from Six RSG Case Studies; Rough Sleeping and Complex Needs Process Evaluation: Learning from Six SIB Case Studies. ↩