Research and analysis

Evaluation of the Safer Streets Fund round 3, year ending March 2022

Published 10 April 2024

Applies to England and Wales

1. Executive summary

1.1 Safer Streets Fund round 3

The government launched the £25 million funding for the third round of the Safer Streets Fund (SSF3) in March 2021 to support locally based solutions that aimed to prevent or reduce violence against women and girls (VAWG) in public spaces, and improve perceptions and feelings of safety in public spaces for all, with a particular emphasis on improving the safety of public spaces for women and girls. VAWG in this report is used to describe a range of crimes, including rape and other sexual offences, stalking, ‘upskirting’, and many others. VAWG crimes are known to disproportionately affect women and girls, but men and boys can also be victims (more detail is outlined in Section 2.1).

The Fund was open to civil society organisations (charities, community and voluntary organisations, social enterprises and cooperatives), local authorities, Police and Crime Commissioners (PCCs), and the British Transport Police.

SSF3 funded 57 successful projects, delivered between August 2021 and March 2022. Each project delivered a different combination of interventions across 8 categories^{[footnote 1]} and 23 discrete intervention types. Most projects aimed to address multiple types of VAWG, anti-social behaviour (ASB) and acquisitive crime across a range of public spaces including high streets and commercial areas, including the night-time economy (NTE), on public transport or at transport hubs, and parks and open spaces.

1.2 Evaluation overview

The government commissioned Verian (formerly known as Kantar Public) to conduct a process and impact evaluation of SSF3 to understand the specific impact of SSF3 funded interventions and the factors affecting the processes of implementation in funded areas.

The process evaluation explored the setup, delivery and implementation experience of SSF3 bid areas to uncover the key enablers and barriers of delivering interventions, and to identify good practice emerging from SSF3. The impact evaluation aimed to determine the net impact of the funding across the target populations in funded areas through in-person and online surveys. This was supplemented by using the Qualitative Impact Protocol (QuIP) methodology, which used qualitative data to understand the causal factors underpinning changes to perceptions of safety. Evaluation of the Fund began in September 2021, and qualitative and quantitative data collection took place from February to July 2022.

1.3 Crime and safety in context

In SSF3 areas, Verian asked members of the general public (users of public spaces and local residents) and recipients of training delivered through SSF3 interventions (training beneficiaries) about their experience of VAWG, ASB and acquisitive crime (local residents only), and their current perceptions and feelings of safety in public spaces. Overall, 26% of users of public spaces, 14% of local residents and 19% of training beneficiaries had experienced or witnessed VAWG in the previous month. Incidence was statistically significantly higher among women and 16 to 24-year-olds, with verbal harassment being the most common type experienced or witnessed. When asked about perceptions of safety, overall, all audiences felt safer across all types of public spaces during the day compared to at night. Women generally felt less safe than men both during the day and at night, across all types of spaces.

It is within this context that SSF3 interventions were delivered and against these perceptions and experiences that impact is considered as the main focus of the report.

1.4 Findings relating to the impact of SSF3 and future implications

Overall, there was no evidence of statistically significant impacts from the delivery of SSF3 on improving perceptions of safety in public spaces at the Fund level. Similarly, there was no statistically significant evidence of impacts on perceptions of the amount of acquisitive crime or ASB in the local area. However, the funding was associated with emerging positive and statistically significant impacts on:

• perceptions of a reduction in VAWG during the day among local residents
• confidence in local police to deal with incidents of VAWG among residents in areas with guardianship initiatives
• confidence in the local police to deal with incidents of VAWG among local residents and training beneficiaries
• likelihood to report VAWG to police among training beneficiaries

Further analysis explored whether there was a difference in perceived changes in safety in areas with interventions focused on particular public spaces. Specifically, among users of public spaces, perceptions of safety in high streets / shopping areas or on public transport during the day were statistically significantly improved in funded areas, with interventions focused on these space types compared to other funded areas. This suggests that interventions targeted toward specific space types were effective in improving perceptions of safety during the day in those corresponding spaces.

Measures of a positive influence on wider outcomes were particularly noted among direct beneficiaries of the Fund who had completed training. Analysis of outcomes among direct recipients of SSF3-funded training or education initiatives also suggested:

• a positive influence on trainees’ attitudes towards VAWG
• improvements in awareness and understanding of VAWG
• confidence in their ability to identify and respond to incidents of VAWG
• to support victims of VAWG crime

Given the delivery of interventions addressed neighbourhood crime and ASB, wider analysis was also undertaken to measure impact. Analysis of police recorded crime (PRC) data on the estimated impact on overall neighbourhood crime found that SSF3 interventions did not have a statistically significant impact on the number of neighbourhood crimes.

The lack of overall impact detected in improving perceptions of safety could be explained by a variety of factors, including intervention choice and dosage, implementation and evaluation methodology and timeframes, and external influences on perceptions of safety. It is not possible to disentangle the contribution of these possible factors to provide an explanation with certainty. QuIP analysis highlighted some of the challenges to realising this impact within the scope and reach of the SSF3 interventions, and within the delivery and evaluation period, due to the complex factors that influence perceptions of safety. Competing influences that are worsening perceptions of safety include personal experiences of VAWG, awareness of local or national incidents of VAWG, and perceptions that other types of crime are increasing. While QuIP analysis showed that some SSF3 interventions (for example, increased CCTV, improving the look and feel of public spaces, presence of street guardians) were directly linked to improved perceptions of safety for some interviewees at an individual level, these competing factors make changing perceptions of safety at scale particularly difficult; it will require time and sustained effort to shift such perceptions within this complex landscape.

Limitations on the evaluation design reduced the ability to detect impact; for example, it was not possible to conduct a baseline survey prior to the start of intervention delivery. Furthermore, some SSF3 interventions had a short timeframe overall (4 to 5 months), with some (for example, education/training, guardianship initiatives) only running for a limited period or of relatively limited numbers. As such, the time available for impacts to be realised and measured was limited. Awareness and use of SSF3 interventions among users of public spaces and residents is a key factor in this – only around half of users of public spaces (58%) and residents (44%) were aware of at least one intervention that was new or increased in the previous 6 months. Where this was the case, awareness often did not correspond with what was actually being funded in the area – that is, people noticed other interventions not funded by SSF3 or failed to notice interventions that were. Together, this evidence suggests that it will be particularly difficult to change perceptions of safety quickly and at a level to detect significant impact or outweigh the effects of negative factors.

It is possible that the few impacts seen will filter through to improved feelings and perceptions of safety for the general public in the longer term. However, this does not assume that the Fund or its individual interventions will necessarily lead to larger-scale impacts over time. It is not possible to conclude this with certainty and will be a challenge given the wider context of crime and safety. Sustained interventions may also help to achieve longer-term change; however, this is likely to be reliant on areas continuing this work over time. Many SSF3 areas put in place interventions designed to be sustained beyond the lifetime of the funding, such as CCTV and lighting. In addition, some interventions, such as those involving technology, apps and communications, were intended to or could continue with limited ongoing cost, which may support longer-term change.

1.5 Lessons learned from the process evaluation

Learning from the process evaluation provides a range of insights that may be useful for the future design and commissioning of similar programmes.

Focus and parameters of the Fund

Funded areas had a mixed perception of the Fund overall. Some stakeholders welcomed the focus of SSF3 on VAWG and a broadened scope of interventions to include attitudinal and culture change interventions. However, bid areas also commented on some tensions between the Fund’s aims and its parameters. This included the requirement of bid areas to pinpoint interventions to specific geographical locations, and some criticism of the funding of physical/environmental interventions (such as streetlighting and CCTV), which were considered by some stakeholders as not addressing the root causes of VAWG (that is, changing attitudes and behaviours relating to VAWG). Some bid areas also decided to exclude some interventions, including those aimed at tackling deep-rooted attitudes, requiring longer delivery timescales or not sustainable beyond the funding period. The main reasons were either a perception that the intervention was not deliverable, or the aims not realisable within the delivery timeframes. Some bid areas also saved interventions for other contemporaneous funding rounds, such as Safety of Women at Night (SWaN) but were later unsuccessful at acquiring funding.

For future funding rounds, if attitudinal and behavioural based interventions are in scope, the Home Office should consider guidance on targeting interventions at particular population groups, as well as the current focus on targeting particular public spaces. Future funding rounds may also benefit from sufficient gaps between rounds, or more clearly delineated staggering of rounds, where applications are likely to be submitted by the same organisations.

Application process

Reflecting on the application stage, funded areas shared their recommendations to future bidders of similar funding programmes on how to approach bid design. Funded areas recommended consulting broadly with a wide variety of stakeholder groups (including VAWG organisations) for strategic, expert and delivery input, and with future beneficiaries to ensure that projects are fit for purpose and impactful. Stakeholders also advised rigorous planning and preparation at the bid design stage, including assembling internal teams, and identifying and engaging prospective delivery partners. Stakeholders recommended that working with partners during the bid design stage was facilitated by clear objectives and roles and responsibilities. Finally, funded areas recommended considering what can realistically be delivered within the timeframes and designing interventions based on feasibility.

Delivering SSF3 interventions

The delivery of education programmes was a welcome addition to SSF3 from funded areas and stakeholders since education programmes had not previously been in scope of SSF funding. Education programmes were largely focused on changing attitudes towards women and girls and improving awareness of VAWG issues and crimes. Communications campaigns targeting particular population groups, support services and some personal safety initiatives shared similar characteristics.

Expert input, including from VAWG and voluntary and community sector (VCS) organisations, was considered particularly important for communications campaigns, support services, education campaigns and some personal safety initiatives to develop messaging, and resources for skilled support work. There was also a greater requirement for marketing and engagement with beneficiaries to encourage notice and take-up of these types of interventions. Since these interventions were more reliant on working with external delivery partners with expertise, early partner engagement at the bid stage was strongly recommended. Communications campaigns, support services, education programmes and some personal safety initiatives were also commonly cited as being impacted by short delivery timescales. A common view across delivery stakeholders was that embedding effective interventions aimed at key groups and addressing attitudes and behaviour change requires sufficient time for partner and beneficiary engagement, and for outcomes to be realised and measurable in an evaluation.

Environmental interventions, along with guardianship initiatives, were more focused on tackling issues in particular crime hotspots than targeting particular population groups. Generally, environmental interventions and guardianship initiatives were expected to increase feelings of safety in public spaces, increase the identification and deterrence of perpetrators, and consequently reduce crimes. These interventions required rigorous scoping phases to pinpoint locations. Effective partnership working and early engagement with suppliers was considered crucial to establish communication channels between organisations and teams, and enable decisions and changes to be approved efficiently. The impact of environmental interventions and guardianship initiatives required that interventions were clearly visible to the public. It is recommended prospective bidders incorporate effective scoping and location planning and the necessary signage and advertisements to ensure that the interventions are fully visible.

Funded areas reported several national circumstances which posed barriers to delivery, including COVID-19, bad weather and the knock-on effects of Brexit, such as global shortages and supply chain issues. Funded areas considered that careful contingency planning and responsive delivery were important across all intervention types to mitigate against these challenges. This included risk identification and planning at the pre-implementation phase, and then responsively pivoting delivery methods, where needed, such as moving training online in response to COVID-19.

1.6 Overall recommendations for future funding: design, delivery and evaluation

Across all findings, a clear set of implications for future iterations of the SSF programme have emerged. Some are based on the evaluation evidence and our own independent conclusions, while other ideas were received directly from the local SSF3 bid teams, based on their project design and delivery experience.

A. Design

Recommendations to bid areas on bid design: In the design of their bids, bid areas are encouraged to consult broadly with a wide range of stakeholder groups, including experts in the specific crime types, and the target beneficiaries of interventions. Bid areas should also engage future delivery partners during the design phase, and conduct as much upfront project planning as possible before mobilisation of projects. Finally, bid areas are encouraged to plan their interventions around what is feasible within the timeframes.

Recommendations for the Home Office on Fund design: Future rounds can maximise impact by scaling up the dosage (meaning the intensity of funded activity, such as volume and/or duration) to address the root causes of VAWG and bring about lasting and measurable change. To enable this, local delivery organisations need more time to develop their bids and the funding to focus on the longer-term sustainability of their interventions. It is therefore important that local bid areas receive clear and timely communications about what specific activities will be in scope for future SSF funding, thereby minimising uncertainty and ensuring continuity between different funding rounds.

B. Delivery

Recommendations to bid areas on project delivery: In the delivery of interventions targeting particular population groups, it is recommended that bid areas use the expertise of VAWG organisations and delivery partners and wider market interventions to encourage take-up and participation. For interventions targeting specific public spaces and/or crime hotspots, bid areas are encouraged to ensure effective partnerships between public sector bodies – such as the local authority and the Office of the Police and Crime Commissioner (OPCC) – to mobilise local area changes and use clear signage and advertisements to ensure that these interventions are sufficiently visible to the public.

Recommendations to the Home Office on enabling project delivery: The key implications for the Home Office to enable project delivery revolve around granting more time and providing better information and guidance. Many bid areas reported that crucial processes – such as consulting with local communities, onboarding delivery partners, running recruitment and procurement processes – required more time to implement effectively than the Fund allocated. There was a perceived lack of guidance around how delivery teams could ensure the sustainability of their interventions beyond the funding period, therefore further guidance to bid areas on sustainability is recommended.

C. Evaluation

Recommendations to bid areas and the Home Office for future evaluations: For SSF and similar programmes, we recommend focusing the evaluation effort on a clearly defined set of short-term outcomes (which can be directly attributed to the local interventions and demonstrate the difference they have achieved or are on track to achieving), rather than attempting to evidence longer-term impacts (which will take more time to materialise and are subject to an array of external influences).

In addition, it is generally good practice to build in appropriate time at the beginning of an evaluation to collect baseline data before interventions are being implemented, as well as scheduling any post-intervention data collection after the funding period.

Acknowledgements

Verian relied on the contributions from several parties to evaluate the Safer Streets Fund round 3 and report on our findings. First, our thanks go to our partners Dr James Copestake and Fiona Remnant and their team at Bath Social & Development Research (BDSR) who were responsible for the Qualitative Impact Protocol (QuIP) strand of this evaluation. We are also grateful to Home Office analysts and policy colleagues for their regular guidance and feedback throughout the evaluation. Our thanks also go to the anonymous peer reviewers who provided us with comments during the reporting stage and finally, to the evaluated project staff and their local residents who gave their time generously to the evaluation process.

2. Introduction

This section provides the background for the Safer Streets Fund round 3 (SSF3) and its intended purpose.

2.1 About the SSF3 context, aims and objectives

Tackling violence against women and girls (VAWG) is a government priority. News coverage of the high-profile murders of several women brought concerns about the safety of women and girls and the prevalence of VAWG in public spaces to the forefront. Crimes of VAWG are many and varied. They include rape and other sexual offences, stalking, domestic abuse, ‘honour-based’ abuse (including female genital mutilation and forced marriage and ‘honour’ killings), ‘revenge porn’ and ‘upskirting’, as well as many others. While different types of VAWG have their own distinct causes and impacts on victims and survivors, what these crimes share is that they disproportionately affect women and girls (Home Office, 2021a).

For example, the Crime Survey for England and Wales (CSEW) reported that in the year ending March 2022, 27% women and 5.7% men aged 16 and over had experienced sexual assault (including attempts) since the age of 16 (ONS, 2023). Women are also less likely to feel safe walking alone in public settings than men (ONS, 2021a), with many victims experiencing VAWG in public spaces – for example 37% of rape offences were reported to be outside the victim or offender’s home (ONS, 2021b).

In July 2021, the cross-government Tackling Violence Against Women and Girls Strategy was published to help ensure that women and girls are safe everywhere – at home, online, at work and on the streets (Home Office, 2021a). This was followed by a complementary cross-government Tackling Domestic Abuse Plan published in March 2022 (Home Office, 2022). These documents aim to transform the whole of society’s response to these crimes with actions to prevent abuse, support victims and pursue perpetrators, as well as to strengthen systems to respond to VAWG. They were informed by the Violence Against Women and Girls Call for Evidence conducted in 2020 (and reopened in March 2021), which received an unprecedented response – over 180,000 responses in total (Home Office, 2020). The government has also invested £5 million into the Safety of Women at Night Fund (SWaN), funding interventions which focus on preventing VAWG in public spaces at night, including in the night-time economy (Home Office, 2021b).

As part of the Tackling Violence Against Women and Girls Strategy (Home Office, 2021a), the government announced immediate steps to provide reassurance to women and girls and to protect their safety in public spaces, including an additional £25 million for the SSF3 announced in March 2021 (Home Office, 2021c). This third round focused on ensuring the safety of public places for all, with a particular emphasis on improving the safety of public spaces for women and girls.

It is important to note that throughout this report, the phrase ‘violence against women and girls’ (abbreviated to VAWG) is used to describe a range of crimes, including rape and other sexual offences, stalking, ‘upskirting’, and many others. VAWG crimes are known to disproportionately affect women and girls, but men and boys can also be victims. References of VAWG in this report do not include domestic abuse, as this was out of scope for the evaluation. VAWG is the official terminology adopted by the Home Office. In the surveys, the phrase ‘sexual harassment and gender-based violence’ was used instead, to make questions more accessible and digestible for respondents. To remain consistent with the official Home Office terminology, the phrase ‘sexual harassment and gender-based violence’ is referenced as ‘VAWG’ throughout the report.

The Fund:

• was open to civil society organisations that met certain criteria (charities, community and voluntary organisations, social enterprises and cooperatives), local authorities, PCCs, and the British Transport Police
• had a maximum value of £550,000 per bid and up to 3 bids per PCC geographic area
• funded interventions that aimed to prevent or reduce VAWG crime occurring in public spaces, and improve feelings of safety
• sought innovative approaches to help build the evidence base on VAWG prevention.

Following the announcement of the additional funding in spring 2021, the government delivered the application and award process under tight timescales. This was to allow funds to be made available to successful bidders by summer 2021 to give them as much time as possible for delivery before completion in March 2022. There were 89 bids, of which 57 were awarded funding, with successful bidders having until March 2022 to implement their plans and use their awarded funding as set out in their application.

3. Evaluation design and methodology

This section outlines the aims of the SSF3 evaluation, process to design the evaluation, and the evaluation methodology in brief. It includes key information for interpreting aspects of the report.

3.1 Evaluation aims and research questions

The Home Office commissioned Verian to conduct a process and impact evaluation of SSF3. The aim was to understand the impact of SSF3 interventions on perceptions of crime and feelings of safety in public spaces, and the factors affecting the processes of implementation in funded areas.

The central purpose of the impact evaluation was to determine the impact of SSF3 interventions overall; it was not intended to explore the local impact of individual projects, but rather the net impact of the funding across the target population in funded areas. The overall purpose of the process evaluation was to explore the setup, delivery and implementation experience of SSF3 areas, uncover the key enablers and barriers of delivering interventions, and to identify good practice emerging from SSF3. The specific design and focus of the impact and process evaluations were refined and adapted based on feasibility during the scoping and evaluation design phase.

3.2 Scoping phase methodology

This phase included a review of successful bid documentation, depth interviews with key stakeholders and VAWG experts, and data scoping to understand the locations and scale of interventions, as well as evaluation requirements and expectations. This was collated to determine an appropriate evaluation design for the process and impact evaluations^{[footnote 2]}.

3.3 Impact evaluation methodology

Impact evaluation was designed to assess the impact of the interventions at the Fund level, rather than exploring the impact on each of the 57 funded projects. It was agreed with the Home Office during the scoping as the most appropriate approach to achieve the key objectives for the evaluation, which was to demonstrate both overall impact and give an indication of interventions that are more/less successful in achieving the intended outcomes.

3.3.1 Summary of survey modes and respondents

Verian collected survey data in 56 of the 57 areas^{[footnote 3]} via 3 modes, each able to capture a different audience and/or public space (linked to specific interventions and where they took place). Each survey included both respondents from within SSF3 areas and those from comparison areas across 3 different audiences: users of public spaces, local residents and direct beneficiaries of training^{[footnote 4]}. Each survey audience was targeted through a different sampling framework and provided different sets of questions tailored to the aims and purposes of the in-scope interventions. A core selection of metrics were captured for all audiences, including perceptions of and change in perceptions of safety and crime, experience/witnessing of VAWG, and awareness of interventions.

Surveys were conducted in line with the Market Research Society code of conduct. Prior to giving consent to participate, potential interviewees were informed of the sensitive nature of the survey, and that participation was voluntary and individual responses would remain confidential. The survey took approximately 15 minutes to complete and participants were provided with information about VAWG sources of support following completion.

Users of public spaces: Face-to-face survey

Aims: The face-to-face survey aimed to capture the impact of public-facing initiatives, such as guardianship initiatives (guardians are ‘everyday citizens who are present when an offender in search of crime opportunities intersects with a potential crime target’) that were likely to be seen or used by both local and transient populations in public spaces, such as high streets / shopping areas, public parks or other green spaces, and public transportation stations.

Sample: The survey took place in 16 funded (n=1,084) and 13 comparison (n=600) areas.

A total of 16 bids were identified as delivering public-facing initiatives in scope for face-to-face survey data collection. These bid areas were asked to provide details on the exact locations where their interventions would be most visible/accessible to users of public spaces.

A further 13 comparison areas were identified. Suggestions of comparison areas were gleaned from bid areas for similar areas to collect comparison data, relying on local insight to ensure that the set of comparison areas was reasonably similar to the set of SSF3 areas. The suitability of the comparison areas was checked against several data sources, including:

• police incidents data
• deprivation statistics
• data from the Department for Transport

Interviewers were stationed at each of the bid and comparison locations. The interviewers approached members of the public using or passing through those public spaces, asking them to participate in the survey. Members of the public were offered a £5 shopping voucher as a thank you for taking part.

All interviews were conducted between 9am and 9pm. In some locations, where funded interventions were focused on NTE areas, interviews were only conducted in the evening (between 5pm and 9pm).

Each interviewer had a target number of interviews to be conducted within their shift; this ensured balance in the numbers of interviews in each area.

Local residents: Address-based online survey (ABOS)

Aims: The ABOS survey aimed to capture the impact of public-facing initiatives that were intended to reach and be relevant for local residential communities in funded areas, such as CCTV and streetlighting.

Sample: The survey took place in 49 funded (n=3,761) and 10 (n=1,335) comparison areas.

A total of 49 bids were identified as delivering public-facing initiatives in scope for ABOS survey data collection.

A stratified random sample of 15,000 households in areas covered by SSF3 projects was drawn. The sample was selected to ensure an equal number of addresses for each bid. Within each bid, residential addresses were sorted by Lower Super Output Area (LSOA), postcode and first line of address. An equal probability sample of addresses was drawn from within each bid area to ensure an even spread across the addresses within that area.

A sample of 5,000 comparison addresses was also drawn. The comparison addresses were selected using a form of propensity score matching to ensure similarity to the resident population of SSF3 areas. Areas were matched on a range of area-level variables:

• the demographics of the local resident population as recorded in the Census
• deprivation
• rural/urban classification
• PRC for violence and sexual assault

Letters were sent to sampled addresses in both the SSF3 areas and the comparison group, inviting resident adults to complete an online survey. Households received 2 separate reminders, and some received paper versions of the questionnaire.

Training beneficiaries: Email survey

Aims: The email survey aimed to capture the impact of VAWG education^{[footnote 5]} and training interventions^{[footnote 6]} delivered directly to individuals. These audiences included:

• university staff and students^{[footnote 7]}
• teachers
• community workers or volunteers
• NTE workers
• police
• security
• other professional services

Sample: The survey took place in 18 funded areas with individuals who had been direct beneficiaries of SSF3-funded training (n=232), and individuals in similar roles who had not taken part (comparison; n=134).

A total of 18 bids were identified as delivering interventions directly to individuals.

Local bid areas identified these beneficiaries. All relevant bid areas were contacted to determine how best to reach the intervention beneficiaries. Bid area stakeholders either distributed the survey links to beneficiaries directly or sent the survey links to intermediaries (for example, training providers) to distribute. All beneficiaries were eligible for the survey.

To reach suitable counterfactual respondents, bid areas were asked to share the survey link with individuals in similar roles who had not taken part in the interventions. These respondents were identified and contacted in the same way as the training beneficiaries – bid area stakeholders identified individuals in similar roles (for example, police officers in the organisation/area who did not take part in SSF3-funded training), and either distributed the survey link directly or via intermediaries.

Reported data

The final sample sizes reported reflected all data collected across all modes in funded and comparison areas. All audiences were used in the impact modelling reporting in Section 5 of this report. However, only data from residents and users of public spaces within funded areas, and training beneficiaries in funded areas who received training as part of funded activity, has been used in descriptive analysis. This has been included to provide context for experiences of crime and perceptions of safety, as well as their awareness of the funded interventions, presented across Section 4 and Section 5. While a selection of core metrics (for example, perceptions of and change in perceptions of safety and crime, experience/witnessing of VAWG, awareness of interventions) were asked to all, the audiences were largely treated as distinct population groups. As such, the analysis sought to examine evidence of impact (that is, change) within each group. Any observed differences in findings between groups are not described. However, the analysis did not make direct statistical comparisons between groups, nor was the data pooled to create one large sample.

3.3.2 Impact analysis

Retrospective impact measurement

The key impact measure for SSF3 was changed perceptions of safety among respondents in public spaces during the day and after dark / at night. We also examined perceptions of crime, confidence in local police and likelihood of reporting crime to police. Due to the timing of the interventions and evaluation, it was not possible to obtain a true baseline or ‘pre’ intervention assessment of key metrics/indicators. The main measurement of impact was therefore based on a retrospective approach in direct questioning of survey respondents. Respondents were asked to provide responses on key metrics for how they feel today and then the degree to which this has changed in the past 6 months. A scale of 0 to 10 was used for the measurement of change over time, from 0 being very negative change to 10 being very positive change. A 10-point answer scale was chosen (as opposed to a 4- or 5-point scale) as it provides more sensitivity to detect smaller degrees of change and so offers slightly more precision for an analysis based on retrospective recall.

Impact models

Section 5 includes estimates of the impacts of SSF3 interventions on key outcomes and impact measures as measured through the surveys. The impacts are estimated by comparing outcomes for respondents in SSF3 areas against those of respondents in the comparison areas, while attempting to account for differences between the funded and comparison groups. Any statistically significant difference at the 95% confidence level in the scale of retrospective change reported between the 2 groups is the impact attributed to SSF3 interventions.

The impacts are all estimated through hierarchical linear regression models. The models include several variables to account for differences between the funded and comparison groups. This includes both characteristics of individuals (such as age and education status) and characteristics of areas. Under the assumption that all relevant differences between the 2 groups are adequately accounted for, the remaining difference between the 2 groups can be interpreted as the impact of SSF3 interventions. Further information about these models are in Appendix B.

Although the models estimate the average impacts of SSF3 interventions, in reality, impacts will vary between individuals: some individuals will not be affected by the interventions at all, while others will be positively impacted, and others negatively.

3.3.3 Estimating impacts using police recorded crime data

The evaluation also integrated PRC data to estimate secondary impacts for neighbourhood crime. Broadly, this compared trends in the number of recorded crimes in funded areas to the equivalent trends in other similar comparison areas.

The Home Office supplied PRC data – quarterly counts of the number of crimes recorded in each LSOA for burglary, robbery, theft from the person and vehicle crime^{[footnote 8]}. For some areas where this data was incomplete, further records of PRC were downloaded from police.uk.

In summary, Verian used Generalized Synthetic Control Models to estimate impacts. These models compare the number of recorded crimes in SSF3 areas against the number of crimes recorded in a set of similar areas which were not part of SSF3 or previous SSF rounds. The process ensures that these comparison areas are broadly similar to SSF3 areas regarding a range of characteristics (described in Appendix C). The models weight these comparison areas such that the trend in recorded crime before introducing SSF3 closely matches the trend in SSF3 areas. The rationale is that if the historic trends are closely matched, it is reasonable to think that this weighted average of the comparison areas can provide a good counterfactual estimate, that is, a good estimate of what would have happened in the bid area without SSF3. Impacts were estimated based on the number of crimes recorded in April to June 2022. Further information about how impacts have been estimated using these data sources is given in the Appendix C.

3.4 Qualitative Impact Protocol

As part of the impact evaluation of SSF3, Verian conducted a QuIP study in partnership with Bath Social and Development Research (Bath SDR), the curators of the QuIP methodology. QuIP is a non-experimental, qualitative approach to impact evaluation that aims to collect credible information directly from intended beneficiaries about the drivers of change relating to specific ‘domains’ of their life and over a pre-defined period of change. Through the analysis of data collected from in-depth interviews, the QuIP identifies beneficiaries’ perceptions of change and the key factors they perceive to be attributing to the change. This approach is particularly useful in complex contexts, such as that of SSF3, where a variety of factors that are hard to disentangle can influence the outcomes of an intervention.

A standard QuIP study involves 24 interviews, a ‘double QuIP’ (that is, 48 interviews) was delivered for the SSF3 evaluation. The QuIP focused on the perceptions of safety, and the influencing factors and drivers of change of these perceptions within a population of members of the public (male and female) within funded areas. A trained QuIP analyst from Bath SDR coded and analysed the QuIP data using a Causal Map App to identify causal chains in the data^{[footnote 9]}.

Participants were selected based on whether they had indicated an overall experience of positive (n=25) or negative change (n=23) in their response to the survey. Changes in perceptions of safety was explored with QuIP participants by asking whether their perception of safety had changed at all over the recall period (approximately 9 months – since October 2021) and crucially why and how they perceived things to have changed.

3.5 Process evaluation methodology

The process evaluation aimed to explore the experiences of local bid teams and their journeys from bid development, planning and implementation, to delivery and early outcome measurement of interventions. A mix of qualitative research methods were used at 2 points in the delivery of SSF3 interventions: at the start of delivery (baseline data collection) and once intervention delivery had concluded. The insights from the process evaluation shed light on facilitators and barriers to the successful design and delivery of SSF3 interventions, and lessons for the Home Office on the administration of the Fund overall. There were 3 main phases of data collection.

Baseline data collection (December 2020 to February 2021)

Strategic leads within bid areas completed a baseline questionnaire to capture data on their aims and expectations for the interventions they were about to implement or were in early stages of implementing. This exercise aimed to capture a Fund-level view of design and implementation experience and the creation of intervention logic models (ILMs)^{[footnote 10]}.

Case study research (March to June 2022)

The purpose of the case study research was to explore the individual bid-level detail in a select number of bid areas, in terms of their setup, implementation and monitoring of interventions, and to explore to what extent the ILMs were being realised in practice within those cases. Case studies were purposively sampled based on achieving breadth and depth of insight across a range of selection criteria (see Appendix B, Section 8.2.1). The case study fieldwork comprised 3 core activities:

• a document review
• case study interviews
• case study focus groups in some areas

A total of 27 interviews and 5 focus groups were conducted.

Follow-up data collection (March to June 2022)

A follow-up questionnaire captured data at end, or in the final stages, of implementation delivery to understand how interventions were implemented and delivered in reality, including the actual outputs, outcomes and impacts (as far as was measurable). It also explored bid teams’ views on their successes, challenges and lessons learned.

Following the conclusion of all fieldwork, researchers synthesised findings from the 3 data collection exercises to develop overarching findings under each of the main reporting headings, based on the research questions of the process evaluation. A workshop was conducted with stakeholders in the Home Office in September 2022 to discuss the main findings.

4. Overview of funded interventions, experience and perceptions of crime and safety

This section outlines the crimes, public spaces and interventions that were the focus of SSF3-funded activity, using data from surveys conducted with users of public spaces, local residents and training beneficiaries who took part in SSF3 interventions such as education or training. It includes descriptive statistics of experiences and perceptions of crime, police and feelings of safety in a range of public spaces.

4.1 Crimes in focus

All bids included interventions to target VAWG crimes. Nearly all funded areas sought to address street harassment (98%) and serious forms of VAWG like sexual assault (96%) and rape (84%). Most funded areas also sought to target secondary crimes and behaviours, including anti-social behaviour (ASB) (86%) and acquisitive crime (84%) and other crime types (82%). In terms of acquisitive crime, robbery (75%) and theft from the person (40%) were the most common secondary crimes targeted. Of those addressing other crime types, violent crimes were the most targeted (61%).

Figure 1: VAWG crimes targeted by bid areas

4.2 Public spaces in focus

The funded interventions targeted a range of public spaces, with the majority (88%) targeting more than one type of public space. Across the different types of public spaces, 82% of bids targeted commercial areas including town centres and high streets, NTE areas, shopping centres and other high-traffic areas that both residents and a wider transient audience may use. Next most commonly targeted were non-residential areas and residential areas, both of which were a focus for three-fifths of bids (60%). Non-residential areas included streets, walkways, footpaths or canal paths that are used as travel zones or ‘corridors’ between spaces where footfall may occur at various points throughout the day. Residential areas are those predominantly occupied by homes or other accommodation and are likely to be most frequently populated by residents.

Around half of funded areas (53%) targeted parks and open spaces^{[footnote 11]}. A similar proportion targeted their interventions toward public transport (49%). Those targeting transport focused on public transport vehicles or terminals such as on buses, trains, trams or taxis and at associated stations or stops that may be used both by local residents and by a wider, more transient audience. University areas were being targeted by just over a third of funded areas (37%). This type of area included university campuses and facilities such as walkways, campus night-time venues or education buildings, and residential areas occupied by students, such as university accommodation.

4.3 Interventions

Across the 57 funded projects, each delivered a combination of interventions across 8 distinct categories and 23 intervention types. Activity ranged from one project delivering just one intervention to another delivering 17 different interventions, with the majority (86%) delivering 5 or more. The most common intervention categories were environmental interventions, like CCTV and lighting (91%), education and training programmes (88%) and communications campaigns (79%). Over half of projects interventions included guardianship initiatives (58%), followed by interventions to enhance personal safety (40%) and support services (18%) aimed at victims of VAWG crime.

When looking at the proportion of spend against each category, the overall composition of the Fund was very similar. Environmental interventions received the greatest proportion of spend (45% of total Fund), followed by educational programmes (19%) and communications (10%). The same 3 categories were therefore both the most common interventions across bids, and collectively were the focus for three-quarters of the overall Fund allocation. However, in some cases, the prevalence of the intervention category was not a direct indication of the proportion of total funding. For example, although 79% of projects feature communications and campaigns, only 10% of Fund spend was allocated to this intervention, as they were less cost intensive than other types of interventions. See Appendix A for a breakdown of interventions delivered.

4.4 Perceptions of VAWG

It is important to consider recent experience of VAWG crime amongst the target populations during the SSF3 delivery period. While funded areas had a focus on VAWG, a key aim of the Fund was to improve safety of public spaces for all. As such, across all surveyed audiences, people of all genders were surveyed. As anyone may experience sexual violence or harassment, questions relating to this type of crime used the phrasing ‘sexual harassment and gender-based violence’ to ensure they were inclusive and clear about what was being asked.

Due to the timing of the evaluation in relation to the delivery of the Fund, it was not possible to capture a baseline measure prior to implementation. Surveys were conducted between March and July 2022 following the implementation of the Fund, and relied on retrospective questions to capture respondents’ perceptions and experience. Respondents were asked to indicate their current perceptions of safety in public spaces and their experiences of VAWG in the previous month, and the extent to which their feelings of safety had changed since October 2021 (prior to implementation of the Fund); see Section 5.1.

4.4.1 Experience of VAWG

At the point of survey, overall, 26% of users of public spaces, 14% of local residents and 19% of training beneficiaries reported they had personally experienced or witnessed VAWG in public spaces in the last month. Overall, across all audiences, respondents most commonly reported witnessing VAWG, as opposed to both experiencing and witnessing, or experiencing only. Users of public spaces were most likely to report both experiencing and witnessing VAWG (7%), compared to training beneficiaries (5%) and local residents (3%).

Among users of public spaces and local residents, women (32% and 17%, respectively) were statistically significantly more likely than men (19% and 12%, respectively) to report they had experienced or witnessed VAWG in public spaces in the last month. Regardless of gender, those aged 16 to 24 were statistically significantly more likely to report having experienced or witnessed VAWG compared to all other age groups, with around one-third of this age group in both audiences (35% users of public spaces, 32% local residents) reporting they had experienced or witnessed VAWG in the last month.

Across all audiences, among those who had experienced or witnessed VAWG, the most common public spaces where this occurred were in the street in a commercial or shopping area, in the street in a residential area, and in a pub or bar. Users of public spaces (42%) and training beneficiaries (40%) were most likely to report that this occurred in the street in a commercial or shopping area. Local residents were most likely to report this had occurred in the street in a residential area (30%). The most common type of VAWG experienced or witnessed among all audiences was verbal harassment, including unwelcome comments, jokes or sounds. This type of harassment was reported by almost two-fifths (37%) of users of public spaces, one-third of training beneficiaries (32%) and a quarter of local residents (24%).

Among local residents, women were statistically significantly more likely to have experienced verbal harassment (27%) compared with men (21%). Residents with a disability were statistically significantly more likely to report experiencing or witnessing emotional or psychological violence and abuse (10%), compared with those without a disability (6%).

Among both local residents and users of public spaces, 16- to 24-year-olds were statistically significantly more likely to report experiencing or witnessing all forms of VAWG compared to older age groups.

4.4.2 Perceptions of safety in public spaces

All audiences were asked the extent to which they feel safe or unsafe in different types of public spaces, during the day and after dark / at night. Overall, all audiences felt safer across all types of public spaces during the day compared to at night. Overall, users of public spaces and local residents felt safest in high street or shopping areas during the day (86% and 81% feel safe, respectively), and least safe in parks and open spaces at night, where only 31% of users of public spaces and local residents feel safe. Training beneficiaries overall felt safest at their place of employment or study during the day (88%), and least safe in areas where there are pubs, bars or clubs at night (31%). Feelings of safety were examined by a range of demographic characteristics, and key statistically significant differences are reported under each space type. Overall, similar patterns emerged across space types for gender, age, disability and ethnicity. Women generally felt statistically significantly less safe compared to men, both during the day and at night, across all types of spaces. Across most spaces and particularly at night, younger age groups, individuals with a disability, and Asian, black and mixed ethnicity respondents reported feeling less safe.

High street / shopping areas

Most users of public spaces (86%) and local residents (81%) report feeling safe in high streets or shopping areas during the day, reducing to around half (48% and 51%, respectively) who report feeling safe after dark / at night. Across both audiences, women and those with a disability were statistically significantly more likely to report feeling unsafe, both during the day and at night. For example, among users of public spaces, 62% of women and 54% of those with a disability report feeling unsafe in this type of space at night, compared to 30% of men and 43% of those without a disability. Younger respondents, and Asian, black and mixed ethnicity respondents were also statistically significantly more likely to report feeling unsafe in these spaces after dark / at night.

Parks and open spaces

Around three-quarters of users of public spaces (77%) and local residents (72%) report feeling safe in parks and open spaces during the day. In contrast, among both audiences, less than one-third (31%) report feeling safe after dark / at night. Overall, across both audiences, women, those with a disability and younger respondents were statistically significantly more likely to report feeling unsafe in these spaces during the day and at night.

Transport hubs

At transport hubs such as train or bus stations, 82% of users of public spaces and 70% of both local residents and training beneficiaries report feeling safe in these spaces during the day, reducing to 51% of users of public spaces, 42% of residents and 32% of training beneficiaries feeling safe after dark / at night. Among users of public spaces and local residents, women, younger age groups and those with a disability were statistically significantly more likely to feel unsafe.

Night-time economy

Respondents of all surveys were asked about their perceptions of safety in NTE areas, those with pubs, bars and clubs. Three-quarters of users of public spaces (76%), 67% of training beneficiaries and 56% of residents report feeling safe in NTE areas during the day, reducing to 42% of users of public spaces, and around one-third of residents (34%) and training beneficiaries (31%) who feel safe at night. As seen in other space types, among users of public spaces and residents, women, 16- to 24-year-olds, those with a disability, and Asian respondents were statistically significantly more likely to feel unsafe in NTE areas than others. Among training beneficiaries, only gender was statistically significant, whereby women were statistically significantly more likely to report feeling unsafe during the day (34%) and at night (75%) compared to men (13% and 32%, respectively).

Residential areas and areas of work or study

Local residents were asked about their perceptions of safety in quiet streets close to their home. As seen in other space types, the proportion of residents who feel safe during the day (78%) was higher than at night (50%). Women, 16- to 24-year-olds and people with a disability were statistically significantly more likely to feel unsafe in streets close to their home.

Training beneficiaries were asked about their perceptions of safety in the area where they work or study, and at their place of employment or study. During the day, the majority of training beneficiaries report feeling safe at their place of work or study (88%), and in the area where they work or study (84%). At night, 62% report feeling safe at their place of work or study, and 49% in the area where they work or study. Among training beneficiaries, only gender played a statistically significant role, whereby women felt more unsafe in the area near where they work or study both during the day and at night, and at their place of work or study at night.

5. Impact evaluation

This section presents results from impact modelling using survey data for all audiences. It includes findings from QuIP interviews to understand potential mechanisms of change with users of public spaces and local residents. It also examines the influence of training and education programmes on outcome measures with training beneficiaries through descriptive data. For more information about data collection methodology, please refer to Section 3.3 and Appendix C; for current perceptions of crime and safety against which these outcomes and impact were measured, please refer to Section 4.

5.1 Results of impact modelling

The key impact measure for SSF3 was changed perceptions of safety among respondents in public spaces during the day and after dark / at night (Section 5.1.1) and perceptions of crime (Section 5.1.2). Additional impact measures included confidence in the local police and likelihood of reporting crime to local police (Section 5.3.1).

5.1.1 Impact on perceptions of safety in public spaces

Verian asked respondents to indicate, on a scale from 0 (less safe) to 10 (much safer), how much their feeling of safety changed in the 6 months prior to the interviews (since October 2021) in different public spaces. These were quiet streets close to home (only for residents), high street or shopping area nearby, public transportation, parks and open spaces, and finally, where there are pubs, bars and clubs.

There was no statistically significant evidence of improvements in perceptions of safety in public spaces due to SSF3 interventions. As shown in Figures 2 and 3, this was the case both for users of public spaces and residents of these areas, respectively. For local residents, in particular (Figure 3), this evidence suggests that any impacts are likely to have been small; the estimated impacts are all close to zero, and the 95% confidence intervals (represented by the vertical bars) are fairly small. Broadly, these confidence intervals can be interpreted as a range which is likely to contain the true average impact of the interventions across SSF3 areas, if the assumptions of the analysis are approximately true.

For users of public spaces (Figure 2), some of the estimated impacts are slightly higher, such as perceptions of safety in open spaces at night. However, the smaller sample size for this survey means the results are more uncertain than the results for local residents, as shown by the wider confidence intervals. As a result, these findings are not statistically significant.

Figure 2: Impact estimates on perceived changes in safety among users of public spaces

Figure 3: Impact estimates on perceived changes in safety among local residents

Further analysis explored whether there was a difference in perceived changes in safety in areas with interventions focused on particular public spaces. For example, we examined whether funded areas that delivered interventions focusing on high street / shopping areas had a different effect on perceived changes of safety in that space type (high streets / shopping areas) than funded areas where interventions focused on all other space types. This was similarly replicated for public transport, parks and open spaces and close to home. For local residents, areas with interventions focused on different types of public space (close to home, high street / shopping areas, public transport, open spaces and pubs/bars/clubs) do not appear to have been more effective at improving perceptions of safety in the corresponding spaces than other funded areas. However, there was some evidence from users of public spaces that projects with interventions specifically focused on high streets / shopping areas or on public transport were more effective in these spaces than other funded projects. This was found for increased feelings of safety during the day only (that is, not for changes in perceptions after dark / at night), with impact estimates for both spaces (high streets / shopping areas and public transport) statistically significant at the conventional 95% level (see Appendix C, Section 8.3.8 for results). In other words, there was evidence to suggest that interventions targeted toward specific space types were effective in improving perceptions of safety during the day in the corresponding spaces.

5.1.2 Impact on perceptions of crime

In addition to perceptions of safety outcomes, the survey asked all respondents about their perceptions of change in crime in the area, and their confidence in the police in relation to VAWG. For perceived changes in crime, respondents were asked how much they thought on a scale from 0 (significantly decreased) to 10 (significantly increased) certain types of crime had changed in the last 6 months. Crime types were VAWG, both during the day and at night, acquisitive crime (residents only) and ASB.

Local residents reported that they perceived a reduction in VAWG during the day (Figure 4). This was the only statistically significant finding regarding perceptions of change in crime across the 3 audiences and crime types. There was some evidence that SSF3 projects led to changes in perceptions of crime moving in the right direction, that is, perceptions of a reduction in VAWG and ASB among users of public spaces. However, these results were not statistically significant at the 95% level (Figure 5).

Figure 4: Impact estimates on perceptions of change in crime types among local residents

Figure 5: Impact estimates on perceptions of change in crime types among users of public spaces and training beneficiaries

Given some indication of perceptions of a reduction in VAWG, further analysis was conducted for perceptions of change in VAWG in local areas among users of public spaces and residents, either at night or during the day. This was to explore the extent to which this apparent reduction in the perceived prevalence of VAWG was due to changes within certain demographic groups, projects or areas. The estimated impacts did not vary substantially by the types of interventions being delivered, or by age or gender (see Appendix C, Section 8.3.8 for results).

5.2 Factors influencing the impact of SSF3

The limited impact of SSF3 on the perceptions and feelings of safety and crime among respondent groups may, in part, be explained by contextual factors relating to the Fund and its target outcome measures.

5.2.1 Awareness of interventions

There was a high degree of general awareness of crime prevention interventions; 82% of users of public spaces, 70% of residents and 87% of training beneficiaries were aware of at least one type of intervention. Despite good general awareness of crime prevention interventions, there was less awareness of new interventions that had been delivered as part of SSF3 with only 58% of users of public spaces, 44% of residents and 68% of training beneficiaries saying at least one intervention was new or increased within the previous 6 months. Awareness of all intervention types was higher among training beneficiaries than other audiences, which was expected given the nature of this audience as direct recipients of SSF3 interventions.

However, while respondents may be generally aware of different types of crime prevention or intervention activity, they were not necessarily aware of, or able to accurately identify, what was new or increased in public spaces or in their local area. Only a small minority who said they were aware of or had noticed these types of intervention correctly identified interventions that were known to have been delivered through SSF3 in their local area. Respondents across funded areas and audiences indicated awareness of interventions such as adverts, CCTV and lighting, even in areas where these interventions were not funded by SSF3. For example, adverts were a common intervention but only correctly recognised by 5 to 6% of respondents. It may be that respondents had noticed activity or interventions that were funded locally, or via other schemes such as the SWaN Fund. During the process evaluation, stakeholders commented that where interventions were not considered highly visible, and steps had not been taken to ensure noticeability (such as engaging communities in the intervention or placing advertisements), they were considered to be less impactful on users of public spaces (Section 6.3).

5.2.2 Perceived impact of interventions

Despite low levels of awareness of specific interventions being delivered in their area, most respondents were positive about the potential impact of these types of interventions on their perceptions of safety and levels of crime. At least half of users of public spaces think these interventions will have some impact on their perceptions of safety in public spaces, and on levels of VAWG crime, and a further one-fifth think they will have a big impact. Conversely, around one-fifth think SSF3 interventions will have no or little impact. For example, 18% of users of public spaces think that SSF3 interventions will have a big impact on levels of VAWG crime, 58% think they will have some impact and 17% think they will have no impact. The process evaluation found that some stakeholders involved in the design and delivery of SSF3 reported doubts about the potential impact of CCTV and streetlighting specifically. Some stakeholders considered that these interventions were not addressing the root causes of VAWG and could be sending the wrong message about how VAWG could and should be addressed.

5.2.3 Factors influencing feelings and perceptions of safety in public spaces

There are complex and multi-faceted personal, social and environmental factors that interact to influence perceptions and feelings of safety in both positive and negative ways. While the wider impact data does not evidence statistically significant change in perceptions of safety, the QuIP analysis can support the identification of the factors that influence change in perceptions and mechanisms by which change occurs. The QuIP analysis provides some evidence demonstrating examples of individuals whose perceptions have changed over the period of SSF3 delivery and as a direct result of SSF3 interventions. This suggests that while impact may not be visible at the Fund level, the interventions themselves may still be having a positive effect in a more isolated way.

The most commonly cited factors influencing positive and negative change over the recall period are presented in Figures 6 and 7 respectively, alongside detailed analysis of the key factors attributed to change. The QuIP findings highlight the complex interaction of factors that influence experiences of change, ranging from specific interventions noticed to social factors, including prior experiences of VAWG and crime. In summary, analysis of the QuIP finds that:

• some participants identified they had experienced a positive change in perception of safety in at least one public space type during the recall period, most commonly this specifically related to high streets and parks
• some identified negative change in perception of safety in at least one public space type; this includes prominent overlap, whereby individuals experience positive change in some spaces and negative change in others
• some interventions noted by interviewees included those that are likely to have been delivered through the SSF3 programme; in some instances, positive change in perceptions of safety was attributed to the intervention(s); considering that QuIP participants were not prompted to discuss interventions and they were unaware that the research related to SSF3 specifically, it is of note that they discussed such interventions; these included references to CCTV, street lighting, police presence and environmental maintenance schemes (Figure 6); also of note are unintended consequences of interventions; in a few instances, interventions were noted to reduce feelings of safety
• other social and personal factors are at play that have negatively and/or positively influenced a change in perceptions of safety during the recall period; these include factors such as the presence of others, perceptions and experiences of VAWG, and other crimes such as ASB

Causal maps have been included below to visually present the findings from the QuIP analysis. The following should be noted when reading QuIP maps:

Understanding the arrows: Maps should be read from left to right, with the direction of the arrowhead on each link reflecting the direction of causation or influence.

Factor labels: The analyst created the factor labels iteratively, which may include additional information relating to the factor. Factors relating to general influences are presented in parentheses (for example, social factor). Details of factors relating to specific intervention categories are presented by a semicolon (for example, for the label ‘Environment: CCTV’, Environment is the intervention category or type, and CCTV provides detail on the specific intervention). Where not attributed to either general influence or intervention-specific factors, this reflects influences noted by interviewees that do not fit in either category.

Factors included: Factors are only shown in maps if they are identified in the data more than once.

It should also be noted that the data presented from the QuIP is qualitative in nature, meaning any examples of change and impact highlighted in the QuIP are at an individual level, relating to individual participants’ experiences.

Figure 6: Factors leading to positive changes in perceptions of safety

Figure 7: Factors leading to negative changes in perceptions of safety

5.2.4 Views on interventions

SSF3 interventions were mostly mentioned in relation to positive impact on perceptions of safety, but sometimes interventions were referenced in relation to driving negative perceptions of safety, as demonstrated in Figure 7. This suggests that the referenced interventions had some unintended consequences in some instances.

Increased police presence

The presence and activities of police was identified as a key factor influencing changes in perceptions of police; however, there is a mixed picture of whether this led to positive or negative changes.

Some participants attributed increased police presence to them feeling safer – specifically on the high street. These sources noticed ‘more officers on the beat recently’ both ‘patrolling’ on foot and in police cars. A few of these sources came from bid areas where interventions targeted policing based on the bid information available for this round of the Fund. Similarly, one participant noted a decrease in police presence or a lack of police response, which had in turn negatively impacted their perceptions of safety.

At the same time, one participant highlighted that police presence / policing activity had led to them feeling less safe over the recall period.

Environmental interventions

Various environmental interventions were noted, mostly by participants who had responded positively in the survey.

Renovations and improved maintenance, particularly in parks, were reported to be positively changing perceptions of safety. This included cutting and clearing overgrown grass/trees and improved refuse collection and litter picking. As a result of these changes, participants said the spaces felt safer, with a few noting this was because they perceived them to be less hazardous or threatening. Based on the bid information available, a few participants reporting improved maintenance were from areas where these types of interventions took place.

A few participants mentioned that new/improved street lighting contributed to them feeling safer on the streets and/or in parks. However, according to the bid information available, none of these areas received interventions of this type.

The impact of both renovations and lighting on one individual is demonstrated in the quote below.

“[The lighting] has been a real benefit. But as much as the lighting, it’s the clearing of those trees, a lot more natural light comes in, it’s opened up the whole area. I don’t think anyone was hiding in the bushes, but I think people feel they can see everything. You didn’t realise it was an issue until it had been cleared. It was a good idea to do it.”

A few participants also reported installations of CCTV as factors positively changing their perception of safety. One of these participants came from a bid area where there was a targeted intervention of this nature.

A couple of participants also mentioned security measures within pubs as having a negative impact on perceptions of safety. For example, one individual outlined how pub staff wearing body cameras reduced their feelings of safety as it made them think that “there must be a reason why they are having to wear these”. This highlights an unintended consequence of this type of intervention.

Guardianships

A few participants noted street wardens/guardians as influencing their perceptions of safety; one person from a bid area where these types of interventions were planned stated that this increased their feelings of safety according to the bid information available.

Hypothetical interventions

In addition, as part of the QuIP, interviewees mentioned interventions that could be done, or that they would like to see done, to improve their feelings safety in public spaces. The interventions referenced by participants include CCTV, lighting, safety hubs and street wardens, all of which are aligned with the interventions that were delivered through SSF3; this mirrors individual perceptions that interventions should create change to crime and safety. This suggests that while it is too early to evidence the impact of SSF3 interventions, the types of interventions funded are aligned with those that QuIP respondents individuals say could or would improve their feelings of safety.

5.2.5 Social factors

Presence of others

As shown in Figure 6, some participants reported that lockdown restrictions easing meant there were more people using public spaces, some of which went on to add that this had increased their sense of safety. A couple mentioned that less people were living on the streets, and this also made them feel somewhat safer.

Community activities

New community activities were described as having a noteworthy positive impact on safety through developing new social relationships, building a sense of community and providing a place for support. This includes references to social meetings, workshops (creative workshops, knitting) and exercise sessions (yoga) within the community. One participant reflected on how a sense of community created through activities helped with feeling safe and encouraging reporting of VAWG:

“If you feel comforted and you feel welcomed, then you are more likely to…report it [domestic abuse].”

These activities were unrelated to the Fund and were only mentioned by a few participants.

Awareness of VAWG

Some participants reported that their awareness of VAWG had increased, leading to them feeling less safe. Specific forms of VAWG referenced were harassment, assault, spiking and rape. Participants reported that this increased awareness of VAWG came from high-profile examples covered in the media and word of mouth.

Anti-social behaviour

Some participants said that increased ASB had reduced their sense of safety, particularly mentioning more fighting, drinking and throwing objects (including bottles) in public spaces, especially at night. Some participants linked this increase in ASB to the current economic situation and the increased levels of alcohol consumption to continuing “patterns of consumption” embedded from lockdown.

Perception of other crimes

Some participants perceived that crime had increased in their area during the recall period, which had reduced their sense of safety. The crime most frequently reported to be increasing was theft, including a few specific references to dog theft. There were also references to gun/knife crime and drug dealing. Some of these participants reported that their perception of crime had increased due to increased awareness of crime from media coverage, word of mouth and social media.

Other factors

In line with perceptions of increased crime, a few participants theorised this was related to the current economic situation. For example, one participant reflected that increases in mugging and theft in his area may result from financial hardship being experienced. Increased homelessness – “more rough sleeping” – since October 2021 was another social factor reported by a couple of sources to have reduced perceptions of safety.

5.2.6 Personal factors

Experience of VAWG

In addition to perceptions of VAWG, outlined above, a few participants reported personally experiencing or observing VAWG during the recall period, and that these experiences had negatively affected their perception of safety. Their accounts referenced experiences of stalking and sexual harassment.

5.3 Other measures of impact

5.3.1 Impact on level of confidence in the police and likelihood to report crime

Respondents were also asked to indicate on a scale of 0 (significantly decreased) to 10 (significantly increased) how in the last 6 months:

• their confidence in the police to deal with VAWG in public spaces had changed
• how their likelihood to report VAWG to the police had changed

Impacts appear to be quite strong, positive and statistically significant on both of the above outcomes for training beneficiaries. There was a statistically significant increase in training beneficiaries’ confidence in local police to deal with VAWG and in their likelihood to report to local police. There was also some evidence that SSF3 interventions led to an improvement in confidence in local police to deal with VAWG-related crimes among local residents in funded areas compared with residents in the comparison areas, showing a small yet statistically significant improvement. However, there was no evidence of an impact on this outcome for users of public spaces. Similarly, there was no evidence of an impact on the likelihood to self-report VAWG to police among residents and users of public spaces.

Further analysis was conducted for respondents’ change in confidence in local police to deal with VAWG among residents and users of public spaces. The results of these analyses are shown in Appendix C, Section 8.3.10. There was no substantial difference in estimated effects by age or gender. There was also no clear evidence that impacts varied by the types of interventions for users of public spaces, although there was some evidence that the effects were positive and statistically significant for residents in areas with guardianship initiatives.

Additionally, and perhaps counterintuitively, confidence in local police statistically significantly decreased in NTE areas that received SSF3, relative to residential/campus areas that also received SSF3. There is no clear evidence to determine why this finding was statistically significant. However, as shown in the QuIP interviews (Section 5.2.3), there is a range of factors that may have influenced confidence in the police unrelated to SSF3 interventions, besides the presence of police having both positive and negative impacts on feelings of safety at an individual level.

5.4 Outcomes relating to training and education interventions

Although there appears to be limited evidence of SSF3 interventions having an impact on key measures, there was some evidence of influence on several outcome metrics (shorter term and direct effects of the interventions), suggesting positive influences for training beneficiaries who participated in SSF3-funded training or education programmes.

5.4.1 Attitudes towards VAWG

Training beneficiaries were asked the extent to which they agree or disagree with 6 attitudinal statements relating to VAWG. Overall, there were high levels of disagreement across all 6 statements, ranging from 79% who disagree with the statement ‘Many women mistakenly interpret innocent remarks as being sexist’ to 95% who disagree with the statements ‘women who wait weeks or months to report sexual harassment are probably lying’ and ‘a man is less responsible for rape if he is drunk or affected by drugs at the time’. As there was already broad disagreement with these statements among this audience, it may be expected that there was little scope to further influence attitudes following training or education received through SSF3 interventions. Despite this, across the 6 statements, around one-third (32% to 35%) said they disagree more, suggesting that for a substantial proportion of beneficiaries, the training or education received still had a positive influence on attitudes towards VAWG. Only a small minority (1% to 6%) said that they agreed more with these statements following education or training. Due to small base sizes among this audience, there were no statistically significant differences by demographic characteristics.

5.4.2 Influence of training on VAWG awareness, knowledge, and confidence

Nearly all (90% to 99%) training beneficiaries reported that their awareness and understanding of VAWG, knowledge and confidence had seen moderate or high improvement since taking part in the training or education activity. Specifically, almost half reported a high improvement in their awareness and understanding of VAWG (47%), and their knowledge of how to identify when women and girls could be at risk (46%), and how equipped they feel to intervene if they witness VAWG (49%). Two-fifths (42%) reported a high improvement in their knowledge of how to identify predatory behaviour. Respondents reported highest improvement in relation to their confidence to support victims of VAWG (67%) and their confidence to identify and respond to incidents of VAWG (55%). Only a minority (1% to 4%) reported no or low improvement across these statements.

5.4.3 Action taken as a result of training

Most training beneficiaries agreed that they had taken action across 4 action statements referring to applying knowledge and skills. Specifically, 84% agreed they had applied the knowledge they gained in their work, 78% agreed they had applied the skills they learned in their work, 70% agreed they had taken steps to keep themselves safe from sexual harassment or gender-based violence, and 78% agreed they had taken steps to keep others safe. Nearly all (90%) of trainees agreed they had found the training useful for their job and 92% agreed they would recommend the training to others.

5.5 Impact on police recorded neighbourhood crime

As described in Section 4.1, as well as tackling VAWG, most funded areas were also attempting to address various forms of acquisitive crime in the local area. Funded areas were generally trying to either reduce all neighbourhood crime (burglary, robbery, theft from the person and vehicle crime), or specific crime types within neighbourhood crime^{[footnote 12]}. Estimated impacts per LSOA on overall neighbourhood crime were calculated using Generalized Synthetic Control Models (see Appendix C, Section 8.3.6 for further details regarding these models). In each case, these impact estimates only include SSF3 areas with a stated focus on neighbourhood crime.

Based on this analysis, SSF3 interventions did not have a statistically significant impact on the number of neighbourhood crimes recorded in these local areas as of April to June 2022 (see Table 6 in Appendix C, Section 8.3.6 for the results, and Table 7 for area-level results). The number of crimes recorded during this 3-month period was similar to the estimated counterfactual, that is, the number expected in the absence of the Fund. This suggests there has not been a reduction in PRC attributable to SSF3 interventions.

There was only a relatively short time between the point at which SSF3 interventions started to be implemented (October 2021) and when impacts on PRC have been assessed (April to June 2022). It is possible that it will take longer for projects to impact crime levels, and so it may be too soon to observe any impact on PRC. Additionally, the effects of COVID-19 on crime may have made it less likely that SSF3 interventions would have a noticeable effect on PRC. Nationally, police recorded neighbourhood crime remained considerably lower in the year ending June 2022 than prior to the pandemic; for example, police recorded theft offences across England and Wales were 18% lower than in the year ending March 2020, and police recorded robberies were 23% lower (ONS, 2022). As such, there was the potential influence of statistical floor effects on some outcomes measures due to the data being distorted by the pandemic; that is, where crime levels were already low, there was then very little variation in the data expected within bid and comparison areas, making it more difficult to detect change.

5.6 Key findings from the impact evaluation

Perceptions of safety in public spaces

Survey evidence: The evidence from the surveys of users of public spaces and residents suggests that overall, there were no statistically significant improvements in perceptions of safety in public spaces where SSF3 interventions were delivered. Across all types of public spaces, during the day and after dark / at night, impact estimates of changes in perceptions are close to zero, indicating that any overall impacts are likely to be small. Further analysis explored whether there was a difference in perceived changes in safety in areas with interventions focused on particular public spaces. Among users of public spaces, there was some evidence that projects with interventions focused on high streets / shopping areas or on public transport were statistically significantly more effective at improving perceptions of safety in these spaces than other funded projects. Specifically, where interventions were focused on high street / shopping areas or public transport, perceptions of safety in these spaces were statistically significantly improved during the day compared to funded areas where interventions focused on all other space types.

QuIP evidence: Findings from the QuIP suggest that while higher-level impact on perceptions of safety has not occurred, SSF3 interventions have resulted in some improved feelings of safety at an individual level. Stakeholders interviewed as part of the process evaluation also shared anecdotal evidence of some impacts on perceptions of safety in public spaces, particularly as a result of increased CCTV (Section 6.5.3). Furthermore, the QuIP results help contextualise the difficulty in improving perceptions of safety by highlighting the many competing factors working to worsen perceptions of safety among the public.

Perceptions of levels of crime and confidence in local police

Survey evidence: When examining the impact on perceptions of prevalence of crime, local residents reported a statistically significant perceived reduction in levels of VAWG during the day. This was the only statistically significant finding regarding perceptions of change in crime across the 3 audiences and crime types (VAWG, ASB and acquisitive crime). There was some evidence that SSF3 projects led to perceptions of crime moving in the right direction (that is, perceptions of a reduction in VAWG and ASB crime) among users of public spaces. However, these results were not statistically significant (Section 5.1.2).

When analysing the impacts on confidence in police, and likelihood to report VAWG to the police, there was a strong, positive and statistically significant increase in training beneficiaries’ confidence in local police to deal with VAWG and in their likelihood to report to local police. There was also some evidence that SSF3 interventions led to an improvement in confidence in local police to deal with VAWG-related crimes among local residents in SSF3 areas as compared with residents in the comparison areas, showing a small yet statistically significant improvement. There was no evidence of an impact on this measure for users of public spaces. Similarly, there was no evidence of an impact on the likelihood to self-report VAWG to police among either users of public spaces or residents.

Outcomes of training and education programmes

Survey evidence: Analysis of outcomes among recipients of training or education through SSF3 suggests a positive influence on their attitudes towards VAWG, as well as improvements in awareness and understanding of VAWG, and confidence in their ability to identify and respond to incidents of VAWG, and to support victims of VAWG. Findings also reinforce the degree to which training attendees valued the training and already found ways to utilise it in their life or work. These themes also emerged during the process evaluation where stakeholders commented on direct feedback from training beneficiaries on the success and impact of courses (Section 6.5.1). This further demonstrates that SSF3 interventions have had positive effects on individuals and created change in behaviours for others.

Wider secondary evidence

Analysis of PRC on the estimated impact on overall neighbourhood crime found that SSF3 interventions did not have a statistically significant impact on the number of neighbourhood crimes.

Contextual factors

It is possible that the few impacts seen will filter through to improved feelings and perceptions of safety for the general public in the longer term. However, this does not assume that the Fund or its individual interventions will necessarily lead to larger-scale impacts over time. It is not possible to conclude this with certainty and will be a challenge given the wider context of crime and safety. As highlighted by QuIP analysis, there are complex and competing factors that worsen perceptions of safety such as personal experiences of VAWG, awareness of local or national incidents of VAWG, and perceptions that other types of crime are increasing.

Limitations on the evaluation design also reduced the ability to detect impact; for example, it was not possible to conduct a baseline survey prior to the start of intervention delivery. Furthermore, some SSF3 interventions had a short timeframe overall (4 to 5 months), with some (for example, education/training, guardianship initiatives) only running for a limited period or of relatively limited numbers. Awareness and use of SSF3 interventions among users of public spaces and residents is a key factor in this – only around half of users of public spaces and residents were aware of at least one intervention that was new or increased in the previous 6 months. Where this was the case, awareness often did not correspond with what was actually being funded in the area, that is, people noticed other interventions not funded by SSF3 or failed to notice interventions that were. Together, this evidence suggests that it will be particularly difficult to change perceptions of safety quickly and at a level to detect significant impact or outweigh the effects of negative factors.

Overall conclusion for impact evaluation

While we cannot conclude that SSF3 interventions have achieved their intended impact overall, there is evidence to suggest they have had a positive effect on individuals and at a more local level.

6. Process evaluation

This section provides findings from the qualitative data collection, including lessons and approaches to bid design, delivery processes, facilitators and barriers to delivery, perceived outcomes of SSF3 interventions, and plans for future sustainability. The data collection and analysis methods are outlined in Section 3.5 and Appendix B, Section 8.2.1.

6.1 Bid design processes

This section describes the processes used to develop funding bids, including:

• the processes and types of evidence used to identify issues and areas
• the principles which framed intervention selection
• the consultation and engagement processes with delivery partners and future beneficiaries

Overall, most funded areas reported that the bid development process was a positive experience. Many areas agreed that the Fund presented an opportunity to share ideas and strengthen collaborative working between partners and increase their understanding of VAWG issues.

However, many noted that the short time available (6 weeks) for the development of high value and complex bids was a challenge in terms of staff capacity and quality of bids, and had knock-on effects on delivery. One bid area suggested that at least 3 months should be required to allow for robust bid development. The time pressure created stress for some staff, requiring them to de-prioritise other work or to work outside their usual working hours.

Alongside this, there were some comments that the overlap across multiple funding rounds was problematic. They were still mobilising SSF2 interventions when they were required to begin work on the SSF3 bid and were then required to start the SWaN bid application before receiving an outcome on SSF3.

6.1.1 Establishing bid application teams

Bid development was typically a collaborative process, with multiple teams and organisations contributing to a variety of design activities. Bids were generally led or coordinated by local authorities (including community safety and domestic violence/VAWG teams) and/or PCC teams, police constabularies and other local police teams. Many other organisations contributed to bid development by attending regular bid meetings to discuss ideas. This included VAWG and other third-sector organisations, subject matter experts, potential delivery partners and providers, student unions, community and resident groups, businesses and business forums.

6.1.2 Utilising Home Office application guidance

Many funded areas that responded to the pre-implementation qualitative questionnaire agreed that guidance from the Home Office was easy to understand, and that they understood the bid instructions and what was expected from them. Some areas said that the process and forms had been simplified since SSF2, although others commented that the tendering processes were complicated and the online portal for submission (Jaegger portal) was confusing.

Some areas provided suggestions for improving the Home Office guidance to bidders. These included:

• providing more accurate and detailed information on timeframes for funding decisions and receipt of funds
• additional detail on the scope of interventions that were acceptable (for example, the indicative case studies provided in the SSF3 prospectus were largely physical/environmental)
• more guidance on how to complete the financial forecasting section of the bid, including detail on the purpose of the headings
• greater clarity on the foci of other upcoming funding rounds, such as SWaN, and inclusion/exclusion criteria for each, so that bid areas understood what they could apply for in different bids
• longer gaps between funding rounds, or more clear delineated staggering of rounds, which are likely to be accessed by the same organisations
• more detail on the geographic considerations for the bid

6.1.3 Utilising past experience

Many funded areas used learning from their bidding experience during SSF1 and SSF2 to inform the way they developed their SSF3 bids. Some said that familiarity with the bidding process and the information requirements facilitated better planning of the bid process, early registration on the portal, and easier identification and analysis of secondary data. Some stated that they were more realistic about what could be achieved within the delivery timeframe, while others ensured they factored in enough time for key processes like planning permission, procurement and recruitment, and invested in earlier partner engagement.

“Partnership working and insufficient upfront engagement caused difficulties in SSF1 and SSF2. Consequently, partner engagement with a realistic discussion about previous difficulties and the levels of support needed for success have significantly assisted SSF3.” (PCC’s office)

6.1.4 Identifying issues, areas and crime types

To develop their bids, funded areas triangulated evidence from multiple sources, including crime data, bid leads’ and partner organisations’ local knowledge, public surveys, and the views of beneficiary groups to identify key issues and locations. Some benchmarked their data against nearby locations or national averages to demonstrate disproportionately high levels of crime.

Local knowledge among bid leads and partners was used to supplement PRC data, which was not considered an accurate measure of prevalence of VAWG crimes due to under-reporting. New or existing resident/public surveys supplemented official data to understand public perceptions of safety. The information from surveys was used to pinpoint specific areas where members of the public, and especially women and girls, felt unsafe.

The views of specific beneficiary groups were also gathered to understand key issues and localise interventions. For example, one case study area held a meeting with councillors, key members of the community, and park users (for example, the rugby and football club and bowlers) to understand issues that needed to be addressed from their perspective, and consulted with young people on the appropriate locations for digital benches.

6.1.5 Choosing interventions

Home Office Guidance for Bidders (Home Office, 2021c) sets out information on eligible activities, stating that bidders should not restrict their bids to only ‘more traditional Safer Street Fund interventions’ (like CCTV and street lighting) (p. 8), but were ‘strongly encouraged to think creatively’ (p. 6) and invest in areas such as ‘educational products and programmes…[which] could involve focusing on attitudinal change and awareness raising’ around VAWG (p. 8).

In general terms, 3 key principles emerged as guiding intervention selection: feasibility, complementarity and breadth.

Feasibility: Some funded areas mentioned they sought interventions that could be implemented within the available funding and delivery timeframe, and organisational capabilities (albeit with funding for additional staff posts or capacity in some cases).

“My primary objective in this was to look at good ideas, good initiatives, see what we could look to deliver in a reasonable timescale.” (Police)

Complementarity: Some funded areas stated they designed their interventions to complement ongoing work under SSF2. In practice, this often meant choosing interventions aimed at VAWG attitude and behaviour change, since SSF2 interventions were more narrowly focused on environmental interventions. Some chose interventions closely aligned with other ongoing work in the VAWG sphere. Some bid areas excluded interventions related to the NTE, anticipating the upcoming SWaN application.

“The SWaN programme came out at such a perfect time I think for us, because there were so many things that we wanted to do, that didn’t fit into Safer Streets 3… We did really consider how they all tied together.” (Local authority)

Breadth: Some funded areas wanted to try to tackle VAWG from multiple angles. These bids generally included a range of interventions targeting different audiences, including the general public, professionals in specific organisations, and school or university students.

6.1.6 Stakeholder and public consultation

All funded areas conducted some form of consultation, most with a broad range of stakeholders and using a range of consultation activities. Most prevalently, areas used surveys to collect views from partner agencies, the police and members of the community to identify crime hotspots and locations where people felt unsafe. Areas also conducted focus groups with stakeholders and members of the community to identify target locations and specifically held focus groups with VAWG experts to learn about ‘what works’ to prevent VAWG crimes and improve feelings of safety. Additionally, areas utilised existing community networks such as local meetings, reached out to specific groups such as academics and the police, and sought individual feedback from members of the public through door-knocking and social media.

Case study: Consulting with stakeholders during bid development (North East of England)

One bid area was passionate about tackling VAWG crime and keen to implement behavioural change interventions. To help design the bid and decide which outcomes to focus on, the area consulted 32 VAWG organisations, representing the diversity of communities across 6 local authority areas to ensure that the concerns and needs of these local communities were heard and considered. Feedback from these consultations identified a need for more presence on public transport networks, which informed the focus of the funding. In conjunction with a youth sector organisation, the PCC’s office designed and delivered a school-based education programme focused on acceptable behaviour on public transport, as well as elements of bystander intervention and peer challenge among secondary school-aged children.

Alongside this, in consultation with transport companies, bystander training was delivered to their bus drivers and transport staff. Uptake on the course among bus drivers was not as high as they had hoped given the number of staff on the transport network. This was attributed to not enough relationship-building or communication with the bus drivers at early stages of the bid design. The PCC’s office had wanted to factor in more consultation with bus companies, but this was not possible given the short timeframes.

Stakeholders commented that if they were to deliver these interventions again, they would build in more costs for early-stage development and relationship-building with stakeholders, particularly with bus companies.

The expertise of VAWG organisations was seen as particularly important in supporting the targeting of resources on spaces, audiences and interventions. One area mentioned the significant contribution of the VAWG sector:

“They were one of the strongest voices around the table realistically, when it came to the decision making…They were a really strong voice within our ability to say we want to do legacy work [that is, educational work with longer-term impact].” (PCC’s office)

Some areas noted that consultation was helpful in challenging their assumptions. For example, consultation with a group of sex workers showed they viewed CCTV and street lighting as protective not punitive, as the bid team had assumed. Similarly, a public survey suggested that women’s feelings of safety improved with CCTV and street lighting, also against bid area expectations.

In contrast, other areas faced barriers which they felt precluded using consultation to better understand local needs and concerns. Some felt the limited timeframe of 6 weeks to be a significant barrier to meaningful and robust consultation. One area reported deciding against a survey of harder to reach women due to the time constraint. Language and cultural issues were also highlighted as significant barriers in consulting with hard-to-reach groups.

6.1.7 Partner identification and engagement

Funded areas brought in a wide variety of delivery partners, as stipulated by Home Office guidance (Home Office, 2021c). Several bid areas identified enablers which helped them to identify suitable partners: having worked well with them in the past, whether on prior SSF projects or VAWG-related work; knowing they had a good reputation; and/or through a competitive selection approach in which potential partners pitched their ideas. Some bid areas that had received SSF1/SSF2 funding stated that their own track record in successfully delivering prior SSF interventions helped them to attract partners.

Across funded areas, early engagement of suppliers and other delivery partners during the bid design stages was considered a facilitator to later successful delivery when implemented. Where funded areas engaged with relevant partners at the project design stage, they felt better placed to mobilise once the outcome of funding was released. Slow mobilisation due to procurement and supply chain issues – for example, as a result of Brexit and COVID-19 – were exacerbated where funded areas did not engage with suppliers and partners early enough.

6.2 Perceptions of the Fund

Funded areas were also asked about their views of SSF3 and what it was trying to achieve, as well as the views they heard from consulted partners and stakeholders. Bid area stakeholders had mixed responses to the SSF3 programme and its parameters. Most stakeholders welcomed the uplift in resources provided by SSF3, describing it as a ‘kick-start’ of funding, or as plugging gaps in resources for tackling VAWG and improving feelings of safety. Stakeholders in one funded area applauded the expanded focus of SSF3, the focus on VAWG, and eligibility of educational interventions, compared with the parameters of prior rounds:

“I really felt that I’ve had creative free rein to do something very innovative and very different, and I really welcome that. I think it allows innovative practice around different ways to tackle some of those really harmful views that are posed towards women.” (Third-party delivery partner)

While it did not seem to present a problem for most bid areas, some funded areas perceived a tension between the Fund’s central aim to tackle VAWG and its parameters. For example, one funded area found the requirement for bids to focus on specific geographic areas confusing, as they believed VAWG should be addressed by changes in attitudes and behaviours that are not addressed through interventions in specific geographic areas. Environmental, guardianship and support services interventions were generally more targeted at specific public spaces/geographical areas; whereas educational, communications, personal safety and more targeted support services interventions tended to target specific beneficiary groups or audiences without reference to geography.

Some areas felt that the application guidance, including the case study examples, emphasised physical interventions in public spaces which they felt sent an inappropriate message as to how VAWG should be tackled, such as by physical changes rather than by addressing the root causes of male violence (misogynistic attitudes). Relatedly, one VAWG organisation commented that interventions in public spaces would not address VAWG perpetrated by a man they know, which commonly occurs in private spaces.

“…. we gave our feedback to the Home Office … that CCTV and street lighting is not going to fix violence against women and girls, they needed to be putting money into behaviour change.” (PCC’s office)

One case study area experienced some initial reticence from VAWG organisations to contribute to bid design but noted that they were later reassured when it was clear that this funding round included education and communications interventions.

6.3 Intervention delivery processes

This section describes the delivery processes undertaken by funded areas for each of the intervention categories.

A. Environmental/physical Interventions

Almost all funded areas started the implementation of environmental/physical interventions through scoping activities and site surveys. These were used to confirm the requirements for what materials needed to be purchased, installed and upgraded, and then begin the procurement process via external agencies and approved vendors. Funded areas were also required to obtain permission and submit planning applications depending on the location of installation.

The installation of materials and upgrading infrastructure like CCTV, alarms, ANPR, and streetlighting was usually conducted by existing suppliers who had the knowledge of the equipment and target areas. Delivery partners managed the installation on site with oversight by local authority community safety teams. However, in some cases, funded areas’ internal teams scheduled work for installations and tested resources.

Some funded areas spoke about engaging the community in cleaning the areas, graffiti removal, art installations and alleygating. This was done to improve the look and feel of the area and increase community pride. Some funded areas also ensured the visibility of the intervention by investing in accompanying signages for CCTV, streetlighting and video doorbells. The physical changes to the spaces were also sometimes communicated through media outlets and local press, local councillors and community groups such as ‘Friends of the Park’.

Most of the funded areas reported no changes in the delivery of these interventions compared to their original plans. For those that did, the main changes were: an increase in the number of CCTV cameras, bins or other equipment put in place due to an area receiving additional funding; a decrease in equipment due to procurement and supply issues; and changes to the location of CCTV cameras, landscaping and graffiti removal, for instance, based on feedback from stakeholders and/or the community.

B. Education and training programmes

Mobilisation of educational training on personal safety or the safety of others also required a range of start-up activities, such as procuring training materials, planning the roll out of training, developing training resources, and deciding on the mode of delivery.

Specific training (such as VAWG training) was developed through consultations with community groups and VCS organisations. This included topics such as understanding VAWG, enabling the identification of predatory behaviour, and building police confidence to respond to these issues. In some cases, the VAWG partners undertook background work to pull together the relevant information and documentation needed for the delivery of the course.

Most of the funded areas advertised training using social media or other direct channels to maximise engagement. This was done in accordance with the spaces where the training was delivered, particularly when target audiences were mobilised to sign up for training programmes such as personal safety, understanding VAWG, and others. In areas where training was conducted in schools and universities, delivery teams and their delivery partners engaged with the authorities to obtain permissions and negotiate grant agreements.

In most areas, training providers/experts administered education and training programmes. The most common type of training delivered was bystander training, but they also delivered other training sessions on positive sexual attitudes, consent, misogyny and relationship safety. Bystander training was also delivered to groups outside of schools and universities, such as to key members of the community and NTE staff. They created the training resources and delivered them in person or online. Conversely, some bids delivering training to specific external organisations (such as venues, workers and ‘train the trainer’ programmes) typically used internal trainers and in-house materials.

Most funded areas reported no change in delivery plans for education and training interventions compared to their original plans. Of those that did, the main changes were reductions in the number of courses and/or trainees due to low engagement or COVID-19 restrictions, or additional training sessions due to additional funding received. Some funded areas had to adapt to changing timescales due to COVID-19 restrictions or related issues, such as staff sickness, recruitment or supplier issues. Other areas changed the target audience or content of training sessions during delivery due to feedback from stakeholders or trainees, or issues with suppliers.

C. Guardianship initiatives

The start-up activities for guardianship initiatives included scoping for target locations, a period of engagement with relevant providers and workers and contracting external providers.

Funded areas also spoke about partnering with external organisations providing guardianship services (for example, Street Angels and Street Pastors) or training courses for individuals to act as guardians, such as for community volunteers; for example, partnerships to deliver training to staff working in pubs, bars and clubs in the area to enable them to act as guardians by recognising and offering support to women and girls who feel unsafe.

The police were key delivery partners in guardianship initiatives focused on patrols. Where interventions involved establishing safety points or night hubs, bid leads designated responsibility to local PCCs to appoint plain-clothed police officers in the targeting of public spaces. In most cases, PCCs and local authorities typically shared oversight of these initiatives. Some bids reported training workers to act as guardians (for example, NTE workers) and engaged audiences through their employer organisations (for example, licensed premises).

Funded areas also engaged with local businesses for the Safe Space accreditation schemes that ensured the well-being of their customers and visitors. Some bid areas spoke about recruiting and training street pastors or street angels, along with the provision of equipment to volunteers such as personal alarms and torches. Guardianship initiatives are promoted to the general public through a mix of communications channels. This was accompanied by setting up signage and posters to create visibility for these initiatives and optimise their usage.

Most funded areas reported no change in their delivery plans, though where plans did change, it was largely due to additional funding or transfer of funding from a programme strand which underspent, thus allowing for additional interventions such as additional guardian training or equipment.

D. Communication campaigns

Most funded areas reported designing communications campaigns in-house through SSF3 project managers and media teams, while others commissioned external organisations with area expertise. Some areas contracted suppliers for the management and delivery of communications campaigns. In some cases, funded areas consulted the local community on the development of materials and collaborated with advisory partners (such as VAWG experts) for more specialist messaging.

“In-depth research will be conducted by the external media company to provide a full understanding of how best to deliver the messages around VAWG to specific groups.” (PCC’s office)

Funded areas used a range of communications channels to reach their target audiences. Most prevalently, this included utilising existing networks, leveraging marketing from contracted suppliers, and dissemination through direct community channels as well as social media groups. Delivery teams and delivery partners also engaged with stakeholders to gain permissions for delivery of interventions in different locations as well as established other governance mechanisms. Subsequently, funded areas reported organising local media campaigns, spanning a wide range of modes, including social media, radio, leaflets and posters. The topics of these campaigns ranged from reporting crimes, recognising inappropriate behaviours, safety and awareness, and bystander campaigns. Additionally, some funded areas followed up the campaigns with community events where community safety teams and the police could engage directly with the public.

Most funded areas’ delivery plans did not change. Where plans did change, the main reasons were changes to delivery timelines and additional events added due to community feedback and additional funding. Some funded areas also reported some changes to the mode or location of campaigns due to feedback and/or learnings from the iterative nature of the campaign.

E. Youth outreach and support services

Funded areas developed different avenues to engage with youth and the community, including mobile services, setting up community groups/hubs and organising youth volunteers. One such initiative included Youth Engagement Vehicle, which provided engagement activities to young people directly from an accessible and mobile vehicle, allowing specific spaces and communities to be targeted.

Very few funded areas delivered support services (18% of funded areas as detailed in Section 4.3). Of those that did, this included a sex worker outreach project, one-to-one support to victims with several areas including help reporting to the police, and one-to-one mentoring with school students identified as being vulnerable to harmful behaviour. Support services were almost entirely delivered by voluntary sector partners and community organisations due to their connections within the communities across different bid areas. All the funded areas reported the delivery of support interventions going according to the initial plan, except for some slight delay to delivery periods, which was accounted for in the contingency plans.

F. Personal safety initiatives

Start-up activities for apps included identification of developers, contract signing and working with creative agencies on designs.

App development was typically externally tendered to design and development agencies. In some cases, the SafeZone app was embedded within the pre-existing IT infrastructures of organisations where they were implemented, such as universities. Bid areas worked with organisations as delivery partners when targeting specific audience groups (for example, university students).

Funded areas also reported distributing personal safety equipment either following training courses or as part of communication campaigns to the general public, especially those focused on individuals most at risk of VAWG. The equipment included a wide range of personal safety tools, such as personal alarms, drink protectors and spiking kits.

Nearly all delivery plans went ahead as outlined, with only minor changes occurring due to additional funding. There were some supply chain issues which delayed the dissemination of equipment.

6.4 Conditions for successful delivery

This section presents the enablers of and barriers to delivery, as perceived by funded areas. The section opens with the conditions at the Fund level and it includes some intervention-specific examples. We drew data for this analysis from all strands of the process evaluation to identify the key enablers and barriers to successful delivery. This analysis led to the development of the ‘conditions for success’ as areas commonly discussed enablers and barriers as 2 sides of the same coin.

Figure 8: Conditions for successful delivery

A. Sufficient time for planning and delivery

Funded areas overwhelmingly felt the limited timeframe was the most prominent barrier to achieving delivery milestones, encouraging shifts in public perception of safety, and measuring impact. They felt that the ongoing impacts of contextual factors, such as COVID-19 and the global shortages and supply chain issues viewed to be worsened by Brexit, were compounded by tight timescales, and this had a major effect on their ability to deliver the programme.

Additionally, stakeholders often felt that the short delivery timelines, alongside issues with resource allocation, prevented substantial cultural shifts and change to be achieved.

“I think it would be really unrealistic to think that a short period of time, and a relatively speaking, small pot of money, would have tackled those issues.” (Local authority)

“This short time period does not give enough time to embed some of the work, especially working with young people/victims around a difficult subject, gaining trust, also not a long enough time period to measure impact.” (PCC’s office)

With more time, areas felt they could deliver more meaningful, long-lasting and impactful changes, and carry out more robust evaluation of their bids.

B. Mitigation plans against potential risks, including COVID-19

Planning contingencies for projects helped funded areas overcome barriers, particularly where they were at risk of being impacted by contextual factors such as COVID-19, bad weather and the knock-on effects of Brexit, such as global shortages and supply chain issues. In cases where delivery had to be adapted, this was often facilitated by advanced contingency planning.

Most funded areas reported that external factors did impact delivery. As in SSF1 and SSF2, COVID-19 and its knock-on effects continued to pose many barriers to delivery, which required careful contingency planning. The pandemic also meant that there was less footfall in previously busier areas (according to areas) which impacted the number of people available to engage with campaigns and use night hubs and safe spaces that were delivered during the winter of 2021 and early months of 2022. This was compounded by the winter weather, where darker and colder nights and increased rainfall also impacted footfall in public spaces. It also led to initiatives such as Street Pastors and Street Angels pausing delivery for periods of time. In some cases, areas could not mitigate against these issues, resulting in delivery elements being delayed or not being delivered at all.

Funded areas reported successful delivery of interventions in spite of these challenges when they could adapt delivery models and/or pivot to other activities.

“Despite best efforts [the Council] were unsuccessful in establishing any Neighbourhood Watches. However, efforts were diverted to other areas and [the district council] was successful in establishing a female focus group.” (Local authority)

Adapting delivery models was particularly important for education interventions, such as training sessions delivered in schools. Where areas could not adapt to online sessions, they often did not achieve target numbers of sessions delivered.

“Delivery could not take place until quarter 4 due to schools not allowing visitors in during quarter 3; one session booked for January had to be rescheduled in April due to Covid [COVID-19] outbreak at school. Another school had to drop out, due to time limit restraints.” (Local authority)

In one area, there was very low attendance at the personal safety sessions delivered by a VAWG organisation, which posed a key challenge to delivery in terms of reaching target audiences and avoiding underspend. The organisation adapted their original delivery model from 2 half-days to one-hour sessions, which were felt to be more realistic and accessible for university students. This resulted in increased attendance at the remaining sessions.

“When they were reduced to an hour, we got a lot more interest – we were getting better attendance, still not great attendance, but a lot better than it was.” (Third-sector delivery partner)

However, sometimes unavoidable barriers such as staff sickness and low team resource because of the pandemic hindered delivery of interventions, with most interventions affected in some way across bids.

“70% of project could not be supplied and installed in time due to [COVID-19] impact on staffing, global shortage of equipment and difficulty fitting work with planned road closures.” (Local authority)

Procurement and supply chain issues mostly affected areas implementing environmental and personal safety interventions. Funded areas reported that lengthy procurement processes and supply chain delays were exacerbated by staff shortages because of the pandemic and existing global shortages that were already impacted by Brexit. Some areas also reported delays in obtaining equipment due to increased demand from other SSF bids.

“Some [personal safety] items were not in stock, and we had to choose alternatives due to high demand from other Safer Streets-funded authorities and also resource shortages.” (PCC’s office)

C. Delivery by experts, including VAWG organisations

Across interventions, the input and expertise of third-party experts and professionals was recognised as a key facilitator of success. Expert delivery partners were commissioned to deliver specific interventions. Similar to SSF2, funded areas found it challenging to deliver interventions when internal delivery teams did not have the required sector knowledge and/or experience.

The third-party/external organisations onboarded by bids for their expertise ranged from specialists for training, campaign management teams, community organisations, contract vendors, and non-profit institutions. The external delivery partners such as community groups, VAWG organisations and non-profits were especially brought onto projects to design and deliver communication, educational and guardianship initiatives.

Case Study: Working with voluntary and community sector organisations (West of England)

One case study area focused on delivering education and training programs for workers and the general public and commissioned voluntary and community sector organisations (VCS) to work alongside the local Violence Reduction Unit (VRU).

The VCS having special expertise of VAWG in the context of the local area was tasked with gathering crucial evidence of transport hubs where women felt unsafe. Through their knowledge of VAWG, they aided the local VRU by developing widespread campaigns to be launched in these spaces, that effectively generated awareness among men and boys. The VCS also helped in developing the training video for police working with female sex workers as well as delivering training packages to the professionals that focused on disclosure of VAWG crimes. This multi-agency collaboration between the local VRU and VCS organisations created an opportunity to reach a larger audience as well as generate widespread impact through training and campaigns.

“I think the [bid area] VRUs are exceptional in terms of how they look after their providers, and build those relationships.” (Third-party delivery partner)

Delivery by professionals was vital to provide professional insight which informed a wide range of decisions. For example, for communications campaigns, funded areas relied on third-party organisations to design materials, identify target delivery dates to maximise publicity of campaigns, and ensure a joined-up approach to campaign launches such as releasing social media posts at the same time. Additionally, expert knowledge was required to identify the most suitable locations for physical campaign materials; for example, ensuring that areas with higher footfall were targeted for posters and leaflets. For education interventions, many funded areas utilised the expert knowledge of sexual violence experts to successfully deliver specific training courses.

Expert knowledge was also important for areas developing apps or using technology. One bid sought to develop a bespoke personal safety app for use on public transport. The third-party supplier they used was highly experienced and had previously developed many features of the app that the delivery team required. As a result, they could build in these features more easily and tailor them to the specific needs of the bid area.

“Having suppliers who are experienced, both in the work that they do and also working with police – that’s definitely helped.” (PCC’s office)

Similarly to SSF1 and SSF2, areas often commented on the benefit of having designated project management professionals to oversee SSF3 delivery in the area and ensure that projects and partners were meeting expectations. However, recruitment to these roles was often a challenge due to short timeframes to recruit suitable staff, and the short-term nature of the posts that made the roles less appealing to prospective candidates.

“The biggest barrier was getting the right people in, the roles that we’d identified a role, but we didn’t necessarily have somebody who we could second or appoint to that role.” (PCC’s office)

Funded areas also felt the expertise and guidance provided by VAWG organisations during delivery was invaluable, and bids felt that working with these organisations positively impacted their ability to deliver interventions. VAWG organisations were involved in developing and delivering training, including bystander training programmes. They were also consulted in the development of campaign and communication materials to ensure appropriate messaging and terminology in materials, delivered engagement events, and distributed personal safety equipment. In some areas, working with VAWG organisations enabled bid teams to create projects that were co-produced by victims of VAWG.

“All of the VAWG organisations mobilised quickly, using the funding allocated to meet the objectives set. There was positive communication, interaction and more importantly, co-production with experts by experience, which made these delivery partners notably the most impacting and successful of the projects.” (PCC’s office)

However, there were some challenges with communication and aligning expectations between VAWG organisations and SSF3 core delivery teams. VAWG organisations sometimes had different expectations of their role in programme delivery or felt that interventions could be delivered in alternative ways to what was planned, sometimes causing tension or confusion between them and the bid delivery teams.

There were also challenges to working with other non-VAWG organisations, particularly issues with communication, differences in ways of working, and aligning project goals. For example, some funded areas found that non-VAWG third-sector organisations wished to consider wider issues beyond VAWG, such as the cost-of-living crisis, rather than focusing specifically on the project aims.

Some funded areas also struggled with communication with external organisations. Stakeholders provided examples of cases where not knowing the outputs and timescales of delivery partners hindered overall progress. Funded areas found that working with delivery professionals was positive when there were clear roles and responsibilities established with the experts, and regular communication was maintained.

D. Effective partnership working processes

As mentioned, funded areas often utilised external/third-party delivery partners. In some cases, projects were also overseen by multiple organisations, including PCCs and local authorities, who worked collaboratively to deliver different aspects of the programme. Effective and supportive partnership working processes were therefore the most prevalent enabler of successful delivery across interventions at every stage of design and delivery. Overall, partnerships were effective when there was shared buy-in into the projects’ aims and there was a clear role for partners to play; for example, community organisations providing intelligence on local communities. On the other side, partnerships were less effective when there was a lack of time to build relationships and clarify roles and responsibilities.

This was particularly noted for interventions which required a combination of community intelligence and local delivery levers such as for the installation of CCTV.

“We worked with a lot of different agencies, because we couldn’t have delivered half of the stuff that we have done if we didn’t (onboard partners). So, I’ve done a lot of the networking and partner engagement, as we couldn’t deliver 16 projects on our own.” (Local authority)

“There was about 10 of us on that task and finish group, with different partners owning different aspects of the project. It was a really good way of spreading the load, so it wasn’t one person responsible for all of the projects in ensuring they get delivered.” (PCC’s office)

To manage teams and partnerships, funded areas reported setting up communication channels, such as regular meetings and Microsoft Teams channels, to track project progress. This ensured that project managers had a constant oversight and could track progress on the ground. The discussions were mainly around implementing action plans, ensuring delivery assurance, and maintaining relationships.

“Project management principles were used with the Safer Streets steering group to ensure that a project plan was kept up to date and actions were recorded to enable all delivery teams to be aware of outstanding actions, issues and risks.” (PCC’s office)

Some areas also established a comprehensive plan to provide team-wide project updates and action plans. In most cases, working committees and steering groups oversaw decision-making and operational processes, chaired by the project lead or chief officers. Commonly, funded areas spoke about the benefit of SSF3 in enhancing and strengthening partnership working, both between lead strategy partners (such as the police and local authority) and with external delivery partners, such as third-party providers and VCS organisations.

In particular, funded areas reported positive experiences working with VCS organisations and found them to be knowledgeable, committed to the aims of the project and easy to work with. VCS organisations commonly supported delivery of community engagement initiatives; for example, set windows for coordinated activities (‘days of action’), and guardianship initiatives such as Street Angels and Neighbourhood Watch schemes.

Case study: Working with a diverse group of partners (East Midlands of England)

One case study area worked with a wide variety of different partners on multiple interventions. This included a VAWG organisation, a university, and voluntary and community groups. The area described how their connections with voluntary and community groups in the area enabled them to extend their ParkWatch scheme in the local park (ParkWatch schemes are schemes where police and local community members work together and aim to reduce crime and anti-social behaviour in parks. ). Due to underspend in other delivery strands, Delivery Leads could utilise their existing networks and offer a local ‘Friends of the Park’ group an incentive to support ParkWatch activities and provide monthly updates to the council’s park management committee. This resulted in additional data that informed their SSF3 programme.

“For that small amount of money to the voluntary group, we’re actually going to get quite a lot of data back over a period of time.” (Local authority)

Their partnership with the local university to deliver a range of interventions was a positive but challenging experience. Though the SSF3 team had a good working relationship with the university, the bureaucracy and university governance added a layer of complexity, and ultimately some tension, to delivering interventions. These factors also contributed to further delays as decisions were required to be signed off by the university’s community safety manager, and finance and procurement systems.

“It was the complete right way to go – I think they were great partners to have on board, but it was extremely challenging because it was external to us, so we lost control in that way, and I think that was particularly challenging.” (Local authority)

Conversely, a lack of integrative team working and relationship-building with partners hindered delivery in a variety of ways. Overall, funded areas felt that SSF3 provided a shared purpose and resource to enhance partnerships, creating a legacy for future projects. Areas generally spoke about improved relationships with existing partners, and the benefit of leveraging these in the design and delivery of SSF3.

E. Visibility of interventions

Funded areas commented on the need for interventions to be clearly visible to the community to have an impact, including CCTV, guardianship initiatives and environmental improvement projects. Clear visibility (including accompanying signage for CCTV) was considered a facilitator of bringing more people to the area and receiving positive feedback on the spaces. Where interventions were regarded as not visible enough, this was considered a barrier to reach and impact. One area considered that visibility was particularly important for interventions aimed at preventing and addressing VAWG as it provides reassurance to the general public that the issue is being recognised and action is being taken by the police and local authorities.

Case study: The importance of the visibility of interventions (London)

This main focus of one case study area was the delivery of night hubs where the public could access medical attention and sexual advice, report crimes to plain-clothed police officers, and receive support when trying to get home safely. Hubs were run from a church next to a nearby underground station.

The night hubs were not felt to be hugely successful due to their location and lack of visibility to the public. The bid team felt that with more time to consult on locations for the hubs, and more lead-in time to organised delivery teams, the hubs would have been set up on a busier street and therefore be better utilised.

“It was the visibility that was the issue for us, the hub was too tucked away.” (PCC’s office)

That being said, the case study felt that the key success of their bid was greater partnership working between the bid team and the local police, which is expected to facilitate sustained closer working going forward.

Moreover, stakeholders considered that a core facilitator of making a difference for a small local community is the visibility of interventions, such as the signage accompanying new CCTV and the obvious placing of an art installation.

F. Engaging intended beneficiaries

Involving the community, including young people, in intervention consultation and delivery was also considered key to successful delivery, since it increased the likelihood that interventions would be noticed and used. In one case study, engaging community interactivity in the look, feel and movement around the spaces encouraged ongoing community maintenance of the area. In another case study, engaging young people in the design of a youth project was considered key to its subsequent impact on young people. In some bid areas, community engagement was felt to have encouraged increased awareness and use of spaces during delivery.

“We’re actually listening to what people are saying and trying to provide initiatives and resources…we wanted to make sure that we were showing that we’re actually listening to them.” (PCC’s office)

Including community groups was also crucial to guardianship initiatives, particularly to deliver ParkWatch schemes and set up Neighbourhood Watch groups.

Case study: Engaging the community when designing environmental interventions (South East of England)

One case study area focused on a suite of interventions concentrated on a popular walkway in the city. The upgrades to the walkway with CCTV, lighting and other improvements were chosen to address the perception of high crime that emerged from their surveys, even though data showed that crime was not particularly high. The walkway was also chosen because it was popularly used by an intersection of the population since it serviced university students, school children and parents, and was a route to the prison.

Stakeholders shared anecdotal feedback they had received from local communities that there was increased use of the walkway and residents reported feeling safer in the area. This was considered to be partly attributed to the interventions which encouraged active community participation in the space, either through the recruitment of volunteers to clean up the space, participation through seeing art, and interactivity through facilitative technology.

“It’s that broken window effect – if they’ve invested some time into it, they’re less likely to have that damage around it.” (Council)

Most funded areas reported their experience with community and beneficiary engagement to be positive. However, a few thought that the lack of in-person activities due to COVID-19 restrictions hindered building strong ties with beneficiaries. Most stakeholders were happy with the level of attendance and take-up of community engagement exercises. However, some funded areas faced issues engaging and recruiting prospective trainees of education interventions due to delays caused by school and university closures during the lockdowns. Some mitigation activities were implemented to boost engagement, such as re-designing training sessions to be virtually conducted and the distribution of promotional materials in the community to mobilise people to sign up for the sessions.

In one area, there was very low attendance at the personal safety sessions delivered by a VAWG organisation, which posed a key challenge to reaching target audiences and avoiding underspend. The organisation adapted their original delivery model from 2 half-days to one-hour sessions, which were felt to be more realistic and accessible for the audience of university students. This resulted in increased attendance at the remaining sessions.

6.5 Sustainability of interventions

For most interventions, funded areas had thought about plans for their continuation beyond the funding period. Stakeholders commented on sustainability measures that were in place to ensure continuation after the funding period finished. The feasibility, methods and barriers to sustainability of interventions are categorised in this section.

6.5.1 Interventions that are self-sustaining and do not require future funding

Some interventions were considered the most sustainable as they do not rely on any future funding sources. Communication campaigns and educational programmes were largely considered self-sustaining because once these materials were created, they could be re-used in future campaigns or training programmes. Materials also tended to remain available online or active on social media channels.

“We have materials for both campaigns, we can promote these when needed. Funding may be needed for targeted social media ads or printing; however, this would only be a small amount as the main cost of the campaign is creative content.” (Local authority)

For educational programmes, some areas used a train the trainer model to ensure sustainability. For example, in one case study area, a sexual violence charity provider delivered female rough sleeper training to local organisations. The area thought that this would ensure a legacy within organisations to continue the training and embed into safeguarding practices.

Funded areas had considered risks to sustainability and the mitigating actions needed to minimise these. The main risk to sustainability for educational programmes was the loss of momentum, where training is no longer being delivered in establishments and schools or being used in daily practices. A few funded areas planned to mitigate this by following up with companies and schools to encourage continuation and roll out. Staff turnover in organisations / teachers who received training was also considered a risk. Funded areas had different plans to mitigate this, with some planning to embed programmes into the curriculum of schools or making teaching packs available with lesson plans for teachers to utilise without training. A few funded areas also mentioned outdated training materials as a risk, with plans to review materials annually to mitigate this risk and ensure they remained current.

For communication campaigns, a certain amount of funding was necessary to continue campaign activity or materials in some areas. In some cases, campaigns were still ongoing and future events were already planned. One funded area secured continued funding from the OPCC who planned to continue to build on and deliver campaigns. However, some areas were hopeful to secure SSF4 funding. For communications campaigns, some funded areas considered that messages could lose impact and become out of date. To mitigate this, plans were in place to refresh messages and continue reinforcement of messages through the police and other stakeholders.

6.5.2 Interventions where future funding had been secured or where there was some longevity of SSF3, such as through continued contracted services

For some interventions, future funding had already been secured, either through alternative sources or with SSF3. These interventions were largely considered sustainable. This was the case for environmental interventions and guardianship initiatives where there were plans in place to ensure further local funding would be available after SSF3, usually either from the police or local council. In the case of guardianship initiatives, the police would continue resourcing patrols, maintaining the presence of these guardianship figures out in the community. For environmental interventions, the local council would often be taking on the upkeep of services, such as landscaping or graffiti removal. Equipment provided by environmental interventions such as CCTV or lighting were considered to be durable with a long-life span, as was equipment such as bodycams, bikes or service cars.

Often the equipment was under warranty in case of breakdown, but in some cases, future maintenance costs were being covered by SSF3. For example, in one bid area, maintenance costs had been budgeted into the project to ensure that CCTV and lighting received funding beyond the lifetime of the project. This was achieved by including a 3-year maintenance plan into the project. However, most often, future maintenance costs were expected to be covered either by the local council or the police.

“Lighting upgrades and security mirrors will be incorporated into county council regular maintenance works.” (PCC’s office)

Case study: Building in sustainability plans for interventions (South of Wales)

In one case study area, sustainability plans were in place for guardianship initiatives including digital benches and vehicles (A digital bench is a public bench that allows people to charge their phones whilst out in the community, in case they need to call for help. The benches were also designed to encourage congregation of young people in a pre-planned space that was appropriate and in a well-lit area.). The digital benches were under a maintenance plan with the council, where 10 more benches were to be installed on council ground with the council youth service funding the delivery costs. The park and leisure team plan to make these benches part of their monthly monitoring visits, ensuring they are still fit for purpose. If any update or maintenance are needed, this will be fed back to the youth service who are committed to maintaining these benches through ongoing costs within the youth service. The council fleet team will also maintain vehicles in this area using some of the SSF3 budget. Costings having been worked out on a 7-year life span of the vehicle, with the council paying a seventh of the cost to replace it per annum, plus an additional 20% of that replacement cost to cover maintenance fees.

For both environmental and guardianship interventions, in a couple of cases, interventions were being sustained through the community or volunteers. For example, in one funded area, there were plans for voluntary gardeners to be responsible for the upkeep of a community planting project. For guardianship initiatives in one funded area, the Street Pastor programme would continue through voluntary roles.

For guardianship initiatives, there was a risk that funding would not be available or secured. To mitigate this, there were plans for the council or police to prioritise these activities for the future and the work to be incorporated into business as usual. A few funded areas noted that the police could move resources or prioritise other needs. To mitigate this, ASB and crime would stay monitored, ensuring hotspot areas were patrolled and targeted.

For environmental interventions, vandalism and damage were considered a risk to sustainability; however, mitigation plans included ongoing area monitoring and having CCTV and signage in place to deter vandalism. Furthermore, for some areas, damage would be covered by the warranties. Areas identified a risk of communities losing interest and deprioritising upkeep of interventions. To prevent this from happening, some funded areas planned to have ongoing promotion of initiatives, which one area was hoping to achieve through SSF4. There is a risk to budget cuts for interventions requiring continued funding and maintenance; some funded areas had thought about this in advance, with one area using interventions less reliant on this, such as by installing LED lighting.

6.5.3 Interventions requiring future funding where source was unknown or not yet secured

Interventions where the further funding source was unknown or not yet secured were considered the least sustainable. Personal safety initiatives, support services and strategy and systems had more instances than other interventions where future funding had not yet been secured. Often bid areas were hopeful of receiving SSF4 funding to continue these interventions.

Unique to personal safety initiatives was the potential continuation of the safety apps through an annual subscription paid by users. Therefore, a few bids considered that apps would continue on that basis. Personal safety equipment was considered to have longevity and last well beyond the funding period. However, there is a risk to the breakdown or destruction of personal safety equipment. For strategy and systems, there is a risk of capacity issues with staffing and not having the same ability to continue the role to the same standard as before. For support services, there is a risk that funding will not be secured; however, some areas planned to share evaluations of services with funding commissioners and plans for youth providers to present projects to a range of commissioners for further funding.

6.6 Perceived outcomes and impact of interventions

To develop a framework for what bid areas felt they would achieve through their interventions, the evaluation gathered data from the pre-implementation qualitative questionnaire to develop programme-level Intervention Logic Models (ILMs) for each intervention type (see Appendix B, Section 8.2.3 for more detail). Funded areas were asked to describe the expected outcomes and impacts^{[footnote 13][footnote 14]}, of their interventions and the hypothesised causal links between them. We analysed the responses across intervention categories to develop logic models for each intervention type. These outputs sought to answer the question: why were these interventions chosen, and how are they expected to bring about change. The detailed interventions logic models for each intervention category are in Appendix B, Section 8.2.3.

After delivery of interventions was completed, bid areas were asked about the actual outcomes and impacts of SSF3 interventions, and how they compared to their original expectations. Overall, bid areas struggled to articulate the main changes as a result of SSF3, with many stakeholders commenting that it was too early to tell.

This section summarises bid areas’ perceptions of the outcomes and impacts of their interventions, and comments on how they align with initial expectations.

6.6.1 Knowledge and awareness

When asked about their expectations before delivery began, many bid areas delivering education programmes, communication campaigns and personal safety initiatives expected to see increased awareness of VAWG issues and crimes. They often considered this a more immediate outcome than a longer-term impact. In the case of educational programmes, some funded areas felt that there had been an improved awareness and reporting of VAWG crime as a result of training. The evidence was usually from feedback directly provided by training beneficiaries, such as through course evaluation surveys and through social media posts about the project. One bid area provided feedback from a member of the public who participated in a training programme:

“I learned about different types of sexual harassment that I was unaware of. All the video clips were really relevant for us to utilise.” (Local training participant)

The feedback from bid areas did not suggest that communications campaigns and personal safety initiatives had also led to these outcomes. However, this may be because local areas had not collected data from these beneficiaries (mostly users of public spaces). As detailed in Section 5.2.1 above, awareness of these interventions was also generally low, which may have mitigated funded areas’ ability to detect or measure their outcomes.

6.6.2 Attitudes, perceptions and feelings of safety

All funded areas expected increased feelings of safety in public spaces as a long-term impact of SSF3 interventions, noting that this is the primary aim of the Fund. Funded areas generally suggested that their interventions were helping some people to feel safer in their communities; however, the available evidence at the time to detect these impacts was limited. Bid areas often relied on anecdotes to report any change. For example, in one area, community members provided feedback online that they felt safer due to the new CCTV installation. In another bid area, users of public transport reported how comfortable and secure they felt on the metro due to the increased presence of staff.

When considering changes in attitudes and perceptions as an immediate outcome of interventions, this was a common expectation for educational programmes, such as those aimed at men and boys. Local evaluation feedback from education programmes indicated a potential improvement in attitudes and behaviours towards women and girls in bid areas delivering relevant interventions. The feedback included an improvement in understanding what misogyny is, understanding what a personal boundary is and understanding how their language and behaviour could impact others. This was evidenced through before and after questionnaires, feedback from those who had taken the courses, and feedback from teachers leading the training. Surveys conducted with training beneficiaries also showed that nearly all (90% to 99%) beneficiaries reported improvements in awareness and understanding of VAWG, their knowledge of how to identify predatory behaviour and where women and girls may be at risk, and their confidence to identify and respond to incidents and support victims of VAWG.

At the time of data collection, there was limited evidence from funded areas on the outcomes of guardianship initiatives and support services in improving attitudes and perceptions despite these generally expected outcomes of those intervention types.

6.6.3 Behaviour change and crime reduction

Funded areas expected that new or improved CCTV systems would increase identification of perpetrators as a short-term outcome and reduce crime in the area as a longer-term impact. In line with expectations, one area reported that CCTV helped to prevent acquisitive crime:

“Male seen to be stealing purses from bags, again patrols directed to male who was searched, detained and property recovered and returned to victims before they were even aware their property had been stolen.”

A few bid areas also shared that there had been a reduction in VAWG crimes and other general crimes (such as ASB) based on PRC statistics and reports direct from CCTV operators. One area reported a drop in VAWG within the first 3 months of initiatives and another reported a drop in sexually-orientated crimes. Other areas reported drops in other crime types due to CCTV:

“The improvements in the area are noticeable, and in particular from the service users in the CCTV monitoring office, they have been used to tackle ASB and crime in hotspots. One area where there is a House of Multiple Occupancy has seen an immediate reduction in ASB incidents.” (Police)

“Early days, but already seen…[a] drop in anti-social, unacceptable and criminal behaviours, and…sexual orientated crime.” (Third-party delivery partner)

Case study: Training to identify perpetrators (East Anglia)

In one case study area, VAWG training was developed and delivered to CCTV controllers and security personnel from NTE venues identified via the Security Industry Authority network, with input from charity sector organisations. Training content focused on improving identification of, and responses to, VAWG. Further VAWG awareness training videos were developed and rolled out to police officers across Cambridgeshire and Peterborough. The training aimed to increase understanding of VAWG, enable identification of predatory behaviour, and build police confidence to respond to these issues.

The bid area was happy with the training outputs that had been delivered and reported positive feedback. They felt that the training helped to raise awareness of VAWG among CCTV controllers and security personnel, and potentially assisted a CCTV controller to intervene when they identified VAWG behaviour.

The bid area felt that key to the success of the project was having a dedicated and fully resourced project manager to lead the work and resourced through the bid funding. This ensured the work was delivered and prioritised across all interventions delivered. They also felt that partnership working was another key facilitator, as it allowed for a wider reach of the training rolled out.

Funded areas expected that increased reporting would be both an immediate outcome and a longer-term impact of personal safety initiatives, such as apps. A few bid areas perceived that there had been an increase in reporting incidents since SSF3. This was taken from PRC statistics or data recorded through personal safety apps. One bid area reported that within the first 3 months of initiatives being active, more incidents were reported due to the ease of doing so through the app. A few bid areas also mentioned an increase in the use of public spaces, such as increased use in a footpath by pedestrians and families captured through CCTV.

The outcomes of personal safety apps to collect data and intelligence, and strategic and system-wide initiatives such as local data collection and research, were not readily shared by funded areas at the time of completing the questionnaire and/or interviews.

6.6.4 Additional perceived outcomes

Bid areas shared several additional outcomes which were not initially raised as expectations of different intervention categories, but which were nevertheless detected after intervention delivery had concluded. These included:

• improved partnership working; for example, in guardianship initiatives, a couple of areas mentioned improved working relationships between the police, NTE staff and street pastors; in education, one area mentioned there was improved engagement between schools and colleges as a result of delivering the in-school programmes
• cost savings for public organisations, including the NHS and police in guardianship initiatives, as there was less pressure on these resources
• preventing other unwanted crimes and behaviours such as littering

Overall, funded areas could more readily identify outcomes which were a detectable and direct result of the interventions and where data could be collected directly from known beneficiaries, such as for educational programmes. This is consistent with the Fund-level impact evaluation, which found more consistent effects of training than interventions targeting general users of public spaces and/or local residents. In the case of impacts, funded areas generally shared some anecdotal evidence, but felt that they were too early to measure or detect.

6.7 Key findings from the process evaluation

This final section summarises the key findings and lessons from the process evaluation.

6.7.1 Design and delivery of interventions

Funded areas had a mixed perception of the Fund overall and shared diverse feedback from partner organisations and stakeholders, including VAWG and VCS organisations. Some areas and stakeholders welcomed the focus of SSF3 on VAWG and a broadened scope of interventions to include attitudinal and culture change interventions. However, areas also commented on some tensions between the Fund’s aims and its parameters. This included the requirement to pinpoint interventions to specific geographical locations, and some criticism of the funding of physical/environmental interventions (such as streetlighting and CCTV), which were considered by some stakeholders as not addressing the root causes of VAWG. Some areas also decided to exclude some interventions when deciding what to deliver, including interventions aimed at tackling deep-rooted attitudes, interventions requiring long timescales and consultation processes, and interventions which would not be sustainable beyond the funding period.

The delivery of education programmes was a welcome addition to SSF3 from funded areas and stakeholders since they had not previously been in the scope of Safer Streets funding. Education programmes were largely focused on changing attitudes towards women and girls and improving awareness of VAWG issues and crimes. Communications campaigns targeting particular population groups, support services and some personal safety initiatives shared similar characteristics to education programmes in their focus on the audience group (for example, general public, victims of sexual harassment, men and boys, NTE workers and others) rather than addressing particular crime or safety issues in specific locations. These interventions also aimed to change attitudes towards women and girls and improve awareness of VAWG issues and crimes.

Expert input, including from VAWG and voluntary and community sector (VCS) organisations, was particularly important for education programmes, communications campaigns, support services and some personal safety initiatives to develop messaging, and resources for skilled support work. There was also a greater requirement for marketing and engagement with beneficiaries to encourage notice and take-up of these types of interventions. Since these interventions were more reliant on working with external delivery partners with specialised expertise, early partner engagement at the bid stage is strongly recommended. These interventions were also commonly cited as being impacted by short delivery timescales. A common view across delivery stakeholders was that embedding effective interventions aimed at key groups and addressing attitudes and behaviour change requires sufficient time for partner and beneficiary engagement, and for outcomes to be realised and measurable in an evaluation.

Based on anecdotal evidence from stakeholders, education programmes and communications campaigns may lead to improved awareness of VAWG issues and how to intervene, and improved attitudes and behaviours towards women and girls. Some interesting additional outcomes also emerged, such as relationship-building between women attending training courses and improved partnerships with organisations where training was delivered including schools and NTE venues.

Environmental interventions, along with guardianship initiatives, were more focused on tackling issues in particular crime hotspots rather than on particular population groups. Generally, these interventions were expected to increase feelings of safety in public spaces, increase the identification and deterrence of perpetrators and consequently reduce crimes. These interventions required rigorous scoping phases to pinpoint locations. Effective partnership working and early engagement with suppliers is considered crucial to establish communication channels between organisations and teams and enable decisions and changes to be approved efficiently. The impact of these environmental interventions and guardianship initiatives particularly requires that interventions are clearly visible to the public. It is recommended that prospective bidders should incorporate effective scoping and location planning and the necessary signage and advertisements to ensure that the interventions are fully visible.

6.7.2 Perceived outcomes of interventions

Funded areas were able to share some perceived early outcomes of their interventions. Evidence sources were most commonly anecdotal feedback to bid areas, comments on social media, and discreet evaluation exercises following activities. Common outcomes reported by bid areas were an increased awareness of VAWG issues/crimes, and how to intervene and improved attitudes and behaviour towards women and girls (a result of education programmes specifically). While bids generally felt that impacts were too early to measure, some bids did report some evidence of increases in reporting and a reduction in VAWG crimes identified from self-monitoring of local police data. In areas where bids had conducted post-implementation surveys of communities, some reported improved feelings of safety in public spaces.

There were some reports of improved perceptions of public spaces (corroborated by some improved perceptions of areas in the impact evaluation) and the community feeling more ownership of public spaces due to the presence of guardians in particular. A few areas also reported enhanced links between local authorities and PCCs and VCS organisations, some cost savings for the police and NHS through the employment of public guardians, and potentially some reduction in other crime types as well, such as ASB (although not supported by the quantitative impact evidence).

6.7.3 Sustainability of interventions

Considering the sustainability of interventions beyond the funding period, education programmes and communications campaigns were generally considered the most sustainable intervention types where resources and materials could be re-used and re-circulated. Environmental interventions and guardianship initiatives were more commonly considered to require further funding from departmental budgets in the local authority and PCC to be sustainable.

7. Summary of key findings, implications and recommendations

This final section brings together the overall findings from all elements of the evaluation and summarises the implications of this and recommendations for future funding.

7.1 Summary of findings relating to the impact of SSF3

Overall, there was no evidence of statistically significant impacts from the delivery of SSF3 on improving perceptions and feelings of safety in public spaces.

However, there is evidence that the funding was associated with emerging positive and statistically significant impacts on:

• perceptions of a reduction in VAWG during the day, among local residents
• confidence in local police to deal with incidents of VAWG among residents in areas with guardianship initiatives
• confidence in the local police to deal with incidents of VAWG among local residents and training beneficiaries
• likelihood to report VAWG to police among training beneficiaries

Overall, there was no statistically significant evidence of impacts on perceptions of the amount of acquisitive crime or ASB in the local area. There was some evidence that SSF3 projects led to perceptions of crime moving in the right direction (that is, perceptions of a reduction in VAWG and ASB crime) among users of public spaces. However, these results were not statistically significant (Section 5.1.2).

Further analysis explored whether there was a difference in perceived changes in safety in areas with interventions focused on particular public spaces. Among users of public spaces, there was some evidence that projects with interventions specifically focused on high streets / shopping areas or on public transport were statistically significantly more effective at improving perceptions of safety in these spaces than other funded projects. Specifically, where interventions focused on high street / shopping areas or public transport, perceptions of safety in these spaces were statistically significantly improved during the day compared to funded areas where interventions focused on all other space types.

Measures of a positive influence on wider outcomes were particularly noted among direct beneficiaries of the Fund, who had participated in education or training. Analysis of outcomes among direct recipients of SSF3-funded training or education initiatives also suggested:

• a positive influence on trainees’ attitudes towards VAWG
• improvements in awareness and understanding of VAWG
• confidence in their ability to identify and respond to incidents of VAWG
• to support victims of VAWG crime

Further, among users of public spaces and residents, QuIP interview analysis showed that some SSF3 interventions were directly linked to improved perceptions of safety for some interviewees, suggesting the mechanisms themselves are effective for some at an individual level. Where QuIP interviewees had reported improved perceptions of safety, some linked these to known SSF3 interventions such as increased CCTV, improving the look and feel of public spaces, through clearance such as cutting back overgrown areas/trees, and increased refuse collection / litter picking, and the presence of street wardens/guardians.

It is possible that these impacts will filter through to improved feelings and perceptions of safety for the general public longer term; however, this is currently uncertain and will be a challenge given the wider context of crime and safety.

Impact is challenging within the scope of the Fund and its delivery

QuIP analysis highlighted some of the challenges to realising impact within the scope and reach of SSF3 interventions, and within the delivery and evaluation period due to the complex factors that influence perceptions of safety. Competing influences that are worsening perceptions of safety include personal experiences of VAWG, awareness of local or national incidents of VAWG, and perceptions that other types of crime are increasing. While QuIP analysis showed that some SSF3 interventions were directly linked to improved perceptions of safety for some interviewees at an individual level, these competing factors make changing perceptions of safety particularly difficult; it will require time and sustained effort to shift such perceptions within this complex landscape.

Awareness and use of SSF3 interventions among users of public spaces and residents is a key factor in allowing impact to occur: only around three-fifths of users of public spaces (58%) and less than half of residents (44%) were aware of at least one intervention that was new or increased in the previous 6 months. Where this was the case, awareness with what was actually being funded in the area – that is, people noticed other interventions not funded by SSF3 or failed to notice interventions that were. Together, this suggests that it will be particularly difficult to change perceptions of safety quickly and at a level to detect significant impact or outweigh the effects of negative factors.

Overall, most funded areas were happy with what they achieved within the delivery period, despite some challenges. Stakeholders found it easier to overcome barriers and deliver interventions within the funding period when they had been involved in previous rounds of Safer Streets, utilised existing networks to deliver interventions, and had experienced teams that worked together effectively. Despite this, bid teams largely felt it was too early to see a major impact (Section 6.5). Bid teams also reflected concerns about the design of SSF3 and emphasis on public spaces, which was seen as incompatible with changing attitudes and behaviours relating to VAWG. Where the design seemed to emphasise environmental interventions in specific spaces, many felt focus needed to be more in attitudinal and behavioural change (particularly amongst boys and men). Furthermore, bid teams felt the timeline was unrealistic to achieve significant change for VAWG. There was a perceived need to calibrate expectations concerning the extent of change that could be achieved in a short period of delivery and provide a longer timeline to allow for more impactful interventions.

Sustained interventions will help achieve longer-term change, which is reliant on areas to continue this work over a longer period. This is supported by the many SSF3 bids that put in place interventions designed to be sustained beyond the lifetime of the funding. As well as environmental interventions such a CCTV and lighting, some interventions, such as those involving technology, apps, communications, or education and training, could continue with limited ongoing cost, supporting longer-term change.

However, the impact is not guaranteed, even with time. As the QuIP showed, there are many competing factors (for example, personal experiences, awareness of local or national incidents of VAWG, perceptions that other types of crime are increasing) that – if unchanged – will hinder or negate any impact achieved by SSF3 interventions.

7.2 Summary of lessons learned related to the design and delivery of SSF3 interventions

7.2.1 Lessons learned on bid design for prospective bidders

SSF3 bidding teams generally welcomed the Fund as an uplift in resources. Some bid areas also commented on the benefits of the Fund to plug gaps in addressing VAWG not afforded by previous rounds. There is evidence from bid areas that designing funding bids for SSF3 interventions was a collaborative process involving lead bidding organisations (local authorities and PCCs) and several stakeholders in the public, private and VCS sectors representing experts, delivery partners and future beneficiaries. A wide range of data sources were used to develop bids, including local police data, public perceptions collected in public surveys and the views of local organisations. Bid areas which had previous experience of delivering SSF1- and SSF2-funded interventions shared how these experiences improved their approaches to planning, partner engagement and the assessment of the feasibility of interventions. Overall, bid areas selected their interventions based on their feasibility (implementable within funding period), complementarity (complemented pre-existing provision and services in the area), and breadth (tackling issues with multiple intervention types).

Reflecting on the lessons learned during the application stage, as outlined in Section 6.1, bid areas also shared their recommendations to future bidders of similar funding programmes on how to approach bid design. The summarised recommendations for prospective bidders are:

• consult broadly with a wide variety of stakeholder groups to collect a wide set of perspectives on the areas, issues and population groups to be targeted and the most appropriate interventions to employ
• consult with intended beneficiaries in the community such as women and girls using public spaces, local residents and young people
• consult with VAWG organisations – if funding is aimed at tackling VAWG – to learn from their specialist expertise and sound understanding of the needs of women and girls
• ensure that the ideas of consultee organisations are considered in bid design
• identify and engage prospective partners early in the bid design stage to allow time to get good buy-in and build strong relationships
• do as much upfront planning and preparation as possible at the bid design stage, including preparing internal teams for administrative processes and designating project leads to facilitate faster mobilisation once awards are announced
• consider what can realistically be delivered in the given timeframes for delivery and spending

7.2.2 Lessons learned on delivery for practitioners

Delivery of interventions was generally considered to have gone well. The processes to facilitate pre-implementation and start-up included setting up project management processes, recruiting to roles and galvanising team members and engaging third-party delivery partners. After this, mobilisation and implementation included ensuring that delivery partners and delivery team members were clear on aims, expectations and timescales, onboarding partners and establishing contracts, and engaging beneficiaries of interventions.

Bid areas also shared that several contextual factors impacted delivery. The ongoing impacts of COVID-19 posed many barriers to delivery, including the postponement or re-pivoting of planned in-person activities such as street walks and training sessions. The pandemic also meant there was less footfall in previously busier areas, which impacted the number of people available to engage with campaigns and use night hubs and safe spaces. Furthermore, supply chain issues (due to both Brexit and COVID-19) caused issues for the procurement of materials such as those required for the installation of streetlighting and CCTV.

Reflecting on the perceived enablers and barriers, bid areas provided several conditions of success to inform future delivery of similar interventions:

• with more time, bid teams felt they could have delivered more meaningful, long-lasting and impactful changes, and carry out more robust evaluation of the outcomes and impacts
• working with experts and delivery professionals was positive when relationships were developed early on, and the expertise and guidance provided by VAWG organisations was particularly invaluable
• partnerships were most effective when there were clear roles and responsibilities established, and communication channels between bid areas and partners were maintained throughout delivery
• careful contingency planning and responsive delivery were considered important for mitigating against contextual barriers
• the inclusion of the community in intervention design and delivery increased the likelihood that interventions would be noticed and used in the longer term
• clear visibility of interventions targeting the general public was considered a facilitator to having a greater impact on users of public spaces; where not considered visible enough, bid areas considered this to be a barrier to reach and impact

7.3 Overall recommendations to the Home Office for future funding rounds – design, delivery and evaluation

Across all findings, a clear set of implications for the Home Office for the design of future funding rounds has emerged. Some implications are based on the evidence and our recommendations on how to mitigate or enhance results in the future, while others are recommendations from bid teams from their experiences of delivering SSF3 and other funds.

Fund design

Focus on increasing scale, duration and sustainability. Achieving impact will be difficult, but early evidence found in this evaluation suggests the interventions are having a positive effect which, over time, could create positive change in perceptions of safety and overall incidence of crime. However, future rounds need to focus on scaling up the dosage, duration and sustainability of interventions to ensure they have sufficient scale and longevity to produce lasting change. Providing a longer delivery timeframe is needed to enable interventions to address the systemic root causes of VAWG through changing awareness, attitudes and lasting culture change.

Increase continuity of interventions between places or across rounds. This could include enabling bid areas to work together on large-scale initiatives, and/or combine funding from multiple rounds to upscale initiatives over time.

Consider how the Fund’s parameters might reflect its aims. In this case, this would include recognising that not all types of interventions eligible for funding (for example, educational interventions designed to shift attitudes and behaviours) would require identification of a specific public space in a defined geography.

Ensure there are gaps between funding rounds, or more clear delineated staggering of rounds which are likely to be accessed by the same organisations. Bid areas reported overloading staff tasked with writing multiple bids simultaneously or in quick succession.

Provide clear and timely communications on what future funding rounds will fund. Bid areas reported some confusion on the different funding parameters between SSF3 and SWaN, which led to them omitting interventions from the SSF3 bid, which they incorrectly believed would be eligible for SWaN funding. Communication on what kinds of interventions are eligible under each round of funding is important.

Bid design

Extend the bidding window. Consultation, engagement and finding the right partners are crucial elements to bid design but are time intensive. The 6-week bidding window (3 June to 15 July 2021) for SSF3 limited some areas from engaging in proper consultation and/or in exploring potential new partnerships that would have benefitted their bid (Home Office, 2021b).

Provide more accurate and detailed information on timeframes for the release of funding decisions and receipt of funds. This is so bid areas can plan accordingly and provide more guidance on how to complete the financial forecasting section of the bid, which some bid areas found difficult to complete.

Delivery

Extend the delivery period. Many interventions required onboarding external delivery partners, consulting the public and audience groups, running recruitment and procurement processes and advertising/marketing interventions, which bid areas considered required more time than was provided to do effectively.

Provide guidance on how bid areas should plan for the sustainability of their interventions, and tailor this to intervention type. Bid areas paid considerable attention to sustainability in the design and delivery of their interventions, and while some intervention types were considered to have a natural legacy beyond the funding period, many bid areas were unsure how they could continue the interventions which required specific ongoing resources (for example, support services).

Evaluation

Focus future evaluation on the short-term outcomes which can be directly traced back to interventions and will demonstrate effects in the absence of impact, which will take considerable time to change and is subject to other influences. For example, given the timing of evaluation, it is more reasonable to expect to see effects of training on the knowledge, understanding and attitudes of those who receive the training than on wider perceptions of safety among members of the general public. The latter will likely take considerable time to change and is subject to many other influences.

Consider the funding period as it relates to the evaluation. Any future funding round should be clear on what impact can reasonably be expected given the delivery period and subsequent time for evaluation. Build in more time at the beginning to ensure sufficient time for baseline data collection, as well as after the end of the funding period, to measure change at the most appropriate time. This approach could support measurement of outcomes and longer-term impacts of future funded interventions.

8. Appendices

8.1 Appendix A: Summary of SSF3 area activity

Intervention categories were broken down into a set of specific types within them (23 types in total). Table 1 summarises the interventions and their sub-types and provides data on the number and percentage of bids delivering interventions for each. Since bid areas also provided an estimate cost for each of their interventions, the table also shows the percentage of expected spend for that sub-category compared with the category overall, and the percentage of spend on that sub-category across the Fund.

Looking at the prevalence of different types of interventions, the most common interventions delivered across bids were CCTV (74% of bids), education programmes and communication campaigns to change attitudes towards VAWG (both 53%), and streetlighting (51%); these also reflected the interventions with the greatest proportion of Fund spend across the projects.

Table 1: Summary of SSF3 interventions and expected spend

	#of bids	% bids	% category spend	% Fund spend
Environmental / physical (91% of bids) Interventions focusing on the physical features of an environment such as the lighting, CCTV cameras, signage, and look and feel of public spaces				45%
Streetlighting	29	51%	23%	10%
CCTV	42	74%	49%	22%
Clearance Interventions improving visibility in spaces/walkways (e.g. trimming hedges)	16	28%	5%	2%
Signage Interventions adding or increasing signage and maps	11	19%	2%	1%
Design and planning Interventions focusing on town planning, urban design and architecture to design of safer routes and spaces	9	18%	3%	1%
Area look and feel Interventions focusing on making a space more appealing and presentable	20	33%	12%	5%
Other	10	18%	5%	2%
Educational programmes (88% of bids) Discreet education and training programmes that are targeted and time limited (e.g. training sessions for male university students on positive sexual attitudes)				19%
Personal safety Education and training focusing on protecting yourself and avoiding danger	19	33%	11%	2%
Attitudes and perceptions Education and training focusing on changing people’s attitudes and perceptions of VAWG (e.g. educating boys on consent)	30	53%	47%	9%
Professionals in services Education and training focusing on improving access and delivery of services for victims of VAWG (e.g. training for support workers)	9	16%	8%	2%
Bystander training Training specifically focusing on supporting bystanders to know when and how to intervene and challenge VAWG in public spaces	27	47%	28%	5%
Other	3	5%	5%	1%
Communications and campaigns (79% of bids) Interventions promoting key messages on welfare, safety and VAWG				10%
Public attitudes Communications and campaigns focusing on changing attitudes and perceptions (e.g. campaigns challenging misogyny)	31	53%	75%	8%
Signposting to services Communications and campaigns focusing on signposting victims and others to services and support networks	14	26%	19%	2%
Bystander campaigns Campaigns specifically focusing on supporting bystanders to know when and how to intervene and challenge VAWG in public spaces	7	12%	5%	<1%
Other	3	5%	2%	<1%
Strategic and system-wide (72% of bids) Interventions focusing on services and systems supporting safety and preventing VAWG				8%
Leadership roles Employment of individuals to oversee a number of interventions (e.g. project manager)	27	47%	49%	4%
Project costs General project costs (not role specific)	15	26%	23%	2%
Research / analysis	17	30%	25%	2%
Other	3	5%	3%	<1%
Guardianship initiatives (58% of bids) Interventions increasing/employing individuals to promote safety and welfare in public spaces				8%
Taxi marshals Interventions recruiting/employing individuals to support transport to and from public spaces via taxis	4	7%	4%	0%
Public guardians Interventions recruiting/employing individuals to be present within public streets to promote and support safety	18	32%	36%	3%
Equipment for public guardians Interventions providing equipment/resources to individuals fulfilling public guardian roles (incl. police)	9	16%	15%	1%
Safety points Interventions providing a designated point/area in a public space where individuals can receive help/support and access resources	15	23%	39%	3%
Other	3	5%	6%	<1%
Personal safety (40% of bids) Interventions supporting women and girls who may at risk of VAWG in public spaces				6%
Apps	16	28%	79%	5%
Equipment (e.g. alarms, whistles, etc.)	12	21%	16%	1%
Other	5	9%	5%	<1%
Support Services (18% of bids) Interventions providing support or outreach to vulnerable audiences and/or victims				2%
Youth work Interventions providing youth work to vulnerable young people	5	9%	52%	1%
Victim support Interventions providing specialist support to victims of VAWG	5	9%	48%	1%

8.2 Appendix B: Process evaluation methodology

The process evaluation aimed to explore the experiences of local bid teams and their journeys from bid development, planning and implementation, to delivery and early outcome measurement of their interventions. Verian used a mix of qualitative research methods at 2 points in the delivery of SSF3 interventions: at the start of delivery (baseline data collection) and once the interventions had concluded. The 2 qualitative questionnaires, at baseline and follow-up, sought to provide a big picture understanding of delivery at the Fund level. These questionnaires provide a Fund-level perspective of implementation experiences and intervention logic models (ILMs) for SSF3 interventions synthesised across the Fund. Case studies supplemented this to provide a more local-level picture of how the overarching themes of implementation experience and intervention logic were realised in individual cases.

8.2.1 Data collection

Qualitative questionnaires (December to February 2021 and March to June 2022)

Stakeholders in funded areas were asked to complete 2 qualitative questionnaires: a baseline questionnaire to capture data at the start of intervention implementation and a follow-up questionnaire to capture data at the final stages, or after the delivery, of interventions.

The purpose of the pre-implementation questionnaire was to collect data from the bid areas on their aims and expectations for the interventions they were about to implement or were in early stages of implementing. The qualitative questionnaire was sent to bid areas to complete in December 2021, with fieldwork closing in February 2022. There were 48 responses (out of 57 bids). A core output from this task was creating the ILMs^{[footnote 15]}.

The post-implementation questionnaire was the final point of data collection for the process evaluation and was conducted after delivery was complete. The purpose was to collect data from the bid areas on how the interventions had been implemented and delivered in reality, including the actual outputs, outcomes and impacts (as far as was measurable). It also explored bid teams’ thoughts on their successes, challenges and lessons learned. The qualitative questionnaire was sent to bid areas in May 2022 for completion in June 2022. There were 45 responses (out of 57 bids) completed by bid areas.

Case study research (March to June 2022)

The case study research aimed to explore the individual bid-level detail for a select number of bid areas, in terms of their setup, implementation and monitoring of interventions, and to explore to what extent the ILMs were being realised in practice within those cases. Case studies were sampled via a purposive sampling approach to ensure a mix of cases across intervention types, breadth of focus, targeted outcomes, targeted population groups, lead organisation, overall bid cost, approaches to consultation, and whether the area had previously received SSF3. The case study fieldwork comprised 3 core activities: a document review; case study interviews; and case study focus groups in some areas. A total of 27 interviews and 5 focus groups were conducted across the case study areas between March to June 2022.

A total of 8 case studies were selected: 6 for breadth of exploration across a range of interventions, public spaces and audience groups being targeted in that area; and 2 because they had narrow, discreet scopes (in one case, one intervention type was delivered; in the other, one public area was targeted).

Document review: Each of the case study leads conducted an initial document review prior to further data collection. This included a detailed review of the bid document from the area, the responses to the pre-implementation qualitative questionnaire, and any local strategies or plans for the delivery of SSF3 provided by the case study area. This phase informed the purposive sampling for interviews in each case study area and the tailoring of interview discussion guides.

Interviews: 3 or 4 interviews were conducted in each of the 6 ‘main’ case study areas, and up to 3 interviews were conducted in each of the 2 smaller case studies. The sample in each area always included the Strategic Lead in the case study area, who was the main decision-maker for SSF3 oversight and delivery in the local area. Interviews with Strategic Leads focused on the process of bid design and rationale for selecting interventions, expectations for longer-term impact, design and delivery considerations and sustainability of interventions. A purposive sample of Delivery Leads was also interviewed based on the recommendations of the Strategic Lead. The interviews with the Delivery Leads focused on activities for mobilisation and start-up, day-to-day management processes and evidence of immediate outcomes and early impacts.

Focus groups: Focus group participants were sampled to meet one of 2 criteria: participants were members of the local community (for example, residents or workers in the local area); or the participants were beneficiaries of interventions (for example, attendees of a training programme). The focus groups explored this audience’s views on the design and delivery process, as well as any emerging or expected impacts.

8.2.2 Intervention choice rationale

Bid areas articulated their specific rationale for intervention selection based on the crime and safety issues in their area which they aimed to address.

A. Educational interventions

The main problem which educational interventions sought to address was the lack of understanding, and sometimes confusion, among some target groups (such as professionals in public services or NTE workers, students, and bystanders in public spaces) around what constitutes a VAWG offence and how to recognise risky behaviour, as well as how to challenge and/or report VAWG.

Bid areas that choose to deliver training programmes in schools cited a variety of reasons for their choice. One bid area reported that work in schools was important because a recent Ofsted report found that children:

“…often don’t see the point of challenging or reporting harmful behaviour [towards women and girls] because it is viewed as ‘normalised experiences’.” (PCC’s office)

Education programmes were therefore considered a way to challenge negative attitudes and normalisation of VAWG. Another bid area chose to work in schools to tackle low self-worth among potential perpetrators, based on evidence that this can help to reduce violent behaviour.

Furthermore, some training programmes aimed at professionals were considered an impactful way of reaching female population groups who were at particular risk of VAWG but may have been hard to reach for support services interventions (such as rough sleepers and sex workers).

B. Environmental interventions

Environmental interventions were intended primarily as preventative measures, to deter crime, and, in the case of CCTV, to identify and catch offenders. They were also intended to address low perceptions of safety in public spaces and low use by the public of some areas, even though, in some cases, these were not crime hotspots.

C. Communications campaigns

The main line of reasoning behind communications and campaign interventions was the need to affect a cultural shift in attitudes to VAWG among the general public via large-scale communication campaigns to reach a large number of people:

“Repeated exposure to the media messages via a variety of methods will increase the likelihood of the message being seen by a large number of the population, and the message retained by the audience.” (PCC’s office)

Some campaigns targeted men specifically. In one bid area, the communication campaign was addressing the issue that men may not understand what they can do to help women feel safer. Another was based on men’s attitudes and behaviours needing to change in order to reduce VAWG. A secondary rationale for communications campaigns was to deter crime by letting offenders know that the authorities were taking measures against VAWG, or, in one case, to send ‘subliminal messages’ to potential offenders, via posters with images of eyes advising of CCTV.

D. Guardianship interventions

Guardianship initiatives were intended primarily to deter VAWG crime and address poor feelings of safety for women and girls while in the NTE by providing space, people or places to turn to when in need. However, the specific aims of these differed significantly across bids, with 2 aims the most common. Some bid areas also sought to address a prevalent low confidence in the police and taxi drivers amongst women; others aimed to tackle inappropriate behaviour towards women in the NTE, for instance among taxi drivers and staff at NTE venues.

E. Personal safety interventions

Personal safety initiatives aimed to tackle high crime and/or poor feelings of safety among women and girls. One bid area wanted to tackle limited use of personal safety equipment (such as alarms and whistles) by women and girls by ‘normalising’ their use. Apps were proposed to tackle low awareness of VAWG and under-reporting of VAWG incidents by victims and bystanders, as well as addressing a gap in personal safety advice, particularly for students.

F. Support services

The key problem which these interventions sought to address were low perceptions of safety among women and girls and young people due to low use of public areas. They also sought to address under-reporting of VAWG incidents, such as via public phone/text reporting services, and by supporting the reporting of stalking or harassment in public spaces by victims through independent domestic violence advisors.

G. Strategic and system-wide interventions

The rationale for further research and analysis was to fill gaps in understanding and evidence around VAWG to inform current VAWG strategy or future VAWG programmes; for example, development of a Safer Parks Standard to help the council design safer parks. It also included the need to enhance professional knowledge related to VAWG crime by engaging with local communities to better understand their needs and priorities. VAWG leadership roles were intended to ensure more effective responses to VAWG.

8.2.3 Intervention logic models

The ILMs are organised by which activities and outputs lead to the most common outcomes and impacts. This is illustrated by the coloured outlined boxes throughout the ILMs.

Figure 9: Education logic model

Figure 10: Environmental logic model

Figure 11: Communications and campaigns logic model

Figure 12: Guardianship initiatives logic model

Figure 13: Personal safety logic model

Figure 14: Support services logic model

Figure 15: Strategy and systems logic model

8.3 Appendix C: Impact evaluation

8.3.1 Scoping phase methodology

Verian began the evaluation with a detailed scoping and evaluation design phase. The scoping phase aimed to establish a detailed understanding of the bid areas and their projects, target crimes, target audiences, and the intended outcomes and mechanisms for change. This phase included the following activities:

Document and bid review: A review of all successful bids to determine:

1. How VAWG is measured.
2. Crimes, audiences and spaces being targeted.
3. Types of interventions being undertaken.
4. The expected outcomes and impact projects hoped to achieve.
5. The governance, monitoring and evaluation arrangements planned for implementation.

Some outcomes of this bid analysis are presented in Section 4.1 of this report.

Stakeholder interviews: 17 interviews with stakeholders (10) and VAWG experts (7) were conducted to understand the expected outcomes and impact of SSF3 in greater detail, considerations and concerns relating to the evaluation design, and the needs and expectations for evaluation outputs and outcomes. Expert stakeholders included individuals from the Home Office, ONS, police, academics from universities and VAWG charities.

Data scoping: Informed by expert and stakeholder interviews, to identify and assess potential administrative and other data sources, including police reported crime data, that could be useful to understand funded areas, develop counterfactuals and/or be used for impact analysis.

Analysis and mapping: Following the bid review and interviews, the above components were pulled together to consolidate into a summary of bid areas and interventions, evaluation requirements and expectations, and considerations/implications for the final evaluation design. This was used to understand the locations and scale of interventions and determine an appropriate evaluation design for the process and impact evaluations.

Design workshop and report: The proposed design was presented to the Home Office for discussion. The final approach was then developed into a design report, detailing the full design of the evaluation.

8.3.2 Survey methodology and demographic breakdown of respondents

Users of public spaces (face-to-face survey):

The face-to-face survey was designed to capture the impact of public-facing initiatives that were likely to be seen or used by both local and transient populations in public spaces, such as high streets / shopping areas, public parks or other green spaces, and public transportation stations.

Sample and counterfactual selection:

A total of 16 bids were identified as delivering public-facing initiatives in scope for face-to-face survey data collection. These bid areas were asked to provide details on the exact locations where their interventions would be most visible/accessible to users of public spaces to identify the spots for researcher intercepts.

A further 13 comparison areas were identified. Suggestions of comparison areas were, gleaned from bid areas for similar areas to collect comparison data, relying on local insight to ensure that the set of comparison areas was reasonably similar to the set of SSF3 areas. The suitability of the comparison areas was checked against several data sources, included police incidents data, deprivation statistics, and data from the Department for Transport.

Fieldwork summary:

Fieldwork was conducted from 24 March 2022 to 13 April 2022, and then resumed from 6 to 31 May 2022 after a pause due to the purdah restrictions.

The survey took no longer than 15 minutes to complete. Shifts took place throughout the week, starting from 9am and running to 9pm latest in all areas.

The final achieved sample size was n=1,684 overall, comprising n=1,016 females and n=668 males. There were n=1,084 respondents from SSF3 areas and n=600 from comparison areas. The survey aimed to capture responses from a broad range of individuals who were using the public spaces. Table 2 outlines the demographic breakdown of respondents in funded areas.

Table 2: Users of public spaces survey respondents demographic breakdown

	Total n=1,084	n	%
Gender	Men	541	49.9
	Women	543	50.1
Age	16-24	386	35.6
	25-34	251	23.1
	35-49	204	18.8
	50-54	43	4.0
	55-59	57	5.3
	60-64	57	5.2
	65-74	61	5.6
	75 or over	26	2.4
Education	A postgraduate degree	136	12.9
	An undergraduate degree	247	23.3
	A-Levels, O Levels, GCSEs	443	41.9
	Any other qualifications (e.g. BTEC, Diplomas, Trade Apprenticeships)	135	12.7
	No qualifications	98	9.2
Ethnicity	Asian or Asian British	129	11.9
	Black, black British, Caribbean or African	85	7.9
	Mixed or multiple ethnic groups	40	3.6
	White	814	75.1
	Other ethnic group	18	1.5
Disability	Yes, affects a lot	85	37.9
	Yes, affects a little	96	42.8
	Not at all	39	17.4

Notes:

1. Due to data weighting and/or missing data, sub-group totals may not total exactly the overall weighted base.

Local residents (Address-based online survey, ABOS)

The ABOS survey was designed to capture the impact of public-facing initiatives that were intending to reach and be relevant to local residential communities in funded areas.

Sample and counterfactual selection:

A total of 49 bids were identified as delivering public-facing initiatives in scope for ABOS survey data collection. A sample of 15,000 households in areas covered by SSF3 projects was drawn. Letters were sent to those sampled addresses, inviting resident adults to complete an online survey. Households received 2 separate reminders and some households were sent paper versions of the questionnaire.

A sample of 5,000 addresses comparison addresses was drawn. Households received an invitation by post to complete the survey online, 2 reminder letters, and some received paper versions of the questionnaire. The comparison addresses were selected using a form of propensity score matching to ensure that the comparison sample was similar to SSF3 areas. Areas were matched on a range of area-level variables: the demographics of the local resident population as recorded in the Census, deprivation, rural/urban classification, and PRC for violent and sexual assault.

Fieldwork summary:

Fieldwork was conducted from 16 May 2022 to 6 July 2022.

The final achieved sample size was n=5,096 overall, comprising n=2,753 females and n=2,262 males; a minority of respondents (n=81) did not provide their gender. There were n=3,761 respondents from SSF3 areas and n=1,335 from comparison areas. The survey aimed to capture responses from residents with a broad range of demographic characteristics. Table 3 outlines the demographic breakdown of respondents in funded areas.

Table 3: Local residents survey respondents demographic breakdown

	Total n=3,761	n	%
Gender	Men	1,909	51.7
	Women	1,785	48.3
Age	16-24	730	19.5
	25-34	751	20.1
	35-49	771	20.6
	50-54	246	6.6
	55-59	232	6.2
	60-64	201	5.4
	65-74	323	8.6
	75 or over	261	7.0
Education	A postgraduate degree	429	12.2
	An undergraduate degree	479	13.5
	A-Levels, O Levels, GCSEs	1,487	42.2
	Any other qualifications (e.g. BTEC, Diplomas, Trade Apprenticeships)	367	10.4
	No qualifications	764	21.7
Ethnicity	Asian or Asian British	389	10.8
	Black, black British, Caribbean or African	128	3.5
	Mixed or multiple ethnic groups	93	2.6
	White	2,956	81.9
	Other ethnic group	43	1.2
Disability	Yes, affects a lot	310	36.1
	Yes, affects a little	328	38.2
	Not at all	207	24.1

Notes:

1. Due to data weighting and/or missing data, sub-group totals may not total exactly the overall weighted base.

Training beneficiaries (Email survey)

Purpose/aim:

The email survey was conducted in 18 funded areas, aiming to capture the impact of VAWG education and training interventions delivered directly to individuals. These audiences included university staff and students, teachers, community workers or volunteers, NTE workers, police, security and other professional services ^{[footnote 16]}.

Sample and counterfactual selection:

A total of 18 bids were identified as delivering interventions direct to individuals.

To reach this audience, all relevant bid areas were contacted to determine how best to reach the intervention beneficiaries. Bid area stakeholders either distributed the survey links to beneficiaries directly or sent the survey links to intermediaries (for example, training providers) to distribute.

To reach suitable counterfactual respondents, bid areas were asked to share the survey link with individuals in similar roles, who had not participated in the interventions.

Fieldwork summary:

Fieldwork was conducted from 16 May 2022 to 6 July 2022.

The final sample size was n=367, comprising n=66 male and n=298 female respondents (n=3 declined to answer). There were n=232 respondents who had been direct beneficiaries of SSF3 interventions with the remaining n=134 coming from individuals in those areas who had not participated (comparison). Table 4 outlines the demographic breakdown of beneficiary respondents.

Table 4: Training beneficiaries survey respondents demographic breakdown

	Total n=232	n	%
Gender	Men	59	25.7
	Women	170	74.3
Age	16-24	29	12.5
	25-34	38	16.5
	35-49	67	29.0
	50-54	31	13.6
	55-59	29	12.5
	60-64	17	7.5
	65-74	2	1.1
	75 or over	1	0.3
Education	A postgraduate degree	57	25.7
	An undergraduate degree	70	31.9
	A-Levels, O Levels, GCSEs	48	22.0
	Any other qualifications (e.g. BTEC, Diplomas, Trade Apprenticeships)	41	18.6
	No qualifications	4	1.7
Ethnicity	Asian or Asian British	5	2.1
	Black, black British, Caribbean or African	2	1.1
	Mixed or multiple ethnic groups	9	4.1
	White	205	91.9
	Other ethnic group	2	0.8
Disability	Yes, affects a lot	2	4.7
	Yes, affects a little	21	48.8
	Not at all	17	39.5

Notes:

1. Due to data weighting and/or missing data, sub-group totals may not total exactly the overall weighted base.

8.3.3 Selection of comparison areas for the ABOS survey

The comparison sample for the ABOS survey comprised 5,000 addresses, designed to look similar in aggregate to the set of addresses making up the ABOS sample in SSF3 areas. To select suitable comparison areas, Verian used a form of propensity score matching to define the similarity of LSOAs in SSF3 areas and potential comparison areas. The comparison sample was drawn to match the distribution of this propensity score. This is a way of ensuring the SSF3 area and comparison area samples are matched simultaneously across a range of area-level characteristics.

The first step was to define a pool of potential comparison areas. This comprised the 18,564 LSOAs in England and Wales, which were:

• in a police force area where at least one SSF3 project was taking place
• not part of any projects from rounds 1 or 2 of the Safer Streets Fund
• with a geographic centroid more than 5 miles from the centroid of any LSOA receiving interventions as part of any Safer Streets round

This last point was to ensure LSOAs close to SSF areas were excluded from the comparison sample, given the possibility of some geographic spillover of the effects of SSF interventions.

The second step was to define the similarity of LSOAs. Verian fit a logistic regression model predicting which LSOAs were part of SSF3 as a function of:

• Index of Multiple Deprivation
• proportion of adult residential population that was aged 18 to 29
• ONS rural/urban classification
• ratio of violence/sexual assaults recorded by police in the quarter prior to SSF3 (April to June 2021) to the number of residents
• geographic region
• address density
• 6 ‘principal components’ derived from over 50 indicators from the Census, representing a range of characteristics of residents within an LSOA

This model produced a ‘propensity score’, which can be interpreted as the estimated probability of an LSOA being in a bid area rather than being in the set of potential comparison areas.

The final step was to draw a sample of addresses from the pool of comparison areas. Verian divided the propensity score into 10 strata to create an equal number of LSOAs with SSF3 interventions in each stratum. Then a random sample of comparison addresses was drawn, with an equal number of addresses in each stratum. This approach ensured the set of addresses in the comparison sample was approximately similar to the set of addresses in the sample of SSF3 areas regarding the variables described above. In this selection process, the potential comparison LSOAs were given a weight proportionate to the number of residential addresses in the LSOA to account for the fact that different LSOAs contain different numbers of residents.

Table 5 provides summary statistics of area-level characteristics for the SSF3 ABOS sample, the comparison ABOS sample, and the whole set of potential comparison areas from which the comparison sample was drawn.

Table 5: Summary area-level statistics for the ABOS samples and the pool of potential comparison areas

	SSF3 sample	Comparison sample	Pool of potential comparison areas
Total population size: mean (std.dev)	1,779 (536)	1,729 (417)	1,674 (349)
Index of Multiple Deprivation rank: mean (std.dev)	9,482 (7,941)	10,249 (8,033)	16,998 (9,670)
Proportion of adult residents aged 18-29: mean (std.dev)	0.32 (0.17)	0.32 (0.17)	0.19 (0.09)
Ratio of police recorded violence/sexual offences to total population: mean (std.dev)	0.03 (0.03)	0.02 (0.02)	0.008 (0.007)
Address density: mean (std.dev)	2,375 (1,580)	2,665 (1,776)	1,456 (1,398)
Urban/rural classification: % Urban	99%	96%	74%
Region: % North East England	6%	2%	4%
Region: % North West England	8%	4%	4%
Region: % Yorkshire and The Humber	6%	5%	6%
Region: % East Midlands	12%	20%	12%
Region: % West Midlands	10%	11%	35%
Region: % East of England	10%	7%	4%
Region: % London	4%	4%	2%
Region: % South East England	18%	19%	19%
Region: % South West England	14%	16%	10%
Region: % Wales	10%	12%	3%

8.3.4 Survey weighting

Weighting the SSF3 samples

For the ABOS survey conducted with local residents in SSF3 areas, Verian used weighting to ensure that the sample achieved in SSF3 areas represented the resident population of these areas at the overall level.

The team weighted the survey data using reliable ONS LSOA-level population statistics for the following variables:

• age by gender
• ethnicity
• household structure
• disability
• marital status
• highest education level

These LSOA benchmark population statistics were then aggregated up to the whole Fund level.

When weighting the data, each SSF3 area was given equal weight (regardless of the population of each area). This is because, when evaluating the success of the Fund, Verian wanted to examine the mean effect it has had across the SSF3 areas (rather than giving more weight to schemes that covered a larger population).

It was not possible to take a similar approach to the face-to-face (F2F) and email surveys. This is because there are no equivalent reliable statistics about the profiles of these populations. The F2F survey covered the transient population of people using certain public spaces at certain times of the day. However, most of the available statistics cover the resident population of an area so it would not be appropriate to use these statistics as many of the respondents to the F2F survey would not have lived within those areas. Consistent with the ABOS survey, each SSF3 area was given equal weight in the F2F survey.

For the email survey, there was no reliable, consistent information about the profile of people attending these training courses across SSF3 areas. Most SSF3 projects were able to provide information about the number of participants invited to the survey. Verian used this information to calculate a weight which adjusts for different response rates for different training interventions. This weight prevents anyone receiving training interventions being over- or under-represented in the survey data. There were 4 training interventions where the training provider or the SSF3 project lead did not provide information about the number of participants invited. For participants receiving these interventions, Verian assumed that the response rate to the survey would have been the average response rate observed for the other training interventions, where this information was known.

Weighting the comparison samples

Verian developed Propensity Score Matched (PSM) weights for each survey to ensure the samples in SSF3 areas and comparison areas were broadly comparable. The first step in deriving the survey weights was for Verian to fit a logistic regression model predicting whether a respondent was from an SSF3 area or from the comparison sample.

The following predictor variables were used in these models for each of the surveys: gender, age, ethnicity, prior highest education level, disability status, sexual orientation and marital/relationship status. In addition:

For the ABOS survey, the models included information about the number of people in the households.

For the F2F survey, the following variables were included:

• the type of area in which the survey was conducted (residential area, high street or shopping area, NTE area, transport hub, park/green space or footpath/cycleway)
• the respondent’s reasons for visiting the area (work or study, shopping, going to a restaurant, going to bars, pubs, clubs or similar venues, or other reason)
• how frequently the respondent typically visits the area
• what times of day the respondent typically visits the area

For the email survey, the following variables were included:

• how frequently the respondent typically visits the area
• what times of day the respondent typically visits the area
• job sector
• whether the respondent was invited to the survey through their employer or via another organisation
• whether they work in a single fixed place or multiple locations

These models gave a predicted probability for each respondent of being in an SSF3 area. The PSM weighting algorithm then assigned weights to respondents in the comparison areas, giving greater weight to respondents with higher predicted probabilities (those which are most similar to respondents in SSF3 areas). This process accounts for many of the initial differences between the SSF3 samples and the comparison samples.

8.3.5 QuIP: methodology

A standard QuIP study involves 24 interviews, a ‘double QuIP’ (that is, 48 interviews) was delivered for the SSF3 evaluation. The QuIP focused on the perceptions of safety, and the influencing factors and drivers of change of these perceptions within a population of members of the public (male and female) within funded areas.

The QuIP sought to understand the drivers influencing change in relation to the following areas:

• safety in residential areas (day vs. night)
• safety on the high street (day vs. night)
• safety on public transport (day vs. night)
• safety in parks (day vs. night)
• safety in pubs/bars/clubs (day vs. night)

Participants were sampled using a combination of purposive and randomised sampling from across the 3 modes of the impact evaluation surveys – users of public spaces, local residents, and training beneficiaries, where permission for recontact was received.

Survey data was used to find respondents who had reported changes in their perceptions of safety over the recall period, specifically in relation to questions regarding their personal safety and perceptions of sexual harassment and gender-based violence. Just over half of the sample (n=25) had reported positive change and slightly less than half (n=23) had reported negative change, to summarise their experiences. A trained QuIP analyst from Bath SDR coded and analysed the QuIP data respectively, using the Causal Map App to identify causal chains in the data.

8.3.6 Impact analysis using PRC data

To measure changes in PRC before and after introducing SSF3 interventions, Verian used the Generalized Synthetic Control Method (Xu, 2017). This method compares the trend in reported crime for each bid area against a set of comparison areas that are not part of the Fund or part of other SSF rounds. The comparison areas are weighted such that the trend in recorded crime before introducing the Fund closely matches the trend in each bid area. The rationale is that if the historic trends are closely matched, it is reasonable to think that this weighted average of the comparison areas can provide a good counterfactual estimate, that is, a good estimate of what would have happened in the bid area without the interventions.

Verian used historic administrative crime data (either from the Home Office DataHub or police.uk) for the quarter 1 period of 2016 to 2017 (April to June 2016) to the quarter 1 period of 2021 to 2022 (April to June 2021), to generate a synthetic control area for each of the successful bid areas^{[footnote 17]}. To ensure that only LSOAs with similar characteristics were included, for each bid the set of potential comparison areas was limited to those which were reasonably similar to the bid LSOAs in terms of:

• population size
• proportion of the population aged 10 to 15
• proportion of the population who are students
• proportion of the population of an ethnic group other than white
• size of the LSOA (in hectares)
• housing density (number of properties per hectare)
• proportion of homes which are flats
• workday to resident population ratio
• Census data about employment in different industry sectors
• the number of recorded COVID-19 cases

This approach means the counterfactual estimates are based on comparison areas which have both similar population characteristics and similar historic PRC trends as the SSF3 bid areas.

Further, to help ensure that potential spillover effects did not affect these comparison areas, any LSOA that was within 5 miles of an LSOA involved in any round of SSF was excluded from the set of potential comparison areas.

The models then compared the number of crimes recorded in each bid area against this synthetic control for the latest available quarter of data – quarter 1 April to June 2023. The difference between the number of crimes recorded and the synthetic control is the estimated impact of SSF3 interventions for that bid.

It is possible that differences between the number of crimes recorded in bid areas and the counterfactual estimates were not caused by SSF3 interventions but by other factors. One key external factor is COVID-19. The pre-intervention trends used to build the models go up to quarter 1 (April to June) 2022 (as based on the policy funded year). As such, most of the impact of COVID-19 will have occurred before this point and so will already be incorporated into the impact estimates. However, it is possible that the ongoing effects of the pandemic will have had different impacts in different areas at different times, and this could have added some bias to the analysis.

Crime types

Some SSF3 projects focused on reducing overall neighbourhood crime, while others focused on one or more specific crime types within neighbourhood crime (a combination of burglary, robbery, theft from the person and/or vehicle crime). For each project, the models counted only the specific crime types that the project was aiming to address. This ensures that the analysis aligned with the stated objectives of each project. Table 7 lists the relevant crime types for each project.

Results

Table 6 shows the overall results of this analysis across SSF3 areas for all neighbourhood crime and for each crime type within neighbourhood crime. Each estimate represents the difference between the observed (reported) crime and the expected counterfactual in the Fund’s absence. The table includes a 95% confidence interval (CI), which represents the statistical uncertainty in the estimates. Table 7 also gives the detailed results for each bid. As shown in table 7, Police and Crime Commissioners (PCCs) were invited to submit up to 3 proposals for their areas, consulting relevant local authorities and other partners in their area. Primary, Secondary, Tertiary refer to the 3 possible proposals from a PCC.

Table 6: Impact analysis results (Generalized Synthetic Control Method)

Crime type	Observed (mean per LSOA)	Counterfactual (mean per LSOA)	Impact estimate (per LSOA)	Lower CI	Upper CI	p-value
All neighbourhood crime	8.03	6.57	1.46	-1.37	4.29	0.312
Burglary	3.42	3.09	0.33	-0.34	1.00	0.336
Robbery	1.02	0.91	0.11	-0.04	0.27	0.152
Theft	1.46	1.22	0.24	-0.23	0.71	0.320
Vehicle crime	3.78	3.42	0.36	-0.03	0.74	0.067

Table 7: Impact analysis results by bid (Generalized Synthetic Control Method)

Please see Appendix C: Table 7.

8.3.7 Impact analysis using survey data

Overview of impact analysis using survey data

Verian used hierarchical regression models to estimate the impact of SSF3 on outcome measures collected through the 3 surveys (ABOS, F2F, email). The outcome measures for each model were measured on a 0 to 10 scale, typically representing a measure of respondents’ perceptions of change over the previous 6 months (the period over which SSF3 interventions were being implemented). For instance, one outcome measure captured respondents’ perceptions of how their feelings of safety on quiet residential streets near their house changed over the previous 6 months.

Each model included a term for whether the respondent was surveyed within an SSF3 area or a comparison area; the estimate for this term represents the estimated difference between outcomes for an individual in an SSF3 area and a similar individual in another similar area where SSF3 interventions were not taking place. Assuming all relevant differences between the SSF3 areas and the comparison areas are accounted for in the model, this can be interpreted as an estimate of the impact attributable to SSF3.

To support this assumption, the model included a set of variables to account for differences between respondents in SSF3 areas and comparison areas, described below. The models also included separate intercepts for each area, to account for the clustered nature of the sample; that is, the tendency of individuals in the same area to be more similar than individuals in different areas. These area-level intercepts will also partially account for differences between the SSF3 areas and the comparison areas that are not directly captured in the model.

A key limitation of this approach is that it was not possible to verify the assumption that all relevant differences between areas were accounted for in the models. There will be some unobserved differences between the areas that remain unaccounted for. It is possible that these differences may explain some or all of any estimated impact. Similarly, it is possible that these differences might mask the true impact of the SSF3 intervention.

Model selection

A process of cross-validation was run to select the best fitting model; that is, the model that minimised the likelihood that any observed impact resulted from other unaccounted for differences between the SSF3 areas and the comparison samples.

For each survey mode, the cross-validation involved:

1. Splitting the data into 3 random subsets.
2. Fitting a model using data for 2 of the 3 subsets.
3. Using the output of this model to predict the outcome values of the third subset.
4. Comparing these predicted values against the actual values.

When comparing different model specifications, the stronger specification will generally predict which are closer to the actual values.

This process was repeated for each outcome measure within each survey mode, testing 3 separate model specifications of increasing complexity. These 3 specifications were as follows:

Specification 1 included 3 core elements: an indicator for whether the respondent was surveyed as part of an SSF3 area or a comparison area; the survey weight (see Section 8.3.4); and area-level intercepts. The SSF3 indicator was the key variable of interest, as this represented the estimated impact of SSF3 interventions.

Specification 2 added a set of individual-level characteristics (demographic variables) as well as information about the type of area in which the interview was conducted (for both ABOS and the F2F survey) and information about respondents’ jobs (for the email survey). The individual-level characteristics were:

• age
• gender
• ethnicity
• highest education level
• disability status
• sexual orientation
• marital/relationship status
• for ABOS, the number of adults in the household and if the respondent is parent/guardian of a child in the household

The information about types of areas for the ABOS survey included:

• urban/rural status
• deprivation of the area using ranking on the Index of Multiple Deprivation (IMD)
• the number of violent and sexual offences recorded by police in the area in April to June 2021
• the population of the area
• several summary area-level variables derived from using factor analysis on a wide range of Census variables covering the demographic characteristics of people living in an area

The information about types of areas for the F2F survey identified each area as being:

• a residential area or university campus
• a high street or shopping area
• an NTE area
• a transport hub (such as by a railway station)
• a park or green space

The information about respondents’ jobs for the email survey was:

• area of employment (for example, sector)
• whether they are based in a single location or travel between locations
• how the respondent was invited to the survey (whether through their employer or another organisation)

As described in Section 8.3.3, the propensity score matching method used to derive the survey weight already attempts to account for many of the observed differences between the SSF3 areas and the comparison samples. However, it is reasonable to think that including these variables separately in the regression models will better account for these differences than the survey weight alone.

Specification 3 added a further set of individual-level variables related to respondents’ use of public spaces. For ABOS and the email survey, this included the frequency of using certain public spaces:

• parks and open spaces
• public transportation
• shopping centres and high streets
• pubs/bars/clubs

It also included the typical time they frequented these places:

• morning
• afternoon
• evening/night

For the F2F survey, this included:

• the respondent’s reasons for visiting the area in which they were surveyed (such as working/studying, shopping, visiting a pub or bar, and so on)
• how often the respondent typically visits the area
• the times of day the respondent typically visits the area

These variables were intended to account for the different ways in which members of the public in the SSF3 areas may use these public spaces, compared with members of the public in the comparison areas. The cross-validation suggested there was very little difference in the performance of the 3 model specifications. Specification 2 was used for all the impact models presented in this report, as it most commonly had the best cross-validated performance, although the difference was very marginal. The cross-validation process also demonstrates that the impact results are not sensitive to these choices about how the models were constructed.

Estimated impacts

Feelings of safety:

Tables 8 to 13: Key outputs of SSF3 impact models – respondents’ own perceived change in safety in public spaces

Think about a scale where 0 means you feel much less safe and 10 means you feel much more safe. Since October 2021 (in the last 6 months), how have your feelings of safety changed in [public space] in this area during the day/at night?

Feelings of safety during the day in each of these areas	Survey	Impact estimate	95% CI: Lower bound	95% CI: Upper bound	p-value
In quiet streets close to your home	ABOS	-0.04	-0.15	0.08	0.536
High street / Shopping area	ABOS	-0.01	-0.12	0.11	0.915
Public transportation	ABOS	0.00	-0.10	0.11	0.930
Parks / Open spaces	ABOS	0.01	-0.09	0.12	0.838
Pubs, bars and clubs	ABOS	-0.01	-0.11	0.08	0.787

Feelings of safety during the day in each area	Survey	Impact estimate	95% CI: Lower bound	95% CI: Upper bound	p-value
High street / Shopping area	F2F	0.00	-0.22	0.23	0.979
Public transportation	F2F	-0.04	-0.19	0.11	0.595
Parks / Open spaces	F2F	0.05	-0.21	0.30	0.730
Pubs, bars and clubs	F2F	-0.02	-0.18	0.13	0.782

Feelings of safety during the day in each area	Survey	Impact estimate	95% CI: Lower bound	95% CI: Upper bound	p-value
Public transportation	Email	0.05	-0.24	0.34	0.734
Pubs, bars and clubs	Email	0.15	-0.14	0.44	0.320
Near where you work or study	Email	0.21	-0.06	0.47	0.140
Place of employment/study	Email	0.19	0.08	0.47	0.179

Feelings of safety at night in each area	Survey	Impact estimate	95% CI: Lower bound	95% CI: Upper bound	p-value
In quiet streets close to your home	ABOS	-0.06	-0.17	0.05	0.266
High street / Shopping area	ABOS	-0.02	-0.15	0.11	0.720
Public transportation	ABOS	-0.02	-0.13	0.09	0.759
Parks / Open spaces	ABOS	0.00	-0.12	0.12	0.987
Pubs, bars and clubs	ABOS	-0.04	-0.15	0.07	0.510

Feelings of safety at night each of these areas	Survey	Impact estimate	95% CI: Lower bound	95% CI: Upper bound	p-value
High street / Shopping area	F2F	0.15	-0.09	0.38	0.256
Public transportation	F2F	0.00	-0.14	0.12	0.949
Parks / Open spaces	F2F	0.20	-0.07	0.47	0.169
Pubs, bars and clubs	F2F	0.10	-0.02	0.22	0.097

Feelings of safety at night in each of these areas	Survey	Impact estimate	95% CI: Lower bound	95% CI: Upper bound	p-value
Public transportation	Email	-0.03	-0.44	0.32	0.875
Pubs, bars and clubs	Email	0.22	-0.20	0.59	0.280
Near where you work or study	Email	0.18	-0.15	0.46	0.2861
Place of employment/study	Email	0.20	-0.13	0.50	0.2411

Perceptions of crime

Table 14: Key outputs of SSF3 impact models – respondents’ own perceived change in crime

Since October 2021 (the last 6 months), how much do you think acquisitive crime has changed in the area where you live?

Since October 2021 (the last 6 months), how much do you think anti-social behaviour has changed?

Since October 2021 (the last 6 months), how much do you think sexual harassment or gender-based violence in public spaces has changed in this area… during the day/at night?

Perceived change in crime	Survey	Impact estimate	95% CI1: Lower bound	95% CI1: Upper bound	p-value
Acquisitive crime	ABOS	0.02	-0.12	0.15	0.804
Anti-social behaviour	ABOS	-0.09	-0.20	0.02	0.124
Sexual harassment / Gender-based violence during the day	ABOS	-0.12	-0.21	-0.03	0.009
Sexual harassment / Gender-based violence at night	ABOS	-0.08	-0.19	0.03	0.139
Anti-social behaviour	F2F	-0.18	-0.58	0.22	0.396
Sexual harassment / Gender-based violence during the day	F2F	-0.39	-0.86	0.07	0.127
Sexual harassment / Gender-based violence at night	F2F	-0.32	-0.69	0.04	0.117
Anti-social behaviour	Email	-0.33	-0.71	0.04	0.088
Sexual harassment / Gender-based violence during the day	Email	0.05	-0.24	0.33	0.752
Sexual harassment / Gender-based violence at night	Email	-0.09	-0.45	0.28	0.650

Notes:

1. CI is defined as the confidence level. Results presented in bold text indicate the results are statistically significant at the 5% level.

Engagement with police

Table 15: Key outputs of SSF3 impact models – respondents’ own perceived change in crime

Since October 2021 (the last 6 months) how has your confidence in the local police to deal with sexual harassment or gender-based violence in public spaces in this area changed?

Since October 2021 (the last 6 months), how has your likelihood to report sexual harassment or gender-based violence to the local police in this area changed?

Engagement with police	Survey	Impact estimate	95% CI1: Lower bound	95% CI1: Upper bound	p-value
Confidence in local police to deal with sexual harassment or gender-based violence in public spaces	ABOS	0.15	0.02	0.29	0.026
Likelihood to report sexual harassment or gender-based violence to local police in this area	ABOS	0.05	-0.09	0.20	0.459
Confidence in local police to deal with sexual harassment or gender-based violence in public spaces	F2F	0.00	-0.26	0.27	0.976
Likelihood to report sexual harassment or gender-based violence to local police in this area	F2F	0.09	-0.16	0.34	0.523
Confidence in local police to deal with sexual harassment or gender-based violence in public spaces	EMAIL	0.93	0.48	1.33	0.000
Likelihood to report sexual harassment or gender-based violence to local police in this area	F2F	1.00	0.37	1.57	0.002

Notes:

1. CI is defined as the confidence level. Results presented in bold text indicate the results are statistically significant at the 5% level.

8.3.8 Further Analysis 1: SSF3 for specific area-based interventions

Verian conducted further analysis to explore the extent to which the certain key impact estimates were associated with characteristics of individuals, areas or SSF3 projects. Given the small sample size of the email survey, these further analyses were limited to the ABOS survey and F2F survey data.

The first form of additional analysis explored the extent to which individuals’ perceptions of safety in different public spaces were associated with the types of spaces SSF3 interventions were targeting. As part of the bid review, Verian mapped the planned interventions of each SSF3 project against the kinds of public spaces in which those interventions primarily operate. For example, some projects included interventions aimed at transport hubs, while others focused on NTE areas. The initial impact analysis described above was concerned with the overall changes in perceptions of safety in different public spaces. However, it is plausible that effects would have been greater in areas where there was a greater focus on interventions targeting that particular kind of space.

To test this, Verian fit additional hierarchical regression models for the measures related to changes in perceptions of safety. These models were a very similar form to the impact models described previously, but included one additional variable indicating whether an SSF3 project had any interventions targeting the relevant type of public space. The estimate for this variable represents the difference in the estimated impacts for areas with SSF3 interventions specifically targeting that kind of public space and other SSF3 areas.

Tables 16 and 17: Key outputs of SSF3 impact models – respondents’ own perceived change in feelings of safety

Think about a scale where 0 means you feel much less safe and 10 means you feel much more safe. Since October 2021 (in the last 6 months), how have your feelings of safety changed in [public space] in this area during the day/at night?

Feelings of safety during the day in each of these areas	Survey	Impact estimate	95% CI1: Lower bound	95% CI1: Upper bound	p-value
In quiet streets close to your home	ABOS	0.00	-0.11	0.11	0.984
High street / Shopping area	ABOS	0.09	-0.04	0.22	0.171
Parks / Open spaces	ABOS	-0.06	-0.16	0.05	0.291
Public transportation	ABOS	-0.05	-0.16	0.05	0.327
Pubs, bars and clubs	ABOS	0.00	-0.11	0.10	0.957
High street / Shopping area	F2F	0.51	0.05	0.97	0.047
Parks / Open spaces	F2F	-0.06	-0.40	0.27	0.721
Public transportation	F2F	0.21	0.04	0.39	0.041
Pubs, bars and clubs	F2F	0.07	-0.14	0.28	0.534

Feelings of safety at night in each of these areas	Survey	Impact estimate	95% CI1: Lower bound	95% CI1: Upper bound	p-value
In quiet streets close to your home	ABOS	-0.04	-0.15	0.07	0.477
High street / Shopping area	ABOS	0.08	-0.07	0.23	0.312
Parks / Open spaces	ABOS	-0.01	-0.13	0.11	0.859
Public transportation	ABOS	-0.02	-0.13	0.09	0.761
Pubs, bars and clubs	ABOS	-0.03	-0.14	0.09	0.680
High street / Shopping area	F2F	0.00	-0.49	0.49	0.990
Parks / Open spaces	F2F	0.08	-0.27	0.43	0.663
Public transportation	F2F	0.01	-0.15	0.18	0.926
Pubs, bars and clubs	F2F	0.05	-0.11	0.20	0.571

Notes:

1. CI is defined as the confidence level. Results presented in bold text indicate the results are statistically significant at the 5% level.

8.3.9 Further Analysis 2: Differences between different types of interventions

The second form of further analysis explored the extent to which more or less positive effects were associated with different types of SSF3 interventions. To explore this, Verian fit further hierarchical regression models; these had the broadly the same form as the impact models, although instead of including an indicator to identify respondents in SSF3 areas, the models included a series of indicators identifying whether a given intervention type was delivered in each area. The coefficients for these variables represent the estimated effect associated with that intervention type. For ABOS, the intervention types were classified as following:

• educational
• environmental
• communications
• guardianship initiatives
• personal safety initiatives
• strategy
• support services
• other interventions

For F2F, the interventions were grouped in a slightly different way because all the areas covered by the F2F survey had some form of education and strategy initiatives, and almost all had some form of guardianship initiative. Therefore, education, strategy and other interventions were grouped together for this analysis for the F2F survey, and divided guardianship initiatives into street guardianship, police patrols and safety points.

This analysis focused on impacts on 2 outcomes of interest. The first was perceived change in sexual harassment or gender-based violence in public spaces; the second, change in the respondent’s confidence in the local police to deal with sexual harassment or gender-based violence in public spaces.

Table 18: Perceived change in sexual harassment or gender-based violence

Intervention type	Survey	Impact estimate	95% CI: Lower bound	95% CI: Upper bound	p-value
Educational	ABOS	0.11	-0.07	0.29	0.245
Environmental	ABOS	-0.13	-0.32	0.06	0.177
Comms	ABOS	-0.08	-0.21	0.05	0.204
Guardianship initiatives	ABOS	0.04	-0.07	0.15	0.460
Personal safety initiatives	ABOS	-0.02	-0.13	0.08	0.668
Strategy	ABOS	0.05	-0.06	0.16	0.350
Support Services	ABOS	-0.11	-0.23	0.01	0.081
Other interventions	ABOS	-0.09	-0.22	0.04	0.116
Environmental	F2F	-0.37	-1.12	0.19	0.423
Comms	F2F	-0.10	-0.70	0.62	0.835
Personal safety initiatives	F2F	0.19	-0.30	0.64	0.604
Support Services	F2F	0.30	-0.19	0.70	0.388
Street Guardians	F2F	0.16	-0.42	0.58	0.675
Police patrols	F2F	-0.31	-0.76	0.58	0.406
Safety Points	F2F	0.21	-0.32	0.63	0.550
Educational/strategy/other	F2F	-0.24	-0.99	0.80	0.721

Table 19: Confidence in local police

Intervention type	Survey	Impact estimate	95% CI1: Lower bound	95% CI1: Upper bound	p-value
Educational	ABOS	-0.10	-0.35	0.16	0.467
Environmental	ABOS	0.17	-0.09	0.44	0.202
Comms	ABOS	-0.07	-0.25	0.11	0.471
Guardianship initiatives	ABOS	0.21	0.06	0.36	0.005
Personal safety initiatives	ABOS	-0.04	-0.19	0.11	0.621
Strategy	ABOS	0.05	-0.10	0.20	0.541
Support Services	ABOS	-0.03	-0.19	0.14	0.767
Other interventions	ABOS	- 0.03	-0.21	0.14	0.728
Environmental	F2F	0.08	-0.51	0.68	0.833
Comms	F2F	0.26	-0.34	0.81	0.499
Personal safety initiatives	F2F	-0.23	-0.62	0.18	0.410
Support Services	F2F	0.11	-0.29	0.48	0.680
Street Guardians	F2F	0.28	-0.14	0.70	0.320
Police patrols	F2F	0.23	-0.17	0.65	0.414
Safety Points	F2F	0.29	-0.13	0.68	0.298
Educational/strategy/other	F2F	-0.68	-1.39	0.12	0.196

Notes:

1. CI is defined as the confidence level. Results presented in bold text indicate the results are statistically significant at the 5% level.

8.3.10 Further Analysis 3: How effects vary between sub-groups

The third form of further analysis explored the extent to which effects measured in both the ABOS and F2F survey varied by key demographic and area characteristics. Demographic characteristics were gender and age. Area characteristics for the ABOS survey included whether the area was urban (city/town or conurbation) or rural and the IMD decile. Whereas for the F2F survey, the area characteristics included the specific type of area being used – high street/shopping area, night-time economy, residential/campus, transport hub, and park/green space/footpath/cycle way.

To explore this, Verian fit further hierarchical regression models; these had broadly the same form as the overall impact models, with the addition of an interaction term for either gender, age or type of area. These coefficients represent the extent to which estimated effects vary between groups.

Taking the first example (Table 20, below), the estimated effect of SSF3 interventions on perceptions of the change in sexual harassment and gender-based violence very slightly decreased with age in the ABOS analysis – the estimated impact reduces by 0.001 points for each year of age, although this is not statistically significant. Considering differences between types of area, in the F2F analysis, the estimated effect in high streets and shopping areas was 0.92 points lower than in residential areas and university campus areas.

This analysis focused on impacts on 2 outcomes of interest. The first was perceived change in sexual harassment or gender-based violence in public spaces. The second was change in the respondent’s confidence in the local police to deal with sexual harassment or gender-based violence in public spaces.

Table 20: Outcome – perceived change in sexual harassment or gender-based violence in public spaces

Sub-group	Crime types	Difference in estimated impact	Lower CI1	Upper CI1	p-value
Gender: difference in estimated impact associated with being female compared to male	ABOS	0.090	-0.090	0.280	0.308
Age: difference in estimated impact associated with an increase in age of one year	ABOS	-0.001	-0.006	0.004	0.686
Rural urban: City/town vs Conurbation	ABOS	-0.050	-0.280	0.180	0.667
Rural urban: Rural vs Conurbation	ABOS	-0.230	-0.883	0.360	0.442
IMD decile	ABOS	-0.020	-0.060	0.020	0.263
Gender: difference in estimated impact associated with being female compared to male	F2F	-0.090	-0.440	0.240	0.595
Age: difference in estimated impact associated with an increase in age of one year	F2F	0.005	-0.010	0.020	0.341
Type of area: High street / shopping area vs. residential/ university campus area	F2F	-0.920	-1.760	-0.090	0.035
Type of area: NTE area vs. residential/ university campus area	F2F	-0.730	-1.870	0.370	0.205
Type of area: Transport hub vs. residential/university campus area	F2F	-0.610	-1.630	0.350	0.228
Type of area: Park or green space/footpath or cycle way vs. residential/university campus area	F2F	-0.950	-1.940	0.030	0.062

Notes:

1. CI is defined as the confidence level. Results presented in bold text indicate the results are statistically significant at the 5% level.

Table 21: Outcome – change in confidence in the local police to deal with sexual harassment or gender-based violence in public spaces

Sub-group	Crime types	Difference in estimated impact	Lower CI1	Upper CI1	p-value
Gender: difference in estimated impact associated with being female compared to male	ABOS	0.000	-0.240	0.250	0.995
Age: difference in estimated impact associated with an increase in age of one year	ABOS	-0.003	-0.010	0.004	0.378
Rural urban: City/town vs Conurbation	ABOS	0.110	-0.210	0.430	0.512
Rural urban: Rural vs Conurbation	ABOS	0.140	-0.720	1.000	0.753
IMD decile	ABOS	0.020	-0.030	0.070	0.350
Gender: difference in estimated impact associated with being female compared to male	F2F	0.130	-0.220	0.490	0.481
Age: difference in estimated impact associated with an increase in age of one year	F2F	-0.002	-0.012	0.009	0.782
Type of area: High street / shopping area vs. residential/ university campus area	F2F	-0.410	-1.320	0.470	0.371
Type of area: NTE area vs. residential/ university campus area	F2F	-1.560	-2.730	-0.350	0.011
Type of area: Transport hub vs. residential/university campus area	F2F	-0.350	-1.360	0.670	0.505
Type of area: Park or green space/footpath or cycle way vs. residential/university campus area	F2F	0.120	-0.880	1.160	0.821

Notes:

1. CI is defined as the confidence level. Results presented in bold text indicate the results are statistically significant at the 5% level.

References

Home Office (2020) Violence Against Women and Girls (VAWG) Call for Evidence. Available online [accessed 13/01/2024]

Home Office (2021a) Tackling violence against women and girls’ strategy. Available online [accessed 11/01/2024]

Home Office (2021b) Safety of Women at Night Fund. Available online [accessed 13/01/2024]

Home Office (2021c) Safer Streets Fund Round 3: Protecting Public Spaces, Guidance for Bidders. Available online [accessed 11/01/2024]

Home Office (2022) Tackling Domestic Abuse Plan. Available online [accessed 13/01/2024]

ONS (2021a) Perceptions of personal safety and experiences of harassment, Great Britain: 2 to 27 June 2021. Office for National Statistics. Available online [accessed 11/01/2024]

ONS (2021b) Nature of sexual assault by rape or penetration, England and Wales. Office for National Statistics. Available online [accessed 11/01/2024]

ONS (2022) Crime in England and Wales: year ending June 2022. Office for National Statistics. Available online [accessed 11/01/2024]

ONS (2023) Sexual offences prevalence and victim characteristics, England and Wales. Office for National Statistics. Available online [accessed 11/01/2024]

Xu, Y. (2017) ‘Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models’. Political Analysis, vol. 25(1), pp. 1-20. Available online [accessed 11/01/2024]

1. The 8 main categories of intervention were: education and training programmes; environmental (for example, CCTV/Lighting); communications; strategy and systems; guardianship initiatives; personal safety; support services; other. ↩

2. Further detail is provided in Appendix C, Section 8.3.1. ↩

3. One bid used their full funding to start developing an app for use on public transportation; they therefore had nothing that could be measured quantitatively for the impact evaluation and were therefore excluded. ↩

4. Further details of survey modes, bid and comparison area selection, and demographic breakdown of respondents are provided in Appendix C, Section 8.3.2. ↩

5. Educational products and programmes which focus on attitudinal/behavioural change and improving understanding and confidence preventing and/or responding to VAWG and to prevent VAWG in public spaces. ↩

6. Bystander action refers to actions taken by a person (or persons) not directly involved as subject(s) or perpetrator of violence against women to identify, speak out about or seek to engage others in responding to VAWG incidents. ↩

7. Interventions in schools / with young people under age 16 were excluded from the evaluation due to time and budget restrictions. ↩

8. LSOAs are small geographic areas with an average population of around 1,500 people. ↩

9. Further detail on the QuIP methodology and recruitment is provided in Appendix C, Section 8.3.5. ↩

10. To develop the ILMs, the research team extracted and analysed the questionnaire responses on expected activities, outcomes, impacts and connecting mechanisms by categorising the interventions into their overarching categories and then conducting analysis within each intervention category to derive an overall logic for each intervention type. ↩

11. Including small local parks – for example, a green space in a residential area, recreational areas such as green spaces or parks with a playground or similar – and larger parks and open spaces, for example, those that attract visitors. ↩

12. The public data about PRC does not separate out commercial burglary and residential burglary, or robbery of business property and robbery of personal property. The neighbourhood crime estimates therefore include all of these crime types. ↩

13. An outcome is defined in the ILMs as the direct change of an intervention, meaning, the measurable change we might expect to see as a direct result of a project’s outputs. For example, through the introduction of an app where a person can report crimes, bids may hope for an immediate increase in reporting crimes in the area. ↩

14. An impact is defined as the wider long-term changes which we expect will follow the outcome and which might be influenced by other outcomes. Using the same example, a longer-term impact of the app may be that people feel safer in their communities due to improved mechanisms in place for reporting. ↩

15. To develop the ILMs, the research team extracted and analysed the questionnaire responses on expected activities, outcomes, impacts and connecting mechanisms by categorising the interventions into their overarching categories and then conducting analysis within each intervention category to derive an overall logic for each intervention type. ↩

16. Interventions in schools / with young people aged 16 and under, were excluded from the evaluation due to time and budget restrictions. ↩

17. Wherever possible, we used the Home Office’s DataHub data for this analysis, due to its superior geo-coding, but in police force areas where the LSOA level data was judged to be relatively incomplete, we used public data from police.uk data instead. ↩