Research and analysis

Non-detection multipliers for measuring tax gaps

Published 9 July 2020

Please note that this is a technical paper that has been written for a specialist analytical audience and contains complex statistical terms

1. Summary

This Working Paper explains HMRC’s plan to develop and implement new, UK-specific non-detection multipliers for use in estimating the tax gap.

Non-detection multipliers are used as part of the methodology to estimate the tax gap from audit data. The multiplier values are applied to audit results to account for non-compliance which is missed or not fully investigated in an audit. They have been applied as part of the methodology for estimating the tax gap for direct taxes since HMRC first began publishing tax gap statistics.

The HMRC tax gaps team have conducted a review of different approaches to estimating non-detection multipliers, explored their application in international tax gap estimation, and tested out different approaches to see whether they can be applied in the UK.

We have selected an estimation method based on expert judgment as the most suitable approach and have implemented this for some of our new tax gap estimates in Measuring tax gaps 2020 edition. We plan to carry out a programme of development to introduce new non-detection multipliers in future editions of ‘Measuring tax gaps’.

2. Background

HMRC publishes an annual report ‘Measuring tax gaps’ which estimates the difference between the amount of tax that should, in theory, be paid to HMRC and what is actually paid. Full details of where to locate the ‘Measuring tax gaps’ series of reports and other papers cited in this note are included in section 6.

There are broadly 2 main methods of tax gap measurement:

1. Top-down: comparing the implied tax due from consumer expenditure data with tax receipts

2. Bottom-up: building up from our own operational data and management information to identify and estimate areas of tax loss

In HMRC, we use random enquiry programme data and operational audit data to produce our bottom-up estimates. When using bottom-up methodologies there are 2 main challenges:

• estimating what we have missed from cases we did not audit
• estimating what we missed in cases we did audit

The first challenge is addressed either by use of a random sampling design or through statistical modelling of operational audit data to correct for bias in sample selection, such as the Extreme Values methodology. These methods enable analysts to extrapolate from the results of the sample of audited cases to the wider taxpayer population.

The second challenge is addressed by using non-detection multipliers. These are multiplication factors that uplift total under declarations identified in audits to a higher value to take account of non-compliance that auditors were unable to detect.

Non-detection arises for several different reasons and this can differ between data collected from randomly selected audits and data collected from operational audits. Reasons include, for example:

• auditor capability – experienced auditors are more likely to uncover more risks and therefore larger under declarations than comparatively new auditors, this is particularly an issue in random enquires as the audits have not been targeted based on risk, so auditors must identify all risks in a case
• taxpayer co-operation – when a taxpayer does not co-operate or actively takes steps to conceal activity, this can impact an audit and alter the course of action undertaken by the auditor
• availability of data – there may be a higher risk of non-detection for line items on a tax return where there is no third-party data available to confirm the taxpayer’s self-reported income

For our estimates based on data from the random enquiry programme, we currently apply multipliers developed by the US tax authority, the Internal Revenue Service (IRS), these are described in more detail in sections 3.d and 4 below.

In 2013, HMRC invited the International Monetary Fund to undertake a comprehensive review of our methodologies to estimate the tax gap (see International Monetary Fund, 2013). The study recommended that we replace the existing IRS multipliers with domestically derived UK multipliers to improve the robustness of our tax gap estimates. This recommendation was further endorsed by the Office for Statistics Regulation in their 2019 ‘Compliance Check of Measuring Tax Gaps statistics’ (see Office for Statistics Regulation, 2019).

In the following sections we set out our work to date to develop new multipliers and our plans to update all multipliers in future editions of ‘Measuring tax gaps’.

3. Approaches to non-detection estimation

Results of audits should, in principle, be adjusted for non-detection biases due to differences in the capacity of auditors to identify all non-compliance and/or the scope of audits. The inclusion of a non-detection multiplier produces a theoretically comprehensive tax gap estimate.

There are a variety of ways of deriving a non-detection multiplier. We have studied the available literature on non-detection multipliers and their use in tax gap estimation. The main methods are described below.

a. No multiplier

When a non-detection multiplier is not applied to audit results, the resulting tax gap estimate should be treated as a lower-bound estimate of the tax gap or the ‘detectable gap’. Resulting estimates reflect what would be found if the entire population were audited on the same basis as the random sample – that is, with the same limitations that affected detection in the random audits. Whilst this is methodologically straightforward, we do not propose adopting this method as it would not provide a comprehensive estimate of the tax gap.

b. Detection controlled estimation

Detection Controlled Estimation (DCE) is an econometric approach which assumes different auditors have different abilities to detect liabilities (see Erard and Feinstein 2011). An uplift is derived by scaling up what was detected by a particular auditor to what the ‘best’ auditor would have found in the audit, taking into account an assumed distribution of non-compliance.

This approach can provide a multiplier specific to each line item on a tax return and for each auditor based on their detection rates. The data requirements are high.

Each auditor must have completed at least 15 cases covering broadly the same line items. Characteristics of auditors (for example, grade, years of experience) and other information from the audit (for example, line items on return, third-party information) are required as explanatory variables in the model. This approach scales up results to the observed capability of the highest performing auditor, which still may not equate with full detection.

The IRS pioneered the use of DCE and report having to pool together multiple years (2008 to 2013) of data from their random audit programme (National Research Program), which has a sample size of approximately 13,000 returns each year, in order to derive estimates using DCE (see Internal Revenue Service, 2019).

Drawing on the work of the IRS, and following consultation with academics at the University of Westminster, HMRC attempted to derive DCE estimates using internal data from our random enquiry programme. This was not successful as we had insufficient observations per caseworker to build a robust model.

We also considered whether the IRS’s multipliers derived using the DCE method could be applied to HMRC’s data. However, these multipliers have been created for discrete sub-populations which do not map to HMRC’s taxpayer population.

We concluded that applying the IRS’s DCE non-detection multipliers to our UK tax gap estimates would not provide any greater robustness than continuing to use the IRS non-detection multipliers based on third-party information.

c. Secondary case review by expert auditors

Under a secondary case review approach, audits are passed to a separate group of experts to review where non-detection may have occurred (for example, items that weren’t reviewed exhaustively) and estimate the potential value of undetected risks.

The group of experts could, for instance, be the most experienced auditors who provide an estimate of the value of remaining risks in the case. If cases are passed for review when they are still open, additional checks could be conducted and the difference between the initial revenue losses identified by the first auditor compared to the second.

This technique requires additional resource investment and is only as powerful as the abilities of the experts conducting the secondary review.

We have considered whether secondary case review by expert auditors would be a practical way to estimate non-detection in the UK. There is a high resource cost associated with this method as expert reviews would be required into audits from each tax and customer group covered by a bottom-up estimation methodology to account for variations in risks.

In addition, there would be an opportunity cost, as the expert resource would likely be diverted from other, high-risk casework. For HMRC, the costs would be high as we currently use ten bottom-up models to estimate the UK tax gap, which would each require a separate review.

The introduction of a live secondary case review would also introduce additional time delays in the audit process, and still would not guarantee that all risks were captured comprehensively. For these reasons we do not propose taking forward this method for addressing the issue of non-detection.

d. Data-matching with third-party information

In this approach, audits are first conducted without access to third-party information then compared retrospectively to third-party data on actual income sources. This is the approach the IRS used to develop multipliers prior to the introduction of Detection Controlled Estimation.

It was based on data from their Taxpayer Compliance Measurement Program from the late 1970s and early 1980s (see Internal Revenue Service, 1988). We currently use the IRS’s multipliers developed under this technique as part of the calculation of the UK tax gap.

One limitation of this approach is that it doesn’t provide an indication of the magnitude of non-detection for items where there is no additional data to cross reference, which may include areas of high risk – such as income received in cash. Secondly, since the original IRS research was conducted, it has become standard practice for auditors to have access to most, if not all, of the available data. As a result, the multipliers do not represent the current types of non-detection issues present in the UK random enquiry programme.

We considered replicating this approach to develop UK-specific multipliers. However, it would not be suitable as third-party information is already available to HMRC auditors throughout the audit process – therefore we would not have any additional information we could check our auditors’ assessments against.

e. Expert judgement

In this case a panel of experts, for example, experienced auditors, estimate how much tax generally goes undetected in different types of audits, without reference to a specific case. This can be explored through a structured series of questions detailing specific scenarios that could contribute to non-detection (for example, lack of cooperation from the taxpayer, lack of third-party data).

One means to achieve this is the Delphi technique, whereby respondents provide their independent assessments through a series of rounds to reach a consensus view of the multiplier in the final round (see Dalkey and Helmer, 1963, for a description of its development and original application).

A drawback of this approach is that, unlike an analytical technique based on audit data, there is unlikely to be a way to replicate the final outcome, and the results will be contingent upon the specific experts on the panel.

However, this method has a comparatively small resource cost and can be helpful in identifying risks and opportunities to the detection of revenue losses specific to different tax types and customer groups in the UK tax system.

It also has the advantage that if there are substantial operational changes that are likely to impact levels of detection, a Delphi exercise can be conducted again to explore if and how the non-detection multipliers should be adjusted.

4. Current non-detection multipliers used in UK tax gap estimates

In ‘Measuring tax gaps 2020’ we have used 2 sets of non-detection multipliers. The first set are based on IRS analysis of third-party information and the second have been derived using the Delphi technique.

Internal Revenue Service Non-Detection Multipliers

Currently HMRC use the IRS multipliers which were derived using data-matching to third-party information, as detailed in section 3.d. HMRC adopted the IRS multipliers in 2005 when we first estimated tax gaps from our random enquiry programme data. We considered it best practice to include multipliers so that our estimates were theoretically comprehensive and to avoid knowingly underestimating the tax gap.

The IRS multipliers varied according to the types of risk they found in a case, for example, under reporting income or over stated expenses. As this categorisation did not align fully with the types of non-compliance seen in the UK random enquiry programme, non-compliance in the UK sample was subdivided into broad risk categories and the multiplier for the closest matching risk applied to that proportion.

Although the IRS estimates did not directly map onto the types of taxpayers and risks seen in the UK tax system, they were the best option available and analytically highly sound.

These non-detection multipliers are currently applied to the random enquiry data collected for 3 areas. The value of the multipliers varies by tax head and is consistent year-on-year. The values of the current multipliers used for random enquiry data are given in Table 1.

Table 1: Current non-detection values in ‘Measuring tax gaps 2020’

Random Enquiry Programme	Multiplier for central estimate	Multiplier for lower estimate	Multiplier for upper estimate
Self Assessment (business)	1.908	1.000	3.075
Self Assessment (non-business)	1.260	1.000	1.928
Employer compliance for small businesses	n/a	n/a	n/a
Corporation Tax for small businesses	1.376	1.000	1.859

The multipliers given in Table 1 have been applied to estimates based on random enquiry data in all publications of ‘Measuring tax gaps’ beginning in 2009.

Delphi non-detection multipliers

For ‘Measuring tax gaps 2020 edition’ we have introduced new multipliers for use in estimates based on operational audit data within Self Assessment. These non-detection multipliers have been derived using the Delphi technique.

As discussed in Section 3.e, the Delphi technique is a method for bringing together ‘expert judgement’ which can be used to derive non-detection multipliers. More specifically, the Delphi technique is a consultative method to gather expert opinion in a systematic way and establish consensus.

It involves multiple rounds of gathering anonymous responses via questionnaire. The method assumes that group judgements provide a more robust response than individual ones. Individual judgements are checked by the group anonymously to reach a consensus.

The Delphi technique is designed to overcome some of the pitfalls that can sway opinion when asking a group to reach a consensus, such as:

1. Powerful personalities – mitigated by having small groups, and asking surveys to be completed anonymously
2. Group pressures – mitigated by having small groups, and asking surveys to be completed anonymously
3. Effects of status – mitigated by ensuring lower graded auditors are in different groups to very highly graded auditors and asking surveys to be completed anonymously

To run the Delphi exercises we gathered together auditors who were subject matter experts from different regions of the UK. We then split the auditors into smaller groups, limiting the number of auditors from each location to reduce the chances of the auditors knowing each other. We met with the auditors and iteratively came to a consensus over 3 rounds of consultation.

Within the first round of consultation, we met with each group of auditors and outlined broad reasons why non-detection may occur. Following this we invited group discussion of reasons why non-detection may occur in their specific tax area, for example, a lack of third-party data for specific risks, or lack of taxpayer cooperation in an enquiry.

The possible reasons for non-detection discussed were brought together into a questionnaire. All auditors were then invited to provide their own, independent estimate of the losses associated with the different areas of non-detection.

The responses to the first questionnaire were then collated and analysed to inform the structure of the second questionnaire. The second questionnaire informed the auditors of the group consensus for each question (the average for each question) and asked them to provide a second, independent estimate of the losses associated with each area, this time taking into account everything that has been discussed before, including the group consensus.

The responses from the second questionnaire were collated and analysed to inform the final questionnaire, which again reported back to auditors the average assessment from the second round. Once the auditors had completed the final questionnaire the results were collated and the average of the third-round assessments was adopted as the non-detection multiplier.

5. Discussion and plans for further development

HMRC plan to replace the IRS non-detection multipliers detailed in section 4 in future editions of ‘Measuring tax gaps’.

We have considered several approaches to non-detection, as detailed in section 3. In summary, we evaluated each point as follows:

a. No multiplier – we intend to develop new non-detection multipliers rather than publishing without multipliers as we wish to continue providing a comprehensive tax gap estimate.

b. Detection controlled estimation – we have explored whether solely data-driven methods, such as detection- controlled estimation, could be applied but found that our data did not enable us to create robust models.

c. Secondary case review – the extremely high resource costs, lack of certainty of uncovering all under declarations and probable time delays associated with secondary case review of audits by experts mean we do not intend to further explore this method.

d. Data-matching to third-party information – this method could not be replicated within HMRC as auditors already have access to third-party information throughout audits.

e. Expert judgement – whilst there are issues with this method in that we do not have a way to replicate results, it gives us a consensus view of non-detection, is specific to the UK tax system, provides further operational insight into specific areas, and has a relatively low resource requirement.

Within ‘Measuring tax gaps 2020 edition’ we introduced a new set of non-detection multipliers which were derived using the Delphi technique. This is a form of expert judgement. We were able to successfully develop a method where we came to a group consensus on the level of non-detection in discrete populations.

Our plan is to begin the process of developing new non-detection multipliers, using the Delphi technique, for the remaining bottom-up tax gap models in the summer of 2020, with a view to replacing existing multipliers in future editions of ‘Measuring tax gaps’. We will update users on progress in subsequent editions of ‘Measuring tax gaps’ where appropriate.

6. References

Dalkey, N.C. and Helmer, O., 1963, ‘An experimental application of the Delphi method to the use of experts’, Management Science 9(3), 458 to 467.

Erard, B. and Feinstein, J., 2011, ‘The Individual Income Reporting Gap: What We See and What We Don’t’, Washington DC, Internal Revenue Service – Tax Policy Centre Research Conference.

HM Revenue and Customs, 2020, ‘Measuring Tax Gaps: 2020 Edition’, London, HM Revenue and Customs.

Internal Revenue Service, 1988, ‘Income Tax Compliance Research: Gross Tax Gap Estimates and Projections for 1973-1992: Supporting Appendices to Publication 7285’. Washington, DC, Internal Revenue Service.

Internal Revenue Service Research, Analysis & Statistics, 2019, ‘Federal Tax Compliance Research: Tax Gap Estimates for Tax Years 2011-2013’, Publication 1415 (Rev. 9-2019), Washington DC, Internal Revenue Service. International Monetary Fund, 2013, Country Report No. 13/314, United Kingdom: Technical Assistance Report — Assessment of HMRC’s Tax Gap Analysis.

Office for Statistics Regulation, 2019, ‘Compliance Check of Measuring Tax Gaps statistics’, letter from Assessment Programme Lead, Office for Statistics Regulation, to Head of Profession for Statistics, HM Revenue and Customs.