Quality and methodology

Question 1

1.  Quality summary

Accepted Answer

1.1 Important points about the UK House Price Index (UK HPI) data

It is important to note that:

to allow only pure price change to feed into the measurement of house price inflation, the UK HPI is mix adjusted to allow for differences between houses sold in different periods, for example the type or size of property
house price inflation is the rate at which the prices of residential property purchased in the UK rise and fall
the UK HPI is a joint production by HM Land Registry, Land and Property Services Northern Ireland (LPSNI), Office for National Statistics (ONS) and Registers of Scotland
UK HPI data is published monthly with historical data available from 1969 (the data is quarterly prior to 1995)
low number of sales transactions in some local authorities and London boroughs (such as Orkney Islands, Na h-Eileanan Siar, Shetland Islands and City of London) can lead to volatility at these levels (geographies with low number of sales transactions should be analysed in the context of their longer-term trends rather than focusing on monthly movements)

1.2 Overview of the UK HPI

The UK HPI captures changes in the value of residential properties and uses sales data collected on residential housing transactions, whether for cash or with a mortgage. Properties have been included:

in England and Wales since January 1995
in Scotland since January 2004
in Northern Ireland (NI) since January 2005

A derived series has been calculated back to 1968. Data is available at a national and regional level, as well as counties, local authorities and London boroughs.

Northern Ireland data is available on a quarterly basis. House prices, transaction volumes and growth rates are held constant for each month within the quarter. In the 2 months following the end of a quarter, when Northern Ireland results for the most recent quarter are not yet available, indices, house prices and growth rates for Northern Ireland are carried forward from the previous quarter. They are subsequently revised when the quarter is complete.

1.3 Uses and users of the UK HPI

The production of house price statistics is relevant for many purposes and has a variety of users who make use of housing market statistics to make a wide variety of decisions including provision of housing, whether to buy, and whether to lend. Main users and their uses of housing market statistics include:

central government – monitoring economic performance, policy making and regulation
local authorities – monitoring and developing housing policies
financial institutions – making decisions on whether to lend, how much to lend and setting interest rates
housing associations – assessing the number of people in housing need and making decisions on whether to purchase or build property to meet that need
housebuilders – assessing whether and where demand for new housing exists, and the returns received on homes built or converted
estate agents and letting agencies – tracking the number of properties sold and the price for which they were sold, as well as the types of properties and their location, and advising potential sellers on the achievable selling price of their property

1.4 Strengths and limitations of the UK HPI

The UK HPI is not as timely in publishing as other house price index measures published in the UK because it is based on completed sales at the end of the conveyancing process, rather than advertised or approved prices.

The strength of the UK HPI is that it has wide coverage of both cash and mortgage transactions and a large data source (land registrations such as that maintained by HM Land Registry) allowing data to be published down to a local authority level with further breakdowns available by property type, buyer status, funding status and property status.

Estimates for the most recent months are provisional and are likely to be updated as more data is incorporated into the index. While changes to estimates are small at the headline level, these can be larger at lower geographies due to the fewer transactions used. Further details on why our estimates change can be found in our revisions policy.

1.5 Recent improvements to the UK HPI

To reduce the size and impact of revisions in the UK HPI, from 12 December 2017 amendments were made to our estimation model when calculating our first estimate. Large historic downward revisions were mainly being driven by our new build estimate being too high, as the number of new build transactions in our first estimate was too small to produce reliable results. As the UK HPI is a mix of new (around 10%) and existing buildings (around 90%) this then impacted our first published estimate at the headline level. We have improved our estimation model to account for this. The impact of this methodology improvement is that the size of our revisions has since reduced and are no longer always downward. Due to this change, we also no longer publish a new build and existing build breakdown for the first two months. So, while our first estimates are more accurate you are losing some granularity in the breakdowns available in our first estimates.

Further detail on our approach was provided in our blog with the impact of this improvement discussed further in section 3.6: Recent methodological improvements.

Question 2

2.Quality  characteristics of the UK HPI

Accepted Answer

2.1 Relevance

The UK HPI was developed in response to the National Statistician’s Review of House Price Statistics (2010) and built on the October 2014 consultation on the development of a definitive house price index (PDF, 280KB) and subsequent published response.

We continue to seek feedback on how we can develop the UK HPI further. Through our LinkedIn UK HPI group we are working with users, stakeholders and our partners to enhance the information and data supplied to ensure the index continues to offer maximum public value.

If you would like to share your feedback on the UK HPI, you can register your interest to join the UK HPI group or contact us for more information.

2.2 Accuracy and reliability

Accuracy

The amount of time between the sale of a property and the registration of this information varies. It typically ranges between 2 weeks and 2 months. Occasionally the interval between sale and registration is longer than 2 months; this is particularly true for new builds. For example, our first estimate for a month is calculated using around 40% of the transactions that will ultimately get registered with the second and third estimates calculated based on around 80% and 90% of the final registered transactions. This means our estimates get revised as more data is incorporated into the index. Revision tables are published alongside the UK HPI release each month to help users evaluate the accuracy and reliability of our estimates.

We have several checks in place to minimise processing and data input errors, these include validation checks on data and data cleaning to remove erroneous data.

Find out more detail on our procedures in place to quality assure our data sources in our Quality Assurance of Administrative Data.

Reliability

In analysing historic UK HPI revisions (particularly those towards the end of 2016) we found that estimates were always revised down wards. Our analysis found that the large downward revisions were mainly being driven by the first published estimate for ‘new builds’ being too high. As the UK HPI is a mix of new (around 10%) and existing buildings (around 90%) this then impacts our first published estimate at the headline level.

Transactions involving the creation of a new register, such as new builds, are more complex and an increase in applications has created a backlog. As a result, processing takes longer. Read about what we’ve done to reduce our backlog, our service standards and our future plans.

To account for new builds taking more time to process, from 12 December 2017 we made amendments to our estimation model when calculating our first estimate. Further detail on this amendment can be found in section 3.

The impact of the processing backlog on the UK HPI is that we no longer publish a new build and existing build breakdown in our first and second estimate.

We will continue to monitor revisions to the UK HPI to ensure our first estimates remain robust as the backlog reduces. Revisions tables are published alongside the UK HPI data downloads each month to provide further transparency to users.

2.3 Output quality

The UK HPI represents growth in average house prices within a geographic area, this may differ to the growth of individual properties within that geography.

While average prices and growth rates at higher geographies are robust, low sales transactions in some local authorities can lead to volatility at these levels. While efforts are made to account for this volatility, the change in the price in these local levels can be influenced by the type and number of properties sold in any given period. Geographies with low number of sales transactions should be considered in the context of their longer-term trends rather than focusing on monthly movements.

2.4 Coherence and comparability

Currently, there are many different sources of house price statistics published in addition to the UK HPI. There will be differences in the data published by each source, as there are differences in both the data and methodology used. For example, Rightmove use asking prices, sources such as Nationwide and Halifax use their own mortgage approvals data, while the UK HPI uses data at the end of the conveyancing process, calculated based on completed sales. This means that the UK HPI is generally more complete than the other measures with coverage of both cash and mortgage transactions for the whole of the UK and available at a more granular level, however the consequence of this is that its publication is not as timely.

You can also read about the difference between the various house price index measures and their strengths and limitations in our publication Comparing house price indices in the UK.

2.5 Concepts and definitions

The UK HPI is calculated following recommended international best practice as defined in the Eurostat Residential Property Price Index handbook.

In December 2016 we commissioned an additional phase of expert peer review of the methods by a senior economist at the International Monetary Fund. This review confirmed that the UK HPI meets international standards and proposed some additional enhancements. These potential enhancements are summarised in a International Monetary Fund Working (IMF) Paper. We will investigate these proposed enhancements as part of our development plan.

2.6 Geography

Property postcodes are mapped to higher level geographies using the National Statistics Postcode Lookup and the Postcode Directory which can be accessed through the Open Geography portal from the Office for National Statistics.

2.7 Accessibility and clarity

The UK HPI data is published under Open Government Licence. Data is made available in several formats each month:

reports are provided for the UK, England, Scotland and Wales, with a separate link to the Northern Ireland House Price Index report which is published quarterly
the underlying UK HPI data can be downloaded in comma-separated values (CSV) format through UK HPI data downloads, or Terse RDF Triple Language (Turtle) formats, and the SPARQL query generated in the background view
the Search the UK House Price Index tool allows customers to produce printable reports derived from the UK House Price Index data

More details on related releases can be found on the announcements on GOV.UK. If there are any changes to the pre-announced release schedule, public attention will be drawn to the change and the reasons for the change will be explained fully.

2.8 Timeliness and punctuality

The UK HPI follows a monthly publication schedule and is lagged by 2 months. This lag is due mainly to the time it takes to collect and process the data.

The exact time of publication of the HPI depends on data availability and delivery, that’ why the HPI statistical bulletin is published either on the second or third Wednesday of the second month after the reference period.

Calendar release dates are available on the website and provide 12 months’ advance notice of release dates. In the unlikely event of a change to the pre-announced release schedule, public attention will be drawn to the change and the reasons for the change will be explained fully at the same time, as set out in the Code of Practice for Statistics.

2.9 The trustworthiness of our data

We commit to the pillars, principles and practices of the Code of Practice for Statistics in producing the UK HPI.

The Office for Statistics Regulation has designated the UK House Price Index as a National Statistic. A letter from the Director General for Regulation details the actions that were taken to meet the requirements as set out in the UK HPI assessment report.

Question 3

3.  Methods used to produce the UK HPI

Accepted Answer

The UK HPI brings together several comprehensive administrative data sources. There is an increasing reliance on the use of administrative data in the production of official statistics. This has been recognised by the Office for Statistics Regulation, who published a Regulatory Standard for the quality assurance of administrative data. While the data sources used in the UK HPI are summarized in this section, more comprehensive information on these data sources and how we have assessed the quality of them can be found in our Quality Assurance of Administrative data.

The data sources used to produce the UK HPI fall into 2 distinct categories; price data or property attributes data.

The price data provides details of the final transaction price at which a residential property has been sold (this source of data does include limited property attributes data) whilst the property attributes data provides details regarding the actual property, such as the type of property (for example detached, semi-detached), the size of the property (number or rooms or floor space) and the location of the property. Combining the detailed property attributes data with the price data provides a comprehensive and robust dataset required for use in a hedonic regression model, which is defined in section 3.2.

3.1 Main data sources

Price Data: HM Land Registry for England & Wales

Data on residential property transactions for England and Wales, collected as part of the official registration process are provided by HM Land Registry for properties that are sold for value.

There are a few registrations excluded from the data used in the UK HPI. These include sales of residential property not sold for value such as the sale of part of a property, a transfer between parties on divorce, ‘Right to buy’ sales at a discount, remortgages etc. In addition, all commercial transactions of residential properties are excluded from the dataset (where transactions of residential properties involve a transfer to a corporate body, company or business).

Access the quality assurance of administrative data used in the production of UK HPI – HM Land Registry data.

Price Data: Registers of Scotland house price data

Similar to HM Land Registry data, residential property sales for Scotland are provided by Registers of Scotland (RoS) for use in the calculation of the UK HPI. The Scotland price data provided for the UK HPI is a by-product of the Scottish property registration process, whereby property sales are submitted to RoS on completion of a sale.

The coverage of the RoS data differs to the Price Paid Data slightly in that transactions relating to residential properties where the buyer or seller is a corporate body, company or business are included within this dataset. Its coverage is like that of the LPSNI as these transactions cannot be separately identified on the dataset.

Access the quality assurance of administrative data used in the production of UK HPI – Registers of Scotland data.

Price Data: Northern Ireland price data

Land and Property Services/Northern Ireland Statistics & Research Agency calculate the Northern Ireland House Price Index. This data is provided to the Office for National Statistics on a quarterly basis and combined with the Great Britain data to give overall figures for the UK.

Property attribute data: UK Finance – Regulated Mortgage Survey

The Regulated Mortgage Survey (RMS) is UK Finance’s version of the Mortgage Product Sales Data (PSD) that all regulated lenders report to the Financial Conduct Authority (FCA). This is detailed transaction level data on mortgage completions. Starting in April 2005, the RMS now contains over 12 million individual mortgage sale records.

The RMS is the only comprehensive source of data available for the type of borrower and provides the necessary data to allow the UK HPI to be produced according to whether the buyer is a first-time buyer or an existing owner.

Access the quality assurance of administrative data used in the production of UK HPI – UK Finance.

Property attribute data: Valuation Office Agency Council Tax Valuation list

The main source of property attributes data that is used to supplement house price data for England and Wales is administrative data taken from the Council Tax Valuation list maintained by the Valuation Office Agency (VOA). The VOA has been responsible for banding properties for Council Tax since the tax was first introduced in 1993; before then, the VOA was responsible for the earlier system of domestic rates. The Council Tax Valuation list is a robust source of property attributes (such as the size of the property) data that covers, in principle, all residential properties in England and Wales.

Access the quality assurance of administrative data used in the production of UK HPI – Valuation Office Agency Council Tax Valuation Lists.

Property attribute data: Land and Property Services Northern Ireland valuation vist

Land and Property Services maintain the list of Northern Ireland properties which are valued for rating purposes. The data are maintained and validated similarly to the VOA data described above. The valuation list database contains all the property attributes required for the hedonic regression used in the production of the NI Residential Property Price Index (RPPI).

Access the quality assurance of administrative data used in the production of UK HPI and Northern Ireland House Price Index – Land and Property Services Northern Ireland valuation list.

Property attribute data: Energy Performance Certificates Scotland

Scottish Energy Performance Certificates (EPC) are used to provide the floor area of the property and the number of habitable rooms required for calculation of the UK HPI for Scotland. This is matched against the price paid data provided by RoS (using the address details of the property).

EPCs were introduced to comply with European legislation which requires that an EPC be provided on construction, sale or rental of a building to a new tenant.

The legislation establishing the EPCs, The Energy Performance of Buildings (Scotland) Regulations 2008, came into force towards the end of 2008, meaning that EPCs exist for all residential property transactions that have taken place since January 2009.

Access the quality assurance of administrative data used in the production of UK HPI – Scottish Energy Performance Certificates.

Property attribute data: Acorn classification

A key determinant of house prices is the demographic characteristics of the area in which the property is located, such as the affluence of those people living in the area. A well-established geo-demographic segmentation of the UK is available through the Acorn dataset, produced and licensed by CACI Ltd. Acorn segments postcodes into categories and groups by analysing significant social factors and behaviours. Access the quality assurance of administrative data used in the production of UK HPI – Acorn (CACI Ltd).

3.2 How we process the data

Adjusting for the mix of properties

A house price index (HPI) is a series that tracks the changes in the price of property relative to the price it had at a reference period. Changes in the series represent increases and decreases in house prices. One of its main features is the mix-adjustment of the monthly transactions to remove the effect of changing composition of sold properties, to make sure we are comparing like with like. The index uses hedonic regression to perform the mix adjustment.

Valuing a property

In a hedonic regression, properties are defined in terms of a set of characteristics, each of which contributes to the price paid for a property. For example, the number of bedrooms, or the location of the property will contribute to the amount paid, but no features can be priced in isolation.

Each month our price data (for example, HM Land Registry) is matched to the property attribute data (for example VOA) data so that for each transaction we have a price and a set of characteristics.

A regression model is used to estimate the value of each characteristic from the set of properties during a period. For example, the model might estimate the effect that every additional room and each different location have in the sale price in a certain month. Then, the price of a property can be calculated by combining the values assigned to each of its features. This method allows us to estimate the prices of properties with every combination of features (such as number of rooms and regions), even if that combination did not trade in the period.

A fuller description of this method and other alternative methods for calculating house price indices can be found in the Handbook on Residential Property Price Index.

In the case of the UK HPI, the price-determining characteristics are as follows:

local authority district in Great Britain and housing market area in Northern Ireland
Acorn area classification variable (groups)
property type (such as detached, semi-detached, terraced, flat)
floor area (metres squared)
number of rooms
new or old property

Equation 1

Mathematically, a semi-log model is used, of the form:

In this equation, the logarithm of the price paid is used because house prices tend to be log-normally distributed - meaning. the frequency distribution of the log of the price is bell-shaped.

The price determining characteristics are combined to give a predicted price for each property. These predicted prices are then averaged using a geometric mean, which involves multiplying the ‘n’ predicted prices together, and then taking the nth root. We consulted with international experts and the GSS Methodological Advisory Committee when deciding which was the most appropriate average to use. The general view was that the geometric mean was the preferred measure, as it is less distorted by high values. The ratio of the geometric mean average prices in successive time periods then gives the price index.

The Acorn area classification code is available at different levels of detail. The group level, comprising 18 groups, is used in the regression analysis for GB, whereas the 6 category level is used in NI due to the smaller population. The room’s variable is treated as a categorical variable, where a coefficient is calculated for each number of rooms in the property, up to a maximum of eight, rather than a rate per room. The quality of the rooms data in NI is insufficient to allow it to be included in the NI model.

It is not possible to measure all the characteristics that may influence prices. For example, qualitative factors relating to the condition of the properties, amount of traffic, distance to shopping or places of work etc are not measured. Consequently, it is not possible to explain all of the variation in prices that is observed. However, the characteristics used in the equations in this study generally explain around 80% of the variation.

A process flow of this can be found within the Summary of UK HPI production process within ONS (PDF, 315KB).

Missing characteristics

When running the hedonic regression model, some properties may be missing one or more of their price determining characteristics – for instance, floor area may not be available. These properties are still used in the regressions, but are given less weight in the calculations depending on the importance of the missing variable as a price determinant. For instance, floor area is found to have more of a bearing on price than whether the property is new or old and existing property, so a property with missing floor area will have a lower weight than one missing the new or old existing property indicator.

A key determinant of house prices is the demographic characteristics of the area in which the property is located. The UK HPI uses the socio-demographic classification, known as Acorn (produced and licensed by CACI Ltd), in the hedonic regression model to measure the affluence of the area.

Prior to 20 December 2023’s publication, property transactions in Great Britain were excluded from the regression model if their Acorn classification was missing. From 20 December 2023’s publication, these properties are included in the regression model from January 2023’s data onwards, but are given less weight in the calculations, as described above. This methodology improvement aligns how transactions with missing Acorn classification are used in the Great Britain model and Northern Ireland model, increasing coherence across the UK and improving the quality of UK HPI statistics.

Calculating first estimates

As noted previously in this article; it typically takes between 2 weeks and 2 months to complete the registration process, with purchases of new properties taking longer than pre-existing properties.Further analysis also seemed to suggest that more expensive properties are registered slightly quicker than other properties. As the number of new build transactions in our first estimate is too low to enable us to produce reliable estimates for this breakdown the ‘new or old property’ variable has been removed from our estimation model when calculating our first estimates.

Equation 2

Since 12 December 2017 (October 2017 estimates) our first estimate is calculated as follows:

This approach has only been applied in calculating the first estimate. The calculation of previous periods remains as stated in Equation 1. Further details on the reasons for this change and its impact can be found in section 3.5.

Weighting the price index

The UK HPI is mix-adjusted to allow for the fact that different houses are sold in different periods; by annually updating the weights (section 3.4.1 of ‘Official House Prices Explained (PDF,974KB)’ provides a useful worked example of the need for mix-adjustment). The mix-adjustment weights are calculated annually and are calculated using the latest complete years’ worth of transactions.

For example, the weights used in 2018 are calculated based on transactions in 2017. In constructing these weights all properties need to have a complete set of characteristics. This is achieved by imputing for missing values using a ‘nearest neighbour’ method, whereby the missing value is replaced by a non-missing value ‘donated’ from a randomly chosen property with the same values for the non-missing variables.

It should be noted that imputation also takes place for buyer status (for example first time or former owner-occupier) and cash or mortgage which means that average predicted prices, and hence price indices, can be calculated for these variables. Buyer status is obtained in the first instance by matching data from UK Finance with data from HM Land Registry and Registers of Scotland. This will, by definition, not cover cash purchases; for these, it is assumed that they are all purchases by former owner-occupiers.

Publication of average prices

The process of mix-adjustment requires that, in each January, a fixed basket of properties is updated to reflect changes in the composition of properties being sold. This basket is then used to produce modelled prices for the year, before the basket is then updated again in the subsequent January. This means that the average prices produced from a fixed basket in 2016 are not directly comparable with the average price produced using the 2017 basket as they will reflect a different mix of properties

To produce comparable house prices over time a base set of average prices is uprated with the price index. To ensure the base set of transactions remains representative, the base period for the price series will be updated every five years, and the whole of the average price series rescaled to align with the new base period.

For example, the initial price series calculated for the UK HPI is January 2015 as the base for average prices. This base will be updated every 5 years (so the next update will move the base to January 2020). This is illustrated in Table 1.

Table 1: Example of re-referencing in the UK HPI

Year	Price index (2015=100)	2015 base year price series	Actual average price	2020 base year price series
2010	86	170,000		172000
2011	85	168,000		170000
2012	85	169,000		170000
2013	87	173,000		175000
2014	94	187,000		189000
2015	100	198,000	198,000	200000
2016	107	212,000		214000
2017	112	221,000		224000
2018	115	228,000		230000
2019	120	238,000		240000
2020	125	248,000	250,000	250000
2021	130	257,000		260000

Note: the data from 2018 onwards is constructed data for illustrative purposes only.

In the example (which is based on annual data, as opposed to monthly for ease of presentation), the growth remains unchanged by the re-referencing of the price base period from 2015 to 2020 but the average prices are all scaled up by 0.8%, which is the percentage difference between the actual price for 2020 and the 2015-based average price for that year. This approach ensures that a set of comparable average prices are published, and that these prices remain representative of the current market.

3.3 How we analyse the data

Once the data has been aggregated the resulting series are analysed by various breakdowns, over time, and against other published sources of house price growth. Any unexpected movements within the series are explored through the record level data. Monthly curiosity meetings are held to review the new data and discuss any long-term trends in the data and its drivers.

3.4 How we quality assure the data

Quality assurance of each of our data sources can be found in our published guidance Quality Assurance of Administrative Data.

In running our hedonic regression each month, test statistics are analysed to ensure the model has run correctly and fit successfully. This includes analysing the R squared of the model (model fit) and significance of the explanatory variables. An R squared of around 0.8 is achieved. This means that 80% of the variation in price is captured by the explanatory variables. An R squared of 0.8 is high. The old ONS HPI had an R squared of around 0.7.

Following the calculation of new estimates, revision analysis is conducted. This includes:

revision analysis between first and subsequent estimates at a country and regional level
revision analysis between new revised estimates and previous published estimates at all published geographical levels

Revisions tables at a country level are published alongside the UK HPI data downloads each month to provide further transparency to users.

3.5 How we present the data

The UK HPI release is published by HM Land Registry monthly on GOV.UK. Data is made available through:

UK and country specific reports
CSV downloadable data
interactive tool

High level summaries are also published by the Office for National Statistics and Registers of Scotland which direct users to the main release on GOV.UK.

Data collected from HM Land Registry, Registers of Scotland and HMRC stamp duty land tax is available from 1995 for England and Wales, 2004 for Scotland and 2005 for Northern Ireland. A longer UK HPI back series has been derived by using the historic path in the (now discontinued) ONS HPI (Excel, 3.7MB). Through this, a derived series is available back to 1968 at the regional level and above.

Local authorities, boroughs, counties and metropolitan areas

A 3-month moving average has been applied to estimates below the regional level to reduce volatility caused by low sales transactions in some local authorities and London boroughs. For example, at the local authority level, the published estimate for March is a simple average of the calculated estimates for January, February and March.

Volatility remains in estimates for local authorities with fewer transactions, such as Shetland Islands, Orkney, City of London and Na h-Eileanan. These series should be considered in the context of their long-term trend rather than focusing on their monthly movements.

A 3 month moving average is not applied for Northern Ireland estimates as they are only available on a quarterly basis.

Seasonal adjustment

The purpose of seasonal adjustment is to remove systematic calendar-related variation associated with the time of the year; that is, seasonal effects.

This facilitates comparisons between consecutive time periods. While the headline index for the UK HPI is non-seasonally adjusted, seasonally adjusted series are made available within the downloadable data. Seasonally adjusted average prices and indices are available at a regional level and above.

Find out more detail on the process for seasonal adjustment on the Office for National Statistics website. Seasonal adjustment parameters are reviewed on an annual basis.

3.6 Recent methodological improvements

This section provides further detail on the recent Improvements to the UK HPI summarized in section 1.5.

Revisions in the UK HPI

The amount of time between the sale of a property and the registration of this information varies. It typically ranges between 2 weeks and 2 months. Occasionally the interval between sale and registration is longer than 2 months. Because of the lags involved in registering properties (particularly new builds), the UK HPI when it is first published is based on around 40% of the total volume of transactions in the most recent month. Consequently, the UK HPI is subject to revisions with the latest month being revisable in subsequent publications.

While revisions themselves are unavoidable given the nature of the data, some users commented about the size and direction of these revisions, which were more evident towards the end of 2016, as presented in Figure 1. While one might expect revisions to be both upwards and downwards, those of the UK HPI were always downwards.

Figure 1: 12 month growth rate for first and final estimate of UK HPI, Great Britain

Notes:
1. UK HPI has a 12 month revision policy; data within this period are still subject to revision.
2. Great Britain data is presented here only as Northern Ireland data is processed separately by LPSNI.

Our analysis found that the large revisions were mainly being driven by the first estimate for ‘new builds’ being too high. While new builds ultimately make up around 10% of all transactions, they contributed to over 80% of the revision at the end of 2016.

Transactions involving the creation of a new register, such as new builds, are more complex and need more time to process. This means they can take longer to appear in HM Land Registry’s register. This leads to there being fewer new build transactions available to us when constructing our first estimate which impacts the reliability of our first estimate for this breakdown.

The fewer new build transactions in the first estimates is highlighted in Table 2 which presents the number of transactions used in the first estimate during 2016 for England and Wales.

Table 2: Number of transactions in first estimate, England and Wales

Time	New build	Existing property
July 2016	131	26,930
August 2016	128	30,213
September 2016	85	29,988
October 2016	96	28,634
November 2016	138	24,438
December 2016	131	28,189

Note: data for Scotland and Northern Ireland are not presented here.

While the number of new build transactions reported in the first estimate is small, by comparison, by the third estimate they account for around one tenth of all transactions (around 7,000 to 8,000 transactions). The number of new build transactions reported in the first estimate is too small to produce reliable results for the first monthly estimate. We also found evidence that more expensive properties are registered slightly quicker than other properties, explaining why new build estimates were subsequently revised down, as less expensive new build transactions were incorporated into the index at a later point. These fewer new build transactions in the first estimate are not representative of new build transactions which are registered slightly later.

As seen in figure 1 from around March 2017 the difference between the first and final estimate reduced. In May 2017 (March 2017 data) we introduced a temporary adjustment factor, based on observed revisions in previous periods, to help control for the volatility of estimates for new build transactions. While this approach did improve UK HPI revisions, a better long-term solution was investigated and is now implemented. Several approaches were tested before choosing this longer-term solution. These included:

calculating bias adjustment factors using a time series approach
calculating a bias adjustment factor based on observed revisions in previous periods
a reduced estimation model when calculating the first estimate

The implemented solution (reduced estimation model) was selected as it resulted in the smallest revisions over time for countries of Great Britain. It also resulted in both positive and negative revisions for the periods tested. Two additional benefits we found for the reduced estimation model over a bias adjustment factor approach were:

it was easier to implement it into our production process
it responded more quickly to changes in the structure of the transaction data over time (such as a reduction in the backlog)

Our implemented methodological improvement and its impact (if it had been applied historically) are presented in more detail below.

Methodological improvement

Our estimation model contains the following variables:

local authority district
Acorn area classification
floor space (metres squared)
number of rooms
new or existing property

As the number of new build transactions in the first estimate is too small to produce reliable results the ‘new or existing property’ variable is now excluded from the estimation model in calculating the first estimate. This method was implemented from the October release (12 December 2017). As noted, the change to the estimation model is only applied when calculating the first estimate; the estimation model for other periods are unaffected. For example, in calculating our second estimate for a month we now have (since March publication) more than 2,500 new build transactions. We estimate this accounts for around 40% of the new build transactions we will ultimately receive. This is a sufficient number of transactions for the ‘new or existing property’ variable to be included in the estimation model.

Figure 2: 12 month growth rate for first, final and first estimate under new method, Great Britain

The impact of applying this improvement historically is presented in Figure 2 above. While revisions would still be observed, under the new approach, they would be much smaller and both positive and negative. Our intention to implement this change was published within About the UK House Price Index and is an amendment to our methodology to improve the accuracy of our first estimates.

HM Land Registry’s speed of service provides detailed information on average completion times for new applications and existing applications.

We will continue to monitor revisions to the UK HPI each month to ensure our first estimates remain accurate and reliable. Revisions tables are published alongside the UK HPI data downloads each month to provide further transparency to users.