The FCO’s commitment to transparency
The FCO is committed to increasing public knowledge about foreign policy and the work of the FCO. As the Minister for Europe stated to Parliament in December 2013, the FCO remains fully committed both to complying with our public records obligations and to doing so with maximum transparency.
The special collection files
The Foreign and Commonwealth Office (FCO), like all government departments, maintains an archive of its departmental files. We hold around 600,000 of these files, most of which are not yet due for transfer to The National Archives (TNA).
The FCO holds a further collection of files, also estimated at 600,000, which are known as “special collections” – those which are outside the normal FCO filing system. These special collection files are generally older than the departmental files and were created mainly by the FCO or its predecessors (such as the Colonial Office).
Under the Public Records Act, government departments are required to transfer files selected for permanent preservation to TNA in line with TNA’s collection policy and the statutory timetable. However the FCO’s transfer of special collection files has been delayed because we have given priority to the transfer to TNA of departmental files and the colonial administration files. This means we continue to hold a large collection of special collection files.
It’s important to note that all the files we hold which are overdue for transfer are retained in compliance with the Public Records Act. Our retention of these files is authorised through a legal instrument (known as a Lord Chancellor’s Instrument) which, at the FCO’s request, has been granted by the Lord Chancellor. This Instrument lasts until the end of 2014 and was granted in order to allow the FCO time to develop a release programme. The scale of the operation to review and release the files should not be underestimated.
In 2013, an external company created an inventory of the special collections (see link below) and initially estimated we held 1.2 million special collection files. However they subsequently revised this estimate to 600,000 following a reassessment of the amount of microform material.
The exact number of files will only be known when the files are prepared for transfer to TNA, at which stage each file is individually listed.
View the Foreign Office’s Archive Inventory
Release of the special collection records
The Minister of State for Europe, the Rt Hon David Lidington, informed Parliament on 12 December 2013 that the FCO was developing plans for the review and release of its legacy records.
A high level plan for the selection, review and release of the special collection files was presented by the FCO to the Lord Chancellor’s Advisory Council on National Records and Archives at its meeting on 14 November 2013 and we will be providing the Council with an update on progress in May 2014.
We have divided the special collections into four categories as a first step towards planning the release of these records: High, Medium and Low priority categories and a separate category for the Hong Kong records (due partly to the large volume of records on microform). This prioritisation has taken into account feedback from a wide variety of sources and interested parties. We are continuing to work closely with interested parties as we develop our release plans.
Our initial assessment is that 10% of the special collection records should be considered high priority and reviewed for release first (see table below). Over a six-year period starting this year (2014), we will aim to review all high priority special collection records and to prepare for transfer to TNA all of those records which fall within TNA’s collection policy, subject to legal exemptions. Special collection material we assess to be high priority includes colonial records, which complement TNA Colonial Office (CO) and Dominions Office (DO) series, and records relating to the two World Wars (such as claims files relating to World War II and the Control Commission for Germany series). The first special collection record series we will prepare for transfer to TNA will consist of around 4,000 files relating to British Nationals who suffered Nazi persecution, and their subsequent claims for compensation. These case files contain unique and detailed first person accounts of the Holocaust. The TNA Records Decision Panel has accepted FCO’s proposal that these files should be transferred to TNA.
FCO’s initial prioritisation of the special collection records
|Priority||Estimated number of files to be reviewed for permanent preservation|
|Medium and low priority||273961|
|Records from the former British administration of Hong Kong||267956|
|Status as public records to be determined (e.g. some of this material may have been commercially published and may not fall within the scope of the Public Records Act)||8220|
|Requires further assessment (e.g. legacy formats which require special equipment to access such as legacy microform and reel-to-reel audio/film)||27|
Following the transfer to TNA of selected high priority special collection files, we plan to focus on the review and release of selected medium and low priority files, and a further phase of the project dedicated to the Hong Kong files. We will publish details of these phases of the project in due course.
The table below provides a broad categorisation by subject of the special collection records.
|Category||Estimated number of items|
|Colonial office records / Colonial period||5801|
|Commercial publications or likely to be in the public domain||801|
|Confidential print (material reprinted from original records. Some of this material may already be at TNA in its original form)||7676|
|Foreign Compensation Commission and claims||170917|
|Industrial and commercial||2867|
|Information Research Department and intelligence-related material||23210|
|Other government departments or international organisation material||15953|
|Other diplomatic material||3597|
|Other material currently being assessed||1382|
|Private Office and senior management papers||4468|
|Registers and indices||21342|
|World War II interest inc Allied Control Commission||10801|
We will be providing regular updates via this page on our release plans and on our progress in making the special collection records available publicly.
The role of the Independent Reviewer
On 12 December 2013 the Minister for Europe announced in a written statement to Parliament the appointment of Professor Tony Badger as Independent Reviewer of the special collections. Professor Badger is Paul Mellon Professor of American History and Master of Clare College Cambridge. Professor Badger has made the following statement about his role as Independent Reviewer:
“It is difficult to overestimate the scale of the process of releasing over 600,000 files. By way of comparison, the release of the migrated colonial archive, which I have just overseen, involved just under 20,000 files. Last year the FCO transferred approximately 12,000 files (chiefly colonial administration files).
As the Independent Reviewer, I intend:
i) to come to a more complete understanding of how the special collections came into being, why it has taken so long to secure the release of the papers, and to make public those findings.
ii) to approve the determination of the prioritisation of the materials to be released. While this eclectic collection clearly contains a good deal of low-level administrative material that has little or no historical value, the collection also contains some extraordinarily valuable papers that should be transferred to The National Archives as soon as possible. The prioritisation needs to be decided in consultation with the academic community, in addition to The National Archives, and to take into account areas of current public interest and concern.
iii) to provide assurance that the Archive Management Team has the capability to drive this release forward without further delaying the release of papers under annual transfer arrangements.
iv) to provide assurance that files and documents are not retained or redacted unnecessarily in the process of sensitivity review.
v) to provide assurance about the selection of files to be transferred to The National Archives. In the case of the migrated colonial archive, it was agreed from the start that every document should be transferred, subject to legal exemptions. Given the size of the special collections and the routine administrative nature of significant parts of the material, it would be both impractical and unnecessary to transfer every document. But it is imperative that there should be public confidence in the process of decision-making in the FCO and The National Archives.
As in the oversight of the transfer of the migrated archive, I intend to consult widely with interested academics and to encourage, and to respond to, queries from journalists and interested members of the public.
Colonial Administration files
The Foreign and Commonwealth Office has made available to the public a large collection of files from former British territories, sometimes known as the “migrated archives”. The files are now available at the National Archives.
In June 2011 the Foreign Secretary appointed Professor Badger from the University of Cambridge as the Independent Reviewer to oversee the release of these files.
Professor Badger approved a timetable for the transfer of the migrated archive files to The National Archives. The first tranche of files was released at TNA on 18 April 2012, the second tranche on 27 July 2012, the third tranche on 28 September 2012, the fourth tranche on 30 November 2012, the fifth tranche on 26 April 2013, the sixth tranche on 30 July 2013, the seventh tranche on 27 September 2013 and the eighth and final tranche was made available on 29 November 2013.
Tranche 8 comprises a total of 1,907 files from Malta, Singapore, Tanganyika, Trinidad and Tobago, Turks and Caicos Islands, Uganda, West Indies, Western Pacific and Zanzibar. It also includes 913 Kenya Land files and 70 Related files.
The Kenya Land files deal with various aspects of land settlement and transfer, schemes of finance, correspondence with individuals and international institutions and aid to the Kenyan Government. These files were created by the British High Commission in Nairobi.
The Related files are records created by the Colonial Office Intelligence and Security Department and the records management branch of the FCO. These records provide background information on the colonial administration files and how they have been managed over time, including some destruction records.
Professor Badger has made the following statement about the Tranche Eight release:
I welcome the release of the eighth and final tranche of the colonial administration files. I am satisfied through my work as Independent Reviewer that the release of these files has been a very conscientious and transparent process and that the Foreign Secretary’s goal to release every paper in this collection (subject to legal exemptions) has been achieved.
I am aware that there is academic and public interest in the processes and decision-making governing the redaction (blocking) of content in the colonial administration files under legal exemptions. I have personally inspected a number of justifications for closure which the FCO has submitted to the Lord Chancellor’s Advisory Council on National Records and Archives, including all Cyprus justifications for an entire tranche.
I am confident that the level of redaction is very low and that frequently it is only small amounts of text in a document (such as an individual’s name) which are withheld. It is much rarer for whole documents to be withheld. At my request, the FCO has already published closure and retention statistics for the colonial administration files on this Web page up to and including Tranche 7.
The FCO has provided me with an assurance that the Tranche 8 statistics will be published as soon as soon as they are available.
I have made a video podcast providing an overview of my role as Independent Reviewer:
I am also preparing a report for the Foreign Secretary on the release of the colonial administration files which will be made available on this page. The files were transferred in alphabetical order of the colonial territory concerned with the exception of prioritised release for Kenya, Cyprus, British India Ocean Territories (BIOT) and Malaya files where there has been particular interest.
An updated version of the transfer timetable is shown below.
Timetable for release of the Colonial Administration files
|Territory||Timetable for release|
|Bahamas (batch 1)||Available|
|Bahamas (batch 2)||Available|
|Basutoland (batch 1)||Available|
|Basutoland (batch 2)||Available|
|British Indian Ocean Territories and Seychelles||Available|
|Ceylon (batch 1)||Available|
|Ceylon (batch 2)||Available|
|Ceylon (batch 3)||Available|
|Ceylon (batch 4)||Available|
|Cyprus (batch 1)||Available|
|Cyprus (batch 2)||Available|
|Cyprus (batch 3)||Available|
|Cyprus (batch 4)||Available|
|Cyprus (batch 5)||Available|
|Cyprus (batch 6)||Available|
|Gilbert and Ellice Islands||Available|
|Kenya (batch 1)||Available|
|Kenya (batch 2)||Available|
|Kenya (batch 3)||Available|
|Kenya Land Files||Available|
|Malta (batch 1)||Available|
|Malta (batch 2)||Available|
|Malta (batch 3)||Available|
|Malta (batch 4)||Available|
|Related files (see FAQs)||Available|
|Seychelles||See British Indian Ocean Territories|
|Singapore (batch 1)||Available|
|Singapore (batch 2)||Available|
|Singapore (batch 3)||Available|
|Tanganyika (batch 1)||Available|
|Tanganyika (batch 2)||Available|
|Turks and Caicos||Available|
The Foreign Secretary has appointed Professor Tony Badger, Paul Mellon Professor of American history and Master of Clare College at the University of Cambridge, to provide independent oversight of the transfer. The Foreign Secretary announced the appointment of the independent reviewer in a Written Ministerial Statement to Parliament on 30 June 2011. The transfer of colonial administration files (as with all public records) is also subject to scrutiny by the Lord Chancellor’s Advisory Council on National Records and Archives.
Releasing the files
In some cases, it is possible to release all of the files for a particular territory in a single batch. However, this is not always possible because of the number of files from some territories. Where the timetable indicates release over an extended period, this means that the files for that territory will become available in a number of separate batches. This approach ensures that files are available as soon as they have been prepared for public release.
The transfer of such a large collection of files involves a significant amount of work by both FCO and TNA staff. The transfer process from FCO to TNA includes the review of files for any residual sensitivity, physical preparation to ensure minimal deterioration over time, and the cataloguing of files so they can be identified via the TNA Catalogue (using the TNA reference number FCO 141). Further information on the UK public records system is available on the TNA Website.
All of the files in the FCO’s possession will be transferred to the TNA subject to any legal exemptions.
Information withheld by the FCO under legal exemptions
On average, the FCO only withholds 1% of the information contained in files eligible for transfer to the TNA.
Following the sensitivity review process, whole files, whole documents or parts of documents may be closed or retained for a specified and approved period if the content is deemed to be sensitive. If part of a document is withheld, sensitive content will be redacted (blanked out).
Information can be closed under Freedom of Information Act (FOI) exemptions. Information can only be closed if the FCO makes an application for closure to the Lord Chancellor’s Advisory Council on National Records and Archives. Once the specified period of closure has expired, the FCO must re-review the closed material and either release it or submit an application for a further period of closure. The exemptions which the FCO uses most frequently for closure are Section 27 (international relations), Section 38 (Health and safety) and Section 40 (personal information).
Closed information is held by The National Archives (TNA) securely and separately from open records. Under Section 66 of the Freedom of Information Act, TNA is responsible for answering Freedom of Information requests for closed material (in consultation with government departments). Further information on the legal basis and process for closing information is available in the TNA publication Access to Public Records (p.7ff.)
The FCO may also retain information under Section 3(4) of the Public Records Act. Applications for retention are made to the Lord Chancellor’s Advisory Council and the subsequent Lord Chancellor’s Instruments are signed by the Lord Chancellor. Retained information is held by the FCO for a specified period and the FCO is responsible for answering any Freedom of Information requests for retained material (please see the FCO’s home page on gov.uk for details of how to make an FOI request to the FCO). The commonest grounds for retention are described on pages 26-27 of the TNA’s Access to Public Records. Records can be retained on grounds of sensitivity, for business use by the department or because of a backlog of records awaiting review.
In November 2011, the Lord Chancellor authorised the retention of all of the colonial administration files for two years in order to allow time for the FCO to review and release the files. This specific authorisation expires in December 2013 and any information which the FCO continues to withhold after that date from these files must be closed or retained by further applications to the Lord Chancellor’s Advisory Council.
In some instances, the Lord Chancellor has given his approval for the retention of large categories of records of a similar character across government departments. This means departments are not required to apply to the Lord Chancellor’s Advisory Council on a case-by-case basis to retain qualifying records. The most widely used is the Security and Intelligence Instrument which covers records relating to security or intelligence service activities. Some migrated archive records are retained under this Instrument.
How many closures and retentions are there for the colonial administration files?
Top Secret colonial administration files
Prior to the to the relocation of the FCO’s records branch to Hanslope Park, near Milton Keynes, a collection of Top Secret colonial administration files (170 boxes) was held in a records store in the FCO’s Old Admiralty Building in London (and prior to that in Curtis Green, London, a former FCO building). The files were known to be in existence in 1991 but the FCO has not yet been able to confirm definitively whether this collection of files is still in existence. A witness statement given to the High Court in 2011 documents an extensive search carried out for the Kenya Top Secret colonial administration files. None of these Top Secret Kenya files have been found.
The FCO has also uncovered evidence that some Top Secret colonial administration files may have been included in a review of FCO post files from late 1991 to 1992 and that this review may have led to the destruction of some of the Top Secret colonial administration files. To date, this evidence only relates to Singapore files. An unregistered Library and Records Department file entitled “Singapore: pre-independence records” contains a list of Top Secret colonial administration files from Singapore. The list is stamped “Destroyed under statute. Sect 3(6) Public Records Act 1958”.
However, since the original search for the Top Secret Kenya files was carried out, the FCO discovered towards the end of 2012 three Top Secret Cyprus colonial administration files which are believed to have come from the original Top Secret collection. These files were discovered during a routine search of a miscellaneous collection of papers. The FCO continues to search for evidence for the destruction or indeed preservation of the Top Secret files.
Kenya colonial administration files
171 Kenya files were released as part of Tranche 5 on 26 April 2013. With the exception of any legally exempted material, this completed the transfer of Kenya colonial administration files to The National Archives.
British Guiana colonial administration files
The FCO’s current assessment is that there are no records from British Guiana amongst the colonial administration files. There are two reasons for this assessment.
First, the FCO has not identified any physical files from British Guiana among the collection of migrated archive records which were transferred to the UK.
Second, there is documentary evidence of the destruction of British Guiana records in a Colonial Office Intelligence and Security Department file from the 1960s entitled “Annual Return of Accountable Documents: British Guiana”. This file, which was released on 27th July 2012 as part of tranche 2 (TNA reference FCO 141/13100) contains correspondence about the destruction of British Guiana records as well as lists of files which have been destroyed.
The audit trail for destruction is not complete, but the file indicates that significant numbers of British Guiana records were destroyed.
Southern Rhodesia colonial administration files
Some Southern Rhodesia files are held at The National Archives (TNA) in the DO 154 series, chiefly from the early to mid-nineteen sixties. The selection of these files for transfer to the Public Record Office (subsequently the TNA) is documented in a file list held by the FCO which also contains the following note from 2001: “All files on this list have been sent to the PRO [Public Record Office, now The National Archives]. All other files from 1955 to 1965 have been destroyed.”
There is further evidence for the destruction of Southern Rhodesia files in an unregistered Library and Records Department (LRD) file entitled “Southern Rhodesia pre-independence records” which will be released to the TNA by the end of 2013. It contains a number of file lists stamped “Destroyed under statute. Sect. 3(6) Public Records Act 1958.” The same file records advice from the UK mission in Salisbury “that practically all pre U.D.I. material had been destroyed”.
There are references in the LRD file to 1965-1969 Southern Rhodesia files held at Curtis Green. This is a former FCO building in London where Top Secret migrated archive files were originally held and it is assumed that the Curtis Green files were part of this collection. No evidence for the destruction of the 1965-1969 files has yet been found nor has the FCO been able to confirm that these files are still in existence.
As part of a recent audit of holdings at Hanslope Park, we identified nine sets of loose papers relating to Southern Rhodesia (between 50 and 100 pages in total and now collated into six files). These papers may have been formerly held by either the Colonial Office or the Southern Rhodesia colonial administration, although one set of papers clearly originates from the Governor’s Office in Southern Rhodesia. The papers deal with appointments (such as the appointment in 1964 of Sir Thomas Hugh William Beadle as Deputy Governor) and with the Central African Power Corporation. These papers form part of the Tranche 7 transfer.
Permanent Under Secretary’s Department files
Following an initial release in 2005, a second tranche of files from the Permanent Under Secretary’s Department (PUSD) was released at The National Archives on 23 May 2013. These files cover the years 1939-1951. This transfer of PUSD papers to TNA was part of a twin thematic release alongside a tranche of the Cabinet Secretary’s miscellaneous papers released by the Cabinet Office. Both sets of papers are intelligence-related and the majority date from the Second World War.