Consultation outcome

DefCARS enabling changes: Consultation response

Updated 23 May 2023

1. Introduction

1.1. The SSRO conducted a public consultation on enabling changes to the Defence Contract Analysis and Reporting System (DefCARS) from 26 January 2023 to 21 March 2023. The consultation document was circulated to the SSRO’s Reporting and IT sub-group and published on the SSRO’s website. The Reporting and IT sub-group has representatives from the Ministry of Defence (MOD), industry and ADS, who provide feedback to the SSRO on its reporting guidance and DefCARS. A consultation workshop with attendees from the Reporting and IT sub-group was held on 20 February 2023.

1.2. The SSRO decided to consult on the enabling changes to DefCARS in advance of the MOD concluding the implementation of its Review of Legislation, which will result in changes to the information that contractors have to submit in statutory reports. The enabling changes will support future updates to DefCARS, which will be needed to take account of these amendments to the legislation, and will deliver improvements identified in the SSRO’s DefCARS Future Technology Strategy. We currently expect new and amended reporting requirements to take effect from April 2024, but this may be subject to change.

1.3 We would like to thank respondents for sharing their views either in writing or at the workshop. We have summarised the key comments raised by stakeholders and the SSRO’s responses to these comments in this document. Where changes have been made to the SSRO’s proposed approach, section 5 on next steps details those changes and our proposed implementation plan.

2. The consultation process

2.1. The SSRO described a number of enabling changes it intended to make to DefCARS in its consultation document. The enabling changes that the SSRO consulted on were:

• uploading data to DefCARS using data upload templates;
• how historic reports might be made available;
• how uploaded data might be validated;
• how compliance issues might be raised on uploaded data; and
• how the SSRO might capture feedback from DefCARS users.

2.2. The SSRO received six written responses to its consultation as shown in Table 1. All of the responses provided have been taken into account and have informed the SSRO plans.

Table 1: Number of responses

	Government	Industry	Trade Association
Number of responses	1	4	1

2.3. We have published five consultation responses as part of this response document at Appendix 1, where the respondents indicated we could do so. Respondents other than the MOD and the Defence Single Source Advisory Group (DSAG)^{[footnote 1]} are not identified in the main body of this document but are identified in Appendix 1.

3. Issues raised by respondents to specific consultation questions

3.1. The key comments raised in the responses are detailed below, grouped by each question, alongside the SSRO’s response.

Data upload

3.2. The SSRO explained how it intended to replace some parts of the graphical user interface (GUI) in reports, with contractors using data upload templates to provide elements of their report. This alternative method would, in the SSRO’s view, reduce the amount of manual data entry to the system. The SSRO proposed that such a shift in approach should be introduced incrementally over time rather than in one development phase.

Q1. Do you support the SSRO’s preferred approach for selected data to be uploaded to DefCARS using a data upload template and the remainder via the current GUI?

Q2. Do you agree that the SSRO should roll out the introduction of file upload incrementally (page by page over time) rather than by making significant changes in one development phase?

Q3. Do you agree that cost data using a contractor’s reporting structure in QCR submissions would be a good candidate to test upload capability?

3.3. DSAG and industry stakeholders were either supportive of the SSRO’s preferred approach for selected data to be uploaded to DefCARS using a data upload template or they stated that, while broadly supportive, they did not have enough information to fully assess the proposal at this time.

3.4. In terms of the timing of the proposed change, stakeholders said they did not think that data upload should be used as a method for submitting information in DefCARS ahead of the legislative changes that the MOD is introducing. Their view was that it was preferable for changes to be implemented once, rather than processes being re-worked potentially on multiple occasions. An industry respondent stated that the stability of the system was important to avoid additional contractor costs.

3.5. One industry respondent stated that they thought the proposed change to data upload would be beneficial. Another contractor said they supported the proposal but did not want to use a combination of data upload and the GUI when preparing and submitting reports. Other industry respondents stated that a single report should utilise data upload or the GUI but not both. One industry respondent said that they could understand why some pages would remain in the GUI. An approach of changing one report at a time was suggested by an industry respondent in their written response and by a number of attendees at the workshop. The MOD raised concerns about the SSRO running two approaches to reporting with some contractors utilising data upload and others not.

3.6. One industry respondent said that a complete example of an upload report in flat-file format would help them to better understand what the SSRO intended and allow them to assess any changes they would need to make to their internal reporting processes to facilitate uploads. The same respondent also said that the SSRO’s proposals appeared to be compulsory for all DefCARS users.

3.7. At the workshop on 20 February, stakeholders asked how data that was uploaded to DefCARS could be made available to users to allow them to share it within their own organisations. Some of the options for making this data available were discussed at the workshop. Stakeholders also made this point in their written responses.

3.8. Stakeholders supported the SSRO undertaking a trial in advance of rolling out data upload capability on a permanent basis. They indicated that an assessment of the trial based on user feedback should be undertaken before full implementation for all users. Several industry respondents confirmed that they would be prepared to be involved in a trial and stated that any testing should be done in parallel alongside their normal reporting and that examples of guidance and data upload template would be provided in advance of any trial. Industry stakeholders also unanimously supported conducting the trial on a single page of the Quarterly Contract Report (QCR) as the SSRO had proposed.

SSRO response

3.9. The SSRO has concluded that there is support from contractors to use data upload as a way of reporting information and some industry stakeholders have identified that this could make the submission process more efficient. The SSRO proposes to trial data upload capability on a single page in the QCR (as explained in the consultation document) with a number of contractors who have volunteered to be part of the test phase. These contractors will be asked to submit their upload as part of their QCR submission as a proof of concept in a beta environment that they will be given access to well in advance of the end of October 2023 deadline for QCR submissions. We envisage there will be a direct link between the beta environment and the normal database meaning that a contractor will be able to complete their QCR in the normal fashion without duplicate effort if significant difficulties are incurred with the upload. The contractors will submit their QCRs but one of the pages which they will complete will be via data upload rather than in the GUI. Contractors will continue to log in to the existing GUI interface to make report submissions. Testing data upload as part of a normal submission rather than separately is sensible to avoid running two approaches and creating additional work for the contractors. During the trial, we will provide guidance and an upload template, which will allow those contractors involved to assess any impact on their current reporting processes. The SSRO will consider the outcomes from the trial before further consultation with stakeholders on wider reporting changes.

3.10. Following assessment of the trial, the SSRO will decide whether there are any significant barriers to rolling out data upload more widely in DefCARS. We do not intend to introduce data upload capability within DefCARS for all users to adopt until after the MOD has made its planned changes to the legislation, as we recognise that introducing this in advance could be more disruptive for contractors.

3.11. The SSRO is of the view that if we proceed with data uploading of reports there will remain some limited information which is better included in each report in the GUI (for example, some of the general requirements associated with Regulation 22) where it would make sense for this master data to be entered via the GUI once and auto-populated in future reports, rather than having to complete reports uploaded to DefCARS which could be inadvertently inconsistent. While we recognise that stakeholders have indicated a preference for an either/or approach, we think there are benefits to the mixed model. Information can be carried forward between reports using the GUI which will ensure data consistency and ultimately will reduce the amount of data which contractors have to repeatedly upload and the time taken to do so. Any roll out of data upload capability would be done on a report by report basis, as suggested by stakeholders, rather than the page by page basis which had been originally proposed. Data upload would also be a mandated approach for all contractors to avoid the SSRO maintaining dual approaches for reporting, which would create additional costs.

3.12. We understand the feedback from contractors that they like to use the way data is currently presented in DefCARS to assist with their internal quality assurance of their information, and we intend to issue guidance to contractors to help them visualise and present the data they provide in the data upload format. There are different ways in which data can be visualised, and providing guidance to contractors on how to do this will allow for companies to flexibly present their data internally for quality assurance in the format that works for them. We do not consider that this will be a burdensome task for contractors to undertake.

Training and support

3.13. The SSRO asked stakeholders to identify any training or support needs they think they would have to be able to upload data to DefCARS.

Q4. What training and/or other support do you think the SSRO would need to provide to enable flat file data upload to proceed most effectively?

3.14. Industry respondents asked for templates and guidance to be provided in advance of trialling them. They also suggested that the SSRO should use multiple methods for training industry to ensure that any change in the current approach to reporting is fully understood and embedded.

SSRO response

3.15. During the trial, contractors will be provided with the templates and guidance required to make the upload sufficiently in advance of the end of October 2023 QCR reporting deadline. This will allow them to understand the approach being trialled and ask any clarification questions of the SSRO. During the trial, contractors will be able to assess the impact on their own internal processes and feedback any concerns to the SSRO. The SSRO will be available during the trial to respond to queries in a timely manner.

3.16. If data upload capability is rolled out following the changes to the reporting requirements, the SSRO recognises that it will need to prioritise the on-going training and support which industry will need over a period of time to understand all of these changes and to be able to continue to meet their existing reporting obligations. The SSRO keeps its support offer to contractors under review and makes improvements based on on-going feedback provided by contractors.

Improvements to preparing and submitting reports

3.17. The SSRO asked if stakeholders considered that the data upload proposals would reduce the preparation and submission time for reports. At the consultation workshop, the SSRO asked about the granularity of data in uploads and whether contractors thought cost data should be reported in £ or £ million.

Q5. Do you agree that the upload functionality proposed in this consultation document would reduce the preparation and submission time for reports?

3.18. One industry respondent confirmed that they thought data upload would be a more efficient way of preparing and submitting reports. Another agreed but felt it was more likely to benefit report submission rather than report preparation. Other stakeholders said they required more information to be able to answer this question.

3.19. DSAG said that they had concerns about reporting in £ as this in their view was expanding what the Regulations required and could lead to rounding errors and would require contractors to amend existing information systems they use to prepare reports. This issue was also discussed at the consultation workshop.

SSRO response

3.20. The SSRO recognises that it may be too early for stakeholders to be confident in their responses to this question and will provide further clarity to enable it to be answered more fully in the future. We fully anticipate that data upload will reduce the time it takes contractors to submit their reports. Preparation time may increase initially until the new method has been embedded.

3.21. The SSRO has considered the way contractors are currently reporting, with roughly half reporting in millions to six decimal places and the other half to less than six decimal places in the last set of QCR submissions made at the end of April 2023. Reporting in millions to six decimal places is the same as reporting in £s. Contractors may therefore be having to convert £ figures to £ millions in order to report in line with the SSRO’s current reporting guidance. Ahead of any consultations on reporting guidance changes prior to April 2024, we will ask contractors to use £ million to at least three decimal places in the trial. This will be consistent with the SSRO’s current reporting guidance.

Historic reports

3.22. The SSRO asked if contractors should only be able to access historic reports in Excel downloads rather than also in the GUI, even if in a previous format of the GUI. Maintaining multiple versions of the GUI is something the SSRO would like to move away from to further simplify DefCARS and reduce system maintenance costs.

Q6. Do you support the SSRO’s preferred approach for the availability of data in historic reports to be limited only to an Excel download?

3.23. Stakeholders indicated they were generally supportive of this proposal. The SSRO’s proposal was as set out in Exhibit 1.

Exhibit 1 – Historic reports proposal

Users will be able to view and download data in the following ways:

In the GUI contractors will:

• View data entered in the current format of the page.
• Access historic data in Excel downloads.

In Excel report downloads contractors will see:

• The whole report as an Excel download in the existing format for data entered prior to upload capability being available.
• Partial report downloads in the existing format for sections of the report that have been entered via the GUI.
• For data entered via the upload template, the totality of this data can be downloaded into the same spreadsheet template format as used in the upload.

3.24. An industry respondent said that historic reports should be available in full report format and should not be split between GUI and uploaded data, and stated that the SSRO needed to ensure that the same reports were always available to the MOD and industry.

3.25. One industry respondent suggested that the SSRO should ensure that download report formats were consistent with DefCARS as they had experienced some issues with this in the past. Another asked that the format of downloads should be fixed.

SSRO response

3.26. The SSRO plans to implement the proposal in Exhibit 1 as this will simplify the approach to providing contractors with the data they have submitted. Historic reports will be available in their entirety in an Excel download. If data upload capability is rolled out, it will mean that all information that makes up a statutory report will only be downloadable in a single document if the report was submitted prior to the roll out. If data is available in an upload template that the contractor can manipulate into a format that they have defined as most useful to them, then the SSRO considers that it would be duplicate effort to develop standardised reports for all contractors. The SSRO provides management information for the MOD based on reported data when it is requested to do so. MOD teams may choose to share its management information products with individual contractors as part of contract management discussions.

3.27. Where a contractor submits data in the GUI and via upload, any Excel download report and the upload templates which relate to the report will be accessible on the same page in DefCARS, currently the ‘submission’ page.

3.28. The SSRO will aim to ensure that download report formats always reflect the data reported in DefCARS, but columns will not be fixed to allow contractors to be able to more easily manipulate formats to meet their own needs. Where there are issues identified with the accuracy of reports based on data submitted in DefCARS, we will aim to resolve them as soon as possible.

Corrections

3.29. DefCARS currently allows any report that has ever been submitted in the system to be corrected. The SSRO consulted on changing this approach to manage the amount of data which needs to be held in the DefCARS database. Any correction report that is submitted, even a minor correction to a report, currently results in a duplicate set of the report’s data (including any associated attachments) being captured in the DefCARS database. The SSRO asked specifically in the consultation for MOD views on this proposal.

Q7. Do you agree that correction functionality should only be available for a limited period of time?

3.30. A stakeholder confirmed that they agreed that the ability to correct reports should be removed but indicated that it might be better to implement this rule for some report types rather than for all. One industry respondent said that if the SSRO introduced a control around when corrections could be made, this would need to be clearly communicated to stakeholders. Another stakeholder said that any rule needed to be linked to any time limits on raising compliance issues on reports, as if the time for raising issues on a report had passed then the contractor may not need to correct data. A suggestion was made that corrections could be limited to the last submitted report of a particular type.

3.31. DSAG confirmed that they did not think the SSRO should implement this proposal as issues raised between the MOD and the contractor could take some time to resolve and the ability to correct should always be available. The MOD said that it thought that contractors should aim to be reporting data correctly first time and that corrections should only be in response to compliance issues raised.

SSRO response

3.32. The SSRO intends that historic corrections will not be possible in the future, after a set period of time from submission. We do not consider that there will be a high volume of requests to correct historic data, especially if the time by which compliance action can be taken has passed. The SSRO also indicated in the consultation that it did not want to have to maintain out-dated versions of the GUI as this was not efficient. We will consult on the approach to historic corrections further as part of changes we will make to DefCARS that will be required when the legislation is amended. In circumstances where historic data cannot be corrected in DefCARS, the SSRO will be able to correct the data in the database if it is notified, but would only seek to do this in exceptional circumstances where there was agreement between the MOD and the contractor that the data should be corrected.

Compliance and validation of upload data

3.33. The SSRO explained that it did not want to make significant changes to the way data is validated or compliance issues are raised in the system. Where data continues to be entered in the GUI, existing validations will apply. Data being uploaded into DefCARS would be rejected if it did not pass validation. The SSRO also indicated that it wanted to move to applying Government Digital Service Standards in relation to data validation and other aspects of how DefCARS operates. Compliance issues will continue to be raised in the GUI, even if the data has been uploaded.

Q8. Do you have any views about the approach to data validation?

Q9. Do you have any comments in the approach to raising compliance issues against data in upload templates?

3.34. Industry stakeholders were supportive of the proposed approach to data validation and urged that it should not interrupt or delay completion of reports. They felt the current approach worked well and therefore stated that their preference was for data validation to occur in DefCARS and not in upload templates. Templates with macros could be problematic in some contractors’ IT environments. Contractors asked for DefCARS to flag the validations that remained on data after it had been uploaded, and for these flags to be visible for those staff within contractors who oversee their reporting arrangements. One contractor suggested that information about all amendments made to an upload template should be available to ensure a complete audit trail of changes.

3.35. There was also support for compliance issues continuing to be raised in the current way in the GUI and not in the upload template. One industry respondent queried how raising issues in an upload template might be possible. Another indicated that they thought the current approach of viewing issues in pop-ups was time consuming.

SSRO response

3.36. The SSRO intends to implement an approach where data in an upload template would not be accepted into the database unless all relevant validations were met prior to submission. These red warnings would be highlighted upon upload and would need to be corrected by the submitter to ensure all expected data is included and is in the correct format to be accepted. The approach to validation will be tested during the trial and we will seek feedback on any issues which contractors experienced.

3.37. The SSRO will continue to apply the current approach to raising and responding to compliance issues in relation to uploaded data. Issues would appear in the GUI rather than the upload template. The SSRO, as explained in its DefCARS Future Technology Strategy, intends to phase out use of pop-ups over time.

User feedback

3.38. The SSRO asked stakeholders whether DefCARS could be used to collect feedback from users on DefCARS performance and other aspects of reporting.

Q10. Do you have views on the mechanism by which and the frequency at which to seek feedback from users?

3.39. Stakeholders were supportive of the proposal to collect user feedback in DefCARS. They suggested that feedback on certain matters like the cost of reporting should not be captured in the system as DefCARS users might not be best placed to provide this information on behalf of the contracting company. Respondents emphasised that any capture of feedback should not be requested while a user was using the system, but should be requested when they have finished their use of it and prior to logging out. One respondent asked if the feedback would be compulsory.

SSRO response

3.40. Based on the support for this proposal, the SSRO will seek to develop a mechanism within DefCARS for collecting feedback from users. Details of what will be requested will be shared with the Reporting and IT sub-group prior to implementation. Responses will not be made compulsory but the system may have prompts to ask users for feedback.

4. Other general comments

4.1. Respondents did not provide comments on any matters not covered by the ten consultation questions covered in section 3.

5. Next steps

5.1. The responses the SSRO received have assisted in the prioritisation of DefCARS developments to deliver these enabling changes. During 2023/24, we will trial data upload capability with associated validation and compliance changes ahead of the planned implementation of new and amended legislation at the beginning of 2024/25. We agree that a new approach to putting data into the system should not be implemented before these changes in 2024/25. The approach to user feedback and historic reports will follow this trial and be introduced towards the end of 2023/24 or at a later stage. Exhibit 2 provides a summary of the SSRO’s plans in relation to the enabling changes on which it consulted.

Exhibit 2 – SSRO DefCARS development plans

5.2. The trial of data upload capability will use the actual and forecast costs which are submitted, broken down by the contractor’s own reporting structure in the Quarterly Contract Reports. We will task our system developer with developing the capability during Q1 and Q2 of 2023/24. We aim to issue initial guidance and templates on the approach to contractors who have agreed to take part in the trial in July 2023 and will explain the approach to the Reporting and IT sub-group at its meeting on 19 July. This will give time to clarify any issues with the SSRO and make guidance amendments in advance of the trail in October 2023 for the QCR submissions that are due by 31 October 2023. In November 2023 we will evaluate the approach and hear feedback from users to allow us to consult further as part of planned consultations on reporting guidance changes and make any necessary revisions to that approach from April 2024. During 2024/25 we will need to expand use of data upload within DefCARS to ensure it supports the amended reporting requirements with the potential for workarounds to be needed between April 2024 and when permanent data uploading can be delivered.

5.3. After the conclusion of the trial on data upload, our next priority will be to develop an approach to capturing user feedback in DefCARS. We recognise from the responses we have received that some of the proposed areas are out of scope.

5.4. Following development of a user feedback approach, we will implement an approach to managing historic reports and corrections in line with the proposal in Exhibit 2 and our proposals relating to corrections. Until data upload is rolled out, contractors will continue to be able to see data in the latest format of the GUI and in other cases via Excel download. Contractors will also be able to correct data in the usual way until the SSRO confirms that it will be implementing a change in data correction approach.

5.5. The SSRO will continue to keep the Reporting and IT sub-group updated with its progress in implementing these enabling changes. We retain a focus on ensuring DefCARS is as efficient and effective a reporting system as possible. The SSRO continues to provide support to all DefCARS users through its helpdesk function to ensure that any identified issues are resolved as soon as possible.

Appendix 1: Consolidated stakeholder responses

Respondent 1: DSAG

Q1. Do you support the SSRO’s preferred approach for selected data to be uploaded to DefCARS using a data upload template and the remainder via the current GUI?

Based on the consultation paper we do not feel we have enough information to fully assess the proposal at this time, coupled with the regulatory change in reporting requirements, and the low level of MOD use of reports, this may not be the right time to undertake this initiative. We believe that the detailed technical proposals should be explored and fully understood to understand the value for money in making this investment in change (for the SSRO and industry). Finally, we are concerned that a composite approach of some reports being uploaded and others by the GUI will be inefficient for contractors to manage in terms of IT modification and subsequent change.

Q2. Do you agree that the SSRO should roll out the introduction of file upload incrementally (page by page over time) rather than by making significant changes in one development phase?

We do not support this approach. While a phased approach could be taken in testing the uploads, a phased roll out to the live environment will be costly and burdensome for contractors. Many companies have made significant investments in IT systems, in moving to an upload approach, companies would largely like to make this change to their systems once and not be changing the systems progressively over a period. Again, a detailed review with a full example of the flat-file system is required to assess if one change should be made (once the review of legislation is complete) or in stages.

Q3. Do you agree that cost data using a contractor’s reporting structure in QCR submissions would be a good candidate to test upload capability?

We support this approach. Using the QCR will generate faster feedback and will have the most benefit for suppliers submitting this data on a regular basis. As for question 2, this should be in the test environment with volunteer companies, not a live progressive roll out, alternatively the SSRO could consider a parallel process where companies use the GUI in the regulated timescales, and then submit later using the flat-file.

Q4 - What training and/or other support do you think the SSRO would need to provide to enable flat file data upload to proceed most effectively?

Training should be delivered in a number of ways: 1. R&IT meetings. 2. Via the SSRO website, with document-based guidance and possibly video clips of sample uploads being performed. 3. Dedicated training sessions for companies. 4. Guidance notes This should cover the flat-file upload and changes to queries and compliance resultant from not using the GUI.

Q5 - Do you agree that the upload functionality proposed in this consultation document would reduce the preparation and submission time for reports?

This depends on the detail of the upload system, its coverage, the compliance system and ability to download reports. We are however concerned about the approach to data granularity and roundings. The proposal is to use data uploads to the level of pence, whilst some GUI inputs are made £’m to 3d.p. With the lack of use of reports, and reports should be used for strategic MOD management, this level of data is unlikely to represent value for money. This is expanding the data requirement, may create rounding issues. It may also require industry again to change their IT systems that have been built to generate data at the current levels, leading to more cost and delay. We believe that the proposal to require uploads to the level of pence requires further consideration and consultation with those compiling reports within industry. Not having a user-friendly view of an upload file within Excel may be a retrograde step.

Q6 - Do you support the SSRO’s preferred approach for the availability of data in historic reports to be limited only to an Excel download?

If the data downloaded is in a format that is intuitive for the user (not a flatfile), we would support this approach. Industry has a statutory duty to answer questions from the MOD, unless it can generate reports in a meaningful format that corresponds with what the MOD can generate, it may impede their compliance with these statutory requirements. Historic reports should be available in a full report format. Industry downloaded reports must be complete, they cannot be split between GUI and uploaded data, to do so would give industry no useable data over what has been reported. We understand that this requirement causes the SSRO issues with the way that DefCARS data is structured, but believe this issue that the SSRO must resolve to ensure (i) a user-friendly interface for industry to review and (ii) both industry and MOD have visibility in the same format.

Q7 – Do you agree that correction functionality should only be available for a limited period of time?

We do not agree with this approach. The ability to correct reports should not be time bound. Regulation 49 specifies the time limits for compliance and penalty notices within several areas including reporting. Issues raised between the contractor and MoD can take a long time to resolve. Time-bounding corrections to reports, will not permit the best/most accurate data to be recorded. This is often not due to the contractor performance, but the lack of MOD review (1.5 of the SSRO Annual Compliance Report: 26% of contract reports and 5% of supplier reports are reviewed).

Q8 – Do you have any views about the approach to data validation?

Validation in DefCARS is the appropriate method, macro enabled upload templates will not be permitted in many companies IT systems. We support the SSRO approach. The administrator/reviewers within industry must be able to submit and review a single report that includes all the ‘amber’ warnings created by DefCARS. If the reviewer cannot operate with a single report, quality and timeliness may decline. We would like to see enhancements when resubmitting an upload flat file (e.g. to compliance questions). We would like the upload system to inform the user when making a re-upload of how many changes are being made, otherwise, again, the quality of submissions may decline.

Q9 – Do you have any comments in the approach to raising compliance issues against data in upload templates?

Compliance issues should be raised in a user-friendly format (e.g. GUI), not a flat file.

Q10 – Do you have views on the mechanism by which and the frequency at which to seek feedback from users?

The SSRO’s suggested approach appears proportionate, and will gather timely data, when the user has just experienced the system. This should be when exiting the system, not during a session (as the .gov websites generally do).

Respondent 2: Rolls-Royce

Q1. Do you support the SSRO’s preferred approach for selected data to be uploaded to DefCARS using a data upload template and the remainder via the current GUI?

Rolls Royce believes the proposed change would be beneficial. By using Excel, the team believes it will be easier to navigate and similarly be more efficient to input. The improved functionality of DefCARS as a result of the standardised Excel templates is perceived as a bonus.

Q2. Do you agree that the SSRO should roll out the introduction of file upload incrementally (page by page over time) rather than by making significant changes in one development phase?

Rolls Royce would prefer to have a full roll out as opposed to an incremental roll out as this would facilitate streamlined approaches when populating each section of the required reports as opposed to using different approaches for different sections.

Q3. Do you agree that cost data using a contractor’s reporting structure in QCR submissions would be a good candidate to test upload capability?

Rolls Royce agrees that cost data would be the most useful section of our reports to trial. However, before the trial is released the team would appreciate a few months’ notice to view and test the proposed template so they can become accustomed to it.

Q4 - What training and/or other support do you think the SSRO would need to provide to enable flat file data upload to proceed most effectively?

The most effective way of rolling out the flat file is to provide us with templates as soon as possible such that we can test the proposed layout and usability of navigation. Rolls Royce would also like some brief bullet pointed guidance against each section to ensure that filling out of the templates is done right first time.

Q5 - Do you agree that the upload functionality proposed in this consultation document would reduce the preparation and submission time for reports?

Rolls Royce considers that this would be a more efficient way of preparing and submitting reports. The standard templates also improving the operational performance of DefCARS will mean we can be more efficient when navigating the current system.

Q6 - Do you support the SSRO’s preferred approach for the availability of data in historic reports to be limited only to an Excel download?

Rolls Royce understands this approach. However, it does create a concern – some time ago we had an issue where the Excel download from DefCARS was not displaying exactly the same data as the DefCARS system. If only Excel downloads are to be available in future, we would need certainty that this issue has been resolved.

Q7 – Do you agree that correction functionality should only be available for a limited period of time?

The cut off time needs to be clearly defined so that it can be communicated out well in advance.

Q8 – Do you have any views about the approach to data validation?

Rolls Royce understands the approach being proposed. However, the key element here will be to ensure that validation of file uploads does not interrupt or build significant delay into population and completion of the reports.

Q9 – Do you have any comments in the approach to raising compliance issues against data in upload templates?

No comment on this question.

Q10 – Do you have views on the mechanism by which and the frequency at which to seek feedback from users?

Again, the key element here will be to ensure that seeking feedback does not build delay into the population and completion of reports.

Respondent 3: Babcock International Group

Q1. Do you support the SSRO’s preferred approach for selected data to be uploaded to DefCARS using a data upload template and the remainder via the current GUI?

We understand the need to try and make data input more friendly. It may be more appropriate to take a batch approach by tackling one report at a time for all pages that are suitable for a flat file upload. There are some pages that require minimal input so should be unchanged. It may be more appropriate to wait until the impact of the legislation change works its way through.

Q2. Do you agree that the SSRO should roll out the introduction of file upload incrementally (page by page over time) rather than by making significant changes in one development phase?

As mentioned above it may be more appropriate to tackle on a report-by-report basis. This will demonstrate real progress being made. Some pages will be similar in nature across relevant reports.

Q3. Do you agree that cost data using a contractor’s reporting structure in QCR submissions would be a good candidate to test upload capability?

We agree with this approach, but it may be more beneficial to have a test environment to iron out any glitches.

Q4 - What training and/or other support do you think the SSRO would need to provide to enable flat file data upload to proceed most effectively?

We concur with DSAG on this question.

Q5 - Do you agree that the upload functionality proposed in this consultation document would reduce the preparation and submission time for reports?

It is felt that there would be little change to the preparation process, but it may speed up the upload process.

Q6 - Do you support the SSRO’s preferred approach for the availability of data in historic reports to be limited only to an Excel download?

We concur with DSAG on this question.

Q7 – Do you agree that correction functionality should only be available for a limited period of time?

Whilst we support DSAG on this point, some reports should have correction functionality removed. This would be more applicable to QCRs and to a lesser extent ICRs as these are updated regularly and makes earlier submissions redundant.

Q8 – Do you have any views about the approach to data validation?

We concur with DSAG on this question.

Q9 – Do you have any comments in the approach to raising compliance issues against data in upload templates?

Any issues raised should be in the existing format.

Q10 – Do you have views on the mechanism by which and the frequency at which to seek feedback from users?

Any feedback should be gathered at the end of a user’s session prior to system exit. It may be more appropriate to have a regular slot at the Reporting and IT Subgroup meetings for this purpose. This would generate discussion across all parties. The R&IT Subgroup meeting time could be extended to accommodate (say from 2 hours to 2 ½ hours) but understand possible time constraints and maybe focus on common topics.

Respondent 4: Leonardo

We welcome the chance to provide input to your review. We provide our response as complementary to DSAG’s paper.

As advised at the workshop on 20th February, we have found it difficult to fully visualise the proposals and how they would operate in practice. Therefore, we are uncertain as to whether the costs of introducing the proposals is value for money.

We have previously explained our UK company comprises a number of different businesses on different physical sites with differing reporting solutions to the regulatory requirements. The different solutions are borne of the history of contracting arrangements. The impact of any proposed changes to reporting needs to be understood in terms of the cost implementation, along with implications of further changes resulting from the Command Paper. It will be important to understand if the cost to implement the proposed changes to reporting will be one off or if there would be a similar cost each time there is a change to the reporting requirements and therefore DefCARS templates.

In the absence of any further details of the proposals, we have endeavoured to answer the questions below.

Q1. Do you support the SSRO’s preferred approach for selected data to be uploaded to DefCARS using a data upload template and the remainder via the current GUI?

The principle of utilising an upload flat-file, we support.

However, understanding of that proposed suggests needing to use both GUI and flat-file upload approaches to populate a single contract report, which we do not support. A single contract report should utilise either GUI or flat-file.

Further clarification of that proposed is also required, for example:

• As MOD’s use of DefCARS reports is very low, would a VFM assessment of current reporting prior to investing in further changes to reporting processes be worthwhile? Would this proposal be more efficient and effective post a full review of the purpose and current use of each reporting requirement?
• A complete example of a report’s flat-file, to understand better that intended and assess implications to current processes is required. Additionally, once templates have been agreed, they would need to be available in advance of launch, to allow implementation by contractors (which to date is unknown in terms of time and cost).
• The proposal to use a flat-file appears compulsory for all users. This needs to be considered in conjunction with implementation costs/time for contractors.
• Para 5.17 “ …a flat-file approach allows contractors to report contract-specific information in more detail, if necessary”. We are concerned that information required will increase over time, increasing burden on contractors, when current regulatory reporting requirements are utilised minimally.
• Whilst we concur with continuous improvements which can be beneficial, from a user perspective there needs to be a balance. The system needs to be stable to avoid additional costs (for eg. IT investment, changes to internal process, training etc) associated with updates to flat-file formats or any changes to DefCARS.
• Para 5.16 states current DefCARS capability of uploading files (as attachments) for non-standard reports will continue. We assume this functionality will remain where one needs to provide additional support / information.
• What is the proposed format of the download functionality or view within DefCARS of that submitted via flat-file? There needs to be a user-friendly format for downloads, similar to that currently available. (Flat-file format is an approach to transferring, storing and manipulating data, not review).

Q2. Do you agree that the SSRO should roll out the introduction of file upload incrementally (page by page over time) rather than by making significant changes in one development phase?

Concerns remain over a page by page implementation as this will require continual changes potentially to IT systems, internal processes etc. Post satisfactory testing, it would be preferable to move an entire contract report to be submitted using a flat-file approach. It is important to have sight of the user interface to allow view/download of the flat-file element as part of any testing phase. As mentioned above, this cannot be the flat-file format itself and would be a retrograde step for contractors.

Q3. Do you agree that cost data using a contractor’s reporting structure in QCR submissions would be a good candidate to test upload capability?

Yes we agree. As mentioned above, examples of the formats of the full flat-files for each report along with examples of the interface for users within DefCARS of the flat-file submitted pages and download formats needs to be made available prior to any testing, to assess impact of the change.

Q4 - What training and/or other support do you think the SSRO would need to provide to enable flat file data upload to proceed most effectively?

We suggest consideration of the following in terms of support:

• Demonstration of an upload of a flat-file, walk through of a query and response/correction to a report;
• Guidance on the above;
• Specific training courses;
• Potential IT support as necessary for implementation of a flat-file;
• Continued on-boarding to assist specific contractors as required;
• The available output (MI reports) from DefCARS to aid review by a contractor (flat-files are not designed for review).

Q5 - Do you agree that the upload functionality proposed in this consultation document would reduce the preparation and submission time for reports?

Without seeing the exact formats of the excel flat-files (both in terms of the physical format and requirements of each cell), it is difficult to make the assessment. Based on the little information available, initial views suggest no change to the preparation/compilation time of any report. However, potentially there may be a small reduction in submission time, as the submission of the flat-file would negate the need to use the copy and paste functionality.

Q6 - Do you support the SSRO’s preferred approach for the availability of data in historic reports to be limited only to an Excel download?

Yes if the question is referring to any report submitted using the GUI functionality. Concern remains with the flat-file format being the format used with queries and interface for users to review data submitted via flat-file. We recommend further consideration of a format to aid review by users as opposed to a download of a flat-file, this would be a retrograde step for contractors and potentially impact time in answering any queries raised. Construct of MI reports may address and resolve the above issue. The MI reports would need to be available simultaneously with the move to flat-files and available to all stakeholders. Better understanding of how corrections to reports would work is required. Currently some information within DefCARS flows from one tab/page to another, we wonder how this will be affected by the proposed changes?

SSRO advise their proposal would mean two downloads for each contract report submitted, one with the GUI entered information and another for the flat-file submitted data. This would not assist speed and simplicity of the regime and is seen as a retrograde step, which would not be acceptable. As mentioned above (question 1), a single contract report needs to be submitted using one approach, not two.

Q7 – Do you agree that correction functionality should only be available for a limited period of time?

No, we do not agree with this approach. The correction functionality should always be available for the last submitted report, it is essential that the MoD have the correct information in any report. Additionally, there should not be a requirement to correct historical reports as well as current reports (para 5.39). This would not be VFM.

Q8 – Do you have any views about the approach to data validation?

Para 5.46 states at the point of upload of the flat-file, validation occurs, with the file being accepted or rejected. The SSRO confirmed that there would be a two step approach: (1) upload file and validate; (2) once all validation flags cleared, the flat-file could be submitted. It was preferable that any validation was against the interface as opposed to the flat-file itself, as this would aid understanding of the validation flags occurring and make it easier to see changes required.

Q9 – Do you have any comments in the approach to raising compliance issues against data in upload templates?

The existing approach to viewing and responding to compliance issues within DefCARS will be retained”. o How will this work in practice with that the SSRO are proposing? The paper suggests one will be required to download the flat-file in order to view, but the query itself would reside in DefCARS? Further understanding and demonstration is required.

• As mentioned previously, a flat-file is to facilitate ease of transfer, storage and manipulation of data, not review. Flat-files establish a database, which should enable “live/interactive” population of a report format, for review within DefCARS. The use of an interface we do not see as inhibiting changes to the reporting requirements within DefCARS.
• Any improvements to the way queries/issues are processed in DefCARS would be welcome.

Q10 – Do you have views on the mechanism by which and the frequency at which to seek feedback from users?

Is the feedback intended to be mandatory or compulsory? Feedback requests whilst using the system was not supported, this would be an unnecessary distraction. Further understanding of the format of feedback required was seen as being helpful in order to comment on the timing. Feedback on DefCARS performance or possibly DefCARS functionality guidance could be provided, but costs associated with report compilation/submission was unlikely, due to lack of knowledge by the submitter. The general view was an open question at the end of each session or each report submission to understand if there were any difficulties with the system would be acceptable. Users who had experienced issues could provide further details or answer further questions, depending on the format of feedback design.

Respondent 5 – The MOD

The MOD response did not reference the consultation questions but did make the following comments.

Data Upload

• Paragraph 5.22 - By introducing the capability gradually is the SSRO concerned about dual running, especially for new users.
• Paragraph 5.27 - As this is new maybe a reminder to contractors to seek advice from the SSRO, rather than going to the MOD commercial team, as they probably will not be able to advise.

Historic report access and report corrections

• Paragraph 5.32 – Is there a risk that the auto-populated correction report may pull down any existing errors in the original submission.
• Paragraph 5.37 - We assume that this means SSRO and SSAT will have to make sure errors are not allowed to roll on.
• Paragraph 5.39 - Correcting historic data, and not allowing a contractor to roll into historic data is important. We are not sure how big an issue this is.

The kick off meetings with our suppliers have set out expectations of Supplier Reports – the basics of what good looks like.

We have also set out that we do not expect iterative reporting as iterative reporting:

1. Reduces any confidence in supplier’ internal quality assurance;
2. Dilutes the significance of the statutory timeframes for completion;
3. Creates delay within the rates assessments and agreements.

We have therefore stated to suppliers that we would only expect reports to change:

1. In response to issues identified and flagged in the system (to achieve compliance); and/or
2. To reflect an agreed set of rates following assessment and negotiation.

Data validation and compliance

• Paragraphs 5.43 and 5.49 – Reports should be right first time. If you won’t stop a contractor from submitting a report with errors, can DefCARS not automatically send a report to the contractor, as soon as they submit their report, that highlights all the Amber Validation warnings and direct the contractor to correct, copying in the DT/SSAT/SSRO.

User feedback

• Paragraph 5.52 - Perhaps a sort of deep dive, where the SSRO notices higher than normal or basic errors, reaching out to the contractor. An intervention, sitting down face to face or MS Teams to MS Teams. Does the SSRO have access to a list of user errors.
• Paragraph 5.54 – We support the feedback mechanism being built into DefCARS.

---

1. DSAG is a group supported by the ADS Group. ↩