Guidance

How an open standards proposal is assessed

This guide explains the questions and process used to assess an open standard proposal before it is sent to the Open Standards Board.

All open standards that are considered for adoption across government must be assessed by the GDS open standards team and the open standards panel before they can be formally proposed. Read more about the open standards selection process.

The Open Standards Board agreed the 47 criteria used in the assessment. These questions are based on the EU CAMSS (common assessment method for standards and specifications).

Challenge owner responsibilities

You must:

  • use the list of core assessment questions below to assess the proposed standard(s)
  • answer the questions with yes or no, and explain your answer
  • send your completed questions to the Open Standards Panel for evaluation

You can also suggest additional questions that provide more context to the proposed open standard. Any answers attached to these questions can be used as a draft by other challenge owners but must be relevant to each proposal that uses them.

Open Standards team responsibilities

The Open Standards team will:

  • provide support to you and access to the online forms you will need to fill in
  • publish the agreed assessment criteria for the proposal
  • collect and publish questions suggested by you as a resource for other Challenge Owners

Assessment questions for the challenge owner

Formal specification

  1. Does it address and aid interoperability between public administrations?
  2. Does it address and aid the development of digital services in government?
  3. Are the functional and non-functional requirements for the use and implementation of the specification clearly defined?
  4. Is it possible to implement the specification across different domains?
  5. Is it largely independent from products of single providers, either open source or proprietary?
  6. Is it largely independent from specific platforms?
  7. Has the standard been written so that it can be delivered or used with more than one technology (for example XML and JSON)?
  8. Has the specification been sufficiently developed and existed long enough to overcome most of its initial problems?
  9. Are there existing or planned mechanisms to assess its conformity and implementation - for example conformity tests, certifications and plugfests?
  10. Does it have sufficient detail, consistency and completeness for the use and development of products?

Implementation of the formal specification

  1. Does it provide current implementation guidelines and documentation for the implementation of products?
  2. Does it provide a reference (or open source) implementation?
  3. Does it address backwards compatibility with previous versions?
  4. Are the underlying technologies for implementing it proven, stable and clearly defined?

Openness

  1. Is information on the terms and policies for the establishment and operation of the standardisation organisation publicly available?
  2. Is participation in the creation process of the formal specification open to all relevant stakeholders (such as organisations, companies or individuals)?
  3. Is information on the standardisation process publicly available?
  4. Is information on the decision-making process for approving formal specifications is publicly available?
  5. Are the formal specifications approved in a decision-making process which aims at reaching consensus?
  6. Are the formal specifications reviewed using a formal review process with all relevant external stakeholders (such as public consultation)?
  7. Can all relevant stakeholders formally appeal or raise objections to the development and approval of formal specifications?
  8. Is relevant documentation of the development and approval process of formal specifications publicly available (such as preliminary results and committee meeting notes)?

Access to the formal specification

  1. Is the documentation publicly available for implementation and use at zero or low cost?
  2. Is the documentation of the intellectual property rights publicly available (is there a clear and complete set of licence terms)?
  3. Is it licensed on a royalty-free basis?

Versatility/flexibility of the proposed standard

  1. Has the formal specification been used for different implementations by different vendors/suppliers?
  2. Has the formal specification been used in different industries, business sectors or functions?
  3. Has interoperability been demonstrated across different implementations by different vendors/suppliers?

End user effect of the formal specification

  1. Do the products that implement it have a significant market share of adoption?
  2. Do the products that implement it target a broad spectrum of end-uses?
  3. Does it have strong support from different interest groups?
  4. Is there evidence that the adoption of it supports improving efficiency and effectiveness of organisational process?
  5. Is there evidence that the adoption of it makes it easier to migrate between different solutions from different providers?
  6. Is there evidence that the adoption of it positively impacts the environment?
  7. Is there evidence that the adoption of it positively impacts financial costs?
  8. Is there evidence that the adoption of it positively impacts security?
  9. Is there evidence that the adoption of it can be implemented alongside enterprise security technologies?
  10. Is there evidence that the adoption of it positively impacts privacy?
  11. Is it largely compatible with related (not alternative) formal specifications in the same area of application?
  12. Is there evidence that the adoption of it positively impacts accessibility and inclusion?

Maintenance of the formal specification

  1. Does it have a defined maintenance organisation?
  2. Does the maintenance organisation provide sufficient finance and resource to control short-to-medium-term threats?
  3. Does the maintenance organisation have a public statement on intention to transfer responsibility for maintenance of it, if the organisation were no longer able to continue?
  4. Does it have a defined maintenance and support process?
  5. Does it have a defined policy for version management?
  1. Is this an existing European standard or an identified technical specification in Europe? (Note: CEN, CENELEC or ETSI are the European standards bodies. Technical specifications provided by organisations other than CEN, CENELEC or ETSI can be under consideration to become a European standard or an identified technical specification in Europe.)
  2. Does this specification or standard cover an area different from those already identified or currently under consideration as an identified European standard or specification?

Open Standards Panel evaluation

The process of evaluation starts once you have completed the core assessment questions on the online form.

A Standards Panel will evaluate the proposal based on your answers to the core assessment questions and the evaluation questions listed below.

If the Standards Panel approves the proposal you will be asked to prepare an Open Standards Profile using the online template.

The Standards Panel will submit both the proposal and profile to the Open Standards Board with their recommendations.

You will be informed if the Standards Panel rejects the proposal and is not submitting it to the Open Standards Board.

Evaluation questions

The Open Standards Panel will evaluate the proposal with the following questions.

Context

  1. What needs does the standards profile meet?
  2. Which organisations or functional areas should refer to this standards profile?

Recommendation

  1. What is the Standards Panel’s recommendation to the Open Standards Board? Describe which stakeholder groups and outcomes the standard profile meets and how.
  2. Why does the panel consider this is the most effective course of action? What is the scope?

Assessment overview

  1. A brief summary of the assessment: what are the main pros and cons relating to applicability, maturity, openness, intellectual property rights, market support, potential and coherence discovered during the assessment?
  2. Alternatives considered: what other proposals and standards were considered and why are these not being recommended?

Implementation

  1. What effect would implementation of this standards profile have on service delivery?
  2. How does this approach deal with any backwards compatibility issues?
  3. What might be on the horizon (for example are there any threats relating to this approach or the associated standards)?
  4. What are the benefits or opportunities relating to this approach or the associated standards (economic, social and environmental)?
  5. When will implementation begin and when is it likely that it will be completed?
  6. Are there any non-technical barriers that remain which will need to be addressed before successful implementation could be achieved (such as legal, organisational)?
  7. What trials have been undertaken? Describe any reference implementations that have been built or tested, and pilot projects or any plugfests etc.

Process

  1. List the people involved in the development of the proposal and the assessment of the standards.
  2. List the members of the standards panel involved in the evaluation of this standards profile.
  3. What date should this standards profile be reviewed? Take into account any anticipated changes in technology or standards.
  4. Was it necessary to notify the European Commission under Directive 2015/1535/EU (which replaced Directive 98/34/EC on 7 October 2015)? If so, what stage has been reached? Is the outcome known?
Published 1 April 2013