How to evaluate Digital Outcomes and Specialists suppliers
How to evaluate digital outcomes, digital specialists and user research participants suppliers.
This guide is for evaluating digital outcomes, digital specialists and user research participants suppliers. If you’re evaluating user research lab suppliers, you should read the how to hire user research studios guide.
What happens at the evaluation stage
You must evaluate shortlisted suppliers to find the one that best meets your requirements.
You need to use the methods and criteria you published with your requirements to:
- assess suppliers, for example in an interview or with a written proposal
- score suppliers
You can then award a contract to the supplier with the highest score.
Make the evaluation process fair
The evaluation process needs to be fair, open and transparent. You should:
- ask suppliers to provide information in the same format, whichever method you choose, for example give suppliers a template to fill in so that it’s easier for you to assess them
- not look at any suppliers’ written responses until after the deadline
- consider the logistics for each method, for example where the interview will happen or what equipment might be needed
- keep a record of how you’ve scored each supplier
- score all suppliers against the same criteria using a scoring template
- use the same evaluation team for each assessment method
- give feedback to each supplier after you’ve made a decision
Read about how to buy services fairly.
What you can evaluate on
You must evaluate outcomes and specialists suppliers on:
- technical competence, for example how well the supplier’s skills or proposal meet your needs
- cultural fit, for example how the supplier will work in your organisation
- price of the proposal
You must evaluate user research participants suppliers on:
- technical competence, for example how well the supplier’s skills or proposal meet your needs
- availability
- price of the proposal
Read about:
The evaluation process
- Plan the evaluation stage, for example methods you’ll use to assess suppliers and who needs to be involved.
- Invite shortlisted suppliers to be assessed, for example in an interview or by providing a written proposal.
- Assess suppliers using the assessment methods you published with your requirements, taking notes as you go.
- Score suppliers using a scoring template and the evidence from your notes.
- Meet with the other people who are evaluating suppliers to agree an overall score for each supplier.
- Award a contract to the supplier with the highest score and give feedback to unsuccessful suppliers.
1. Plan the evaluation stage
Who should be on your evaluation team
You need an evaluation team with the right skills and expertise to find the supplier who best meets your needs.
Your evaluation team should be made up of people who understand what you’re buying. This may include people with digital expertise in your organisation or another public sector organisation. You could also ask digital experts outside the public sector to help you evaluate.
Consider who needs to be involved in each assessment, for example you may use different people to assess supplier work histories than to assess a proposal.
You must ensure that no one on your evaluation team has a conflict of interest, for example doesn’t have shares in a supplier who has been shortlisted.
Your evaluation team should have a minimum of:
- 3 people for digital outcomes
- 2 people for digital specialists and user research participants
What your evaluation team needs to do
The people on your evaluation team will need to:
- plan the order of the assessment methods and if you’ll do any at the same time, for example ask suppliers to provide both a written case study and a proposal
- prepare for assessments
- assess supplier responses, for example evaluate case studies or go to interviews
- write up their individual evaluations and give a score to each supplier
- agree an overall score for each supplier
Use the same team for each assessment method.
How your evaluation team will manage the process
Write a timetable for the evaluation process as early as possible and ensure that everyone in the team is available when they are needed.
You need to plan how written responses from suppliers will be received. Make sure that:
- you give suppliers an email address to send their written responses to
- no one in your evaluation team has access to the email account that written responses are sent to
- no one in your evaluation team sees any supplier responses before the deadline
- you keep an audit trail of who has access to the email account that written responses are sent to
If a supplier wants to change their written response before the deadline, ask them to resend it. You must use the latest version received before the deadline.
Prepare your assessment templates
Add your evaluation criteria and how the supplier responded to your essential and nice-to-have skills and experience to the templates you’re using for each assessment method.
2. Invite shortlisted suppliers to be assessed
When you email shortlisted suppliers, provide all the information they’ll need to participate in the assessment stage.
You can only give suppliers additional information to write a proposal if:
- it expands on your published requirements
- it wouldn’t have affected a supplier’s decision to apply for the work
- you send the information to all shortlisted suppliers
If suppliers have any questions, you must:
- only answer them if suppliers need the information to write their proposals
- send their questions and your answers to all shortlisted suppliers
- remove any reference to the supplier’s name or any confidential information about the supplier
You must invite all the shortlisted suppliers to all the assessments you run.
Tell suppliers how the assessment process will work, including:
- what the timetable for the assessment process is, for example any deadlines and dates for meetings
- how long each assessment will be, for example how much time is allocated for a presentation
- where each assessment will be, for example the address of your office
- who will be at the assessment, for example interview panel members
- who they should bring (if in person) to the assessment, for example they should bring everyone who will working on the project
- what equipment will be available for the assessment, for example wifi or projector
- what templates and formats you need them to use, for example for a case study or a work profile
- how you want them to break down their price, for example the price should show how many roles they are proposing, for how many days and what the day rates are for each role
- where they should send any written response, for example case study, work history or proposal
If your assessment timetable changes, give suppliers new dates as soon as you know them and make sure suppliers still have enough time to respond.
3. Assess suppliers
Use as many of the assessment methods you published with your requirements as you need to. You don’t have to use them all if you find a supplier that meets your needs after the first assessment stage.
You can only open suppliers’ written proposals after the response deadline.
In each assessment, suppliers will give you evidence of how they meet your evaluation criteria. Your evaluation team should take notes as it goes so you can measure how well the evidence they provide meets your requirements.
You can ask suppliers questions to clarify things they’ve said.
Read about the ways to assess suppliers.
4. Score suppliers
You must score shortlisted suppliers against your criteria after each assessment. Use the scoring template and the evidence from your notes. Your evaluation team shouldn’t share scores until all suppliers have been scored on every assessment method.
Read how to score suppliers.
5. Agree an overall score
When your evaluation team has finished scoring all suppliers, you need to agree a score for each criteria. You can’t take an average of the evaluation team’s scores.
You must:
- discuss why each evaluator gave each supplier the score they did for each criteria
- reach a consensus on each supplier’s score for each criteria
Use these agreed scores and the weightings you published with your requirements to calculate each supplier’s overall score.
Read more about how to score suppliers.
6. Award a contract and give feedback to unsuccessful suppliers
The winning supplier is the one with the highest overall score. When you’ve identified the winning supplier, you must notify the remaining suppliers of your decision at the same time. You can notify suppliers by email.
Notify the successful supplier
Tell the winning supplier you’ll award them a contract.
Read about how to award a contract.
Notify unsuccessful suppliers
Suppliers need to know if and why they weren’t successful so they can plan for other work and improve any future applications they make.
You must:
- tell unsuccessful suppliers that you will not be awarding them a contract
- give feedback explaining why a supplier was unsuccessful, including the advantages of the successful bid compared to the unsuccessful bid
- give positive feedback where appropriate
- only comment on that supplier’s scores, so don’t share specific details of other unsuccessful supplier’s scores
- give only the final agreed scores, not individual evaluator scores
- give the scores of the winning supplier
What happens if you don’t find the right supplier
You don’t have to award a contract if you can’t find a suitable supplier. You must tell all remaining suppliers that:
- you haven’t found one that meets your needs
- you’re not going to award a contract
You may choose to review what you need and then publish new requirements.
What to keep a record of
You must keep a record of how you’ve made your decisions and any communication you have with suppliers.
Where to get help
You can get help assessing technical competence from:
- digital experts in your organisation
- government digital communities of practice
- experts outside your organisation
You can get help with the evaluation process from:
- your commercial team
- the Crown Commercial Service (CCS)
Read about:
Last updated 1 October 2019 + show all updates
-
Buyers must give feedback to unsuccessful suppliers that explains the advantages of the winning bid. You must get agreement in writing from the supplier if you want to use the written evidence of their essential and nice-to-have skills and experience in the evaluation stage.
-
First published.