Research and analysis

Research to support the independent TEF review: survey of HE careers advisors

Published 21 January 2021

Applies to England

Research overview

A questionnaire for HE careers advisors was designed by the DfE HE Social Research Team to meet the requirements of the Teaching Excellence and Student Outcomes (TEF) independent review.

The questionnaire aimed to establish:

• What are careers advisors’ current levels of awareness of TEF and use of the TEF in their career advisory role?
• How useful do careers advisors think TEF is for pupils making HE choices?
• How may careers advisors use provider-level and subject-level TEF in the future?

Methods

• UCAS administered the survey via their monthly newsletter to careers advisors registered on their careers advisors database.
• Analysis was undertaken by DfE.
• The research was undertaken in March 2019.

Response

The questionnaire received 226 responses:

• 210 centres had dedicated careers advisors; 31 centres had a part-time advisor, 121 centres had either 1 or 2 dedicated advisors and 58 centres had 3 or more advisors
• 72 advisors had been advising for 0 to 5 years, 46 for 6 to 10 years and 96 for over 10 years
• 163 centres in the UK, 32 from an EU country and 31 from outside the EU
• 111 centres with less than 250 Year 12 and 13 pupils, 54 with 250 to 500 pupils and 36 with over 250 pupils

Respondent centres

Type of centre	Number of centres
State schools	38
Academies and free schools	37
Independent schools	37
Colleges	33
Specialist schools, academies and colleges	8
Education/careers organisations	5
Other UK	5
International schools and colleges – EU based	20
Education/careers organisations – EU based	12
International schools and colleges – Outside EU	24
Education/careers organisations – Outside EU	7
Total	226

Note: we cannot confirm whether this sample is fully representative of all careers advisors. However many of the key careers advice services centres are represented in the research.

Knowledge of TEF

Overall, 73% of careers advisors had heard of the TEF, with 55% of those who had heard of it considering they knew a fair amount or a lot about it.

Careers advisors’ knowledge of TEF was compared by location of advisor and to applicants in an applicant based companion survey.

Knowledge of TEF before taking the survey	All careers advisors	UK advisors	EU advisors	Outside EU advisors	All applicants
I had heard of the name, TEF	50%	53%	41%	42%	41%
I had heard of gold, silver and bronze awards	59%	61%	56%	55%	28%
I had heard of provisional awards	17%	20%	9%	10%	6%
No, I had not heard of any of the above	27%	24%	34%	33%	47%

For those who knew about TEF, how knowledgeable did they think they were?	All careers advisors	UK advisors	EU advisors	Outside EU advisors	All applicants
I had heard of TEF, but didn’t know anything about it	10%	8%	10%	19%	30%
I knew a little bit about TEF	35%	31%	43%	53%	44%
I knew a fair bit about TEF	38%	42%	24%	29%	22%
I knew a lot about TEF	17%	19%	24%	0%	4%

Applicant source ^{[footnote 1]}

Insights

• 73% of all advisors had heard of TEF (76% UK advisors, 66% EU advisors, 67% advisors outside EU) and 53% of applicants had heard of TEF.
• UK careers advisors are relatively more knowledgeable about TEF than EU and international advisors.
• Over the whole sample of careers advisors 40% are relatively knowledgeable about TEF (55% of the 73% who had heard of TEF knew a fair bit or a lot about it) compared to 14% of all applicants (26% of the 63% who had heard of TEF knew a fair bit or a lot about it).
• These findings suggests careers advisors knowledge of TEF is not reaching applicants as well as it could.

Knowledge of TEF – UK careers advisors

For the UK, 76% of careers advisors had heard of TEF, with 61% of those who had heard of TEF considering they knew a fair amount or a lot about TEF.

Knowledge of TEF before taking the survey	All UK careers advisors	State schools	Academies or free schools	Independent schools	Colleges	All applicants
I had heard of the name, TEF	53%	45%	38%	60%	58%	43%
I had heard of gold, silver and bronze awards	61%	53%	43%	73%	73%	31%
I had heard of provisional awards	20%	16%	16%	27%	18%	7%
No, I had not heard of any of the above	24%	32%	46%	16%	9%	45%

For those who knew about TEF, how knowledgeable did they think they were?	All UK careers advisors	State schools	Academies or free schools	Independent schools	Colleges	All applicants
I had heard of TEF, but didn’t know anything about it	8%	8%	10%	6%	7%	31%
I knew a little bit about TEF	31%	38%	15%	29%	33%	44%
I knew a fair bit about TEF	42%	31%	55%	55%	37%	21%
I knew a lot about TEF	19%	23%	20%	10%	23%	4%

Applicant source ^{[footnote 1]}

Insights

• Advisors at colleges (91%) and independent schools (84%) appear more likely to have heard of TEF and the awards than state schools (68%) and academies or free schools (54%).
• Of those who had heard of TEF, advisors at academies or free schools (75%) were most likely to consider they know a fair amount or a lot about TEF compared to independent schools (65%), colleges (60%) and state schools (54%).

Note: analysis by UK sixth form type uses small sample sizes for the sub-sample of those who knew about TEF (n<30 for each type), and we know the careers advisor sample may not be representative of all careers advisors and sixth form types. Further research is required to confirm the accuracy and robustness of these findings.

Informing students about TEF

42% of careers advisors who had heard of the TEF had informed students about the TEF to at least a moderate degree.

Statement	Yes, a lot	Yes, a moderate amount	Yes, a little	No, not at all	Unsure	Total
I have informed students about the TEF	15%	27%	31%	26%	1%	100%
I have informed parents about the TEF	10%	20%	27%	42%	1%	100%
I have informed teachers about the TEF	7%	14%	25%	53%	1%	100%
I have used the TEF to help individual students make decisions about where to study	12%	21%	28%	39%	0%	100%

Insights

A number of careers advisors indicated they were aware of the TEF, but had not fully used it with their students. Reasons given included:

• advisors lack of knowledge and lack of time to gain knowledge
• advisors are yet to be convinced about TEF’s accuracy, credibility, reliability or objectivity
• other choice factors are more significant to students
• TEF lacks departmental or course level awards

Comments from careers advisors

Asked a student to look at it when they couldn’t decide between institutions that were very similar.

We display the awards in our sixth form area for students to see.

Used in careers lesson on How to research HE and the use of league tables.

TEF is not necessarily a reliable measure of overall university quality. Top universities do not always achieve gold awards.

The results looked very mixed. If a university is good at ensuring its students respond to surveys/questions, they may score better than other institutions.

Informing students about TEF – UK careers advisors

42% of UK careers advisors who had heard of the TEF had informed students about the TEF to at least a moderate degree.

Statement	Yes, a lot	Yes, a moderate amount	Yes, a little	No, not at all/Unsure	Total
I have informed students about TEF	15%	27%	33%	25%	100%
I have informed parents about TEF	8%	20%	27%	45%	100%
I have informed teachers about TEF	7%	16%	24%	53%	100%
I have used TEF to help individual students make decisions about where to study	9%	21%	31%	39%	100%

School type or college	Informed students to at least a moderate degree	Informed parents to at least a moderate degree	Informed teachers to at least a moderate degree	Helped individual students to at least a moderate degree
State school	39%	33%	29%	16%
Academy or free school	45%	20%	5%	35%
Independent school	38%	27%	21%	42%
College	43%	28%	17%	20%

Insights

• UK careers advisors had a very similar pattern of use of TEF to the average number of careers advisors.
• Comparison with the small number of advisors based outside the EU suggests use by advisors outside the EU may be higher than UK or EU based advisors, but sample sizes are too small draw robust conclusions.
• In terms of how different careers advisors at different types of sixth form used TEF:
  • academies or free schools, and colleges appear most likely to inform students about TEF
  • independent schools appear most likely to use TEF to help individual students to make decisions
  • state schools appear most likely to share information about TEF to both teachers and parents

Note: small sample sizes for sixth form types who had heard of TEF (n<30).

Value of TEF

Careers advisors’ views about the value of TEF are broadly positive.

Statement	Agree	Neutral	Disagree	Not sure	Total
TEF gives useful information about the teaching and educational experience my students can expect	56%	13%	11%	20%	100%
A single rating like TEF makes it easier for my students to use the information when making decisions	55%	11%	20%	14%	100%
The independence and standardisation of TEF assessment process makes it a valuable resource for students	53%	18%	12%	17%	100%
TEF makes it more confusing for students when trying to decide where to apply	24%	18%	40%	18%	100%
TEF duplicates information already available and is not needed	26%	19%	34%	21%	100%

Insights

Whilst careers advisors views of the value of TEF to students are broadly positive, applicants themselves rate the value of TEF higher than careers advisors:

• useful information from TEF: 70% applicants, 56% advisors
• ease of use of a single rating: 67% applicants, 55% advisors
• value of TEF independence and standardisation: 63% applicants, 53% advisors
• TEF adds confusion:17% applicants, 24% advisors
• TEF duplicates information and is not needed: 18% applicants, 26% advisors

Value of TEF – UK, EU and international careers advisors

% agreeing with statements	TEF gives useful information	Single TEF rating is easier to use	Independence and standardisation is valuable	TEF makes deciding where to apply more confusing	TEF duplicates information already available
All	56%	55%	53%	24%	26%
UK	54%	54%	52%	26%	28%
EU	66%	59%	61%	21%	29%
International	60%	57%	50%	17%	13%

Insights

• EU careers advisors appear to value TEF more than UK and international advisors in terms of it providing useful information, being easier to use, and valuing the independence and standardisation.
• International advisors are less likely than UK and EU advisors to agree that TEF adds confusion or duplicates information.

Value of TEF – UK careers advisors

% agreeing with statements	TEF gives useful information	Single TEF rating is easier to use	Independence and standardisation is valuable	TEF makes deciding where to apply more confusing	TEF duplicates information already available
State schools	51%	57%	65%	41%	32%
Academies or free schools	38%	50%	44%	21%	33%
Independent schools	59%	57%	54%	19%	19%
Colleges	65%	55%	45%	19%	32%

Insights

Across different types of sixth form in the UK the picture is more mixed:

• Colleges and independent schools are most likely to agree that TEF provides useful information about teaching and educational experience, and least likely to agree TEF makes decision making more confusing.
• Independent schools are also least likely to agree TEF duplicate information already available.
• State schools are most likely to agree the independence and standardisation of TEF is valuable, and also that TEF makes applying to HE more confusing.
• Academies or free schools are most likely to agree the a single rating is easier to use (though this a lower rate of agreement than all other sixth form types).

Importance of TEF compared with other decision-making factors

When considered alongside other decision-making factors TEF rating of HE provider is least important of 15 factors:

Decision-making factor	% saying important or extremely important
Academic entry requirements	95%
Graduate prospects	94%
Quality of teaching facilities	90%
Quality of staff/teaching	88%
Good student reviews	82%
The range of modules	81%
Work experience as part of course	76%
Earning potential of graduates	70%
Challenging course content	67%
Institution pre-application communication	63%
Cost of tuition fees	62%
Position in league tables	61%
Incentives on offer	61%
TEF rating of subject at institution	48%
TEF rating of institution	37%

Insights

Factors are rated in a different order and have a different importance rating for careers advisors and applicants^{[footnote 1]}, for example applicants:

• highest importance: quality of staff/teaching: 95%
• lowest importance: TEF rating of institution: 40%
• importance of TEF rating of subject: 51%

There were some differences between careers advisor by location and sixth form type:

• 97% of EU and 100% of international advisors rated graduate prospects most important
• 64% of International advisors rated subject TEF important; 12th of 15 factors, the only sub-group of advisors not rating it as 14th of 15
• by comparison, 59% of EU advisors rated subject TEF important, as did 42% of UK advisors
• academies or free schools rated the importance of subject and provider TEF the highest at 45% and 38% respectively; colleges the lowest at 38% and 28%

Note: Whilst TEF was ranked lower compared to other factors, many of the factors that are rated highly are composite elements within the TEF rating.

Applicant source ^{[footnote 1]}

Anticipated use of subject-level and provider-level TEF ratings

The majority of careers advisors favoured using both subject-level and provider-level TEF ratings.

If both provider-level and subject-level ratings were available, which do you think you would use?

If both provider-level and subject-level ratings were available, which do you think you would use?	Percentage
Both provider and subject ratings	80%
Only provider ratings	2%
Only subject ratings	9%
Neither provider or subject ratings	9%

Insights

Most advisors say they would use both provider-level and subject-level TEF, but subject-level is slightly more popular:

• 9% say they would use only subject-level
• 2% would only use provider-level

Careers advisor use of subject-level TEF

After hearing about subject-level TEF:

• 71% of advisors say they are likely or very likely to use subject-level TEF with their students
• international advisors are more likely to think they will use subject-level TEF (82%) than EU advisors (73%) and UK advisors (69%)
• advisors from state schools (73%) and independent schools (72%) are more likely to think they will use subject-level TEF than advisors from academies or free schools (67%) and from colleges (62%)

Note: this used a more sensitive scale than the previous question, which includes 8% unlikely or very unlikely to use subject-level TEF and 21% unsure or don’t know.

What impact would subject-level TEF ratings have on the likelihood of encouraging students to apply?

Award	Much more likely to apply	A little more likely to apply	It wouldn’t impact my decision	A little less likely to apply	Much less likely to apply	Total
Gold	41%	29%	26%	0%	4%	100%
Silver	16%	45%	35%	3%	1%	100%
Bronze	3%	23%	48%	23%	3%	100%
Provisional	2%	9%	66%	18%	5%	100%
No award	3%	1%	61%	17%	18%	100%

Impact of receiving additional TEF information

Having additional information about the TEF has encouraged advisors to consider using the TEF more widely with students.

Careers advisors were presented with further information about the TEF throughout the survey. They were asked whether this additional information would make them more or less likely to use the TEF in their role as an advisor in the future.

How more likely are you to use TEF	More likely to use TEF	No difference	Less likely to use TEF	Don’t know	Total
All advisors	42%	44%	2%	12%	100%
All advisors not aware of TEF prior to the survey	51%	27%	0%	22%	100%
UK advisors	39%	49%	2%	10%	100%
EU advisors	47%	40%	0%	13%	100%
Advisors outside EU	53%	27%	3%	17%	100%
State schools	57%	39%	3%	1%	100%
Academies or free schools	33%	46%	0%	21%	100%
Independent schools	30%	60%	5%	5%	100%
Colleges	27%	53%	0%	20%	100%

Insights

Provision of additional information about TEF has encouraged advisors to consider using TEF or using TEF more widely with their students.

Advisors from centres outside the UK were more likely than UK advisors to think they would use TEF now they had additional information about it.

This reflects findings that advisors outside the UK generally knew less about TEF than advisors in the UK and around a third were not aware of TEF prior to the survey compared to around a quarter in the UK.

Views about TEF – evidence from careers advisor comments in the survey

Careers advisors considered TEF was useful to assist decision making, but other factors could be more important

It will be a reference for my students if they can’t decide which university to go to.

At the end of the day there are other factors that come in to play – notably the reputation / Russell Group status, the location, parents wishes and the customer service available, whether the unis are supportive – the personal touch.

The TEF provides only part of the information that students need when choosing a university. It makes a difference to choice at the margins, but other factors influence students more significantly.

EU and International advisors in particular felt they needed more TEF information which would enable them to use the TEF with applicants

I hadn’t heard of the TEF prior to this survey, but will look into it and be likely to use it in my role.

I wasn’t aware of the focus on outcomes and employability (I thought it was a quality award for teaching) and this emphasis on employability and student experience would be highly relevant for our students who are often eligible for WP schemes.

Careers advisors are yet to be fully convinced about TEF’s accuracy, credibility, reliability or objectivity – though in some cases this appeared to be due to lack of knowledge

I am not convinced by the method used to establish whether and which TEF a university deserves, therefore I would not want to rely on this.

I’m not sure it is an accurate reflection and true benchmark of the teaching and learning environment at the university. I don’t know what it really measures…

Advisors thought that subject-level TEF would be more useful than provider-level. Caveats about the credibility of TEF processes and usefulness of other choice factors remained in the comments.

This is what I felt was missing from the TEF, so if this is being included, this will be useful.

More specificity would certainly make TEF a more useful resource.

Far more useful than a rating for the whole university due to variation between faculties.

We always try to make students aware of the information available so that they can research and use it themselves – but we have and will continue to make them aware of how TEF decisions are made so that they can choose how much store to put by them.

1. Applicant source – UCAS Applicant Omnibus survey 2019 with DfE questions ↩ ↩² ↩³ ↩⁴