The distributors, which include the Big Lottery Fund and Arts Council England, receive about £1,400 million a year between them to give out as grants but currently spend about 6.5 per cent on grant administration costs.
Mr Penrose has written to distributors calling on them to match best practice in comparable charities, which is about five per cent.
The move is the latest in a number of changes to reform the lottery, including increasing the share of lottery income that goes to arts, heritage and sport, and ensuring the Big Lottery Fund focus their grants on the voluntary and community sector.
Mr Penrose said: “Lottery players have a right to expect their cash to be used wisely and effectively on good causes. We pledged to cut Lottery running costs because it’s vital to public confidence that as much money as possible goes to the arts, sport, heritage and charities while still ensuring that funds are managed properly.”
Distributors have also been asked to cap their total running costs - including research, technical support and public involvement in grant decisions - to eight per cent in line with best practice among charities carrying out similar activities.