Fiona Dickie, Pubs Code Adjudicator, provides insights on the journey towards a more data-driven approach.
In our recent webinar reviewing the key findings of the Data in Regulation report, panellist Fiona Dickie answers questions from panel facilitator Naomi Nicholson on using data to influence the behaviours of regulated bodies and moving away from qualitative data towards a more data-driven approach.
Note: The following content is an edited transcript from the Data in Regulation webinar. The views expressed are personal.
Naomi Nicholson: Fiona, you kindly provided a great case study for this research, all about how the Pubs Code Adjudicator (PCA) is using data to influence the behaviours of the bodies that you regulate to achieve good regulatory outcomes. Could you say a bit about how you did this? What some of the challenges were and what you see is the benefits of taking this approach?
Fiona Dickie: Yes, I'm really happy to tell you a bit more about what we do with data. The Pubs Code Adjudicator is a relatively new regulator; the Pubs Code was introduced in 2016. What the Pubs Code does, is it regulates the relationship between the tied pub tenant (TPT) and their big pub company. Parliament, through select committees, had received evidence for over 10 years about concerns over the treatment of tied tenants. A tied tenant is one who rents the pub premises from a landlord but is also contractually obliged to buy their alcohol from that landlord. Parliament was concerned that the contractual relationship was not equal and that there were inequalities of bargaining power in that relationship. Parliament decided to intervene in a discordant industry by regulating to ensure fairness to tied pub tenants in their business relationship.
There are currently about 8,000 regulated tied pubs and six regulated pub companies. To come under the Pubs Code, the pub company has to have more than 500 tied pubs. It's the biggest players in the industry who are now subject to statutory regulation.
So, the Pubs Code Adjudicator, we're a tiny regulator. We currently have a staff team of around ten. We regulate in a commercial environment. The bodies that we regulate are in competition with each other: competition for the best pub operators; competing with their business models; and their pubs are in competition. Our role is really to find where the PCA can exert the most influence in driving up behaviours and standards.
Our experience has been that the, albeit valuable intelligence that we get through Code disputes, inquiries to the office, other information that comes to us wasn't providing us with a robust evidence base or robust data to enable us to leverage big change. We came into being at a time when there was discord within the industry, and there has been significant pressure to show progress, to show the impact of the Code and improving the lot of the tied tenant of these big pub companies. Nobody gave the PCA a quantitative objective, but there was a lot of narrative evidence about what was happening in the tied pub trade. But, being subjected to this expectation and pressure to show the impact of the Code, it became clear to us that data was going to be a really important tool.
Being subjected to this expectation and pressure to show the impact of the Code, it became clear to us that data was going to be a really important tool.
We started surveying the attitudes of TPTs annually – their experiences of Code rights and their attitudes to their pub company. We started initially with a small survey of 400 tied tenants and a small number of in-depth interviews. We decided over time that was too small a sample to give us the leverage that we wanted with the pub companies. Importantly, what it didn't enable us to do was to compare the survey results of the six pub companies. In order to do that, we needed a statistically significant sample of all six pub companies. So that's what we did from 2023, which was our fourth annual survey.
This year (2024) is our fifth. We increased our survey sample to 1,200 tied tenants, which is 15% of the regulated industry. The explicit goal of doing that was to be able to compare the conduct and compare the results of the different pub companies and put that data in the public domain so that it would be available to all.
We did that in 2023 for the first time and it enables us to do several things; it enables us to obtain insights for our regulatory priorities. We ask a range of questions about different areas of the Code. The Code is, in the main, a transparency obligation on the pub companies. It requires the pub companies to give tied tenants important information before they sign up for the pub. It requires them to ensure the tenant is trained, is advised to get professional advice and has a sustainable business plan for the pub. It also requires transparency around the rent that is proposed and that be analysed and supported with evidence. Transparency around premises insurance, obligations and in other areas. So, there are lots of points in the tenancy cycle where Pubs Code rights arise. We've developed survey questions which look at those interactions and test tenant satisfaction with them. That enables us to inform our work for where to target our very limited resources in our regulatory interactions. It certainly informs our conversations with the pub companies, there's no doubt about that.
The other thing that it enables us to do is to measure the impact of the Code year on year. That ability will be even greater this year, when we have the second survey of the increased survey sample size of 1,200. That's very important for us to be able to do and important for the pub companies as well because our experience is that the pub companies are genuinely interested in this data. They are used to looking at data about how their businesses are running and this is part of the data that is available to them. We publish everything – our questions and all the data that we collect. We, of course, do some analysis in our presentation: these are the key points, these are the highlights and we do design and present around that, but all the raw data is there in any event for the companies and others to have. It also provides insights for our priorities; it enables us to measure Code impact and progress year on year.
It also provides a competitive incentive for the pub companies. Because there is a prize here, which is to be the pub company with the most satisfied tied tenants, or the pub company with most improved performance, or the pub company whose Business Development Managers, who have the day-to-day relationship with the pub tenant are dealing with their pub tenants most fairly.
It’s been a journey to get this far. It's been a journey to develop the questions that we think are right. We have done a lot of internal work in developing the survey, which has been a challenge. It's been a journey in learning how to market and communicate this data that we have. We now have a communications partner who helps us communicate that to the pub trade in a relevant and meaningful way and also seeks to attract media attention.
There are of course challenges around it – there's a need to be fair around the data because, some of it does need interpretation. There are, for example, variables in the results depending upon the type of operator agreement and the pub companies have different estates. So sometimes, we are comparing apples with oranges. We have to ensure that we're fair and transparent and balanced about doing that. But, after doing this five times, our experience is that, for a small organisation, we've become quite good at it.
It is an area that we really benefit from leveraging external expertise because we procure the survey. We have a three-year contract, which we will re-procure this year but, at the moment, our contractor is Ipsos. So, we are able to leverage all that additional expertise and people power to produce a quality end product.
Naomi Nicholson: I think it’s impressive that as a small regulator you’ve got one product that you’re focusing on, and it does so much for you – that risk-based regulation, measuring your impact and progress and influencing the regulated bodies with that positive competition. You mentioned in the report case study that you moved away from qualitative information in the survey towards a more data-driven approach. Why do you think that the data-led approach/ the numbers have had more traction with the bodies that you regulate?
Fiona Dickie: Before the birth of the Pubs Code and the PCA, the pubs trade was awash with a narrative of dissatisfaction. It was not easy to show that, and it was not easy to gain traction by looking at narrative. It was useful for us to see that dissatisfaction was there, but it was not something that we found particularly useful in our communications with pub companies, because it was simply one tenant’s or a number of tenants’ description of their particular experience. There was much more power for us in having the data. It became very clear to us that having statistically significant data was much more important because we really wanted to be able to communicate with the industry and the pub companies in a way that was impactful and objective which our current survey really does.
I think the way that you communicate data is important, because we are able to package the data in different formats and media. Whether it be delivering bar charts and graphs and smiley face graphics on social media and such, it just has a marketability that is wholly different from a narrative approach. As I said, the pub companies, they really value it. They want to know it matters to them and that they are succeeding - it's commercially valuable to know how their business relationship is working.
Further information
 | Fiona Dickie Pubs Code AdjudicatorFiona Dickie served as the Deputy Pubs Code Adjudicator since November 2017 before being appointed Pubs Code Adjudicator from 3 May 2020. She was a Vice President of the Valuation Tribunal for England from 2009 until 2020 and is a Judge of First-tier Tribunal (Property Chamber), having been appointed in 2013 (after serving as a Lawyer Chair of its predecessor tribunal from 2006). She was also appointed as a Road User Charging Adjudicator in 2004. Fiona has been a mediator in civil disputes since 2005 and a member of the Chartered Institute of Arbitrators since 2018. |
 | Naomi Nicholson Regulatory ConsultantNaomi is an experienced regulatory leader, currently working as Director of her own regulatory consultancy. |
