RAJAR - Treat With Caution...

Happy RAJAR Day!

Yes, today, July 31st is RAJAR Day. Four times each year radio programmers up and down the nation receive official confirmation of 'how they are doing'. As always we should expect some truly erratic and seemingly inexplicable results. 
 
Below you'll find my thoughts on RAJAR, together with some background on what is one of the world's biggest ongoing research studies, plus a few tips on how to use the data productively. 

Years ago, my routine on RAJAR day would begin with a long drive down to the offices of Research Services Limited in Harrow. The results were made available at 2:30pm prompt. I remember the agony of peeling open the brown envelope and taking my first glimpse at the latest results. "We're up .... yipeeee!" or "Oh b***er.....we've taken a hit". (Generally back then, in those Halcyon days before the airwaves were opened up, my station, Signal Radio, enjoyed strong, stable results).

Then, arriving back at base it was out with the graph paper and felt tip pens for some serious graph drawing!  (I'm really showing my age here!) I would often stay up all night preparing my 'Weekday and Weekend Trends Over Time' which I would share with my team the following day.

These days it's all much quicker and, somehow, far more clinical. A small file download at 10:30am provides subscriber stations with all the raw data they need in order to start processing graphs and preparing News releases.

What Is RAJAR?

RAJAR (Radio Joint Audience Research Limited) was established in 1992 to operate a single audience measurement system for the radio industry - BBC and commercial stations. Previously the BBC had their own system and commercial radio used something called JICRAR, so it was commonplace for the two sectors to 'rubbish' each others results. RAJAR brought all that to an end......at least that was the plan!

The first RAJAR survey was in Quarter 4, 1992. That was a 'bumpy' transition, with some isolated cases of major variance between audiences measured by the two systems. At least three Programme Controllers and several presenters lost their jobs at this time. Both Viking and TFM suffered terribly and, at Signal, we saw an apparent 38% decline in listening. What an introduction to RAJAR for me! I was lucky, though. My Managing Director at the time, Barry Machin, was extremely research-literate, with a logical, common sense approach meaning he simply didn't, couldn't believe that in the space of just three months over a third of our audience had gone away. Of course he was right and, sure enough, as things stabilised the station recovered fully, going on to make big increases becoming the Number One AM/FM combo in England and Wales, (in terms of audience share). Barry went on to chair the Commercial Radio Association's Research Committee for a number of years.

In common with the current system, the first RAJAR research was based on seven day self-completion diaries, personally placed and collected by interviewers. Respondents complete their diaries using a list of relevant stations which are listed across each diary page. Diaries are then placed with respondents who are selected according to specific criteria to meet pre-defined sampling quotas.

ACCURACY
Having spent four years in close proximity to the most outspoken critic of diary-based research, Kelvin MacKenzie, (I was his Group Programme Director between 2001 and 2005), you may be surprised to read that I am, in fact, amazed by the consistency of RAJAR over time. However, the interpretation of RAJAR and any tweaks to output based on apparent performance are areas fraught with danger.

Before you open the Champers....or jump off a high building, come to that....read on about the risks of taking your audience research at face value.

Rather than focusing on RAJAR as one specific research study, let's look at statistics in general:

Confidence Limits

The table to the right sets out the 'confidence intervals' which can be given to percentages based on the size of sample used. The larger the sample, the narrower the band and the closer the observed percentage is likely to be to the true figure. Taking this to its extreme, if every member of the population is questioned, and they all tell the truth, the survey should, in theory, be 100% accurate.

For example, taking a sample size of 650, and an observed percentage at around 20%, the table shows that there are 95 chances in 100 of the true percentage lying between 15% and 25%; in other words, the sample percentage of 20% is estimated to be accurate to plus or minus 5%.

So, if your station's reported reach is, say, 24%, it could, in reality, be anywhere between 19% and 29%. That's a simple statistical fact.

Obviously, once you start drilling down into the demographics or half-hourly numbers, (as we all do), these confidence limits are exaggerated and this is where many programmers have made big mistakes with knee-jerk reactions to apparent huge swings within dayparts and/or demographics.

---

A RAJAR Diary

RECALL V REALITY
One of the major recurring issues is based on the fact that people generally find it difficult to remember what they listened to and when. Furthermore, how often do you find yourself 'exposed' to radio without actually knowing what the station is? Diary research tends to favour the established 'heritage' stations as these are more likely to be front of mind than lesser brands. Kelvin MacKenzie famously went out on a limb and challenged the establishment, putting forward a compelling case for electronic measurement. The bitter war of words is now history, but RAJAR did commit to a move away from the paper diary system and now online diaries are also used. The online diary station selection is carried out by the respondent, each person is asked to look through a set of station names onscreen that are available in the area and invited to select all the stations which they might listen to or hear in various situations. 

---

THE PROBLEM WITH RECALL
Here's a true story concerning a former senior manager at one of the stations I used to programme. We'll call him 'Mr. X'. The individual in question is a total radio enthusiast, extremely tuned-in to the medium. Shortly after he had parted company with the station 'Mr. X's wife answered the door to somebody from the research company Ipsos/MORI. They wanted to place a diary with the household but, before doing so, had to check that no-one in the home worked in radio. Well, having just left the station 'Mr. X' could honestly declare no vested interests and accepted the diary, relishing the prospect of spending time painstakingly logging his listening patterns over the next seven days.

A week later 'Mr. X' was putting the kids to bed when the door bell rang. 'Mrs. X' answered the door again and it was the same Ipsos/MORI fieldworker who had come back to pick up the completed diary. Of course, 'Mr. X', despite his best intentions, had completely forgotten about the survey and shouted downstairs, "....I won't be a minute...", then hurriedly spent five minutes scribbling in the diary, desperately trying to remember what he'd been listening to over the course of the week. And this is an ABC1 radio professional. Imagine what goes on in 'ordinary' households!

Oh, and a question I often get asked is "how much do RAJAR respondents get paid?" The answer is nothing, although they do receive a pen!

RAJAR Analysis and Reaction

There's a real skill to RAJAR analysis. My advice is as folllows:

• Always look for long term trends;
  
• Challenge anything that seems unusual;
  
• Check sample sizes and quotas;
  
• Be careful with timeframes....especially if you are carrying six or twelve months of data;
  
• Never, EVER make reactionary changes to output based on a single quarter's data....unless RAJAR supports your previously held 'gut' feel;
  
• Look at relativities. If you are trending down, say, on Sunday mornings how does your station's share compare with that of other stations in similar landscapes?
• Please resist the temptation to look at the 'Discretes' (e.g. 3 months of data within a 6 month roll). This can be very misleading because of the reduced and unbalanced samples;
• Sample Point Analysis? Forget it! 15 diaries in isolation mean nothing.

RAJAR critic Kelvin MacKenzie

RAJAR DAY AND KELVIN
Kelvin MacKenzie is on record as having no confidence whatsoever in RAJAR. He passionately believed that his company was being denied millions of pounds in revenue because the diary system disadvantaged his national ‘flagship’ station, talkSPORT.

Strangely, though, four times a year something very odd occurred. I would ring Kelvin on his mobile with a ‘heads up’ on RAJAR, expecting him to say something like, “….ah, we all know it’s a load of b****cks….”, but instead, he would want to know every minute detail. “Where are we up…..WHY?”, “Where are we down….WHY? What are you gonna do about it? And WHEN?" Suddenly, for a few moments at least, RAJAR had become the Holy Grail!

It was a similar story at The Revolution with Steve Penk. Steve, like me, has become a cynical 'old soldier' and, bears deep scars from his time as a presenter when RAJAR was a matter of life or death. But now it's really important! As the sole owner of the radio station Steve's horizons have broadened and commercial performance is now uppermost in his thoughts. RAJAR is crucial because it has a major impact on national revenue. In fact there is a direct correlation between audience - as measured by RAJAR - and the share of national advertising revenue the station receives.

So today I want to pass on my best wishes to Steve and all my other radio friends for a kind set of figures for Q2 2013.

---

Draw a Line and Move On

When you examine your RAJAR results today, please bear in mind that you may have had a larger than usual change in POPULATION,  This is due to the incorporation of the 2011 census data, for the first time.  This may impact on your reach '000's and your percentage reach figures. 

Whatever your results today remember, this is just a moment in time. There's nothing you can do to change what's going to be published at midnight so celebrate, commiserate or whatever, then move on. There'll be another set of results along in three months time. Bet you can't wait!