Lies, Damned Lies, and Mobile Statistics

Pinocchio

When I started in mobile in 1999, we were starved of any kind of numeric validation for our work. How many people had a mobile phone? How many used SMS? How many had ever texted into shortcodes?

These questions had to be ignored or ridden over roughshod in order to sell in any work, or we simply relied on a good idea that kind of made sense.

Nowadays though there are more stats than you could possibly know what to do with. While it’s hard (sometimes) to argue with the accepted authorities such as Comscore, Nielsen et al, the industry has now become polluted by the interests of various parties trying to sell their own micro-discipline of mobile with what can only be described as slanted, if not entirely misleading, statistics. Conducted in-house or with the connivance of a research company who are clearly being paid to deliver something that makes good PR, we are now under an avalanche of questionable, and often contradictory factoids about our industry.

This has been raised again today by a PR release masquerading as fact on no less a site than AdvertisingAge with some extraordinary claims. Let’s have a look shall we? Please note this research is entirely about CPG/FMCG goods – that means bread, vodka, toothpaste, tinned cabbage etc…

  1. 47% of men check product availability of CPG goods while in store. There are two things inherently wrong with this. Firstly, if you are in store and you are looking for a product, you go to the aisle, you don’t check on your mobile for it. Secondly – let’s be serious – has anyone ever checked the availability of individual items before going to a supermarket? No – if they don’t have your brand, you buy another. If they don’t have any, you shrug your shoulders or maybe tut a little bit
  2. 51% of men compare prices on CPG goods while in-store. Oh I can see it now. “This cabbage is $1.29 but if I go to comparemycabbage.mobi I can probably get it from WalMart for $1.19, and I’m totally prepared to drive across town for that!”. Men do not compare prices on CPG goods – we chuck it in the cart and move on, often deciding based on which has our favorite color packaging. Do I have data to support this assertion – of course not, but you know I’m right.
  3. 47% of men find coupons or deals on CPG goods while in store. OK, so your vanilla Stoli costs $18.99. What do you do? Think about the price, feel it’s a bit much, search online for Vanilla Stoli vouchers, download it, and present at checkout in the store you’re already in. Is that likely? If you’ve ever done this please let me know. Clues: there are no vanilla stoli vouchers on the mobile web; most retailers simply don’t accept mobile vouchers full stop; the chances of you finding one that works in the store you’re standing are virtually 0%. And men would never do this – again no stats, call it male intuition or something.

This is just one example of a ‘generous’ reading of stats to come up with a load of lies. This may of course have begun as a serious exercise, but something has gone seriously wrong. These stats could well apply to the hi-tech or auto space, where men are waaay ahead of women in their mobile participation and where these activities make some kind of sense. To me it looks like they tried to get some data to support CPG, the stats didn’t look good, so they rolled in a few categories as well to beef it up.

So this is not a one off. Here are my top 3 unlikely but regularly cited mobile stats that are almost entirely complete bollocks. I welcome your favorites in the Comments:

  1. Average teenage girl sends 4,005 text messages monthly. Let’s think about that: 133 texts per 15-hour waking day, 9 texts per hour, one text every 6 and a half minutes. Let’s assume that she’s in class for 6 hours per day, that means a text every 4 minutes. Oh, and that’s the AVERAGE teen girl. Let’s assume that a high-texting teen is x3 the average, which seems reasonable, nay conservative, in any behavioral law-of-averages, she’s sending one every 8.1 seconds. Relentless.
  2. 19% of US consumers have used a QR code. Really. I am more pro-QR than most, but let’s be clear about terms. What is a consumer? I take it we are including everyone from new-borns up to pensioners, right, because that they all consume. The USA has 314m total population. The Census Bureau cites 72% of these as being adult. If 45% of adults are smartphone owners, that gives us a total potential market for QR use of 102m people. Now if 19% of consumers really have used a QR code, that means there should be 59.6m QR-loving freaks out there, or, 3 out of every 5 smartphone owner. Go out and ask 10 of your non-industry friends if they scan. If more than 5 say yes, I’ll build you your very own QR code campaign…
  3. Mobile is the most consumed media channelOK, so I personally think this one is true. But would you trust InMobi, who make their living from selling mobile media space, to tell it to you? Especially when you have the likes of eMarketer telling you that TV and online are still waaay ahead? Would you? And then there’s Mary Meeker (apparently asleep on the job and still showing stats from 2011 on slide 19) who has mobile at 10%, behind even radio

So, there are as many stats out there as there are arguments you may want to win. What are your favorite mobile stats? Which have you used most shamelessly to sell work even though you know it was dreamed up for nefarious ends? And are you one of the necromancers whose job it is to come up with this? Dish the dirt below the line…

Author, Tim Dunn.
21

I'm Director of Mobile at Roundarch Isobar. While I've specialised in mobile for over 13 years, I now work across broad digital marketing strategy, and am interested in interactions, UX, media and marketing theory, and innovation of almost any description. Click About Me for more and info and what this blog is about. Follow me on Twitter @timmcdunn or connect on LinkedIn

14 Comments.
  1. Cameron Wall says:

    Nice work Tim.

    Very true, I also shifted from fixed Web to Mobile in 99′ as it was easy to see where it was all heading. Being able to realise the promise of location and context with a device was the Wholly Grail…we all thought. Then the media agencies realised that if this Mobile thing sticks then their revenues would dive. Why?

    The reason is that having a client with a media annual spend of $5M that you could scatter-gun at TV, Print, OOH etc. and some Web everyone was happy and lunch was had. Enter Mobile, so measurable that if it had any chance of the client investing 4% of that budget and succeeding in reach then you could bet your bottom dollar that that $5M budget next year would be less.

    Stats are great, we all love them and enjoy hearing them, however we really don’t care until the bottom line moves and we start digging. It’s only then we make changes. The unfortunate thing is that there is not that many people skilled up in Mobile marketing and advertising that can reach all the bands.

    • Tim Dunn says:

      It certainly must have been simpler when media boys just had to tick boxes then go out for lunch – halcyon days… I take your point about measurability being a problem for some interest groups compared to traditional media, but quite honestly it’s just plain old inertia and conservatism that holds us back in media land. Still.

  2. Simon Liss says:

    Tim, I too am bored of mobile stats. Conference after conference, deck after powerpoint deck, I see hockey stick graphs, showing explosive growth, surveys, polls, dubious research all designed to prove the importance of mobile, when really, we all know how pervasive the technology is. To anyone touting stats as a basis for strategy, I say remember they are historical, they represent was has happened, and as such can only act as a rear view mirror. In such a fast moving environment they are almost as useless as predictions (although predictions are even harder to disprove) and don’t move us on one inch in terms of developing better experiences.

    • Tim Dunn says:

      Right – I start every one of my training courses with a very clear ‘all these stats are out of date’. I guess the joy of mobile is that they do only go up, or have done for 5 years or so – even that trend could start to plateau as we reach maturity. That’ll be worrying!

  3. Matt Bowden says:

    It’s scary that these sorts of “stats” are banded about as fact when it’s pretty obvious that they’re seriously flawed, as you’ve highlighted here quite nicely.

    As a case in point I saw three different sets of stats yesterday on Smartphone penetration in the UK (varying from 38% to 60%). Now I get that it will depend on when and where you are citing this information from but articles like this from ComScore are seriously damaging their reputation.

    I think its fair to say that we’ve all been guilty of positioning stats to fit our needs or tell a story we want to tell but I personally try not to veer too far from the truth as you will inevitably get caught out by some smartass.

  4. Ian Kater says:

    You should hear how these “stats” guys unrelentingly shed light on “how apps are everything” in the media. Bullshit. The truth of the matter is, feature phones — which cannot support these apps — still account for 70-80% of the global phone market and that will be the case for a long time. Sometimes I just think it’s a space full of idiots who don’t know what they’re doing.

    • Tim Dunn says:

      Ah yes – apps + messaging + mobile web – the holy trinity.

      You make a good point, but it’s definitely one for another blog post on another day!

  5. Terry McCusker says:

    Tim, you’ve Americanised your spelling, ‘color’, ‘cart’ (not shopping trolley) are you go to be saying ‘sidewalk’ and ‘gas station’ next as well? Other than the spelling very insightful. Depending on who you listen to smart phone ownership in the UK at the end of Q4 2012 was either 65% or 50%. People bleat these stats in presentations and they become gospel. There are too many whitepapers and reports from mobile vendors who have their own commercial interests at heart. Unfortunately the MMA’s research is usually outdated as soon as its published. Forrester, in the UK at least, seem to be the most believable. People like ComScore want to talk up the market to protect the interest of their clients.

  6. Ross Sleight says:

    Hi Tim

    Whilst I am the first to call out on stats obfuscation in the case of the above I think its a combination of bad journalism and misinterpretation of the report.

    In the actual report from Millennial http://www.millennialmedia.com/mobile-intelligence/mobile-intel-series/ the cohort is defined as users who “frequently engage with CPG content or ads on their mobile devices”. The in store behaviour that you correlate with this cohort as being around CPG goods, is actually the groups total mobile behaviour and does not refer to their interaction or purchase of CPG goods. Thus if I have interacted with a Kraft recipe add or app, and am thus in the CPG cohort and I then scan a barcode in say an electrical shop, then my activity here is defined as an in store interaction. The report quite clearly states this

    “While Consumer Goods products are sometimes purchased from a mobile device, especially by men, most Consumer Goods products are purchased from retail locations, such as grocery and national retail chains. Very few of these goods are sold directly from the manufacturer.
    Therefore, a consumer’s in-store mobile behavior can also be indicative of mobile Consumer Goods behavior”

    Now whether overall in store mobile behaviour is indicative of in store behaviour whilst buying CPG is debatable and tenuous, but the survey is sound on total mobile in store behaviour if you look at the source material.

    Its important that we ensure veracity of stats.

    • Tim Dunn says:

      Thanks Ross – that’s all true. The quote that you cite correlating CPG behavior to general in-store is a staggering sleight-of-hand (no pun intended!). They are basically stating that as we behave in one way in electronics stores, we might as well assume that’s the same for CPG, which is nonsense. As I state in the main article, they must have started with the best of intentions, realized the numbers didn’t add up, and rolled in more data to look good.

      The logic they imply is about as robust as ‘York City once beat Man City, Man City are (were) champions, therefore York are the best team in England’

  7. Si Crowhurst says:

    Tim,

    Nice article.

    The real problem, as I see it, stems from the fact that decent market analysis data is deemed too expensive for businesses to invest in. Particularly where people still don’t necessarily see/understand the growing importance of the channel. This leaves a void and creates the opportunity for stat creation as you have well described.

  8. mark stone says:

    Superb article. I’m so disappointed that comparemycabbage.mobi is not working.
    As Mark Brill always said “97% of all statistics are made up”…

Leave a Reply