Monday 7 October 2013

Bonus Question
Does missing vowels matter?

Delicious vowels
A charge often levelled by (some) Only Connect viewers is that the final round - in which quickfire word puzzles are solved on the buzzer - carries too much weight. All too often, it is said, a team works hard to establish a lead over the first three-quarters of the contest only to have their surely deserved victory snatched away by a team who are slightly quicker on the draw. My team's win (I make up one third of the Board Gamers along with Hywel Carver and Jamie Karran) over the Globetrotters last week seemed to stir the hornet's nest again, as our 19-16 deficit after the connecting walls turned into a 26-21 victory. (Although it probably didn't help that our somewhat unorthodox wall strategy made us look like dribbling idiots.)

Setting aside the rather more philosophical question of why one round of an established quiz show should carry less weight for a moment, I thought it would be interesting to see if I could find out just how powerful (or not) missing vowels really is. To answer, I did what any statistician would do, and looked for the data. Through a combination of question editor David Bodycombe's own records (thanks David!), episode summaries at Mastermind champion Dave Clark's blog (thanks Dave!), and my own trawling through old episodes on YouTube (thanks Dav- I mean thanks me!), I put together what I'm pretty sure is the only comprehensive dataset of regular series Only Connect scoring in the universe. With it I'm able to look at all sorts of fun Only Connect-related questions (some of which will follow in later posts) including whether or not the missing vowels round is particularly important.

First things first, what about all these come-from-behind victories I've heard so much about? It turns out that they're not that common after all. In 99 episodes over the first seven series there have been just 14 occasions where a team losing after the walls have gone on to win - and two of those were on a tie-break. What's more, these turnarounds are seldom particularly dramatic. Of the 14, five were carried out by teams who were losing by just one point after the walls, while four were by a team trailing by two. The biggest deficit that's ever been overturned in missing vowels is four points; you can score more than that in one question in the earlier rounds.

Continuing our journey into the unremarkable let's have a look at how many points are scored on average in each of the four rounds. This should give us a very rough impression of how each round contributes to a team's performance. Check out the fancy table below.

Average (combined) points scored per
Only Connect episode, series 1-7.
It may surprise some that it's the connecting wall that comes out on top, with a whopping 13.5 points scored per episode between the two teams. Missing vowels is second with 11.3 points per show, while sequences and connections are a bit further back with 8 and 7.4 points respectively. It's important to note that these figures aren't directly comparable as each round has slightly different scoring mechanisms - a team can't score more than ten in the wall round, for instance - but the numbers still provide some food for thought. There are also some curious trends across the series (the data for which can be viewed here) - teams seem to be getting gradually better at the first round (or perhaps the questions are getting easier) - but those are left to the interested reader. Overall though, it's fairly remarkable how well-balanced the rounds are, particularly in more recent series.

It seems, then, that while missing vowels is an important part of the show it's nowhere near as make-or-break as the Internet (or at least the vocal parts of it) would have you believe. On average you'll see a turnaround once in seven shows, or roughly twice a series - the rest of the time you may as well not bother with the final round at all; the outcome would be the same. Moreover, when it does happen, it'll more than likely be a lead of only one or two points that's overturned, which for a relatively large part of a show seems pretty reasonable to me.

So why has this myth persisted? The most likely reason is that while the first three rounds take a relatively long time, missing vowels seldom lasts more than two minutes. Points are therefore scored at a much quicker rate than earlier in the show giving the impression that the round as a whole is cheap. The walls, meanwhile, have their points 'disguised'; they're only added for both teams at the very end of the round.

One final observation is that there seems little reason why a team leading before missing vowels might be expected to go on to lose. After all, a team who are better over the first three rounds are, one might expect, likely to be better in the last one as well. Pleasingly, this hypothesis is borne out by the data: there's a (statistically) significant association between a team's performance in missing vowels and their lead before it.

In conclusion:

10 comments:

  1. Good work. And thank you for doing your table in LaTeX.

    I feel like this article is missing an important reason in favour of missing vowels: it's the only 'fair' round in the show. As in, the only one where the teams are competing on identical questions in the same way.

    In Only Connect terms, some questions are absolute sitters, and some are nasty lateral thinking ones that you only have a hope on when you can see all 4. There are some questions which most teams would get 5 points on*, and those are distributed completely at random.

    *An example: what do you think comes fourth in a sequence that begins Mount Lhotse?

    ReplyDelete
    Replies
    1. Yeah agreed. I wasn't really trying to write 'in support of missing vowels', more to just present the numbers that contradict the widely held view that they're too powerful. Last night's show was a pretty great example of the volatility of the earlier rounds, although I shan't spoilers it here. I don't doubt it's night on impossible to strike a balance, though.

      As for Mount Lhotse, I've no idea, but I'd probably guess Everest?

      Delete
    2. I'm biting my tongue quite hard but I'll say just this - the Mount Lhotse question went to a team of travel writers, and they took all three clues.

      Delete
    3. I can well imagine being nervous and biding your time, but I think if we had that question (and I had no idea it was a past q, I just looked up the 4th highest mountain on Wikipedia), we would all have been thinking Everest and probably gone for it.

      In fairness, I think the sequence beginning 'Adenine' (which we had) is similarly easy for anyone into pub quizzes / with biology AS-level, and we still saw the second clue before buzzing.

      Delete
    4. Hywel you have the worst memory. You are fired from the team.

      Delete
    5. Amazing how statistics can be manipulated, isn't it...
      The wall round high average shows that both teams do well on it. It's very very rare (never happened?) for a team to score 10 and another to score 0. The fact that the average is over 10 would back this up.
      The vowels round is totally different. 11-0 and 6-5 provide the same statistic,

      Delete
    6. That's a fair point, although it is tied up with the fact that, as Hywel notes, missing vowels is the only truly 'fair' round in terms of teams directly competing on the same questions. It's also rare to get particularly big swings in missing vowels, especially in recent series where the round is so short you usually only see around 12 clues.

      In any case, the purpose of this post was not to say that the walls are super powerful, but that there's not much evidence that missing vowels are particularly dominant. Having spent some time with these data, I think it would take a more creative statistician than me to show otherwise.

      Delete
    7. Oh, and as an addendum/correction (just seen your posts on twoplustwo) - my own team only managed one missing vowels turnaround in our run (in our first game where we were down by three) :)

      Delete
  2. Congratulations on your win - there's some further interesting discussion on this topic in the comments here:

    http://lifeaftermastermind.blogspot.co.uk/2010/10/only-connect-first-round-review.html

    ReplyDelete
    Replies
    1. Hi Aaron, thanks - closer than we'd have liked but 'a win's a win', as they (axiomatically) say...

      I'm an avid reader of that particular blog, but had forgotten about that particular post, so was nice to be reminded - thanks. It partly goes to show how important it is to get as many data as possible; that series had some particularly crazy missing vowels rounds (such as the Epicureans one highlighted) and as a whole that series stands out with an unusually high missing vowels total (an average of 14.6 game compared to 12.9 in second).

      That comment thread raises some other interesting questions though, which I plan to look at over the coming weeks. Should hopefully throw up some other nice statistical tidbits.

      Delete