Not knowing what we don’t know

Every now and then the Guardian newspaper publishes an extract from the Secret Footballer offering some anecdotes and insight, from someone who played at the highest level, on what makes professional footballers ‘tick’ and what the fans get out of the game.

In talking about the fans the Secret Footballer commented recently that it is not so much that many fans know very little about football, the point is that they do not know how little they know:

‘Some people are so stupid that they have no idea how stupid they are. Professor of psychology at Cornell University, David Dunning, argues that in order to know how good you are at something, it requires exactly the same skills as it does to be good at that thing in the first place – which means if you’re absolutely no good at something at all then you lack exactly the skills you need to know that you’re absolutely no good at it. Understand?’

This sounds harsh but a few minutes listening to a football phone-in or reading a fans forum will show, that people are not so much saying ‘stupid things’ but that they talk with absolute certainty about tactics or team choices for a game they may not even have played, and certainly not played or managed at the highest level. This is odd even ‘stupid’ as there is absolutely no way of knowing whether, say, the two substitutions they wanted the manger to make would really have made any difference to the outcome of the game at all. This is not a put down of phone-ins, if you are in the mood they are good entertainment, but I do understand that you might get miffed as a professional listening to people talk dogmatically of things they do not know, a bit like teachers having to listen to politicians talk about education, I suppose.

So what does Professor David Dunning, cited by the secret footballer, say? His web site tells us that his work focuses on:

‘two related phenomena. First I am interested in why people tend to have overly favorable and objectively indefensible views of their own abilities, talents, and moral character. For example, a full 94% of college professors state that they do “above average” work, although it is statistically impossible for virtually everybody to be above average. Second, I am interested in how people bolster their sense of self-worth by carefully tailoring the judgments they make of others. That is, people tend to make judgments of others that reflect favorably back on themselves, doing so even when the self is not under explicit scrutiny.’

In one of his papers* Dunnings concluded that:

‘Taken together, these findings reaffirm the notion that poor performers show little insight into the depth of their deficiencies relative to their peers. They tend to think they are doing just fine relative to their peers when, in fact, they are at the bottom of the performance distribution. Yet, in each of these familiar circumstances, poor performing participants did not seem to know how poorly they were doing.
[] Part of why the dramatic overestimation demonstrated by poor performers is so fascinating is precisely because they show dramatic overconfidence on tasks about which they have likely received substantial feedback in the past.’

What I like about the paper is that rather than doing ‘pretend research’, ie how people behave in the artificial environment of the lab, the team explored the concept of self-evaluation of competence in ‘ecological valid’ or real life contexts. In particular they asked undergraduate students to estimate how well they had performed on course exams and asked members of college debate teams to evaluate their tournament performance. A criticism here is that, like much similar work, this is skewing research in psychology to university contexts where it is much easier to get access to willing participants. In this case they are picking up on more successful learners and not the crushing lack of self esteem of some who fail completely in the education system – see my previous blog.

It was however good that the paper offered some positive ideas about how to deal with this problem of over-optimistic self-evaluation – we need (not their phrase) to learn how to learn.

But it did leave me with some further thoughts about assessment of performance and, in the spirit of offering opinions on something I know not a lot about, it seems to me that part of this phenomenon of over optimism is that if you really did know what was involved in ‘complex performance’, would you would start to try and learn it in the first place? In many cases naïve optimism sounds like a good idea. The pity of it is that with greater understanding you get better at it but you realise that it could all go wrong and that your performance falls short of the ideal – one reason to explain the phenomenom of sports people (including I think the Secret Footballer) and entertainers falling out of love with what they once dreamed of doing as their performance becomes assessed and taken apart.

A second thought I offer is more about our reference point in making a self evaluation. It is all very well to ask people to assess against a norm or a supposed average but in reality we are always going to look at our nearest salient points of reference. Thus, for example, in thinking about how well off we are, we will blank out the huge salaries and bonuses that some are earning but feel genuine rage if someone we worth gets an extra bonus or earns more than us when they ‘don’t deserve it’. Self-assessment comes up too in the context of health and weight studies, for even if the data show we are getting chunkier and chunkier our relative assessment of being overweight changes to adjust to the sizes of those around us, in our eyes we are not overweight at all.

It could be then that distorted self-evaluation serves a purpose or is simply our default. Perhaps this is why lack of self esteem is so disempowering. It is also, to be honest, a lot more fun expressing strong opinions on things you know relatively little about than offer measured opinions on something you know is complex and uncertain. So perhaps we should be relaxed and try to find out when distorted self evaluation really matters and when it does, yes, we need to understand that we may well be long way off from where we think we are and need to start engaging in a rather long and difficult process of analysis and feedback. If it doesn’t really matter, hold on to your naïve optimism, try the phone-in but do expect to be found out at some point.

* Ehrlinger, J., Johnson, K., Banner, M., Dunning, D., & Kruger, J. (2008). Why the unskilled are unaware? Further explorations of (lack of) self-insight among the incompetent. Organizational Behavior and Human Decision Processes, 105, 98-121.

1 thought on “Not knowing what we don’t know

  1. Can’t help but think of Socrates’ ἓν οἶδα ὅτι οὐδὲν οἶδα.. I only know what thing: I know nothing.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s