[This entry supports a recent talk on the subject to research students] 

Over the last couple of years I became interested in theory and theorising in social research. I have done this not because I started with a great deal of confidence, or indeed background knowledge, of theory as a concept but, quite the reverse, an awareness of a gap in my understanding. However as I got into the topic I noticed that other colleagues struggle with the concept too and this gave me the push to continue looking into this area. [1]

A logical starting point in trying to understand theory was to get an idea of the core claims made for theory and for me this comes back to language; theoretical concepts provide the language we use to organise our thinking about something. This seems quite slight a contribution but it isn’t, for once you can name a thing you can think about and act on it. For example when you visit another country and don’t understand the language  you cannot express thoughts,  share ideas, or enter into a social world. Of course the frustrating thing here is that you have the concepts but cannot express them, but imagine not being able to give name to the concepts in the first place. Language is absolutely core to what we are as human beings and theorising is a language activity.

Of course when it comes to theorising we can get by without the kind of obsession that social researchers often bring to naming and debating theories but theses obsessions comes from a desire to push boundaries.

Theories can be bigged up  but this does not need to be the case.   For example in education research Vygotsky introduced the idea of a zone of proximal development (ZPD) – the difference between what a learner understands at present and what a learner can do with the support of a more knowledgeable other [2]. This concept helps organise the work of a teacher by focusing attention on gaps in the learners’ understanding; what is the learner ready to grasp; and how can the learner be best supported.  Setting it out in this way you might say isn’t a ZPD obvious as a concept? Well yes, but it only appears obvious as someone named it in the first place.

Theories come in different forms [3] and much of the problem many of us have with theory is that the same word covers different ideas. Theory could, for example, be about introducing a new concept, but for some it could also be about modelling relationships between variables. Theoretical contributions can also cover the study of key thinkers, ideas about  how we should see the world or how we should carry out research into understanding the world [4].

One of the problems, or at least challenges, that arises in talking about theory is that people do not understand that other people might use theory in a different way to themselves. For some researchers explanatory theory is about noticing and explaining cause and effect, if X then people tend to do Y. In contrast, other researchers might want to understand why people behave as they do and, rather than say what causes such behaviour, they are interested in explaining the  consequences –  if people act like X then Y is likely to be an outcome. Thus it not surprising that we all struggle with the word theory and in vivas the question ‘what is your theoretical contribution?’  is such an unsettling question as the student is having to second guess what the examiner means [5]. [Of course the examiner can help here by clarifying what he or she has in mind, but the student can also be proactive and set out what they see as a theoretical contribution too]. Theory can end up becoming a stick to beat up other researchers (you don’t understand X, you have misapplied Y, you have not read Z) which can be quite misplaced as few theories lend themselves to easy interpretation.

In spite of its different associations I think when it comes to theory there is a common idea that theory should explain and not be ‘merely’ descriptive. Just as in natural science, the models and formulae that some social scientists put forward can offer good representations of what is happening but they are not properly theories if they don’t say how it all fits together [6]. In putting forward  an explanation theorists should have a broader idea about the way that human being behave – as sociologists put it an engagement with agency and structure. For example, Bourdieu’ s idea that there are different kinds of capital (social, cultural and economic) which shape social activity would be a model, a useful model but an underdeveloped one,  without the accompanying ideas of habitus (how our life experiences develop within us habits, skills, and dispositions) and field (implicit and explicit rules of behavior, and valuations put on that behaviour in particular sites of activity) [7]. Likewise  Bronfenbrenner’s ecological systems approach (the idea that the individual exists within interlocking micro, macro, exo, meso systems, some of which are directly experiences, some are not)  would be much less without Bronfenbrenner’s view that the point of social development was to develop personal agency [8].

Another point about theories is that they are transferable. Theorists seek to abstract out what is important in a situation and so let others apply and of course reinterpret these theories in other contexts. Theories seem to integrate different findings and there is, in both natural and social science, a view that theories should often provide the least complicated explanation.  For example Einstein’s theory of relativity is so thought provoking as it is expressed so simply (E=MC^2) and more recently in economics Piketty’s (cited earlier) formula r>g  (where r is the rate of return on capital and g economic growth)  captures something about wealth and inequality so elegantly  that Piketty has an audience beyond the field of economics. Maybe we exaggerate the value of simplicity but at a pragmatic level theories which try to tell too much are difficult to apply.

If theory is complicated then so too is theorising. According to Swedberg, theorising is what you do when you make a theory.  For Swedberg theorising is speculating and theorising calls for a mix of logic, intuition and  creativity – a different way of thinking [9].  Swedberg draws on the pragmatic thinker Peirce who introduced the term abduction to capture the kind of intuitive guesswork we use when reaching for an explanation. The metaphor of the detective is irresistible here and indeed Peirce’s account of guessing draws on his experience of tracking down a watch – a very expensive watch that had been loaned to him. In his account Peirce seems to have a sixth sense as to who stole the watch and relies more on a hunch to identify the thief rather than on logic or deduction. [10]

Conan Doyle’s detective Sherlock Holmes provides another metaphor for guessing, that of detective work. Famously, on meeting Watson for the first time, Holmes says ‘ah you have come from Afghanistan’. Watson wants to know how Holmes had guessed this. Holmes explains:

Here is a gentleman of a medical type, but with the air of a military man. Clearly an army doctor, then. He has just come from the tropics, for his face is dark, and that is not the natural tint of his skin, for his wrists are fair. He has undergone hardship and sickness, as his haggard face says clearly. His left arm has been injured. He holds it in a stiff and unnatural manner. Where in the tropics could an English army doctor have seen much hardship and got his arm wounded? Clearly in Afghanistan.  [11]

Here there is a leap of imagination that brings various observations together around a broader idea. The problem  is that the account is fictional and indeed the Holmes stories are popular as they are comforting (truth will come out) when  in reality  many different hunches or guesses are possible based on the same data.  Indeed there were many other ways to account for the appearance of  Watson than he had just come from Afghanistan, but that would spoil the story. An additional source of confusion in Holmes is that his methods are described as deduction, but they read more like abduction in Peirce’ sense [12].

A further metaphor for theorising is that of the investigative journalism. For example, Stacey Dooley as a documentary film maker models the reflection needed for theorising very well. She also asks the  difficult questions which a social researcher  needs to do  without antagonising her interviewees. Once the interviews are over she steps back and questions the significance of what she has heard in a way that the qualitative researcher might understand. She often finds no easy solutions or conclusions [13].

Detective work and investigative journalism only take you so far as metaphors, social researchers may make hunches, be obsessive about their inquiries, and go round in circles but a crucial difference is that social researchers have to engage in a very self-conscious way with what has been said about a topic or an idea in the past. In discussing their methods researchers such as Wright Mills [14] and Eco [15] show how much they value past literature and take detailed notes on all that they have read. They see theorising as a kind of shuttling between the data they have collected and the theories they have explored.

The idea of comparing data to theory is helpful to describe theorising in academic contexts and Michael Polanyi sheds further light. Polanyi believed that in coming to know something we need  to switch between focal (the goal of the activity) and subsidiary awareness (the particular features of an activity).  (Focal awareness might be the message we want to convey, and subsidiary awareness the grammatical structures we use to construct the message).  Polanyi suggested that we can only really  engage with the focal once we have worked really hard on the particulars.  The paradox is that we need to background the subsidiary in order to see the focal but this can be done only by putting a great deal of effort into mastering what is subsidiary in the first place.  Once we do seem able to gain a focal attention it is very difficult to look back at the subsidiary. This is the idea of tacit knowledge, things we know but find difficult to say – for example the experienced driver who cannot tell you what he or she does when driving a car, or the native language speaker who uses but cannot explain the grammatical forms they are using when speaking. [16]

Polanyi is useful as he shows that theorising is a huge challenge as it requires a step change in the way we think. However, this step change cannot be forced as we need to know in detail what we are theorising about. Perhaps because theorising is such a challenge, and to a degree a process of forgetting, researchers often describe theoretical breakthroughs as a revelation –  they get very excited when it all falls into place. As a example a recent review of Silent Spring  [17] cites Rachel Carson reflecting on her ‘aha moment’ – this is the moment when she can see the disparate data she has on environmental destruction fits into a pattern – the over use of pesticides such as DDT. Carson, like others, uses the metaphor of the jigsaw puzzle:

Recently some of my thinking on all this has begun to fit together like the pieces of a jigsaw puzzle … a great light is breaking in my mind’

and she adds

I have a comforting feeling that  what I shall now be able to achieve  is a synthesis of widely scattered facts, that have not heretofore been considered in relation to each other. It is now possible to build up…a really  damning case against the use of these  chemicals as they are now inflicted upon us.’ [18]

I don’t think it is possible to legislate for the kind of thinking that theorising captures but I can make some suggestions. If trying to theorise it is surely a good idea to step back physically from the data, literally go outside and just think what is the central idea in all you have been looking at. Try to notice what lies behind what you have found out.  Try talking to other people about your ideas, they wont listen to you, but rehearse what is running through your head. Be kind on yourself, don’t assume what you have found is unimportant or not worth knowing, it usually is. Compare your central idea to the literature but be critical of the literature. Try to notice the gaps or things that are not being said. I hear a lot from students who say they have not really understood something they have read,   they don’t quite get it. But it could be that they don’t get it because there is something  in the literature that is missing, not ‘getting it’ might  be a really important step to some new insight.  If you want to be more organised in theorising try looking too at the same data through different perspectives, a kind of theoretical triangulation. Few people do this but it can be illuminating [19] .

Finally whatever you do, be kind to yourself, this is hard.  Try to be gently assertive, put forward your ideas and be open about how you came to reach your conclusions. Argue persuasively but with kindness. Accept there are other ways to look at the data. It is not self-indulgence to theorise if you are a social researcher, it is the job.

[1] I have written about this in Hammond, M. (2018). ‘An interesting paper but not sufficiently  theoretical’: What does theorising in social  research look like?Methodological Innovations May- August 2018: 1-10.

[2] See for example Van der Veer, R. and J. Valsiner (1994). The Vygotsky Reader. Oxford, England, Blackwell.

[3] See for example Abend, G. (2008). The meaning of ‘theory’. Sociological Theory,26(2): 173-199 and Krause, M. (2016). The meanings of theorizing. The British Journal of Sociology,67(1): 23-29.

[4] Most articles will fit into these categories, in the talk I gave we looked at Davis, F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340 as an example of modelling;  Fuchs, C. (2015). Reading Marx in the Information Age: A media and communication studies perspective on capital. London: Routledge as an example of interrogating key thinkers; Piketty, T. (2015). About capital in the twenty-first century. American Economic Review, 105(5), 48-53 as an example of drawing attention to a relationship between variables; Rawls, J. (2009). A Theory Of Justice.MA: Harvard University Press as an example of speculating on what should happen; Reay, D. (2004). Gendering Bourdieu’s concepts of capitals? Emotional capital, women and social class. The Sociological Review, 52(2_suppl), 57-74 as an example of developing a new concept.

[5] See for example the aptly titled Kiley, M. (2015). I didn’t have a clue what they were talking about’: PhD candidates and theory. Innovations in Education and Teaching International, 52(1): 52-63.

[6] Here we looked as an example at Banerjee, A. V., Cole, S., Duflo, E., & Linden, L. (2007). Remedying education: Evidence from two randomized experiments in India. The Quarterly Journal of Economics, 122(3), 1235-1264. This was a study about using support teachers to support children falling behind in schools in India. It was well written and drew out data on achievement. Unlike some experimental studies it did offer an explanation for the success of an intervention but this was underdeveloped.

[7] There is a mass of work on Bourdieu but on forms of capital see for example Bourdieu, P. (1986). The forms of capital in J. Richardson (ed) Handbook of Theory and Research for the Sociology of Education. NY, USA: Greenwood  241-258 as well as examples of applying the theory creatively in for example Reay earlier and Williams, S. (1995). Theorising class, health and lifestyles: can Bourdieu help us?,  Sociology of Health & Illness 17(5): 577-604.

[8] Bronfenbrenner, U. (1979). The Ecology of Human Development, MA, USA: Harvard  University Press.

[9] Swedberg is insightful throughout see for example:

Swedberg, R. (2015). “Before Theory Comes Theorizing or How to Make Social Science More Interesting, British Journal of Sociology 2015 Annual Public Lecture.” [online] https://www.youtube.com/watch?v=34DME7lCe1I

Swedberg, R. (2016). Before theory comes theorizing or how to make social science more interesting.The British Journal of Sociology 67(1): 5-22

Swedberg, R. (2016). Can you visualize theory? On the use of visual thinking in theory pictures, theorizing diagrams, and visual sketches. Sociological Theory34(3): 250-275.

[10] Peirce’s ideas of guessing can be found in his collected works ( 1907, ca., Guessing. MS 687) and the account of the watch is in a republished article Peirce, C. (1929) Guessing, The Hound and Horn, 2, 267-282

[11] Conan Doyle (1887) see also discussion in Sebok (1981) and Walton (2015).

Conan Doyle, A. (1887). A Study in Scarlet, Project Gutenberg [online]. https://www.gutenberg.org/files/244/244-h/244-h.htm.

Sebok, T., & Umiker-Sebok, J. (1981). You Know My Method; A juxtaposition of Charles S. Peirce and Sherlock Holmes. Indiana, USA: Gaslight Publications and more analytically Walton, D. (2015). Argument Evaluation and Evidence. Cham, CH: Springer.

[12] The fashion now is to show the detective can overlook what is important, and does not get it right without a great deal of backtracking, as in Wallender, Spiral, The Bridge

Wallender:  https://en.wikipedia.org/wiki/Kurt_Wallander

Spiral:  https://en.wikipedia.org/wiki/Spiral_(TV_series)

The Bridge: https://en.wikipedia.org/wiki/The_Bridge_(2011_TV_series)

[13] For example The Billion Pound Party – Stacey Dooley Investigates The DUP [online] https://www.youtube.com/watch?v=DBNY8rhng6E

[14] Mills, C. W. (1959). The Sociological Imagination. Harmondsworth: Penguin.

[15] Eco, U. (2015). How to Write a Thesis. MA: MIT Press.

[16] Polanyi, M. (1958). Personal knowledge: Towards a Post-Critical Philosophy. Chicago, USA: University of Chicago Press.

[17] Crist, M. (2019). “Rachel Carson’s Forebodings.” London Review of Books 41(11).

[18] Carson, R. (1962). Silent Spring. MA, USA, Houghton Miff.

[19]   Jackson, A. Y. and L. Mazzei (2011). Thinking with Theory in Qualitative Research: Viewing data across multiple perspectives, London: Routledge is unusual in looking at the same data from different standpoints (including Derrida; Foucault; Butler; and Delueze). Mol (2002) The Body Multiple, Duke University Press is an interesting example as the authors separate out the empirical and the theoretical in a study that looks at the day-to-day diagnosis and treatment of atherosclerosis in a Dutch hospital.