Big Data and Education

Reading through contributions to debates on big data and education (see [1] – [9] below) reminds us that there are two perspectives on education: education as science and education as wisdom of practice.

Education as science is about generalisation (if x what is likely to happen?), measurable outcomes (chiefly grades, if appropriate retention too) and observable processes of teaching and learning. Education as science seeks ‘objectivity’ and appeals in particular to policy makers and organisations who want to know what is worth funding and what works.

Education as wisdom of practice is about recognising complexity. From experience teachers know that no two classes are ever the same and without wisdom gained over time they will lack the strategies and techniques to get the best out of their learners. Wise teachers recognise uncertainty both in what is being measured and how you measure it. They ask how and why people learn, or fail to learn, and build up mental pictures of their learners. They know that teaching and learning is emotional. The appeal to wisdom of practice plays very well with practitioners who understand that ‘it is always complicated’.

Big data (and learner analytics) appeal instinctively to our first group, those who see education as a science. This on the grounds that:

  • We can interrogate more data, more quickly, than ever before. For example in England the national pupil database (NPD) [1] provides a huge data file on pupil attainment, attendance and progression which can be broken down by sector, gender ethnicity, first language, free school meals and so on. It can help show what is happening and what is having an impact. To take one example it has helped us understand not just about ethnicity, but different dimensions to ethnicity, and it has recalibrated how we think about impact of multi culturalism in schools and identified an achievement gap among white working class boys.
  • We can be serendipitous. Indeed other than its size the NPD is not big data, it is a snapshot survey with predictable data fields. Of more interest to big data enthusiasts are explorations of hitherto unlinked sets of data. For example projects in several universities aim to explore what data on library usage (not just visits, but what is accessed online) have on the attainment of students [2] [3]. It is not, of course, surprising if library usage and attainment are in some way related but we have never been able to explore the association in as much detail as now.
  • We can get just in time data which can be fed back to learners, for example to ‘nudge’ students to visit the library [2] [4]. This kind of early feedback can be very important in distance learning where a host of online activity is automatically tracked. [4]
  • We can use big data in order to generate powerful models which assist decision-making at policy, institution and teacher levels. Such modelling has enabled the construction of so called dashboards for learners allowing them to compare their activity against both the performance of the group and ideal profiles [2]. Of course this monitoring has come under scrutiny for overgeneralisation. More nuanced commentators have noted that big data analysis throws up association rather than proof of causality [6] It is also essential for researchers to consider the ethics of it all [3]. Indeed ethical issues extend beyond respecting confidentiality, and awareness of the data protection legislation, to wider questions such as whether we really want to live in a society in which data on our activity is automatically generated. [7]

The Big Data movement is not immediately attractive for our second group, those who believe in the wisdom of practice. In particular:

  • Those using Big Data often ignore that teachers collect data all the time both formally and informally. For example teachers know which assignments seem to engage students and which do not, and they can adjust when they see pupils bored and unhappy. Above all, they have the back story about the students they teach which means that they can interpret actions sensitively and intelligently [8]. Practitioners bring an understanding of context and develop compelling metaphors for thinking and learning which inform and explain practice.
  • Big data may be big but it does measure everything. Rather it is only measuring what can be measured not what it is appropriate to measure. One, perhaps unintended, consequence is that outcome measures are dominating thinking about education but this is swamping debates about educational values, for example what should be taught and how we should assess it.
  • Those obsessing about Big Data simply miss the fact that teaching is a complex mix of different types of knowledge. Insights from data analysis often confirm what teachers have long thought and often articulated among themselves. They are left asking why no-one spoke to them in the first place.

Is there a way forward between teaching as science and teaching as art? Probably not, but:

  • First we can understand that different stakeholders might be asking different questions about data and it is quite understandable that they do so. For example one statistical association that many institutions have found is that those attending a library induction will often achieve better outcomes than those miss the induction. This is useful for managers wanting to make decisions about resourcing libraries and for course leaders in planning their courses and promoting such induction. What of course it does not tell us is how to teach a class nor should we conclude that if you attend library induction you will get a better degree. Big data might not provide a story about learning but that is fine as long as we recognise the limitations.
  • Second, we can use data as one more source of information about teaching and learning. Indeed as teachers we can misread the signs and cues and we can become highly focussed on what we do, on our performance, rather than what the learners do, both in and out of formal teaching sessions. In practice teacher performance may turn out to be less important than we think, it is the design for teaching that matters more and this is something for which Big Data can help us [9].
  • Third, we can take the Big Data movement with a pinch of salt but can accept the wider case that we should make more use of available data to inform teaching. Indeed most practitioners do understand that so-called objective data about learner activity can help, especially when explored as part of practitioner research projects. What we need is, perhaps, a Small Data revolution, in which local data is used to support practitioner inquiry before we talk about a Big Data movement.

Update

This post was in anticipation of a one day event on Big Data at the University of Warwick.  At the event Ben Williamson presented a more disturbing view of learning analytics or at least subjected the claims made by private educational providers such as Pearson to critical scrutiny. He also noted the scale of the funding and ambition of commercial organisations to provide comprehensive profiling of learners, going far beyond the rather tentative examples of higher education. For more go to https://codeactsineducation.wordpress.com/2015/12/09/education-big-data-imaginary/

References

[1] For more on the National Pupil Database in England go to: https://www.gov.uk/guidance/national-pupil-database-apply-for-a-data-extract

[2] A JISC report gives a useful snapshot of learning analytics in higher education: in Sclater, N. (2014) The Current State of Play in UK Higher and Further Education published at:http://repository.jisc.ac.uk/5657/1/Learning_analytics_report.pdf

[3] A parallel report looks at legal and ethical issues: Sclater, N. (2014) Code of Practice for Learner Analytics, at http://analytics.jiscinvolve.org/wp/2014/12/04/jisc-releases-report-on-ethical-and-legal-challenges-of-learning-analytics/

[4] Opportunities for formative use of data are presented in Ferguson, R. and Buckingham Shum, S. (2012). Social learning analytics: five approaches. In: 2nd International Conference on Learning Analytics & Knowledge, 29 Apr – 02 May 2012, Vancouver, British Columbia, Canada, pp. 23–33.

[5] A feel for learning analytics in respect to online activity is given in, for example, Fidalgo-Blanco, A. et al (2015) Using Learning Analytics to improve teamwork assessment, Computers in Human Behavior, 47 (2015) 149–156 though this is not specific to DL.

[6] An argument for understanding complexity is put by Beer, C. et al (2012) Analytics and complexity: Learning and leading for the future, presented at ASCILITE2012 Future challenges, sustainable futures. http://www.ascilite.org/conferences/Wellington12/2012/images/custom/beer,colin_-_analytics_and.pdf

[7] At the time of writing there is some consternation among librarians in Japan that a newspaper breached confidentiality by publishing the names of books that novelist Haruki Murakami, 66, took out as a teenager from his school library, this all seems rather innocent: http://www.theguardian.com/books/2015/dec/02/librarians-in-uproar-after-borrowing-record-of-haruki-murakami-is-leaked

[8] This is argued robustly by Buckingham Shum, S.  Learning analytics: white rabbits and silver bullets, University of Technology, Sydney in Williamson, B. (ed.) 2015. Coding/Learning: Software and digital data in education. Stirling: University of Stirling at: https://codeactsineducation.files.wordpress.com/2015/08/coding_learning_-_software_and_digital_data_in_education.pdf

[9] For an exploration of learning as in this data rich world try Goodyear, P. (2015) Teaching as Design in (ed P. Kandlbinder) A Review of Higher Education Vol. 2. www.herdsa.org.au/publications/journals/ herdsa-review-higher-education-vol-2

2 thoughts on “Big Data and Education

  1. Really useful analysis.

    A few more points to consider:

    From the wisdom of practice perspective (which is aligned nicely with Sennett’s recent work on craft) good teachers are often quite tacitly “testing the waters” and “feeling the grain”, seeing how far they can go in a direction. It’s more like playing tennis than watching numbers change on a share trading screen (which might be a more familiar thing to some of big data’s political supporters).

    The danger is that the focus on data pulls us away from the craft. We expect too much from the data and lose our feel for the game. Worse still, new players might think that they can achieve mastery through manipulating the data. Notice how I slipped into a sports metaphor? It is very close. And we know what can happen to cricketers, for example, when they spend too much time analysing their game (England might now be recovering from that over-scientific phase).

    A couple of weeks ago a young PhD student called Federico Botta did a really insightful talk on how he uses data in his teaching. He is a statistics and complexity expert, but teaching in the business school. He has got the balance worked out perfectly, but starts with the craft of teaching and then uses data carefully to inform the craft. So if that’s what the real statistics experts are doing it would make sense to learn from them (rather than politicians who think they understand data). The video of Federico’s talk should be online soon (). Here is his web site http://www.fedebotta.com

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s