Big Data and Education

Reading through contributions to debates on big data and education (see [1] – [9] below) reminds us that there are two perspectives on education: education as science and education as wisdom of practice.

Education as science is about generalisation (if x what is likely to happen?), measurable outcomes (chiefly grades, if appropriate retention too) and observable processes of teaching and learning. Education as science seeks ‘objectivity’ and appeals in particular to policy makers and organisations who want to know what is worth funding and what works.

Education as wisdom of practice is about recognising complexity. From experience teachers know that no two classes are ever the same and without wisdom gained over time they will lack the strategies and techniques to get the best out of their learners. Wise teachers recognise uncertainty both in what is being measured and how you measure it. They ask how and why people learn, or fail to learn, and build up mental pictures of their learners. They know that teaching and learning is emotional. The appeal to wisdom of practice plays very well with practitioners who understand that ‘it is always complicated’.

Big data (and learner analytics) appeal instinctively to our first group, those who see education as a science. This on the grounds that:

• We can interrogate more data, more quickly, than ever before. For example in England the national pupil database (NPD) [1] provides a huge data file on pupil attainment, attendance and progression which can be broken down by sector, gender ethnicity, first language, free school meals and so on. It can help show what is happening and what is having an impact. To take one example it has helped us understand not just about ethnicity, but different dimensions to ethnicity, and it has recalibrated how we think about impact of multi culturalism in schools and identified an achievement gap among white working class boys.
• We can be serendipitous. Indeed other than its size the NPD is not big data, it is a snapshot survey with predictable data fields. Of more interest to big data enthusiasts are explorations of hitherto unlinked sets of data. For example projects in several universities aim to explore what data on library usage (not just visits, but what is accessed online) have on the attainment of students [2] [3]. It is not, of course, surprising if library usage and attainment are in some way related but we have never been able to explore the association in as much detail as now.
• We can get just in time data which can be fed back to learners, for example to ‘nudge’ students to visit the library [2] [4]. This kind of early feedback can be very important in distance learning where a host of online activity is automatically tracked. [4]
• We can use big data in order to generate powerful models which assist decision-making at policy, institution and teacher levels. Such modelling has enabled the construction of so called dashboards for learners allowing them to compare their activity against both the performance of the group and ideal profiles [2]. Of course this monitoring has come under scrutiny for overgeneralisation. More nuanced commentators have noted that big data analysis throws up association rather than proof of causality [6] It is also essential for researchers to consider the ethics of it all [3]. Indeed ethical issues extend beyond respecting confidentiality, and awareness of the data protection legislation, to wider questions such as whether we really want to live in a society in which data on our activity is automatically generated. [7]

The Big Data movement is not immediately attractive for our second group, those who believe in the wisdom of practice. In particular:

• Those using Big Data often ignore that teachers collect data all the time both formally and informally. For example teachers know which assignments seem to engage students and which do not, and they can adjust when they see pupils bored and unhappy. Above all, they have the back story about the students they teach which means that they can interpret actions sensitively and intelligently [8]. Practitioners bring an understanding of context and develop compelling metaphors for thinking and learning which inform and explain practice.
• Big data may be big but it does measure everything. Rather it is only measuring what can be measured not what it is appropriate to measure. One, perhaps unintended, consequence is that outcome measures are dominating thinking about education but this is swamping debates about educational values, for example what should be taught and how we should assess it.
• Those obsessing about Big Data simply miss the fact that teaching is a complex mix of different types of knowledge. Insights from data analysis often confirm what teachers have long thought and often articulated among themselves. They are left asking why no-one spoke to them in the first place.

Is there a way forward between teaching as science and teaching as art? Probably not, but:

• First we can understand that different stakeholders might be asking different questions about data and it is quite understandable that they do so. For example one statistical association that many institutions have found is that those attending a library induction will often achieve better outcomes than those miss the induction. This is useful for managers wanting to make decisions about resourcing libraries and for course leaders in planning their courses and promoting such induction. What of course it does not tell us is how to teach a class nor should we conclude that if you attend library induction you will get a better degree. Big data might not provide a story about learning but that is fine as long as we recognise the limitations.
• Second, we can use data as one more source of information about teaching and learning. Indeed as teachers we can misread the signs and cues and we can become highly focussed on what we do, on our performance, rather than what the learners do, both in and out of formal teaching sessions. In practice teacher performance may turn out to be less important than we think, it is the design for teaching that matters more and this is something for which Big Data can help us [9].
• Third, we can take the Big Data movement with a pinch of salt but can accept the wider case that we should make more use of available data to inform teaching. Indeed most practitioners do understand that so-called objective data about learner activity can help, especially when explored as part of practitioner research projects. What we need is, perhaps, a Small Data revolution, in which local data is used to support practitioner inquiry before we talk about a Big Data movement.

Update

This post was in anticipation of a one day event on Big Data at the University of Warwick.  At the event Ben Williamson presented a more disturbing view of learning analytics or at least subjected the claims made by private educational providers such as Pearson to critical scrutiny. He also noted the scale of the funding and ambition of commercial organisations to provide comprehensive profiling of learners, going far beyond the rather tentative examples of higher education. For more go to https://codeactsineducation.wordpress.com/2015/12/09/education-big-data-imaginary/

References

[1] For more on the National Pupil Database in England go to: https://www.gov.uk/guidance/national-pupil-database-apply-for-a-data-extract

[2] A JISC report gives a useful snapshot of learning analytics in higher education: in Sclater, N. (2014) The Current State of Play in UK Higher and Further Education published at:http://repository.jisc.ac.uk/5657/1/Learning_analytics_report.pdf

[3] A parallel report looks at legal and ethical issues: Sclater, N. (2014) Code of Practice for Learner Analytics, at http://analytics.jiscinvolve.org/wp/2014/12/04/jisc-releases-report-on-ethical-and-legal-challenges-of-learning-analytics/

[4] Opportunities for formative use of data are presented in Ferguson, R. and Buckingham Shum, S. (2012). Social learning analytics: five approaches. In: 2nd International Conference on Learning Analytics & Knowledge, 29 Apr – 02 May 2012, Vancouver, British Columbia, Canada, pp. 23–33.

[5] A feel for learning analytics in respect to online activity is given in, for example, Fidalgo-Blanco, A. et al (2015) Using Learning Analytics to improve teamwork assessment, Computers in Human Behavior, 47 (2015) 149–156 though this is not specific to DL.

[6] An argument for understanding complexity is put by Beer, C. et al (2012) Analytics and complexity: Learning and leading for the future, presented at ASCILITE2012 Future challenges, sustainable futures. http://www.ascilite.org/conferences/Wellington12/2012/images/custom/beer,colin_-_analytics_and.pdf

[7] At the time of writing there is some consternation among librarians in Japan that a newspaper breached confidentiality by publishing the names of books that novelist Haruki Murakami, 66, took out as a teenager from his school library, this all seems rather innocent: http://www.theguardian.com/books/2015/dec/02/librarians-in-uproar-after-borrowing-record-of-haruki-murakami-is-leaked

[8] This is argued robustly by Buckingham Shum, S.  Learning analytics: white rabbits and silver bullets, University of Technology, Sydney in Williamson, B. (ed.) 2015. Coding/Learning: Software and digital data in education. Stirling: University of Stirling at: https://codeactsineducation.files.wordpress.com/2015/08/coding_learning_-_software_and_digital_data_in_education.pdf

[9] For an exploration of learning as in this data rich world try Goodyear, P. (2015) Teaching as Design in (ed P. Kandlbinder) A Review of Higher Education Vol. 2. www.herdsa.org.au/publications/journals/ herdsa-review-higher-education-vol-2