OPUS Collection:

Using causal models to bridge the divide between big data and educational theory

2023-09-01T00:00:00Z

Title: Using causal models to bridge the divide between big data and educational theory Authors: Kitto, K; Hicks, B; Buckingham Shum, S Abstract: An extraordinary amount of data is becoming available in educational settings, collected from a wide range of Educational Technology tools and services. This creates opportunities for using methods from Artificial Intelligence and Learning Analytics (LA) to improve learning and the environments in which it occurs. And yet, analytics results produced using these methods often fail to link to theoretical concepts from the learning sciences, making them difficult for educators to trust, interpret and act upon. At the same time, many of our educational theories are difficult to formalise into testable models that link to educational data. New methodologies are required to formalise the bridge between big data and educational theory. This paper demonstrates how causal modelling can help to close this gap. It introduces the apparatus of causal modelling, and shows how it can be applied to well-known problems in LA to yield new insights. We conclude with a consideration of what causal modelling adds to the theory-versus-data debate in education, and extend an invitation to other investigators to join this exciting programme of research. Practitioner notes What is already known about this topic ‘Correlation does not equal causation’ is a familiar claim in many fields of research but increasingly we see the need for a causal understanding of our educational systems. Big data bring many opportunities for analysis in education, but also a risk that results will fail to replicate in new contexts. Causal inference is a well-developed approach for extracting causal relationships from data, but is yet to become widely used in the learning sciences. What this paper adds An overview of causal modelling to support educational data scientists interested in adopting this promising approach. A demonstration of how constructing causal models forces us to more explicitly specify the claims of educational theories. An understanding of how we can link educational datasets to theoretical constructs represented as causal models so formulating empirical tests of the educational theories that they represent. Implications for practice and/or policy Causal models can help us to explicitly specify educational theories in a testable format. It is sometimes possible to make causal inferences from educational data if we understand our system well enough to construct a sufficiently explicit theoretical model. Learning Analysts should work to specify more causal models and test their predictions, as this would advance our theoretical understanding of many educational systems.

Towards more replicable content analysis for learning analytics

2023-03-13T00:00:00Z

Title: Towards more replicable content analysis for learning analytics Authors: Kitto, K; Manly, CA; Ferguson, R; Poquet, O Abstract: Content analysis (CA) is a method frequently used in the learning sciences and so increasingly applied in learning analytics (LA). Despite this ubiquity, CA is a subtle method, with many complexities and decision points affecting the outcomes it generates. Although appearing to be a neutral quantitative approach, coding CA constructs requires an attention to decision making and context that aligns it with a more subjective, qualitative interpretation of data. Despite these challenges, we increasingly see the labels in CA-derived datasets used as training sets for machine learning (ML) methods in LA. However, the scarcity of widely shareable datasets means research groups usually work independently to generate labelled data, with few attempts made to compare practice and results across groups. A risk is emerging that different groups are coding constructs in different ways, leading to results that will not prove replicable. We report on two replication studies using a previously reported construct. A failure to achieve high inter-rater reliability suggests that coding of this scheme is not currently replicable across different research groups. We point to potential dangers in this result for those who would use ML to automate the detection of various educationally relevant constructs in LA.

Aligning the Goals of Learning Analytics with its Research Scholarship: An Open Peer Commentary Approach

2023-01-01T00:00:00Z

Title: Aligning the Goals of Learning Analytics with its Research Scholarship: An Open Peer Commentary Approach Authors: Ferguson, R; Khosravi, H; Kovanović, V; Viberg, O; Aggarwal, A; Brinkhuis, M; Buckingham Shum, S; Chen, LK; Drachsler, H; Guerrero, VA; Hanses, M; Hayward, C; Hicks, B; Jivet, I; Kitto, K; Kizilcec, R; Lodge, JM; Manly, CA; Matz, RL; Meaney, MJ; Ochoa, X; Schuetze, BA; Spruit, M; Van Haastrecht, M; Van Leeuwen, A; Van Rijn, L; Tsai, Y-S; Weidlich, J; Williamson, K; Yan, VX Abstract: NA

Pendulum waves: A model of Thomae's function

2023-12-01T00:00:00Z

Title: Pendulum waves: A model of Thomae's function Authors: Feng, X; Lu, C; Schulte, J; Shan, Z; Liu, G Abstract: The pendulum wave apparatus exhibits cyclic pendulum patterns, including wave-like motion and pendulums alignments. This work presents a complete analytical solution to the times and numbers of pendulum alignment, which is shown to be a subset of Thomae's function. Based on the properties of this function and basic number theory, a comprehensive analysis of pendulum patterns is presented.