Measurement of Teacher's Orchestration Load : A Framework and a Case Study on Tool Flexibility

Abstract

Teacher orchestration in Computer-Supported Collaborative Learning (CSCL) environments demands managing multiple tasks across different social levels, often under tight constraints, leading to an increased orchestration load. This load represents the cognitive and physical effort teachers invest in real-time coordination of learning activities, which remains underexplored, particularly how the flexibility of orchestration tools influences this burden. In response to this gap, we propose a comprehensive framework for tracking and characterizing orchestration load, focusing on the intensity and dynamics of teachers' actions while implementing CSCL scripts. The framework integrates multimodal data sources, including observable orchestration actions, physiological metrics, and self-reported insights, to comprehensively analyze the orchestration load. We illustrate the applicability of the framework through a case study comparing two CSCL orchestration tools: PyramidApp, which offers pre-configured, structured enactment of the Pyramid collaborative learning flow pattern, and EthicApp, a more flexible tool allowing real-time design adjustments for a variety of instructional designs. Our findings reveal that PyramidApp facilitates a streamlined orchestration process with lower intensity and reduced teacher workload. At the same time, EthicApp's high flexibility increases orchestration intensity, requiring more cognitive effort from teachers during real-time phase configuration. These results underscore the importance of balancing flexibility with usability in orchestration tool design, as overly flexible environments may overwhelm teachers, especially during high cognitive load scenarios. This study contributes a methodological framework for evaluating orchestration tools and highlights key design considerations for reducing orchestration load in CSCL environments.