New Lenses on Learning Design: Course Types and Indicators from Large-Scale Moodle Data

AI summary

Overview

Research area: Learning analytics and learning design (LD) in technology-enhanced learning; the paper is listed under arXiv category cs.CY and studies course structures extracted from institutional Moodle data.

Technical level: Intermediate. The statistical toolkit (PCA, k-means, Fisher's exact test, Cramér's V, OLS fits) is standard for learning-analytics readers, but the paper explains its choices in accessible terms and the indicators themselves are simple threshold rules.

Scope (one sentence): An analysis of structural Moodle variables from 606 courses at a large German distance university (FernUniversität in Hagen) that derives three empirical course "types" (Compact, Expansive, Interactive) and proposes three composite indicators of learning design — activity-mode balance, structural clarity, and analytics readiness — computed from routinely collected LMS data alone.

What This Paper Is About

Most learning-analytics work describes courses through what students do (clicks, engagement logs), not through what instructors have designed — the activities, sections, and resource arrangements visible in a learning management system. Large-scale empirical evidence on how online courses are actually structured is scarce, and computable indicators of design itself are underdeveloped.

The authors ask two questions: (RQ1) what learning-design types characterise Moodle courses at scale, and how do they differ by discipline, activity load, and interface; and (RQ2) how can learning-design indicators be derived from LMS course data, and do they meaningfully distinguish those types? They answer both using design-side data only — no student interaction logs.

Key Contributions

1. A large-scale empirical characterisation of authentic course designs: structural data on 606 courses (mean 7.3 sections, SD 5.7; mean 50.6 activities, SD 64.5) from two semesters (summer 2025 and winter 2025/2026), drawn from real institutional Moodle instances rather than manually coded samples.
2. Three empirically derived learning-design types: Compact (n = 135), Expansive (n = 425), and Interactive (n = 46), obtained via PCA followed by k-means, and statistically characterised across discipline, activity load, and interface.
3. A transparent, reusable procedure for deriving LD indicators: a two-stage method combining theoretically motivated constructs, expert-set thresholds, and validation against empirical distributions — explicitly designed to work without access to instructor intent.
4. Three new higher-order LD indicators: activity-mode balance, structural clarity, and analytics readiness, each aggregating multiple LMS variables, and each shown to differentiate significantly across all three LD types.

Main Findings

• Three types, not a single spectrum. PCA retained components explaining approximately 92% of variance; PC1 (34.7%) acts as a course-scale axis (activities, sections, page content) and PC2 (17.2%) captures structural clarity plus activity mix; together they explain roughly 52% of variance. The types reflect qualitatively different organising logics: Compact courses scale sections and activities in tight proportion (within-type r = .77), Expansive courses spread activities across many sections with variable orderliness (r = .47), and Interactive courses decouple the two, concentrating many activities into few sections (r = .47, but structurally distinct).
• Clusters are a pragmatic segmentation, not natural kinds. Silhouette width was 0.19 for k = 3 versus 0.30 for k = 2 in the main text (the appendix silhouette figure reports ≈0.22 for k = 3); the authors kept three clusters for interpretability and note that courses form a continuum.
• Discipline differences are real but moderate. Expansive designs dominate HASS (75.0%), STEM (85.8%), and Commerce (86.2%), with no significant differences among those three after Bonferroni correction. Psychology is the outlier: the most balanced distribution and the only discipline with a substantial Interactive minority (24.7%), differing significantly from all others (p = .001). STEM vs. Law also differed (p = .002), reflecting Law's larger share of Compact courses (31.5%). Overall FET: p < .001, Cramér's V = 0.24.
• Activity load is the strongest discriminator. All 46 Interactive courses were "large" (80+ activities); Compact courses were almost exclusively small (36.5%) or medium (12.6%), with a single large course (1.0%); Expansive spanned all bands. FET: p < .001, Cramér's V = 0.50, a large effect, and all three pairwise comparisons were significant (p_adj < .001).
• Interface matters least. All types relied predominantly on the default "topics" format; FET was significant (p < .001) but the effect small (Cramér's V = 0.13), driven by Interactive courses' comparatively greater use of "tiles" (topics–tiles contrast p_adj < .001; one-topic did not differ from either).
• Activity composition separates Interactive courses. Interactive courses combined higher collaboration with higher assessment use (within-type r = .48 between the two ratios), versus r = −.08 for both Compact and Expansive. Compact courses showed the highest collaboration ratio with minimal assessment.
• Density costs orderliness in Interactive courses. Section density and mean orderliness correlated r = .27 for Compact, r = .00 for Expansive, and r = −.48 for Interactive — concentration of activities into few sections comes with measurably lower layout consistency.
• All three indicators differentiate the types (p < .001 for each). Activity-mode balance: Cramér's V = 0.26, all pairwise p_adj < .01, strongest separation Compact vs. Expansive (V = 0.35). Structural clarity: V = 0.27, all pairwise p_adj < .001, strongest separation Compact vs. Interactive (V = 0.57). Analytics readiness: V = 0.27, all pairwise p_adj < .05, with Interactive vs. Compact (V = 0.46) and Interactive vs. Expansive (V = 0.40) the strongest contrasts.
• Indicator profiles per type (Table 4). Content-heavy design: 0.7% of Compact, 26.6% of Expansive, 13.0% of Interactive. Collaboration-heavy: 81.5% of Compact, 66.1% of Expansive, 65.2% of Interactive; mixed: 17.8%, 4.0%, 21.7% respectively. Note: the running text in Section 3.2 attributes the 4.0% and 21.7% figures to "collaboration-heavy" and the 66.1%/65.2% figures differently from Table 4, so the table and prose are not fully consistent on this one row.
• Structural clarity and analytics readiness diverge from intuition. Compact courses were almost never cluttered (0.7%) and most often "clear" (30.4%); Interactive courses were cluttered in 41.3% of cases and clear in only 10.9%; Expansive courses were mostly "mixed" (79.1%). Yet Interactive courses were the best instrumented: 76.1% moderate and 13.0% high analytics readiness, the only type with a substantial high-readiness share. Compact courses had no high-readiness designs at all; Expansive were 68.0% low.
• Analytics readiness is not a restatement of the clustering. The authors stress it does not enter the clustering solution directly yet discriminates strongly once introduced, which they read as independent corroboration of the typology.

Methodology in Plain English

The team exported course-structure data from institutional Moodle for every course active and visible to students in two semesters. No clickstream or user logs were included. After cleaning (removing duplicates, non-regular teaching, and variables with heavy missingness or plugin-specific scope), they worked with 14 structural variables such as number of sections, number of activities, page items, assessment/receptive/productive activity counts, unique activity types, assessment ratio, nested surface ratio, collaboration ratio, LA block ratio, mean orderliness (0–1 regularity of activity-type sequencing), section density, and the standard deviation of activities per section.

To find course types, they standardised the variables and ran PCA — chosen over clustering raw variables because instructors' design choices make the variables co-vary, and because PCA yields more stable, interpretable dimensions. They then ran k-means on the PCA scores, using elbow and silhouette diagnostics plus interpretability to pick k = 3, and checked robustness with Adjusted Rand Index against k-means on the full variable space and with average silhouette width, Calinski-Harabasz, and Davies-Bouldin indices.

To characterise the types, they cross-tabulated them against discipline (HASS, STEM, Psychology, Commerce, Law), activity load (small = fewer than 30 activities; medium = 30–79; large = 80 or more), and interface (one-topic, tiles, topics), using Fisher's exact test with Bonferroni correction. Scatter plots with within-type Pearson correlations and OLS fit lines explored how variables relate inside each type.

For the indicators, they first converted simple variables into categorical bands (assessment diversity: 0, 1–3, 4–10, 11+ items; section density: <5, 5–<15, ≥15 activities per section; orderliness: <0.33, 0.33–<0.66, ≥0.66), setting cut-offs iteratively through expert discussion and checking them against the observed distributions rather than using fixed conventions. These bands were then combined with further LMS variables into three indicators with explicit rules — e.g., "collaboration-heavy" requires a collaboration ratio ≥ 0.20 and an assessment ratio ≥ 0.20; "cluttered" requires high section density with low/medium orderliness; "high analytics readiness" requires an LA block ratio > 0 and ≥ 100 nested items. Each indicator's distribution across types was tested with FET using Monte Carlo approximation (B = 199,999 resamples) and reported with Cramér's V.

Why This Matters

The paper shifts design analytics from classifying individual activities to composing higher-order indicators of design, and shows this can be done from data institutions already hold, with no manual coding and no access to instructor intent. That is a practical route to scaling learning-design research beyond single-institution, hand-coded samples (the paper contrasts its approach with prior work such as 6,040 Moodle offerings across four years and ten faculties, or 102 Canvas courses at a Norwegian university).

Real-world applications:

• Institutional course-design audits: faculties can see what proportion of their courses are Expansive, Compact, or Interactive, and whether those proportions match disciplinary intentions — Psychology's 24.7% Interactive share versus 2.8% in STEM is exactly this kind of signal.
• Targeted design support: because Compact courses score high on structural clarity but zero on high analytics readiness, and Expansive courses are 68.0% low-readiness, support staff can direct analytics-instrumentation help where it is missing instead of where it is already present.
• Course (re)design review for navigation quality: the density–orderliness trade-off measured in Interactive courses (r = −.48) gives a concrete diagnostic for courses that concentrate activities into few, visually crowded sections.
• Lightweight benchmarking across platforms or institutions, since the indicator rules use generic LMS constructs (sections, activities, ratios, blocks) rather than Moodle-specific plugins.

Industry relevance: LMS vendors and ed-tech platforms can expose these indicators natively in instructor dashboards; the paper's framing of "analytics readiness" as a design property is directly actionable for product teams building adoption and instrumentation features, and for institutions procuring or configuring course templates. The authors also connect the work to generative AI pressures on course design, arguing that instructors' course-level choices are currently "largely invisible at scale" to both researchers and institutions.

Future Directions

• Link types and indicators to student behaviour and outcomes. Student interaction logs were not included and learning outcomes are not systematically captured in Moodle, so the study can describe designs but cannot say whether a type affects engagement or performance. Longitudinal linkage is the authors' headline next step.
• Test threshold robustness and transferability. The cut-offs were set by expert discussion and validated against one institution's distributions; the authors call for sensitivity analysis plus validation across other institutions, LMS platforms, and disciplines.
• Add instructor rationales. LMS data shows how a course is organised but not why; complementing indicators with instructor explanations would move the work from characterising design to understanding design reasoning.
• Track design change over time. The analysis is a snapshot of two semesters; following courses longitudinally would show how designs evolve, including under the pressure of new generative-AI tools and reassessed assessment practices.

Target Audience

Learning-analytics and learning-design researchers who want a replicable template for deriving design-side indicators from LMS logs; institutional learning-technology teams, educational developers, and quality-assurance staff at universities running Moodle or comparable platforms; and data scientists or product managers in ed-tech who need interpretable, threshold-based course metrics rather than black-box models. Readers new to the area will find the methods section approachable; readers wanting causal claims about learning outcomes will find the paper explicitly defers them.

Authors’ abstract

Although learning design (LD) plays a central role in shaping online courses, large-scale empirical evidence on how courses are actually structured remains scarce, and existing analytics work has characterised courses through student engagement with specific learning activities rather than through indicators of the design itself. This study addresses both gaps through an analysis of learning management system (LMS) data from 606 courses at a distance university. Using LMS data, we first applied dimensionality reduction and clustering to a structural variable set, from which three distinct LD types emerged empirically: Compact, Expansive, and Interactive and statistically characterised how their design elements differ across disciplines, activity load, and interfaces. Building on this typology, we derived three LD indicators, activity-mode balance, structural clarity, and analytics readiness, each combining multiple LMS-derived variables into higher-order constructs that target a different aspect of LD. All three indicators differentiated significantly across the LD types and added pedagogically meaningful dimensions for interpretation. These contributions advance the operationalisation of LD in technology-enhanced learning research by demonstrating that higher-order LD indicators can be derived directly from routinely collected LMS data, and offer empirical insight into how LD manifests at scale within a large distance university.