Enhancing Procedural Writing Through Personalized Example Retrieval: A Case Study on Cooking Recipes

AI summary

Overview

Research area: Human-Computer Interaction, specifically educational technology and adaptive learning systems, with substantial contributions from natural language processing (LLM fine-tuning, example retrieval) and the learning sciences (example-based learning, self-explanation).

Technical level: Intermediate. The system architecture is explained clearly, but understanding the fine-tuning, regression prediction, and BM25 retrieval steps requires some familiarity with NLP concepts.

One-sentence scope: The paper designs, implements, and evaluates RELEX, a system that adaptively retrieves a higher-quality, topically similar cooking recipe as a personalized example to help learners improve their own procedural writing.

What This Paper Is About

Procedural writing (recipes, lab protocols, manuals, instructions) is a domain-dependent skill that many learners struggle to master, and example-based learning is a proven way to help them improve. However, conventional approaches give every learner the same static expert example with the same generic explanations, regardless of what they actually wrote or already know, which makes the feedback feel redundant, too complex, or irrelevant. This paper builds and tests a system that instead selects a different, better, and more relevant example for each learner based on their specific input and inferred skill level, and enriches it with explanations targeting only the weaknesses the learner actually has.

Key Contributions

1. The RELEX system and its personalized example retrieval pipeline. A four-stage online process that (a) predicts the quality of the learner's recipe with a fine-tuned LLM, (b) retrieves recipes of higher quality from a large database, (c) filters them to those relevant to the learner's specific missing elements, and (d) uses BM25 to select the semantically most similar one in real time.

2. RELEXset, a large annotated recipe resource. A dataset of over 180,000 rated, cleaned recipes derived from RecipeNLG, with quality scores obtained by standardizing food.com user ratings per rater and averaging them, plus an annotation layer of roughly 45 writing suggestions drawn from a professional recipe-writing handbook and general procedural-writing principles.

3. A recipe quality predictor built through two-stage fine-tuning. A DistilRoBERTa model first adapted to the cooking domain through masked language modeling on RecipeNLG, then fine-tuned as a regression model to predict a recipe's averaged star rating (MAE of 0.39 versus 0.42 for a mean-prediction baseline, chosen partly for real-time inference without GPUs).

4. A design process and controlled evaluation. Ten semi-structured user interviews yielding 22 user stories and five user requirements, combined with literature-derived requirements, then a 2x2 controlled study with 200 participants (adaptive vs. non-adaptive examples, reflective prompts vs. none) plus a static-support control group.

Main Findings

• Personalized examples improved writing performance. Participants who received tailored example recipes produced higher-quality recipes than those who received non-adaptive examples, as reported in the abstract and introduction.

• Personalized examples increased revision activity. Learners who got tailored examples revised their recipes more than learners without adaptive feedback, addressing the revising-behavior research question.

• User experience was more positive with adaptation. Users receiving tailored examples rated the tool more favorably than those in the non-adaptive condition.

• Quality prediction was modest but generalizable. The fine-tuned predictor achieved an MAE of 0.39 versus a baseline of 0.42; the difference was not statistically significant, but the model generalizes to unseen recipes rather than predicting a constant, which the authors argue makes it more trustworthy in practice. The pipeline compensates for this uncertainty by only retrieving recipes whose quality is comfortably above the learner's predicted score.

• Design requirements converged from two sources. User needs and learning-science literature independently pointed toward the same features: relevant/similar examples, on-demand access to multiple examples, color-coded highlighting of important elements, short and dismissible explanations, and built-in self-reflection prompts.

Methodology in Plain English

The work proceeds in three parts: building the data and models, building the interface, and running an experiment.

Offline, before any learner uses the tool: The researchers took a public recipe collection (RecipeNLG) and attached real user ratings scraped from food.com to more than 180,000 of them. Because different raters use different scales, each person's ratings were normalized against their own average, then averaged across users to produce a single quality score ("stars") per recipe. A small, efficient language model (DistilRoBERTa) was first trained on all the recipe text to absorb cooking-domain vocabulary, then fine-tuned to output a predicted star rating for any recipe. Separately, recipes were automatically annotated with writing suggestions using rules and regular expressions, drawn from a professional recipe-writing handbook and from general principles of good procedural writing, covering structure, clarity, and specificity.

Online, when a learner submits a recipe: The system predicts that recipe's quality score, pulls candidate recipes from the database that score meaningfully higher, keeps only candidates that address the specific improvement suggestions relevant to the learner's weaknesses, and uses BM25 (a classic keyword-based ranking method) to pick the most topically similar one. The chosen recipe is then displayed with color-coded highlights and short, dismissible tips, alongside a reflection space that asks the learner to compare the example to their own recipe.

Evaluation: A 2x2 controlled study with 200 participants varied whether examples and feedback were adaptive or not, and whether reflection prompts were present or not, with an additional control group that received only static procedural writing support. The researchers measured learning experience, writing performance, and revision behavior.

Why This Matters

This paper is a concrete demonstration that LLM-based quality prediction and retrieval can be combined with example-based learning theory to produce feedback that is genuinely individualized rather than generic. It extends example-based learning, which has mostly been validated on well-structured tasks like algebra, into heuristic domains where there is no single correct answer, and it shows a practical path for choosing a learning example based on the learner's demonstrated skill rather than only on topical similarity.

Real-world applications:

• Culinary education, where chef apprentices commonly omit ingredients and lack detail in their written recipes.
• Technical documentation and instruction manuals, where incorrect step ordering and ambiguity cause real operational failures.
• Laboratory protocols in the life sciences, which require both domain knowledge and procedural precision.
• Writing instruction generally, including second-language learners and professional communication training, where example comparison is a standard pedagogical technique.

Industry relevance: The pipeline is designed for real-time inference without GPUs, which matters for edtech platforms that need low latency. The overall pattern, scoring user-generated content with a fine-tuned model, retrieving better examples, and attaching targeted explanations, transfers directly to automated feedback products, content quality assessment, and retrieval-augmented learning tools.

Future Directions

• Generalizing beyond cooking. The pipeline's structure is domain-agnostic, but the annotations and the quality predictor are recipe-specific. Testing whether the same approach works for lab protocols, technical manuals, or legal writing is the obvious next step.

• Disentangling which component drives the benefit. The study varied adaptivity of both the example and the annotations together. Separate experiments could isolate whether the gain comes from the example's quality, its relevance, the targeted explanations, or the reflection prompts.

• Improving and stress-testing the quality predictor. The prediction improvement over the mean baseline was not statistically significant. Better calibrated or stronger models, and analysis of how prediction error propagates into example selection, remain open problems.

• Measuring durable learning rather than immediate revision. The study assesses short-term writing performance and revision behavior. Whether learners abstract general rules that transfer to future, unrelated procedural writing tasks is untested.

Target Audience

Researchers in human-computer interaction, educational data mining, and the learning sciences who study adaptive learning and example-based instruction; NLP practitioners interested in applying LLMs to educational content; edtech designers and developers building automated writing feedback tools; and instructors in culinary, technical writing, or laboratory training contexts who want evidence on whether personalized examples outperform static ones.

Authors’ abstract

Writing high-quality procedural texts is a challenging task for many learners. While example-based learning has shown promise as a feedback approach, a limitation arises when all learners receive the same content without considering their individual input or prior knowledge. Consequently, some learners struggle to grasp or relate to the feedback, finding it redundant and unhelpful. To address this issue, we present RELEX, an adaptive learning system designed to enhance procedural writing through personalized example-based learning. The core of our system is a multi-step example retrieval pipeline that selects a higher quality and contextually relevant example for each learner based on their unique input. We instantiate our system in the domain of cooking recipes. Specifically, we leverage a fine-tuned Large Language Model to predict the quality score of the learner's cooking recipe. Using this score, we retrieve recipes with higher quality from a vast database of over 180,000 recipes. Next, we apply BM25 to select the semantically most similar recipe in real-time. Finally, we use domain knowledge and regular expressions to enrich the selected example recipe with personalized instructional explanations. We evaluate RELEX in a 2 x 2 controlled study (personalized vs. non-personalized examples, reflective prompts vs. none) with 200 participants. Our results show that providing tailored examples contributes to better writing performance and user experience.