Losses that Cook: Topological Optimal Transport for Structured Recipe Generation

AI summary

Overview

Research area: Natural Language Processing, specifically structured text generation (recipe generation) and loss function design for small language models.

Technical level: Intermediate. The paper assumes familiarity with cross-entropy training, loss weighting, edit distance metrics, and text embeddings, but the core idea (comparing predicted and gold ingredient lists as point clouds) is explained in accessible terms.

Scope: The paper evaluates composite training objectives — focal, Dice, and a newly proposed topological (optimal-transport) loss — for fine-tuning small language models to generate structured cooking recipes with correct ingredients, quantities, times, and temperatures.

What This Paper Is About

Standard recipe generation models are trained with cross-entropy, which treats every token as equally important — so omitting "eggs" from a carbonara costs the model as little as misplacing a comma. This paper asks whether changing the training loss, rather than the model size, can make small language models produce recipes that are not just fluent but actually usable: correct ingredients, plausible quantities, and executable steps. The authors introduce a topological loss that treats ingredient lists as geometric objects in embedding space and penalizes structural deviations from the gold recipe.

Key Contributions

1. A topological loss for ingredient sets. A new objective that represents the predicted ingredient section as a soft, differentiable point cloud (a probability-weighted average over the model's embedding matrix) and compares it to the gold ingredient point cloud using Sinkhorn divergence, an entropic-regularized optimal transport distance.

2. A systematic comparison of composite objectives. Cross-entropy is combined with focal loss, Dice loss, and the topological loss in controlled single-custom-loss configurations (0.6 weight on CE, 0.4 on the custom loss), plus a mixed objective (0.6 CE + 0.2 Dice + 0.2 Topo), all fine-tuned on the same data with LoRA.

3. A recipe-specific evaluation suite. Beyond ROUGE-1 and BERTScore, the paper defines Ingredient Recall, Quantity Precision, Action Precision, Action and Step Edit Distances, and Time and Temperature Precision, with an NER pipeline validated against Gemini 2.0 and human annotations.

4. A human preference and error study. Three expert annotators compared CE-fine-tuned and Topo+Dice-fine-tuned outputs on 64 recipes, with inter-annotator agreement and per-error-type analysis.

Main Findings

• Fine-tuning beats larger pretrained models on task metrics. On the Qwen3-4B backbone, non-fine-tuned Gemini 2.0 scored ROUGE-1 15.0847, BERTScore 88.50, Action Precision 43.80, Ingredient Recall 37.47, and Step Distance 48.60, while non-fine-tuned Qwen3-14B scored ROUGE-1 25.2393, BERTScore 85.69, Action Precision 42.12, Ingredient Recall 28.25. All fine-tuned 4B variants improved on these task-specific measures.

• The topological loss gives the most consistent recipe-specific gains. For Qwen3-4B, Topological achieved Action Precision 59.68, Quantity Precision 63.93, and Ingredient Recall 48.59, versus CE's 45.09, 50.94, and 35.98 respectively, while also reaching the lowest Action Distance (30.49) and Step Distance (34.09).

• Dice loss excels at numerical precision. Dice recorded the best Temperature Precision (74.58) among single-loss configurations on Qwen3-4B, versus 61.93 for CE and 65.59 for Topological.

• The mixed Topo+Dice objective gives the best overall balance. It achieved the best ROUGE-1 (31.9045), BERTScore (90.99), Quantity Precision (65.09), and Time Precision (61.95), while matching the best edit distances (Action Distance 30.49, Step Distance 34.09).

• Focal loss underperforms on culinary constraints. Focal improved BERTScore (89.94) and Ingredient Recall (43.09) over CE, but lagged Dice and Topological on most task metrics, and the authors' significance tests indicated BERTScore gains are not always reliable.

• Trends hold across architectures and scales. On SmolLM3-3B, Topological reached the best ROUGE-1 (25.9079), BERTScore (89.7789), Action Precision (37.5597), Quantity Precision (51.8961), and Ingredient Recall (36.9899) with the lowest edit distances (41.5958 / 42.0973). On Qwen2.5-1.5B, Topological again led on ROUGE-1 (16.86), BERTScore (87.79), Ingredient Recall (29.29), and edit distances (45.39 / 51.98).

• Human annotators preferred Topo+Dice. Across 64 recipes and 3 annotators (192 judgments), Topo+Dice was preferred over CE for ingredients (38% vs 10%), procedure (46% vs 9%), numbers (42% vs 9%), and overall quality (62% vs 11%), with win rates of 79%, 81%, 88%, and 84% when excluding ties. The paper reports all differences significant at p < 0.01.

• Topo+Dice sharply reduced errors. Annotators flagged errors in 7.8% of Topo+Dice evaluations versus 24.0% for CE, a 67.5% reduction. Step mismatches fell from 14 to 2, missing ingredients from 14 to 3, bad quantities from 13 to 2, and hallucinations from 4 to 0. Time-related "Other" errors rose slightly (7 vs 3).

• Evaluation extraction is reliable. Qwen3-4B NER agreement with Gemini 2.0 on 60 recipes ranged from F1 91.67 (temperature) to 97.35 (instructions); Gemini-to-human agreement ranged from 94.44 to 99.67. Recomputing task metrics with Gemini as extractor shifted scores by roughly +1 point (about +2.5 for step distance), nearly identically across CE and Topo+Dice, preserving the ranking.

Methodology in Plain English

The authors take a subset of 5,000 recipes from the RECIPE-NLG corpus, restricted to pasta, rice, and sandwiches, and convert all imperial measurements to metric using Gemini 2.0 Flash. They add 235 manually curated cooking questions covering missing-ingredient identification, substitution, scaling, quantity reasoning, time estimation, and temperature specification.

They fine-tune a pretrained Qwen3-4B model using LoRA (rank 8, alpha 16, dropout 0.05, applied to q_proj, k_proj, v_proj, o_proj) with bfloat16 training, a learning rate of 10⁻⁴ for 2 epochs, batch size 2, and gradient accumulation of 3. Each custom loss is blended with cross-entropy at 0.6/0.4, and a mixed configuration uses 0.6 CE + 0.2 Dice + 0.2 Topo. The Sinkhorn epsilon is 0.05 and the focal gamma is 2.

The topological loss works like this: rather than scoring token-by-token, the model's output logits for the ingredient section are turned into probabilities via softmax and used to compute a weighted average of the token embedding matrix. This produces a "soft" point cloud for the prediction, which is compared against a point cloud built from the gold ingredient embeddings. Sinkhorn divergence measures how far apart the two clouds are geometrically, so substituting "pepper" for "salt" costs more than producing a near-identical list. Shared tokens like "flour" have zero transport cost.

For evaluation, all models generate 1,000 recipes in JSON with schema enforcement via the Outlines library. A Qwen3-4B NER pipeline extracts quantities, units, times, temperatures, and primary actions from the generated text, and these extracted entities drive the recipe-specific metrics.

Why This Matters

Impact on research. The work demonstrates that the training objective, not just model scale, is a lever for structured generation quality. It brings optimal transport into the loss function for a structured NLG task and shows that geometric alignment of embedding point clouds can outperform token reweighting schemes like focal loss. The recipe-specific metric suite also offers a template for evaluating other structured generation tasks where fluency metrics are insufficient.

Real-world applications:

• Consumer cooking assistants and recipe-generation apps that need correct quantities and times, not just readable text.
• Meal-planning and grocery-list systems where a missing or wrong ingredient cascades into a broken shopping list.
• Dietary-constraint and allergen filtering tools, where ingredient recall directly determines safety.
• Food-service and catering software that needs scalable, numerically plausible recipe drafting.

Industry relevance. The approach requires no increase in model size or inference-time complexity — the losses affect training only — which makes it attractive for teams deploying small, cheap models. The code is released at https://github.com/DarthReca/losses-cook.

Future Directions

1. Scaling to broader cuisines and dietary constraints. The current domain is intentionally narrow (pasta, rice, sandwiches), and the authors state improvements may not transfer without additional adaptation.

2. Entity-level rather than sequence-level comparison. The topological loss compares entire token sequences, so it captures geometric similarity but does not explicitly parse or match individual ingredients; future work could operate on parsed ingredient entities.

3. Better time reasoning. Time-related errors increased for Topo+Dice (7 vs 3 in the "Other" category), and the authors identify temporal specification as an area for improvement.

4. Safety and nutrition validation. The paper calls for explicit safety- and nutrition-aware validation, and the limitations note that the system is not guaranteed to produce safe recipes or respect allergies.

Target Audience

Researchers working on structured text generation, controllable NLG, or alternative training objectives for language models; practitioners building recipe, instruction, or JSON-structured generation systems on small models; and evaluation-focused researchers interested in domain-specific metric suites beyond ROUGE and BERTScore. Readers need only a working understanding of fine-tuning and loss weighting, not deep optimal transport background, since the topological loss is presented conceptually with illustrative figures.

Authors’ abstract

Cooking recipes are complex procedures that require not only a fluent and factual text, but also accurate timing, temperature, and procedural coherence, as well as the correct composition of ingredients. Standard training procedures are primarily based on cross-entropy and focus solely on fluency. Building on RECIPE-NLG, we investigate the use of several composite objectives and present a new topological loss that represents ingredient lists as point clouds in embedding space, minimizing the divergence between predicted and gold ingredients. Using both standard NLG metrics and recipe-specific metrics, we find that our loss significantly improves ingredient- and action-level metrics. Meanwhile, the Dice loss excels in time/temperature precision, and the mixed loss yields competitive trade-offs with synergistic gains in quantity and time. A human preference analysis supports our finding, showing our model is preferred in 62% of the cases.