TalentCLEF at CLEF2026: Skill and Job Title Intelligence for Human Capital Management

AI summary

Overview

• Research area: Natural Language Processing applied to Human Capital Management (HCM), specifically multilingual job–person matching and job–skill matching.
• Technical level: Intermediate. The paper is a lab/task description rather than an experimental study; it assumes familiarity with information retrieval metrics (MAP, MRR, Precision@K) and with concepts such as ESCO terminology, fine-tuning, and prompt engineering.
• Scope: This paper announces the second edition of the TalentCLEF evaluation lab at CLEF2026, defining two shared tasks, their datasets, and their evaluation protocols for benchmarking NLP systems in talent management.

What This Paper Is About

Companies struggle to match the right candidates to the right jobs and to understand which skills a given role actually requires, but most existing research relies on private data that cannot be shared or compared across teams. TalentCLEF addresses this by providing open, reproducible, multilingual benchmarks built from realistic synthetic data that preserves privacy. This second edition introduces two tasks: one that ranks candidate résumés for a job description, and one that predicts the skills associated with a job title and labels each as specific or transversal.

Key Contributions

1. Launch of the second TalentCLEF evaluation lab at CLEF2026, building on the first edition, with an explicit focus on fairness, multilinguality, and cross-industry adaptability.
2. Task A – Contextualized Job-Person Matching, which shifts from the previous edition's job-title similarity approach to matching synthetic job descriptions against synthetic candidate résumés in context-rich settings.
3. Task B – Job-Skill Matching with Skill Type Classification, which extends the prior year's skill prediction task by improving dataset quality and adding a new annotation layer that classifies each skill as either specific or transversal to the job title.
4. An evaluation protocol combining retrieval performance and fairness, using Mean Average Precision (MAP) as the official metric for both tasks, with Mean Reciprocal Rank (MRR) and Precision@K reported alongside, and a gender-bias assessment via Rank-Biased Overlap (RBO) for Task A.

Main Findings

• Prior edition participation: The first TalentCLEF edition attracted 76 teams and generated 280 system submissions, which the authors cite as evidence of community interest and the motivation for a second edition.
• Labor market context: The paper states that more than 1.1 billion jobs could be transformed by the end of the decade, that approximately 70% of the skills currently used in most occupations are expected to change, and that in 2024, 74% of employers reported difficulties finding candidates with suitable skills.
• Task A data: Participants receive a development set and a test set comprising a total of 100 job descriptions and 300 synthetic résumés. The documents are synthetically generated from structured data originally derived from real job descriptions and résumés, so the paper states there are no privacy risks associated with the data.
• Task B data: Three subsets are provided. The training set contains 5,000 job titles linked to their relevant skills; the development set contains 200 job titles normalized to ESCO terminology and enriched with skill type annotations; the test set comprises 500 job titles.
• Evaluation setup: Tasks A and B are evaluated on the Codabench platform using standard information retrieval metrics, with MAP as the official metric for both tasks. Task A is evaluated in two monolingual settings (English and Spanish) and one cross-lingual setting (en–es), plus a fairness assessment measuring performance with respect to gender bias using RBO.
• Recognition and submission: The top two teams in each monolingual Task A evaluation, the best cross-lingual team, and the model with the best bias control will be highlighted during the workshop; for Task B, the two best-performing teams will be recognized. All receive a certificate of achievement. Registration is via the CLEF conference website, with predictions uploaded through the CodaLab competition system.
• No 2026 results: Because this paper announces a lab that has not yet run, it reports no system results, rankings, or scores for either task.

Methodology in Plain English

The authors define a shared challenge rather than running their own experiments. They generate synthetic job descriptions and résumés from structured data that originally came from real documents, which lets them publish realistic multilingual material without exposing anyone's personal information. Each task then gives participants a training or development portion for building and tuning systems and a held-out test portion for the official scoring round, so every team is compared on identical inputs.

For Task A, a system takes a job description and must return a ranked list of candidate profiles ordered by relevance, with participants free to use techniques such as data augmentation, fine-tuning, prompt engineering, information extraction, or representation learning. For Task B, a system takes a job title, retrieves the skills that best match it, and additionally decides whether each skill is specific or transversal to that title. Scoring happens on Codabench using retrieval metrics, with MAP as the headline number, and Task A additionally includes a separate check for gender-related bias using Rank-Biased Overlap.

Why This Matters

The paper argues that progress in NLP for talent management is limited by a lack of shared, high-quality, publicly comparable resources: most studies use private data, the few public datasets lack consistent annotation criteria, and standardized evaluation frameworks for both performance and fairness are missing. TalentCLEF is positioned as a response to that gap, providing an open, reproducible, multilingual environment in which methods can be directly compared.

Real-world applications the work points toward:

• Talent matching and recruitment: ranking candidate profiles against job openings.
• Skill gap detection: identifying which skills an organization lacks relative to the roles it needs to fill.
• Upskilling and reskilling programs: designing targeted training based on the skills a job actually requires.
• Fairness auditing of hiring technology: evaluating whether matching systems behave differently across genders, which the paper frames as especially relevant for sensitive applications like HCM.

Industry relevance: The tasks target the shift employers have made from role-based to skill-based talent management, and they are hosted by an industry machine learning group (Avature Machine Learning) together with an academic group (NLP & IR Group at UNED), reflecting direct commercial interest in deployable, multilingual, fairness-aware hiring tools.

Future Directions

• Results of the 2026 edition: The performance of submitted systems, the comparative value of the techniques encouraged (data augmentation, fine-tuning, prompt engineering, information extraction, representation learning), and the outcomes of the fairness assessment are all still open.
• Extending multilinguality: Task A is evaluated only in English and Spanish and in the en–es cross-lingual pair; how systems generalize to other languages and language pairs is not addressed in this paper.
• Broadening skill type and taxonomy coverage: Task B uses ESCO normalization and a binary specific/transversal distinction on a development set of 200 job titles, leaving room to explore richer taxonomies and other skill-type dimensions.
• Scaling context-aware matching: The paper contrasts isolated-entity matching with context-aware matching enabled by LLMs; how far context-rich approaches can be scaled and how they trade off against technical and scalability constraints remains an open question.
• Other fairness dimensions: Only gender bias is measured, via RBO, and only for Task A, so other demographic or fairness criteria are not covered.

Target Audience

This paper is most useful for NLP and information retrieval researchers who plan to participate in or follow the TalentCLEF tasks, and for practitioners building talent-matching, skill-extraction, or job-recommendation systems who want a standardized benchmark. It is also relevant to fairness researchers interested in bias evaluation in hiring contexts, to teams working with multilingual and cross-lingual retrieval, and to human resources technology organizations seeking reproducible ways to compare approaches. Readers looking for experimental results or model comparisons will not find them here, since the paper describes a challenge that is yet to be run.

Authors’ abstract

This paper presents the second edition of the TalentCLEF Challenge, which will run as an evaluation lab as part of CLEF 2026. The aim of TalentCLEF is to promote the development of systems and methods that use Natural Language Processing (NLP) in the field of Human Capital Management (HCM), fostering approaches that ensure fairness in results, operate across multiple languages, and adapt to diverse industries. To this end, TalentCLEF establishes public benchmarks where research teams can compare methods and share findings, moving the field toward more practical and impactful NLP solutions that effectively address the real needs of workforce management. This year's lab will feature two tasks designed to foster the development and evaluation of systems that support key HCM activities such as talent matching, upskilling, reskilling, and skill gap detection: (i) Task A - Contextualized Job-Person Matching, focused on retrieving and ranking suitable candidates for specific job positions using context-rich and privacy-preserving data; and (ii) Task B - Job-Skill Matching with Skill Type Classification, centered on identifying relevant skills for a given job title and classifying them by their type within the job profile. TalentCLEF website: https://talentclef.github.io/talentclef/