Two-Step Occupation Coding

AI summary

Overview

• Research area: Natural Language Processing, specifically occupation coding (mapping free-text job titles to occupational taxonomies) and Named Entity Recognition for labor market research.
• Technical level: Intermediate to Advanced. The paper assumes familiarity with text classification, sequence labeling, Transformer models, and standard evaluation metrics, though the central idea is explained conceptually.
• Scope: The paper proposes and evaluates a two-step pipeline that separates job title detection from taxonomy classification for German occupational data, comparing it against single-step end-to-end occupation coding.

What This Paper Is About

Occupation coding links job titles written in free text to standardized occupational taxonomies, and it is usually solved end-to-end in one step that both finds job titles and assigns codes. This paper argues that such single-step methods conflate two conceptually distinct tasks: deciding whether a text span is a job title at all, and classifying a recognized title against a taxonomy. The goal is to show that splitting these tasks into a Named Entity Recognition (NER) step followed by a separate classification step improves accuracy, robustness, and interpretability.

Key Contributions

1. A two-step occupation coding approach that explicitly separates job title identification from taxonomy-based classification, in contrast to existing single-step, end-to-end approaches. The authors describe this as one of the first applications of pipeline decomposition to occupation coding.
2. A domain-specific NER model trained with a noise-aware, multi-step fine-tuning strategy that integrates occupational taxonomies and other word lists as additional training resources.
3. A retrained occupation coding classifier that focuses exclusively on matching previously identified job titles to the taxonomy. Multiple classifiers are systematically compared: Support Vector Machines (SVM), logistic regression, and XGBoost.
4. A margin-based confidence criterion for occupation coding, proposed as an alternative to the commonly used absolute confidence thresholds. The authors also publish source code and evaluation scripts for reproducibility.

Main Findings

• Two-step beats one-step: The two-step approach achieved an F1 score of 57.26% compared to 46.94% for the one-step approach implemented in the occupationCoding R package, using the same training data (the list of occupations provided by the Federal Employment Agency).
• Precision drives the gain: The improvement in F1 was mainly driven by higher precision, while recall remained almost unchanged. This indicates that the advantage lies in reducing false positives, because the NER step restricts the classifier's input to occupational entities and filters out non-occupational fragments and OCR noise.
• SVM was the best classifier: Among the three classifiers compared on a test set split from the standardized word lists, the SVM achieved the highest accuracy with an F1 score of 94.92%. The authors note that accuracy on standardized word lists is significantly higher than on real-world data.
• Confidence margin slightly outperforms absolute threshold: Evaluated on historical and contemporary real-world data, the confidence margin approach achieved an F1 score of 43.96%, compared to 43.89% for the absolute threshold approach. Both outperformed the baseline of accepting all predictions, and the combination of both thresholding methods achieved an identical F1 score to the confidence margin approach alone.
• Combined confidence is multiplicative: The overall confidence is computed as the product of the NER confidence and the occupation coding confidence (conf = conf_NER · conf_OC).
• JobBERT dataset results: On the JobBERT dataset (a subset of ESCO containing 15,462 occupation instances and 1,369 unique occupations), the proposed two-step method reached recall of 82.86% and precision of 30.14%. For comparison, the paper reports micro Recall@1 figures taken from Decorte et al. (2021): 19.30% for SBERT, 19.19% for JobBERT, and 22.48% for a fine-tuned JobBERT. Applied thresholds were an absolute threshold of t = 0.6 and a confidence margin of m = top1 − top2 = 0.3.
• Comparability caveat: The authors emphasize that these approaches are only partially comparable, since JobBERT and SBERT rely on contextual semantic information from skill-related data, whereas the proposed method is entity-based. They argue that semantic-based approaches are conceptually superior but must be properly trained, otherwise entity-based approaches can outperform them.
• High recall, lower precision profile: On the JobBERT evaluation, the system is described as coverage-oriented, capturing most job titles with few false negatives at the cost of reduced precision.
• Thresholding evidence is illustrative: The authors state that statistically robust conclusions about the thresholding strategies would require a broader evaluation across multiple data splits and repeated runs.
• NER stability check: The NER model contains multiple stochastic components, so it was trained five times with different random seeds and the standard deviation of the evaluation metrics was reported. The specific precision, recall, and F1 values for NER are presented in Table 2, whose contents are not included in the provided text.

Methodology in Plain English

The approach breaks occupation coding into two consecutive stages.

Step 1 – Finding job titles. A German cased BERT model from the Bavarian State Library is used as the base model. The authors address two problems: noisy input (OCR errors and spelling mistakes) and the variability of job titles across time and regions. To handle noise, they use Noise-aware Training, injecting synthetic OCR-style errors (insertions, deletions, substitutions) on the fly during training so each input gets a new noisy variant before tokenization. To handle variability, they use transfer learning plus multi-stage fine-tuning: an intermediate pre-training stage is trained on additional data, and a final stage is trained on real-world vocational education and training (VET) data. Additional data include word lists of search terms from the Federal Employment Agency and the KldB taxonomy in both its 2020 and 1988 versions. Because this additional data contains no "O" labels (tokens outside any entity) and no OCR noise, it cannot be used directly for the final model, which is why the intermediate stage exists. The training data combine historical documents (VET documents from East and West Germany and reunified Germany, including training regulations and systematic job title listings, covering the period since 1976 and comprising 159 document pages, digitized so they may contain OCR errors) with contemporary data crawled from X (formerly Twitter), which is born-digital but contains spelling variations and non-standardized job titles. Both sources are used for NER, but only the contemporary KldB-annotated data is used for occupation coding.

To compensate for label imbalance, the authors applied class weighting and oversampling, deliberately removed a proportion of "O" tokens, normalized and clamped the class weights, and selected the best model across epochs by micro-averaged entity-level F1. Annotations use the BIO format with B-OCCUPATION, I-OCCUPATION, and O tags. A single OCCUPATION entity class was used, merging job titles and higher-level job title groups after confusion analysis showed the model struggled to distinguish them.

Step 2 – Mapping titles to codes. The classifier is retrained because it no longer needs to find job titles in free text. Three classical classifiers were compared using scikit-learn: an SVM, XGBoost, and logistic regression, all operating on TF-IDF vectorized inputs. Classical classifiers were chosen deliberately to maintain comparability with the one-step approach and to isolate the effect of task separation. Although one might expect a decision tree method to suit the hierarchical KldB taxonomy, XGBoost treats all classes as flat categories, and the SVM performed best.

Confidence handling. Instead of asking whether a prediction exceeds a fixed absolute threshold, the margin-based approach evaluates how confident the model is about its top prediction relative to the second-highest one. The paper illustrates this with two examples where the same top job ID ("26302-103 – Elektrotechnische/r Assistent/in") and second-highest job ID ("41412-100 – Physikalisch-technische/r Assistent/in") are returned, and neither exceeds a threshold of 0.8. In the first example the gap between the two scores is large and a margin-based approach would return the top job ID; in the second the gap is small and no job ID would be recognized.

Evaluation setup. Annotated data were split 70:20:10 into training, test, and validation sets at segment level rather than token level, to avoid breaking multi-word entities. Metrics are precision, recall, and F1. Training used a warm-up phase over 10% of steps with the learning rate gradually increased to a target of 2·10⁻⁵ and then decreased.

Why This Matters

Impact on research. The paper reframes occupation coding as two distinct problems rather than one, arguing that this separation lets error sources be attributed more clearly to either identification or classification. It also introduces a margin-based confidence criterion into a field where absolute thresholds have been the norm, and it points out that no standard benchmark dataset exists for occupation coding while contributing a publicly available evaluation setup.

Real-world applications:

• Labor market research on surveys and administrative data, where job titles must be classified automatically for statistics.
• Job search platforms, where matching job seekers to relevant postings depends on correctly interpreting free-text job titles.
• HR departments filling positions, where unstructured job titles must be aligned with standardized occupational categories.
• Vocational institutions aligning their programs with labor market demands.

Industry relevance. Automated coding promises lower processing costs, faster production of statistics, and higher consistency in coding decisions compared to manual assignment. Because the proposed method tolerates OCR errors and non-standardized titles, it is relevant to organizations processing digitized historical archives, social media data, or noisy job postings. The authors caution, however, that the results should be interpreted as a methodological improvement rather than a ready-to-use occupation coding system.

Future Directions

• Stronger classifiers: Training a Transformer-based classifier instead of classical machine learning models is expected to further enhance recognition rates.
• Semantic context recognition: The current approach depends on explicitly mentioned job titles, which is error-prone, especially in contemporary social media data where occupations are often paraphrased. Future work plans to recognize occupations from semantic context, similar to JobBERT or SBERT, but within a two-step framework. The existing NER model already supports other entity types such as skills, working activities, and work equipment, providing a foundation for richer semantic understanding.
• Multilingual extension: Extending the system to English and other languages is a stated goal. Because the KldB taxonomy is aligned with ISCO, annotated ground truth data could be generated automatically to train multilingual models.
• Standardized benchmarking and robustness: Establishing a standardized benchmark dataset for occupation recognition and classification remains open, and the authors note that robust conclusions on thresholding would require broader evaluation across multiple data splits and repeated runs. A systematic bias analysis is also identified as beyond the scope of this paper but as a possible future evaluation.

Target Audience

This paper is most useful to researchers and practitioners in labor market research, official statistics, and computational social science who work with occupational taxonomies and free-text job titles. It also suits NLP researchers interested in pipeline decomposition, noisy-text NER, or confidence estimation for classification. Readers working on German-language processing, historical document digitization, or vocational education and training data will find the domain-specific details directly applicable, and engineers building job-matching or HR systems will find the conceptual framing and published code useful.

Authors’ abstract

Occupation coding links job titles in free text to occupational taxonomies and is a core task in labor market research. Existing approaches typically address this problem in a single end-to-end step, jointly identifying job titles and assigning occupational codes. This paper presents a novel two-step approach that separates these tasks. In the first step, a domain-specific Named Entity Recognition (NER) model identifies occupational titles in continuous text, even under noise such as OCR errors. In the second step, the extracted job titles are mapped to a taxonomy, enabling the classifier to focus exclusively on this mapping. We demonstrate that this separation improves accuracy, robustness, and interpretability compared to single-step approaches. The method has been developed for German documents but is transferable to other languages. We further introduce a margin-based confidence criterion for occupation coding, replacing common absolute thresholds. To support reproducibility, we publish the source code and evaluation scripts.