ESG-Kor: A Korean Dataset for ESG-related Information Extraction and Practical Use Cases

AI summary

Overview

Research area: Natural Language Processing — domain-specific dataset construction and text classification for ESG (Environmental, Social, Governance) information in Korean.

Technical level: Intermediate. Readers should be comfortable with pre-trained language model fine-tuning, sentence embeddings, and standard classification evaluation, but no specialized ESG finance background is required.

Scope (one sentence): The paper introduces ESG-Kor, a manually labeled Korean dataset of 118,946 sentences drawn from corporate sustainability reports, benchmarks several Korean pre-trained language models on ESG classification, and demonstrates two downstream use cases (SME reports and ESG key-issue extraction).

What This Paper Is About

ESG information is published mostly as unstructured text in sustainability reports, with no standard format, and smaller companies often publish no report at all — so pulling out just the ESG-relevant content from large document sets is hard. Specialized-domain datasets are expensive to build because they need expert annotators, and most existing resources and pre-trained models are English-centric, leaving non-English specialized domains, including Korean ESG, with almost no resources.

The goal is to build a large Korean ESG dataset, verify that fine-tuning existing Korean pre-trained language models on it works, and show that the resulting model is useful beyond simple classification.

Key Contributions

1. ESG-Kor dataset construction. A dataset of 118,946 sentences from 173 Korean corporate sustainability reports (published 2002–2021), manually labeled by three annotators into five classes (Environmental, Social, Governance, Relevant, Irrelevant) following rules derived from ESG rating agencies including MSCI, Refinitiv, and KCGS. The authors describe it as the first constructed large-scale dataset associated with ESG and the only Korean-based professional NLP benchmark dataset manually labeled in the ESG field.

2. Benchmarking of Korean pre-trained language models. Fine-tuning and evaluation of KoBERT, DistilKoBERT, KB-ALBERT, and KLUE-RoBERTa (base and large) on four-class and binary ESG classification, reported per sector and overall.

3. Two practical use cases. Extension of the trained model to small and medium enterprise (SME) ESG reports, and an ESG key-issue extraction pipeline combining the classification model with sentence embeddings and cosine-similarity retrieval, validated by human evaluation.

4. A reusable construction template. The paper argues that the full construction process can serve as a benchmark for building ESG datasets in other languages, and releases the dataset via a public repository (https://github.com/nowzer0/ESG-Kor).

Main Findings

• Strong overall classification accuracy: KLUE-RoBERTa-large achieved 85.87% on four-class classification and 86.67% on binary classification across the full ESG-Kor dataset.
• Even the smallest model performed meaningfully: DistilKoBERT recorded the lowest four-class accuracy at 84.19%, indicating the dataset is usable across model types rather than only with the largest model.
• Sector-wise accuracy varied considerably: Under KLUE-RoBERTa-large, four-class accuracy ranged from 83.82% (Electronics) to 92.64% (Aviation); binary accuracy ranged from 86.12% (Electronics) to 93.54% (Heavy industry).
• Domain-specific pre-training helped despite fewer parameters: KB-ALBERT, a finance-specialized model with 9M parameters, performed as well as or better than the larger KoBERT (92M) and DistilKoBERT (28M), which the authors attribute to its financial-document pre-training.
• The dataset reflects real-world sector differences: The finance sector showed a notably higher proportion of Social-class sentences and fewer Environmental sentences than Governance, consistent with less environmentally impactful activities in that sector.
• Class imbalance is substantial: Of the 118,946 sentences, Social accounts for 45,782 and Environmental for 19,789, while Governance accounts for 8,819 and Relevant for 5,704; Irrelevant accounts for 38,852.
• Sentences fit within standard model limits: Token lengths range from 3 to 233 with an average of 42 and standard deviation of 20, so no input is truncated by the 512-token limits typical of pre-trained models.
• Transfer to SMEs succeeded: On a 403-sentence SME validation set (254 relevant, 149 irrelevant), the ESG-Kor-trained model reached 86.72% binary accuracy.
• Key-issue extraction was rated highly: On Hyundai Construction sustainability reports (6,403 sentences, of which 4,588 were classified as ESG-related), a human evaluation with 10 participants scored extracted content between 4.4 and 4.9 on a 1–5 scale, with an average of 4.67; Climate Change scored highest at 4.9 and Corporate Behavior lowest at 4.4.
• Annotation disagreements were rare: Sentences deleted due to annotator disagreement comprised less than 0.1% of the total dataset.

Methodology in Plain English

Data collection. The team gathered 173 Korean sustainability reports in PDF form. They surveyed 53 sectors published between 2002 and 2021, then focused on automobile, aviation, electronics, finance, and heavy industry because those had more than 20 published reports and relatively more data. They added POSCO (steel) and NAVER (IT) as additional high-profile sources.

Preprocessing. Only text was extracted from the PDFs — tables and images were excluded. Sentences were split with the Korean sentence splitter (KSS), numerical formats were unified, special characters and duplicate sentences were removed, and spelling was checked with the py-hanspell API. Humans then manually reviewed automatically mis-split, overly long, or overly short sentences and either re-segmented or deleted them.

Labeling. Three annotators with relevant knowledge organized labeling rules based on standards from MSCI, Refinitiv, and KCGS, then labeled sentences into five classes. Labeling was two-stage: clear sentences were labeled first, then ambiguous ones were discussed among all three annotators and kept only if everyone agreed; disagreement meant deletion. The process ran 2 hours a day, five days a week, at roughly 500–600 sentences per 2 hours, with each annotator labeling about 2,500 sentences per week — about four months of labeling clear sentences, two months of discussion on hard cases, and two months of cross-checking, totaling roughly 8–9 months.

Model training. Each sentence is encoded by a [CLS] token through the model layers, and a classification layer produces class probabilities via softmax; training minimizes cross-entropy loss. Hyperparameters were a batch size of 64, maximum length 256, the AdamW optimizer, epsilon 1e-08, and learning rates of 1e-04 (KoBERT), 5e-04 (DistilKoBERT), 5e-05 (KB-ALBERT), and 5e-05 (KLUE-RoBERTa). The data was split 70% training, 15% validation, 15% test, and all experiments ran on four NVIDIA GeForce RTX 3090 GPUs. The four-class task excludes the R class; the binary task merges E, S, G, and R into "relevant" and keeps Irrelevant separate.

Use case 1 (SMEs). The team crawled the 2022 ESG reports of three SMEs, applied the same preprocessing and labeling process, and built a 403-sentence binary validation set.

Use case 2 (key-issue extraction). The trained classifier first filters ESG-related sentences from a report. Those sentences are embedded with KoSimCSE (RoBERTa backbone) into the same vector space as MSCI's ESG key issues, and the top 10 most similar sentences per detailed issue are retrieved by cosine similarity. Because one detailed issue could dominate the results, the retrieved sentences were recombined per main issue, guaranteeing at least one highest-similarity sentence for each detailed issue and filling the remaining slots by similarity within the same main category. Evaluation covered Hyundai Construction's 2010–2020 reports — a construction company not in ESG-Kor — and was scored by 10 human participants.

Why This Matters

Impact on research. ESG-Kor fills a gap the paper identifies explicitly: machine learning work in ESG is limited by the lack of expert-labeled data, and most available resources are English. A 118,946-sentence, expert-labeled, non-English dataset gives researchers a reproducible benchmark and a documented construction protocol that can be adapted to other languages. The paper also reports computational cost (parameters and elapsed time) alongside accuracy, which supports realistic comparisons.

Real-world applications:

• SME ESG support: SMEs make up roughly 99% of Korean enterprises and often lack the capacity to produce ESG disclosures; the 86.72% SME accuracy suggests the model can act as an auxiliary tool for extracting and classifying their ESG content.
• Corporate report drafting and monitoring: Companies can extract or classify ESG information from various sources to compile sustainability reports and monitor their external ESG performance.
• Investor and regulator use: Individual investors, investment firms, and government or regulatory bodies can extract and classify company ESG information for investment decisions, supervision, and policy development.
• ESG scoring: Since rating agencies use key issues as the basis for scores, the key-issue extraction pipeline points toward more objective, automated ESG scoring — the paper notes that agencies' criteria differ and are undisclosed, producing inconsistent ratings for the same company.
• Information triage in large text volumes: The retrieval pipeline reduces the manual effort of finding issue-relevant passages in the large, diverse stream of corporate disclosures.

Industry relevance. The work sits directly at the intersection of financial NLP and sustainability reporting, two areas where practitioners already need scalable text processing. The finding that a finance-domain model (KB-ALBERT, 9M parameters) matched larger general models is practically relevant for organizations with constrained compute.

Future Directions

• Extend beyond plain text. The dataset currently excludes tables and graphs inside sustainability reports, which the authors acknowledge also contain crucial ESG information; expanding to multi-modal content is stated as planned work.
• Move from extraction to automatic scoring. The paper stops at extracting and classifying ESG information and identifies automatic ESG level scoring as a required next step, framing the current work as its prerequisite.
• Test larger language models. The authors chose BERT-family models for resource accessibility and cite data-privacy concerns for corporate internal use of LLMs; comparing against models such as Llama is left open.
• Generalize the construction process across languages and countries. The paper positions its pipeline as a template for ESG datasets in other languages, but does not itself demonstrate a second language or a cross-lingual transfer result.

Target Audience

• NLP researchers working on domain-specific or low-resource-language datasets, who want a documented construction protocol and a benchmark to compare against.
• Korean-language NLP practitioners building classification or retrieval systems for corporate or financial text.
• ESG and sustainable-finance analysts, data providers, and rating-agency technologists interested in automating information extraction and eventually scoring.
• Corporate sustainability and disclosure teams, especially at SMEs, who need tooling to handle ESG reporting at low cost.
• Regulators and policy researchers examining how objective, comparable ESG information could be extracted and supervised across companies.

Authors’ abstract

With the expansion of pre-trained language model usage in recent years, the importance of datasets for performing tasks in specialized domains has significantly increased. Therefore, we have built a Korean dataset called ESG-Kor to automatically extract Environmental, Social, and Governance (ESG) information, which has recently gained importance. ESG-Kor is a dataset consisting of a total of 118,946 sentences that extracted information on each ESG component from Korean companies' sustainability reports and manually labeled it according to objective rules provided by ESG evaluation agencies. To verify the effectiveness and applicability of the ESG-Kor dataset, classification performance was confirmed using several Korean pre-trained language models, and significant performance was obtained. Additionally, by extending the ESG classification model to documents of small and medium enterprises and extracting information based on ESG key issues and in-depth analysis, we demonstrated potential and practical use cases in the ESG field.