DimABSA: Building Multilingual and Multidomain Datasets for Dimensional Aspect-Based Sentiment Analysis

AI summary

Overview

Research area: Natural Language Processing — aspect-based sentiment analysis (ABSA), dimensional/affective sentiment representation, multilingual and multidomain dataset construction, and large language model (LLM) benchmarking.

Technical level: Intermediate. The paper is accessible to readers familiar with basic sentiment classification and standard ABSA task formulations, but it assumes comfort with evaluation metrics, regression, and LLM fine-tuning concepts.

Scope: The paper introduces DimABSA, a multilingual, multidomain ABSA resource annotated with continuous valence–arousal (VA) scores, defines three dimensional ABSA subtasks and a unified evaluation metric, and benchmarks prompted and fine-tuned LLMs on it.

What This Paper Is About

Existing ABSA research labels sentiment with coarse categories such as positive, negative, or neutral, which cannot distinguish subtle affective differences like good versus excellent, or very rude versus a bit far. This paper replaces those categorical labels with continuous valence (negative–positive) and arousal (sluggish–excited) scores on a 1–9 scale, and builds a manually annotated multilingual, multidomain resource that pairs VA scores with traditional ABSA elements (aspect terms, aspect categories, opinion terms). The goal is to establish dimensional ABSA as a task with datasets, subtask definitions, a joint metric, and LLM baselines.

Key Contributions

1. The DimABSA datasets. The first multilingual, dimensional ABSA resource manually annotated with both traditional ABSA elements and VA scores, containing 76,958 aspect instances across 42,590 sentences, spanning six languages (Chinese, English, Japanese, Russian, Tatar, Ukrainian) and four domains, organized as 10 datasets.
2. Three new subtasks bridging categorical and dimensional ABSA: Dimensional Aspect Sentiment Regression (DimASR), Dimensional Aspect Sentiment Triplet Extraction (DimASTE), and Dimensional Aspect Sentiment Quadruplet Prediction (DimASQP).
3. A unified metric, continuous F1 (cF1), which folds VA prediction error into the standard F1 formulation so that categorical and continuous outputs can be scored in a single measure.
4. A comprehensive LLM benchmark covering zero-/few-shot prompting of closed-source models and supervised fine-tuning of open models across all subtasks, languages, and domains.

Main Findings

• Task definitions and metrics. DimASR (text + aspects → V#A) is a regression task measured with RMSE. DimASTE (text → A, O, V#A) combines extraction and regression, and DimASQP (text → A, C, O, V#A) adds aspect-category classification, both measured with cF1.
• Dataset scale and composition. 76,958 aspect instances across 42,590 sentences, covering the restaurant (rest), laptop (lap), hotel (hot), and finance (fin) domains, using sources including ACOS, the Yelp Open Dataset, Amazon Reviews 2023, Rakuten Travel, chABSA, EDINET, SemEval-2016, SIGHAN-2024, Google Reviews, PTT, Mobile01, and MOPS.
• Translation-based low-resource extension. Tatar and Ukrainian data were produced by automatically translating the Russian dataset with Yandex Translate, with manual corrections applied to 45.5% of translated instances for Tatar and 35.6% for Ukrainian.
• Annotation agreement. Tuple-level F1 is highest for single aspect terms and lower for finance-domain data; the reported finance F1 values are 0.761 for jpn-fin and 0.604 for zho-fin (aspect-only), versus 0.892/0.778/0.703 for eng-rest on (A)/(A,C)/(A,C,O). Annotators consistently agree less on arousal than on valence.
• Long-tailed category distributions. In the restaurant domain, 18 categories exist and the top five cover 97.89% of aspect instances while the remaining 13 cover 2.11%. The laptop domain has 148 categories, with the top five covering 37.20% and the top ten 54.18%. The hotel domain has 47 categories, with the top five covering 51%.
• U-shaped VA distributions. Across datasets, arousal is lowest around neutral valence and increases toward both negative and positive extremes. Chinese and Japanese show more compact VA distributions with lower dispersion, and the finance domain shows a more constrained arousal distribution.
• Benchmark results — DimASR (RMSE, lower is better). Average scores: GPT-5 mini zero-shot 2.7595, Kimi K2 Thinking zero-shot 2.3441, GPT-5 mini one-shot 2.1549, Kimi K2 Thinking one-shot 1.8731, Qwen3 14B 2.1889, Ministral-3 14B 2.3149, Llama-3.3 70B 2.5674, GPT-OSS 120B 1.1924. The fine-tuned 120B model gives the substantial and consistent improvement; the 14B and 70B fine-tuned models do not consistently beat prompting baselines.
• Benchmark results — DimASTE (cF1, higher is better). Average scores: 0.3528 (GPT-5 mini zero-shot), 0.3792 (Kimi K2 Thinking zero-shot), 0.3481 (GPT-5 mini one-shot), 0.3859 (Kimi K2 Thinking one-shot), 0.2875 (Qwen3 14B), 0.1784 (Ministral-3 14B), 0.4640 (Llama-3.3 70B), 0.4571 (GPT-OSS 120B). Closed-source models perform best on English, followed by Russian and Ukrainian; Tatar remains the lowest-performing language even after fine-tuning.
• Benchmark results — DimASQP (cF1, higher is better). Average scores drop across all models: 0.2254, 0.2542, 0.2340, 0.2695 for the closed-source settings, 0.1451 (Qwen3 14B), 0.0989 (Ministral-3 14B), 0.3862 (Llama-3.3 70B), 0.3727 (GPT-OSS 120B). The laptop domain shows a significant decrease, attributed to its larger and more diverse category set, while the restaurant domain drops less sharply.
• Kimi K2 Thinking versus GPT-5 mini. Kimi-K2-Thinking achieves higher average performance than GPT-5 mini under both zero-shot and one-shot settings.
• Few-shot behavior. DimASR benefits immediately from one example because a single example calibrates the continuous VA scale, whereas DimASTE and DimASQP show unstable early performance. Performance generally plateaus around 32 shots, and up to 256 shots few-shot prompting still underperforms fine-tuned benchmarks (GPT-OSS 120B for DimASR; Llama-3.3 70B for DimASTE and DimASQP) on all datasets except English.
• VA output distributions. Zero-shot prompting produces a random grid-like VA distribution, one-shot immediately begins to align with the gold distribution, and higher shot counts (up to 64 and 256) produce distributions that resemble the gold standard more closely, although visual alignment does not continue to improve performance after saturation.
• Categorical comparison. Partitioning the data into positive (V>5.5), neutral (4.5<=V<=5.5), and negative (V<4.5) subsets shows the fine-grained DimABSA data are more challenging than the categorical counterparts, while the conversion yields a new multilingual categorical ABSA dataset.
• Community uptake. The dataset was used for Track A of SemEval-2026 Task 3, attracting over 300 participants. No further breakdown of participant numbers is reported.

Methodology in Plain English

The authors assembled data rather than training a new architecture. They took four widely studied ABSA domains — restaurant, laptop, hotel, and finance — across four high-resource languages (English, Japanese, Russian, Chinese), and extended coverage to two lower-resource languages (Tatar, Ukrainian) by machine-translating the Russian data and having native speakers review and correct it. Existing labeled datasets were used only for training, while newly curated reviews and reports were annotated and split into train, development, and test sets.

Annotation proceeded in two phases. First, annotators extracted categorical tuples — the aspect term, its aspect category, and the opinion term — following guidelines that required extracting all valid tuples, including sentiment modifiers such as very good or a little slow, and matching spans exactly. Each sentence was annotated independently by two annotators; a third adjudicated disagreements, and instances without consensus among the three were discarded. Second, VA ratings were collected on a 1–9 scale using the Self-Assessment Manikin (SAM) scale with VA emojis as references. Each tuple was rated by five annotators, and the final rating discarded outliers beyond the mean ± 1.5 standard deviations before averaging.

The authors then defined three subtasks and evaluated them with two model families: closed-source models accessed via API for zero-shot and few-shot prompting (GPT-5 mini, Kimi K2 Thinking), and open models fine-tuned with 4-bit QLoRA (Qwen3 14B, Ministral-3 14B, Llama-3.3 70B, GPT-OSS 120B). Fine-tuning used the AdamW optimizer with a linear learning rate scheduler, a learning rate of 2e-5, batch size 4, 5 training epochs, and NVIDIA H200 GPUs. For evaluation, DimASR uses RMSE over the VA space, while DimASTE and DimASQP use cF1, in which a prediction first has to match all categorical elements exactly, after which its credit is reduced by the normalized Euclidean distance between predicted and gold VA (normalized by the maximum possible distance in the VA space).

Why This Matters

Impact on research. The paper shifts ABSA from discrete polarity labels to a continuous affective representation, provides the first multilingual and multidomain dimensional ABSA resource, and supplies a metric (cF1) that other researchers can reuse whenever a task mixes categorical extraction with continuous regression. It also establishes baselines showing that these subtasks remain challenging, particularly for low-resource languages such as Tatar and for tasks requiring aspect-category classification.

Real-world applications:

• Product and service review analytics that need to separate mildly positive feedback from strongly positive feedback, or mild annoyance from strong anger, rather than collapsing both into one label.
• Financial reporting analysis, where the finance domain is included and VA scores are annotated for aspects such as investment securities and profit-related terms.
• Mental-health-adjacent and social-media monitoring, since the VA framework has been applied to stance detection, misinformation identification, biosocial markers for mental health, and emotion dynamics analysis.
• Cross-lingual customer-experience monitoring for companies operating in Chinese, English, Japanese, Russian, Tatar, and Ukrainian markets.

Industry relevance. The authors expressly forbid commercial use of the data, so direct commercial deployment is restricted, but the benchmark and metric are directly usable for evaluating vendor sentiment systems, and the categorical version of the dataset offers a standard multilingual ABSA resource for teams that need conventional polarity labels.

Future Directions

• Cross-cultural measurement invariance. The limitations section notes that interpretations of valence and arousal vary across cultures, which affects cross-lingual comparability; testing measurement invariance and expanding language coverage are named as important future work.
• Improving token-based VA prediction. The authors identify the constraints of token-based VA prediction in regression as a key remaining limitation of current LLMs on DimASR.
• Hybrid extraction–regression robustness. Persistent difficulty with the joint extraction and regression tasks of DimASTE and DimASQP, especially for low-resource languages, is flagged as a central open challenge.
• Scaling low-resource language performance. Tatar remains the lowest-performing language even after fine-tuning, raising the question of how to handle languages whose structural properties are difficult for pretrained models to capture.

Target Audience

Researchers and practitioners in sentiment analysis and affective computing who need multilingual, fine-grained resources; dataset and benchmark builders interested in annotation protocols and agreement measurement; LLM evaluation researchers looking for challenging structured-prediction tasks; and industry teams working on multilingual review or customer-feedback analytics who want to understand how far current models get on dimensional sentiment tasks.

Authors’ abstract

Aspect-Based Sentiment Analysis (ABSA) focuses on extracting sentiment at a fine-grained aspect level and has been widely applied across real-world domains. However, existing ABSA research relies on coarse-grained categorical labels (e.g., positive, negative), which limits its ability to capture nuanced affective states. To address this limitation, we adopt a dimensional approach that represents sentiment with continuous valence-arousal (VA) scores, enabling fine-grained analysis at both the aspect and sentiment levels. To this end, we introduce DimABSA, the first multilingual, dimensional ABSA resource annotated with both traditional ABSA elements (aspect terms, aspect categories, and opinion terms) and newly introduced VA scores. This resource contains 76,958 aspect instances across 42,590 sentences, spanning six languages and four domains. We further introduce three subtasks that combine VA scores with different ABSA elements, providing a bridge from traditional ABSA to dimensional ABSA. Given that these subtasks involve both categorical and continuous outputs, we propose a new unified metric, continuous F1 (cF1), which incorporates VA prediction error into standard F1. We provide a comprehensive benchmark using both prompted and fine-tuned large language models across all subtasks. Our results show that DimABSA is a challenging benchmark and provides a foundation for advancing multilingual dimensional ABSA. We publicly released the DimABSA dataset, which was used for Track A of SemEval-2026 Task 3, attracting over 300 participants.