SciPostLayoutTree: A Dataset for Structural Analysis of Scientific Posters

AI summary

Overview

Research area: Computer Vision, specifically document/poster layout structure analysis (document structure analysis, or DSA) and multimodal relation prediction.

Technical level: Intermediate. The paper combines dataset construction, tree-structure prediction, and beam search decoding; the concepts are approachable, but familiarity with layout analysis, transformer encoders, and tree edit distance metrics helps.

One-sentence scope: The paper introduces a dataset of roughly 8,000 scientific posters annotated with reading order and parent-child relations as DFS-ordered trees, and proposes an extended relation-prediction model with bounding-box features and beam search that improves accuracy on spatially challenging relations.

What This Paper Is About

Structural analysis research has mostly targeted textual documents, leaving scientific posters — a major medium of academic communication — largely unexamined. Posters present structures that documents do not, such as reading transitions that move upward or horizontally and parent-child links that span long distances. The authors build a poster dataset with tree-structured annotations and develop a baseline model, called Layout Tree Decoder, that predicts reading order and parent-child relations for posters.

Key Contributions

1. SciPostLayoutTree dataset: approximately 8,000 scientific posters annotated with reading order and parent-child relations, represented as DFS-ordered trees, built by extending the SciPostLayout dataset (which contains BBox annotations for approximately 8,000 posters collected from the web). After excluding posters containing only a single BBox, 7,849 annotated posters remain, split 6,853 / 498 / 498 into training, validation, and test sets.
2. Characterization of poster structure as a novel challenge: a comparative statistical analysis against DocHieNet shows the dataset contains more upward, horizontal, and long-distance relations, which are infrequent in documents.
3. Layout Tree Decoder: an extension of the Document Relation Graph Generator (DRGG) by Chen et al. (2025) that adds a BBox Embedding (position and category information) and beam search to capture sequence-level plausibility.
4. Empirical evidence that these two components are complementary: BBox Embedding mainly improves parent-child prediction, beam search mainly improves reading order prediction, and their combination (DRGG-BEBS) achieves the highest scores across all metrics and across five visual backbones.

Main Findings

• The dataset is larger and more visually oriented than document benchmarks: posters contain approximately 25 BBoxes on average versus approximately 12 for DocHieNet documents, and figures are more numerous. The overall category statistics are: Title 7,842 (1.00 ± 0.04 per poster), Author Info 7,692 (0.98 ± 0.15), Section 41,321 (5.26 ± 2.43), Text 52,725 (6.72 ± 4.89), List 23,761 (3.03 ± 3.31), Figure 38,625 (4.92 ± 3.62), Table 5,655 (0.72 ± 1.26), Caption 14,966 (1.91 ± 2.66), and 192,587 BBoxes in total (24.54 ± 9.58 per poster).
• Trees are shallower and wider than in documents: SciPostLayoutTree has a mean tree depth of 3.37 (± 0.56), tree width of 15.24 (± 7.49), and 0.96 (± 2.47) children per node, compared with DocHieNet's 3.16 (± 0.90), 9.41 (± 6.60), and 0.93 (± 2.78). The number of children per node follows a skewed distribution, so models must often order many children sharing the same parent.
• Reading order is not constrained to a single direction: as in DocHieNet, most transitions go downward over short distances, but the poster dataset contains more transitions toward Top and Top-Right regardless of distance, plus frequent horizontal transitions to Right and Left.
• Parent-child relations are more directionally diverse and more non-local: relations are primarily downward, but upward and horizontal relations are more common than in DocHieNet, and they span longer distances, making parent identification a non-local problem among roughly 25 candidate BBoxes.
• Annotation quality is high: on 100 randomly selected test posters, agreement between the original annotations and independent annotations by two additional annotators was STEDS = 0.91, parent-child pair agreement = 0.97, and reading-order pair agreement = 0.94.
• Model errors are not primarily caused by annotation noise: the Spearman rank correlation between per-poster model STEDS and inter-annotator STEDS is moderate (ρ = 0.40); correlations are high among models and among annotators but low between them.
• Both extensions improve all three overall metrics consistently across backbones (Table 2): for example with ResNet-50, DRGG scores STEDS 68.74, REDS 75.07, TED 8.83; DRGG-BE 84.24 / 86.44 / 4.41; DRGG-BS 76.79 / 83.14 / 6.65; and DRGG-BEBS 88.45 / 90.40 / 3.22. With ViT, DRGG-BEBS reaches 90.04 / 91.73 / 2.78. All marked improvements are significant over DRGG at p < 0.005 by Wilcoxon signed-rank test.
• Beam search helps hardest cases in reading order: improvements are limited for downward and short-distance relations where baseline accuracy is already high, but substantial for upward, horizontal, and long-distance relations. For instance, with InternImage, Top-Right accuracy improves by 3.0 percentage points and the (16, ∞) distance bin by 3.3 points over DRGG-BE.
• BBox Embedding helps hardest cases in parent-child prediction: accuracy gains over DRGG-BS are larger for spatially challenging directions and distances, with large gains such as Top (+32.6 points with DiT), Top-Left (+23.3 points with DiT), and the (16, ∞) distance bin (+28.3 points with ResNet-50).
• Residual failure modes remain: reading order failures account for 7.00% of all predictions and parent-child failures for 3.59%. In parent-child errors the Root node was frequently mispredicted as the parent regardless of the correct parent's direction or distance; most of these involve Text, List, Figure, or Table nodes that should have had a Section as parent, and in 60% of cases the correct Section node was in the tree but not on the rightmost path and thus not a selectable candidate.
• An illustrative hard case is a poster where all Figures and Tables attach as children to the bottom-left Section following a Z-shaped reading order, on which the predicted tree receives a low STEDS of 42.70.
• Two alternatives did not help: models incorporating OCR text features showed no improvement in prediction accuracy, and evaluated VLMs (GPT-5, Gemini 3 Pro) showed lower performance than DRGG-based models.

Methodology in Plain English

The task is defined following Comp-HRDoc: given a set of BBoxes on a poster, each with center position, size, and a category (one of eight: Title, Author Info, Section, Text, List, Table, Figure, Caption), the model predicts a reading-order sequence and, for each BBox, its parent. A virtual Root node representing the poster is always placed first in the reading order, so the predicted sequence has length N + 1, and the two predictions together form a DFS-ordered tree where reading order and hierarchy are structurally consistent.

For annotation, the authors extended SciPostLayout with tree annotations. Scientific posters have an explicit and consistent reading order intended by authors, which allows a unique tree per poster. Professional annotators from an external vendor produced one annotation per poster under a written guideline (Root represents the poster; Title, Author Info, and Section always take the Root as parent; Text, List, Table, and Figure take the Section containing them if one exists, otherwise the Root; Captions take their corresponding Figure or Table; siblings are ordered by reading priority, and the DFS traversal follows that order). All annotations were reviewed by in-house supervisors and the authors, and consistency was measured on 100 randomly selected test posters. Posters with only a single BBox were excluded; 98% of validation and test posters have non-overlapping authors.

The model starts from DRGG, which extracts visual features per BBox using a visual backbone, an FPN, ROI Align, and a Box Head, then applies a relation feature extractor separately for row and column roles. The authors add a BBox Embedding: normalized top-left and bottom-right coordinates are encoded into a position embedding via an MLP, and the category is embedded separately (the Root uses the fixed region (0, 0, 1, 1) and a special "Root" class). Visual, position, and category features are concatenated and passed through a Transformer encoder; an MLP then scores every node pair and produces a subsequent score matrix S and a parent score matrix P. Tree decoding follows the reorganization by Wang et al. (2024): the reading order is built by repeatedly picking the highest-scoring unvisited successor, and parents are chosen from a stack representing the rightmost path from the root to a leaf. Beam search (beam width 20) replaces greedy choices in both the reading-order and parent-selection loops to favor sequences with higher total scores. Training uses cross-entropy losses on S and P, summed into a total loss.

Experiments compare four variants — DRGG, DRGG-BE (BBox Embedding), DRGG-BS (Beam Search), and DRGG-BEBS (both) — across five visual backbones: ResNet-50, ViT, Swin, DiT, and InternImage. Evaluation uses Tree Edit Distance (TED), Semantic Tree Edit Distance Score (STEDS), and Reading Edit Distance Score (REDS), plus per-direction and per-distance accuracies computed as the ratio of ground-truth relations included in the predicted tree (relations involving the Root are excluded since direction and distance are undefined for them).

Why This Matters

Scientific posters are a primary medium of academic communication, and misreading their structure can hinder access to the research they convey. Most structure-analysis research has focused on textual documents, so posters — with their wider layouts, more figures, and more frequent upward, horizontal, and long-distance relations — represent a distinct and underexplored technical challenge. A public dataset and a strong baseline make this problem measurable and comparable for future work. The authors also frame reliable structural analysis as a basis for information-access technologies and for layout assessment, where discrepancies between model predictions and designer intentions could flag confusing layouts.

Real-world applications:

• Structure-aware text-to-speech synthesis that reads a poster's content in the correct order.
• Question answering over posters, where answers depend on knowing which text belongs under which section.
• Retrieval using a table of contents or outline derived from predicted hierarchy.
• Layout assessment for authors and designers, using predicted-versus-intended structure differences as a signal of a confusing poster.

Industry relevance: the work comes from OMRON SINIC X Corporation, and the dataset and code are publicly released on Hugging Face and GitHub, making it directly usable by teams building document AI, accessibility, publishing, and conference-support tools. The finding that performance gains hold across five different visual backbones suggests the approach is not tied to a single architecture choice, which matters for practical deployment.

Future Directions

• Reducing systematic directional and distance bias: error analyses showed that even beam-search models frequently mispredicted reading orders biased toward downward and short-distance relations, so new decoding strategies are needed.
• Capturing structural organization rather than only pairwise relations: 60% of parent-child errors involved a correct Section node that was present in the tree but excluded from the selectable rightmost-path candidates, which the authors say is difficult to fix in later decoding steps.
• Handling semantically grouped, long-sequence layouts: the low-STEDS Z-shaped example suggests a need to capture semantic grouping and structural plausibility over longer sequences.
• Improving baseline components further: OCR text features gave no improvement, and evaluated VLMs (GPT-5, Gemini 3 Pro) underperformed the DRGG-based models, leaving open how text and large vision-language models could be used effectively for this task.

Target Audience

Researchers and practitioners in document layout analysis, computer vision, and multimodal document understanding who are interested in benchmarks and baselines for hierarchical structure prediction. It is also relevant to accessibility and information-retrieval engineers building structure-aware interfaces for posters, and to scientists interested in how the structure of their own posters is machine-interpretable.

Authors’ abstract

Scientific posters play a vital role in academic communication by presenting ideas through visual summaries. Analyzing reading order and parent-child relations of posters is essential for building structure-aware interfaces that facilitate clear and accurate understanding of research content. Despite their prevalence in academic communication, posters remain underexplored in structural analysis research, which has primarily focused on papers. To address this gap, we constructed SciPostLayoutTree, a dataset of approximately 8,000 posters annotated with reading order and parent-child relations. Compared to an existing structural analysis dataset, SciPostLayoutTree contains more instances of spatially challenging relations, including upward, horizontal, and long-distance relations. As a solution to these challenges, we develop Layout Tree Decoder, which incorporates visual features as well as bounding box features including position and category information. The model also uses beam search to predict relations while capturing sequence-level plausibility. Experimental results demonstrate that our model improves the prediction accuracy for spatially challenging relations and establishes a solid baseline for poster structure analysis. The dataset is publicly available at https://huggingface.co/datasets/omron-sinicx/scipostlayouttree. The code is also publicly available at https://github.com/omron-sinicx/scipostlayouttree.