Precise Top-Layer Fabric Segmentation for Fabric Destacking with Edge- and Shape-Aware Deep Networks

AI summary

Overview

• Research area: Computer vision for robotics — semantic segmentation of the topmost fabric layer in stacked fabric piles (fabric destacking) for garment automation.
• Technical level: Advanced. The paper assumes familiarity with encoder-decoder segmentation networks, multi-branch/multi-task training with auxiliary supervision, and edge/shape-based losses. The writing is accessible, but the method builds on ResNet50 backbones, Dice loss, and gated/edge-aware segmentation literature.
• Scope: The paper proposes a training architecture with two auxiliary supervision branches (edge-aware and shape-aware) added to a standard encoder-decoder segmentation backbone, and evaluates it on a real collar-fabric dataset of 235 annotated images.

What This Paper Is About

When fabrics are stacked for garment manufacturing, a robot needs to know exactly which pixels belong to the top layer so it can grab and separate that single piece. This is hard because the boundary between one fabric layer and the next is often faint or irregular, and because stacked fabrics are usually similar in color and texture to each other. The paper's goal is to train a segmentation network that predicts the top-layer fabric mask accurately, both at the pixel level and in overall shape.

Key Contributions

1. An edge-aware branch for training supervision. A parallel branch takes the decoder's highest-resolution output, produces a single-channel predicted edge probability map via a 1×1 convolution and sigmoid, and is supervised by ground-truth edge masks derived from the segmentation masks. This targets the blurry, irregular fabric boundaries that standard segmentation methods handle poorly.
2. A shape-aware branch that uses CAD-derived reference shapes. The predicted segmentation probability map is concatenated with a binary reference shape mask generated from the CAD model, passed through a lightweight CNN, then a fully connected layer and sigmoid, producing a scalar probability of whether the prediction aligns with the ideal shape. The branch is pre-trained as an independent binary classifier on synthetic mask pairs before joint fine-tuning.
3. A composite training loss. The total loss is the segmentation loss plus weighted edge and shape losses (L_total = L_seg + λ_edge·L_edge + λ_shape·L_shape), where L_seg combines pixel-wise cross-entropy and Dice loss, and both L_edge and L_shape are binary cross-entropy terms.
4. An ablation study on a real fabric dataset quantifying the contribution of each branch in terms of IoU, Pixel Accuracy, and Edge Root Mean Squared Error.

Notably, only the segmentation backbone is used at inference; the two branches exist purely to shape the backbone's weights during training.

Main Findings

• Dataset used: Experiments ran on a real collar fabric dataset of 235 images with pixel-level annotations, each accompanied by a semantic label mask, a ground-truth edge mask, and an ideal shape label derived from the corresponding CAD model.
• Ablation results (Table I):
  • Baseline (no branches): IoU 93.25%, PA 95.87%, ERMSE 5.03 pixels.
  • Baseline + edge-aware branch: IoU 96.24%, PA 96.72%, ERMSE 3.19 pixels.
  • Baseline + edge-aware + shape-aware branch (proposed): IoU 96.80%, PA 97.50%, ERMSE 2.58 pixels.
• The edge-aware branch is the larger single gain. Adding it raised IoU from 93.25% to 96.24% and PA from 95.87% to 96.72%, while lowering ERMSE from 5.03 to 3.19 pixels, which the authors attribute to sharper and more accurate masks.
• The shape-aware branch adds further improvement on every metric. After the edge branch, it pushed IoU to 96.80%, PA to 97.50%, and produced the lowest ERMSE of 2.58 pixels, indicating better alignment between predicted and ideal shapes.
• The two branches are complementary. The paper states the results confirm both branches are beneficial for accurate segmentation, with the shape branch improving global structure and the edge branch improving boundaries.
• Qualitative behavior: Figure 3 shows the proposed architecture delineating fabric boundaries and preserving overall shape even in regions with ambiguous edges and complex contours.
• Robustness to limited data is claimed in the conclusion, along with outperforming strong baselines; however, the reported table contains only the three ablation configurations, so head-to-head numbers against named external methods such as UNet or GSCNN are not reported in the provided content.
• Training setup: PyTorch, Adam optimizer, initial learning rate 1×10⁻⁴, batch size 8, 300 epochs, early stopping based on validation loss. Hardware was an NVIDIA RTX 3060 GPU (12GB VRAM), Intel Core i9-10900F CPU, 16GB RAM, CUDA 11.7 with cuDNN.
• Evaluation metrics: IoU, Pixel Accuracy (PA), Edge Mean Squared Error (EMSE), and Edge Root Mean Squared Error (ERMSE). EMSE is defined mathematically but only ERMSE values appear in Table I.

Methodology in Plain English

The researchers start from a standard segmentation network shaped like an hourglass: an encoder (a pre-trained ResNet50) shrinks the image into feature maps at four levels of abstraction, and a decoder progressively upsamples them, fusing in high-resolution encoder features through skip connections. The final decoder output is the highest-resolution one, matching the input image size, and a convolution plus sigmoid turns it into a per-pixel probability that each pixel is top-layer fabric.

On top of this backbone, they bolt on two extra heads that exist only during training. The first is an edge head: it looks at that same high-resolution decoder output and predicts where the fabric outlines are, and it is graded against edge masks derived from the human-annotated segmentation masks. This forces the network to care about contours, not just regions. The second is a shape head: it takes the network's own prediction, stacks it with a binary reference silhouette taken from the product's CAD model, and runs a small CNN plus a fully connected layer to output a single number — how likely is it that this predicted shape is the right shape? The shape head is pre-trained separately on synthetic mask pairs first, then fine-tuned together with everything else.

All three objectives are summed into one loss with weighting hyperparameters. At test time, the extra heads are discarded: only the backbone runs, but its weights have been shaped by the extra supervision. The authors then train three versions of the model — backbone alone, backbone plus edge head, and backbone plus both heads — on the 235-image collar dataset and compare IoU, pixel accuracy, and edge distance error.

Why This Matters

• Impact on research: The paper shows a lightweight, task-specific route to improving segmentation of visually ambiguous, layered, deformable objects. Rather than building a heavier architecture, it injects structural knowledge — edges and CAD silhouettes — as training-time supervision and then discards the auxiliary heads, keeping inference identical to the baseline backbone. The ablation isolates how much each type of supervision buys.
• Real-world applications:
  • Robotic pick-and-place of a single fabric piece from a stack in garment assembly lines.
  • Fabric alignment, folding, and sewing automation for flexible fabric components.
  • Industrial destacking tasks where CAD models of parts are already available and can serve as free shape priors.
  • Quality control and grasping-point selection that depend on knowing the exact top-layer outline.
• Industry relevance: Garment production is labor-intensive and depends on consistent handling of floppy, similar-looking materials. A segmentation method that runs on modest hardware (a single RTX 3060 with 12GB VRAM) and requires only 235 annotated real images, with supervision boosted by existing CAD data, is practical for deployment. The authors explicitly state the method will be integrated into their pick-and-place system.

Future Directions

• Deployment in the pick-and-place system. The authors name this directly as future work, to demonstrate versatility and practical applicability.
• Generalization beyond collars. The evaluation covers one collar fabric dataset of 235 images; whether the same edge-and-shape supervision transfers to other garment parts, fabrics with patterns, or non-garment layered materials is an open question.
• Head-to-head comparison with named baselines. The related work discusses UNet, GSCNN, DCAN, CASENet, BiseNet, BASNet, and EGNet, but the reported table contains only the three ablation configurations, so how the method compares to those specific architectures on this dataset is not reported.
• Reducing dependence on CAD models. The shape branch relies on a reference shape mask generated from the CAD model in the image plane, which raises the question of how the method behaves when no CAD model exists or when the fabric deforms relative to its design silhouette.
• Efficiency characterization. Inference latency and throughput on the target robotic system are not reported, which would matter for real-time grasping.

Target Audience

This paper suits researchers and engineers working on robotic manipulation of deformable materials, industrial computer vision for apparel and textile manufacturing, and applied segmentation practitioners who want a concrete example of auxiliary-task supervision (edges plus shape priors) improving a standard encoder-decoder backbone. It is also useful for readers interested in multi-branch training architectures where auxiliary heads are used only during training and dropped at inference. Readers need prior exposure to semantic segmentation and deep network training to follow the methodology fully.

Authors’ abstract

Fabric destacking requires precise segmentation of the topmost fabric layer, a task complicated by subtle fabric boundaries and high visual similarity between fabric layers. Existing semantic and edge-based segmentation approaches often struggle with these complexities, limiting the performance of robotic manipulation for different tasks. In this work, a novel segmentation training architecture tailored for top-layer fabric segmentation in stacked fabrics is proposed. The method extends the classical encoder-decoder framework by introducing two specialized branches - an edge-aware branch and a shape-aware branch - that are used to supervise the backbone network for better tuning. The edge-aware branch enhances boundary delineation, while the shape-aware branch guides the network to capture and align the overall fabric shape with reference masks derived from Computer Aided Design (CAD) models. Experiments on a real-world fabric dataset demonstrate that the training approach outperforms established baselines, verifying the effectiveness of the multi-branch design through both quantitative results and ablation studies.