Defect Spectrum: A Granular Look of Large-Scale Defect Datasets with Rich Semantics
ECCV 2024

Shuai Yang *
HKUST(GZ)		 Zhifei Chen *
HKUST(GZ)		 Pengguang Chen
Smartmore		 Xi Fang
Smartmore
Yixun Liang
HKUST(GZ)		 Shu Liu
Smartmore		 Yingcong Chen
HKUST(GZ)

  • Paper

  • Dataset

  • Code

Abstract

Defect inspection is paramount within the closed-loop manufacturing system. However, existing datasets for defect inspection often lack the precision and semantic granularity required for practical applications. In this paper, we introduce the Defect Spectrum, a comprehensive benchmark that offers precise, semantic-abundant, and large-scale annotations for a wide range of industrial defects. Building on four key industrial benchmarks, our dataset refines existing annotations and introduces rich semantic details, distinguishing multiple defect types within a single image. With our dataset, we were able to achieve an increase of 10.74% in the Recall rate, and a decrease of 33.10% in the False Positive Rate (FPR) from the industrial simulation experiment. Furthermore, we introduce Defect-Gen, a two-stage diffusion-based generator designed to create high-quality and diverse defective images, even when working with limited defective data. The synthetic images generated by Defect-Gen significantly enhance the performance of defect segmentation models, achieving an improvement in mIoU scores up to 9.85 on Defect-Spectrum subsets.

overview

Dataset Reannotation

Current industrial datasets often lack the granularity needed for detailed defect inspection, with many offering only binary masks or occasionally misclassifying defects. We introduce the Defect Spectrum to provide detailed, large-scale annotations for a wide range of industrial defects, enhancing the precision of defect inspection systems. Drawing from four industrial benchmarks, Defect Spectrum refines annotations, ensuring accurate representation of subtle defects and filling in missed ones. Our dataset uniquely offers rich semantic annotations, identifying multiple defect types in one image. Additionally, we include descriptive captions for each sample, paving the way for future Vision Language Model studies.
The annotation comparison in the MVTec dataset. The first row shows the defect image. Rows 2 and 3 show the original annotation and our improved annotation.
The annotation comparison in the VISION dataset. The first row shows the defect image. Rows 2 and 3 show the original annotation and our improved annotation.

Defect-Click

Pixel mask annotation, especially under Defect Spectrum standards, is a demanding task. We introduce "Defect-Click," an advanced interactive tool that automatically segments defects based on user clicks, leveraging pretrained knowledge of industrial defects. Using Defect-Click, we achieve a 60% time-saving in annotation, though the project still took 580 working hours. This tool significantly streamlines the defect annotation process in the industrial domain.

Defect-Gen

Many industrial datasets suffer from a limited number of defective samples, even for the most extensive datasets like VISION. To combat this deficiency, we introduce "Defect-Gen," a two-stage diffusion-based generator. This generative model excels in producing diverse and high-quality images, even when trained on limited data.
We demonstrate the performance boost achieved using our synthetic data in the following figure. It illustrates how varying synthetic data quantities affect DeepLabV3+(a) and MiT-B0(b) performance. Notably, transformer-based models like MiT-B0 benefit more from synthetic data than CNN-based models.

Citation