[2106.13804] Single Image Texture Translation for Data Augmentation

Significance

Image-to-image translation with single source-reference pair

Keypoints

  • Propose a method for single image texture translation
  • Demonstrate use of propose method for data augmentation by experiments

Review

Background

Image-to-image translation has been a key application of deep neural networks. CycleGAN is one of the most successful model for this task, which can translate a horse image to a zebra, or a photo into a painting by Monet. However, a trained CycleGAN can map one-to-one relationship between the two domains, meaning that it requires re-training the model to translate a source image to another reference image. The authors try to address this issue and propose an image translation method with single source-reference pair. Fast image translation enables the use of the proposed framework as a data augmentation method, which is shown by experiments.

Keypoints

Propose a method for single image texture translation

The proposed Single Image Texture Translation (SITT) method consists of a pair of encoder-decoders for encoding and decoding the content (source) and the texture (reference) of the input images. The encoder-decoders are trained with adversarial loss, cycle-consistency loss (or latent regression loss), VGG19 perceptual loss, and the KL divergence loss. 210628-1 Schematic illustration of the proposed method For the content encoder, positional norm is employed to better preserve the structural information by re-injecting the information to the decoder.

Demonstrate use of propose method for data augmentation by experiments

Quality of the image translation is evaluated by FID, LPIPS, and the VGG loss scores compared with baseline models including ArtStyle, CycleGAN, SinGAN, FUNIT, and TuiGAN. 210628-2 Quantitative comparison of image translation quality 210628-4 Qualitative comparison of image translation quality 210628-3 Computation time comparison by sec/iter It can be seen that the proposed SITT outperforms baseline methods while being faster.

Owing to its speed and quality, the authors exploit SITT as a data augmentation method for challenging datasets such as the Plant Pathology 2020. 210628-5 Exemplar images of data augmentation for Plant Pathology dataset 210628-6 Top-1 classification result of healthy/sick leaf classification on Plant Pathology dataset

The authors show that augmentation with SITT works complementarily with other augmentation methods 210628-7 SITT works complementarily with other image augmentations Results of few-shot image classification and SITT augmentation on other in-the-wild datasets are referred to the original paper.

Related

Share

Comment

#image-generation #multi-modal #language-model #retrieval-augmentation #robotics #forecasting #psychiatry #instruction-tuning #diffusion-model #notice #graph-neural-network #responsible-ai #privacy-preserving #scaling #mixture-of-experts #generative-adversarial-network #speech-model #contrastive-learning #self-supervised #image-representation #image-processing #object-detection #pseudo-labeling #scene-text-detection #neural-architecture-search #data-sampling #long-tail #graph-representation #zero-shot #metric-learning #federated-learning #weight-matrix #low-rank #vision-transformer #computer-vision #normalizing-flow #invertible-neural-network #super-resolution #image-manipulation #thread-summarization #natural-language-processing #domain-adaptation #knowledge-distillation #scene-text #model-compression #semantic-segmentation #instance-segmentation #video-understanding #code-generation #graph-generation #image-translation #data-augmentation #model-pruning #signal-processing #text-generation #text-classification #music-representation #transfer-learning #link-prediction #counterfactual-learning #medical-imaging #acceleration #transformer #style-transfer #novel-view-synthesis #point-cloud #spiking-neural-network #optimization #multi-layer-perceptron #adversarial-training #visual-search #image-retrieval #negative-sampling #action-localization #weakly-supervised #data-compression #hypergraph #adversarial-attack #submodularity #active-learning #deblurring #object-tracking #pyramid-structure #loss-function #gradient-descent #generalization #bug-fix #orthogonality #explainability #saliency-mapping #information-theory #question-answering #knowledge-graph #robustness #limited-data #recommender-system #anomaly-detection #gaussian-discriminant-analysis #molecular-graph #video-processing