Sketch Understanding and Applications

SketchyCOCO: Image Generation from Freehand Scene Sketches

Intro: We introduce the first method for automatic image generation from scene-level freehand sketches. Our model allows for controllable image generation by specifying the synthesis goal via freehand sketches. The key contribution is an attribute vector bridged generative adversarial network called edgeGAN which supports high visual-quality image content generation without using freehand sketches as training data. We build a large-scale composite dataset called SketchyCOCO to comprehensively evaluate our solution. We validate our approach on the task of both objectlevel and scene-level image generation on SketchyCOCO. We demonstrate the method’s capacity to generate realistic complex scene-level images from a variety of freehand sketches by quantitative, qualitative results, and ablation studies.

Computer Vision and Pattern Recognition (CVPR) (*oral) , 2020
[Paper] [Code]

Language-based Colorization of Scene Sketches

Intro: This paper proposed the first language-based user-customizable colorization system for scene sketches. We also proposed the first architecture for text-based instance-level segmentation of scene sketches.

ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH Asia 2019) (*oral)
[Paper] [Code]

SketchyScene: Richly-Annotated Scene Sketches

Intro: This paper constructed the first large-scale dataset of scene sketches called SketchyScene. We demonstrate the potential impact of SketchyScene by training new computational models for semantic segmentation of scene sketches.

European Conference on Computer Vision (ECCV), 2018
[Paper] [Code]

Non-Photorealistic Rendering

PencilArt: A Chromatic Penciling Style Generation Framework

Chengying Gao, Mengyue Tang, Xiangguo Liang, Zhou Su, Changqing Zou

Computer Graphics Forum (CGF), 2018

Color Restoration and Transfer

An edge-refined vectorized deep colorization model for grayscale-to-color images

Zhuo Su, Xiangguo Liang, Jiaming Guo, Chengying Gao, Xiaonan Luo

Neurocomputing, 2018

L0 Gradient-Preserving Color Transfer

Dong Wang, Changqing Zou, Guiqing Li, Chengying Gao, Zhuo Su, Ping Tan

Computer Graphics Forum (CGF), 2017

Image Inpainting

Data-Driven Image Completion for Complex Object

Chengying Gao, Yanmei Luo, Hefeng Wu*, Dong Wang

Signal Processing: Image Communication, 2017




Back to top