Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields

Yongcheng Jing*
Yang Liu*
Yezhou Yang
Zunlei Feng
Yizhou Yu
Dacheng Tao
Mingli Song

ECCV2018 [Paper]
Code [GitHub]
Supplementary [Supp]



    Content & Style           Small Stroke Size          Middle Stroke Size          Large Stroke Size          Mixed Stroke Size

Example stylized results with different stroke sizes. All these results are produced by one single model in real-time using our proposed algorithm. We did not do any image pre- or post- processing before and after forwarding.

Abstract

The Fast Style Transfer methods have been recently proposed to transfer a photograph to an artistic style in real-time. This task involves controlling the stroke size in the stylized results, which remains an open challenge. In this paper, we present a stroke controllable style transfer network that can achieve continuous and spatial stroke size control. By analyzing the factors that influence the stroke size, we propose to explicitly account for the receptive field and the style image scales. We propose a StrokePyramid module to endow the network with adaptive receptive fields, and two training strategies to achieve faster convergence and augment new stroke sizes upon a trained model respectively. By combining the proposed runtime control strategies, our network can achieve continuous changes in stroke sizes and produce distinct stroke sizes in different spatial regions within the same output image.


Demo Video




Network Architecture




Results of Continuous Stroke Size Control

By interpolating between the output feature maps in the StrokePyramid, our algorithm can achieve arbitrary intermediate stroke sizes. We zoom in on the same region (red frame) to observe the variations of stroke sizes



Results of Spatial Stroke Size Control

Our algorithm allows flexible spatial stroke size control during stylization. The result produced by our single model can have mixed stroke sizes, which is more consistent with an artist’s artwork in reality.





This webpage template was borrowed from Richard Zhang.