Paper

  • Z. Wang, J. Yang, H. Jin, E. Shechtman, A. Agarwala, J. Brandt and T. Huang, “DeepFont: Identify Your Font from An Image”, ACM International Conference on Multimedia (ACM MM) , 2015. [Full Paper].
  • Z. Wang, J. Yang, H. Jin, J. Brandt, E. Shechtman, A. Agarwala, Z. Wang, S. Kong, and T. Huang, “DeepFont: A System for Font Recognition and Similarity”, ACM International Conference on Multimedia (ACM MM) , 2015. [Tech Demo]
  • Z. Wang, J. Yang, H. Jin, E. Shechtman, A. Agarwala, J. Brandt and T. Huang, “Real-World Font Recognition Using Deep Network and Domain Adaptation”, International Conference on Learning Representations (ICLR) workshop, 2015
  • Z. Wang et. al., "Font Recognition and Font Similarity Learning Using A Deep Neural Network", US Patent No. 9501724. [USPTO Link]


Abstract

As font is one of the core design concepts, automatic font identification and similar font suggestion from an image or photo has been on the wish list of many designers. We study the Visual Font Recognition (VFR) problem, and advance the state-of-the-art remarkably by developing the DeepFont system. First of all, we build up the first available large-scale VFR dataset, named AdobeVFR, consisting of both labeled synthetic data and partially labeled real-world data. Next, to combat the domain mismatch between available training and testing data, we introduce a Convolutional Neural Network decomposition approach, using a domain adaptation technique based on a Stacked Convolutional Auto-Encoder (SCAE) that exploits a large corpus of unlabeled real-world text images combined with synthetic data preprocessed in a specific way. Moreover, we study a novel learning-based model compression approach, in order to reduce the DeepFont model size without sacrificing its performance. The DeepFont system achieves an accuracy of higher than 80% (top-5) on our collected dataset, and also produces a good font similarity measure for font selection and suggestion. We also achieve around 6 times compression of the model without any visible loss of recognition accuracy.


Tutorial and Demo





























  • Technical Tutorial (click to view): presented in NVidia GPU Technology Conference (GTC), 2015, by my collaborator Dr. Jianchao Yang.
  • A more recent DeepFont tutorial, presented in NVidia GPU Technology Conference (GTC), 2016,  by my internship mentor, Dr. Hailin Jin.
  • See more related in the Press page.



Dataset

[Click to Download AdobeVFR Dataset]

  • The AdobeVFR dataset is the first large-scale benchmark set (2383 fonts) consisting of both synthetic and real-world text images, for the task of font recognition.
  • The AdobeVFR dataset is super fine-grain, with highly subtle categorical variations, leading itself to a new challenging dataset for object recognition.
  • The substantial domain mismatch between synthetic and real-world data makes the AdobeVFR dataset an ideal subject for domain adaptation and transfer learning research.
  • The AdobeVFR dataset also promotes the new problem area of understanding design styles with deep learning.



Implementation

  • We recommend using cuda-convnet as well as anna package (developed by UIUC IFP for training deconvolutional networks). 



Zhangyang (Atlas) Wang​

  • DeepFont demo in Adobe products.
  • Adobe MAX conference 2015: DeepFont session