Implementation:


  • Unfortunately, the source codes of our own implementations were developed under research grants from industry leaderships. Therefore, they are protected by certain IPs and will not be made publicly available.
  • However, we are more than happy to assist your research-purpose comparison experiments with our SR methods (CSC, EPI, and JSR).
  • We could also help test other competitive SR algorithms (BCI, LSE and IER) mentioned in our papers, using our own implementations. We however cannot guarantee that our implementations are always tuned to their optimal performances (Please refer to original authors).  
  • To obtain our assistance, please send an email to: masterwant@gmail.com, specifying the following information:
    • Your name, affiliation, and purpose (research paper, industrial project, or else)
    • Which method(s) you would like to compare with: CSC, EPI, JSR; BCI, LSR, IER.
    • Your own LR images for testing, and desired SR factor
    • Your own HR groundtruth images (optional)
  • Make sure include "[JSR Implementation]" in email title, in case it is blocked by the (quite strict) spam filters. If you are not replied in two weeks, please send out another reminder email to make sure your email was successfully delivered into the inbox.
  • All codes are developed by: Jianchao Yang, Zhaowen Wang, Yingzhen Yang, and Zhangyang Wang, in UIUC IFP group. We request your to properly cite our paper(s) if you use SR results supplied by us.


With much progress being made, it is recognized that external and internal SR methods each suffer from their certain drawbacks. 

  • External SR methods produce plausible image appearances, but no guarantee that an arbitrary input patch can be well matched or represented by the external dataset of limited size. When dealing with some unique features that rarely appear in the given dataset, external SR methods are prone to produce either noise or over smoothness. 
  • Internal SR methods search for more relevant references but with a very limited number. Their performances degrade especially for irregular patches without any discernible repeating pattern. The mismatches of internal examples often lead to more visual artifacts.


However, their complementary properties inspire us to propose the joint SR,  that adaptively utilizes both external and internal examples for the SR task.  Experimental results demonstrate that joint SR outperforms existing state-of-the-art methods for various test images of different definitions and scaling factors, and is also significantly more favored by user perception. 

Insight:

Super-resolution (SR) algorithms aim to constructing a high-resolution (HR) image from one or multiple low-resolution (LR) input frames. This problem is essentially ill-posed because much information is lost in the HR to LR degradation process. Thus SR has to refer to strong image priors, that range from the simplest analytical smoothness assumptions, to more sophisticated statistical and structural priors learned from natural images. 

Popular SR methods rely on example-based learning techniques. Classical example-based methods learn the mapping between LR and HR image patches, from a large and representative external set of image pairs, and is thus denoted as external SR. Meanwhile, images generally possess a great amount of self-similarities; such a self-similarity property motivates a series of internal SR methods.

Paper:


  • Z. Wang, Y. Yang, Z. Wang, S. Chang, J. Yang, and T. Huang, “Learning Super-Resolution Jointly from External and Internal Examples”, IEEE Transactions on Image Processing (TIP), 2015. 
  • Z. Wang, Y. Yang, Z. Wang, S. Chang, W. Han, J. Yang, and T. Huang, "Self-Tuned Deep Super Resolution",  In IEEE CVPR workshop on Deep Learning in Computer Vision (DeepVision), 2015.
  • Z. Wang, Z. Wang, S. Chang, J. Yang and T. Huang, “A Joint Perspective Towards Image Super-Resolution: Unifying External- and Self-Examples”, In Proceedings of IEEE Winter conference on Applications of Computer Vision (WACV), 2014.

Zhangyang (Atlas) Wang​

Results: