Neural Universal Discrete Denoiser

I_05_Taesup Moon(Photo)
Taesup Moon

Abstract :

In this talk, I will present a novel framework of applying deep neural network (DNN) to discrete denoising problem. DNN has recently shown remarkable performance improvements in diverse applications, and most of the success are based on the supervised learning framework. While successful in many applications, it is not straightforward to apply such framework to the discrete denoising problem, in which a denoiser tries to estimate an unknown finite-valued clean data based on its noisy observation. The reason is because the ground-truth label for a denoiser is the clean data subject to the estimation and is clearly not available for training a denoiser. In this talk, I follow the framework of DUDE (Discrete Universal DEnoiser) and devise a novel way of training a DNN as a discrete denoiser without any ground-truth labels. The key idea is to define “pseudo-labels” based on an unbiased estimate of the true loss of a denoiser and use them as targets for DNN parameters to optimize for. The resulting scheme is dubbed as Neural DUDE, and our experiments show that Neural DUDE significantly outperforms the original DUDE, which is the state-of-the-art on several discrete denoising problems. Furthermore, I will show that Neural DUDE overcomes the critical limitation of DUDE, namely, it is much more robust to the choice of the hyper-parameter and has a concrete way of choosing the best hyper-parameter for given data. Such property becomes an attractive feature of Neural DUDE in practice. Finally, I will conclude with some potential future research directions, such as extending the framework to the denoising of continuous-valued data.

This work will also be presented at NIPS 2016 and is a joint work with Seonwoo Min, Byunghan Lee and Sungroh Yoon (SNU).


Research Interests

  • Machine Learning / Deep Learning
  • Data Science
  • Signal Processing
  • Information Theory


Professional Experience

  • 2015 ~ present, Assistant Professor, DGIST, Daegu, Korea (http://mind.dgist.ac.kr)
  • 2013 ~ 2015, Research Staff Member, Samsung Advanced Institute of Technology (SAIT), Suwon, Korea
  • 2012 ~ 2013, Postdoctoral Scholar, Department of Statistics, University of California at Berkeley, Berkeley, CA, USA
  • 2008 ~ 2012, Scientist, Yahoo! Labs, Sunnyvale, CA, USA



  • 2008 Ph.D. in Electrical Engineering, Stanford University, Stanford, CA
  • 2004 M.S. in Electrical Engineering, Stanford University, Stanford, CA
  • 2002 B.S. in Electrical Engineering, Seoul National University, Seoul, Korea


Awards & Honors

  • 2016~2019, NRF Young Researcher Program Award (300M KRW / 3 years)
  • 2004-2008, Samsung Scholarship
  • 2002-2004, Full Scholarship from The Korean Foundation for Advanced Studies
  • 2002, AT&T Asia Pacific Leadership Award
  • 1998-2002 General Electric (GE) Scholarship