Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network

Seon Joo Kim
Yonsei University
Nov. 30 15:10~15:40


We present a novel deep learning framework that models the scene dependent image processing inside cameras. Often called as the radiometric calibration, the process of recovering RAW images from processed images (JPEG format in the sRGB color space) is essential for many computer vision tasks that rely on physically accurate radiance values. All previous works rely on the deterministic imaging model where the color transformation stays the same regardless of the scene and thus they can only be applied for images taken under the manual mode. In this paper, we propose a data-driven approach to learn the scene dependent and locally varying image processing inside cameras under the automode. Our method incorporates both the global and the local scene context into pixel-wise features via multi-scale pyramid of learnable histogram layers. The results show that we can model the imaging pipeline of different cameras that operate under the automode accurately in both directions (from RAW to sRGB, from sRGB to RAW) and we show how we can apply our method to improve the performance of image deblurring.



Seon Joo Kim received the B.S. and M.S. degrees from Yonsei University, Seoul, South Korea, in 1997 and 2001, respectively, and the Ph.D. degree in computer science from the University of North Carolina, Chapel Hill, in 2008. He has been an Assistant Professor with the Department of Computer Science, Yonsei University, since 2013. His research interests include computer vision, computer graphics/computational photography, and HCI/visualization.



  • University of North Carolina, Chapel Hill, N.C. USA
    – Ph.D., Computer Science, August 2008
  • Yonsei University, Seoul, Korea
    – M.S., Electrical & Electronic Engineering, February 2001
  • Yonsei University, Seoul, Korea
    – B.S., Electronic Engineering, February 1997


Professional Career

  • 2017.03-Current, Associate Professor, Yonsei University, Dept. of Computer Science
  • 2013.03-2017.02, Assistant Professor, Yonsei University, Dept. of Computer Science
  • 2012.01-2012.11, Assistant Professor, SUNY Korea, Stonybrook University, USA
  • 2011.06-2011.11, Research Scientist, UIUC Advanced Digital Sciences Center, Singapore
  • 2009.10-2011.06, Research Fellow, National University of Singapore, Singapore
  • 2008.09-2009.09, Postdoctoral Researcher, University of North Carolina, Chapel Hill, N.C
  • 2002.08-2008.08, Research Assistant, University of North Carolina, Chapel Hill, N.C
  • 2005.05-08, Intern, GE Global Research Center
  • 2004.05-08, Intern, Cortex Surveillance Automation
  • 2001.05-2002.05, Research Assistant, Signal Processing Center, Yonsei University, Seoul, Korea
  • 1999.05-2001.02, Research Assistant, Yonsei University, Seoul, Korea


Awards and Honors

  • 2013, 2015, Best Teaching Award (Yonsei University)
  • 2016, Bronze Prize in Samsung Humantech Paper Award
  • 2016, Okawa Foundation Research Award
  • 2015, Naver Young Faculty Award (Naver Corp.)
  • 2014, Best Paper Award, Pacigic Graphics
  • 2013, Outstanding Reviewer, CVPR