Energy-aware Computational Imaging

  • Authors: R. Rithe, P. Raina, A. P. Chandrakasan
  • Sponsorship: Foxconn

With computational imaging becoming an integral part of photography, enabling this functionality on portable multimedia device is important. Portable device processors lack hardware support for computational imaging. Software-based applications add significant complexity and reduce battery life. This project aims to implement computational photography algorithms on-chip, to enable computational photography on portable multimedia devices, as in Figure 1.

While integrating computational photography is integrated into the camera processor or the processor in a portable multimedia device, power consumption becomes a key concern. This project explores power reduction techniques at different stages of the design, such as algorithmic optimization to reduce computational complexity, architectural optimizations to achieve shared/reconfigurable and parallel-pipelined architecture, and circuit level optimizations to operate at low voltage (VDD ≤ 0.5V).

A bilateral filter [1] , is an edge-preserving filter that takes into account not just the spatial location of pixels but also the intensity variation, is commonly used in several computational photography applications, such as high dynamic range (HDR) imaging [2] , low-light image enhancement [3] [4] , tone management, etc. This project implements a bilateral grid [5] as a core processing unit, as in Figure 2, that is shared among multiple applications, such as HDR imaging, low-light imaging, haze removal, and color enhancement. The grid can be accessed directly to implement other applications that require use of bilateral filtering. The project explores energy-scalable image processing using a scalable bilateral grid budget to achieve energy vs. quality trade-offs. Grid dimensions can vary depending on the energy budget. The design can achieve real-time processing for 10 megapixel images.

  1. C. Tomasi and R. Manduchi, “Bilateral filtering for grey and color images,” IEEE Int. Conference on Computer Vision, pp. 836-846, Jan. 1998. [↩]
  2. F. Durand and J. Dorsey, “Fast bilateral filtering for the display of high-dynamic range imaging,” ACM SIGGRAPH, pp. 257-266, July 2002. [↩]
  3. E. Eisemann and F. Durand, “Flash photography enhancement via intrinsic relighting,” ACM SIGGRAPH, pp. 673-678, Aug. 2004. [↩]
  4. G. Petschnigg, M. Agrawala, H. Hoppe, R. Szeliski, M. Kohen, and K. Toyama, “Digital photography with flash and no-flash image pairs,” ACM SIGGRAPH, pp. 664-672, Aug. 2004. [↩]
  5. J. Chen, S. Paris, and F. Durand, “Real-time edge-aware image processing with the bilateral grid,” ACM SIGGRAPH, pp. 1031-1039, July 2007. [↩]

Faculty & Topics