Introduction

What is computational photography?

• Combining CV, CG, optics&sensors.

Analog photography

• optics（光学器件） to focus light on an image plane （成像平面）
• film （胶卷） to capture focused light（聚焦光）(chemical process)
• dark room（暗室） for limited postprocessing（后处理） (chemical process)

Digital photography

• optics to focus light on an image plane
• digital sensor to capture focused light (electrical process)
• on-board processor（机载处理器） for postprocessing (digital process)

Computational photography

Optics + illumination + sensors + computation

• generalized optics between scene and sensor
• unconventional sensing and illumination(照明)
• arbitrary（任意）computation between sensor and image

Overcome limitations of digital photography

• Image enhancement and photographic look (图像增强与摄影效果)

Image enhancement and photographic look

• High dynamic range (HDR) imaging (Homework!)

High dynamic range (HDR) imaging

Create realistic new imagery

• Image blending(混合) and harmonization(协调)

Image blending(混合) and harmonization(协调)

Post-capture(捕获后的图像合成) image compositing(合成)

• Computational zoom (计算变焦) (Homework!)

Computational zoom

Process image collections

• Auto-stitching（缝合） images into panoramas（全景图像）

Auto-stitching（缝合） images into panoramas

Process (very) large image collections

• Using the Internet as your camera

  Using the Internet as your camera
  • reconstructing cities from Internet photos
  • time-lapse（延时） from Internet photos

Generalized optics between scene and sensor

Capture more than 2D images

• Light-field cameras for plenoptic（全光） imaging
• post-capture refocusing（Homework!）

post-capture refocusing*

Measure 3D from a single 2D image

• Coded aperture (光圈) for single-image depth and refocusing

Coded aperture (光圈) for single-image depth and refocusing

Remove lenses（镜头） altogether

• FlatCam: replacing lenses with masks

FlatCam: replacing lenses with masks

Unconventional sensing and illumination

Measure depth

• Time-of-flight sensors(飞行时间传感器) for real-time depth sensing

Time-of-flight sensors(飞行时间传感器) for real-time depth sensing

Measure light in flight

• Streak(条纹) camera for femtophotography()

Streak(条纹) camera for femtophotography

Measure photons（光子） selectively

• Structured light for epipolar imaging（Homework!）

Structured light for epipolar imaging

Course fast-forward and logistics

Topics to be covered

Digital photography

• optics and lenses
• color
• exposure
• aperture (光圈)
• focus and depth of field
• image processing pipeline

Image manipulation (操作) and merging（合并）

• image filtering （滤波）
• image compositing （合成）
• image blending （混合）
• image warping （变形）
• morphing （变形）
• high-performance image processing

Types of cameras

• geometric camera models
• light-field cameras
• coded cameras
• lensless cameras
• compressive cameras
• hyperspectral cameras

Active illumination and sensing

• time-of-flight sensors
• structured light
• computational light transport
• transient imaging
• non-line-of-sight imaging
• optical computing

Course logistics

• Course web : http://graphics.cs.cmu.edu/courses/15-463