Problem Statement
Modern smartphone computational photography pipelines tackle two fundamental challenges in imaging: limited dynamic range and high sensor noise, especially in low-light conditions. Existing academic work (e.g., Blind2Unblind, GDP) largely targets smartphone RAW burst photography, which is difficult to replicate in an open research setting — the RAW burst pipelines of companies like Apple are closed-source, and capturing aligned multi-exposure bursts requires specialized hardware access.
Instead, we propose to leverage the physical advantages of a full-frame mirrorless camera (Sony ILCE-7M4) — its larger sensor provides substantially better per-pixel SNR and wider native dynamic range than any smartphone — to build and evaluate an HDR + denoising pipeline that is both reproducible and academically rigorous.
Our core question: For full-frame RAW capture, which combination of diffusion-based denoising and multi-exposure HDR merging produces the best final image quality?
Goals & Deliverables
Core Pipeline (Plan to Deliver)
Capture a set of full-frame RAW images under controlled conditions. Apply two denoising strategies independently and evaluate quality with SNR, PSNR, and SSIM against a ground-truth clean reference:
- Diffusion-based: GDP (Generative Diffusion Prior) applied to single RAW frames
- Multi-frame averaging: Aligned multi-exposure stack merge for stationary scenes
Build a benchmark comparing four pipeline variants across multiple test scenes:
| Variant | Description | Key Trade-off |
|---|---|---|
| D only | Diffusion denoising on each exposure, then HDR merge | Best for dynamic scenes; no static requirement |
| D → M | Diffusion first, then multi-frame merge | Clean inputs to merger; may over-smooth |
| M → D | Multi-frame merge first, then diffusion refinement | Preserves structure; diffusion as post-processing |
| M only | Multi-frame merge only (classical baseline) | Requires static scene; no deep learning |
Based on benchmark results, assemble the best-performing pipeline and demonstrate end-to-end HDR reconstruction with quantitative and qualitative evaluation.
Stretch Goals (Hope to Deliver)
Apply a tone mapping operator to the merged HDR output so the final image is visually faithful to human perception, with preserved highlight and shadow detail.
Train a diffusion model specifically on our captured full-frame RAW data, rather than relying solely on ImageNet-pretrained models, to better match our sensor's noise characteristics.
Design a set of color-grading function calls (exposure, saturation, hue shift, LUT application, etc.), then connect them to an LLM so users can describe a desired look in plain language and receive a graded output automatically.
Proposed Pipeline
The figure below shows the full pipeline we plan to build. The innovative pipeline (top row) extends the baseline with white balance correction and optional LLM-driven color grading. The reproduced baseline (bottom row) follows the original Blind2Unblind + HDR merge approach for direct comparison.
— Our Innovative Pipeline —
Short / Mid / Long LDR
Color Temperature Correction
GDP / Blind2Unblind
Camera Response Estimation + Fusion
NL → Params → LUT
— Reproduced Baseline (from Papers) —
burst photography
Self-supervised, no pairs needed
Frame alignment + fusion
Schedule
| Week | Tasks |
|---|---|
| Week 1 | RAW data capture with Sony A7M4; set up denoising baselines (Blind2Unblind, GDP); implement evaluation metrics (SNR, PSNR, SSIM) |
| Week 2 | Implement multi-frame HDR merging (camera response curve estimation + exposure fusion); white balance / color temperature correction module |
| Week 3 | Build benchmark: run all four pipeline variants (D only, D→M, M→D, M only) on test dataset; quantitative + qualitative analysis |
| Week 4 | Finalize best pipeline; stretch goals (tone mapping, LLM color grading); write-up and demo preparation |
Resources
Reference Papers
Wang et al., CVPR 2022 arXiv:2203.06967
Proposes a global-aware mask mapper and re-visible loss for self-supervised denoising without clean reference images, achieving state-of-the-art performance on real-world noise.
Fei et al., CVPR 2023 arXiv:2304.01247
Exploits a pre-trained unconditional DDPM as a generative prior for unified image restoration (super-resolution, deblurring, inpainting, colorization, low-light enhancement, HDR recovery) in an unsupervised setting.
Hasinoff et al., ACM TOG 2016 DOI:10.1145/2980179.2980254
Presents a practical burst photography pipeline for mobile imaging that combines frame alignment, robust merging, and HDR reconstruction to improve image quality in challenging low-light scenes.
Computing Platform
MacBook M3 Pro — 16-core GPU
Lenovo Legion — NVIDIA RTX 5090 GPU
Hardware
Full-frame 33 MP BSI-CMOS sensor. Larger pixel pitch and lower read noise compared to smartphone sensors, providing a stronger baseline SNR and enabling cleaner HDR bracket capture.