Natural Color Rendering

CS 184 Final Project · Final Report

Lu Jin Xujia Liu Zhehao Wu Zihan Yi

Burst HDR Merge / Denoise Order Diffusion Refinement Tone Mapping Ablation

Showcase & Links: Final deliverables (video, slides, and website).
Website: https://none-momo.github.io/CS-184-NDR/final_report.html
Video: Google Drive link
PPT: Google Slides link

Abstract

We build a computational photography pipeline for high-ISO burst RAW imaging and display-ready color rendering. Our system compares multiple processing orders (denoise→merge, merge→denoise, and coarse→merge→refine), integrates an optional diffusion-based learned refinement backend, and evaluates tone-mapping / color finishing variants (Reinhard, filmic/Hable, and ACES). The report emphasizes visual quality, pipeline clarity, and reproducible evaluation, aligning with the graphics-oriented focus of CS 184. We learned that in a graphics setting, the finishing stage (tone mapping and color) often dominates perceived quality, and that clear comparisons (tables, metrics, zoom-ins) are essential for communicating results.

Technical Approach

We evaluate three end-to-end processing orders (Pipeline A/B/C) that reorder the same core stages (classical denoising, HDR merge/alignment, diffusion refinement, and finishing). For readability, we summarize A/B/C explicitly in the Pipeline Overview section and then present denoising and tone-mapping effects separately in the Results.

Unique decisions and enhancements. We implemented a modular C++ pipeline interface that allows swapping denoisers and reordering stages with a single flag. We integrated a diffusion backend as an optional post-merge refiner via a C++↔Python bridge, and performed a tone mapping ablation to align the project with CS 184’s emphasis on visual quality.

Pipeline Overview

Our system is organized as three end-to-end pipeline variants that reorder the same core stages (denoising, HDR merge/alignment, and finishing). This report separates denoising effects from tone mapping effects to keep the results easy to read.

Pipeline	Order	Intuition	When it helps
Pipeline A	classical denoise (per-frame) → HDR merge → finishing	Remove sensor noise early so fusion operates on cleaner inputs	Fast baselines; stable improvement on high ISO
Pipeline B	HDR merge → diffusion refine (merged) → finishing	Learned prior cleans the merged image while preserving global structure	Strong denoising at very low light; may over-smooth textures
Pipeline C	coarse denoise → HDR merge → diffusion refine → finishing	Hybrid cascade: light classical pre-clean + diffusion fine-tune	More robust across scenes; more knobs but better trade-offs

In the sections below, we first isolate the denoising vs. no-denoising comparison (using DSC08538/DSC08539), then isolate the tone mapping ablation (using three additional scenes).

Denoising vs. No Denoising

We use two low-light scenes (DSC08538 and DSC08539) to highlight the effect of denoising. The goal is to show how much noise can be removed without destroying small light sources and subtle gradients in dark regions.

Where to look: background chroma noise in dark areas, the halo around point lights, and whether denoising washes out faint details.

Scene Pair — DSC08538 and DSC08539

For each scene: No-mapping (baseline) → Denoise → Method3 (tone map). Hover to zoom for close inspection.

DSC08538

DSC08538 · No-mapping (baseline)

DSC08538 · Denoise

DSC08538 · ACES-luminance

DSC08539

DSC08539 · No-mapping (baseline)

DSC08539 · Denoise

DSC08539 · ACES-luminance

Result summary. Denoise Denoise mainly reduces chroma/luma noise in very dark regions, but under a no-mapping baseline the improvement can be hard to perceive due to under-exposure. Applying ACES-luminance improves display visibility (midtone lift + smoother highlight roll-off around point lights), making denoising benefits easier to evaluate. Focus on: (i) dark-sky noise texture, (ii) halos near light sources, (iii) preservation of faint structures vs. over-smoothing.

Tone Mapping Ablation

After denoising and merging, the same scene-referred signal can look dramatically different once mapped to display space. We therefore compare three tone-mapping operators side-by-side on three scenes, using No-mapping as the baseline.

Where to look: highlight roll-off (avoid hard clipping), shadow lift (visibility without noise blow-up), and color consistency.

Tone Mapping Ablation — IMG_4849

Indoor / structured content. Inspect edges and smooth gradients for banding or contrast loss. Hover to zoom; compare highlight roll-off, midtone contrast, and color shift.

No-mapping

Hasselblad

Guided filmic

ACES (luminance)

Tone Mapping Ablation — IMG_5239 2

Additional scene (filename contains space). Useful to validate robustness of the reporting pipeline. Hover to zoom; compare highlight roll-off, midtone contrast, and color shift.

No-mapping

Hasselblad

Guided filmic

ACES (luminance)

Tone Mapping Ablation — IMG_8457

Low-light sample for generalization. Focus on highlight roll-off and color casts. Hover to zoom; compare highlight roll-off, midtone contrast, and color shift.

No-mapping

Hasselblad

Guided filmic

ACES (luminance)

Method / Pipeline Variants Table

We group methods into (i) HDR+ reproduction baselines that isolate alignment/merge behavior (Average vs. Robust), (ii) our end-to-end pipeline orders (A/B/C), and (iii) a tone mapping ablation that compares three display-mapping operators.

A. HDR+ Reproduction Baselines (Alignment + Merge)

Baseline	Order	Key idea	Improves	Failure mode
Average	align → temporal mean	Aligned averaging across burst frames (simplest HDR+ baseline).	Noise reduction (≈ √N)	Blur/ghosting if residual misalignment exists
Robust	align → robust merge	Adaptive fusion that down-weights inconsistent regions to suppress mismatch artifacts.	Sharper details; fewer ghosts	More compute; sensitive to weighting/window choices

B. End-to-End Pipeline Orders (Our System)

Pipeline	Order	Intuition	When it helps	Trade-off
Pipeline A	classical denoise → HDR merge → finish	Reduce per-frame sensor noise first so fusion operates on cleaner inputs.	Fast, stable improvements at high ISO	May smooth fine detail before merge
Pipeline B	HDR merge → diffusion refine → finish	Apply a learned prior after merging to clean noise while preserving global structure.	Strong denoising in very low light	Higher runtime; risk of over-smoothing textures
Pipeline C	coarse denoise → HDR merge → diffusion refine → finish	Hybrid cascade: light classical pre-clean stabilizes merge, then diffusion fine-tunes.	More robust across scenes; better trade-off	More knobs; tuning required

C. Tone Mapping Ablation (3 Operators)

Tone mapping	What it does	Typical visual effect	What to look for
Hasselblad	Filmic-style curve tuned for photographic contrast and smooth roll-off.	More “photo-like” contrast; smoother highlights.	Highlight roll-off, midtone contrast, shadow lift
Guided-filmic	Guided/edge-aware filmic mapping to preserve structure under compression.	Better structure preservation; reduces flat-looking compression.	Edge halos, local contrast around boundaries
ACES-luminance	ACES fitted curve applied through luminance for stable color appearance.	Cinematic highlight compression; consistent color response.	Color cast shift, highlight saturation, shadow noise

Result summary. Denoise mainly reduces chroma/luma noise in very dark regions; however, under the no-mapping (near-linear) baseline, the improvement can be difficult to perceive because the image remains under-exposed. To avoid confounding denoising with display mapping, we fix the finishing operator and apply ACES-luminance, which lifts midtones and provides smoother highlight roll-off around point lights. With ACES-luminance, denoising gains become more interpretable: (i) background noise texture in the dark sky becomes more uniform, (ii) halos and shimmer near small light sources are reduced, and (iii) faint structures are either preserved or slightly smoothed depending on the denoising strength. Overall, ACES-luminance improves readability (display visibility) while denoise improves noise statistics; the main trade-off is potential loss of micro-texture in extremely low-contrast regions.

Quantitative Metrics (Grouped by Scene)

To keep the report readable, we group proxy metrics by scene (image_id) and enforce a consistent variant order. We report no-reference proxies (MAD noise σ, Tenengrad sharpness, local contrast) because ground truth is unavailable.