mw

mike.wong | pixel.surgeon

Multi-scale Image Decomposition (publication)

This image decomposition method was first shared as a chapter in GPU Zen edited by Wolfgang Engel, and I included a GLSL compute shader implementation in the book’s Github repo. I originally planned to build a vanity effect Nuke plug-in using this method but since my plug-in business (via artixels) never gained enough traction since 2015, so I simply kind of open-sourced it.

The method uses an image filter I designed named Sub-window Variance filter (SVF) which uses a statistical edge model to define the image details to be decomposed. When one applies the filter iteratively, the original image becomes decomposed into multiple detail layers of different degree of fineness. By manipluating and re-integrating these detail layers, it is easy to create visual effect such aging or de-aging, see below an example.

In 2021, I published a follow-up article (ICVR) in IEEE Internation Conference on VR to explain the theoretical detail of the SVF with a MATLAB implementation on my Github for other researchers. One key design of SVF is its use of Summed Area Table as an acceleration means for real-time processing. Below is another example of using SVF for local contrast enhancement (Top: Original, Bottom: Enhanced). I also used SVF for automated dataset patch selection in my MC denoising research.

  • Pixel-space Diffusion Model from scratch
  • Resonance for Nuke
  • Escher for Nuke plug-in suite
  • Deep Residual Denoising for MC Rendering (research publication)
  • Conquer: Lahad Datu (TakLuk)
  • VigGen 02 – Unseen (Runway Gen3-Alpha)