Assignment 4: Out Of The Blue

The blue spot, that is. What blue spot? This one: The Digital Sensor Reflection Effect. Or maybe it is this one: The SCT f/6.3 focal reducer's pupil ghost? A focal reducer causing this problem is particularly insidious because it's supposed to work with any lens... so how could one modify all lenses to solve the problem?

Basically, these issues are not able to be fixed optically (at least not by modifying existing opticas), so we're going to try to do it computationally. Here's my post on DPReview suggesting this could be fixed in postprocessing: Jinfinance Focal Reducer vs. Metabones Speed Booster.

Your Project

You are to write a program that, given a P6 8-bit-per-pixel-color-channel PPM file, simply outputs a copy of that file with any central blue spot removed. Easy, right?

Well, sort-of easy. There are plenty of ways to detect spots. In fact, that's what my DeOrbIt tool does so it can remove so-called "white orbs" from Fuji X10 images. I had a paper and presentation on this, FUJIFILM X10 white orbs and DeOrbIt, paper number 8660-4, at the IS&T/SPIE Electronic Imaging conference in San Francisco. Here are the slides and paper preprint. However, these blue spots can have different shapes depending on the aperture used; in fact, sometimes they will even be rectangular thanks to the shape of the sensor or other masking. They also can fall-off very gradually due to defocus. Thus, I think detecting spots will be problematic. Of course, especially if you're a grad student, feel free to try to prove me wrong. ;-)

My suggestion is to not detect spots at all. Basically, I'm suggesting that the blue spot is more like a locally incorrect color balance. There is some amount of blue added to some pixels near the center of the frame... and not added to pixels that are more distant from the center. So, let's detect individual pixels that have their color off and correct them.

The key is color histogramming. A pixel whose color is essentially some blue plus the color of another pixel more distant from the center probably should have it's color match that of the pixel more distant from the center. This matching will often be ambiguous, so we'll turn it into a popularity contest. We want pixels to be like the most popular ones that we trust to be good role models. What?

It's simple really. Our trust comes from two things. Popularity is simply frequency of occurrence. After all, if every pixel away from the center was a particular shade of green, it would be a darn good bet that the center ones should be that same green too. Trust comes from not being subject to the localized color imbalance. We don't really know how big the central blue spot is, but the farther away from the center a pixel is, the more we trust it is not affected by a blue spot. Thus, I'm suggesting that you simply build a histogram weighted by the distance from the center of the frame.

For each pixel, you can imagine a family of potential color corrections involving removal of blue. Ok, it isn't really always the exact same blue, but it is a surprisingly consistent color bias for any particular optical system that has this issue, so your program can be given an R-G-B vector describing the color. Simply look-up each of those potential corrections and replace the pixel with the one that has the highest weighting in the histogram.

Will this really work? Well, no. There probably will be lots of errors. Your goal is to see how well you can make this approach work. There are various subtle tweaks that you can easily make to try to improve performance. For example, how many buckets should your histogram have? Another tweakable parameter comes from the observation that the level of blue added by a spot should be generally decreasing as one moves radially away from the center. Be creative. Try a few tweaks.

Sample Images

Well, here they are. You're looking at PNG files, but the PPM files are what you'll get if you click on what you see.

Stuff To Know About

Reading a PPM (Portable Pixel Map) file is actually very easy, and I'm making it even easier by giving you code to read and write them. Here is C code to read and write P6 files using memory mapping for the input.

Due Dates, Submission Procedure, & Such

For full consideration, your project should be submitted no later than December 14, 2015. Note that grad students will need to present their talks about their solutions on December 14, which means that you will have time to tweak yours after hearing about theirs.... Submit your .tar or .tgz file here:

Your email address is .
Your password is

Which type of student are you?
Undergraduate registered for EE599
Graduate registered for EE699


http://aggregate.org/CACS/ Cameras as Computing Systems