Credible repair of Sony ARW2 raw image data

The current test version of KARWY-SR is:

http://aggregate.org/DIT/KARWY-SR/karwy/test/karwy.html

That's the beta test version 20180318 of the free program KARWY-SR, which runs locally in your web browser to perform "Stripe Removal." The stripes we're talking about are the ones like the above, which run horizontally across the sensor in landscape orientation and are probably related to the PDAF pixels on the sensor. This version only can repair this artifact in compressed Sony ARW2 raw files, and the result is also a compressed ARW2 file. Simply drag & drop ARW2 file(s) below, wait typically less than 30 seconds, and save the "Link to processed file" to a file with a name ending in .arw. The image displayed is just the original JPEG thumbnail; it has not been repaired, but is displayed to make it easier for you to know which file is which when converting multiple ARW2 files.

We created the original KARWY (pronounced car-we) to credibly repair compression artifacting in Sony ARW2 raw files. However, the original version is compiled C code which requires Adobe DNG Converter, ExifTool, ImageMagick and XZ as helpers, making installation a pain. Thus, we made KARWY available to run remotely on our servers via a WWW form interface. That means it does its computations much faster than KARWY-SR, but you have to wait as big raw files are sent to and from the server. We hope to add functionality of the original version to KARWY-SR... but for now, the two tools are separate, and you should use KARWY-SR first and then KARWY to repair a file suffering both types of artifacting.

Revision History

The original version posted was alpha test version 20180315. It did fairly well, but was a little over-aggressive in its repairs and had a little too much logic trying to avoid making a mess out of non-artifact things that looked like artifacts. The 20180318 version uses a much simpler, and actually more aggressive, artifact recognition process, but is much gentler about making repairs. It's enough improved that we aren't worrying about keeping the previous version live.

Neither of the above versions has the texture generation repair logic enabled yet, although the code is there and "repaired" pixels are appropriately marked with error bounds for the texture ehnacement process.

Examples

It turns out to be rather difficult to get really nasty artifacts from these stripes -- the example above has been processed to significantly enhance how bad that looks. Still, it does happen and some cameras are more prone than others. The Sony A7III seems particularly prone, I was able to get some visible effects using my Sony RX100-V in extreme lighting, and my Sony A7RII seems virtually immune.

I am posting test images that I've shot here. There will be at least one artifacted image posted for each camera that I could make produce the artifact. Both the original and KARWY-SR-processed ARW2 raws are linked for each shot there, along with JPEGs for the original, the repaired version, and the auto-levels-boosted difference between the two.

Authorship & Acknowledgements

This program was created by Henry Dietz at the University of Kentucky (also known as ProfHankD) as a research prototype; you use it at your own risk. The ARW2 decoding logic is derived from dcraw. The program is written in C, but has been converted into javascript using emscripten, with javascript interface routines from zfedoran's dcraw.js. Both dcraw and the javascript interface routines are covered by the GNU General Public License, and full source code is available by request (and will be posted once KARWY-SR is stable).

An Open Note To Sony

The core code that performs this credible repair is extremely simple. Running in camera with direct access to the raw frame buffer, it would barely require more time than a single read pass over the image. For a particular camera model, it could be significantly sped-up by only looking at the rows known to be prone to this artifact. Even with the decoding and re-encoding of an ARW2 file, it runs in about 1 second as native C code; it takes less than 30 seconds in this interpreted javascript version running inside a WWW browser. I would be very happy to help Sony implement this repair in a firmware update.

However, this repair also trivially could be implemented in-camera using PlayMemories -- and I could have implemented it for you using OpenMemories if you hadn't disabled PlayMemories support in the cameras most seriously affected by this artifact. So, let me once again repeat my plea that Sony should consider continuing and expanding support for PlayMemories to allow 3rd-party developers, including open-source developers like myself, to make camera applications... most of which would be doing novel things, not just implementing repairs Sony could implement as firmware updates. I would recommend allowing running "unsigned" camera applications to permanently mark the camera as "warranty voided" -- that would not discourage 3rd-party developers such as myself -- but also implement a review process whereby Sony could certify that an app was safe and then sign it so users would not have their warranty voided. I and many others have been doing novel things inside Canon PowerShot cameras using CHDK for about a decade, but CHDK's user interface is not well integrated and Canon's software infrastructure cannot support secure apps, while Sony's Linux-based PlayMemories environment easily could. A marketplace for 3rd-party apps could be as big a thing for cameras as it has been for cell phones.

I would be happy to discuss any of the above with Sony engineers....


The Aggregate The only thing set in stone is our name.