Lossy Compression of RAW

There has been some discussion recently thanks to the huge pixel count of Nikon’s D800 regarding the size of the NEF (“raw”) files.  They’re 75MB, 14-bit, uncompressed.  Nikon offers a form of lossy-compressed NEF but a lot of people think this is a Bad Thing™ because they know that JPEG is “lossy” and everybody knows that is bad.

However, it’s not proper to compare the two.  JPEG compression has quality problems because it loses information between pixels causing noise and artifacts around sharp edges (text being the best example), and because it loses some resolution (the number of bits worth of color detail) as well resulting in somewhat less than 8 bits of information per channel.

JPEG loses information you would not normally see.  Nikon’s NEF compression loses information you cannot use.  Let me explain…

First, you have to think how many bits you need on any single pixel. The dynamic range of the image is not relevant because the lossy-ness is only per-pixel and thus no odd artifacts around sharp edges.

Second, you can’t display or print with a resolution higher than 8 bits per channel.

Third, of all mathematical operations only subtraction loses significant bits. Note that clipping (i.e., increasing the brightness beyond maximum value) and contrast enhancements are a kind of subtraction.

With those in mind, you can keep only 9 bits of information for any pixel and still be able to create a full-detail image. A 9-bit number can hold values from 0-511. The value with 9 significant bits (i.e. has a 1 in the most significant bit) and the least amount of information is 256 and so still has full-detail when reduced to 8-bit (0-255) for display/print.

What is important is what 9 bits to keep. A 14-bit AtoD converter has 16384 output values. Ideally you want to encode this such that each pixel is stored in a way that it holds 9 bits of detail data (the mantissa) and where in the original 14 bits the 9 bits reside (the exponent). Yes, it’s effectively a floating-point number. It’s not too difficult to think how to store this efficiently.

Nikon doesn’t do it explicitly in this way but the non-linear encoding curve they use is close. Their 12-bit compression stores about 700 values (varies from camera to camera) and their 14-bit compression stores about 3000 values. The bottom line is that even compressing 12-bit/4096 values to 700 values keeps 7.75 bits of information and compressing 14-bit/16384 values to 3000 values has 9.3 bits of information. [source]

Therefore, given the 8-bit limits when it comes to printing and display, it seems to me that lossy-compressed 14-bit NEFs have all the information you could possibly use… though perhaps not all you could want.


Leave a Reply




You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>