The End of Moore's Road: Sensor Edition

So I've been looking for a new camera, one that can record 4K 60fps video. It doesn't exist unless you count the $6000 Canon 1DX Mark II which is huge and weighs a ton. It's hard to go back to 1080p after seeing the rich details 4K offers but it's also hard to go back to 30fps after watching the smooth motion 60fps is capable of.

Right now you can get 4K or 60fps despite the fact that there are a number of relatively inexpensive ~$1000 cameras that can do near native 1080p @ 240fps. The bandwidth and processing requirements are similar but companies don't seem to see the need. Oh well. It's hard to choose 4K or 60fps even though either would be an improvement over my current 1080p30 setup. I guess that makes me Buridan's Ass.*

Anyway, one thing I discovered during my search is that image sensor quantum efficiency is around 60%.

Wow! This means that there is under a stop left of high ISO performance left before nature itself places a hard limit on improvement. This milestone sort of snuck up on me even high ISO performance is one of the most discussed aspects of digital imaging. Although this quantum efficiency level, quantitatively speaking, isn't as impressive as the technological records achieved in trying to reach absolute zero, semiconductor process sizes approaching the size of a handful of atoms, or even something like Vantablack, it's significant from a photographic standpoint.

There is probably around another stop of sensitivity available by replacing the color mosaic layer used in sensors with a 3 chip array which is sometimes used in video cameras. That approach represents significant enough higher cost and complexity that it will probably be a last resort after signal processing approaches have been exhausted.

Resolution wise, there's still plenty of room. Although diffraction limited optics exist - something which also amazes me - most lenses do not exhibit that high performance wide open. But as manufacturing improves and exotic lens shapes - such as those used in the Nokia 808 pictured below - become feasible, a diffraction limited f/2.8 lens can resolve nearly 400 megapixels on full frame; the current full frame megapixel champ maxes at around 50. Looking at Sony's sensors, the IMX318 has the finest pixel pitch at a computationally convenient 1.0 micrometers which implies a full frame scaling of 864 megapixels. If per pixel full color accuracy is desired, 1600 "Bayer megapixels" will be required to approximate 400 full color megapixels. Given the state of semiconductor manufacturing, this is definitely within the realm of possibility. In that sense, the effects of the end of Moore's law probably lie beyond the diffraction wall.

NO SPHERICS ALLOWED! (Image from test-mobile.fr)

Dynamic range capability is related to signal to noise ratios. Quantum efficiency has a role in improving the signal quality while improvements to read noise can boost SnR in tandem. But current technologies impose another limitation: full well capacity. Full well capacity tends to be lower with smaller photosites. However, multiple photosites with smaller full well capacity are equivalent to the same sized single photo site with its larger full well capacity. According to DPReview, read noise is the only reason that sensors using larger photosites marginally outperform similar sensors with smaller photosites. If photon counting technology can be developed, read noise is not only essentially eliminated, but full well capacity is no longer an issue. This implies enormously higher dynamic range capability.

All of these improvements, save 3CCD, are dealing with a single sensor. But huge gains in image quality can be obtained by using multiple sensors for 3D imaging or processing tricks like the multiple lens/sensor gimmick approach by L16 or Leica Huawei P9. So while the high ISO sensor performance race is finished, there are plenty of events left.

* If Sony keeps its June release cadence, maybe the Alpha a6400 or RX100V will have 4K60. The GH5 might but I'm hoping the feature hits smartphones first.