Cross-talk can be divided to different components such as optical cross-talk and diffusion cross-talk. In the simulator, however, all cross-talk is determined by one single parameter, i.e. PSF standard deviation. Cross-talk is also assumed to be constant in the whole image sensor area (typically optical cross-talk is most severe on the edges of the sensor area, but here we don’t simulate optics).

Diffusion cross-talk is resulted in when photons are absorbed and generated electrons diffuse to the neighboring pixels. In order to avoid diffusion cross-talk the modern Back-Side Illuminated (BSI) consumer image sensors are typically very thin. In order to improve the sensitivity for Near Infra Red (NIR) radiation a considerably thicker sensor is required which, however, increases the cross-talk substantially unless a fully depleted sensor design is utilized. Consequently, in applications requiring good sensitivity for both NIR and visible light a fully depleted thick BSI sensor is optimal.

You can find our simulator here. We hope that the ability to simulate cross-talk makes our simulator a more practical tool. If you have any questions about it please post a comment. Any feedback is welcome.

In security and surveillance applications identifications is often done from a stored video stream. Especially in the low light outdoor applications the signal-to-noise ratio in single video frames is often too low for identification. However, one can look after periods in the stored video stream when the subject remains more or less still. By merging frames together during such a period the signal-to-noise ratio and thus the identification can be improved considerably.

When multiple frames are merged together Pixpolar’s MIG technology enables much better image quality than conventional technologies. The below Figure presents a simulation result comparing the signal-to-noise ratio of merged frames in case of MIG and high-end CMOS Image Sensor (CIS).

Figure 1. Simulation result presenting the signal-to-noise ratio enhancement when multiple frames are merged together in high-end CIS and MIG technologies. The simulation was done for low intensity Near Infra-Red (NIR) radiation which dominates in the darkest conditions (850 nm, 50 photons / second / pixel in the lightest areas). The Quantum Efficiency (QE) in CIS was set to 28 % and in MIG to 70 %. Dark current in CIS was 0 e-/s/pixel and in MIG 1 e-/s/pixel. The read noise in both sensors was 1 e-. The frame rate was 25 fps and in total 50 frames was merged together. The simulation was done with Pixpolar’s image sensor simulator found at http://imager-simulator.appspot.com.

Technical Note:
To get good Quantum Efficiency (QE) in NIR the sensor should be relatively thick. Due to MIGs fully depleted backside illuminated structure the MIG sensor thickness can be optimized unlike in CIS where cross talk becomes more and more an issue when thickness is increased. The QE values for the simulation were chosen to best reflect what is achievable in the technologies without compromising too much other performance parameters (the QE of CIS was picked up from an existing high-end sensor). The low dark current values used in the simulation were set expecting that both sensors were cooled down. The higher value in MIG is due to thicker and more optimal structure for NIR.

Pixpolar’s MIG pixels incorporate a Non-Destructive Correlated Double Sampling (NDCDS) read-out method, which enables low noise read-outs without destroying the captured signal. In conventional technologies the signal is always destroyed when low noise CDS read-out is done. The difference between the read-out methods is seen in the image quality when frames are merged together. In case of traditional destructive CDS, the read noise of the merged image adds up in a root mean square fashion whereas in case of NDCDS the read noise is reduced.

Download as flyer: pixpolar_flyer_security_surveillance.pdf

More detailed technical description: pixpolar_technical_note_security_surveillance.pdf

Please don’t hesitate to comment if you have any questions related to the video!

Note that the y-axis in the measurement data is voltage and the x-axis is time. The higher the measurement signal is the more there are electrons in the corresponding MIG. If you have any questions about the video please feel free to post a comment.

Please check out our presentation about the 1T MIG pixel concept.

There has been some debate whether or not anyone needs 41 MP in their cameras, but that’s just pointless. The idea of putting 41 MP into a mobile phone camera is not to produce 41 MP images, the output images have only 5 MP by default. Oversampling is used to reduce image noise, problems related to Bayer color filters and of course to introduce digital zoom. The camera has it’s own dedicated processor to digest the huge amount of data from the sensor. It can deliver full HD 1080p 30fps with 4x loss-less zoom.

Nokia 808 PureView has just set the benchmark for future mobile phone cameras.

Jussi Seppälä

It is a very common problem that Scalado aims to solve with their object removal technology. The idea is simple, their app takes several images from the scene and detects all moving objects and a static background. Nice part is that they don’t just extract the static background but they let the user select which objects they want to keep in the image.

Besides comparing different readout methods, the simulator can be used for many different purposes. Students can use it as a learning tool to study how different noise components and image sensor parameters affect the image quality in various lighting conditions. Professionals can compare how different image sensors could perform in their application. Photographers can use it to get deeper understanding of digital image formation process.

The simulator should work with newest versions of most popular desktop browsers. Please feel free to use the simulator for whatever purpose!

Pixpolar

Images taken in low light with present technology include a tradeoff between two artifacts: image noise and image blur. In the below figures you can see two example pictures. The left one has lots of image noise and the right one is blurry. The blurry image is not easy on eyes and if you look at it long enough you will most probably get a headache. The image with lots of noise is easier to look at, and if you must decide, you would probably go for noise instead of blur. That’s also the most probable decision that the automatic mode of any present camera would make.

The noise is not a problem if the signal is large enough, it’s the signal-to-noise ratio that matters. In photography, the more light is captured the larger the signal is and the better the resulting signal-to-noise ratio is. Now the question is, how can you capture lots of light in low light? The answer is that you increase the exposure time, but, as most things in life, increasing exposure time has its side effects. To get a sharp image using a long exposure time your camera and the subject you are trying to capture should be relatively still during the whole exposure. If your camera has an image stabilizer the pitch and yaw motion can be compensated, at least to some degree, but it cannot correct camera roll motion or subject movement.

One way to avoid image blur with long exposure times is to divide the long exposure time into shorter intervals, i.e., to take multiple pictures with fast shutter speeds. Each image would be sharp, but the signal-to-noise ratio would be relatively low. When they all are combined using intelligent software, the signal-to-noise ratio can be improved substantially. There are already some cameras that utilize this idea, for example Sony’s NEX-5 (review here).

Whereas Sony is implementing image blur correction algorithms into their cameras, Adobe has developed a post processing algorithm that can correct image blur (video here). As the processing power in hand held devices is constantly increasing, the future of computational photography looks really interesting.

Jussi Seppälä

P.S. We made an html5 application to demonstrate the idea of using NDCDS readout to correct image blur. The app was made for Pixpolar’s demonstration needs and therefore it’s not optimized for many different platforms. You can check it out if you have an html5 ready browser running on a relatively powerful desktop environment. Have fun with our demonstrator!

Pictures taken with conventional cameras are two dimensional light intensity maps. Light field pictures include also information of the directions of the light rays which can be used in several ways:

- The focus of an image can be set to any distance or the whole image could be made sharp.
- One can measure the distance of the objects in an image that enables 3D modeling.
- Objects in any given distance from the camera could be cropped out easily.
- etc.

Light field imaging will change the way we use cameras and digital images. The development of this technology is early in it’s life cycle and it will take some time until it will replace conventional cameras. One major area that needs to be improved is the resolution.

During the past decade, the number one selling point of digital cameras has been the number of pixels in the image sensor. Lytro says that their camera can capture 11 megarays. It most probably means that they have 11 mega pixel sensor in their camera. What it doesn’t mean is that the images will have 11 mega pixel resolution.

Light field cameras include a lens array and the number of those lenses determine more or less the end image resolution. Behind each lens in the lens array there is some amount of pixels which are used for determining the direction of the light rays. For example, with 11 mega pixel image sensor and with 25 pixels reserved for each lens in the lens array the picture resolution would be 0.44 megapixels only.

In this manner to have a HD resolution picture with a light field camera one would need to have approximately 92 megapixel sensor (720 x 1280 x 4 x 25 = 92.16M). Have you seen one for sale? I haven’t.

For sensor manufacturers light field imaging means that the pixel race is not over and it will become even more important when light field imaging starts to get more consumer attention.

Scaling down pixels in conventional CCD and CMOS image sensor technologies means that the performance is severely compromised due to cross talk, dynamic range and other issues. Totally new image sensor technologies are therefore required to utilize the full potential of light field imaging.

It looks like there will be interesting times ahead for us enthusiastic photographers.

Jussi Seppälä