A camera can not solve the problem, then use two: for dual camera you need to understand these

Lei Feng network (search "Lei Feng network" public number concerns) : Author of this article 7Tens, WeChat public number: ICshare sponsor.

The highlights of high-end smart phones are becoming less and less, and Thunder Rabbit can’t show the advantages of mobile phones. So even more domineering dual camera debut. The dual cameras mentioned here are not like previous smartphones, one camera before and one behind, but one shared two rear cameras, simulating a pair of human eyes to achieve more camera functions and better camera effects.

In general, the camera interface of the current smart phone is an MIPI (Mobile Industry Processor Interface) interface. Before the mobile phone platform has only 2 way MIPI interface, respectively to the front camera and rear camera. To do dual cameras requires the platform to support at least three MIPI interfaces. In fact, in the previous high-end platform, in order to achieve a higher pixel, has been using a dual ISP (Image Signal Processing, image signal processor), for example, in order to support 16M camera, will use 2 way 8M ISP. This type of platform is likely to have only 2 way MIPI, but this can not stop the engineer to do the front single camera + rear dual camera.

The question is, what can a dual camera do?

1, dual cameras can range, can do distance-related applications

The human eye can easily locate the distance of an object, but when one person closes one of the eyes, the positioning ability will drop a lot. Dual cameras are applications that simulate human eyes. Simply put, if the distance is measured, it is calculated by an algorithm. The angle θ1 and θ2 of the object to be photographed with the left/right camera, plus a fixed y value (ie, the center distance between the two cameras), it is very easy to calculate the z value. (that is, the distance from the object to the Camera).

However, it is also easy to predict that if the center distance between the two cameras is too small, the calculable object distance will be very close. If you want to figure out a longer distance, you must pull the distance between the left and right cameras.

As shown in the figure above, since the dual cameras can determine the distance of the subject through the algorithm, it is very easy to make some special effects through this feature, such as:

A. Bokeh

One of the most outstanding features of SLR cameras is the large aperture. Because the dual cameras can measure the distance of different objects, aiming at objects requiring large apertures and blurring other objects at different distances, the effect of large aperture can be easily achieved.

(Original)

(Focus on the beauty focus, blur the background and the hand of the mushroom)

(focus on mushroom, blur background and beauty)

B, background replacement

Since the distance can be measured, the subject in the object to be photographed can be extracted and the background can be changed, and it can be simpler than the PS to perform the map.

C, background effects

Since it is possible to measure distances and separate themes and backgrounds, it is easy to do anything with the background and not to describe it too much here.

D, measuring distance

This figure clearly identifies the distances of different objects. This distance information is identified in different colors. When the AP has obtained the distance information of different objects, it can do the above-mentioned various functions.

2, dual cameras can do optical zoom

The optical zoom is mainly about using different FOVs (viewing angles) for the left and right cameras, so that the two cameras have different views. When the user needs a wide-angle photo, the left camera with a viewing angle of 85 degrees is used to obtain a wide-angle effect. When the user needs a telephoto photograph, a right camera with a viewing angle of 45 degrees is used to obtain a telephoto effect.

In order to make the overlap of objects photographed by the left and right cameras high, the dual camera modules of the optical zoom cannot be as large as the distance from the camera module of the application, but it is necessary to arrange the left and right cameras as close as possible.

If the two cameras have different FOVs, a large FOV, a small FOV, and then an algorithm to achieve the effect between the two optical lenses, optical zoom can be easily achieved.

If you do not use dual cameras, the text is not clear after the picture is enlarged

If using a dual camera, the text is still clear after zooming in

This figure is a combination of a wide-angle image and a telephoto image. The median attitude image is calculated by the algorithm so that the details are not distorted.

3, dark light enhancement

In general, dimming is done in two cameras: a standard camera with RGBG and a black and white camera with RGBG filters removed. RGBG is used to obtain the color of the object, and the black-and-white camera is used to obtain a better amount of light to determine the intensity of the light intensity of the object being photographed. Then merge the two pictures for better dark light enhancement.

There are currently two fusion methods:

With black and white pictures as the main body, the color of each pixel obtained on the color picture is pasted on the black and white picture, and the two pictures are merged.

Taking the color picture as the main body, the light intensity of each image obtained on the black and white picture is compensated to the color picture, and the two pictures are merged.

As to which method is more suitable for integration, it is possible that the benevolent see wise men and see wisdom, and they will not discuss this.

In the same way, to make the dark light enhanced, such dual camera modules are also required to be as close as possible in order to allow a high degree of overlapping of objects shot by the left and right cameras.

It needs to be explained that Huawei P9 actually chooses the module of this method.

Of course, some people in the industry also said that the current results of this algorithm are not obvious. Dim light compensation is really helpful for the user, especially when shooting night scenes. However, some customers think that Sony's and Samsung's Dual PD technology is very good, and they prefer to use the Dual PD camera for dimming compensation.

In the end is dual camera or Dual PD dim light compensation effect is good, we can compare Huawei P9 and Samsung's Galaxy S7 edge, there will be an answer.

This generally uses a color + black and white camera. The light intensity of the picture is obtained by the black and white camera to compensate the dark light of the picture.

4, 3D shooting and 3D modeling

The algorithm of 3D shooting and 3D modeling is actually similar to the distance application, but its accuracy is even higher, and sometimes even infrared distance measurement is needed to make a more accurate distance judgment.

Referring to the dual-camera algorithm, we must mention ISP (Image Signal Processing). The main function of the ISP is to post-process the signals output by the front-end image sensor. The main functions include linear correction, noise removal, and dead pixel removal. Interpolation, white balance, automatic exposure control, etc., depend on the ISP to be able to better restore the details of the scene under different optical conditions, ISP technology to a large extent determines the imaging quality of the phone.

In the era of functional machines, ISPs are all built on cameras. Different pixel cameras have different performance ISPs. As the mobile phone camera pixels are getting higher and higher, the requirements for ISP performance are getting higher and higher. If the ISP is integrated into the camera Sensor, it will inevitably cause the camera's module to be too large and even affect the camera effect. So in the era of smart machines, ISPs are generally on the main chip SoC. In order to achieve better results, some brand customers even resort to cost plus an ISP to achieve better and more professional photographic results.

A good camera algorithm requires a good ISP, and ISPs and algorithms complement each other and are indispensable. Dual cameras require more ISP performance. First of all, in order to allow the signals of the left and right cameras to be processed at the same time, a single ISP can no longer meet the requirements of dual cameras. This requires a dual ISP to implement this feature.

Take the dark light enhancement as an example, the color/black and white images enter the respective ISP channels and calibration channels respectively, and then the two pictures are matched (for example, the same part of the two pictures is extracted to remove only the part captured by one camera). Then, the relevant pictures are processed through algorithms such as occlusion, detection, and compensation. Finally, the two pictures are merged to achieve color enhancement. Of course, in fact, the ISP cooperates with the algorithm to do far more than this picture.

Of course, there is also a small episode in it. After all, there are two ISPs. The two ISPs have some problems with processing speed and processing power. In order to ensure that the two ISPs can sample at the same time, the pictures taken by the dual cameras need to be taken at the same time. One solution is to have the Sensor have a sync signal pin. The synchronization signals of the two cameras are connected, and each time the image is read, the pictures are time stamped, and the ISP ensures that the pictures taken by the left and right cameras are shot at the same time through time stamps, and finally the fusion is performed.

It is different from shooting of general 3D movie. The two cameras on the phone cannot produce enough visual difference during the shooting of the image because the distance between the two cameras is different from that of the human eye. And in order to allow people to more clearly get 3D visual effects. So often the algorithm needs to be enhanced.

Because the distance can be measured, the subsequent dual camera can not only achieve 3D photography, but also can perform 3D modeling. By this time, I think the importance of dual cameras will be more important.

Other effects are enhanced, such as HDR, increased resolution, and in fact, single camera can also be achieved. However, dual cameras can make the effect better, and they will not be listed one by one.

summary:

Currently, these features are the most common features of dual camera phones. Bokeh/replacement, the effect of dark light brings more photo effects to the user; optical zoom allows us to experience the camera function of the zoom function; but personally feel that the most exciting of the future is the 3D function.

This year VR is so hot. Where does the VR material come from? Still rely on the optimization of the dual camera algorithm. If the 3D camera and modeling algorithm matures, dual cameras will become more popular.

In the previous section we talked about the application of dual cameras and their principles. Now we will focus on the dual camera industry chain.

1, dual camera algorithm provider

Because the algorithm needs to cooperate with the ISP, the algorithm and the ISP are mutually complementary. If you want to do the algorithm well, you must also have a good ISP.

As the main platform supplier, Qualcomm/MediaTek has its own ISP, so it has also developed its own dual camera algorithm. As for the good or bad of its algorithm, it has yet to be tested by the market.

As a supplier to Sensors, Sony, Samsung, and OV are also actively developing dual-camera algorithms, and they have not yet seen mass production. However, in the era of functional machines, there is no ISP. When these Camera sensor suppliers are doing 2M/5M, they have to match their own ISP, so these suppliers have their own experience in developing ISP. Therefore, the development of dual camera algorithms is relatively related to experience.

Apple acquired Linx last year and also has multiple camera patents and algorithms. Whether it will be used in their own dual camera models, look at this year's iPhone7. In theory, Linx has enough algorithms, so Apple won't buy a dual-camera IP license with a big manufacturer.

In addition to these platforms, sensor vendors and smart phone brand companies will develop dual camera algorithms themselves. Other vendors will list them:

Founded in 1996 in Huajing, Taiwan, mainly develops independent ISP chips. Some high-end mobile phones, cameras and cars have their own ISP case. Dual camera phones are also useful for their ISP's. Distance applications, optical zoom, and dimming compensation all contribute.

An Israeli company. The advantages of its algorithm are mainly in optical zoom and dim light compensation. In terms of depth of field, there is also some research. From the perspective of media promotion, the hTC organic type uses its algorithm.

Hongsoft was founded in 1994 and is headquartered in the United States. It has technology centers in Shanghai, Hangzhou and Nanjing. Rainbow's strengths are optical zoom and dimming compensation. The second optical zoom and dark light supplements in the previous article come from ArcSoft.

Shanghai Xingxinwei is a company established in 2011 and is mainly engaged in the research and development of image processors. Currently, the products are mainly used in the automotive market. As one of the few companies that develops independent ISPs, it is currently developing dual-camera algorithms and ISPs. After the dual camera market, X-Chip will be a very potential dark horse.

The dual camera is also just out, so each algorithm has its own advantages and disadvantages. However, judging from the effects of the dual camera phones that have been introduced so far, the algorithms of each one have yet to be improved. After a variety of algorithm capabilities are added, it is bound to make dual cameras become the standard mobile phone.

2, dual camera sensor supplier

Since the two cameras need to synchronize time stamps when taking pictures, Camera Sensor needs to have synchronization signals. Sensors that currently have this synchronization signal include Sony, Samsung, OV, and Gecom. So the dual camera Sensor mainly uses these products.

3, dual camera module suppliers

At present, there are many module factories that can do dual cameras, including Lite-On, Sunny, Trust, Namuga, O-Film, Samsung Electromechanical, and Titanium. However, there are mainly mass production experience of Lite-On, Sunny and Truly. Huawei's models mainly use Lite-On and Sunny's modules. O-Film and Samsung Electromechanics, with their strong factory capabilities, are now entering the field of dual camera modules. Namuga maintains a good communication relationship with various algorithm companies, and is also a supplier of Samsung mobile phones, and has gradually made efforts in the field of dual camera modules.

However, different functions have different requirements for modules. We continue to use the four functions mentioned above as an example:

Distance applications, the general approach is the size of the camera, the common camera specifications such as 13M + 2M, 13M + 5M. As shown below:

Or is to put two cameras far away, as shown below:

Through these two methods, the distance from the object to the lens can be better calculated.

Optical zoom dual camera module, the most important is the two cameras need to have different FOV, similar to the following figure:

Different FOVs produce different focus points, and then the optical zoom function can be achieved through an algorithm.

The general dimming enhancement is performed by an RGB full-color camera + a MONO black and white camera, as shown below:

In terms of algorithm, the amount of incoming light is mainly read by the black-and-white camera on the right to compensate for the color of the left RGB camera.

3D shooting and modeling, this approach is somewhat similar to distance-related modules. Only 3D has higher precision for distance. In this case, it is more necessary to distance the two cameras farther away. Some of them even use infrared-assisted positioning to achieve distance measurement, and finally achieve 3D shooting and modeling.

The difficulties of dual camera module production:

Dual camera approach, there are two general: a total of substrate, or a total of stents. As shown below:

If a common substrate is used, two Camera Sensors are collectively placed on the same substrate, and then an FPC can be pulled out from the substrate. If there is a common bracket, as shown above, the Sensor is fixed by the bracket and each Sensor has its own substrate and its own FPC.

The advantages of a common substrate: The two Sensors can sit on the same plate and resist falling.

The disadvantage of the common substrate: low yield, resulting in expensive.

The advantages of a total stent: high yield, good prices.

Disadvantages of the common bracket: Because it is two independent Sensor modules, it needs to be in the same plane through AA calibration, which is difficult and has a poor fall resistance.

In short, both methods have their own advantages and disadvantages. At present, only Huawei uses a common substrate method. But either way, the current yield rate is poor, so the cost is still very high.

If the algorithm is improved and the module yield is improved, the dual camera still has many advantages worthy of everyone to do.

However, due to the different requirements placed on the module for different functions, dual cameras may not be able to meet the requirements of the camera.

Since Google launched Project Tango, it was smart to propose a concept of three cameras: when distance measurement and 3D modeling are required, two distant cameras can be used; if optical zoom and dim light compensation are used, it can be used Two close shots. Even this Depth Sense can combine infrared to measure distance more accurately.

In any case, with the evolution of dual camera algorithms and the increase in VR demand, cameras are playing a bigger role. Dual-camera, even dual-camera + post-three-camera phones are likely to become more common next year.