{"id":2451,"date":"2020-01-20T12:18:05","date_gmt":"2020-01-20T12:18:05","guid":{"rendered":"https:\/\/blogs.mathworks.com\/headlines\/?p=2451"},"modified":"2020-01-20T12:18:05","modified_gmt":"2020-01-20T12:18:05","slug":"computer-vision-algorithm-removes-the-water-from-underwater-images","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/headlines\/2020\/01\/20\/computer-vision-algorithm-removes-the-water-from-underwater-images\/","title":{"rendered":"Computer vision algorithm removes the water from underwater images"},"content":{"rendered":"

Underwater photography is hard to get right. Special filters, artificial lights, and top-of-the-line underwater cameras can help, but there\u2019s still a lot of water between the camera and the object in the photo. We\u2019ve become accustomed to the blue-green tint of underwater photography.<\/p>\n

 <\/p>\n

\"\"<\/a>

Before: A coral reef in the Red Sea, Israel. Image Credit Matan Yuval, Marine Imaging Lab, University of Haifa<\/p><\/div><\/p>\n

 <\/p>\n

How would the ocean look without water? \u00a0What are the true colors of a coral reef? Thanks to a new computer vision algorithm called Sea-thru,<\/a> we can see what an underwater scene would look like if it was photographed through the air instead of water.<\/p>\n

 <\/p>\n

\"\"<\/a>

After: The same image as above, after processing with Sea-thru algorithm. Image Credit: Matan Yuval, Marine Imaging Lab, University of Haifa<\/p><\/div><\/p>\n

 <\/p>\n

Sea-thru removes the visual distortion that occurs as light travels through the water to produce a color-accurate image. Sea-thru was developed by Dr. Derya Akkaynak, and engineer and oceanographer, and Dr. Tali Treibitz, an electrical engineer. And the results are not only stunning but also a physically accurate correction.<\/p>\n

According to Scientific American<\/a><\/em>, \u201cSea-thru’s image analysis factors in the physics of light absorption and scattering in the atmosphere, compared with that in the ocean, where the particles that light interacts with are much larger. Then the algorithm effectively reverses image distortion from water pixel by pixel, restoring lost colors.\u201d<\/p>\n

 <\/p>\n

<\/a>

Image Credit: Tom Shlesinger, Institute of Global Ecology, Florida Tech<\/p><\/div><\/p>\n

 <\/p>\n

Akkaynak developed Sea-thru<\/a> as a post-doctoral fellow at the Marine Imaging Lab<\/a>, run by Dr. Tali Treibitz, at the University of Haifa. Sea-thru is licensed to SeaErra Ltd.<\/p>\n

The reason for the blue-green tint in underwater photos is the way light travels through deep water, where blue and violet wavelengths are absorbed least compared to the other wavelengths. The more water the light travels through, the less red, yellow, and orange wavelengths reach the object. In coastal waters, the blue and green light is absorbed faster, leaving more red light and causing the dominant brown hue.<\/p>\n

There\u2019s also an issue of small particles in the water, which create backscatter or haze. The further the photographer is from the object in the photo, the hazier the object becomes, much like looking at an object through a light fog.<\/p>\n

How Sea-thru works<\/h2>\n

Sea-thru<\/a> works on raw images or videos taken with natural lighting, removing the need for expensive and difficult to set-up artificial underwater lighting. It also helps in reconstructing images of objects that are further away from the light source, where the underwater strobes don’t reach.<\/p>\n

The computer vision algorithm is a physics-based color reconstruction algorithm designed for underwater RGB-D images, where D<\/em> stands for the distance <\/em>from the camera to the object. The Sea-thru algorithm requires the distance between each pixel in the scene from the camera, as almost all parameters governing loss of colors and contrast depend on distance in some non-linear way.<\/p>\n

For her research, Akkaynak created a distance (range) map by capturing multiple images of the same scene. She captured images with a single camera from slightly different angles so that all of them overlap. Included in the images is a color chart placed by the object of interest. Then, using a commercial photogrammetry software called Agisoft Metashape Professional, Sea-thru built a 3D reconstruction<\/a> of each scene from these images and exported the distance maps once the 3D reconstruction is complete.<\/p>\n

 <\/p>\n

<\/div>\n

\"\"<\/a>

The darkest pixels used in the backscatter estimation are shown in red. Full research poster here<\/a>. Image credit: Akkaynak et.al.<\/p><\/div><\/p>\n

 <\/p>\n

The range maps are grayscale .tiff<\/em> files, which are then read into MATLAB<\/a> and combined with the RAW images. The images are processed according to this guide<\/a>. MATLAB is used to estimate backscatter by searching the image for very dark pixels or shadowed pixels. In the image above, the darkest pixels used in backscatter estimation are shown in red. The backscatter in the images is subtracted with MATLAB.<\/p>\n

In the next step, Sea-thru computed the relevant parameters of color attenuation using only the pixels and the depth map D and inverts them to reveal original colors. The attenuation of each wavelength of light is calculated with MATLAB. With this information, Sea-thru inverts the image to reveal the true color.<\/p>\n

 <\/p>\n

 <\/p>\n

\"\"<\/a>

A coral reef in the Red Sea, Israel. Image Credit: Matan Yuval, Marine Imaging Lab, University of Haifa<\/p><\/div><\/p>\n

 <\/p>\n

Sea-thru works on videos as well as still images. See a side-by-side video from Lake Tanganyika, Zambia below:<\/p>\n