Today's post is by guest blogger Isaac Bruss. Isaac has been a course developer for MathWorks since 2019. His PhD is in computational physics from UMass Amherst. Isaac has helped launch and support several courses on Coursera, on topics such as robotics, machine learning, and computer vision. Isaac's post is based on a lesson from the Image Processing for Engineering and Science Specialization on Coursera.
In this post, I’ll analyze a video of a container filling with liquid. Now, that might not sound like the most exciting thing in the world, but imagine, for example, you're a quality control engineer responsible for ensuring that the rate of filling bottles is consistent. You need to determine if this rate is constant or varies over time. To answer this question, you’d need to process each individual frame from the video file. But where would you start?
Well, in this case, I started out with the Video Viewer App to become more familiar with the video. This app allows you to see the video's size, frame rate, and total number of frames along the bottom bar. By using this app, you can quickly get a sense of what I’m working with by playing the video and navigating between frames.
Creating the Segmentation Function
With the goal of measuring the liquid within the container, I needed a segmentation function that can separate the liquid from the background. To ensure I develop a robust algorithm, I export a few representative frames to create and test a segmentation function.
The Color Thresholder App is a powerful tool for interactively segmenting images based on color. This app is particularly useful for images that have distinct color differences between the object of interest and the background. In this case, I used the region selection tool to highlight pixels of interest within the L*a*b* color space. This allowed me to isolate the dark liquid from both the purple background and white foam at the top.
Once I'm satisfied with my thresholding results, I export this segmentation function from the app as a “.m” file named “liquidMask”.
Check the Segmentation Function
This custom function was developed using only the few frames I exported from the video. But to be confident in the results, I need to ensure that it accurately isolates the dark liquid in all the frames. To do this, I first create a video reader object.
v = VideoReader("liquidVideo.mp4")
Then I loop through the frames using the hasFrame function. Inside the loop, each frame is passed into the custom liquidMask function, and the resulting mask is displayed side-by-side using the montage function.
while hasFrame(v) % Loop through all frames
img = readFrame(v); % Read a frame
bw = liquidMask(img); % Apply the custom segmentation function
The resulting segmented video frames look great! Further morphological operations to better segment the region of interest are possible, but to get a quick example up and running they aren’t needed.
Analyzing Liquid Percentage
Now I can move on to the final part: calculating the percentage of the container that is filled with water at each frame.
To get started, I need to rewind the video to the beginning using the CurrentTime property of the VideoReader.
Next, I initialize two variables to store values for each frame: the percentage of the container filled, and the time stamp.
percents = zeros(nFrames,1);
times = zeros(nFrames,1);
Now I’m ready to use a for-loop to read through the video, segment each image, and calculate the percentage of the container filled at each frame. The percentage calculation is done by counting the number of pixels that are filled with liquid and dividing it by the total number of pixels.
for i = 1:nFrames % Loop through all frames
img = readFrame(v); % Read a frame
bw = liquidMask(img); % Create the binary mask using the custom function
liquidPart = nnz(bw); % Calculate the number of liquid (true) pixels
perc = liquidPart/numel(bw); % Calculate percentage filled with liquid
time = v.CurrentTime; % Mark the current time
percents(i) = perc; % Save the fill-percentage value
times(i) = time; % Save the time stamp
Now that the data has been gathered, I plot the percentage of the container filled versus time to see how quickly the liquid fills the container.
title('Percentage of Container Filled with Liquid vs Time')
Look at that! The fill rate is not consistent. From the plot, I see that the flow slowed down around 8 seconds, increased at 15 seconds, and slowed again at 20 seconds.