Quantitative Video Analysis: Measuring a Container Filling with Liquid

作者 Steve Eddins, April 12, 2023

12 次查看（过去 30 天） | 0 个赞 | 0 个评论

Today's post is by guest blogger Isaac Bruss. Isaac has been a course developer for MathWorks since 2019. His PhD is in computational physics from UMass Amherst. Isaac has helped launch and support several courses on Coursera, on topics such as robotics, machine learning, and computer vision. Isaac's post is based on a lesson from the Image Processing for Engineering and Science Specialization on Coursera.

Introduction

In this post, I’ll analyze a video of a container filling with liquid. Now, that might not sound like the most exciting thing in the world, but imagine, for example, you're a quality control engineer responsible for ensuring that the rate of filling bottles is consistent. You need to determine if this rate is constant or varies over time. To answer this question, you’d need to process each individual frame from the video file. But where would you start?

Well, in this case, I started out with the Video Viewer App to become more familiar with the video. This app allows you to see the video's size, frame rate, and total number of frames along the bottom bar. By using this app, you can quickly get a sense of what I’m working with by playing the video and navigating between frames.

Creating the Segmentation Function

With the goal of measuring the liquid within the container, I needed a segmentation function that can separate the liquid from the background. To ensure I develop a robust algorithm, I export a few representative frames to create and test a segmentation function.

The Color Thresholder App is a powerful tool for interactively segmenting images based on color. This app is particularly useful for images that have distinct color differences between the object of interest and the background. In this case, I used the region selection tool to highlight pixels of interest within the L*a*b* color space. This allowed me to isolate the dark liquid from both the purple background and white foam at the top. 

Once I'm satisfied with my thresholding results, I export this segmentation function from the app as a “.m” file named “liquidMask”.

Check the Segmentation Function

This custom function was developed using only the few frames I exported from the video. But to be confident in the results, I need to ensure that it accurately isolates the dark liquid in all the frames. To do this, I first create a video reader object.

v = VideoReader("liquidVideo.mp4")	

v = 
  VideoReader with properties:

   General Properties:
            Name: 'liquidVideo.mp4'
            Path: 'C:\MATLAB Drive\Play\Blog Posts\video analysis'
        Duration: 24
     CurrentTime: 0.1000
       NumFrames: <Calculating...> learn more

   Video Properties:
           Width: 120
          Height: 216
       FrameRate: 10
    BitsPerPixel: 24
     VideoFormat: 'RGB24'

Then I loop through the frames using the hasFrame function. Inside the loop, each frame is passed into the custom liquidMask function, and the resulting mask is displayed side-by-side using the montage function.

while hasFrame(v) % Loop through all frames

    img = readFrame(v); % Read a frame

    bw = liquidMask(img); % Apply the custom segmentation function

    montage({img,bw},'BorderSize',[20,20],'BackgroundColor',[0.5,0.5,0.5]);

    drawnow

end

The resulting segmented video frames look great! Further morphological operations to better segment the region of interest are possible, but to get a quick example up and running they aren’t needed. 

Analyzing Liquid Percentage

Now I can move on to the final part: calculating the percentage of the container that is filled with water at each frame.

To get started, I need to rewind the video to the beginning using the CurrentTime property of the VideoReader.

v.CurrentTime = 0;	

Next, I initialize two variables to store values for each frame: the percentage of the container filled, and the time stamp.

nFrames = v.NumFrames;

percents = zeros(nFrames,1);

times = zeros(nFrames,1);	

Now I’m ready to use a for-loop to read through the video, segment each image, and calculate the percentage of the container filled at each frame. The percentage calculation is done by counting the number of pixels that are filled with liquid and dividing it by the total number of pixels.

for i = 1:nFrames % Loop through all frames

    img = readFrame(v); % Read a frame

    bw = liquidMask(img); % Create the binary mask using the custom function

    liquidPart = nnz(bw); % Calculate the number of liquid (true) pixels

    perc = liquidPart/numel(bw); % Calculate percentage filled with liquid

    time = v.CurrentTime; % Mark the current time

    percents(i) = perc; % Save the fill-percentage value

    times(i) = time; % Save the time stamp

end

Now that the data has been gathered, I plot the percentage of the container filled versus time to see how quickly the liquid fills the container.

figure

plot(times,percents*100)

title('Percentage of Container Filled with Liquid vs Time')

xlabel('Time (seconds)')

ylabel('Percentage Filled')

Look at that! The fill rate is not consistent. From the plot, I see that the flow slowed down around 8 seconds, increased at 15 seconds, and slowed again at 20 seconds. 

This post is based on a lesson from the Image Processing for Engineering and Science Specialization on Coursera. There you’ll be able to complete many projects like this one, from segmenting cracks in images of concrete to counting cars in a video of traffic.