{"id":9682,"date":"2023-05-08T09:02:27","date_gmt":"2023-05-08T13:02:27","guid":{"rendered":"https:\/\/blogs.mathworks.com\/student-lounge\/?p=9682"},"modified":"2023-05-08T09:02:27","modified_gmt":"2023-05-08T13:02:27","slug":"get-invisible-design-an-invisibility-cloak-using-matlab-and-simulink","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/student-lounge\/2023\/05\/08\/get-invisible-design-an-invisibility-cloak-using-matlab-and-simulink\/","title":{"rendered":"Get Invisible! Design an Invisibility Cloak using MATLAB and Simulink"},"content":{"rendered":"
\n
<\/div>\n

In today’s post, Maitreyee Mordekar joins us to talk about designing an invisibilty cloak. Over to you, Maitreyee..<\/em><\/p>\n

<\/div>\n
<\/div>\n
Let\u2019s get ready to design an invisibility cloak illusion! In this blog, we will go over the process of implementing an illusion of an Invisibility Cloak using a simple image processing technique of color-based thresholding and segmentation. The solution implements the concepts of performing image segmentation and analysing the regions of interest using interactive apps. At this point, I solemnly swear that I am up to no good \ud83d\ude09<\/div>\n
\"InvisibilityCLoak.gif\"<\/div>\n
Fig 1: Showcasing the Simulink model<\/span><\/div>\n

Unlocking the Algorithm (Alohamora Algorithm)<\/h2>\n
The illusion works on the concept of color-based segmentation and image multiplication. For all of you who are looking for the magical invisibility cloak, it is located here<\/span><\/a> (Don\u2019t be disappointed to find just the code and not a cloak here \ud83d\ude09 ).<\/div>\n
The algorithm multiplies a colour thresholder mask to a pre-configured background image to display the background in the coloured cloak area giving an illusion of invisibility. If the above sentence was a little overwhelming to understand, don\u2019t worry. We will go over this step by step!<\/div>\n
The illusion of creating an invisibility cloak has 3 parts:<\/div>\n
    \n
  1. Setup of the Background Image<\/li>\n
  2. Tune the foreground color (aka creating our invisible cloth)<\/li>\n
  3. Manipulate the image from the camera video stream<\/li>\n<\/ol>\n

    1. Setup of the Background Image<\/h3>\n
    The very first step would be to for us to capture a background image with a stationary camera and a stationary background. For this, we would have to ensure that we have a camera connected to our system and have the MATLAB Support Package for USB Webcams<\/span><\/a> installed. Then we set it up using the following command in the MATLAB Command Window:<\/div>\n
    \n
    \n
    cam = webcam;<\/span><\/div>\n<\/div>\n<\/div>\n
    Once the camera is setup, we take a snapshot of the background and save it into the MATLAB Workspace. Let\u2019s say following is the image that is captured.<\/div>\n
    \n
    \n
    backgroundImage = snapshot(cam);<\/span><\/div>\n<\/div>\n<\/div>\n
    \"\"<\/div>\n
    Fig 2: The background image<\/span><\/div>\n
    Save this as backgroundImage.<\/span><\/div>\n
    \n
    \n
    save(‘backgroundImage’<\/span>,‘backgroundImage’<\/span>);<\/span><\/div>\n<\/div>\n<\/div>\n

    2. Tune the Foreground Color<\/h3>\n
    Now, we must tune to detect and segment colors from the cloth that we would like to use as our invisibility cloak. For this example, we are using an orange object.<\/div>\n
    Let\u2019s detect the colors of the orange object using the Color Thresholder App<\/span><\/a>.<\/div>\n
    \"\"<\/div>\n
    Fig 3: Tuning the orange-colored foreground<\/div>\n
    \n
    \n
    colorThresholder<\/span><\/div>\n<\/div>\n<\/div>\n
    You can use the image with the colored object in the App and adjust the sliders to detect only the orange color. You can export<\/span> the detected color mask and then use it in Simulink.<\/div>\n
    \"Threshold.gif\"<\/div>\n
    Fig 4: Segmenting the foreground color using Color-Thresholder App<\/span><\/div>\n
    Also, if there are regions falsely detected as the cloak color, you can refine the detected mask using Image Segmenter App<\/span><\/a>.<\/div>\n
    \n
    \n
    imageSegmenter<\/span><\/div>\n<\/div>\n<\/div>\n
    The white pixels represent the areas that are detected as orange<\/span> and the black are areas that are detected as not orange. <\/span><\/div>\n
    \"\"<\/div>\n
    Fig 5: Segmented, orange-colored object<\/span><\/div>\n
    To understand the implementation in detail, you can watch the video on Making Vehicles and Robots See: Image Segmentation and Analysis<\/span><\/a> that introduces the concepts of performing image segmentation and analysing the regions of interest using interactive apps.<\/div>\n

    3. Image Manipulation of the Camera Video Stream<\/h3>\n
    The above two steps were carried out to setup the system by using the MATLAB Command Window. This step is the image manipulation to help us get the illusion will be performed using Simulink. Here, our foreground is the orange-colored object, and the background is everything else in an image.<\/div>\n
    The method adds up two images which are separate from each other that eventually leads us an illusion of invisibility.<\/div>\n

    Revealing the background (Revelio Background)<\/h4>\n
    In the first image, we take the background and multiply it with the orange-colored object mask. That leaves us with an image which has the background image at the region of the image frame which had the orange object. The rest of the image would have pixels which are black (i.e. a pixel value of 0).<\/div>\n
    \"\"<\/div>\n
    Fig 6: Algorithm for revealing the background<\/span><\/div>\n

    Obscuring the foreground (Obscuro Foreground)<\/h4>\n
    For the second image, we manipulate the streaming input video to stream everything except the part of the image which we have in Image 1. For this, we work with an operation that is an inversion of the mask that we used in the last image. We use the not operation so that the 0s in the image become 1s and vice versa. We finally multiply this image with the incoming video frame.<\/div>\n
    \"\"<\/div>\n
    Fig 7: Algorithm for obscuring the foreground<\/span><\/div>\n

    Adding the Images (Confirmarum Images)<\/h4>\n
    The last step is to add the two images together. Here, we add the foreground and the background images to give us the invisibility cloak illusion.<\/div>\n
    \"\"<\/div>\n
    Fig 8: Adding the background and foreground images<\/span><\/div>\n

    Final Simulink Model<\/h4>\n
    Now that we have understood the algorithm, the next step is to configure it all and run it to become invisible. You can find the models in this File Exchange submission<\/span><\/a>.<\/div>\n
    \"\"<\/div>\n
    Fig 9: Final Simulink model<\/span><\/div>\n

    Assumptions<\/h2>\n
    Now, the big question is, at what instances does this algorithm work \u2013 what have we assumed? This is a simple algorithm that performs color thresholding. We assume that:<\/div>\n
      \n
    1. The web camera is stationary<\/li>\n
    2. The background of the image is stationary does not have any moving objects<\/li>\n
    3. The lighting conditions do not change<\/li>\n<\/ol>\n
      Do check the example out to have some fun with image processing and make yourself invisible \ud83d\ude42 . Mischief managed!<\/div>\n<\/div>\n