{"id":5585,"date":"2022-09-02T14:50:16","date_gmt":"2022-09-02T18:50:16","guid":{"rendered":"https:\/\/blogs.mathworks.com\/videos\/?p=5585"},"modified":"2022-09-02T14:50:16","modified_gmt":"2022-09-02T18:50:16","slug":"using-a-parfor-loop-to-store-the-html-of-many-web-pages","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/videos\/2022\/09\/02\/using-a-parfor-loop-to-store-the-html-of-many-web-pages\/","title":{"rendered":"Using a parfor Loop to Store the HTML of Many Web Pages"},"content":{"rendered":"<p>Previously, I used a parfor loop to make a <a href=\"https:\/\/blogs.mathworks.com\/videos\/2021\/06\/10\/using-parfor-to-make-many-web-requests\/\">bunch of web requests<\/a>. Here I want to use a parfor loop to download the HTML for a large number of pages on our website, then store the content in a string array, so that it can be searched later by another web app. I will repurpose the code I developed before.<\/p>\n<p>Features covered in this <a href=\"https:\/\/blogs.mathworks.com\/videos\/2015\/10\/29\/matlab-code-along-videos\/\">code-along<\/a> style video include:<\/p>\n<ul>\n<li>webread<\/li>\n<li>parfor<\/li>\n<\/ul>\n<p><div class=\"row\"><div class=\"col-xs-12 containing-block\"><div class=\"bc-outer-container add_margin_20\"><videoplayer><div class=\"video-js-container\"><video data-video-id=\"6311610453112\" data-video-category=\"blog\" data-autostart=\"false\" data-account=\"62009828001\" data-omniture-account=\"mathwgbl\" data-player=\"rJ9XCz2Sx\" data-embed=\"default\" id=\"mathworks-brightcove-player\" class=\"video-js\" controls><\/video><script src=\"\/\/players.brightcove.net\/62009828001\/rJ9XCz2Sx_default\/index.min.js\"><\/script><script>if (typeof(playerLoaded) === 'undefined') {var playerLoaded = false;}(function isVideojsDefined() {if (typeof(videojs) !== 'undefined') {videojs(\"mathworks-brightcove-player\").on('loadedmetadata', function() {playerLoaded = true;});} else {setTimeout(isVideojsDefined, 10);}})();<\/script><\/div><\/videoplayer><\/div><\/div><\/div><\/p>\n<p>Play the video in full screen mode for a better viewing experience.\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":"<div class=\"thumbnail thumbnail_asset asset_overlay video\"><a href=\"https:\/\/blogs.mathworks.com\/videos\/2022\/09\/02\/using-a-parfor-loop-to-store-the-html-of-many-web-pages\/?dir=autoplay\"><img decoding=\"async\" src=\"https:\/\/cf-images.us-east-1.prod.boltdns.net\/v1\/static\/62009828001\/5bd62db7-85ee-44fb-8f1d-8902472a86e6\/0b54d955-0e23-42d7-8896-ee9161215256\/1280x720\/match\/image.jpg\" onError=\"this.style.display ='none';\"\/><\/p>\n<div class=\"overlay_container\">\n      <span class=\"icon-video icon_color_null\"><time class=\"video_length\">28:17<\/time><\/span>\n      <\/div>\n<p>      <\/a><\/div>\n<p>Previously, I used a parfor loop to make a bunch of web requests. Here I want to use a parfor loop to download the HTML for a large number of pages on our website, then store the content in a string&#8230; <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/videos\/2022\/09\/02\/using-a-parfor-loop-to-store-the-html-of-many-web-pages\/\">read more >><\/a><\/p>\n","protected":false},"author":133,"featured_media":5597,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[27,4],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/posts\/5585"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/users\/133"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/comments?post=5585"}],"version-history":[{"count":6,"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/posts\/5585\/revisions"}],"predecessor-version":[{"id":5609,"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/posts\/5585\/revisions\/5609"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/media\/5597"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/media?parent=5585"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/categories?post=5585"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/tags?post=5585"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}