{"id":615,"date":"2018-10-18T15:22:31","date_gmt":"2018-10-18T15:22:31","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=615"},"modified":"2018-10-18T15:22:31","modified_gmt":"2018-10-18T15:22:31","slug":"deep-beer-designer","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2018\/10\/18\/deep-beer-designer\/","title":{"rendered":"Deep Beer Designer"},"content":{"rendered":"This post is from Ieuan Evans, who has created a very unique example combining deep learning with LSTM and beer. (Please drink responsibly!)\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">I love craft beer. Nowadays, there are so many choices that it can be overwhelming, which is a great problem to have! Lately I have found myself becoming lazy when it comes to carefully selecting beers in a bar, and I tend to just go for the beer with the best sounding name.<\/span>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">I started to wonder: Could MATLAB automatically analyze a list of names and select a beer for me? Why stop there? Could I get MATLAB to design a unique beer just for me?<\/span>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">In this example, I will show how to classify beer styles given the name, how to generate new beer names, and even automatically generate some tasting notes too.<\/span>\r\n<h6><\/h6>\r\n<p style=\"text-align: center;\"><strong>Happe Hill Hefeweizen<\/strong><\/p>\r\n\r\n\r\n<div id=\"attachment_619\" style=\"width: 235px\" class=\"wp-caption alignnone\"><img aria-describedby=\"caption-attachment-619\" decoding=\"async\" loading=\"lazy\" class=\"wp-image-619 size-medium\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2018\/10\/beer_image7-225x300.jpg\" alt=\"The nice blur effect is from image processing toolbox.\" width=\"225\" height=\"300\" \/><p id=\"caption-attachment-619\" class=\"wp-caption-text\">\"A rich and fruity traditional Marthe Belgian yeast. Full-bodied with nutty undertones and a slightly sweet fruit flavor.\"<\/p><\/div>\r\n<p style=\"text-align: center;\">(MATLAB-generated name and tasting notes. Not bad!)<\/p>\r\n\r\n\r\n<hr width=\"50%\" \/>\r\n\r\n<h2>Import Data<\/h2>\r\n<h6><\/h6>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">There are two data sources available for this example:<\/span>\r\n<ul class=\"S3\">\r\n \t<li class=\"S4\">A dataset of craft beers from Kaggle: <a href=\"https:\/\/www.kaggle.com\/nickhould\/craft-cans\">https:\/\/www.kaggle.com\/nickhould\/craft-cans<\/a><\/li>\r\n \t<li class=\"S4\">A beer list of beers from the Cambridge Beer Festival in the UK: <a href=\"https:\/\/www.cambridgebeerfestival.com\/products\/cbf45-beer\">https:\/\/www.cambridgebeerfestival.com\/products\/cbf45-beer<\/a><\/li>\r\n<\/ul>\r\n<!--\r\n<h3 class=\"S5\">Craft Beers Dataset<\/h3>\r\n-->\r\n<span style=\"font-size: 14px;\">Load the craft beers data from Kaggle.<\/span>\r\n<div class=\"CodeBlock\">\r\n<div class=\"inlineWrapper\">\r\n<pre class=\"S6\">rng(0)\r\nfilename = \"beers.csv\";\r\ndataKaggle = readtable(filename,'TextType','string','Encoding','UTF-8');<\/pre>\r\n<\/div>\r\n<span style=\"font-size: 14px;\">View a random sample of the data.<\/span>\r\n<div class=\"CodeBlock\">\r\n<div class=\"inlineWrapper\">\r\n<pre class=\"S6\">idx = randperm(size(dataKaggle,1),10);\r\ndisp(dataKaggle(idx,[\"name\" \"style\"]))\r\n<\/pre>\r\n<\/div>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td style=\"padding: 1px 5px; text-align: center;\"><strong>Name<\/strong><\/td>\r\n<td style=\"padding: 1px 5px; text-align: center;\"><strong>Style<\/strong><\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 0px 5px;\">_______________________________________<\/td>\r\n<td style=\"padding: 0px 5px;\">_______________________________________<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Walloon (2014)\"<\/td>\r\n<td style=\"padding: 1px 5px;\">\"Saison \/ Farmhouse Ale\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Yoshi's Nectar\"<\/td>\r\n<td style=\"padding: 1px 5px;\">\"California Common \/ Steam Beer\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"1327 Pod's ESB\"<\/td>\r\n<td style=\"padding: 1px 5px;\">\"Extra Special \/ Strong Bitter (ESB)\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Parade Ground Coffee Porter\"<\/td>\r\n<td style=\"padding: 1px 5px;\">\"American Porter\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Perpetual Darkness\"<\/td>\r\n<td style=\"padding: 1px 5px;\">\"Belgian Strong Dark Ale\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"La Frontera Premium IPA\"<\/td>\r\n<td style=\"padding: 1px 5px;\">\"American IPA\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Canyon Cream Ale\"<\/td>\r\n<td style=\"padding: 1px 5px;\">\"Cream Ale\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Pace Setter Belgian Style Wit\"<\/td>\r\n<td style=\"padding: 1px 5px;\">\"Witbier\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Squatters Hop Rising Double IPA\"<\/td>\r\n<td style=\"padding: 1px 5px;\">\"American Double \/ Imperial IPA\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Good Vibes IPA\"<\/td>\r\n<td style=\"padding: 1px 5px;\">\"American IPA\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<h6><\/h6>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">Load the data from the Cambridge Beer Festival, which in addition to names and styles, also contains tasting notes. Extract the data using the HTML parsing tools from Text Analytics Toolbox.<\/span>\r\n<pre>url = \"https:\/\/www.cambridgebeerfestival.com\/products\/cbf44-beer\";\r\ncode = webread(url);\r\ntree = htmlTree(code);<\/pre>\r\n<span style=\"font-size: 14px;\">Extract the beer names.<\/span>\r\n<pre>subtrees = findElement(tree,\"span[class=\"\"productname\"\"]\");\r\nname = extractHTMLText(subtrees);<\/pre>\r\n<span style=\"font-size: 14px;\">Extract the tasting notes.<\/span>\r\n<pre>subtrees = findElement(tree,\"span[class=\"\"tasting\"\"]\");\r\nnotes = extractHTMLText(subtrees);\r\ndataCambridge = table(name,notes);<\/pre>\r\n<span style=\"font-size: 14px;\">Visualize the tasting notes in a word cloud. The <\/span><span style=\"font-size: 14px; font-family: Consolas;\">wordcloud<\/span><span style=\"font-size: 14px;\"> function in Text Analytics Toolbox creates word clouds directly from string data.<\/span>\r\n<pre>figure\r\nwordcloud(notes);\r\ntitle(\"Tasting Notes\")\r\n<\/pre>\r\n<h2><img decoding=\"async\" loading=\"lazy\" width=\"560\" height=\"420\" class=\"alignnone size-full wp-image-641\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2018\/10\/wordcloud.png\" alt=\"\" \/><\/h2>\r\n<h2><\/h2>\r\n<span style=\"color: #e67e22; font-size: 18px;\"><strong>Classify Beer Style<\/strong><\/span>\r\n\r\n<span style=\"font-size: 14px;\">First, using the Kaggle data, create a long short-term memory (LSTM) deep learning model to classify the beer style given the name.\r\nVisualize the distribution of the beer styles using a word cloud.<\/span>\r\n<pre>textData = dataKaggle.name;\r\nlabels = categorical(dataKaggle.style);\r\nfigure\r\nwordcloud(labels);\r\ntitle(\"Beer Styles\")<\/pre>\r\n<img decoding=\"async\" loading=\"lazy\" width=\"560\" height=\"420\" class=\"alignnone size-full wp-image-643\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2018\/10\/wordcloud2.png\" alt=\"\" \/>\r\n\r\nAs you can see in the wordcloud, the styles are very imbalanced, with some styles containing only a few instances. To improve the model, remove the styles with fewer than 5 instances, and then split the data into 90% training and 10% testing partitions.\r\n<h6><\/h6>\r\n(The details of the data preparation can be found in the full example file)\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">Convert each beer name to a sequence of integers, where each integer represents a character.<\/span>\r\n\r\n<!--\r\n<pre>cvp = cvpartition(labels,'HoldOut',0.1);\r\ntextDataTrain = textData(training(cvp));\r\nlabelsTrain = labels(training(cvp));\r\ntextDataTest = textData(test(cvp),:);\r\nlabelsTest = labels(test(cvp),:);\r\n<\/pre>\r\n-->\r\n<!--Convert each beer name to a sequence of integers, where each integer represents a character. The sequences are row vectors of integers of varying lengths.\r\n<pre>XTrain = cellfun(@double,textDataTrain,'UniformOutput',false);\r\nXTest = cellfun(@double,textDataTest,'UniformOutput',false);<\/pre>\r\n-->\r\n<!--\r\n<pre>XTrain(1:5)<\/pre>\r\n-->\r\n\r\n<span style=\"font-size: 14px;\">The responses are the beer styles.<\/span>\r\n<pre>YTrain = labelsTrain;\r\nYTest = labelsTest;\r\nYTrain(1:6)<\/pre>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td style=\"padding-left: 10px;\">ans = 6x1 string array<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">American Pale Lager<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">American IPA<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">American Double \/ Imperial IPA<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">American IPA<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">Oatmeal Stout<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<h6><\/h6>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">Next create the deep learning network architecture. Use a word embedding layer to learn an embedding of characters and map the integers to vectors. Use a bidirectional LSTM (BiLSTM) layer to learn bidirectional long-term dependencies between the characters in the beer names.<\/span>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">To learn stronger interactions between the hidden units of the BiLSTM layer, include an extra fully connected layer of size 50. Use dropout layers to help prevent the network from overfitting.<\/span>\r\n<h6><\/h6>\r\n<pre>numFeatures = 1;\r\nembeddingDimension = 100;\r\nnumCharacters = max([XTrain{:}]);\r\nnumClasses = numel(categories(YTrain));\r\nlayers = [\r\n    sequenceInputLayer(numFeatures)\r\n    wordEmbeddingLayer(embeddingDimension,numCharacters)\r\n    bilstmLayer(200,'OutputMode','last')\r\n    dropoutLayer(0.5)\r\n    fullyConnectedLayer(50)\r\n    dropoutLayer(0.5)\r\n    fullyConnectedLayer(numClasses)\r\n    softmaxLayer\r\n    classificationLayer];\r\n<\/pre>\r\n<span style=\"font-size: 14px;\">Specify the training options.<\/span>\r\n<pre>options = trainingOptions('adam', ...\r\n    'MaxEpochs',100, ...\r\n    'InitialLearnRate',0.01, ...\r\n    'GradientThreshold',2, ...\r\n    'Shuffle','every-epoch', ...\r\n    'ValidationData',{XTest,YTest}, ...\r\n    'ValidationFrequency',80, ...\r\n    'Plots','training-progress', ...\r\n    'Verbose',false);<\/pre>\r\n<span style=\"font-size: 14px;\">Train the network.<\/span>\r\n<pre>beerStyleNet = trainNetwork(XTrain,YTrain,layers,options);<\/pre>\r\n<img decoding=\"async\" loading=\"lazy\" width=\"1736\" height=\"1039\" class=\"alignnone size-full wp-image-645\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2018\/10\/training_progress.png\" alt=\"\" \/>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">Here, we can see that the model overfits. The model has effectively memorized the training data, but not generalized well enough to get as high accuracy on the test data.<\/span>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">This is perhaps expected: lots of beer names don't given that much away when it comes to the style, so the network has little to work with. Some are easy to classify since they contain the style of the beer in the name.<\/span>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">For example, what style of beer do you think the following are? Can you beat the classifier?<\/span>\r\n<pre>idx = [1 4 5 8 9 10 12 14 15 17];\r\ntextDataTest(idx)\r\n\r\n<\/pre>\r\n<h6><\/h6>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td style=\"padding-left: 10px;\">ans = 10x1 string array<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Sophomoric Saison\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Divided Sky\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Honey Kolsch\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Alaskan Amber\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"California Lager\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Brotherhood Steam\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Angry Orchard Apple Ginger\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Long Leaf\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"This Season's Blonde\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Raja\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">Compare your guesses vs. predictions made by the network vs. the correct labels<\/span>\r\n<pre>YPred = classify(beerStyleNet,XTest);\r\ndisp(table(textDataTest(idx),YPred(idx),YTest(idx),'VariableNames',[\"Name\" \"Prediction\" \"True\"]))<\/pre>\r\n<h6><\/h6>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td style=\"padding: 0px 5px; text-align: center;\"><strong>Name<\/strong><\/td>\r\n<td style=\"padding: 0px 5px; text-align: center;\"><strong>Prediction<\/strong><\/td>\r\n<td style=\"padding: 0px 5px; text-align: center;\"><strong>True<\/strong><\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 2px 5px;\">______________________________<\/td>\r\n<td style=\"padding: 1px 2px 5px;\">______________________________<\/td>\r\n<td style=\"padding: 1px 2px 5px;\">______________________________<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Sophomoric Saison\"<\/td>\r\n<td style=\"padding: 1px 5px;\">Saison \/ Farmhouse Ale<\/td>\r\n<td style=\"padding: 1px 5px;\">Saison \/ Farmhouse Ale<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Divided Sky\"<\/td>\r\n<td style=\"padding: 1px 5px;\">American Amber \/ Red Ale<\/td>\r\n<td style=\"padding: 1px 5px;\">American IPA<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Honey Kolsch\"<\/td>\r\n<td style=\"padding: 1px 5px;\">K\u00f6lsch<\/td>\r\n<td style=\"padding: 1px 5px;\">K\u00f6lsch<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Alaskan Amber\"<\/td>\r\n<td style=\"padding: 1px 5px;\">American Amber \/ Red Ale<\/td>\r\n<td style=\"padding: 1px 5px;\">Altbier<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"California Lager\"<\/td>\r\n<td style=\"padding: 1px 5px;\">American Amber \/ Red Lager<\/td>\r\n<td style=\"padding: 1px 5px;\">American Amber \/ Red Lager<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Brotherhood Steam\"<\/td>\r\n<td style=\"padding: 1px 5px;\">American Pale Wheat Ale<\/td>\r\n<td style=\"padding: 1px 5px;\">California Common \/ Steam Beer<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Angry Orchard Apple Ginger\"<\/td>\r\n<td style=\"padding: 1px 5px;\">Cider<\/td>\r\n<td style=\"padding: 1px 5px;\">Cider<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Long Leaf\"<\/td>\r\n<td style=\"padding: 1px 5px;\">Munich Helles Lager<\/td>\r\n<td style=\"padding: 1px 5px;\">American IPA<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"This Season's Blonde\"<\/td>\r\n<td style=\"padding: 1px 5px;\">Cream Ale<\/td>\r\n<td style=\"padding: 1px 5px;\">American Blonde Ale<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 1px 5px;\">\"Raja\"<\/td>\r\n<td style=\"padding: 1px 5px;\">Fruit \/ Vegetable Beer<\/td>\r\n<td style=\"padding: 1px 5px;\">American Double \/ Imperial IPA<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<!-- Name Prediction ____________________________ __________________________ \"Sophomoric Saison\" Saison \/ Farmhouse Ale \"Divided Sky\" American Amber \/ Red Ale \"Honey Kolsch\" K\u00f6lsch \"Alaskan Amber\" American Amber \/ Red Ale \"California Lager\" American Amber \/ Red Lager \"Brotherhood Steam\" American Pale Wheat Ale \"Angry Orchard Apple Ginger\" Cider \"Long Leaf\" Munich Helles Lager \"This Season's Blonde\" Cream Ale \"Raja\" Fruit \/ Vegetable Beer -->\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">So, can I use this network to select a beer for me? Suppose the test set contains all the beers available at a bar. I tend to go for some kind of IPA. Let's see which of these beers are classified as an IPA. This could be any of the class labels containing \"IPA\".<\/span>\r\n<h6><\/h6>\r\n<pre>classNames = string(beerStyleNet.Layers(end).Classes);\r\nidx = contains(classNames,\"IPA\");\r\nclassNamesIPA = classNames(idx)<\/pre>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td style=\"padding-left: 10px;\">ans = 5x1 string array<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"American IPA\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"American IPA\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"American White IPA\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Belgian IPA\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"English India Pale Ale (IPA)\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<pre>[YPred,scores] = classify(beerStyleNet,XTest);\r\nidx = contains(<span style=\"color: #000000;\">string<\/span>(YPred),\"IPA\");\r\nselection = textDataTest(idx);\r\n<\/pre>\r\n\r\n<span style=\"font-size: 14px;\">Let's see what proportion of these actually are labelled as some kind of IPA.<\/span>\r\n<pre>accuracyIPA = mean(contains(string(YTest(idx)),\"IPA\"))\r\n<\/pre>\r\n<h6><\/h6>\r\naccuracyIPA = 0.7241\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">View the top 10 predictions sorted by classification score. And to make it even more exciting let's exclude any names with \"IPA\" in the name<\/span>\r\n<h6><\/h6>\r\n<pre>topScores = max(scores(idx,:),[],2);\r\n[~,idxSorted] = sort(topScores,'descend');\r\nselectionSorted = selection(idxSorted);\r\n<span class=\"com\">% remove with IPA in the name<\/span>\r\nidx = contains(selectionSorted,[\"IPA\" \"India Pale Ale\"]);\r\nselectionSorted(idx) = [];\r\nselectionSorted(1:10)\r\n<\/pre>\r\n<h6><\/h6>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td style=\"padding-left: 10px;\">ans = 10x1 string array<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"American Idiot Ale (2012)\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Citra Faced\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Hopped on the High Seas (Calypso)\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Bengali Tiger\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"The Sword Iron Swan Ale\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"The 26th\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Isis\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"En Parfaite Harmonie\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Sanctified\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Sockeye Maibock\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">Looks like some good suggestions!<\/span>\r\n<h6><\/h6>\r\n<!-- ans = 10\u00d71 string array \"American Idiot Ale (2012)\" \"Citra Faced\" \"Hopped on the High Seas (Calypso)\" \"Bengali Tiger\" \"The Sword Iron Swan Ale\" \"The 26th\" \"Isis\" \"En Parfaite Harmonie\" \"Sanctified\" \"Sockeye Maibock\" -->\r\n\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-659 size-medium\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2018\/10\/20161030_142421-300x169.jpg\" alt=\"\" width=\"300\" height=\"169\" \/>\r\n<h6><\/h6>\r\n<h6><\/h6>\r\n<span style=\"color: #e67e22; font-size: 18px;\"><strong>Generate New Beer Names<\/strong><\/span>\r\n<h4><\/h4>\r\n<span style=\"font-size: 14px;\">We have created a deep network that does a reasonable job of finding a beer for me. My next desire is for MATLAB to design a beer for me. First it needs a name.\r\nTo do this, I'll use an LSTM network for sequence forecasting which predicts the next character of a sequence.\r\nTo improve the model, I'll also include the beer names from the Cambridge Beer Festival in the UK. Validation data is not helpful here, so we will train on all the data.<\/span>\r\n<h2><\/h2>\r\n<pre>textData = [dataKaggle.name; dataCambridge.name];<\/pre>\r\n<span style=\"font-size: 14px;\">To help with the generation, replace all the space characters with a \"\u00b7\" (middle dot) character, insert a start of text character at the beginning, and an end of text character at the end.<\/span>\r\n<pre>startOfTextCharacter = compose(\"\\x0002\");\r\nwhitespaceCharacter = compose(\"\\x00B7\");\r\nendOfTextCharacter = compose(\"\\x2403\");\r\n<\/pre>\r\n<span style=\"font-size: 14px;\">For the predictors, insert the start of text character before the beer names. For the responses, append the end of text character after the beer names. Here, the responses are the same as the predictors, shifted by one time step.<\/span>\r\n<pre>textDataPredictors = startOfTextCharacter + replace(textData,\" \",whitespaceCharacter);\r\ntextDataResponses = replace(textData,\" \",whitespaceCharacter) + endOfTextCharacter;\r\n<\/pre>\r\n<pre>XTrain = cellfun(@double,textDataPredictors,'UniformOutput',false);\r\nYTrain = cellfun(@(Y) categorical(cellstr(Y')'),textDataResponses,'UniformOutput',false);<\/pre>\r\n<span style=\"font-size: 14px;\">View the first sequence of predictors and responses.<\/span>\r\n<pre>XTrain{1}<\/pre>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td style=\"padding-left: 10px;\">ans = 1x9<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px; font-size: 12px; font-family: consolas;\">2 80 117 98 183 66 101 101 114<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<pre>YTrain{1}\r\n<\/pre>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td style=\"padding-left: 10px;\">ans = 1x9 categorical<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px; font-size: 12px; font-family: consolas;\">P u b \u00b7 B e e r \u2403<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<h6><\/h6>\r\n<h6><\/h6>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">Construct the network architecture.<\/span>\r\n<pre>numFeatures = 1;\r\nnumClasses = numel(categories([YTrain{:}]));\r\nnumCharacters = max([XTrain{:}]);\r\n\r\nlayers = [\r\n    sequenceInputLayer(numFeatures)\r\n    wordEmbeddingLayer(200,numCharacters)\r\n    lstmLayer(400)\r\n    dropoutLayer(0.5)\r\n    fullyConnectedLayer(numClasses)\r\n    softmaxLayer\r\n    classificationLayer];\r\n<\/pre>\r\n<span style=\"font-size: 14px;\">Specify the training options.<\/span>\r\n<pre>options = trainingOptions('adam', ...\r\n    'InitialLearnRate',0.01, ...\r\n    'GradientThreshold',2, ...\r\n    'Shuffle','every-epoch', ...\r\n    'Plots','training-progress', ...\r\n    'Verbose',false);\r\n\r\n<\/pre>\r\n<span style=\"font-size: 14px;\">Train the network.<\/span>\r\n<pre>beerNameNet = trainNetwork(XTrain,YTrain,layers,options);\r\n<\/pre>\r\n<img decoding=\"async\" loading=\"lazy\" width=\"1736\" height=\"1039\" class=\"alignnone size-full wp-image-647\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2018\/10\/training_progress_2.png\" alt=\"\" \/>\r\n<h4><\/h4>\r\n<span style=\"font-size: 14px;\">Here, the network might look like it is not doing particularly well. Again, this might be expected. To get high accuracy, the network must generate the training data exactly. We don't want the network to overfit too much because the network will simply generate the training data.<\/span>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">Generate some beer names using the<\/span><span style=\"font-size: 14px; font-family: Consolas;\"> generateText <\/span><span style=\"font-size: 14px;\">function, which is included in the full example file at the end of the post.<\/span>\r\n<pre>numBeers = 30;\r\ngeneratedBeers = strings(numBeers,1);\r\nfor i = 1:numBeers\r\n    generatedBeers(i) = generateText(beerNameNet,startOfTextCharacter,whitespaceCharacter,endOfTextCharacter);\r\nend\r\n<\/pre>\r\n<span style=\"font-size: 14px;\">Sometimes, the network might simply predict beer names from the training data. Remove them.<\/span>\r\n<pre>idx = ismember(generatedBeers,textData);\r\ngeneratedBeers(idx) = [];\r\n<\/pre>\r\n<span style=\"font-size: 14px;\">View the generated beers.<\/span>\r\n<pre> generatedBeers<\/pre>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td style=\"padding-left: 10px;\">generatedBeers =<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Firis Amber\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Sprecian Claisper\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Worther Pale Ale\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Ma's Canido Winter Ale\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Hop Roust\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Honey Fuddel Pilsner\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Slowneck Lager\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"CuDas Colora Lager\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"No Ryer Pilsner\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding-left: 30px;\">\"Dark Light IPA\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<span style=\"color: #e67e22; font-size: 18px;\"><strong>Generate Tasting Notes<\/strong><\/span>\r\n<h5><\/h5>\r\n<span style=\"font-size: 14px;\">We have our beer names, we now need some tasting notes.\r\nSimilar to the name generator, create a tasting note generator from the Cambridge Beer Festival notes.<\/span>\r\n<pre>textData = dataCambridge.notes;<\/pre>\r\n<span style=\"font-size: 14px;\">As before, to help with the name generation, replace all the space characters with a \"\u00b7\" (middle dot) character, insert a start of text character at the beginning, and an end of text character at the end.<\/span>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">Once again, define the network architecture, specify the training options, and train the network.<\/span> <em>(details are found in the main example file - link at the very end of this post)<\/em>\r\n\r\n<img decoding=\"async\" loading=\"lazy\" width=\"1736\" height=\"1039\" class=\"alignnone size-full wp-image-649\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2018\/10\/training_progress_3.png\" alt=\"\" \/>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">Generate some tasting notes using the generateText function, listed at the end of the example.<\/span>\r\n<br\/>\r\n\r\n<pre>numBeers = 5;\r\nfor i = 1:numBeers\r\n    generatedNotes = generateText(beerNotesNet,startOfTextCharacter,whitespaceCharacter,endOfTextCharacter)\r\nend\r\n<\/pre>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td style=\"padding: 8px 30px;\">\"This pale ale has a good assertive pale and full-bodied and lagerong aftertaste.\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 8px 30px;\">\"A full-bodied Imperial stout with flavour with a slight but fuity bite from The Fussion of roasted, malty flavours and a delicate character that is also present in the aftertaste with a silk stout. Unfined.\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 8px 30px;\">\"Light copper traditional bitter with good malt flavours. Brewed with the finest English Maris Otter taste and a rowner fruit and bitter sweet finish.\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 8px 30px;\">\"Stout brewed with a variety of flavoursomen. Unfined.\"<\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"padding: 8px 30px;\">\"Mixed malt and fruit start thise in the boil.\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<h5><\/h5>\r\n<span style=\"font-size: 14px;\">Perfect! I can now get started on brewing my own perfect beer. You can run the code many times to generate more names and tasting notes. My favorite design that I have seen so far is:<\/span>\r\n<h6><\/h6>\r\n<h6><\/h6>\r\n<strong>Hopky Wolf IPA<\/strong>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\"><em>\"This Double IPA has a big malt backbone and flavours of grapefruit, orange and lemon with an underlying floral quality and tent complex. Well balanced aroma reflects its taste. It's hopped with a blend of Fuggle and Golding hops.\"<\/em><\/span>\r\n<h6><\/h6>\r\n<span style=\"font-size: 14px;\">Now I just need MATLAB to automate the brewing process...<\/span>\r\n<h5><\/h5>\r\n\r\n\r\n<script language=\"JavaScript\"> <!-- \r\n    function grabCode_a710f144b80042c592b9fe35aab1fc59() {\r\n        \/\/ Remember the title so we can use it in the new page\r\n        title = document.title;\r\n\r\n        \/\/ Break up these strings so that their presence\r\n        \/\/ in the Javascript doesn't mess up the search for\r\n        \/\/ the MATLAB code.\r\n        t1='a710f144b80042c592b9fe35aab1fc59 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\r\n        t2='##### ' + 'SOURCE END' + ' #####' + ' a710f144b80042c592b9fe35aab1fc59';\r\n    \r\n        b=document.getElementsByTagName('body')[0];\r\n        i1=b.innerHTML.indexOf(t1)+t1.length;\r\n        i2=b.innerHTML.indexOf(t2);\r\n \r\n        code_string = b.innerHTML.substring(i1, i2);\r\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\r\n\r\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \r\n        \/\/ in the XML parser.\r\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\r\n        \/\/ doesn't go ahead and substitute the less-than character. \r\n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\r\n\r\n        copyright = 'Copyright 2018 The MathWorks, Inc.';\r\n\r\n        w = window.open();\r\n        d = w.document;\r\n        d.write('<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add copyright line at the bottom if specified.\r\n        if (copyright.length > 0) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\\n');\r\n\r\n        d.title = title + ' (MATLAB code)';\r\n        d.close();\r\n    }   \r\n     --> <\/script><p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\">Copyright 2018 The MathWorks, Inc.<br><a href=\"javascript:grabCode_a710f144b80042c592b9fe35aab1fc59()\"><span class=\"get_ml_code\">Get the MATLAB code <noscript>(requires JavaScript)<\/noscript><\/span><\/a><br><br>\r\n      <br><\/p><\/div><!--\r\na710f144b80042c592b9fe35aab1fc59 ##### SOURCE BEGIN #####\r\n\r\n%% Deep Beer Designer\r\n% I love craft beer. Though nowadays, I have become lazy when it comes to carefully \r\n% selecting beers in a bar and I tend to just go for the beers with names that \r\n% sound good to me.\r\n% \r\n% I have gotten lazier even still, and I would like MATLAB to automatically \r\n% analyze the names and select a beer for me. Why stop there? Could I get MATLAB \r\n% to design a beer for me?\r\n% \r\n% I will go through some interesting deep learning techniques and show how \r\n% to classify beer styles given the name, how to generate new beer names, and \r\n% how to generate some tasting notes too.\r\n%% Import Data\r\n% For this blog post, I will use two data sources:\r\n% \r\n% * This dataset of craft beers from Kaggle: <https:\/\/www.kaggle.com\/nickhould\/craft-cans \r\n% https:\/\/www.kaggle.com\/nickhould\/craft-cans> \r\n% * This beer list of beers from the Cambridge Beer Festival in the UK: <https:\/\/www.cambridgebeerfestival.com\/products\/cbf45-beer \r\n% https:\/\/www.cambridgebeerfestival.com\/products\/cbf45-beer> \r\n%% Craft Beers Dataset\r\n% Load the craft beers data from Kaggle.\r\n\r\nrng(0)\r\nfilename = \"beers.csv\";\r\ndataKaggle = readtable(filename,'TextType','string','Encoding','UTF-8');\r\n%% \r\n% View a random sample of the data.\r\n\r\nidx = randperm(size(dataKaggle,1),10);\r\ndisp(dataKaggle(idx,[\"name\" \"style\"]))\r\n%% Cambridge Beer Festival Dataset\r\n% Load the data from the Cambridge Beer Festival. Extract the data using the \r\n% HTML parsing tools from Text Analytics Toolbox.\r\n\r\nurl = \"https:\/\/www.cambridgebeerfestival.com\/products\/cbf44-beer\";\r\ncode = webread(url);\r\ntree = htmlTree(code);\r\n%% \r\n% Extract the beer names.\r\n\r\nsubtrees = findElement(tree,\"span[class=\"\"productname\"\"]\");\r\nname = extractHTMLText(subtrees);\r\n%% \r\n% Extract the tasting notes.\r\n\r\nsubtrees = findElement(tree,\"span[class=\"\"tasting\"\"]\");\r\nnotes = extractHTMLText(subtrees);\r\ndataCambridge = table(name,notes);\r\n%% \r\n% Visualize the tasting notes in a word cloud. The |wordcloud| function \r\n% in Text Analytics Toolbox creates word clouds directly from string data.\r\n\r\nfigure\r\nwordcloud(notes);\r\ntitle(\"Tasting Notes\")\r\n%% Classify Beer Style\r\n% First, using the Kaggle data, create a long short-term memory (LSTM) deep \r\n% learning model to classify the beer style given the name.\r\n% \r\n% Visualize the distribution of the beer styles using a word cloud.\r\n\r\ntextData = dataKaggle.name;\r\nlabels = categorical(dataKaggle.style);\r\nfigure\r\nwordcloud(labels);\r\ntitle(\"Beer Styles\")\r\n%% \r\n% The styles are very imbalanced, with some styles containing few instances. \r\n% To improve the model, remove the styles with fewer than 5 instances.\r\n\r\n[counts,names] = histcounts(labels);\r\nidx = counts < 5;\r\ninfrequentStyles = names(idx);\r\nidx = ismember(labels,infrequentStyles);\r\n\r\ntextData(idx) = [];\r\nlabels(idx) = [];\r\nlabels = removecats(labels);\r\n%% \r\n% Remove the beers without a labeled style at all.\r\n\r\nidx = ismissing(labels);\r\ntextData(idx) = [];\r\nlabels(idx) = [];\r\n%% \r\n% Next, split the data into 90% training and 10% testing partitions.\r\n\r\ncvp = cvpartition(labels,'HoldOut',0.1);\r\ntextDataTrain = textData(training(cvp));\r\nlabelsTrain = labels(training(cvp));\r\n\r\ntextDataTest = textData(test(cvp),:);\r\nlabelsTest = labels(test(cvp),:);\r\n%% \r\n% To input the beer names into the network, we must convert them to numeric \r\n% sequences.\r\n% \r\n% Convert each beer name to a sequence of integers, where each integer represents \r\n% a character.\r\n\r\nXTrain = cellfun(@double,textDataTrain,'UniformOutput',false);\r\nXTest = cellfun(@double,textDataTest,'UniformOutput',false);\r\n%% \r\n% The sequences are row vectors of integers of varying lengths.\r\n\r\nXTrain(1:5)\r\n%% \r\n% The responses are the beer styles.\r\n\r\nYTrain = labelsTrain;\r\nYTest = labelsTest;\r\nYTrain(1:10)\r\n%% \r\n% Next create the deep learning network architecture. Use a word embedding \r\n% layer to learn an embedding of characters and map the integers to vectors. Use \r\n% a bidirectional LSTM (BiLSTM) layer to learn bidirectional long-term dependencies \r\n% between the characters in the beer names.\r\n% \r\n% To learn stronger interactions between the hidden units of the BiLSTM layer, \r\n% include an extra fully connected layer of size 50. Use dropout layers to help \r\n% prevent the network from overfitting.\r\n\r\nnumFeatures = 1;\r\nembeddingDimension = 100;\r\nnumCharacters = max([XTrain{:}]);\r\nnumClasses = numel(categories(YTrain));\r\n\r\nlayers = [\r\n    sequenceInputLayer(numFeatures)\r\n    wordEmbeddingLayer(embeddingDimension,numCharacters)\r\n    bilstmLayer(200,'OutputMode','last')\r\n    dropoutLayer(0.5)\r\n    fullyConnectedLayer(50)\r\n    dropoutLayer(0.5)\r\n    fullyConnectedLayer(numClasses)\r\n    softmaxLayer\r\n    classificationLayer];\r\n%% \r\n% Specify the training options.\r\n\r\noptions = trainingOptions('adam', ...\r\n    'MaxEpochs',100, ...\r\n    'InitialLearnRate',0.01, ...\r\n    'GradientThreshold',2, ...\r\n    'Shuffle','every-epoch', ...\r\n    'ValidationData',{XTest,YTest}, ...\r\n    'ValidationFrequency',80, ...\r\n    'Plots','training-progress', ...\r\n    'Verbose',false);\r\n%% \r\n% Train the network.\r\n\r\nbeerStyleNet = trainNetwork(XTrain,YTrain,layers,options);\r\n%% \r\n% Here, we can see that the model overfits. The model has effectively memorized \r\n% the training data, but not generalized well enough to get as high accuracy on \r\n% the test data.\r\n% \r\n% This is perhaps expected: lots of beer names don't given that much away \r\n% when it comes to the style, so the network has little to work with. Some are \r\n% easy to classify since they contain the style of the beer in the name.\r\n% \r\n% For example, what style of beer do you think the following are? Can you \r\n% beat the classifier?\r\n\r\nidx = [1 4 5 8 9 10 12 14 15 17];\r\ntextDataTest(idx)\r\n%% \r\n% Compare your guesses to the predictions made by the network.\r\n\r\nYPred = classify(beerStyleNet,XTest);\r\ndisp(table(textDataTest(idx),YPred(idx),'VariableNames',[\"Name\" \"Prediction\"]))\r\n%% \r\n% Compare your guesses and the classifiers predictions the correct labels.\r\n\r\ndisp(table(textDataTest(idx),YPred(idx),YTest(idx),'VariableNames',[\"Name\" \"Prediction\" \"True\"]))\r\n%% \r\n% So, can I use this network to select a beer for me? Suppose the test set \r\n% contains all the beers available at a bar. I tend to go for some kind of IPA. \r\n% Let's see which of these beers are classified as an IPA. This could be any of \r\n% the class labels containing \"IPA\".\r\n\r\nclassNames = string(beerStyleNet.Layers(end).Classes);\r\nidx = contains(classNames,\"IPA\");\r\nclassNamesIPA = classNames(idx)\r\n[YPred,scores] = classify(beerStyleNet,XTest);\r\nidx = contains(string(YPred),\"IPA\");\r\nselection = textDataTest(idx);\r\n%% \r\n% Let's what proportion of these actually are labelled as some kind of IPA.\r\n\r\naccuracyIPA = mean(contains(string(YTest(idx)),\"IPA\"))\r\n%% \r\n% View the top 10 predictions sorted by classification score.\r\n\r\ntopScores = max(scores(idx,:),[],2);\r\n[~,idxSorted] = sort(topScores,'descend');\r\nselectionSorted = selection(idxSorted);\r\nselectionSorted(1:10)\r\n%% \r\n% It looks like the network has picked some of the obvious beers with \"IPA\" \r\n% in the name. This is not that exciting. Let's instead look at the top 10 that \r\n% do not contain \"IPA\".\r\n\r\nidx = contains(selectionSorted,[\"IPA\" \"India Pale Ale\"]);\r\nselectionSorted(idx) = [];\r\nselectionSorted(1:10)\r\n%% \r\n% Looks like some good suggestions!\r\n%% Generate New Beer Names\r\n% We have created a deep network that does a reasonable job of finding a beer \r\n% for me. My next desire is for MATLAB to design a beer for me. First it needs \r\n% a name.\r\n% \r\n% To do this, I'll use an LSTM network for sequence forecasting which predicts \r\n% the next character of a sequence.\r\n% \r\n% To improve the model, I'll also include the beer names from the Cambridge \r\n% Beer Festival in the UK. Validation data is not helpful here, so we will train \r\n% on all the data.\r\n\r\ntextData = [dataKaggle.name; dataCambridge.name];\r\n%% \r\n% To help with the generation, replace all the space characters with a \"\u00b7\" \r\n% (middle dot) character, insert a start of text character at the beginning, and \r\n% an end of text character at the end.\r\n\r\nstartOfTextCharacter = compose(\"\\x0002\");\r\nwhitespaceCharacter = compose(\"\\x00B7\");\r\nendOfTextCharacter = compose(\"\\x2403\");\r\n%% \r\n% For the predictors, specify insert the start of text character before \r\n% the beer names. For the responses, append the end of text character after the \r\n% beer names. Here, the responses are the same as the predictors, shifted by one \r\n% time step.\r\n\r\ntextDataPredictors = startOfTextCharacter + replace(textData,\" \",whitespaceCharacter);\r\ntextDataResponses = replace(textData,\" \",whitespaceCharacter) + endOfTextCharacter;\r\n%% \r\n% Convert the predictors to sequences of integers. Convert the responses \r\n% to categorical sequences.\r\n\r\nXTrain = cellfun(@double,textDataPredictors,'UniformOutput',false);\r\nYTrain = cellfun(@(Y) categorical(cellstr(Y')'),textDataResponses,'UniformOutput',false);\r\n%% \r\n% View the first sequence of predictors and responses.\r\n\r\nXTrain{1}\r\nYTrain{1}\r\n%% \r\n% Construct the network architecture.\r\n\r\nnumFeatures = 1;\r\nnumClasses = numel(categories([YTrain{:}]));\r\nnumCharacters = max([XTrain{:}]);\r\n\r\nlayers = [\r\n    sequenceInputLayer(numFeatures)\r\n    wordEmbeddingLayer(200,numCharacters)\r\n    lstmLayer(400)\r\n    dropoutLayer(0.5)\r\n    fullyConnectedLayer(numClasses)\r\n    softmaxLayer\r\n    classificationLayer];\r\n%% \r\n% Specify the training options.\r\n\r\noptions = trainingOptions('adam', ...\r\n    'InitialLearnRate',0.01, ...\r\n    'GradientThreshold',2, ...\r\n    'Shuffle','every-epoch', ...\r\n    'Plots','training-progress', ...\r\n    'Verbose',false);\r\n%% \r\n% Train the network.\r\n\r\nbeerNameNet = trainNetwork(XTrain,YTrain,layers,options);\r\n%% \r\n% Here, the network might look like it is not doing particularly well. Again, \r\n% this might be expected, to get high accuracy, the network must generate the \r\n% training data exactly. We don't want the network to overfit too much because \r\n% the network will simply generate the training data.\r\n% \r\n% Generate some beer names using the |generateText| function, listed at the \r\n% end of this example.\r\n\r\nnumBeers = 30;\r\ngeneratedBeers = strings(numBeers,1);\r\nfor i = 1:numBeers\r\n    generatedBeers(i) = generateText(beerNameNet,startOfTextCharacter,whitespaceCharacter,endOfTextCharacter);\r\nend\r\n%% \r\n% Sometimes, the network might simply predict beer names from the training \r\n% data. Remove them.\r\n\r\nidx = ismember(generatedBeers,textData);\r\ngeneratedBeers(idx) = [];\r\n%% \r\n% View the generated beers.\r\n\r\ngeneratedBeers\r\n%% Generate Tasting Notes\r\n% We have our beer names, we now need some tasting notes.\r\n% \r\n% Similar to the name generator, create a tasting note generator from the \r\n% Cambridge Beer Festival notes.\r\n\r\ntextData = dataCambridge.notes;\r\n%% \r\n% As before, to help with the generation, replace all the space characters \r\n% with a \"\u00b7\" (middle dot) character, insert a start of text character at the beginning, \r\n% and an end of text character at the end.\r\n\r\ntextDataPredictors = startOfTextCharacter + replace(textData,\" \",whitespaceCharacter);\r\ntextDataResponses = replace(textData,\" \",whitespaceCharacter) + endOfTextCharacter;\r\n\r\nXTrain = cellfun(@double,textDataPredictors,'UniformOutput',false);\r\nYTrain = cellfun(@(Y) categorical(cellstr(Y')'),textDataResponses,'UniformOutput',false);\r\n%% \r\n% Define the network architecture.\r\n\r\nnumFeatures = 1;\r\nnumClasses = numel(categories([YTrain{:}]));\r\nnumWords = max([XTrain{:}]);\r\n\r\nlayers = [\r\n    sequenceInputLayer(numFeatures)\r\n    wordEmbeddingLayer(200,numWords)\r\n    lstmLayer(400)\r\n    dropoutLayer(0.5)\r\n    fullyConnectedLayer(numClasses)\r\n    softmaxLayer\r\n    classificationLayer];\r\n%% \r\n% Specify the training options.\r\n\r\noptions = trainingOptions('adam', ...\r\n    'MaxEpochs',500, ...\r\n    'InitialLearnRate',0.01, ...\r\n    'GradientThreshold',2, ...\r\n    'Shuffle','every-epoch', ...\r\n    'Plots','training-progress', ...\r\n    'Verbose',false);\r\n%% \r\n% Train the network.\r\n\r\nbeerNotesNet = trainNetwork(XTrain,YTrain,layers,options);\r\n%% \r\n% Generate some tasting notes using the |generateText| function, listed \r\n% at the end of the example.\r\n\r\nnumBeers = 5;\r\nfor i = 1:numBeers\r\n    generatedNotes = generateText(beerNotesNet,startOfTextCharacter,whitespaceCharacter,endOfTextCharacter)\r\nend\r\n%% \r\n% Perfect! I can now get started on brewing my own perfect beer. You can \r\n% run the code many times to generate more names and tasting notes. My favorite \r\n% design that I have seen so far is:\r\n%% Hopky Wolf IPA\r\n% _\"This Double IPA has a big malt backbone and flavours of grapefruit, orange \r\n% and lemon with an underlying floral quality and tent complex. Well balanced \r\n% aroma reflects its taste. It's hopped with a blend of Fuggle and Golding hops.\"_\r\n% \r\n% Now I just need MATLAB to automate the brewing process...\r\n%% Text Generation Function\r\n% The |generateText| function generates text character by character, starting \r\n% with the start of text character and reconstructs the text using the special \r\n% characters. The function samples each character using the output prediction \r\n% scores using the |datasample| function (Statistics and Machine Learning Toolbox). \r\n% The function stops predicting when the network predicts the end-of-text character \r\n% or when the generated text is 500 characters long.\r\n\r\nfunction generatedText = generateText(net,startOfTextCharacter,whitespaceCharacter,endOfTextCharacter)\r\n\r\ngeneratedText = \"\";\r\n\r\nX = double(char(startOfTextCharacter));\r\nvocabulary = string(net.Layers(end).Classes);\r\n\r\nmaxLength = 500;\r\nwhile strlength(generatedText) < maxLength\r\n    % Predict the next character scores.\r\n    [net,characterScores] = predictAndUpdateState(net,X,'ExecutionEnvironment','cpu');\r\n    \r\n    % Sample the next character.\r\n    newCharacter = datasample(vocabulary,1,'Weights',characterScores);\r\n    \r\n    % Stop predicting at the end of text.\r\n    if newCharacter == endOfTextCharacter\r\n        break\r\n    end\r\n    \r\n    % Add the character to the generated text.\r\n    generatedText = generatedText + newCharacter;\r\n    \r\n    % Get the numeric index of the character.\r\n    X = double(char(newCharacter));\r\nend\r\n\r\ngeneratedText = replace(generatedText, ...\r\n    [startOfTextCharacter whitespaceCharacter], ...\r\n    [\"\" \" \"]);\r\n\r\ngeneratedText = string(join(textwrap(generatedText,45),newline));\r\n\r\nend\r\n\r\n\r\n\r\n\r\n##### SOURCE END ##### a710f144b80042c592b9fe35aab1fc59\r\n--><!-- AddThis Sharing Buttons below -->\t<\/div>\r\n\r\n","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img decoding=\"async\"  class=\"img-responsive\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2018\/10\/beer_image7-225x300.jpg\" onError=\"this.style.display ='none';\" \/><\/div><p>This post is from Ieuan Evans, who has created a very unique example combining deep learning with LSTM and beer. (Please drink responsibly!)\r\n\r\nI love craft beer. Nowadays, there are so many choices... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2018\/10\/18\/deep-beer-designer\/\">read more >><\/a><\/p>","protected":false},"author":156,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[5],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/615"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/users\/156"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/comments?post=615"}],"version-history":[{"count":18,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/615\/revisions"}],"predecessor-version":[{"id":5375,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/615\/revisions\/5375"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media?parent=615"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/categories?post=615"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/tags?post=615"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}