{"id":5249,"date":"2020-10-30T09:03:18","date_gmt":"2020-10-30T13:03:18","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=5249"},"modified":"2021-04-06T15:45:41","modified_gmt":"2021-04-06T19:45:41","slug":"deep-wine-designer","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2020\/10\/30\/deep-wine-designer\/","title":{"rendered":"Deep Wine Designer"},"content":{"rendered":"This post is another from Ieuan Evans, who brought us Deep Beer Designer<\/em><\/a>, back today to talk about wine! It's a longer post than usual, but packed with useful information for Deep Learning for Text. If you're looking for a unique text example, this is for you!<\/span>\r\n
<\/h6>\r\n

\"\"<\/h2>\r\n

<\/h6>\r\n
<\/h6>\r\nFollowing my last blog post about how to use MATLAB to choose the perfect beer<\/u><\/strong><\/a>, I decided to explore what I could do with deep learning and wine.<\/span>\r\n
<\/h6>\r\nSimilar to beer, the number of different choices of wine is overwhelming. Before I can select a wine, I need to work out what grape varieties of wine I actually like.<\/span>\r\n\r\nCould MATLAB help me with this? If I describe my\r\nperfect wine to MATLAB, will it be able to select one for me? Which characteristics of different wine varieties stand out?<\/span>\r\n
<\/h6>\r\nThere are two common approaches to a text classification problem like this:<\/span>\r\n
<\/h6>\r\n
    \r\n \t
  1. Train a long short-term memory (LSTM) network that treats text data as a time series and learns long-term dependencies between time steps.<\/span><\/li>\r\n \t
  2. Train a convolutional neural network (CNN) that treats text data as hyperspectral images and learns localized features by applying sliding\r\nconvolutional filters.<\/span><\/li>\r\n<\/ol>\r\nIn this blog post, I will focus on the second approach: classifying wine grape varieties given a description by converting them to images and using a CNN.<\/span>\r\n
    <\/h6>\r\n

    Background: Word Embeddings<\/h2>\r\nTo convert text to hyperspectral images, we can use a word embedding that maps a sequence of words to a 2-D array representing an image. In other words, a word embedding maps words to high-dimensional vectors. These vectors sometimes have interesting properties. For example, given the word vectors corresponding to Italy, Rome, Paris, and France you might discover the relationship:<\/span>\r\n
    <\/h6>\r\n

    Italy \u2013 Rome + Paris \u2248 France<\/em><\/p>\r\n\r\n

    <\/h6>\r\nThat is, the vector corresponding to the word Italy<\/em> without the components of the word Rome<\/em> but with the added components of the word Paris<\/em> is\r\napproximately equal to the vector corresponding to France<\/em>.<\/span>\r\n
    <\/h6>\r\nTo do this in MATLAB, we can use the Text Analytics Toolbox\u2122 Model for fastText English 16 Billion Token Word Embedding<\/em> support package. This word embedding maps approximately 1,000,000 English words to 1-by-300 vectors. Let's load the word embedding using the fastTextWordEmbedding function.<\/span>\r\n
    emb = fastTextWordEmbedding;<\/pre>\r\n
    <\/h6>\r\nLet's visualize the word vectors Italy, Rome, Paris<\/em>, and France<\/em>. By calculating the word vectors using the word2vec<\/span> function, we can see the relationship:<\/span>\r\n
    <\/h6>\r\n
    \r\n
    italy = word2vec(emb,\"Italy\");\r\nrome = word2vec(emb,\"Rome\");\r\nparis = word2vec(emb,\"Paris\");\r\n\r\nword = vec2word(emb,italy - rome + paris)<\/pre>\r\n<\/div>\r\n\u00a0 \u00a0word = \"France\" <\/span>\r\n
    <\/h6>\r\n
    Words to Images<\/h2>\r\nTo use a word embedding to map a sequence of words to an image, let's split the text into words using tokenizedDocument<\/span> and convert the words to a sequence vectors using doc2sequence<\/span>.<\/span>\r\n
    \r\n
    str = \"The rain in Spain falls mainly on the plain.\";\r\ndocument = tokenizedDocument(str);\r\nsequence = doc2sequence(emb,document);<\/pre>\r\nLet's view the hyperspectral images corresponding to this sequence of words.<\/span>\r\n\r\n<\/div>\r\n
    figure\r\nI = sequence{1};\r\nimagesc(I,[-1 1])\r\ncolorbar\r\nxlabel(\"Word Index\")\r\nylabel(\"Embedding Feature\")\r\ntitle(\"Word Vectors\")<\/pre>\r\n\"\"\r\n
    <\/h6>\r\nThe resulting image is not particularly exciting. It is a C-by-S array, where C is the number of features of the word embedding (the embedding\r\ndimension) and S is the number of words in the text (the sequence length). When formatted as 1-by-N hyperspectral image with C channels, you can\r\ninput this data to a CNN and apply sliding filters of height 1. These are known as 1-D convolutions.<\/span>\r\n
    <\/h6>\r\n
    Load Wine Reviews Data<\/h2>\r\nLet's download the Wine Reviews<\/span><\/a> data from Kaggle and extract the data into a folder named wine-reviews. After downloading the data, we can read the data from winemag-data-130k-v2.csv into a table using the readtable<\/span> function. The data contains special characters such as the \u00e9 in Ros\u00e9, so we must specify the text encoding option too.<\/span>\r\n
    <\/h6>\r\n
    filename = fullfile(\"wine-reviews\",\"winemag-data-130k-v2.csv\");\r\ndata = readtable(filename,\"Encoding\",\"UTF-8\");\r\ndata.variety = categorical(data.variety);<\/pre>\r\n
    <\/h6>\r\n
    Explore Wine Reviews Data<\/h2>\r\nTo get a feel for the data, let's visualize the text data using word clouds. First create a word cloud of the different grape varieties.<\/span>\r\n
    <\/h6>\r\n
    figure; \r\nwordcloud(data.variety);\r\ntitle(\"Grape Varieties\")\r\n<\/pre>\r\n
    <\/h6>\r\n\r\n\"\"\r\n
    <\/h6>\r\nTo quickly verify whether text classification might be possible, let's quickly create word clouds for a selection of classes and inspect the differences between them. If you have Text Analytics Toolbox installed, then the wordcloud function automatically preprocesses string input. For better visualizations, let's also remove a list of common words and the grape varieties from the text.<\/span>\r\n
    <\/h6>\r\n
    labels = [\"Gew\u00fcrztraminer\" \"Chardonnay\" \"Nebbiolo\" \"Malbec\"];\r\ncommonWords = [\"wine\" \"Drink\" \"drink\" \"flavors\" \"finish\" \"palate\" \"notes\" \"aromas\"]; \r\n\r\nfigure\r\nfor i = 1:4\r\n  subplot(2,2,i)\r\n  label = labels(i);\r\n  idx = data.variety == label;\r\n\r\n  str = data.description(idx);\r\n  documents = tokenizedDocument(str)\r\n  documents = removeWords(documents,commonWords);\r\n  documents = removeWords(documents,labels);\r\n  str = joinWords(documents);\r\n\r\n  wordcloud(str);\r\n  title(label)\r\nend<\/pre>\r\n
    <\/h6>\r\n\"\"\r\n
    <\/h6>\r\nThe word clouds show that the distributions of words amongst each grape variety are different. Even though words like \"fruit\" and \"berry\" appear to commonly describe some of these varieties, the word clouds show that the distributions of words among each grape variety are different.<\/span>\r\n
    <\/h6>\r\nThis shows that there are grounds to train a classifier on the text data. Excellent!<\/span>\r\n
    <\/h6>\r\n
    \"\"<\/h6>\r\n
    Prepare Text Data for Deep Learning<\/h2>\r\n
    <\/h6>\r\nTo classify text data using convolutions, we need to convert the text data into images. To do this, let's pad or truncate the observations to have a constant length S and convert the documents into sequences of word vectors of length C using the pretrained word embedding. We can then represent a document as a 1-by-S-by-C image (an image with height 1, width S, and C channels).<\/span>\r\n
    <\/h6>\r\nTo convert text data from a CSV file to images, I have a helper function at the end of this post called transformTextData<\/span>. It creates a tabularTextDatastore object and uses the transform function with a custom transformation function that converts the data read from the tabularTextDatastore object to images for deep learning.<\/span>\r\n
    <\/h6>\r\nIn this example, we'll train a network with 1-D convolutional filters of varying widths. The width of each filter corresponds the number of words the filter can see (the n-gram length). The network has multiple branches of convolutional layers, so it can use different n-gram lengths.<\/span>\r\n
    <\/h6>\r\n \r\n
    Clean up Data<\/h3>\r\nRemove reviews without a label.<\/span>\r\n
    <\/h6>\r\n
    idxMissing = ismissing(data.variety);\r\ndata(idxMissing,:) = [];<\/pre>\r\n
    <\/h6>\r\nRemove any reviews where the grape variety is not one of the top 200 varieties in the data. (If you can't find a wine you like in the top 200 choices available, MATLAB probably can't help you.)<\/span>\r\n
    <\/h6>\r\n
    numClasses = 200; \r\n[classCounts,classNames] = histcounts(data.variety);\r\n\r\n[~,idx] = maxk(classCounts,numClasses);\r\nclassNames = classNames(idx);\r\n\r\nidx = ismember(data.variety,classNames);\r\ndata = data(idx,:);<\/pre>\r\nRemove the unused categories from the data.<\/span>\r\n
    data.variety = removecats(data.variety);\r\nclassNames = categories(data.variety);<\/pre>\r\n
    <\/h6>\r\n
    Partition Data<\/h3>\r\nTo help evaluate the performance of the network, let's partition the data into training, testing, and validation sets. Let's set aside 30% of the data for validation and testing (two partitions of 15%).<\/span>\r\n
    <\/h6>\r\n\r\n\r\n\r\n
    cvp = cvpartition(data.variety,'HoldOut',0.3);\r\n\r\nfilenameTrain = fullfile(\"wine-reviews\",\"wineReviews_\" + numClasses + \"_classes_Train.csv\");\r\ndataTrain = data(training(cvp),:);\r\nwritetable(dataTrain,filenameTrain,\"Encoding\",\"UTF-8\");\r\n\r\ndataHeldOut = data(test(cvp),:);\r\ncvp = cvpartition(dataHeldOut.variety,'HoldOut',0.5);\r\n\r\nfilenameValidation = fullfile(\"wine-reviews\",\"wineReviews_\" + numClasses + \"_classes_Validation.csv\");\r\ndataValidation = dataHeldOut(training(cvp),:);\r\nwritetable(dataValidation,filenameValidation,\"Encoding\",\"UTF-8\");\r\n\r\nfilenameTest = fullfile(\"wine-reviews\",\"wineReviews_\" + numClasses + \"_classes_Test.csv\");\r\ndataTest = dataHeldOut(test(cvp),:);\r\nwritetable(dataTest,filenameTest,\"Encoding\",\"UTF-8\");<\/code><\/td>\r\n\"\"<\/a>\r\n\r\nNeed a break from reading code? Read how researchers are using MATLAB for making better beer and wine in this article<\/strong><\/a>.<\/em><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
    <\/h6>\r\nBring in the description and variety fields from the table.<\/span>\r\n
    <\/h6>\r\n
    miniBatchSize = 128;\r\nttdsTrain = tabularTextDatastore(filenameTrain, ...\r\n'SelectedVariableNames',[\"description\" \"variety\"], ...\r\n'ReadSize',miniBatchSize);<\/pre>\r\n
    <\/h6>\r\n
    Specify Input Size<\/h3>\r\nTo input the text data into the network, we need to convert the text to images with a fixed size by padding or truncating the sequences. Ideally, we need to choose a value that minimizes both the amount of padding added to the sequences and the amount of data discarded due to truncation. Let's try to approximate the number of words in each review by counting the number of spaces and plotting the sequence lengths in a histogram.<\/span>\r\n
    <\/h6>\r\n\"\"\r\n
    <\/h6>\r\nMost of the reviews contain 80 or fewer words. Let's use this as our sequence length by specifying 80 in our custom transform function. The transformTextData function, takes the data read from a tabularTextDatastore object and returns a table of predictors and responses.<\/span>\r\n
    <\/h6>\r\n
    sequenceLength = 80;\r\ntdsTrain = transform(ttdsTrain, @(data) transformTextData(data,sequenceLength,emb,classNames));<\/pre>\r\n
    <\/h6>\r\nThe predictors are 1-by-S-by-C arrays, where S is the sequence length and C is the number of features. The responses are the categorical labels.<\/span>\r\n
    preview(tdsTrain)<\/pre>\r\n\"\"\r\n
    <\/h6>\r\nFor validation, let's also create a transformed datastore containing the validation data using the same steps.<\/span>\r\n
    ttdsValidation = tabularTextDatastore(filenameValidation, ...\r\n    'SelectedVariableNames',[\"description\" \"variety\"], ...\r\n    'ReadSize',miniBatchSize);\r\ntdsValidation = transform(ttdsValidation, @(data) transformTextData(data,sequenceLength,emb,classNames))<\/pre>\r\n
    Define Network Architecture<\/h2>\r\nLet's now define the network architecture for the classification task, we can use deepNetworkDesigner to create the network.<\/span>\r\n
    <\/h6>\r\n\r\n\r\n\r\n
    \"\"<\/td>\r\nThe following describes the network architecture:\r\n