{"id":1837,"date":"2018-08-23T16:28:30","date_gmt":"2018-08-23T20:28:30","guid":{"rendered":"https:\/\/blogs.mathworks.com\/developer\/?p=1837"},"modified":"2018-08-23T16:28:30","modified_gmt":"2018-08-23T20:28:30","slug":"gitlab-jenkins-workflow","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/developer\/2018\/08\/23\/gitlab-jenkins-workflow\/","title":{"rendered":"Git to the Lab!"},"content":{"rendered":"

Hello everyone, for this post, I'd like to introduce Mariano Lizarraga Fernandez as a guest blogger. He has a great story to share where he demonstrates a nicely tuned workflow between a Jenkins server and and a GitLab repository. Take it away Mariano!<\/em><\/p>\r\n

The Big Picture<\/h3>\r\n

In this post we will present a workflow to show how we can run a MATLAB or Simulink Test suite on a Continuous Integration server every time we push to a git repository branch. <\/p>\r\n

We will not describe in detail what Continuous Integration is and everything it involves. For those interested, we recommend this post<\/a> as an introduction to the topic. This post introduces the concept of interfacing MATLAB with Jenkins<\/a>, a well-known continuous integration server. Feel free to explore this and other CI related posts by looking at the Continuous Integration category.<\/u><\/a><\/p>\r\n

In the following we assume that you have some basic familiarity with git and have a Jenkins server running (even locally) that has access to the git repository under test. We will be showing interactions between Jenkins and GitLab<\/a> (a well-known open source git repository manager) and therefore you will need to install the GitLab plugin<\/a> in Jenkins. <\/p>\r\n

Let's go through the Workflow step by step.<\/h3>\r\n

For purposes of this post, we will assume a simplified continuous integration workflow as follows:<\/p>\r\n

    \r\n
  1. Changes are made to code or a model, staged, and committed to a git repository's testing branch.<\/li>\r\n
  2. Changes are pushed to a git remote's testing branch.<\/li>\r\n
  3. The remote repository notifies (via a web hook) the Jenkins CI server that there has been a new push event.<\/li>\r\n
  4. The Jenkins server pulls the changes from the git repository and runs a test suite.<\/li>\r\n
  5. If a test fails, it creates a bug issue in GitLab, if the test passes, it creates a merge request.<\/li>\r\n <\/ol>\r\n

    Here is a picture illustrating this workflow:<\/p>\r\n

    <\/p>\r\n

    Getting Jenkins Ready<\/h3>\r\n

    We have written before in this blog how to go about setting up Jenkins<\/a> to launch MATLAB unit tests. So here we'll focus on configuring it, so every time you push to the testing<\/em> branch, a Jenkins workflow is triggered. <\/p>\r\n

    Start by creating a Freestyle project in Jenkins and configure your Source Code management tab to use git. Type the full address of your remote git repository, configure your credentials and make sure you set it to build from your testing<\/em> branch. <\/p>\r\n

    <\/p>\r\n

    Next, configure your project to build when a change is pushed to GitLab. Take note of the GitLab CI Service URL, as you will need it later to configure GitLab. <\/p>\r\n

    <\/p>\r\n

    Finally, configure launching MATLAB with no display<\/a>, changing directory to the Project's workspace, and running a script that we'll be writing in the following sections: <\/p>\r\n

    <\/p>\r\n

    After that, save your changes. Your freestyle project is ready. Now off to GitLab.<\/p>\r\n

    Getting GitLab ready<\/h3>\r\n

    In GitLab the first thing to do is to configure the Web Hook that will notify Jenkins when a push occurs. To do that, navigate to your settings and in the Integrations subsection, add the URL you wrote from the previous section, check the push events checkbox and click the Add Webhook button: <\/p>\r\n

    <\/p>\r\n

    At this point you can test that your web hook triggers a build in Jenkins. To do that, in the same Integrations subsection in GitLab, select your webhook from the list, and using the Test<\/em> button, select Push Events. You should immediately get a blue ribbon at the top indicating that your web hook executed successfully <\/p>\r\n

    <\/p>\r\n

    Finally, since we will be using GitLab's RESTful API to create issues and merge requests, you'll need to get a Personal Access Token. To do that, go to your personal settings, (in the rightmost corner on a standard GitLab instance) and from there go to your Access Tokens subsection. This will take you to the Access Token creation page. Select a Name, choose an expiration date (you can also leave it blank) and for the scope check the API checkbox. <\/p>\r\n

    <\/p>\r\n

    After you click the green Create personal access token <\/em> button, you will be taken to a page that shows you the token. Make sure you record that token somewhere as you will not be able to see it again. We will use that token when we write our MATLAB script to run the tests in Jenkins. <\/p>\r\n

    Testing Script<\/h3>\r\n

    We will use a variation of the script published for this Simulink's Blog post<\/a>. <\/p>\r\n

    Convert it to a function to receive one argument:<\/b> The argument will determine if it ran in continuous Integration or locally. This will allow us to test things locally then push it to server to run there. <\/p>

    function<\/span> runMyTests (ranInCI)\r\n\r\nimport matlab.unittest.TestRunner<\/span>\r\nimport matlab.unittest.plugins.TAPPlugin<\/span>\r\nimport matlab.unittest.plugins.ToFile<\/span>\r\n\r\nworkspace = pwd;\r\n\r\nif<\/span> ranInCI == 1\r\n    resultsLocation = fullfile(pwd, 'results'<\/span>);\r\n\r\n.\r\n.\r\n.\r\n<\/pre>\r\n      
    Collect Data about the current commit<\/b>: So we can put comments on the issues or merge request about which commit passed and needs to be merged (or failed and needs to be fixed). <\/p>
    % Collect data from the current commit to include in the<\/span>\r\n% body of the issue or merge request.<\/span>\r\n[~, commit] = system('git rev-parse --short HEAD'<\/span>);\r\n[~, branch] = system('git rev-parse --abbrev-ref HEAD'<\/span>);\r\n<\/pre>\r\n      
    Do a web GET or POST to GitLab's API:<\/b> We will use webwrite and webread to create the issues and merge requests as necessary. Note that in you will need the private token created in GitLab. <\/p>
    if<\/span> ranInCI == 1\r\n    copyfile(tapResultsFile, fullfile(resultsLocation, 'TAPResults.tap'<\/span>));\r\n    totalFailures = sum(vertcat(results(:).Failed));\r\n    writeOptions = weboptions('MediaType'<\/span>,'application\/x-www-form-urlencoded'<\/span>,...<\/span>\r\n                              'HeaderFields'<\/span>, ...<\/span>\r\n                              {'PRIVATE-TOKEN'<\/span>, gitPrivateToken});\r\n    if<\/span> totalFailures == 0\r\n        % If no failures, create a merge request<\/span>\r\n        writeMergeURL = [gitLabAPI 'projects\/'<\/span> ProjectID '\/merge_requests?'<\/span>];\r\n        webwrite(writeMergeURL, ...<\/span>\r\n                 'title'<\/span>, ...<\/span>\r\n                 ['[Jenkins] Test(s) Passed for commit '<\/span> strtrim(commit)], ...<\/span>\r\n                 'description'<\/span>, ...<\/span>\r\n                 ['Continous Integration passed on commit '<\/span> strtrim(commit) ...<\/span>\r\n                     ' of branch '<\/span> strtrim(branch) ...<\/span>\r\n                     '. Merge request ready for review'<\/span>], ...<\/span>\r\n                 'labels'<\/span>, 'merge_request'<\/span>, ...<\/span>\r\n                 'source_branch'<\/span>, strtrim(branch), ...<\/span>\r\n                 'target_branch'<\/span>, 'master'<\/span>, ...<\/span>\r\n                 'approvals_before_merge'<\/span>, '3'<\/span>, ...<\/span>\r\n                  writeOptions);\r\n    else<\/span>\r\n        % Tests failed, create a test failure issue in GitLab<\/span>\r\n        writeIssueURL = [gitLabAPI 'projects\/'<\/span> ProjectID '\/issues?'<\/span>];\r\n        webwrite(writeIssueURL, ...<\/span>\r\n                 'title'<\/span>, ...<\/span>\r\n                 ['[Jenkins] Test(s) Failing in commit '<\/span> strtrim(commit)], ...<\/span>\r\n                 'description'<\/span>, ...<\/span>\r\n                 ['Continous Integration Failed when running on commit '<\/span> ...<\/span>\r\n                 strtrim(commit) ' of branch '<\/span> strtrim(branch) '.'<\/span>], ...<\/span>\r\n                 'labels'<\/span>, 'bug'<\/span>, ...<\/span>\r\n                  writeOptions);\r\n    end<\/span>\r\nend<\/span><\/pre>\r\n      
    Putting it all Together<\/h3>\r\n      
    Now you are ready to test the complete workflow. You can take a look at the video below showing the complete process. Note, the video is a silent film, no need to crank up your audio. <\/p>\r\n\r\n