Hi everyone, I hope that you all had a nice holiday season and that your new year is off to a great start. Amy is here again to follow up on some good practices she has found lead to good deployabilty of her MATLAB code in toolbox form.
How do you organise your toolbox code?In my previous post, I wrote about the MATLAB toolbox packaging feature. Today I want to share some best practices on how to organise your development environment to streamline the toolbox development and packaging process. I've also written the Toolbox Tools to help you get started developing your own MATLAB toolbox.
Version control and why it should be used
A software development workflow typically uses:
- version control or source control to manage changes to source code over time;
- a package management tool to automate the process of installing, upgrading, configuring and removing packaged software.
A version control system maintains a detailed history of all modifications made to your source files. This allows you to easily maintain multiple version of your software, and to review and revert those changes when required. More importantly, version control facilitates collaborative software development as it enables multiple developers to contribute and combine their code into a main development area or code repository. Some organisations also use their version control system to distribute software to end users; however, a dedicated package management tool may be more suitable to automatically handle the install, upgrade and uninstall process for your users.
Toolbox development workflow
When writing a toolbox in MATLAB, you typically:
- Check out a copy of the source code from a central code repository. I will call this local copy a "sandbox".
- Make and test code changes.
- Check the changes back into the repository for version control and to make it available to other contributors.
- Package the code and release it to end users when ready.
As well as toolbox code that's distributed to end users, you often have other resources and files to support the ongoing cycle of development, testing, and release of your toolbox. These need to be kept separate from the distribution code.
How to organise your sandbox
A sandbox for a MATLAB toolbox contains:
- Shipping toolbox files (code and documentation),
- Files used to package the toolbox,
- Files used to automate the configuration of the MATLAB environment for development, packaging and testing,
- An archive of all toolbox releases.
So when developing and packaging a toolbox, how should you organise this content in MATLAB? The screenshot below shows the convention we've adopted internally at MathWorks and have found it to work successfully for our customers. Adopting a convention within your team actively promotes consistency and familiarity.
What to package?
You should only package and distribute toolbox code and documentation to your end users. The other files, used for supporting the toolbox development and packaging process, are stored in the repository for development only.
The project file (.prj) contains information about the toolbox and is used for packaging the .mltbx file. It is stored outside the packaged or shipped toolbox in your development sandbox.
The released code and documentation files are in a subfolder of the project root directory /tbx. The toolbox code is stored in the /<tbxname> subfolder, alongside a Contents.m file summarising the MATLAB files in the folder. You will replace <tbxname> with the actual name of your toolbox. This enables users to query content and version information of the toolbox using the help <tbxname> or ver <tbxname> commands. From R2016a onwards, MATLAB users can also query toolbox versions programmatically using the matlab.addons.toolbox.toolboxVersion API .
The /tbx/doc subfolder contains supporting documentation or tutorials to demonstrate the toolbox functionalities. To display the custom help files in the MATLAB Help browser, the toolbox should include an info.xml and helptoc.xml file. With R2016b you're able to tell MATLAB to generate template xml files. The toolbox can also contain published examples which will be stored under the /examples subfolder.
Test files should be stored in the /tests folder. This facilitates running test suites in the folder when tests are written using the MATLAB Unit Testing Framework.
Switching between sandbox and installed code
When writing a toolbox, you want to be able to run and test your sandbox code directly. This requires the sandbox code to be on the MATLAB path. Once the toolbox is ready to be released, you package the toolbox code into a .mltbx file and test the packaged code to ensure it will install and run correctly for end users. For this, it is important that the sandbox code is not on the path and that the installed or packaged code is executed. For these reasons, it is useful to write scripts for adding and removing sandbox code from the MATLAB path. The Toolbox Tools provides template scripts addsandbox.m and rmsandbox.m for this, which you can customise to your own development environment.
Prior to R2016a, MATLAB makes a note of which packaged folders are on your path at packaging time, and these are added to the user path when the toolbox is installed. From R2016a onwards, the toolbox packaging tool allows you to specify folders to be added to the user's MATLAB path when a toolbox is installed. By default, MATLAB includes all toolbox folders that are on your path when you create the toolbox. You can then exclude any folders from being added to the user path by deselecting them as shown below.
All toolbox releases should be archived in the /releases folder for ease of reference. Each toolbox release should be annotated with the correct version number. A release script, i.e. release.m, automates the toolbox release process. This script:
- runs checks for the correct MATLAB version and dependencies,
- generates documentation and publishing examples,
- runs tests,
- packages the toolbox.
You can also find an example release script in Toolbox Tools.
You might ask how strictly do I have to follow these guidelines? Are there any common pitfalls to avoid when organising my sandbox content? All will be revealed in the next post.
Please leave us any comment or feedback below. Don't forget to also download Toolbox Tools!
Get the MATLAB code
Published with MATLAB® R2017a
8 CommentsOldest to Newest
Hello Andy, our company (Parrot Analytics) have Matlab license since 2014 and we have Matlab 2014a. From your blog article, version control is something that’s available only in Matlab 2016a. Is this a separate toolbox on its own or it comes with main Matlab license?
Are you referring to the matlab.addons.toolbox.toolboxVersion API? This is not a separate toolbox, it is a feature included in the main MATLAB license.
Amy and Andy:
What are the best practices for including packaged apps and compiled applications in this workflow? These apps and compiled applications would be based on the code in the toolbox. The apps would need to be included as a part of the toolbox. The compiled applications would be used for users without access (or willingness to learn) Matlab.
Great question, and thanks for your comment. I’ll be talking about best practices on packaging MATLAB apps within a toolbox in an upcoming post. But in a nutshell, you want to avoid files being duplicated or packaged more than once in the toolbox(.mltbx) and app(.mlappinstall) files. My suggestion is to only package the app launcher or main file in the mlappinstall file. All other code and dependencies are packaged in the mltbx file. This is particularly relevant if you have more than one app that shares the same toolbox code.
Users can only install and access content of an mltbx file from within MATLAB. Standalone applications will have to be packaged and distributed separately to users who do not have access to MATLAB.
Hope this makes sense!
[…] Best Practices – Adapt, then Adopt! […]
I’m a bit confused about how to set up the package. When using the packaging tool, should I set the Toolbox Folder to tbx/ or tbx/<tbxname>/? Should the root folder be added to the MATLAB path or does it not matter?
When using the packaging tool, the toolbox folder should point to ‘/tbx’ as it contains both the toolbox code and documentation that you want to package and distribute to your users. It is optional whether the project root folder is on the MATLAB path. However, toolbox folders you want added to the user path when the .mltbx is installed have to be on your MATLAB path when packaging.
[…] Take it away Amy! What are some common pitfalls when packaging a toolbox? In one of my previous posts, I described how to set up your development environment in MATLAB when developing and packaging a […]