I'm pleased to introduce Peter Webb as this week's guest blogger. Peter has worked on the MATLAB Compiler since 1995. This is the first in a series of posts about how to make your M-files compile more easily.
Contents
The MATLAB Compiler and the deployment tools create deployable applications by "freezing" a collection of MATLAB functions and state information into a portable package. Deployed applications run using the MATLAB Component Runtime (MCR), which "thaws" the functions and state back into executable form. The Compiler creates either standalone executables or shared libraries (DLLs). Many of the differences between MATLAB and the MCR arise because a single user-written application may make use of multiple Compiler-generated shared libraries; in this case the shared libraries share the MCR's global data.
The MCR differs from MATLAB in five ways:
- The MCR has no desktop.
- The MCR only executes encrypted M-files.
- Global MCR resources, like the Handle Graphics root object and the random number seed, are shared between multiple libraries working together.
- The M-File and Java paths are fixed and cannot be changed.
- Certain MATLAB and toolbox features are not available to deployed applications.
Because the MCR differs from MATLAB, M-Files written to run using the MCR may sometimes differ from M-Files that run only in MATLAB. Generally speaking, the differences between MATLAB and the MCR fall into four categories:
- Finding functions at compile time
- Path management at runtime
- Non-deployable features
- Differences in behavior
This article describes some general guidelines and then focuses on the problem of finding functions at compile time; I will address the other three types of differences in subsequent posts.
General Guidelines
The process of compiling and deploying an application is more efficient if the application follows these guidelines.
- Don't create or use non-constant static state anywhere (in M, C++ or Java code). Code that uses global variables, for example, will likely yield unexpected results if other applications share the MCR's global resources. Don't do this:
global data;
y = data + x;
data = x;- Instead, create a MATLAB handle object to store the shared data, and pass that object to each function that needs access to the shared data.
- Compiled applications cannot read, parse, create or manipulate M-files at runtime. The MCR will only execute encrypted M-Files; as a result, all of the M-files in a deployed application are encrypted. MATLAB functions like fopen and help will read encrypted M-Files as unintelligible binary data. Deployed applications cannot, for example, create and then execute M-files at runtime. This will compile, but won't run:
fp = fopen('Compute.m', 'w'); fprintf(fp, 'function x = Compute(y)\n'); fprintf(fp, ' x = y + 17;\n'); fclose(fp); rehash; z = Compute(10);
- This behavior is by design. You must ensure that your application contains all the M-files it needs at compile time.
- Don't rely on the current directory (cd) or changes to the MATLAB path (addpath) to control the execution of M-files. The path of a deployed application is determined at compiled time, and remains forever fixed. The current directory is never on the path in a deployed application. In the code below, cd-ing to the stringFcns directory does not make stringFcns/add shadow mathFcns/add. The order of these functions at runtime depends on what their order was at compile time. This causes errors:
cd mathFcns z = add(x, y); cd ../stringFcns s = add(s1, s2);
- Avoid this problem by either using different function names, e.g., addstring and addnum or MATLAB Objects.
- Use the isdeployed function (available in M, C++ and Java) to execute deployment specific code paths, or to protect MATLAB-only code (~isdeployed). isdeployed is true when run in the MCR, and false when run in MATLAB. For example, deployed applications must use deployprint, rather than print, to send data to the printer:
if ~isdeployed print else deployprint end
- Provide graceful degradation for applications that rely on non-deployable functions. Typically, this means using isdeployed to provide deployable alternatives to non-deployable functions, or at least to issue a precise warning message.
switch (userInput) % There's no editor available to deployed applications; % print a warning message. case 'EditMFile' if isdeployed warning('MyApp:NonDeployable', ... ['The editor is not available ', ... 'in compiled applications.']); else edit(mfile); end % Deployed applications can't call DOC. But they can % redirect a help query to The MathWorks help site. case 'Help' if ~isdeployed doc(mfile); else web('http://www.mathworks.com/support'); end end
Finding Functions at Compile Time
In order to produce a deployable package that is smaller than a MATLAB installation, the MATLAB Compiler attempts to narrow the set of deployed functions to only those required by the application. The Compiler calls the MATLAB function depfun to compute the dependent functions of each M-File it compiles. This process differs from the mechanism by which MATLAB determines which functions to call.
MATLAB searches for the best match for a function at runtime, when the types of the inputs and the contents of the MATLAB path are known precisely. depfun analyzes M-files at compile-time, when much less information is available; as a result it may omit functions that your application requires. Functions may be omitted because they are non-deployable or invoked only as callbacks or by eval.
Non-deployable Functions
Licensing restrictions prevent most design time functions from being compiled or deployed. A design time function is one that changes or augments the basic functionality of a program. Design time functions include the MATLAB editor, most toolbox GUI-based tools, like the Image Processing Toolbox function imtool, and functions that create M-Files, like the Fuzzy Logic Toolbox function anfis. This code will compile, but won't run:
[fis,error,stepsize] = anfis(trainingData);
Attempting to execute non-deployable functions generates MATLAB:UndefinedFunction errors at runtime. Check your application's code to make sure it uses only deployable functions. mcc generates a list of excluded functions into a file called mccExcludedFunctions.log. Search this file for the string "because of compilability rules" to find the non-deployable functions depfun excluded from compilation. Because of the conservative nature of depfun's compile-time analysis, mccExcludedFiles.log may be quite large, and may list functions that your application does not require.
Detecting Dependencies on Callback Functions
The MATLAB Compiler's dependency analysis statically analyzes M-file functions to determine what other M-files they depend on. This analysis cannot determine the types of any variables or examine the contents of any strings. As a result, callback functions, which are often mentioned only in strings, can be overlooked; the Compiler will produce an application that appears to work but which will generate runtime errors when it attempts to invoke the missing callbacks.
For example:
set(gca, 'ButtonDownFcn', 'LogButtonPress');
If LogButtonPress is not included in the deployed package, this program will fail. Include LogButtonPress by explicitly specifying it on the mcc command line, or inserting a %#function pragma in the M-file that uses LogButtonPress.
Specifying a callback using mcc:
mcc [other options] -a dir1/dir2/LogButtonPress.m
Using a %#function pragma
%#function LogButtonPressIncluding Functions Called by eval
MATLAB's eval function poses the same kind of problem for the Compiler's dependency analysis that callbacks do: the Compiler cannot examine text strings to determine if they contain function calls. Applications that use eval fail at runtime if the functions invoked by eval are not included in the deployed package.
For example, this code that uses eval to create dynamically named MATLAB variables requires the function getdata:
for n = 1:count eval(sprintf('x%d = getdata(%d);', n, n)); end
The solutions to problems created by eval are the same as those for callbacks: use the %#function pragma, or explicitly deploy the function in question by listing it as an argument to your mcc command.
Keep Reading
I plan to address issues and workarounds surrounding runtime path management, non-supported functions and differences in function behavior in future posts. In the meantime, you can refer to the documentation for the MATLAB Compiler or post follow-up questions here.
Get
the MATLAB code
Published with MATLAB® 7.6
Peter,
Thank you for addressing an area which has long represented a headache for me. I do have a few questions though:
1. Could you give a somewhat more complete definition of a “non-constant static state”? You give the example of a global variable, but what else falls into this category?
2. I’ve never understood why the necessity exists for explicitly coding in the deployprint instead of print. If print doesn’t exist in compiled code (which I also don’t understand the reasons for), can’t the compiler recognize this and do the substitution at compile time? I find that having to include many variations based on isdeployed makes the code much more difficult to maintain and comprehend.
3. Can you talk about the issues associated with UNC paths? One perpetual challenge for me is that I deploy software to many users on a windows network, where everybody needs to be able to access the same directory on the server. Unfortunately, different computers have the server mapped differently, so I can’t use drive letters. The best I’ve been able to do is apply a system call to (’net use’) and then parse it for certain key words (which is kludgy, at best). Is there a prefered method for dealing with this sort of thing?
Thanks,
Dan
Dan,
In response to your questions:
1. By non-constant static state, I mean global data that the program both reads and writes. State that gets initialized once and is then “read only” is OK. Multiple instances of the MCR can inadvertently share state through:
* The HG root
* Java global state (for example, Java singleton objects)
* The properties of a figure window, like UserData: set(1, ‘UserData’, 25)
* MATLAB Singleton objects.
2. Compiled applications must use DEPLOYPRINT on Microsoft Windows because of a difference in the way they display figure windows. Compiled applications render their graphics into Java frames. MATLAB’s printing relies on access to the native (i.e., Microsoft Windows) graphical data, which is not available in compiled applications. Therefore, since compiled applications render their graphics differently, they must print differently as well.
The compiler can’t substitute one call for another at compile time because that would require modifying the M-file in question, which we don’t do as a matter of policy.
3. Regarding UNC paths, I think probably the best solution is to use identical drive letters; failing that, your solution (which works :-)) seems OK. I don’t know of any other “best practices,” in this area.
Hi Peter,
Its good to see some coverage of a somewhat misunderstood topic.
Just wanted to comment on one thing you stated at the start of your article : “The M-File and Java paths are fixed and cannot be changed.”
I have actually used “javaaddpath(’/opt/user/lib/java’)” to allow for some custom Java classes to be utilised. (Although i was stung by that function CLEARING global variables…)
I would like to ask a question concerning global variables: If two compiled programs that run on the same computer use global variables with the same name, does the MCR assume that they are the same?
Actually I have run two different compiled programs on the same computer and both used (and are still using) a global variable. They have the same name in both programs. In both programs the global variables contain a connection to the same database.
Program 1 runs continuously. Program 2 is called seldomly and closed after a short time. Program 2 clears its own global variable when being closed. Program 1 still operates properly. Also, program 2 creates initializes its global variable only if it is empty. If program 2 accessed the global variable of program 1, program 2 would never ever function. However, program 2 operates as expected. These programs are very different from each other.
Was it just luck that both programs worked properly? I used R2007a when compiling the two programs. Will problems occur when switching to R2008a or R2008b?
Dear Loren,
I have some problem for making standalone application from both m-file(GUI) and simulink model(including stateflow). Would you please help me to perceive how the matlab can perform this task for me?
Regards,
Hamidreza Nourzadeh
Hamidreza-
Peter may have more specific information, but from what I know, the Compiler is intended for only MATLAB-based applications, not ones from Simulink and Stateflow. For Simulink and Stateflow, RTW (Real-Time Workshop) is the path to the embedded code world.
–Loren
Rodney,
The side-effect you noticed with regard JAVAADDPATH clearing global variables is one of the reasons we don’t recommend the use of JAVAADDPATH in compiled applications. Some of the commands that manipulate the path might seem to work at first, but they almost always have some unintended side effects. And besides, changing the path sort of violates the model of a deployed application: once compiled, an application is not supposed to “change” — there should be no way to run new code, or modify the application’s functionality.
OkinawaDolphin,
The only way that two or more compiled applications can communicate with (or affect) one another is through the filesystem. For example, one application could write a file that another could read.
However, compiled shared libraries can affect one another via interactions through shared global state (such as the Handle Graphics root object) if they are both linked into the same executable.
So, if your program 1 and program 2 are independent executables, they will maintain their global variables completely separately, and you should have no problems. This behavior is not scheduled to change in any upcoming release.
Hamidreza,
Loren’s correct. The MATLAB Compiler is not designed to create applications from Simulink models or Stateflow diagrams. For that, you need to use Real Time Workshop.
OkinawaDolphin-
Here’s some more info for you.
There’s a very basic description of the recommended way to do this in this SWAT.
Basically, you have to use two separate toolchains to do the deployment, and manually link them up — RSim for Simulink/Stateflow, and the MATLAB Compiler for the GUI. (The SWAT talks about GUIDE guis specifically, but m-file GUIs made w/o GUIDE would be the same.)
There’s no one-button solution.
Should it be OK to use “persistent” variables in a deployed application? What if I have two copies of the program running at once?
Hi, so is it a licensing issue that I get this:
C:\Archivos de programa\MATLAB\R2007b\toolbox\ident\ident\armax.m
called by C:\MATLAB\GUI\DEst.m
(because of toolbox compilability rules)
in the excluded files log?
armax its supposed to be used in a standalone windows application I’m trying to deploy.
Thanks
Marcial,
Yes, the exclusion of ARMAX from your application is a result of licensing; because the ARMAX function performs model estimation, it cannot be deployed.
For more details on the deployability of toolbox functions, see the toolbox deployability table.
Why is it that compiled exe from matlab open invariably a command window? . is there any way to get gui of the compiled program directly start on the user system.
1. or only way to get rid of this problem is to get our programs compiled into dll and use VB or other programs for front end /gui and call these gui.
does this not seem a drawback/limitation .
kindly throw some light on the issue.
Working in a corporate environment and I being a Matlab user in minority, it is difficult to get the MCR “officially” installed. Also having an update every 6 months would be difficult to move forward. Is there a way to create compiled applications so that they are fully transportable? Single EXE or single directory? Adding registry entries etc. is also out of the question.
… any suggestions or other comments?
Prasanna,
You can’t make the DOS command window go away, but you can make it start up minimized:
http://www.mathworks.com/support/solutions/data/1-1W0CIG.html?solution=1-1W0CIG
Stephen,
Compiled MATLAB applications are similar to Java programs and .NET assemblies. All three of these kinds of applications require relatively large virtual machines to run them: Java applications need the JVM, .NET assemblies require the CLR and MATLAB applications need the MCR.
That architecture does not lend itself to single file solutions. You could, of course, package the MCR with each of your executables or shared libraries, and each of your users could install it locally. You might also be able to install the MCR locally on your machine and then share the installation directory across your internal network.
There is currently no way to build a single binary that contains all the code required for a MATLAB Compiler-generated application; we don’t support static linking because of the inefficiencies it creates.
I’m still somewhat confused and unfortunately have only limited access to the Compiler Toolbox so cannot simply guess-and-check-and-learn. So here goes:
1. My startup.m has an addpath call in it. When I build a project that calls a function that lives only in the directory that I’ve added, will the depfun magic find that function and add it to the mcc command-line? I know I can explicitly add it to the build command with either “-a” or by dropping the file into the “Other files” part of the deploytool, but my question is why the compiler can’t grab the function at compile-time and automatically include it in the executable. (It seems that it can’t.) If I just included the path with a “-I” would it work?
I understand the compiler has to “freeze” the M-code at compile-time, and can’t use the path at run-time to dynamically locate and run new M-code sitting somewhere on disk. My question has to do with the “right” way to link in code from a library that I use across many projects. Do I have to “-a” every M-file in the library? What do I need to do so that I can rely on depfun to pick up the right functions (and maybe only those that it needs) with as little intervention as possible?
2. Similarly: It’s only the state of the path at compile-time that matters, right? So both me and the compiler should/do ignore any addpath and rmpaths in the code that is to be compiled.
3. A different topic: We’ve seen some evidence that use of “disp()” a lot on in a stand-alone compiled application slows it down considerably, compared to running from the MatLab command-line. Turning off all disp statements makes a remarkable improvement. Haven’t tried something like fprintf() instead yet. Does this make sense? If so, why? Or is something else going on? (Windows I/O buffering is mis-tuned, we’re idiots, . . .?)
4. Someone (”Dan K”, above) mentioned “issues with UNC paths” but I didn’t understand what was wrong with actually using UNC paths. We’ve just started doing this across a Windows network and it seems to work. This way we avoid the use of drive letters, which can be user- and logon-state- dependent. So why not use them . . .?
Thanks for helping clarify some very important points already,
-tom
Tom,
1. Using ADDPATH in a startup.m file is generally not a good way to manage the use of libraries of M-files in compiled applications. Some alternatives:
* MCC’s -a switch (as you noted above)
* Using a bundle file to automate the inclusion of -a switches on the MCC command line. See the Compiler’s documentation: http://www.mathworks.com/access/helpdesk/help/toolbox/compiler/f13-998373.html.
* Use ADDPATH in an interactive MATLAB session and then save the path (via SAVEPATH). This permanently modifies the MATLAB path, and therefore the path used by MCC.
2. Yes, the state of the path at COMPILE TIME is what matters. Any addpath/rmpath calls in the compiled code will have no effect on what gets included in the deployed application (though they may cause runtime errors for the deployed application, as indicated in the main body of this posting).
3. I’m not sure why the use of disp() should affect performance. This bears further investigation.
4. R2007b enhanced MATLAB’s support for UNC paths, thereby masking the underlying problem: Microsoft DOS commands do not uniformly support UNC paths. MATLAB commands that use DOS commands cannot support UNC paths either. In MATLAB, cd to a UNC directory and then execute system(’dir’); this command will fail because DOS doesn’t support DIR when the current directory is a UNC path.
Regarding the distribution of the compiled code and the MCR. It is my understanding that the ‘deploytool’ GUI (which is new to R2008a??) creates a single install program for the compiled code written by the user AND the MCR. Thereby eliminating the need to distribute the MCR separately to the compiled code.
Is this correct?
Tim.
By the way the spam protection asking me the sum of 3 + 9 seems to think 12 is not the right answer!!! I’ve had to refresh to get a new question.