Loren on the Art of MATLAB

Creating C++ Shared Libraries and DLLs 29

Posted by Loren Shure,

Guest blogger Peter Webb returns with another in an occasional series of postings about application deployment.

Contents

A previous post demonstrated how to use MATLAB Compiler to create standalone executables. In this article, I'll demonstrate how to use MATLAB Compiler to create C and C++ shared libraries or DLLs.

You create a shared library or DLL to add MATLAB-based functionality to an application you're developing. You select one or more MATLAB-based functions to include in the shared library, and MATLAB Compiler generates a binary file that you link against your application. Shared libraries generated by MATLAB Compiler are compatible with the Microsoft Visual Studio development environment on Windows and with the GNU C/C++ tool suite on most UNIX platforms. The list of supported compilers changes from one release to the next; be sure to check that list before starting your project.

Building a Shared Library

To illustrate the process of creating and using shared libraries, I'll use a cryptographic algorithm, the Vigenere cipher. The program I've written consists of two parts:

  • libvingenere: A shared library containing two MATLAB functions encrypt and decrypt.
  • vigenere.cpp: A C++ main program that calls the functions in libvigenere.

The source code for the MATLAB functions and the main program is available on MATLAB Central. The download package also includes VigenereDetails.html, which describes the implementation of the Vigenere cipher in MATLAB.

The mcc command invokes MATLAB Compiler both from within interactive MATLAB and at the system (DOS or UNIX shell) prompt. Use mcc to build a shared library from the MATLAB encrypt and decrypt MATLAB functions:

 mcc -v -W cpplib:libvigenere -T link:lib encrypt decrypt

This mcc command breaks down into four parts:

  • -v: Turn on verbose output.
  • -W cpplib:libvigenere: Generate C++ wrapper code. Name the generated library libvigenere.
  • -T link:lib: Invoke a C/C++ compiler to create a platform-specific binary shared library file from the generated code.
  • encrypt decrypt: Place the encrypt and decrypt functions in the shared library. Generate C++ wrapper functions for each.

This command generates several files. Two of them are relevant here:

  • libvigenere.dll: The platform-specific binary: the shared library itself. On most Unix systems, this file ends with a .so extension: libvigenere.so.
  • libvigenere.h: Declarations of the C++ wrapper functions and C++ type conversion classes and utilities.

The Generated Interface

MATLAB Compiler generates many different kinds of functions: initialization, termination, error and print handling and, of course, the functions you selected to compile into the library.

In our example, MATLAB Compiler generates a C++ entry point for encrypt and decrypt. Stripped of some bookkeeping annotation, these functions look very much like the corresponding MATLAB functions.

void encrypt(int nargout, mwArray& ciphertext,
             const mwArray& plaintext, const mwArray& key);
void decrypt(int nargout, mwArray& plaintext,
             const mwArray& ciphertext, const mwArray& key);

The generated functions differ from the MATLAB functions in two significant ways:

  • The C++ functions explicitly declare the types of their arguments.
  • The C++ functions return void, passing back results in output parameters provided in the argument list.

Let's look at the encrypt function in detail. For comparison, here's encrypt's MATLAB function signature:

 function ciphertext = encrypt(plaintext, key)

The encrypt MATLAB function has one output and two inputs. The encrypt C++ function has zero outputs (the void return type) and four inputs. The first C++ input indicates the number of outputs requested by the caller. As indicated by the MATLAB function, the number of outputs may be zero or one. Passing any other value will result in an error. The second C++ input is the output argument, passed by reference. encrypt will overwrite any data in this argument with the encrypted message. The third and fourth C++ input arguments are the function inputs, the plaintext and the encryption key. They are passed by constant reference which means the encrypt function cannot change their contents.

Unlike MATLAB, C++ requires all variables have declared, immutable types. Variables in MATLAB have types as well (matrix, cell array, structure, etc.), but the type of a MATLAB variable can change at any time. To accommodate this dynamic behavior in C++, the MATLAB Compiler provides the mwArray data type. All the functions generated by the MATLAB Compiler take mwArray inputs and return mwArray outputs. The mwArray API allows you to create mwArray objects from most C++ data types and to extract native C+++ data from a returned mwArray.

Calling Functions in a Shared Library

Now that I've created a shared library, I need to write a program to call the library's public functions. The program performs six tasks:

  • Includes the shared library's header file.
  • Parses command line arguments.
  • Initializes the MATLAB Compiler Runtime's global state and the shared library's state.
  • Creates input arguments; convert native C++ data types to MATLAB data types.
  • Invokes one or more functions from the shared library.
  • Shuts down the library and the MATLAB Compiler Runtime.
  • Below, I demonstrate how these steps translate into the code of the example's main program, vigenere.cpp.

    Step 1: All programs that use a MATLAB Compiler-generated shared library must include the library's header file. The header file has the same base name as the compiled library, libvigenere in this case, and a .h extension.

    // vingenere.cpp: Encrypt and decrypt using the Vigenere cipher.
    #include "libvigenere.h"
    #include <iostream>

    Step 2: The main program parses the command line. The first argument, a switch, determines the type of action: -e means encrypt, -d decrypt. The second argument is the message, and the third, the key.

    int main(int ac, const char *av[])
    {
        // Encrypt or decrypt? Determined by command line switch
        bool enc = strcmp(av[1], "-e") == 0;

    Step 3: Initialize the runtime and start the library before calling any functions exported from the runtime or the library. Failure to initialize will cause your program to crash. Always check for success (the initializers return false if they fail) and issue error messages as necessary.

        // Initialize the MATLAB Compiler Runtime global state
        if (!mclInitializeApplication(NULL,0))
        {
            std::cerr << "Could not initialize the application properly."
                      << std::endl;
            return -1;
        }
        // Initialize the Vigenere library
        if( !libvigenereInitialize() )
        {
            std::cerr << "Could not initialize the library properly."
                      << std::endl;
            return -1;
        }

    Step 4: Convert the C++ strings from the command line (the message and the key) into MATLAB strings by creating mwArray objects. These declarations cannot appear before the initialization calls in Step 3.

        // Must declare all MATLAB data types after initializing the
        // application and the library, or their constructors will fail.
        mwArray text(av[2]);
        mwArray key(av[3]);
        mwArray result;

    Step 5: Invoke the exported functions. Encrypt or decrypt as indicated by the command line switch. Note that the C++ functions have the same name as their MATLAB counterparts, and that all return values must be passed in by reference. The mwArray class defines

        // Initialization succeeded. Encrypt or decrypt.
        if (enc == true)
        {
            // Encrypt the plaintext text with the key.
            // Request one output, pass in two inputs
            encrypt(1, result, text, key);
        }
        else
        {
            // Decrypt the ciphertext text with the key.
            // Request one output, pass in two inputs
            decrypt(1, result, text, key);
        }
        std::cout << result << std::endl;

    Step 6: Shut down the library and the runtime, in the opposite order of initialization (library first, then runtime).

        // Shut down the library and the application global state.
        libvigenereTerminate();
        mclTerminateApplication();
    }

    Creating and Running the Application

    MATLAB Compiler uses the mbuild function to compile the code it generates into a shared library. mbuild knows how to invoke the C/C++ compiler with the correct switches so the compiler can find the required include files and libraries. You can use mbuild to create your own executables and link them with MATLAB Compiler-generated shared libraries. On Windows, for example, issue this command:

    mbuild vigenere.cpp libvigenere.lib

    On UNIX, you link against a .so file instead of a .lib file:

    mbuild vigenere.cpp libvigenere.so

    In both cases, mbuild produces an executable called vigenere.

    Encrypt the message with the secret key.

    >> !vigenere -e "Algebra, the Music of the Reason" "MATLAB"
       MLZPBSM LSEAYUKTCA FSDHFLRXLSPZ

    The first argument is the message, the second argument the key. Note the leading exclamation point -- this command runs the vigenere executable from within MATLAB, using the system command. You can run these commands on the UNIX or DOS command line as well, but you have to configure your environment correctly.

    Decrypt to verify the encryption worked:

    >> !vigenere -d "MLZPBSM LSEAYUKTCA FSDHFLRXLSPZ" "MATLAB"
    ALGEBRA THE MUSIC OF THE REASON

    Because the alphabets in the Vigenere square only include letters and the space character, the decrypted message lacks punctuation marks. Extending the algorithm to handle punctuation is an exercise left to the reader.

    And extra points if you can figure out why I chose that phrase (aside from its poetic merit, which is in the eye of the beholder).

    Next: Using Multiple Shared Libraries

    I've shown you how to combine several MATLAB-based functions into a single shared library, and how to incorporate that library into a host application. In a later posting, I'll demonstrate how to use multiple shared libraries from a single host application.

    What else can I tell you about shared libraries? Let me know here.


    Get the MATLAB code

    Published with MATLAB® 7.11

    29 CommentsOldest to Newest

    Hi Peter,

    In your previous post :
    http://blogs.mathworks.com/loren/2011/01/06/matlab-data-types-as-arguments-to-standalone-applications/

    you have written:
    “I leave you with exercise for the reader: how would you pass a MATLAB structure to a standalone executable? I have a few ideas, but none of them are really elegant.”

    Do you plan to share your ideas? I would love knowing which trick you are using.

    Aurélien
    http://blog.developpez.com/matlab4geek/

    Dear Loren,

    Quick question: can you declare more than three mwArray in the standalone cpp file that calls the Matlab function? Just try declaring four mwArray without even calling the Matlab function. I always get Matlab crash errors.

    It is interesting how all examples of using Matlab compiler generated c++ shared library in a c++ code have three or less mwArraydefined, two for input and one for output.

    Thanks a lot
    Bir

    Bir-

    Of course you can have as many inputs as you want, just like in MATLAB functions. The examples are trying to not be too complicated. If you have a crash, you should send your code to tech support (link on right of blog) so they can see what’s happening.

    -loren

    Loren,

    What about thread safety? Can I call the functions in the shared library from multiple threads?
    Thanks,

    Tom

    Tom,

    You can call the libraries from multiple threads, but only one thread can be active in a library at any time. The generated libraries are protected by semaphores, which only allow one user thread into the generated library at any one time. Other user threads that try to call into the shared library will block (wait) until the shared library is “free”.

    I have lcc as the compiler
    The mcc command:
    mcc -v -W cpplib:libvigenere -T link:lib encrypt decrypt
    failed with error:

    Error libvigenere.cpp: 155 syntax error; found `void’ expecting `;’
    Error libvigenere.cpp: 155 missing parameter type
    Error libvigenere.cpp: 155 syntax error; found `&’ expecting `)’
    Error libvigenere.cpp: 155 skipping `&’ `ciphertext’ `,’
    Error libvigenere.cpp: 155 syntax error; found `const’ expecting `{‘
    Error libvigenere.cpp: 155 redeclaration of `mwArray’ previously declared at libvigenere.cpp 155
    Error libvigenere.cpp: 155 syntax error; found `&’ expecting `;’
    Error libvigenere.cpp: 155 undeclared identifier `plaintext’
    Error libvigenere.cpp: 156 illegal expression
    :
    and more errors.
    What went wrong?
    Thanks.

    Very Nice article, you asked at the bottom what else we might want to know about the topic –

    All the examples I see are using mbuild to generate the driver application. In most instances, if calling a matlab library from a larger application, this will not be the case. For instance, I am trying this process on a linux system with a QT 4.7.1 project as a driver application.

    Perhaps an explanation of the mbuild driven compiler flags, or an example of how to generate a custom Makefile to build a driver application would help

    Mike,

    I could go on at great length about compiler switches and Makefiles. I was a little wary to, not knowing that there was an interest. Now that I know there is, I’ll see what I can put together. Thanks.

    Interesting post. And I’m impatient to read your next article, because I’m trying to load a C++ dll into Matlab, and because of multiple shared libraries, I receive a lot of Warning messages like
    “Warning: The data type ‘error’ used by function CallMDL does not exist” or
    Warning: The function ‘Write_Eeprom’ was not found in the library “

    How can I fix that?

    Merci,

    Very useful post.

    If I understand correctly, this provides a method to call a matlab compiled library from C++.

    Is it possible to call the library from a matlab script (or compiled executable)?

    Loren/Peter

    These compiler series have been most informative. I am also curious about the deployment of compiled dll’s from the VBA environment. I understand there is a Matlab Builder EX that does a nice job of it, but is one required to use that toolbox in order to call compiled dll’s from, say, Excel VBA? The documentation is, understandably, fairly mum on the topic. Is there a way to use the regular Matlab Compiler to build VBA deployable dll’s, and if so, what additional considerations are there on the VBA side?

    Thanks much, Joe

    Hi Peter, great article!
    I have a question for you, have you any advice for instances when the Initialize() function returns false? From my debugging, I’ve found that “mclInitializeComponentInstance” is causing the hangup, seemingly because the MCR is not instantiated properly. Do you have any suggestions?

    Cousin Chris,

    if mclInitializeComponentInstance() fails, something is usually wrong with the installation of the MCR or the compiled library. If you’ve got a specific, reproducible case, email support@mathworks.com. If they can’t help you, the details will eventually find their way to me…

    Leon,

    You can’t load shared libraries generated by the MATLAB Compiler back into MATLAB. So no, you can’t call a MATLAB Compiler-generated shared library from a MATLAB script.

    And if you want to call some MATLAB functions from a standalone application, you can always use the MATLAB Compiler to build the standalone application for you. See this article for details:

    http://blogs.mathworks.com/loren/2010/11/18/deploying-standalone-applications/

    I have created a shared library which contains one function. I am able to execute its code but I am having problems with one of the parameters. This parameter is an structure that contains several data. Every time a try to access this variable it fails. I also want to return an array of structures.

    Could you please tell about a sample that works with this kind of data ?

    Thanks in advance.

    Nice work to the comments responses. I have a problem with the linking of the program during compilation on visual studio it tells me error 2019 that all files i have loaded cannot be linked.Any help?

    Dear Loren,
    I have created a dll from matlab code.I am able to run this from cpp. However I need to run this dll from Matlab itself. I am using loadlibrary function and calllib methods in matlab to do this. However it is not working. Am I missing something here ?

    >> loadlibrary('TestDll' , 'TestDll')
    >> calllib('TestDll', 'TestDllInitialize')
    >> calllib('TestDll', 'mlxTestDll', 0 , [], 0 , [])
    

    TestDll.dll is the name of the shared library

    Sumit,

    DLLs created with the MATLAB Compiler cannot be loaded back into MATLAB with loadlibrary (or any other mechanism). They are intended strictly for use outside of MATLAB.

    is it possible to compile a .m file to a shared library .dll, then create a c mex file wrapper calling the library function and compile this, so I can distribute the function to other 3rd party users without giving them access to my original .m file source code?

    Paul,

    The DLLs created by the MATLAB Compiler cannot be loaded back into MATLAB, either directly or via MEX files. If you want to distribute your functions to other MATLAB users while hiding your source code, you could take a look at the PCODE function. While not, strictly speaking, cryptographically secure, PCODE is not human-readable. Depending on your requirements, it might be secure enough.

    I have a C++ library compiled with the Matlab compiler (2010b).

    The C++ code using the library works perfectly when I test it with a very simple calling executable. The MCR and library initialize just fine.

    However when called from the actual client, which uses a JVM, I get a crash in the library initialization call: Initialize().

    Is this a known problem? What is going wrong?

    Dear Loren, we have written a single library that is being used by several threads in the same process. Basically we observed that only one thread is able to call the shared library, while the others are blocking its execution.

    Is it true that the shared libraries generated with mcc are protected by semaphores, which only allow one thread into the generated library (exclusive execution)? If yes, Is it possible to disable these semaphores?

    Best,
    rado

    Rado,

    It is true that the libraries protect themselves with semaphores. They do so because the underlying execution engine (the MCR) is not thread safe. This means that even if you could disable the semaphores, you wouldn’t want to, since you’d likely get incorrect results or program failures.

    If you truly need parallelism, currently your best (and only) option is to use separate processes. If your client can speak any of the standard web protocols (HTTP or JSON) or Microsoft’s proprietry extended versions, it’s pretty simple to set up web-based WCF clients in separate processes using WCF. (Of course, your servers have to run on Windows machines in that case.) See my WCF post for details.

    Michael,

    I can’t diagnose the problem unless I have more information. However, rumors on the Net suggest that you might try recompiling your libraries with -nojvm, which ensures the libraries themselves don’t try and connect to or start a JVM. Of course, this only works if your MATLAB functions don’t use Java.

    Hi,
    I followed this tutorial, however, I get a runtime exception
    “The procedure entry point TerminateProcess could not be located in the dll mclmcrrt7_14.dll

    Could anyone please help me on this?

    These postings are the author's and don't necessarily represent the opinions of MathWorks.