aiQUANT

In addition to Matlab, The Mathworks offer a product called Simulink which is a design platform for implementing and testing model-based systems. Most of my experience using Simulink borders on modelling Control Systems and DSP architectures. However, as the Mathworks have pointed out in a recent webinar, Simulink can be extended to develop and test finance based models such as a trading system or sub-systems that form part of a larger model.

I decided to implemented a Hilbert Transformer model for price data using Simulink which is shown below:

The model has input “simin” which takes the price data from the matlab workspace as discrete points one at a time and applies unit delay (represented by “1/Z”) and gain which is basically multiplication by the shown number. Individual Real and Complex parts are combined to create complex numbers which is then outputted to the workspace.

Simply put, the Hilbert Transform converts price data into complex number form so that one may go about calculating different measures that are not possible to calculate using just the price itself. My previous post mentions measures that are calculated using hilbert transformed price. The relationship between the actual price and the hilbert transformed price is

for a hilbert price .

As one would expect the model implemented above suffers lag. This is confirmed by comparing the Actual price with the price reconstructed from the hilbert complex numbers:

The model does not induce any loss or gain in magnitude, but there is a lag of 4 bars. This implies that the hilbert price for the current bar actually corresponds to the price 4 bars ago - something we should consider when using hilbert prices to deduce other measures.

Other Links:

1. The wikipedia page covers basics of the hilbert transform.
2. The Mathworks have a webinar which has a section on using Simulink for designing and testing Algorithmic Trading models. You can view this webinar here. Its round about 40:00mins into the presentation where the section on Simulink begins.
3. Simulink totorial can be found here.
4. There a number of webinars and example models at the Mathworks website for one to get started using simulink.

Bookmark & Share

This is a collection of notes made when using the Matlab compiler under Matlab 6 (2003) and Matlab 7 (2006) which may be of use to Matlab compiler users.

Matlab 6 notes

The Matlab compiler is a useful utility which enables you to turn your Matlab code into C/C++ and compile it. The compiler doesn’t really compile your code at all… it translates your Matlab code into C line for line. The C/C++ code produced is unreadable do don’t count on being able to modify it or even understand it! Mex files can be included, watch out for relative paths names though, some documentation on this on Matlab homepage.

Don’t use the remez filter generation command in Matlab as the compiler can’t compile it. There is a fix but it is complicated, see this.

Other functions such as eval and keyboard are unsupported by the compiler. The save command only supports .mat format, not ascii format.

Make sure you test the executable you’ve produced on other machines especially if you intend to deploy the code for other users! I have found that code would work fine on the machine it was compiled on but produce run-time errors on other machines due to missing dlls & incorrect relative paths.

When you supply a command line argument to the executable when running from the DOS command line, it is passed to the internal code as a string even if it is a number. You need to use a command such as

to convert these into numbers. Of course now if you run this code under Matlab, it doesn’t work… I used a compile variable to execute different parts of code depending on whether we were running under Matlab or in a compiled version of the code:

If you run the compiler command (mcc) from a DOS prompt or Perl script etc, you need to make sure you’ve got the Matlab path saved as a Windows variable or else the compiler can’t find any libraries such as the wavelet toolbox. Use the mccsavepath utility to do this.

When giving executable models to other users, you also need to supply the Matlab run-time libraries. These are supplied with matlab compiler, but can also be downloaded from the mathworks homepage.

Matlab 7 notes

For Matlab 7, the compiler has been revised extensively. It now encrypts your code along with the built-in Matlab functions you’ve used. It presumably decrypts these at run-time.

A few more issues:

You need to be VERY CAREFUL here as the compiler makes no effort to change either directory structures or filenames. If you are using some bespoke algorithm/ method in your system which you regard as proprietary and you name your filename after this, e.g. 16TapRLS.m then the one who uses your deployed application is going to see it!

Compiler doesn’t like working over networked file stores, sometimes it fails for strange reasons.

C files with a mex interface used to be called .dll files are now called .mexw32. Matlab 7 only uses .mexw32 file and will helpfully rename your .dll files to .dll.old when you recompile your .c files rendering them useless under Matlab 6.

Some things have been improved since version 6:

Introduction of isdeployed function to indicate if you are running in Matlab or compiled environment. The following code can be used to handle this whilst retaining backwards run compatability to Matlab 6.

Compiled code still can’t handle numerical inputs from command line.

Compiled code can now load .mat files in same way as native code i.e.

The mex function has the following format:

The Matlab mex documentation gives a full explanation of how to access arguments and return values. The mex function allows multiple values to be returned from a C-function. However, the function arguments are passed by address and can therefore be changed within the C-code; these changes are reflected in the matlab environment.
An example shows a mex function called product, written in a file called product.c as:

The mex call in matlab could be:

The value of c will be 200, b will be 20 but a will now be 200 also. The example seems obvious but can be easily overlooked when dealing with more complex functions.

Other notes:

• If the mex function creates static variables, these are created dynamically the first time the function is called and remain persistent for future calls.

• The matlab clear function will clear only matlab variables from the workspace; clear all will properly clear any variables, such as static variables, allocated by the mex function.

• The mex compiler error LINK : fatal error LNK1168: cannot open your_mexfunction.dll for writing is caused by static variables persisting after your_mexfunction has been called in the Matlab environment. The clear all command will remedy this.

• The compiled mex function has a .dll extension under Matlab 6 (Matlab 7 uses .mexw32). If the function has been called, the DLL file cannot be deleted until clear your_mexfunction or clear all is used.

• Creating a mex build script is difficult since matlab string variables cannot be referenced within the mex command. However, the mex command can be simplified by using the triple dot ‘…’ directive to separate the command. Particularly useful for very large build commands, e.g.

Bookmark & Share

I recently started to teach myself OOP techniques for Matlab. To start with I looked for free online resources and found some good links:

• link 1
• link 2
• link 3

Following a specialized textbook on the subject has its own benefits and so I’ve decided to buy the following book:

It covers pretty much everything you need to know about Matlab OOP:

A Guide to MATLAB Object-Oriented Programming is the first book to deliver broad coverage of the documented and undocumented object-oriented features of MATLAB®. Unlike the typical approach of other resources, this guide explains why each feature is important, demonstrates how each feature is used, and promotes an understanding of the interactions between features. Assuming an intermediate level of MATLAB programming knowledge, the book not only concentrates on MATLAB coding techniques but also discusses topics critical to general software development. It introduces fundamentals first before integrating these concepts into example applications. In the first section, the book discusses eight basic functions: constructor, subsref, subsasgn, display, struct, fieldnames, get, and set. Building on the previous section, it explores inheritance topics and presents the Class Wizard, a powerful MATLAB class generation tool. The final section delves into advanced strategies, including containers, static variables, and function fronts. With more than 20 years of experience designing and implementing object-oriented software, the expert author has developed an accessible and comprehensive book that aids readers in creating effective object-oriented software using MATLAB.

Hopefully the concepts discussed in the book will come in handy when I decide to create my own Matlab toolboxes.

Bookmark & Share

Very interesting paper that proposes an algorithm for Vectorizing “loopified” Matlab code:

A Dimension Abstraction Approach to Vectorization in Matlab
Birkbeck, Neil; Levesque, Jonathan; Amaral, Jose Nelson
International Symposium on Code Generation and Optimization, 2007. CGO apos;07. Volume , Issue , 11-14 March 2007 Page(s):115 - 130

I briefly skimmed through this paper and found a few interesting ideas such as the “Loop Pattern Database” and the Vectorization algorithm which I think are pretty neat. There is a powerpoint presentation accompanying the paper which I have attached below for easy view.

| View | Upload your own

Bookmark & Share

The good thing with Matlab is that it provides excellent library of functions and great felxibility to implement something in very less time. One of the other great things is that you can write your code in another language and run it in the Matlab environment. Matlab’s MEX (Matlab-EXecutable) feature allows C code to compile into a Matlab compatible executable so that it runs much faster than its M-file equivalent. Under Windows the MEX function is compiled into a .dll or win32 binary file. If you are running Matlab in other environments then the function would need to be compiled into an appropriate format:

Usually the Matlab compiler automatically identifies the appropriate format of the target file.

In this example I show a MEX function for calculating the mean of an array. We will then use this function in our Genetic Algorithm to show that it improves the speed of our code. I keep the explanation brief as there are numerous resources elsewhere which explain how to write MEX functions.

The standard interface for a MEX function written C takes the form:

• nlhs (Type = int): This paramter represents the number of “left hand side” arguments.

• plhs (Type = array of pointers to mxArrays): This parameter is the actual output arguments. As we will see in our example code that, an mxArray is MATLAB’s structure for holding data and each element in plhs holds an mxArray of data.

• nrhs (Type = int): Similar to nlhs, this paramter holds the number of “right hand side” arguments.

• prhs (Type = const array of pointers to mxArrays): This array hold all of the pointers to the mxArrays of input data.

In a similar fashion our MEX function code for calculating the mean of an array is:

The resulting .dll file runs 40% faster than its .m file equivalent which slightly improves the speed of our Genetic Algorithm.

The bigger window shows profiling with the MEXed code.

• .m file = approx 1 second per generation

• .dll MEX = approx 0.6 seconds per generation

We will move to a complete C++ model in a couple of weeks when we look at a more challenging problem using a self-evolving Prediction Function.

Resources:

• http://cnx.org/content/m12348/latest/
  
• http://www.codeproject.com/samples/mexFunction.asp

Bookmark & Share

As a Matlab user I’ve found writing code that make use of very large data sets can sometimes run very slow.  Memory performance is not increased at the same rate as CPU performance if the structure of the code is such that it is ”memory-bound”.  But with a little knowledge about how Matlab stores and accesses data, we can avoid inefficient memory usage and hence improve execution time.

As an example let us consider two similar snippets of code below:

Code A1 time = 56.56 seconds.  Code A2 time = 0.032 seconds

We see a very big difference in execution times for both codes which do the same thing!  So why is code A2 faster?  Matlab does not require the user to declare the types and sizes of variables before they are used.  The drawback of this is that each time you use an array variable for instance, Matlab must allocate memory for the a new larger array and copy existing data into it.  This can be very memory intensive and hence slow down your code as in A1.  Code A2 has a section of memory already allocated before it enters the for loop and as a result runs 1750% faster.  Try it!

Another example is to do with the fact that Matlab favours Columns over Rows:

 Code B1 time = 1.34 seconds.  Code B2 time = 1.01 seconds

Code B2 is faster because it traverses the elements of the 2-D array by going down the columns in the inner loop, which is similar to the ”fast cache” mechanism CPUs use to reduce the time taken to access buffers in main memory.  Similar results are noted for arrays of N-dimension.

Main points:

• Preallocate large vectors and matrices before populating them with data they need to hold.
• When processing 2-D or N-D arrays, access your data in columns rather than rows.
• Avoid creating unnecessary variables unless they are essential to your algorithm.
• Try to do most of your processing within variables that already exist.

Bookmark & Share

Part 5: Programming Issues

I wrote code for the Genetic Algorithm in MATLAB and it is terribly slow.  A run for 50 Generations averages to about 1 second per generation.  Here are some performance measures obtained using Matlab Profiler:

The FitnessFunction function is the biggest culprit and takes almost 100% of the total simulation time.  Evaluation of the fitness function is the most time consuming part in the Genetic Algorithm.  This is generally the case even with simpler fitness functions because on a relative scale, operators such as crossover and mutation require very little algebraic manipulation.  Looking at the function itself we get more detail:

We find that evaluations for alpha and beta take the most time.  I shall attempt to speed up the algorithm by writing a MEX function.  In doing so we expect the simulation time to drop by many orders of magnitude as compiled MEX code would not need to run under Matlab’s interpreted mode.

Bookmark & Share

Have you ever been frustrated with Matlab when you forget that longwinded command sequence that took ages to get just right and would like to use again, but you didn’t write it down! The Matlab diary helps alleviate this problem. Basically it copies all typed input, and textual output in the command window into a file that you can refer back to at a later date.

To setup the diary such that you get 1 file for each day you use Matlab:

1. Create a directory in which to store your diary files.
2. Add the following line to your startup.m file:

Note, one thing to be aware of is that if you accidentally display a large matrix, it will get added to your diary, so the files can get rather large. Here’s an example of what’s in my startup script:

Bookmark & Share

I attended this seminar in Oxford in May 2007; the notes relate to Matlab 7.1 which was current at that point.

Did you know…?

• You can automatically generate reports from matlab, containing code, figures etc, in HTML, Word,
• PowerPoint or XML form.  They are formatted using cell mode; see “Publishing Results” under “Desktop Tools and Development Environment” in Matlab Help.
• There are compiler tools to generate Excel plugins as well as .dlls or .exes
• There is a Distributed Computing Toolbox which provides support for distributing processing across clusters of PCs (allows larger, distributed data arrays, as well as parallel processing)
• Matlab should be able to make use of multi-core processors on a single machine, to support parallel operations
• The Matlab import wizard can import from a range of file formats e.g. Excel, jpeg etc.  There is a tick-box to create an m-file which can be used to automate the import process.
• There are in-built functions to support reading of XML files, e.g. xmlread, xmlwrite
• You can do things to data by right-clicking on it in the data window
• When plotting, you don’t need to remember all the commands to pretty-up the result.  Just click the “show plot tools” icon on the figure and make the changes you want graphically.  Then you can save the commands which have the same effect, using “File -> Generate M-File”.  You can also do more sophisticated things like this such as adding more data to your plot, or creating sub-plots.
• Cell mode allows you to divide up files into sections which may be independently tested; also used for publishing (can apply formatting to comments in cells)
• You can add your favourite functions to the GUI shortcut bar
• 3D plots can be rotated around for viewing at different angles
• The statistics toolbox has functions for viewing and manipulating multi-dimensional data, including an optimisation tool to iterate to find an optimal output given multiple input variables
• There is a try - catch statement sequence that can be used to trap errors
• You can call functions in C .dlls, using functions such as calllib, loadlibrary, libfunctions etc.  Note the use of functions such as libpointer and libstruct, to create C-style pointers and structures.  Can use this to avoid the pain of mex wrapping?  See “Matlab Interface to Generic DLLs” under “External Interfaces” in the help.
• If you right-click on a breakpoint, you can set up conditions for breaking, which can include any valid matlab code; e.g. you can run a whole function to evaluate whether or not to break…
• There is an m-lint tool which analyses your code for problems in a similar way to lint in C.  Run it on all files in a directory, from the drop-down box in the directory window.
• Running your code from the Profiler tool gives you all kinds of profile info including code coverage, “m-lint” warnings, time spent in each function etc.
• The depfun function will report all the dependencies of a .m file (the Profiler can also be used for this purpose).  The Dependency Report tool (run from the drop-down box in the directory window) can do this for all files in a directory
• If you put TODO, FIXME or NOTE in comments, then these can be picked up and listed for all files in a directory, using the TODO/FIXME report tool, run from the drop-down box in the directory window.
• Using the Contents Report tool, you can automatically create a Contents.m file for a directory, listing descriptions for each file in the directory.  The contents can then be viewed using help <dirname>
• The Help Report tool will display any help that you have written for all files in a directory.  Note that if you use the strings Example, See also, or Copyright, these are picked out separately by the Help Report tool.
• If you need to create a large array and have a problem with available memory, try the undocumented command system_dependent memstats.  This shows the available blocks of contiguous memory.  Arrays must be held in contiguous memory, so you may need to re-root to get the area you need.
• You can set up a timer which “wakes up” and runs a function every so often.  This can allow you to implement a simple form of multi-tasking between different matlab functions, by scheduling functions to run from the timer (see notes on function handles, below)

Bookmark & Share