venerdì 28 dicembre 2012

A MetaStock class library in C#

Metastock is a popular data format for storing end-of-day and intraday stock quotes data.

It's a very old format with a structure that is a total mess. I myself have spent a lot of time figuring how to properly read quotes in a jungle of different files and formats.

Here I present the actual C# code I use to interface with the database that I update every evening from my data vendor.

Information on the metastock data structure is scattered over the web. I've collected what seemed to work and implemented as a C# class. A special thank to Julian M Bucknall for providing the code to convert from MBF4 (Microsoft Binary Format single precision) to standard floating point IEEE number.

The library is organized as follows:
  • the struct MetaStockRow stores one day of data
   public struct MetaStockRow  
   {  
    public DateTime Day;  
    public double Open;  
    public double High;  
    public double Low;  
    public double Close;  
    public double Volume;  
    public double OpenInterest;  
   }  
  • a set of MetaStockRow(s) forms a MetaStockTable, that is, the whole data for a stock symbol
  • a set of MetaStockTable(s) forms a MetaStockCollection, which is useful when you have to work with more than one stock at a time. A special feature of MetaStockCollection is the capability to intersect dates, that is,  making sure that the stocks you are working with have the same data points (days).
  • MetaStockTables and MetaStockCollections are generated by the class MetaStockReader which is the main object that performs the low-level reading on the database files. 
How to use the code:
  • Create a MetaStockReader object
  • Use AddPath() to tell it where your metastock folders are located (usually there are more than one). 
  • Create a MetaStockCollection object
  • Insert stocks in the collection with AddSymbol()
  • Trim and synch dates in the collection with FilterPeriod() and IntersectDates()
  • Get daily closes with GetCloseArray()

An example:

 MetaStockReader MR = new MetaStockReader();       
 MR.AddPath(@"C:\MetaStock\NYSE");   
 MR.AddPath(@"C:\MetaStock\AMEX");   
   
 MetaStockCollection ML = new MetaStockCollection();                
 ML.AddSymbol("MSFT",MR);  
 ML.AddSymbol("BRK.A",MR);  
   
 ML = ML.FilterPeriod(DateTime.Parse("01/01/2000"),DateTime.Parse("31/12/2011"));  
 ML = ML.IntersectDates();  
   
 double[] s1 = ML[0].GetCloseArray();  
 double[] s2 = ML[1].GetCloseArray();  

If you need to work with just one stock, you can avoid using MetaStockCollection and work directly with MetaStockTable objects and access to its MetaStockRow(s).

There is a little more in the code of what I've shown here, but if you have a look at the sources you can easily discover by yourself all the other features. I must warn you the code quality is rather poor, because I haven't spent much time in improving it. I stopped when it "worked" (shame on me!).

Download the C# source code here.


sabato 1 dicembre 2012

Saltarelle vs JSIL -- Raytracer Demo

I've compared the performance of Saltarelle-compiler vs. another C# to Javascript compiler: JSIL by Kevin Gadd.

Differently from Saltarelle, JSIL operates at intermediate language (IL) level. After the C# program is compiled by Visual Studio, JSIL examines the MSIL instruction codes contained in the dll files and translates them into javascript.

This allows a greater compatibility and portability of .NET applications, but probably introduces an unnecessary layer of abstraction between the original C# code and the javascript output but the additional code needed to simulate all the .NET library has a negative impact on the execution of the compiled code.

This is shown by the following test.

I've converted the Raytracer Demo featured on the JSIL website to the Saltarelle compiler, with the aim to compare the execution speed of the two versions.

Adapting the code to Saltarelle was easy but not straightforward. Saltarelle was missing some needed System classes that I had to re-implement, namely BitmapColorStopWatch, Console and Random. I tried to keep them as similar as possible to the JSIL version so to not affect the result of the test. I also had to write the import code for mersenne.js, a small library the demo uses for generating random numbers (probably because javascript's Math.random() lacks the random seed feature).

Result of the test

I ran the test on a four core Athlon AMD processor with Windows 7 and Chrome browser. The result is summarized in the table:

SaltarelleJSIL
Script code size (.js)108 KB15 MB
Average time/pixel0.006 ms0.093 ms

So, on average, Saltarelle is 15 times faster than JSIL. Impressive!

The output script code is also very small-sized with no loading time, while the JSIL version has to download 15 MB of stuff before it can start.

Check by yourself

Run the above test directly in your browser:
You can also download the sources hosted on GitHub .