Pre-inclusion Bias: How to create a false strategy

In the previous post I described a simple rule to double the returns of a mean reversion strategy. In this post, I show how pre-inclusion bias can take a losing strategy and make it a winning one.

Recently I had reader send me the rules for a stock trend following strategy. He knew these are the strategies I have been researching lately. The rules were few and I had time, so I coded it up.

Continue reading

Norgate Data Review

I am frequently asked what data provider I use and recommend for stocks. I have been using Norgate Data for four years and recommend them to anyone looking for data. This review will focus on US Stocks and AmiBroker integration which is what I use daily. Norgate Data has data for the Australian and US markets, forex and futures data. They integrate with AmiBroker and RightEdge.

Continue reading

Is synthetic XIV/VXX data safe to use?

I have done several posts about trading XIV & VXX. In these posts (here, here and here) I refer to using synthetic data before these ETFs started trading. I supported the use of the data due to the very high correlation of daily returns during the overlap period. With a correlation of .97, I thought great the data should be good to use for backtesting.

Continue reading

Start Dates, Correlation and Random Strategy

In my last post I showed research on how optimization results can be mean reverting. Sometimes, my research keeps getting side tracked as I think of random ideas to look at. In this post, we look at the random walk my research took starting from my mean reverting optimization research. I will show how changing the start date can have a big change in the results, correlation of 1990’s to now, and random data and how it correlates.

Continue reading

Is your data in good shape? Would you know it if it was not?

At the end of last year, I was working with a client and we were having problems with code I had written. We would get different results depending on who ran the code. After comparing trade lists and doing some debugging, we discovered that their database was missing several symbols. These symbols existed in my database but did not exist in theirs. We were both using Norgate Data, which I highly recommend and can read my review here.

Continue reading

To dividend adjust or not to dividend adjust? That is the question.

About once a month, someone asks how important it is to have dividend adjusted data. Or someone will comment they do not want to use Norgate Data because they do not adjust for dividends (it does but it is not enabled by default). My answer has been “without dividend adjusted data, your results may be understated.” It has always bothered me that I could not give a better answer. In my post, “How much does not having survivorship free data change test results?” I covered other data issues but not this one. Since Norgate Data makes it easy to have two databases, one with the dividend adjustments and one without, it was time to run tests and determine how much of a difference it makes.

Continue reading

Premium Data from Norgate Investor Services Review

THIS REVIEW IS OLD. READ THE NEW REVIEW: Norgate Data Review

Old Review below.

I am frequently asked what data provider I use. A year ago my data provider was CSI Data. Then I heard about Premium Data from Norgate Investor Services and the one feature that enticed me to look at them closer: historical S&P500 index constituent data. At that time, I was maintaining the data by hand. Each month I would have to determine which stocks had been added or deleted from the index. I would need to look for name changes in the current and historical list. Not a hard task but time consuming and easy to make mistakes. The thought of not having to do this was very enticing.

This review will focus on US Stocks and AmiBroker integration. Premium data has data for the Australian and Singapore markets, integrate with multiple other platforms and have forex and futures data. For more information go to Premium Data from Norgate Investor Services.

Continue reading