I am frequently asked what data provider I use. A year ago my data provider was CSI Data. Then I heard about Premium Data from Norgate Investor Services and the one feature that enticed me to look at them closer: historical S&P500 index constituent data. At that time, I was maintaining the data by hand. Each month I would have to determine which stocks had been added or deleted from the index. I would need to look for name changes in the current and historical list. Not a hard task but time consuming and easy to make mistakes. The thought of not having to do this was very enticing.
This review will focus on US Stocks and AmiBroker integration. Premium data has data for the Australian and Singapore markets, integrate with multiple other platforms and have forex and futures data. For more information go to Premium Data from Norgate Investor Services.
[Edit 7/16/2014 and 1/19/2015]
This entire review is of the new Norgate Data updating platform NDU) which has not yet been released commercially. It does not apply to the existing updating platform or data that’s advertised on the Premium Data website. In order to get access to the new platform, you need to be an existing Premium Data subscriber and email support requesting participation in the NDU beta testing program. The data provided by the new platform can only be accessed via AmiBroker. It has been pointed out that some of the features I have described below do not apply to the regular program. Since, I never did the regular program but jumped straight to the Beta without even doing a trail, I cannot comment on what the regular program has.[End edit]
Disclosures: I have paid for my data. The links on this page are affiliate links.
While investigating Premium Data, these were the features I focused on:
- Data back to at least 1995
- Delisted stock data
- Data quality
- The data can be adjusted for dividends and one time capital gains
- A watch list of the S&P500 index constituents
- A simple way of determining if a stock was in the S&P500 on a particular date
- Data integration with AmiBroker
- Good customer service especially when dealing with bad data
- Reasonably priced compared to CSI data
We will look at each area in more detail with each area receiving a letter grade.
Data back to at least 1995
Premium Data has US stock data back to 1950 which is much further back than I would ever test. One can purchase data back to either 1950 or 1985. I choose 1985.
Delisted stock data
This is an additional package added to the basic package. Coming from CSI Data, there is a difference on how stock symbols are treated when delisted. CSI data symbols are in the format of XYZ-123. Where, XYZ is the current symbol ticker and 123 is a unique number given to every company that does not change. For example Sun Microsystems symbol was SUNW. The symbol in CSI was SUNW-5968. Then they changed their symbol to JAVA. The new symbol in CSI became JAVA-5968. I could use this number to keep track of stocks and not have to worry about symbol changes.
Premium Data does things differently. Any currently traded stock has just the symbol. When a stock gets delisted, then they append the year and month. For example Sun Microsystems symbols is JAVA-201001. One nice benefit of this is that in your trade list it is easy to spot stocks that are delisted.
This proved to be the hardest part of my evaluation. The first tests were running several of my strategies through both Premium Data and CSI. The results were close but not super close. This gave me confidence that both data sets were similar. The problem is how does one say which data source is correct without a third data source to compare against.
I could compare about a dozen symbols with Bloomberg’s data. Eight symbols agreed between CSI and Bloomberg. Four symbols agreed with Premium Data. Of the eight symbols were Premium Data did not agree there were a couple of obvious data errors, which I reported to Norgate Investor Services.
The data can be adjusted for dividends and one time capital gains
There seems to be some misconception that Premium Data Services cannot do this. I have had several people write saying they will not use the data because it cannot. The issue is that this feature is not turned on by default. In their support page, they walk you through on how to do this.
I compared Premium Data against CSI data and got the same results for ten dividend yielding stocks.
A watch list of the S&P500 index constituents
When investigating this feature, I thought I had died and gone to data heaven. Not only did Premium Data have a watch list for S&P500 historical constituents, they also have the data for the following indexes: DJIA, NASDAQ 100, Russell 1000, Russell 2000, Russell 3000, S&P 100, S&P 400, and S&P 600. In the past I had made a watch list of my trading universe. This now could save me a step and I could always use the Russell 3000. What a time saver.
Comparing my S&P500 historical list to Premium Data’s they were very similar. Nearly perfect match for the last 5 years.
One important note on this, the historical constituent data is currently a beta program for AmiBroker users. You need to ask to get into this program. I do not know if this data is available to other platforms.
Grade: B (A+++ for multiple index data, C because it is a beta program)
A simple way of determining if a stock was in the S&P500 on a particular date
Determining if a stock is currently in the index could not be simpler. Here is sample AmiBroker code.
inIndex = IsIndexConstituent(“$SPX”); // test for in SP500
inIndex = IsIndexConstituent(“$RUA”); // test for in R3000
Data integration with AmiBroker
Premium Data integrates directly into AmiBroker. After going through Premium Data’s setup instructions, which are very good, you need only to run their data update program and your data is updated in AmiBroker. Nice and simple. Much easier than what I was doing before.
One thing to note, which does not affect me, is that there are no intraday updates. Meaning if you run the update program during market hours, one still is getting the previous day’s close. Not a 20 minute delay snapshot of the current day. This may be important for people who like to run their scans right before the close.
Good customer service
I am always on the lookout for bad data. Sometimes it is one day with a bad price. Other times, it is missing data for several days. Either way, it is important that my data provider quickly look into the issue and get back to me. Data is my life blood.
While comparing against CSI Data, I found some obvious data errors. I sent these errors to Norgate Investor Services. Within 24 hours they had responded that it was an error and they were fixing it. The fix appearing within 48 hours which is great turnaround time.
They have also been very good about responding to any technical support emails. They are located in Australia, so emails are answered at night for those of us in the United States.
Reasonably priced compared to CSI data
Given the index data and the tight AmiBroker integration, I would pay a little bit more. Here are the costs:
Daily data: $297/year
Currently listed US Data History back to 1985: $75 (single time fee)
Delisted US Data History back to 1985: $232 (single time fee)
First year cost: $604
Subsequent years cost: $297
I am paying $680 per year for CSI Data. Another benefit is that I am using a different data provider for my daily signal generation which is costing $300 per year. I will be able to drop them. After the first year, may data costs will go from $980 to $297, this will be a substantial savings.
Premium Data allows the creation and linking of multiple AmiBroker databases. This is helpful because I like keeping three databases: one with 2 years of data for daily scans, one with 11 years of data for most of my back testing and the third with 25 years of data when I am curious about results in the 1990’s.
Premium Data can be installed on two different computers at the same time.
Here we have a screenshot of the update program
I have been using Premium Data exclusively for my testing for the last three months and have been very happy. The only minor negative is that the constituent data program is in beta. The last time I asked there was no date on when it would come out of beta.
Even without the constituent data, I would recommend Premium Data. Having the historical index constituent data makes me highly recommend Premium Data.
Click the logo below to get more information.