Using Our Data With AmiBroker

Important news for AmiBroker users!

The information below describes the solution we have for customers who wish to have "MetaStock" format data for use in charting/analysis applications in addition to AmiBroker.

We also have a completely new and different updating platform that provides a direct plug-in solution for AmiBroker and a superior data environment for back-testing in general.

If you don't have any subscription yet, visit our sister site to learn about the Norgate Data Updater ("NDU").

If you are an existing Premium Data subscriber interested in switching to a Norgate Data subscription, please start with the information provided here.

Compatability with Windows 10 for ARM (June 2021)

This note is the result of testing the Tools > PremiumData maintenance script for AmiBroker on a Mac M1 machine running the Insider Preview of Windows 10 for ARM via Parallels.

Windows 10 for ARM aims to provide backwards-compatibility for x86 and x64 applications like AmiBoker and Premium Data. But that effort appears to be still a work in progress.

In particular, Windows 10 for ARM doesn't appear to support OLE (Object Linking and Embedding) which is used by both AmiBroker and Premium Data.

The end result is that the Premium Data maintenance script routine for AmiBroker cannot run.

This is an essential part of the operation and therefore Premium Data is not compatible with AmiBroker on Windows 10 for ARM.

Overview

Premium Data holds its data in the "MetaStock" data format. AmiBroker has its own routine for creating an AmiBroker database to access MetaStock format data, instructions for which can be found here.

As an alternative, we provide integration scripts that can create AmiBroker databases for you using the same MetaStock plugin provide by AmiBroker. The advantage of using our scripts is that they also organize the data in the databases for you and subsequently maintain them to account for symbol additions, removals and changes. For stock market databases, the integration scripts also provide pre-generated Premium Data Watch Lists for AmiBroker and subsequently maintain those.

Requirements

1. The integration scripts work with both the 32 and 64-bit versions of AmiBroker. However, they will ONLY work with AmiBroker version 5.60 or higher (6.00 or above HIGHLY recommended). If you have an earlier version of AmiBroker, you can get the latest official release from the AmiBroker downloads area.
2. You need to have Internet Explorer/Microsoft Edge installed and functional on your machine, even if you don't use it as your default browser.
3. The integration scripts are designed for use with a Premium Data subscription or free trial. Integration is only completed successfully after a data update is performed. Please bear this in mind when purchasing historical data without a subscription to updates or without having a free trial for updates.

Installation

There are separate integration scripts for each of our ASX, US, Futures and Forex services. If you have subscribed to more than one service, then you will need to run each relevant script (this will not interfere with your existing AmiBroker workspaces - new ones will be created). Installers for the various integration scripts are available from our Downloads area.

Click on a link below to find instructions (and illustrations) on how to integrate our data with AmiBroker.

ASX Stocks
US Stocks
Forex
Futures

FAQ

What maintenance do I need to do each day/week?
Run the maintenance script (Tools > XXX-PremiumData) to pick up any code changes, name changes, new listings and delistings as well as changes to the index constituents and watchlists.
As the majority of the database maintenance is done over the weekend, we recommend that the maintenance script is run every Monday (as a minimum).

I seem to be missing some recently listed/changed symbols. What do I need to do to be able to see them?
Run the Maintenance as above.

My AmiBroker database is missing a lot of data (or seems to be corrupt). How can I fix this?
Run the maintenance script (Tools > XXX-PremiumData) to refresh the AmiBroker database.

If the AmiBroker database still appears to be corrupt, you can always create a fresh one. The instructions below refer to the US-PremiumData database but are equally applicable to any AmiBroker PremiumData database.

Show Instructions

Hide Instructions

1. If the database in question is currently the default AmiBroker database, select another one in AmiBroker to be the default (Tools > Preferences > Data). This can be reversed later.
   Note: If you don't have another AmiBroker database to use as the default, simply create an empty folder on your machine and give it a name like "tempdefault". Then select that folder to be the default database.
2. Make sure that AmiBroker is closed.
3. Go to the AmiBroker folder (c:\Program Files\AmiBroker) and rename the existing US-PremiumData sub-folder to something like US-PremiumData(BAD) so as to set it aside.
4. Run the US-PremiumData integration script (which you can get from the Downloads area of our website). This will create a fresh US-PremiumData database under the AmiBroker folder.
5. You may have had your own Watch Lists (tls files) in the "\US-PremiumData(BAD)\WatchLists" folder. If so, copy them across to the new "\US-PremiumData\WatchLists" folder. Similarly with Layouts.
6. Start AmiBroker and open the US-PremiumData database (c:\Program Files\Amibroker\US-PremiumData).
7. Click Tools > US-PremiumData to perform symbol maintenance on this database (Tools > US-PremiumData).
8. Once everything is up and running, you can safely delete the "c:\Program Files\Amibroker\US-PremiumData(BAD)" folder.

My AmiBroker database is still not displaying charts after running the Tools > XXX-PremiumData maintenance script. How can I fix this?
In AmiBroker go File > Database settings and ensure that the Data source is set to "MetaStock data plug-in" (and not "local database") and Local Data Storage is disabled. If not, change them, click OK and then run Tools > XXX-PremiumData.

I am missing the Tools > XXX-PremiumData entry. How can I fix this?
Sometimes after upgrading (or reinstalling) AmiBroker, the Tools menu gets reset. You can manually add the symbol maintenance script to the Tools menu as follows:

ASX Stocks

Hide ASX Stocks

Go Tools > Customize
Click the Tools Tab
Click the New button
Call the item ASX-PremiumData
Under the Tools tab, highlight the item ASX-PremiumData
In the Command item put: wscript.exe
In the Arguments item put: //E:javascript Stocks-PremiumData.js ASX
In the Initial directory put: C:\Trading Applications\\bin
Click Close
Then go Tools > ASX-PremiumData

US Stocks

Hide US Stocks

Go Tools > Customize
Click the Tools Tab
Click the New button
Call the item US-PremiumData
Under the Tools tab, highlight the item US-PremiumData
In the Command item put: wscript.exe
In the Arguments item put: //E:javascript Stocks-PremiumData.js US
In the Initial directory put: C:\Trading Applications\\bin
Click Close
Then go Tools > US-PremiumData

Futures

Hide Futures

Go Tools > Customize
Click the Tools Tab
Click the New button
Call the item Futures-PremiumData
Under the Tools tab, highlight the item Futures-PremiumData
In the Command item put: wscript.exe
In the Arguments item put: //E:javascript Futures-PremiumData.js
In the Initial directory put: C:\Program Files\AmiBroker\Futures-PremiumData\Scripts (or C:\Program Files (x86)\AmiBroker\Futures-PremiumData\Scripts)
Click Close
Then go Tools > Futures-PremiumData

Forex

Hide Forex

Go Tools > Customize
Click the Tools Tab
Click the New button
Call the item Forex-PremiumData
Under the Tools tab, highlight the item Forex-PremiumData
In the Command item put: wscript.exe
In the Arguments item put: //E:javascript Forex-PremiumData.js
In the Initial directory put: C:\Program Files\AmiBroker\Scripts (or C:\Program Files (x86)\AmiBroker\Scripts)
Click Close
Then go Tools > Forex-PremiumData

When I run Tools > XXX-PremiumData, it opens a blank window and nothing happens. What should I do?
This issue can arise when there is a problem with Microsoft Edge or Internet Explorer, or with IE Mode in Edge, and/or when an extension or some sort of so-called "browser helper object" has been installed into Internet Explorer (possibly by malware, spyware, adware or other "add-on"/toolbar that may have been inadvertently installed). Such add-ons and extensions interfere with the normal operation of the Microsoft browsers (which we use to display the status of the maintenance script).
Microsoft have details on how to fix this in Internet Explorer here.
Details on how to disable extensions in Edge can also be found here.

How do I reorder the watchlists?
Using Windows Explorer, navigate to "C:\Program Files\AmiBroker\XXX-PremiumData\Watchlists" and edit the "index.txt" file. This must be done while AmiBroker is not running.
You can move your preferred watchlists to the top of the list (be sure to move the items and not copy them as the system cannot handle duplicate entries). Alternatively, if you just want to reorder the watchlists alphabetically, just delete the index.txt file.

How can I use Markets, Groups, and Watchlists in an Exploration?
Due to the automated set of markets, groups, and watchlists, it is now very easy to specify in an Exploration a filter to limit your scan to a specific set of securities.

Click here for some examples

Hide examples

On the ASX data, explore only Fully Paid Orindaries have closed above their 30 day moving average
Filter = InWatchListName("Fully Paid Ordinary") AND C > MA(C,30)

On the ASX data, explore only those securities that can be shorted on CMC Markets and have closed below their 30 day moving average
Filter = InWatchListName("CMC ASX Share Shortable CFDs Underlying") AND C < MA(C,30)

On the ASX data, explore only the securities that are within the S&P/ASX 300 index that have closed above their 30 day moving average
Filter = InWatchListName("S&P ASX 300") AND C > MA(C,30)

On the US data, explore only only exchange-traded securities (ie no OTC) that have closed above their 30 day moving average
Filter = GroupID(0) == 1 AND C > MA(C,30)

On the US data, explore only NYSE traded securities that are within the S&P 1500 index and that have closed above their 30 day moving average
Filter = MarketID(0) == 5 AND AND InWatchListName("S&P 1500") AND C > MA(C,30)

On the US data, explore only those securities which have CBOE Equity options available to be traded and are in the S&P 500, that have closed above their 30 day moving average
Filter = InWatchListName("CBOE Equity Options Underlying") AND InWatchListName("S&P 500") AND C > MA(C,30)

How do I access the watchlists in my AFL code?
You can reference the watchlists by name. To do this, put a filter into your scans/explorations with:
    Filter = InWatchListName("watchlist name here");
    for example:
    Filter = InWatchListName("S&P 500");

The Dow Jones Industrial Average only shows a few days of data. How can I see more?
Prior to Amibroker v5.60 there was a bug in the MetaStock data plugin. To workaround this, click File -> Database Settings and set the number of bars to 31000.

I receive the error message "Automation server can't create object" or "Could not start initial objects" when trying to run the Tools > XXX-PremiumData from within AmiBroker. How do I fix this?
Firstly, make sure that Internet Explorer is installed in your machine and fully functional, even if you don't use it as your default browser.

Otherwise, this error may be due to parts of the Windows Operating System not being properly installed or modified by a badly behaving program. We have only seen this twice with Windows Vista users and we suspect it is due to a not-so-Vista-compatible piece of software interfering with the registry.

Here was how it was fixed

Hide

Click Start, All Programs, Accessories, right-click on Command Prompt, and then click Run as Administrator.

Type the following command and press ENTER.
regsvr32 scrrun.dll

Note that this may result in an error message being reported. This is normal.

Now reboot your PC, start AmiBroker, then try the Tools > XXX-PremiumData again.

For 64-bit Windows type:
cd c:\windows\syswow64\
then
RegSvr32 scrrun.dll

How can I increase the scanning/backtest speed of AmiBroker?

1. Put your data on a SSD drive (or purchase a SSD drive to replace your main system drive). Your exploration will run approximately 8 times quicker.
2. Prevent your virus scanner from performing real-time scanning of both AmiBroker Database and the actual data storage locations (e.g. C:\Program Files\AmiBroker\ASX-PremiumData and C:\Trading Data). Since there are no executable programs in these folders, scanning them is superfluous. Your exploration will run approximately 2 times quicker.
3. Click Tools > Preferences then click on the Data tab. If you increase the In-memory cache size to 20000 (max symbols) that should cover growth in this area for a while. 20000 is the maximum limit within AmiBroker. The Max MegaBytes can also be increased. Perhaps try increasing this to 1000MB (if you have at least 2GB of RAM) and increase further if you have more RAM. The 64 bit version of AmiBroker allows you to increase this to many Gigabytes - 4000MB is a good choice if you have the RAM to spare. This will cause the second and subsequent backtest/scan to perform at lightning-quick pace.
4. For most trading systems/scans you do not need the full data history for any given security. This can be set in File -> Database Settings. For example, if your scans only require the last year of trading history, try changing the number of bars to 300. When you wish to perform backtesting, remember to increase this level.
5. Use the a 64 bit operating system and the 64 bit version of AmiBroker. Our own tests have shown the 64 bit version to increase performance by 30%. Note: This requires you to have purchased or upgraded to the AmiBroker Professional Edition.

How do I speed up scans/explorations of the Australian Stock Exchange database?
The ASX database contains a lot (20,000+) of exchange traded options (ETOs) and Warrants (3000+). If you don't trade them and don't need the ETO and/or Warrant data we recommend you remove it with the following steps:

1. Start Windows Explorer (or click your My Computer icon) and navigate to Drive C, Trading Data, Stocks, ASX.
2. Delete the ETOs and/or Warrants Folder as preferred.
3. Start Premium Data.
4. Click the Configuration Tab.
5. Ensure that "Warrants" and/or "Exchange Traded Options", as preferred, are unticked for both intraday and end of day.
6. Click the Save button.
7. Exit Premium Data.
8. Start AmiBroker and click Tools > ASX-PremiumData. This will then remove the ETOs and/or Warrants from your AmiBroker database too.

How can I transfer my AmiBroker database to a new computer?

Show transfer steps

Hide Transfer Steps

The instructions below refer to US Stocks, but are equally applicable to ASX stocks and are broadly applicable to Forex or Futures.

These steps cover the transfer of both the AmiBroker symbols database and the underlying MetaStock-format price database. The latter may be installed rather than copied across.

1. Network your old machine and your new one, or use some external medium to handle the transfer (large USB memory stick of 4-8GB or an external USB hard drive).
2. Back-up the entire "AmiBroker" folder to the external medium (usually c:\Program Files\AmiBroker or c:\Program Files(x86)\AmiBroker).
3. Back-up the entire "Trading Data" folder to the external medium (usually c:\Trading Data).
4. Install AmiBroker on your new machine (get the AmiBroker program installer from the AmiBroker website).
5. Transfer the contents of the "AmiBroker" folder from the external medium to the "AmiBroker" folder on the new machine (C:\Program Files\AmiBroker).
6. Transfer the "Trading Data" folder to the new machine (c:\Trading Data).
7. Install the Premium Data updating application on the new machine (get the program installer from the Downloads area of our website).
8. Configure the Premium Data updating application so that its recognizes the data history location (open the application, click the Configuration tab and make sure that the entry for "Stocks Folder" reads - "c:\Trading Data\Stocks").
9. Run an update with the updating application.
10. Run the Amibroker integration script (which you can get from the Downloads area of our website).
11. Start AmiBroker and open the US-PremiumData database (c:\Program Files\Amibroker\US-PremiumData).
12. Click Tools > US-PremiumData in AmiBroker to perform symbol maintenance on this database (Tools > US-PremiumData).

How do I backtest on delisted data?

Assuming you have purchased historical data for US delisted, and already have the data installed, follow these steps:

For US Stocks:

Hide US Delisted Stock access

1. In your backtest formula, you will have a Buy condition.... Buy = {your buy condition} eg. Buy = Cross( MACD(), Signal() );
   Change this to
2. Buy = {your buy condition} AND (MarketID(0) == 3 OR MarketID(0) == 4 OR MarketID(0) == 5 OR MarketID(0) == 12 OR MarketID(0) == 13)
   This will ensure the backtest only runs against Major exchange-listed (ie AMEX, Nasdaq, NYSE, NYSE Arca) and delisted securities.
3. Then run your backtest against All securities (no filter set)

After running a backtest I still have an open position in a delisted stock. How can I get my backtest to exit this position?

Sometimes when you are backtesting with data that includes delisted stocks, you end up with an open position that is never closed. Here is some code that should be added to your trading system to simulate exiting the position on the final bar:

Show code

Hide Code

// For backtesting, we never want to be left holding a delisted stock,
// however not all versions of AmiBroker allow us to detect whether a security is delisted
// To simulate this we will provide a sell signal on the
// second last bar only if the last bar's date is >= 30 days from today's date.
// Also, we will not enter on the second or last bar of the trade
// as we will then be unable to exit the position if trade delay is set to 1 day.
// This code has been collaboratively developed by Norgate Investor Services
// www.premiumdata.net and its users and is made freely available to the AmiBroker community.
// Known issue: For highly illiquid securities that trade very infrequently will be considered to
// be inactive. This is probably fine for the purposes of backtesting as you would not want to be
// attempting to trade a rarely-traded security.

NonTradedPeriod = 30; //in calendar days
SecurityIsInactive = LastValue(DateTimeDiff(Now(5),DateTimeAdd(DateTime(), NonTradedPeriod, inDaily))) >= 0;

OnSecondLastBarOfInactiveSecurity = BarIndex() == (LastValue(BarIndex()) -1) AND SecurityIsInactive;
OnLastTwoBarsOfInactiveSecurity = BarIndex() >= (LastValue(BarIndex()) -1) AND SecurityIsInactive;

Buy = Buy AND NOT OnLastTwoBarsOfInactiveSecurity;
Sell = Sell OR OnSecondLastBarOfInactiveSecurity;

// If you have a short selling system you will also need to include the following:

Short = Short AND NOT OnLastTwoBarsOfInactiveSecurity;
Cover = Cover OR OnSecondLastBarOfInactiveSecurity;

My futures data keeps showing non-trading days such as holidays and weekends. How can I remove them?

In AmiBroker there is an option under the View menu to "Pad non-trading days". If you uncheck this your charts will go back to normal.

When backtesting Futures, I receive the error message "not entered because requested size is less than minshares/minposvalue" or "not entered because of insufficient funds or wrong position size/value". How do I fix this?

Futures can only be traded in whole contracts (i.e. "round lots"), so the RoundLotSize in AmiBroker is set to 1 (unlike stocks where RoundLotSize=0). Backtests on Futures can be quite sensitive to this setting.

Here are some things that you can check:
Do you have enough equity to cover the margin required for a round lot?
What is your base currency set to (Tools > Preferences > Currencies)? If your base currency is different to the currency of the Futures contract then your available equity may have been reduced due to the currency conversion.

When backtesting, Error 47 occurs. I am using SetForeign/RestorePriceArrays.

When SetForeign is used, AmiBroker needs to keep the contents of the foreign symbol referenced in SetForeign in its in-memory cache. If the number of foreign symbols accessed exceeds the cache size then error 47 will be given. You should increase the In-memory cache size in Tools -> Preferences -> Data. Ensure that the max symbols and max megabytes has enough size to cache all symbols. You can monitor the usage of the cache by clicking Tools -> Performance Monitor.

Where can I get help on using AmiBroker?
For help with using AmiBroker see the AmiBroker website. The AmiBroker staff are also quick to respond to support emails.
Some great books have been written on trading systems that use AmiBroker as the underlying anlaysis software by Howard Bandy.
There are also some AmiBroker user forums available on the Internet - The Official AmiBroker Forum, the old AmiBroker Yahoo Group (this group is due to be archived in July 2017 - all content from 1998 remains searchable), and the Unofficial AmiBroker Users Forum. Other generic forums also have threads on AmiBroker coding including: Aussie Stock Forums (Software and Data), EliteTrader and Trade2Win.
Should you require it, paid assistance is also available from AmiBrokerCoding.Com, TheChartist and Alvarez Quant Trading.