assetHistory(string Name, int Mode): int

Parameters:

Returns:

assetSource(string Filename, string URL, string Header, string Body, string Format)

Parameters:

Remarks:

• Price history is stored in the History folder under Name plus .t6 extension for D1 data, or Name plus .t1 for T1 data. The file name can be modified with the History string. T1 or M1 data downloaded from a broker is split into separate years, EOD data or data from an online source is stored in a single file. The Quandl database name, an appended ".US" by Stooq, and '/', '.', or ':' characters are automatically stripped from the historical data file names. If the name is otherwise different to the asset name, use file_copy to copy the downloaded file to the correct asset name (f.i. file_copy("History\\ICE.t6","History\\LONDONICE.t6").
• Price history is only downloaded in the INITRUN or in the main function, and only when existing price history is not already up to date. For this the most recent online price is compared with the history file date, and if the difference is less than half the data resolution, no new history is downloaded. If FOR_ASSET is set, UpdateDays are also considered. For downloading data after the INITRUN and regardless of existing history, set the OVERRIDE flag.
• If the same price history is downloaded multiple times, for instance by several different scripts, make sure that the begin of already existing data matches the longest needed data period. Newly downloaded data is only merged to the end of existing history, but not to the begin.
• The raw data from online sources (Google, Quandl, etc) is stored in History\history.csv or History\history.json. If the price source does not deliver price data, but some error message instead, a "invalid response" error is displayed. The error message can then be read from the downloaded raw file. The assetHistory function will attempt twice to download the data before giving up.
• If a download failed for some reason, delete the resulting partial history file (if any). Otherwise subsequent downloads might attempt to append to it or assume that the data is up to date. Usual reasons for failed downloads from online sources are wrong symbols or exceeding a quoata limit. The reason can be normally found in the downloaded history.csv or history.json file, which contains the response from the online source.
• When downloading data from a broker, the asset is subscribed automatically. The recent values of the asset's Spread, PipCost, etc. are updated to the Assets.csv file even if no price history is downloaded. This way new assets can be added and available assets can be set up to their current parameter values for further simulations.
• Price history from brokers is downloaded either for the number of recent years given by NumYears, or for the time period given by StartDate and EndDate plus optionally a LookBack year. If NumYears is set to -1, no data is downloaded, but Log\Assets.csv is still updated. Price history from online sources is downloaded for a fixed number of years up to the current day, depending on the source.
• Price data from a broker or from online sources with limited history (FROM_IEX, FROM_CRYPTOC, FROM_SOURCE) is appended at the begin of an existing data file. Price data from other sources is not appended, but replaces the previous file unless it's already up to date. Make sure that the time resolution of the downloaded prices matches the resolution of the historical data files. Normally the resolution is 1 tick for .t1 or .t2 files, 1 minute for .t6 files with year number, or 1 day for .t6 files without year number.
• For backtesting price history, BarPeriod must be at or above the price history resolution. Set it to 1440 for testing D1 price data. For testing only parts of a multi-year .t6 file, use an 8-digit StartDate or EndDate.
• Price history from online sources is temporarily stored in a dataset of T6 or T2 records with handle H_HISTORY. It can be used to evaluate the data without loading the history file.
• Check the correct spelling of the asset name or code. Spelling often differs slightly, f.i. the same stock can be named BRK.B, BRK_B, BRK-B, or BRK-B.US dependent on the website or price source.
• Outliers in the price history are automatically fixed on download dependent on the Outlier variable. The first detected outliers will trigger Warning 035. Some assets, such as cryptocurrencies or penny stocks, have a high volatility and require reducing the default outlier sensitivity when downloading prices of those assets.
• A price history file with length 0 prevents downloading prices for that particular year. This way, by storing a 0-byte file, price data that is not available or that is of bad quality can be excluded from download attempts and from the backtest.
• For T1 data, the price server of the broker must support quote-based price data; this is the case for FXCM, but not for Oanda, IB, or MT4(MT5 brokers. Dependent on the price server speed, downloading T1 prices can take a long time, such as a whole day for one year of price quotes. If the LEAN flag is not set, ask and bid prices are stored in the .t1 file, otherwise only ask prices. 
• Adjusted prices can be split-adjusted, dividend-adjusted, or both. For short-term trading backtests is split-adjustment the best choice, for long-term trading dividend- and split-adjustment. Since dividend adjustment has little effect on short-term backtests, you normally use dividend- and split-adjusted prices for all backtests.
• Comments about some predefined data sources (all trademarks are property of their owners):
  • Google prices are adjusted for splits only, unadjusted prices are not available. Google download is known to fail on certain assets for no apparent reason, even though they can be downloaded manually. The service was recently not available.
  • Yahoo data is split and dividend adjusted; UNADJUSTED is supported. Yahoo EOD data was reported to contain 2 entries on dividend payment days, one with the current and one with previous day's data. Yahoo data of some European assets could be of poor quality; check the resulting price curve if in doubt.
  • AlphaVantage data loads relatively slow when there is much traffic. The data has good quality. UNADJUSTED is supported. The AV API key must be present under "AVApiKey = ..." in Zorro.ini. Free data is limited to 5 requests per minute and 500 requests per day.
  • Stooq requires a country identifier appended to stock or ETF names. If it is missing, ".US" is automatically appended. The data loads fast and is of good quality. UNADJUSTED is supported. There is a limit to the number of downloads per day, identified through the IP address.
  • EOD requires a country identifier appended to stock or ETF names. If it is missing, ".US" is automatically appended. The free subscription only supports one year data and only 20 requests per day. Prices are split and dividend adjusted, UNADJUSTED is supported. The key token must be entered under "EODKey = ...."  in Zorro.ini. Zorro users get a discount on the all-in-one data package when they order on this link.
  • Quandl data from the WIKI (stocks), CHRIS/CME and CHRIS/ICE (futures), BITFINEX and GDAX (Bitcoin and other cryptocurrencies) EOD databases are supported. Databases may be discontinued, so check it before on the Quandl website. Quandl databases in different formats can be downloaded with the dataDownload function. Quandl data loads fast and has usually good quality. The Quandl key must be present under "QuandlKey = ..." in Zorro.ini and the asset's database code (f.i. "WIKI/AAPL" or "BITFINEX/ETHBTC") must be passed to Name.
  • IEX provides EOD data for US equities. UNADJUSTED is supported; otherwise data is split- and dividend-adjusted. The IEX API token must be present under "IEXlKey = ..." in Zorro.ini.
  • Bittrex provides recent price history for a large number of cryptocurrencies. Format: BTC-ETH returns ETH prices in BTC. The history length is normally several years for D1 data, several months for H1 data, and several days for M1 data. It is recommended to insert a 4 seconds delay between subsequent loadHistory calls.
• CryptoCompare provides recent price history for almost all existing cryptocurrencies. Format: ETH/BTC returns ETH prices in BTC. The history length is up to 3 years for D1 data, 1 year for H1 data, and 1 month for M1 data. Since digital currencies are not traded at a central exchange, prices from different sources can be different.
• MT4/MT5 servers usually have no long price history, so the command will print an error message or only partially download the history.
• IB market data must be subscribed before it can be downloading. For downloading stock M1 data, send a SET_PRICETYPE,2 command for switching to the last traded price, because ask/bid prices are sometimes in poor quality.
• FCXM servers provide tick price history from 2002 for currencies, and from 2008 for index CFDs.
• Before downloading prices, check if they are already available in the History folder or on the Zorro download page. M1 and options EOD data for US stocks can also be purchased from oP group.
• Some data sources, such as Quandl, AlphaVantage, EOD, provide also earnings reports and other fundamental data in JSON format. It can be retrieved with http_transfer and parsed with strstr and strvar commands.
  

Example:

// Update M1 price history of all assets from selected broker
function main()
{
  NumYears = 2;
  while(loop(Assets))
    assetHistory(Loop1,1);
}

// Download D1 prices from Quandl
string Name;
while(Name = loop("AAPL","MSFT","GOOGL"))
  assetHistory(strf("WIKI/%s",Name),FROM_QUANDL);

// Download and plot Bitcoin prices from Bitfinex
function run()
{
  BarPeriod = 1440;
  assetHistory("BITFINEX/BTCUSD",FROM_QUANDL);
  assetAdd("BTCUSD");
  asset("BTCUSD");
  set(PLOTNOW);
}

// Download unadjusted SPY data and store it in "SPYuna.t6"
History = "*una.t6";
assetHistory("SPY",FROM_STOOQ|UNADJUSTED);

// Download AAPL prices from Stooq, using assetSource
assetSource("AAPL.t6",
"https://stooq.pl/q/d/l/?s=AAPL.US&d1=20000101&d2=20201024&i=d",
0,0,"+%Y-%m-%d,f3,f1,f2,f4,f6");
assetHistory(0,FROM_SOURCE);

// Download Bitcoin OHLC prices from Coin.io
int Month, Year = 2019, MinutesPerMonth = 31*1440;
for(Month = 1; Month <= 12; Month++) {
  string URL = strf("https://rest.coinapi.io/v1/ohlcv/BTC/USD/history?period_id=1MIN&time_start=%04d-%02d-01T00:00:00&limit=%d",
    Year,Month,MinutesPerMonth);
  string Name = strf("BTCUSD_%04d.t6",Year);
  assetSource(Name,URL,
    "X-CoinAPI-Key: my-coin-api-key",0,
    "[,time_period_end,%Y-%m-%dT%H:%M:%SZ,price_high,price_low,price_open,price_close,,volume_traded");
  assetHistory(0,FROM_SOURCE);
}

// Download Bitcoin BBO ticks from Kraken via Coin.io
var From = dmy(20190101), To = dmy(20200101);
for(; From < To; From += 1.) {
  string Start = strdate("%Y-%m-%dT00:00:00",From),
    End = strdate("%Y-%m-%dT00:00:00",From+1.);
  assetSource("BTCUSD_2019.t2",
    strf("https://rest.coinapi.io/v1/quotes/KRAKEN_SPOT_BTC_USD/history?time_start=%s&time_end=%s&limit=100000",
      Start,End),
    "X-CoinAPI-Key: my-coin-api-key",0,
    "[,time_exchange,%Y-%m-%dT%H:%M:%SZ,ask_price,ask_size,bid_price,bid_size");
  assetHistory(0,FROM_SOURCE);
}

See also: