IB / TWS Bridge
Interactive Brokers™ main advantages are access to many exchanges, a huge number of supported assets, and relatively low trading costs. The IB bridge allows direct trading with IB, either through IB's Trader Workstation (TWS), or with the IB Gateway. For long-term automated trading the IB Gateway is preferable, since the TWS, a bloated java program, normally stops every 24 hours and interrupts the connection.
For trading with Gateway, select IB API (not FIX CTCI). Open Configuration/API/Settings and make sure that the socket port is set to 4002. Enter an individual number from 1..8 in Zorro's User field for connecting to the Gateway via socket port 4002. The Password field can be left empty.
For trading with the TWS, open File/Global Configuration/API/Settings (see screenshot below) and select Enable ActiveX and Socket Clients. Make sure that the socket port is set to 7496. Deselect Read-Only API when you want to send orders. Enter an individual number from 101..108 in Zorro's User field for connecting to the TWS via socket port 7496. The Password field can be left empty.
Up to 8 Zorros can connect to the same Gateway or to the same TWS, but only a single TWS or a single Gateway per
user is supported by IB. The connection will break down when a second Gateway or TWS instance is opened, even on a different PC.
||1 .. 8 for the Gateway,
101 .. 108 for the TWS
Trading with IB
The included plugin supports IB paper trading accounts that begin with the letter "D". A plugin for all accounts is included in Zorro S or available for subscription on the Zorro download page. For opening an account with IB, the normal choice is a RegT Margin account. After opening the account, make sure to withdraw enough from your initial deposit as to not exceed the account size limit of the free Zorro version. If you own Zorro S, consider a Portfolio Margin account with its lower margin costs. IB paper trading accounts are only available after opening a real account.
Due to the high margin and lot size requirements, most Z systems are not really suited for IB, as they would require a large Margin setting for not skipping too many trades, and even then achieve only a fraction of the annual return due to the low leverage. Only exception is Z8 that is especially tailored for IB. The main advantage of IB is that most public financial assets can be traded. This allows many new trade systems that exploit specific inefficiencies, f.i. volatility, or seasonal effects of particular stocks or treasuries.
Historical price data for backtests can not be downloaded from IB, but they are available from many other Internet sources, f.i. from Yahoo™ or Quandl™ by using the Download script or the assetHistory or dataDownload function.
For trading an asset, some prerequisites must be fulfilled:
- Subscribe market data on the IB website; otherwise you'll get a "market data not subscribed" error message at session start. Forex market data is free, most other data requires a monthly fee. For subscribing market data, enter your IB account management page, and select Trade Configuration / Market Data. For the usual US stocks and ETFs, subscribe the "U.S. Securities Snapshot and Futures Value Bundle" and the "US Equity and Options Add-On Streaming Bundle". For demo trading, activate Share Market Subscription for sharing your subscriptions with the paper trading account.
- Obtain trading permission from IB for all asset types you want to trade, under Trade Configuration / Permissions (see image below). You must fill in a form about your trading experience and your financial situation. It takes about 24 hours to get approval for trading all asset types. Penny stocks require a special approval.
- Remove trading restrictions. Open File / Global Configuration / API / Precautions / Bypass Order Precautions for API Orders and enable Suppress market cap check. Otherwise orders of more than 5 contracts will not be executed and you'll get an "over limit" error message. Alternatively, you can increase order precaution limits individually per asset type.
- Make sure that all traded assets are listed in the TWS Monitor and their prices are visible before connecting to IB (even when using the Gateway). A system can not be started when its assets prices or history are currently not available due to holidays or outside market hours.
Not all assets can be traded all the time. Index (IND) assets are read only, but you can trade index ETFs, Futures, or CFDs. Be aware that you have to pay interest for CFDs, so they are no equivalent to ETFs. Some assets, such as CFDs and gold/silver commodities, are not available to US citizens. For getting permission to trade them you have to open an UK or other account outside the US. The default minimum sizes for currency pairs are in the 20,000 contracts range, but entering a lower minimum size in the asset list is also possible. Orders are then marked as 'odd lot' and cause higher transaction costs. Some stocks (STK) and some other assets can be not shortable temporarily (dependent on market liquidity) or permanently. Short trades for closing a long position are normally accepted.
For setting up the IB asset list and for simulating IB trading in the
backtest, margin requirements can be found on
maintenance or end-of-day margin is often higher than the initial margin.
Any asset you want to trade must be represented by a line in the asset list. Asset symbols (Symbol column) are converted to IB symbols in the following way:
- Any symbol name containing a '/' slash at the 4th position (f.i. EUR/USD) is assumed a currency pair. Is is split in symbol (first three characters) and counter currency (last three characters), gets the asset type CASH and is routed to the IDEALPRO exchange. Currency pairs are displayed with a dot (f.i. EUR.USD) in the TWS, and the purchased amount is listed separately per currency in the TWS account window.
- Any symbol name containing '-' hyphen characters (f.i. AAPL-STK-NYSE-USD, or XAGUSD-CMDTY) is split in strings separated by hyphens. String 1 is the asset, string 2 the type (STK, OPT, FUT, FUTX, IND, FOP, WAR, CASH, CFD, STKCFD, FUND, EFP, BAG, BOND, CMDTY), string 3 normally the exchange and string 4 the currency. If the currency is omitted, USD is assumed. If the exchange is omitted, smart routing is used with no primary exchange. The following exchange codes are supported: SMART,
XETRA. Details about smart routing and the supported exchanges can be found (or not) in the IB documentation.
- Two special symbol types are supported by the bridge. The type STKCFD trades as a CFD, but gets the price quotes and historical prices of the underlying stock. This is useful for stock CFDs (f.i. AAPL-STKCFD) which can be traded, but get no price quotes from IB. The type FUTX can be used for downloading historical data of expired futures contracts.
- Options and futures options have symbol names in the format Underlying-OPT/FOP-Expiry-Strike-P/C-Exchange, f.i. ES-FOP-20161218-1350.0-C-GLOBEX. Futures have symbol names in the format Underlying-FUT/FUTX-Expiry-TradingClass-Exchange-Currency, f.i. SPY-FUT-20161218-SPY1C-GLOBEX-USD. Some futures, f.i. ES, require the symbol instead of the trading class, f.i. ES-FUTX-20170317-ES-GLOBEX. If the currency is omitted, USD is assumed. The symbol names of options and futures are normally automatically generated by calling the contract function.
- If the name contains neither a slash nor a hyphen, is it assumed a stock or ETF in USD. SMART routing with primary exchange ISLAND is used. These parameters supposedly work for all US stocks and ETFs.
A example file AssetsIB.csv with currencies, CFDs, and stocks is included; we do however not guarantee the accuracy and correctness, so use it at your own risk.
Extra data and broker commands
The IB bridge supports the following additional data streams:
- marketVal: Not supported in historical data,
bid-ask spread in live data.
- marketVol: Trade volume per minute in historical
data; accumulated volume since market open in live data. Only for exchange-traded assets, not for currencies.
Multiple IB accounts are supported from Zorro 1.84 on. The account can be set
up in the account list.
The IB bridge supports the brokerCommand function with the following additional commands
(extra commands, f.i. for market depth data or news messages, can be implemented
on user request):
- GET_POSITION (Positions for forex pairs are not supported, but for single currencies, f.i. "EUR").
(1 = ask/bid, 2 = last trade; for exchange-traded assets only)
- GET_BOOK (market depth must be
subscribed and the exchange must be coded in the asset symbol)
Known IB API issues
In comparison to other broker APIs, the IB API has several limitations that must be worked around in the script, and can make an IB trading script a bit more complicated:
- Limited history. IB allowed only 60 requests of historical data every 10 minutes. This limitation restricts automated trading strategies to few assets and short lookback periods. More assets and long lookback periods, as with some Z systems, cause a "pacing violation" error message at session start. In that case use the PRELOAD flag and get recent historical data from other sources. Update 2017: According to their new API specifications, IB has removed the limit for M1 or low-resolution data (the current limitations can be found under http://interactivebrokers.github.io/tws-api/historical_limitations.html#gsc.tab=0). The assetHistory function can be used for downloading up to one year of historical M1 data for most available assets from IB. For long lookback periods or for asset types such as options and futures, historical prices are only available in reduced resolution, such as 1 hour or 1 day.
- Trades. The IB API is NFA compliant and does not control individual trades. So all trades are strictly virtual and controlled by Zorro only. The account portfolio only reflects the net sum of all open positions by all connected Zorros. The NFA flag must be set for all IB accounts, even outside the US.
All trades are IOC by default; partial fills are possible. Stops are handled by Zorro and not sent to IB. Closing positions (f.i. by manual intervention) is not reported through the API and thus won't automatically close the corresponding trades in the Zorro session. The brokerCommand(GET_POSITION) function and the LotsPool variable can be used for comparing open positions between Zorro and IB as long as it's not a currency pair. Use cancelTrade for removing manually closed positions and for keeping the session in sync with the account.
- Market hours. Outside market hours - normally 9:30 .. 15:00 ET for US-traded assets - price quotes are often not available
from IB, so you'll get a "unavailable at this time" message
when attempting to connect. Therefore begin the session when the market is
open. If PRELOAD is set and the API does not respond to a price request, Zorro will attempt to retrieve the last known historical price. This is indicated by a "frozen price" message. If an order can not be immediately executed, it is cancelled, so there won't be any pending orders except for Zorro-controlled entry conditions. Sometimes orders can be executed outside market hours, but with higher transaction costs. Therefore trading outside market hours should generally be avoided.
- Asset parameters. Except for price and bid/ask spread, asset parameters are not available through the IB API. Leverage, commission, pip cost, and lot size of traded assets must be located on the IB website and entered manually in the asset list spreadsheet. Typical leverage on a RegT account is 33 for most Forex pairs, 4 for stocks and ETFs, and 20 for CFDs. The parameters need not be 100% accurate, but should not be too far off in the interest of realistic backtests. You can find examples (with no guarantee of accuracy and correctness) of major currencies, CFDs, and stocks in the included file AssetsIB.csv. Commissions are not included.
- Prices. The prices can be switched with the
SET_PRICETYPE command between Close (Ask/Bid) and Last price.
Strategies normally use the Close price, but with some assets - for
instance, with indexes such as VIX - the Last price is more recent.
- Account status. The API provides a lots of account parameters, but none
of them is 100% equivalent to balance and equity. Therefore proxies are used: Equity is set to the
Net Liquidation Value, Balance
is set to Available Funds, and MarginVal is set to the Current Maintenance Margin. Non-US accounts sometimes display account parameters in wrong currency immediately after connection. This is corrected after about a minute.
- Limited result accuracy. Profits displayed in the TWS by closing a trade can be very different from the real trade profit, since the API returns only the profit based on the average of all open positions. Total profit will also differ slighty because transactions costs and leverage can not be retrieved through the API, therefore approximate values are used on the Zorro side.
- Connection reliability. The IB API will produce warnings during operation, some harmless, some more serious. In the Gateway log you'll see ominous messages such as
"Invalid incoming request type 0". They are by design
(heatbeat of the used IB API library) and can be safely ignored. Warnings of orders outside business hours can also be ignored as long as the order is still executed. A more serious matter is that the Gateway tends to occasionally log out and in by itself, displaying messages like
"Connectivity between IB and Trader Workstation has been lost". Even without logging out, the connection to price servers is sometimes briefly interrupted with the message
"Data Farm connection lost". Normally all this does not require user intervention, since
Zorro automatically reconnects and continues the session in such cases. However during log out time Zorro can not react on price events. So an IB script should detect offline periods by comparing bar times, and check asset positions (see above) for making sure that no trades were closed during that time.
A user reported a Gateway bug, causing a loss of connection after some time when requesting
futures prices. This problem did not happen with other assets but futures, and
also not with earlier Gateway versions or when connecting to the TWS. If could be
fixed by setting TickTime to 2000
or above, this way reducing the price request rate.
- Single session only. You can not connect with your IB user name to the TWS and the Gateway at the same time - not even for only checking your account. When you log in with the TWS, the Gateway connection breaks down. After terminating your TWS session you must therefore manually re-login from the Gateway. Zorro will automatically resume the session after an interruption, but while not connected it can not handle entries, stops, or profit targets. For working around this problem, you can register a second user for your IB account, and open the TWS with this second user name. This workaround does not work for IB paper accounts, which are single-user only.
- No hibernation. The PC must not be reset, restarted, switched off, hibernate, or go in suspend mode while connected to the IB API. Otherwise the API may crash and require a Zorro restart.
On the other hand, the IB API offers many interesting features that are not available in Forex broker APIs, such as volume data, market depth data, options and futures trading, and news messages. Especially market depth is of interest for HFT strategies since it gives insight in short-term trends. If you need additional features for your automated trading strategy, please contact us. But be aware that those additional data are mostly unavailable in historical data and thus can not be easily used for simulations and backtests.
Brokers, Broker plugin,
► latest version online