Preparing Accurate Data for Algo Trading: Broker Data Feed Differences

Introduction

In the world of algorithmic trading, data is king. The quality, accuracy, and reliability of your data can significantly impact the performance of your trading strategies. This article will focus on the nuances of forex broker data feed, the importance of accurate historical data, and the impact of timezone and Daylight Saving Time (DST) differences on your trading strategies. We will focus on two popular forex brokers (Dukascopy and Darwinex) and discuss how to prepare your data correctly to ensure your strategies align with your broker’s data directly in StrategyQuant X.

IMPORTANT NOTE

Please be advised that the foreign exchange (forex) and crypto market are decentralized markets, which means that there is no single, centralized exchange or data source. As a result, the data feeds provided by different brokers and financial institutions may differ in terms of exchange rates, quotes, and other information.

These variations can be attributed to factors such as market liquidity, differences in data sources, the methodologies used to calculate exchange rates, and discrepancies in the timing of data updates. Although efforts are made to ensure accuracy and consistency in the data provided, we cannot guarantee the complete reliability or uniformity of the forex data across different brokers or data feeds.

Forex Broker Data Feed Differences: Dukascopy and Darwinex

Forex brokers often have different data feeds, which can lead to discrepancies in price quotes, spreads, and liquidity. These differences can affect the performance of your algorithmic trading strategies.

Dukascopy, for instance, is known for its high-quality data feed, which includes historical tick data going back many years. This is particularly useful for strategies that rely on high-frequency data.

Other option to free forex data is Darwinex that provides data feed, focusing on providing high-quality but the history is shorter than Dukascopy data feed.

The key takeaway here is that the choice of broker can impact your strategy’s performance. It’s crucial to understand the characteristics of your broker’s data feed and adjust your strategies accordingly.

Please note that every broker can have different data feed so you need to verify the results in the target platform first. How to do backtest for each platform, you will find at the end of the article where have prepared roadmap.

Timezone Differences in Historical Data

Timezone differences can significantly impact the interpretation of historical data. Forex markets operate 24 hours a day, and different brokers may use different time zones for their data feeds. This can lead to discrepancies in the opening and closing times of candles, which can affect technical analysis and indicator calculations. For instance, a broker using GMT time have their daily candle close at a different time than a broker using New York time. This can lead to different interpretations of the same market event. To mitigate this, it’s essential to adjust your historical data to match the timezone used by your broker. This will ensure that your backtests accurately reflect the market conditions you will face in live trading.

What is timezone?

A timezone is a region of the globe that observes a uniform standard time for legal, commercial, and social purposes. Timezones tend to follow the boundaries of countries and their subdivisions instead of strictly following longitude, because it is convenient for areas in close commercial or other communication to keep the same time.

EST+7:

EST stands for Eastern Standard Time, which is the time zone of the East Coast of the United States and Canada, and some countries in the Caribbean. It is 5 hours behind Coordinated Universal Time (UTC-5). When you see EST+7, it means the local time is 7 hours ahead of Eastern Standard Time. This would be equivalent to UTC+2, which is the timezone for countries like Egypt, South Africa, and most of Eastern Europe.

Daylight Saving Time (DST): DST is the practice of setting the clock ahead by one hour from Standard Time during the warmer part of the year, so that evenings have more daylight and mornings have less. The idea is to make better use of natural daylight during the evenings, and also to conserve energy by reducing the need for artificial lighting in the evening.

However, not all places observe DST and those that do may start and end DST at different times. In regions that observe DST, clocks are usually set forward one hour in late winter or early spring, and then set back again one hour in the autumn to standard time.

EET:

EET stands for Eastern European Time. It is a timezone which covers Eastern Europe, the Middle East and parts of Africa. EET is 2 hours ahead of Coordinated Universal Time (UTC+2). Countries in this timezone include Finland, Greece, Israel, Egypt, and South Africa, among others.

Daylight Saving Time (DST): DST is the practice of setting the clock ahead by one hour from Standard Time during the warmer months of the year, so that evenings have more daylight and mornings have less. The intent is to make better use of natural daylight during the evenings, and also to conserve energy by reducing the need for artificial lighting in the evening.

In regions that observe DST, the Eastern European Time becomes Eastern European Summer Time (EEST), and the offset becomes UTC+3. However, not all places within the EET timezone observe DST, and those that do may start and end DST at different times. It’s important to check the specific DST rules for each country.

Dukascopy

Raw data source in the Quant Data Manager is in the UTC timezone, so you need to clone the data to your target platform. If you use MT4 from Dukascopy the timezone  is EST+7.

Darwinex

Raw data source in the Quant Data Manager is in the UTC timezone, so you need to clone the data to your target MT4/5 platform which is EST+7

IC Markets

This broker currently using EST+7 timezone

OANDA

This broker currently using EST+7  timezone

RoboMarkets (RoboForex)

This broker uses UTC+2 with DST EET on the MT4/5 platform

IMPORTANT NOTE

Always make sure that the timezone is the same. This info can be outdated. If you are not sure about your timezone, please contact your broker support or our support.

DST Differences and Impact on Strategies

Daylight Saving Time (DST) can add another layer of complexity to your data preparation. Not all countries observe DST, and those that do might not change their clocks at the same time. This can lead to an hour’s difference in data for a couple of weeks a year, which can impact your strategies. To account for DST, you should adjust your historical data to match the DST rules followed by your broker. This will ensure that your backtests accurately reflect the timing of the market events during the DST period. 

Algo Trading Strategy Backtest Accuracy: Tick vs M1

The granularity of your data can significantly impact the accuracy of your backtests. Tick data, which records every price change, provides the most accurate representation of the market. However, it can be resource-intensive to handle, especially for longer historical periods.

Minute data (M1), on the other hand, provides a balance between accuracy and computational efficiency. It records the open, high, low, and close prices for each minute, which is sufficient for most strategies.

However, for strategies that rely on high-frequency data, such as scalping or arbitrage strategies, tick data might be necessary to accurately backtest the strategy.

Starting with Accurate Historical Data

Starting with accurate historical data is crucial for developing reliable trading strategies. Here are some steps to ensure the accuracy of your data:

 1. Source your data from reliable providers: This could be your broker or a reputable third-party provider. Ensure that the data is clean and free of errors.

 2. Adjust for timezone and DST differences: Make sure your data matches the timezone and DST rules used by your broker.

 3. Choose the right granularity: Depending on your strategy, choose between tick data and M1 data. Remember, the higher the granularity, the more accurate your backtests will be, but also the more resource-intensive.

 4. Update your data regularly: The forex market is dynamic, and historical data can become outdated quickly. Regularly updating your data ensures that your strategies stay relevant.

Tutorial on how to download and prepare the data

FAQ

Q: How do I know if my broker’s data feed is reliable?

A: You can check the reliability of your broker’s data feed by comparing it with data from a reputable third-party provider. If there are significant discrepancies, it might indicate issues with the broker’s data feed. We also recommend to verify the results with the target platform first by doing backtest in the MetaTrader4/5, Tradestation and Multicharts.

Q: How do I adjust my data for timezone differences?

A: You can adjust your data for timezone differences by cloning in Quant Data Manager, as you can see on the following screen: 

Q: What is the impact of DST on my trading strategies?

A: DST can affect the timing of market events, which can impact your trading strategies. For instance, if your strategy relies on the opening price of the daily candle, DST can shift this time by an hour or if you have same strategies based on higher timeframes like H4 or D1, in this case will have an impact on OHLC data candles and if your entry level is on different price, the results of the strategy will be different. 

Q: Should I use tick data or M1 data for my backtests?

A: The choice between tick data and M1 data depends on your strategy. If your strategy relies on high-frequency data, tick data might be necessary. However, for most strategies, M1 data provides a good balance between accuracy and computational efficiency.

Q: I developed strategy on specific timezone. Can I use the strategy on the different timezone?

You need to verify the strategy on the different timezone first, if are the backtestst same you can use it. When the backtest is not the same, the impact on strategy is high and you should develop new strategy. 

Conclusion

Preparing accurate data for algorithmic trading can be a complex task, but it’s crucial for the success of your strategies. By understanding the nuances of forex broker data feeds, adjusting for timezone and DST differences, and choosing the right granularity for your data, you can ensure that your backtests accurately reflect the market conditions you will face in live trading. Remember, the quality of your data is as important as the quality of your trading strategy. So, invest the time and resources necessary to ensure your data is accurate and reliable. Your trading performance will thank you for it. The best option on how to get accurate backtest of the strategies is backtesting with broker data feed in the target trading platform. For forex markets we have good experience with MetaTrader 5. Most brokers provide latest data, so you can verify results directly in your trading platform. For Futures we have good experience. 

WE HIGHLY RECOMMEND TO VERIFY THE RESULTS IN THE TARGET TRADING PLATFORM FIRST, BEFORE YOU WILL TRADE REAL MONEY. 

ROADMAP to other articles

Data analysis

How to be prepared for data changes

Recommendation for export data from Tradestation

Historical data in SQ – quality backtest requires quality data

Backtesting 

Metatrader 4/5 backtesting

Multicharts backtesting, data preparation

Tradestation backtesting, data session

How to set tick pip step/size and point value correctly for CFD’s

Best practice on how to build multi-timeframe strategies

Comparing and using the same backtest settings

Stockpicker Backtest Limitations

Export strategy from StrategyQuant and test or trade it in MetaTrader

Troubleshooting 

What to do, if the backtest in the platform does not match with StrategyQuant X?

The strategy tried to place stop/limit order at incorrect price