Evaluating the trading performance of strategies

A common problem for a trader is to know when his strategy has lost the advantage, or in short, when there is such a situation that does not fit today’s strategies. Strategies generated by data mining can experience performance degradation quite quickly, and these snippets can help analyze such situations. This episode is the introduction to a series of articles in which I will introduce new WHAT-IF snippets to help us deal with this problem. In the following episodes, we will also introduce snippets based on statistical quality control logic, and finally, we will conduct a study comparing whether these snippets statistically reduce the risk profile of strategies.

In today’s part, I have prepared a snippet that allows the strategy developer to measure the maximum drawdown in the in-sample period (the training period) and then terminate the strategy when it reaches the percentage value of this drawdown in the out-of-sample period.

.

I’ve prepared two versions of the WHAT-IF Analysis

 

1.Training/Test logic

e WHAT-IF snippet, you can set the percentage threshold of the percentage drawdown in the sample. For example, you can choose 50% of IS DD or 150% of IS DD. Finally, the strategy turns off during the out-of-sample period when it reaches the percentage drawdown measured in the DD sample.

2. Training/Validation/Test logic

This WHAT-IF snippet works with training/validation test logic. This way of analysis is based on the hold-out method. In the hold-out method for model selection, the dataset is split into three different sets – training, validation, and test dataset.

Source: https://vitalflux.com/hold-out-method-for-training-machine-learning-model/

 

The What If snippet has two mods.

 

  1. The maximum drawdown is measured in the training part of the data, and the strategy stops trading in the validation and testing part of the data after reaching the maximum training drawdown.
  2. The WHAT-IF snippet determines the maximum drawdown in the validation part of the data and stops trading in the testing part of the data after reaching the percentage drawdown measured in the validation data part.

 

In Strategy Quant X you can set the training/validation/testing in the Data section.

 

More information about how to use and set training/validation/testing you can find here.

 

Let’s take a look at the use of these snippets in practice:

 

Training/Testing What-IF snippet

 

In the figure above, we see a simple RSI strategy backtest divided into an in-sample period and an out-of-sample period. We can see that the strategy reaches the maximum drawdown in the out-of-sample period.

 

After using the WHAT-IF snippet, we can see what happens when the strategy exits the trade after reaching 50% of the maximum drawdown from the sample part of the data. The strategy in Out of sample will end the trade early. At the bottom of the database, we can see how the statistics of each strategy have changed. The drawdown in What If Out Of Sample is lower.

 

Training/Validation/Testing WHAT-IF snippet

 

Snippet works in two modes:

Mode =  1

Operates under the assumption that the stratégy trading is turned off when the percentage value of the maximum drawdown from the training period is reached. This logic is applied both in the validation and testing period.

Mode =  2  

Works with the assumption that the stratégy trading is turned off when the percentage value of the maximum drawdown is measured in the validation period. This logic is applied in the test period.

 

 

Basic backtest

In the following figure, we see the initial backtest of the strategy divided into Training/Validation/Testing.

 

 

Mode = 1

After using the WHAT-IF snippet, we see that the strategy failed to achieve 50% of its drawdown measured in the Training period. 50% of the drawdown was reached only in the test part of the data, where the trading was stopped.

 

Mode =  2

In this mode, I set a drawdown of 100% in the validation part of the data. In the testing part, the strategy reached a drawdown of 100% quite quickly and stopped trading.

 

These “WHAT-IF” snippets are an introduction to the topic. The goal is to work out the entire workflow. As for implementation in platforms, → the above solution can be implemented in MT4/MT5 and also in Tradestion/Multicharts. Anyway, in terms of modularity, the best platform is Metatrader 5.

At the end of the blog, I would like to open a discussion about ideas that could be integrated into the snippets. If you have ideas, please post them in the discussion below the article, or directly to [email protected]

 

Both What if snippet you can download here.

 

clonex

clonex

5 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Bee trader
Bee trader
March 2, 2022 10:01 am

Hi Clonex,

Thanks for the writeup and snippets. Have you considered using a method of fitting a distribution like the Beta and then use the statistical properties to stop trading if there is a deviation beyond the allowable/chosen threshold? Tim Masters talks about this in his books.

Thanks
Bee Trader

Bee trader
Bee trader
Reply to  clonex
March 2, 2022 4:51 pm

Which SQC methods are you considering for your next research?

Emmanuel
Emmanuel
April 11, 2022 6:03 pm

Excellent !!!!!! Thank you very much Clonex for this awesome work !!!!!

Continue reading