Coding session transcript from 26/10/2021

The first coding session was a successful experiment, thank you EmanuelE, beetrader, clonex for your questions and the chat.

We will continue with coding sessions
it will be bi-weekly, always in the middle and end of month.

The next coding session will be on 18th November at 14:00 CET time, which is 13:00 London time, 8am New York time, 5am San Francisco time, 0:00 am Sydney Time.

All people who filled the coding session interest form will be notified about every next session (until they unsubscribe from session mailing list), if you want to get an invitation, please fill the form here.

Transcript of coding session – it is not 100% match, it is filtered and grouped by the question theme for better reading.

Calling external programs – Python, #F – from SQ

Beetrader
(2) How do I add an external script like F# or Python script as part of the custom project? (3) is there a way to link SQX and a database like mysql or Microsoft SQL using JDBC?

EmmanuelE
Bee, this is an interesting question. Some bridge .Net-java are available on internet. It would be interesting to see how to include a dll in SQX.
I am using ML.Net library, i would like to know if it possible to implement ML.Net library in SQX via a bridge .net-java

Mark
Hello Emmanuel – calling .NET from Java – honestly I haven’t tried it yet, I experimented with Python. It should be very similar.
The example article on Python from SQ is published here: https://strategyquant.com/doc/programming-for-sq/calling-python-from-sq-java-introduction/
The second example there with the use of ProcessBuilder can be used to invoke any external script or program. It is not specific to SQ, SQ snippets are written in Java, so you can look for Java examples for calling .NET or anything else.
If you need some bridge and add a custom JAR library to SQ, there is article about this: https://strategyquant.com/doc/programming-for-sq/using-custom-jar-libraries/

EmmanuelE
in fact, it is like any language, we just have to insert at the beginning “import xx.yy” , I guess we can import any dll, like any java program
It would be interesting to share, data, object, (signal, indicator) directly from a trading software to SQX
Python can be helpful , I am using ML .NET, it is very similar. and there is lot of improvement in ML .NET
I am working a lot with Neural Network

Mark (SQ)
I’m not sure it is that simple. generally, you can call custom methods from custom Java JAR, or Python or .NET libraries in SQ snippets, but you should know what you are doing. It isn’t that easy to create a custom indicator that will be computed in other code – it is technically possible, but it might be very slow.
as for using neural networks, it depends on what exactly you want to do with SQ and NN. Perhaps if you’ll describe more what exactly you want to do, I can think about how to do it

EmmanuelE
I found some bridge between NET Java : https://www.javonet.com/about/what-is-javonet/

Mark (SQ)
yes, there are ways to call .NET from Java, this itself is not a problem. The “problem” is how to connect your library to SQ, it depends on what exactly you want to do

Developing new Monte Carlo test

Beetrader
(1) How do I extend SQX so I can implement a different Monte Carlo permutation test in addition to the 2 that are in built. I am thinking of implementing Tim Masters Permutation test?

Mark
I’m not familiar with the exact algorithms of his permutation tests – is he making permutations of the resulting trades, or something else?
In any case you can have a look at how the current MC tests are implemented – the source code of each test is available.
If you’d describe better what you want to do I can give you some hints
I’ll be able to help you the best in this live session if you’ll give me better description on what exactly you want to do – in case of MC test the description of algorithm to implement. Or if you already started then the snippet code that you have so far and what problem you run into.
I can give only general answers to general questions.
But I’ll try to look at Tim Masters Permutation test if my time allows and publish it in the transcript of this chat

Beetrader
He permutes the timeseries before feeding them into the algo. Uses Logarithms to find differences e.g. of close prices, randomises that and recreates the timeseries.

EmmanuelE
I see we can develop indicators , signals, column of databank. Can we access trade to implement a new kind of test ? Can we add a new cross check ?
Is the Monte Carlo cross check available ? as an example ?

Mark (SQ)
you can extend much more than this. Yes, you can easily create a new Monte Carlo method, there already is one example in our codebase: https://strategyquant.com/codebase-category/mc-retest/
this way you can add new simulation method for MC.
Also, full source code of all the existing Monte Carlo tests is available in Code Editor in SQ.

EmmanuelE
Excellent !!!!!!!!!!!!!!!!! I was looking for this information !!!

Mark (SQ)
you can also access the strategy trades or even a batch of strategies to do your own analysis, this is best done using CustomAnalysis snippet as a part of Custom project workflow, an example: https://strategyquant.com/doc/programming-for-sq/example-per-strategy-custom-analysis/
custom analysis can be run after your builder or retester finisheds, it can perform its own analysis and (for example) delete the strategies that don’t pass your validation. Or use JDBC to connect to your database and upload something there

EmmanuelE
this is awesome !!!!!!
We can do a lot !!!!

Mark (SQ)
yes, there are many things that SQ can do that are not well documented yet, we are trying to improve it now

EmmanuelE
This give a lot of flexibility !!!!!
Is there a code , where we create a portfolio ?
Is there a code of Walk Forward test ?

Clonex
Yes it is possible to work with WFO. We add examples ASAP – during 1. half of November

EmmanuelE
It would be helpful to see how the strategies a selected from run to the next run
I would like to create portofolio with the WF trade results in it

Mark (SQ)
do you mean portfolio of WF equities?

EmmanuelE
yes

Mark (SQ)
it should be possible. The simplest is to do it in Custom analysis snippet, I’ll make an example of that

Example creating portfolio from WF results: Per databank custom analysis snippet.

Linking SQ and database

beetrader
(3) is there a way to link SQX and a database like mysql or Microsoft SQL using JDBC?

Mark (SQ)
generally yes, but link what exactly? do you want to load data for backtest from DB, or store strategy results to DB?

Beetrader
I want to store results into database so that later I can run comparisons with live action. I currently do this manually in QA.

Mark (SQ)
do you want to do this in SQ? it is possible using Custom analysis snippet, but what exactly do you want to store to database? strategy results, or trades?

Beetrader
ideally I want to store everything into the database. I know I can store the strategy xml, in which case do I just create a custom task to read the xml and save what I want into the database?

Mark (SQ)
yes, and you can store everything else in the same way. You have access to metrics (Netprofit, # of trades, etc) as well as complete list of trades for every strategy in the databank.
look at the Custom analysis example here: https://strategyquant.com/doc/programming-for-sq/example-per-strategy-custom-analysis/
it can be called for every strategy in the databank. Then just get the things from ResultsGroup and save them

Strategy with all its data is stored in ResultsGroup object: https://strategyquant.com/doc/programming-for-sq/working-with-resultsgroup/
Custom analysis can be run after your Builder or Retester finishes, or as a part of custom project. It can perform its own analysis and (for example) delete the strategies that don’t pass your validation. Or use JDBC to connect to your database and upload something there
Again, SQ Snipppets are in pure Java, so you can simply add code to connect to your database through JDBC and save whatever you want there.

QuantAnalyzer – Martingale MM simulation

Beetrader
how can I create a martingale money management in Quant Analyser. I tried to follow the existing code but couldn’t find a way to refer to previous trades.

Mark (SQ)
you are right, you cannot refer to previous trades in MM, I must think about it – if it is possible somehow. I’ll post better answer and maybe some example in the chat transcript

EmmanuelE
I have the same, I want to create another money management as well with a different way to buy/sell/hedge.

EmmanuelE
I need as well to refer to previous trade, to open multiple trade

Beetrader
I sent some examples of how we can start to improve the documentation. If you have a way for user submitted examples then you can check and review before posting on the api documentation.

This answer is still worked on, it will be published in the next days.

Update Builder config on the run

Beetrader
SQ in the custom projects is there a way to update the builder task with output from a previous say pythong script which calculates e.g. ranges I want to use as stop loss and take profit.

Mark (SQ)
it should be possible, but it will be quite tricky. I must look at how to do it, I’ll try to make an example for it.

This answer is still worked on, it will be published in the next days.