SQ X updates

Here’s where we can discuss SQ X updates nad news.

This topic was renamed from SQ4 Early preview. The old, no longer relevant posts are or will be deleted and archived.

The point is in clearing up our forum from old, non-relevant stuff.

mabi wrote:

notch wrote:

I’m currently weening myself off the addiction to SQ3. SQ4 is truly a powerful piece of kit, I’m overwhelmed!! Thank you SQDEVT!!!

+1

Guys, it seems you are enjoying Mark’s fruit of labour – and I am not! My Genetic evolution has been running 48 + and not one strat generated!!

Before that I imported some strats previously randomly generated into Initial population. Didn’t work either!

Hi Mark,

SR106 looks realy nice. Not errorfree but the direction of the software is right. If you go forward in this way you can change the world.

thomas

thank you, I’m glad you like the new version. We are working hard to make it the best 🙂

Jojo wrote:

My Genetic evolution has been running 48 + and not one strat generated!! Before that I imported some strats previously randomly generated into Initial population. Didn’t work either!

If you don’t see any strategies within 30 mins – 1 hour something is wrong. Try to switch to Random generation. If no strategy passes the filters it might mean that the filters are too strict or not configured properly, you’ll see it more quickly in Random generation.

Mark

I know this has come up before but I just feel like I want to say it again. The idea of strategy quant being able to run on a GPU is probably our collectively largest possible gain. The nvidia P100 and V100 processors are literally hundreds of times faster than what we are all running.

I just spent some time looking into google cloud processing and if this was possible we could all do batch processing for a few hours at a cost of around $8 per hour.  This would be HUGE development!!!!

What are your thought on this?

jmtc1230 wrote:

Mark I know this has come up before but I just feel like I want to say it again. The idea of strategy quant being able to run on a GPU is probably our collectively largest possible gain. The nvidia P100 and V100 processors are literally hundreds of times faster than what we are all running. I just spent some time looking into google cloud processing and if this was possible we could all do batch processing for a few hours at a cost of around $8 per hour. This would be HUGE development!!!! What are your thought on this?

Hi jmtc1230, What do you mean “we could all” run batch processing at $8. Would it mean we share the cost by pooling our strats together for the purpose of processing?; have different logins, keeping our strats separate? I guess some of us have a lot more than others to process – how would we determine access/use/proportional cost? Sounds like a good idea – if only I can understand it better.

Mark, the cost of the google cloud environment costs around $5 to $10 per hour depending on the virtual machine and options that you select.   What I am planning to do is purchase the use of this space for a block of time, say 10 hours which will cost around $80 or so. I can then get 200x to 500x more work done in that time than on my personal computer.  I will gladly pay $80 for this ability at select times to get so much work done.

Josh

I said it few times already, the type of “real” trading engine that is implemented in new SQ4 is not suitable for computation using GPU.

But we are planning to make a simpler backtesting engine that can run on GPU, mainly for the purpose of training Neural Networks, or for very fats backtests using Selected Timeframe precision. So it is in middle or long term plan, but the exact architecture hasn’t been yet designed.

There are more important features for us to finish asap.

Now its a bit clear. So its not a collective effort you are talking about. Right Josh?

if you’ll check the pricing you’ll find out that renting power from Amazon/Google etc. is much more expensive than buying your own hardware, or even your own dedicated server.

So there isn’t much value in doing this.

As for some collective effort – it is a good idea, but difficult to implement and manage. It is something we’d like to provide ourselves in the near-mid future when grid will be finished – we will have multiple servers and we can rent the power for shorter periods of time so you can speed up some things.
But again, this is much more difficult to make than it seems.

JoJo, it could be but it does not have to be collective. It’s really just renting a virtual machine from Google with up to 96 Cores in it, and a Tesla P 100 GPU to be able to develop at very high speed with a windows environment.

Mark for me I see that this is really a GPU or no GPU conversation. Because to buy a Tesla P 100 it would cost around six or $7000  where is to rent one from Google would cost around say eight dollars per hour.   This would offer around 300 times more processing power than our CPUs can do and I would plan to for small blocks of time. Imagine getting 300 days of processing done in 1 day

If we are talking about staying only in a CPU environment only then I agree with you it is not worth it  and would be better to build our own systems.

Jojo and jmtc123,

Why dont you try daily strategies.  I ran for 2 days on EurUsd and i now have daily strategies that works on 21 markets with that 1 session.  All thought i have to spend some more computing power on it i suppose. Now crosschecks for additional markets have some bugs. So i split up the data and  use 2013-2018 as IS and export 1986-20121231  and use that as  OS  in additional markets. On my 2013 corei7 i (4 cores/8 threads) get 400000 strategies per hour on doing this retesting on 2 additional markets.

Why have this workflow. Well 99,999% of all strategies get rejected at IS does not matter if it is a longer or shorter period it is generated on so it is better to move the OS out of calculation why spend cpu testing on OS data when it already failed the IS.

the SQX build 106 shows the problem of slower with the CPU usage back to 70-80% only again,  which was fixed in RC5.  Mark  can you confirm and fix it in next release?

Mark,

also , May i ask for  each version update , did you also make improvement on the Wizard? the full mechanical generation still  get no nice strategy for me, my bet still relay on my own strategy plus some  variations  using the strategy builder. thank you.