Reply

Optimizing using real tick data slow and 100% disk activity

7 replies

mikeyc

Customer, bbp_participant, community, 878 replies.

Visit profile

5 years ago #114853

Hi,

 

I’m using real tick data in SQ, and noticed that during optimisation, the disk is pretty much at 100% activity the whole time.  Now it’s an SSD disk, so fast, but optimisation takes a very long time.

 

I need real tick because I need real spread, not fixed spread.

 

Is there any way of making SQ use RAM rather than the disk with real tick data? Will SQ use RAM if there is enough?

 

Thanks,

 

Mike

 

PS. Same 100% disk activity applies to building strategies using tick data rather than tick simulation.

0

Karish

Customer, bbp_participant, community, 445 replies.

Visit profile

5 years ago #135711

I got a solution for you: how about adding a Spread Limiter that you would set your maximum spread and if the spread reaches this maximum spread value your order wont be entered,

that way you can use fixed spread when testing/building, that’s what i do..

0

mikeyc

Customer, bbp_participant, community, 878 replies.

Visit profile

5 years ago #135712

I got a solution for you: how about adding a Spread Limiter that you would set your maximum spread and if the spread reaches this maximum spread value your order wont be entered,

that way you can use fixed spread when testing/building, that’s what i do..

 

That is not a usuable solution.

 

Say you use tick simulation with fixed spread 1 pip.  SQ will have lots of trades where spread > 1 pip in the real tick data, giving what looks like a very profitable strategy.

 

A spread filter in the MQ4 will mean all those trades will not be placed in real trading.  Completely different trading results.

 

The solution is somehow for SQ to use more RAM and not the disk.

0

Mark Fric

Administrator, sq-ultimate, 3 replies.

Visit profile

5 years ago #135726

unfortunately it is not possible to switch from disc to memory by changing some settings.

When you use real tick data then they are never loaded into memory for backtesting, because of their size, they are always loaded from disc for every test.

 

But I agree that if you have enough memory it should be served from memory, but it is something we’ll be able to make only in the new version.

Mark
StrategyQuant architect

0

Threshold

Customer, bbp_participant, community, 715 replies.

Visit profile

5 years ago #135732

Surprised SSD isn’t running it fast enough.

Have you done an optimization on 1minute data and then another on tick data and checked if there is even any difference if you find the mean spread and go slightly wider than it? Opt 1m data then retest on tick? Worth a shot. Same with generating.

I make these sacrifices on MT4 because it literally takes 1 month to optimize 1 EA Wizard strategy sometimes longer. So on a D1 system I’ll delete all the 1 min data, sometimes even the 5 min data so its only optimizing from m5 or m15, after its done I go back and retest on m1. There’s been 0 difference in the results but the strategies are not spread or slippage sensitive. Still, my point on comparison might help you save a lot of time.

0

mikeyc

Customer, bbp_participant, community, 878 replies.

Visit profile

5 years ago #135806

unfortunately it is not possible to switch from disc to memory by changing some settings.

When you use real tick data then they are never loaded into memory for backtesting, because of their size, they are always loaded from disc for every test.

 

But I agree that if you have enough memory it should be served from memory, but it is something we’ll be able to make only in the new version.

 

Hi Mark,

 

If I load a symbol with real tick data, but test using M5 bars with tick simulation, does SQ use RAM only in this case?

 

Because sometimes I like to test with fixed spread quicky, and other times with real tick and real spread, but don’t want to have to load the data twice (once with real tick data and another symbol with M1 bar data).  If I can just load with tick data, use bar timeframes with tick simulation and fixed spread and only use RAM that would be great.

 

Regards,

 

Mike

0

mabi

Customer, bbp_participant, community, 262 replies.

Visit profile

5 years ago #135809

I downloaded and installed ramdisk on 5 gig of my ram. It works fine creates a virtual disk in your ram. Copying from 3 disk simultaneously i got up to 1000mb/sek . It is free ! Download in bottom of page.

 

https://www.softperfect.com/products/ramdisk/

0

Karish

Customer, bbp_participant, community, 445 replies.

Visit profile

5 years ago #135812

I downloaded and installed ramdisk on 5 gig of my ram. It works fine creates a virtual disk in your ram. Copying from 3 disk simultaneously i got up to 1000mb/sek . It is free ! Download in bottom of page.

 

https://www.softperfect.com/products/ramdisk/

This is awesome!, thanks buddy!

0

Viewing 8 posts - 1 through 8 (of 8 total)