Hefty RAM usage (memory management bug?)

Hi,

 

with the attached settings file which is just using price patterns and building on EURUSD M30 and additionally GBPUSD M30 (both from 2001 to 2014 without OOS) with “Selected timeframe only”-mode, I am getting HEFTY RAM-usage of up to 11GB (my upper limit that I´ve set for StrategyQuant via the commandline as otherwise my RAM would went full). The hefty usage starts out of sudden, not sure exactly why, usually it´s after it has done a few genetic generations it jumps up hefty while I am doing nothing special and just watch SQ.

 

This seems to MAINLY happen when building strategies by just using price patterns. When throwing all other indicators into the mix, it isn´t going that heavy so quickly, but still after a few hours too.

 

I am not sure if this a memory management leak within StrateqyQuant, as I can´t really imagine why the RAM-usage would jump up so heavy out of sudden while SQ is not really doing any new special tasks and RAM-usage was about 6GB all the time before when it did the same thing and my database size being limited to already 100 only.

 

In any case, once max RAM usage is reached, SQ becomes slow and takes ages to generate new strategies and the interface gets extremely sluggish. So basically going unuseable at that point.

 

Can there something be done to improve SQ`s memory footprint? Am I doing something wrong? Because I would imagine the same problem would exist for other people as well?

 

Thank you!

I see, but still, there seems to be some memory leak. I mean I should be able to use all available cores without the RAM usage going crazy like that.

 

Population size can never be high enough for diversity. From previous experiences with genetic algos I am always using sizes beyond 1000 for best results. More is always better there if you have the time 🙂 Generations are set to 4000 but are usually stopped anyway after a fitness stagnation for 5 generations, so this value is just theoretically.

we will look into this, we are completely rebuilding StrategyQuant platform for the new version, and we are paying special attention to performance and possible memory problems.

 

So hopefully it will behave much better in the new version.

Very great to hear this, really looking forward to SQ 4 ! 🙂

FYI I noticed that the memory leaks occur when stopping then restarting strategy generation, via the stop/start button if I start and from scratch it just runs and runs, but if the stop button is pushed then it starts occurring.

Yes, exactly. Whenever you finished any kind of operation that has been running for a while (e.g. strategy generating) and then start something else, it just “adds on top” of the memory. Like if the memory from the previous process that you did run is never released again. That´s why I restart SQ after every “big task”.

 

I also found that whenever population size > 250, memory usage raises HEAVILY. If I just use a population size of 250, all is fine, but as soon as I start to use 300 or more, the memory usage almost doubles up. Very “funny” behavior.

 

I have partly addressed this with my SQ tuning tutorial which I´ve posted here in the Forum, but I really hope it gets improved with SQ 4 though.

yes, it will be improved in the new SQ4, it should use less memory and be faster in general