Not logged in

Forums

Viewing 15 posts - 1 through 15 (of 95 total)

Forums>StrategyQuant (formerly named Genetic Builder)>General Discussion>[How To] Tune SQ for max performance and reduce memory usage by 4 to 5 times

  • #113468 |
    Customer
    602 Posts

    Updated 10. April 2016 to reflect the default garbage collector change in Java 9:

     

    Hi Guys,

     

    throughout the last year I have been fine-tuning SQ by the various Java (which SQ runs on) options that you can pass on to it during start-up. I am now satisfied with the performance and on the same side heavy reduction in memory usage that I´ve achieved through fine-tuning the Java options.

     

    Just use this line to launch SQ with max performance and as low as possible memory usage:

     

    StrategyQuant64.exe -J-server -J-Xmx13g -J-XX:+DisableExplicitGC -J-XX:+AggressiveOpts -J-XX:+UseSerialGC

     

                                                        ^^ please adjust the “-J-Xmx13g” to the memory you want to assign to SQ (and which you really have free!) as max allowed amount. E.g. if you want to allow it 6GB max, use “-J-Xmx6g” instead.

     

    Additionally to this launch-line, I´d also recommend to replace the JVM from Oracle to the one from Zulu, you will get even more speed and less RAM usage with this one as it is a straight build of the OpenJDK sources (the original Java) without Oracles branding and other things added to it. You can get it here: http://www.azulsystems.com/products/zulu/downloads . There also currently is a dedicated page for the new Java 9 beta which also works fine and is even faster than Java 8, but only up to version 9.0.0.2 since changes in Java 9 that have been adapted in later builds won´t work anymore with SQ 3.8.1, you can download it here: http://cdn.azul.com/zulu-pre/bin/zulu9.0.0.2-ea-jdk9.0.0-win_x64.zip .

     

    Just install it and afterwards rename the directory “j64” in your StrategyQuant directory to “j64.old”. Create a new “j64” directory and copy the complete Zulu installation (usually from C:\Program Files\Java…..) into the newly created “j64” directory.

     

    Please compare on your own how SQ runs without any parameters and with my command line and the JVM replaced. You´ll see that when starting to build strategies that alone in the first 2 minutes of building RAM usage will be half with my command line than SQ without my command line. Then have fun building strategies even faster and with less RAM than before now and make some money hopefully:)

    #129336
    Customer
    312 Posts

    Hello GeekTrader,

     

    That a nice tips! Have You compared does it change results of testing/retesting/optimising strategies? 

     

    What I try to say is: Does Your changes apply on results of strategies ? 

    btn_viewmy_160x33.png

    #129337
    Customer
    602 Posts

    No, this doesn´t and can´t at all change any of the results for the operations SQ performs. This is purely related to performance tuning / memory tuning, the results which SQ generates in any of it´s operations, regardless whichever command line arguments you pass to SQ / Java, will never change. It´s just the speed and memory usage that can change with these options.

    #129366
    Administrator
    3071 Posts

    Hello Lorenz,

     

    thank you very much for this testing and optimizations 🙂

    I have never tested these advanced Java options, it seems they could have a big impact.

    I’ll test it and possibly include them as default settings in the future release.

    Mark
    StrategyQuant architect

    #129378
    Customer
    602 Posts

    Hi Mark,

     

    you are welcome and have a test on your own of course please. Just trying to get SQ to work as fast as possible with as low as possible memory usage.

     

    Honestly though, there still seems to be some memory leaking somewhere in SQ. Running the same task (strategy building) for half a day now in a loop and RAM usage is increasing steadily while the database of strategies it should store is still just 200 (which was filled up about 20 minutes after the start of building strategies). So in theory the RAM usage should not steadily increase, but it does, hour by hour and keeps on growing, until a point where RAM is simply full and SQ needs a restart and then this starts over again…. It´s strange too that SQ does not release the memory at all even after stopping the generating, it just sticks with that high usage and neither the “Perform Memory Cleanup” helps nor a Windows-based RAM optimizer which forces the process to releae unused memory. So that really must be a leak instead in SQ.

     

    But I know you said SQ4 got a complete redesign also in terms of memory handling, so it might be worthless right now to look into that for 3.8 if 4.0 will be changed in that aspect anyway.

    #129508
    Customer
    366 Posts

    Thank you.  I can now run the WF matrix without my PC freezing!

    #129521
    Customer
    21 Posts

    Hi guys!

     

    Where to put the code? I have no idea how to start the SQ with this code. Please help me. Thank you!

     

    StrategyQuant64.exe -J-server -J-xmx13g -J-XX:+AggressiveOpts -J-XX:+UseParallelGC -J-XX:+OptimizeStringConcat

    Automated Trading, PA & Renko systems, PC - i7-5820k, 480 GB SSD, 16 GB RAM 

    #129522
    Customer
    312 Posts

    Hello Deenius,

     

    Please find shortcut of SQ on Your desktop. Then click right mouse button and go into PROPERIES,

     

    There You will find full patch like c:\StrategyQuant\strategyquant64.exe After this make space and pase above formula.

    Here is example:

     

    nerwo4.png

    btn_viewmy_160x33.png

    #129523
    Customer
    21 Posts

    Thank you!

     

    But, I’m doing something wrong. I have 16GB RAM, but I still see only 6GB of nemory in my SQ. I don’t know why?

     

     

    http://postimg.org/image/rgpghuhlh/full/

    Automated Trading, PA & Renko systems, PC - i7-5820k, 480 GB SSD, 16 GB RAM 

    #129538
    Administrator
    3071 Posts

    you have to start SQ with parameters if you want it to use more memory, check this:

    https://strategyquant.com/support/starting-strategyquant-with-more-memory/

    Mark
    StrategyQuant architect

    #129554
    Customer
    21 Posts

    Now it works! 😀   Thank you very much Mark. 

    Automated Trading, PA & Renko systems, PC - i7-5820k, 480 GB SSD, 16 GB RAM 

    #129605
    Customer
    312 Posts

    StrategyQuant64.exe -J-server -J-xmx13g -J-XX:+AggressiveOpts -J-XX:+UseParallelGC -J-XX:+OptimizeStringConcat

     

    Strange, when I input those part what Mark posted in his article then memory limit goes up. But when I pase above formula then nothing is changing. Still around 5.5GB limit for memory. That makes me wondering does other parameters are working or not. 

    btn_viewmy_160x33.png

    #129670
    Customer
    602 Posts

    It doesn´t work because it has to be:

     

    -J-Xmx13g

     

    not

     

    -J-xmx13g

     

    Java command line options are case-sensitive.

    #129671
    Customer
    602 Posts

    P.S.: Please note I´ve updated the first post with a new SQ launch line based on my latest ongoing findings for optimum performance and lowest memory usage.

    #129782
    Customer
    734 Posts

    Thanks for this I’ll try it out. I’m also starting a similar thread but optimizing performance through strategy generate settings: removing the unnecessary.

Viewing 15 posts - 1 through 15 (of 95 total)

You must be logged in to reply to this topic.