Multiple genetic evolution running at the same time

Hello,

 

I’m using SQ to find stocks strategies and I would like to run multiple genetic evolution, one for each stock at the same time, is it possible?

 

For example, let my computer running a genetic evolution for Apple, and another one for Intel, another one for Cisco, and so on.

 

To find the best strategy for each stock independently and at the same time.

 

Regards

Yes after you install sq , copy paste the installation folder (duplicate) and run 2 instances of the program . each instance change the data settings , one apple and the second cisco and more instances and more data as your pc can handle .

Yes, exactly like this. Personally I run 22 instances on one server with 64GB RAM 🙂

lol geek whats your cpu ? dual xeon **** number ?

6-core 4930k @ 4.5GHz. Already 12 instances would load it up fully, but running 22, I don´t have to restart the genetic optimization all few hours as 22 instances keep it busy for 17 hours:)

@geektrader,

I have the same 4930K but my cpu has only 3.8 GHZ not more.

 

Question1:

Do you overclock your system and use WaterCooling ?

 

 

If I run more than 2 instances of the SQ at the same machine the SQ´s crashes after a while.

 

Question2:

do some SQ crashes on the system or freezes ?

 

Question3:

What settings do you use to start the SQ?

Hi 🙂

 

1) Yes I do overclock it to 4.5Ghz and use air-cooling with a fat 1200g CPU cooler on it. Temperatures are absolutely fine below 80°C (TJ Max of that CPU is 95°C).

 

2) No, not even the 22 SQ instances crash here, not any of them ever did. You do copy them to different folders, right? And you do have enough HD space and RAM too? Otherwise it won´t work. I´ve created a batch file to automate this.

 

3) Launch-line is “StrategyQuant64.exe -J-server -J-Xmx2600m -J-XX:+DisableExplicitGC -J-XX:+AggressiveOpts”  I have 64GB of RAM, hence 22 x 2,6GB = 57,2GB max RAM usage of all instances. You always have to take care so that all instances will never take more RAM than you have in total of course. I ONLY use these 22 instances to run genetic optimization on 14 years of M30 data in tick simulation mode. For that, 2,6GB per instance are enough. For strategy generating on M30 14 years of data, SQ needs at least 5000MB RAM per instance when set to use just 1 CPU core in the settings dialog – otherwise RAM usage goes up heavy per instance if you set it to more cores. So in this case I am only using 12 instances (which maxes out the CPU too as the 4930k has 6 cores + 6 HT) and if each instance uses the full 5GB, my total RAM usage is at 60GB. However, 12 instances on that CPU are the only way to leave no resources unused since SQ, at least when generating strategies, barely uses more than 1,5 cores on average even when it´s set to use 12 cores. So I am simply launching those 12 instances, all set to use only 1 CPU core, and can max out the computer fully during strategy generation too instead of wasting my time because SQ is so ineffective with multi-threading when generating strategies. I seriously hope that gets a lot more efficient in SQ 4…

 

How I figured out these values? All by trial and error… for example for generating stratiges on M15, RAM usage will be different. So you have to experiment….

 

Good luck to you:)

I never have had an SQ crash, not on my old AMD custom build, or on my new dell poweredge using intel xeons. I been running 4 simultaneous instances of SQ on the poweredge for the last week with around 20million+ random gens, no problems.
I have another PowerEdge now and I plan to do the same thing. Each has 8 threads, ~ 3.5GHZ, 32GB ram. Running zulu performance mod.

Geektrader nice setup and everything you explained regarding cpu and HT is perfectly correct . 

 

tnickel If the application crushes with 2 instances , These are the following things that you must think off  .

Illiminate each point by point until u fix the error . or let a pro check it to know .. 

 

1- Check your windows version and reset your windows to factory reset ( CLEAN WINDOWS )  … This is the first thing that you can easily do as elimination

2- After a clean installation open “Task manager and keep it open check cpu usage when idle , ram usage and disk usage ” just idle

3- Begin to load first instance and check the consumption ( and paste here what you see ) 

 

If these 3 failed i would suggest strongly to check the RAM as hardware , they might cause problems specially with financial application and pure math , Thats why companies used to go for xeon motherboards as it support ECC rams that dont do mistakes with calculation and so on . But i think intel i7 with normal rams and motherboard are pretty ok . 

Motherboard can affect the ram too . but u ll go for checking the ram first .

 

I Beleive Geektrader have a nice Motherboard probably a X99 deluxe or a rampage which is helping him with stability . 

hi gentmat,

this happens on more then two instances.

If I start only two instances all works fine.

 

The crash happens on different machines.

AMD Bulldozer(16GB), I7-4790(32GB), I7-3930(64GB), I7-3770(32GB)

 

I think this is not a hardware defect

I will do a ram stresstest and look.

 

The cooling of the CPU is good, I checked this.

 

I thing there is a bug in the SQ 3.8.1. The SQ has a memory leak and the memory leak causes the freeze.

 

I will check this settings

“StrategyQuant64.exe -J-server -J-Xmx2600m -J-XX:+DisableExplicitGC -J-XX:+AggressiveOpts”

 

Hm… it is possible that XEON is more stable ?

 

thomas

Then its not hardware .
1- Change orancle java to Zulu , replace j64 with zulu (delete inside folder and replace with zulu , it barely leaks).
2- Put windown 2008 or 2012 .
3- Xeon is not the thing affect stability what i told u is the motherboards used for xeons accept ECC rams , thats why companies used xeons even for little financial apps. but after 2014 cpu architecture changed now u can have normal motherboard for i7 that accept such rams (lower freq than gamining consumer rams but more stable ) Btw thats not my opinion thats a fact.
4- go to options and manually choose the thread number of each instance so if u have 8 cores , 16 threads and u have 2 opened instances then assign 4 threads for each instance .

Strange, at least I can tell you that I don´t have any crashes nor memory leaks in 3.8.1. I´ve just finished another run of 22 instances on machine #1 and 7 instances on machine #2, all good.

Hi geektrader,

22 * 8 threads = 176 Thread on machine #1

and

7*8 threads=56 threads on machine #2

 

or

 

22*1=22 threads on machine #1

and 7 threads on machine #1

 

???

 

 

With zulu or oracle java ?

 

thomas

Actually 22 x 12 threads on machine #1 and 7 x 8 on machine #2, yes. Using Zulu.