Reply

Optimal instances of SQ3 per core and scripts for managing them

21 replies

Threshold

Customer, bbp_participant, community, 723 replies.

Visit profile

7 years ago #115144

What is the optimal # of SQ3 instances you run per core?

I do 1 SQ instance per core and and its multithreaded so I do 2 threads per SQ on my old Dell Poweredges.

Are there programs or scripts out there for managing multiple instances of software? My new server has 32 cores, thats 32 instances of SQ3, maybe 16, I’ll probably be installing and running on it for optimal strategy generation speeds. Thats a lot of work changing their settings and everything if I want them all doing the exact same thing.

0

statistic

Subscriber, bbp_participant, community, 31 replies.

Visit profile

7 years ago #137399

Averaging about 80-85% usage. 32SQs 2 threads each. 64SQs with 1 thread each would have been too much work.

I hope Mark will fix it in SQ4, basicly what we doing is manual Parallelization, I think we are in 2016 when it is not common to see such OLD way of doing things.

Last century it would be fine, but not now. I know it is cool to see so many Apps at the same time, but each eats memory/CPU instead of using ONE instance which does Parallel testing, it is not that difficult. I was doing the same in 2005 with MT4 but not now 🙁

If we use this work around can we avoid using Save/Delete , as SQ3 will not handle it very well over 5K strategies.

I suggest we can create a “shared” folder for BIN (Windows will load it in memory as it is used too many times) and  “Databank” folders so SQ3 will save all strategies to the disk.

I wish SQ:DataBank would match “Temp”>Build Strategy>databank , in this case we do not need to bother to use SAVE option, but can automate the Daily Harvesting. Secondly the Strategies in DataBank folder are 78kb vs 1MB if I save from SQ DataBank, which makes huge difference if you check 50k strategies.

Also it would be good to change the naming convention to randomized 8 alphanumeric this way we can use common folder for all 64 SQ instances.

So Threshold, GeekTrader – I’m impressed what you do but I’m not impressed why we do it. (it should be avoided, as it is Software limitation) -> and this is not Cool 🙁

0

Threshold

Customer, bbp_participant, community, 723 replies.

Visit profile

7 years ago #137400

Agree. The good news is I now get about 3million random generations every ~12hours.
BTW I never deal with more than 200 Strategies. 200 elite/very high quality strategies. I found using smaller numbers of high-quality strategies loaded into genetic evolution improves speeds greatly. Each time Gen Evo loops back to ‘generation 1’ SQ will only need to reload 200 strategies instead of 5k or 10k or 20k which takes much less time ultimately after doing 2000 Genetic evolution ‘loops’. This is good if you use the infinite Genetic evo loop feature.
Pretty sure we’ll be impressed with SQ4 when its all said and done. Being able to have the EA Wizard freedom of manual strategy creation and using SQ’s testing and optimization on them makes just those 2 features all worth the wait for me, never mind all the others features coming.

0

Patrick

Customer, bbp_participant, community, 424 replies.

Visit profile

7 years ago #137747

can we compare i7 to xeon ?

 

On i7 4790k i have 4 cores/8threads so i can use it for 4 instance of SQ each with 2 threads, right?

 

On xeon E26xx there is 32 cores/64 threads so it can be 32 instances of SQ.

 

so its seems Xeon should be better, because its 8x more powerfull (32Sq on xeon vs 4SQ on i7) 

 

Also price for this xeon will be around 4000$ vs i7 for 1500$, so Xeons seems to be beter in price also.

 

Question is, in real conditions, is Xeon better than i7? Did anyone compare?

0

Threshold

Customer, bbp_participant, community, 723 replies.

Visit profile

7 years ago #137748

For ‘SQ4 you will want more cores.

0

clonex / Ivan Hudec

Customer, bbp_participant, community, sq-ultimate, contributor, author, editor, 271 replies.

Visit profile

7 years ago #137753

For ‘SQ4 you will want more cores.

This is why im waiting. Anyway huge thanks everybody here for posting his insights !

0

gentmat

Customer, bbp_participant, community, 234 replies.

Visit profile

7 years ago #137792

Threshold use that 

 

affinity Settings and dedicate each core to 1 sq instance / I would dedicate 2cores for 1 sq

1- Go to Windows Choose a SQ instace (Dedicate 2 cores)

2-Open this instance and limit it to 4 threads , or put to max as this application can not jump to other instances anymore)

3- Do that 6 times as you have 12 cores.

 

Step by Step Affinity Settings 

http://www.techrepublic.com/blog/windows-and-office/change-the-processor-affinity-setting-in-windows-7-to-gain-a-performance-edge/

0

Viewing 6 replies - 16 through 21 (of 21 total)

1 2