Portfolio optimiser

31 replies

mabi

Customer, bbp_participant, community, 261 replies.

Visit profile

7 years ago #115657

I seems Quantanalyser craches on out of ram when closing in on 9 gig of ram usage when using the portfolio analyser. Is there a fix for this as with SQ to give it more ram ?

 

Thanks

0

Tamas

Customer, bbp_participant, community, sq-ultimate, 73 replies.

Visit profile

7 years ago #140059

Yes, correlations are computed only for unique combinations of strategies.

 
Tomas

0

mikeyc

Customer, bbp_participant, community, 877 replies.

Visit profile

7 years ago #140060

Hi,

 

On my machine, the portfolio optimizer uses at most 35% CPU, seems to be hitting one core more than the others.  Looks like the multi-threading aspect is very poor?

 

Any thoughts?

 

Regards,

 

Mike

0

mikeyc

Customer, bbp_participant, community, 877 replies.

Visit profile

7 years ago #140070

Hi SQ Team,

 

New version doesn’t work for me.  290 strategies (STR files loaded), want exactly 10 in portfolio. Brute force selection.  Correlation number of closed positions by hour, allow negative correlation, max correlation 0.4

 

Runs for a few minutes and then just doesn’t write anything more into log window, doesn’t respond to any button clicks, but continues to use 35% cpu and 3.7Gb Ram.

 

Have to kill the process.

 

Very disappointing.

0

mabi

Customer, bbp_participant, community, 261 replies.

Visit profile

7 years ago #140075

Why by hour ? That is really slow i agree a. Cpu usage on my server is low aswell do not know what it is doing and it takes along time to end the process. I have loaded 650 str. Setting by day seems fine thought even thought the Cpu usage are around 10 % which is about 4 cores .

0

mabi

Customer, bbp_participant, community, 261 replies.

Visit profile

7 years ago #140083

@ Mickey ,

 

You can check the hour correlation later. I find that if i run per day and setting 0,1 and start to build portfolios usung  say 3 to 5 in each it finds alot and then i restart and go to 5-7 if it finds 0,1 there aswell increase to 7- 9. Then it gets harder but they will come eventually if you have buildt correctly. I then pick the best ones ( portfolios) say 20 and merge those to a singel strategy and add it to the rest. Now i run again from the start 3-5 , etc. So now i will have portfolios that are at 0,01-0,02 (even 1 hour) with up to 15 members so far. I also noticed that the CPU usage increases by time. It is now up to 50% which means 20 cores or 40 threads. !!

0

Tamas

Customer, bbp_participant, community, sq-ultimate, 73 replies.

Visit profile

7 years ago #140097

Hello, 
 
does anybody else have a problem with Optimiser?
 
@ Mickey could you please send me your logs?
 
What about the memory consumption and OutOfMemoryError? 
 
Sincerely,
Tomas

0

mabi

Customer, bbp_participant, community, 261 replies.

Visit profile

7 years ago #140099

@Tamas,

 

Can we have merging portfolios to singel strategies made so that we do not need to do 1 by one but many at the time ?

 

Thanks

0

Tamas

Customer, bbp_participant, community, sq-ultimate, 73 replies.

Visit profile

7 years ago #140103

Hello,

no it is not possible to merge portfolios to single strategies in the current version of QA.

 

We will consider to add this feature in the future.

 

Sincerely,

Tomas 

0

mabi

Customer, bbp_participant, community, 261 replies.

Visit profile

7 years ago #140138

@Tomas

 

It is for sure possible already today but you can only do it one at the time. Manually clicking 4000 times seems to be an impossible task . To be able to join smaller portfolios with lower correlation to each other to find low stagnation must be an obvious benefit. Playing with it i manage almost ot bring down stagnation to 90 days over 10 years still keeping correlation below 0,1 and i bet you can get stagnation down much lower then that, It is possible that Genetic evolution can find this by it self maybe some time next year since combinations are closing in to infinity if you have many singelstrategies   🙁

0

mabi

Customer, bbp_participant, community, 261 replies.

Visit profile

7 years ago #140165

When a portfoilo is merged to a simple strategy the information if what strategies it contains is lost. Am i missing something here. Seems unessecary to be able to merge it  if you do not know which strategies it contains. if you only have one or 2 i guess it can be okey but what if you have 100 ?

0

mabi

Customer, bbp_participant, community, 261 replies.

Visit profile

7 years ago #140184

Just some info,

 

QA gives up after computing 125 Mio portfolios if using 5-10 as max. When doing 10-20 it dies at 25 mio. Luckely it does not hang and are still responsive just just quit doing anything.I ran 4  QA sessions result are the same on all 4.

0

mikeyc

Customer, bbp_participant, community, 877 replies.

Visit profile

7 years ago #140330

 

Yes, correlations are computed only for unique combinations of strategies.

 
Tomas

 

 

Hi Tomas,

 

I think you might be missing the point on cutting down on computation.

 

If I have 100 single strategies, and want to make a porfolio with 10 strategies with no two strategies having a correlation > 0.5

 

First iteration selected by QA4:

 

1,2,3,4,5,6,7,8,9,10

 

Strategy pair 1,3 and 7,8 have correlation > 0.5

 

Therefore this portfolio is rejected, and there is no point ever including 1,3 or 7,8 again together in any future portfolio.

 

However, with Brute Force, QA4 will now select:

 

1,2,3,4,5,6,7,8,9,11

 

as the next possible portfolio to compute, but there’s no point, as we already know 1,3, and 7,8 fail the correlation test.  So this portfolio can be rejected without any computation or testing since it contains at least one failing pair of strategies.

 

 

QA4 should store each pair found in memory that fail the correlation test as it goes, and remove from the list of all possible combinations those that include any of these pairs without testing.

 

 

 

This way the number of combinations to test falls at greater and greater speed as portfolios are computed.

0

mikeyc

Customer, bbp_participant, community, 877 replies.

Visit profile

7 years ago #140587

Or to look at it another way, since no one cares the current Genetic / Brute Force portfolio builder is hopelessly useless…

 

At the moment I am trying to construct a portfolio that has minimal stagnation.

 

There are 1549 simple strategies to choose from.  A quick calculation shows that the number of unique combinations of strategy pairs is 1,198,926.

 

So, first run through the 1,198,926 pairs, calculate the correlation of each pair

 

Remove from the list of simple strategies all those that are not in the list of pairs that pass the correlation test.  Depending on the correlation settings and treshold, the remaining pool of simple strategies will be somewhere between 0 and 1549.  If the remaining strategies is less than the min # of strategies to form a portfolio, then no portfolio exists.

 

If there are at least the minimum # strategies to meet the min portfolio size, create the portfolios in a brute force manner, there’s no need to measure the correlation, any portfolio now created will have a correlation below the threshold.  Just calculate the fitness (stagnation in my case) and sort accordingly.

 

That is the optimal way to construct the portfolios.

0

Mark Fric

Administrator, sq-ultimate, 2 replies.

Visit profile

7 years ago #140609

mike, I understand your point. This is a good idea how to optimize portfolios creation.

 

 

But why do you say current portfolio builder is useless?

Even without this optimization it works, it just takes more time.

Mark
StrategyQuant architect

0

mikeyc

Customer, bbp_participant, community, 877 replies.

Visit profile

7 years ago #140610

mike, I understand your point. This is a good idea how to optimize portfolios creation.

 

 

But why do you say current portfolio builder is useless?

Even without this optimization it works, it just takes more time.

 

 

It does, but expected end time 100’s of years.

0

Viewing 15 replies - 16 through 30 (of 31 total)

1 2 3