Portfolio optimiser
31 replies
mabi
7 years ago #115657
I seems Quantanalyser craches on out of ram when closing in on 9 gig of ram usage when using the portfolio analyser. Is there a fix for this as with SQ to give it more ram ?
Thanks
Tamas
7 years ago #140059
mikeyc
7 years ago #140060
Hi,
On my machine, the portfolio optimizer uses at most 35% CPU, seems to be hitting one core more than the others. Looks like the multi-threading aspect is very poor?
Any thoughts?
Regards,
Mike
mikeyc
7 years ago #140070
Hi SQ Team,
New version doesn’t work for me. 290 strategies (STR files loaded), want exactly 10 in portfolio. Brute force selection. Correlation number of closed positions by hour, allow negative correlation, max correlation 0.4
Runs for a few minutes and then just doesn’t write anything more into log window, doesn’t respond to any button clicks, but continues to use 35% cpu and 3.7Gb Ram.
Have to kill the process.
Very disappointing.
mabi
7 years ago #140075
Why by hour ? That is really slow i agree a. Cpu usage on my server is low aswell do not know what it is doing and it takes along time to end the process. I have loaded 650 str. Setting by day seems fine thought even thought the Cpu usage are around 10 % which is about 4 cores .
mabi
7 years ago #140083
@ Mickey ,
You can check the hour correlation later. I find that if i run per day and setting 0,1 and start to build portfolios usung say 3 to 5 in each it finds alot and then i restart and go to 5-7 if it finds 0,1 there aswell increase to 7- 9. Then it gets harder but they will come eventually if you have buildt correctly. I then pick the best ones ( portfolios) say 20 and merge those to a singel strategy and add it to the rest. Now i run again from the start 3-5 , etc. So now i will have portfolios that are at 0,01-0,02 (even 1 hour) with up to 15 members so far. I also noticed that the CPU usage increases by time. It is now up to 50% which means 20 cores or 40 threads. !!
Tamas
7 years ago #140097
mabi
7 years ago #140099
@Tamas,
Can we have merging portfolios to singel strategies made so that we do not need to do 1 by one but many at the time ?
Thanks
Tamas
7 years ago #140103
Hello,
no it is not possible to merge portfolios to single strategies in the current version of QA.
We will consider to add this feature in the future.
Sincerely,
Tomas
mabi
7 years ago #140138
It is for sure possible already today but you can only do it one at the time. Manually clicking 4000 times seems to be an impossible task . To be able to join smaller portfolios with lower correlation to each other to find low stagnation must be an obvious benefit. Playing with it i manage almost ot bring down stagnation to 90 days over 10 years still keeping correlation below 0,1 and i bet you can get stagnation down much lower then that, It is possible that Genetic evolution can find this by it self maybe some time next year since combinations are closing in to infinity if you have many singelstrategies 🙁
mabi
7 years ago #140165
When a portfoilo is merged to a simple strategy the information if what strategies it contains is lost. Am i missing something here. Seems unessecary to be able to merge it if you do not know which strategies it contains. if you only have one or 2 i guess it can be okey but what if you have 100 ?
mabi
7 years ago #140184
Just some info,
QA gives up after computing 125 Mio portfolios if using 5-10 as max. When doing 10-20 it dies at 25 mio. Luckely it does not hang and are still responsive just just quit doing anything.I ran 4 QA sessions result are the same on all 4.
mikeyc
7 years ago #140330
Yes, correlations are computed only for unique combinations of strategies.
Tomas
Hi Tomas,
I think you might be missing the point on cutting down on computation.
If I have 100 single strategies, and want to make a porfolio with 10 strategies with no two strategies having a correlation > 0.5
First iteration selected by QA4:
1,2,3,4,5,6,7,8,9,10
Strategy pair 1,3 and 7,8 have correlation > 0.5
Therefore this portfolio is rejected, and there is no point ever including 1,3 or 7,8 again together in any future portfolio.
However, with Brute Force, QA4 will now select:
1,2,3,4,5,6,7,8,9,11
as the next possible portfolio to compute, but there’s no point, as we already know 1,3, and 7,8 fail the correlation test. So this portfolio can be rejected without any computation or testing since it contains at least one failing pair of strategies.
QA4 should store each pair found in memory that fail the correlation test as it goes, and remove from the list of all possible combinations those that include any of these pairs without testing.
This way the number of combinations to test falls at greater and greater speed as portfolios are computed.
mikeyc
7 years ago #140587
Or to look at it another way, since no one cares the current Genetic / Brute Force portfolio builder is hopelessly useless…
At the moment I am trying to construct a portfolio that has minimal stagnation.
There are 1549 simple strategies to choose from. A quick calculation shows that the number of unique combinations of strategy pairs is 1,198,926.
So, first run through the 1,198,926 pairs, calculate the correlation of each pair
Remove from the list of simple strategies all those that are not in the list of pairs that pass the correlation test. Depending on the correlation settings and treshold, the remaining pool of simple strategies will be somewhere between 0 and 1549. If the remaining strategies is less than the min # of strategies to form a portfolio, then no portfolio exists.
If there are at least the minimum # strategies to meet the min portfolio size, create the portfolios in a brute force manner, there’s no need to measure the correlation, any portfolio now created will have a correlation below the threshold. Just calculate the fitness (stagnation in my case) and sort accordingly.
That is the optimal way to construct the portfolios.
Mark Fric
7 years ago #140609
mike, I understand your point. This is a good idea how to optimize portfolios creation.
But why do you say current portfolio builder is useless?
Even without this optimization it works, it just takes more time.
Mark
StrategyQuant architect
mikeyc
7 years ago #140610
mike, I understand your point. This is a good idea how to optimize portfolios creation.
But why do you say current portfolio builder is useless?
Even without this optimization it works, it just takes more time.
It does, but expected end time 100’s of years.