Reply

/temp directory filling up HD

4 replies

geektrader

Customer, bbp_participant, community, 542 replies.

Visit profile

6 years ago #113037

Hi,

 

I already informed Mark about this a while ago, but wanted to make it a official “bug”. When building on a large amount of data (7 pairs, 5M timeframe in my case), the /temp/orders folder in the StrateqyQuant directory gets filled up quickly. After a 40 minutes run, I am getting this:
 

 

…………………….

09.01.2015  09:06         5.444.547 ffc6b0e3-12b4-47ee-ae4e-51379f4c6825.ord
09.01.2015  08:52           454.749 ffe2575a-8e70-4fbd-af59-95177b115782.ord
09.01.2015  09:10         8.700.000 fff31b96-c89f-4066-bac1-85635f15976f.ord
09.01.2015  08:47           562.368 fff8f412-4af6-4392-aba9-c0df1b7a50fb.ord
            3444 File(s) 14.389.177.863 bytes
 
14GB of data in that directory. I´d suggest to use a more save-spacing methods to store these informations. Would that be possible? Otherwise I´ll need a 3TB drive after a few days of running 🙂
 
Thank you:)

0

tnickel

Customer, bbp_participant, community, sq-ultimate, 457 replies.

Visit profile

6 years ago #128093

hi geektrader,

very interesting.

 

How much Strategies are in the database?

All 3444 Strategies ? or less.

 

I think the tmp-Foulder is very importend if the SQ crashes, he need to restore his strategies

 

thomas

https://monitortool.jimdofree.com/

0

geektrader

Customer, bbp_participant, community, 542 replies.

Visit profile

6 years ago #128098

Now it looks like this:

 

……..

09.01.2015  13:03         8.700.000 ffd7f027-c897-487c-ab79-ce79b2ed250f.ord
09.01.2015  13:11         3.940.578 ffdec21b-a3ca-4916-865c-b52aab456e2e.ord
09.01.2015  13:41         7.378.818 ffded889-bce0-4680-9acd-a7922f10b4f8.ord
09.01.2015  13:59         1.586.271 fff36079-73b7-4965-8917-6fafe6f4f9cd.ord
09.01.2015  13:47           286.404 fff49a73-cb14-4ab0-bb0e-2d7c1f43dd0a.ord
09.01.2015  14:51           734.019 fffd8eee-00c3-49da-bf0d-bf77300ec826.ord
            9043 File(s) 20.270.884.203 bytes
 
It seems to save a file for every generated strategy that has a positive profit, not just the ones in the database – not sure though. You see, now it´s 9043 files with 20GB and unfortunately my SQ has locked up after I returned to the computer.
 
Memory usage of the task is at 8GB (out of 12gb I allowed via commandline) and the window is just “grey” in the thumbnail-preview of the taskbar now. Also the CPU usage is at 0%. I can´t seem to to re-activate it. To bad 🙁 Seems like there are some bugs that need to be fixed in that relation.
 
Yes sure it needs to restore strategies from there. I am just asking to make these files a lower size, maybe GZIP them instead to save them in such a big format.

0

Mark Fric

Administrator, sq-ultimate, 3 replies.

Visit profile

6 years ago #128140

yes, we use temp folder to store strategies in disc in case of crashes and also to save operating  memory.

 

But the strategy format is not really size-efficient now, we’ll make it much more efficient in the new version, so it should solve this problem.

Mark
StrategyQuant architect

0

geektrader

Customer, bbp_participant, community, 542 replies.

Visit profile

6 years ago #128145

Perfect, thanks 🙂

0

Viewing 5 posts - 1 through 5 (of 5 total)