Page 1 of 2

Slow Importing Speed

PostPosted: Wed Sep 16, 2015 2:09 am
by TennisDKT
Hello,

I am importing around 500k hands, the first 300k went at around 160h/s and then the speed dropped to 10h/s. I stopped the import, restarted and then started the import from where the previous import finished and it is still going at around 7h/s at the moment.

Any ideas?

Thanks

Re: Slow Importing Speed

PostPosted: Wed Sep 16, 2015 2:42 am
by WhiteRider
We're aware that import speeds drop over large imports and until we can address this we recommend splitting your larger imports into batches. We understand that this is inconvenient, but fortunately these large imports are pretty rare so hopefully it isn't too much of a problem in the long run.

Re: Slow Importing Speed

PostPosted: Wed Sep 16, 2015 2:47 am
by TennisDKT
I have done this.. I have put 5-10 files into directories and it still is slow.

Re: Slow Importing Speed

PostPosted: Wed Sep 16, 2015 2:53 am
by WhiteRider
In that case please enable logging (Configure > Options) and restart PT4. Reproduce the problem then attach the PokerTracker4.log file from C:\Users\yourname\AppData\Local\PokerTracker 4\ (which you can access from the PT4 File menu) to a Support Ticket along with the hand history files you imported so that we can see what is going on.

Re: Slow Importing Speed

PostPosted: Wed Sep 16, 2015 2:58 am
by TennisDKT
Done.

Re: Slow Importing Speed

PostPosted: Wed Sep 16, 2015 4:06 pm
by TennisDKT
2015/09/17 08:03:49.038 [27118]: [B0831000][StatManager] Warning: Cache Update Took: 36.62s
2015/09/17 08:04:23.592 [34554]: [B0831000][StatManager] Warning: Cache Update Took: 34.55s
2015/09/17 08:04:23.641 [00049]: [B0831000][DatabaseVariables] Stopping Thread
2015/09/17 08:04:23.641 [00000]: [B072D000][DatabaseVariables::Entry] Thread End: 0 queries remain.


This is what I see in the log, would this be causing my hands to import at 5 h/s ?

I haven't heard back from support and importing hands it literally taking forever.

Thanks

Re: Slow Importing Speed

PostPosted: Wed Sep 16, 2015 4:15 pm
by kraada
Yes, that's an extremely long time to update the cache. Have you tried doing a REINDEX? If not try that first that might fix this entirely. If not you may need to rebuild your cache (which takes considerably longer). What version of PostgreSQL are you running?

Re: Slow Importing Speed

PostPosted: Wed Sep 16, 2015 4:30 pm
by TennisDKT
Thank you for the fast reply.

I am trying a REINDEX now...

PostgreSQL v9.3.5

Re: Slow Importing Speed

PostPosted: Wed Sep 16, 2015 4:45 pm
by TennisDKT
REINDEX fixed it. Now back to 150 h/s

Thank you

Re: Slow Importing Speed

PostPosted: Thu Sep 17, 2015 12:41 am
by TennisDKT
REINDEX fixes is periodically. I have to keep on doing REINDEX otherwise it comes to a crawl.

Do you recommend doing a REBUILD CACHE? Is that my only option?

Thanks