Page 1 of 6

Clustering taking too long

PostPosted: Tue Dec 01, 2009 9:14 am
by Vigilanty
Hi I recently tried to import about 5million hands and and errored out at about 2million hands (I didn't write down the error). I figured that was okay for now and went to do housekeeping and the clustering has been running for 14.5 hours now. this doesn't make any sense since when I previously did a 1 million hand import (same computer) I was able to complete all housekeeping in 2 hours so I was expecting it to run around 4 hours. What should I do? The hand import took forever do I really have to start from scratch?

Re: Clustering taking too long

PostPosted: Tue Dec 01, 2009 10:00 am
by kraada
How much free hard drive space is left on the drive the database uses?

Re: Clustering taking too long

PostPosted: Tue Dec 01, 2009 6:01 pm
by Vigilanty
I'm not at home right now to check exactly but over 200 gigs for sure. I'll double check this later tonight and lay down another reply... maybe it even finished housekeeping by now who knows.

Re: Clustering taking too long

PostPosted: Tue Dec 01, 2009 6:49 pm
by Vigilanty
ya ok so I was right. I have 408g free of 596g so like 75% free I should be fine in that regard... cluster been running for 24h now... do I stop and re-import? maybe do 1 mill hands at a time?

Re: Clustering taking too long

PostPosted: Tue Dec 01, 2009 7:14 pm
by kraada
What antivirus/antispyware software are you using? Are you still seeing data being written in your PostgreSQL/Data/Base directory? What version of PostgreSQL are you using?

Re: Clustering taking too long

PostPosted: Tue Dec 01, 2009 8:40 pm
by Vigilanty
Using symantec... I don't know if it was being written because I re-stared, 24h was ridiculous. Using version 8.3 of postgresql.

Re: Clustering taking too long

PostPosted: Wed Dec 02, 2009 5:24 am
by WhiteRider
Please read our Firewall Guide and make sure you have everything set up correctly - Symantec has issues.

Re: Clustering taking too long

PostPosted: Wed Dec 02, 2009 10:48 am
by Vigilanty
Okay I added the indicated exceptions. The data folder has gotten bigger since I last checked. It's at 17.7GB right now is that normal... and now 17.7GB so it's growing...that's a good sign? I'd rather not restart if I don't have to as indicated in the help guide. What I did was: ended housekeeping, deleted the database,created a new database, loaded a million hands, when I saw that it ran for 7 hours I saw your post whiterider (and thanks to both you and kraada, great timely support, love it) and added the firewall exceptions, so I guess leave it for now and see what happens? I'm off to class. Let me know if there's anything else I should check or if a restart was crucial.

Re: Clustering taking too long

PostPosted: Wed Dec 02, 2009 11:25 am
by kraada
Just leave it; if the folder is changing size that means PostgreSQL is continuing to cluster and you should let it finish.

Re: Clustering taking too long

PostPosted: Wed Dec 02, 2009 5:26 pm
by MrBarbegris
His point is still well taken, though. My db maintenance -- mainly updating the cache -- takes hours and hours as well, and I don't have nearly that many hands and I've got oodles of disk space too.