Imported 1.5 Million hands.. REALLY slow..

[genemish]

clustering just started on it.. (7 mins ago)

[kraada]

*keeps his fingers crossed for you*

[genemish]

on 11th hour.. Updating Cache now.. so it's almost done but still took a looooooong time.

so how do I fix my clustering issue?

[genemish]

running housekeeping again, just to see how long it runs without any imports..

and oddly enough, the postgre.exe process is all over the place during the clustering phase.. which is never the case if I run housekeeping after import...

really confused now..

[WhiteRider]

Please let us know how long it took this time.
It's possible that because less rearranging of files on disk needs to be done the processor is working more intensively - hopefully this means it won't take as long?

[genemish]

it tooks only 3h:03m.. on a DB that has ~3.2MM hands.. that's really good news.

so the problem is narrowed down to clustering during import.. it's taking long and postgre.exe is not really jumping in CPU usage..

[kraada]

Have we confirmed that the slow clustering time is there for a second cluster as well? The first time cluster is run on a database it is always going to take longer than subsequent clusters.

[genemish]

Just to make sure it's clear..

DB1 (main) - import #1 (1.5MM hands) .. housekeeping took 14hours... import #2 (additional 1.7MM hands) .. housekeeping took 16hours and postgre.exe process was quiet during both clustering phases..

DB2 (test) - import #1 (1.5MM hands) .. housekeeping took 13.5 hours... postgre.exe process was quiet during clustering phase

last night i re-ran housekeeping on DB1 (3.2MM Hands in it) w/o any imports - total housekeeping took 3hours.. postgre.exe process was using 20-30% of CPU during clustering and other phases..

so the issue is with clustering process IF I import hands..

[kraada]

Are you behind a domain? Is there anything else odd about your networking setup?

Are you using AVG's Resident Shield feature? If so, try setting the PostgreSQL directory to be excluded from that function and that might speed things up, depending on how exactly AVG is interacting here.

* open AVG -> Tools -> Advanced settings
* in the left pane select Resident Shield -> Exceptions
* use the "Add Path" button to choose the directory to be excluded
* confirm the changes using the OK button

[genemish]

i am not behind a domain.. it's a regular setup..

i will check AVG settings..

but how come clustering works well during "non-import" housekeeping and drags after the import?