[ADMIN] CPU Load question / PgBouncer config

[Benjamin Krajmalnik]

I am experiencing a little higher CPU load than I would like to see, and
was wondering if it has to do with the number of connections (although
many are idle).

I am running PG 9.0.4/amd64 on FreeBSD 8.1., dual boxes running
streaming replication.

Hardware is a 16 core box with 96GB RAM, using 6GB of shared buffers.

RAID 1 for OS, 12 drive RAID 10 for data, RAID 1 for logs.

I am using pgbouncer as a connection pooler, with 200 max client
connections, a default pool size of 100, and a reserve of 80.

The number of backends running are between 40 and 70. I monitor the
number of non-idle connections. Average is about 20, with non-frequent
spikes going to about 40.

My gut feeling is that I can probably reduce the pool size somewhat.

Would reducing the number of backends relieve some of the load on the
server (I am seeing a 5 min load of 12 more often than I would like to)?

Also, for this size setup, what would be the recommended values for the
pool sizes? I am certain the values are way above what they should be,
but nbot certain.



I posted here instead of the pgbouncer group hoping someone has
experienced similar things.



Prresently, I have the following load,





last pid: 96231; load averages: 12.18, 11.07, 10.97
up 112+19:23:29 11:02:05



and this is the result from checking the state of pgbouncer:



pgbouncer=# SHOW POOLS;

database | user | cl_active | cl_waiting | sv_active | sv_idle |
sv_used | sv_tested | sv_login | maxwait

-----------+-----------+-----------+------------+-----------+---------+-
--------+-----------+----------+---------

ishield | xxxxxxx | 48 | 0 | 48 | 4 |
2 | 0 | 0 | 0

pgbouncer | pgbouncer | 1 | 0 | 0 | 0 |
0 | 0 | 0 | 0

(2 rows)





pgbouncer=# SHOW POOLS;

database | user | cl_active | cl_waiting | sv_active | sv_idle |
sv_used | sv_tested | sv_login | maxwait

-----------+-----------+-----------+------------+-----------+---------+-
--------+-----------+----------+---------

ishield | xxxxxxx | 55 | 0 | 55 | 0 |
2 | 0 | 0 | 0

pgbouncer | pgbouncer | 1 | 0 | 0 | 0 |
0 | 0 | 0 | 0

(2 rows)







Thanks in advance,



Benjamin

[k***@rice.edu]

Post by Benjamin Krajmalnik
I am experiencing a little higher CPU load than I would like to see, and
was wondering if it has to do with the number of connections (although
many are idle).
I am running PG 9.0.4/amd64 on FreeBSD 8.1., dual boxes running
streaming replication.
Hardware is a 16 core box with 96GB RAM, using 6GB of shared buffers.
RAID 1 for OS, 12 drive RAID 10 for data, RAID 1 for logs.
I am using pgbouncer as a connection pooler, with 200 max client
connections, a default pool size of 100, and a reserve of 80.
The number of backends running are between 40 and 70. I monitor the
number of non-idle connections. Average is about 20, with non-frequent
spikes going to about 40.
My gut feeling is that I can probably reduce the pool size somewhat.
Would reducing the number of backends relieve some of the load on the
server (I am seeing a 5 min load of 12 more often than I would like to)?
Also, for this size setup, what would be the recommended values for the
pool sizes? I am certain the values are way above what they should be,
but nbot certain.
I posted here instead of the pgbouncer group hoping someone has
experienced similar things.
Prresently, I have the following load,
last pid: 96231; load averages: 12.18, 11.07, 10.97
up 112+19:23:29 11:02:05
pgbouncer=# SHOW POOLS;
database | user | cl_active | cl_waiting | sv_active | sv_idle |
sv_used | sv_tested | sv_login | maxwait
-----------+-----------+-----------+------------+-----------+---------+-
--------+-----------+----------+---------
ishield | xxxxxxx | 48 | 0 | 48 | 4 |
2 | 0 | 0 | 0
pgbouncer | pgbouncer | 1 | 0 | 0 | 0 |
0 | 0 | 0 | 0
(2 rows)
pgbouncer=# SHOW POOLS;
database | user | cl_active | cl_waiting | sv_active | sv_idle |
sv_used | sv_tested | sv_login | maxwait
-----------+-----------+-----------+------------+-----------+---------+-
--------+-----------+----------+---------
ishield | xxxxxxx | 55 | 0 | 55 | 0 |
2 | 0 | 0 | 0
pgbouncer | pgbouncer | 1 | 0 | 0 | 0 |
0 | 0 | 0 | 0
(2 rows)
Thanks in advance,
Benjamin

Hi Benjamin,

For a 16-core box a load of 12 is only at 75% capacity, so it sounds like you
are doing just fine. It does seem like your connection pool is a bit large
for your hardware. A common recommendation is:

(number_of_cores * 2) + effective_spindle_count

or for your box: (16 * 2) + 6 = 38

It looks like you are about 2.5X that on your system.

Regards,
Ken