[ADMIN] Deadlock on "select ... for update"?

[Craig James]

Several times recently one of our databases has gotten stuck with the
following situation:

postgres=# select datname, procpid, usename, current_query from
pg_stat_activity where current_query != '<IDLE>';
datname | procpid | usename |
current_query
------------+---------+----------+--------------------------------------------------------------------------------------------------------
emolecules | 13503 | customerdb | select tableid from hitlist_table_pool
where hitlistid <= 0 for update
emolecules | 32082 | customerdb | select tableid from hitlist_table_pool
where hitlistid <= 0 for update
emolecules | 17974 | customerdb | select tableid from hitlist_table_pool
where hitlistid <= 0 for update
emolecules | 31299 | customerdb | select tableid from hitlist_table_pool
where hitlistid = 0 limit 1 for update
emolecules | 30247 | customerdb | select tableid from hitlist_table_pool
where hitlistid = 0 limit 1 for update
postgres | 1705 | postgres | select datname, procpid, usename,
current_query from pg_stat_activity where current_query != '<IDLE>';
emolecules | 28866 | customerdb | <IDLE> in transaction
emolecules | 21394 | customerdb | select tableid from hitlist_table_pool
where hitlistid = 0 limit 1 for update
emolecules | 22237 | customerdb | select tableid from hitlist_table_pool
where hitlistid = 0 limit 1 for update
(9 rows)

It's obvious that they're all waiting ... but for what? The "<IDLE>"
process looks like the culprit, but how do I figure out what it's doing?

The next time this happens, what queries can I run to help diagnose what's
going on?

This is PG 8.4.4 on Ubuntu 10.

Thanks,
Craig

[Scott Marlowe]

On Tue, Nov 29, 2011 at 11:15 AM, Craig James

Post by Craig James
Several times recently one of our databases has gotten stuck with the
postgres=# select datname, procpid, usename, current_query from
pg_stat_activity where current_query != '<IDLE>';
  datname   | procpid | usename  |
  current_query
------------+---------+----------+--------------------------------------------------------------------------------------------------------
 emolecules |   13503 | customerdb | select tableid from hitlist_table_pool
where hitlistid <= 0 for update
 emolecules |   32082 | customerdb | select tableid from hitlist_table_pool
where hitlistid <= 0 for update
 emolecules |   17974 | customerdb | select tableid from hitlist_table_pool
where hitlistid <= 0 for update
 emolecules |   31299 | customerdb | select tableid from hitlist_table_pool
where hitlistid = 0 limit 1 for update
 emolecules |   30247 | customerdb | select tableid from hitlist_table_pool
where hitlistid = 0 limit 1 for update
 postgres   |    1705 | postgres | select datname, procpid, usename,
current_query from pg_stat_activity where current_query != '<IDLE>';
 emolecules |   28866 | customerdb | <IDLE> in transaction
 emolecules |   21394 | customerdb | select tableid from hitlist_table_pool
where hitlistid = 0 limit 1 for update
 emolecules |   22237 | customerdb | select tableid from hitlist_table_pool
where hitlistid = 0 limit 1 for update
(9 rows)
It's obvious that they're all waiting ... but for what?  The "<IDLE>"
process looks like the culprit, but how do I figure out what it's doing?
The next time this happens, what queries can I run to help diagnose what's
going on?
This is PG 8.4.4 on Ubuntu 10.

Does this help?

http://wiki.postgresql.org/wiki/Lock_Monitoring

[Craig James]

Post by Scott Marlowe
On Tue, Nov 29, 2011 at 11:15 AM, Craig James

Post by Craig James
Several times recently one of our databases has gotten stuck with the
postgres=# select datname, procpid, usename, current_query from
pg_stat_activity where current_query != '<IDLE>';
datname | procpid | usename |
current_query
------------+---------+----------+--------------------------------------------------------------------------------------------------------
emolecules | 13503 | customerdb | select tableid from hitlist_table_pool
where hitlistid<= 0 for update
emolecules | 32082 | customerdb | select tableid from hitlist_table_pool
where hitlistid<= 0 for update
emolecules | 17974 | customerdb | select tableid from hitlist_table_pool
where hitlistid<= 0 for update
emolecules | 31299 | customerdb | select tableid from hitlist_table_pool
where hitlistid = 0 limit 1 for update
emolecules | 30247 | customerdb | select tableid from hitlist_table_pool
where hitlistid = 0 limit 1 for update
postgres | 1705 | postgres | select datname, procpid, usename,
current_query from pg_stat_activity where current_query != '<IDLE>';
emolecules | 28866 | customerdb |<IDLE> in transaction
emolecules | 21394 | customerdb | select tableid from hitlist_table_pool
where hitlistid = 0 limit 1 for update
emolecules | 22237 | customerdb | select tableid from hitlist_table_pool
where hitlistid = 0 limit 1 for update
(9 rows)
It's obvious that they're all waiting ... but for what? The "<IDLE>"
process looks like the culprit, but how do I figure out what it's doing?
The next time this happens, what queries can I run to help diagnose what's
going on?
This is PG 8.4.4 on Ubuntu 10.

Does this help?
http://wiki.postgresql.org/wiki/Lock_Monitoring

Yes, thanks! That's exactly what I needed.

Craig