Sam Hooke

These are rough notes from whatever I was working on, interested in or thinking about at the time. They vary greatly in quality and length, but prove useful to me, and hopefully to you too!

Kombu "timed out" bug: connection hangs forever in Kombu 4.1.0

Kombu connects to RabbitMQ. However in Kombu 4.1.0 there is a bug where it waits forever if the broker is down rather than timing out. This affects Celery too. There is a fix in trunk but at time of writing there is no new release of Kombu which contains this fix. A workaround is to downgrade to Kombu 4.0.2.

# kombu==4.1.0
# hangs forever
# kombu==4.0.2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/venv/lib/python2.7/site-packages/celery/app/", line 235, in ping
    timeout=timeout, **kwargs)
  File "/opt/venv/lib/python2.7/site-packages/celery/app/", line 436, in broadcast
    limit, callback, channel=channel,
  File "/opt/venv/lib/python2.7/site-packages/kombu/", line 304, in _broadcast
    chan = channel or self.connection.default_channel
  File "/opt/venv/lib/python2.7/site-packages/kombu/", line 819, in default_channel
  File "/opt/venv/lib/python2.7/site-packages/kombu/", line 802, in connection
    self._connection = self._establish_connection()
  File "/opt/venv/lib/python2.7/site-packages/kombu/", line 757, in _establish_connection
    conn = self.transport.establish_connection()
  File "/opt/venv/lib/python2.7/site-packages/kombu/transport/", line 130, in establish_connection
  File "/opt/venv/lib/python2.7/site-packages/amqp/", line 302, in connect
  File "/opt/venv/lib/python2.7/site-packages/amqp/", line 485, in drain_events
    while not self.blocking_read(timeout):
  File "/opt/venv/lib/python2.7/site-packages/amqp/", line 490, in blocking_read
    frame = self.transport.read_frame()
  File "/opt/venv/lib/python2.7/site-packages/amqp/", line 240, in read_frame
    frame_header = read(7, True)
  File "/opt/venv/lib/python2.7/site-packages/amqp/", line 415, in _read
    s = recv(n - len(rbuf))
socket.timeout: timed out
<< Previous: Debugging RabbitMQ
Next: RabbitMQ handshake timeout caused by hostname change >>