Sam Hooke

Kombu "timed out" bug: connection hangs forever in Kombu 4.1.0

Kombu connects to RabbitMQ. However in Kombu 4.1.0 there is a bug where it waits forever if the broker is down rather than timing out. This affects Celery too. There is a fix in trunk but at time of writing there is no new release of Kombu which contains this fix. A workaround is to downgrade to Kombu 4.0.2.

# kombu==4.1.0
# hangs forever
# kombu==4.0.2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/venv/lib/python2.7/site-packages/celery/app/", line 235, in ping
    timeout=timeout, **kwargs)
  File "/opt/venv/lib/python2.7/site-packages/celery/app/", line 436, in broadcast
    limit, callback, channel=channel,
  File "/opt/venv/lib/python2.7/site-packages/kombu/", line 304, in _broadcast
    chan = channel or self.connection.default_channel
  File "/opt/venv/lib/python2.7/site-packages/kombu/", line 819, in default_channel
  File "/opt/venv/lib/python2.7/site-packages/kombu/", line 802, in connection
    self._connection = self._establish_connection()
  File "/opt/venv/lib/python2.7/site-packages/kombu/", line 757, in _establish_connection
    conn = self.transport.establish_connection()
  File "/opt/venv/lib/python2.7/site-packages/kombu/transport/", line 130, in establish_connection
  File "/opt/venv/lib/python2.7/site-packages/amqp/", line 302, in connect
  File "/opt/venv/lib/python2.7/site-packages/amqp/", line 485, in drain_events
    while not self.blocking_read(timeout):
  File "/opt/venv/lib/python2.7/site-packages/amqp/", line 490, in blocking_read
    frame = self.transport.read_frame()
  File "/opt/venv/lib/python2.7/site-packages/amqp/", line 240, in read_frame
    frame_header = read(7, True)
  File "/opt/venv/lib/python2.7/site-packages/amqp/", line 415, in _read
    s = recv(n - len(rbuf))
socket.timeout: timed out

These are rough notes from whatever I was working on, interested in or thinking about at the time. They vary greatly in quality and length, but prove useful to me, and hopefully to you too!

← Previous: Manually invoking Celery app control commands from a Python package
Next: Celery pinging →