Summary

TCP connections can cause process to hang

Metadata

Description

As reported by Jim Ursetto, his Spiffy server (which is running on localhost behind a NGINX proxy) will stop responding at some point. It is currently not 100% clear when exactly this happens.

As Jim says:

> I believe I’ve tracked down the problem, although not the solution yet. It seems the file descriptor table is filled up with half open sockets. lsof shows: > > … > chickadee 13361 jim 1019u sock 0,7 0t0 2365037 can't identify protocol > chickadee 13361 jim 1020u sock 0,7 0t0 2366414 can't identify protocol > chickadee 13361 jim 1021u sock 0,7 0t0 2368047 can't identify protocol > chickadee 13361 jim 1022u sock 0,7 0t0 2368343 can't identify protocol > > And this message on Linux seems to occur when sockets are half-open (or half-closed, if you are a pessimist).

And later:

> my hunch is that this happens when the connecting side hangs up while we are still sending. Which is probably obvious from my previous description of the problem. I was going to start attacking it by inserting strategic sleeps at various locations to tryßßßßßßßßually trigger it, although you probably have a better way. Obviously you do appear to catch all errors in spiffy, so I’m not sure if one of those error handlers is neglecting to close a socket, if the socket close fails and isn’t reported, or if this is a deeper bug inside the tcp unit.

I hear faint echoes of #340...

Changes and comments

[2018-05-01 16:39:54 UTC] kooda wrote:

I’ve investigated this bug a little, and it seems to be caused when reading or writing to the socket causes a broken pipe exception. The sockets seem to stay half-open even after the ports have been closed.

Forcing the socket closing with `file-close` (in spiffy) makes the issue disappear.

I’ll have to dig a bit more, I don’t know what the POSIX spec says about the state of sockets in this case.

[2018-05-24 11:43:12 UTC] kooda changed milestone from 4.14.0 to 5.1

[2019-04-07 12:23:24 UTC] sjamaan changed milestone from 5.1 to 5.2

[2019-04-07 12:23:24 UTC] sjamaan wrote:

Getting ready for 5.1, moving tickets which won't make it in to 5.2.

[2019-08-25 17:35:28 UTC] felix changed milestone from 5.2 to 5.3

[2021-04-17 11:48:28 UTC] felix changed milestone from 5.3 to 5.4

[2021-04-17 11:48:28 UTC] felix wrote:

I think this is too obscure to be fixed any time soon, shifted to 5.4

[2023-11-06 22:11:22 UTC] felix changed milestone from 5.4 to someday