[tac_plus] Orphaned connections with dual-stack process

JJ Crawford jj at fb.com
Tue Jun 19 18:40:18 UTC 2018


Currently in all releases of tac_plus there is the possibility to leak file descriptors due to how accept() calls are handled. This should only impact versions where both v4 and v6 listeners exist or some other setup where multiple listeners are being utilized.

If you are using both ipv4 and ipv6 bound and you get a v4 and v6 request at the same time the for loop will accept both before hitting the fork call to handle the request. This results in one of the requests being properly handled and closed but the other open descriptor will be orphaned and remain in close_wait status indefinitely, ultimately leading to hitting the open file limit and stopping all further requests from being processed.

This can be seen in the trace below… both fd 11 and 12 are accepted, but only fd 12 is closed since newsockfd is reassigned before anything is done with fd 11.

poll([{fd=4, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 2, 180000) = 2 ([{fd=4, revents=POLLIN}, {fd=5, revents=POLLIN}])
accept(4, {sa_family=AF_INET, sin_port=htons(41359), sin_addr=inet_addr("172.16.4.5")}, [16]) = 11
accept(5, {sa_family=AF_INET6, sin6_port=htons(47166), inet_pton(AF_INET6, "2601:600:9f80:3073::", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 12
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f2760df0a50) = 3420565
close(12)                               = 0
poll([{fd=4, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 2, 180000) = 1 ([{fd=4, revents=POLLIN}])

I temporarily got around this by checking newsockfd after accept, and if it is valid (>=0) breaking out of the loop to fully process that request before checking for more. This really isn’t ideal as in theory you could block all requests on the other sockets from being processed if the first socket always has connections pending, but it works for the interim.

Thanks,
-jj
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.shrubbery.net/pipermail/tac_plus/attachments/20180619/b1443547/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stranded-socket.diff
Type: application/octet-stream
Size: 690 bytes
Desc: stranded-socket.diff
URL: <http://www.shrubbery.net/pipermail/tac_plus/attachments/20180619/b1443547/attachment.obj>


More information about the tac_plus mailing list