tree 4c0f0c9c7d8ce0ff0d6c9dc51810e490f9edbcb3
parent 4a102a9fc000dc0abf9ad27698bb50311beba794
author Avery Pennarun <apenwarr@gmail.com> 1411108571 -0400
committer Avery Pennarun <apenwarr@gmail.com> 1470782345 -0400

tornado: disable epoll and add a test that shows why.

Here's a simple test program that shows the problem with epoll.

import socket, select, os
s1, s2 = socket.socketpair()
fd = os.dup(s1.fileno())
e = select.epoll()
e.register(s1, select.EPOLLIN)
e.poll(1)
s2.close()
e.poll(1)
e.poll(1)
os.close(3)
os.close(fd)
e.poll(1)

libcurl apparently still has a bug where it will sometimes close and replace
its underlying file descriptor *before* telling epoll (or not even telling
epoll at all).  select() would have no problem with this, since the fd is
the same before and after, but epoll remembers the identity of the fd, which
persists even if the file no longer exists in this process (like if it was
inherited by a subprocess).  Unfortunately, it keeps returning the original
fd number, not the new one, which is both useless and confusing.  And to
make things worse, it doesn't poll on the new file descriptor that inherited
the old number!

This causes these possible symptoms:

1. if a subprocess inherits our libcurl fd, we'll keep getting epoll events
   on it under some conditions, even after we think we've closed it.  When
   the remote http server dies, we'll just get endless EOF events on a fd
   we can't possibly read anymore since it doesn't even exist in this
   process.

2. if the remote server *doesn't* close our
   connection right away, catawampus might just sit forever (or until
   a local timeout if one exists) waiting for the server to respond on the
   new socket.  The server might in fact respond, but we'd never get an
   event about it.  This could explain strange catawampus delays.  This could
   happen regardless of whether a subprocess has a copy of the original fd
   or not; the main thing is epoll is not listening on the new fd.

The "right" fix would be to overhaul libcurl to always unregister its fds
before closing or replacing them, but that's too hard (because libcurl's fd
logic is very confusing).  Instead, let's just bludgeon it by switching to
good old-fashioned select(), which is more resilient to such things.

select() is less scalable than epoll() if you have zillions of fds (which is
what tornado was originally built to handle), but our use case doesn't need
this.

b/17575492

Change-Id: I1d57a7d8defa48e0b003e10b59afa474d4c4e2bd