http_download fixes

+ fix b/7384266
+ fix http_download.py as standalone file downloader, and
  add a unit test to try to keep from breaking it again.
+ log download progress. Also write to a file, suitable
  to read into a UI somewhere.
+ don't stringify username and password, as this broke
  downloads where no authentication was required.
+ add unit test to actually fetch content from a local
  webserver, not just mock the httpclient and check the
  arguments
+ add unit test to fetch content from a local webserver
  which requires digest authentication.
+ Fix interop between our digest client and digest server
  implementations, always according to the "be liberal
  in what you accept" principle:
  + in client, don't require www-authenticate parameters
    to be quoted. RFC 2617 has a number of examples where
    they are not.
  + in server, don't require commas in the Authorization
    header to be followed by a space. Our client does not
    send spaces, and apparently IE doesn't either:
    https://github.com/jatsz/curtain/commit/faee5b610e14c83dc2fd8fdbd010172ff0db86ac
+ Ensure that filling the filesystem is handled, deleting
  the file and sending failure back to the ACS.
+ Fix Tornado unit test HTTP server to allow connection
  via either IPv4 or IPv6. Tornado 2.4 completely revamps
  this part of the code, so rather than push it upstream
  I think we'll update to the latest Tornado.

Change-Id: I0fb1f6558f8d8bd616106b10016c09c59bbf400a
1 file changed