使用python请求将0字节文件上传到owncloud会挂起

Question

我正在尝试上传一个0字节的文件,其中包含对owncloud的请求.我想为此使用类似文件的对象.通常我会这样做:

file_obj = io.BytesIO(b'')
response = requests.put('http://localhost/remote.php/webdav',
                                    auth=('xxx', 'xxx'),
                                    data=file_obj)

但它冻结了.如果我中断进程,我会看到它与堆栈跟踪挂起的位置:

Traceback (most recent call last):
  File "/home/julian/cc/client/.venv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 376, in _make_request
    httplib_response = conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/julian/cc/client/cc/storage/webdav.py", line 360, in <module>
    main()
  File "/home/julian/cc/client/cc/storage/webdav.py", line 351, in main
    data=file_obj)
  File "/home/julian/cc/client/.venv/lib/python3.5/site-packages/requests/api.py", line 120, in put
    return request('put', url, data=data, **kwargs)
  File "/home/julian/cc/client/.venv/lib/python3.5/site-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/julian/cc/client/.venv/lib/python3.5/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/julian/cc/client/.venv/lib/python3.5/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/home/julian/cc/client/.venv/lib/python3.5/site-packages/requests/adapters.py", line 376, in send
    timeout=timeout
  File "/home/julian/cc/client/.venv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    body=body, headers=headers)
  File "/home/julian/cc/client/.venv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 378, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib64/python3.5/http/client.py", line 1174, in getresponse
    response.begin()
  File "/usr/lib64/python3.5/http/client.py", line 282, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python3.5/http/client.py", line 243, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib64/python3.5/socket.py", line 575, in readinto
    return self._sock.recv_into(b)
KeyboardInterrupt

Wireshark告诉我要求sendet以下请求,这似乎没问题,但从来没有得到答案:

PUT /remote.php/webdav/test.txt HTTP/1.1
Host: localhost
Connection: keep-alive
Accept-Encoding: gzip, deflate
Authorization: Basic ***********
Transfer-Encoding: chunked
User-Agent: python-requests/2.9.1
Accept: */*
Content-Length: 0

如果我发送一个空字符串,它的工作原理:

response = requests.put('http://localhost/remote.php/webdav/test.txt',
                                    auth=('xxx', 'xxx'),
                                    data='')

HTTP流:

PUT /remote.php/webdav/test.txt HTTP/1.1
Host: localhost
Content-Length: 0
Accept-Encoding: gzip, deflate
Connection: keep-alive
User-Agent: python-requests/2.9.1
Accept: */*
Authorization: Basic ******



HTTP/1.1 204 No Content
Date: Thu, 24 Mar 2016 16:14:28 GMT
Server: Apache/2.4.10 (Debian) PHP/5.6.19
X-Powered-By: PHP/5.6.19
Set-Cookie: xxx=xxx; path=/; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: oc_sessionPassphrase=xxxx; path=/; httponly
Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; frame-src *; img-src * data: blob:; font-src 'self' data:; media-src *; connect-src *
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: Sameorigin
X-Robots-Tag: none
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Set-Cookie: xxx=xxx; path=/; HttpOnly
OC-FileId: xxxxx
Content-Length: 0
ETag: "xxx"
OC-ETag: "xxx"
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8

这也有效:

def chunker(file_obj):
    buf = None
    while buf != b'':
        print('iter')
        buf = file_obj.read(16*1024)
        yield buf

file_obj = io.BytesIO(b'')
response = requests.put('http://localhost/remote.php/webdav/test.txt',
                                    auth=('xxx', 'xxx'),
                                    data=chunker(file_obj))

任何想法为什么这不适用于类似文件的对象？我使用的是最新版本的请求(2.9.1)和Python 3.5.

Answer 1

似乎有一个错误requests.没有指定auth请求使用Transfer-Encoding: chunked并在请求last-chunk结束时发送正确,但auth没有last-chunk发送和标头混淆.

来自http://tools.ietf.org/html/rfc7230#section-4.1:

 chunked-body   = *chunk
                  last-chunk
                  trailer-part
                  CRLF

 chunk          = chunk-size [ chunk-ext ] CRLF
                  chunk-data CRLF
 chunk-size     = 1*HEXDIG
 last-chunk     = 1*("0") [ chunk-ext ] CRLF

 chunk-data     = 1*OCTET ; a sequence of chunk-size octets

和http://tools.ietf.org/html/rfc7230#section-3.3.2:

发送方不得
在包含Transfer-Encoding标头字段的任何消息中发送Content-Length标头字段.

没有auth争论

f = io.BytesIO(b'')
requests.put('http://localhost:8000/asdf', data=f)

发送的请求是

PUT /asdf HTTP/1.1?
Host: localhost:8000?
User-Agent: python-requests/2.9.1?
Transfer-Encoding: chunked?
Accept-Encoding: gzip, deflate?
Connection: keep-alive?
Accept: */*?
?
0?
?  

(⏎表示上面的CRLF).但是,如果你指定auth

requests.put('http://localhost:8000/asdf', auth=('asdf', 'fdsa'), data=f)

请求是

PUT /asdf HTTP/1.1?
Host: localhost:8000?
Transfer-Encoding: chunked?
Accept-Encoding: gzip, deflate?
User-Agent: python-requests/2.9.1?
Authorization: Basic YXNkZjpmZHNh?
Content-Length: 0?
Connection: keep-alive?
Accept: */*?
?

这两个Transfer-Encoding和Content-Length指定,它不应该这样做,而最后一块未被发送,所以服务器处于等待更多的块来和requests等待响应.