How to mock raise urllib errors

Question

After reading this in the python docs, I am catching the HTTPError and URLError exceptions in get_response_from_external_api that the make_request_and_get_response (via urllib's urlopen call) can raise:

foo.main.py

from urllib.request import urlopen
import contextlib
from urllib.error import HTTPError, URLError

def make_request_and_get_response(q):
    with contextlib.closing(urlopen(q)) as response:
        return response.read()

def get_response_from_external_api(q):
    try:
        resp = make_request_and_get_response(q)
        return resp
    except URLError as e:
        print('Got a URLError: ', e)
    except HTTPError as e:
        print('Got a HTTPError: ', e)

if __name__ == "__main__":
    query = 'test'
    result = get_response_from_external_api(query)
    print(result)

While testing the get_response_from_external_api method, I am trying to mock raising the HTTPError and URLError exceptions:

foo.test_main.py

from foo.main import get_response_from_external_api

import pytest
from unittest.mock import patch, Mock
from urllib.error import HTTPError, URLError

def test_get_response_from_external_api_with_httperror(capsys):
    with patch('foo.main.make_request_and_get_response') as mocked_method:
        with pytest.raises(HTTPError) as exc:
            mocked_method.side_effect = HTTPError()  # TypeError
            resp = get_response_from_external_api(mocked_method)

            out, err = capsys.readouterr()
            assert resp is None
            assert 'HTTPError' in out
            assert str(exc) == HTTPError

def test_get_response_from_external_api_with_urlerror(capsys):
    with patch('foo.main.make_request_and_get_response') as mocked_method:
        with pytest.raises(URLError) as exc:
            mocked_method.side_effect = URLError()  # TypeError
            resp = get_response_from_external_api(mocked_method)

            out, err = capsys.readouterr()
            assert resp is None
            assert 'URLError' in out
            assert str(exc) == URLError

But I get a TypeError: __init__() missing 5 required positional arguments: 'url', 'code', 'msg', 'hdrs', and 'fp'. I am new to python mocks syntax and looking for examples.

I have read this answer but I cannot see how this can be applied in my case where the return value of the urllib.urlopen (via get_response_from_external_api) is outside of the scope of the except-block. Not sure if I should instead mock the whole urllib.urlopen.read instead as seen here?

Answer 1

无需模拟的一部分urlopen-通过模拟函数引发异常，可以确保urlopen不会被调用。

由于您正在创建这些异常来检查您的错误处理代码是否正常运行，因此它们不需要完整-它们仅包含满足测试所需的最少信息。

HTTPError需要五个参数：

• 一个网址
• HTTP状态码
• 错误信息
• 请求头
• 类似于文件的对象（响应的主体）

出于模拟目的，这些全都可以是None，但是构造看起来像真正错误的对象可能会有所帮助。如果要读取“文件状对象”，则可以传递io.BytesIO包含示例响应的实例，但是根据问题中的代码，这似乎没有必要。

>>> h = HTTPError('http://example.com', 500, 'Internal Error', {}, None)
>>> h
<HTTPError 500: 'Internal Error'>

URLError需要一个参数，该参数可以是字符串或异常实例。出于模拟目的，字符串就足够了。

>>> u = URLError('Unknown host')
>>> u

URLError('Unknown host')

这是该问题的代码，将上述内容考虑在内进行了修改。无需将模拟函数传递给自身-只需传递任意字符串即可。我删除了这些with pytest.raises块，因为异常被捕获在您的代码的try / except块中：您正在测试代码本身处理异常，而不是异常渗入到测试函数中。

from foo.main import get_response_from_external_api

import pytest
from unittest.mock import patch, Mock
from urllib.error import HTTPError, URLError

def test_get_response_from_external_api_with_httperror(capsys):
    with patch('foo.main.make_request_and_get_response') as mocked_method:
        mocked_method.side_effect = HTTPError('http://example.com', 500, 'Internal Error', {}, None)
        resp = get_response_from_external_api('any string')
        assert resp is None
        out, err = capsys.readouterr()
        assert 'HTTPError' in out


def test_get_response_from_external_api_with_urlerror(capsys):
    with patch('foo.main.make_request_and_get_response') as mocked_method:
        mocked_method.side_effect = URLError('Unknown host')
        resp = get_response_from_external_api('any string')
        assert resp is None
        out, err = capsys.readouterr()
        assert 'URLError' in out

最后，您需要颠倒尝试try除外块的顺序- HTTPError是的子类URLError，因此您需要先对其进行测试，否则它将由该except URLError块处理。