列出所有郊区并检查是否在python中被破坏

Question

我有一个在django开发的网站,其中包括大约25000个郊区.我需要一些东西来列出网站中的所有网址,并检查链接是否定期中断,所以我更喜欢做一些我可以作为脚本运行的东西.

我应该遵循哪种方法？任何的想法 ？

Answer 1

这是一个基于优秀@sneawo答案的改进课程.功能包括:

• 自动加载所有urlconfs settings.ROOT_URLCONF;
• 递归加载包含的urlconfs;
• 测试期望关键字参数的url模式(通过默认参数);
• 测试不同的http代码;
• 测试仅适用于登录用户的网址.

欢迎改进.

    from django import test
    from django.core.urlresolvers import reverse
    from django.conf import settings
    import importlib

    class UrlsTest(test.TestCase):

        def test_responses(self, allowed_http_codes=[200, 302, 405],
                credentials={}, logout_url="", default_kwargs={}, quiet=False):
            """
            Test all pattern in root urlconf and included ones.
            Do GET requests only.
            A pattern is skipped if any of the conditions applies:
                - pattern has no name in urlconf
                - pattern expects any positinal parameters
                - pattern expects keyword parameters that are not specified in @default_kwargs
            If response code is not in @allowed_http_codes, fail the test.
            if @credentials dict is specified (e.g. username and password),
                login before run tests.
            If @logout_url is specified, then check if we accidentally logged out
                the client while testing, and login again
            Specify @default_kwargs to be used for patterns that expect keyword parameters,
                e.g. if you specify default_kwargs={'username': 'testuser'}, then
                for pattern url(r'^accounts/(?P<username>[\.\w-]+)/$' 
                the url /accounts/testuser/ will be tested.
            If @quiet=False, print all the urls checked. If status code of the response is not 200,
                print the status code.
            """
            module = importlib.import_module(settings.ROOT_URLCONF)
            if credentials:
                self.client.login(**credentials)
            def check_urls(urlpatterns, prefix=''):
                for pattern in urlpatterns:
                    if hasattr(pattern, 'url_patterns'):
                        # this is an included urlconf
                        new_prefix = prefix
                        if pattern.namespace:
                            new_prefix = prefix + (":" if prefix else "") + pattern.namespace
                        check_urls(pattern.url_patterns, prefix=new_prefix)
                    params = {}
                    skip = False
                    regex = pattern.regex
                    if regex.groups > 0:
                        # the url expects parameters
                        # use default_kwargs supplied
                        if regex.groups > len(regex.groupindex.keys()) \
                            or set(regex.groupindex.keys()) - set(default_kwargs.keys()):
                            # there are positional parameters OR
                            # keyword parameters that are not supplied in default_kwargs
                            # so we skip the url
                            skip = True
                        else:
                            for key in set(default_kwargs.keys()) & set(regex.groupindex.keys()):
                                params[key] = default_kwargs[key]
                    if hasattr(pattern, "name") and pattern.name:
                        name = pattern.name
                    else:
                        # if pattern has no name, skip it
                        skip = True
                        name = ""
                    fullname = (prefix + ":" + name) if prefix else name
                    if not skip:
                        url = reverse(fullname, kwargs=params)
                        response = self.client.get(url)
                        self.assertIn(response.status_code, allowed_http_codes)
                        # print status code if it is not 200
                        status = "" if response.status_code == 200 else str(response.status_code) + " "
                        if not quiet:
                            print(status + url)
                        if url == logout_url and credentials:
                            # if we just tested logout, then login again
                            self.client.login(**credentials)
                    else:
                        if not quiet:
                            print("SKIP " + regex.pattern + " " + fullname)
            check_urls(module.urlpatterns)

Answer 2

在django-extensions中使用show-urls命令作为起点.(文件)

python manage.py show_urls