Python:获取URL路径部分

Question

如何从网址获取特定的路径部分？例如,我想要一个对此进行操作的函数:

http://www.mydomain.com/hithere?image=2934

并返回"hithere"

或对此进行操作:

http://www.mydomain.com/hithere/something/else

并返回相同的东西("hithere")

我知道这可能会使用urllib或urllib2,但我无法从文档中找出如何仅获取路径的一部分.

Answer 1

使用urlparse提取URL的路径组件:

>>> import urlparse
>>> path = urlparse.urlparse('http://www.example.com/hithere/something/else').path
>>> path
'/hithere/something/else'

使用os.path .split 将路径拆分为组件:

>>> import os.path
>>> os.path.split(path)
('/hithere/something', 'else')

dirname和basename函数为您提供了两个分割; 也许在while循环中使用dirname:

>>> while os.path.dirname(path) != '/':
...     path = os.path.dirname(path)
... 
>>> path
'/hithere'

Answer 2

最佳选择是在使用posixpathURL的路径组件时使用该模块.os.path当在基于POSIX和Windows NT的平台上使用时,此模块具有与POSIX路径相同的接口并始终如一地操作.

示例代码:

#!/usr/bin/env python3

import urllib.parse
import sys
import posixpath
import ntpath
import json

def path_parse( path_string, *, normalize = True, module = posixpath ):
    result = []
    if normalize:
        tmp = module.normpath( path_string )
    else:
        tmp = path_string
    while tmp != "/":
        ( tmp, item ) = module.split( tmp )
        result.insert( 0, item )
    return result

def dump_array( array ):
    string = "[ "
    for index, item in enumerate( array ):
        if index > 0:
            string += ", "
        string += "\"{}\"".format( item )
    string += " ]"
    return string

def test_url( url, *, normalize = True, module = posixpath ):
    url_parsed = urllib.parse.urlparse( url )
    path_parsed = path_parse( urllib.parse.unquote( url_parsed.path ),
        normalize=normalize, module=module )
    sys.stdout.write( "{}\n  --[n={},m={}]-->\n    {}\n".format( 
        url, normalize, module.__name__, dump_array( path_parsed ) ) )

test_url( "http://eg.com/hithere/something/else" )
test_url( "http://eg.com/hithere/something/else/" )
test_url( "http://eg.com/hithere/something/else/", normalize = False )
test_url( "http://eg.com/hithere/../else" )
test_url( "http://eg.com/hithere/../else", normalize = False )
test_url( "http://eg.com/hithere/../../else" )
test_url( "http://eg.com/hithere/../../else", normalize = False )
test_url( "http://eg.com/hithere/something/./else" )
test_url( "http://eg.com/hithere/something/./else", normalize = False )
test_url( "http://eg.com/hithere/something/./else/./" )
test_url( "http://eg.com/hithere/something/./else/./", normalize = False )

test_url( "http://eg.com/see%5C/if%5C/this%5C/works", normalize = False )
test_url( "http://eg.com/see%5C/if%5C/this%5C/works", normalize = False,
    module = ntpath )

代码输出:

http://eg.com/hithere/something/else
  --[n=True,m=posixpath]-->
    [ "hithere", "something", "else" ]
http://eg.com/hithere/something/else/
  --[n=True,m=posixpath]-->
    [ "hithere", "something", "else" ]
http://eg.com/hithere/something/else/
  --[n=False,m=posixpath]-->
    [ "hithere", "something", "else", "" ]
http://eg.com/hithere/../else
  --[n=True,m=posixpath]-->
    [ "else" ]
http://eg.com/hithere/../else
  --[n=False,m=posixpath]-->
    [ "hithere", "..", "else" ]
http://eg.com/hithere/../../else
  --[n=True,m=posixpath]-->
    [ "else" ]
http://eg.com/hithere/../../else
  --[n=False,m=posixpath]-->
    [ "hithere", "..", "..", "else" ]
http://eg.com/hithere/something/./else
  --[n=True,m=posixpath]-->
    [ "hithere", "something", "else" ]
http://eg.com/hithere/something/./else
  --[n=False,m=posixpath]-->
    [ "hithere", "something", ".", "else" ]
http://eg.com/hithere/something/./else/./
  --[n=True,m=posixpath]-->
    [ "hithere", "something", "else" ]
http://eg.com/hithere/something/./else/./
  --[n=False,m=posixpath]-->
    [ "hithere", "something", ".", "else", ".", "" ]
http://eg.com/see%5C/if%5C/this%5C/works
  --[n=False,m=posixpath]-->
    [ "see\", "if\", "this\", "works" ]
http://eg.com/see%5C/if%5C/this%5C/works
  --[n=False,m=ntpath]-->
    [ "see", "if", "this", "works" ]

笔记:

• 在基于Windows NT的平台os.path上ntpath
• 在基于Unix/Posix的平台os.path上posixpath
• ntpath不会\正确处理反斜杠()(参见代码/输出中的最后两种情况) - 这posixpath就是推荐的原因.
• 记得要用 urllib.parse.unquote
• 考虑使用 posixpath.normpath
• RFC 3986/未定义多个路径分隔符()的语义.然而,折叠多个相邻的路径分隔符(即它处理,并相同)posixpath//////
• 尽管POSIX和URL路径具有相似的语法和语义,但它们并不相同.

规范性参考文献:

• IEEE Std 1003.1,2013 - Vol.1:基本定义 - 第4.12节:路径名解析
• GNU C库参考手册 - 第11.2节:文件名
• IETF RFC 3986:统一资源标识符(URI):通用语法 - 第3.3节:路径
• IETF RFC 3986:统一资源标识符(URI):通用语法 - 第6节:规范化和比较
• 维基百科:URL规范化

Answer 3

Python 3.4+解决方案:

from urllib.parse import unquote, urlparse
from pathlib import PurePosixPath

url = 'http://www.example.com/hithere/something/else'

PurePosixPath(
    unquote(
        urlparse(
            url
        ).path
    )
).parts[1]

# returns 'hithere' (the same for the URL with parameters)

# parts holds ('/', 'hithere', 'something', 'else')
#               0    1          2            3

Answer 4

注意 Python3 import 已更改为from urllib.parse import urlparseSee documentation。下面是一个例子：

>>> from urllib.parse import urlparse
>>> url = 's3://bucket.test/my/file/directory'
>>> p = urlparse(url)
>>> p
ParseResult(scheme='s3', netloc='bucket.test', path='/my/file/directory', params='', query='', fragment='')
>>> p.scheme
's3'
>>> p.netloc
'bucket.test'
>>> p.path
'/my/file/directory'