如何确定Python中嵌套数据结构的类型?

Guy*_*der 7 python types

我目前正在将一些 Python 翻译成 F#,特别是神经网络和深度学习。

为了确保正确转换数据结构,需要 Python 中嵌套类型的详细信息。该类型()函数工作简单类型而不是嵌套类型。

例如在 Python 中:

> data = ([[1,2,3],[4,5,6],[7,8,9]],["a","b","c"])
> type(data)
<type 'tuple'>
Run Code Online (Sandbox Code Playgroud)

只给出第一级的类型。对元组中的数组一无所知。

我希望像 F# 那样做

> let data = ([|[|1;2;3|];[|4;5;6|];[|7;8;9|]|],[|"a";"b";"c"|]);;

val data : int [] [] * string [] =
  ([|[|1; 2; 3|]; [|4; 5; 6|]; [|7; 8; 9|]|], [|"a"; "b"; "c"|])
Run Code Online (Sandbox Code Playgroud)

返回独立于值的签名

int[][]*字符串[]

*         is a tuple item separator  
int [] [] is a two dimensional jagged array of int  
string [] is a one dimensional array of string
Run Code Online (Sandbox Code Playgroud)

这可以或如何在 Python 中完成?

TLDR;

目前我正在将 PyCharm 与调试器一起使用,并在变量窗口中单击单个变量的查看选项以查看详细信息。问题是输出包含值以及混合的类型,我只需要类型签名。当变量类似于 (float[50000][784], int[50000]) 时,这些值会成为障碍。是的,我现在正在调整变量的大小,但这是一种解决方法,而不是解决方案。

例如

使用PyCharm 社区

(array([[ 0.,  0.,  0., ...,  0.,  0.,  0.],
        [ 0.,  0.,  0., ...,  0.,  0.,  0.],
        [ 0.,  0.,  0., ...,  0.,  0.,  0.],
        ...,     
        [ 0.,  0.,  0., ...,  0.,  0.,  0.],
        [ 0.,  0.,  0., ...,  0.,  0.,  0.],
        [ 0.,  0.,  0., ...,  0.,  0.,  0.]], dtype=float32),
  array([7, 2, 1, ..., 4, 5, 6]))
Run Code Online (Sandbox Code Playgroud)

使用Spyder

使用 Spyder 变量查看器

将Visual Studio 社区与适用于 Visual Studio 的 Python 工具结合使用

(array([[ 0.,  0.,  0., ...,  0.,  0.,  0.],    
        [ 0.,  0.,  0., ...,  0.,  0.,  0.],  
        [ 0.,  0.,  0., ...,  0.,  0.,  0.],  
        ...,   
        [ 0.,  0.,  0., ...,  0.,  0.,  0.],  
        [ 0.,  0.,  0., ...,  0.,  0.,  0.],  
        [ 0.,  0.,  0., ...,  0.,  0.,  0.]], dtype=float32),  
  array([5, 0, 4, ..., 8, 4, 8], dtype=int64)) 
Run Code Online (Sandbox Code Playgroud)

编辑:

由于这个问题已经被盯上了,有人显然是在寻找更多细节,这是我的修改版本,它也可以处理numpy ndarray。感谢Vlad提供初始版本。

也因为使用了运行长度编码的变体,所以不再使用 ? 对于异构类型。

# Note: Typing for elements of iterable types such as Set, List, or Dict 
# use a variation of Run Length Encoding.

def type_spec_iterable(iterable, name):
    def iterable_info(iterable):
        # With an iterable for it to be comparable 
        # the identity must contain the name and length 
        # and for the elements the type, order and count.
        length = 0
        types_list = []
        pervious_identity_type = None
        pervious_identity_type_count = 0
        first_item_done = False
        for e in iterable:
            item_type = type_spec(e)
            if (item_type != pervious_identity_type):
                if not first_item_done:
                    first_item_done = True
                else:
                    types_list.append((pervious_identity_type, pervious_identity_type_count))
                pervious_identity_type = item_type
                pervious_identity_type_count = 1
            else:
                pervious_identity_type_count += 1
            length += 1
        types_list.append((pervious_identity_type, pervious_identity_type_count))
        return (length, types_list)
    (length, identity_list) = iterable_info(iterable)
    element_types = ""
    for (identity_item_type, identity_item_count) in identity_list:
        if element_types == "":
            pass
        else:
            element_types += ","
        element_types += identity_item_type
        if (identity_item_count != length) and (identity_item_count != 1):
            element_types += "[" + `identity_item_count` + "]"
    result = name + "[" + `length` + "]<" + element_types + ">"
    return result

def type_spec_dict(dict, name):
    def dict_info(dict):
        # With a dict for it to be comparable 
        # the identity must contain the name and length 
        # and for the key and value combinations the type, order and count.
        length = 0
        types_list = []
        pervious_identity_type = None
        pervious_identity_type_count = 0
        first_item_done = False
        for (k, v) in dict.iteritems():
            key_type = type_spec(k)
            value_type = type_spec(v)
            item_type = (key_type, value_type)
            if (item_type != pervious_identity_type):
                if not first_item_done:
                    first_item_done = True
                else:
                    types_list.append((pervious_identity_type, pervious_identity_type_count))
                pervious_identity_type = item_type
                pervious_identity_type_count = 1
            else:
                pervious_identity_type_count += 1
            length += 1
        types_list.append((pervious_identity_type, pervious_identity_type_count))
        return (length, types_list)
    (length, identity_list) = dict_info(dict)
    element_types = ""
    for ((identity_key_type,identity_value_type), identity_item_count) in identity_list:
        if element_types == "":
            pass
        else:
            element_types += ","
        identity_item_type = "(" + identity_key_type + "," + identity_value_type + ")"
        element_types += identity_item_type
        if (identity_item_count != length) and (identity_item_count != 1):
            element_types += "[" + `identity_item_count` + "]"
    result = name + "[" + `length` + "]<" + element_types + ">"
    return result

def type_spec_tuple(tuple, name):
    return name + "<" + ", ".join(type_spec(e) for e in tuple) + ">"

def type_spec(obj):
    object_type = type(obj)
    name = object_type.__name__
    if (object_type is int) or (object_type is long) or (object_type is str) or (object_type is bool) or (object_type is float):            
        result = name
    elif object_type is type(None):
        result = "(none)"
    elif (object_type is list) or (object_type is set):
        result = type_spec_iterable(obj, name)
    elif (object_type is dict):
        result = type_spec_dict(obj, name)
    elif (object_type is tuple):
        result = type_spec_tuple(obj, name)
    else:
        if name == 'ndarray':
            ndarray = obj
            ndarray_shape = "[" + `ndarray.shape`.replace("L","").replace(" ","").replace("(","").replace(")","") + "]"
            ndarray_data_type = `ndarray.dtype`.split("'")[1]
            result = name + ndarray_shape + "<" + ndarray_data_type + ">"
        else:
            result = "Unknown type: " , name
    return result
Run Code Online (Sandbox Code Playgroud)

我不会认为它已经完成,但到目前为止它已经完成了我需要的一切。

L3v*_*han 5

正如我所评论的,这在 Python 中是不可能的,因为列表是无类型的。

你仍然可以假装这样做:

def typ(something, depth=0):
    if depth > 63:
        return "..."
    if type(something) == tuple:
        return "<class 'tuple': <" + ", ".join(typ(ding, depth+1) for ding in something) + ">>"
    elif type(something) == list:
        return "<class 'list': " + (typ(something[0], depth+1) if something else '(empty)') + ">"
    else:
        return str(type(something))
Run Code Online (Sandbox Code Playgroud)

<class 'tuple': <<class 'list': <class 'list': <class 'int'>>>,<class 'list': <class 'str'>>>>这将返回您的示例的字符串。

编辑:为了使它看起来更像 F#,你可以这样做:

def typ(something, depth=0):
    if depth > 63:
        return "..."
    if type(something) == tuple:
        return " * ".join(typ(ding, depth+1) for ding in something)
    elif type(something) == list:
        return (typ(something[0]) if something else 'empty') + " []"
    else:
        return str(type(something, depth+1)).split("'")[1]
Run Code Online (Sandbox Code Playgroud)

int [] [] * str []这将在您的示例中返回。


Vla*_*lad 3

手动完成的一种方法是:

def type_spec_iterable(obj, name):
    tps = set(type_spec(e) for e in obj)
    if len(tps) == 1:
        return name + "<" + next(iter(tps)) + ">"
    else:
        return name + "<?>"


def type_spec_dict(obj):
    tps = set((type_spec(k), type_spec(v)) for (k,v) in obj.iteritems())
    keytypes = set(k for (k, v) in tps)
    valtypes =  set(v for (k, v) in tps)
    kt = next(iter(keytypes)) if len(keytypes) == 1 else "?"
    vt = next(iter(valtypes)) if len(valtypes) == 1 else "?"
    return "dict<%s, %s>" % (kt, vt)


def type_spec_tuple(obj):
    return "tuple<" + ", ".join(type_spec(e) for e in obj) + ">"


def type_spec(obj):
    t = type(obj)
    res = {
        int: "int",
        str: "str",
        bool: "bool",
        float: "float",
        type(None): "(none)",
        list: lambda o: type_spec_iterable(o, 'list'),
        set: lambda o: type_spec_iterable(o, 'set'),
        dict: type_spec_dict,
        tuple: type_spec_tuple,
    }.get(t, lambda o: type(o).__name__)
    return res if type(res) is str else res(obj)


if __name__ == "__main__":
    class Foo(object):
        pass
    for obj in [
        1,
        2.3,
        None,
        False,
        "hello",
        [1, 2, 3],
        ["a", "b"],
        [1, "h"],
        (False, 1, "2"),
        set([1.2, 2.3, 3.4]),
        [[1,2,3],[4,5,6],[7,8,9]],
        [(1,'a'), (2, 'b')],
        {1:'b', 2:'c'},
        [Foo()], # todo - inheritance?
    ]:
        print repr(obj), ":", type_spec(obj)
Run Code Online (Sandbox Code Playgroud)

这打印:

1 : int
2.3 : float
None : (none)
False : bool
'hello' : str
[1, 2, 3] : list<int>
['a', 'b'] : list<str>
[1, 'h'] : list<?>
(False, 1, '2') : tuple<bool, int, str>
set([2.3, 1.2, 3.4]) : set<float>
[[1, 2, 3], [4, 5, 6], [7, 8, 9]] : list<list<int>>
[(1, 'a'), (2, 'b')] : list<tuple<int, str>>
{1: 'b', 2: 'c'} : dict<int, str>
[<__main__.Foo object at 0x101de6c50>] : list<Foo>
Run Code Online (Sandbox Code Playgroud)

问题是你想走多远,检查多深,并在速度和准确性之间进行权衡。例如,您想浏览一个大列表中的所有项目吗?您想要处理自定义类型(并追踪这些类型的共同祖先)吗?

值得一读,虽然我不确定它是否适用,这个关于类型提示的 PEP 。