从python中的列表中删除重复的JSON对象

Question

我有一个dict列表,其中特定值重复多次,我想删除重复的值.

我的列表:

te = [
      {
        "Name": "Bala",
        "phone": "None"
      },
      {
        "Name": "Bala",
        "phone": "None"
      },
      {
        "Name": "Bala",
        "phone": "None"
      },
      {
        "Name": "Bala",
        "phone": "None"
      }
    ]

用于删除重复值的函数:

def removeduplicate(it):
    seen = set()
    for x in it:
        if x not in seen:
            yield x
            seen.add(x)

当我调用这个函数时,我得到了generator object.

<generator object removeduplicate at 0x0170B6E8>

当我尝试迭代生成器时,我得到了 TypeError: unhashable type: 'dict'

有没有办法删除重复值或迭代生成器

Answer 1

您可以通过字典理解轻松删除重复键,因为字典不允许重复键,如下所示 -

te = [
      {
        "Name": "Bala",
        "phone": "None"
      },
      {
        "Name": "Bala",
        "phone": "None"
      },
      {
        "Name": "Bala",
        "phone": "None"
      },
      {
        "Name": "Bala",
        "phone": "None"
      },
      {
          "Name": "Bala1",
          "phone": "None"
      }      
    ]

unique = { each['Name'] : each for each in te }.values()

print unique

输出 -

[{'phone': 'None', 'Name': 'Bala1'}, {'phone': 'None', 'Name': 'Bala'}]

Answer 2

因为你不能添加dict到set. 从这个问题：

您正在尝试将 adict用作另一个dict或在set. 这不起作用，因为键必须是可散列的。

作为一般规则，只有不可变对象（字符串、整数、浮点数、冻结集、不可变元组）是可散列的（尽管可能有例外）。

>>> foo = dict()
>>> bar = set()
>>> bar.add(foo)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: unhashable type: 'dict'
>>> 

相反，您已经在使用if x not in seen，因此只需使用列表：

>>> te = [
...       {
...         "Name": "Bala",
...         "phone": "None"
...       },
...       {
...         "Name": "Bala",
...         "phone": "None"
...       },
...       {
...         "Name": "Bala",
...         "phone": "None"
...       },
...       {
...         "Name": "Bala",
...         "phone": "None"
...       }
...     ]

>>> def removeduplicate(it):
...     seen = []
...     for x in it:
...         if x not in seen:
...             yield x
...             seen.append(x)

>>> removeduplicate(te)
<generator object removeduplicate at 0x7f3578c71ca8>

>>> list(removeduplicate(te))
[{'phone': 'None', 'Name': 'Bala'}]
>>>