如何查找嵌套列表中所有列表共有的元素？

Question

我有一个大型嵌套列表,嵌套列表中的每个列表都包含一个格式为浮点数的列表.但是,除少数例外情况外,嵌套列表中的每个列表都是相同的.我想提取嵌套列表中所有列表共有的数字.我的问题的一个简单示例如下所示:

nested_list = [[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0],
              [2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0],
              [1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0],
              [2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0]]

在下面的例子中,我想提取以下内容:

common_vals = [2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0]

我尝试使用set intersectionctions来解决这个问题,但是因为我无法使用它来处理嵌套列表的所有元素.

Answer 1

你可以使用reduce和set.intersection:

>>> reduce(set.intersection, map(set, nested_list))
set([2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0])

使用itertools.imap的内存有效的解决方案.

时间比较:

>>> lis = [[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0],
              [2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0],
              [1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0],
              [2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0]]
>>> %timeit set.intersection(*map(set, lis))
100000 loops, best of 3: 12.5 us per loop
>>> %timeit set.intersection(*(set(e) for e in lis))
10000 loops, best of 3: 14.4 us per loop
>>> %timeit reduce(set.intersection, map(set, lis))
10000 loops, best of 3: 12.8 us per loop
>>> %timeit reduce(set.intersection, imap(set, lis))
100000 loops, best of 3: 13.1 us per loop
>>> %timeit set.intersection(set(lis[0]), *islice(lis, 1, None))
100000 loops, best of 3: 10.6 us per loop


>>> lis = [[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0],
              [2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0],
              [1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0],
              [2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0]]*1000
>>> %timeit set.intersection(*map(set, lis))
10 loops, best of 3: 16.4 ms per loop
>>> %timeit set.intersection(*(set(e) for e in lis))
10 loops, best of 3: 15.8 ms per loop
>>> %timeit reduce(set.intersection, map(set, lis))
100 loops, best of 3: 16.3 ms per loop
>>> %timeit reduce(set.intersection, imap(set, lis))
10 loops, best of 3: 13.8 ms per loop
>>> %timeit set.intersection(set(lis[0]), *islice(lis, 1, None))
100 loops, best of 3: 8.4 ms per loop


>>> lis = [[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0],              [2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0],
              [1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0],
              [2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0]]*10**5
>>> %timeit set.intersection(*map(set, lis))  
1 loops, best of 3: 1.92 s per loop
>>> %timeit set.intersection(*(set(e) for e in lis))
1 loops, best of 3: 2.17 s per loop
>>> %timeit reduce(set.intersection, map(set, lis))
1 loops, best of 3: 2.14 s per loop
>>> %timeit reduce(set.intersection, imap(set, lis))
1 loops, best of 3: 1.52 s per loop
>>> %timeit set.intersection(set(lis[0]), *islice(lis, 1, None))
1 loops, best of 3: 913 ms per loop

结论:

就效率而言,Steven Rumbalski的解决方案显然是最好的解决方案.

Answer 2

Ashwini Chaudhary的解决方案很优雅,但对于大型输入来说可能效率很低,因为它创建了许多中间集.如果你nested_list很大,请执行以下操作:

>>> set.intersection(set(nested_list[0]), *itertools.islice(nested_list, 1, None))
set([2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0])

Answer 3

试试这个,这是最简单的解决方案:

set.intersection(*map(set, nested_list))

或者,如果您更喜欢使用生成器表达式,那么就内存使用而言,这应该是一种更有效的解决方案:

set.intersection(*(set(e) for e in nested_list))