字符串列表中字符串出现的双重列表理解

Question

我有两个列表列表：

text = [['hello this is me'], ['oh you know u']]
phrases = [['this is', 'u'], ['oh you', 'me']]

我需要拆分文本，使短语中出现的单词组合成为单个字符串：

result = [['hello', 'this is', 'me'], ['oh you', 'know', 'u']

我尝试使用 zip() 但它连续遍历列表，而我需要检查每个列表。我还尝试了一个 find() 方法，但从这个例子中它也会找到所有字母 'u' 并将它们变成一个字符串（就像在单词 'you' 中它使它变成 'yo', 'u'）。我希望 replace() 在用列表替换字符串时也能工作，因为它可以让我做类似的事情：

for line in text:
        line = line.replace('this is', ['this is'])

但是尝试了所有方法，在这种情况下，我仍然没有找到任何适合我的方法。你能帮我解决这个问题吗？

Answer 1

与原始海报澄清：

给定文本 pack my box with five dozen liquor jugs 和短语five dozen

结果应该是：

['pack', 'my', 'box', 'with', 'five dozen', 'liquor', 'jugs']

不是：

['pack my box with', 'five dozen', 'liquor jugs']

每个文本和短语都会转换为 Python 单词列表['this', 'is', 'an', 'example']，以防止“u”在单词内匹配。

文本的所有可能的子短语均由 生成compile_subphrases()。首先生成较长的短语（更多单词），因此它们先于较短的短语进行匹配。'five dozen jugs'总是优先于'five dozen'or进行匹配'five'。

使用列表切片来比较短语和子短语，大致如下：

    text = ['five', 'dozen', 'liquor', 'jugs']
    phrase = ['liquor', 'jugs']
    if text[2:3] == phrase:
        print('matched')

使用这种比较短语的方法，脚本会遍历原始文本，并用挑选出的短语重写它。

texts = [['hello this is me'], ['oh you know u']]
phrases_to_match = [['this is', 'u'], ['oh you', 'me']]
from itertools import chain

def flatten(list_of_lists):
    return list(chain(*list_of_lists))

def compile_subphrases(text, minwords=1, include_self=True):
    words = text.split()
    text_length = len(words)
    max_phrase_length = text_length if include_self else text_length - 1
    # NOTE: longest phrases first
    for phrase_length in range(max_phrase_length + 1, minwords - 1, -1):
        n_length_phrases = (' '.join(words[r:r + phrase_length])
                            for r in range(text_length - phrase_length + 1))
        yield from n_length_phrases
        
def match_sublist(mainlist, sublist, i):
    if i + len(sublist) > len(mainlist):
        return False
    return sublist == mainlist[i:i + len(sublist)]

phrases_to_match = list(flatten(phrases_to_match))
texts = list(flatten(texts))
results = []
for raw_text in texts:
    print(f"Raw text: '{raw_text}'")
    matched_phrases = [
        subphrase.split()
        for subphrase
        in compile_subphrases(raw_text)
        if subphrase in phrases_to_match
    ]
    phrasal_text = []
    index = 0
    text_words = raw_text.split()
    while index < len(text_words):
        for matched_phrase in matched_phrases:
            if match_sublist(text_words, matched_phrase, index):
                phrasal_text.append(' '.join(matched_phrase))
                index += len(matched_phrase)
                break
        else:
            phrasal_text.append(text_words[index])
            index += 1
    results.append(phrasal_text)
print(f'Phrases to match: {phrases_to_match}')
print(f"Results: {results}")

结果：

$python3 main.py
Raw text: 'hello this is me'
Raw text: 'oh you know u'
Phrases to match: ['this is', 'u', 'oh you', 'me']
Results: [['hello', 'this is', 'me'], ['oh you', 'know', 'u']]

要使用更大的数据集测试此答案和其他答案，请在代码开头尝试此操作。它会在单个长句子上生成数百个变体来模拟数百个文本。

from itertools import chain, combinations
import random

#texts = [['hello this is me'], ['oh you know u']]
theme = ' '.join([
    'pack my box with five dozen liquor jugs said',
    'the quick brown fox as he jumped over the lazy dog'
])
variations = list([
    ' '.join(combination)
    for combination
    in combinations(theme.split(), 5)
])
texts = random.choices(variations, k=500)
#phrases_to_match = [['this is', 'u'], ['oh you', 'me']]
phrases_to_match = [
    ['pack my box', 'quick brown', 'the quick', 'brown fox'],
    ['jumped over', 'lazy dog'],
    ['five dozen', 'liquor', 'jugs']
]