假设我有100000个电子邮件正文,其中2000个包含一个像"快速的棕色狐狸跳过懒狗"或"lorem ipsum dolor sit amet"这样的公共字符串.我可以/应该使用哪些技术来"挖掘"这些短语？我对挖掘单个单词或短语并不感兴趣.此外,我需要过滤掉我已经知道在所有邮件中出现的短语.
例:
string mailbody1 = "Welcome to the world of tomorrow! This is the first mail body. Lorem ipsum dolor sit AMET. Have a nice day dude. Cya!";
string mailbody2 = "Welcome to the world of yesterday! Lorem ipsum dolor sit amet Please note this is the body of the second mail. Have a nice day.";
string mailbody3 = "A completely different body.";
string[] mailbodies = new[] {mailbody1, mailbody2, mailbody3};
string[] ignoredPhrases = new[] {"Welcome to …