将重复的元素删除到文件中

Question

你好，我有一个巨大的文件，这是一个头部：

>Sequence1:p
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
>Sequence2:ok
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
>Sequence3/lo
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
>Sequence:LJ
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
>Sequence3/lo
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
ATTGGAGAGA
>Sequence:YU
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
ATTAGAG
>Sequence:LJ
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
ATTGGAGAGA
....

如您所见，该文件由几个序列组成：

它们总是以它的名字 : 开头，>name 后跟字母。

在这里，我想通过名称删除重复的序列

在示例中：

>Sequence:LJ并且>Sequence3/lo出现两次。

然后我只想继续并获得一个没有重复序列的新文件：

>Sequence1:p
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
>Sequence2:ok
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
>Sequence3/lo
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
>Sequence:LJ
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
ATTGGAGAGA
>Sequence:YU
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
ATTAGAG

有人对 bash 代码或其他东西有什么想法吗？

Answer 1

$ awk '/>/{c=a[$1]++}!c' file
>Sequence1:p
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
>Sequence2:ok
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
>Sequence3/lo
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
>Sequence:LJ
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
>Sequence:YU
AAAAAAACCCCCTTTGGGGAGAGAGAGGAACACAGATAATGATAAGTAGATATGATTATAGTAG
CAGAYAGTATGAGTAGTAAGTGAATTAGTAGTAGTAGATGATGA
ATTAGAG