根据共同的2列正确连接两个文件

Question

我有两个文件,我试图加入/合并基于列1和2.它们看起来像这样,file1(58210线)比file2(815530线)短得多,我想根据字段1和2索引找到这两个文件的交集:

file1:

2L      25753   33158
2L      28813   33158
2L      31003   33158
2L      31077   33161
2L      31279   33161
3L      32124   45339
3L      33256   45339
...

file2:

2L      20242   0.5     0.307692307692308
2L      22141   0.32258064516129        0.692307692307692
2L      24439   0.413793103448276       0.625
2L      24710   0.371428571428571       0.631578947368421
2L      25753   0.967741935483871       0.869565217391304
2L      28813   0.181818181818182       0.692307692307692
2L      31003   0.36    0.666666666666667
2L      31077   0.611111111111111       0.931034482758621
2L      31279   0.75    1
3L      32124   0.558823529411765       0.857142857142857
3L      33256   0.769230769230769       0.90625
...

我一直在使用以下几个命令,但最终得到不同数量的行:

awk 'FNR==NR{a[$1$2]=$3;next} {if($1$2 in a) print}' file1 file2 | wc -l
awk 'FNR==NR{a[$1$2]=$3;next} {if($1$2 in a) print}' file2 file1 | wc -l

我不确定为什么会发生这种情况,并且我在比较之前尝试过排序,以防我在任一文件中有重复的行(基于列1和2),但它似乎没有帮助.(对此原因的任何见解也表示赞赏)

我怎样才能合并文件,以便只有file2具有相应列的行,1并且2在file1打印3时file1添加了列,看起来像这样:

2L      25753   0.967741935483871       0.869565217391304    33158
2L      28813   0.181818181818182       0.692307692307692    33158
2L      31003   0.36    0.666666666666667    33158
2L      31077   0.611111111111111       0.931034482758621    33161
2L      31279   0.75    1    33161
3L      32124   0.558823529411765       0.857142857142857    45339
3L      33256   0.769230769230769       0.90625    45339

Answer 1

awk 'NR==FNR{a[$1,$2]=$3;next} ($1,$2) in a{print $0, a[$1,$2]}' file1 file2

看:

$ cat file1
2L      5753   33158
2L      8813   33158
2L      7885   33159
2L      1279   33159
2L      5095   33158
$
$ cat file2
2L      8813    0.6    1.2
2L      5762    0.4    0.5
2L      1279    0.5    0.9
$
$ awk 'NR==FNR{a[$1,$2]=$3;next} ($1,$2) in a{print $0, a[$1,$2]}' file1 file2
2L      8813    0.6    1.2 33158
2L      1279    0.5    0.9 33159
$

如果那不是您想要的,请澄清并发布更具代表性的样本输入/输出.

上述代码的注释版本提供了请求的解释:

awk ' # START SCRIPT

# IF the number of records read so far across all files is equal
#    to the number of records read so far in the current file, a
#    condition which can only be true for the first file read, THEN 
NR==FNR {

   # populate array "a" such that the value indexed by the first
   # 2 fields from this record in file1 is the value of the third
   # field from the first file.
   a[$1,$2]=$3

   # Move on to the next record so we don't do any processing intended
   # for records from the second file. This is like an "else" for the
   # NR==FNR condition.
   next

} # END THEN

# We only reach this part of the code if the above condition is false,
# i.e. if the current record is from file2, not from file1.

# IF the array index constructed from the first 2 fields of the current
#    record exist in array a, as would occur if these same values existed
#    in file1, THEN
($1,$2) in a {

   # print the current record from file2 followed by the value from file1
   # that occurred at field 3 of the record that had the same values for
   # field 1 and field 2 in file1 as the current record from file2.
   print $0, a[$1,$2]

} # END THEN

' file1 file2 # END SCRIPT

希望有所帮助.

Answer 2

如果您想逐行加入文件,请使用以下命令:

join -o 1.2,1.3,2.4,2.5,1.4 <(cat -n file1) <(cat -n file2)

当您更新问题时:

join -o 1.1,2.2,2.3,1.2 <(sed 's/[[:space:]]\+/@/' file1|sort) \
    <(sed 's/[[:space:]]\+/@/' file2|sort)|sed 's/@/\t/'

首先用一些非空格字符替换每行中的第一个分隔符,然后对两个输入文件进行排序.然后join用来进行实际的连接.过滤掉其输出以用空格替换非空格字符.

这是文件的输出,如下所示:

xyz]$ join -o 1.1,2.2,2.3,1.2 <(sed 's/[[:space:]]\+/@/' file1|sort) \
<(sed 's/[[:space:]]\+/@/' file2|sort)|sed 's/@/\t/'

2L  25753 0.967741935483871 0.869565217391304 33158
2L  28813 0.181818181818182 0.692307692307692 33158
2L  31003 0.36 0.666666666666667 33158
2L  31077 0.611111111111111 0.931034482758621 33161
2L  31279 0.75 1 33161
3L  32124 0.558823529411765 0.857142857142857 45339
3L  33256 0.769230769230769 0.90625 45339