我想使用 bash 脚本语言对结构化文本进行操作。然而,我的知识使这项任务非常具有挑战性。
输入样本:
"4-QUEIJOS": Mucarela Provolone Catupiry Ricota Oregano
"A-MODA": Mucarela Presunto Calabresa Bacon Tomate Milho Oregano
"ALHO-E-OLEO": Mucarela Alho oleo Oregano
"PEITO-DE-PERU-ESPECIAL": Mucarela Peito-de-Peru Catupiry Oregano
Run Code Online (Sandbox Code Playgroud)
输出样本
"4-QUEIJOS": ["mucarela", "provolone", "catupiry", "ricota", "oregano"],
"A-MODA": ["mucarela", "presunto", "calabresa", "bacon", "tomate", "milho", "oregano"],
"ALHO-E-OLEO": ["mucarela", "alho", "oleo", "oregano"],
"PEITO-DE-PERU-ESPECIAL": ["Mucarela", "peito-de-peru", "catupiry", "oregano"]
Run Code Online (Sandbox Code Playgroud)
正如您在上面看到的,我们需要:
顶部樱桃是除最后一行之外每行末尾的逗号。
使用sed
$ sed -E ':a;s/([^ ]*) ([^ ]*)/\1"\2",/;ta;s/(:)(.*")/\1 [\L\2]/;$s/,$//;s/,/& /g' input_file
"4-QUEIJOS": ["mucarela", "provolone", "catupiry", "ricota", "oregano"],
"A-MODA": ["mucarela", "presunto", "calabresa", "bacon", "tomate", "milho", "oregano"],
"ALHO-E-OLEO": ["mucarela", "alho", "oleo", "oregano"],
"PEITO-DE-PERU-ESPECIAL": ["mucarela", "peito-de-peru", "catupiry", "oregano"]
Run Code Online (Sandbox Code Playgroud)
小智 5
我只是决定要深入研究 sed,更具体地说,是为了了解上面答案中 @HatLess 的 sed 功夫。我运行了发布的命令,--debug并花了更多时间研究正则表达式和其他 sed-isms。得到一个解决问题的答案是一回事,了解到底发生了什么以及你做了什么才能得到答案是另一回事 - 所以这是我对上述答案的逐一解释...因为我不满足于仅仅记住公式或模式!
揭开引擎盖看看香肠是如何制作的,这是真正理解这些东西的唯一方法,尤其是像 sed 这样的东西。这就像学习音乐的基础知识一样,一旦弄清楚了模式,创作自己的交响乐并不是一件遥不可及的事情。
让我们分解这个 sed 命令/脚本并逐步进行:
sed --debug -E ':a;s/([^ ]*) ([^ ]*)/\1"\2",/;ta;s/(:)(.*")/\1 [\L\2]/;$s/,$//;s/,/& /g' input_file
请注意,单引号括住 sed 将运行的命令集,并用分号分隔 sed 命令。
注释程序执行
--debug
使用扩展正则表达式
-E
标记要跳转到迭代的位置
:a是一个可以返回使用的标签t a
向值添加引号和逗号
s/([^ ]*) ([^ ]*)/\1"\2",/
最初匹配"4-QUEIJOS": Mucarela模式
"4-QUEIJOS":Mucarela删除空格似乎是一个聪明的“技巧”,因此下一次迭代可以将捕获组 2 设置为下一个单词,依此类推,直到所有值都正确格式化......然后稍后将空格添加回来。
s # sed substitute command, (i.e.: s/old/new/)
/ # start search pattern from here
( # start a capture group (group 1)
[^ # begin negated set (inversely matches set)
# space character
] # end (negated) set (or, set of non-space chars)
* # select as many of these sets as found
) # end capture group (group 1)
# space between first and second capture group
( # begin capture group (group 2)
[^ # begin negated set (inversely matches set)
# space character
] # end (negated) set (or, set of non-space chars)
* # select as many of these sets as found
) # end capture group (group 2)
/ # replace above found items with items below
\1 # represents string in capture group 1
"\2", # surround group 2 w/ quotes and trailing comma
/ # end the replacement
Run Code Online (Sandbox Code Playgroud)
分支到标签“a”...
t a # if above match was successful, jump back to
# position label 'a' (start from the beginning)
# replacing the second group pattern with "x",
# until there are no more matches.. then go on
Run Code Online (Sandbox Code Playgroud)
恢复单词之间的空格并使值小写
s/(:)(.*")/\1 [\L\2]/
# matches ':"Mucarela","Provolone","Catupiry","Ricota","Oregano"'
# group 1 = ':'
# group 2 = '"Mucarela","Provolone","Catupiry","Ricota","Oregano"'
s # sed substitute command, (i.e.: s/old/new/)
/ # start search pattern from here
( # start a capture group (group 1)
: # look for the colon char
) # end capture group (group 1)
( # start a capture group (group 2)
. # match any char, including space
*" # match any number of chars up to last quote
) # end capture group (group 2)
/ # replace above found groups with items below
\1 # represents string in capture group 1
# output a space after the first item(s)
[\L\2] # set group2 lowercase + surround w/ brackets
/ # end the replacement
Run Code Online (Sandbox Code Playgroud)
删除最后一行的最后一个逗号
$ s/,$//
# matches ',' at the end of the last line read from the file and removes it
$ # match on the last line in the input file
s # sed substitute command, (i.e.: s/old/new/)
/ # start search pattern from here
,$ # match comma at end of line
// # replace with nothing (delete)
Run Code Online (Sandbox Code Playgroud)
将所有逗号替换为逗号和空格
s/,/& /g
# matches ',' and replaces with `, `
s # sed substitute command, (i.e.: s/old/new/)
/,/ # match on a comma
& / # replace comma with itself (comma) and space
g # do this for all commas on the line
Run Code Online (Sandbox Code Playgroud)
现在是逐个播放(为了简洁起见,仅显示处理的第一行和最后一行的一部分)。在本练习中,我将原始数据添加到名为“input_file”的文件中,并对其运行 sed 命令,就像上面提供的答案一样。
--debug 输出的前几行
这显示了 sed 解释的命令(上面详细描述了)
SED PROGRAM:
:a
s/([^ ]*) ([^ ]*)/\1"\2",/
t a
s/(:)(.*")/\1 [\L\2]/
$ s/,$//
s/,/& /g
Run Code Online (Sandbox Code Playgroud)
从输入文件中读入第一行数据
INPUT: 'input_file' line 1
Run Code Online (Sandbox Code Playgroud)
其余的内容在某种程度上是不言自明的,或者我添加了从 sed 手册页引用的“注释”。
# pattern to be operated on from the input file
PATTERN: "4-QUEIJOS": Mucarela Provolone Catupiry Ricota Oregano
# :a is a label for 'b' and 't' commands
# 'b a' means to branch to label 'a', unconditionally.
# if 'a' is omitted, branch to end of script.
# 't a' means to branch to label 'a', conditioned on
# a s/// doing a successful substitution since the last
# input line was read and since the last t or T command,
# if label 'a' is omitted, branch to end of script.
COMMAND: :a
# look for this pattern and do the quotes and comma thing...
COMMAND: s/([^ ]*) ([^ ]*)/\1"\2",/
MATCHED REGEX REGISTERS
regex[0] = 0-21 '"4-QUEIJOS": Mucarela'
regex[1] = 0-12 '"4-QUEIJOS":'
regex[2] = 13-21 'Mucarela'
Run Code Online (Sandbox Code Playgroud)
接下来-再做一次
# the above produced this as an output
PATTERN: "4-QUEIJOS":"Mucarela", Provolone Catupiry Ricota Oregano
# because s/// did a successful substitution since the last input line
# was read and since the last t or T command, branch to label 'a'
COMMAND: t a
# starting back at label 'a' for another iteration
COMMAND: :a
COMMAND: s/([^ ]*) ([^ ]*)/\1"\2",/
MATCHED REGEX REGISTERS
regex[0] = 0-33 '"4-QUEIJOS":"Mucarela", Provolone'
regex[1] = 0-23 '"4-QUEIJOS":"Mucarela",'
regex[2] = 24-33 'Provolone'
Run Code Online (Sandbox Code Playgroud)
另一个迭代...另一个替代
# the above produced this as an output
PATTERN: "4-QUEIJOS":"Mucarela","Provolone", Catupiry Ricota Oregano
# branch to label 'a' for another iteration
COMMAND: t a
COMMAND: :a
COMMAND: s/([^ ]*) ([^ ]*)/\1"\2",/
MATCHED REGEX REGISTERS
regex[0] = 0-44 '"4-QUEIJOS":"Mucarela","Provolone", Catupiry'
regex[1] = 0-35 '"4-QUEIJOS":"Mucarela","Provolone",'
regex[2] = 36-44 'Catupiry'
Run Code Online (Sandbox Code Playgroud)
...
# the above produced this as an output
PATTERN: "4-QUEIJOS":"Mucarela","Provolone","Catupiry", Ricota Oregano
# branch to label 'a' for another iteration
COMMAND: t a
COMMAND: :a
COMMAND: s/([^ ]*) ([^ ]*)/\1"\2",/
MATCHED REGEX REGISTERS
regex[0] = 0-53 '"4-QUEIJOS":"Mucarela","Provolone","Catupiry", Ricota'
regex[1] = 0-46 '"4-QUEIJOS":"Mucarela","Provolone","Catupiry",'
regex[2] = 47-53 'Ricota'
Run Code Online (Sandbox Code Playgroud)
说完最后一句话..
#... last word of the line to add quotes and a comma to coming right up
PATTERN: "4-QUEIJOS":"Mucarela","Provolone","Catupiry","Ricota", Oregano
COMMAND: t a
COMMAND: :a
COMMAND: s/([^ ]*) ([^ ]*)/\1"\2",/
MATCHED REGEX REGISTERS
regex[0] = 0-63 '"4-QUEIJOS":"Mucarela","Provolone","Catupiry","Ricota", Oregano'
regex[1] = 0-55 '"4-QUEIJOS":"Mucarela","Provolone","Catupiry","Ricota",'
regex[2] = 56-63 'Oregano'
Run Code Online (Sandbox Code Playgroud)
进行了替换..再次分支...
PATTERN: "4-QUEIJOS":"Mucarela","Provolone","Catupiry","Ricota","Oregano",
COMMAND: t a
COMMAND: :a
COMMAND: s/([^ ]*) ([^ ]*)/\1"\2",/
Run Code Online (Sandbox Code Playgroud)
自上一个分支以来没有更多替换..下一次检查将继续
PATTERN: "4-QUEIJOS":"Mucarela","Provolone","Catupiry","Ricota","Oregano",
COMMAND: t a
# did not branch back to a... now let's enclose the values in a list/ brackets
COMMAND: s/(:)(.*")/\1 [\L\2]/
MATCHED REGEX REGISTERS
regex[0] = 11-64 ':"Mucarela","Provolone","Catupiry","Ricota","Oregano"'
regex[1] = 11-12 ':'
regex[2] = 12-64 '"Mucarela","Provolone","Catupiry","Ricota","Oregano"'
Run Code Online (Sandbox Code Playgroud)
继续
# last command produced this... good job
PATTERN: "4-QUEIJOS": ["mucarela","provolone","catupiry","ricota","oregano"],
# is the last line in the file, remove last comma
COMMAND: $ s/,$//
# no match for this one, must not be the last line from file.. moving on
# look for commas and add a space after them
COMMAND: s/,/& /g
MATCHED REGEX REGISTERS
regex[0] = 24-25 ','
# result
PATTERN: "4-QUEIJOS": ["mucarela", "provolone", "catupiry", "ricota", "oregano"],
Run Code Online (Sandbox Code Playgroud)
你能看一下吗……一切都在这条线上完成了!
END-OF-CYCLE:
"4-QUEIJOS": ["mucarela", "provolone", "catupiry", "ricota", "oregano"],
Run Code Online (Sandbox Code Playgroud)
从要迭代的文件中读取的新行...
INPUT: 'input_file' line 2
PATTERN: "A-MODA": Mucarela Presunto Calabresa Bacon Tomate Milho Oregano
Run Code Online (Sandbox Code Playgroud)
重复循环直到我们到达最后一行的最后一部分......
# result from previous operation
PATTERN: "PEITO-DE-PERU-ESPECIAL": ["mucarela","peito-de-peru","catupiry","oregano"],
# are we on the last line in the file? yes? k, remove comma at end of line
COMMAND: $ s/,$//
MATCHED REGEX REGISTERS
regex[0] = 75-76 ','
Run Code Online (Sandbox Code Playgroud)
很好 - 最后一行缺少行尾逗号 - 只需添加空格
PATTERN: "PEITO-DE-PERU-ESPECIAL": ["mucarela","peito-de-peru","catupiry","oregano"]
# check for commas, and replace ',' with ', '
COMMAND: s/,/& /g
MATCHED REGEX REGISTERS
regex[0] = 37-38 ','
PATTERN: "PEITO-DE-PERU-ESPECIAL": ["mucarela", "peito-de-peru", "catupiry", "oregano"]
Run Code Online (Sandbox Code Playgroud)
就在那里……最后一行。
END-OF-CYCLE:
"PEITO-DE-PERU-ESPECIAL": ["mucarela", "peito-de-peru", "catupiry", "oregano"]
Run Code Online (Sandbox Code Playgroud)