执行脚本的多个实例 - GNU Parallel

Question

我目前正在尝试使用GNU Parallel，但是我没有成功。安装非常简单。我的目标是运行 python 脚本youtube-dl的两个并行实例及其相应的参数。使用并行执行脚本的正确方法是什么？

平行线

parallel 'youtube-dl -w --no-warnings -o "/media/video1.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn"' 'youtube-dl -w --no-warnings -o "/media/video2.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla"'

如果单独运行命令而不并行运行，它会起作用：

youtube-dl -w --no-warnings -o "/media/video1.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn"

youtube-dl -w --no-warnings -o "/media/video2.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla"

Answer 1

对于只有两个流，只需在第一个命令末尾添加一个与号 (&) 就可以了，这样就更容易了。像这样：

youtube-dl -w --no-warnings -o "/media/video1.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn" &
youtube-dl -w --no-warnings -o "/media/video2.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla" &

# Wait for both downloads to finish
wait

如果你想使用 GNU Parallel...类似这样的东西 - 尽管我通常会被告知更好的方法...:-)

创建一个名为的文件work.txt，其中包含以下内容：

/media/video1.%(ext)s http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn
/media/video2.%(ext)s http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla

那么命令将是：

parallel --colsep " " youtube-dl -w --no-warnings -o {1} {2} < work.txt

另一种选择是简单地将两个命令放入一个文件中并将其发送到 的GNU Parallel标准输入，如下所示：

将此存储在work.txt

youtube-dl -w --no-warnings -o "/media/video1.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn"
youtube-dl -w --no-warnings -o "/media/video2.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla"

然后就跑

parallel < work.txt