将地面真实掩模覆盖在图像上

Question

在我的项目中，我从视频中提取了帧，在另一个文件夹中，我有每个帧的地面实况。\n我想将视频每帧的地面实况图像（在我的例子中，它是显着性预测地面实况）映射到其相关帧图像。作为示例，我有以下框架：

\n

\n

以下是真实掩码：

\n

\n

以下是真实值在框架上的映射。

\n

\n

我怎样才能做到这一点。另外，我有两个文件夹，每个文件夹内都有几个文件夹，每个文件夹内都有存储的帧。如何对这些批量数据进行操作？

\n

这是我的文件夹的层次结构：

\n

框架_文件夹：文件夹_1、文件夹_2、......

\n

\xe2\x94\x9c\xe2\x94\x80\xe2\x94\x80 frames\n\xe2\x94\x82   \xe2\x94\x9c\xe2\x94\x80\xe2\x94\x80 601   (601 and 602  and etc are folders that in the inside there are image frames that their name is like 0001.png,0002.png, ...)\n\xe2\x94\x82   \xe2\x94\x9c\xe2\x94\x80\xe2\x94\x80 602\n       .\n       .\n       .\n\xe2\x94\x82   \xe2\x94\x94\xe2\x94\x80\xe2\x94\x80 700\n\n\n \xe2\x94\x9c\xe2\x94\x80\xe2\x94\x80 ground truth\n    \xe2\x94\x82   \xe2\x94\x9c\xe2\x94\x80\xe2\x94\x80 601   (601 and 602  and etc are folders that in the inside there are ground truth masks that their name is like 0001.png,0002.png, ...)\n    \xe2\x94\x82   \xe2\x94\x9c\xe2\x94\x80\xe2\x94\x80 602\n           .\n           .\n           .\n    \xe2\x94\x82   \xe2\x94\x94\xe2\x94\x80\xe2\x94\x80 700\n

\n

\n

更新：\n使用@hk Chengrex提出的答案，我遇到了错误。当路径中只有一个文件夹时，它工作得很好，但是当我根据我面临的问题放置多个文件夹（不同视频的帧）时，出现以下错误。详细信息如下：

\n

\n

 multiprocessing.pool.RemoteTraceback: \n"""\nTraceback (most recent call last):\n  File "/home/user/miniconda3/envs/vtn/lib/python3.10/multiprocessing/pool.py", line 125, in worker\n    result = (True, func(*args, **kwds))\nTypeError: process_video() takes 1 positional argument but 6 were given\n"""\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File "/home/user/Video_processing/Saliency_mapping.py", line 69, in <module>\n    pool.apply(process_video, videos)\n  File "/home/user/miniconda3/envs/vtn/lib/python3.10/multiprocessing/pool.py", line 357, in apply\n    return self.apply_async(func, args, kwds).get()\n  File "/home/user/miniconda3/envs/vtn/lib/python3.10/multiprocessing/pool.py", line 771, in get\n    raise self._value\nTypeError: process_video() takes 1 positional argument but 6 were given\n

\n

Answer 1

我需要经常做类似的事情。按照我最喜欢的 StackOverflow 风格，这里有一个可以复制和粘贴的脚本。我希望代码本身是不言自明的。您可以调整和尝试一些内容（例如，颜色贴图、叠加样式）。它multiprocessing.Pool用于更快的批处理，调整蒙版大小以匹配图像的形状，假设蒙版格式正确.png，并取决于您发布的文件结构。

import os
from os import path
import cv2
import numpy as np

from argparse import ArgumentParser
from multiprocessing import Pool


def create_overlay(image, mask):
    """
    image: H*W*3 numpy array
    mask: H*W numpy array
    If dimensions do not match, the mask is upsampled to match that of the image

    Returns a H*W*3 numpy array
    """
    h, w = image.shape[:2]
    mask = cv2.resize(mask, dsize=(w,h), interpolation=cv2.INTER_CUBIC)

    # color options: https://docs.opencv.org/4.x/d3/d50/group__imgproc__colormap.html
    mask_color = cv2.applyColorMap(mask, cv2.COLORMAP_HOT).astype(np.float32)
    mask = mask[:, :, None] # create trailing dimension for broadcasting
    mask = mask.astype(np.float32)/255

    # different other options that you can use to merge image/mask
    overlay = (image*(1-mask)+mask_color*mask).astype(np.uint8)
    # overlay = (image*0.5 + mask_color*0.5).astype(np.uint8)
    # overlay = (image + mask_color).clip(0,255).astype(np.uint8)

    return overlay

def process_video(video_name):
    """
    Processing frames in a single video
    """
    vid_image_path = path.join(image_path, video_name)
    vid_mask_path = path.join(mask_path, video_name)
    vid_output_path = path.join(output_path, video_name)
    os.makedirs(vid_output_path, exist_ok=True)

    frames = sorted(os.listdir(vid_image_path))
    for f in frames:
        image = cv2.imread(path.join(vid_image_path, f))
        mask = cv2.imread(path.join(vid_mask_path, f.replace('.jpg','.png')), cv2.IMREAD_GRAYSCALE)
        overlay = create_overlay(image, mask)
        cv2.imwrite(path.join(vid_output_path, f), overlay)


parser = ArgumentParser()
parser.add_argument('--image_path')
parser.add_argument('--mask_path')
parser.add_argument('--output_path')
args = parser.parse_args()

image_path = args.image_path
mask_path = args.mask_path
output_path = args.output_path

if __name__ == '__main__':
    videos = sorted(
        list(set(os.listdir(image_path)).intersection(
                set(os.listdir(mask_path))))
    )

    print(f'Processing {len(videos)} videos.')

    pool = Pool()
    pool.map(process_video, videos)

    print('Done.')

输出：

编辑：使其在 Windows 上运行；变成。pool.apply​pool.map