使用Android NDK中的Libavfilter库实现多输入过滤器图形

Question

我正在尝试使用overlay具有多个输入源的过滤器,用于Android应用.基本上,我想在静态图像上叠加多个视频源.我查看了ffmpeg附带的示例并基于此实现了我的代码,但事情似乎没有按预期工作.

在ffmpeg过滤示例中,似乎只有一个视频输入.我必须处理多个视频输入,我不确定我的解决方案是否正确.我试图找到其他例子,但看起来这是唯一的例子.

这是我的代码:

AVFilterContext **inputContexts;
AVFilterContext *outputContext;
AVFilterGraph *graph;

int initFilters(AVFrame *bgFrame, int inputCount, AVCodecContext **codecContexts, char *filters)
{
    int i;
    int returnCode;
    char args[512];
    char name[9];
    AVFilterInOut **graphInputs = NULL;
    AVFilterInOut *graphOutput = NULL;

    AVFilter *bufferSrc  = avfilter_get_by_name("buffer");
    AVFilter *bufferSink = avfilter_get_by_name("buffersink");

    graph = avfilter_graph_alloc();
    if(graph == NULL)
        return -1;

    //allocate inputs
    graphInputs = av_calloc(inputCount + 1, sizeof(AVFilterInOut *));
    for(i = 0; i <= inputCount; i++)
    {
        graphInputs[i] = avfilter_inout_alloc();
        if(graphInputs[i] == NULL)
            return -1;
    }

    //allocate input contexts
    inputContexts = av_calloc(inputCount + 1, sizeof(AVFilterContext *));
    //first is the background
    snprintf(args, sizeof(args), "video_size=%dx%d:pix_fmt=%d:time_base=1/1:pixel_aspect=0", bgFrame->width, bgFrame->height, bgFrame->format);
    returnCode = avfilter_graph_create_filter(&inputContexts[0], bufferSrc, "background", args, NULL, graph);
    if(returnCode < 0)
        return returnCode;
    graphInputs[0]->filter_ctx = inputContexts[0];
    graphInputs[0]->name = av_strdup("background");
    graphInputs[0]->next = graphInputs[1];

    //allocate the rest
    for(i = 1; i <= inputCount; i++)
    {
        AVCodecContext *codecCtx = codecContexts[i - 1];
        snprintf(args, sizeof(args), "video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
                    codecCtx->width, codecCtx->height, codecCtx->pix_fmt,
                    codecCtx->time_base.num, codecCtx->time_base.den,
                    codecCtx->sample_aspect_ratio.num, codecCtx->sample_aspect_ratio.den);
        snprintf(name, sizeof(name), "video_%d", i);

        returnCode = avfilter_graph_create_filter(&inputContexts[i], bufferSrc, name, args, NULL, graph);
        if(returnCode < 0)
            return returnCode;

        graphInputs[i]->filter_ctx = inputContexts[i];
        graphInputs[i]->name = av_strdup(name);
        graphInputs[i]->pad_idx = 0;
        if(i < inputCount)
        {
            graphInputs[i]->next = graphInputs[i + 1];
        }
        else
        {
            graphInputs[i]->next = NULL;
        }
    }

    //allocate outputs
    graphOutput = avfilter_inout_alloc();   
    returnCode = avfilter_graph_create_filter(&outputContext, bufferSink, "out", NULL, NULL, graph);
    if(returnCode < 0)
        return returnCode;
    graphOutput->filter_ctx = outputContext;
    graphOutput->name = av_strdup("out");
    graphOutput->next = NULL;
    graphOutput->pad_idx = 0;

    returnCode = avfilter_graph_parse_ptr(graph, filters, graphInputs, &graphOutput, NULL);
    if(returnCode < 0)
        return returnCode;

    returnCode = avfilter_graph_config(graph, NULL);
        return returnCode;

    return 0;
}

filters函数的参数传递给avfilter_graph_parse_ptr它,它看起来像这样:[background] scale=512x512 [base]; [video_1] scale=256x256 [tmp_1]; [base][tmp_1] overlay=0:0 [out]

在调用avfilter_graph_config警告之后调用中断: Output pad "default" with type video of the filter instance "background" of buffer not connected to any destination和错误Invalid argument.

我做得不对的是什么？

编辑:这是我发现的两个问题:

1. 看起来描述avfilter_graph_parse_ptr有点模糊.该ouputs参数表示图表的当前输出列表,在我的情况下是graphInputs变量,因为这些是buffer过滤器的输出.该inputs参数表示图表的当前输入的列表,在这种情况下,这是graphOutput变量,因为它表示buffersink过滤器的输入.

2. 我scale用过滤器和单个输入做了一些测试.似乎需要的AVFilterInOut结构名称.我试过不同的版本:, . 他们都没有工作,我也找不到任何与此相关的文件.avfilter_graph_parse_ptrinin_1in_link_1

所以问题仍然存在.如何实现具有多个输入的过滤器图形？

Answer 1

我找到了一个解决问题的简单方法.这涉及替换avfilter_graph_parse_ptrwith avfilter_graph_parse2并将the buffer和buffersinkfilters 添加到filters参数中avfilter_graph_parse2.

因此,在您有一个背景图像和一个输入视频的简单情况下,filters参数的值应如下所示:

buffer=video_size=1024x768:pix_fmt=2:time_base=1/25:pixel_aspect=3937/3937 [in_1]; buffer=video_size=1920x1080:pix_fmt=0:time_base=1/180000:pixel_aspect=0/1 [in_2]; [in_1] [in_2] overlay=0:0 [result]; [result] buffersink

在avfilter_graph_parse2将所有的图形连接和初始化所有的过滤器.输入缓冲区和输出缓冲区的过滤器上下文可以在结尾处从图形本身中检索.这些用于从过滤器图形中添加/获取帧.

代码的简化版本如下所示:

AVFilterContext **inputContexts;
AVFilterContext *outputContext;
AVFilterGraph *graph;

int initFilters(AVFrame *bgFrame, int inputCount, AVCodecContext **codecContexts)
{
    int i;
    int returnCode;
    char filters[1024];
    AVFilterInOut *gis = NULL;
    AVFilterInOut *gos = NULL;

    graph = avfilter_graph_alloc();
    if(graph == NULL)
    {
        printf("Cannot allocate filter graph.");        
        return -1;
    }

    //build the filters string here
    // ...

    returnCode = avfilter_graph_parse2(graph, filters, &gis, &gos);
    if(returnCode < 0)
    {
        cs_printAVError("Cannot parse graph.", returnCode);
        return returnCode;
    }

    returnCode = avfilter_graph_config(graph, NULL);
    if(returnCode < 0)
    {
        cs_printAVError("Cannot configure graph.", returnCode);
        return returnCode;
    }

    //get the filter contexts from the graph here

    return 0;
}