如何从仅使用 GPU 访问创建的 IDXGISurface 中获取像素数据？

Question

概括地说，我想要完成的是捕获（部分）屏幕并将捕获的内容转换为数字图像格式。以下步骤概述了我认为的解决方案：

\n

\n
1. 设置Direct3D11CaptureFramePool并订阅其FrameArrived事件
\n
2. FrameArrived访问事件委托中的像素数据
\n
3. 将图像数据传递到Windows Imaging Component进行编码
\n

\n

我的问题在于步骤 2：虽然我可以获得捕获的帧，但获得对表面的 CPU 读取访问权限失败。这是我的FrameArrived事件委托实现（完整重现如下）：

\n

void on_frame_arrived(Direct3D11CaptureFramePool const& frame_pool, winrt::Windows::Foundation::IInspectable const&)\n{\n    if (auto const frame = frame_pool.TryGetNextFrame())\n    {\n        if (auto const surface = frame.Surface())\n        {\n            if (auto const interop = surface.as<::Windows::Graphics::DirectX::Direct3D11::IDirect3DDxgiInterfaceAccess>())\n            {\n                com_ptr<IDXGISurface> dxgi_surface { nullptr };\n                check_hresult(interop->GetInterface(IID_PPV_ARGS(&dxgi_surface)));\n\n                DXGI_MAPPED_RECT info = {};\n                // Fails with `E_INVALIDARG`\n                check_hresult(dxgi_surface->Map(&info, DXGI_MAP_READ));\n            }\n        }\n    }\n}\n

\n

调用Map()失败并显示E_INVALIDARG，调试层提供了额外的、有用的错误诊断：

\n

\n

DXGI 错误：IDXGISurface::Map：该对象不是使用允许 CPU 访问的 CPUAccess 标志创建的。[其他错误#42：]

\n

\n

那么，既然我知道出了什么问题，我该如何解决这个问题呢？具体来说，如何从仅使用 GPU 访问创建的表面中提取像素数据？

\n

---

\n

以下是完整的重现。它最初是使用“Windows 控制台应用程序 (C++/WinRT)”项目模板创建的。应用的唯一更改是“预编译头：使用 (/Yu)” \xe2\x86\x92 “预编译头：不使用预编译头”，以将其保留为单个文件。

\n

它创建一个命令行应用程序，该应用程序期望窗口句柄作为其唯一的参数（十进制、十六进制或八进制）。

\n

#include <winrt/Windows.Foundation.h>\n#include <winrt/Windows.Graphics.Capture.h>\n#include <winrt/Windows.Graphics.DirectX.Direct3D11.h>\n#include <winrt/Windows.Graphics.DirectX.h>\n\n#include <Windows.Graphics.Capture.Interop.h>\n#include <windows.graphics.capture.h>\n#include <windows.graphics.directx.direct3d11.interop.h>\n\n#include <Windows.h>\n#include <d3d11.h>\n#include <dxgi.h>\n\n#include <cstdint>\n#include <stdio.h>\n#include <string>\n\n\nusing namespace winrt;\nusing namespace winrt::Windows::Graphics::Capture;\nusing namespace winrt::Windows::Graphics::DirectX;\nusing namespace winrt::Windows::Graphics::DirectX::Direct3D11;\n\n\nvoid on_frame_arrived(Direct3D11CaptureFramePool const& frame_pool, winrt::Windows::Foundation::IInspectable const&)\n{\n    wprintf(L"Frame arrived.\\n");\n\n    if (auto const frame = frame_pool.TryGetNextFrame())\n    {\n        if (auto const surface = frame.Surface())\n        {\n            if (auto const interop = surface.as<::Windows::Graphics::DirectX::Direct3D11::IDirect3DDxgiInterfaceAccess>())\n            {\n                com_ptr<IDXGISurface> dxgi_surface { nullptr };\n                check_hresult(interop->GetInterface(IID_PPV_ARGS(&dxgi_surface)));\n\n                DXGI_MAPPED_RECT info = {};\n                // This is failing with `E_INVALIDARG`\n                check_hresult(dxgi_surface->Map(&info, DXGI_MAP_READ));\n            }\n        }\n    }\n}\n\n\nint wmain(int argc, wchar_t const* argv[])\n{\n    init_apartment(apartment_type::single_threaded);\n\n    // Validate input\n    if (argc != 2)\n    {\n        wprintf(L"Usage: %s <HWND>\\n", argv[0]);\n        return 1;\n    }\n    auto const target = reinterpret_cast<HWND>(static_cast<intptr_t>(std::stoi(argv[1], nullptr, 0)));\n\n    // Get `GraphicsCaptureItem` for `HWND`\n    auto interop = get_activation_factory<GraphicsCaptureItem, IGraphicsCaptureItemInterop>();\n\n    ::ABI::Windows::Graphics::Capture::IGraphicsCaptureItem* capture_item_abi { nullptr };\n    check_hresult(interop->CreateForWindow(target, IID_PPV_ARGS(&capture_item_abi)));\n    // Move raw pointer into smart pointer\n    GraphicsCaptureItem const capture_item { capture_item_abi, take_ownership_from_abi };\n\n    // Create D3D device and request the `IDXGIDevice` interface...\n    com_ptr<ID3D11Device> device = { nullptr };\n    check_hresult(::D3D11CreateDevice(nullptr, D3D_DRIVER_TYPE_HARDWARE, nullptr,\n                                      D3D11_CREATE_DEVICE_BGRA_SUPPORT | D3D11_CREATE_DEVICE_DEBUG, nullptr, 0,\n                                      D3D11_SDK_VERSION, device.put(), nullptr, nullptr));\n    auto dxgi_device = device.as<IDXGIDevice>();\n    // ... so that we can get an `IDirect3DDevice` (the capture frame pool\n    // speaks WinRT only)\n    com_ptr<IInspectable> d3d_device_interop { nullptr };\n    check_hresult(::CreateDirect3D11DeviceFromDXGIDevice(dxgi_device.get(), d3d_device_interop.put()));\n    auto d3d_device = d3d_device_interop.as<IDirect3DDevice>();\n\n    // Create a capture frame pool and capture session\n    auto const pool = Direct3D11CaptureFramePool::Create(d3d_device, DirectXPixelFormat::B8G8R8A8UIntNormalized, 1,\n                                                         capture_item.Size());\n    auto const session = pool.CreateCaptureSession(capture_item);\n    [[maybe_unused]] auto const event_guard = pool.FrameArrived(auto_revoke, &on_frame_arrived);\n\n    // Start capturing\n    session.StartCapture();\n\n    // Have the system spin up a message loop for us\n    ::MessageBoxW(nullptr, L"Stop capturing", L"Capturing...", MB_OK);\n}\n

\n

Answer 1

您必须创建一个可由CPU 访问的2D 纹理，并将源帧复制到此 2D 纹理中，然后您可以对其进行Map。例如：

void on_frame_arrived(Direct3D11CaptureFramePool const& frame_pool, winrt::Windows::Foundation::IInspectable const&)
{
  wprintf(L"Frame arrived.\n");

  if (auto const frame = frame_pool.TryGetNextFrame())
  {
    if (auto const surface = frame.Surface())
    {
      if (auto const interop = surface.as<::Windows::Graphics::DirectX::Direct3D11::IDirect3DDxgiInterfaceAccess>())
      {
        com_ptr<IDXGISurface> surface;
        check_hresult(interop->GetInterface(IID_PPV_ARGS(&surface)));

        // get surface dimensions
        DXGI_SURFACE_DESC desc;
        check_hresult(surface->GetDesc(&desc));

        // create a CPU-readable texture
        // note: for max perf, the texture creation
        // should be done once per surface size
        // or allocate a big enough texture (like adapter-sized) and copy portions
        D3D11_TEXTURE2D_DESC texDesc{};
        texDesc.Width = desc.Width;
        texDesc.Height = desc.Height;
        texDesc.ArraySize = 1;
        texDesc.CPUAccessFlags = D3D11_CPU_ACCESS_READ;
        texDesc.Format = DXGI_FORMAT_B8G8R8A8_UNORM;
        texDesc.MipLevels = 1;
        texDesc.SampleDesc.Count = 1;
        texDesc.Usage = D3D11_USAGE_STAGING;
        com_ptr<ID3D11Device> device;
        check_hresult(surface->GetDevice(IID_PPV_ARGS(&device))); // or get the one from D3D11CreateDevice

        com_ptr<ID3D11Texture2D> tex;
        check_hresult(device->CreateTexture2D(&texDesc, nullptr, tex.put()));

        com_ptr<ID3D11Resource> input;
        check_hresult(interop->GetInterface(IID_PPV_ARGS(&input)));

        com_ptr<ID3D11DeviceContext> dc;
        device->GetImmediateContext(dc.put()); // or get the one from D3D11CreateDevice

        // copy frame into CPU-readable resource
        // this and the Map call can be done at each frame
        dc->CopyResource(tex.get(), input.get());

        D3D11_MAPPED_SUBRESOURCE map;
        check_hresult(dc->Map(tex.get(), 0, D3D11_MAP_READ, 0, &map));

        // TODO do something with map

        dc->Unmap(tex.get(), 0);
      }
    }
  }
}