使用由Camera预览填充的SurfaceTexture作为Jelly Bean中的Renderscript输入分配

Question

在Jelly Bean中,是否可以使用由相机预览填充的SurfaceTexture创建Renderscript分配？我正在Android源代码树中构建我的应用程序,所以我可以使用@hide API,例如Allocation.setSurfaceTexture().但是我想避免使用RS Graphics弃用的API.这里的类似问题没有得到完全回答,也没有特定于JB的问题.

尝试以下代码时出现以下问题:

• 进入Renderscript的数据始终为零
• 对于重复调用的onFrameAvailable回调,我必须updateTexImage(),因为当我调用Allocation.ioReceive()时,它在第一次之后不再被回调,并且logcat中存在"无效的EGLDisplay".然而我认为ioReceive()是要走的路 - 它内部也是updateTexImage().
• 支持的分配类型包括RGBA8888但不包括NV21(这是相机预览格式),RS代码如何能够以这种方式格式化数据？

(我知道我正在使用的设备确实支持所请求的VGA分辨率).

public class SampleRSCPCActivity extends Activity implements SurfaceTexture.OnFrameAvailableListener {
final static int DO_KERNEL = 0;
private static final String TAG="SAMPLERSCP";
private static Camera mCamera;
private Camera.Parameters mParams;
private int mFrameWidth, mFrameHeight;
private static SurfaceTexture mST;
private RenderScript mRS;
private Allocation mInAllocation;
private Allocation mOutAllocation;
private ScriptC_mono mScript;

public void onCreate(Bundle savedInstanceState) {

    super.onCreate(savedInstanceState);

    Log.i(TAG, "onCreate()");
    createGUI();
    createCamera();
    createRSEnvironment();
}

public void onPause() {
    Log.i(TAG, "onPause");
    mCamera.stopPreview();
    mCamera.release();
    mCamera = null;
    super.onPause();
}

private void createCamera() {
    mCamera = Camera.open();
    mParams = mCamera.getParameters();

    mFrameWidth = 640;
    mFrameHeight = 480;
    mParams.setPreviewSize(mFrameWidth, mFrameHeight);
    mParams.setPreviewFormat(ImageFormat.NV21);

    mCamera.setParameters(mParams);
}

private void createRSEnvironment () {
    mRS = RenderScript.create(this);
    mScript = new ScriptC_mono(mRS, getResources(), R.raw.mono);

    Type.Builder b = new Type.Builder(mRS, Element.U8(mRS));

    int usage = Allocation.USAGE_SCRIPT | Allocation.USAGE_IO_INPUT;
    mInAllocation = Allocation.createTyped(mRS, b.setX(mFrameWidth).setY(mFrameHeight).create(), usage);
    mOutAllocation = Allocation.createTyped(mRS, b.setX(mFrameWidth).setY(mFrameHeight).create());

    Log.i(TAG, "Getting SurfaceTexture from input Allocation");
    mST = mInAllocation.getSurfaceTexture();

    mST.setOnFrameAvailableListener(this);

    try {
      Log.i(TAG, "Setting SurfaceTexture for camera preview");
      mCamera.setPreviewTexture(mST);

      Log.i(TAG, "Starting preview");
      mCamera.startPreview();
      } catch (IOException e) {
      Log.e(TAG, "Oops, something got wrong with setting the camera preview texture");
    }
}

private void createGUI() {
    requestWindowFeature(Window.FEATURE_NO_TITLE);
    getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN, 
            WindowManager.LayoutParams.FLAG_FULLSCREEN);

    setContentView(R.layout.main);
}

private Handler handler = new Handler() {
    @Override
    public void handleMessage(Message msg) {
        if (msg.what == DO_KERNEL)
          Log.i(TAG, "Calling RS kernel");
          mST.updateTexImage();
          // mInAllocation.ioReceive();
          mScript.forEach_root(mInAllocation, mOutAllocation);

          Log.i(TAG, "Finishing RS");
          mRS.finish();
          Log.i(TAG, "Ok");
    }
};

public void onFrameAvailable(SurfaceTexture st) {
  Log.i(TAG, "onFrameAvailable callback");
  handler.sendEmptyMessage(DO_KERNEL);
}

}

RS代码非常简单,只是尝试检测非空数据:

void root(const uchar *v_in, uchar *v_out, uint32_t x, uint32_t y) {

if (*v_in != 0)
  rsDebug("input data non null !", *v_in);
*v_out = (x / 640) * 255;

}

Answer 1

跟进我自己的问题:

事实证明,无法从相机填充的SurfaceTexture读取NV21缓冲区.我不得不修改Android源代码来试验这个(对于专家:获取SurfaceTexture的当前缓冲区,然后将其锁定以获得真正的缓冲区指针 - 我在RS驱动程序的rsdAllocationIoReceive()中执行此操作).这样可以避免从相机到RS执行缓冲区复制.

最新的JB(MR1版本)有一个名为LivePreview的测试应用程序,它在摄像头预览上执行RS处理.但是,它使用应用程序分配的预览回调缓冲区,然后将其复制到输入分配.有趣的是,它使用新的ScriptIntrinsicRgbToYuv类进行颜色转换.转换的大部分是手动编码的霓虹灯组件,因此可能非常快.

甚至可能是Nexus 10(它有一个支持Mali的RS驱动程序)在GPU上执行此操作,我很乐意让我的手在这个设备上玩.(捐赠者欢迎;-)!)