在WebGL和Canvas 2D中模拟基于调色板的图形

LGB*_*LGB 9 javascript scaling canvas webgl color-palette

目前,我正在使用2D画布上下文从JavaScript以大约25fps的速率绘制生成的图像(从像素到像素,但在生成的帧之后一次刷新为整个缓冲区).生成的图像每个像素始终是一个字节(整数/类型数组),固定调色板用于生成RGB最终结果.还需要缩放以采用画布的大小(即:进入全屏)和/或根据用户请求(放大/缩小按钮).

画布的2D上下文可以用于此目的,但是我很好奇WebGL是否可以提供更好的结果和/或更好的性能.请注意:我不想通过webGL放置像素,我想将像素放入我的缓冲区(基本上是Uint8Array),并使用该缓冲区(一次)刷新上下文.我不太了解WebGL,但是使用所需的生成图像作为某种纹理会以某种方式工作吗?然后我需要以大约25fps的速率刷新纹理,我猜.

如果WebGL以某种方式支持色彩空间转换,那真是太棒了.对于2D上下文,我需要将每个像素的1个字节/像素缓冲区转换为RGBA for JavaScript中的imagedata ...现在通过改变画布的高度/宽度样式来缩放(对于2D上下文),因此浏览器会缩放然后是图像.但是我想它可能比WebGL可以用hw支持做得慢,并且(我希望)WebGL可以提供更大的灵活性来控制缩放,例如使用2D上下文,浏览器即使我不想要也会做抗锯齿做(例如:整数缩放因子),也许这就是它有时可能很慢的原因.

我已经尝试过学习几个WebGL教程,但所有这些教程都以对象,形状,3D立方体等开头,我不需要任何 - 经典 - 对象来渲染2D上下文也可以做什么 - 希望对于同样的任务,WebGL可以是更快的解决方案!当然,如果根本没有WebGL的胜利,我会继续使用2D上下文.

需要说明的是:这是用JavaScript完成的某种计算机硬件模拟器,它的输出(连接到它的PAL电视上可以看到)通过画布上下文呈现.该机器具有256个元素的固定调色板,在内部它只需要一个字节用于像素来定义其颜色.

gma*_*man 17

您可以使用纹理作为调色板,使用不同的纹理作为图像.然后,您从图像纹理中获取一个值,并使用它从调色板纹理中查找颜色.

调色板纹理为256x1 RGBA像素.您的图像纹理是您想要的任何尺寸,但只是单个通道ALPHA纹理.然后,您可以从图像中查找值

    float index = texture2D(u_image, v_texcoord).a * 255.0;
Run Code Online (Sandbox Code Playgroud)

并使用该值在调色板中查找颜色

    gl_FragColor = texture2D(u_palette, vec2((index + 0.5) / 256.0, 0.5));
Run Code Online (Sandbox Code Playgroud)

您的着色器可能是这样的

顶点着色器

attribute vec4 a_position;
varying vec2 v_texcoord;
void main() {
  gl_Position = a_position;

  // assuming a unit quad for position we
  // can just use that for texcoords. Flip Y though so we get the top at 0
  v_texcoord = a_position.xy * vec2(0.5, -0.5) + 0.5;
}    
Run Code Online (Sandbox Code Playgroud)

片段着色器

precision mediump float;
varying vec2 v_texcoord;
uniform sampler2D u_image;
uniform sampler2D u_palette;

void main() {
    float index = texture2D(u_image, v_texcoord).a * 255.0;
    gl_FragColor = texture2D(u_palette, vec2((index + 0.5) / 256.0, 0.5));
}
Run Code Online (Sandbox Code Playgroud)

然后你只需要一个调色板纹理.

 // Setup a palette.
 var palette = new Uint8Array(256 * 4);

 // I'm lazy so just setting 4 colors in palette
 function setPalette(index, r, g, b, a) {
     palette[index * 4 + 0] = r;
     palette[index * 4 + 1] = g;
     palette[index * 4 + 2] = b;
     palette[index * 4 + 3] = a;
 }
 setPalette(1, 255, 0, 0, 255); // red
 setPalette(2, 0, 255, 0, 255); // green
 setPalette(3, 0, 0, 255, 255); // blue

 // upload palette
 ...
 gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 256, 1, 0, gl.RGBA, 
               gl.UNSIGNED_BYTE, palette);
Run Code Online (Sandbox Code Playgroud)

而你的形象.它只是一个alpha图像,所以只有1个频道.

 // Make image. Just going to make something 8x8
 var image = new Uint8Array([
     0,0,1,1,1,1,0,0,
     0,1,0,0,0,0,1,0,
     1,0,0,0,0,0,0,1,
     1,0,2,0,0,2,0,1,
     1,0,0,0,0,0,0,1,
     1,0,3,3,3,3,0,1,
     0,1,0,0,0,0,1,0,
     0,0,1,1,1,1,0,0,
 ]);

 // upload image
 ....
 gl.texImage2D(gl.TEXTURE_2D, 0, gl.ALPHA, 8, 8, 0, gl.ALPHA, 
               gl.UNSIGNED_BYTE, image);
Run Code Online (Sandbox Code Playgroud)

您还需要确保两个纹理都gl.NEAREST用于过滤,因为一个表示索引而另一个是调色板,在这些情况下值之间的过滤没有意义.

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
Run Code Online (Sandbox Code Playgroud)

这是一个有效的例子:

var canvas = document.getElementById("c");
var gl = canvas.getContext("webgl");

// Note: createProgramFromScripts will call bindAttribLocation
// based on the index of the attibute names we pass to it.
var program = twgl.createProgramFromScripts(
    gl, 
    ["vshader", "fshader"], 
    ["a_position", "a_textureIndex"]);
gl.useProgram(program);
var imageLoc = gl.getUniformLocation(program, "u_image");
var paletteLoc = gl.getUniformLocation(program, "u_palette");
// tell it to use texture units 0 and 1 for the image and palette
gl.uniform1i(imageLoc, 0);
gl.uniform1i(paletteLoc, 1);

// Setup a unit quad
var positions = [
      1,  1,  
     -1,  1,  
     -1, -1,  
      1,  1,  
     -1, -1,  
      1, -1,  
];
var vertBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(positions), gl.STATIC_DRAW);
gl.enableVertexAttribArray(0);
gl.vertexAttribPointer(0, 2, gl.FLOAT, false, 0, 0);

// Setup a palette.
var palette = new Uint8Array(256 * 4);

// I'm lazy so just setting 4 colors in palette
function setPalette(index, r, g, b, a) {
    palette[index * 4 + 0] = r;
    palette[index * 4 + 1] = g;
    palette[index * 4 + 2] = b;
    palette[index * 4 + 3] = a;
}
setPalette(1, 255, 0, 0, 255); // red
setPalette(2, 0, 255, 0, 255); // green
setPalette(3, 0, 0, 255, 255); // blue

// make palette texture and upload palette
gl.activeTexture(gl.TEXTURE1);
var paletteTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, paletteTex);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 256, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE, palette);

// Make image. Just going to make something 8x8
var image = new Uint8Array([
    0,0,1,1,1,1,0,0,
    0,1,0,0,0,0,1,0,
    1,0,0,0,0,0,0,1,
    1,0,2,0,0,2,0,1,
    1,0,0,0,0,0,0,1,
    1,0,3,3,3,3,0,1,
    0,1,0,0,0,0,1,0,
    0,0,1,1,1,1,0,0,
]);
    
// make image textures and upload image
gl.activeTexture(gl.TEXTURE0);
var imageTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, imageTex);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.ALPHA, 8, 8, 0, gl.ALPHA, gl.UNSIGNED_BYTE, image);
    
gl.drawArrays(gl.TRIANGLES, 0, positions.length / 2);
Run Code Online (Sandbox Code Playgroud)
canvas { border: 1px solid black; }
Run Code Online (Sandbox Code Playgroud)
<script src="https://twgljs.org/dist/twgl.min.js"></script>
<script id="vshader" type="whatever">
    attribute vec4 a_position;
    varying vec2 v_texcoord;
    void main() {
      gl_Position = a_position;
    
      // assuming a unit quad for position we
      // can just use that for texcoords. Flip Y though so we get the top at 0
      v_texcoord = a_position.xy * vec2(0.5, -0.5) + 0.5;
    }    
</script>
<script id="fshader" type="whatever">
precision mediump float;
varying vec2 v_texcoord;
uniform sampler2D u_image;
uniform sampler2D u_palette;
    
void main() {
    float index = texture2D(u_image, v_texcoord).a * 255.0;
    gl_FragColor = texture2D(u_palette, vec2((index + 0.5) / 256.0, 0.5));
}
</script>
<canvas id="c" width="256" height="256"></canvas>
Run Code Online (Sandbox Code Playgroud)

要设置动画,只需更新图像,然后将其重新上传到纹理中

 gl.texImage2D(gl.TEXTURE_2D, 0, gl.ALPHA, 8, 8, 0, gl.ALPHA, 
               gl.UNSIGNED_BYTE, image);
Run Code Online (Sandbox Code Playgroud)

例:

var canvas = document.getElementById("c");
var gl = canvas.getContext("webgl");

// Note: createProgramFromScripts will call bindAttribLocation
// based on the index of the attibute names we pass to it.
var program = twgl.createProgramFromScripts(
    gl, 
    ["vshader", "fshader"], 
    ["a_position", "a_textureIndex"]);
gl.useProgram(program);
var imageLoc = gl.getUniformLocation(program, "u_image");
var paletteLoc = gl.getUniformLocation(program, "u_palette");
// tell it to use texture units 0 and 1 for the image and palette
gl.uniform1i(imageLoc, 0);
gl.uniform1i(paletteLoc, 1);

// Setup a unit quad
var positions = [
      1,  1,  
     -1,  1,  
     -1, -1,  
      1,  1,  
     -1, -1,  
      1, -1,  
];
var vertBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(positions), gl.STATIC_DRAW);
gl.enableVertexAttribArray(0);
gl.vertexAttribPointer(0, 2, gl.FLOAT, false, 0, 0);

// Setup a palette.
var palette = new Uint8Array(256 * 4);

// I'm lazy so just setting 4 colors in palette
function setPalette(index, r, g, b, a) {
    palette[index * 4 + 0] = r;
    palette[index * 4 + 1] = g;
    palette[index * 4 + 2] = b;
    palette[index * 4 + 3] = a;
}
setPalette(1, 255, 0, 0, 255); // red
setPalette(2, 0, 255, 0, 255); // green
setPalette(3, 0, 0, 255, 255); // blue

// make palette texture and upload palette
gl.activeTexture(gl.TEXTURE1);
var paletteTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, paletteTex);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 256, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE, palette);

// Make image. Just going to make something 8x8
var width = 8;
var height = 8;
var image = new Uint8Array([
    0,0,1,1,1,1,0,0,
    0,1,0,0,0,0,1,0,
    1,0,0,0,0,0,0,1,
    1,0,2,0,0,2,0,1,
    1,0,0,0,0,0,0,1,
    1,0,3,3,3,3,0,1,
    0,1,0,0,0,0,1,0,
    0,0,1,1,1,1,0,0,
]);
    
// make image textures and upload image
gl.activeTexture(gl.TEXTURE0);
var imageTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, imageTex);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.ALPHA, width, height, 0, gl.ALPHA, gl.UNSIGNED_BYTE, image);
    
var frameCounter = 0;
function render() {  
  ++frameCounter;
    
  // skip 3 of 4 frames so the animation is not too fast
  if ((frameCounter & 3) == 0) {
    // rotate the image left
    for (var y = 0; y < height; ++y) {
      var temp = image[y * width];
      for (var x = 0; x < width - 1; ++x) {
        image[y * width + x] = image[y * width + x + 1];
      }
      image[y * width + width - 1] = temp;
    }
    // re-upload image
    gl.activeTexture(gl.TEXTURE0);
    gl.texImage2D(gl.TEXTURE_2D, 0, gl.ALPHA, width, height, 0, gl.ALPHA,
                  gl.UNSIGNED_BYTE, image);
    
    gl.drawArrays(gl.TRIANGLES, 0, positions.length / 2);
  }
  requestAnimationFrame(render);
}
render();
Run Code Online (Sandbox Code Playgroud)
canvas { border: 1px solid black; }
Run Code Online (Sandbox Code Playgroud)
<script src="https://twgljs.org/dist/twgl.min.js"></script>
<script id="vshader" type="whatever">
    attribute vec4 a_position;
    varying vec2 v_texcoord;
    void main() {
      gl_Position = a_position;
    
      // assuming a unit quad for position we
      // can just use that for texcoords. Flip Y though so we get the top at 0
      v_texcoord = a_position.xy * vec2(0.5, -0.5) + 0.5;
    }    
</script>
<script id="fshader" type="whatever">
precision mediump float;
varying vec2 v_texcoord;
uniform sampler2D u_image;
uniform sampler2D u_palette;
    
void main() {
    float index = texture2D(u_image, v_texcoord).a * 255.0;
    gl_FragColor = texture2D(u_palette, vec2((index + 0.5) / 256.0, 0.5));
}
</script>
<canvas id="c" width="256" height="256"></canvas>
Run Code Online (Sandbox Code Playgroud)

当然,假设您的目标是通过操纵像素在CPU上执行动画.否则,您可以使用任何常规的webgl技术来操纵纹理坐标或其他任何东西.

您也可以类似地更新调色板动画.只需修改调色板并重新上传即可

 gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 256, 1, 0, gl.RGBA, 
               gl.UNSIGNED_BYTE, palette);
Run Code Online (Sandbox Code Playgroud)

例:

var canvas = document.getElementById("c");
var gl = canvas.getContext("webgl");

// Note: createProgramFromScripts will call bindAttribLocation
// based on the index of the attibute names we pass to it.
var program = twgl.createProgramFromScripts(
    gl, 
    ["vshader", "fshader"], 
    ["a_position", "a_textureIndex"]);
gl.useProgram(program);
var imageLoc = gl.getUniformLocation(program, "u_image");
var paletteLoc = gl.getUniformLocation(program, "u_palette");
// tell it to use texture units 0 and 1 for the image and palette
gl.uniform1i(imageLoc, 0);
gl.uniform1i(paletteLoc, 1);

// Setup a unit quad
var positions = [
      1,  1,  
     -1,  1,  
     -1, -1,  
      1,  1,  
     -1, -1,  
      1, -1,  
];
var vertBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(positions), gl.STATIC_DRAW);
gl.enableVertexAttribArray(0);
gl.vertexAttribPointer(0, 2, gl.FLOAT, false, 0, 0);

// Setup a palette.
var palette = new Uint8Array(256 * 4);

// I'm lazy so just setting 4 colors in palette
function setPalette(index, r, g, b, a) {
    palette[index * 4 + 0] = r;
    palette[index * 4 + 1] = g;
    palette[index * 4 + 2] = b;
    palette[index * 4 + 3] = a;
}
setPalette(1, 255, 0, 0, 255); // red
setPalette(2, 0, 255, 0, 255); // green
setPalette(3, 0, 0, 255, 255); // blue

// make palette texture and upload palette
gl.activeTexture(gl.TEXTURE1);
var paletteTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, paletteTex);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 256, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE, palette);

// Make image. Just going to make something 8x8
var width = 8;
var height = 8;
var image = new Uint8Array([
    0,0,1,1,1,1,0,0,
    0,1,0,0,0,0,1,0,
    1,0,0,0,0,0,0,1,
    1,0,2,0,0,2,0,1,
    1,0,0,0,0,0,0,1,
    1,0,3,3,3,3,0,1,
    0,1,0,0,0,0,1,0,
    0,0,1,1,1,1,0,0,
]);
    
// make image textures and upload image
gl.activeTexture(gl.TEXTURE0);
var imageTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, imageTex);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.ALPHA, width, height, 0, gl.ALPHA, gl.UNSIGNED_BYTE, image);
    
var frameCounter = 0;
function render() {  
  ++frameCounter;
    
  // skip 3 of 4 frames so the animation is not too fast
  if ((frameCounter & 3) == 0) {
    // rotate the 3 palette colors
    var tempR = palette[4 + 0];
    var tempG = palette[4 + 1];
    var tempB = palette[4 + 2];
    var tempA = palette[4 + 3];
    setPalette(1, palette[2 * 4 + 0], palette[2 * 4 + 1], palette[2 * 4 + 2], palette[2 * 4 + 3]);
    setPalette(2, palette[3 * 4 + 0], palette[3 * 4 + 1], palette[3 * 4 + 2], palette[3 * 4 + 3]);
    setPalette(3, tempR, tempG, tempB, tempA);

    // re-upload palette
    gl.activeTexture(gl.TEXTURE1);
    gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 256, 1, 0, gl.RGBA,
                  gl.UNSIGNED_BYTE, palette);
    
    gl.drawArrays(gl.TRIANGLES, 0, positions.length / 2);
  }
  requestAnimationFrame(render);
}
render();
Run Code Online (Sandbox Code Playgroud)
canvas { border: 1px solid black; }
Run Code Online (Sandbox Code Playgroud)
<script src="https://twgljs.org/dist/twgl.min.js"></script>
<script id="vshader" type="whatever">
    attribute vec4 a_position;
    varying vec2 v_texcoord;
    void main() {
      gl_Position = a_position;
    
      // assuming a unit quad for position we
      // can just use that for texcoords. Flip Y though so we get the top at 0
      v_texcoord = a_position.xy * vec2(0.5, -0.5) + 0.5;
    }    
</script>
<script id="fshader" type="whatever">
precision mediump float;
varying vec2 v_texcoord;
uniform sampler2D u_image;
uniform sampler2D u_palette;
    
void main() {
    float index = texture2D(u_image, v_texcoord).a * 255.0;
    gl_FragColor = texture2D(u_palette, vec2((index + 0.5) / 256.0, 0.5));
}
</script>
<canvas id="c" width="256" height="256"></canvas>
Run Code Online (Sandbox Code Playgroud)

稍微相关的是这个tile shader示例 http://blog.tojicode.com/2012/07/sprite-tile-maps-on-gpu.html


dav*_*ink 5

大概你正在构建一个大约512 x 512(PAL大小)的javascript数组......

WebGL片段着色器绝对可以很好地完成您的调色板转换.食谱会是这样的:

  1. 使用仅包含两个跨越视口的三角形的"几何"设置WebGL.(GL是所有三角形.)如果你还没有GL流畅,这是最大的麻烦.但它并没有那么糟糕.花一些时间在http://learningwebgl.com/blog/?page_id=1217.但它将是~100行的东西.入场费.
  2. 构建内存帧缓冲区大4倍.(我认为纹理总是必须是RGBA?)并用像素值填充每四个字节R组件.使用新的Float32Array来分配它.您可以使用值0-255,或将其除以0.0到1.0.我们将它作为纹理传递给webgl.每一帧都会改变.
  3. 构建一个256 x 1像素的第二个纹理,这是您的调色板查找表.这个永远不会改变(除非可以修改调色板?).
  4. 在片段着色器中,使用模拟的帧缓冲区纹理作为查找调色板.调色板中的第一个像素在像素中间的位置(0.5/256.0,0.5)处被访问.
  5. 在每个帧上,重新提交模拟的帧缓冲区纹理并重绘.将像素推送到GPU是很昂贵的......但是按照现代标准,PAL尺寸的图像非常小.
  6. 奖励步骤:您可以在现代高分辨率显示器上增强片段着色器以模仿扫描线,隔行视频或其他可爱的仿真工件(荧光点?),所有这些都是免费的!

这只是一个草图.但它会奏效.WebGL是一个非常低级的API,非常灵活,但非常值得付出努力(如果你喜欢那样的东西,我会这样做.:-)).

同样,对于整体WebGL指导,建议http://learningwebgl.com/blog/?page_id=1217.