更新于 

实时渲染

渲染到纹理

所谓渲染到纹理

就是将 渲染结果 作为纹理贴到另一个三维物体上去,
实际上就是 动态生成图像

应用场景
动态模糊
景深效果

帧缓冲区对象和渲染缓冲区对象

还记得WebGL的绘制结果存储在哪个缓冲区中吗?

答:

  • 默认情况下在 颜色缓冲区 中绘图
  • 开启DEPTH_TEST时,还会用到 深度缓冲区
  • 总之,绘制结果存储在 颜色缓冲区

帧缓冲区对象

帧缓冲区对象(framebuffer object)

可以用来代替 颜色缓冲区深度缓冲区

为什么在帧缓冲区中绘制的过程又称为离屏绘制(offscreen drawing)

答:

  1. 因为绘制在帧缓冲区的对象不会直接显示在画布上
  2. 帧缓冲区的内容一般会做如下处理:
    • 进行一些处理再显示
    • 直接用其中的内容作为 纹理图像
帧缓冲区对象的结构

绘制操作不是直接发生在 帧缓冲区 中的,
而是发生在帧缓冲区 所关联的对象(attachment) 上。

帧缓冲区的3个关联对象
  • 颜色关联对象 color attachment 代替颜色缓冲区
  • 深度关联对象 depth attachment 代替深度缓冲区
  • 模板关联对象 stencil attachment 代替模板缓冲区

渲染缓冲区对象

渲染缓冲区对象(renderbuffer object)

帧缓冲区的每一个关联对象可以有两种类型:

  • 纹理对象
    存储了纹理图像
  • 渲染缓冲区对象
    表示一种 更加通用 的绘图区域,可以向其中写入多种类型的数据

如何实现渲染到纹理

实现步骤

实现目的: 把WebGL渲染出的图像作为纹理使用
实现方式:

  • 纹理对象作为颜色关联对象关联到 帧缓冲区对象上
    纹理对象 ← 颜色关联对象
  • 渲染缓冲区对象作为深度关联对象关联到 帧缓冲区对象上
    渲染缓冲区对象 ← 深度关联对象
小比喻:雪媚娘

绘图就像烹饪一样,对应的对象就像是 食材工序 一样。
WebGL的缓冲区对应烹饪中的几个要素一样:

  • 颜色缓冲区:食材
  • 深度缓冲区:层次

但是只使用颜色缓冲区和深度缓冲区就好像做一道工序简单的菜,
食材 新鲜,层次 直观
就比如要做西红柿炒鸡蛋,
食材(颜色缓冲区)有鸡蛋、番茄,工序(深度缓冲区)就是先放鸡蛋再放番茄。

但如果想要做 雪媚娘 这样工序比较繁琐的甜点,
你需要考虑 皮料(糯米粉、凉白开),
并且皮料的工序已经比较复杂,所以我们可以将准备皮料的工序单独取出来,
和其他工序区别开来,这就相当于帧缓冲区的 离屏绘制

那么上面使用帧缓冲区对象渲染到纹理的目的和方式也可以对应如下的描述:
实现目的: 自制雪媚娘
实现方式: 将食谱(帧缓冲区对象)中的食材(颜色关联对象)指向糯米皮(纹理对象)
至于糯米皮怎么制作的,又是单独的一道工序了(即纹理绘制)

创建步骤

step 1

创建 帧缓冲区对象

1
gl.createFramebuffer()

step 2

创建 纹理对象 并设置其尺寸和参数

1
2
3
4
gl.createTexture()
gl.bindTexture()
gl.texImage2d()
gl.Parameteri()

step 3

创建 渲染缓冲区对象

1
gl.createRenderbuffer()

step 4

绑定渲染缓冲区对象 并设置尺寸

1
2
gl.bindRenderbuffer()
gl.renderbufferStorage()

step 5

分配 纹理对象 到帧缓冲区的 颜色关联对象

1
gl.framebufferRenderbuffer()

step 6

分配 渲染缓冲区对象 到帧缓冲区的 深度关联对象

1
gl.framebufferRenderbuffer()

step 7

检查帧缓冲区 是否正确配置

1
gl.checkFramebufferStatus()

step 8

在帧缓冲区中 进行绘制

1
gl.bindFramebuffer()

FramebufferObject

FramebufferObject Code
FramebufferObject运行结果
FramebufferObject运行结果
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
let VSHADER_SOURCE = `
attribute vec4 a_Position;
attribute vec2 a_TexCoord;
uniform mat4 u_MvpMatrix;
varying vec2 v_TexCoord;
void main(){
gl_Position = u_MvpMatrix * a_Position;
v_TexCoord = a_TexCoord;
}
`

let FSHADER_SOURCE = `
#ifdef GL_ES
precision mediump float;
#endif
uniform sampler2D u_Sampler;
varying vec2 v_TexCoord;
void main(){
gl_FragColor = texture2D(u_Sampler, v_TexCoord);
}
`
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
// 离屏绘制尺寸
let OFFSCREEN_WIDTH = 256;
let OFFSCREEN_HEIGHT = 256;
function main() {
let canvas = document.getElementById('webgl')
let gl = getWebGLContext(canvas)
if (!initShaders(gl, VSHADER_SOURCE, FSHADER_SOURCE)) {
console.log('failed to init shaders')
return
}
let program = gl.program
program.a_Position = gl.getAttribLocation(program, 'a_Position')
program.a_TexCoord = gl.getAttribLocation(program, 'a_TexCoord')
program.u_MvpMatrix = gl.getUniformLocation(program, 'u_MvpMatrix')
let cube = initVertexBuffersForCube(gl) // 初始化立方体顶点信息
let plane = initVertexBuffersForPlane(gl) // 初始化平面定点信息
let texture = initTextures(gl) // 初始化纹理
let fbo = initFramebufferObject(gl) // 初始化帧缓冲区对象
gl.enable(gl.DEPTH_TEST) // 打开深度缓冲区

let viewProjMatrix = new Matrix4() // 视图透视矩阵 为颜色缓冲区做准备
viewProjMatrix.setPerspective(
30, canvas.width/canvas.height, 1.0, 100.0
)
viewProjMatrix.lookAt(
0.0, 0.0, 7.0,
0.0, 0.0, 0.0,
0.0, 1.0, 0.0
)

let viewProjMatrixFBO = new Matrix4(); // 帧缓冲器对象使用的透视矩阵
viewProjMatrixFBO.setPerspective(
30.0, OFFSCREEN_WIDTH/OFFSCREEN_HEIGHT, 1.0, 100.0
)
viewProjMatrixFBO.lookAt(
0.0, 2.0, 7.0,
0.0, 0.0, 0.0,
0.0, 1.0, 0.0
)

let currentAngle = 0.0
let tick = function () {
currentAngle = animate(currentAngle)
draw(gl, canvas, fbo, plane, cube, currentAngle, texture, viewProjMatrix, viewProjMatrixFBO)
window.requestAnimationFrame(tick, canvas)
}
tick()
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112

// 初始化立方体顶点缓冲区
function initVertexBuffersForCube(gl) {
let vertices = new Float32Array([ // 顶点缓冲区数组
1.0, 1.0, 1.0, -1.0, 1.0, 1.0, -1.0,-1.0, 1.0, 1.0,-1.0, 1.0, // v0-v1-v2-v3 front
1.0, 1.0, 1.0, 1.0,-1.0, 1.0, 1.0,-1.0,-1.0, 1.0, 1.0,-1.0, // v0-v3-v4-v5 right
1.0, 1.0, 1.0, 1.0, 1.0,-1.0, -1.0, 1.0,-1.0, -1.0, 1.0, 1.0, // v0-v5-v6-v1 up
-1.0, 1.0, 1.0, -1.0, 1.0,-1.0, -1.0,-1.0,-1.0, -1.0,-1.0, 1.0, // v1-v6-v7-v2 left
-1.0,-1.0,-1.0, 1.0,-1.0,-1.0, 1.0,-1.0, 1.0, -1.0,-1.0, 1.0, // v7-v4-v3-v2 down
1.0,-1.0,-1.0, -1.0,-1.0,-1.0, -1.0, 1.0,-1.0, 1.0, 1.0,-1.0 // v4-v7-v6-v5 back
])
let texCoords = new Float32Array([ // 纹理缓冲区数组
1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, // v0-v1-v2-v3 front
0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, // v0-v3-v4-v5 right
1.0, 0.0, 1.0, 1.0, 0.0, 1.0, 0.0, 0.0, // v0-v5-v6-v1 up
1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, // v1-v6-v7-v2 left
0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 1.0, // v7-v4-v3-v2 down
0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 1.0 // v4-v7-v6-v5 back
])
let indices = new Uint8Array([ // 索引缓冲区数组
0, 1, 2, 0, 2, 3, // front
4, 5, 6, 4, 6, 7, // right
8, 9,10, 8,10,11, // up
12,13,14, 12,14,15, // left
16,17,18, 16,18,19, // down
20,21,22, 20,22,23 // back
])
let o = new Object();
o.vertexBuffer = initArrayBuffersForLaterUse(gl, vertices, 3, gl.FLOAT)
o.texCoordBuffer = initArrayBuffersForLaterUse(gl, texCoords, 2, gl.FLOAT)
o.indexBuffer = initElementArrayBufferForLaterUse(gl, indices, gl.UNSIGNED_BYTE)
if (!o.vertexBuffer || !o.texCoordBuffer || !o.indexBuffer) return null
o.numIndices = indices.length
gl.bindBuffer(gl.ARRAY_BUFFER, null)
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, null)

return o
}

// 初始化平面顶点缓冲区
function initVertexBuffersForPlane(gl) {
let vertices = new Float32Array([
1.0, 1.0, 0.0, -1.0, 1.0, 0.0, -1.0,-1.0, 0.0, 1.0,-1.0, 0.0 // v0-v1-v2-v3
])
let texCoords = new Float32Array([
1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0
])
let indices = new Uint8Array([
0, 1, 2, 0, 2, 3
])
let o = new Object()
o.vertexBuffer = initArrayBuffersForLaterUse(gl, vertices, 3, gl.FLOAT)
o.texCoordBuffer = initArrayBuffersForLaterUse(gl, texCoords, 2, gl.FLOAT)
o.indexBuffer = initElementArrayBufferForLaterUse(gl, indices, gl.UNSIGNED_BYTE)
if (!o.vertexBuffer || !o.texCoordBuffer || !o.indexBuffer) return null
o.numIndices = indices.length
gl.bindBuffer(gl.ARRAY_BUFFER, null)
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, null)
return o
}

// 初始化ARRAY_BUFFER
function initArrayBuffersForLaterUse(gl, data, num, type) {
let buffer = gl.createBuffer()
if (!buffer) {
console.log('failed to init Array Buffer')
return null
}
gl.bindBuffer(gl.ARRAY_BUFFER, buffer)
gl.bufferData(gl.ARRAY_BUFFER, data, gl.STATIC_DRAW)
buffer.num = num
buffer.type = type
return buffer
}

// 初始化索引缓冲区
function initElementArrayBufferForLaterUse(gl, data, type) {
let buffer = gl.createBuffer()
if (!buffer) {
console.log('failed to init element array buffer')
return
}
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, buffer)
gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, data, gl.STATIC_DRAW)
buffer.type = type
return buffer
}

// 初始化纹理对象
function initTextures(gl) {
let texture = gl.createTexture()
if (!texture) {
console.log('failed to create the Texture object')
return null
}
let u_Sampler = gl.getUniformLocation(gl.program, 'u_Sampler')
if (!u_Sampler) {
console.log('failed to get the storage location of u_Sampler')
return null
}
let image = new Image()
image.onload = function () {
gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, 1)
gl.bindTexture(gl.TEXTURE_2D, texture)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR)
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, image)
gl.uniform1i(u_Sampler, 0)
gl.bindTexture(gl.TEXTURE_2D, null)
}
image.src = '../img/sky_cloud.jpg'
return texture
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
// 初始化帧缓冲区对象
function initFramebufferObject(gl) {
let framebuffer
let texture
let depthBuffer
let error = function () {
if (framebuffer) gl.deleteFramebuffer(framebuffer)
if (texture) gl.deleteTexture(texture)
if (depthBuffer) gl.deleteRenderbuffer(depthBuffer)
return null
}
framebuffer = gl.createFramebuffer() // 创建帧缓冲区
if (!framebuffer) {
console.log('failed to create frame buffer object')
return error()
}
texture = gl.createTexture() // 创建纹理缓冲区
if (!texture) {
console.log('failed to create texture object')
return error()
}
gl.bindTexture(gl.TEXTURE_2D, texture)
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, OFFSCREEN_WIDTH, OFFSCREEN_HEIGHT, 0, gl.RGBA, gl.UNSIGNED_BYTE, null)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR)
framebuffer.texture = texture // 存入纹理对象

depthBuffer = gl.createRenderbuffer() // 创建渲染缓冲区
if (!depthBuffer) {
console.log('failed to create renderbuffer object')
return error()
}
gl.bindRenderbuffer(gl.RENDERBUFFER, depthBuffer)
gl.renderbufferStorage(gl.RENDERBUFFER, gl.DEPTH_COMPONENT16, OFFSCREEN_WIDTH, OFFSCREEN_HEIGHT)

// 将纹理对象与渲染对象绑定到帧缓冲区对象上
gl.bindFramebuffer(gl.FRAMEBUFFER, framebuffer)
gl.framebufferTexture2D(
gl.FRAMEBUFFER,
gl.COLOR_ATTACHMENT0,
gl.TEXTURE_2D,
texture,
0
)
gl.framebufferRenderbuffer(
gl.FRAMEBUFFER,
gl.DEPTH_ATTACHMENT,
gl.RENDERBUFFER,
depthBuffer
)
let e = gl.checkFramebufferStatus(gl.FRAMEBUFFER)
if (gl.FRAMEBUFFER_COMPLETE !== e) {
console.log('frame buffer is incomplete: ' + e.toString())
return error()
}
gl.bindFramebuffer(gl.FRAMEBUFFER, null)
gl.bindTexture(gl.TEXTURE_2D, null)
gl.bindRenderbuffer(gl.RENDERBUFFER, null)
return framebuffer
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
// 绘制
function draw(gl, canvas, fbo, plane, cube, angle, texture, viewProjMatrix, viewProjMatrixFBO) {
// 先绘制在帧缓冲区上
gl.bindFramebuffer(gl.FRAMEBUFFER, fbo)
gl.viewport(0, 0, OFFSCREEN_WIDTH, OFFSCREEN_HEIGHT)
gl.clearColor(0.2, 0.2, 0.4, 1.0)
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT)
drawTextureCube(gl, gl.program, cube, angle, texture, viewProjMatrixFBO)

// 再绘制在canvas上
gl.bindFramebuffer(gl.FRAMEBUFFER, null)
gl.viewport(0, 0, canvas.width, canvas.height)
gl.clearColor(0.0, 0.0, 0.0, 1.0)
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT)
drawTexturedPlane(gl, gl.program, plane, angle, fbo.texture, viewProjMatrix)
}

let g_modelMatrix = new Matrix4()
let g_mvpMatrix = new Matrix4()

// 绘制带纹理的立方体
function drawTextureCube(gl, program, o, angle, texture, viewProjMatrix) {
g_modelMatrix.setRotate(
20.0, 1.0, 0.0, 0.0
).rotate(
angle, 0.0, 1.0, 0.0
)
g_mvpMatrix.set(
viewProjMatrix
).multiply(
g_modelMatrix
)
gl.uniformMatrix4fv(program.u_MvpMatrix, false, g_mvpMatrix.elements)
drawTexturedObject(gl, program, o, texture)
}

// 绘制平面
function drawTexturedPlane(gl, program, o, angle, texture, viewProjMatrix) {
g_modelMatrix.setTranslate(
0, 0, 1
).rotate(
20.0, 1.0, 0.0, 0.0
).rotate(
angle, 0.0, 1.0, 0.0
)
g_mvpMatrix.set(
viewProjMatrix
).multiply(
g_modelMatrix
)
gl.uniformMatrix4fv(program.u_MvpMatrix, false, g_mvpMatrix.elements)
drawTexturedObject(gl, program, o, texture)
}

// 绘制纹理对象
function drawTexturedObject(gl, program, o, texture) {
initAttributeVariable(gl, program.a_Position, o.vertexBuffer)
initAttributeVariable(gl, program.a_TexCoord, o.texCoordBuffer)
gl.activeTexture(gl.TEXTURE0)
gl.bindTexture(gl.TEXTURE_2D, texture)
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, o.indexBuffer)
gl.drawElements(gl.TRIANGLES, o.numIndices, o.indexBuffer.type, 0)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
// 获取attribute
function initAttributeVariable(gl, a_attribute, buffer) {
gl.bindBuffer(gl.ARRAY_BUFFER, buffer)
gl.vertexAttribPointer(a_attribute, buffer.num, buffer.type, false, 0, 0)
gl.enableVertexAttribArray(a_attribute)
}

let ANGLE_STEP = 30
let last = Date.now()
function animate(angle) {
let now = Date.now()
let elapsed = now - last
last = now
let newAngle = angle + (ANGLE_STEP * elapsed) / 1000.0;
return newAngle % 360
}

渲染流程详解

创建帧冲区对象

1
framebuffer = gl.createFramebuffer()
createFramebuffer用于创建帧缓冲区对象
deleteFramebuffer用于删除帧缓冲区对象

创建纹理对象并设置尺寸和参数

1
2
3
4
5
6
7
8
9
texture = gl.createTexture() // 创建纹理对象
gl.bindTexture(gl.TEXTURE_2D, texture)
gl.texImage2D(
gl.TEXTURE_2D, 0, gl.RGBA,
OFFSCREEN_WIDTH, OFFSCREEN_HEIGHT,
0, gl.RGBA, gl.UNSIGNED_BYTE, null
)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR)
framebuffer.texture = texture // 存入纹理对象

创建渲染缓冲区对象

1
depthBuffer = gl.createRenderbuffer();
createRenderbuffer创建渲染缓冲区对象
deleteRenderbuffer删除渲染缓冲区对象

绑定渲染缓冲区并设置其尺寸

1
2
3
4
5
gl.bindRenderbuffer(gl.RENDERBUFFER, depthBuffer)
gl.renderbufferStorage(
gl.RENDERBUFFER, gl.DEPTH_COMPONENT16,
OFFSCREEN_WIDTH, OFFSCREEN_HEIGHT
)
bindRenderbuffer绑定渲染缓冲区
renderbufferStorage创建并初始化渲染缓冲区的数据区

深度关联对象的渲染缓冲区宽高必须与纹理缓冲区一致

将纹理对象关联到帧缓冲区对象

1
2
3
4
5
gl.bindFramebuffer(gl.FRAMEBUFFER, framebuffer); // 绑定帧缓冲区
gl.framebufferTexture2D(
gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0,
gl.TEXTURE_2D, texture, 0
)
bindFramebuffer绑定帧缓冲区对象
framebufferTexture2D指定颜色和渲染关联对象
为什么颜色关联对象存在0后缀?

答: 在OpenGL中有很多颜色关联对象,但在WebGL中只有1个,
所以就只将序号为0的颜色关联对象分给了WebGL。

将渲染缓冲区对象关联到帧缓冲区对象

1
2
3
4
gl.framebufferRenderbuffer(
gl.FRAMEBUFFER, gl.DEPTH_ATTACHMENT,
gl.RENDERBUFFER, depthBuffer
)
framebufferRenderbuffer

检查帧缓冲区的配置

1
2
3
4
5
let e = gl.checkFramebufferStatus(gl.FRAMEBUFFER)
if(gl.FRAMEBUFFER.COMPLETE !== e){
console.log('Framebuffer object is incomlete: ' + e.toString())
return error()
}
checkFramebufferStatus

在帧缓冲区进行绘图

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
function draw(gl, canvas, fbo, plane, cube, angle, texture, viewProjMatrix, viewProjMatrixFBO) { 
// 先绘制在帧缓冲区上
gl.bindFramebuffer(gl.FRAMEBUFFER, fbo)
gl.viewport(0, 0, OFFSCREEN_WIDTH, OFFSCREEN_HEIGHT)
gl.clearColor(0.2, 0.2, 0.4, 1.0)
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT)
drawTextureCube(gl, gl.program, cube, angle, texture, viewProjMatrixFBO)

// 再绘制在canvas上
gl.bindFramebuffer(gl.FRAMEBUFFER, null)
gl.viewport(0, 0, canvas.width, canvas.height)
gl.clearColor(0.0, 0.0, 0.0, 1.0)
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT)
drawTexturedPlane(gl, gl.program, plane, angle, fbo.texture, viewProjMatrix)
}
viewport定义离线绘图的绘图区域

开启消隐功能

gl.CULL_FACE

运行FramebufferObject时,矩形正反两面都被贴上了纹理,
这是因为WebGL默认 绘制图形的正反两面
实际上WebGL提供消隐功能(culling function)

1
gl.enable(gl.CULL_FACE)

这样让WebGL不再绘制图形背面,能够提高绘制速度

未开启消隐

未开启消隐运行结果
未开启消隐运行结果

开启了消隐

开启了消隐运行结果
开启了消隐运行结果