一个新的 UI 系统:GGUI
分类 | 先决条件 |
---|---|
操作系统 | Windows / Linux / Mac OS X |
后端 | x64 / CUDA / Vulkan |
从 v0.8.0开始,Taichi 添加了一个新的 GGUI系统。 新的 GUI 系统使用 GPU 进行渲染,使得 3 维场景的渲染更快。 这就是为什么这一新系统被命名为 GGUI。 本文档将介绍其提供的 API。
IMPORTANT
如果你选择 Vulkan 作为后端, 确保你已经 install the Vulkan environment.
note
我们推荐您通过 examples/ggui_examples
提供的示例熟悉 GGUI
创建一个窗口
ti.ui.Window(name, res)
可以创建一个窗口。
window = ti.ui.Window('Window Title', res = (640, 360), pos = (150, 150))
The argument res
means resulotion(width and height) of the window, pos
means the position of the window which origins from the left-top of your main screen.
以下三类对象可以显示在 ti.ui.window
上:
- 2D 画布,可用于绘制简单的2D几何形状,如圆形、三角形等。
- 3D 场景,可用来渲染3D网格和粒子,具有可配置的相机和光源。
- Immediate mode GUI components, for example buttons and textboxes.
2D 画布
创建画布
以下代码获取了一个覆盖整个窗口的 Canvas
对象。
canvas = window.get_canvas()
在画布上绘画
canvas.set_background_color(color)
canvas.triangles(vertices, color, indices, per_vertex_color)
canvas.circles(vertices, radius, color, per_vertex_color)
canvas.lines(vertices, width, indices, color, per_vertex_color)
canvas.set_image(image)
参数 vertices
, indices
, per_vertex_color
, 和 image
必须是 Taichi field。 如果提供了 per_vertex_color
,color
将会被忽略。
画布中几何图案的位置/中心的相对位置以在 0.0
和 1.0
之间的浮点数表示。 对于 circles()
and lines()
,radius
和 width
参数都是窗口高度的相对值。
画布在每帧后被清空。 务必在渲染循环中调用这些方法。
3D 场景
创建场景
scene = ti.ui.Scene()
配置相机
camera = ti.ui.Camera()
camera.position(pos)
camera.lookat(pos)
camera.up(dir)
camera.projection_mode(mode)
scene.set_camera(camera)
配置光源
添加点光源
调用 point_light()
将点光源添加到场景中。
scene.point_light(pos, color)
请注意,你需要为每一帧调用 point_light()
方法。 与 canvas()
方法类似,请在你的渲染循环中调用该方法。
3D 几何图形
scene.lines(vertices, width, indices, color, per_vertex_color)
scene.mesh(vertices, indices, normals, color, per_vertex_color)
scene.particles(vertices, radius, color, per_vertex_color)
参数 vertices
, indices
, per_vertex_color
和 image
都被设计为 Taichi fields 类型。 如果提供了 per_vertex_color
,color
将会被忽略。
几何体的位置/中心应设在世界空间的坐标上。
note
如果一个网格有 num
个三角形, indices
应该是一个大小为(num * 3)
的 1D 标量 field,而不是一个 向量 field。
normals
是 scene.mesh()
的一个可选参数。
- An example of drawing 3d-lines
import taichi as ti
ti.init(arch=ti.cuda)
N = 10
particles_pos = ti.Vector.field(3, dtype=ti.f32, shape = N)
points_pos = ti.Vector.field(3, dtype=ti.f32, shape = N)
@ti.kernel
def init_points_pos(points : ti.template()):
for i in range(points.shape[0]):
points[i] = [i for j in ti.static(range(3))]
init_points_pos(particles_pos)
init_points_pos(points_pos)
window = ti.ui.Window("Test for Drawing 3d-lines", (768, 768))
canvas = window.get_canvas()
scene = ti.ui.Scene()
camera = ti.ui.Camera()
camera.position(5, 2, 2)
while window.running:
camera.track_user_inputs(window, movement_speed=0.03, hold_key=ti.ui.RMB)
scene.set_camera(camera)
scene.ambient_light((0.8, 0.8, 0.8))
scene.point_light(pos=(0.5, 1.5, 1.5), color=(1, 1, 1))
scene.particles(particles_pos, color = (0.68, 0.26, 0.19), radius = 0.1)
# Draw 3d-lines in the scene
scene.lines(points_pos, color = (0.28, 0.68, 0.99), width = 5.0)
canvas.scene(scene)
window.show()
Advanced 3d Geometries
scene.lines(vertices, width, indices, color, per_vertex_color, vertex_offset, vertex_count, index_offset, index_count)
scene.mesh(vertices, indices, normals, color, per_vertex_color, vertex_offset, vertex_count, index_offset, index_count, show_wireframe)
scene.particles(vertices, radius, color, per_vertex_color, index_offset, index_count)
scene.mesh_instance(vertices, indices, normals, color, per_vertex_color, vertex_offset, vertex_count, index_offset, index_count, show_wireframe)
The additional arguments vertex_offset
, vertex_count
, index_offset
and index_count
control the visible part of the particles and mesh. For the mesh()
and mesh_instance()
methods, set whether to show wireframe mode through setting show_wireframe
.
:::example
- Example of drawing a part of the mesh/particles
# For particles
# draw the 2-th to 7-th particles
scene.particles(center, radius,
index_offset = 1,
index_count = 6)
# For mesh
# 1. with indices
scene.mesh(vertices, indices,
index_offset = user_defined_first_indices_index,
index_count = user_defined_index_count,
# vertex_offset is set to 0 by default, and it is not necessary
# to assign vertex_offset a value that otherwise you must.
vertex_offset = user_defined_vertex_offset)
# usually used as below:
# draw the 11-th to 111-th mesh vertexes
scene.mesh(vertices, indices,
index_offset = 10,
index_count = 100)
# 2. without indices (similar to the particles' example above)
scene.mesh(vertices,
vertex_offset = user_defined_first_vertex_index,
vertex_count = user_defined_vertex_count)
- An example of drawing part of lines
import taichi as ti
ti.init(arch=ti.cuda)
N = 10
particles_pos = ti.Vector.field(3, dtype=ti.f32, shape = N)
points_pos = ti.Vector.field(3, dtype=ti.f32, shape = N)
points_indices = ti.Vector.field(1, dtype=ti.i32, shape = N)
@ti.kernel
def init_points_pos(points : ti.template()):
for i in range(points.shape[0]):
points[i] = [i for j in range(3)]
# points[i] = [ti.sin(i * 1.0), i * 0.2, ti.cos(i * 1.0)]
@ti.kernel
def init_points_indices(points_indices : ti.template()):
for i in range(N):
points_indices[i][0] = i // 2 + i % 2
init_points_pos(particles_pos)
init_points_pos(points_pos)
init_points_indices(points_indices)
window = ti.ui.Window("Test for Drawing 3d-lines", (768, 768))
canvas = window.get_canvas()
scene = ti.ui.Scene()
camera = ti.ui.Camera()
camera.position(5, 2, 2)
while window.running:
camera.track_user_inputs(window, movement_speed=0.03, hold_key=ti.ui.RMB)
scene.set_camera(camera)
scene.ambient_light((0.8, 0.8, 0.8))
scene.point_light(pos=(0.5, 1.5, 1.5), color=(1, 1, 1))
scene.particles(particles_pos, color = (0.68, 0.26, 0.19), radius = 0.1)
# Here you will get visible part from the 3rd point with (N - 4) points.
scene.lines(points_pos, color = (0.28, 0.68, 0.99), width = 5.0, vertex_count = N - 4, vertex_offset = 2)
# Using indices to indicate which vertex to use
# scene.lines(points_pos, color = (0.28, 0.68, 0.99), width = 5.0, indices = points_indices)
# Case 1, vertex_count will be changed to N - 2 when drawing.
# scene.lines(points_pos, color = (0.28, 0.68, 0.99), width = 5.0, vertex_count = N - 1, vertex_offset = 0)
# Case 2, vertex_count will be changed to N - 2 when drawing.
# scene.lines(points_pos, color = (0.28, 0.68, 0.99), width = 5.0, vertex_count = N, vertex_offset = 2)
canvas.scene(scene)
window.show()
- Details of mesh instancing
num_instance = 100
m_transforms = ti.Matrix.field(4, 4, dtype = ti.f32, shape = num_instance)
# For example: An object is scaled by 2, rotated by rotMat, and translated by t = [1, 2, 3], then
#
# The ScaleMatrix is:
# 2, 0, 0, 0
# 0, 2, 0, 0
# 0, 0, 2, 0
# 0, 0, 0, 1
#
# The RotationMatrix is:
# https://en.wikipedia.org/wiki/Rotation_matrix#General_rotations
#
# The TranslationMatrix is:
# 1, 0, 0, 1
# 0, 1, 0, 2
# 0, 0, 1, 3
# 0, 0, 0, 1
#
# Let TransformMatrix = TranslationMatrix @ RotationMatrix @ ScaleMatrix, then the final TransformMatrix is:
# 2 * rotMat00, rotMat01, rotMat02, 1
# rotMat10, 2 * rotMat11, rotMat12, 2
# rotMat20, rotMat21, 2 * rotMat22, 3
# 0, 0, 0, 1
...
# Draw mesh instances (from the 1st instance)
scene.mesh_instance(vertices, indices, transforms = m_transforms, instance_offset = 1)
- Example of setting wireframe mode
window = ti.ui.Window("Display Mesh", (1024, 1024), vsync=True)
canvas = window.get_canvas()
scene = ti.ui.Scene()
camera = ti.ui.Camera()
# slider_int usage
some_int_type_value = 0
def show_options():
global some_int_type_value
window.GUI.begin("Display Panel", 0.05, 0.1, 0.2, 0.15)
display_mode = window.GUI.slider_int("Value Range", some_int_type_value, 0, 5)
window.GUI.end()
while window.running:
...
# if to show wireframe
scene.mesh_instance(vertices, indices, instance_count = 100 , show_wireframe = True)
canvas.scene(scene)
show_options()
window.show()
note
If indices
is not provided, consider using like this:
scene.mesh(vertices, normals, color, per_vertex_color, vertex_offset, vertex_count, wireframe)
If indices
is provided, consider using like this:
scene.mesh(vertices, indices, normals, color, per_vertex_color, vertex_offset, index_offset, index_count, wireframe)
渲染场景
您可以在画布上渲染场景。
canvas.scene(scene)
Fetching Color/Depth information
img = window.get_image_buffer_as_numpy()
window.get_depth_buffer(scene_depth)
depth = window.get_depth_buffer_as_numpy()
After rendering the current scene, you can fetch the color and depth information of the current scene using get_image_buffer_as_numpy()
and get_depth_buffer_as_numpy()
, which copy the gpu data to a NumPy array(cpu). get_depth_buffer()
copies the GPU data to a Taichi field (depend on the arch
you choose) or copies data from GPU to GPU.
:::example
- Example of fetching color information
window = ti.ui.Window("Test for getting image buffer from ggui", (768, 768), vsync=True)
video_manager = ti.tools.VideoManager("OutputDir")
while window.running:
render_scene()
img = window.get_image_buffer_as_numpy()
video_manager.write_frame(img)
window.show()
video_manager.make_video(gif=True, mp4=True)
- An example of fetching the depth data
window_shape = (720, 1080)
window = ti.ui.Window("Test for copy depth data", window_shape)
canvas = window.get_canvas()
scene = ti.ui.Scene()
camera = ti.ui.Camera()
# Get the shape of the window
w, h = window.get_window_shape()
# The field/ndarray stores the depth information, and must be of the ti.f32 data type and have a 2d shape.
# or, in other words, the shape must equal the window's shape
scene_depth = ti.ndarray(ti.f32, shape = (w, h))
# scene_depth = ti.field(ti.f32, shape = (w, h))
while window.running:
render()
canvas.scene(scene)
window.get_depth_buffer(scene_depth)
window.show()
GUI 组件
GGUI 的 GUI 组件设计遵循 Dear ImGui API。
gui = window.get_gui()
with gui.sub_window(name, x, y, width, height):
gui.text(text)
is_clicked = gui.button(name)
new_value = gui.slider_float(name, old_value, min_value, max_value)
new_color = gui.color_edit_3(name, old_color)
显示窗口
调用 show()
来显示一个窗口。
...
window.show()
请 仅 在每一帧结束时再调用该方法完成渲染。
用户输入处理
如需获取上一个方法调用后发生的事件:
events = window.get_events()
在 events
中每一个 event
是 ti.ui.Event
的一个实例。 它支持如下属性:
event.action
, 可以是ti.ui.PRESS
,ti.ui.RELEASE
, 或ti.ui.MOTI
event.key
: 与此事件相关的键。
如需获取光标位置:
window.get_cursor_pos()
如需检查一个键是否被按下:
window.is_pressed(key)
以下是一个来自 mpm128 的用户输入处理的示例:
while window.running:
# keyboard event processing
if window.get_event(ti.ui.PRESS):
if window.event.key == 'r': reset()
elif window.event.key in [ti.ui.ESCAPE]: break
if window.event is not None: gravity[None] = [0, 0] # if had any event
if window.is_pressed(ti.ui.LEFT, 'a'): gravity[None][0] = -1
if window.is_pressed(ti.ui.RIGHT, 'd'): gravity[None][0] = 1
if window.is_pressed(ti.ui.UP, 'w'): gravity[None][1] = 1
if window.is_pressed(ti.ui.DOWN, 's'): gravity[None][1] = -1
# mouse event processing
mouse = window.get_cursor_pos()
# ...
if window.is_pressed(ti.ui.LMB):
attractor_strength[None] = 1
if window.is_pressed(ti.ui.RMB):
attractor_strength[None] = -1
图像 I/O
如需在窗口中将当前帧写入图像文件:
window.save_image(filename)
Note that you must call window.save_image()
before calling window.show()
.
离屏渲染
GGUI 支持在不显示窗口的情况下保存图像帧。 这也被称为“无头”渲染。 如需启用此模式,请在初始化窗口时将参数 show_window
设置为 False
。
window = ti.ui.Window('Window Title', (640, 360), show_window = False)
Then you can call window.save_image()
as normal and remove the window.show()
call at the end.