一个新的 UI 系统：GGUI

分类	先决条件
操作系统	Windows / Linux / Mac OS X
后端	x64 / CUDA / Vulkan

从 v0.8.0开始，Taichi 添加了一个新的 GGUI系统。 新的 GUI 系统使用 GPU 进行渲染，使得 3 维场景的渲染更快。 这就是为什么这一新系统被命名为 GGUI。 本文档将介绍其提供的 API。

IMPORTANT

如果你选择 Vulkan 作为后端, 确保你已经 install the Vulkan environment.

note

我们推荐您通过 examples/ggui_examples 提供的示例熟悉 GGUI

创建一个窗口

ti.ui.Window(name, res) 可以创建一个窗口。

window = ti.ui.Window('Window Title', res = (640, 360), pos = (150, 150))

The argument res means resulotion(width and height) of the window, pos means the position of the window which origins from the left-top of your main screen.

以下三类对象可以显示在 ti.ui.window 上：

• 2D 画布，可用于绘制简单的2D几何形状，如圆形、三角形等。
• 3D 场景，可用来渲染3D网格和粒子，具有可配置的相机和光源。
• Immediate mode GUI components, for example buttons and textboxes.

2D 画布

创建画布

以下代码获取了一个覆盖整个窗口的 Canvas 对象。

canvas = window.get_canvas()

在画布上绘画

canvas.set_background_color(color)
canvas.triangles(vertices, color, indices, per_vertex_color)
canvas.circles(vertices, radius, color, per_vertex_color)
canvas.lines(vertices, width, indices, color, per_vertex_color)
canvas.set_image(image)

参数 vertices, indices, per_vertex_color, 和 image 必须是 Taichi field。 如果提供了 per_vertex_color ，color 将会被忽略。

画布中几何图案的位置/中心的相对位置以在 0.0 和 1.0 之间的浮点数表示。 对于 circles() and lines()，radius 和 width 参数都是窗口高度的相对值。

画布在每帧后被清空。 务必在渲染循环中调用这些方法。

3D 场景

创建场景

scene = ti.ui.Scene()

配置相机

camera = ti.ui.Camera()
camera.position(pos)
camera.lookat(pos)
camera.up(dir)
camera.projection_mode(mode)
scene.set_camera(camera)

配置光源

添加点光源

调用 point_light() 将点光源添加到场景中。

scene.point_light(pos, color)

请注意，你需要为每一帧调用 point_light() 方法。 与 canvas() 方法类似，请在你的渲染循环中调用该方法。

3D 几何图形

scene.lines(vertices, width, indices, color, per_vertex_color)
scene.mesh(vertices, indices, normals, color, per_vertex_color)
scene.particles(vertices, radius, color, per_vertex_color)

参数 vertices, indices, per_vertex_color 和 image 都被设计为 Taichi fields 类型。 如果提供了 per_vertex_color ，color 将会被忽略。

几何体的位置/中心应设在世界空间的坐标上。

note

如果一个网格有 num 个三角形， indices 应该是一个大小为(num * 3) 的 1D 标量 field，而不是一个 向量 field。

normals 是 scene.mesh() 的一个可选参数。

1. An example of drawing 3d-lines

import taichi as ti

ti.init(arch=ti.cuda)

N = 10

particles_pos = ti.Vector.field(3, dtype=ti.f32, shape = N)
points_pos = ti.Vector.field(3, dtype=ti.f32, shape = N)

@ti.kernel
def init_points_pos(points : ti.template()):
    for i in range(points.shape[0]):
        points[i] = [i for j in ti.static(range(3))]

init_points_pos(particles_pos)
init_points_pos(points_pos)

window = ti.ui.Window("Test for Drawing 3d-lines", (768, 768))
canvas = window.get_canvas()
scene = ti.ui.Scene()
camera = ti.ui.Camera()
camera.position(5, 2, 2)

while window.running:
    camera.track_user_inputs(window, movement_speed=0.03, hold_key=ti.ui.RMB)
    scene.set_camera(camera)
    scene.ambient_light((0.8, 0.8, 0.8))
    scene.point_light(pos=(0.5, 1.5, 1.5), color=(1, 1, 1))

    scene.particles(particles_pos, color = (0.68, 0.26, 0.19), radius = 0.1)
    # Draw 3d-lines in the scene
    scene.lines(points_pos, color = (0.28, 0.68, 0.99), width = 5.0)
    canvas.scene(scene)
    window.show()

Advanced 3d Geometries

scene.lines(vertices, width, indices, color, per_vertex_color, vertex_offset, vertex_count, index_offset, index_count)

scene.mesh(vertices, indices, normals, color, per_vertex_color, vertex_offset, vertex_count, index_offset, index_count, show_wireframe)

scene.particles(vertices, radius, color, per_vertex_color, index_offset, index_count)

scene.mesh_instance(vertices, indices, normals, color, per_vertex_color, vertex_offset, vertex_count, index_offset, index_count, show_wireframe)

The additional arguments vertex_offset, vertex_count, index_offset and index_count control the visible part of the particles and mesh. For the mesh() and mesh_instance() methods, set whether to show wireframe mode through setting show_wireframe.

:::example

1. Example of drawing a part of the mesh/particles

# For particles
# draw the 2-th to 7-th particles
scene.particles(center, radius,
index_offset = 1,
index_count = 6)

# For mesh
# 1. with indices
scene.mesh(vertices, indices,
index_offset  = user_defined_first_indices_index,
index_count   = user_defined_index_count,
# vertex_offset is set to 0 by default, and it is not necessary
# to assign vertex_offset a value that otherwise you must.
vertex_offset = user_defined_vertex_offset)

# usually used as below:
# draw the 11-th to 111-th mesh vertexes
scene.mesh(vertices, indices,
index_offset  = 10,
index_count   = 100)

# 2. without indices (similar to the particles' example above)
scene.mesh(vertices,
vertex_offset = user_defined_first_vertex_index,
vertex_count  = user_defined_vertex_count)

2. An example of drawing part of lines

import taichi as ti

ti.init(arch=ti.cuda)

N = 10

particles_pos = ti.Vector.field(3, dtype=ti.f32, shape = N)
points_pos = ti.Vector.field(3, dtype=ti.f32, shape = N)
points_indices = ti.Vector.field(1, dtype=ti.i32, shape = N)

@ti.kernel
def init_points_pos(points : ti.template()):
    for i in range(points.shape[0]):
        points[i] = [i for j in range(3)]
        # points[i] = [ti.sin(i * 1.0), i * 0.2, ti.cos(i * 1.0)]

@ti.kernel
def init_points_indices(points_indices : ti.template()):
    for i in range(N):
        points_indices[i][0] = i // 2 + i % 2

init_points_pos(particles_pos)
init_points_pos(points_pos)
init_points_indices(points_indices)

window = ti.ui.Window("Test for Drawing 3d-lines", (768, 768))
canvas = window.get_canvas()
scene = ti.ui.Scene()
camera = ti.ui.Camera()
camera.position(5, 2, 2)

while window.running:
    camera.track_user_inputs(window, movement_speed=0.03, hold_key=ti.ui.RMB)
    scene.set_camera(camera)
    scene.ambient_light((0.8, 0.8, 0.8))
    scene.point_light(pos=(0.5, 1.5, 1.5), color=(1, 1, 1))

    scene.particles(particles_pos, color = (0.68, 0.26, 0.19), radius = 0.1)
    # Here you will get visible part from the 3rd point with (N - 4) points.
    scene.lines(points_pos, color = (0.28, 0.68, 0.99), width = 5.0, vertex_count = N - 4, vertex_offset = 2)
    # Using indices to indicate which vertex to use
    # scene.lines(points_pos, color = (0.28, 0.68, 0.99), width = 5.0, indices = points_indices)
    # Case 1, vertex_count will be changed to N - 2 when drawing.
    # scene.lines(points_pos, color = (0.28, 0.68, 0.99), width = 5.0, vertex_count = N - 1, vertex_offset = 0)
    # Case 2, vertex_count will be changed to N - 2 when drawing.
    # scene.lines(points_pos, color = (0.28, 0.68, 0.99), width = 5.0, vertex_count = N, vertex_offset = 2)
    canvas.scene(scene)
    window.show()

3. Details of mesh instancing

num_instance  = 100
m_transforms = ti.Matrix.field(4, 4, dtype = ti.f32, shape = num_instance)


# For example: An object is scaled by 2, rotated by rotMat, and translated by t = [1, 2, 3], then
#
# The ScaleMatrix is:
# 2, 0, 0, 0
# 0, 2, 0, 0
# 0, 0, 2, 0
# 0, 0, 0, 1
#
# The RotationMatrix is:
# https://en.wikipedia.org/wiki/Rotation_matrix#General_rotations
#
# The TranslationMatrix is:
# 1, 0, 0, 1
# 0, 1, 0, 2
# 0, 0, 1, 3
# 0, 0, 0, 1
#
# Let TransformMatrix = TranslationMatrix @ RotationMatrix @ ScaleMatrix, then the final TransformMatrix is:
#   2 * rotMat00,     rotMat01,       rotMat02, 1
#       rotMat10, 2 * rotMat11,       rotMat12, 2
#       rotMat20,     rotMat21,   2 * rotMat22, 3
#              0,            0,              0, 1
...

# Draw mesh instances (from the 1st instance)
scene.mesh_instance(vertices, indices, transforms = m_transforms, instance_offset = 1)

4. Example of setting wireframe mode

window = ti.ui.Window("Display Mesh", (1024, 1024), vsync=True)
canvas = window.get_canvas()
scene = ti.ui.Scene()
camera = ti.ui.Camera()

# slider_int usage
some_int_type_value = 0
def show_options():
    global some_int_type_value

    window.GUI.begin("Display Panel", 0.05, 0.1, 0.2, 0.15)
    display_mode = window.GUI.slider_int("Value Range", some_int_type_value, 0, 5)
    window.GUI.end()

while window.running:

    ...
    # if to show wireframe
    scene.mesh_instance(vertices, indices, instance_count = 100 , show_wireframe = True)

    canvas.scene(scene)
    show_options()
    window.show()

note

If indices is not provided, consider using like this:

scene.mesh(vertices, normals, color, per_vertex_color, vertex_offset, vertex_count, wireframe)

If indices is provided, consider using like this:

scene.mesh(vertices, indices, normals, color, per_vertex_color, vertex_offset, index_offset, index_count, wireframe)

渲染场景

您可以在画布上渲染场景。

canvas.scene(scene)

Fetching Color/Depth information

img = window.get_image_buffer_as_numpy()
window.get_depth_buffer(scene_depth)
depth = window.get_depth_buffer_as_numpy()

After rendering the current scene, you can fetch the color and depth information of the current scene using get_image_buffer_as_numpy() and get_depth_buffer_as_numpy(), which copy the gpu data to a NumPy array(cpu). get_depth_buffer() copies the GPU data to a Taichi field (depend on the arch you choose) or copies data from GPU to GPU.

:::example

1. Example of fetching color information

window = ti.ui.Window("Test for getting image buffer from ggui", (768, 768), vsync=True)
video_manager = ti.tools.VideoManager("OutputDir")

while window.running:
    render_scene()
    img = window.get_image_buffer_as_numpy()
    video_manager.write_frame(img)
    window.show()

video_manager.make_video(gif=True, mp4=True)

2. An example of fetching the depth data

window_shape = (720, 1080)
window = ti.ui.Window("Test for copy depth data", window_shape)
canvas = window.get_canvas()
scene = ti.ui.Scene()
camera = ti.ui.Camera()

# Get the shape of the window
w, h = window.get_window_shape()
# The field/ndarray stores the depth information, and must be of the ti.f32 data type and have a 2d shape.
# or, in other words, the shape must equal the window's shape
scene_depth = ti.ndarray(ti.f32, shape = (w, h))
# scene_depth = ti.field(ti.f32, shape = (w, h))

while window.running:
    render()
    canvas.scene(scene)
    window.get_depth_buffer(scene_depth)
    window.show()

GUI 组件

GGUI 的 GUI 组件设计遵循 Dear ImGui API。

gui = window.get_gui()
with gui.sub_window(name, x, y, width, height):
    gui.text(text)
    is_clicked = gui.button(name)
    new_value = gui.slider_float(name, old_value, min_value, max_value)
    new_color = gui.color_edit_3(name, old_color)

显示窗口

调用 show() 来显示一个窗口。

...
window.show()

请 仅 在每一帧结束时再调用该方法完成渲染。

用户输入处理

如需获取上一个方法调用后发生的事件：

events = window.get_events()

在 events 中每一个 event 是 ti.ui.Event 的一个实例。 它支持如下属性：

• event.action, 可以是 ti.ui.PRESS, ti.ui.RELEASE, 或 ti.ui.MOTI
• event.key: 与此事件相关的键。

如需获取光标位置：

• window.get_cursor_pos()

如需检查一个键是否被按下：

• window.is_pressed(key)

以下是一个来自 mpm128 的用户输入处理的示例：

while window.running:
    # keyboard event processing
    if window.get_event(ti.ui.PRESS):
        if window.event.key == 'r': reset()
        elif window.event.key in [ti.ui.ESCAPE]: break
    if window.event is not None: gravity[None] = [0, 0]  # if had any event
    if window.is_pressed(ti.ui.LEFT, 'a'): gravity[None][0] = -1
    if window.is_pressed(ti.ui.RIGHT, 'd'): gravity[None][0] = 1
    if window.is_pressed(ti.ui.UP, 'w'): gravity[None][1] = 1
    if window.is_pressed(ti.ui.DOWN, 's'): gravity[None][1] = -1

    # mouse event processing
    mouse = window.get_cursor_pos()
    # ...
    if window.is_pressed(ti.ui.LMB):
        attractor_strength[None] = 1
    if window.is_pressed(ti.ui.RMB):
        attractor_strength[None] = -1

图像 I/O

如需在窗口中将当前帧写入图像文件：

window.save_image(filename)

Note that you must call window.save_image() before calling window.show().

离屏渲染

GGUI 支持在不显示窗口的情况下保存图像帧。 这也被称为“无头”渲染。 如需启用此模式，请在初始化窗口时将参数 show_window 设置为 False。

window = ti.ui.Window('Window Title', (640, 360), show_window = False)

Then you can call window.save_image() as normal and remove the window.show() call at the end.