In the course of the last 7 years, deep learning has revolutionized many fields, including computer vision and computer graphics. In this talk, I will give an introduction and an overview of convolutional neural networks (CNNs), as well as generative adversarial networks (GANs) which have been extremely effective for a variety of generative tasks.
Traditional methods for motion comparison consider features from individual characters. However, the semantic meaning of many human activities is usually defined by the interaction between them, such as a high-five interaction of two characters. There is little success in adapting interaction-based features in activity comparison, analysis and synthesis, as they either do not have a fixed topology or are in high dimensional. In this talk, I will review some of the existing work in modelling the interactions in human activity. I will also introduce some of the recent work from our research group on the related topics. Finally, the challenges and future research directions will be discussed in the talk.
In shape analysis, several interesting applications require us to reason about the arrangement, layout, or geometric relationships between individual regions of a shape, as opposed to focusing on their individual properties in isolation. For simplicity, I refer to these non-unary properties as the structure of a shape. Applications such as high-level semantic editing of 2D or 3D objects, robust asset extraction from images or videos, exploration of large shape collections, assisted shape editing, or (semi-) automatic content generation, benefit greatly from working with this structure. However, shape structure has been less accessible to deep learning, since both the spatially local convolutions and the regular grid representation used traditionally in deep learning make working with structure unnecessarily hard. In this talk I am going to present alternatives we found to be more effective in our current research, such as graph representations for learning 2D layouts, and hierarchical graph networks for learning 3D shape structure, and I will show how the resulting structure-aware approach benefits several of the applications mentioned above.
近年来，随着计算机视觉技术的发展，对于图像识别的研究取得了长足的进展。尤其是以深度卷积网络为代表的广泛应用，越来越多的研究人员借助深度神经网络的强大学习能力来处理图像类别、属性以及像素类别的信息。目前大多数研究的侧重点在于对识别系统进行改进以提高对这些信息的识别性能。尽管识别性能有所提高，目前的研究在复杂的应用场景中仍然需要面对诸多的挑战。本次报告立足于图像基础维度信息的研究，使用深度学习将基础信息拓展到高层次图像识别应用中。
Virtual indoor scenes are a critical resource for furniture and interior design companies, architects, virtual and augmented reality experience designers, and computer vision and robotics researchers. Thus, a system which can synthesize novel indoor scenes would be valuable. In this tutorial, I’ll take students on a brief tour of this field of indoor scene synthesis: from its origins in rule/constraint-based systems, through the introduction of data-driven methods, and culminating with state-of-the-art research in deep generative models of scenes. We'll spend majority of our time discussing the most cutting-edge methods, including analyzing their strengths and weaknesses. The tutorial will conclude with a look toward the future of the field: open problems, opportunities, and ongoing work.
Visualization help people understand the data in an intuitive way. As the complexity of data increases, how to make the visualization visual clearly is challenging. This work will introduce our latest two works in visualization enhancements. One is about the visual design in scatterplot, Winglets, to strengthen the association perception and associating uncertainty. The other is enhancement technique, to improve the interactivity for web-based visualization.
智能计算机图形属人工智能与计算机图形学的交叉融合研究，强调从大量图形图像数据中提取特征和模式，在特征空间表达问题的语义约束并简化求解算法，有效提高了数据处理的效率和质量。由于深度学习技术为面向应用的特征学习提供了强有力的计算框架，迅速的在计算机图形学领域得到了广泛应用。本报告主要内容为如何应用深度学习进行图形图像理解编辑与仿真。通过计算机图形学的应用研究，报告如何针对图像和网格这两类数据的处理设计网络结构以高效计算其特征，控制网络中的梯度流动得到高质量处理结果。
Sketch-based interfaces have been adopted by a lot of 3D content creation applications, due to its ability of depicting complicated ideas with simple forms. In this talk, I will give a brief overview of sketch-based 3D content creation techniques, and will focus more on my recent related research on this topic.
In data visualization large amounts of data are converted into images in order to use the human eye to find interesting patterns that can later be analysed by data mining techniques. In my talk I will give brief overview about various data visualization techniques and will mention the most important research questions of this field.
The real world exhibits an abundance of non-stationary textures. Examples include textures with large scale structures, as well as spatially variant and inhomogeneous textures. While existing example-based texture synthesis methods can cope well with stationary textures, non-stationary textures still pose a considerable challenge, which remains unresolved. In this talk, I will first introduce our work on analysis and controlled synthesis of inhomogeneous textures, where traditional (non-deep) patch-based method is used. Then our recent work will be introduced on employing deep generative model for non-stationary texture synthesis. The power of deep neural networks make our method can cope with very challenging textures, which, to our knowledge, no other existing method can handle.
We present SAGNet, a structure-aware generative model for 3D shapes. Given a set of segmented objects of a certain class, the geometry of their parts and the pairwise relationships between them (the structure) are jointly learned and embedded in a latent space by an autoencoder. The encoder intertwines the geometry and structure features into a single latent code, while the decoder disentangles the features and reconstructs the geometry and structure of the 3D model. Our autoencoder consists of two branches, one for the structure and one for the geometry. The key idea is that during the analysis, the two branches exchange information between them, thereby learning the dependencies between structure and geometry and encoding two augmented features, which are then fused into a single latent code. This explicit intertwining of information enables separately controlling the geometry and the structure of the generated models. We evaluate the performance of our method and conduct an ablation study. We explicitly show that encoding of shapes accounts for both similarities in structure and geometry. A variety of quality results generated by SAGNet are presented.
Nowadays designers have to create different layouts for different screen sizes and orientations to get desirable results. This makes GUI design more time-consuming. Also, multiple specifications can be hard to maintain and to keep consistent. To solve this problem, we proposed a novel approach for constraint-based graphical user interface (GUI) layout based on OR-constraints (ORC) in standard soft/hard linear constraint systems. Our aim was to develop a technology that can adapt layouts to screens with different sizes, orientations, and aspect ratios based on only one single layout specification.
Nowadays UAV is getting popular, the applications based on low cost drones are getting very common, such as aerial video, large scene reconstructions, photographics by drone. However, for these applications, it's very hard for user to simultaneously control the drone's positions and orientations. In our study, we have been trying to make it automatic by predefining trajectories and real time flight command generations. In this talk, the newest research works of VCC@SZU in this field will be introduced to the students.
With rapid developments in 3D scanning and photogrammetry, it is now possible to efficiently capture static and dynamic objects in the real world. This attracts intensive research interests in object acquiring, analyzing, reconstructing and understanding. Nonetheless, obtaining a faithful 3D representation of real-world static and dynamic objects still remains an open problem due to various technical challenges. In this talk, I would like to introduce these difficulties, together with my work in the past few years with respect to solutions on reconstructing buildings, articulated objects, blooming flowers, and inferring human interaction motions and beyond.
In this talk, I will introduce our recent works on learning Deep Hierarchical Models for the context-aware understanding and structure-aware generation of 3D shapes. Complex 3D shapes are typically represented as a hierarchy of objects, which is evidenced by the widely adopted hierarchical scene graphs of CAD models. Deep convolutional neural networks achieve hierarchical representation learning with the help of pooling operations. Such feature hierarchies, however, do not explicitly capture and meaningfully model the spatial relations between different neuron activations, thus often resulting in unreasonable understanding and implausible generation. Our key insight is that 3D structures can be effectively characterized by a hierarchical organization of its constituent parts, which explicitly encodes part relations such as proximity, symmetry, etc. Through extending our GRASS model, a recursive neural network architecture for the hierarchical modeling of 3D shape structures, we develop a series of deep hierarchical models both for shape understanding and shape generation. We show that deep hierarchical models, with explicit relation modeling, attains much more powerful context-aware analysis and structure-aware synthesis than CNNs.
Robots can perform complex tasks in highly dynamic and unstructured environments. However, the vision system can be complex and prone to errors and designing the perception and control software for autonomous operation remains a major challenge. Reinforcement learning algorithms hold the promise of enabling robots to automatically learn new behaviors through experience. In this talk, I will discuss how the robots use reinforcement learning algorithms to learn manipulation skills from visual observation of the manipulator, without any prior knowledge of configuration or joint state. I will include several state-of-the-art reinforcement learning algorithms and give several recent applications for different vision-guided robotic manipulation tasks.
Autonomous 3D acquisition of outdoor environments poses special challenges. Different from indoor scenes, where the room space is delineated by clear boundaries and separations (e.g., walls and furniture), an outdoor environment is spacious and unbounded (thinking of a campus). Therefore, unlike for indoor scene where the scanning effort is mainly devoted to the discovery of boundary surfaces, scanning an open and unbounded area requires actively delimiting the extent of scanning region and dynamically planning a traverse path within that region. We approach the planning of an energy-efficient auto-scanning course for outdoor scenes, through formulating a discrete-continuous optimization of robot scanning paths. The discrete optimization computes a topological map, through solving an online Orienteering Problem (OP), which determines the scanning goals and paths on-the-fly. The dynamic goals are determined as a collection of visit sites with high reward of visibility-to-unknown. A visit graph is constructed via connecting the visit sites with edges weighted by traversing cost. This topological map evolves as the robot scans via deleting outdated sites that are either visited or become rewardless and inserting newly discovered ones. The continuous part optimizes the traverse paths geometrically between two neighboring visit sites via maximizing the information gain of scanning along the paths. The discrete and continuous processes alternate until the traverse cost of the current graph exceeds the remaining energy capacity of the robot. Our experiments demonstrate the effectiveness and advantages of the proposed method.
Introduction to Deep Learning and GANs （Dani Lischinski）
