Course Abstract

Introduction to Generative Adversarial Networks and their Applications （Daniel Cohen-Or）　

Abstract：In this tutorial talk we will describe Generative Adversarial Networks (GANs). We will start by explaining the motivation and the theory behind these networks, which are considered by some as one of the most important developments in Artificial Intelligence. Next we will survey various extensions and applications of GANs, including the recent exciting cycleGAN/dualGAN architecture.

Deeply Understanding Complex Human Activities and Scenes in the Wild Convexity Analysis （Yu Qiao）　

Abstract：Human activities and scenes understanding is receiving extensive research interests in computer vision nowadays due to its wide applications in surveillance, human-computer interface, sports video analysis, and content based video retrieval. The challenges of action and scene recognition come from background clutter, viewpoint changes, and motion and appearance variations. In this lecture, I will report our continuous efforts (CVPR13,14,15,16, ICCV13, ECCV 14, 15, TIP14, TIP 17, IJCV 16) to address these challenges. These works range from mining middle level parts, multi-view encoding of local descriptors, hierarchical model, to utilizing deep models for action recognition and detection. Experimental results on large public datasets (e.g. UCF101, HMDB51, SUN, LSUN) demonstrate the effectiveness of the proposed methods.

Deep Learning Techniques for Sketch-Based Face Modeling and Automatic Image/Video Stylization （Yizhou Yu）　

Abstract：Face modeling has been paid much attention in the field of visual computing. There exist many scenarios, including cartoon characters, avatars for social media, 3D face caricatures as well as face-related art and design, where low-cost interactive face modeling is a popular approach especially among amateur users. In this lecture, I present a deep learning based sketching system for 3D face and caricature modeling. This system has a labor-efficient sketching interface, that allows the user to draw freehand imprecise yet expressive 2D lines representing the contours of facial features. A novel CNN based deep regression network is designed for inferring 3D face models from 2D sketches. Our system also supports gesture based interactions for users to further manipulate initial face models. Both user studies and numerical results indicate that our sketching system can help users create face models quickly and effectively. A significantly expanded face database with diverse identities, expressions and levels of exaggeration is constructed to promote further research and evaluation of face modeling techniques. Color and tone stylization in images and videos strives to enhance unique themes with artistic color and tone adjustments. It has a broad range of applications from professional image postprocessing to photo sharing over social networks. I also present a novel deep learning architecture for exemplar-based image stylization, which learns local enhancement styles from image pairs. Image stylization can be efficiently accomplished with a single forward pass through our deep network. To extend our deep network from image stylization to video stylization, we exploit temporal superpixels (TSPs) to facilitate the transfer of artistic styles from image exemplars to videos. Experiments on a number of datasets for image stylization as well as a diverse set of video clips demonstrate the effectiveness of our deep learning architecture.

Deeply Understanding Complex Human Activities and Scenes in the Wild Convexity Analysis （Andrei Sharf）　

Abstract：Convolutional neural networks (CNNs) require data from a regular domain, which prohibits direct processing of geometric data that is irregular by nature, such as point clouds and meshes. We present an approach for point-cloud analysis using a neural network that samples geometric information and maps it to a representation suitable for CNN consumption. A sensor, composed of a regular grid of probes embedded in space, samples vector-field values induced by the geometry. The captured information can then be readily processed by a CNN, extracting high-level features. Having the sensor module as an integral part of the network enables an iterative scheme in which the net applies multiple geometric transformations to both the sensor and the shape. We formulate several variations of the sensor module and discuss their merits, describe approaches to a few shape-analysis tasks using this framework, reason about its scalability, and present preliminary experiments and results on shape detection and classification.

3D Fabrication For Everyone （Ariel Shamir）　

Abstract：With the progress of 3D printing and other fabrication techniques, the creation of real 3D object has become more accessible. This allowed the emergence of a new and exciting "makers movement".However, if designing a 3D object for virtual world (movies, games) was a difficult task for non-experts, then designing 3D objects that need to be manufactured and function is even more challenging. In this talk, I will describe several solutions that try to meet this challenge. One solution allows novice users to create new objects by taking parts from existing designs. Others allows the creation of connectors between objects, or designing foldable objects automatically. In all cases, behind the scene the system takes care of tedious and difficult tasks such as connecting constraints and physical feasibility.

Data driven appearance modeling （Yue Dong）　

Abstract：Digitally reproducing the visual richness of the appearance of real-world materials is a challenging and important problem in computer graphics. In this talk, I will introduce several works in data driven appearance modeling and acquisitions. The main axis of those works are data coherency, I will discuss how different data coherency leads to various solutions and the relationship between those methods. Finally, I will also present our latest work that adapting deep learning techniques for appearance modeling.

Visual Analytics of Big Data （Huamin Qu）　

Abstract：Big data is large and complex. Researchers have developed advanced data mining and machine learning techniques to reveal patterns in big data. However, people without a mathematics or computer science background may find these methods and findings difficult to understand. Also not all problems can be solved by automatic methods. Data visualization, which turns data into visual forms and keep human in the analytical loop, is widely considered a key to big data analytics. In this talk, I will first introduce the opportunities and challenges for visual analytics of big data. I will then present ongoing research projects at HKUST and demonstrate how visualization helps reveal rumor propagation on social media such as Twitter and WeChat, learning behaviors of students on MOOC platform, and human mobility patterns based on mobile phone and transportation data.

High Quality Hex Mesh Generation （Jin Huang）　

Abstract：Mesh is the most popular discretization method in many fields. What is a good mesh and how to get a good mesh are very basic problems. In this talk, I will briefly introduce some basic knowledge about them, then focus on our progress of answering these two questions. First, we developed a very efficient method of estimating condition number, the numerical stability indicator of a mesh. Second, we show how to address the key challenges in hex remeshing, the “holy grail” problem in remeshing, and discuss the possible future works.

From Symmetry to Functionality: An Evolution to Understand 3D Shapes （Hao Zhang）　

Abstract：Symmetry is ubiquitous in nature and man-made artifacts. In this talk, I will first show how symmetry analysis and organization can play a crucial role in understanding a 3D shape in a hierarchical way. In particular, it may seem surprising that one can infer the generative history of a pattern by analyzing its symmetry alone. Furthermore, I will show how a symmetry-induced hierarchical representation is the key to allow a machine to learn a generative and structure-aware model for 3D shapes. With an intimate connection between symmetry and functionality (e.g., symmetric parts in a 3D object tend to perform the same function), symmetry provides a first cue to understand object functionality, but the key missing piece is how an object interacts with its environment to perform its functions. In the second part of my talk, I will cover our recent works on functional understanding of 3D shapes, evolving from analyses in a static setting to inferring part motion from a single static snapshot.

Holistic 3D Scene Understanding （Manolis Savv）　

Abstract：The tremendous progress of RGBD sensing technology and recent growth of organized 3D model repositories has led to a deluge of data that captures and represents the structure and semantics of real-world scenes. With this increasing availability of 3D scene data, research challenges are shifting towards analysis and high-level understanding of scene semantics.This talk will describe recent work exploring 3D scene understanding along three axes: i) large-scale acquisition and semantic labeling of RGBD reconstructions, ii) leveraging large-scale virtual 3D scene data to jointly predict the volumetric occupancy and semantics of a scene from single RGBD frames, and iii) investigating the impact of different rendering methods when generating data from virtual scenes to train deep learning methods for tasks such as normal prediction, semantic segmentation, and edge prediction.

Network visualization and its application to academic influence analysis （Lei Shi）　

Abstract：Visually analyzing academic influence graphs, such as those in the citation network, poses challenge to many fields of the computer science research. How can we summarize a large graph according to user's interest? In particular, how can we illustrate the impact of a highly influential paper through the summarization? Can we maintain the sensory node-link visual metaphor while preserving both the influence flow patterns and fine readability? How to visualize the evolutionary pattern as the influence propagates? This talk starts from basics on network visualization, including the design rationale, various layout considerations and evaluation. After that, we demonstrate the visualization and data mining techniques developed in recent years to solve academic influence graph analysis problem.

Global Structure-from-Motion and Its Application （Zhaopeng Cui）　

Abstract：Structure-from-motion (SfM) is a fundamental problem in 3D computer vision, with the aim of recovering camera poses and 3D scene structure simultaneously given a set of 2D images. SfM methods can be broadly divided into incremental and global methods according to their ways to register cameras. Incremental methods register cameras one by one, while global SfM methods solve all cameras simultaneously from all available relative motions. As a result, global SfM has better potential in both reconstruction accuracy and computation efficiency than incremental SfM. In this talk, I will first present our latest global SfM system which could deal with all kinds of camera motions and datasets. Then I will present our new robust video alignment algorithm based on global SfM, which can align videos taken at different times with substantially different appearances, in the presence of moving objects and moving cameras with slightly different trajectories.

Gradient Domain Manipulation （Dani Lischinski）　

Abstract：Gradient domain manipulation is an important technique with multiple applications in image and video tone mapping and editing. In this talk, I will explain the relevant underlying mathematical theory and show how operating in the gradient domain can be used for tone mapping of High Dynamic Range (HDR) images, as well as for seamless image cloning and stitching.

Data Visualization: an Introduction （Oliver Deussen）　

Abstract：In data visualization large amounts of data are converted into images in order to use the human eye to find interesting patterns that can later be analysed by data mining techniques. In my talk I will give brief overview about various data visualization techniques and will mention the most important research questions of this field.

Style and Functionality Analysis of 3D Shapes （Ruizhen Hu）　

Abstract：The majority of man-made objects are designed to serve a certain function and process a certain style, and they are often reflected by the geometry of the objects. In recent years, many efforts in shape analysis have developed methods that extract high-level structural and semantic information from geometric shapes and scenes, especially involving man-made objects. Style and functionality have been receiving increasingly more attention in shape analysis and geometric modeling. In this talk, I will talk about recent developments that incorporate style and functionality aspects in the analysis of 3D shape.

Learning Image Representation for Classification and Semantic Segmentation （Di Lin）　

Abstract：Image classification and semantic segmentation have been long standing topics in high-level computer vision. They emphasize understanding of scene context and object. Towards image classification, we address the problem regarding appropriate representation of the scene structure and differentiation among object categories. Compared to image classification, semantic segmentation task faces more challenges. Not only the global information of the image, but also the underlying relationship among pixels need to be captured by the image representation. Therefore large-scale data is vital to enable reliable learning of representation. We propose a weakly-supervised learning system for semantic segmentation, which can be used in practice to improve segmentation with cost-effective data.

Pattern-Aware Arrangement of Graphic Elements （Pengfei Xu）　

Abstract：Arrangement of graphic elements is a fundamental task in many text and graphic editing scenarios. Traditional interactive techniques such as snapping and arrangement commands are provided by most graphic editors to aid the users in accomplishing this task. However, these techniques often ignore the underlying patterns of the elements, requiring the users to perform explicit and tedious operations to achieve the goal of the manipulation. It is expected that the amount of user interaction can be reduced by exploiting the patterns of the elements. Nevertheless, designing effective pattern-aware interaction tools is challenging due to the ambiguities in the patterns presented by the elements and the operations issued by the user. In this talk, I will introduce two effective pattern-aware interaction techniques to aid the users in the arrangement of elements. In particular, I will introduce 1) GACA, a group-aware command-based arrangement tool for arranging multiple groups of elements with a single command, and 2) A framework for automatic global beautification of layouts of graphic elements with gestural interface for editing the patterns of the elements. These two techniques adopt different strategies to resolve the ambiguities that may arise during manipulation, ensuring their usability.

Capture and Reconstruction of Blooming Flowers （Qian Zheng）　

Abstract：Advances in affordable three-dimensional (3D) acquisition devices now provide new opportunities in capturing and modelling 3D real-life phenomena in time. In my talk I will present our recent work on capturing and reconstructing spatially and temporally coherent sequences of blooming flowers, and talk about some common techniques for registration and tracking.