Uncut Aerial Video via a Single Sketch

Computer Graphics Forum (Proceedings of PG 2018)

Hao Yang      Ke Xie      Shengqiu Huang     Hui Huang*
Shenzhen University

Figure 1: Given a 2D sketch (yellow) that the user casually drew on a map, a safe and suitable camera trajectory can be automatically generated in 3D, along which the drone’s flying velocity (color coded) is also optimized to better film the scene in a continuous manner.

Nowadays UAV filming is getting popular, more and more stunning aerial videos appearing online. Nonetheless, making a good uncut aerial video with only one-long-shot for the large-scale outdoor scenes is still quite challenging, no many eye-catching pieces available yet. It requires users to have both consummate drone controlling skill and good perception of filming aesthetics. If totally manual, the user has to simultaneously adjust the drone position and the mounted camera orientation during the whole flyby while trying to keep all operation changes executed smoothly. Recent research has proposed a number of planning tools for automatic or semi-automatic aerial videography, however, most requires rather complex user inputs and heavy computations.

In this paper, we propose a user-friendly system designed to simplify the input and automatically generate continuous camera moves to capture compelling aerial videos that users prefer to see without any post cutting or editing. Assume a rough 2.5D scene model that includes all the regions of interest are available, users are only required to casually draw a single sketch on the 2D map. Our system will analyze this rough sketch input, compute the corresponding quality views in 3D safe flying zone, and then create a globally optimal camera trajectory passing through regions of user interest via solving a combinatorial problem. At end, we optimize the drone flying speed locally to make the resulting aerial videos more visually pleasing.

Figure 2: Algorithm overview: starting from a simple 2D user sketch (a), our system can analyze it and sample a set of waypoints (b); then in pre-computed safe flight zone (c), by sampling above the sketch point and positioning virtual cameras on every height sample targeting different regions of interest, we are able to select local view candidates (d); an instance of STSP is constructed next to obtain the path with the lowest total cost (e); after flight speed optimization, the final 6D (position, orientation, speed) camera trajectory is generated (f).

Figure 3: How drone fly: (a) the constant speed input (green) and the output (purple) of speed optimization; (b) comparison of the optical flow before and after speed optimization.

Figure 4: A 6D camera trajectory is automatically generated from a simple 2D sketch. Please see the supplementary video for our beautiful uncut shots by the sea.

Figure 5: Two very different camera trajectories automatically generated from two different sketches drew at the same region around City Bay. Please see the supplementary video.

Figure 6: Various camera moves generated from six sketches drew by six different users for the same university scene.

We would like to thank the anonymous reviewers for their valuable comments. This work was supported in parts by NSFC (61522213, 61761146002, 61861130365), China Postdoc Foundation (2017M622780), 973 Program (2015CB352501), Guangdong Science and Technology Program (2015A030312015) and Shenzhen Innovation Program (KQJSCX20170727101233642).

title = {Uncut Aerial Video via a Single Sketch},
author = {Hao Yang and Ke Xie and Shengqiu Huang and Hui Huang},
journal = {Computer Graphics Forum},
volume = {37},
number = {7},
pages = {191--199},  
year = {2018},

Downloads (faster for people in China)

Downloads (faster for people in other places)