Smart Video Manipulation Leveraging Object Trajectories and Background Frames

Conventional video editing interfaces primarily represent a video as a sequence of frames aligned on a timeline, which makes object-level editing both time-consuming and unintuitive. In this work, we propose an object-centric interaction framework for long-shot video manipulation by introducing operators on three semantic levels: background mosaics, object trajectories, and camera motions. The static background is estimated independently, while object movements are encoded as 3D space–time trajectories that serve as primary interaction primitives. Temporal object manipulations are achieved through direct and intuitive trajectory editing. Furthermore, camera operations are modeled as a scalable and movable aperture, enabling users to simulate pan, tilt, and zoom effects by constructing new camera paths. Experimental demonstrations highlight that the proposed representation and interaction operators enable efficient and user-friendly execution of complex, high-level video editing tasks.