Sceneseer-3D Scene Generation from Text Description

Automatic scene generation using text offers a unique multimedia approach to classic story telling and human computer interaction with 3D graphics. In this paper, we present a newly developed system that generates 3D scenes from natural language text input. Our system is intended to bene�t non-graphics domain users and applications by providing advanced scene production through an automatic system. Scene descriptions are constructed in real-time using a method for depicting spatial relationships between different objects. The advantage of natural language editing interfaces is strongest when performing semantic operations at the sub-scene level acting on group of objects. We learn how to manipulate these sub-scenes by analyzing existing 3D scenes. We are parsing user’s natural language command and then transforming it into a semantic scene graph which is used to retrieve corresponding matching sub-scenes from the databases. We then augment this retrieved sub-scene by adding other objects from the text description. Finally, a new 3D scene is synthesized by aligning the augmented sub-scene with the user’s current scene, where new objects are spliced into the environment, possibly triggering appropriate adjustments to the existing scene arrangement.