A method and apparatus that renders a mesh from one or more virtual camera views of an object, the one or more virtual camera views being projections of the mesh onto a plurality of 2D planes; inputs the one or more virtual camera views into a fully-convolutional model for real-time segmentation; generates one or more 2D pixel-wise masks based on the fully-convolutional model, the 2D pixel-wise masks being associated with one or more segmented elements; performs remeshing and mesh optimization using the one or more 2D pixel-wise masks; and generates an output mesh based on the remeshing and mesh optimization.