This project aims at the task of interactive object segmentation. We revive the old ideas on level set segmentation which framed object annotation as curve evolution. The Level Set Method can handle objects with complex shapes and topological changes such as merging and splitting, thus able to deal with occluded objects and objects with holes. We propose Deep Extreme Level Set Evolution (DELSE) that combines powerful CNN models with level set optimization in an end-to-end fashion. Our method learns to predict evolution parameters conditioned on the image and evolves the predicted initial contour to produce the final result. Carefully designed energy functions ensured that the curve was well aligned with image boundaries, and generally "well behaved". Our model is interactive by incorporating user clicks on the extreme boundary points and allows further correction on erroneous boundary.

The boundary performance of segmentation models is extremely important for the task of interactive annotation and some other real world applications. We framed object segmentation task as curve evolution, to avoid "scattered island" artifacts in pixel-wise classification models.

Instead of using explicitly represented curve like polygons, we used level set to implicitly represent the curve. This allows us to handle objects with complex shapes and topological changes such as merging and splitting.

Extreme points are encoded as a heat map and concatenated with the image, which are then passed to the encoder CNN. A multi-branch architecture is used to predict the initial level set function (a rough estimation through Level Set Branch) and evolution parameters (Motion Branch & Modulation Branch) used in level set evolution. Then DELSE evolves the initial estimated contour by iteratively updating the level set function.