, the first unified AI model for real-time object segmentation in video, unlocking new possibilities for video editing and mixed reality.){kind=link}
Meta has unveiled the Segment Anything Model 2 (SAM 2), its first unified model capable of identifying which pixels belong to a target object in an image or video. SAM 2 can segment any object and consistently follow it across all frames of a video in real-time, unlocking new possibilities for video editing and mixed reality experiences.
Segmentation, the process of identifying which image pixels belong to an object, assists with tasks like analysing scientific imagery or editing photos. The original Segment Anything Model, released last year, inspired new AI-enabled image editing tools in Facebook’s apps, such as Backdrop and Cutouts on Instagram. SAM has also catalysed diverse applications in science, medicine, and numerous other industries. For instance, SAM has been used in marine science to segment sonar images and analyse coral reefs, in satellite imagery analysis for disaster relief, and in the medical field for segmenting cellular images and aiding in detecting skin cancer.
The new Segment Anything Model 2 (SAM 2) extends these capabilities to video. SAM 2 can segment any object in an image or video and consistently follow it across all frames of a video in real-time. Existing models have struggled with video segmentation due to the complexity of objects moving quickly, changing in appearance, and being concealed by other objects or parts of the scene. Facebook has addressed many of these challenges in developing SAM 2.
Meta believes this research can unlock new possibilities such as easier video editing and generation, and allow new experiences to be created in mixed reality. SAM 2 could also be used to track a target object in a video to aid in faster annotation of visual data for training computer vision systems, including those used in autonomous vehicles. It could also enable creative ways of selecting and interacting with objects in real-time or live videos.
In line with its open science approach, Meta is sharing its research on SAM 2 so others can explore new capabilities and use cases. The company says it is eager to see what the AI community will achieve with this research.
Visit https://about.fb.com/