Meta has launched two new models, SAM 3 and SAM 3D, which enhance its open-source Segment Anything computer vision tools. SAM 3 focuses on identifying and tracking objects and people in images and videos using text prompts. It allows users to specify detailed descriptions of objects they want to segment, improving accuracy over previous models. For example, if a user types “red baseball cap,” SAM 3 can locate all instances of that item in the visual input. This capability is significant because it overcomes limitations of earlier models that struggled with complex descriptions. SAM 3D takes this a step further by reconstructing recognized objects and people in three dimensions. Users can transform 2D images into 3D models, which could be particularly impactful for applications like virtual reality or 3D modeling in various fields, including robotics and creative industries. For instance, someone could reconstruct a likeness of a deceased relative from a photo and use it in videos or virtual environments. While SAM 3 is fully open-sourced, SAM 3D has limited availability but will share some model checkpoints and code for the research community. Both models can be accessed through Meta's Segment Anything Playground, requiring no special expertise. Users can upload images or videos and input prompts to manipulate the visuals. Meta aims to integrate these capabilities into applications like its “View in Room” feature on Facebook Marketplace, allowing users to visualize how decor items look in their own homes.