Object Classification | Frigate
Short summary
Frigate lets you train lightweight MobileNetV2 models to classify tracked objects into finer categories (like identifying specific pets or detecting safety gear). The models require consistent predictions across multiple frames before assigning labels, which prevents false positives but means you need to carefully curate training data to avoid overfitting.
- Classification requires a two-step verification: confidence score above threshold (default 0.8) plus 60% consensus across at least 3 attempts, preventing single-frame misclassifications from triggering assignments.
- Training data quality matters far more than quantity—using dozens of nearly identical images actually degrades performance through overfitting, so you should start small with visually distinct examples and add hard cases (low confidence predictions, unusual angles/lighting) over time.
- The `none` class needs the opposite training approach from named classes: it should include visual diversity (small crops, partial views, false positives) so the model can recognize when something doesn't match your known categories, otherwise borderline detections get forced into named classes by default.
Full summary
Frigate lets you train custom MobileNetV2 models to classify tracked objects more granularly—think identifying whether a person is a delivery driver or resident, or whether a dog is your pet or a stranger's. The classification results show up in Frigate's UI, MQTT topics, Home Assistant sensors, and the HTTP API. Training requires an internet connection to download base weights, but once trained, everything runs offline. The process is lightweight on CPU during inference, though training itself hammers system resources for 1-3 minutes. You'll need a CPU with AVX + AVX2 instructions to make it work.
You define classes as the visual categories your model learns to distinguish. For a person object, you might create classes like "delivery_person," "resident," and "stranger," plus an automatic "none" class for things that don't fit. There are two ways to apply results: sub_label (one specific identity per object, like naming individual cats) or attribute (multiple independent properties, like whether someone's wearing a helmet). The system only assigns labels when two conditions are met—the confidence score hits your threshold (default 0.8) and after at least 3 attempts, 60% agree on the same class. This prevents false positives by demanding consistency across frames.
Training happens in the Frigate UI in two steps: first you name the model and define your classes, then you label example images generated from detected objects. The real trick is avoiding overfitting by keeping your training set small and visually diverse. Don't bulk-train dozens of nearly identical images—that teaches the model to recognize that exact moment rather than the actual class. Start with a small representative set, then iteratively improve by prioritizing hard examples (low confidence scores or unusual lighting/distance) that appear in the Recent Classifications tab over time.