PolicyLM-1.7B | Model library
Short summary
PolicyLM is a 1.7-billion-parameter open-source model that flags policy violations in text by scoring messages against custom rules defined at inference time, without needing retraining. It returns violation scores and threshold-based flags for content moderation across 19 languages.
- Custom policies can be written and applied at inference time without model retraining, making it adaptable to different platform rules on the fly.
- The model scores multiple policy categories in a single pass and is designed for low-latency screening of live chat, comments, and user posts.
- It's released under Apache 2.0 and evaluated on multilingual text, but teams must validate score thresholds on their own data and should only use it for individual messages, not images or conversation context.
Full summary
PolicyLM-1.7B is a content moderation classifier built by Musubi with 1.7 billion parameters, designed to flag policy violations in text without generating responses. Instead of producing free-form outputs, it scores messages against custom policies or a built-in safety taxonomy and returns violation flags. The key advantage is that you can define custom policies at inference time—no retraining needed—which makes it practical for live applications like chat, comments, and user-generated posts. It's open-weight and released under Apache 2.0, so you can run it yourself.
The model is built for speed, scoring multiple policy categories in a single pass and working across 19 languages. You pass in a message along with policy definitions (each with a violation rule, non-violation rule, and exception override), and you get back per-category scores and boolean flags. The example in the article shows a threatening message that also requests credentials; PolicyLM returns scores of 0.96 and 0.99 respectively, flagging both violations. The scores are what matter most—teams need to set their own thresholds based on their specific data and tolerance levels.
One important limitation: PolicyLM is meant for individual text messages only, not images, conversation threads, or situations requiring explanations. You're responsible for validating score thresholds on your own content before deploying it. It's a screening tool first, useful for filtering at scale, but it won't handle every edge case your platform might encounter.