Click any tag below to further narrow down your results
Links
The article shows how LLMs use discrete role tags (like <user>, <think>, <tool>) to structure context but also rely on writing style, causing them to misclassify malicious commands in tool-tagged text as user instructions. Using linear probes to measure internal “roleness” (CoTness, Userness, etc.), the authors reveal why prompt injections succeed and suggest focusing research on the science of roles to defend against such attacks.
- LLMs classify roles more by writing style/content than by the actual tags surrounding text, so tool-tagged content that reads like a command gets internally treated as a user instruction
- Researchers used linear probes on hidden activations (wrapping neutral text in different role tags) to show this "content overrides tag" effect directly in the model's internal representations
- This explains why prompt injections achieve near-100% success against frontier models in practice even though those same models score well on static benchmarks that only test memorized attack patterns
- The fix isn't more attack examples but strengthening models' internal tracking of role boundaries—a "science of roles" approach to defense
This article shows how to use the Claude Compliance API to feed full chat and file content into your SIEM, then write detections for AI-specific threats that live in the content. It covers prompt injections in uploaded files, jailbreaks, system-prompt leaks, data-laundering requests, sensitive disclosures, and insider data-flow risks.
- Claude's Compliance API streams full chat and file content into your SIEM, not just auth/admin logs—so an unrecognized identity querying compliance_api_accessed can see every chat in the org.
- Uploaded files can hide jailbreak instructions via zero-width characters, white-on-white text, or base64 blobs, so file_uploaded/project_document_created events need scanning for instruction-like language and suspicious encoding.
- Detections should focus on the assistant's output, not the prompt: a jailbreak attempt only matters if disallowed content or leaked system instructions actually appear in the response.
- Requests to encode, paraphrase, or strip PII from text are exfiltration red flags, and only the model's actual response confirms whether sensitive data got through.
Google integrates its computer use tool into Gemini 3.5 Flash, allowing agents to see, reason and act across browser, mobile and desktop environments for tasks like continuous software testing and accessibility audits. It uses adversarial training plus optional safeguards—explicit confirmations and auto-stop triggers—to reduce prompt-injection risks, and is accessible via the Gemini API and Enterprise Agent Platform.
- Gemini 3.5 Flash now has computer use built directly into the main model, merging what was previously a standalone Gemini 2.5 feature so the same model handles chat, function-calling, and UI automation across browser, mobile, and desktop.
- Google used targeted adversarial training to curb prompt-injection attacks, plus optional safeguards—mandatory user confirmation before sensitive/irreversible actions and auto-stop on suspicious prompt patterns.
- Available now via the Gemini API and Gemini Enterprise Agent Platform, with a live demo on Browserbase and reference code in the docs.
This post breaks down Amazon’s lawsuit against Perplexity over its Comet AI agentic browser, which browses and transacts on users’ behalf while disguising itself as Chrome. It explains how these browsers work, the security risks they introduce—like prompt injection attacks—and why sites like Amazon demand transparent agent identification.
- Amazon's lawsuit claims Comet disguises itself as Chrome, hiding the AI agent's true identity from Amazon's systems.
- Agentic browsers share the user's actual cookies/passwords/session data, so a compromised AI agent can act with full access as if it were the user.
- Amazon's complaint cites a public report of prompt-injection attacks hijacking embedded AI assistants to steal private data.
- Amazon argues Comet's covert automation bypasses and degrades its personalized shopping features built over years.
This issue covers a new AI-built SQL parser for PostHog that’s 70× faster, Linear’s move from styled-components to StyleX for better React performance, and a deep dive into misleading database benchmarks. It also highlights tools like Gemini 3.5 Flash’s computer use feature, the peerd automation extension, a breakdown of prompt injection vulnerabilities, and other AI and dev updates.
- PostHog built an AI-generated SQL parser that's 70x faster than their old C++ ANTLR version, using property-based tests to guide the model, without changing the query language for users
- Linear is replacing styled-components with StyleX to reduce runtime style recalculations and cut style-sheet overhead, since styled-components hasn't kept up with React
- LLMs can't reliably distinguish system/user role tags because they follow tone rather than tag hierarchy, letting attackers embed commands like "ignore previous" inside user-level text to bypass safeguards
Gray Swan cofounders Zico Kolter and Matt Fredrikson explain why AI systems need a different security mindset, focusing on indirect prompt injection, agent vulnerabilities and correlated failures. They walk through automated red teaming tools like Shade and the Gray Swan Arena, discuss guardrails, and argue that bigger models aren’t inherently safer and require bespoke security, identity management, and compliance measures.
- Human red-teamers ranked only fourth in robustness testing against browser-based agents, behind specialized automated red-teaming models.
- Scaling model size doesn't automatically improve safety, and agents introduce new vulnerabilities distinct from traditional IT security risks.
- The "lethal trifecta" (untrusted data, private data, exfiltration paths) creates attack surfaces that make a major prompt-injection breach feel inevitable.
- Effective AI defense will require machine-driven interpreters and agent-native identity/permissions systems, since humans can't keep pace with automated attack tools.
Trail of Bits crafted four malicious agent skills that bypass Cisco, Vercel, and ClawHub scanners by padding code or embedding payloads in docx and bytecode. The article shows how Siri-AI’s integration with private cloud compute and search can expose messages, emails, and calendars through prompt injection. It also covers a PyPI worm that auto-executes a hades-setup.pth hook to steal cloud and GitHub tokens, urging audits and immediate token rotation.
- Trail of Bits bypassed Cisco, Vercel, and ClawHub agent-skill scanners using tricks like 10,000 newlines of padding, or hiding payloads in .docx files and compiled Python bytecode—none of the scanners caught the buried triggers.
- Siri-AI's use of Gemini via Apple Private Cloud Compute pulls in messages, email, notes, and calendars, making it vulnerable to prompt injection that can leak private data or trigger unwanted actions like forwarding sensitive material.
- Socket found a PyPI supply-chain attack hijacking 19 packages (37 malicious wheels) that drop a hades-setup.pth hook to run an embedded Bun runtime, stealing AWS/GCP/Azure, GitHub, npm, and SSH credentials and exfiltrating them to GitHub repos while disguising traffic as Anthropic API calls.
- Affected package names include bramin, executor-engine, executor-http, funcdesc, coolbox, dynamo-release, and magique—any match warrants full credential rotation and a CI/CD log audit.