Someone built a multi-GPU homelab for local AI inference using a Framework Desktop, RTX 5090 eGPU, and two DGX Spark units, routing requests between local and cloud models depending on latency needs. They're currently at 60-70% local inference and plan to move toward 100%, though they admit the eGPU was a mistake and you don't need this much hardware to start.

• The Qwen 3.8 27B model on the 5090 eGPU hits 150+ tokens/second for fast inference, while the Deepseek v4 Flash models on the DGX Sparks handle slower but higher-quality batch processing and background jobs.
• A custom routing plugin (Arch-Router) decides whether each request goes to local models or cloud/frontier models based on latency requirements, currently keeping about 60-70% of workloads local.
• The hardware stack is overkill for most people—started with just a Mac mini and kept adding. The eGPU specifically isn't worth the investment.

Kipi is a self-hosted OSINT platform that ingests documents and screenshots to build a typed entity graph, then runs automated WHOIS, DNS, certificate and web pivots. It applies real-time graph analytics (centrality, communities, pathfinding) and generates a sourced, graded brief while letting analysts confirm every finding. The only external dependency is an Anthropic API key; all data and processing run locally.

• Kipi turns dumped documents/screenshots into a typed entity graph and runs an autonomous investigation (WHOIS, DNS, certs, web pivots) with only an Anthropic API key required, everything else self-hosted and mostly keyless.
• Its 75-second demo traces two seed domains into a Russian-language crypto casino affiliate fraud network with 20,000+ affiliates stealing 60-80% of deposits, run through a Reykjavík shell company set up right after Brian Krebs exposed its predecessor.
• Every extracted fact is sourced and graded on an evidence scale (A = DNS record down to analyst leads), with analysts able to confirm, correct or reject each node/edge.
• Licensed under Elastic License 2.0: free to self-host or use for paid investigations, but can't be repackaged or resold as a service.

Mistral OCR 4 extracts text from PDFs, DOCs and more while also returning bounding boxes, block types and per-word confidence. It supports 170 languages, runs in a single container for self-hosted deployments, and outperforms rivals on human and automated benchmarks.

• Mistral OCR 4 doesn't just extract text—it returns bounding boxes, block types (titles, tables, equations, signatures), and per-word confidence scores, and can be self-hosted in a single container for data-sovereignty needs
• Beat every tested competitor in human evaluations across 600+ documents/12+ languages with a 72% average win rate, and scored 85.20 on OlmOCRBench and 93.07 on OmniDocBench
• Roughly 4x faster than some enterprise OCR providers, and 8x cheaper with 17x lower latency than leading agentic document parsers on finance datasets with charts/figures
• Priced at $4 per 1,000 pages via API ($2 with batch discount) or $5 per 1,000 pages through the no-code Document AI interface

Stash lets your AI agent keep memory across conversations, so you don’t have to repeat context every time. It’s an open-source, self-hosted layer that uses Postgres, pgvector, and an eight-stage consolidation pipeline to turn raw observations into structured knowledge. It plugs into any MCP-compatible agent with a single Docker command.

• Stash gives AI agents persistent memory across sessions using Postgres + pgvector, run via a self-hosted Docker stack (no cloud dependency).
• A background service runs an eight-stage consolidation pipeline (fact extraction, relationship mapping, causal links, goal tracking, failure patterns, hypothesis checks, confidence decay) that only processes new entries, keeping the knowledge base efficient as it grows.
• It works with any MCP-compatible agent (OpenAI Agents, Claude Desktop, Continue, Cursor, Windsurf, Ollama, OpenRouter) via a single Docker command setup.
• It's open source (Apache 2.0), letting users customize consolidation rules, swap models, or plug in custom retrieval logic.