Click any tag below to further narrow down your results
Links
PyPI has launched a redesigned project page with a new Security tab that explains where each package file came from, using attestations and provenance states. It also adds warnings for suspicious publishing changes, like a package switching build sources or losing its attestations.
- Each release is labeled Full Provenance (every file has an attestation) or No Provenance (none do), with the note that no provenance doesn't automatically mean malicious.
- Six warning types flag mismatches: incomplete, inconsistent, change of, and loss of provenance, plus late file warnings for files uploaded more than 14 days after release.
- The work was funded by the OpenSSF and built with Kabu Creative, following the OpenSSF Attestations Style Guide, and grew out of user research with the Python community.
Scrapling is an open-source Python library that combines fast HTTP fetching, stealth browser automation, adaptive parsing, and concurrent crawling in one package. It can be driven from Python code, the terminal, or AI agents through an MCP server.
- Handles the full scraping stack: plain HTTP requests, JavaScript rendering, Cloudflare Turnstile handling, and background capture of the API calls behind a page.
- Spider API is Scrapy-like, with concurrency limits, AutoThrottle that adjusts per-domain delays from server response, robots.txt compliance, pause/resume, and JSON/CSV/XML/JSONL export.
- Element tracking relocates selectors after a site changes its layout, and the library can turn whole sites into clean Markdown for RAG pipelines.
SLayer is an open-source Python semantic layer that sits on top of a data warehouse so AI agents and people can query metrics without writing SQL by hand. You define a column like revenue once, then query it with expressions such as avg(revenue) or time_shift(sum(revenue), -1, 'year').
- Unlike traditional BI semantic layers, which need every metric and aggregation predefined, SLayer lets agents combine metrics, apply transforms like time shifts, and change aggregations on the fly.
- It handles read-only database connectivity, SQL generation, and row-level security, so the LLM doesn't have to write raw SQL.
- It exposes the same definitions over MCP, REST, CLI, Python, Flight SQL, and a Postgres-style SQL API, and imports configs from dbt, Cube, and Ossie.
- Install is via uv, and it can be added to Claude Code as an MCP server with a demo Jaffle Shop database for a quick start. It sends aggregated usage stats once a day unless you opt out.
This is Hugo van Kemenade's live-tooted account of releasing Python 3.15.0, covering the blockers, rc3 delay, and final steps from October 1 to 9, 2026. It's a behind-the-scenes look at how a CPython release actually gets shipped.
- The final release was pushed back by a surprise rc3, because last-minute lazy-import fixes needed time for proper testing.
- Early blockers included docs.python.org and buildbot outages from an incompatible pyOpenSSL upgrade, and a flaky test that was fixed by raising a timeout.
- The release process has many manual gates, including Windows installer signing that needs button clicks, so no release can be signed without the whole release team being notified.
- Van Kemenade followed a seven-day cooldown for dependencies, sticking with 3.14 for release tooling instead of the new rc3 despite pydantic and aiohttp adding 3.15 support.
mimic is a tool that intercepts mobile app API traffic, extracts authentication credentials, and uses Claude to generate Python client libraries you can call directly. You capture real requests from your phone, and it handles the auth reuse and multi-step call chaining automatically.
- Captures app traffic via mitmproxy, extracts stable auth tokens (bearer tokens, session IDs, device IDs), then generates a working Python client without writing boilerplate
- The generated client handles token rotation — on a 401 from an idempotent request, it re-pulls auth from the captured session and retries automatically
- Works around certificate pinning with Frida-based unpinning, but can't bypass DPoP sender-constrained tokens since those require fresh signatures per request
Apache Iceberg released Python 0.12.0 with 470+ merged PRs adding REST catalog view support, concurrent commit handling, incremental append scans, and geometry type support. The release fixes numerous correctness bugs and bumps minimum dependencies for PyArrow and PyIceberg-core.
- REST catalog now supports creating, loading, listing, and dropping Iceberg views with a View object API
- Writes now retry on concurrent commits and validate for conflicts instead of failing silently
- 40+ bug fixes address critical issues like partition pruning, null/NaN handling, timestamp parsing, and decimal encoding
- Minimum PyArrow bumped from 17.0.0 to 18.0.0; PyIceberg-core requirement increased to >=0.10.1,<0.11.0
Tau is a command-line AI agent that handles coding tasks like explaining repos, writing tests, and fixing errors. It's designed as a teaching project—clean, modular code that shows how to build a coding agent without the complexity of production systems.
- The architecture splits into three layers (tau_ai for model translation, tau_agent for the reusable brain, tau_coding for the app wrapper), making each part readable and independent
- It works with multiple model providers (OpenAI, Anthropic, OpenRouter, local models) through a provider-neutral event system, so you can swap backends without changing the core
- Sessions persist as append-only JSONL files with branching and compaction support, letting you resume work and inspect the full conversation history
A 378-page book teaching practical approaches to building ML models by prioritizing data quality over algorithm complexity, covering data collection, cleaning, labeling, and synthetic data generation with Python. Published February 2024, it emphasizes responsible AI and the role of subject-matter experts in model development.
- Shifts focus from algorithm optimization to data quality as the foundation for robust, fair, and interpretable ML models
- Covers concrete techniques: data imputation, cleaning, labeling, augmentation, and synthetic data generation with scikit-learn code examples
- Introduces "small data" concept and strategies for handling missing data, addressing bias, and building ethical AI systems
Teams often end up rebuilding the same runtime code—connection handling, retries, logging—in hundreds of standalone Python scripts. SeaTunnel fixes this by moving those concerns into a shared runtime and connector framework, leaving developers to define just the source, transform, and sink for each data pipeline.
- Hundreds of standalone Python scripts each duplicate the same runtime plumbing—connection setup, retries, logging, checkpointing, thread pools, error recovery.
- SeaTunnel separates pipeline definition (Source, Transform, Sink) from execution, letting its runtime handle connections, error handling, scaling and monitoring.
- A unified Connector Framework gives every connector (MySQL, Oracle, Kafka, S3, REST APIs, etc.) the same interface and lifecycle, so adding a data source just means plugging in a connector instead of rebuilding retry/thread logic.
This snippet shows how to call NVIDIA’s integrate API to run the moonshotai/kimi-k2.6 chat model. It covers setting headers, payload fields (model, tokens, temperature) and handling both JSON and stream responses.
- Sample code calls NVIDIA's integrate API (https://integrate.api.nvidia.com/v1/chat/completions) to run moonshotai/kimi-k2.6, requiring only an NVIDIA_API_KEY swapped into the Authorization header.
- Default payload uses temperature 1, top_p 1, a 16,384 token cap, and a fixed seed of 0 for reproducible outputs.
- Same code handles both streaming (line-by-line printing) and non-streaming (full JSON dump) responses by toggling the stream flag and Accept header.
- Snippet is portable and meant to be adapted to any endpoint provider, not just NVIDIA's hosted API.
@rasbt on X
Sebastian Raschka is announcing a new book, available now from Manning and up for preorder on Amazon with shipping in a few weeks. The post itself is just the two purchase links, so the book's subject isn't stated in the content provided.
- The book is already shipping from Manning's site.
- The Amazon edition is open for preorder, with delivery expected in a few weeks.
SpiderFoot is an open-source Python 3 framework for automating OSINT reconnaissance via a web UI or CLI. It includes over 200 modules, a YAML-driven correlation engine, data exports, TOR support and integrates with tools like Nmap, SHODAN and HaveIBeenPwned. For teams and large-scale scans, SpiderFoot HX adds cloud hosting, multi-user collaboration, REST APIs and change alerts.
- SpiderFoot has 200+ modules pulling from SHODAN, HaveIBeenPwned, GreyNoise, AlienVault OTX and more, with a YAML-based correlation engine running 37 pre-built rules to link findings like leaked emails to vulnerable subdomains.
- It's free, MIT-licensed, and runs via a local web UI or CLI (Python 3.7+, SQLite backend), chaining into tools like Nmap, CMSeeK and DNSTwist for port scans and typo-domain checks.
- Targets span IPs, ASNs, emails, phone numbers and even Bitcoin wallets, with most modules working without paid API keys.
- The paid SpiderFoot HX adds cloud hosting, multi-user/2FA access, Splunk/ElasticSearch/Slack integration, attack-surface monitoring with change alerts, and a REST API.
This gist provides a single-file, dependency-free implementation of a GPT-style transformer, complete with a custom autograd engine, training loop using Adam, and inference routine. It trains on a list of names, demonstrating both the core algorithm and a brief benchmark discussion for a GPU-based microgpt.cu variant.
- A complete GPT training pipeline—autodiff engine, transformer architecture, and Adam optimizer—fits in under 300 lines of pure Python with zero dependencies.
- The custom Value class implements backpropagation from scratch, proving you don't need PyTorch/TensorFlow to understand how gradients flow through a transformer.
- The model is genuinely tiny (16-dim embeddings, 4 heads, 16-token context, a few thousand params) yet demonstrates every core GPT component: embeddings, RMS-norm, scaled dot-product attention, residuals, and a feed-forward MLP.
- It's explicitly a stripped-down teaching tool, sacrificing batching and GPU speed to expose the raw mechanics of training a character-level name generator in 1,000 steps.
This article introduces Pointblank, a Python library designed to streamline data validation. It emphasizes user-friendly features, automated validation suggestions, and customizable reports to enhance team communication about data quality issues.
- Pointblank's DraftValidation feature uses AI to scan a dataset and auto-generate suggested validation rules, cutting setup time
- Reports are built for team communication, not just error logs, translating validation results into actionable insights
- It works across Polars, Pandas, and SQL databases, with YAML config support for CI/CD pipelines and shared validation rules
- Supports automated alerting (e.g. Slack notifications) when data quality thresholds for warnings/errors are breached
The removal of Python's Global Interpreter Lock (GIL) marks a significant shift in the language's ability to handle multithreading and concurrency. With the introduction of PEP 703, developers can now compile Python with or without the GIL, enabling true parallelism and reshaping how systems are designed, particularly in data science and AI. This change presents both opportunities and challenges, requiring developers to adapt to new concurrency patterns.
- PEP 703 lets Python be compiled with or without the GIL, making the lock optional rather than removing it outright everywhere
- Enables true multi-core parallelism for CPU-bound Python code instead of relying on multiprocessing or async workarounds
- Particularly impactful for data science and AI workloads that need concurrent computation
- Developers will need to adapt to new concurrency patterns and potential thread-safety issues that the GIL previously masked