Click any tag below to further narrow down your results
+ machine learning
(2)
+ inference optimization
(1)
+ optimization
(1)
+ neural nets
(1)
+ differentiable programming
(1)
+ handwired
(1)
+ python
(1)
+ transformers
(1)
+ convolutional neural networks
(1)
+ ben affleck
(1)
+ query planning
(1)
+ llm reproducibility
(1)
+ ai sql
(1)
+ quail
(1)
+ cloud
(1)
Links
Handwired converts regular JavaScript control flow (if-else, loops, state) into matrix multiplication and ReLU operations without any training required. The resulting code stays fully differentiable and runs on GPU, enabling use cases like training frozen programs end-to-end or parallelizing sequential JS code.
- Transforms conditional logic into arithmetic using ReLU tricks (e.g., if-else becomes a weighted sum of both branches selected by a binary mask), keeping everything differentiable without training weights
- Collapses all operations into matmul and ReLU primitives, which GPUs can heavily optimize through fusion, sparsity, and reordering techniques
- Enables hybrid workflows: freeze a handwired program and train it alongside a generic neural net, or run sequential JavaScript at GPU-level parallelism (demo processes 100k JSON documents in milliseconds)
Ben Affleck discusses his technical knowledge of machine learning and computer vision, explaining convolutional neural networks and tensors in detail. He reveals he's visited Google and OpenAI's labs to see their video transformer work, leveraging his celebrity status to get private access.
- Affleck has hands-on experience with Python, GPUs, and visual effects workflows that incorporate machine learning, stemming from his ownership of a small VFX company before Artists Equity.
- He understands the technical mechanics of CNNs—converting images to tensors (numerical representations of RGB pixel values) and using edge detection and feature extraction to identify objects in video for compositing.
- He's gained private access to Google and OpenAI's video transformer research by directly calling and asking, using his name to get meetings most people couldn't arrange.
Quail is a new inference engine that runs AI-SQL queries (SQL with LLM function calls) much faster than existing approaches by co-optimizing query planning and GPU inference. It reduces wasted compute from CPU overhead and KV cache mismanagement, cutting costs by 4-5x compared to naive implementations.
- Running AI-SQL queries on vLLM naively is 27x slower than the theoretical optimum because of CPU scheduling overhead and unnecessary KV cache evictions that force reprocessing of tokens.
- Quail jointly plans queries and inference execution to minimize KV cache waste and keep the GPU busy, achieving 360 documents/second on a medical adverse event filtering task that would cost $0.37 on an H100 versus $1.75 with GPT-5 nano.
- The system handles real workloads like filtering 100,000 IMDB reviews through two AI predicates in under 6 minutes for under $0.37, demonstrating practical viability for production AI-SQL pipelines.
Microsoft will invest A$25 billion (US$18 billion) by 2029 to expand Australia’s digital infrastructure, AI supercomputing capabilities and cloud capacity. The move aims to boost commercial cloud services and AI/GPU offerings for local customers.
- Microsoft is investing A$25 billion (~US$18 billion) into Australia by 2029, its largest-ever single investment there
- Funds will expand data centers, Azure cloud capacity, and AI/GPU supercomputing infrastructure across multiple states, adding to existing NSW and Victoria facilities
- Aims to cut latency, meet data-sovereignty requirements, and give local finance, mining, government, and healthcare sectors faster access to Azure AI tools and OpenAI models
- Includes partnerships with universities and research institutes to build AI talent and drive commercial R&D
This article digs into why repeated LLM calls can produce different outputs even at zero temperature. It shows that floating-point non-associativity and kernel implementation details—rather than thread scheduling or atomic adds—are the real sources of run-to-run variation and outlines ways to make inference fully reproducible.
- Bit-identical results across thousands of runs of the same GPU matmul disprove the standard "concurrency + floating-point non-associativity" explanation for LLM nondeterminism.
- The actual cause is that libraries like cuBLAS/cuDNN nondeterministically pick among multiple reduction/tiling algorithms at runtime, each summing floats in a different order.
- Forcing a single deterministic algorithm choice (via flags like CUBLAS_WORKSPACE_CONFIG, torch.use_deterministic_algorithms(True), and seeded CUDA RNG) makes LLM inference output bit-for-bit reproducible.