Click any tag below to further narrow down your results
Links
The author compares DNA genomes and large language model weights as passive information sequences—scores that only become meaningful when processed by cellular machinery or inference engines. Both arise from massive search processes (evolution and gradient descent) that compress vast experiences into lossy representations, leaving much uninterpreted “junk.” This parallel explains why it’s hard to pinpoint where specific functions reside in either genome or model.
- DNA and LLM weights are both inert symbol strings that only produce meaning when run through an interpreter (ribosomes/enzymes vs. inference engines)
- Both are compressed, lossy residues of massive iterative search—4 billion years of natural selection vs. gradient descent over trillions of examples—not literal copies of the experiences that shaped them
- Neither system stores discrete facts or instructions at a single location (no "gene for a hand," no weight for "Paris is the capital of France"); competence is distributed across the whole structure, which is why pinpointing function in either genome or model is so hard
This article explores an unconventional method for classifying text by leveraging compression algorithms. The author demonstrates how to concatenate labeled documents, compress them, and use the compressed sizes to predict labels for new texts. While the method shows promise, it is computationally expensive and generally underperforms compared to traditional classifiers.
- Compression-based classification (concatenate labeled texts, measure compressed-size increase) hits a 0.749 macro F1 with gzip on 4 categories of 20 Newsgroups, versus 0.88 for multinomial Naive Bayes.
- lzma pushes accuracy up to 0.897, beating Naive Bayes, but takes 32 minutes versus over 5 minutes for gzip on just 1,353 test cases—far too slow to be practical.
- The technique reframes text classification as an information-theory problem, showing compression algorithms implicitly model word probability distributions.
This article explores how Python 3.14's zstd module enables efficient text classification through incremental compression. It outlines a method where text is classified based on the size of compressed output from different class-specific compressors, demonstrating improved speed and accuracy over traditional methods.
- Python 3.14's zstd module supports incremental compression, letting classifiers update on new data in tens of microseconds instead of recompressing everything from scratch.
- Classification works by feeding a new document to per-class compressors and picking the class whose compressed output size increases least.
- Tunable parameters (window size, compression level, rebuild frequency) let you trade off speed against accuracy for a given use case.
- Benchmarked on 20 Newsgroups, the compression-based classifier shows competitive learning ability while avoiding traditional ML pipeline complexity.