6 min read
|
Saved February 14, 2026
|
Copied!
Do you care about this?
The author benchmarks a custom lexer against Dart's official scanner, only to find that I/O operations are the real bottleneck due to excessive syscalls. By packaging files into tar.gz archives, the author reduces syscall overhead, resulting in a significant speedup in I/O performance.
If you do, here's more
The author built an ARM64 assembly lexer for Dart code, achieving a performance improvement of 2.17 times over the official Dart scanner. After benchmarking with 104,000 files, the results revealed that I/O operations were the real bottleneck, taking five times longer than the lexing process itself. The NVMe SSD could read at 5-7 GB/s, but the actual I/O speed was only 80 MB/s due to excessive syscalls—over 300,000 for opening, reading, and closing files.
To address the syscall overhead, the author packaged the Dart files into 1,351 tar.gz archives. This drastically reduced the number of syscalls from over 300,000 to around 4,000. The I/O time dropped from 14.5 seconds to just 339 milliseconds, leading to a total speedup of 2.27 times, even accounting for decompression time. The compression ratio was impressive too, with 1.13 GB of Dart code reduced to 169 MB. This method explained why pub.dev uses tar.gz archives, as it minimizes HTTP requests, saves bandwidth, speeds up extraction, and reduces syscall overhead.
The article highlights that modern storage solutions are fast, but their performance suffers significantly with small file access due to syscall overhead. This insight has implications for various systems, such as build processes and log management, where handling numerous small files can slow down operations. The author suggests further optimizations, such as using faster compression algorithms and parallelizing decompression, to enhance performance even more.
Questions about this article
No questions yet.