Pingkit is a toolkit designed for training reproducible, capacity-aware models using transformer activations. It offers features for extracting embeddings, training neural architectures, and creating custom probes tailored to specific research needs. The toolkit is integrated with Hugging Face models and provides various utilities for data processing and model training.

• Pingkit extracts embeddings directly from transformer activations for downstream probing/analysis rather than relying on final-layer outputs alone.
• It's built to train "capacity-aware" probes, letting researchers control model complexity to avoid overstating what information is linearly/nonlinearly decodable from activations.
• Integrates with Hugging Face models, so it can plug into existing pretrained transformer workflows without custom extraction code.
• Emphasizes reproducibility, aiming to standardize probing experiments that are often ad hoc and hard to replicate across papers.