Zero KV-Cache.
Pure Linear Intelligence.
Pulse v1 eliminates self-attention with Continuous State Convolutions and 1.58-bit ternary quantization. Strict linear-time complexity, zero KV-cache explosion, and verified reasoning at under 100M pretraining tokens.
Engineered for pure efficiency.
How Pulse v1 replaces Transformer attention mechanisms with constant-memory continuous state convolutions.
O(1) Fixed Circular State Buffer
Transformers consume hundreds of megabytes per batch as context grows to 4k tokens. Pulse v1 maintains a strictly fixed 8.8 MB state buffer across any sequence length—enabling infinite context streaming on edge hardware.
1.58-Bit Ternary Core
55M parameters quantized to {-1, 0, +1} ternary matrices. Reduces weight storage to 10.9 MB with integer addition kernels.
52.40% WinoGrande at 92M Tokens
Matches SmolLM-135M (which required 600 Billion tokens) with over 6,500x less training data, validating extreme data efficiency in linear state spaces.
19,000 Tok/s GPU Throughput
Zero CPU fallback DirectML BitNet execution on AMD Radeon RX 9070 XT. Full pretraining cycle completed in 84 minutes.
Small Language Model Comparison
Evaluated strictly under official lm-evaluation-harness specifications against standard models.
| Architecture | Format | Pretrain Tokens | Compute Budget | WinoGrande (0-Shot) | Runtime State |
|---|---|---|---|---|---|
| Pythia-70M | 16-Bit FP16 | 300 Billion | Cluster Weeks | 50.60% | 140 MB (KV-Cache) |
| GPT-2 (124M) | 16-Bit FP16 | 10 Billion | Multi-GPU Days | 51.30% | 380 MB (KV-Cache) |
| SmolLM (135M) | 16-Bit FP16 | 600 Billion | Massive Cluster | 52.40% | 450 MB (KV-Cache) |
| Pulse v1 (55M) | 1.58-Bit Ternary | 92.1 Million (3,200x less) | 84 Mins (1x GPU) | 52.40% (Matched) | 8.8 MB (Fixed O(1)) |