Tilde Research introduces Parallax, a parameterized local linear attention mechanism
It corrects boundary bias in softmax attention by adding a learned covariance correction, improving extrapolation and hardware efficiency.
See the latest news and media coverage for Tilde Research. We track all announcements, press releases, and industry mentions in real time, all in one place.
AI research laboratory advancing interpretability, architectures, and pretraining
tilderesearch.comLast updated
In short: Tilde Research completed an $8M seed round and introduced new attention and training-efficiency methods
It corrects boundary bias in softmax attention by adding a learned covariance correction, improving extrapolation and hardware efficiency.
It prevents MLP neuron death by enforcing row-norm uniformity and orthogonality, achieving SoTA on nanoGPT speedrun and 1.1B pretraining.
Nitrobrew reduces communication and memory by transmitting hidden states instead of logits, achieving 3x faster throughput.
The One Layer Deeper competition challenges participants to train models that productively extrapolate beyond training depth, using modular squaring as the public task.
GitHub - tilde-research/MoMoE-impl: Memory optimized Mixture of Experts · 4 · Inicia sesión o regístrate en ThreadsEntérate de qué hablan otras personas y únete a...
Track Tilde Research and your other target companies to get real-time alerts and weekly summaries delivered straight to your inbox.
Browse news for competitors to Tilde Research and other trending companies.