LL3 debuts on April 18th. It feels like a year since then as far as progress. There was a 4bit bnb quant out in 4 hours after the release. Meta and the model wall… What’s the point? The second one person gets access the model is essentially public… Very large context sizes and fine tunes … Read more
Long-term digital archiving is actually much more involved than it appears. On one hand, you might think you can simply make a copy of something and store it indefinitely. On the other, it would be naïve to consider this a secure long-term storage method. Numerous natural factors, including magnetism, gravity, and environmental conditions (temperature, moisture, … Read more
<rant> Generally, the current thought is to train a model on a ton of data- broad data, usually. I think the idea is to turn it into a human. But is this really what we need. I see modular models A model that knows all Maths A model that knows all Art and etc and … Read more
Is memory good or… Memory increases complexity History creates complexity How to maintain memory in AI Memory creates bias Knowledge creates bias..etc