2024-04-15: Infini-Attention
LLMs with Infinite Memory
🔷 Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.
Here’s today at a glance:
⚠️ Infini-Attention
Google designed a memory system for transformers that allows storing of state. This compressive memory system scales better for very long sequences. A compressive memory maintains a fixed number of parameters to store and recall information with a bounded storage and computation costs. In the compressive memory, new information is added to the memory by changing its parameters, with the objective that this information can be recovered back later on.
What’s happening is that a summary of the information to date (a compressed version of it) is kept in the memory store, allowing the model to not compute anew for each successive token. This potentially breaks transformers out of the quadratic increase in computation with increasing length problem.
This system is similar in some ways to the other non-Trans…
