VideoMamba
on post transformer architectures
🔷 Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.
Here’s today at a glance:
🎥 VideoMamba
The Shanghai AI Lab has a paper on VideoMamba. Recall that Mamba is the leading state space model architecture, the best explanation of which is:
Transformers are the workhorse of modern sequence modeling, achieving remarkable performance on a variety of tasks, but they have unavoidable inefficiencies. Specifically, the memory and compute used for generating every output token growslinearlywith the input length. This means that generating n tokens requires O(n^2) compute, making training with long sequence lengths practically impossible.Recently, State Space Models (SSMs) have emerged as a challenger to the Transformer architecture. These models can be interpreted as a type of recurrent neural networks (RNNs), which use a fixed-size memory that does not grow with the sequence length. This makes training…
