Real-World AI Engineering
Building It
🔷 Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.
Here’s today at a glance:
⛭ Real-World AI Engineering
This blog post from former Google Brain research scientist, and current Reka Labs founder Yi Tay is an amazing window into large-scale engineering problems in AI.
People always assume it’s simply a question/debate of accelerator choice (TPUs vs GPUs etc) and all GPU clusters are created equal. For us, this soon proved to be false. As we sampled across different service providers, we find that the variance of hardware quality differs vastly even for the same hardware, i.e., GPUs (H100s). Note that here, hardware refers to overall cluster quality and not necessarily the chips or accelerators per se. Just like a lottery. Basically:Not all hardware is created equal. The variance of cluster quality across hardware providers is so high that it is literally a lottery pertaining to …
