“Inference on my Mac mini is slow”

We’ve all played with an inference model locally on our laptop. In this talk, you will follow Scaleway’s journey from that experiment to serving trillions of tokens across dozens of models. We will walk through the challenges, the metrics to watch, the pitfalls to avoid… and how to get the most out of the big box full of GPUs that just blew up your budget.