This is a short post to explore differences between Inference vs Training in AI.

Training in AI

Training is when you’re building an AI “model”. This is the part where there’s lots of iterations and tweaks. You will try different algorithms and different “hyper parameters” for your algorithms.

All the time you’re trying to maximize how much your algorithm is actually learning by comparing how well it performs against your original data versus data it’s never seen before (called validation data). Some models work really well against the “training” data but perform poorly against new data. That’s called over fitment. You don’t want that.

When your model is ready, you deploy it and monitor for various types of drift and quality. When you deploy the model it is doing “Inference”.

Differences between Inference vs Training in AI

Training vs inference are very different modes. They need different type of compute. Different people are involved. And are completely different states. It’s like comparing the people needed to build a technology (the developers) versus those that are needed to keep it running (ops management, site reliability, support staff, etc).

When you’re reading benchmarks or doing strategic planning for your AI projects be aware of these two modes. For those a bit distant from AI, that’s what these two terms mean.