Key terms
What is AI inference?
Inference is running a trained AI model to produce an output, as opposed to training it. Each response a user receives is inference, and its cost, speed and energy use are separate questions from how the model was trained.
The full answer
Inference is running a trained AI model to produce an output, as opposed to training it. Each response a user receives is inference, and its cost, speed and energy use are separate questions from how the model was trained.
Read moreHow LLM API pricing worksThe inference economics crisis (opinion)