Inference: what it means.
An inference application takes new input and uses a model to produce an output. In a vision example, an image is supplied to a trained network and the application uses the resulting classification or detection. (NVIDIA: edge AI)
Training and inference can occur on different hardware and in different locations. The fact that a model trained successfully does not establish that it meets the timing, memory or output-quality needs of a deployed device. (NVIDIA: edge AI)
Put the definition to work.
Recognizing an object in a new camera image is an inference task. Learning the model from examples is training; evaluating a deployed model still requires application-specific testing. (NVIDIA: edge AI)
Questions to ask next.
Use these questions as a reading checklist, not as a device-selection recommendation. Exact manufacturer documentation takes priority over a general definition.
- Model and input requirements
- Output-quality validation on real data
- Latency, memory and deployment environment
For the broader explanation, compare ai processing roles. This page defines the term; the companion guide connects it to the rest of the system.
Common Inference questions.
Is inference always performed in the cloud?
No. Edge AI uses local inference near the source of data. Cloud and local deployments are alternative or complementary arrangements. (NVIDIA: edge AI)