Llama 3.1 Nemotron 70B Instruct
llama-3.1-nemotron-70b-instructWhere to run it
3 hosts serve Llama 3.1 Nemotron 70B Instruct. The comparison is on blended price, 3 parts input to 1 part output.
| Host | In / out per 1M | Model id | |
|---|---|---|---|
NVIDIA NIMInference llama-3.1-nemotron-70b-instruct | - | llama-3.1-nemotron-70b-instruct | |
DeepInfraInference Llama-3.1-Nemotron-70B-Instruct | $0.60 / $0.60 | Llama-3.1-Nemotron-70B-Instruct | |
Fireworks AIInference llama-v3p1-nemotron-70b-instruct | $0.90 / $0.90 | llama-v3p1-nemotron-70b-instruct |
Capabilities
Core
- Streaming
- Function calling
- JSON mode
- Structured outputs
Multimodal
- Vision
- Image input
- Audio
- Image generation
- Video
Advanced
- Reasoning
- Tool calling
- MCP
- Prompt caching
- Batch API
- Embeddings
- Takes in
- Text
- Produces
- Text
- Open weights
- License: Llama Community LicenseRuntimes: Ollama, TGI, llama.cpp, vLLM
Similar models
This model has no listed price, so these are current models with similar capabilities.
Smaller NVIDIA models
No similar NVIDIA model keeps its key capabilities.
Other frontier labs
Open-weight picks
Model line
Newer and older versions, at each lab's own price.
- UndatedLlama 3.1 Nemotron 70B InstructLatestNot listed
- UndatedLlama3.1 Nemotron 70B Instruct$0.12 / $0.30
Call it
Using NVIDIA NIM's API.
from litellm import completion
response = completion(
model="nvidia_nim/llama-3.1-nemotron-70b-instruct",
messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)Quick answers
How much does Llama 3.1 Nemotron 70B Instruct cost?
Its lowest listed price is at DeepInfra: $0.60 per million input tokens and $0.60 per million output tokens.
Which APIs offer Llama 3.1 Nemotron 70B Instruct?
Llama 3.1 Nemotron 70B Instruct runs on 3 APIs: NVIDIA NIM, DeepInfra and Fireworks AI.
What is Llama 3.1 Nemotron 70B Instruct's context window?
131K tokens, with up to 131K tokens in one reply.
Is Llama 3.1 Nemotron 70B Instruct being retired?
No retirement date is scheduled for it on any API we track.
Is there a newer version of Llama 3.1 Nemotron 70B Instruct?
No. Llama 3.1 Nemotron 70B Instruct is the newest model in its line.
What is Llama 3.1 Nemotron 70B Instruct's API model id?
Use llama-3.1-nemotron-70b-instruct with NVIDIA NIM. Other hosts use their own ids: Llama-3.1-Nemotron-70B-Instruct on DeepInfra, llama-v3p1-nemotron-70b-instruct on Fireworks AI.
Sources
Raw catalog entry for llama-3.1-nemotron-70b-instruct on NVIDIA NIM
{
"litellm_provider": "nvidia_nim",
"source": "https://integrate.api.nvidia.com/v1/models"
}