I would guess that most people will get them through the LM Studio UI.
No clue...
They are exactly the same size and were built from the same
I've been using QWEN2.5 7B for SEP and feel that it's a meaningful step up from just using paddle. Maybe not "you should buy a new gpu for this", but definitely "best use it if you have it." I'd expect QWEN3 8B is probably in the same boat.
Since OCR stuff takes "relatively" little tokens and you usually only have a few pictures that need translation I don't really see why someone shouldn't use the 32B version running on CPU though. All that's needed for the IQ3_XS version for example would be 16GB of ram. That's like $30 if you buy the Chinese stuff.
On v2.027? Was the project created on the same version?
I can't even manage to make it run on Linux.
Huh... I had no idea it works like that. Interesting.
from openai import OpenAI
client = OpenAI(
base_url = "https://integrate.api.nvidia.com/v1",
api_key = "$NVIDIA_API_KEY"
)
completion = client.chat.completions.create(
model="deepseek-ai/deepseek-v3.1-terminus",
messages=[{"role":"user","content":""}],
temperature=0.1,
top_p=0.7,
max_tokens=16384,
extra_body={"chat_template_kwargs": {"thinking":True}},
stream=True
)
for chunk in completion:
reasoning = getattr(chunk.choices[0].delta, "reasoning_content", None)
if reasoning:
print(reasoning, end="")
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="")I don't know what its problem could be since it's supposedly using the same v1 format as the rest.
const response = await fetch(apiUrl, {
method: 'POST',
headers: Object.assign({
'Content-Type': 'application/json'
},
apiKey ? {
'Authorization': `Bearer ${apiKey}`
} : {}
),
body: JSON.stringify({
model: activeLLM,
messages: messages,
temperature: llmTemp,
top_k: llmTopK,
top_p: llmTopP,
min_p: llmMinP,
repeat_penalty: llmRepeatPen,
max_tokens: maxTokenValue
})
});
I didn't have time to really test it, yet. Are there any downsides? Or would you basically recommend everyone use their browser instead?
Hi Shisaye,
So maybe the NVIDIA endpoint is also rejecting the extra response_format or extra_body fields. Your fetch works fine because it doesn’t include those fields, while the OpenAI client adds them by default. That could explain why you don’t see the error on DeepSeek API, OpenRouter, or LM Studio, but I do on NVIDIA integrate API.
import OpenAI from 'openai';
const openai = new OpenAI({
apiKey: '$NVIDIA_API_KEY',
baseURL: 'https://integrate.api.nvidia.com/v1',
})
async function main() {
const completion = await openai.chat.completions.create({
model: "deepseek-ai/deepseek-v3.1-terminus",
messages: [{"role":"user","content":""}],
temperature: 0.2,
top_p: 0.7,
max_tokens: 16384,
chat_template_kwargs: {"thinking":true},
stream: true
})
for await (const chunk of completion) {
const reasoning = chunk.choices[0]?.delta?.reasoning_content;
if (reasoning) process.stdout.write(reasoning);
process.stdout.write(chunk.choices[0]?.delta?.content || '')
}
}
main();import OpenAI from 'openai';
const openai = new OpenAI({
apiKey: '$NVIDIA_API_KEY',
baseURL: 'https://integrate.api.nvidia.com/v1',
})
async function main() {
const completion = await openai.chat.completions.create({
model: "nvidia/llama-3.3-nemotron-super-49b-v1.5",
messages: [{"role":"system","content":"/think"}],
temperature: 0.6,
top_p: 0.95,
max_tokens: 65536,
frequency_penalty: 0,
presence_penalty: 0,
stream: true,
})
for await (const chunk of completion) {
process.stdout.write(chunk.choices[0]?.delta?.content || '')
}
}
main();