OpenAI · AI Inference · Hugging Face
Run a vLLM Server on HF Jobs in One Command
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
★ Tier-1 Source
It's the quickest way to stand up a model for tests, evals, or batch generation. (If you're after a managed, production-ready service instead, that's what Inference Endpoints are for, more on when to pick which at the end.)
Key facts
- The --max-model-len 32768 --max-num-seqs 256 flags are specific to this model: Qwen3.5-122B is a hybrid Mamba/attention architecture with a 256K-token default context, which doesn't leave enough
- tensor-parallel-size should match the number of GPUs in the flavor ( h200x2 → 2, h200x8 → 8)
- expose 8000 routes the container's port through HF's public jobs proxy (see the Serve Models guide for the full reference)
- Which returns the usual OpenAI-style JSON, with choices[0].message.content holding "Hello
Summary
Huggingface_hub >= 1.20.0: pip install -U "huggingface_hub>=1.20.0". --expose 8000 routes the container's port through HF's public jobs proxy (see the Serve Models guide for the full reference). 6a381ca1953ed90bfb947332 is your job ID.