vllm-project/vllm
What it can do
A high-throughput and memory-efficient inference and serving engine for LLMs (GitHub project description) source
What it can't do
TBD
Minimum hardware
TBD
Setup difficulty
2/5 — Single documented command (docker run / pip / npm / brew install) source
Freshness
not fetched yet
Running cost (per month, roughly)
VPS: TBD — minimum RAM not verified
Local electricity: $ —/mo watts / 1000 × hours/day × 30 × 0.1831 $/kWh source
