跳到正文
DEV Community· Kirill Lukyanov·· 37 分鐘前AI 評分67

企業 AI 伺服器選購指南:7B–70B 模型配置與雲端成本比較

An AI Server for Your Company: What to Buy for 7B–70B Models and What It Costs vs the Cloud

AI 導讀

文章按俄羅斯 2026 年 11 月市場價格,整理企業運行 7B–70B 模型所需的 GPU 記憶體,並比較租用、自購伺服器與雲端 API 成本。70B Q4 模型約需 40 GB 記憶體,另加 5–15 GB 上下文;自建 RTX 4090 伺服器運行 32B 模型,估算每月 ₽17–21K。

正文

"Should we buy an AI server?" — this question hides three different ones: how much GPU memory you actually need, rent or buy, and where your data lives. Here are the numbers I collected for November 2026 (Russian market prices, verified sources), plus a worked example: a RAG document assistant for 10 employees.

How much memory: 7B, 32B, 70B

You pay for memory, not parameters. At full precision each parameter costs 2 bytes — a 70B model takes ~140 GB. In practice everyone runs 4-bit quantization (Q4), which cuts that roughly 4×:

Model Q4 memory + context Good for
7B–8B 5–6 GB +1–4 GB Drafts, summarization, ticket triage
32B 18–20 GB +2–8 GB Document Q&A, reasoning — "not a toy anymore"
70B ~40 GB +5–15 GB Deep analysis, long documents, near-flagship quality

First rule of buying a server: look at VRAM, not at the marketing tier. One RTX 4090 (24 GB) fits a 32B Q4 model with context room to spare. For 70B you need a 96 GB card (RTX PRO 6000) or multiple cards.

Renting GPUs in Russia (November 2026)

Public hourly rates from Russian providers (immers.cloud, Selectel, HOSTKEY, Teraferma, aggregated by GPURadar):

GPU VRAM From, ₽/hr 24/7 per month
RTX 3090 24 GB 29 ~21 000 ₽
RTX 4090 24 GB 55 40–60 000 ₽
RTX 5090 32 GB 72 ~52 000 ₽
RTX PRO 6000 96 GB 195 ~140 000 ₽
A100 80 GB 212 ~152 000 ₽
H100 80 GB 291 ~210 000 ₽
H200 141 GB 423 ~305 000 ₽

Hourly billing × 720 hours destroys the "leave it running" economics. For always-on workloads, dedicated monthly servers win: a dedicated RTX 4090 server starts at 37 000 ₽/month (HOSTKEY) versus ~55 000 ₽/month for the same card as cloud instance (Selectel).

Building your own rack

GPU prices are ugly right now: RTX 4090 is discontinued and retails at $4 899 new (Newegg, Nov 17, 2026; ~$3 400 used on eBay) against the original $1 599. RTX 5090 lists at $1 999 MSRP but sells from $6 550. A server-class RTX PRO 6000 Blackwell 96 GB starts at ₽2 586 700 in Russian retail (VAT included).

A realistic "32B in production" build with a new 4090: ~500–650 thousand ₽ all-in. Running costs: ~450–550 W under load → 330–400 kWh/month → ₽2 000–2 800 of electricity. Depreciation over 3 years adds ₽14–18K/month. Total: ₽17–21K per month for your own 32B server, data never leaves the office.

The worked example: document assistant for 10 people

Typical RAG setup: 10 employees × 20 questions/day, ~8K tokens of retrieved context per query, ~500-token answers → ~35M input + 2.2M output tokens per month:

Path Per month Data location
DeepSeek V4.1 Flash (off-peak) ~600 ₽ leaves the country
GigaChat Lite ~2 400 ₽ Russian cloud
YandexGPT Pro 5.1 (async) ~15 000 ₽ Russian cloud
RTX 4090 rental (10h × 22 days) ~18 000 ₽ provider DC
Own rack with 4090 17–21 000 ₽ your office

Three takeaways: the cheapest foreign API is pennies but usually fails the security review; Russian cloud APIs on this scale already cost more than owning hardware — and scale linearly with headcount; a stable workload above ~6 hours/day makes owned or dedicated hardware cheaper than API at a single-department scale.

Rules of thumb

  1. Start with an API pilot on a Russian model — ₽2–3K gives you real load statistics to size the GPU.
  2. Don't buy memory "for 70B" without proof — verify 32B isn't enough on your tasks first; the difference is ~2M ₽.
  3. Hybrid beats extremes: local 32B for routine, route hard questions to a cloud model.
  4. Data residency decides early: prompts with contracts and personal data sent to a foreign API is cross-border transfer under Russian law (152-FZ) — local models close this question physically.

Software stack

vLLM for production (dozens of concurrent sessions per card via batching) or Ollama for simple cases, exposed as an OpenAI-compatible API so internal apps don't care whether the model lives in your closet or in OpenAI's datacenter.

Originally published at klukyanov.ru — I write about local LLMs and AI implementation in Russian companies.

來源:DEV Community · dev.to