benchmark_20260310T170013.md 2.8 KB

Benchmark Results - 20260310T170013

Model Selection (6-slot / 2-socket)

Slot Socket Role Model Composite Score
1 Node 1 (port 11434) General (locked) llama3.2:3b 0.814
2 Node 1 (port 11434) General (locked) llama3.1:8b 0.623
5 Node 1 (port 11434) General (rotate) gemma3:12b-it-q4_K_M 0.481
3 Node 0 (port 11435) Coding (locked) deepseek-coder-v2:16b 0.723
4 Node 0 (port 11435) Coding (locked) qwen2.5-coder:7b 0.655
6 Node 0 (port 11435) Coding (rotate) qwen2.5-coder:14B 0.57

Detailed Metrics

codellama:34b

  • Category: coding
  • Coding Quality: 0.833
  • General Quality: 0.586
  • Avg Tokens/sec: 3.2
  • Latency (ms): 4235.4
  • Coding Composite: 0.437
  • General Composite: 0.326

    deepseek-coder-v2:16b

  • Category: coding

  • Coding Quality: 0.833

  • General Quality: 0.885

  • Avg Tokens/sec: 23.5

  • Latency (ms): 1568.5

  • Coding Composite: 0.723

  • General Composite: 0.746

    qwen2.5-coder:14B

  • Category: coding

  • Coding Quality: 0.85

  • General Quality: 0.931

  • Avg Tokens/sec: 6.6

  • Latency (ms): 2229.7

  • Coding Composite: 0.57

  • General Composite: 0.607

    llama3.1:8b

  • Category: general

  • Coding Quality: 0.823

  • General Quality: 0.877

  • Avg Tokens/sec: 11.8

  • Latency (ms): 2202.0

  • Coding Composite: 0.599

  • General Composite: 0.623

    qwen2.5-coder:7b

  • Category: coding

  • Coding Quality: 0.85

  • General Quality: 0.91

  • Avg Tokens/sec: 12.5

  • Latency (ms): 1431.0

  • Coding Composite: 0.655

  • General Composite: 0.682

    gemma3:12b-it-q4_K_M

  • Category: general

  • Coding Quality: 0.873

  • General Quality: 0.966

  • Avg Tokens/sec: 6.1

  • Latency (ms): 5941.9

  • Coding Composite: 0.439

  • General Composite: 0.481

    llama3.2:3b

  • Category: general

  • Coding Quality: 0.89

  • General Quality: 0.954

  • Avg Tokens/sec: 23.0

  • Latency (ms): 754.8

  • Coding Composite: 0.786

  • General Composite: 0.814

Scoring Formula

  • Composite = quality * 0.45 + token_speed_normalized * 0.30 + latency_score * 0.25
  • Speed normalized against 40 tok/sec ceiling (hardware-observed max)
  • Coding quality (per-prompt): code_gen: has_def×0.20 + has_return×0.20 + has_docstring×0.15 + has_type_hint×0.15 + has_code_block×0.10 + has_assert×0.08 + has_test_def×0.07 + has_import×0.05 debug: has_def×0.30 + has_return×0.30 + has_code_block×0.25 + has_assert×0.15 refactor: has_def×0.25 + has_return×0.25 + has_code_block×0.20 + has_type_hint×0.15 + has_import×0.15
  • Category: override dict → quality delta (coding_avg - general_avg >= 0.1) → name pattern (coder/codestral/codellama/starcoder) → general