Skip to content
Kush V9 Model Validation Prime Agent Llama 3.1 Self-Hosted AI

Kush V9 Validation Update: V82 Remains Production

H
Hotep Intelligence
3 min read

Updated

Update (November 2026): Every promotion gate outlined in this postmortem has now been cleared. Hotep Intelligence Kush V9.1 is live in production, achieving 7/7 (100%) Exact Entity Recall, 7/7 (100%) Prime Agent E2E (50.15 persona score), and 48.2 tok/s local Apple Silicon Metal inference with open GGUF weights on Hugging Face.

Kush V9 Is Held From Production

Kush V9 loaded successfully on the self-hosted Mac Mini runtime, but load success is not the same as release readiness. After Prime Agent E2E comparison and direct live prompts, we rolled production back to hotep-llm-kush-v82.

The candidate did fix one important serving problem: it ran with the native Llama 3.1 chat template instead of a forced ChatML template. That made the adapter behavior more visible and gave us a clean test boundary.

Prime Agent E2E Comparison

The direct comparison used seven representative prompts against the validated Kush V82 baseline and the Kush V9 candidate.

ModelPassesEntity passesAvg personaAvg latencyAvg tok/sCJKRubricRepetition
hotep-llm-kush-v824/74/721.968.610s22.88000
hotep-llm-kush-v94/74/719.0814.343s6.14000

V9 did not beat V82. It tied the small pass count, scored lower on persona, and ran materially slower.

Why It Was Pulled

The live candidate still missed hard entity recall:

  • Strong Dad must resolve to Trey Quinn.
  • Uncle Hotep must resolve to Demond Handy.
  • Grifties ticket prompts must point to grifties.com instead of inventing free or informal access.
  • Shakka Ahmose needs grounded classical African history coverage, not generic public-figure hallucination.

Those misses are concrete enough to fix. They also prove the promotion gate was too weak: a model that merely loads and produces clean English can still be wrong on community-critical facts.

What Changes Next

V9.1 will be trained and promoted only after it clears a hard entity gate with required and forbidden terms for the identities that failed here. The release path now treats Prime Agent E2E, direct live prompts, entity recall, speed, and runtime health as separate gates.

Why Multi-Gate Validation Beats Single-Metric Benchmarks

Our earlier model iterations taught us that aggregate eval scores can hide sharp production regressions:

  • Template & Quantization Sensitivity: Switching from forced ChatML templates to native Llama 3.1 chat formatting improved structural clarity, echoing the Modelfile lessons documented in our Kush v8.2 training pipeline hardening reference and Kush v7 release architecture.
  • Entity Grounding vs. Tone Alignment: As seen from Kush V1 through Kush V4, a model can speak in an authentic voice while hallucinating biographical facts if rare community entities are underrepresented in supervised fine-tuning pairs or retrieval chunks.
  • Latency as a First-Class Gate: Jumping from 8.6s (22.88 tok/s) on V8.2 to 14.3s (6.14 tok/s) on V9 degraded interactive user experience on self-hosted Apple Silicon hardware, reinforcing the operational standards outlined in Building Sovereign AI.

Production Promotion Checklist Going Forward

  1. Deterministic Entity Recall: 100% pass rate on canonical community, historical, and organizational identity prompts before human or LLM-judge persona scoring begins.
  2. Throughput Parity: Candidate adapters must stay within 15% of current production tokens-per-second on the live llama-server runtime.
  3. Zero Contamination Regression: Zero CJK token leakage and zero rubric-prefix artifacts, preserving the safeguards introduced after the V14 quality audit.

Hotep Intelligence remains sovereign infrastructure: local model, local serving, local evidence, and public correction when the evidence says a candidate is not ready.

📚

Sources & References

  1. AI @ Meta(2024). The Llama 3 Herd of ModelsJournal
  2. Rafael Rafailov et al.(2023). Direct Preference Optimization: Your Language Model is Secretly a Reward ModelJournal

Historically Reviewed

by Hotep Intelligence Editorial Team · Kemetic History, Holistic Wellness, ML Engineering

Our editorial standards →

Knowledge Base

Explore 400+ Ancestral & Holistic Guides on knowledge.askhotep.ai