Defense Software Firm Publishes Public AI Model Leaderboard, Kimi K3 Debuts High

AIThis post was created with the assistance of artificial intelligence (AI).

VigilSAR Defense LLM Benchmark — The public benchmark page — aggregate results public, task set private. Source: vigilsar.com

In a move that highlights the evolving landscape of artificial intelligence in defense, VigilSAR has publicly shared its latest AI model rankings on a dedicated leaderboard. This leaderboard assesses which language models are trustworthy for intelligence, surveillance, and reconnaissance (ISR) work, focusing on their reasoning, reporting, and restraint capabilities — not just general trivia. The goal? To determine which models can be reliably used in sensitive defense scenarios.

The evaluation was conducted with 14 models over 300 tasks on July 17, 2026. The scores are publicly available, but crucially, the specific test questions remain private, ensuring models cannot be trained on the exact tasks. A separate, private, held-out set exists to verify the models’ true capabilities, with the difference between public and private scores indicating potential memorization or overfitting. This transparency approach aims to provide a fair assessment amid claims and marketing hype.

Leading the pack is Claude Fable-5, with a score of 67.77, earning its spot in the confident Band A. A notable newcomer, Kimi K3 from Moonshot, made a strong debut at #3 with a score of 64.65. This entry falls into Band B, surpassing all GPT and Gemini models on the leaderboard. The results are organized into confidence bands, rather than exact ranks, with overlapping confidence intervals ensuring a nuanced view of model performance.

Among the models, the GPT-5.x family occupies Bands C and D, while Gemini models sit in Bands E and F. One particular model is scored as ‘sovereign-deployable,’ indicating that deployment readiness plays a role in the overall score. The evaluation’s design explicitly states that vendor claims are not considered evidence. Instead, the operators prioritize measuring models’ real capabilities, choosing not to be swayed by marketing claims or unverified assertions.

To maintain transparency, the site includes features such as confidence intervals, held-out score gaps, and a pinned reference row. They also publish economic data, like the cost-per-correct-answer, giving a comprehensive view of each model’s practical value. This approach ensures that the ranking reflects both performance and real-world usability in defense applications.

For those interested in exploring the full results, you can visit the public leaderboard. The entire effort stems from a commitment to honest, evidence-based evaluation, rather than relying solely on vendor claims. As the landscape of AI in defense continues to grow, VigilSAR’s transparent testing offers a valuable glimpse into which models are truly trusted for sensitive ISR tasks.