Stock Qwen2.5-Coder-7B outperformed fine-tuned competitors but lost originality points

StacksNG was designed for the Africa Deep Tech Challenge 2026 as an offline coding assistant for Nigerian fintech APIs like Paystack, Flutterwave, Monnify, and Termii. To eliminate API key dependencies and hallucinations, developers implemented a RAG pipeline with 780 scraped chunks processed locally via llama.cpp. Though the technical foundation was sound, the project failed to advance past the first round. Without clear feedback on scoring, a deeper examination of judging criteria became necessary.

The "Originality" score of 3 raised questions about how innovation is measured in AI submissions. Internal benchmarks showed Qwen2.5-Coder-7B consistently outperformed many heavily fine-tuned models, delivering accurate code for local payment gateways while avoiding the drift seen in smaller, over-optimized models. Yet the judging tool penalized the use of an off-the-shelf model rather than recognizing the efficiency of combining it with a precise RAG system.

Reviewing other finalists revealed a pattern: judges prioritized the act of fine-tuning over practical results. High-scoring entries often fine-tuned smaller models on niche datasets, even when those models produced flawed or logically inconsistent code compared to Qwen2.5-Coder-7B. The emphasis on custom weights—rather than performance—meant fine-tuned approaches earned higher originality marks, despite their limitations.

A RAG architecture was better suited for fintech assistants because API changes render fine-tuned models obsolete quickly, while vector stores only require updates to their chunks. StacksNG’s local, API-key-free setup ensured low-latency, privacy-focused operation without compromising accuracy. However, the 3-point score suggested that "originality" was tied to training novelty rather than the practical innovation of building a dependable offline system.

The penalty for using a stock model—despite its superior real-world performance—highlights a broader issue. If the goal was to identify the most effective tool for Nigerian developers, Qwen2.5-Coder-7B’s results should have been the deciding factor. Instead, the judging framework favored manual effort over engineering effectiveness, rewarding fine-tunes for their process rather than their outcomes.

The mismatch between technical performance and scoring reflects a common flaw in AI competitions: equating "originality" with manual intervention. A smart RAG integration that achieves superior results with minimal customization represents a meaningful engineering achievement, yet the system treated it as less innovative than a flawed fine-tuned model. The result was a high-performing tool misjudged by a framework that valued effort over effectiveness.