Getting an LLM-powered feature to work in a demo takes an afternoon. But getting it to produce the same clean, structured answer every time is a bigger challenge.
It’s also what gives devs pause when using AI. In Devpost’s AI Trends Report 2026, 40% named reasoning errors as a top blocker when building with AI, and 32% named unpredictable outputs. Only API cost (43%) ranked higher.
Yet AI use keeps climbing, with people building more complicated projects. On Devpost, one in four community hackathon submissions is now agentic, up from almost none two years ago. In company hackathons, it’s one in ten.
Making AI output dependable takes more than a better prompt. It’s an engineering problem that needs focused time regular sprints don’t leave. A hackathon can give teams that time.
Inside our dev team’s internal hackathon
Rather than focus on the ‘magic’ of LLMs, our engineering team ran a 48-hour internal hackathon on mastering the engineering behind the curtain. The hackathon targeted the orchestration layer around the model: structured, repeatable interactions built on raw API implementations rather than free-form text inputs.
The challenge was to build a functional feature that goes beyond string-in, string-out. The engineering team chose the focus areas together and landed on two.
Prompt management. Prompts moved into structured templates or logic-driven builders, so versions could be tracked and swapped without a full redeploy.
Output parsing. Models returned formats like JSON, with logic to handle malformed responses so the app stays stable when the model doesn’t.
Functional projects in 48 hours
Every participating developer submitted a functional project. Projects were shared company-wide, and within days, other teams began stress-testing one of the features.
“Hackathons give teams a structured, time-bound space to explore AI and build working projects. I strongly believe the fun, playful element is a big part of why they work. People try things they’d never risk in a regular sprint.”
— Richard Murby, Director of Business Development, Devpost
How to replicate this in your hackathons
Narrow the challenge. Ours had one target: structured, repeatable output.
Run a series of hackathons. Ours was the first of several short hackathons, each focused on one goal. Across Devpost’s data, developers who have joined six or more hackathons ship AI projects to production at 2.2 times the rate of first-timers.
Consider who picks the topic. Our engineers chose the focus areas together, which kept the work tied to what the team wanted to master.
Match the wrap-up to your team. We skipped pitch presentations for a technical retro, which let each developer walk through what worked and the challenges they hit. For our team, that left more room for honest feedback.
Most teams already have access to good models. What they need is time to build around them. Read the full breakdown of our hackathon.