Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard

Developers behind Memdoor released a multi-agent programming workflow named team-solve that coordinates five Qwen3.8-27B model roles to solve complex competitive coding problems. According to github.com's report, the setup rebuilds the multi-agent method of GVS5H and achieved an average score of 9.8 out of 10 across five passes on the first ten LiveCodeBench Hard problems. That performance topped both the authors' original 27B harness score of 9.6 and a standalone Fable 5 run that scored 8.0.

This piece covers the architecture, benchmark scores, hosting constraints, and operational commands for running the team-solve pipeline. It is intended for software engineers and systems builders who evaluate multi-agent orchestration frameworks against single frontier models for difficult automated programming benchmarks.

What shipped in A team of 5 Qwen3.8-27B models

The release provides a directed acyclic graph workflow executed by the underlying mario engine. The pipeline separates problem-solving tasks into five distinct agent roles comprising a planner, two parallel ideators, a solver, and a review manager, supplemented by a repair worker. Each agent executes inside a completely fresh context window without shared chat history, reading and writing state strictly through markdown files on disk. The planner writes plan.md alongside an anchor ledger in notes.md, while ideator A proposes a direct provable algorithm and ideator B proposes a technique from a different algorithmic family.

Deterministic shell commands govern the testing phases rather than model-based assertions. The solve step writes candidate code to solution.py, which the test command evaluates against sample inputs and outputs before writing logs to test-output.log and test-status.txt. The review agent analyzes captured execution logs to supply repair instructions back into notes.md when tests fail. A final verify command runs check-samples.sh against sample tests, allowing nonzero exit codes to fail the run while passing samples finish the workflow.

What improved in A team of 5 Qwen3.8-27B models

Rebuilding the harness as a Memdoor workflow improved execution reliability and achieved competitive parity against larger frontier systems. The team-solve implementation resolved 49 out of 50 total attempted runs across five passes on the ten selected LiveCodeBench Hard tasks. The runs cost $35.79 in total API spend across 15.3 million output tokens, or $38.77 when counting sixteen attempts that paused, hit time limits, or halted for fixes. Runs posted a median model duration of 40 minutes and a median cost of $0.39 per problem.

Four core platform modifications enabled the workflow to match the original prompts byte for byte. Memdoor added reasoning fallback extraction when a model generates no visible text, direct support for top_p and top_k configuration parameters on model lines, a mark_cutoff flag for token caps, and independent four-hour execution timers. Infrastructure routing was also adjusted from local FP8 vLLM instances to OpenRouter providers, routing 203 calls to Modal, 27 to CoreWeave, and 115 to Wafer before locking to FP8 providers when Modal transfer speeds dropped to 14 tokens per second.

What you gain from A team of 5 Qwen3.8-27B models

Teams gain a reproducible local engineering workflow that solves programming problems without paying frontier model pricing. In verification tests on a sample requiring equal counts of three characters across strings up to 200,000 characters, the solver selected ideator A's prefix-state method running in linear time with a dictionary of pair differences. A brute-force check on 300 random test strings confirmed zero mismatches, and the resulting Python script processed 200,000 characters in 0.04 seconds without requiring manager intervention.

Engineers also gain isolation from cascading context drift because agents interact only through files stored in the filesystem. Resiliency updates prevent dropped streams from killing entire runs. Setting max_retries to 1 on agent steps allows failed streaming calls to recover, while the partition flag lets developers resume stalled executions while preserving completed work from preceding planner and ideator nodes. Gateway version 1.692.0 further ensures that workspace agent resolution succeeds without workspace-level lookups failing immediately on initialization.

How to get A team of 5 Qwen3.8-27B models

The framework installs via a bash script and connects to cloud model aggregators or local runtime engines. Run the installation script and configure an OpenRouter API key:

curl -fsSL https://memdoor.ai/install.sh | bash
memdoor connect openrouter

Local inference setups can bypass cloud API keys by connecting directly to local servers like Ollama, llama.cpp, vLLM, or LM Studio:

memdoor connect local

Clone the repository and copy the workflow template to your global or project directory:

git clone --depth 1 https://github.com/guregodevo/memdoor
mkdir -p ~/.memdoor/workflows
cp -r memdoor/examples/workflows/team-solve ~/.memdoor/workflows/

To run a problem, copy the sample files into a working directory containing task.md alongside paired tests/N.in and tests/N.out files, then trigger execution:

mkdir my-problem && cd my-problem
cp -r ../memdoor/examples/workflows/team-solve/sample/* .
memdoor workflow run team-solve --wait

Users can monitor execution visually inside the terminal interface by typing /workflow in the Memdoor TUI. Individual model assignments inherit whatever default is pinned in the client or set in the project configuration.

What to watch after A team of 5 Qwen3.8-27B models

Two specific failure modes in the underlying loop design require operational attention. In Pass 5 on problem abc399_e, round one produced a working program that solved 32 of 44 hidden tests, but the manager invented an invalid counterexample with an incorrect expected output. The team rewrote the solution to pass the invalid case, resulting in regression because the public samples failed to expose the error. Adding small-input brute-force checks against manager counterexamples would prevent similar degradations.

In Pass 4 on problems arc196_c and arc196_d, valid programs that passed all hidden tests were overwritten during subsequent rounds because the manager failed to emit an explicit completion signal. While both runs eventually recovered correct programs at round eight and the final round, they racked up $3.44 and $4.24 in token charges, representing the two costliest runs in the dataset. Retaining the latest program that successfully cleared public test samples will safeguard against unnecessary late-round code rewrites.