03Project 03 of 3

MRS Studio, a self-hosted multi-agent assistant

An assistant designed for a small company: chat and voice in, scheduled workflows, business tools. It runs on my own server; Claude is an option chosen per conversation.

Designed and directed alone
Since March 2026
TypeScript (NestJS, React)
Python services · Docker · vLLM

8specialist agents behind one assistanton 14 October 2026
2 × RTX 5090self-hosted inference, image and video generation, local workers for Claude Codeon 14 October 2026
176-193tokens per second written by Flash Next on a single RTX 5090 (Strata)bench of 11 October 2026
7 daysof model calls kept in full at the gateway, so any call can be re-read and replayed as it was sentrolling window, since 10 August 2026

The big picture

Web researchMemoryEmailCalendarMain
assistantDocumentsMediaWorkflowsSettings

One door per domain. The main assistant only routes; each specialist works in its own context and returns a summary with its sources.

InputsWeb chat · streamingVoice · Parakeet / KokoroScheduled workflows

Hub · NestJSMain assistant8 specialistsWorkflow engine

Tools · MCPHybrid RAG memoryWeb · self-hosted SearXNGIBM iEmail · calendarDocuments · OCR

InferenceGateway · OpenAI APIGPU 0 · Qwen MoE and Flash Next, shared with ComfyUIGPU 1 · Qwen denseClaude · Agent SDK

Everything runs on my own server, except Claude’s cloud models. Claude (in orange) is chosen per conversation. Claude Code, the engine behind it, can work either with Anthropic’s models in the cloud or with the server’s own models. Each model is a provider set in the app, behind an OpenAI-compatible API: changing a model does not touch the assistant’s code.

How the assistant delegates

Main assistant

"Summarise the latest vLLM release notes"

Modeldecides: answer itself or delegate

Simple tools, no business logicdate · files · workspaces

Specialist · web research

Its own instructions and method

Modelloops over several turns, separate context

search_web · fetch_pageSearXNG, then reads the pages

The main assistant has no business tool. It receives only each specialist’s summary and sources, so its context stays short. Scheduled workflows go through the same specialists.

Each specialist has its own method

A specialist has its own instructions, its own tools and a declared scope.

How it finds information

Question

By meaningEmbeddingGemma-300M

By exact wordsBM25

FusionRRF

Rerankercross-encoderre-sorts 8 candidates per result kept, 40 at least

Contextthe best results + linked notes

Two complementary searches, fused, then ranked by a more precise model. Searching by meaning finds “prefers black coffee” when asked about “favourite drinks”; a keyword search (BM25) catches the exact names (a program, a table). The reranker is multilingual.

Inference on my own hardware

Chat and voice

Workflows

Specialists

Coding workers (project 01)

GatewayOpenAI-compatible API

one model at a time per GPU
every call logged

GPU 0 · MoEQwen 3.6 35B-A3B · NVFP4

volume: summarise, list
shared with Flash Next and ComfyUI

GPU 1 · denseQwen 3.8 27B · NVFP4 · NInfer

judgment: cite sources, review
always loaded

ClaudeAgent SDK · option

Two kinds of local model. In a mixture-of-experts model (MoE), only a small part of the model works on each word, which makes it fast; a dense model works as a whole on each word: slower, but more reliable.

An OpenAI-compatible gateway

Every model sits behind this one gateway, which loads each on its card when needed and logs every call. The MoE serves volume, the dense model serves judgment, and Flash Next serves local coding sessions.

A mixture-of-experts model on one card

Flash Next, a Qwen mixture-of-experts model, runs on a single RTX 5090 with Strata, an engine derived from llama.cpp that keeps the experts in system memory and the most used ones on the card: 176 to 193 tokens per second.

The dense model: eight conversations at once

Since 13 October 2026 the dense model runs on NInfer, a young open-source inference engine, in an in-house 4-bit quantization (NVFP4). Its KV cache, the memory that each conversation’s context takes on the card, now holds 604,224 tokens, against 330,107 under vLLM, the engine it used before: room for eight conversations of 75,000 tokens each. With eight conversations at once it writes about 1,000 tokens per second in total on short prompts, and about 790 with 32,000 tokens of context each (bench of 13 October 2026).

How it generates images and video

Media specialiston a model that sees images

Gatewayunloads the MoE during the render

ComfyUI · GPU 0ImageEditVideoUpscale to 4K

Looks at the result before answering

The specialist looks at what it produced before it answers. Hence the rule: generation is reserved for models that can see. On a blind model, an image request produced a story with links to images that were never created. Generation runs on GPU 0, the card with the fastest link to the rest of the machine, because it is the one that loads and unloads models most often.

How it is evaluated

Retrieval

A set of test questions with known answers, replayed at every change to the search: how often the right note comes back among the first results (recall@k).

Prompts

A prompt change is run on real chat turns and judged by a second model; it counts as done only after a statistical test suited to small samples.

Design choices

Why not LangGraph or LangChain?

Evaluated twice, in June and again in August against the 2026 Deep Agents releases. My workflow engine already covers what LangGraph offers (steps, persistent state, pause and resume, scheduling) without adding a dependency. The framework’s releases broke three times in four months, and its default interface at the time did not work with the inference server used here (vLLM). A framework would make sense for a product sold to clients who change the orchestration, kept behind MCP, the standard tool interface.

Decision of 23 June, reviewed on 24 August 2026

Why local models?

Sensitive data stays on the machine, unless Claude is chosen for a conversation. The cost is fixed instead of paid per token. And it is a lot more fun this way.

What does multi-agent mean, concretely?

The specialists above are agents, each with its own context. Development follows the same principle: Claude orchestrates, and local Qwen models do the big reading jobs in parallel (project 01).

How do I limit hallucinations and silent failures?

An answer must come from a source. The web specialist used to answer from the model’s own memory in 17 of 24 test runs; since its first rule requires a search before any answer, that dropped to 0 of 60, and a run in which no search succeeded reports a failure instead of passing on an unsourced answer. Every search result also tells the model what it has not seen: how many results are shown, how many more were found, what was cut, whether a source is down. Each search tool says what it covers and which tool holds the rest: the assistant once called 9 files “everything” in a space holding 28 documents, because it had asked the wrong tool. Finally, nothing fails quietly: an error is shown and logged, and when a tool refuses, the model gets one sentence saying what it tried and why.

17 of 24 without a search, then 0 of 60 · August 2026

Limits

One user
MRS Studio is my own company’s assistant: one real user, me. It is not a hosted product with customers.
Local models, not a frontier model
The assistant runs on local models of the 27 to 35 billion parameter class: slower and less capable than a frontier model. That is why routing is simple, each specialist sticks to one domain, and Claude stays an option per conversation.
A shared GPU
The MoE, Flash Next and image and video generation share GPU 0. A render evicts the MoE, which reloads afterwards; a local coding session takes the whole card, and the MoE and renders are refused until it is released, 20 minutes after the last coding request.