, Rudvanth Labs
01Research tracks
We keep the list short on purpose. A four-person lab with twelve research directions has none.
How do you prove an AI system is working?
Most evaluation in industry is a spreadsheet of vibes. We work on evaluation sets that encode domain-specific notions of correctness, on graders that are themselves validated against human judgement, and on the reporting format that lets a non-technical approver read a result and act on it.
How do you make an answer defensible?
Retrieval quality, not generation quality, is what usually determines whether an enterprise system is trustworthy. Our work here concentrates on citation-faithful generation, retrieval over documents with real structure, tables, appendices, amendments, cross-references, and calibrated refusal when the corpus does not contain the answer.
How do you teach a general model a specific world?
Regulated domains carry vocabulary, precedence rules and edge cases that no general model has internalised. We study where the boundary sits between prompting, retrieval, fine-tuning and explicit symbolic constraints, and how to make that choice on evidence rather than fashion.
How do you run this where it actually has to run?
A great deal of useful AI has to operate on a factory floor, inside a hospital network, or on infrastructure with a fixed monthly budget. We work on quantisation and distillation trade-offs measured against task-level evaluation rather than perplexity, and on serving architectures for intermittent connectivity.
02Publication policy
Methods, evaluation harnesses, benchmark definitions and negative results are published. Negative results especially: “we tried this and it did not work, here is the measurement” is the single most useful thing a small lab can contribute, and almost nobody does it.
The two exceptions are client data, never, under any circumstance, in any form, and the specific product implementations we intend to commercialise. We will describe how something works before we hand over the thing itself.
Where a research note draws on client work, it is published only with written consent and with the domain sufficiently abstracted that the client is not identifiable unless they want to be.
Some of our research initiatives are being developed in stealth and will be announced as they mature. Where that is the case we will say so, rather than leave a gap and let it read as inactivity.
03Rudvanth Research Index
Research notes carry a permanent reference. RN-001 stays RN-001, so a result can be pointed at in a paper, a procurement document or an issue thread without anyone having to describe which blog post you meant.
A grounded answer with a citation attached is not the same as a grounded answer. Early design notes on grounding-bench, an open benchmark for whether a generated claim actually follows from the source it points at.
Read the notePublic benchmarks reward answering. Enterprise deployments are killed by answering when you should not have. Notes on why calibrated abstention deserves its own test suite, and what we are building.
Read the noteWriting from the delivery side, what we learn building these systems in production, and the things we wish we had known earlier.
Read the engineering notesWe work with academic groups and industry partners on shared benchmarks and evaluation sets, particularly where a domain has no public standard. Write to [email protected].