MattHicks.com Programming on the Edge

All projects

84.0%
LongMemEval-S question answering, with a local 9-billion-parameter model doing both the answering and the grading
99.4%
LongMemEval-S retrieval: the right memory in the top five results
84%
SWE-bench Verified pilot: 21 of 25 real-world bug fixes resolved with no human involvement, on a local 27-billion-parameter model

Benchmarks run September 2026. The SWE-bench figure is a 25-task pilot on Astropy and Django, not the full 500-task set.

Sigil

The agent framework under Voidcraft and ShopMagic.

In production

Sigil is a Scala 3 framework for AI agents that hold real conversations, with people and with each other. It runs in production today, under Voidcraft and ShopMagic, and is closed source.

What agents built on Sigil can do

  • Hold multi-party conversations, with several people and several agents in the same thread.
  • Use tools, and find new capabilities when they need them.
  • Remember what matters across sessions, and recall it when it becomes relevant.
  • Stream their work live to every device in the conversation, including tool progress as it happens.
  • Run on any model, hosted or on your own hardware.

Benchmarks

Memory and tooling carry the most weight when the model is small. In these benchmarks, the answering, grading, and coding were all done by small open models on local hardware.

LongMemEval-S tests whether an assistant can answer questions about details buried in long histories of past conversations, across 500 questions. Sigil answered 84.0% correctly, with a local 9-billion-parameter model doing both the answering and the grading. The right memory was in the top five retrieved results 99.4% of the time, and remembering added only about 720 tokens of context per question.

SWE-bench Verified gives an agent real bug reports from open-source Python projects and checks the fix against the project's own tests. In a 25-task pilot on Astropy and Django, a local 27-billion-parameter model resolved 21, with no human involvement. Each task ran in its own container, graded by the official harness, unchanged.