evaluation — independent software & tools

7 independent evaluation listings the big search engines bury.

• factory

  github · Free · by crewhaus

  Open-source compiler for AI agents. Write one crewhaus.yaml; compile it to a CLI, a Slack bot, and an eval harness from the same spec. Apache-2.0.

• never-ask-twice

  github · Free · by Marcelle-Labs

  Enterprise Support MemoryAgent on Qwen Cloud that remembers customer context across sessions and proves it with a live re-ask-rate ablation.

• litmux

  github · Free · by Topsailpediculati120

  Build unit tests for AI prompts to compare models, track performance, and prevent regressions.

• llm-compress

  github · Free · by luckystar-pear

  Compress context data to optimize memory and performance in C++ large language model applications within the llm-cpp toolkit.

• doc2dataset

  github · Free · by reisel-g

  📄 Ingest documents into structured datasets for LLMs, ensuring numeric integrity and easy export across multiple frameworks with doc2dataset.

• llm-eval

  github · Free · by wahhyun

  Evaluate large language models with tools for performance and consistency scoring as part of the llm-cpp C++ infrastructure toolkit.

• mcp-evals

  npm · Free · by mclenhard

  GitHub Action for evaluating MCP server tool calls using LLM-based scoring