The best Llm Evaluation newsletters

Pre-registered, publicly documented ML experiments with specific numbers, covering LLM-as-a-judge reliability, API versus local model costs, and EWC versus flat L2 regularisation. Readers get a transparent look at what each experiment tested, whether its gates passed or failed, and why.

A personal blog-style newsletter by Aidan Cooper covering hands-on experiments with large language models, especially chess benchmarks, alongside essays on software engineering practice. It offers original experiments, methodology details and reflective takes on engineering team culture.

A technical blog from GovTech Singapore covering how the government builds, evaluates and governs AI systems for public service use. Posts include benchmark design, agent observability engineering, and leadership perspectives on central versus embedded AI engineering.

A hands-on course that builds an AI evaluation and regression-gating harness lesson by lesson, covering golden datasets, trend tracking, severity-weighted policies and CI exit codes. Each issue walks through code design decisions and verified test results.

Practical guides to evaluating AI agents with tools such as Promptfoo and DeepEval, covering agent-as-a-judge grading, hallucination testing and RAG metrics. Each post explains how to design evidence-based tests so teams can diagnose failures rather than guess.

Practitioner-written posts on building production AI features, covering LLM evaluation methods, voice AI infrastructure, and data sovereignty options in Australia. Readers get concrete technical tradeoffs and practical guidance for shipping reliable non-deterministic systems.

A practical newsletter for full-stack JavaScript developers aiming to think like architects, covering React, Node, software architecture and production AI systems such as LLM agents and evals. Readers get concrete, code-level guidance like testing LLM judges and debugging agent behaviour, plus occasional promotion of a Node.js book and sponsors.

A company blog from an AI evaluation and observability platform covering AI agent research, industry developments, and product announcements. Readers get paper breakdowns, critical commentary on AI news, and engineering context for debugging agents in production.

MakerPulse is a practitioner-focused newsletter on building with AI, mixing tools, tutorials and analysis. Recent issues cover collaborative creative workflows with AI and results from the author's AgentPulse benchmark of frontier and open models on structured versus creative writing.