Tag
confidence-intervals
- Apr 7, 2026 Why Monte Carlo Error Falls as the Square Root of the Sample Count The convergence rate follows from the variance of a sum, and the practical consequences are a fixed digit budget per factor of a hundred in work and an error bar that is itself estimated. scientific computing
- May 13, 2025 Confidence Intervals for Accuracy on Small Evaluation Sets A score on a few hundred items is a sample from a binomial, and the interval around it is usually wider than the differences being reported. How to compute it and when the independence assumption fails. inference systems
- Oct 16, 2024 Confidence Intervals for the Difference Between Two Benchmark Runtimes Why two overlapping intervals can still hide a real difference, how the standard error of a difference is computed from paired or independent samples, and what a ratio estimate needs before it is trustworthy. performance engineering
- Apr 28, 2024 Quantiles from Finite Samples and Their Confidence Bands What a p99 computed from a few hundred observations actually estimates, how the order statistic maps to a binomial interval, and why tail claims need sample sizes that frequently do not exist. performance engineering
- Aug 7, 2023 Sampling Error, Skid, and How Many Samples a Profile Needs Why a profile is a binomial estimate, how instruction skid biases samples toward long-running instructions, and what confidence a given sample count actually supports. observability