LimitLine

Validation: Checking LimitLine’s Math

Generated 2026-11-17 · code version d899f86 (2026-11-17) · Node.js 22.23.4

775 of 775checks pass, across 34 reference cases

What Was Tested

A script loads LimitLine’s calculation modules (spc-core.js, stats-core.js and lean-core.js), runs each case below, and writes this page. These are the same source files that are built into the app at limitline.org/app. The script runs them in Node.js, not in a browser; both use the same IEEE double-precision arithmetic.

Each row compares one number from LimitLine with the expected value. Under each expected value is where it comes from:

Published values are rounded, so the tolerance is set by how many digits the source prints. These are the same tolerances LimitLine’s automated tests use.

What This Does Not Cover

Variables Control Charts 47/47

Individuals (I-MR), X̄-R and X̄-S limits, and the control-chart constants they use.

NIST Individuals Chart: Flow Rate, 10 Batches

9/9 pass

Data: nist-6322-flow-rate.csv

StatisticExpectedLimitLineDifferenceToleranceResult
Moving ranges (rounded to 0.1)2, 2.3, 1.4, 3.5, 3.4, 1.4, 0.2, 1.2, 1.5printed in the source2, 2.3, 1.4, 3.5, 3.4, 1.4, 0.2, 1.2, 1.5noneexact matchPASS
Mean (center line)50.81printed in the source50.810± 1e−4PASS
Average moving range MR̄1.8778printed in the source1.877777777778−2.2e−5± 1e−4PASS
Individuals UCL55.8041printed in the source55.80408983452−1.0e−5± 0.001PASS
Individuals LCL45.8159printed in the source45.815910165481.0e−5± 0.001PASS
MR chart UCL = 3.267 × MR̄6.1347726computed here from the source’s data and formula6.1347−7.3e−5± 0.001PASS
Points beyond the 3-sigma limits0printed in the source0noneexact matchPASS
Phase I: limits from the first 10 of 13 points, UCLThree extra points (60, 50, 51) added after the baseline must not move the limits.55.8041printed in the source55.80408983452−1.0e−5± 0.001PASS
Phase I: limits from the first 10 of 13 points, LCL45.8159printed in the source45.815910165481.0e−5± 0.001PASS

NIST Table of Control-Chart Constants A2, D3, D4 (n = 2 to 10)

27/27 pass

LimitLine derives d2 and d3 by numerical integration and c4 from the gamma function; this compares the resulting factors with NIST’s printed table.

Show All 27 Checks
StatisticExpectedLimitLineDifferenceToleranceResult
A2, n = 21.88printed in the source1.879969818288−3.0e−5± 0.001PASS
D3, n = 20printed in the source00± 0.001PASS
D4, n = 23.267printed in the source3.266523688828−4.8e−4± 0.001PASS
A2, n = 31.023printed in the source1.0233259525863.3e−4± 0.001PASS
D3, n = 30printed in the source00± 0.001PASS
D4, n = 32.575printed in the source2.574593665255−4.1e−4± 0.001PASS
A2, n = 40.729printed in the source0.7285974499089−4.0e−4± 0.001PASS
D3, n = 40printed in the source00± 0.001PASS
D4, n = 42.282printed in the source2.2820546448095.5e−5± 0.001PASS
A2, n = 50.577printed in the source0.5768190730159−1.8e−4± 0.001PASS
D3, n = 50printed in the source00± 0.001PASS
D4, n = 52.115printed in the source2.114496137029−5.0e−4± 0.001PASS
A2, n = 60.483printed in the source0.48324654313692.5e−4± 0.001PASS
D3, n = 60printed in the source00± 0.001PASS
D4, n = 62.004printed in the source2.003831266449−1.7e−4± 0.001PASS
A2, n = 70.419printed in the source0.41928346042232.8e−4± 0.001PASS
D3, n = 70.076printed in the source0.07570367850434−3.0e−4± 0.001PASS
D4, n = 71.924printed in the source1.9242963214963.0e−4± 0.001PASS
A2, n = 80.373printed in the source0.3725274556687−4.7e−4± 0.001PASS
D3, n = 80.136printed in the source0.1361723798821.7e−4± 0.001PASS
D4, n = 81.864printed in the source1.863827620118−1.7e−4± 0.001PASS
A2, n = 90.337printed in the source0.3366969357212−3.0e−4± 0.001PASS
D3, n = 90.184printed in the source0.18401834324911.8e−5± 0.001PASS
D4, n = 91.816printed in the source1.815981656751−1.8e−5± 0.001PASS
A2, n = 100.308printed in the source0.30826327064752.6e−4± 0.001PASS
D3, n = 100.223printed in the source0.22302445808462.4e−5± 0.001PASS
D4, n = 101.777printed in the source1.776975541915−2.4e−5± 0.001PASS

Published c4 and d2 Values

4/4 pass
StatisticExpectedLimitLineDifferenceToleranceResult
c4, n = 20.7979printed in the source0.79788−2.0e−5± 1e−4PASS
c4, n = 50.94printed in the source0.93999−1.0e−5± 1e−4PASS
c4, n = 250.9896printed in the source0.989644.0e−5± 1e−4PASS
d2, n = 253.931printed in the source3.93063−3.7e−4± 0.001PASS

X̄-R and X̄-S Limits Recomputed by Hand with NIST’s Printed Constants

7/7 pass

The data are a small LimitLine test set (5 subgroups of 5), not a published example. Expected values are computed in this script from the raw data with NIST’s printed A2 = 0.577, D4 = 2.115 and c4 = 0.9400, without using LimitLine’s code.

Data: xbar-r-test-5x5.csv

StatisticExpectedLimitLineDifferenceToleranceResult
Grand mean X̿10.036computed here from the source’s data and formula10.0360± 1e−12PASS
X̄ chart UCL (X̿ + 0.577 R̄)10.33604computed here from the source’s data and formula10.33594591797−9.4e−5± 5e−4PASS
X̄ chart LCL (X̿ − 0.577 R̄)9.73596computed here from the source’s data and formula9.7360540820329.4e−5± 5e−4PASS
R chart UCL (2.115 R̄)1.0998computed here from the source’s data and formula1.099537991255−2.6e−4± 5e−4PASS
R chart LCL (D3 = 0 for n = 5)0computed here from the source’s data and formula0noneexact matchPASS
X̄-S chart UCL (X̿ + 3 s̄ / (c4 √5))10.32015959673computed here from the source’s data and formula10.320162619733.0e−6± 1e−4PASS
S chart UCL (B4 from c4 = 0.9400)0.4158743549349computed here from the source’s data and formula0.41589416701482.0e−5± 0.001PASS

Attribute Control Charts 27/27

c, p, np, u and Laney P′ charts.

NIST c Chart: Defects on 25 Wafers

4/4 pass

Data: nist-6331-wafer-defects.csv

StatisticExpectedLimitLineDifferenceToleranceResult
c̄ (center line)16printed in the source160± 1e−4PASS
UCL28printed in the source280± 1e−4PASS
LCL4printed in the source40± 1e−4PASS
Wafers beyond the limits (1-based)24printed in the source24noneexact matchPASS

NIST p Chart: 30 Wafers of 50 Chips

8/8 pass

NIST prints the fraction defective per wafer. Counts are fraction × 50. Limits are computed here from those data with the standard formula p̄ ± 3√(p̄(1 − p̄)/n), independently of LimitLine’s code.

Data: nist-6332-wafer-proportions.csv

StatisticExpectedLimitLineDifferenceToleranceResult
Total defectives347computed here from the source’s data and formula347noneexact matchPASS
p̄ = 347 / 15000.2313333333333computed here from the source’s data and formula0.23133333333330± 1e−12PASS
UCL0.4102391185947computed here from the source’s data and formula0.41023911859470± 1e−12PASS
LCL0.05242754807193computed here from the source’s data and formula0.052427548071930± 1e−12PASS
UCL to 4 decimals0.4102computed here from the source’s data and formula0.41023911859473.9e−5± 0.001PASS
Wafers beyond the limits (1-based)15, 23computed here from the source’s data and formula15, 23noneexact matchPASS
np chart center = n·p̄11.56666666667computed here from the source’s data and formula11.566666666670± 1e−12PASS
np chart UCL20.51195592974computed here from the source’s data and formula20.511955929740± 1e−12PASS

Montgomery u Chart: Defects in Computers, 20 Samples of 5

3/3 pass

The center line is printed. The limits are computed from the printed center line as 1.93 ± 3√(1.93/5) and compared at 2 decimals.

Data: montgomery-u-computers.csv

StatisticExpectedLimitLineDifferenceToleranceResult
ū (center line)1.93printed in the source1.930± 1e−12PASS
LCL0.07computed here from the source’s data and formula0.06613305195891→ 0.07−3.9e−3same at 2 decimalsPASS
UCL3.79computed here from the source’s data and formula3.793866948041→ 3.793.9e−3same at 2 decimalsPASS

Montgomery u Chart: Dyeing Rolls of Cloth, Unequal Inspection Units

5/5 pass

The center line is printed. Per-roll limits are computed from ū = 153/107.5 with ū ± 3√(ū/n) and compared at 2 decimals.

Data: montgomery-u-cloth-rolls.csv

StatisticExpectedLimitLineDifferenceToleranceResult
ū (center line)1.423printed in the source1.4232558139532.6e−4± 5e−4PASS
UCL, roll 2 (8 units, widest)2.69computed here from the source’s data and formula2.688626427923→ 2.69−1.4e−3same at 2 decimalsPASS
UCL, roll 3 (13 units, narrowest)2.42computed here from the source’s data and formula2.41589419127→ 2.42−4.1e−3same at 2 decimalsPASS
LCL, roll 20.16computed here from the source’s data and formula0.1578851999839→ 0.16−2.1e−3same at 2 decimalsPASS
LCL, roll 30.43computed here from the source’s data and formula0.4306174366369→ 0.436.2e−4same at 2 decimalsPASS

Laney P′ Chart: 16 Months, Sample Sizes 8,755 to 22,300

7/7 pass

Data: spcforexcel-laney-p.csv

StatisticExpectedLimitLineDifferenceToleranceResult
p̄0.4774printed in the source0.4773725037135−2.7e−5± 1e−4PASS
σz = MR̄ / 1.128 (article’s MR̄ 10.4108)9.229432624113computed here from the source’s data and formula9.229389043457−4.4e−5relative ± 2e−3PASS
Jan-07 LCL (%)32.96printed in the source32.95668793931→ 32.96−3.3e−3same at 2 decimalsPASS
Jan-07 UCL (%)62.52printed in the source62.51781280338→ 62.52−2.2e−3same at 2 decimalsPASS
Jul-07 LCL (%)38.24printed in the source38.23883615181→ 38.24−1.2e−3same at 2 decimalsPASS
Jul-07 UCL (%)57.24printed in the source57.23566459089→ 57.24−4.3e−3same at 2 decimalsPASS
Months outside the limitsJul-07printed in the sourceJul-07noneexact matchPASS

Process Capability 11/11

Cp, Cpk, Pp, Ppk and the Cpk confidence interval.

NIST Capability Example: USL 20, LSL 8, Mean 16, s 2

9/9 pass

NIST gives summary statistics only. To drive LimitLine’s code, a 6-point dataset with mean exactly 16 and sample s exactly 2 is built, and σ within is set to 2 to mirror the handbook. In the app, Cp/Cpk use σ within = MR̄/d2, so loading this file there reproduces Pp = 1.000 and Ppk = 0.667, not Cp/Cpk.

Data: nist-616-capability-mean16-s2.csv

StatisticExpectedLimitLineDifferenceToleranceResult
Data set mean16computed here from the source’s data and formula160± 1e−4PASS
Data set s2computed here from the source’s data and formula20± 1e−4PASS
Cp1printed in the source10± 1e−4PASS
Cpu0.6667printed in the source0.6666666666667−3.3e−5± 1e−4PASS
Cpl1.3333printed in the source1.3333333333333.3e−5± 1e−4PASS
Cpk0.6667printed in the source0.6666666666667−3.3e−5± 1e−4PASS
Pp (overall s = 2)1printed in the source10± 1e−4PASS
Ppk (overall s = 2)0.6667printed in the source0.6666666666667−3.3e−5± 1e−4PASS
One-sided (USL only): Cpk = Cpu0.6667printed in the source0.6666666666667−3.3e−5± 1e−4PASS

Cpk 95% Confidence Interval: Cpk 1.17, 50 Points

2/2 pass

LimitLine uses Bissell’s approximation Cpk ± 1.96·√(1/(9n) + Cpk²/(2(n − 1))).

StatisticExpectedLimitLineDifferenceToleranceResult
Lower 95% limit0.92printed in the source0.9206097774316→ 0.926.1e−4same at 2 decimalsPASS
Upper 95% limit1.42printed in the source1.419390222568→ 1.42−6.1e−4same at 2 decimalsPASS

Gauge R&R (Measurement Systems Analysis) 13/13

Crossed Gauge R&R by the ANOVA method. Like Minitab’s default, LimitLine removes the part × operator interaction when its p-value is above 0.05.

AIAG Gauge Study: 10 Parts × 3 Operators × 3 Trials, ANOVA Mean Squares and Variance Components

9/9 pass

Data: aiag-msa4-gage-rr.csv

StatisticExpectedLimitLineDifferenceToleranceResult
MS part9.81799printed in the source9.817992716049→ 9.817992.7e−6same at 5 decimalsPASS
MS operator1.58363printed in the source1.583631111111→ 1.583631.1e−6same at 5 decimalsPASS
MS repeatability (pooled)0.03997printed in the source0.03997327635328→ 0.039973.3e−6same at 5 decimalsPASS
Part × operator interaction p-value0.974printed in the source0.974106404329→ 0.9741.1e−4same at 3 decimalsPASS
Interaction removed from the model (p > 0.05)removedprinted in the sourceremovednoneexact matchPASS
Variance component, operator0.0514553printed in the source0.05145526115859→ 0.0514553−3.9e−8same at 7 decimalsPASS
Variance component, part-to-part1.08645printed in the source1.086446604411→ 1.08645−3.4e−6same at 5 decimalsPASS
Variance component, total gauge R&RThe printed 0.0914253 is the sum of the rounded 0.03997 + 0.0514553; the unrounded sum is 0.0914285, so the check allows 5 parts in 100,000.0.0914253printed in the source0.091428537511873.2e−6relative ± 5e−5PASS
Variance component, total variation1.17788printed in the source1.177875141923→ 1.17788−4.9e−6same at 5 decimalsPASS

AIAG Gauge Study: % Contribution (ANOVA Method)

4/4 pass

Data: aiag-msa4-gage-rr.csv

StatisticExpectedLimitLineDifferenceToleranceResult
% contribution, Total gauge R&R7.76printed in the source7.762158675209→ 7.762.2e−3same at 2 decimalsPASS
% contribution, Repeatability3.39printed in the source3.393676878861→ 3.393.7e−3same at 2 decimalsPASS
% contribution, Reproducibility4.37printed in the source4.368481796348→ 4.37−1.5e−3same at 2 decimalsPASS
% contribution, Part-to-part92.24printed in the source92.23784132479→ 92.24−2.2e−3same at 2 decimalsPASS

Regression and Numerical Accuracy 22/22

NIST Statistical Reference Datasets with certified values. Longley is a notoriously ill-conditioned regression; NumAcc2 and NumAcc4 test whether means and standard deviations keep their digits when values are large and close together. “Correct digits” is the log relative error, LRE = −log10(|value − certified| / |certified|).

NIST StRD Longley: 6 Predictors, 16 Observations

17/17 pass

Data: nist-strd-longley.csv

Show All 17 Checks
StatisticExpectedLimitLineDifferenceToleranceResult
Coefficient, Intercept−3482258.634596NIST certified value−3482258.634596−6.5e−9relative ± 1e−8PASS
Coefficient, GNPDEFL15.06187227137NIST certified value15.061872271382.5e−12relative ± 1e−8PASS
Coefficient, GNP−0.03581917929259NIST certified value−0.03581917929259−6.5e−16relative ± 1e−8PASS
Coefficient, UNEMP−2.020229803817NIST certified value−2.020229803817−3.1e−15relative ± 1e−8PASS
Coefficient, ARMED−1.033226867174NIST certified value−1.033226867174−3.1e−15relative ± 1e−8PASS
Coefficient, POP−0.05110410565358NIST certified value−0.051104105653583.2e−15relative ± 1e−8PASS
Coefficient, YEAR1829.151464614NIST certified value1829.1514646146.1e−12relative ± 1e−8PASS
Standard error, Intercept890420.3836074NIST certified value890420.38360759.1e−8relative ± 1e−8PASS
Standard error, GNPDEFL84.91492577477NIST certified value84.914925774788.9e−12relative ± 1e−8PASS
Standard error, GNP0.03349100777224NIST certified value0.033491007772253.4e−15relative ± 1e−8PASS
Standard error, UNEMP0.4883996816517NIST certified value0.48839968165175.0e−14relative ± 1e−8PASS
Standard error, ARMED0.2142741631617NIST certified value0.21427416316172.2e−14relative ± 1e−8PASS
Standard error, POP0.2260732000694NIST certified value0.22607320006942.3e−14relative ± 1e−8PASS
Standard error, YEAR455.4784991422NIST certified value455.47849914234.7e−11relative ± 1e−8PASS
Residual standard deviation304.854073562NIST certified value304.8540735623.1e−11relative ± 1e−9PASS
R²0.9954790045773NIST certified value0.9954790045773−1.3e−15relative ± 1e−10PASS
F statistic330.2853392346NIST certified value330.2853392345−6.8e−11relative ± 1e−8PASS

NIST StRD NumAcc4: 1001 Values Near 10,000,000

3/3 pass

8.3 correct digits is the best possible for s here: 10000000.1 cannot be stored exactly in binary floating point. Stata 17’s published scores are shown for comparison.

Data: nist-strd-numacc4.csv

StatisticExpectedLimitLineDifferenceToleranceResult
Mean, correct digits (Stata 17: 15.7)10000000.2NIST certified value10000000.215.7 correct digits1.9e−9≥ 15 correct digitsPASS
Standard deviation, correct digits (Stata 17: 8.3)0.1NIST certified value0.10000000055888.3 correct digits5.6e−10≥ 8 correct digitsPASS
Lag-1 autocorrelation, correct digits (Stata 17: 10.7)−0.999NIST certified value−0.998999999981410.7 correct digits1.9e−11≥ 10.5 correct digitsPASS

NIST StRD NumAcc2: 1001 Values Near 1.2

2/2 pass

The lag-1 autocorrelation is computed by the stats module but not shown in the app today.

Data: nist-strd-numacc2.csv

StatisticExpectedLimitLineDifferenceToleranceResult
Standard deviation, correct digits (Stata 17: 15.6)0.1NIST certified value0.115.6 correct digits−2.8e−17≥ 15.5 correct digitsPASS
Lag-1 autocorrelation, correct digits (Stata 17: 16.0)−0.999NIST certified value−0.999all digits1.1e−16≥ 15.5 correct digitsPASS

Run Charts 97/97

Anhøj run-chart rules: a signal when the longest run on one side of the median is too long or the line crosses the median too few times.

Anhøj Limits for 10 to 30 Useful Points

21/21 pass

Each row feeds LimitLine a series with exactly that many points off the median and reads back its two limits.

Show All 21 Checks
StatisticExpectedLimitLineDifferenceToleranceResult
n = 10: longest-run limit / fewest crossings6 / 2printed in the source6 / 2noneexact matchPASS
n = 11: longest-run limit / fewest crossings6 / 2printed in the source6 / 2noneexact matchPASS
n = 12: longest-run limit / fewest crossings7 / 3printed in the source7 / 3noneexact matchPASS
n = 13: longest-run limit / fewest crossings7 / 3printed in the source7 / 3noneexact matchPASS
n = 14: longest-run limit / fewest crossings7 / 4printed in the source7 / 4noneexact matchPASS
n = 15: longest-run limit / fewest crossings7 / 4printed in the source7 / 4noneexact matchPASS
n = 16: longest-run limit / fewest crossings7 / 4printed in the source7 / 4noneexact matchPASS
n = 17: longest-run limit / fewest crossings7 / 5printed in the source7 / 5noneexact matchPASS
n = 18: longest-run limit / fewest crossings7 / 5printed in the source7 / 5noneexact matchPASS
n = 19: longest-run limit / fewest crossings7 / 6printed in the source7 / 6noneexact matchPASS
n = 20: longest-run limit / fewest crossings7 / 6printed in the source7 / 6noneexact matchPASS
n = 21: longest-run limit / fewest crossings7 / 6printed in the source7 / 6noneexact matchPASS
n = 22: longest-run limit / fewest crossings7 / 7printed in the source7 / 7noneexact matchPASS
n = 23: longest-run limit / fewest crossings8 / 7printed in the source8 / 7noneexact matchPASS
n = 24: longest-run limit / fewest crossings8 / 8printed in the source8 / 8noneexact matchPASS
n = 25: longest-run limit / fewest crossings8 / 8printed in the source8 / 8noneexact matchPASS
n = 26: longest-run limit / fewest crossings8 / 8printed in the source8 / 8noneexact matchPASS
n = 27: longest-run limit / fewest crossings8 / 9printed in the source8 / 9noneexact matchPASS
n = 28: longest-run limit / fewest crossings8 / 9printed in the source8 / 9noneexact matchPASS
n = 29: longest-run limit / fewest crossings8 / 10printed in the source8 / 10noneexact matchPASS
n = 30: longest-run limit / fewest crossings8 / 10printed in the source8 / 10noneexact matchPASS

Anhøj Figure 2 Case: 24 Points, 2 on the Median

6/6 pass

The series reproduces the figure’s pattern (1 = above the median, −1 = below, 0 = on it), not its original values.

Data: anhoj-figure2-pattern.csv

StatisticExpectedLimitLineDifferenceToleranceResult
Useful points (not on the median)22printed in the source22noneexact matchPASS
Longest run10printed in the source10noneexact matchPASS
Crossings3printed in the source3noneexact matchPASS
Longest-run limit7printed in the source7noneexact matchPASS
Fewest-crossings limit7printed in the source7noneexact matchPASS
Signalyesprinted in the sourceyesnoneexact matchPASS

Anhøj Limits for 31 to 100 Useful Points

70/70 pass

Same method as above, against the longer table in the qicharts2 vignette.

Show All 70 Checks
StatisticExpectedLimitLineDifferenceToleranceResult
n = 31: longest-run limit / fewest crossings8 / 11printed in the source8 / 11noneexact matchPASS
n = 32: longest-run limit / fewest crossings8 / 11printed in the source8 / 11noneexact matchPASS
n = 33: longest-run limit / fewest crossings8 / 11printed in the source8 / 11noneexact matchPASS
n = 34: longest-run limit / fewest crossings8 / 12printed in the source8 / 12noneexact matchPASS
n = 35: longest-run limit / fewest crossings8 / 12printed in the source8 / 12noneexact matchPASS
n = 36: longest-run limit / fewest crossings8 / 13printed in the source8 / 13noneexact matchPASS
n = 37: longest-run limit / fewest crossings8 / 13printed in the source8 / 13noneexact matchPASS
n = 38: longest-run limit / fewest crossings8 / 14printed in the source8 / 14noneexact matchPASS
n = 39: longest-run limit / fewest crossings8 / 14printed in the source8 / 14noneexact matchPASS
n = 40: longest-run limit / fewest crossings8 / 14printed in the source8 / 14noneexact matchPASS
n = 41: longest-run limit / fewest crossings8 / 15printed in the source8 / 15noneexact matchPASS
n = 42: longest-run limit / fewest crossings8 / 15printed in the source8 / 15noneexact matchPASS
n = 43: longest-run limit / fewest crossings8 / 16printed in the source8 / 16noneexact matchPASS
n = 44: longest-run limit / fewest crossings8 / 16printed in the source8 / 16noneexact matchPASS
n = 45: longest-run limit / fewest crossings8 / 17printed in the source8 / 17noneexact matchPASS
n = 46: longest-run limit / fewest crossings9 / 17printed in the source9 / 17noneexact matchPASS
n = 47: longest-run limit / fewest crossings9 / 17printed in the source9 / 17noneexact matchPASS
n = 48: longest-run limit / fewest crossings9 / 18printed in the source9 / 18noneexact matchPASS
n = 49: longest-run limit / fewest crossings9 / 18printed in the source9 / 18noneexact matchPASS
n = 50: longest-run limit / fewest crossings9 / 19printed in the source9 / 19noneexact matchPASS
n = 51: longest-run limit / fewest crossings9 / 19printed in the source9 / 19noneexact matchPASS
n = 52: longest-run limit / fewest crossings9 / 20printed in the source9 / 20noneexact matchPASS
n = 53: longest-run limit / fewest crossings9 / 20printed in the source9 / 20noneexact matchPASS
n = 54: longest-run limit / fewest crossings9 / 21printed in the source9 / 21noneexact matchPASS
n = 55: longest-run limit / fewest crossings9 / 21printed in the source9 / 21noneexact matchPASS
n = 56: longest-run limit / fewest crossings9 / 21printed in the source9 / 21noneexact matchPASS
n = 57: longest-run limit / fewest crossings9 / 22printed in the source9 / 22noneexact matchPASS
n = 58: longest-run limit / fewest crossings9 / 22printed in the source9 / 22noneexact matchPASS
n = 59: longest-run limit / fewest crossings9 / 23printed in the source9 / 23noneexact matchPASS
n = 60: longest-run limit / fewest crossings9 / 23printed in the source9 / 23noneexact matchPASS
n = 61: longest-run limit / fewest crossings9 / 24printed in the source9 / 24noneexact matchPASS
n = 62: longest-run limit / fewest crossings9 / 24printed in the source9 / 24noneexact matchPASS
n = 63: longest-run limit / fewest crossings9 / 25printed in the source9 / 25noneexact matchPASS
n = 64: longest-run limit / fewest crossings9 / 25printed in the source9 / 25noneexact matchPASS
n = 65: longest-run limit / fewest crossings9 / 25printed in the source9 / 25noneexact matchPASS
n = 66: longest-run limit / fewest crossings9 / 26printed in the source9 / 26noneexact matchPASS
n = 67: longest-run limit / fewest crossings9 / 26printed in the source9 / 26noneexact matchPASS
n = 68: longest-run limit / fewest crossings9 / 27printed in the source9 / 27noneexact matchPASS
n = 69: longest-run limit / fewest crossings9 / 27printed in the source9 / 27noneexact matchPASS
n = 70: longest-run limit / fewest crossings9 / 28printed in the source9 / 28noneexact matchPASS
n = 71: longest-run limit / fewest crossings9 / 28printed in the source9 / 28noneexact matchPASS
n = 72: longest-run limit / fewest crossings9 / 29printed in the source9 / 29noneexact matchPASS
n = 73: longest-run limit / fewest crossings9 / 29printed in the source9 / 29noneexact matchPASS
n = 74: longest-run limit / fewest crossings9 / 29printed in the source9 / 29noneexact matchPASS
n = 75: longest-run limit / fewest crossings9 / 30printed in the source9 / 30noneexact matchPASS
n = 76: longest-run limit / fewest crossings9 / 30printed in the source9 / 30noneexact matchPASS
n = 77: longest-run limit / fewest crossings9 / 31printed in the source9 / 31noneexact matchPASS
n = 78: longest-run limit / fewest crossings9 / 31printed in the source9 / 31noneexact matchPASS
n = 79: longest-run limit / fewest crossings9 / 32printed in the source9 / 32noneexact matchPASS
n = 80: longest-run limit / fewest crossings9 / 32printed in the source9 / 32noneexact matchPASS
n = 81: longest-run limit / fewest crossings9 / 33printed in the source9 / 33noneexact matchPASS
n = 82: longest-run limit / fewest crossings9 / 33printed in the source9 / 33noneexact matchPASS
n = 83: longest-run limit / fewest crossings9 / 34printed in the source9 / 34noneexact matchPASS
n = 84: longest-run limit / fewest crossings9 / 34printed in the source9 / 34noneexact matchPASS
n = 85: longest-run limit / fewest crossings9 / 34printed in the source9 / 34noneexact matchPASS
n = 86: longest-run limit / fewest crossings9 / 35printed in the source9 / 35noneexact matchPASS
n = 87: longest-run limit / fewest crossings9 / 35printed in the source9 / 35noneexact matchPASS
n = 88: longest-run limit / fewest crossings9 / 36printed in the source9 / 36noneexact matchPASS
n = 89: longest-run limit / fewest crossings9 / 36printed in the source9 / 36noneexact matchPASS
n = 90: longest-run limit / fewest crossings9 / 37printed in the source9 / 37noneexact matchPASS
n = 91: longest-run limit / fewest crossings10 / 37printed in the source10 / 37noneexact matchPASS
n = 92: longest-run limit / fewest crossings10 / 38printed in the source10 / 38noneexact matchPASS
n = 93: longest-run limit / fewest crossings10 / 38printed in the source10 / 38noneexact matchPASS
n = 94: longest-run limit / fewest crossings10 / 39printed in the source10 / 39noneexact matchPASS
n = 95: longest-run limit / fewest crossings10 / 39printed in the source10 / 39noneexact matchPASS
n = 96: longest-run limit / fewest crossings10 / 39printed in the source10 / 39noneexact matchPASS
n = 97: longest-run limit / fewest crossings10 / 40printed in the source10 / 40noneexact matchPASS
n = 98: longest-run limit / fewest crossings10 / 40printed in the source10 / 40noneexact matchPASS
n = 99: longest-run limit / fewest crossings10 / 41printed in the source10 / 41noneexact matchPASS
n = 100: longest-run limit / fewest crossings10 / 41printed in the source10 / 41noneexact matchPASS

Time-Weighted Charts (EWMA, CUSUM) 65/65

Charts that accumulate evidence over time to catch small, sustained shifts sooner than a Shewhart chart.

NIST EWMA Chart: 20 Points, λ = 0.3

23/23 pass

NIST uses the long-run (asymptotic) limits. LimitLine uses the exact limits, which start narrower and widen to the same values within a few points; the last point’s limits are compared.

Data: nist-ewma.csv

Show All 23 Checks
StatisticExpectedLimitLineDifferenceToleranceResult
EWMA at point 150.6printed in the source50.6→ 50.600same at 2 decimalsPASS
EWMA at point 249.52printed in the source49.52→ 49.520same at 2 decimalsPASS
EWMA at point 350.56printed in the source50.564→ 50.564.0e−3same at 2 decimalsPASS
EWMA at point 450.18printed in the source50.1848→ 50.184.8e−3same at 2 decimalsPASS
EWMA at point 550.16printed in the source50.15936→ 50.16−6.4e−4same at 2 decimalsPASS
EWMA at point 649.21printed in the source49.211552→ 49.211.6e−3same at 2 decimalsPASS
EWMA at point 749.75printed in the source49.7480864→ 49.75−1.9e−3same at 2 decimalsPASS
EWMA at point 849.85printed in the source49.85366048→ 49.853.7e−3same at 2 decimalsPASS
EWMA at point 950.26printed in the source50.257562336→ 50.26−2.4e−3same at 2 decimalsPASS
EWMA at point 1050.33printed in the source50.3302936352→ 50.332.9e−4same at 2 decimalsPASS
EWMA at point 1150.11printed in the source50.11120554464→ 50.111.2e−3same at 2 decimalsPASS
EWMA at point 1249.36printed in the source49.35784388125→ 49.36−2.2e−3same at 2 decimalsPASS
EWMA at point 1349.52printed in the source49.52049071687→ 49.524.9e−4same at 2 decimalsPASS
EWMA at point 1450.05printed in the source50.05434350181→ 50.054.3e−3same at 2 decimalsPASS
EWMA at point 1549.38printed in the source49.37804045127→ 49.38−2.0e−3same at 2 decimalsPASS
EWMA at point 1649.92printed in the source49.92462831589→ 49.924.6e−3same at 2 decimalsPASS
EWMA at point 1750.73printed in the source50.72723982112→ 50.73−2.8e−3same at 2 decimalsPASS
EWMA at point 1851.23printed in the source51.22906787478→ 51.23−9.3e−4same at 2 decimalsPASS
EWMA at point 1951.94printed in the source51.94034751235→ 51.943.5e−4same at 2 decimalsPASS
EWMA at point 2051.99printed in the source51.98824325864→ 51.99−1.8e−3same at 2 decimalsPASS
UCL (long-run)52.5884printed in the source52.588430914123.1e−5± 2e−4PASS
LCL (long-run)47.4115printed in the source47.411569085886.9e−5± 2e−4PASS
Points beyond the limits0printed in the source0noneexact matchPASS

NIST Tabular CUSUM: 20 Group Means, Target 325

42/42 pass

NIST gives k and h in data units (0.3175 and 4.1959, with σ of the means 0.635); LimitLine takes them in σ units, so k = 0.5 and h = 4.1959 / 0.635. NIST rounds the inputs to 2 decimals before adding, so its last sums can differ by 0.01.

Data: nist-cusum.csv

Show All 42 Checks
StatisticExpectedLimitLineDifferenceToleranceResult
Upper sum S_hi at group 10printed in the source00± 0.011PASS
Upper sum S_hi at group 20printed in the source00± 0.011PASS
Upper sum S_hi at group 30printed in the source00± 0.011PASS
Upper sum S_hi at group 40printed in the source00± 0.011PASS
Upper sum S_hi at group 50.03printed in the source0.032500000000022.5e−3± 0.011PASS
Upper sum S_hi at group 60printed in the source00± 0.011PASS
Upper sum S_hi at group 70printed in the source00± 0.011PASS
Upper sum S_hi at group 80printed in the source00± 0.011PASS
Upper sum S_hi at group 90printed in the source00± 0.011PASS
Upper sum S_hi at group 100printed in the source00± 0.011PASS
Upper sum S_hi at group 110printed in the source00± 0.011PASS
Upper sum S_hi at group 120printed in the source00± 0.011PASS
Upper sum S_hi at group 133.01printed in the source3.0075−2.5e−3± 0.011PASS
Upper sum S_hi at group 144.94printed in the source4.94−1.2e−14± 0.011PASS
Upper sum S_hi at group 157.45printed in the source7.4475−2.5e−3± 0.011PASS
Upper sum S_hi at group 1610.63printed in the source10.63−2.5e−14± 0.011PASS
Upper sum S_hi at group 1711.99printed in the source11.9875−2.5e−3± 0.011PASS
Upper sum S_hi at group 1814.44printed in the source14.4455.0e−3± 0.011PASS
Upper sum S_hi at group 1916printed in the source16.00252.5e−3± 0.011PASS
Upper sum S_hi at group 2019.04printed in the source19.035−5.0e−3± 0.011PASS
Lower sum S_lo at group 10printed in the source00± 0.011PASS
Lower sum S_lo at group 20.01printed in the source0.007499999999989−2.5e−3± 0.011PASS
Lower sum S_lo at group 30printed in the source00± 0.011PASS
Lower sum S_lo at group 40.33printed in the source0.33252.5e−3± 0.011PASS
Lower sum S_lo at group 50printed in the source00± 0.011PASS
Lower sum S_lo at group 60printed in the source00± 0.011PASS
Lower sum S_lo at group 70.56printed in the source0.5575−2.5e−3± 0.011PASS
Lower sum S_lo at group 80.72printed in the source0.715−5.0e−3± 0.011PASS
Lower sum S_lo at group 90.17printed in the source0.17252.5e−3± 0.011PASS
Lower sum S_lo at group 100.25printed in the source0.2555.0e−3± 0.011PASS
Lower sum S_lo at group 110.31printed in the source0.31252.5e−3± 0.011PASS
Lower sum S_lo at group 120printed in the source00± 0.011PASS
Lower sum S_lo at group 130printed in the source00± 0.011PASS
Lower sum S_lo at group 140printed in the source00± 0.011PASS
Lower sum S_lo at group 150printed in the source00± 0.011PASS
Lower sum S_lo at group 160printed in the source00± 0.011PASS
Lower sum S_lo at group 170printed in the source00± 0.011PASS
Lower sum S_lo at group 180printed in the source00± 0.011PASS
Lower sum S_lo at group 190printed in the source00± 0.011PASS
Lower sum S_lo at group 200printed in the source00± 0.011PASS
First signal (group)14printed in the source14noneexact matchPASS
Groups signaling7printed in the source7noneexact matchPASS

Lean Tools 7/7

Published worked examples for OEE, takt time and heijunka.

OEE Worked Example

4/4 pass
StatisticExpectedLimitLineDifferenceToleranceResult
Availability0.8881printed in the source0.8880952380952−4.8e−6± 5e−5PASS
Performance0.8611printed in the source0.8610813226095−1.9e−5± 5e−5PASS
Quality0.978printed in the source0.97804991956835.0e−5± 5e−5PASS
OEE0.7479printed in the source0.74793650793653.7e−5± 5e−5PASS

Takt Time

1/1 pass
StatisticExpectedLimitLineDifferenceToleranceResult
Takt time, seconds (480 min ÷ 240 units)120printed in the source1200± 1e−12PASS

Heijunka: Leveling by Volume

2/2 pass
StatisticExpectedLimitLineDifferenceToleranceResult
Level rate per day100printed in the source1000± 1e−12PASS
Buffer needed to cover Monday100computed here from the source’s data and formula1000± 1e−12PASS

Power and Sample Size 2/2

How many units a t test needs to detect a given difference: the classic published table.

Cohen's Table: Two-Sample T, Power 0.80, α 0.05 Two-Sided

2/2 pass

Cohen’s medium and large effects (d = 0.5, 0.8). For the small effect (d = 0.2), Cohen’s table, built on an approximation, prints 393; the exact noncentral t (as in statsmodels and G*Power) gives 394, which LimitLine also gives, so that row isn't counted here.

StatisticExpectedLimitLineDifferenceToleranceResult
d = 0.5: n per group64printed in the source64noneexact matchPASS
d = 0.8: n per group26printed in the source26noneexact matchPASS

Cross-Checks Against Independent Statistical Software 484/484

These are not published examples. ref/ref_gen.py ran SciPy and statsmodels on seeded random data and stored their answers in ref/refs.json; LimitLine must reproduce them. They test the same calculations by a second, independent implementation, across many more inputs than textbook examples cover.

Probability Distributions

92/92 pass

t, F and chi-square tail areas, the normal CDF, t quantiles, and log-gamma (Python math.lgamma). The p-values for every test in LimitLine come from these functions.

Show All 92 Checks
StatisticExpectedLimitLineDifferenceToleranceResult
t CDF (t = -3.2, df = 1)0.09641124797923independent software output0.09641124797923−5.6e−17relative ± 1e−10PASS
t CDF (t = -3.2, df = 3)0.02466592148135independent software output0.024665921481353.8e−17relative ± 1e−10PASS
t CDF (t = -3.2, df = 9)0.005415651294951independent software output0.005415651294951−8.7e−19relative ± 1e−10PASS
t CDF (t = -3.2, df = 30)0.001619300855977independent software output0.0016193008559779.3e−18relative ± 1e−10PASS
t CDF (t = -3.2, df = 200)0.0007991768707151independent software output0.000799176870715−4.1e−17relative ± 1e−10PASS
t CDF (t = -1.1, df = 1)0.2348538278116independent software output0.2348538278116−2.8e−17relative ± 1e−10PASS
t CDF (t = -1.1, df = 3)0.1758415974673independent software output0.1758415974673−2.2e−16relative ± 1e−10PASS
t CDF (t = -1.1, df = 9)0.1499413809284independent software output0.14994138092843.3e−16relative ± 1e−10PASS
t CDF (t = -1.1, df = 30)0.1400404590438independent software output0.1400404590438−2.1e−15relative ± 1e−10PASS
t CDF (t = -1.1, df = 200)0.1363271860987independent software output0.13632718609871.6e−14relative ± 1e−10PASS
t CDF (t = 0.4, df = 1)0.6211189415908independent software output0.62111894159082.2e−16relative ± 1e−10PASS
t CDF (t = 0.4, df = 3)0.6420324230128independent software output0.64203242301281.1e−16relative ± 1e−10PASS
t CDF (t = 0.4, df = 9)0.6507579541643independent software output0.6507579541643−1.1e−16relative ± 1e−10PASS
t CDF (t = 0.4, df = 30)0.6540047417429independent software output0.65400474174296.7e−16relative ± 1e−10PASS
t CDF (t = 0.4, df = 200)0.655208297837independent software output0.6552082978369−1.2e−14relative ± 1e−10PASS
t CDF (t = 2.5, df = 1)0.8788810584092independent software output0.87888105840920relative ± 1e−10PASS
t CDF (t = 2.5, df = 3)0.956146676496independent software output0.9561466764960relative ± 1e−10PASS
t CDF (t = 2.5, df = 9)0.9830690861585independent software output0.98306908615850relative ± 1e−10PASS
t CDF (t = 2.5, df = 30)0.990942175466independent software output0.9909421754660relative ± 1e−10PASS
t CDF (t = 2.5, df = 200)0.9933884136791independent software output0.99338841367913.3e−16relative ± 1e−10PASS
t CDF (t = 6, df = 1)0.9474315432887independent software output0.94743154328870relative ± 1e−10PASS
t CDF (t = 6, df = 3)0.9953636425539independent software output0.99536364255390relative ± 1e−10PASS
t CDF (t = 6, df = 9)0.999898750339independent software output0.9998987503390relative ± 1e−10PASS
t CDF (t = 6, df = 30)0.9999993028616independent software output0.99999930286160relative ± 1e−10PASS
t CDF (t = 6, df = 200)0.9999999954424independent software output0.99999999544240relative ± 1e−10PASS
F upper tail (F = 0.3, df 1, 5)0.6074354940759independent software output0.60743549407591.1e−16relative ± 1e−10PASS
F upper tail (F = 0.3, df 3, 12)0.8248035702258independent software output0.82480357022581.1e−16relative ± 1e−10PASS
F upper tail (F = 0.3, df 6, 9)0.9217416608607independent software output0.9217416608607−1.1e−16relative ± 1e−10PASS
F upper tail (F = 0.3, df 20, 40)0.9973296186574independent software output0.99732961865740relative ± 1e−10PASS
F upper tail (F = 1.2, df 1, 5)0.3232601902798independent software output0.32326019027985.0e−16relative ± 1e−10PASS
F upper tail (F = 1.2, df 3, 12)0.351512640724independent software output0.3515126407241.6e−15relative ± 1e−10PASS
F upper tail (F = 1.2, df 6, 9)0.3865692436842independent software output0.3865692436842−5.6e−17relative ± 1e−10PASS
F upper tail (F = 1.2, df 20, 40)0.3035030346267independent software output0.3035030346267−4.4e−15relative ± 1e−10PASS
F upper tail (F = 4.5, df 1, 5)0.08735930812736independent software output0.08735930812736−5.6e−17relative ± 1e−10PASS
F upper tail (F = 4.5, df 3, 12)0.02457347884095independent software output0.02457347884095−7.3e−17relative ± 1e−10PASS
F upper tail (F = 4.5, df 6, 9)0.02214050292969independent software output0.02214050292969−2.1e−17relative ± 1e−10PASS
F upper tail (F = 4.5, df 20, 40)2.5950300463e−5independent software output2.5950300463e−5−2.3e−19relative ± 1e−10PASS
F upper tail (F = 30, df 1, 5)0.002764960301305independent software output0.002764960301305−5.2e−18relative ± 1e−10PASS
F upper tail (F = 30, df 3, 12)7.3730324758e−6independent software output7.3730324758e−6−1.2e−20relative ± 1e−10PASS
F upper tail (F = 30, df 6, 9)1.8525315594e−5independent software output1.8525315594e−5−5.8e−20relative ± 1e−10PASS
F upper tail (F = 30, df 20, 40)4.7700820249e−18independent software output4.7700820249e−18−4.3e−32relative ± 1e−10PASS
Chi-square upper tail (x = 0.5, df = 1)0.479500122187independent software output0.479500122187−2.2e−16relative ± 1e−10PASS
Chi-square upper tail (x = 0.5, df = 2)0.7788007830714independent software output0.7788007830714−2.2e−16relative ± 1e−10PASS
Chi-square upper tail (x = 0.5, df = 5)0.9921232932326independent software output0.99212329323260relative ± 1e−10PASS
Chi-square upper tail (x = 0.5, df = 17)0.9999999999488independent software output0.99999999994880relative ± 1e−10PASS
Chi-square upper tail (x = 3, df = 1)0.08326451666355independent software output0.083264516663559.7e−17relative ± 1e−10PASS
Chi-square upper tail (x = 3, df = 2)0.2231301601484independent software output0.2231301601484−5.8e−16relative ± 1e−10PASS
Chi-square upper tail (x = 3, df = 5)0.6999858358786independent software output0.69998583587863.3e−16relative ± 1e−10PASS
Chi-square upper tail (x = 3, df = 17)0.9999304982629independent software output0.99993049826290relative ± 1e−10PASS
Chi-square upper tail (x = 11, df = 1)0.0009111188771537independent software output0.0009111188771537−8.7e−19relative ± 1e−10PASS
Chi-square upper tail (x = 11, df = 2)0.004086771438464independent software output0.0040867714384643.5e−18relative ± 1e−10PASS
Chi-square upper tail (x = 11, df = 5)0.05137998348307independent software output0.05137998348307−9.0e−17relative ± 1e−10PASS
Chi-square upper tail (x = 11, df = 17)0.8565639878403independent software output0.85656398784033.3e−16relative ± 1e−10PASS
Chi-square upper tail (x = 40, df = 1)2.5396285895e−10independent software output2.5396285895e−10−5.2e−26relative ± 1e−10PASS
Chi-square upper tail (x = 40, df = 2)2.0611536224e−9independent software output2.0611536224e−90relative ± 1e−10PASS
Chi-square upper tail (x = 40, df = 5)1.4933679001e−7independent software output1.4933679001e−7−2.1e−22relative ± 1e−10PASS
Chi-square upper tail (x = 40, df = 17)0.001294198533743independent software output0.001294198533743−4.3e−18relative ± 1e−10PASS
Normal CDF (z = -8)6.2209605743e−16independent software output6.2209605743e−169.4e−30relative ± 1e−12PASS
Normal CDF (z = -5)2.8665157188e−7independent software output2.8665157188e−71.2e−21relative ± 1e−12PASS
Normal CDF (z = -3)0.00134989803163independent software output0.001349898031633.3e−18relative ± 1e−12PASS
Normal CDF (z = -1.5)0.06680720126886independent software output0.06680720126886−2.6e−16relative ± 1e−12PASS
Normal CDF (z = 0)0.5independent software output0.50relative ± 1e−12PASS
Normal CDF (z = 0.7)0.7580363477769independent software output0.75803634777691.1e−16relative ± 1e−12PASS
Normal CDF (z = 2.2)0.9860965524865independent software output0.98609655248650relative ± 1e−12PASS
Normal CDF (z = 4)0.9999683287582independent software output0.99996832875820relative ± 1e−12PASS
Normal CDF (z = 6)0.9999999990134independent software output0.99999999901340relative ± 1e−12PASS
t quantile (p = 0.025, df = 1)−12.70620473617independent software output−12.706204736179.2e−14relative ± 1e−9PASS
t quantile (p = 0.025, df = 4)−2.776445105198independent software output−2.7764451051980relative ± 1e−9PASS
t quantile (p = 0.025, df = 15)−2.13144954556independent software output−2.131449545564.4e−14relative ± 1e−9PASS
t quantile (p = 0.025, df = 100)−1.983971518524independent software output−1.9839715185237.7e−14relative ± 1e−9PASS
t quantile (p = 0.1, df = 1)−3.077683537175independent software output−3.0776835371751.1e−13relative ± 1e−9PASS
t quantile (p = 0.1, df = 4)−1.533206274059independent software output−1.533206274059−2.6e−14relative ± 1e−9PASS
t quantile (p = 0.1, df = 15)−1.34060560785independent software output−1.34060560785−7.8e−15relative ± 1e−9PASS
t quantile (p = 0.1, df = 100)−1.290074761347independent software output−1.2900747613464.2e−14relative ± 1e−9PASS
t quantile (p = 0.9, df = 1)3.077683537175independent software output3.077683537175−1.1e−13relative ± 1e−9PASS
t quantile (p = 0.9, df = 4)1.533206274059independent software output1.5332062740592.5e−14relative ± 1e−9PASS
t quantile (p = 0.9, df = 15)1.34060560785independent software output1.340605607857.5e−15relative ± 1e−9PASS
t quantile (p = 0.9, df = 100)1.290074761347independent software output1.290074761346−4.2e−14relative ± 1e−9PASS
t quantile (p = 0.975, df = 1)12.70620473617independent software output12.70620473617−8.2e−14relative ± 1e−9PASS
t quantile (p = 0.975, df = 4)2.776445105198independent software output2.7764451051988.9e−16relative ± 1e−9PASS
t quantile (p = 0.975, df = 15)2.13144954556independent software output2.13144954556−4.4e−14relative ± 1e−9PASS
t quantile (p = 0.975, df = 100)1.983971518524independent software output1.983971518523−7.7e−14relative ± 1e−9PASS
t quantile (p = 0.995, df = 1)63.65674116287independent software output63.65674116287−2.0e−13relative ± 1e−9PASS
t quantile (p = 0.995, df = 4)4.60409487135independent software output4.60409487135−1.1e−13relative ± 1e−9PASS
t quantile (p = 0.995, df = 15)2.946712883475independent software output2.946712883475−1.4e−13relative ± 1e−9PASS
t quantile (p = 0.995, df = 100)2.625890521438independent software output2.625890521438−5.1e−14relative ± 1e−9PASS
log-gamma (0.1)2.252712651734independent software output2.2527126517344.4e−16± 1e−12PASS
log-gamma (0.5)0.5723649429247independent software output0.5723649429247−8.9e−16± 1e−12PASS
log-gamma (1)0independent software output−8.881784197e−16−8.9e−16± 1e−12PASS
log-gamma (2.5)0.2846828704729independent software output0.28468287047291.3e−15± 1e−12PASS
log-gamma (10)12.80182748008independent software output12.801827480087.1e−15± 1e−12PASS
log-gamma (171.3)708.114947039independent software output708.114947039−2.3e−13± 1e−12PASS

Descriptive Statistics (37 Values)

7/7 pass

Quartiles use the (n + 1)p method that Minitab uses (numpy percentile method “weibull”). The mean and standard deviation feed every tool; skewness, kurtosis and quartiles come from the same module but the app doesn’t show them today.

Data: refs-describe.csv

StatisticExpectedLimitLineDifferenceToleranceResult
Mean5.681897368676independent software output5.6818973686760relative ± 1e−13PASS
Standard deviation4.471880036468independent software output4.4718800364680relative ± 1e−13PASS
Skewness (adjusted)1.207487634087independent software output1.207487634087−4.4e−16relative ± 1e−12PASS
Kurtosis (adjusted)1.292806393565independent software output1.292806393565−1.8e−15relative ± 1e−12PASS
Q12.197822189793independent software output2.1978221897930relative ± 1e−13PASS
Median5.170722741533independent software output5.1707227415330relative ± 1e−13PASS
Q37.449728222047independent software output7.4497282220470relative ± 1e−13PASS

Anderson-Darling Normality Test (4 Samples)

8/8 pass

Data: refs-anderson-darling.csv

StatisticExpectedLimitLineDifferenceToleranceResult
Sample 1 (n = 25): AD statistic0.2022517255367independent software output0.20225172553670relative ± 1e−10PASS
Sample 1: p-value0.8634339963817independent software output0.86343399638170± 1e−9PASS
Sample 2 (n = 40): AD statistic0.7966172718201independent software output0.7966172718201−7.1e−15relative ± 1e−10PASS
Sample 2: p-value0.03566547290245independent software output0.035665472902461.5e−15± 1e−9PASS
Sample 3 (n = 200): AD statistic0.4711891733073independent software output0.47118917330732.8e−14relative ± 1e−10PASS
Sample 3: p-value0.2429460060544independent software output0.2429460060544−3.9e−14± 1e−9PASS
Sample 4 (n = 15): AD statistic0.5809110602341independent software output0.5809110602341−3.6e−15relative ± 1e−10PASS
Sample 4: p-value0.1091912875371independent software output0.10919128753712.3e−15± 1e−9PASS

Hypothesis Tests: T Tests, ANOVA, Levene, Chi-Square, Two Proportions

25/25 pass

Data: refs-t-tests.csv · refs-anova-groups.csv

Show All 25 Checks
StatisticExpectedLimitLineDifferenceToleranceResult
1-sample t (μ0 = 9.5): t0.01935998176627independent software output0.019359981766270relative ± 1e−12PASS
1-sample t: p0.984779347686independent software output0.984779347686−2.6e−14relative ± 1e−10PASS
1-sample t: 95% CI low8.601612726835independent software output8.6016127268351.1e−14relative ± 1e−11PASS
1-sample t: 95% CI high10.41502743645independent software output10.41502743645−1.1e−14relative ± 1e−11PASS
1-sample t: p (one-sided, greater)0.492389673843independent software output0.492389673843−1.3e−14relative ± 1e−10PASS
Welch 2-sample t: t−0.8824736934754independent software output−0.8824736934754−1.1e−16relative ± 1e−12PASS
Welch: p0.3838189220058independent software output0.3838189220058−5.0e−16relative ± 1e−10PASS
Welch: df33.44907190079independent software output33.449071900797.1e−15relative ± 1e−12PASS
Welch: 95% CI low−2.591163598292independent software output−2.5911635982922.6e−14relative ± 1e−10PASS
Welch: 95% CI high1.02280181955independent software output1.02280181955−2.6e−14relative ± 1e−10PASS
Pooled 2-sample t: t−0.8172053632849independent software output−0.8172053632849−1.1e−16relative ± 1e−12PASS
Pooled: p0.4187755041075independent software output0.4187755041075−1.5e−15relative ± 1e−10PASS
Pooled: 95% CI low−2.72513188634independent software output−2.72513188634−4.4e−15relative ± 1e−10PASS
Paired t: t−2.976068678971independent software output−2.9760686789714.4e−16relative ± 1e−12PASS
Paired t: p0.008476036680099independent software output0.0084760366800990relative ± 1e−10PASS
Paired t: 95% CI high−0.2006319767431independent software output−0.2006319767431−6.3e−15relative ± 1e−10PASS
One-way ANOVA: F4.764705737524independent software output4.764705737524−1.8e−15relative ± 1e−12PASS
One-way ANOVA: p0.006885553282113independent software output0.006885553282113−4.7e−17relative ± 1e−10PASS
Levene (median): W0.6922417635841independent software output0.6922417635841−1.1e−16relative ± 1e−12PASS
Levene: p0.5629570705917independent software output0.56295707059174.4e−15relative ± 1e−10PASS
Chi-square, 3×3 table [[12,30,18],[20,25,35],[8,14,20]]: χ²6.468289325657independent software output6.468289325657−8.9e−16relative ± 1e−12PASS
Chi-square: p0.1667994104129independent software output0.16679941041290relative ± 1e−10PASS
Chi-square: df4independent software output4noneexact matchPASS
2 proportions, 45/300 vs 30/280 (pooled): z1.537094376756independent software output1.5370943767560relative ± 1e−12PASS
2 proportions: p0.1242701998111independent software output0.1242701998111−2.5e−16relative ± 1e−10PASS

Multiple Regression: 3 Predictors, 30 Observations

30/30 pass

Data: refs-regression.csv

Show All 30 Checks
StatisticExpectedLimitLineDifferenceToleranceResult
Coefficient, intercept1.874689665505independent software output1.874689665505−1.8e−15relative ± 1e−11PASS
Coefficient, x11.361563143009independent software output1.361563143009−2.2e−16relative ± 1e−11PASS
Coefficient, x2−0.8739299498415independent software output−0.8739299498415−4.4e−16relative ± 1e−11PASS
Coefficient, x30.1496412458402independent software output0.14964124584022.5e−16relative ± 1e−11PASS
Standard error, intercept0.1689927843108independent software output0.1689927843108−5.6e−17relative ± 1e−10PASS
Standard error, x10.1769743853737independent software output0.1769743853737−2.8e−17relative ± 1e−10PASS
Standard error, x20.2010811903905independent software output0.20108119039052.8e−17relative ± 1e−10PASS
Standard error, x30.2230514708441independent software output0.2230514708441−5.6e−17relative ± 1e−10PASS
p-value, intercept2.3419510988e−11independent software output2.3419510988e−112.1e−25± 1e−8PASS
p-value, x13.6501351758e−8independent software output3.6501351758e−86.6e−24± 1e−8PASS
p-value, x20.0001886853370603independent software output0.0001886853370603−1.2e−18± 1e−8PASS
p-value, x30.5082121352493independent software output0.50821213524930± 1e−8PASS
Adjusted R²0.8107432932159independent software output0.81074329321590relative ± 1e−11PASS
Model p (F test)3.7114932556e−10independent software output3.7114932556e−104.7e−24± 1e−8PASS
Leverage, row 10.1062954486162independent software output0.1062954486162−6.9e−17relative ± 1e−10PASS
Leverage, row 20.1553432805676independent software output0.15534328056762.8e−17relative ± 1e−10PASS
Leverage, row 30.06425586121756independent software output0.06425586121756−2.8e−17relative ± 1e−10PASS
Leverage, row 40.1440919363799independent software output0.1440919363799−5.6e−17relative ± 1e−10PASS
Leverage, row 50.2929633545587independent software output0.29296335455871.1e−16relative ± 1e−10PASS
Standardized residual, row 1−0.1175203952685independent software output−0.11752039526852.1e−15relative ± 1e−9PASS
Standardized residual, row 2−0.2228715870875independent software output−0.22287158708753.1e−15relative ± 1e−9PASS
Standardized residual, row 3−0.6462074454988independent software output−0.64620744549882.0e−15relative ± 1e−9PASS
Standardized residual, row 4−0.9651196491279independent software output−0.96511964912792.6e−15relative ± 1e−9PASS
Standardized residual, row 50.297067676452independent software output0.2970676764522.3e−15relative ± 1e−9PASS
VIF, x11.572894585264independent software output1.572894585264−2.2e−16relative ± 1e−9PASS
VIF, x21.026032907857independent software output1.0260329078574.4e−16relative ± 1e−9PASS
VIF, x31.604908810459independent software output1.6049088104590relative ± 1e−9PASS
Prediction at (0.5, −0.2, 0.1)2.745221351562independent software output2.745221351562−1.8e−15relative ± 1e−11PASS
95% confidence interval, low2.323258410863independent software output2.323258410863−1.1e−14relative ± 1e−10PASS
95% prediction interval, high4.651613048762independent software output4.6516130487624.3e−14relative ± 1e−10PASS

Non-Normal Data: Box-Cox, Lognormal and Weibull Fits

7/7 pass

Maximum-likelihood fits with the location fixed at 0, as with SciPy’s floc = 0. These drive LimitLine’s non-normal capability.

Data: refs-lognormal-60.csv · refs-weibull-80.csv

StatisticExpectedLimitLineDifferenceToleranceResult
Box-Cox λ (maximum likelihood)−0.07761589539388independent software output−0.077615866431762.9e−8± 1e−6PASS
Lognormal σ0.4395476313864independent software output0.43954763138645.6e−17relative ± 1e−7PASS
Lognormal μ0.972522082197independent software output0.972522082197−1.1e−16relative ± 1e−7PASS
Lognormal 99.865th percentile9.88634847448independent software output9.886348474487.1e−15relative ± 1e−6PASS
Weibull shape2.000148110365independent software output2.0001505280742.4e−6relative ± 1e−5PASS
Weibull scale11.79568421492independent software output11.795715014063.1e−5relative ± 1e−5PASS
Weibull 0.135th percentile0.433653560681independent software output0.43365642448972.9e−6relative ± 1e−4PASS

Gauge R&R Two-Way ANOVA (10 Parts × 3 Operators × 2 Trials)

4/4 pass

Data: refs-gage-rr-synthetic.csv

StatisticExpectedLimitLineDifferenceToleranceResult
SS part63.83469679353independent software output63.834696793537.1e−15relative ± 1e−10PASS
SS operator0.3980597608446independent software output0.39805976084479.5e−15relative ± 1e−10PASS
Interaction p-value (vs error)0.8499763059015independent software output0.8499763059015−2.7e−15± 1e−9PASS
Pooled repeatability SS = interaction SS + error SS0.8816023027272independent software output0.88160230272721.8e−15relative ± 1e−10PASS

Designed Experiment: 2³ Full Factorial, 2 Replicates, Full Model

14/14 pass

Data: refs-doe-2x3-replicated.csv

Show All 14 Checks
StatisticExpectedLimitLineDifferenceToleranceResult
Coefficient, A4.139202170081independent software output4.1392021700811.8e−15relative ± 1e−11PASS
p-value, A7.2262702212e−7independent software output7.2262702212e−7−3.0e−21± 1e−8PASS
Coefficient, B−2.607738056833independent software output−2.6077380568331.2e−14relative ± 1e−11PASS
p-value, B2.3519608957e−5independent software output2.3519608957e−55.4e−19± 1e−8PASS
Coefficient, C0.4531543108019independent software output0.45315431080196.1e−15relative ± 1e−11PASS
p-value, C0.1685042157022independent software output0.1685042157022−5.1e−15± 1e−8PASS
Coefficient, A×B1.597856365582independent software output1.597856365582−4.9e−15relative ± 1e−11PASS
p-value, A×B0.0006956174417653independent software output0.00069561744176531.4e−17± 1e−8PASS
Coefficient, A×C−0.04963003276434independent software output−0.049630032764342.0e−15relative ± 1e−11PASS
p-value, A×C0.8724222505913independent software output0.87242225059139.9e−15± 1e−8PASS
Coefficient, B×C0.1982409079729independent software output0.1982409079729−1.5e−15relative ± 1e−11PASS
p-value, B×C0.5263965727899independent software output0.5263965727899−1.9e−15± 1e−8PASS
Coefficient, A×B×C0.1543435210803independent software output0.15434352108033.7e−15relative ± 1e−11PASS
p-value, A×B×C0.6200488147326independent software output0.6200488147326−7.1e−15± 1e−8PASS

Power and Sample Size (297 Values)

297/297 pass

Noncentral t and F distributions, power for t tests, two proportions and one-way ANOVA, and the sample size each needs (statsmodels’ fractional answer rounded up).

Show All 297 Checks
StatisticExpectedLimitLineDifferenceToleranceResult
Noncentral t CDF (t -2.5, df 3, δ -1)0.1758162407995independent software output0.17581624079953.1e−16± 1e−10PASS
Noncentral t CDF (t -2.5, df 3, δ 0)0.04385332350403independent software output0.043853323504033.5e−17± 1e−10PASS
Noncentral t CDF (t -2.5, df 3, δ 0.7)0.01155634567409independent software output0.01155634567409−3.4e−16± 1e−10PASS
Noncentral t CDF (t -2.5, df 3, δ 2.4)0.000105879242372independent software output0.0001058792423716−4.0e−16± 1e−10PASS
Noncentral t CDF (t -2.5, df 3, δ 5)8.3459984415e−10independent software output8.3459983369e−10−1.0e−17± 1e−10PASS
Noncentral t CDF (t -2.5, df 9, δ -1)0.1073024951327independent software output0.10730249513275.1e−16± 1e−10PASS
Noncentral t CDF (t -2.5, df 9, δ 0)0.01693091384149independent software output0.01693091384149−5.6e−17± 1e−10PASS
Noncentral t CDF (t -2.5, df 9, δ 0.7)0.003042125037528independent software output0.003042125037528−2.4e−16± 1e−10PASS
Noncentral t CDF (t -2.5, df 9, δ 2.4)9.8036877333e−6independent software output9.803687733e−6−3.6e−16± 1e−10PASS
Noncentral t CDF (t -2.5, df 9, δ 5)1.5059179745e−11independent software output1.5059176128e−11−3.6e−18± 1e−10PASS
Noncentral t CDF (t -2.5, df 29, δ -1)0.079855116843independent software output0.0798551168434.0e−16± 1e−10PASS
Noncentral t CDF (t -2.5, df 29, δ 0)0.009162672169213independent software output0.009162672169213−8.3e−17± 1e−10PASS
Noncentral t CDF (t -2.5, df 29, δ 0.7)0.001235966284947independent software output0.001235966284947−3.4e−16± 1e−10PASS
Noncentral t CDF (t -2.5, df 29, δ 2.4)1.6517464693e−6independent software output1.6517464692e−6−1.0e−16± 1e−10PASS
Noncentral t CDF (t -2.5, df 29, δ 5)4.6043098998e−13independent software output4.6040948831e−13−2.2e−17± 1e−10PASS
Noncentral t CDF (t -2.5, df 118, δ -1)0.07005319082556independent software output0.070053190825568.0e−16± 1e−10PASS
Noncentral t CDF (t -2.5, df 118, δ 0)0.006896433721524independent software output0.006896433721524−4.6e−16± 1e−10PASS
Noncentral t CDF (t -2.5, df 118, δ 0.7)0.0008060238929645independent software output0.0008060238929641−3.3e−16± 1e−10PASS
Noncentral t CDF (t -2.5, df 118, δ 2.4)6.7484084093e−7independent software output6.7484084054e−7−3.9e−16± 1e−10PASS
Noncentral t CDF (t -2.5, df 118, δ 5)6.8084030097e−14independent software output6.805667141e−14−2.7e−17± 1e−10PASS
Noncentral t CDF (t -0.3, df 3, δ -1)0.7638660923232independent software output0.76386609232321.1e−16± 1e−10PASS
Noncentral t CDF (t -0.3, df 3, δ 0)0.39188164602independent software output0.39188164602−5.6e−17± 1e−10PASS
Noncentral t CDF (t -0.3, df 3, δ 0.7)0.166069540438independent software output0.166069540438−1.4e−16± 1e−10PASS
Noncentral t CDF (t -0.3, df 3, δ 2.4)0.003917166098301independent software output0.003917166098301−1.0e−16± 1e−10PASS
Noncentral t CDF (t -0.3, df 3, δ 5)7.8532316493e−8independent software output7.8532316428e−8−6.6e−17± 1e−10PASS
Noncentral t CDF (t -0.3, df 9, δ -1)0.7600603480398independent software output0.76006034803983.3e−16± 1e−10PASS
Noncentral t CDF (t -0.3, df 9, δ 0)0.3854953518708independent software output0.38549535187081.7e−16± 1e−10PASS
Noncentral t CDF (t -0.3, df 9, δ 0.7)0.1612339539197independent software output0.16123395391971.7e−16± 1e−10PASS
Noncentral t CDF (t -0.3, df 9, δ 2.4)0.003622414571044independent software output0.003622414571044−6.0e−17± 1e−10PASS
Noncentral t CDF (t -0.3, df 9, δ 5)6.4811557174e−8independent software output6.4811557121e−8−5.3e−17± 1e−10PASS
Noncentral t CDF (t -0.3, df 29, δ -1)0.7586715083063independent software output0.75867150830633.3e−16± 1e−10PASS
Noncentral t CDF (t -0.3, df 29, δ 0)0.3831585466645independent software output0.38315854666451.1e−16± 1e−10PASS
Noncentral t CDF (t -0.3, df 29, δ 0.7)0.1594649776043independent software output0.15946497760432.5e−16± 1e−10PASS
Noncentral t CDF (t -0.3, df 29, δ 2.4)0.003515612456944independent software output0.003515612456944−9.5e−17± 1e−10PASS
Noncentral t CDF (t -0.3, df 29, δ 5)6.003394828e−8independent software output6.0033948257e−8−2.3e−17± 1e−10PASS
Noncentral t CDF (t -0.3, df 118, δ -1)0.7581929720233independent software output0.75819297202335.0e−15± 1e−10PASS
Noncentral t CDF (t -0.3, df 118, δ 0)0.3823524992731independent software output0.38235249927312.6e−15± 1e−10PASS
Noncentral t CDF (t -0.3, df 118, δ 0.7)0.1588549645863independent software output0.15885496458635.8e−15± 1e−10PASS
Noncentral t CDF (t -0.3, df 118, δ 2.4)0.003478954256963independent software output0.0034789542569633.6e−16± 1e−10PASS
Noncentral t CDF (t -0.3, df 118, δ 5)5.8423950463e−8independent software output5.8423950344e−8−1.2e−16± 1e−10PASS
Noncentral t CDF (t 0.8, df 3, δ -1)0.9517887357974independent software output0.9517887357974−1.1e−16± 1e−10PASS
Noncentral t CDF (t 0.8, df 3, δ 0)0.7589005241245independent software output0.7589005241245−1.1e−16± 1e−10PASS
Noncentral t CDF (t 0.8, df 3, δ 0.7)0.5132635944712independent software output0.5132635944712−3.3e−16± 1e−10PASS
Noncentral t CDF (t 0.8, df 3, δ 2.4)0.05651745358777independent software output0.056517453587772.1e−17± 1e−10PASS
Noncentral t CDF (t 0.8, df 3, δ 5)2.7001347972e−5independent software output2.7001347972e−5−1.6e−17± 1e−10PASS
Noncentral t CDF (t 0.8, df 9, δ -1)0.9598318663862independent software output0.9598318663862−1.1e−16± 1e−10PASS
Noncentral t CDF (t 0.8, df 9, δ 0)0.7778435007761independent software output0.7778435007761−3.3e−16± 1e−10PASS
Noncentral t CDF (t 0.8, df 9, δ 0.7)0.5305126123877independent software output0.5305126123877−2.2e−16± 1e−10PASS
Noncentral t CDF (t 0.8, df 9, δ 2.4)0.05544663997066independent software output0.055446639970664.2e−17± 1e−10PASS
Noncentral t CDF (t 0.8, df 9, δ 5)1.686566023e−5independent software output1.686566023e−5−3.0e−18± 1e−10PASS
Noncentral t CDF (t 0.8, df 29, δ -1)0.9627444843152independent software output0.9627444843152−6.7e−16± 1e−10PASS
Noncentral t CDF (t 0.8, df 29, δ 0)0.7848921657248independent software output0.7848921657248−6.7e−16± 1e−10PASS
Noncentral t CDF (t 0.8, df 29, δ 0.7)0.5368910355718independent software output0.5368910355718−8.9e−16± 1e−10PASS
Noncentral t CDF (t 0.8, df 29, δ 2.4)0.05500923283976independent software output0.05500923283976−1.2e−16± 1e−10PASS
Noncentral t CDF (t 0.8, df 29, δ 5)1.4344512887e−5independent software output1.4344512887e−5−1.3e−18± 1e−10PASS
Noncentral t CDF (t 0.8, df 118, δ -1)0.9637433598813independent software output0.9637433598813−1.1e−14± 1e−10PASS
Noncentral t CDF (t 0.8, df 118, δ 0)0.7873407973806independent software output0.7873407973806−1.8e−14± 1e−10PASS
Noncentral t CDF (t 0.8, df 118, δ 0.7)0.5391023962979independent software output0.5391023962979−1.7e−14± 1e−10PASS
Noncentral t CDF (t 0.8, df 118, δ 2.4)0.05485167280971independent software output0.05485167280971−2.2e−15± 1e−10PASS
Noncentral t CDF (t 0.8, df 118, δ 5)1.3583864124e−5independent software output1.3583864124e−5−6.3e−19± 1e−10PASS
Noncentral t CDF (t 2.1, df 3, δ -1)0.9910765400541independent software output0.99107654005413.3e−16± 1e−10PASS
Noncentral t CDF (t 2.1, df 3, δ 0)0.9367173987252independent software output0.93671739872520± 1e−10PASS
Noncentral t CDF (t 2.1, df 3, δ 0.7)0.8302897193734independent software output0.8302897193734−4.4e−16± 1e−10PASS
Noncentral t CDF (t 2.1, df 3, δ 2.4)0.3525372488881independent software output0.3525372488881−2.8e−16± 1e−10PASS
Noncentral t CDF (t 2.1, df 3, δ 5)0.01104486516829independent software output0.011044865168291.7e−18± 1e−10PASS
Noncentral t CDF (t 2.1, df 9, δ -1)0.9970931228641independent software output0.99709312286410± 1e−10PASS
Noncentral t CDF (t 2.1, df 9, δ 0)0.9674408587939independent software output0.9674408587939−1.1e−16± 1e−10PASS
Noncentral t CDF (t 2.1, df 9, δ 0.7)0.8865478853818independent software output0.8865478853818−3.3e−16± 1e−10PASS
Noncentral t CDF (t 2.1, df 9, δ 2.4)0.3728095957465independent software output0.37280959574651.1e−16± 1e−10PASS
Noncentral t CDF (t 2.1, df 9, δ 5)0.004178827465566independent software output0.004178827465566−6.9e−17± 1e−10PASS
Noncentral t CDF (t 2.1, df 29, δ -1)0.9985363596415independent software output0.9985363596415−1.1e−16± 1e−10PASS
Noncentral t CDF (t 2.1, df 29, δ 0)0.9777290209819independent software output0.97772902098190± 1e−10PASS
Noncentral t CDF (t 2.1, df 29, δ 0.7)0.908732843167independent software output0.908732843167−5.6e−16± 1e−10PASS
Noncentral t CDF (t 2.1, df 29, δ 2.4)0.3794022612007independent software output0.3794022612007−9.4e−16± 1e−10PASS
Noncentral t CDF (t 2.1, df 29, δ 5)0.002469005914685independent software output0.002469005914684−4.1e−17± 1e−10PASS
Noncentral t CDF (t 2.1, df 118, δ -1)0.9989200598583independent software output0.9989200598583−5.6e−15± 1e−10PASS
Noncentral t CDF (t 2.1, df 118, δ 0)0.9810704301108independent software output0.98107043011081.2e−15± 1e−10PASS
Noncentral t CDF (t 2.1, df 118, δ 0.7)0.9166289541005independent software output0.9166289541005−3.6e−15± 1e−10PASS
Noncentral t CDF (t 2.1, df 118, δ 2.4)0.3814519750588independent software output0.3814519750588−1.5e−14± 1e−10PASS
Noncentral t CDF (t 2.1, df 118, δ 5)0.002003935692024independent software output0.002003935692024−1.4e−16± 1e−10PASS
Noncentral t CDF (t 4, df 3, δ -1)0.9982750423472independent software output0.99827504234722.2e−16± 1e−10PASS
Noncentral t CDF (t 4, df 3, δ 0)0.9859957719949independent software output0.98599577199490± 1e−10PASS
Noncentral t CDF (t 4, df 3, δ 0.7)0.9572378138883independent software output0.9572378138883−2.2e−16± 1e−10PASS
Noncentral t CDF (t 4, df 3, δ 2.4)0.7468669436674independent software output0.7468669436674−2.2e−16± 1e−10PASS
Noncentral t CDF (t 4, df 3, δ 5)0.2324260189244independent software output0.23242601892441.3e−15± 1e−10PASS
Noncentral t CDF (t 4, df 9, δ -1)0.999915978907independent software output0.9999159789071.1e−16± 1e−10PASS
Noncentral t CDF (t 4, df 9, δ 0)0.9984447858448independent software output0.99844478584480± 1e−10PASS
Noncentral t CDF (t 4, df 9, δ 0.7)0.9918691016604independent software output0.9918691016604−1.1e−16± 1e−10PASS
Noncentral t CDF (t 4, df 9, δ 2.4)0.8631451431995independent software output0.8631451431995−2.2e−16± 1e−10PASS
Noncentral t CDF (t 4, df 9, δ 5)0.2070182053794independent software output0.20701820537941.7e−16± 1e−10PASS
Noncentral t CDF (t 4, df 29, δ -1)0.999995550191independent software output0.9999955501912.2e−16± 1e−10PASS
Noncentral t CDF (t 4, df 29, δ 0)0.9997999680272independent software output0.99979996802720± 1e−10PASS
Noncentral t CDF (t 4, df 29, δ 0.7)0.9981964917791independent software output0.9981964917791−1.1e−16± 1e−10PASS
Noncentral t CDF (t 4, df 29, δ 2.4)0.9176597240056independent software output0.9176597240056−1.1e−16± 1e−10PASS
Noncentral t CDF (t 4, df 29, δ 5)0.1795931465461independent software output0.1795931465461−1.3e−15± 1e−10PASS
Noncentral t CDF (t 4, df 118, δ -1)0.9999993332526independent software output0.99999933325262.2e−16± 1e−10PASS
Noncentral t CDF (t 4, df 118, δ 0)0.9999445529635independent software output0.99994455296350± 1e−10PASS
Noncentral t CDF (t 4, df 118, δ 0.7)0.9992815568917independent software output0.99928155689170± 1e−10PASS
Noncentral t CDF (t 4, df 118, δ 2.4)0.9382836685405independent software output0.93828366854061.9e−15± 1e−10PASS
Noncentral t CDF (t 4, df 118, δ 5)0.1645322718996independent software output0.1645322718995−1.1e−14± 1e−10PASS
Noncentral F upper tail (F 0.6, df 1, 10, λ 0.5)0.5557156520335independent software output0.5557156520335−1.2e−15± 1e−10PASS
Noncentral F upper tail (F 0.6, df 1, 10, λ 4)0.8933063566298independent software output0.8933063566298−4.4e−16± 1e−10PASS
Noncentral F upper tail (F 0.6, df 1, 10, λ 12)0.9962086857743independent software output0.9962086857743−1.1e−16± 1e−10PASS
Noncentral F upper tail (F 0.6, df 2, 27, λ 0.5)0.6292714809071independent software output0.6292714809071−2.4e−15± 1e−10PASS
Noncentral F upper tail (F 0.6, df 2, 27, λ 4)0.8982329788686independent software output0.8982329788686−7.8e−16± 1e−10PASS
Noncentral F upper tail (F 0.6, df 2, 27, λ 12)0.9953805132912independent software output0.9953805132912−2.2e−16± 1e−10PASS
Noncentral F upper tail (F 0.6, df 4, 45, λ 0.5)0.713493575503independent software output0.713493575503−1.3e−15± 1e−10PASS
Noncentral F upper tail (F 0.6, df 4, 45, λ 4)0.9085833090931independent software output0.9085833090931−3.3e−16± 1e−10PASS
Noncentral F upper tail (F 0.6, df 4, 45, λ 12)0.9943746929302independent software output0.9943746929302−1.1e−16± 1e−10PASS
Noncentral F upper tail (F 2.2, df 1, 10, λ 0.5)0.261397607931independent software output0.261397607931−3.4e−15± 1e−10PASS
Noncentral F upper tail (F 2.2, df 1, 10, λ 4)0.7013170488309independent software output0.7013170488309−1.4e−15± 1e−10PASS
Noncentral F upper tail (F 2.2, df 1, 10, λ 12)0.9722017778063independent software output0.9722017778063−1.1e−16± 1e−10PASS
Noncentral F upper tail (F 2.2, df 2, 27, λ 0.5)0.1918285764068independent software output0.1918285764068−3.3e−16± 1e−10PASS
Noncentral F upper tail (F 2.2, df 2, 27, λ 4)0.5687761000603independent software output0.5687761000603−2.1e−15± 1e−10PASS
Noncentral F upper tail (F 2.2, df 2, 27, λ 12)0.9345411935053independent software output0.93454119350530± 1e−10PASS
Noncentral F upper tail (F 2.2, df 4, 45, λ 0.5)0.117159817498independent software output0.1171598174981.4e−16± 1e−10PASS
Noncentral F upper tail (F 2.2, df 4, 45, λ 4)0.3817239425998independent software output0.38172394259987.2e−16± 1e−10PASS
Noncentral F upper tail (F 2.2, df 4, 45, λ 12)0.8265584286474independent software output0.82655842864741.1e−15± 1e−10PASS
Noncentral F upper tail (F 4, df 1, 10, λ 0.5)0.1347068491045independent software output0.1347068491045−4.7e−16± 1e−10PASS
Noncentral F upper tail (F 4, df 1, 10, λ 4)0.5191599032642independent software output0.5191599032642−1.2e−15± 1e−10PASS
Noncentral F upper tail (F 4, df 1, 10, λ 12)0.9165416017982independent software output0.9165416017982−1.1e−16± 1e−10PASS
Noncentral F upper tail (F 4, df 2, 27, λ 0.5)0.05513084031793independent software output0.055130840317931.4e−16± 1e−10PASS
Noncentral F upper tail (F 4, df 2, 27, λ 4)0.2941833588881independent software output0.2941833588881−7.8e−16± 1e−10PASS
Noncentral F upper tail (F 4, df 2, 27, λ 12)0.778985645366independent software output0.7789856453663.3e−16± 1e−10PASS
Noncentral F upper tail (F 4, df 4, 45, λ 0.5)0.01289858350134independent software output0.012898583501349.0e−17± 1e−10PASS
Noncentral F upper tail (F 4, df 4, 45, λ 4)0.09380426953057independent software output0.09380426953057−7.5e−16± 1e−10PASS
Noncentral F upper tail (F 4, df 4, 45, λ 12)0.4626244427256independent software output0.46262444272561.1e−15± 1e−10PASS
Power, 1-sample t (d 0.2, n 5, two-sided)0.06426957404801independent software output0.064269574048011.1e−16± 1e−9PASS
Power, 1-sample t (d 0.2, n 5, larger)0.1024442765543independent software output0.10244427655436.7e−15± 1e−9PASS
Power, 1-sample t (d 0.2, n 10, two-sided)0.08765715402847independent software output0.08765715402846−6.7e−15± 1e−9PASS
Power, 1-sample t (d 0.2, n 10, larger)0.1448180678455independent software output0.1448180678455−9.8e−15± 1e−9PASS
Power, 1-sample t (d 0.2, n 20, two-sided)0.1359562887674independent software output0.13595628876741.3e−14± 1e−9PASS
Power, 1-sample t (d 0.2, n 20, larger)0.2170751816719independent software output0.2170751816719−6.9e−15± 1e−9PASS
Power, 1-sample t (d 0.2, n 50, two-sided)0.2835924146675independent software output0.28359241466752.6e−14± 1e−9PASS
Power, 1-sample t (d 0.2, n 50, larger)0.4012215049932independent software output0.4012215049932−2.3e−14± 1e−9PASS
Power, 1-sample t (d 0.2, n 120, two-sided)0.5844409144364independent software output0.5844409144364−2.0e−14± 1e−9PASS
Power, 1-sample t (d 0.2, n 120, larger)0.7031750281047independent software output0.70317502810486.8e−14± 1e−9PASS
Power, 1-sample t (d 0.5, n 5, two-sided)0.1405166899903independent software output0.1405166899903−2.8e−17± 1e−9PASS
Power, 1-sample t (d 0.5, n 5, larger)0.2389951709035independent software output0.23899517090351.3e−14± 1e−9PASS
Power, 1-sample t (d 0.5, n 10, two-sided)0.2931756065141independent software output0.2931756065141−1.5e−14± 1e−9PASS
Power, 1-sample t (d 0.5, n 10, larger)0.4272898267712independent software output0.4272898267712−1.8e−14± 1e−9PASS
Power, 1-sample t (d 0.5, n 20, two-sided)0.564504418439independent software output0.5645044184392.5e−14± 1e−9PASS
Power, 1-sample t (d 0.5, n 20, larger)0.6951493382443independent software output0.6951493382443−8.8e−15± 1e−9PASS
Power, 1-sample t (d 0.5, n 50, two-sided)0.9338975812614independent software output0.93389758126149.8e−15± 1e−9PASS
Power, 1-sample t (d 0.5, n 50, larger)0.9672067458262independent software output0.9672067458262−5.3e−15± 1e−9PASS
Power, 1-sample t (d 0.5, n 120, two-sided)0.9997422834288independent software output0.99974228342880± 1e−9PASS
Power, 1-sample t (d 0.5, n 120, larger)0.9999279233437independent software output0.99992792334371.1e−16± 1e−9PASS
Power, 1-sample t (d 0.8, n 5, two-sided)0.2809920635174independent software output0.28099206351741.1e−16± 1e−9PASS
Power, 1-sample t (d 0.8, n 5, larger)0.436359186401independent software output0.4363591864011.7e−14± 1e−9PASS
Power, 1-sample t (d 0.8, n 10, two-sided)0.6162327634744independent software output0.6162327634744−1.9e−14± 1e−9PASS
Power, 1-sample t (d 0.8, n 10, larger)0.7544247592493independent software output0.7544247592493−1.6e−14± 1e−9PASS
Power, 1-sample t (d 0.8, n 20, two-sided)0.9238987553886independent software output0.92389875538869.4e−15± 1e−9PASS
Power, 1-sample t (d 0.8, n 20, larger)0.964172800606independent software output0.964172800606−2.2e−15± 1e−9PASS
Power, 1-sample t (d 0.8, n 50, two-sided)0.9998302862283independent software output0.99983028622830± 1e−9PASS
Power, 1-sample t (d 0.8, n 50, larger)0.9999578749289independent software output0.99995787492890± 1e−9PASS
Power, 1-sample t (d 0.8, n 120, two-sided)0.9999999999916independent software output0.99999999999160± 1e−9PASS
Power, 1-sample t (d 0.8, n 120, larger)0.9999999999992independent software output0.99999999999920± 1e−9PASS
Power, 1-sample t (d 1.2, n 5, two-sided)0.5302212511403independent software output0.53022125114033.3e−16± 1e−9PASS
Power, 1-sample t (d 1.2, n 5, larger)0.7119106100632independent software output0.71191061006321.7e−14± 1e−9PASS
Power, 1-sample t (d 1.2, n 10, two-sided)0.9203333855898independent software output0.9203333855898−8.1e−15± 1e−9PASS
Power, 1-sample t (d 1.2, n 10, larger)0.9674752485648independent software output0.9674752485648−4.1e−15± 1e−9PASS
Power, 1-sample t (d 1.2, n 20, two-sided)0.9990954035492independent software output0.99909540354922.2e−16± 1e−9PASS
Power, 1-sample t (d 1.2, n 20, larger)0.9997839137198independent software output0.99978391371980± 1e−9PASS
Power, 1-sample t (d 1.2, n 50, two-sided)0.9999999998935independent software output0.99999999989350± 1e−9PASS
Power, 1-sample t (d 1.2, n 50, larger)0.9999999999908independent software output0.99999999999080± 1e−9PASS
Power, 1-sample t (d 1.2, n 120, larger)1independent software output10± 1e−9PASS
Power, 2-sample t (d 0.2, 5 per group, two-sided)0.05904263425365independent software output0.05904263425365−4.6e−15± 1e−9PASS
Power, 2-sample t (d 0.2, 5 per group, larger)0.08779955011805independent software output0.08779955011805−8.7e−15± 1e−9PASS
Power, 2-sample t (d 0.2, 10 per group, two-sided)0.07082134518931independent software output0.0708213451893−6.1e−15± 1e−9PASS
Power, 2-sample t (d 0.2, 10 per group, larger)0.1123273468601independent software output0.11232734686012.1e−15± 1e−9PASS
Power, 2-sample t (d 0.2, 20 per group, two-sided)0.0945673276173independent software output0.09456732761729−8.1e−15± 1e−9PASS
Power, 2-sample t (d 0.2, 20 per group, larger)0.1530096672282independent software output0.15300966722823.3e−16± 1e−9PASS
Power, 2-sample t (d 0.2, 50 per group, two-sided)0.1676754864961independent software output0.1676754864961−2.6e−14± 1e−9PASS
Power, 2-sample t (d 0.2, 50 per group, larger)0.2572804628117independent software output0.25728046281184.4e−14± 1e−9PASS
Power, 2-sample t (d 0.2, 120 per group, two-sided)0.3385617478128independent software output0.33856174781285.4e−14± 1e−9PASS
Power, 2-sample t (d 0.2, 120 per group, larger)0.4601463445893independent software output0.4601463445893−2.0e−14± 1e−9PASS
Power, 2-sample t (d 0.5, 5 per group, two-sided)0.1076859897976independent software output0.1076859897975−7.3e−15± 1e−9PASS
Power, 2-sample t (d 0.5, 5 per group, larger)0.1788338948398independent software output0.1788338948398−1.5e−14± 1e−9PASS
Power, 2-sample t (d 0.5, 10 per group, two-sided)0.1850956562912independent software output0.1850956562912−1.2e−14± 1e−9PASS
Power, 2-sample t (d 0.5, 10 per group, larger)0.2847634913513independent software output0.28476349135133.8e−15± 1e−9PASS
Power, 2-sample t (d 0.5, 20 per group, two-sided)0.337939028925independent software output0.337939028925−1.7e−14± 1e−9PASS
Power, 2-sample t (d 0.5, 20 per group, larger)0.4633743492964independent software output0.4633743492964−2.2e−16± 1e−9PASS
Power, 2-sample t (d 0.5, 50 per group, two-sided)0.6968934055334independent software output0.6968934055333−3.7e−14± 1e−9PASS
Power, 2-sample t (d 0.5, 50 per group, larger)0.7989361641502independent software output0.79893616415024.5e−14± 1e−9PASS
Power, 2-sample t (d 0.5, 120 per group, two-sided)0.9711087800141independent software output0.97110878001418.1e−15± 1e−9PASS
Power, 2-sample t (d 0.5, 120 per group, larger)0.9866914750475independent software output0.9866914750475−2.3e−15± 1e−9PASS
Power, 2-sample t (d 0.8, 5 per group, two-sided)0.200739452652independent software output0.200739452652−1.1e−14± 1e−9PASS
Power, 2-sample t (d 0.8, 5 per group, larger)0.3133300246889independent software output0.3133300246889−2.1e−14± 1e−9PASS
Power, 2-sample t (d 0.8, 10 per group, two-sided)0.3950692121136independent software output0.3950692121136−1.7e−14± 1e−9PASS
Power, 2-sample t (d 0.8, 10 per group, larger)0.5303874685238independent software output0.53038746852394.9e−15± 1e−9PASS
Power, 2-sample t (d 0.8, 20 per group, two-sided)0.6934041966154independent software output0.6934041966154−1.6e−14± 1e−9PASS
Power, 2-sample t (d 0.8, 20 per group, larger)0.7994081780872independent software output0.7994081780872−3.3e−16± 1e−9PASS
Power, 2-sample t (d 0.8, 50 per group, two-sided)0.977279090654independent software output0.977279090654−5.7e−15± 1e−9PASS
Power, 2-sample t (d 0.8, 50 per group, larger)0.9900258947477independent software output0.99002589474774.4e−15± 1e−9PASS
Power, 2-sample t (d 0.8, 120 per group, two-sided)0.9999873244295independent software output0.99998732442951.1e−16± 1e−9PASS
Power, 2-sample t (d 0.8, 120 per group, larger)0.9999971089792independent software output0.99999710897920± 1e−9PASS
Power, 2-sample t (d 1.2, 5 per group, two-sided)0.3864372610122independent software output0.3864372610122−1.7e−14± 1e−9PASS
Power, 2-sample t (d 1.2, 5 per group, larger)0.535662602192independent software output0.535662602192−2.6e−14± 1e−9PASS
Power, 2-sample t (d 1.2, 10 per group, two-sided)0.7184053765417independent software output0.7184053765417−1.6e−14± 1e−9PASS
Power, 2-sample t (d 1.2, 10 per group, larger)0.8252224525487independent software output0.82522245254873.6e−15± 1e−9PASS
Power, 2-sample t (d 1.2, 20 per group, two-sided)0.9588268143625independent software output0.9588268143625−4.4e−15± 1e−9PASS
Power, 2-sample t (d 1.2, 20 per group, larger)0.9812867561418independent software output0.98128675614180± 1e−9PASS
Power, 2-sample t (d 1.2, 50 per group, two-sided)0.999965601217independent software output0.9999656012171.1e−16± 1e−9PASS
Power, 2-sample t (d 1.2, 50 per group, larger)0.9999919537392independent software output0.99999195373920± 1e−9PASS
Power, 2-sample t (d 1.2, 120 per group, larger)1independent software output10± 1e−9PASS
Power, 2 proportions (0.1 vs 0.05, 50 per group, two-sided)0.1566971371917independent software output0.1566971371917−2.2e−16± 1e−9PASS
Power, 2 proportions (0.1 vs 0.05, 50 per group, larger)0.2423226607327independent software output0.24232266073278.9e−16± 1e−9PASS
Power, 2 proportions (0.1 vs 0.05, 200 per group, two-sided)0.4753633953952independent software output0.4753633953952−1.7e−16± 1e−9PASS
Power, 2 proportions (0.1 vs 0.05, 200 per group, larger)0.6004885508234independent software output0.60048855082341.3e−15± 1e−9PASS
Power, 2 proportions (0.1 vs 0.05, 800 per group, two-sided)0.9674810599019independent software output0.9674810599019−1.1e−16± 1e−9PASS
Power, 2 proportions (0.1 vs 0.05, 800 per group, larger)0.9846730604555independent software output0.98467306045551.1e−16± 1e−9PASS
Power, 2 proportions (0.1 vs 0.05, 3000 per group, two-sided)0.9999999696347independent software output0.99999996963470± 1e−9PASS
Power, 2 proportions (0.1 vs 0.05, 3000 per group, larger)0.9999999950716independent software output0.99999999507160± 1e−9PASS
Power, 2 proportions (0.3 vs 0.2, 50 per group, two-sided)0.2096303634395independent software output0.2096303634395−2.2e−16± 1e−9PASS
Power, 2 proportions (0.3 vs 0.2, 50 per group, larger)0.3108459810388independent software output0.31084598103888.9e−16± 1e−9PASS
Power, 2 proportions (0.3 vs 0.2, 200 per group, two-sided)0.6375108199581independent software output0.6375108199581−2.2e−16± 1e−9PASS
Power, 2 proportions (0.3 vs 0.2, 200 per group, larger)0.7482594588413independent software output0.74825945884131.0e−15± 1e−9PASS
Power, 2 proportions (0.3 vs 0.2, 800 per group, two-sided)0.9962829183347independent software output0.99628291833470± 1e−9PASS
Power, 2 proportions (0.3 vs 0.2, 800 per group, larger)0.998623159011independent software output0.9986231590110± 1e−9PASS
Power, 2 proportions (0.3 vs 0.2, 3000 per group, two-sided)0.999999999999independent software output0.9999999999990± 1e−9PASS
Power, 2 proportions (0.3 vs 0.2, 3000 per group, larger)0.9999999999999independent software output0.99999999999990± 1e−9PASS
Power, 2 proportions (0.5 vs 0.4, 50 per group, two-sided)0.1700223101303independent software output0.1700223101303−2.2e−16± 1e−9PASS
Power, 2 proportions (0.5 vs 0.4, 50 per group, larger)0.260088725766independent software output0.2600887257668.9e−16± 1e−9PASS
Power, 2 proportions (0.5 vs 0.4, 200 per group, two-sided)0.5201179041645independent software output0.52011790416450± 1e−9PASS
Power, 2 proportions (0.5 vs 0.4, 200 per group, larger)0.6432205704507independent software output0.64322057045071.2e−15± 1e−9PASS
Power, 2 proportions (0.5 vs 0.4, 800 per group, two-sided)0.9808052603079independent software output0.98080526030790± 1e−9PASS
Power, 2 proportions (0.5 vs 0.4, 800 per group, larger)0.9915156617608independent software output0.99151566176080± 1e−9PASS
Power, 2 proportions (0.5 vs 0.4, 3000 per group, two-sided)0.9999999976102independent software output0.99999999761020± 1e−9PASS
Power, 2 proportions (0.5 vs 0.4, 3000 per group, larger)0.9999999996615independent software output0.99999999966150± 1e−9PASS
Power, 2 proportions (0.02 vs 0.01, 50 per group, two-sided)0.06939361040559independent software output0.06939361040559−3.2e−16± 1e−9PASS
Power, 2 proportions (0.02 vs 0.01, 50 per group, larger)0.1084983627799independent software output0.10849836277994.4e−16± 1e−9PASS
Power, 2 proportions (0.02 vs 0.01, 200 per group, two-sided)0.1301869974698independent software output0.1301869974698−1.7e−16± 1e−9PASS
Power, 2 proportions (0.02 vs 0.01, 200 per group, larger)0.2052938027556independent software output0.20529380275566.7e−16± 1e−9PASS
Power, 2 proportions (0.02 vs 0.01, 800 per group, two-sided)0.3765916951194independent software output0.3765916951194−1.1e−16± 1e−9PASS
Power, 2 proportions (0.02 vs 0.01, 800 per group, larger)0.5002102260703independent software output0.50021022607031.2e−15± 1e−9PASS
Power, 2 proportions (0.02 vs 0.01, 3000 per group, two-sided)0.8901526697371independent software output0.89015266973711.1e−16± 1e−9PASS
Power, 2 proportions (0.02 vs 0.01, 3000 per group, larger)0.9385502772131independent software output0.93855027721314.4e−16± 1e−9PASS
Power, one-way ANOVA (f 0.1, 3 groups of 5)0.05896536542259independent software output0.058965365422612.2e−14± 1e−9PASS
Power, one-way ANOVA (f 0.1, 3 groups of 15)0.0829289166563independent software output0.0829289166563−7.9e−15± 1e−9PASS
Power, one-way ANOVA (f 0.1, 3 groups of 40)0.1479676280058independent software output0.1479676280058−2.2e−16± 1e−9PASS
Power, one-way ANOVA (f 0.1, 3 groups of 100)0.3186429507235independent software output0.31864295072351.2e−14± 1e−9PASS
Power, one-way ANOVA (f 0.1, 4 groups of 5)0.05948277737515independent software output0.05948277737514−1.3e−14± 1e−9PASS
Power, one-way ANOVA (f 0.1, 4 groups of 15)0.08511539469272independent software output0.08511539469287.4e−14± 1e−9PASS
Power, one-way ANOVA (f 0.1, 4 groups of 40)0.1575861937825independent software output0.15758619378269.5e−14± 1e−9PASS
Power, one-way ANOVA (f 0.1, 4 groups of 100)0.3552591913706independent software output0.35525919137071.2e−13± 1e−9PASS
Power, one-way ANOVA (f 0.1, 6 groups of 5)0.06073809934242independent software output0.0607380993424−2.4e−14± 1e−9PASS
Power, one-way ANOVA (f 0.1, 6 groups of 15)0.09028912877278independent software output0.090289128772795.2e−15± 1e−9PASS
Power, one-way ANOVA (f 0.1, 6 groups of 40)0.1788523428012independent software output0.1788523428011−7.2e−14± 1e−9PASS
Power, one-way ANOVA (f 0.1, 6 groups of 100)0.4290285136216independent software output0.4290285136215−4.9e−15± 1e−9PASS
Power, one-way ANOVA (f 0.25, 3 groups of 5)0.1095296872039independent software output0.1095296872043.5e−14± 1e−9PASS
Power, one-way ANOVA (f 0.25, 3 groups of 15)0.2849724443179independent software output0.2849724443179−1.5e−14± 1e−9PASS
Power, one-way ANOVA (f 0.25, 3 groups of 40)0.6756744946708independent software output0.67567449467087.8e−16± 1e−9PASS
Power, one-way ANOVA (f 0.25, 3 groups of 100)0.9780555505591independent software output0.97805555055915.6e−15± 1e−9PASS
Power, one-way ANOVA (f 0.25, 4 groups of 5)0.1150101405246independent software output0.1150101405246−2.1e−14± 1e−9PASS
Power, one-way ANOVA (f 0.25, 4 groups of 15)0.3165802409043independent software output0.31658024090441.7e−13± 1e−9PASS
Power, one-way ANOVA (f 0.25, 4 groups of 40)0.749404537701independent software output0.74940453770111.3e−13± 1e−9PASS
Power, one-way ANOVA (f 0.25, 4 groups of 100)0.9928175163418independent software output0.99281751634184.3e−15± 1e−9PASS
Power, one-way ANOVA (f 0.25, 6 groups of 5)0.127476854629independent software output0.127476854629−4.2e−14± 1e−9PASS
Power, one-way ANOVA (f 0.25, 6 groups of 15)0.381895625593independent software output0.3818956255931.1e−14± 1e−9PASS
Power, one-way ANOVA (f 0.25, 6 groups of 40)0.8572074314554independent software output0.8572074314553−6.3e−14± 1e−9PASS
Power, one-way ANOVA (f 0.25, 6 groups of 100)0.999336261231independent software output0.9993362612311.1e−16± 1e−9PASS
Power, one-way ANOVA (f 0.4, 3 groups of 5)0.2137435107015independent software output0.21374351070155.6e−14± 1e−9PASS
Power, one-way ANOVA (f 0.4, 3 groups of 15)0.6350412095709independent software output0.6350412095709−1.7e−14± 1e−9PASS
Power, one-way ANOVA (f 0.4, 3 groups of 40)0.9789277028288independent software output0.97892770282881.1e−16± 1e−9PASS
Power, one-way ANOVA (f 0.4, 3 groups of 100)0.9999974799348independent software output0.99999747993480± 1e−9PASS
Power, one-way ANOVA (f 0.4, 4 groups of 5)0.2353286362836independent software output0.2353286362835−3.4e−14± 1e−9PASS
Power, one-way ANOVA (f 0.4, 4 groups of 15)0.7087216415816independent software output0.70872164158171.6e−13± 1e−9PASS
Power, one-way ANOVA (f 0.4, 4 groups of 40)0.9932129067282independent software output0.99321290672827.9e−15± 1e−9PASS
Power, one-way ANOVA (f 0.4, 4 groups of 100)0.9999999605018independent software output0.99999996050180± 1e−9PASS
Power, one-way ANOVA (f 0.4, 6 groups of 5)0.2819146866192independent software output0.2819146866192−6.7e−14± 1e−9PASS
Power, one-way ANOVA (f 0.4, 6 groups of 15)0.8225457871305independent software output0.82254578713058.4e−15± 1e−9PASS
Power, one-way ANOVA (f 0.4, 6 groups of 40)0.9993966418701independent software output0.9993966418701−5.6e−16± 1e−9PASS
Power, one-way ANOVA (f 0.4, 6 groups of 100)0.9999999999932independent software output0.99999999999320± 1e−9PASS
Sample size, 1-sample t (d 0.3, power 0.8, two-sided)90independent software output90noneexact matchPASS
Sample size, 1-sample t (d 0.3, power 0.8, larger)71independent software output71noneexact matchPASS
Sample size, 1-sample t (d 0.3, power 0.9, two-sided)119independent software output119noneexact matchPASS
Sample size, 1-sample t (d 0.3, power 0.9, larger)97independent software output97noneexact matchPASS
Sample size, 1-sample t (d 0.5, power 0.8, two-sided)34independent software output34noneexact matchPASS
Sample size, 1-sample t (d 0.5, power 0.8, larger)27independent software output27noneexact matchPASS
Sample size, 1-sample t (d 0.5, power 0.9, two-sided)44independent software output44noneexact matchPASS
Sample size, 1-sample t (d 0.5, power 0.9, larger)36independent software output36noneexact matchPASS
Sample size, 1-sample t (d 1, power 0.8, two-sided)10independent software output10noneexact matchPASS
Sample size, 1-sample t (d 1, power 0.8, larger)8independent software output8noneexact matchPASS
Sample size, 1-sample t (d 1, power 0.9, two-sided)13independent software output13noneexact matchPASS
Sample size, 1-sample t (d 1, power 0.9, larger)11independent software output11noneexact matchPASS
Sample size per group, 2-sample t (d 0.3, power 0.8, two-sided)176independent software output176noneexact matchPASS
Sample size per group, 2-sample t (d 0.3, power 0.8, larger)139independent software output139noneexact matchPASS
Sample size per group, 2-sample t (d 0.3, power 0.9, two-sided)235independent software output235noneexact matchPASS
Sample size per group, 2-sample t (d 0.3, power 0.9, larger)191independent software output191noneexact matchPASS
Sample size per group, 2-sample t (d 0.5, power 0.8, two-sided)64independent software output64noneexact matchPASS
Sample size per group, 2-sample t (d 0.5, power 0.8, larger)51independent software output51noneexact matchPASS
Sample size per group, 2-sample t (d 0.5, power 0.9, two-sided)86independent software output86noneexact matchPASS
Sample size per group, 2-sample t (d 0.5, power 0.9, larger)70independent software output70noneexact matchPASS
Sample size per group, 2-sample t (d 1, power 0.8, two-sided)17independent software output17noneexact matchPASS
Sample size per group, 2-sample t (d 1, power 0.8, larger)14independent software output14noneexact matchPASS
Sample size per group, 2-sample t (d 1, power 0.9, two-sided)23independent software output23noneexact matchPASS
Sample size per group, 2-sample t (d 1, power 0.9, larger)18independent software output18noneexact matchPASS

How to Rerun These Yourself

  1. Download a dataset below. Each is a plain CSV with a header row, in the order the source prints it.
  2. The Individuals (I-MR) tab with capability, the Pareto chart and the Run chart are on the Free plan. Subgroup and attribute charts and the other analysis tools need Pro.
  3. Open LimitLine and paste the file’s contents into the tool listed. LimitLine lists the header word under “Ignored non-numeric entries”, which is expected.
  4. Compare the result with the case’s expected values above. Or open the same file in Minitab, JMP, R or Excel and compare all three.

Settings to match when you compare with another package: I-MR sigma is MR̄/1.128 (average moving range). Gauge R&R uses the ANOVA method and removes the interaction when p > 0.05. Quartiles use the (n + 1)p method. The 2-sample t test is Welch’s (no equal-variance assumption) unless “Assume equal variances” is ticked. All results are also in machine-readable form in results.json.

FileWhat it isIn LimitLineIn Minitab or elsewhere
nist-6322-flow-rate.csv10 rowsNIST §6.3.2.2 flow rate, 10 batchesIndividuals (I-MR) tab: paste the column (rows are batches 1 to 10).Minitab: Stat › Control Charts › Variables Charts for Individuals › I-MR, column flow_rate.
xbar-r-test-5x5.csv5 rows5 subgroups of 5 (LimitLine test set)Subgroups (X̄-R / X̄-S) tab: paste the five data lines (one subgroup per line).Minitab: Stat › Control Charts › Variables Charts for Subgroups › Xbar-R, “Observations for a subgroup are in one row of columns”.
nist-6331-wafer-defects.csv25 rowsNIST §6.3.3.1 defects per wafer, 25 wafersAttributes tab, c chart: paste the 25 counts (one per line).Minitab: Stat › Control Charts › Attributes Charts › C.
nist-6332-wafer-proportions.csv30 rowsNIST §6.3.3.2 defective chips, 30 wafers of 50Attributes tab, p chart: paste the lines “defectives, sample size” (for the np chart, also enter sample size 50).Minitab: Stat › Control Charts › Attributes Charts › P, subgroup sizes column sample_size.
montgomery-u-computers.csv20 rowsMontgomery p. 318 (via IMSL): defects in 20 samples of 5 computersAttributes tab, u chart: paste the lines “defects, units”.Minitab: Stat › Control Charts › Attributes Charts › U, subgroup sizes column units.
montgomery-u-cloth-rolls.csv10 rowsMontgomery p. 321 (via IMSL): defects in 10 rolls of cloth, units of 50 m²Attributes tab, u chart: paste the lines “defects, units”.Minitab: Stat › Control Charts › Attributes Charts › U, subgroup sizes column units.
spcforexcel-laney-p.csv16 rowsSPC for Excel Laney p′ example, 16 monthsAnalysis tools › EWMA · CUSUM · more › Laney P′: paste the lines (the first two numbers on each line are used: defectives, sample size).Minitab: Stat › Control Charts › Attributes Charts › Laney P′.
nist-616-capability-mean16-s2.csv6 rows6 values with mean 16 and s 2 (built to match NIST §6.1.6)Individuals (I-MR) tab with LSL 8 and USL 20: paste the values and compare Pp and Ppk.Minitab: Stat › Quality Tools › Capability Analysis › Normal (subgroup size 1); compare overall Pp/Ppk.
aiag-msa4-gage-rr.csv90 rowsAIAG MSA (4th ed.) gauge study as printed by SPC for Excel: 10 parts, operators A/B/C, 3 trialsAnalysis tools › Gauge R&R: paste the lines “part, operator, measurement” (the trial column at the end is ignored).Minitab: Stat › Quality Tools › Gage Study › Gage R&R Study (Crossed), ANOVA method.
nist-strd-longley.csv16 rowsNIST StRD Longley (y = TOTEMP first, then 6 predictors)Analysis tools › Regression: paste the file with its header row (first column = Y).Minitab: Stat › Regression › Regression › Fit Regression Model, response TOTEMP, the other six as continuous predictors.
nist-strd-numacc4.csv1001 rowsNIST StRD NumAcc4, 1001 valuesIndividuals (I-MR) tab: the center line is the mean. The app rounds what it shows, so the digit-level result is in results.json.Minitab: Stat › Basic Statistics › Display Descriptive Statistics (increase displayed digits).
nist-strd-numacc2.csv1001 rowsNIST StRD NumAcc2, 1001 valuesIndividuals (I-MR) tab: the center line is the mean. The app rounds what it shows, so the digit-level result is in results.json.Minitab: Stat › Basic Statistics › Display Descriptive Statistics.
anhoj-figure2-pattern.csv24 rowsPattern of the qicharts “Run charts with R” Figure 2 case (above/below/on the median)Analysis tools › Run chart: paste the column. It shows the longest run about the median (10) and flags the shift.Any run-chart tool that applies the Anhøj rules (for example qicharts in R).
nist-ewma.csv20 rowsNIST §6.3.2.4 EWMA example, 20 pointsAnalysis tools › EWMA · CUSUM · more: Chart EWMA, Target 50, σ 2.0539, EWMA weight λ 0.3, EWMA width L 3.Any EWMA chart with the same settings.
nist-cusum.csv20 rowsNIST §6.3.2.3 CUSUM spreadsheet example, 20 group meansAnalysis tools › EWMA · CUSUM · more: Chart CUSUM, Target 325, σ 0.635, CUSUM k 0.5, CUSUM h 6.608.Any tabular CUSUM with k = 0.3175 and h = 4.1959 in data units.
refs-describe.csv37 rows37 gamma-distributed values used for the descriptive-statistics cross-checkIndividuals (I-MR) tab: the center line is the mean. The app rounds what it shows, so the digit-level result is in results.json.Minitab: Stat › Basic Statistics › Display Descriptive Statistics.
refs-anderson-darling.csv280 rows4 samples (n = 25, 40, 200, 15) used for the Anderson-Darling cross-checkAnalysis tools › Capability+: paste one sample’s values at a time and add any spec limit; the Anderson-Darling result is shown.Minitab: Stat › Basic Statistics › Normality Test, Anderson-Darling.
refs-t-tests.csv59 rowsSamples a (n 18), b (n 23), c (n 18, paired with a) for the t-test cross-checksAnalysis tools › Hypothesis tests: paste sample a (and b, or c for the paired test). 1-sample target 9.5.Minitab: Stat › Basic Statistics › 1-Sample t / 2-Sample t / Paired t.
refs-anova-groups.csv39 rows4 groups (n 8, 12, 10, 9) for the one-way ANOVA and Levene cross-checksAnalysis tools › Hypothesis tests › one-way ANOVA or equal variances: paste each group.Minitab: Stat › ANOVA › One-Way; Stat › ANOVA › Test for Equal Variances (Levene).
refs-regression.csv30 rows30 observations, 3 correlated predictors, for the OLS cross-checkAnalysis tools › Regression: paste with the header row; predict at 0.5, -0.2, 0.1.Minitab: Stat › Regression › Regression › Fit Regression Model.
refs-lognormal-60.csv60 rows60 lognormal values for the Box-Cox and lognormal-fit cross-checksAnalysis tools › Capability+ (Box-Cox or Lognormal): paste the column and add a spec limit.Minitab: Stat › Quality Tools › Individual Distribution Identification.
refs-weibull-80.csv80 rows80 Weibull values for the Weibull-fit cross-checkAnalysis tools › Capability+ (Weibull): paste the column and add a spec limit.Minitab: Stat › Quality Tools › Individual Distribution Identification (2-parameter Weibull).
refs-gage-rr-synthetic.csv60 rowsSimulated gauge study, 10 parts × 3 operators × 2 trialsAnalysis tools › Gauge R&R: paste the lines “part, operator, measurement”; LSL 3, USL 7.Minitab: Stat › Quality Tools › Gage Study › Gage R&R Study (Crossed), ANOVA method.
refs-doe-2x3-replicated.csv16 rows2³ factorial, 2 replicates (16 runs), coded −1/+1Analysis tools › DOE, step 2: paste the runs with the header row (factors A, B, C, last column = response).Minitab: Stat › DOE › Factorial › Define Custom Factorial Design, then Analyze Factorial Design (full model).

Sources Cited

Code Fingerprint

SHA-256 of the calculation modules tested for this page (commit d899f86):

FileSHA-256
spc-core.js3d59458120d7364cc33ee507bfee924d200a3563df251464de8b448d557c9b54
stats-core.jsb055813882f583a2a2509ebb062c240dd67aeb25e1e089209c1df04e2fdfd734
lean-core.jsb1b0ba840858a676fabe09be56f7128881d1d2d3c9aebffe31fb48fa4ed7fde8

Found a number that disagrees with your software? Email support@limitline.org with the dataset and both results.

← Back to LimitLine · Privacy policy · Terms of service