MSA Nominal Level Output

The output for a Measurement Systems Analysis for Nominal level data is shown below.

Several unique diagnostic assessments are performed. Pair-wise comparisons are made for each set of measures for each appraiser. Pair-wise comparisons are also made between each appraiser.

Output Statistics

nSample Size
AgreeThe number of items in which agreement occurred
DisagreeThe number of items in which disagreement occurred
P(Agree)The proportion of items in which agreement occurred
P(Disagree) The proportion of items in which disagreement occurred
P(Chance)The proportion of agreement expected due to chance
Kmaxkappa Max
kappaThe kappa Statistic
CI(low)The lower bound of the confidence interval for kappa
CI(high)The upper bound of the confidence interval for kappa
p(kappa)The p-value or statistical significance of kappa
p(Sym)The p-value of the test for symmetry

example — Three Appraisers, One Measure Each

Measurement Systems Analysis - Nominal Data

	File: C:\DATA\LWI\MSA-Nominal\Christie, Lisa, and Patsy.mvp
	Confidence Level = 95.00%

Variables

Christie [Christie] Lisa [Lisa] Patsy [Patsy]

Concordance Analysis (Between Appraisers)

	Group                    n  Agree  Disagree  P(Agree)  P(Disagree)  P(Chance)  Kmax  kappa  CI(low) CI(high) p(kappa)  p(Sym)
	Christie vs Lisa       175    169         6    0.966        0.034      0.510  0.953  0.930   0.875    0.985    0.000*  0.219 
	Christie vs Patsy      175    130        45    0.743        0.257      0.497  0.830  0.489   0.359    0.618    0.000*  0.036*
	Lisa vs Patsy          175    126        49    0.720        0.280      0.496  0.785  0.445   0.311    0.579    0.000*  0.009*
	Average                175    142        33    0.810        0.190      0.501  0.856  0.618     --       --       --      --  
	Overall                175    125        50    0.714        0.286      0.251  0.855  0.618   0.532    0.705    0.000*    --  

example — Two Appraisers, Three Measures Each

Measurement Systems Analysis - Nominal Data

	File: C:\DATA\LWI\MSA-Nominal\Ray and Victoria.mvp
	Confidence Level = 95.00%

Variables

	Ray [Ray1,Ray2,Ray3]
	Victoria [Victoria1,Victoria2,Victoria3]

Specifications

	Acceptable Code = 1

Internal Consistency (Within Appraiser)

Ray - Internal Consistency Analysis

	Group               n  Agree  Disagree  P(Agree)  P(Disagree)  P(Chance)  Kmax  kappa  CI(low) CI(high) p(kappa)  p(Sym)
	m1 vs m2          250    239        11    0.956        0.044      0.337  0.964  0.934   0.895    0.972    0.000*  0.446 
	m1 vs m3          250    233        17    0.932        0.068      0.338  0.976  0.897   0.851    0.944    0.000*  0.333 
	m2 vs m3          250    230        20    0.920        0.080      0.333  0.982  0.880   0.830    0.930    0.000*  0.255 
	Average           250    234        16    0.936        0.064      0.336  0.974  0.904     --       --       --      --  
	Overall           250    227        23    0.908        0.092      0.139  0.972  0.893   0.836    0.950    0.000*    --  

	Category Analysis (m1 vs m2)
	1 vs Not          250    244         6    0.976        0.024      0.500  0.968  0.952   0.914    0.990    0.000*  0.219 
	2 vs Not          250    243         7    0.972        0.028      0.673  0.963  0.914   0.852    0.977    0.000*  0.453 
	3 vs Not          250    244         6    0.976        0.024      0.710  0.972  0.917   0.852    0.983    0.000*  0.687 
	4 vs Not          250    247         3    0.988        0.012      0.792  0.942  0.942   0.878    1.000    0.000*  0.250 
	Overall           250    239        11    0.956        0.044      0.337  0.964  0.934   0.895    0.972    0.000*  0.446 

	Category Analysis (m1 vs m3)
	1 vs Not          250    242         8    0.968        0.032      0.500  0.968  0.936   0.892    0.980    0.000*  0.289 
	2 vs Not          250    242         8    0.968        0.032      0.670  0.951  0.903   0.837    0.969    0.000*  0.289 
	3 vs Not          250    240        10    0.960        0.040      0.705  1.000  0.864   0.783    0.946    0.000*  1.000 
	4 vs Not          250    242         8    0.968        0.032      0.801  1.000  0.839   0.730    0.948    0.000*  1.000 
	Overall           250    233        17    0.932        0.068      0.338  0.976  0.897   0.851    0.944    0.000*  0.333 

	Category Analysis (m2 vs m3)
	1 vs Not          250    240        10    0.960        0.040      0.500  1.000  0.920   0.871    0.969    0.000*  1.000 
	2 vs Not          250    237        13    0.948        0.052      0.664  0.988  0.845   0.764    0.927    0.000*  1.000 
	3 vs Not          250    240        10    0.960        0.040      0.710  0.972  0.862   0.779    0.945    0.000*  0.754 
	4 vs Not          250    243         7    0.972        0.028      0.792  0.942  0.866   0.768    0.963    0.000*  0.453 
	Overall           250    230        20    0.920        0.080      0.333  0.982  0.880   0.830    0.930    0.000*  0.255 

	Category Analysis (All Measures)
	1 vs Not          250    238        12    0.952        0.048      0.250  0.979  0.936   0.864    1.000    0.000*    --  
	2 vs Not          250    236        14    0.944        0.056      0.503  0.968  0.887   0.762    1.000    0.000*    --  
	3 vs Not          250    237        13    0.948        0.052      0.562  0.982  0.881   0.741    1.000    0.000*    --  
	4 vs Not          250    241         9    0.964        0.036      0.692  0.961  0.883   0.697    1.000    0.000*    --  
	Overall           250    227        23    0.908        0.092      0.139  0.972  0.893   0.836    0.950    0.000*    --  

	Category Analysis (m1 vs m2) - Nonconforming Only
	2 vs Not          122    120         2    0.984        0.016      0.519  1.000  0.966   0.919    1.000    0.000*  1.000 
	3 vs Not          122    117         5    0.959        0.041      0.541  0.946  0.911   0.834    0.987    0.000*  0.375 
	4 vs Not          122    119         3    0.975        0.025      0.633  0.933  0.933   0.858    1.000    0.000*  0.250 
	Overall           122    117         5    0.959        0.041      0.347  0.962  0.937   0.808    1.000    0.000*    --  

	Category Analysis (m1 vs m3) - Nonconforming Only
	2 vs Not          121    118         3    0.975        0.025      0.516  0.949  0.949   0.892    1.000    0.000*  0.250 
	3 vs Not          121    112         9    0.926        0.074      0.535  0.982  0.840   0.740    0.940    0.000*  1.000 
	4 vs Not          121    115         6    0.950        0.050      0.653  0.952  0.857   0.746    0.968    0.000*  0.687 
	Overall           121    112         9    0.926        0.074      0.352  0.962  0.885   0.754    1.000    0.000*    --  

	Category Analysis (m2 vs m3) - Nonconforming Only
	2 vs Not          122    118         4    0.967        0.033      0.519  0.966  0.932   0.866    0.997    0.000*  0.625 
	3 vs Not          122    113         9    0.926        0.074      0.541  0.982  0.839   0.738    0.940    0.000*  1.000 
	4 vs Not          122    115         7    0.943        0.057      0.633  0.933  0.844   0.732    0.956    0.000*  0.453 
	Overall           122    112        10    0.918        0.082      0.347  0.962  0.875   0.745    1.000    0.000*    --  

	Category Analysis (All Measures) - Nonconforming Only
	2 vs Not          120    116         4    0.967        0.033      0.280  0.965  0.954   0.842    1.000    0.000*    --  
	3 vs Not          120    109        11    0.908        0.092      0.305  0.964  0.868   0.749    0.987    0.000*    --  
	4 vs Not          120    113         7    0.942        0.058      0.458  0.923  0.892   0.728    1.000    0.000*    --  
	Overall           120    109        11    0.908        0.092      0.125  0.952  0.895   0.818    0.972    0.000*    -- 

Victoria - Internal Consistency Analysis

	Group               n  Agree  Disagree  P(Agree)  P(Disagree)  P(Chance)  Kmax  kappa  CI(low) CI(high) p(kappa)  p(Sym)
	m1 vs m2          250    239        11    0.956        0.044      0.341  0.982  0.933   0.895    0.971    0.000*  0.423 
	m1 vs m3          250    235        15    0.940        0.060      0.339  0.970  0.909   0.865    0.954    0.000*  0.477 
	m2 vs m3          250    230        20    0.920        0.080      0.336  0.976  0.880   0.829    0.930    0.000*  0.490 
	Average           250    235        15    0.939        0.061      0.339  0.976  0.907     --       --       --      --  
	Overall           250    228        22    0.912        0.088      0.141  0.977  0.898   0.841    0.954    0.000*    --  

	Category Analysis (m1 vs m2)
	1 vs Not          250    245         5    0.980        0.020      0.500  0.976  0.960   0.925    0.995    0.000*  0.375 
	2 vs Not          250    246         4    0.984        0.016      0.675  0.975  0.951   0.903    0.999    0.000*  0.625 
	3 vs Not          250    243         7    0.972        0.028      0.687  0.987  0.910   0.845    0.976    0.000*  1.000 
	4 vs Not          250    244         6    0.976        0.024      0.820  1.000  0.867   0.762    0.971    0.000*  1.000 
	Overall           250    239        11    0.956        0.044      0.341  0.982  0.933   0.895    0.971    0.000*  0.423 

	Category Analysis (m1 vs m3)
	1 vs Not          250    240        10    0.960        0.040      0.500  0.968  0.920   0.871    0.969    0.000*  0.344 
	2 vs Not          250    243         7    0.972        0.028      0.682  0.987  0.912   0.848    0.976    0.000*  1.000 
	3 vs Not          250    240        10    0.960        0.040      0.680  0.950  0.875   0.799    0.951    0.000*  0.344 
	4 vs Not          250    247         3    0.988        0.012      0.817  0.978  0.934   0.861    1.000    0.000*  1.000 
	Overall           250    235        15    0.940        0.060      0.339  0.970  0.909   0.865    0.954    0.000*  0.477 

	Category Analysis (m2 vs m3)
	1 vs Not          250    239        11    0.956        0.044      0.500  0.992  0.912   0.861    0.963    0.000*  1.000 
	2 vs Not          250    241         9    0.964        0.036      0.678  0.963  0.888   0.817    0.960    0.000*  0.508 
	3 vs Not          250    237        13    0.948        0.052      0.678  0.963  0.839   0.754    0.924    0.000*  0.581 
	4 vs Not          250    243         7    0.972        0.028      0.817  0.978  0.847   0.736    0.958    0.000*  1.000 
	Overall           250    230        20    0.920        0.080      0.336  0.976  0.880   0.829    0.930    0.000*  0.490 

	Category Analysis (All Measures)
	1 vs Not          250    237        13    0.948        0.052      0.250  0.979  0.931   0.859    1.000    0.000*    --  
	2 vs Not          250    240        10    0.960        0.040      0.518  0.975  0.917   0.789    1.000    0.000*    --  
	3 vs Not          250    235        15    0.940        0.060      0.522  0.967  0.874   0.745    1.000    0.000*    --  
	4 vs Not          250    242         8    0.968        0.032      0.727  0.985  0.883   0.681    1.000    0.000*    --  
	Overall           250    228        22    0.912        0.088      0.141  0.977  0.898   0.841    0.954    0.000*    --  

	Category Analysis (m1 vs m2) - Nonconforming Only
	2 vs Not          122    120         2    0.984        0.016      0.516  1.000  0.966   0.920    1.000    0.000*  1.000 
	3 vs Not          122    116         6    0.951        0.049      0.523  0.966  0.897   0.817    0.977    0.000*  0.687 
	4 vs Not          122    118         4    0.967        0.033      0.684  0.948  0.896   0.797    0.996    0.000*  0.625 
	Overall           122    116         6    0.951        0.049      0.361  0.974  0.923   0.789    1.000    0.000*    --  

	Category Analysis (m1 vs m3) - Nonconforming Only
	2 vs Not          120    118         2    0.983        0.017      0.523  1.000  0.965   0.917    1.000    0.000*  1.000 
	3 vs Not          120    115         5    0.958        0.042      0.522  0.983  0.913   0.838    0.988    0.000*  1.000 
	4 vs Not          120    117         3    0.975        0.025      0.665  0.975  0.925   0.842    1.000    0.000*  1.000 
	Overall           120    115         5    0.958        0.042      0.355  0.987  0.935   0.803    1.000    0.000*    --  

	Category Analysis (m2 vs m3) - Nonconforming Only
	2 vs Not          121    119         2    0.983        0.017      0.525  1.000  0.965   0.917    1.000    0.000*  1.000 
	3 vs Not          121    112         9    0.926        0.074      0.520  0.983  0.845   0.748    0.942    0.000*  1.000 
	4 vs Not          121    114         7    0.942        0.058      0.667  0.975  0.826   0.702    0.950    0.000*  1.000 
	Overall           121    112         9    0.926        0.074      0.356  0.987  0.885   0.752    1.000    0.000*    --  

	Category Analysis (All Measures) - Nonconforming Only
	2 vs Not          119    116         3    0.975        0.025      0.283  1.000  0.965   0.852    1.000    0.000*    --  
	3 vs Not          119    110         9    0.924        0.076      0.281  0.965  0.895   0.782    1.000    0.000*    --  
	4 vs Not          119    113         6    0.950        0.050      0.507  0.949  0.898   0.716    1.000    0.000*    --  
	Overall           119    110         9    0.924        0.076      0.133  0.971  0.913   0.833    0.992    0.000*    --

Concordance Analysis (Between Appraisers)

	Group               n  Agree  Disagree  P(Agree)  P(Disagree)  P(Chance)  Kmax  kappa  CI(low) CI(high) p(kappa)  p(Sym)
	Ray vs Victoria   250    224        26    0.896        0.104      0.343  0.963  0.842   0.786    0.897    0.000*  0.102 

	Category Analysis (Ray vs Victoria)
	1 vs Not          250    242         8    0.968        0.032      0.500  1.000  0.936   0.892    0.980    0.000*  1.000 
	2 vs Not          250    240        10    0.960        0.040      0.675  1.000  0.877   0.802    0.951    0.000*  1.000 
	3 vs Not          250    230        20    0.920        0.080      0.705  0.919  0.729   0.617    0.841    0.000*  0.263 
	4 vs Not          250    236        14    0.944        0.056      0.807  0.876  0.710   0.566    0.853    0.000*  0.180 
	Overall           250    224        26    0.896        0.104      0.343  0.963  0.842   0.786    0.897    0.000*  0.102 

	Category Analysis (Ray vs Victoria) - Nonconforming Only
	2 vs Not          119    112         7    0.941        0.059      0.514  0.983  0.879   0.792    0.966    0.000*  1.000 
	3 vs Not          119    103        16    0.866        0.134      0.538  0.927  0.709   0.577    0.841    0.000*  0.454 
	4 vs Not          119    106        13    0.891        0.109      0.653  0.879  0.685   0.527    0.844    0.000*  0.267 
	Overall           119    101        18    0.849        0.151      0.352  0.935  0.766   0.634    0.899    0.000*    --