<< Click to Display Table of Contents >> Navigation:  Reliability Checks > KAPPA - Inter-Rater-Reliability > Kappa Results > Tipps to improve Kappa Results

As stated before, Kappa was designed for mutually exclusive and exhaustive data and largely relies on probability calculation.

Probability calculation requires a lot of data to produce a relevant result. That is why calculating Kappa on a Class with just one or two Codes does not work well. Also very few Events per document may cause irrelevant Kappa results. See also FAQ about Kappa.

Merge Multiple Data Files

To increase the total number of Events (the larger the data pool, the more reliable the outcome), create a compilation file for each Observer.

▪Use the Start-File-Merge command, as described in Merge Data Files, to create the following compilation files:

▪Merge all completely coded data file of Observer A into a single compilation file (Compilation-A).

▪Do the same for the matching files coded by Obererver B (Compilation-B)

▪Use the files 'Compilation-A' and 'Compilation-B' to calculate KAPPA - Inter-Rater-Reliability.

Make Exhaustive Events

If your data contains gaps on the timeline per Class, let INTERACT create Events to close those gaps using a dummy Code like NULL, N/A or whatever name suits you.

▪Make use of the Make exhaustive command, as is described in Fill Gaps with Data.

This increases the number of Codes (=variables) per Class, which is good for Kappa and it also makes sure the data is exhaustive per Class.

Copy Code to a Generic Kappa Class

Kappa is always calculated per Class, so there is no such thing as an 'over-all' Kappa.

It is possible to copy all (or specific) Codes to a single 'Kappa'-Class. This increases the number of Codes / Variables, which is usually good for the overall results and only look at the results of the Kappa calculation for this special Class.

▪Use the command Move & Combine Codes to copy Codes to this new Class.

For further details, take a look at the Move & Combine YouTube tutorial: