Reliability and validity

Research should be tested for reliability, generalizability, and validity. Generalizability is the ability to make inferences from a sample to the population.

Reliability is the extent to which a measure will produce consistent results. Test-retest reliability checks how similar the results are if the research is repeated under similar circumstances. Stability over repeated measures is assessed with the Pearson coefficient. Alternative forms reliability checks how similar the results are if the research is repeated using different forms. Internal consistency reliability checks how well the individual measures included in the research are converted into a composite measure. Internal consistency may be assessed by correlating performance on two halves of a test (split-half reliability).

The value of the Pearson product-moment correlation coefficient is adjusted with the Spearman-Brown prediction formula to correspond to the correlation between two full-length tests. A commonly used measure is Cronbach's α, which is equivalent to the mean of all possible split-half coefficients. Reliability may be improved by increasing the sample size.

Validity asks whether the research measured what it intended to. Content validation (also called face validity) checks how well the content of the research are related to the variables to be studied. Is the research questions representative of the variables being researched.

It is a demonstration that the items of a test are drawn from the domain being measured. Criterion validation checks how meaningful the research criteria are relative to other possible criteria. When the criterion is collected later the goal is to establish predictive validity. Construct validation checks what underlying construct is being measured. There are three variants of construct validity.

They are convergent validity (how well the research relates to other measures of the same construct), discriminant validity (how poorly the research relates to measures of opposing constructs), and nomological validity (how well the research relates to other variables as required by theory) .

Internal validation, used primarily in experimental research designs, checks the relation between the dependent and independent variables. Did the experimental manipulation of the independent variable actually cause the observed results? External validation checks whether the experimental results can be generalized.

Validity implies reliability: a valid measure must be reliable. But reliability does not necessarily imply validity: a reliable measure need not be valid.