Learn — Data collection, measurement, validity & reliability

5 notes · 15 questions on the topic

Levels of measurement — nominal, ordinal, interval and ratio

The level of measurement is the mathematical property of the numbers a variable is recorded in — whether they merely name, or also rank, or also have equal intervals, or also have a true zero — set out by Stevens in 1946 as nominal, ordinal, interval and ratio. It matters because the level decides which statistics are meaningful: a mean of blood groups is nonsense, a mean of temperatures is fine, and the choice of a statistical test begins with it.

Nominal measurement only names or classifies — sex, blood group, religion, marital status, diagnosis, ward — with no order among the categories; only counts, percentages and the mode are meaningful, the chi-square test is its test, and a dichotomous variable such as alive or dead is nominal with two categories.
Ordinal measurement ranks the categories without equal or known distances between them — pain rated mild, moderate or severe, a single Likert response, socioeconomic class, the stage of a pressure injury; the median, percentiles and range describe it and the rank tests (Spearman's rho, Mann–Whitney, Wilcoxon) analyse it.
Interval measurement has equal intervals between values but no true zero, so ratios mean nothing — temperature in degrees Celsius or Fahrenheit, the calendar year, an IQ score: zero degrees is not the absence of heat and 40 °C is not twice as hot as 20 °C; the mean, the standard deviation, the t-test and Pearson's r are allowed, and a summed Likert scale is treated as interval by convention.
Ratio measurement has equal intervals and a true zero that means the absence of the property — weight, height, haemoglobin in grams per decilitre, pulse rate, blood pressure, income, time, temperature in kelvin — so every statistic applies and one value can be said to be twice another.
The levels form a hierarchy in which each has the properties of those below it, so data can be moved down (haemoglobin grouped into grades of anaemia) but never up, and the rule is to measure at the highest level the variable allows; the mnemonic is NOIR.
The statistic follows the level — nominal takes the mode and chi-square, ordinal the median and the non-parametric tests, interval and ratio the mean, the standard deviation and the parametric tests — and the level of the dependent variable together with the number of groups is what picks the test; discrete against continuous is a different classification, counting against measuring, that cuts across interval and ratio.

Data collection methods — questionnaire, interview, observation and records

Data collection is the systematic gathering of the information a study needs, with tools chosen to match the variables, the design and the participants; it is the empirical phase of the research process, and its quality caps the quality of everything that follows. The main methods are self-report by questionnaire or interview, observation, biophysiological measurement and the use of existing records, and each has a known strength and a known bias.

A questionnaire is a written self-report instrument with closed-ended items (fixed choices — quick and easy to code) or open-ended items (free text — richer but harder to analyse), given by post, online or in person; it is cheap, reaches many, keeps the respondent anonymous and has no interviewer bias, but response rates are low, literacy is needed and nothing can be probed — so its items are short, clear, unambiguous, non-leading, one idea each, in logical order with sensitive items late, and it is pretested before use.
An interview asks the questions orally — structured with a fixed schedule asked the same way to everyone (reliable and comparable), semi-structured with a guide and probes, or unstructured as a conversation for qualitative depth — face to face or by telephone, and in a focus group of six to twelve people with a moderator; it gives higher response, probing and access to the illiterate, at the cost of time, interviewer bias and socially desirable answers.
Observation records behaviour or events as they happen — structured with a checklist, rating scale or category system and time or event sampling, or unstructured as field notes — by a participant observer who joins the group or a non-participant one; it measures what people do rather than what they say, as with hand-washing compliance, and its costs are the Hawthorne effect, observer bias and the need to establish agreement between observers.
Biophysiological measures are in vivo (blood pressure, temperature, oxygen saturation, the electrocardiogram) or in vitro (laboratory values such as haemoglobin and blood sugar); they are objective and precise provided the instrument is calibrated and the procedure standardised for every participant.
Existing records — case sheets, registers, the census, hospital statistics — give secondary data that are cheap, place no burden on participants and cannot react to being studied, but they may be incomplete, inaccurate or collected for a different purpose, and access to them needs permission.
The method is chosen by the nature of the variable, the participants' age and literacy, the design and the resources, more than one method may be combined (triangulation), the collectors are trained and the whole plan is piloted, and whatever the tool, its reliability and validity must be established before the main study.

Rating scales — Likert, visual analogue and semantic differential

A rating scale is a device for turning an attitude, an opinion, a symptom or a perception into a number by having the respondent place a mark or a choice on a graded continuum; scales are the instruments behind most nursing research on pain, anxiety, satisfaction and quality of life. The examinable ones are the Likert scale, the visual analogue scale and the semantic differential, together with the level of measurement each yields.

A Likert scale (Rensis Likert, 1932) is a set of declarative statements about the attitude object, each answered on a graded agreement continuum — usually five points from strongly agree to strongly disagree with a neutral midpoint, sometimes four or seven; about half the statements are worded favourably and half unfavourably to control the habit of agreeing, the unfavourable items are reverse-scored, and the item scores are summed to a total, which is why it is called a summated rating scale.
A visual analogue scale is a horizontal line, conventionally 100 mm long, anchored by the two extremes — no pain at one end, the worst pain imaginable at the other — on which the respondent marks a point; the distance from the left end in millimetres is the score; it is quick and sensitive to change, and the numeric rating scale from 0 to 10 and the faces scale for children are its relatives.
A semantic differential (Osgood) rates a concept — 'the nurse', 'the ward' — on a series of seven-point scales between pairs of opposite adjectives — good and bad, kind and cruel, strong and weak, active and passive — and sums the scores by dimension (evaluation, potency, activity).
The other named scales are Guttman's cumulative scale, Thurstone's equal-appearing intervals, graphic rating scales, checklists and the Q-sort; the Glasgow Coma Scale, the Apgar score and the Braden scale are clinical rating scales whose totals are ordinal.
A scale is built from an item pool drawn from the literature and experts, judged for content validity by a panel, pretested, refined by item analysis and checked for internal consistency with Cronbach's alpha; the biases to design against are acquiescence (agreeing whatever the statement), social desirability, extreme responding and the middle-of-the-scale habit, and a standardised instrument is used with permission and, when translated, back-translated and re-validated.
A single Likert item is ordinal and takes the median and the rank tests; a summed Likert scale and a visual analogue scale are analysed as interval data — the mean, the standard deviation and the t-test — by convention.

Reliability — test-retest, inter-rater and internal consistency (Cronbach's alpha)

Reliability is the degree to which an instrument gives the same result on repeated use under the same conditions — its consistency, stability and precision — and it is the first property demanded of any research tool, because a measure that gives a different reading each time cannot be measuring anything. It is estimated as a coefficient between 0 and 1, and the kind of reliability that matters depends on what could vary: the time, the rater or the items.

Stability is test-retest reliability: the same instrument is given to the same people on two occasions and the two sets of scores are correlated, the coefficient being the correlation between them; it suits stable traits and not changing states such as pain, and the interval must be long enough to defeat memory and short enough to exclude real change.
Equivalence is inter-rater (inter-observer) reliability: two or more raters score the same subjects independently — two nurses staging the same twenty pressure injuries — and their agreement is measured, by percentage agreement, by Cohen's kappa for categorical ratings, which corrects for agreement by chance, or by the intraclass correlation for continuous ones; parallel-forms reliability correlates two equivalent versions of a test.
Internal consistency (homogeneity) asks whether the items of a multi-item scale all measure the same attribute — Cronbach's alpha for Likert-type items, with 0.70 taken as acceptable, 0.80 as good for research and 0.90 for a scale used to decide about an individual; Kuder–Richardson 20 for dichotomous items; and split-half reliability, which correlates the two halves of the scale and corrects with the Spearman–Brown formula.
A coefficient of 0.80 means that 80 per cent of the variance in the scores is true score and 20 per cent is error; an observed score is the true score plus error, random error lowers reliability and systematic error lowers validity, and reliability belongs to the scores obtained in a particular sample rather than to the instrument in the abstract.
Reliability is necessary for validity but not sufficient: an instrument can be reliable without being valid — a weighing scale that always reads two kilograms high — but it cannot be valid without being reliable, which the target analogy shows as shots clustered away from the bullseye against shots clustered on it.
Reliability is raised by clear instructions, standardised administration, training and calibration of the raters, more items (the Spearman–Brown prophecy), the removal of ambiguous items after item analysis and a pilot study that finds the problems first.

Validity — face, content, criterion and construct; pilot study

Validity is the degree to which an instrument measures what it is intended to measure and nothing else — a scale of anxiety that in fact measures depression, or a knowledge test that measures reading ability, may be reliable but is not valid. It is not established by a single coefficient but by accumulating evidence of several kinds, and a pilot study is where a new tool's validity and reliability are first checked before the main study.

Face validity is the weakest evidence — the tool appears, to respondents or lay judges, to measure what it claims — and it matters mainly because a tool that looks sensible gets better cooperation.
Content validity asks whether the items cover the whole domain of the concept in the right proportion, and it is judged by a panel of experts who rate the relevance of each item — infection-control experts judging a hand-hygiene questionnaire — summarised as a content validity index, with 0.80 taken as acceptable; a table of specifications does the same job for a knowledge test.
Criterion-related validity compares the tool's scores with an external criterion or gold standard: concurrent validity when the criterion is measured at the same time — a new anxiety scale against the State-Trait Anxiety Inventory — and predictive validity when the criterion comes later — an entrance test against final marks — each expressed as a validity coefficient, which is a correlation.
Construct validity asks whether the tool measures the theoretical construct itself, through the known-groups technique (patients with and without an anxiety disorder should score differently), convergent validity (agreement with other measures of the same construct), discriminant validity (no agreement with measures of different constructs) and factor analysis (the items cluster as the theory predicts); it is the most important kind and the hardest to establish.
Validity is specific to a population and a purpose — a scale validated in adults is not thereby valid in children, and a translated tool is back-translated and re-validated — and for a screening instrument criterion validity is expressed as sensitivity and specificity against the diagnosis; reliability is necessary for validity but never sufficient, random error lowering the first and systematic error the second.
A pilot study is a small-scale rehearsal of the whole study, often on about a tenth of the intended sample, that tests the tool's clarity and the time it takes, the procedure, the recruitment, the internal consistency and the feasibility, and finds the problems before the main study; pretesting is the tool part of it, and pilot data are not merged with the main results.

Free today: the 10-question daily quiz · the NORCET 11 recall paper · what ₹999 unlocks — once, no auto-renewal.