### Reading a Journal Article; How Do We Know That the Results Represent a Statistically Significant Finding?

Basic Statistical
Analysis for Anesthesiologists

When we read the biomedical literature we depend, to some
extent, on the journal editors to guide us about the veracity of the
observations that are being presented. Beyond that, it is largely the responsibility
of the reader to determine whether the data that is being presented has been
appropriately derived and analyzed. We have spoken before about

__bias__that may have been introduced into the investigation. Some elements of bias may be apparent from the close reading of the article. Some may not be.
The statistical analysis of observations in any given study
allows the reader to determine whether the findings are significant based on
accepted values routinely used. This does not determine whether the finding is
clinically significant in the population that is evaluated or whether there is
statistical or clinical significance in any given population that a clinician
may be considering. As I have said before, very few single articles provide
evidence that should change current practice. The best evidence is obtained
from multiple randomized controlled trials that have been analyzed as a group.

Clinicians require a basic knowledge of the basis of a
statistical analysis in order to understand the language of medical research
and that will be the focus of this short piece.

The Language of Data
Evaluation

Medical studies, especially randomized controlled trials,
present data in many different ways but some of the information in a
statistical analysis is invariable:

1.
Data from a study is presented using two
distinct processes – a description of the sample that is being used for the
analysis, and a determination of the conclusions that can be drawn from the
data. Both of these aspects may be presented in tabular form

2.
As a general rule the characteristics of groups considered
in a study should be as close to each other as possible. A significant
difference between groups is a bias that may render the findings of the study
meaningless.

3.
Variables are, of course, anything that can be
measured or manipulated in a study for the purpose of demonstrating an effect.
These characteristics may be presented as nominal, ordinal, interval, and/or
ratio. Interval data may be presented as continuous or discrete.

4.
Randomization represents the attempt by the
investigator to assure that bias is not introduced into the study by choosing
patients that are intrinsically more or less likely to respond in a given way.

5.
Blinding attempts to remove any bias that is
produced by the investigator, the observer, or other clinicians that may change
the characteristics of care associated with the trial.

6.
Data that is presented in a continuous fashion
numerically has characteristics, which describe the central tendency of a group
of data points. This represents the tendency of an individual observation to be
clustered around a central point – the mean, median, and/or the mode. Only the
mode may be used with all types of data. The mean requires interval/ratio
measurements while the median ordinal may be used with interval/ratio or ordinal
data.

7.
In concert with a description of the central
tendency or data is a description of the variability of the entire data set
around a central value. This variability can be defined in several different
ways, the most common being the standard deviation. Other methods may be noted
based on the characteristics of the data set – the range, the variance, or the
standard error of the mean.

8.
The appropriate use of these characteristics of
the data, that is descriptions of the central tendency and the variability of
the data are important in the interpretation of the data that is presented. For a clinician, interpreting a data set with
an indication of high variability may suggest difficulty in understanding the
true clinical significance of the study.

(P<.05)

There is some probability that in
examining the data derived from clinical trials we arrive at a value that is
only indicative of chance. That is, the value in reality does not represent a
true scientific finding but a random observation. How do we determine whether the
finding is statistically significant or not?

First, investigators usually
presume that there is no difference between an experimental group and a control
group. This is termed the null hypothesis. In analyzing data we must prove that
the individual values were drawn at random from the same population. When this
requirement is met, we use statistical tests to determine the likelihood that
the value that we have derived represents a statistically significant finding,
that is that it is not representative of chance alone.

By using statistical analysis, if
the probability is determined to be less than one in twenty, giving a P
(probability) value of less than .05 (P<.05) then the null hypothesis is
rejected and one can conclude that there is indeed a difference in the group.
That said, even with careful analysis of data that is randomized, blinded, and
analyzed, there always exists a certain probability of accepting or rejecting
the null hypothesis in error, either a type 1 or a type 2 error.

__Type One Error__represents the probability of incorrectly rejecting the null hypothesis when, in fact the null hypothesis is true. (Remember that the null hypothesis assumes that there is no difference between two groups.) This value, the probability of making this mistake is termed the alpha and is usually not more than 5% (Alpha =0.05).

__Type Two Error__is the probability that one fails to reject a null hypothesis that is in fact false. This implies that there is a real difference in the experimental and the control group but analysis of the data did not demonstrate this difference. Obviously, if the original analysis rejects the null hypothesis, then no Type Two Error can be made.

Randomization, blinding, measures
of central tendency, and of variation, variables,
demonstration of the probability that the results are not solely due to chance,
and errors in this determination; these are basic concepts that are required in
determining whether the journal article that you are reading meets the basic
standard of scientific investigation. We will talk about these elements in
detail and other, more complete, offerings.

Rae Brown, M.D.

January 22, 2015