You choose a statistical test by looking at what the objective compares, the variable type, the number of groups and whether the data are normal. Once you answer these four, in most theses only one test is left.
Problem comes when scholars fix the test first. A senior used it, or the guide mentioned it once, and then the objective is pulled and stretched to suit that test. Please do it the other way. Take one objective at a time and let that objective decide the test.
Start from the objective before opening the software
Take your synopsis and write down the objectives, one in each line. Next to each one, note the main verb it uses. Mostly a thesis objective will do one of these four jobs.
- Describe. For example, what is the level of job stress among nurses in Madurai?
- Compare. Is job stress different in government and private hospitals?
- Relate. Does job stress go up or down with years of service?
- Predict. Which factors explain the job stress of these nurses?
A describe objective needs no test. Means, percentages and standard deviations are enough for it. For a compare objective you will use a t-test, ANOVA or chi-square. A relate objective needs correlation, and a predict objective needs regression. So you know the family of test for each objective even before SPSS is opened.
Some objectives don’t match any of the four verbs. Then many times the objective is too vague to test. Take “To study the role of mobile banking among rural women”. It can be read as a comparison, or a correlation, or a regression, and nobody can say which. Rewrite it with a clear verb, and choosing the test becomes easy after that.
Know what kind of variable you measured
The next question is about the outcome variable you measured. Books give many names, but for thesis work three kinds will do.
In categories we only have names, and one name is not higher or lower than the other. Gender, district, type of school and yes/no answers are examples. Ordered categories do have an order, but the gaps between them are not equal. Education level is one, and a single Likert item from “strongly disagree” to “strongly agree” is another. Scale variables are numbers where the gap has a meaning, like age, income, marks, blood pressure or a total score added up from many items.
See, nobody can take the average of a district. The t-test is working with means, and means are there only for scale outcomes, so that is where you use it. If the outcome is a category, you have to count and compare proportions, and in most cases that means chi-square.
In SPSS, open Variable View and look at the Measure column. In many files we get from scholars, this column is never touched, and every variable is still showing the default “Nominal” or “Scale”. Then the menus quietly show options that don’t suit the data.
Count the groups, and check whether they’re the same people
For a comparison, first count the groups. Two groups need one type of test and three or more groups need a different one. Male and female teachers make two groups. Teachers from five districts make five groups.
After that, see whether the groups are different people or the same people measured two times. Say 60 students write a test before a training programme and again after it. That is paired data. Two separate classes of 30 students each are independent. In the SPSS menu the paired-samples t-test and the independent-samples t-test look almost the same, but they answer different questions, and if you pick the wrong one the examiner will ask about it in the viva-voce.
You can check it like this. If each score in group A can be joined to one particular score in group B, because both belong to the same person, your data are paired.
| What the objective is asking | Outcome variable | Groups or predictors | Usual test | If the data are not normal |
|---|---|---|---|---|
| Is the average of one group different from a fixed value? | Scale | One group | One-sample t-test | Wilcoxon signed-rank (one sample) |
| Are two separate groups different? | Scale | Two independent groups | Independent-samples t-test | Mann–Whitney U |
| Did the same people’s scores change? | Scale | Same people, measured twice | Paired-samples t-test | Wilcoxon signed-rank (paired) |
| Is there a difference among 3+ groups? | Scale | 3+ independent groups | One-way ANOVA | Kruskal–Wallis H |
| Are two category variables related? | Categories | One more category variable | Chi-square (independence) | Fisher’s exact, when counts are small |
| Do two measures go up and down together? | Scale | One more scale variable | Pearson correlation | Spearman’s rank correlation |
| Which factors predict the outcome? | Scale | Several predictors | Multiple linear regression | Transform the data, or use a robust method |
| Which factors predict a yes/no outcome? | Two categories | Several predictors | Binary logistic regression | Not needed, as it has no normality assumption |
Check whether the data are normal
The t-test, ANOVA, Pearson correlation and linear regression all expect the data to be roughly normal. Check this for every group before running any of them.
In SPSS, go to Analyze → Descriptive Statistics → Explore. Put the outcome in the Dependent List and the group variable in the Factor List. Click the Plots button there and put a tick on “Normality plots with tests” before you press OK. The output show the Shapiro–Wilk test and a Q–Q plot for each group.
Don’t decide only by Shapiro–Wilk. With 300 or 400 responses it will flag very small departures that are not important. With 20 responses it can miss big ones. So see the Q–Q plot and the histogram also. Many fields accept skewness and kurtosis values between about −2 and +2, but check what your field and your guide expect.
Two points confuse many scholars. In regression, normality is checked on the residuals and not on each variable separately. In the regression dialog, go to Plots, tick the histogram of standardised residuals and check that. And when each group has 30 or more cases, the t-test and ANOVA handle moderate skew quite well, so a significant Shapiro–Wilk by itself is not a reason to change the test.
When the data are roughly normal, go with the usual test in the table above. If the data are clearly skewed, or any group has fewer than about 15 cases, take the non-parametric test from the same table. It answers almost the same question with less assumptions. Whatever you finally run, mention it in the method chapter with the reason, even two lines is enough for this.
What to do with Likert scale data
This is one of the most common doubt scholars bring to us, and even guides don’t agree on it.
Take a single Likert item like “I feel supported by my head of department”. It is ordinal, because how far “agree” is from “strongly agree” is not known to anyone. For single items, medians, percentages, chi-square or non-parametric tests are the safer way.
Most theses, though, don’t analyse items one by one. Five to ten items are added or averaged to get one score for each construct, for example job satisfaction or perceived usefulness. These total scores behave much more like scale data, and many published studies use t-tests, ANOVA and regression on them. Before using the total, check the reliability of each scale with Cronbach’s alpha.
Whichever way you go, take the decision once and follow it in the whole thesis. We have seen examiners write a remark when chapter four used ANOVA on the scores and chapter five called the same scores ordinal.
Tests for relationships and prediction
If an objective asks whether two scale variables move together, use a correlation. Pearson’s correlation is for when both variables are normal. When either one is not normal, or is ordinal, use Spearman’s correlation. Report the coefficient, its direction and the p-value. With 400 people a correlation of 0.25 can come out significant and still be weak, so mention the strength also and don’t stop at significance.
Objectives with words like “factors influencing” or “determinants of” mostly go to regression. When the outcome is a scale score, we run multiple linear regression, and when it is only yes or no, binary logistic regression is the one to take. Look at the VIF values in the regression output to check multicollinearity, because predictors that overlap heavily make each other look unimportant.
In some theses the model is bigger, with one outcome feeding into the next one, and each construct is measured using many items. For such models plain regression may not be enough. Then structural equation modelling in AMOS or SmartPLS is used, and that needs a bigger sample. Work out the numbers with our sample size calculator before data collection starts, since changing the sample afterwards is very difficult.
How to justify the choice in your thesis
Examiners rarely ask a scholar to write out a formula. Mostly they ask why you used this particular test. So in the method chapter, give the reasoning as a small table with the objective, the variables, the test and the assumption you checked.
A sentence like this one works well. “The second objective compares job stress across three hospital types. As stress scores were not normally distributed (Shapiro–Wilk p < .05 in two groups), the Kruskal–Wallis test was used instead of one-way ANOVA.” The examiner reads this and understands why you moved away from ANOVA, without asking you.
After that, report every result in the same order each time. Give the test, the statistic, the degrees of freedom when there are any, the p-value and an effect size. For the format, follow what your university says, or what your guide is asking for. Which format you use is not a big issue for most examiners, but chapter four and chapter five should report results in the same pattern.
Questions scholars ask
Can I use ANOVA for two groups?
Yes, you can. With two groups ANOVA gives the same result as the independent-samples t-test, because F is equal to t squared. Readers expect a t-test for two groups though, so use the t-test unless your guide wants ANOVA everywhere for consistency.
My sample is 30. Is that too small for a t-test?
Not by default. A t-test can run fine with 30, as long as the data in each group are roughly normal. If the data are not normal, or one group is much bigger than the other, go for the non-parametric option and give the reason in your thesis.
Do I need a post-hoc test after ANOVA?
Yes, if the ANOVA result is significant and you have three groups or more. The ANOVA table tells only that the groups are not all same, it will not say which ones. For that we add a post-hoc test. Tukey is the usual choice, and if Levene’s test shows the variances are unequal, take Games–Howell.
What if my p-value is 0.06?
Report the value as 0.06 itself, and at the 0.05 level it is not significant, so say so. In the discussion you are free to write about the effect size. Mention also that with more respondents the result might have come out different. We have seen scholars running the same data again and again with different tests till something comes below 0.05. Please avoid this. Examiners who have read many theses get suspicious when all the p-values are sitting just under 0.05, and it is not honest work also.
Is chi-square still valid if some cells have small counts?
When more than about one fifth of the expected counts are below 5, chi-square becomes unreliable. SPSS prints a note on this just below the table. In a 2 × 2 table, use Fisher’s exact test in its place. Or, where it makes sense, combine some categories.