Sage Research Methods

How do I choose between different research methods?

Pub. Date: 2011

Product: Sage Research Methods

DOI: https://doi.org/10.4135/9781412995559

Methods: Research questions, Research design, Philosophy of research

Keywords: cost benefit analysis, critical thinking, expertise, generalization, innovation, pragmatism, rational

choice, Skills development

Disciplines: Anthropology, Business and Management, Criminology and Criminal Justice, Communication

and Media Studies, Counseling and Psychotherapy, Economics, Education, Geography, Health, History,

Marketing, Nursing, Political Science and International Relations, Psychology, Social Policy and Public Policy,

Social Work, Sociology, Science

Access Date: December 1, 2023

Publishing Company: SAGE Publications Ltd.

City: London

Online ISBN: 9781412995559

© 2011 SAGE Publications Ltd. All Rights Reserved.

[MUSIC PLAYING] Stephen Gorard, thank you very much for talking to us. My first question today is,

how do I choose between different methods?

The standard answer would be on the basis of what it is you want to find out. It would be absurd to

pick a method in isolation from what you’re going to be trying to find out. So think of great questions.

We know, in reality, that there’s an intuitive process. You start with an idea, or a feel, or a theme that

you’re interested in. And as you discover more about it, in fact, what you want to do changes slightly.

So maybe your very last version of the research method questions will be towards the end of the

project. Of course, if you change completely, it’s a very different thing. But for the moment, assume

that you’re working on something. That you would pick the methods that would be most appropriate.

Now, that sounds basic and obvious. And yet, it’s quite revolutionary.

Because in Social Sciences, that very rarely happens. I’m going to give you an analogy. I’ve recently

moved house. And in moving house, I wanted to buy a new one. In the UK, that basically involves

getting a mortgage. And so I’ve been putting quite a lot of my money, or promising a lot of my money,

for 25 years, whatever, to a bank in order to buy a house.

So how do I go about buying a house? Well I went to visit it. I looked at the roots. I looked at the

neighborhood. I looked at the cost of the loan, and how much the repayments would be, how much

the insurance would be, and so on. And I synthesized all that naturally in my head. I came to the

decision that this house was, or was not, worth this amount of money. And in any skill or task that

you or I do we will tend to behave in the same way.

We will naturally collect all relevant data, synthesize them without concern– we might be better or

worse doing it– but that’s how we’ll do it. And come to a decision that this worth it. This is cost effec-

tive. This is, whatever it is. So when I’m doing Social Science research, I’m acting exactly the same

way. What I find odd is that the same people who, like you and me– if they were buying a house or

booking a holiday, or looking after the health of their loved ones, would make rational decisions on

the basis of all of the data available to them, naturally synthesized– would suddenly say, Oh, I’m not

that kind of researcher.

As a researcher, they would do the equivalent of saying, I don’t want to know the price of the house

because I don’t do numbers. Or I’m not going to visit the house because I don’t do that kind of work,

and I don’t do in-depth work. I think if you think of it in terms of that analogy of buying a house or

booking holiday, it’s an utterly ridiculous thing to do. So you don’t choose your methods beforehand,

you choose the methods once you have the questions.

Sage

(c) SAGE Publications Ltd., 2008

Sage Research Methods

Page 2 of 6 How do I choose between different research methods?

And once you have the questions, it’s fairly obvious. If you’re saying, has the extent of which children

from poor families clustered in particular areas or particular schools, or are dealt with by particular

hospitals, has that changed over time? You would have to collect numeric data about attendance at

hospitals and attendance in the schools.

If you want to know, perhaps it has changed, why has it changed? Then you’d presume you’d have to

do some observations, some historical documentary archival analysis. So the methods fall naturally,

almost like ripe fruit, from a decent question. So how do we come up with decent questions? OK.

That’s a very interesting one. And it’s one of those areas that is, I think, currently underdeveloped in

research methods training.

There are a range of different ways of doing it. Most people, perhaps when they start with a PhD,

perhaps just like yourself, what they will be doing is they’ll have and area that they’re really, really

concerned about. We often come with autobiographical reasons. I was particularly concerned with

injustice which I’d come across in my professional life.

One thing, though, is to refine that by reading the literature and so on. The danger of reading too

much literature is you may end up essentially just doing what everyone else does. The beauty of

coming at things fresh, as you do when you start your PhD, is that’s where innovation happens.

That’s good. So, once I come up with a research question, then I have to devise a research design.

And how am I to go about thinking about doing that? I guess you’re going to repeat your answer in

some ways, but– I could say more which is that if developing such questions is an immature area of

research training at the moment.

Research design is almost completely absent. It’s the Cinderella of the research methods arena.

There are courses being taught, results that’s being published, and websites available which purport

to be about research design, but are actually are not. So one of the key things for me about research

design is that it is independent of any methods of investigation.

Take a stereotone example like, longitudinal study? A longitudinal one where I’m going to pursue a

group of cases over a period of time. In my own field of education, I might be tracking some student

through school or college and looking at some changes over time. I could observe.

I could use smell, touch, taste, numeric text data about them. The design you’re following through

time is completely dependent of the methods of data collection and analysis. So that what I would

expect from the student would be the questions lead to the design, and also then to the methods.

But the design doesn’t determine methods.

So if you’re going to do a round of a controlled trial, you could that by looking to see whether people

Sage

(c) SAGE Publications Ltd., 2008

Sage Research Methods

Page 3 of 6 How do I choose between different research methods?

say they are happier at the end of the intervention than they were at the beginning. It doesn’t have

to be complex, statistical analysis or anything like that. That’s a very pragmatic approach to doing

research and coming up with research designs.

And I guess that contrasts, doesn’t it, with some people who seem to start with theory? And I wonder

whether you could talk a little bit about that relationship. Two reasons for that. One would be the field

in which I work. So education, like many in [INAUDIBLE] public policy is automatically an applied

area.

So if you want in-house studies, or crime, or housing or education, you tend to be coming across as

being [INAUDIBLE] a bit, genuine problems that real people have out there. So in a sense, theory,

as I think you mean it, is not really relevant. The government wants to know if we pay adults to go to

literacy evening classes, if we give them incentive, will that improve their attendance and attention?

It’s a perfectly pragmatic question. And we as investigators will investigate it. There’s not a huge

amount of theory. Of course, if it does or doesn’t work, we might then want to explain why incentives

do or don’t work. So that would be the first reason. The second reason is perhaps a bit more con-

troversial. I would say there’s lots of methods training and a lot of methods advice being given by

people who don’t do a lot of research.

There are people who become de facto methods experts, or deemed experts. And what I would ad-

vise all researchers to do is when someone wants to give you advice about research, is look at the

research they’ve done themselves. Because if they know how to do research, they will have done

it. And you’ll be able to see the success of their approach and what they’ve done. And I find it very

disquieting when there are people telling us how we should do research, or if it is possible to do re-

search in a particular way, when they haven’t actually tried it themselves.

And I’m not going to name names, but there are some very high profile people who are deemed to

be experts in method that tell us we must do it this way and that way and another. And I’ve never

seen an actual paper by them of any research at all. How does that come to be? Those are my two

parts to the question. The third angle on it would be, yes of course, the theory can be important, I’ve

investigated theory-driven approaches, but they don’t tend to become pragmatic.

Because the theory, if it’s a genuine theory, generates capital propositions such that, if you’re looking

at human capital theory, people will behave in a certain way and you can test that. If you test it and

they don’t behave in that way, then you’ve got some evidence supports your theory, but not the right

one.

So it’s not like you can have a theory which you stick to. So I’m not one of those people who has a

theory like a religion. And I think there are people who look at research like that. I don’t really under-

Sage

(c) SAGE Publications Ltd., 2008

Sage Research Methods

Page 4 of 6 How do I choose between different research methods?

stand how or why, don’t really want to understand how or why. But, you do get that venerating some

[INAUDIBLE] or something. And actually, and often, enjoying what they call the conceptual loose-

ness.

So they can make it be like an inkblot test. You can make it fit almost anything you want and call it a

lens. I don’t do any of that work, because out there people don’t want it. So theory is what we do at

the end or while we’re doing our research? Theory will generate propositions to be tested. Obviously,

everything is involved in theory.

But I suppose I try– I don’t want to go into the details– but I try [INAUDIBLE] three different types of

theory. So there’s the genuine theory which is to explore stuff, to explain it in a way that can help us.

Or help us to transport solutions from one area to another. Or to generalize findings from one country

or one sector to another.

And the big ticket theory– which is what I think of with religion, which is unaffected by data and I think

would be not noticed if we simply eliminated it. So by that, you mean formal theory, or formalized

theory? I mean when people are using theories as lenses. But they’re trying to use the cache that the

word theory has as an explanation of a certain event.

But not actually testing it, and not be concerned whether it works or not. But just using it, essentially

just like an inkblot test. You can say, well, I’m going to use the concept of happiness, or whatever it

is, to help me explain these interviews. I think what [INAUDIBLE] I’ve never seen anything where if

you eliminated that, it would make any difference to substantive findings in research.

It worries me, I suppose, that it’s a red herring. So if I’m at the start of my research career, then per-

haps the better thing I should do is to develop my own repertoire of skills in different forms of data

gathering and data analysis so that I can adapt to different research questions. Would that be your

advice?

There would be an infinite number of ways that you could research anything. And you can’t possibly

in your last year, or six months, whatever it is, to develop skills in all of them. I think you should– I

think we should all be hungry, to learn more. I mean, that’s what we do. We start learning when we

are finished. So I said yes, we should be looking for more. The main reason that I think we should

be looking wide at a repertoire of skills, is as consumers of research, the key thing is you’re going to

be reading and making critical judgments about existing research.

How can you do that if you’ve got no idea what people have done? I mean they don’t necessarily

make it easy for you by explaining it well. But even if they do, if you’ve got no idea what an ethnog-

raphy is, or you’ve got no idea how you can do a [INAUDIBLE] test or whatever it’s going to be, how

are you going to know whether or not what they’ve said makes sense?

Sage

(c) SAGE Publications Ltd., 2008

Sage Research Methods

Page 5 of 6 How do I choose between different research methods?

I can’t understand a researcher who would suggest that they were mono-method or duo-method. We

all have to have some understanding of all of them. Because otherwise, what you’re going to do with

the stuff that you were reading about which you don’t know? Are you going to say, I will reject all of

it because I don’t do that. Or I will accept all of it, which is really dangerous, isn’t it? So as a critical

consumer, I think we have to have all of that range, but when you come to do your more specific

project, you are bound to do that skills in a particular area, or one or two things.

But then you might have a new project. What I’d like to see from researchers is that they build on

the skills that they’ve got, but perhaps each new project, they’re going to invent a new method. I very

rarely use an off-the-shelf design or an off-the-shelf method for any project. I mean, I very rarely use

the same method twice because the situations don’t recur.

What you do is, you look at what’s available and, perhaps, redesign, or redevelop stuff, or combine

two or more things to make an approach belong to this particular question. [MUSIC PLAYING]

https://doi.org/10.4135/9781412995559

Sage

(c) SAGE Publications Ltd., 2008

Sage Research Methods

Page 6 of 6 How do I choose between different research methods?