MBA Notes: Quantitative And Qualitative Research And Analysis

Key Topics:
- Decision-making tools
- Statistical methods
- Making forecasts
- Assessing cause and effect
- Soft studies
- Carrying out surveys

Finance, marketing, operations and HRM (human resource management) collect an inordinate amount of data and the IT (information technology) department processes it. However, it falls to the application of analysis techniques to interpret the data and explain its significance or otherwise. Bald information on its own is rarely of much use. If staff turnover goes up, customers start complaining and bad debts are on the rise, these facts on their own may tell you very little. Are these figures close to average, or should it be the mean or the weighted average that will reveal their true importance? Even if the figures are bad, you need to know if they are outside the range you might reasonably expect to occur in any event.

Generally, managers prefer to rely on quantitative methods for analysis and there are always plenty of numbers to be obtained. Figures are efficient, easy to manipulate and you should use them whenever you can. But there is also a rich seam of qualitative methods to get valuable information that you cannot obtain well with quantitative methods. These qualitative methods can be used to study human behaviour and more importantly changes in behaviour. Complex feelings and opinions, such as why employee morale is low, customers are complaining or shareholders dissatisfied, cannot be comprehensively captured by quantitative techniques. Using qualitative methods it is possible to study the variations of complex, human behaviour in context.

By connecting quantitative data to behaviour using qualitative methods, a process known as triangulation, you can add an extra dimension to your analysis with people's descriptions, feelings and actions.

In business schools these two methods of analysis are rarely taught together and are even less likely to be taught in the same department, though some marketing professors will manage joined-up analysis in areas such as surveys.

At Rotterdam School of Management, Erasmus University (www.rsm.nl), for example, in 'Quantitative Platform for Business' students investigate the qualitative as well as the quantitative methods available for problem solving within an organization. But EM Lyon (www.em-lyon.com/en) confines its teaching to 'Business Statistics' covering 'the essential quantitative skills that will be required of you throughout the programme'. MIT Sloan School of Management (http://mitsloan.mit.edu/mba/program/firstsem.php) has a teaching module, 'Data, Models, and Decision', in its first semester that 'Introduces students to the basic tools in using data to make informed management decisions'.

That seems heavy on quantitative analysis, covering probability, decision analysis, basic statistics, regression, simulation, linear and nonlinear optimization, and discrete optimization, but devoid of much qualitative teaching matter. But MIT does use cases, and examples drawn from marketing, finance, operations management, and other management functions, in teaching this subject.

Quantitative research and analysis
The purpose of quantitative research and analysis is to provide managers with the analytical tools necessary for making better management decisions. The subject, while not rocket science, requires a reasonable grasp of mathematical concepts. It is certainly one area that many attending business school find challenging. But as figures on their own are often of little help in either understanding the underlying facts or choosing between alternatives, some appreciation of probability, forecasting and statistical concepts is essential. It is an area where, with a modicum of application, an MBA can demonstrate skills that will make them stand out from the crowd.

Decision theory
Blaise Pascal (1623–62), the French mathematician and philosopher who with others laid the foundations for the theory of probability, is credited with inaugurating decision theory, or decision making under conditions of uncertainty. Until Pascal's time, the outcomes of events were considered to be largely in the hands of the gods, but he instigated a method for using mathematical analysis to evaluate the cost and residual value of various alternatives so as to be able to choose the best decision when all the relevant information is not available.

Decision trees
Decision trees are a visual as well as valuable way to organize data so as to help make a choice between several options with different chances of occurring and different results if they do occur. Trees (see Figure below) were first used in business in the 1960s but became seriously popular from 1970 onwards when algorithms were devised to generate decision trees and automatically reduce them to a manageable size.

FIGURE: Example decision tree

Making a decision tree requires these steps to be carried out initially, from which the diagram can be drawn:
- Establish all the alternatives.
- Estimate the financial consequences of each alternative.
- Assign the risk in terms of uncertainty allied with each alternative.

Figure above shows an example decision tree. The convention is that squares represent decisions and circles represent uncertain outcomes. In this example, the problem being decided on is whether to launch a new product or revamp an existing one. The uncertain outcomes are whether the result of the decision will be successful ($/£/€10 million profit), just ok ($/£/€5 million profit) or poor ($/£/€1 million). In the case of launching a new product there is, in the management's best estimate, a 10 per cent (0.1 in decimals) chance of success, a 40 per cent chance it will be ok and a 50 per cent chance it will result in poor sales. Multiplying the expected profit arising from each possible outcome by the probability of its occurring gives what is termed an 'expected value'. Adding up the expected values of all the possible outcomes for each decision suggests, in this case, that revamping an old product will produce the most profit.
The example is a very simple one and in practice decisions are much more complex. We may have intermediate decisions to make, such as should we invest heavily and bring the new product to market quickly, or should we spend money on test marketing. This will introduce more decisions and more uncertain outcomes represented by a growing number of 'nodes', the points at which new branches in the tree are formed.
If the outcomes of the decision under consideration are spread over several years, you should combine this analysis with the net present value of the monetary values concerned. (See Discounted Cash Flow)

Statistics
Statistics is the set of tools that we use to help us assess the truth or otherwise of something we observe. For example, if the last 10 phone calls a company received were all cancelling orders, does that signal that a business has a problem, or is that event within the bounds of possibility? If it is within the bounds of possibility, what are the odds that we could still be wrong and really have a problem? A further issue is that usually we can't easily examine the entire population, so we have to make inferences from samples and, unless those samples are representative of the population we are interested in and of sufficient size, we could still be very wrong in our interpretation of the evidence. 

Central tendency
The most common way statistics are considered is around a single figure that purports in some way to be representative of a population at large.

There are three principal ways of measuring tendency and these are the most often confused and frequently misrepresented set of numbers in the whole field of statistics.

To analyse anything in statistics you first need a 'data set' such as that in Table below.

TABLE: The selling prices of companies' products

The mean (or average)
This is the most common tendency measure and is used as a rough and ready check for many types of data. In the example above, adding up the prices – $/£/€105 and dividing by the number of products – 5, you arrive at a mean, or average, selling price of $/£/€21.

The median
The median is the value occurring at the centre of a data set. Recasting the figures in above Table puts product 4's selling price of $/£/€15 in that position, with two higher and two lower prices. The median comes into its own in situations where the outlying values in a data set are extreme, as they are in our example, where in fact most of the products sell for well below $/£/€21. In this case the median would be a better measure of the central tendency. You should always use the median when the distribution is skewed. You can use either the mean or the median when the population is symmetrical as they will give very similar results.

The mode
The mode is the observation in a data set appearing the most often; in this example it is $/£/€10. So if we were surveying a sample of the customers of the company in this example, we would expect more of them to say they were paying $/£/€10 for their products, though, as we know, the average price is $/£/€21.

Variability
As well as measuring how values cluster around a central value, to make full use of the data set we need to establish how much those values could vary.

The two most common methods employed are the following.

Range
The range is calculated as the maximum figure minus the minimum figure. In the example being used here, that is $/£/€40– $/£/€10 = $/£/€30. This figure gives us an idea of how dispersed the data is and so how meaningful, say, the average figure alone might be.

Standard deviation from the mean
This is a rather more complicated concept as you need first to grasp the central limit theorem, which states that the mean of a sample of a large population will approach 'normal' as the sample gets bigger. The most valuable feature here is that even quite small samples are normal. The bell curve, also called the Gaussian distribution, named after Johann Carl Friedrich Gauss (1777–1855), a German mathematician and scientist, shows how far values are distributed around a mean. The distribution, referred to as the standard deviation, is what makes it possible to state how accurate a sample is likely to be. When you hear that the results of opinion polls predicting elections based on samples as small as 1,000 are usually reliable within four percentage points, 19 times out of 20, you have a measure of how important. (You can get free tutorials on this and other aspects of statistics at Web Interface for Statistics Education (http://wise.cgu.edu).)

Figure below is a normal distribution that shows that 68.2 per cent of the observations of a normal population will be found within 1 standard deviation of the mean, 95.4 per cent within 2 standard deviations, and 99.6 per cent within 3 standard deviations. So almost 100 per cent of the observations will be observed in a span of six standard deviations, three below the mean and three above the mean. The standard deviation is an amount calculated from the values in the sample.

Use this calculator (www.easycalculation.com/statistics/standard-deviation.php) to work out the standard deviation by entering the numbers in your sample.

FIGURE: Normal distribution curve (bell) showing standard deviation


Forecasting
Sales drive much of a business's activities; it determines cash flow, stock levels, production capacity and ultimately how profitable or otherwise a business will be, so, unsurprisingly, much effort goes into attempting to predict future sales. A sales forecast is not the same as a sales objective. An objective is what you want to achieve and will shape a strategy to do so. A forecast is the most likely future outcome given what has happened in the past and the momentum that provides for the business.

The components of any forecast are made up of three components and to get an accurate forecast you need to decompose the historic data to better understand the impact of each on the end result:
- Underlying trend: This is the general direction, up, flat or down, over the longer term, showing the rate of change.
- Cyclical factors: These are the short-term influences that regularly superimpose themselves on the trend. For example, in the summer months you would expect sales of certain products, swimwear, ice creams and suntan lotion, for example, to be higher than, say, in the winter. Ski equipment would probably follow a reverse pattern.
- Random movements: These are irregular, random spikes up, or down, caused by unusual and unexplained factors.

Using averages
The simplest forecasting method is to assume that the future will be more or less the same as the recent past.

The two most common techniques that use this approach are:
- Moving average: This takes a series of data from the past, say the last six months' sales, adds them up, divides by the number of months and uses that figure as being the most likely forecast of what will happen in month 7. This method works well in a static, mature marketplace where change happens slowly, if at all.
- Weighted moving average: This method gives the most recent data more significance than the earlier data since it gives a better representation of current business conditions. So before adding up the series of data each figure is weighted by multiplying it by an increasingly higher factor as you get closer to the most recent data.

Exponential smoothing and advanced forecasting techniques
Exponential smoothing is a sophisticated averaging technique that gives exponentially decreasing weights as the data gets older and conversely more recent data is given relatively more weight in making the forecasting. Double and triple exponential smoothing can be used to help with different types of trend. More sophisticated still are Holt's and Brown's linear exponential smoothing and Box-Jenkins, named after two statisticians of those names, which applies autoregressive moving average models to find the best fit of a time series.

Fortunately, all an MBA needs to know is that these and other statistical forecasting methods exist. The choice of which is the best forecasting technique to use is usually down to trial and error. Various software programs will calculate the best-fitting forecast by applying each technique to the historic data you enter. Then wait and see what actually happens and use the technique that's forecast as closest to the actual outcome. Professor Hossein Arsham of the University of Baltimore (http://home.ubalt.edu/ntsbarsh/Business-stat/otherapplets/ForecaSmo.htm#rmenu) provides a useful tool that allows you to enter data and see how different forecasting techniques perform. Duke University's Fuqua School of Business, consistently ranked among the top 10 US business schools in every single functional area, provides this helpful link (www.duke.edu/~rnau/411home.htm) to all its lecture material on forecasting.

Causal relationships
Often, when looking at data sets it will be apparent that there is a relationship between certain factors. Look at Figure 11.3. It is a chart showing the monthly sales of barbeques and the average temperature in the preceding month for the past eight months.

FIGURE: Scatter diagram example

It's not too hard to see that there appears to be, as we might expect, a relationship between temperature and sales, in this case. By drawing the line that most accurately represents the slope, called the line of best fit, we can have a useful tool for estimating what sales might be next month, given the temperature that occurred this month (Figure below ).

FIGURE: Scatter diagram – the line of best fit

The example used is a simple one and the relationship obvious and strong. In real life there is likely to be much more data and it will be harder to see if there is a relationship between the 'independent variable', in this case temperature, and the 'dependent variable', sales volume. Fortunately, there is an algebraic formula known as 'linear regression' that will calculate the line of best fit for you.

There are then a couple of calculations needed to test if the relationship is strong (it can be strongly positive or even if strongly negative it will still be useful for predictive purposes) and significant. The tests are known as R-squared and the Students t-test, and all an MBA needs to know is that they exist and you can probably find the software to calculate them on your computer already.

Otherwise you can use Web-Enabled Scientific Services & Applications (www.wessa.net/slr.wasp) software, which covers almost every type of statistical calculation. The software is free online and provided through a joint research project with K.U.Leuven Association, a network of 13 institutions of higher education in Flanders.

For help in understanding these statistical techniques, read The Little Handbook of Statistical Practice by Gerard E Dallal of Tufts, available free online (http://gpvec.unl.edu/bcpms/files/Epi/mod3/Project%20Resources/LittleHandbookofStatisticalPracticeDallal.pdf). At Princeton's website (http://dss.princeton.edu/online_help/analysis/interpreting_regression.htm) you can find a tutorial and lecture notes on the subject as taught to its Master of International Business students.

Qualitative research and analysis
Qualitative research is a well-entrenched academic tradition in sociology, history, geography and anthropology; it is widely used in the medical and political fields. It has made much less of a mark in business, perhaps because of its image as a softer, more ethereal discipline. That situation is changing with the growing realization that while quantitative research can reveal what issues are important and even where they lie, it is of rather less use in understanding why they have come about or what to do about them.

Qualitative research comes into its own particularly when these are important factors:

- Complex issues: Quantitative methods are useful for separating out and measuring individual factors, say what percentage of customers are dissatisfied with a product or service and how many will defect. Qualitative methods can help get an understanding of the linkages between these factors and the competing tensions they arouse.
- Stakeholders' differences: Not everyone involved in an organization sees matters from the same perspective. Often the aggregation nature of quantitative methods makes it difficult to fully appreciate the position of less powerful stakeholders. Qualitative research gives individuals a voice in the analytical process.
- Significant recommendations: When the consequences of research are likely to result in recommendations with significant consequences, for example changing work patterns, shutting down a unit or altering pay and conditions, qualitative research allows attitudes and feelings towards potential courses of action to be explored, leading hopefully to a less contentious outcome.

Researchers used to quantitative analysis frequently dismiss qualitative research as 'unscientific' and 'anecdotal'. It certainly doesn't have to succumb to such criticism, as the array of tools used in qualitative research is large and the tools have a well-documented and rigorous methodology for their application.

Observation
The power of observation as a method of gathering data lies in the inconsistency between what people will say in an interview, or on a questionnaire, and what they actually do. It's not that people are necessarily lying, it's just that their capacity for self-deception is often high. Customers may feel foolish admitting they have difficulty finding their way around a shop and so would not record that fact. That doesn't mean that they don't have a problem and that a company would not gain valuable information from finding out about it.

So observations can give valuable insights into how things look from an outsider such as a customer, supplier or prospective employee. But such insights will only be representative of the time the researcher was observing and may not be indicative of the general level of service. They are often used to provide contextual information alongside some other research method.

Observations themselves generally come in one of two forms:

- Participating observation: This is where the observer takes part in at least some aspect of what is being assessed in order to get a better understanding of insider views and experiences. This, for example, could involve going through the whole procedure of making a purchase or using a service, rather than standing on the sidelines watching others. This is the methodology used in mystery shopping.
- Pure observation: Here the observer stays aloof from the situation under assessment so as not to influence it and so perhaps bias the findings.

The great difficulty in carrying out this type of research is being able to record observations accurately. Taking notes can be conspicuous and will almost certainly put those being observed on their guard.

Interviews
Talking and listening to people is the most basic and the most used method of conducting qualitative research. Qualitative interviews can take several forms and can be incorporated into triangulation methods (see below). These are the main interview types:

- Open-ended ad hoc conversations allowing interviewees to drive the discussion with minimum intervention by the interviewer; for example users of a product or service could be asked to give their feelings without being steered towards questions concerning satisfaction or dissatisfaction. This approach can throw up issues that have not been explored by the researcher.
- Open-ended interviews where the broad issues to be covered are stated, but the course of conversation is allowed to decide the order or ways in which questions are asked.
- Semi-structured interviews where the questions are largely planned in advance, with time left for issues that arise mainly as a result of the conversation itself.
- Qualitative questions built into structured surveys and questionnaires, where the main thrust is to gather quantitative data. For example, in an interview carried out to measure staff morale, questions such as 'how do you feel about the new pay scale?' could be interspersed with questions that gather quantitative data such as 'do you now feel: 10 per cent better off; the same; 10 per cent worse off?'.
- Cognitive interviews: These are used to test respondents' understanding of the meaning of questions or statements and are eventually to be used in questionnaires, user instructions and manuals, for example.

Qualitative interviews differ from surveys, for example in that they adhere less to a fixed set of questions but continually probe and cross-check information, building cumulatively on the knowledge gained from earlier answers. Nevertheless, interviewers at some point have to ask the questions that give them the specific data they need. Good interpersonal skills, sensitivity to the respondent, conducting the interviews at an appropriate time and place, using trained interviewers as well as having an appropriate sample are all vital to successful interviewing.

Focus groups
Focus groups are a form of multiple interview, with small groups of around 8 to 10 people selected with specific key attributes in mind: specific knowledge, experience or socioeconomic characteristics, for example. Participants are invited to attend informal discussion sessions of no more than two hours' duration on a particular topic, facilitated by someone knowledgeable about the issues involved, but tactful and firm enough to keep the group in order and on task. Often an incentive is offered for people to attend. The advantages of using a focus group over interviews include efficiency, as you can get 10 opinions in around twice the time it takes to conduct an interview; and by listening to other people's comments, often more ideas, opinions and experiences and insights can be gained. It is also easier to take notes of the discussion as this is expected and less threatening in a group situation. But, as with interviews, it relies on the views of a small sample and so is not truly representative of any body of opinion.

Three variations on focus groups are:

- Neighbourhood forum: These are structured, regular local meetings for local people to consult about issues of local importance. The term local can mean any characteristic that binds people together – young mothers, pensioners, train users.
- Citizens' juries: These involve a small sample of the public spending perhaps a day or two, at most, debating an issue in a quasi-judicial setting. They hear experts present the various sides to an argument, much as in court they would take evidence from witnesses. This approach is used by local government and police forces, but is also used by major local employers to gain insights into local community issues that they might impinge on.
- Brainstorming sessions: These are group meetings designed to stimulate creative thinking to solve a particular problem or address a single issue. There are three steps to brainstorming. Initially the group should try to generate as many ideas as possible, without criticism, welcoming unusual and even apparently impractical or impossible propositions. Next, the propositions should be reviewed briefly to either eliminate the ones universally agreed to be unworkable or to combine ideas to form better solutions. Finally, the handful of feasible solutions are discussed and ranked. All that is needed by way of materials are a flip chart, marker pens and Blu-tack to fix the ideas that have been generated visibly onto walls.


Case studies
A case study is a comprehensive and systematic study of a specific organization, event or subject. They can be written, on film or computer and are usually used where wide-ranging, complex questions have to be addressed and the findings used either as a focus for further discussion, for illustrative purposes or for training.

The case study needs an underlying question – how did the company go about closing down a particular unit, for example. It doesn't answer the question, rather it provides the 'reader' with information from interviews, company and public documents, observations and such sources, from which they can debate and form an opinion.

Triangulation
This is the rather pretentious name given to the combination of qualitative and quantitative research methods; a sensible process that allows researchers to get the best of both worlds. In fact the disciplines already overlap. Quantitative research produces numbers – the number of people questioned, for example, or how many times a particular feeling or opinion was mentioned in an interview. Qualitative methodology can be used to shed light on qualitative issues, such as how strongly people feel about a certain issue. Triangulation strengthens qualitative and quantitative analyses by combining insights from both.

Surveys
The most common research method that combines quantitative and qualitative processes is the survey. This is a near-ubiquitous tool used in organizations to get a handle on almost every aspect from measuring employee morale or assessing customer satisfaction to getting the views of almost any stakeholder group on almost any issue. MBAs will certainly have to know how to get surveys done and, if working in a small organization, they may well have to do it themselves.
Around half of all surveys are conducted face to face, considered best for tackling consumer markets. Next in popularity come telephone, e-mail and web surveys, which work well with companies and organizations. Postal surveys, once very popular, now account for less than 10 per cent of survey work.
See the guidelines for interviewing and questionnaire design.

Survey sample size
The size of the survey undertaken is also important. You frequently hear of political opinion polls taken on samples of 1,500–2,000 voters.

This is because the accuracy of your survey clearly increases with the size of sample, as the following table shows:

So, if on a sample size of 600 your survey showed that 40 per cent of women in the town drove cars, the true proportion would probably lie between 36 and 44 per cent. For small businesses, we usually recommend a minimum sample of 250 completed replies.

ResearchInfo.com (www.researchinfo.com/docs/websurveys/index.cfm) gives the basics of writing a program in order for you to use your own questionnaire on the internet.