| age | treatment_response | eye_color |
|---|---|---|
| 49 | Yes | Black |
| 48 | Yes | Hazel |
| 34 | No | Hazel |
| 34 | Yes | Blue |
| 17 | No | Black |
| 52 | Yes | Blue |
| 54 | Yes | Hazel |
| 43 | Yes | Black |
| 63 | Yes | Hazel |
| 46 | Yes | Brown |
3 Specifying Your Question
Being a data scientist is a little different from being a traditional scientist in that the order in which we do things can be different. In a traditional science world, we often formulate questions and hypotheses, design a study to collect data, engage in data collection, and then analyze the data to produce results. In a data science world, we often are handed a dataset that was collected by someone else and are asked to answer a question that is likely different from the one for which the data were originally collected. Both traditional scientists and data scientists have significant overlap in their day-to-day work, but the idea of being “handed a dataset” and then asked to answer a question is arguably a hallmark of data science work.
Before launching head first into a data science investigation, one of the most important things that you can think about is the specific question you are trying to ask. This can be a surprisingly tricky task, especially when our understanding of the thing we are trying to study is poor. Understanding the nature of the question you are asking is important because
The question can determine whether an analysis is even possible, given the data that you have at hand;
The specifics of a question can determine what type of analysis you will do and what type of models or statistical tools you can employ;
Having a specific question can focus your efforts on something of interest to you or to your audience and can prevent time-wasting side quests and fishing expeditions into the data;
The type of question you ask can determine what kinds of conclusions you can draw from the data.
Often, the question that you intend to ask can initially be stated in general terms. For example, we might want to know “How does the environment affect health?” This is a general question that might be of significant interest to some people. A more specific version of this question might be “How does outdoor air pollution affect hospitalizations for respiratory disease?” In practice, we often iterate on the question based on what data we have available and what kinds of analyses are possible. As you are developing the question you intend to ask, it can be useful to determine what type of question you are asking, as the type of question may lead you to a certain type of analysis. In the next section, we describe the six type of questions we often ask in a data analyses.
3.1 Types of Questions
Understanding the type of question you are asking may be the most fundamental step you can take to ensure that, in the end, your interpretation of any data analysis results is appropriate. In a paper titled “What Is The Question?”, six types of questions are described as common questions that we ask in data analyses (Leek and Peng 2015). In this section we will go over the six types of questions and give some examples of each of them.
3.1.1 Descriptive
A descriptive question is one that seeks to summarize a characteristic of a dataset. The goal is not to make inferences or predictions about unknown phenomena in the world. Rather, the aim is to make factual statements about things in the dataset. Descriptive analyses do not have any uncertainty because they are simply summarizing aspects of the information in the dataset. A key aspect of descriptive questions is that they are not questions about the data-generating process. Rather, descriptive questions are questions about the observed data that we have in front of us.
For example, in a medical study, we might want to determine the proportion of patients in the study who were male. This can be calculated directly from the data, and once we do the calculation we know the answer. If this study were collecting information about the relationship between people’s diets and their health, then as an initial check of the data and before doing any other analysis, we might want to know the mean number of servings of fresh fruits and vegetables eaten per day, or the frequency of viral illnesses each person had in a given period of time. These quantities can all be calculated directly, assuming the proper variables are in the dataset in the first place. There is no need for interpretation of the result itself as the result is a fact, an attribute of the set of data that you are working with.
Descriptive questions are the simplest type of question and can almost always be asked and answered. The only real challenge to asking a descriptive question is the availability of the data. If we do not have data on fruit and vegetable consumption in a health study, for example, we cannot ask what is the mean number of servings of fruits and vegetables that are eaten by the study subjects. But if data are available and are in a suitable form for analysis, then we can answer a descriptive question.
Although descriptive questions are simple, they are critically important because they allow us to learn about our dataset. Furthermore, if a dataset is very large, a descriptive question can be important in and of itself. For example, many companies will want to know their quarterly income based on all their sales and expenses for the quarter. Assuming careful records are kept by the company and all the data are available, the question of how much income was generated is a descriptive question about the company.
3.1.2 Exploratory
An exploratory question is one in which the goal is to analyze the data to see if there are patterns, trends, or relationships between variables. In addressing an exploratory question, we often say that we are conducting an exploratory analysis. These types of analyses are sometimes called “hypothesis-generating” analyses because you are looking for patterns in the data that would support proposing or generating a hypothesis. With exploratory questions, we are starting to ask questions about the underlying data-generating process and moving beyond simply asking questionst of the observed data.
For example, if you had a general thought that diet was linked somehow to viral illnesses, you might explore this idea by examining relationships between a range of dietary factors and viral illnesses. If you found in your exploratory analysis that individuals who ate at least 5 servings a day of fresh fruit and vegetables had fewer viral illnesses than those who ate fewer than 5 servings of fruits and vegetables, you might go on to propose in a future study to test the hypothesis that among adults, eating fresh fruits and vegetables (at least 5 servings a day) is associated with fewer viral illnesses per year.
Exploratory questions tend to focus on visualizations and on applying techniques that allow us to learn a lot about what is going on in a dataset. We often want to understand better the distributions of individual variables and the bivariate relationships between variables. Typically, there is not much complex modeling involved, as the focus is to develop interpretable results from the data. We often aim to identify potentially problematic data points or observations that may have been a result of a problem in the data collection process. Conclusions drawn from an exploratory analysis are usually descriptive in nature, so the statements that we make are still confined to factual statements about the dataset. In this way, exploratory analyses are similar to descriptive analyses.
A key point about exploratory analyses is that you cannot propose a hypothesis and test the hypothesis using the same dataset. The reason, in general, is that if you discover a relationship in a dataset, you are much more likely to “confirm” the existence of that relationship if you re-use the same dataset. Ideally, you would use one dataset to explore the relationships between the variables and then use a completely separate independent dataset to develop evidence for or against that hypothesis. In some cases, you may be able to split a dataset into two independent parts, where you conduct exploratory analysis in one part and then test your hypothesis in the other part. This is a strategy that is commonly used when addressing prediction questions and we will cover it in greater detail in Chapter 7.
3.1.3 Inferential
Inferential questions are fundamentally about asking questions about the underlying state of the world, i.e. the data-generating process. With inferential questions, we are asking about things that are beyond the specific numbers in our dataset, which makes inferential questions some of the most difficult to answer. Suppose we wanted to know how often people in the city of Austin, Texas drink alcoholic beverages. We could gather a group of 50 people and ask them “How many times a week do you drink an alcoholic beverage?” The statistic we might compute would be the mean number of times across all 50 people. This statistic would answer the question for the 50 people that we asked. However, it would not directly answer the question about how often people in the entire city of Austin, Texas drink alcoholic beverages. The reason, simply put, is that we did not ask this question to everyone who lives in the city of Austin. We only asked 50 people. The key problem when it comes to inferential questions is whether we can use the data that we collected from the 50 people to make a statement about the entire city.
With inferential questions we have to make a distinction between two key concepts:
Population: The population is the group of people or items about which we want to answer a question. In the example above, the population was all people living in the city of Austin, Texas. Typically, it is too difficult or even impossible to collect data from the entire population.
Sample/Dataset: The sample (or the dataset) is the specific set of measurements that we have collected from the population. In the example above, the sample is the 50 people to whom we asked the question about alcoholic beverages. Ideally, this sample is representative of the population, in that it was collected in a systematic manner so that the properties of the sample mirror the properties of the population.
One place where the distinction between population and sample routinely comes up is with political polling. Polling is the act of asking questions about voting preferences to a small sample of people and making inferences about a much larger population of people. For a given election, a poll question might be along the lines of “If the election were held today, which candidate would you vote for?” A large poll might sample a few thousand people (usually via phone calls). For a national election, the population of interest might be the entire voting population, which could be many millions of people (depending on the country). For that sample of a few thousand people, we know what percentage would vote for which candidate. This much is certain. But the inferential question is what percentage of the entire voting population will vote for each candidate. Using the small sample to draw a conclusion about this much larger population is the act of inference, and much of the field of statistics is dedicated to understanding under what conditions it would be reasonable to make such an inference.
An example of a sample that is designed to answer questions about the entire United States population is the National Health and Nutrition Examination Survey (NHANES). NHANES is a meticulously designed study whose purpose is to measure the health of the U.S. population in a variety of ways. It is typically conducted over a few years using mobile data collection centers all over the country. Although the survey is large (the 2017–2018 version collected data from over 8,000 people), it is obviously still much smaller than the entire U.S. population. However, because of its careful design and execution, data from the survey have been used in many scientific publications researching the health of the U.S. population.
3.1.4 Predictive
Predictive questions aim at trying to understand the relationship between a specific measure or variable, usually referred to as an outcome, and a set of other variables called predictors. Usually, the goal is to accurately predict the outcome variable using a large number of predictors, without too much thought regarding why certain variables might be related to each other. For example, a scientist might be interested in knowing what predicts whether a person will respond well to a specific medical treatment. The scientist might collect data on different people and their responses to the treatment, and it may turn out that people who are older respond better to the treatment than people who are younger. In this case, we might say that age is a good predictor of the response to treatment. We do not yet know why this is true, but the data indicate that there is a correlation.
Sometimes, knowing that a correlation exists is sufficient because it allows you to make statements about unobserved future events. For example, if a young patient were to walk into a doctor’s office and the doctor prescribed the medical treatment referred to above, the doctor might warn the patient that the treatment is less successful in younger patients and that expectations should be set appropriately. The doctor is making a statement about a future event, i.e., whether the treatment is successful in the current patient. Knowing the exact mechanism of how age affects treatment response is not necessary to make this statement with some accuracy. The point is that the predictive information can be used to make an informed decision, in this case about whether to apply the treatment.
Usually, an outcome is not well-predicted by a single variable like age, and so we must consider a large set of variables to develop a good prediction. For example, in cancer research, one might use the expression of thousands of different genes to predict whether a person will develop a specific type of cancer or not. Predictive questions are usually addressed with predictive models, such as linear regression or logistic regression. Machine learning approaches also play a big role in answering predictive questions. All of these approaches search for some combination of predictor variables that best predicts the outcome. The quality of the prediction that a model makes is described through a variety of metrics that will be covered in detail in Chapter 7.
Predictive questions have something in common with inferential questions in that in both cases we are often trying to understand something that is outside of our dataset. With predictive questions we are trying to understand what an (unobserved) outcome will be given a set of predictor values. It is possible that we will eventually observe that outcome, and then we can compare our prediction to reality. But in many cases, we will never observe that outcome, and the predictions from our models will be used to make decisions about what to do in a given situation.
3.1.4.1 Regression vs. Classification
For prediction questions, we often make a distinction between regression and classification problems. This distinction relates to what type of outcome variable we want to predict.
With regression problems, we are generally trying to predict a continuous outcome variable. In general, with regression problems the quantity that we try to model is the mean response for a given set of predictors. In other words, we want to know given a specific set of values for the predictor variables, what is the expected response? This expected response is sometimes referred to as the conditional mean function, because we are interested in the mean response conditional on the predictor variables taking on a specific set of values.
One example that we might consider a regression problem is predicting blood pressure. Blood pressure is measured in millimeters of Mercury (mmHg). Blood pressure is often broken down into systolic and diastolic blood pressure and expressed as a ratio of the two. Systolic blood pressure can range from less than 120 (normal) to greater than 140 (hypertensive), while diastolic blood pressure can range from less than 80 (normal) to greater than 90 (hypertensive). Clearly, the range of blood pressure in humans is limited to a specific interval, but within the allowable range the measurement of blood pressure can take on a large number of values. In addition, values of blood pressure can be thought of as being equally spaced throughout the range. Therefore, it might be reasonable to think of predicting blood pressure as a continuous regression problem. Ultimately, whether a problem is a regression problem or some other category of problem can often be a matter of approximation. If an outcome can reasonably be considered continuous and its values are equally spaced, then regression modeling is often a useful approach.
Classification problems generally involve trying to predict a categorical outcome variable. The most common classification problem is predicting a binary outcome. For example, physicians may be interested in predicting whether a patient will respond to a certain drug. In this case the outcome variable might be “respond to drug” vs. “did not respond to drug.” In other situations the outcome might be multi-category. With a shopping web site, clothing retailers might be interested in predicting whether a potential customer will purchase shirt, a pair of shoes, or a hat. This would be an example of a 3-category outcome.
3.1.5 Causal
Inferential questions focus on the associations that we observe in a sample or dataset and try to make statements about those associations in a population. For example, a study focused on an inferential question might tell us that people who eat a certain type of foods tend to have fewer viral illnesses. However, the answer to this question does not tell us if eating these foods causes a reduction in the number of viral illnesses, which is what we would call a causal question. A causal question asks whether changing one factor will cause a change in another factor, on average, in a population.
At this point, many people do not see the distinction between inferential questions and causal questions. If we observe that people who eat more fruits and vegetables have fewer viral illnesses, doesn’t that then mean that eating fruits and vegetables causes us to have fewer viral illnesses? In general, the answer is no. The reason is because there may be other factors at play that can make us observe something whose underlying mechanism is different from what we expect. For example, it could be being “health conscious” causes one to eat more fruits and vegetables while also causing us to have fewer viral illnesses. This “health consciousness” can induce a correlation in the data between eating fruits and vegetables and viral illnesses. But it does not mean that the fruits and vegetables caused the viral illnesses to go down. Rather, it was the “health consciousness” that caused both factors to change. We will go deeper into the specifics of this phenomenon (known as confounding) later in Chapter 10.
Answering causal questions is very difficult, especially when we have a poor understanding of the underlying science or the processes that generate the data. Sometimes the design of the data collection allows for the answering of a causal question. An example of this would be data collected in the context of a randomized controlled trial, in which people are randomly assigned to eat a diet high in fresh fruits and vegetables or one that was low in fresh fruits and vegetables. In other instances, even if your data are not from a randomized trial, you can take an analytic approach designed to answer a causal question.
Part of understanding whether it is possible to answer a causal question involves understanding what assumptions do we need to make and whether those assumptions are likely to be true in the real world. We often have to make untestable assumptions about the real world in order to draw causal conclusions, and those assumptions can weaken a conclusion if they are thought to be false. We will discuss in depth the specific nature of these assumptions in Chapter 10.
3.1.6 Mechanistic
From a scientific perspective, even if we can answer a causal question, we may ultimately want to know how one factor causes another factor to change. If we are studying diet and viral illnesses, a causal question asks whether increasing one’s fruit and vegetable consumption reduces the risk of a viral illness. However, the causal question does not necessarily ask why the kind of diet reduces viral illnesses in people. That kind of question is really a mechanistic question. A mechanistic question asks about the underlying mechanisms that cause one factor to change another factor. If we can develop an understanding of these mechanisms by asking mechanistic questions, we have a powerful basis to make predictions about what might happen in other circumstances.
For example, if I flip a light switch in a lecture hall, I can predict that the light will go on. I can make this prediction, not because I have collected data on thousands of light switches and light bulbs, but because I have a basic understanding of how electricity works. If I go on to cut the wire that connects the light switch to the light bulb, I can further predict that flipping the switch will fail to turn the bulb on. Hence, having a mechanistic understanding here allows me to make predictions about the light switch-light bulb system under different conditions.
In general, developing a mechanistic understanding can be exceedingly difficult when the system being studied is very complicated. For example, social scientists are often studying the dynamics of large populations of human beings. Having a mechanistic understanding of how each person interacts with every other person can be very challenging. Similarly, the human biological system is very complicated and biologists have to carefully design their experiments to obtain an understanding of basic processes within the body. However, having such a mechanistic understanding of human biology could unlock the key to treating a variety of diseases that currently have no cure.
One way to think about mechanistic questions is as a series or collection of causal questions. For example, we may learn that some factor \(X\) causes \(Y\) to change (e.g. increasing \(X\) causes \(Y\) to increase). We may later discover that the mechanism behind this process is that \(X\) causes a third factor \(Z\) to change, and then \(Z\) causes \(Y\) to change. In this case, \(Z\) is said to mediate the causal relationship between \(X\) and \(Y\). Another way to think of this scenario is that \(Z\) is part of the mechanism that explains how \(X\) influences \(Y\). To give a concrete example, \(X\) could be “cigarette smoking”, \(Y\) could be “heart disease”, and \(Z\) could be “systemic inflammation in the body”. Then we might conclude that systemic inflammation in the body is the mechanism by which cigarette smoking causes heart disease.
3.1.7 Additional Notes
There are a few additional points about the types of questions that are important to note. First, by necessity, many data analyses answer multiple types of questions. For example, if a data analysis aims to answer an inferential question, descriptive and exploratory questions must also be answered during the process of answering the inferential question. To continue our example of diet and viral illnesses, you would not jump straight to a statistical model of the relationship between a diet high in fresh fruits and vegetables and the number of viral illnesses without having determined the frequency of this type of diet and viral illnesses and their relationship to one another in this sample.
A second point is that the type of question you ask is determined in part by the data available to you (unless you plan to conduct a study and collect the data needed to do the analysis). For example, you may want to ask a causal question about diet and viral illnesses to know whether eating a diet high in fresh fruits and vegetables causes a decrease in the number of viral illnesses, and the best type of data to answer this causal question is one in which people’s diets change from one that is high in fresh fruits and vegetables to one that is not, or vice versa. If this type of dataset does not exist, then the best you may be able to do is either apply causal analysis methods to observational data or instead answer an inferential question about diet and viral illnesses.
3.2 Variable Roles
Variables in a dataset can play multiple different roles depending on the question that is being asked of the dataset. Some key roles that we will see often are outcome, predictor, and confounder.
3.2.1 Outcome
An outcome or response variable can be thought of as the final end point or goal of an analysis, something that we hope to predict or whose variation we hope to explain. In a health study, an outcome might be the occurrence of a given disease or the value of a biomarker like blood pressure. In that case, we might seek to explain why some people get a certain disease and some people do not. Or we might want to explain why some people have higher blood pressure than other people. In a business setting, we might want to know about the sales of a specific product and why sometimes sales are high and sometimes sales are low.
One thing that is common with outcome variables is that there is unexplained variation that we want to ultimately explain. We want to know why this variable is going up or down. The way that we “explain” variation in outcome variables is by relating that variation to other variables of interest called predictors. If we find, for example, that sales of a product tend to be high in the winter and low in the summer, then we could say that the season “explains” the variation in sales for that product.
When visualizing data with plots like a scatter plot, we often place outcome variables on the y-axis of the plot. By convention, we tend to think of the variable on the x-axis as predicting or explaining the variable on the y-axis (i.e. the outcome). In modeling scenarios, we often think of a model as an equation with the outcome on the “left-hand side” of the equation and the other variables on the “right-hand side”. Such notation looks as follows. \[ \text{outcome}\sim\text{predictor} \] where the \(\sim\) symbol separates the left-hand side from the right-hand side of the model. This kind of notation is most explicit when we are doing regression modeling.
3.2.2 Predictor
Predictors are the variables that we use to explain variation in an outcome variable. In the simplest case we might examine a single outcome and a single predictor variable. The relationship between the two might be explored by using summary statistics like the correlation coefficient or with plots like a scatter plot. Examining a bivariate relationship like this is a common exploratory task in the early stages of a data analysis.
In more complex situations, we might want to use multiple predictor variables to explain variation in an outcome. In these cases, we might resort to more formal modeling techniques like linear regression or machine learning. In this setting predictors are sometimes referred to as covariates, as there are multiple variables that co-vary with each other. In some applications, the number of predictor variables can be in the thousands or tens of thousands. In these modeling scenarios, the predictor variables are thought of as being on the “right-hand side” of a modeling equation, such as \[ \text{outcome} \sim \text{predictor}_1 + \text{predictor}_2 + \cdots \]
There is one exception to the idea that predictors are used to explain variation in an outcome. In some applications, we are interested in looking at a set of predictors when there is no outcome variable. In these cases, we often want to understand how the various predictors vary with each other and explain these variations. We might notate this kind of situation as \[ \sim \text{predictor}_1 + \text{predictor}_2 + \cdots \] with an empty outcome. These kinds of analyses fall into a large class of analyses known as multivariate or high-dimensional analysis.
3.2.3 Confounder
Confounders are a kind of predictor that are most commonly discussed when considering causal questions. Confounders represent factors that can obscure or “confound” a relationship between two other variables, usually a key predictor and an outcome. In the worst case scenario, if not considered properly, confounders can make it appear that one factor causes another factor, when in reality no such relationship exists. In general, confounding can result in a biased estimate of the causal relationship between two factors, which could subsequently result in poorly-informed decisions based on the data. We will go much deeper in to the nature of confounding in Chapter 10.
For example, shark attacks occur most commonly when ice cream sales are high, suggesting that ice cream sales cause shark attacks. However, a confounder variable is the summer season, because the summer can cause an increase in ice cream sales and it can also cause people to go to the beach, thereby increasing their exposure to sharks. So while there might be a correlation between ice cream sales and shark attacks, that correlation appears because of the confounding effect of the summer season.
A perhaps more realistic example involves smoking cigarettes and lung cancer. While there is now considerable research that draws a causal link between smoking and lung cancer, there are nevertheless important confounding factors one must consider when studying this relationship. For example, the socio-economic conditions of a person can cause one to start smoking as well as cause one to be in poor health. Therefore, while there is very strong evidence that smoking causes lung cancer, failure to adjust for confounding factors might result in over- or under-estimating the precise effect that smoking has.
3.3 Mapping Questions to Datasets
Once a question has been specified the next task is to determine whether it is even possible to answer that question with the datasets that are available. We can generally answer this question without looking at the data themselves, but rather by looking a the metadata and information about how the datasets were assembled (see Chapter 2). A key task is to map the elements of the question to variables and features in your dataset. In the simplest situations, you can map specific words in the question to variables or columns in your dataset.
Using the example given in the section on predictive questions, we might want to ask the question “Does age predict whether a person will respond to a given medical treatment?” Here, we have bolded the words that we hope to map to variables in a dataset. Now suppose we have the dataset printed in Table 3.1.
The aim is to map specific words in the question to the variable names in our dataset. With something like age, we actually have a column in the dataset named age, which has the same meaning as in our question. For the phrase “respond to a given medical treatment,” we do not have an exact match in the dataset, but we do have a column named treatment_response, which is a binary variable that has the correct meaning for our question. The last column in the data frame, eye_color, does not map to any element of our question, so we can ignore that column.
In many cases, the question we want to answer will not have a direct correspondence with the data that we have. For example, we might want to answer a general question like “What is the effect of the environment on the economy in a community?” We might couple that question with a dataset like the one in Table 3.2.
| air_pollution | % unemployed | average_income |
|---|---|---|
| 16.9 | 20 | 68,000 |
| 16.5 | 8 | 67,040 |
| 9.5 | 82 | 19,612 |
| 9.3 | 94 | 27,110 |
| 1.0 | 88 | 50,583 |
| 18.6 | 17 | 71,744 |
| 19.7 | 36 | 60,982 |
| 13.9 | 75 | 53,182 |
| 23.8 | 45 | 49,269 |
| 15.6 | 56 | 58,343 |
The words “environment” and “economy” are a bit too general to map to a specific variable in this dataset so we will have to decide which variables will stand in for those concepts. The dataset in Table 3.2 has a variable air_pollution which is the average air pollution concentration in that community. This might be a reasonable summary of the “environment” in a community. The other variables, % unemployed and average_income, might be reasonable summaries of the “economy,” but one might have a preference for one or the other, depending on what aspects of the economy are of greatest interest.
One important principle in data analysis is data matching (D’Agostino McGowan et al. 2023), or how well elements of a question map to specific variables in a dataset. With some datasets, we have the exact variables that correspond to the elements of a question we want to ask. Such situations exhibit strong data matching. In other scenarios, we only have rough proxies or surrogates for the specific elements in our question. In situations where the variables are not a perfect match to the question, we say there is weak data matching. In situations with weak data matching, we may decide to go ahead with the analyses anyway, if we feel that we can still learn something from doing the analysis with the imperfect variables.
In addition to mapping elements of a question to variables in our dataset, we also need to determine what role each of these variables will play in subsequent analysis. In the next section, we describe the different roles that variables can play and how that might affect the way we apply data analysis tools.
3.4 Example: College Scorecard Data
The U.S. Department of Education collects data on undergraduate institutions in the U.S. and makes them available via the College Scorecard dataset.1 There are over 3,000 variables in this dataset, so we will not go into great detail for the moment. However, we will use this dataset to highlight what kinds of questions we might want to ask.
For example, one of the variables (ADM_RATE) is the overall admission rate for the school. Some descriptive questions we might want to ask are:
What is the typical admission rate for a college in the dataset?
Which school has the lowest/highest admission rate?
Which schools have an admission rate that is less than 10%?
Each of these descriptive questions can be answered directly by analyzing the ADM_RATE variable in some manner.
There is another variable called REGION that indicates the region of the U.S. where the school is located (i.e., New England, Southeast, Southwest). An exploratory question we could ask is whether the number of schools varies by the region. Do some regions have many more schools than other regions? We might also ask whether there is any relationship between the region in which a school is located and the admission rate for that school. Do New England schools have higher admission rates than schools in the Southwest?
One variable of interest might be C100_4, which is the completion rate for full time students at 4-year colleges. In addition to knowing in general what completion rates look like (a descriptive question), we might want to know what predicts completion rates at different colleges (a prediction question). For example, does the median family income of the students enrolled at that university play a role? What about whether the students at the school have taken out loans to pay for college? We might be interested in identifying the key factors in the dataset that predict the 4-year completion rate at a given school.
Finally, for a causal question, if we were university administrators, we might want to know what we could do to increase the completion rate at our school. Would increasing financial aid or lowering tuition cause an increase in the completion rate? What about changing the admissions criteria? These questions address specific interventions that might be done in order change the completion rate outcome and the dataset may be able to shed some light on those questions.
3.5 Summary
Once we have a solid understanding of our dataset and what the different variables mean, we can begin to understand what kinds of questions can be asked of the data. Specifying the type of question to be asked is an important part of a data analysis because it helps to define what kinds of tools can be applied to the data and what kinds of conclusions can be drawn from teh results. There are six categories of questions that we cover in this chapter and the range from straightforward to address (Descriptive) to very difficult to answer (Mechanistic).
Part of specifying a question to ask of the data is determining what roles the variables in the dataset will play. Three key roles are outcome, predictor, and confounder. Assigning roles to each of the variables will help to determine the modeling strategy and the types of models that are appropriate. We will be covering different modeling strategies starting in Chapter 6.
3.6 Exercises
A baseball fan notices that on days when it is cold (say, below 40 degrees Fahrenheit) the local baseball team doesn’t hit as many home runs as when it is warmer (above 40 degrees). If one were to study this relationship, what would be the outcome variable? What would be the predictor variable?
Asthma is a chronic disease that involves obstruction of the airways in the lungs. A physician in Baltimore, Maryland, who specializes in treating asthma patients, notices that the patients in her clinic tend to have higher than normal blood pressure. She wonders whether all asthma patients in Baltimore have higher than normal blood pressure. What type of question is the physician asking here?
The U.S. Medicare system is a large national health insurance program for people aged 65 years and older. The Medicare program collects data on a variety of factors like hospital admissions and doctor’s office visits so that it can reimburse hospitals and doctors for their services. Researchers working with Medicare can analyze the data to answer specific questions. One researcher looks at the Medicare data and wants to know how many people in Medicare have been admitted to a hospital for a heart attack. Assuming the data are available to answer this question, what type of question is the researcher asking?
You are a data scientist at a travel booking company. You have been tasked with exploring the following research question, which is very broad: Which commercial airline is the most reliable? Brainstorm two different interpretations of the word “reliable” in the context of commercial aviation. Then, for one of these interpretations, develop an updated and more specific research question to explore.
A data scientist is interested in determining whether the amount of rain in a city is related to the daily number of car accidents happening in that city. Her hypothesis is that days when there is more rain tend to have more car accidents. When looking at what data are available she can find data on total rainfall and on bicycle accidents (not car accidents). How would you characterize the level of data matching in this analysis with respect to the original question?
More information about the data can be found at https://collegescorecard.ed.gov↩︎