Introduction to Research Methods

Week 1

 

 

What is research? (develop new knowledge)

 

Economists ground their research in the scientific method (like other sciences):

·                   Predicting or observing an event

·                   Devising an explanation (hypothesis)

·                   Testing the hypothesis by gathering additional information under repeated observations

·                   Accepting, revising, or rejecting the hypothesis

 

Economics makes use of the scientific method, but often differs from some of the other sciences because it is often more akin to social sciences. Economists do not usually conduct controlled experiments, although experimental economics is a growing field within the discipline. Instead they depend on the observation of real-world events, which seldom reoccur under the exact same circumstances (i.e., they defy precise replication).

 

Economics is not an exact science. As a result, economists make use of the principles of economics, including assumptions about how individuals, firms, and government behave. On the basis of these assumptions, economists predict the results of events or policies. Economic reasoning, used to derive research hypotheses from theory, consists of applying economic principles to explain events, predict outcomes, and recommend policies. Economic reasoning, which is merely a form of the critical thinking process, uses deductive reasoning to develop hypotheses from larger economic principles. [Deductive reasoning derives a set of propositions from a few "given" propositions].

 

Economic theories or principles are often developed (or defined ) using inductive reasoning. In fact, theories are often developed in an attempt to make sense of observations by asking "is this observation a particular case or does it fit into a more general pattern or story".  Inductive reasoning frequently is used in more applied research that seeks to define or describe a problem or phenomena that is observable.

 

What tools do economists use to execute research?

Theory driven research

·                   Model building and examining relationships using economic theory to predict relationships

·                   Math (theoretical research)

·                   Econometrics (empirical or applied research)

Problem driven research

·                   Description and examining problems (e.g. business, policy) using economic theory to predict potential solutions to problems

·                   Data

·                   Univariate and Multivariate Statistics

 

Its quantitative not qualitative. Theory is mathematical modeling, design is often dictated by complex mathematical models of behavior and measurement biases, methods are quantitative (remember econometrics?), as is their analytic execution, and results are often reported in numbers quantifying change.

 

Example of difference between two:

·                   Model building: Mincer's log wage function (empirically tested)

·                   Problem solving: How can we achieve parity between AA and whites in wages? (ed, exp, inst factors)

What are the components of the Research Process (see handout)

 

Theory/problem

All research is built upon past work (i.e., you need a thorough lit review). Differences come between theory driven (theory) and problem driven (applied) research.

 

In theory driven research, hypotheses are developed from existing theory and tested. Deductive reasoning drives the model building. Hypothesis can be theoretical and tested with mathematical modeling or empirical and tested with data (econometrics).

 

In applied research, problem statements are developed from practical real-world issues, which are often derived from inductive reasoning. Problem statements are often posed as questions and the research is designed to provide solutions to the problem. Applied research can be theoretical and may not contain data (e.g., Heckman's work on sample selection and theoretical econometrics), but most is empirically grounded.

 

Hypothesis/problem statement

Hypothesis: a statement (or conjecture about) the relationships among the variables that a researcher intends to study. When hypothesis are phrased as testable statements of relations, they are often thought of as predictions, which if confirmed, will support the theory. Care must be taken that the hypothesis can be falsified. (For example, it is sometimes argued that the hypothesis of a utility maximizing individual cannot be rejected because behavior is always defined as utility maximizing. OR The hypothesis that discrimination exists cannot be confirmed or denied because "missing" variables could always be argued to explain wage differentials.)

 

Empirical tests are often based on rejecting the null hypothesis (Ho). The null states that two or more variables are not related. If accumulating evidence suggests that the null hypothesis is false, the researcher indirectly demonstrates that the variables are related.

 

Problem statements can be developed through statistics (e.g., why did the outflow from farming communities increase during the 1990's?), practical difficulties (e.g., what is the optimal location of a bottling plant?), or observation (e.g., why are wages higher in SF than Wichita--are real wages higher???)

 

Sound theoretical and applied research are grounded in the same research principles.

 

Research design--THE ARCHETICT

In order to test the hypothesis (derive a solution to the problem) a research plan must be devised. The plan, or research design, reflects a logical inquiry that will lead to a verifiable test of the hypothesis or to answer the question in the problem statement. The design delineates the steps necessary to test the theory or solve the problem.

·                   Is the research theoretical or empirical?

·                   What are rival alternative hypotheses that must be considered?

·                   How do we structure the inquiry to test the theory? (experimental, quasi experimental, etc)

·                   What research outcomes would lead to rejecting a null hypothesis?

 

Analysis design (modeling)--THE CONTRACTOR

Economists use models as a simplified representation of a real world phenomena. The models outline the relationships between variables that are theoretically identified. Without the models, the research design would be a mere abstraction of how to test theory. However, models without research design often do not logically and coherently test the hypothesis.

 

Ockham's Razor is the operating principle for both theoretical and empirical model building:

Theories and explanations should be as streamed as possible. Ceteris paribus, the simplest theory (i.e., the one with the fewest predictors) is the best. Named after William of Ockham (1285-1349).

 

·                   What variables are we trying to explain (i.e., what are the endogenous variables)?

·                   What are the predictor variables (i.e., what are the exogenous variables)?

·                   What are the control factors (all else equal)?

·                   If empirical, what are the appropriate data? (longitudinal, time series, cross sectional)

 

 

Analysis--THE LABORERS

Analysis is "merely" the execution of the work plan/model. "Grinding through the analysis" is an oft heard phrase, although "grinding" without understanding "the big picture" can produce disasters if the analyst can't see the forest for the trees. (I just need to do one more run or define this variable slightly differently).

 

·                   Are secondary data available or must we rely on primary data collection?

·                   How will the variables be operationalized?

·                   What other potential constructs can the operationalized variables represent? (For example, is IQ an appropriate measure of intelligence or does it measure something else?)

 

Evaluate outcomes and undertake sensitivity analysis

This is the interpretation phase and raises the question "can the research findings be used to reject the null hypothesis or can they be used in support of a rival hypothesis?" Do they answer the problem question raised?

 

Sensitivity analysis--making sure the results hold up under a variety of conditions (e.g., different time periods, different samples, different model specifications)--provides additional evidence that the null can be rejected and, by implication, the hypothesis can be accepted. Sensitivity analysis is also critical in ensuring that the solution to the problem will work under a variety of circumstances.

 

Accept/reject hypothesis or solution to problem

Make sure that rival hypotheses or alternative interpretations cannot replace your interpretation of the results. Critical analysis by yourself and others will help prevent this from occurring at the final stages of research. If the null hypothesis cannot be rejected, try to determine if flaws in the research process existed that may create results that run counter to the hypothesis. Note this happens quite frequently!

 

·                   Was there an error in applying theory or logic?

·                   Was a research designed developed that could not test the hypothesis? For example, did tests set up did not address the issue raised or was ceteris not paribus for some confounding factor?

·                   Was the analysis executed without a plan so it makes no sense?  (Crass empiricism?)

·                   Were the data appropriate? Did the empirical variables measure the appropriate concept?

·                   Did the results only hold up under certain conditions (i.e., does context matter)?

What are some common pitfalls and what are some helpful hints?

 

Pitfalls

·                   Make sure your research design lets you distinguish between correlation and causality, a difficult task. (smoking causes (?) cancer; GDP increases with (?) time (or prices and productivity)). Use both theory and statistics to help sort through the differences.

·                   Make sure your phrasing is correct or your statement could be incorrect. There is a 100 percent probability of dying but a less than 100 percent at dying during any given time period. Men earn more than women only if ceteris is paribus. Don't think research writing is easy.

·                   Don't think you data set provides you with all the information necessary. Secondary data sources, no mater how extensive they are, never have all of the information that you want. Sometimes its critical (e.g., curriculum and earnings in NLSY), sometimes it can be finessed (e.g., HSB and annual earnings without labor supplied). Primary data are generally limited with respect to geographic or population.

·                   Don't think you can perfectly operationalize empirical variables. Prices, income, and quantities are all relatively easy to measure. (Are they really, what is the price of housing?) Things like gender and race seem easy (Are they really?) Tastes and value of a life are harder to quantify.

·                   Don't forget to leave your biases out of research. Know that normative economics has no place in research but recognize that it is ubiquitous. Understand the role that biases play and design your research against them. I can't research topics that are emotional for me.

Hints--QUESTION EVERYTHING

·                   Know that life is probabilistic. Virtually no economic event will occur with certainty. As a result, all (empirical) research and predicting conclusions are based on probability. We can increase the accuracy of our predictions and conclusions, however, by knowing the conditions under which an event is likely to occur.

·                   Undertaking primary research is HARD. Asking questions in the right way takes time in development and swallowing of pride. (Phrasing is critical).

·                   It always takes LOTS longer than expected. I estimate the amount of time it should take me to do something and times by 2.5 and I usually underestimate--and I've been at this research thing since 1975.

·                   Find rewards in small accomplishments and make a high out of putting it all together.

·                   Swallow your pride and take criticism/feedback and ideas.

·                   Know that your critics often provide you with plausible rival hypotheses that you must reject. Use the critics early (to collect data to support your case) and often (to ensure that you are building logically arguments to reject rival hypotheses).

·                   Remember, if it were easy, someone else would have already done it.

How does the course simulate and teach the research process?

 

·                   Hands on classroom. Lecture will be at the first part of most classes but much of class time will be spent addressing (group) questions about research problems. The class is your lab for gaining feedback for what you've done, problem solving, and discussing next steps.

·                   You will do research and it will be a group effort. Most research is not done in isolation. "Independent work in a team environment."

·                   List serv developed for communication.

·                   You must do primary research--get your hands dirty. (15% of your research grade).

·                   Develop and execute a survey for your group research project.

·                   Engage in on-going primary research. One option is the HIRE Center, which is in the field on two on-going projects (phoning welfare clients--applied policy research--and interviewing employers about knowledge and skills of entry-level workers).

 

GO OVER SYLLABUS AND HAVE THEM START DETERMINING RESEARCH GROUP AND PROJECT.