Student and Instructor Resources
This page contains resources and suggestions that may be helpful to students and instructors using the book Theory Construction and Model Building Skills. For students, we provide supplementary materials and exercises to elaborate and reinforce core concepts. For instructors, we make suggestions for class discussion and assignments. Additional resources relevant to Chapter 11 (on exploratory data analysis) are available by clicking the statistics tutorials tab.
Statistical Programs
Statistical Primers
Statistical Primers
Interfaces with SPSS, Excel, Ascii, SAS and STATA Formats
Chapters 13 and 14 covered facets of measurement theory from a theory construction point of view. They emphasized classic test theory, self reports, observer reports and "objective" reports. The chapters did not cover scaling theory. Scaling theory is more quantitatively oriented and focuses on the mapping of measures onto properties of constructs. We have written a primer that discusses scaling theory with an emphasis on creative theory construction. It is introductory and conceptual but underscores yet another facet of measurement theory construction. The primer covers tracelines and item operating characetristics in multi-item scales (including Item Response Theory), conjoint measurement, and multidimensional scaling as illustrative examples of creative scaling theories.
See the primer on scaling theory
See the primer on scaling theory
We make available here a power point file with every figure from the book on a separate slide. You can select and then copy and paste figures of your choosing into your presentations, lectures or handouts.
For causal theories, it often is useful to read an article and draw an influence diagram of the theory the article is evaluating. Students, below are links to six open-access articles that you can use to practice doing so, followed by a link to the diagrams that we drew for each article. You can compare your diagrams with ours. The latter link also includes a description of the heuristics we use when constructing such diagrams. We purposely chose articles that are diverse, including a quantitative research study, a qualitative research study, a meta-analysis, a traditional review, a laboratory experiment, and a study using path analysis. These studies, like all studies, have methodological weaknesses, but we ignore these. Our focus is on theory and how to abstract a causal theory from reading a research article.
Instructors, you likely will want to have students in your class review the articles and heuristics we provide but you also might locate articles that better map onto the interests of your students (or you can have students choose their own articles) and repeat the task of drawing the influence diagrams. We like to assign several students the same article to read and construct an influence diagram for it. We then choose one student to draw his or her influence diagram for the study on the board and explain it, including providing a conceptual definition for every box in the diagram (we provide feedback on the clarity of the conceptual definitions). We then have a second and third student who read the same article draw their diagrams just after the first student and explain their diagrams and conceptual definitions. We then lead a discussion of the discrepancies in the diagrams and definitions, using it to sharpen student skills for extracting theory from articles. If you are a student, you can try this exercise with another student and compare notes.
See the quantitative research article
See the qualitative research article
See the meta-analysis review article
See the traditional review article
See the laboratory experiment article
See the article with path analysis
See our influence diagrams and a description of how we constructed them for the above articles.
For a tutorial on software for drawing influence diagrams, see the supplemental material for Chapter 16 below.
If you are planning a thesis proposal that will use causal thinking in conjunction with structural equation modeling, see our primer for preparing a thesis proposal using causal modeling.
Instructors, you likely will want to have students in your class review the articles and heuristics we provide but you also might locate articles that better map onto the interests of your students (or you can have students choose their own articles) and repeat the task of drawing the influence diagrams. We like to assign several students the same article to read and construct an influence diagram for it. We then choose one student to draw his or her influence diagram for the study on the board and explain it, including providing a conceptual definition for every box in the diagram (we provide feedback on the clarity of the conceptual definitions). We then have a second and third student who read the same article draw their diagrams just after the first student and explain their diagrams and conceptual definitions. We then lead a discussion of the discrepancies in the diagrams and definitions, using it to sharpen student skills for extracting theory from articles. If you are a student, you can try this exercise with another student and compare notes.
See the quantitative research article
See the qualitative research article
See the meta-analysis review article
See the traditional review article
See the laboratory experiment article
See the article with path analysis
See our influence diagrams and a description of how we constructed them for the above articles.
For a tutorial on software for drawing influence diagrams, see the supplemental material for Chapter 16 below.
If you are planning a thesis proposal that will use causal thinking in conjunction with structural equation modeling, see our primer for preparing a thesis proposal using causal modeling.
The second edition of our book eliminated one of the examples on mathematical modeling that was in the first edition and condensed a second example. If you want to read the examples as they appeared in the first edition, click see the examples from the first edition
GENERAL
CHAPTER 7: CAUSAL MODELS
CHAPTER 8: MATHEMATICAL MODELS
CHAPTERS 13 AND 14: THEORY AT THE LEVEL OF MEASUREMENT
CHAPTER 11: EMERGENT THEORY: QUANTITATIVE APPROACHES
Supplementary materials for this chapter are in the Statistics and Tutorials section on this website. Instructors, depending on the students, we sometimes make available a data set and ask students to pursue one or more of the analytic methods described in Chapter 11 and to present their results to the class. This requires students working through the primers and worked examples we provide on this website.
If you have questions about basic or advanced math, there is a great archive of common math questions and answers on the website called Ask Dr. Math.
CHAPTER 10: EMERGENT THEORY: QUALITATIVE/MIXED METHOD APPROACHES
To watch a range of videos on the analysis of qualitative data using the popular Atlas.ti software, click Atlas.ti Videos
To watch a range of videos on the analysis of mixed methods data using the MAXQDA software, click MAXQDA
A data repository for qualitative and mixed methods that makes data sets available to the scientific community is at QDR. The University of North Carolina maintains Dataverse, a repository of both qualitative and quantitative data, as does ICPSR. The classic Human Relations Area Files originated by anthropologist George Murdock are maintained at Yale university. Related to it is the eHRAF World Cultures website that contains ethnographic collections for a wide range of cultures.
To watch a range of videos on the analysis of mixed methods data using the MAXQDA software, click MAXQDA
A data repository for qualitative and mixed methods that makes data sets available to the scientific community is at QDR. The University of North Carolina maintains Dataverse, a repository of both qualitative and quantitative data, as does ICPSR. The classic Human Relations Area Files originated by anthropologist George Murdock are maintained at Yale university. Related to it is the eHRAF World Cultures website that contains ethnographic collections for a wide range of cultures.
CHAPTER 16: READING AND WRITING ABOUT THEORIES
Look at the supplementary materials for Chapter 7 on how to extract causal theories from articles that you read. There also is a primer on how to prepare thesis proposals that use causal models with structural equation modeling SEM in that section. Look at the supplemental materials for Chapters 3 and 4 for examples of evaluating the conceptual logic model associated with a theory.
When writing theories, you may have to present an influence diagram. A very flexible program we use to create influence diagrams is Microsoft Visio, which is available through many universities. It also is availble at a reasonable cost through third party vendors, like Brytesoft and Pmarketonline. Click below for a tutorial on how to use this program and a description of tools for use in Visio.
Download templates and vss file discussed in the tutorial (this is a zip file that, when opened, will make 4 files available). You may have to adjust your Trust Settings in Visio to use these files - see the tutorial.
CHAPTER 6: THOUGHT EXPERIMENTS
The second edition eliminated the appendices for thought experiments, one of which was on moderated moderation or three way interaction effects. If you want access to this appendix, click HERE.
Link to descriptions of secondary data bases and data repositories for secondary data analyses. See also some of the links for Chapter 10.
Watch a video on how to prepare a research poster for a conference (this is an external link to youtube)
Watch a video on how to prepare a research presentation (this is an external link to youtube; although it was made for physiologists, it is apt for social scientists)
CHAPTERS 3 AND 4: CONCEPTUAL LOGIC MODELS AND THEORETICAL CONTRIBUTIONS
Chapter 3 discussed three criteria that journal editors and reviewers often use to judge the theoretical contribution of an article, namely novelty, utility, and scope. Chapter 4 discussed conceptual logic models that theorists use to support the viability of a proposed theoretical expression. Both chapters encourage you to evaluate the theoretical contribution and the quality of the conceptual logic model contained in a research report. Here, we provide two open access articles as well as a document we wrote to evaluate the theoretical contribution and conceptual logic model of each article. (These same articles are used below in the supplemental materials for Chapter 7 for drawing influence diagrams to represent theories in articles).
Students, read the first two articles in the links below, make note of the authors' conceptual logic model and evaluate it, and then identify how the authors make a case for the research's theoretical contribution using the criteria of novelty, utility, and scope. Then compare your analysis to the document we wrote conducting the same exercise. (Article 3 is only used to make a point in our write-up)
For instructors, you can assign students the above task or you can assign articles of your choice that better map onto the substantive interests of your students. Or, you can let students choose their own article and then have them present their analyses to the class. Alternatively, you might have the entire class perform the task on the same target article of your choice and then have different students present their analysis to the rest of the class. You would lead a discussion of the discrepancies between the presentations. Students, if your instructors do not do this, you might try it with one or two other classmates outside of class.
See article 1
See article 2
See article 3
See our analysis of articles 1 and 2
Students, read the first two articles in the links below, make note of the authors' conceptual logic model and evaluate it, and then identify how the authors make a case for the research's theoretical contribution using the criteria of novelty, utility, and scope. Then compare your analysis to the document we wrote conducting the same exercise. (Article 3 is only used to make a point in our write-up)
For instructors, you can assign students the above task or you can assign articles of your choice that better map onto the substantive interests of your students. Or, you can let students choose their own article and then have them present their analyses to the class. Alternatively, you might have the entire class perform the task on the same target article of your choice and then have different students present their analysis to the rest of the class. You would lead a discussion of the discrepancies between the presentations. Students, if your instructors do not do this, you might try it with one or two other classmates outside of class.
See article 1
See article 2
See article 3
See our analysis of articles 1 and 2
CHAPTER 9: SIMULATIONS
Link to NetGo, a programming environment for simulating natural and social phenomena.
Link to RePast, free JAVA based software for agent based simulations.
Link to MASON, free JAVA based software for agent based simulations by a collaborative team at George Mason University
Link to the Journal of Artificial Societies and Social Simulation
Link to RePast, free JAVA based software for agent based simulations.
Link to MASON, free JAVA based software for agent based simulations by a collaborative team at George Mason University
Link to the Journal of Artificial Societies and Social Simulation
CHAPTER 15: THEORY REVISION
In Chapter 15, we discussed Karl Popper's notion of methodological rules, i.e., the specification of methodological practices that rule out, as much as possible, all alternative explanations for the results of a study so that critics cannot question the theory test. Ideally, a negative result should unambiguously disconfirm a theory and a positive result should increase confidence in the theory. From the standpoint of Bayesian epistemology discussed in Chapter 15, this corresponds to designing a study whose Bayes factor is diagnostic rather than uninformative. We present in the primer below 52 "methodological rules" for current day social science research to use for theory testing research to address critics who might seek to dismiss your findings. Students, apply these rules to your thesis proposals and projects and address them, as appropriate, to design a stronger research project. Instructors, consider assigning a research article for your students to critique using the rules. Have different students present their critiques in class and then lead a discussion about disparities that arise or important omissions. This task is more methodologically oriented than theory oriented, so you may not want to use it, depending on your course goals.
A popular method for testing causal models developed in Chapter 7 is structural equation modeling (SEM). The mathematical modeling strategies described in Chapter 8 can be usefully applied to SEM, although such applications are rare. For the connection between the two approaches, see our primer on SEM and math models.
Using Bayesian epistomology, you make judgments about (1) the prior probability (or odds) that your theory is true, (2) the probability that the results of your study would have occurred if your theory was, in fact, true, and (3) the probability that the results of your study would have occurred if your theory was, in fact, false. With this information in hand, Bayes theorem can be used to calculate the probability or odds your theory holds in light of the study results that occurred. This probability is called the posterior probability and when expressed as odds, it is called the posterior odds.
Although we sometimes can't specify exact values for these input probabilities, we usually can form approximate judgments of them. For example, for the prior probability of a theoretical expression, a value of 0.25 means that prior to conducting the study, the existing evidence and logic model suggest the theory is unlikely to be true, 0.50 means the theory is as likely to be true as false, and 0.75 means the existing evidence suggests the theory is fairly likely to be true. You may not find yourself formally invoking Bayes theorem, but often the Discussion section in your research report takes into account, in one form or another, your judgments about the strength of prior evidence for the tested theory as well as the diagnosticity of your study results for that theory (as captured in the Bayes factor - see Chapter 15). In some respects, Bayesian logic almost always enters one's evaluation of research studies, even if it is not done so explicitly.
Presented below is a "Bayes Calculator" that will calculate the posterior odds and posterior probability for a theory based on the three input probabilities identified above. After critically reading a study, consciously form judgments about the three probabilities with respect to a given theoretical expression, namely (1) the prior probability the theoretical expression is true, (2) the probability that the results of the study would have occurred if the theoretical expression is true, and (3) the probability that the results of the study would have occurred if the theoretical expression is false. Chapter 15 helps you think about how to make such judgments (note: if a theory has limited prior evidence, it is not uncommon to set its prior probability at 0.50 or 50-50 in terms of the common parlance for odds). Enter the values into the calculator and then think about the answers that result. The reported Bayes factor is an index, in part, of how well the study was designed as a clean theory test. The more it deviates from 1.0 in either direction, the more informative the results are relative to the theory, The difference between the prior and posterior probabilities also is revealing about study quality and how much you learned from the study relative to the theory. Try experimenting with different input vaues and see how the posterior probability for the theory is affected.
You can only enter values in the top three boxes. The values must be greater than 0 and less than 1
As an example, suppose after reading the Introduction of a study, we decide that the prior probability of a theory being "true" based on past research and the conceptual logic model is 0.70. After a thoughtful analysis of the study design and Introduction, we decide that if the theory is true, it is quite likely that the predicted results will occur, say about 0.90. However, we also identify some plausible confounds and alternative explanations such that we think the predicted results are also somewhat likely to occur even if the theory is false, say with a probability of 0.50. Suppose the predicted results did, in fact, occur. Given this, if we enter these three values into the calculator, we find that the posterior probability for the theory is 0.81. The results are confirmatory and lead to an increase in the prior probability of the theory from about 0.70 to about 0.80. The confounds lessened the diagnosticity of the research, but they did not completely undermine it. The information value of the study also is evidenced by a Bayes factor larger than 1, namely 1.8. Thus, some information gain was garnered from the study, although it could have been better.
If we were to discuss our evaluation of the study with colleagues and/or other scientists, we would seek to determine if they agreed with our general assessment of the prior probability of the theory (around 0.70). We also would seek to determine if they agreed, more or less, with our judgments about the probability of the predicted results given the theory is true and the probability of the predicted results given the theory is false, after identifying for them the various confounds and alternative explanations we felt could be operating. We could enter into the calculator the values our colleagues think operate and compare their results with ours. In this sense, Bayesian thinking is a good "conversation organizer." And, it forces you to think about the results of a study in ways you might not otherwise do.
Note that if the predicted results did not occur in the study, we would need to re-orient our estimates of the probabilities of the Bayes factor to the results that actually did occur. Suppose the theory predicted that the correlation between variables X and Y should be statistically signficant and reasonably large but that this did not turn out to be the case; X and Y were relatively uncorrelated. We might judge that the probability of such a result occuring given the theory is true to be quite low, say 0.10. We might also reason that if the theory is false, then X and Y likely would be uncorrelated, so the probability of the near zero correlation given the theory is false is reasonably large, say 0.80. Using the same prior probability for the theory (0.70) and entering these new probabilities into the Bayesian calculator, the posterior probability for the theory is now 0.23. The Bayes factor is 0.125 and the results are decidely disconfirming relative to the prior probability. Again, in our discussion with others, we would want to build a case for the approximate values we chose for our Bayesian analysis.
Instructors, have your students read a research article to test a theory, focus on a specific theoretical expression within that theory, and then have students form judgments of the three Bayesian probabilities for that expression. (Not all articles will easily fit this task, so you may have to choose the article carefully as students are still acquiring Bayesian thinking skills). Have several students present and justify their analysis to the class and then lead a discussion about any disparities that arise. Students, if your instructor does not do this, then try the exercise with one or two other students in the class on your own. Also, when you conduct a research project to test a theory (e.g., your thesis), think about the study results from a Bayesian perspective.
We emphasize the point from the main text that the above thinking reflects Bayesian epistemology, not Bayesian statistics as applied to social science research. It is a way of thinking about theory in light of study results that is distinct from the more traditional approach of Karl Popper, although there is some overlap. As we have encouraged you to do throughout our text, put on different "thinking hats" as you consider the implications of any given result.
If we were to discuss our evaluation of the study with colleagues and/or other scientists, we would seek to determine if they agreed with our general assessment of the prior probability of the theory (around 0.70). We also would seek to determine if they agreed, more or less, with our judgments about the probability of the predicted results given the theory is true and the probability of the predicted results given the theory is false, after identifying for them the various confounds and alternative explanations we felt could be operating. We could enter into the calculator the values our colleagues think operate and compare their results with ours. In this sense, Bayesian thinking is a good "conversation organizer." And, it forces you to think about the results of a study in ways you might not otherwise do.
Note that if the predicted results did not occur in the study, we would need to re-orient our estimates of the probabilities of the Bayes factor to the results that actually did occur. Suppose the theory predicted that the correlation between variables X and Y should be statistically signficant and reasonably large but that this did not turn out to be the case; X and Y were relatively uncorrelated. We might judge that the probability of such a result occuring given the theory is true to be quite low, say 0.10. We might also reason that if the theory is false, then X and Y likely would be uncorrelated, so the probability of the near zero correlation given the theory is false is reasonably large, say 0.80. Using the same prior probability for the theory (0.70) and entering these new probabilities into the Bayesian calculator, the posterior probability for the theory is now 0.23. The Bayes factor is 0.125 and the results are decidely disconfirming relative to the prior probability. Again, in our discussion with others, we would want to build a case for the approximate values we chose for our Bayesian analysis.
Instructors, have your students read a research article to test a theory, focus on a specific theoretical expression within that theory, and then have students form judgments of the three Bayesian probabilities for that expression. (Not all articles will easily fit this task, so you may have to choose the article carefully as students are still acquiring Bayesian thinking skills). Have several students present and justify their analysis to the class and then lead a discussion about any disparities that arise. Students, if your instructor does not do this, then try the exercise with one or two other students in the class on your own. Also, when you conduct a research project to test a theory (e.g., your thesis), think about the study results from a Bayesian perspective.
We emphasize the point from the main text that the above thinking reflects Bayesian epistemology, not Bayesian statistics as applied to social science research. It is a way of thinking about theory in light of study results that is distinct from the more traditional approach of Karl Popper, although there is some overlap. As we have encouraged you to do throughout our text, put on different "thinking hats" as you consider the implications of any given result.
Instructors, we typically assign 1 or 2 chapters to read before class and then elicit student reactions to the material during class. We sometimes select a subset of the questions at the end of each chapter for class discussion. In the course as a whole, we try to create a supportive, problem solving environment in which all students help each other create contributive theories for a research project or design research that will lead to the creation of such theories.
CHAPTER 5: CONCEPTUAL DEFINITIONS
Instructors, we primarily rely on student reactions to the chapter material and questions at the end of the chapter to organize our class discussion. At some point in the course, we ask students to create a theory and present it to the class. At that time, we ask students to define every construct in their theory after encouraging them to use the strategies in Chapter 5. We encourage other students to give critiques and feedback with respect to the definitions. Another possibility is to assign articles for students to read and to critique the quality of the conceptual definitions in the articles. (However, many articles simply take conceptual definitions as givens).
Instructors, we again primarily rely on student reactions to the chapter material and questions at the end of the chapter to organize class discussion. However, when students present their theory to the class (see Chapter 5 above), we ask them to be explicit about the relationships they posit in the spirit of Chapter 6. We again encourage other students to give their critiques and feedback. Another possibility is to assign articles for students to read and to discuss and critique the quality of relationship elaboration in the articles.
CHAPTER 12: HISTORICALLY INFLUENTIAL SYSTEMS OF THOUGHT
Instructors, a useful exercise for students is for you to choose a topic of interest (e.g., reducing HIV risk behavior, reducing opiate abuse, increasing mental heath service utilization, reducing homelessness, reducing prejudice/discrimination, increasing adherence to treatment protocols) and, prior to the next class, assign each student the task of using one of the grand/influential theories to identify core constructs or processes to gain new perspectives on the problem. You will need to assign a different theory to each student to ensure appropriateness of the theory and to create theoretical diversity across students. Have each student present his or her analysis in class to the other students. Then have the class reduce and organize the different ideas into a workable, integrated theoretical framework. Have the class evaluate its novelty, utility, and scope. Students usually will need to read more about the grand theory they are assigned outside of class, but they also can give the task a try just by using the mindset created by the material in the main text.
BAYES CALCULATOR
An interesting class exercise for thought experiments is to have students, prior to class, generate interesting hypothetical questions representing counterfactuals in their research areas. Then have each student present the counterfactual and discuss possible answers to it, thinking through all the ramifications of the counterfactual in depth For example, students might be asked to generate questions of the form "what if there were no...." (e.g., no teachers, no schools, no SES differences) or to posit downward counterafctuals (a counterfactual that states certain things things are worse than they are) or upward counterfactuals (a counterfactual that states certain things are better than they are). The students as a group then create theoretical speculations around the counterfactuals.