Reading the Scientific Literature: A Tutorial

 

Jon L. Pierce

John W. Newstrom[1]

University of Minnesota Duluth

 

Many students approach the task of reading a scientific journal article much like they would a piece of prose.  They start at the beginning and read it word-for-word until they arrive at the end Boften bewildered, overwhelmed, and wondering >what was it that I just read?=  This confusion, if typical among students, begs for a better approach.  The purpose of this tutorial is to help you read, understand, and even critique such articles more effectively.

 

A bit of background on journal articles may be helpful to you.  There are three-different types of scientific literature (i.e., empirical studies, literature reviews, and theoretical pieces) that you are likely to be exposed to as a student of management and organizational studies.  Articles reporting empirical studies present research findings, conducted by the author of the article.  Empirical research unfolds in the field or in the research laboratory.  These types of articles are likely to be the most challenging to read and comprehend.  In addition, there are articles that present a theory.  A theory is a generalized explanation (i.e., something akin to a >story=) of some phenomenon that provides us with insight into that phenomenon, why and when it exists, what it is related to, and what effects that it produces.  Finally, there are articles that are literature reviews.  In these articles the author attempts to summarize the theory and empirical work that has focused on a particular phenomenon.  The purpose of literature reviews is to provide a current, up-to-date, summary of an existing body of knowledge.

 

Each type of scientific literature has an underlying structure to it.  (For example, empirical studies often contain a title, abstract, introduction and discussion of hypotheses, study design and participants, variables and their measurement, analyses, results, and discussion and conclusion sections.)  You will also find that some portions of these journal articles will be easier to read and comprehend than others.  We believe that there are three key tasks for you to master: 1) knowing how to cut through the peripheral details, 2) learning how to interpret sophisticated statistics, and 3) establishing how to identify the central ideas of an article.  Once you achieve some capability in doing these three things, your job as a student will be easier, richer, and more rewarding.

 

In this tutorial, we will provide you with a road map for reading each of these three different types of literature.  We believe that this >user=s guide to the scientific literature= will prove useful to you in both the short and long term.

 


 

Reading Empirical Articles

 

Some articles report the results of various forms of empirical studies.  These research articles are based on observation, experimentation, data collection, and analyses.  They typically include quantitative and on occasion or qualitative data, which may be primary (directly gathered) or secondary (derived from pre-existing sources).  When examining an empirical study, it is important to understand both who are the authors and where the work was published.

 

Authors

 

Ask yourself these questions:

 

Who is saying it (who is the author)?

 

Why are they saying what is being said (do they have an >axe to grind= or a pre-existing point of view to confirm)?

 

What are their credentials relative to that which is being said?  (For example, are they qualified researchers?  Are they affiliated with any special cause?  Have they conducted and published prior research?  What was their field of study?)

 

Journals

 

Recognize that journals differ markedly in their reputation, often as a function of the quality of articles that have appeared in the past, the editor and members of his/her editorial review board.  Therefore, it is important that you ask (and answer) these questions:

 

What is the reputation of the outlet?

 

Is there a >review board= or are articles screened only by paid editorial staff members?.

 

Are articles submitted by authors, published by invitation, or provided by paid writers?

 

Note that there are substantial differences in the quality of journals in which an empirical articles may appear.  The same is true for literature reviews and articles presenting theory.  Many of these differences stem from two factors: (a) the percentage of articles that are submitted that are ultimately accepted for publication versus being rejected (a lower acceptance rate = greater selectivity and hence presumably higher quality), and (b) whether the article is Ablind@ reviewed (with the authors remaining anonymous to the reviewers) by an independent review panel whose independent judgments to publish or reject are the primary determining factor in the publication of the manuscript.

 


Suggestions for Your Reading Process

 

B Read the title of the article first  Try to establish for yourself what the article is about.

 

B Read the abstract of the article second.  This will set the tone for the article, briefly introduce the underlying theory tested, and provide an overview of the conclusions reached.

 

B Read the introduction third.  This will help you develop a feeling for the background of the study, the issue being explored, the research question(s) being asked and the hypothesis(es) being tested, and make sure that you have a firm grasp on the conceptual definition (i.e., how the variable is defined with other words) for each constructs (variable) in the research question/hypothesis.  Then >dig= for the logic/rationale/reasoning that justifies the hypotheses (i.e., What is the theory that is involved? Does it appear to be >internally consistent= in its reasoning?).

 

B Read the section on the >variables and their measurement= next.  Make sure that you have a good understanding of both the conceptual and >operational definitions= (i.e., how the variable was measured or manipulated) for variables being investigated

 

B Read the >discussion and conclusion= sections fifth.  It is here that you are generally provided with an overview to the study Bits purpose, questions, hypotheses, and important findings.  Just like the abstract, this is an extremely useful summary of the study and its highlights.

 

B Read the >method= and >results= sections next.  These two sections may be the most difficult to read and understand, and possibly the least interesting, as it is here that the complete research design of the study is explained, the population studied described, and the statistical findings/results presented.

 

Dependent upon your statistical skills, you can begin to get a feel for the research findings by following these steps:

 

(a)  look at the mean and standard deviation for each variable.  Examine the mean relative to the response scale that was employed to measure the variable (e.g., a mean of 4.2 on job satisfaction that was measured on a 5-point Likert scale, gives you a feel for how job satisfied/dissatisfied this group of employees are.  The standard deviation gives you a feel for how much spread (variance) there is among the respondents.  This variance is on that specific and unique measure, and for the group of employees studied;

 


(b) look at the alpha coefficient reported for each variable (coefficient alpha is a measure of the reliability of the data employed in the study.  An alpha coefficients less than .70 should be looked upon with extreme caution.  Note that without reliable data it is impossible to have valid research findings;

 

(c) next look at the correlations among the study variables Bthis will give you a feel for what is related to what, the strength and the direction of the relationship among the variables, and

 

(d) look at the >p= statistic that accompanies each correlation.  The p statistic tells you the >likelihood= of observing this size of a correlation between the two variables in question (given this sample size) by chance along (e.g., a p < .05 means that given the sample size employed, the observed correlation could be this strong less than 5 time out of a 100 by chance along;  a p < .01 would mean less than 1 time out of a 100 by chance along).  Generally, we employ a decision rule of p < .05 to report a statistically significant relationship.  Beyond this your statistical education may help you understand some of the analyses employed, or it may be may be somewhat limiting.  Never-the-less, you can read the author=s words and they will most likely tell you what findings are significant and what findings support or fail to support the hypotheses that were presented earlier.

 

B Finally, reread the discussion section (especially the first few paragraphs) and the abstract.  As you read this section try answering several questions:

 

What conclusions (statements of fact) can be drawn from the study?

 

Does the study help resolve the problem or question(s) raised at the beginning?

 

What are the theoretical implications of the findings?

 

Do the results contradict or support past findings.  If there is a contradiction, how does the author explain the discrepancy?

 

What are the practical implications of the findings?

 

Next should come your >personal interpretation= of the study, and its findings:

 

What is the answer to the research question(s) asked? 

 

Were the hypotheses supported? 

 

What meaning do you give to the findings?

 

What are the shortcomings of the present study?  (Note that the author will most likely comment on the shortcomings in their discussion section.

 

What other shortcomings do you see?

 


Finally, place the reading into the context that prompted your reading the article.  What is the question that you are working on, and how does this study fit?  From the context of >why= you read the article in the first place --What new questions do you now have?

 

Cautionary Notes

 

1.         The results section of an empirical paper is often intimidating to the reader, especially when it laden with the statistics.  Take heart of the fact that you are not alone.  Many academics (including us) often find this material difficult to fully absorb and comprehend.  Strive to get something of value from your efforts, however.  Be persistent.

 

Throughout this section, you will most likely find comments noting whether a particular hypothesis was supported or not, and comments on what the findings mean.  This is the important stuff.  Look at the tables and graphs, and pay attention to the means, standard deviations, and correlations, as you are most likely familiar with these.  When reading good academic journals (those with a strong reputation for quality), you can usually rely upon the fact that members of the editorial review board have already looked very carefully at the statistics employed, and made sure they were applied appropriately.

 

2.         Most of the authors whom you will read are quite passionate about their work; they believe strongly in the importance of it, and believe that their measures are the best available.  They are not always >disinterested= parties, and as a result they will want to sell you on the correctness or soundness of their ideas, hypotheses, theories, and the logic of the >story= that they depict in detailing the relationships under investigation.

 

3.         While reading an article, it is extremely difficult to not simultaneously evaluate and judge the work.  We encourage you to >suspend= your initial judgment and disbelief, which can come to many of us very easily and quickly.  As consumers of information, our objective should be to try to >understand= the researcher=s story.  (Understand does not mean to >blindly accept,= however.)  Try to understand the rational for the hypotheses, try to understand previous findings, try to understand the facts, try to understand the findings and how they were obtained, and try to understand the researcher=s interpretation.  After all, in all likelihood s/he has been a student of this subject for a substantial period of time.  We just might be able to learn something from them.  This is especially true of those works that we find published in reputable outlets, by individuals who have the credentials that qualify them to reflect on the phenomenon in question.

 

Assuming that you have succeeded and that you understand what it is that the author has to say it is your turn.  It is at this stage you may decide to take the researcher >to task= by critiquing the work, but you should only do this after you understand what it is that they have done and what it is that they have had to say.  In short, hear them out before passing judgment.

 


The exciting thing about science is that Ait progresses by people taking off on one another=s work (Aronson, 1995, p. 5).  They build on each other=s foundation.  AScience is indeed a cumulative enterprise, and each new study builds on what has (or sometimes, has not) gone before it@ (Jordon & Zanna, 2000, p. 464).  This is why the context for a study, and previous research on related issues and questions is so important.

 

 

Reading Articles that are Literature Reviews

 

Some articles, rather than being data-based and narrow in focus, take a broader perspective.  After identifying a theme or topic that has been rather widely researched, the authors engage in a comprehensive review of all (or many ) of the major studies to identify trends and major conclusions that are supported by the bulk of the research evidence.  Literature reviews are intended to present us with a >state of understanding= of the phenomenon under review.  They provide us with the >best possible= current answer to the research question that drove this particular literature review.

 

There are two types of reviews, one is >narrative= and one is >quantitative= in nature.  The former is best characterized by the author=s reading, interpreting and summarizing for us what it is that the literature tells him/her about a particular topic.  As an example of this traditional literature review the author may be interested in answering the question B What do we now know about the determinants and consequences of self-esteem in the work and organizational context?  The second type of review is quantitative in nature.  Starting during the decade of the 1990s the use of a statistical technique called >meta-analysis= has been employed to provide a more objective summarization of existing empirically-based research findings.  Meta-analysis employs the statistics (e.g., sample sizes, correlations among the variables of interest) reported in previously published studies in order to calculate, for example, the average strength of the relationship between variables.  Through this statistical analyses conclusions about a particular body of knowledge can be drawn.

 

Reading of literature reviews should proceed differently dependent upon the type of review that has been conducted.  If the article is a >narrative-based review,= we suggest employing the process that we detail below.  If the literature review employs a meta-analysis, we suggest that you employ the reading process presented previously in our discussion on reading the empirical literature.

 

Background

 

Just as with empirical articles (and somewhat overlapping them), you should attempt to address the following questions:

 

Who is saying it (who is the author)?

 

Why are they saying what is being said?

 

What are their credentials relative to that which is being said?

 


Where does the article appear?

 

What is the reputation of the outlet?

 

Is there a >review board= or are articles published by invitation.

 

 

The Reading Process

 

Read the title first.

 

Read the abstract second.

 

Read the body of the paper third.

 

Identify the domain of the review in the article.

 

What is the basis for the organization of the review?

 

Chronology (e.g., from earliest study to most recent)

 

Around the dependent variables (i.e., around the >so what= question that examines what was affected by the independent variable)

 

Around the independent variables

 

Around problems with the area of research.  What suggestions are offered on needed future work, and how it might be done?

 

Read the discussion / conclusion section last

 

                                                                             

Reading Articles that Present a Theory

 

ATheory,@ according to Dubin (1976), Atries to make sense out of the observable world by ordering the relationships among elements that constitute the theorist=s focus of attention in the real world@ (p. 26).  Theory has an important role to play in our understanding of that world, and also in guiding our actions to be taken in that world.  Social psychologist Kurt Lewin expressed it well when he stated that Anothing is so practical as a good theory@ (1945, p. 129).  Ultimately, theory should point us in the direction of effective and systematic behavior.  Theory when couple with sound empirical evidence should provide us with grist for our practice of evidence-based management.

 

 

 On >Good Theory= (you may already realize that not all of theory is equally good)


 

According to Klein and Zedeck (2004), good theory:    

 

(1) offers novel insights Bit provides a Asense of discovery and illumination;@

 

(2) is interesting Bit is more than a Aho-hum@ documentation of the obvious;

 

(3) is focused and cohesive B Agood theory illuminates and clarifies, often by organizing, and thus simplifying, a set of previously unorganized and scattered observations@ ... it Arenders real-world processes and phenomena clear and coherent by simplifying and structuring our inchoate understanding of them.  This is only possible if the theory itself is clear and coherent@ (p.932). 

 

They go on to note that good theory:

 

(4) is grounded in the relevant literature, but offers more than a review or integration of this literature; and

(5) presents clearly-defined constructs and offers clear, thorough, and thoughtful       explanations of how and why the constructs in the model are linked BAIf clearly defined constructs are the building blocks of good theory, then thorough and thoughtful propositions linking the constructs Bexplaining what constructs lead to what, when, how, and why Bprovide the mortar@ (p.932). 

 

Finally, they indicate that good theory:

 

 (6) is testable B the constructs are clear and precise; how the constructs are to be measured and how key ideas are to be tested is clearly articulated;

 

(7) in many fields, the theory has practical implications [for example, good organization theory is theory that can be used to address organizational problems (e.g., the causes and consequences of job satisfaction)]; and

 

(8) it is well-written Bthe work presents a clear and logical flow, while it is simultaneously Aclear, focused and interesting@ (p. 933).

 

Employing these eight factors, ask yourself: How well does the theory you are currently reading about meet Klein and Zedeck=s (2004) criteria for a good theory?

 

 

Background

 

The author:

 

Who is saying it (who is the author)?

 


Why are they saying what is being said?

 

What are their credentials relative to that which is being said?

 

The Journal:

 

Where does the article appear?

 

What is the reputation of the outlet?

 

Is there a >review board= or are articles published by invitation.

 

The Reading Process

 

Read the title first.

 

Read the abstract second.

 

Read the body of the paper third.

 

Is it internally valid? (Is there a logical structure to the ideas and arguments that are being presented?)

 

Is it innovative?  (Is there something new here?)

 

Is it provocative?  (Does it elicit an emotional response from the reader?)

 

Is it elegant?  (Is the product a work of art?)

 

Does is exhibit parsimony?  (Is it tight and devoid of extraneous material?)

 

Is it presented in a straight forward manner?  (Is it easy and logical to follow?)

 

Does it build upon existing science (empirical evidence)?  (Alternatively, is it a philosophical or ideological statement grounded primarily in >faith= or the author=s personal experience?)

 

 

A Note on Case Analyses

 


Many students are familiar with >case studies.=  The case study will not be treated here as a >type= of scientific literature.  Case studies are best seen as an educational tool (i.e., an instructional vehicle employed by many of your professors as a way of facilitating your learning a particular subject matter). Case studies typically represent an in-depth (intensive) examination of a single unit of analysis (e.g., person, group, or some organizational phenomenon such as a merger). 

 

Case research, on the other hand, is an important subset of theoretical research.  In case research, the scholar selects one or more cases (e.g., person, group, organizational phenomenon) which are designed to highlight important elements of a topic under study.  The goal of case research is to strategically choose samples, which will allow the researcher to understand issues in the topic of interest (see Eisenhardt, 1989; Yin, 1994).  Data are frequently collected by a researcher employing multiple data collection means (e.g., interviews, organizational memos, and financial records) of the phenomenon under investigation and within its natural setting.  There is no attempt to employ exact variable measurement, experimentation coupled with the use of control groups, and the statistical testing of relationships among variables.  As a scientific tool, case research is particularly well suited for the generation of hypotheses and theory construction.  It is less rigorous and less-well suited for the testing of hypotheses and the making of generalization to other settings --both its internal and external validity are extremely limited.

 

 

Cautionary Notes

 

1.         Most of the authors whom you will read are quite passionate about their work Bespecially when they are developing >new= theory or try to dispel old theory.  These authors are not always >disinterested= parties, and as a result they will want to sell you on the correctness or soundness of their ideas, hypotheses, theories, and the logic of the >story= that they depict in detailing the relationships under investigation.

 

2.         While reading a theoretical article, it is extremely tempting to simultaneously evaluate and judge the work.  We would like to encourage you to >suspend= judgment, as well as to suspend disbelief, which can come to many of us very easily and quickly.  As consumers of information, it should be our objective to try to come to >understand= the theory being discussed.  (Understand does not mean to >blindly accept.)  Try to understand the rational for the hypotheses, try to understand previous findings, try to understand the existing evidence, try to understand how this evidence is being woven into the theory, try to understand the arguments for the new or revised theory. 

                                                                             

 

Conclusion

 

We would like to encourage you to make the regular consumption of the management and organizational sciences literature a part of your formal education experience, and as a part of your >life-long= educational activities.  Contemporary history has quit clearly demonstrated that education is a good investment in yourself, and that that investment should not be completed upon receipt of your formal degree.

 


One important part of this education should consist of your regular reading of the >top-level= journals in your discipline Bthis will consist of your reading theory, literature reviews, and empirical investigations.  The end to which all of this is aimed is both your personal enlightenment, and the ability to make your professional career one driven and characterized by >evidence-based= practices.

 

 

 

 

                                                                                                                       

References

 

Aronson, E.  1995.  Readings about the social animal.  New York: Bedford, Freeman, & Worth Publishers.

 

Dubin, R.  1976.  Theory building in applied areas.  In M. Dunnette (Ed.), Handbook of industrial and organizational psychology, (pp. 17-39).  Chicago: Rand McNally.

 

Eisenhardt, K. M.  1989.  Building theories from case study research.  Academy of Management Review, 14:4, 532-550.

 

Jordon, C. H., & Zanna, M. P.  1999.  How to read a journal article in social psychology.  In R. F. Baumeister (Ed.), The self in social psychology, (pp. 461-470).  Philadelphia: Psychology Press.

 

Klein, K. J., & Zedeck, S.  2004.  Theory in applied psychology: Lessons (re)learned.  Journal of Applied Psychology, 87, 931-933.

 

Lewin, K.  1945.  The Research Center for Group Dynamics at Massachusetts Institute of Technology.

 

Yin, R. K.  1994.  Case study research: Design and methods.  Thousand Oaks CA: Sage Publications.

 

 

Labovitz School of Business and Economics

Department of Management Studies B2007



[1]  We appreciate the constructive contributions made to this document by Dr. Geoffrey G. Bell.