POE+draft+archives

David (10/3): Another article that discusses data, how data becomes evidence and what constitutes evidence can be found in the //International Journal of Qualitative Methods 2(1) Winter, 2003.// In the article titled, **The Nature of "Evidence" In Qualitative Research Methods** the authors Steven Miller and Marcel Fredericks explore how qualitative research data become evidence. Their discussion parallels Lincoln's article describing how data becomes evidence i.e. that data does not constitute evidence but that data becomes evidence when it can be used to support a "claim" (hypothesis?, belief?). Miller's article also parallels Lincoln's in the premise that evidence is only pertinent to the question when analyzed in the paradigm in which the data was gathered and the question was asked. In our opening statement we spoke of the relationship between data, evidence and knowledge as being an iterative relationship but, I am a linear thinker and there seems to be a flow to the relationship. "Data becomes Evidence when it is Analyzed within a specific Paradigm or Method, Evidence supports/leads to Findings which support a Claim or Belief which, when supported by Evidence adds to the body of Knowledge. I may be way off base and I've only just scratched the surface in my readings but this pattern seems to fall out of these two articles. Miller and Fredericks, 2003 Regarding the "politics" aspect of Lincoln's article I have to agree with Gary that it seems that the issue is being "politicized" by the recent actions of the Department of Education. While this is my personal view the actions of the Federal Government seem to define what is acceptable educational research. To remove all references to Pre-Bush education research from Department of Education Web pages and to possibly limit funding for ERIC amounts to Government censorship of any educational research that does not support the administration's views. As Lincoln stated, this is certainly chilling.

Karen: The additional articles Dr. B sent are very helpful. The last article on the list, Ryan& Hood, //Guarding the castle and opening the gates//, offers a suggestion that I believe makes a lot of sense, and seems to be appropriate for us given that it emphasizes broad research methods exposure in graduate education.

Qualitative and Quantitative: Implications for Graduate Education This commitment sets the stage for graduate training that is substantially broadened (Pelligrino & Goldman, 2002). Some suggest that a methods core include a balance of quantitative and qualitative methods (Pelligrino and Goldman, 2003.) Although we are concerned that trading breadth for depth may lead to less methods expertise, the notion that qualitative and quantitative methods are complementary is an important improvement in how the educational community talks about differences. We propose that enabling a mutual respect and regard for different methods is the greatest priority. We suggest this can best be accomplished in coursework devoted to differing epistemologies and philosophies of inquiry.This commitment sets the stage for graduate training that is substantially broadened (Pelligrino & Goldman, 2002). Some suggest that a methods core include a balance of quantitative and qualitative methods (Pelligrino & Goldman, 2003). Although we are concerned that trading breadth for depth may lead to less methods expertise, the notion that qualitative and quantitative methods are complementary is an important improvement in how the educational community talks about differences. We propose that enabling a mutual respect and regard for different methods is the greatest priority. We suggest this can best be accomplished in coursework devoted to differing epistemologies and philosophies of inquiry.

[|Pelligrino and Goldman, 2002]

Is there an article that refutes or disagrees with Lincoln or have we (and by we I mean Gary) found the motherload?
 * Karen:**

Another thought. In higher education we rarely find generalizability or reliability beyond the group to which we have access, even in quantitative research. Is it possible that because of this, qualitative research will find (may have already found) a stronger advocacy by use in the type of questions higher education asks of itself? By this I mean questions about student, faculty, and staff experiences in the higher education community?

I just finished reading the Lincoln article and am more impressed than when I first scanned it. There is a lot in the article, especially if we can follow up on some of the references she provides at the end of the article. Since she provides four questions, we could split up the references for the various sections (if anyone thinks it is worth doing) and see what we find with respect to the four questions that provide the framework for this article.
 * Gary:**

Her last question re: "The "politics" of evidence seems to me more to be the "politicization" of evidence. On the one hand, this seems almost extraneous to the issue that I think we are trying to address; on the other hand, it is like a worst-case scenario of the application of the "politics" question. Besides the ramifications it has with respect to funding concerns, it's an example of the diversionary tactics that keep us from looking at other issues that probably are of more substance (Yes, I realize that is a value judgment on my part). We seem to do this regularly in the political arena, and, apparently, it spills over into other arenas that are impacted by those political processes -- e.g. considerations of what constitutes evidence in the ways we study the world in which we live; and ways in which knowledge is constructed (or interpreted).

The Collins' quotation that leads into the Lincoln article discusses the paradigms we use to interpret and measure "reality". It seems to me that is the crux of our conversation -- shaping the paradigms by which we understand and speak about our "reality". What evidence is there for holding on to, or surrendering, the paradigms that help us to make sense of the world? I'm intrigued with this question from a couple of vantage points: I work in the behavioral sciences, which often are referred to as "soft" sciences. That old dualism of "hard" and "soft", however, is the same false dichotomy that feeds the polarization of quantitative as distinct from qualitative research, "measurable" data as distinct from "interpretive" data, and facts as distinct from stories. New paradigms will be necessary that can help bring these various "ways of knowing" together instead of pushing them apart.

In addition to some of the quotes Karen provided earlier from the Lincoln article, I also liked these: > Much of what we call "data" is itself phenomenological -- that is, socially constructed and "there" only becasue we are attuned to looking for it. (p. 3) > No "evidence" is evidence until we see it from some theoretical, paradigmatic, or metaphysical framework ... Thus what constitutes evidence, and therefore, what justifies it, is the result not only o fwhat questions are posed, but of the framework within which they are posed. (p. 4) > One of the sharpest and most cruel realizations of the postmodern critique has been that all knowledge is partial, incomplete, standpoint-determined,and, therefore, suspect in its claims to universality ... A second painful realization from the postmedern critique has been the growing understanding that some knowledges ... are more equal than others. (p. 8) > The issue of validity in qualitative evidence has taken on the qulaity of heavy freight ... (some) have called for an end to the cult of "criteriology" all together ... The entire question of validity, however, is far more complex than merely assigning new criteria to new methods ... different criteria of rigor and relevance apply when judging the nature of evidence ... Valid knowledges, it seems to me, have less to do with exclusivist, technical admissiblity criteria than it does with inclusionist, accessible knowledges and experiences. (p. 9-10) > Two characteristics of qualitative work mark it also as evidence, in and of itself: findings must be grounded in the situation examined, and if possible, comparisons will be created, and findings from one study should be 'translatable to other studies, theories or problems ... she does not call for "generalizability" but rather for "translatability" ...I would argue that it is possible to have both rigor and vigor in our qualitatve studies, but that frequently, trade-offs will have to be made." (p. 12) > The more dialogical the evidence, the likelier it is that multiple social constructions have been systematically collected, analyzed, and included in any research report. (p. 13)

Gary this is a very good article for you to read. It was one I read in my graduate studies when Yvonna Lincoln was drafting the article. I used it to argue a lot of my points in my dissertation. Great find! Congratulations!
 * Kakali**

Karen: from the Lincoln article that Gary mentions, which is very good by the way. Here are excerpts that will give you an idea of how we can use this article for our discussion.

…no one learns the truth by being told it. Everyone needs to experience truth in some way. > What do we mean by qualitative evidence? What weight do we give to the validityand rigor of that evidence? What are the assumptions behind different forms of evidence? And,what are the “politics” of evidence? Evidence, however, is data brought to bear on specific questions, theories or experiences. Evidence is data with a purpose. The purpose may be historical, theoretical, evaluative or descriptive, but it is arrayed to some larger end, and it is arrayed in a specific kind of order. Evidence represents data to which have been added a layer—or multiple layers—of interpretation and rhetorical strategy. > Data, however, is not evidence until two things happen: first, someone recognizes it as data, and second, an inquirer subjects it to some form of systematic analysis, which turns it into evidence directed toward some question or argument. What Collins argues here, that a set of invisible rules in part determine what constitutes evidence, demonstrates quite well the argument that what evidence is, and what constitutes evidence is determined in large part by the community to which one belongs, and the paradigm—or what counts, in a metaphysical sense, as knowledge—within that community. > Consequently, qualitative researchers take as a primary source of both data and evidence the accounts of meaning-making activities of individuals and groups.

Gary: Hi folks, I haven't had a chance to read it yet, but this article, entitled "On the Nature of Qualitative evidence", looks like it might be helpful -- http://www.usc.edu/dept/chepa/pdf/ASHE_lincoln.pdf. It looks packed with the kind of info I think we're seeking.

I also found the introduction of our CIQM book to have some good stuff, talking about the differences between qualitative and quantitative research and how the nature of "evidence" differs as a result of those respective frameworks. "Qualitative research is many things to many people. Its essence is twofold: a commitment to some version of the naturalistic, interpretive approach to its subject matter and an ongoing critique of the politics and methods of postpostivism" (p. 13). They go on, "...qualitative implies an emphasis on the qualities of entities and on processes and meanings that are not experimentally examined or measured (if measured at all) in terms of quantity, amount, intensity, or frequesncy. Qualitiatve researchers stress the socially constructed nature of reality, the intimate relationship betwwen the researcher and what is studied, and the situational constraints that shape inquiry ... quantitative studies emphasize the measurement and analysis of causal relationship between variables, not processes" (p. 13) They continue by differentiating the two types of research on the bases of five comparisons: uses of postivism and postposititvism, acceptance of postmaodern sensiblities, capturing the individual's point of view, examining the constraints of everyday life, and securing rich descriptions. these five comparisons, "reflect commitments to different styles of research, different epistemologies, and different forms of representation" (p. 16).

In reading that section of the CIQM book, it struck me that comparisons between these ways of doing research, on one level, make no sense at all because they are "measuring" (or "attesting to") completely different phenomena. Quality and quantity are not just alternative ways of viewing the same thing; they are different properties. At least on that level, it seems to me that the discussion of "the politics of evidence" belies this fundamental difference and conflates two properties that are foundationally different into comparable characteristics of the same phenomena. I suspect some of the urge to hold these two together is related to the history of qualitative research "having to prove itself" -- achieving its scientific stripes, as it were. The very act of engaging in that battle, however, violates the distinctiveness and valididty of each of the approaches as it stands on its own. One semantic consideration that helps me in looking at this issue, as it relates to qualitative research, is to think about the use of the term "evidence" not as a noun, but rather in its verb form -- "Evidence: 1. To indicate clearly: EXEMPLIFY; 2. To support by testimony: ATTEST" (Webster's Dictionary).

Some thoughts taken from Darby's Chapter 3 (using symbolic interactionism): > "Symbolic interaction emphasizes the meaning people attach to their social interactions and the world around them" (p. 68). > The Chicago School of thought (Blumer) believes, "the meaning people ascribe to the world cannot be standardized and that the self is continually evolving as it interacts with the world" (p. 69). "Participants use symbols such as language to understand their environment ... threee guiding principles (to this theory): 1) Human beings act toward things on the basis of the meanings that the things have for them; 2) The meanings of such things is derived from, or arises out of, tha social interactions that one has with one's fellows; and 3) these meanings are handled in and modified through an interpretive process used by the person in dealing with the things he or she encounters" (p. 69).

More from that piece tomorrow. Enough for today.

Gary: Hi group, I had a productive meeting with KB today. I'm going to be doing some reading about symbolic interactionism so I'll try to produce some info about evidence as it relates to that area. I think if we come up with evidence regarding the various methods we're planning to consider for our projects, we should be able to put together some useful info for this assignment as well. The Chapter 3's KB provided on the website can be helpful tools for looking at how these methodologies approach the issue of evidence. Any takers?

David, get us started. We are going to engage in a discussion of the iterative relationship between data, evidence and the construction of knowledge in the light of the politics of evidence. As we go along we will consider these ideas as they relate to different methodolgies and forms of analysis.