Tuesday, April 26, 2016

What's the Question?

Recently I wrote about several projects currently on my virtual desk, including fact-checking a book on a health topic. I wrote, “The book, which was researched and written by a journalist, is very interesting. The challenge with this project consists of deciphering some odd citations and tracking down source documents for specific facts and assertions. The latter can be very challenging, as it requires distinguishing between facts and assertions and unraveling how, over many years, those facts and assertions have been communicated and attributed. It’s never as straightforward as it seems.”

Yesterday I ran into a perfect example of this, which I will try to explain without violating my client’s confidentiality.

In her text, she described how people try to alleviate a specific health condition by engaging in any (or all) of eight specific behaviors, and added that the behaviors don’t make people feel better, and actually make the health condition worse. She asked me to “research this and find credible academic papers that show that [the eight specific behaviors] won’t alleviate [the condition].”

That sounds straightforward, but a broad request like that could end up taking an enormous amount of time (and money) and might not yield the desired result. Was it likely that researchers had investigated each of these eight behaviors, either alone or separately? If so, where might this research have been published? Was the research recent enough to be relevant to her purpose?

As always, before research can begin, one must determine if the client’s question is actually the “right” question. What does that mean?

In this case, her “question” was a request for me to find “credible academic papers” that backed up her claim, but what she really wanted was corroboration from a reliable source that the information she had was correct. Of course, in healthcare, peer-reviewed (“academic”) journals are at the apex of the authority hierarchy, but they are not the only reliable source.

In any case, I had a gut feeling that the answer lay elsewhere. This gut feeling came from the very wording of the phrase she wanted me to investigate; I can’t copy it here, of course, but there was just something about it that made me think that the answer did not lie in academia.

So, before I embarked on what could be an extensive (and expensive) search in academic databases for multiple papers that may or may not exist, I started out by looking for a single source for this particular assertion about all these activities, as a group. This involved some very creative and focused searching in Google, and within a few minutes I found the first clues. 

My search results were flooded with many resources that cited the same assertion that my client was questioning, and, interestingly enough, they used nearly the exact language that she had used. I noticed right away that all of these appearances were in the popular press – newspapers, magazines, self-help resources, blogs about the specific health topic – that is, none of them appeared in academic publications or ancillary sources related to academia or professional health care.

This sort of contextual observation is critical to successful searching. Information does not exist in isolation; it is always part of a contextual landscape, and deciphering that landscape and how the facts fit into it is essential for evaluating information as it is discovered, which in turn determines how the search must proceed.

At this point, the challenge was to evaluate these many different sources against each other to determine which of them was the earliest and most authoritative. Why? Because it was critical to determine who had originated the statement in order to learn what that person’s source had been; that reference was where I would find the answer to my client’s question. (And once I determined who had originated the statement, I could also disregard all the repetitions and later quotations, of which there were many dozens, a sort of distracting debris all over the information landscape.)

I traced the earliest appearance of the statement back to a self-help health book that had been published in 2011. With a little digging around online I found a PDF copy of the book (obviously pirated). There was the target statement on page 121, with a reference to the 2008 edition of an annual study published by a prominent professional health organization. Bingo. From there, it was easy.

In five minutes I had tracked down the publications related to this annual study, located the full text of the report for the year in question, and found the answer to the client’s question on two pages from this single document from an authoritative (and free!) source.

In my notes to the client about this particular bit of work, I mentioned that it had been fun to track this down. She replied, “That was so clever! Ever considered a career in investigative journalism?” I responded, “Yes, tracking down that particular information was really interesting and very rewarding! I've always thought that investigative journalism and the sort of research I do are closely related. Though I worked in libraries for years (hospital, insurance, engineering), I've come to love my independent research consulting practice. My clients give their most difficult questions to me, and expect me to find the answers! Much of my work includes a search for the best, most authoritative, most reliable information on a given topic, followed by writing about it in a way that people can understand - that's so similar to journalism.”

No comments:

Post a Comment