The Value and Challenge of Unsolicited References

I've been busy developing a new Information Fluency assessment and training program for Northwestern University's Center for Talent Development. As a result, I'm off my pace for posting new challenges.

The approach with the assessment and training is to focus on investigative searching skills. These are the competencies least engaged by Internet users. For one, they take time, although not as much time as you'd suspect. But the biggest reason is that the approaches are not easily "discoverable." They are best learned by training rather than trial and error.

One of the techniques, which I've written about before, is to investigate pages that link to the page or site being evaluated. This is accomplished by using the link: command. When I conduct workshops, very few people (teachers and students) already know about this operator.

Pages returned using the link: command are valuable because they constitute an unsolicited list of references. For the most part, authors of those pages voluntarily added a link to the site/page in question.

Just looking at a list of unsolicited references is not where the value lies. Herein lies the challenge. It requires reading and seeing patterns in the unsolicited references. This becomes difficult for students.

The easiest way to introduce link: to students is to see who links to bogus sites such as Help Save the Endangered Pacific Northwest Tree Octopus, etc.  The pages returned contain keywords such as 'hoax' which are easy to see in the snippets.

If a site/page is credible and a lot of highly respected organizations link to it that is also easy to detect. For example, links to the Institute for Library and Information Literacy Education include lots of libraries, schools and Websites for teachers. By association, if so many reputable sites link to it, it helps to establish credibility. Of course, just looking to see who links is not enough. What they have to say is also important. Plenty of schools link to the Tree Octopus site. What they say about it is what is important.

Querying link: for an organization you do not know may be helpful. By comparison, querying one you already know (like National Geographic) is a waste of time for the purpose of establishing credibility.

Seeing patterns in link: results is the challenge. And it becomes more difficult when the site/page being investigated doesn't lie at the extremes of universal respect or hoax.

The example I am wrestling with is an assessment item to determine if students know how to use the link: command. I'd also like to determine if they know how to make sense of the results. This is harder to do.  How would you intrepret the results for this query:  link:http://www.rense.com/general72/oinvent.htm   (Copy and paste this string into Yahoo).

The content is about suppression of alternative energy innovations. Over 500 pages link to it in Yahoo and many tend to support it. Does this mean it is credible? Who thinks it is credible is critically important. I'd like to hear how you interpret or wrestle with the results.

Detecting Bias

Objectivity is a characteristic of information that adds to its credibility. An objective article or posting either avoids taking sides or represents them fairly. An objective author typically has to overcome personal biases to write objectively.

Biases are hard to avoid. We all have them. (Did you catch my bias?)

Biases creep into our writing and into our reading. This latter condition can cause confusion for students who think they see bias in an author who really intended none. The bias occurs in the eyes of the beholder. I've seen this recently in students' response to an article by Carl Sagan, believing his intent was to frighten them. I was actually surprised by their conclusion and believe the words he used could unsettle someone--or make them feel existentially small--but that interpretation is in the mind of the reader.

Using bias or objectivity as the basis for accepting or rejecting information is problematic. Foremost, there's the problem of a skewed personal perspective that changes the way information was intended to be received. How can I tell if the problem is the author or me? Without other points of reference that may be impossible to determine.

That leaves a couple of options. Don't base the worth of information solely on whether it evidences bias or not. Include this in the mix of the author's qualifications, the publisher's reputation, the accuracy of claims and what others say about it.

The last point may be the critical one. If you believe in the "wisdom of the crowd" approach, then what a lot of other people think about an author's writing may come close to the truth. There's always the specter of 'group think' where the majority is wrong, but that tends not to happen in an open exchange of information or when information is volunteered freely. This is where checking to see what pages link to an author's work is extremely valuable.

The link: operator can be used to retrieve pages found in a search engine database (Yahoo's is the most powerful in this regard). Most of the external pages that link to a site don't have to. So why did they go to the trouble to put a link to a particular page on their site? Usually, it's to point to something of value (insight, humor, news, etc.) or to warn (as in the case of hoax sites). The best thing to ask before looking at the list of pages that link to information you want to evaluate is: "if this information is legitimate, who do I expect to find linking to it?" This involves some speculation on your part, but if you expect to find experts in a topic linking to, say, the Northwest Tree Octopus site (marine biologists, etc.) and you find none (which is the case), then you have the benefit of a crowd of witnesses expressing their personal views about the information. In the case of the Tree Octopus, people who link to it tend to find the information funny while others claim it's a fabrication. Based on the consensus of views you can determine there is insufficient support that the octopus has taken to the trees or is endangered.

Evidence of bias can also be found in the words an author uses. Strong language (often adjectives such as 'stupid' or worse) are seeded among biased views. The infamous Martin Luther King Jr. site is an example of this. Teaching students to ask, "why did an author use THAT word?" helps to uncover bias. As a student, I'm not sure I recall any of my teachers addressing how to detect bias, but it would have been valuable.

So I leave you with this challenge. Below is a link to a blog by an author who cannot be identified. Since he's self-published, there's no known publisher to fall back on. Using only the author's words, you can learn a lot about the individual. What would you say are this individual's values or biases? Under what circumstances would you use the information he authors? This could be a good activity for a group of students.

http://shroudedindoubt.typepad.com/

For more on the link: operator and how to use it, visit this tutorial.

Link To Evaluation

Knowing who links to a site can be very revealing.

If trustworthy people link to a site and say positive things about it, does that site gain in credibility? I think most people would say yes.

What if they just link to the site and say nothing about it? That's harder to evaluate.

Using the link: command by itself is not an evaluation shortcut. For example, pages that link to http://golfcross.com/ present ambiguous results. Google returns 4 pages: two of them are about hoax sites, another is an account of someone playing golfcross and the fourth is the 21cif website. Since three fourths of the pages seem to indicate there is some suspicious about golfcross, one might be tempted to conclude the sport is really a hoax. But that's not an accurate conclusion.

One of the problems is that Google no longer returns all the pages that link to a page. Within the last year, only a sample of the pages is returned. Try the link: search in Yahoo and you get
lots more (over 400), if you select the option for picking pages from the entire site, not just the home page. That's a good reason to try more than one database when searching.

Hoax-related pages still show up in Yahoo results, but now there are others: travel sites, blogs, wikipedia, and so on. Now it isn't so easy to conclude that the sport is a hoax.

It still requires reading and interpreting the pages that have a link to golfcross to figure out why there's a link there. Always ask: why did this author include the link?

Among the Yahoo results is a blog by Bernie DeKoven. The context of the page is all about fun and games, including wallyball, slamball and this game played with egg-shaped balls. A link to Bernie DeKoven leaves the impression that he is educated, was a teacher, is an author and has pursued game-playing as a serious pastime for years. He seems to be an expert in games. So does his testimony convince you that golfcross is real?

Maybe you know Bernie and maybe you don't. If you do, does his testimony persuade you that golfcross is legitimate? Does he say strong enough things about the sport?

If you could get to know Bernie and ask him why he thinks golfcross is legitimate, that might help. That's where Web 2.0 becomes very valuable. You can ask questions, join personal networks and get a lot more information than if you were just observing from a distance. Of course this takes a little more time than lurking, but it yields better information.

So, what do you think about golfcross?

An opportunity to learn more about Web 2.0 tools and evaluate will start on Feb 9. Consider joining our 4 week course on Power Searching in a Web 2.0 World. Here's a link to look at the course.