Yes, There's a Need - Part 2

Using performance assessments to study how people search is very revealing. Several years ago, we identified five major things that students don't know how to do that keeps them from achieving information fluency. In case you're unfamiliar with our work, these are 1) turning questions into queries, 2) selecting appropriate databases, 3) recognizing relevant results/finding better keywords, 4) evaluating the credibility of information, and 5) using information ethically (creating citations).

Upon closer examination, there are five problem areas related to evaluation alone--the facet of information fluency where students' shortcoming are most evident. These have to do with 1) reading carefully, 2) understanding different types of queries and when to apply them, 3) navigating by browsing effectively, 4) truncating urls and 5) checking for page information. Not knowing about a technique or how to apply it is one thing. Even when students (and adults) know the techniques, going too fast tends to erase any advantage the techniques might provide.

Speed is likely the biggest factor responsible for ineffective, inefficient searching. In terms of efficiency, it is ironic that the slower you go the better you perform.

We've consistently noticed "the-slower-the-better" relationship while conducting research. The faster a person goes, the more likely it is that important clues will be overlooked. The place this really has a negative impact on information fluency is (not) finding better keywords in snippets and (not) tracking down leads that can be used in determining the credibility of an author or an author's work. It is possible to find good clues by scanning, but the speed at which students scan a page renders understanding what is read unlikely. This is why we advise students taking our assessments to take their time.

Speed tends to be the factor that explains why a student scores higher on a pretest than a post-test. Our current assessment work utilizes a pair of 10-item tests.* Students who take less than 20 minutes tend not to score very well. That's 2 minutes per item, most of which require submitting a query or navigating to find answers, reading content to find clues and checking on those clues with a subsequent search. It's very hard to do that in 2 minutes. Deliberately slowing down, taking time to look for clues (because you know they are there) makes a big difference.

Because people have other things they want to do with their time, knowing when to slow down and conduct a careful search is paramount. My advice is, slow down when the stakes start to get high. For some, this will be when a grade or money is on the line, or a performance review or a job. Not every search demands the same careful consideration. But when it does, you need to take the time.

So time yourself. After becoming familiar with the instructions, give yourself two minutes to try the following search activity. Without looking at the possible answers (click 'give me a do-over' rather than 'show me the words'), allow yourself five minutes and do it again. Do you notice a difference in your score?

In what contest is one of the prizes a Hasbro action figure of the winner?

http://21cif.com/rkitp/challenge/v1n3/snippetsleuth/ActionFigure/SS_ActionFigure.swf

--

*If you'd like to try the 10-item tests yourself, sign up for Investigative Searching 20/10 (sign in as guest for access). The assessments are featured in that course.

Searching and Asking Better Questions

An early mistake made in the search process is looking for the wrong thing. From our research this happens around 10% of the time. No matter how you phrase a question, about one in ten people will misinterpret it. Maybe more.

Not only is looking for the wrong thing often the first mistake one can make, it is THE unforgivable one. Using poor keywords, looking in the wrong database, browsing ineffectively are all costly in terms of time, but they an lead to a good answer with persistence and a little luck. Looking for the wrong thing, well that's a different story.

I recall the time we asked students to research the World War II perspectives of people living in the Middle West. Sure enough, there were some students who turned their attention to the Middle East. A fairly easy mistake, but fatal in terms of search success. (By the way these were highly gifted high school students--no one is immune).

Yesterday I led a workshop for grad students at a local university. I used the Declaration of Independence search challenge that I posted on July 4 (see two blogs ago) as a warm-up activity to get an idea about their search skills. It confirmed a reservation I've had about search questions for several years: there is no perfect question.

Unless the question reads like a legal document--and who really wants that--there is almost always a way to misinterpret it. This may be particularly problematic in the case of search challenge questions. It takes a bit of work to come up with a question that doesn't give away the answer. By design, one or more powerful keywords for a search are left out of the question. That's to replicate the reality of what we call the "1 in 5 rule:" on average, there are four other words than the one you used in a query that may be more effective. Finding those better keywords is the most common search challenge.

It is enlightening to see what a group will do with a question. Every time I try this, I think of a better way to pose a question. And I've had a lot of experience designing questions. That's an unintended consequence of using a search challenge, but has a lot of importance for teachers who use questioning in the classroom. It's a mistake to think you've asked a good question until you see what students do with it. If you don't catch the misunderstandings in the beginning, a lot of effort can be wasted, both students' and teachers'.

Here's an illustration using the question: "How many original copies of the Declaration of Independence are known to exist?"

If you take 'original' without considering 'copies' you might think this is a trick question. Of course, there's only one original: the one in the National Archives with all the original signatures on it. 'Copies' is a bit misleading. What the search is about is 'printed copies', of course there is a better keyword than this--and it can be discovered by searching with the word 'copy' or 'copies'. The better keyword is 'Dunlap Broadsides'. That's the printer and the name of the 200 copies of the Declaration of Independence he printed soon after the original was signed.

There were also questions (confusion) among the students about the meaning of 'known to exist'. Many of the grad students found the right answer despite having to overcome these hurdles. Some found a wrong answer. Some were still searching when I called time. If I would have included the term 'printed' in the question, or 'Dunlap Broadsides', more would have found the answer. Of course, that would have made this a pretty weak challenge.

Here's my point: if you're a teacher and want to improve the questions you give your students, try giving it to them first as a search challenge question. See what they do. What keywords do they use in a query? What words do they think are unimportant? What do they find as they search? Do they get stuck? In most cases, I think you'll find a better way to ask your question.

Detecting Bias

Objectivity is a characteristic of information that adds to its credibility. An objective article or posting either avoids taking sides or represents them fairly. An objective author typically has to overcome personal biases to write objectively.

Biases are hard to avoid. We all have them. (Did you catch my bias?)

Biases creep into our writing and into our reading. This latter condition can cause confusion for students who think they see bias in an author who really intended none. The bias occurs in the eyes of the beholder. I've seen this recently in students' response to an article by Carl Sagan, believing his intent was to frighten them. I was actually surprised by their conclusion and believe the words he used could unsettle someone--or make them feel existentially small--but that interpretation is in the mind of the reader.

Using bias or objectivity as the basis for accepting or rejecting information is problematic. Foremost, there's the problem of a skewed personal perspective that changes the way information was intended to be received. How can I tell if the problem is the author or me? Without other points of reference that may be impossible to determine.

That leaves a couple of options. Don't base the worth of information solely on whether it evidences bias or not. Include this in the mix of the author's qualifications, the publisher's reputation, the accuracy of claims and what others say about it.

The last point may be the critical one. If you believe in the "wisdom of the crowd" approach, then what a lot of other people think about an author's writing may come close to the truth. There's always the specter of 'group think' where the majority is wrong, but that tends not to happen in an open exchange of information or when information is volunteered freely. This is where checking to see what pages link to an author's work is extremely valuable.

The link: operator can be used to retrieve pages found in a search engine database (Yahoo's is the most powerful in this regard). Most of the external pages that link to a site don't have to. So why did they go to the trouble to put a link to a particular page on their site? Usually, it's to point to something of value (insight, humor, news, etc.) or to warn (as in the case of hoax sites). The best thing to ask before looking at the list of pages that link to information you want to evaluate is: "if this information is legitimate, who do I expect to find linking to it?" This involves some speculation on your part, but if you expect to find experts in a topic linking to, say, the Northwest Tree Octopus site (marine biologists, etc.) and you find none (which is the case), then you have the benefit of a crowd of witnesses expressing their personal views about the information. In the case of the Tree Octopus, people who link to it tend to find the information funny while others claim it's a fabrication. Based on the consensus of views you can determine there is insufficient support that the octopus has taken to the trees or is endangered.

Evidence of bias can also be found in the words an author uses. Strong language (often adjectives such as 'stupid' or worse) are seeded among biased views. The infamous Martin Luther King Jr. site is an example of this. Teaching students to ask, "why did an author use THAT word?" helps to uncover bias. As a student, I'm not sure I recall any of my teachers addressing how to detect bias, but it would have been valuable.

So I leave you with this challenge. Below is a link to a blog by an author who cannot be identified. Since he's self-published, there's no known publisher to fall back on. Using only the author's words, you can learn a lot about the individual. What would you say are this individual's values or biases? Under what circumstances would you use the information he authors? This could be a good activity for a group of students.

http://shroudedindoubt.typepad.com/

For more on the link: operator and how to use it, visit this tutorial.

Happy July 4!

Here's a new Search Challenge in time for Independence Day. It was inspired by reports of the discovery of a copy of the Declaration of Independence in the National Archives in London. Originally, around 200 copies were printed. The whereabouts of the majority of these is unknown.

http://21cif.com/tutorials/challenge/SC001/SC_034.swf


See if you can rise to the challenge.

Information Forensics goes to School

Yesterday, Dennis O'Connor and I presented "Information Forensics goes to School" at the NECC conference in Washington, D.C.*

We talked about how the investigative aspects of information fluency, which constitutes part of the missing curriculum at most schools, cuts across all subject matters, 21st Century Skills as well as ethics. The ability to track down and evaluate information about authors, publishers, publication dates, content and references is vital to research in any subject. The techniques used to locate and evaluate such information is integral to ICT literacy and the ability to cite information is definitely an aspect of ethics in scholarship. It's deplorable that the task of helping students develop these skills and attitudes is most often left to the school librarian. The entire faculty should share this responsibility.

We didn't actually rant that much during the session, but it needs to be said.

We also introduced a new resource being used by nearly 1,000 students at the Center for Talent Development at Northwestern University this summer: Investigative Searching 20/10. This is a performance-based assessment and training package designed using Moodle and an array of Websites we created specifically for the purpose of diagnosing students' skills. The focus is on investigative searching and evaluation.

The results are as we predicted: even the most gifted students in middle school and high school, on average, cannot locate critical online information and evaluate it. Without someone to show them effective techniques, they browse hoping to stumble upon information that may be helpful to identify an author, a publisher, the date of publication, etc. Couple this with students' general "need for speed"--which contributes to important information being overlooked--and you have a recipe for failure and poor research.

Following a pretest which requires students to demonstrate techniques like querying and truncation, a series of self-paced tutorials steps through 5 main techniques, each followed by a mastery check activity. The 6-8 hour experience is capped off with a posttest of items comparable to the pretest. Students, on average, improve 15 to 20 percentage points. Not bad for a few hours work. The improvement would be even greater with continued use.

Sound like something you could use?

We are offering educators a chance to experience this package starting July 7. You may enroll up to July 28. We'll leave this section just for educators open for 4 weeks, although it takes less than a week to complete it. Our hope is that you will mine it and discover ways to use it with students as part of a systematic approach to online research or as part of an instructional unit. The full preview is available for a modest fee ($25); 8 CPDUs are awarded for completion.

To learn more about Investigative Searching 20/10, visit this link. (log in as guest)

From the tutorial on finding the author, here's a taste of what you'll get.

Find the first and last name of the author whose initials appear at the bottom of this article: http://www.spacestaking.com/about/mission/greenvision.html

As you will discover from the associated tutorials, there are several ways you might try to solve this: browsing, querying and truncation. One of these is the most effective given the circumstances. See if you can do it.



*Our NECC presentation was recorded for anyone interested who wasn't there--I'll post the link when Apple publishes it.

Finding the Black Box

Homing in on information online is similar to locating a downed aircraft's black box in one respect: both send out signals. In the case of the black box, the signal gets stronger as you approach. In the case of information online, the contextual evidence (think clues) gets stronger.

Detecting those clues and interpreting them can be very difficult.

Homing in on information online depends on browsing for keyword evidence. Some people have a distinct advantage in this regard. Masterful browsers tend to have fairly developed language skills. This, coupled with a disposition to read carefully, puts them far ahead of otherwise skilled computer users. I know adults often feel inadequate when working with kids with lightning fast computer abilities. But adults, despite their technological shortcomings, tend to outperform prodigious children and teens when it comes to homing in on information.

I witnessed an example of this last evening when my wife pilot tested an assessment that Information Fluency is preparing to use with Center for Talent Development students at Northwestern University. Pat, not a computer user from birth, took about an hour to complete the 10 item pretest, most of which depends on careful reading and browsing--after all, investigative searching relies on being thorough and examining lots of clues. She only missed 2 items, which is typical of an adult who has mastered searching and evaluation. Middle school and high school students will take about half that time and miss the majority of the items.

The major difference boils down to reading and browsing carefully. Sure, there are other techniques in a skilled online investigator's toolkit, but careful reading and browsing can be used to solve most search challenges.

1. See every word as a clue It's really easy to overlook important keywords when skimming. The words and terms that matter most are typically nouns and numbers. Adjectives become important in detecting bias. Spotting these is easier by slowing down. If you have the attitude that words are clues--not just the specific keyword(s) you are looking for--you will probably have to adjust your speed.

2. Recognize and follow possible connections Here is where terms you weren't looking for become important and why it helps to have a good vocabulary. Synonyms and words used in the context of what you are looking for are all possible connections. It may help to think about what some of these other words are before reading. In terms of the Digital Information Fluency Model, this is known as finding better keywords as you search. You can't predict with 100% accuracy what keywords are necessary before you search. You have to pay attention and find them as you search.

An example may help Let's say you're looking for the publisher of this site: http://www.spacetoday.net. First steps are usually to read the header and footer for the name of an organization or copyright information. Not seeing those, what stands out is the About Us link. Let the browsing begin. The name of the publisher is in the text of this page. In this case, it's an individual: "Spacetoday.net was founded by Jeff Foust." This wasn't a particularly difficult challenge, but many students will miss it because the name is embedded in the text. Moreover, Mr. Foust's role is represented with the word "founded" rather than "published." You have to know what these words mean, how they are used and related. Is the founder always the publisher? Not always. To make sure we've got the publisher, another source of information is required.

Here's your Challenge: Can you find another source of information that confirms that Jeff Foust is actually the publisher? It might take another technique in addition to careful reading and browsing.

Five Keys to Investigative Searching

When it comes right down to it, there aren't a lot of techniques involved in investigative searching. As opposed to speculative searching--what we tend to do most of the time--investigative searching involves clear clues and places to look for information about credibility.

In speculative searching, we're not sure where to look (Google is the default) and what keywords we need to retrieve the information we are looking for.

The goals of investigation are clear: find and evaluate the author, find and evaluate the publisher, find and interpret the meaning of the date, find evidence and evaluate it for accuracy, objectivity and external verification. You don't need to collect all this information unless the stakes for using information are high (like your job depends on it or you'll be punished if you plagiarize or violate fair use). Most of the time, checking a few facts is enough.

It may be helpful to think about investigative search techniques as depending on careful reading and four other keys. Nothing takes the place of reading thoughtfully. Still, the tendency of students is to go as fast as possible, thereby overlooking important information and clues about credibility. I'd say that's the biggest problem and why students trust false information.

The rest of the problem, as I see it, is that students aren't taught some basic techniques. These are the four keys I'm referring to:


Queries [Enter Key]

Using a search engine to track down missing information and check its credibility is the most powerful tool in the investigator’s kit. Depending on the information you need, the database you search will vary. For most queries, Google is preferred. But if you need to look up the registered owner of a website—to find the publisher or someone to contact—that requires a different database: whois.net. If you need to find a list of pages that link to the page you are investigating, then Yahoo is preferred, as it returns more information than Google with the link: command.

There are keyword queries, when you need to check the facts about a person, a publisher or something an author wrote in a web page. If you can’t find information to back up an author’s claim, that lends no support to its credibility. Sometimes, you’ll find that other people have information that contradicts the author you are investigating. This is all part of triangulating: finding information from multiple sources that agrees. When that happens, there’s a better chance the information can be trusted.

There are also string queries. This is when you copy a portion of text and search for the exact phrase in a database. You may place quotes around a passage, but it’s not necessary. Quotes work best around a first and last name or a short phrase. By searching for a string, you will often find other instances of the article or person you are investigating. The other instance may contain additional information you need. You may also discover that the passage you are investigating was plagiarized: copied word for word without being cited.


Truncation [Backspace]

The URL is an important source of information. It can reveal the publisher’s name or whether the site is self-published by the author. Truncation is a good way to navigate toward the root of the site you are investigating. Not all the information you need may be found on a web page, but a directory page may contain important clues such as an author’s name, a publisher or the date an article was written. You truncate by removing parts of the URL with the backspace or delete key, starting from the right and stopping when you reach a folder marker (/).


Browsing [Left Click]

Browsing is a special kind of reading: paying attention to hyperlinks. Normally when you browse you are looking for words or images that stand out. On a web page, browsing is defined by the links you click. Think of successful browsing as a game of HOT and COLD. You are trying to use links to get you closer to information you need, though you’re not exactly sure where that information is hiding. Whenever you click a link you have to scan or read the new page to discover whether you are getting hotter or colder. If you seem to be getting colder, go back and try a different link or technique. If you are getting hotter you will discover keywords and clues that you can use in your investigation.



Page Information [Right Click]

If you are using a browser that provides page information (Firefox does this) you can right-click on a web page to bring up a menu that includes Page Information. Depending on how the web page was coded, information about the last time the page was updated may be provided. If the last updated information is ‘now’ then the coding on the web page doesn’t allow this information to be shown. Knowing the last update of a page can be helpful in determining the age of the material.



This is obviously only an introduction to the techniques, but with them you should be able to solve this challenge:

Who is the author of the Sellafield Zoo? http://www.brookview.karoo.net/Sellafield_Zoo/

Challenge Level: Intermediate (Don't forget careful reading!)

Visual Literacy - First Steps

Here's how I would approach the challenge I posted in the previous blog. The two images, both showing sharks and humans interacting, could be real or fake. One approach is to examine the photos for evidence of 1) tampering and 2) clues for further investigation. A second approach is to search for commentary information about the photos in question.

If the photos are small, it may be difficult to detect any suspicious alterations. In this case, it may be possible to locate larger images by describing what you see. A Google Images search is more powerful than it used to be. A few years ago, too many keywords would bring it to its knees. But now it is possible to use more terms in a query and get results. (That being said, it's always a good idea to use as few keywords as possible since matching each word reduces the likelihood of getting the desired result.)

An image search for the shark and kayaker could simply be shark kayak. This indeed does fetch a larger image--and several related images which could come in handy. A query for the photo of the scuba diver hitching a ride on the shark's fin could simply be shark diver hitch. This query returns a different image of a diver hitching a ride on a shark, which adds some credibility to the claim that a diver might be able to do this. The query shark diver dorsal ride retrieves a larger version of the image shown above.

A closer examination of the shark-kayak photo reveals some irregularities. First, there is a smooth water reflection of the kayaker despite the fact that the surface of the water elsewhere is non-reflective. This suggests that the kayak was pasted into the scene. The photo of the shark could be real. The words on the photo are also helpful: "sitting in a 3.8 meter sea kayak, etc." A Google fact-check of this phrase returns a wealth of pages discussing the photo. The first one of these is: Scary Shark: Real or Not?

Readers' comments are mixed: some think it's real, others don't. This is a good way to use Web 2.0 information to see what the 'crowd' thinks. Taken as a whole, the crowd is usually correct. Of interest here is that a different photo of a kayaker and shark is referenced. This alternate view causes some to doubt whether the photo is actually a fabrication. The reflection is about the same, but now the view of the kayak is close-up and the kayaker is looking a different direction. Now the investigative problem includes the question: were two different photos of the same content faked? Is that likely?

Reader's comments are about 50-50 whether this is real or fake. One of the readers links to snopes.com which claims the photo is REAL. According to snopes it was taken from this source: http://www.whitesharktrust.org/pages/mediaarticle/media25.html. You can see the first appearance of the photo there: a two-page spread in a magazine, hence the line down the middle where the two pages don't exactly match up.

Looking at other returns in Google for the photo reveal a photographer: The author of the photograph is Thomas P. Peschak, a marine biologist at the University of Cape Town’s Marine Biology Research Institute. That name is another great investigative clue. Here's his webpage.

This is what the photographer says about the photo: "When this photograph was first published in Africa Geographic, BBC Wildlife and later in Paris Match and the Daily Mail (London) it resulted in a flurry of e-mails, phone calls and letters from around the world asking if the image was a fake. Following publication no less than 50 websites were hosting colorful discussions about the authenticity of this photograph. The image became probably the most talked about of shark photograph ever and has become my undisputed best seller." [Source]

The other photo--of the diver hitching a ride on a shark's fin--is probably real as well since there are other types of photos of divers hanging onto sharks--too many to all be faked. A similar process could be used to track down references and even who took the picture.

Surprised? Disagree?

At the Intersection of Web 2.0 and Information Fluency

I've had the opportunity to lead a number of workshops this year entitled, "Power Searching in a Web 2.0 World." Sometimes the sessions are 45 minutes long; other times I've had 6 hours. No matter the length of time, it's difficult to wrap one's arms around Web 2.0.

There are a host of new tools that debut weekly in the Web 2.0 world. Just staying current takes hours. And while people are generally interested in what's new, they also want to know how information fluency intersects with this world. That takes a chunk of time by itself.

I usually start by comparing two worlds: Web 1.0 and Web 2.0. There are some simple comparisons. For example, in Web 1.0 you can only lurk. You can't leave your mark. In Web 2.0 you may lurk, but that's not really the way to discover the value that's there. Leaving your mark creates value. That's typically accomplished by blogging, contributing to a wiki, saving a bookmark, uploading an image or a video or commenting on someone's else efforts to do so.

Here are a few more global insights about Web 2.0 from an information fluency perspective:

• As much as Web 2.0 is all about contributing, equally important is networking. These two words sum up most of what there is to know about Web 2.0. At the intersection of Web 2.0 and Information Fluency, the purpose of the environment cannot be overlooked. Finding the 'right' bit of information may require participation and networking; and that takes time.
• You may get to know an author by joining his or her network, you may build trust by being a participant. The investment of time may result in the formation of your own personal learning network--which could be very powerful and long-lasting. Some of the people who benefit most from Web 2.0 have built large networks of colleagues who provide help on demand. For example, I've seen Will Richardson use Twitter during a workshop to harvest responses from his network. The larger your network, the more powerful it becomes (also know as the network effect). Applying Information Fluency skills in this environment can consume a great deal of time. But you may end up with much more than the piece of information you set out to find.
• You have to limit your choices. How many networks can you realistically join? I have memberships in Facebook, Twitter, Digg, Diigo, Delicious, Flickr, Wikipedia and Linked In. I rarely post in most of them. You have to choose where you want to spend your time. Linked In is a good investment if you're trying to build professional (business) relationships. I see a lot of Ed Tech folks in Twitter. I connect mainly with members of my family in Facebook.
  
• Web 2.0 is a public place. The information you contribute tells something about you. The more you share, they more others will know, if they're interested. You may try to hide your identity, but your readers will discover a lot about you even if they never learn your real name. You have limited control over who is able to access your information.
  
• Speculative searching doesn't work as well in Web 2.0 as it does in Web 1.0. Part of the reason for this is that search engines return the most recent information first. For example, if you are looking for a post that's more than a month old in Technorati, you have to page down and down (Google Blog Search makes it possible to search by date range). Another reason is that searching based on tags is less likely to return predictable results than keywords drawn from the context of a whole page. Tags are keywords, of course, but they are chosen by the author to characterize the information as a whole. You aren't searching the whole information. Search for a specific photo in flickr sometime. It's hard to do. You may find pictures with similar qualities but not an exact match. Try it (here's the challenge in this post).
  
• A better use of speculative searching is not to start with ideas that are too specific. Use Web 2.0 to open your eyes to new information. Exploring tagclouds is a brainstorming activity.
  
• Finally--and by no means is this the last word!--the wisdom of crowds is active in Web 2.0. The more people and perspectives that weigh in on a topic makes the information more reliable. This is the principle of collective intelligence behind the genius of Wikipedia. This is the phenomenon behind what becomes a popular item in DIGG. This may even be why Fake Steve Jobs has over 52,000 followers in Twitter, while other Steve Jobs have only a fraction of that. Has the crowd found the Real Steve Jobs? See for yourself.