Fifth BadApple Challenge and Scoring

 

Just added: a fifth Challenge to our new Bad Apple investigation game. This one tests individual skills using a current controversy in the news: The Thomas Jefferson Statue in New York City.

Also new is a scoring feature that assesses investigative skills. Demonstrate Information Fluency by earning a score of 80% accuracy or higher. Pick up valuable investigative tips and tools along the way. Tutorials are suggested if you want to improve your score.

Play today or with your students:  https://21cif.com/tutorials/evaluation/badapple/

Election Year Bias

 

If you are looking for examples of bias, it's hard to beat an election year. The 2020 national election in the United States stands out in this regard. Two sides stand in stark opposition: Republicans and Democrats.

The intent of this article is not to align with one side or the other. Instead, the purpose is to strengthen investigative search skills by engaging in bias detection. The investigative targets are two fund-raising letters, one sent by President Donald Trump and the other by Speaker of the House Nancy Pelosi. Both were mailed during the summer of 2020. Both letters are biased in favor of respective party positions and against the other party. This is completely normal. No matter the candidates, bias for and against are intended to get voters to donate money.

To read the full article and help students better understand bias, click here

The Feature Article is available without a subscription. An individual or school subscription is required to access the Curriculum applications and Assessment resources. 

Spring 18 Full Circle Kit now available

The newest edition of the 21st Century Information Fluency Full Circle Kit focuses on Fact Checking.

This is part three of three in a series on Investigative Searching. Access to the Kit requires an annual membership.

All Kits

Spring '18 Contents

Feature Article: Fake News, Part Three

Researchers at MIT recently published their findings about the spread of false news on Twitter. There is still no substitute for investigative searching. 

Action Zone: Fact Checking Challenge

This Level 3 challenge may be used with the Assessment guide to check students' understanding of fact checking.

Curriculum: Mini Lessons

Using two Websites about fake news to create mini lessons on fake news and fact checking.

Assessment: Fact Checking and Secondary Searching

Five items to measure how well students know when to Fact Checking and how.

WSI: Web Site Investigator

Dennis O'Connor is having students in his online course at the University of Wisconsin-Stout work through the activities in Web Site Investigator. We updated this package recently and it has proven to be an effective way for students to teach themselves effective Web evaluation skills.

It's well-known (from ours and other's research*) that students spend little time evaluating the information they retrieve from the Web.

For this reason, we created a detective game that incorporates information evaluation as the forensics activity. WSI: Web Site Investigator features four Web sites where students look for evidence that the information is credible or suspect:

• The Air Car (a car that runs on air)
• New Zealand Golf Cross (a golf game played with an elliptical ball)
• The Pacific Northwest Tree Octopus (an endangered cephalopod found in trees)
• Sorting Hat Personality Test (an online questionnaire that determines your Hogwarts house)

The WSI package provides a background article on each site and a link to the online site itself. Also included in the package are eight self-paced tutorials on investigative techniques that may be used to determine if the sites are bonafide or bogus:

• Author
• Publisher
• Date
• Bias
• Links To
• Links From
• Accuracy
• Evidence

In addition, there is an Evaluation Wizard for each of the four sites that investigators may use to file a case report to turn in if the package is used as part of a course. Dennis has his students complete and submit a Case Report on one of the sites selected by the student. For the Case Report, each student examines three aspects from the bulleted list above in order to back up a claim that the site is trustworthy or not. The deeper the investigation goes, the more interesting the discoveries--it's actually fun.  The objective is that students understand how investigative techniques may be used and start to question the veracity of EVERYTHING they find online. You can't tell if information can be trusted without investigation.

Try it out for yourself!

* The reader is directed to our new book Teaching Information Fluency for more information on student's lack of evaluation skills.

Citation Challenges

Tracking down missing information for a citation can be frustrating. Even though citation styles like MLA, Chicago and APA make allowances for missing information (e.g., no author, no date), many times all the information is available.
Tracking down elusive information takes some practice, careful scanning and investigative searching techniques.
For this reason, we created the Citation Challenge and Citation Assessment activities.

Citation Challenge offers three challenges of increasing complexity.

• The first level requires players to identify what information satisfies citation requirements. For example, what is the Author's name, the date of publication, the publisher, the Web page, etc.
• Level two is a live Internet search to locate information on an article on a NASA site. The player must locate the name of the author, date, etc. by scanning and browsing.
• Level three is another live search that awards extra points for inputting the missing information in MLA style.
• The whole package is an interactive companion to the Citation MicroModule on the 21cif.com site.

The Citation Assessment offers four levels of difficulty from beginner to advanced. The beginning levels require finding information on a web page with no further searching. The advanced levels require tracking down the original source of the information, using truncation to navigate to directories and metadata to determine last modified dates. Each level has two live searches. Players get a percentage score at the end and can print the results to hand in. We don't recommend using these activities as the basis for a grade. However, they do make good benchmarks for demonstrating skills and progress.

A Shark, a Kayak and a Tree Octopus

Back in May 2009 I posted a visual information fluency challenge about a shark and a kayak.

I'm revisiting that challenge because of a new tool I came across recently. Error Level Analysis is a software application that can be used to detect possible jpeg image tampering or touch-ups. To keep it simple, digital images are composed of pixels. Every time an image is copied, the pixels lose some of their original sharpness. The more an image is copied, the more it degrades. Then if something is touched up or added to the copied image, the new part of the image has less error than its surroundings. Read more here.

If you subject the edited image to error level analysis, the newer parts stand out against a background of noise. A good example of this is found at http://errorlevelanalysis.com/ -- roll over the picture of the model on that page.

I ran an information fluency workshop last week at ICE (Illinois Computing Educators) and we used this tool with the image of the shark and the kayak. We also tried it with an image from the infamous Pacific Northwest Tree Octopus site, shown here.

Try it. The original Shark and Kayak picture is attributed to Thomas Peschak. Using error level analysis, the jpeg on his website does not appear to be the original image.  Go to http://errorlevelanalysis.com/ and paste in the url of the image: http://www.thomaspeschak.com/storage/White%20Shark%20KayakThomas%20P.%20Peschak.jpg.  See the white outlines of the copyright text? You know that wasn't on the original. The outlines around the shark and kayak are unmistakable.

This poses an interesting investigative situation. Based on all the evidence, except for error level analysis, I'd have to conclude that what this photo shows actually happened. There is so much going for the validity of the story: published in a reputable magazine, a credible author/photographer, even Snopes.com says the image is real.

Then why does it appear to be touched up? And what is the extent of the addition? Maybe the shark and the kayak were merely enhanced. Maybe they were added later. Hard to tell.  One thing's for sure: something is different about this image since it was first snapped.

A second image that HAS to be fake comes from the tree octopus site.  There's no doubt that the tree octopus is a fabrication. Yet when you do error level analysis with http://zapatopi.net/treeoctopus/treeocto.jpg, you see what an image that hasn't been photoshopped should look like.  The original photo could have been some type of stuffed or toy octopus intentionally posed on a tree limb--although the octopus appears to have an outline that makes it look out of context.  Blowing up the image might reveal something about that outline.

Clearly, you still have to use your brain and other evidence to tell when an image is real or faked. A tool like error level analysis can help in doing investigation, but you can't rely on it alone.

Challenge: try some favorite images from the web and see what they reveal. Any surprises?

Searching for the Correct Answer

The new report from Project Information Literacy describes the research predicament students face as they go to college. 

• 84 percent of students have difficulty getting started with course-based research
• students take how they learned to research in high school with them to college
• students regard research as a search for the correct answer

It's the last one that shows how far the search process has wandered off course. As Alison J. Head, an investigator on the study has said,

"Not being aware of the diverse resources that exist or the different ways knowledge is created and shared is dangerous. College is a time to find information and learn about multiple arguments, and exploring gets sacrificed if you conduct research in this way."  (quoted in the Chronicle of Higher Education)

I suspect that searching for the "correct" answer stems from research experiences in high school. There certainly could be tensions over adopting this approach in middle school and high school.

First, let me engage in some self-criticism. Many of the Search Challenges that are found on the 21st Century Information Fluency site have a correct answer. In some cases there are multiple correct answers, but this does not reinforce the concept that diverse resources should be investigated. A diet consisting solely of Internet Search Challenges would lead most students to conclude that searching is totally about finding an answer to a question.

21cif Search Challenges, for the most part, exist to improve speculative searching by:

• strengthening ability to turn questions into queries
• helping students identify databases that are likely to contain relevant information
• helping students modify searches by careful examination of snippet results (abstracts)

Are these enough to prepare students for post-high school research challenges? I doubt it. However, those bulleted objectives are all necessary--without them, research is accidental.

The point is that students need more than practice honing skills, strategies and techniques that are useful for finding answers. They need to be challenged to look for rival answers, then evaluate them. If students  learn only to be better speculative searchers in high school, they are half-prepared for what comes next.  Investigative searching looks beyond first answers to see if answers hold up. In the process, fact-checking, investigating authors' credentials, etc. often turns up different points of view. 

Is Investigative Searching being taught in high school? You tell me.

Is the Internet Making Us Stupid?

Multi-tasking and the Internet, both are suspected of making us more stupid.

From Nobel Prize winner Doris Lessing (lecture) to an HP study of online multi-tasking (article), to Nicholas Carr, author of "Is Google Making Us Stupid?" (NPR interview),  there are plenty of warnings against too much of a digital thing.

Ironically, I found all the warnings online.

To a certain extent, I agree that people do a lot of stupid things online. They waste a lot of time and manage to overlook what they are looking for. They accept online versions of a story they would never believe if it were told to their face. They copy and paste, oftentimes reproducing falsehoods or violating guidelines of fair use.

One of the observations Nicholas Carr makes is that shallow reading (skimming, jumping from one thing to the next) commonly occurs online. This has made it harder for him to engage in deeper reading. The temptation is to fall prey to "Internet-influenced attention deficit disorder."

Maybe it's the sheer volume of stuff to read or the tendency of web designers to recreate the equivalent sensory overload of Times Square that turns us into skimmers and multitaskers.  I don't know for sure, but I can relate.

Some of the deepest reading I've done has been online. For me, investigative searching provides balance for shallow skimming and relentless surfing. Taking a little time to track down the source of information, determine if inaccuracies have been overlooked, becoming familiar with who links to the information--these represent deeper thinking. In fact, I've probably done more deeper thinking online paying attention to clues on which credibility depends than I've done with works in print. Juried or edited works in print tend not to require as much investigation.

I don't need to belabor the point. A habit of investigating online information is bound to make one smarter rather than dumber.

Mindless Internet usage, on the other hand, can be a stupid pastime.

Try an Investigative Challenge: Use It or Lose It?

Is Reading Believing?

Unfortunately for many students, reading is believing.

There could have been a time when information that appeared in print may have been trustworthy--thanks to editors and (hopefully) experts who reviewed it before it went to press--but that's no longer a safe assumption.

Because of the Internet, the reader has become the editor.

Before an evaluation can be made, one needs to know something about the author, publisher, references, or content; probably a combination of these things.

So here's an author challenge that's appropriate for students. Look at this blog: The Future is Green. The content is about sustainability, energy, the green movement, etc. But can the views expressed be believed? Knowing some facts about the author (and possibly checking out those facts) provides some perspective on the author's words.
Try this or have your students try it:

1. Locate the author's name (not hard)
2. Find information about the author (requires browsing)
3. Determine if the author knows what he is talking about, based on his experience, education, associations, accomplishments, etc.
4. Fact-check information about the author. Having another source (other than the author) confirm what the author says about himself is always a good idea.

So, should the author be believed? Why or why not?

Yes, There's a Need

Take a good look at this chart. What do you see? A normal distribution with a negative skew? A test that's too hard? A need for improvement?

This chart depicts the performance of high school students attempting to apply investigative techniques to Web pages. Not too good, is it? The average score is 52% That means the vast majority of students in this sample (n = over 350) really struggle with the ability to find and evaluate online information to determine its credibility. The average for middle school students is 45%. Very few are at 80%--what we would consider mastery. (By the way, these are gifted students. The top 1-2%.)

If information fluency was a typical school subject, the majority of students would fail. Fortunately for them, this is not a typical school subject; it is neither widely addressed nor seldom practiced.

The result above is sampled from a new assessment we developed at 21cif. The results aren't surprising. These are the same results we've been getting for years. The computer-savvy generation consistently under-performs when asked to locate and evaluate information found on the Internet. They have real difficulty finding authors, publishers, dates of publication, salient facts and claims and references of information on web pages and blogs. Of course if they can't find those things, they have an even harder time evaluating their credibility.

After a few hours of targeted training in investigative techniques, students improve by 10% on average. With more practice, their gains would be even greater. By the way, adults tend to score 10-15% points higher than high school students. Not because they are better with computers; they take the time to read critically. They have better vocabulary skills. Despite all the new age bells and whistles, search mastery still comes down to careful reading and thinking.

The need for information fluency is why we're in this business. Research merely informs what training is needed. Having an opportunity to train students is the challenge of this business. If we continue to say there are more important things to study, we will continue to see the results above. A generation that cannot research proficiently online will lack the ability to use information resources wisely and profit from them.

Preachy, I admit, but what else can one say about the state of information fluency in schools today? It needs work.

If you would like to learn more about the assessment, I'll share more about it over the next few blogs.

Information Forensics Goes To School

In case you want to view the presentation Dennis O'Connor and I gave at NECC 2009, here it is:

Finding the Black Box

Homing in on information online is similar to locating a downed aircraft's black box in one respect: both send out signals. In the case of the black box, the signal gets stronger as you approach. In the case of information online, the contextual evidence (think clues) gets stronger.

Detecting those clues and interpreting them can be very difficult.

Homing in on information online depends on browsing for keyword evidence. Some people have a distinct advantage in this regard. Masterful browsers tend to have fairly developed language skills. This, coupled with a disposition to read carefully, puts them far ahead of otherwise skilled computer users. I know adults often feel inadequate when working with kids with lightning fast computer abilities. But adults, despite their technological shortcomings, tend to outperform prodigious children and teens when it comes to homing in on information.

I witnessed an example of this last evening when my wife pilot tested an assessment that Information Fluency is preparing to use with Center for Talent Development students at Northwestern University. Pat, not a computer user from birth, took about an hour to complete the 10 item pretest, most of which depends on careful reading and browsing--after all, investigative searching relies on being thorough and examining lots of clues. She only missed 2 items, which is typical of an adult who has mastered searching and evaluation. Middle school and high school students will take about half that time and miss the majority of the items.

The major difference boils down to reading and browsing carefully. Sure, there are other techniques in a skilled online investigator's toolkit, but careful reading and browsing can be used to solve most search challenges.

1. See every word as a clue It's really easy to overlook important keywords when skimming. The words and terms that matter most are typically nouns and numbers. Adjectives become important in detecting bias. Spotting these is easier by slowing down. If you have the attitude that words are clues--not just the specific keyword(s) you are looking for--you will probably have to adjust your speed.

2. Recognize and follow possible connections Here is where terms you weren't looking for become important and why it helps to have a good vocabulary. Synonyms and words used in the context of what you are looking for are all possible connections. It may help to think about what some of these other words are before reading. In terms of the Digital Information Fluency Model, this is known as finding better keywords as you search. You can't predict with 100% accuracy what keywords are necessary before you search. You have to pay attention and find them as you search.

An example may help Let's say you're looking for the publisher of this site: http://www.spacetoday.net. First steps are usually to read the header and footer for the name of an organization or copyright information. Not seeing those, what stands out is the About Us link. Let the browsing begin. The name of the publisher is in the text of this page. In this case, it's an individual: "Spacetoday.net was founded by Jeff Foust." This wasn't a particularly difficult challenge, but many students will miss it because the name is embedded in the text. Moreover, Mr. Foust's role is represented with the word "founded" rather than "published." You have to know what these words mean, how they are used and related. Is the founder always the publisher? Not always. To make sure we've got the publisher, another source of information is required.

Here's your Challenge: Can you find another source of information that confirms that Jeff Foust is actually the publisher? It might take another technique in addition to careful reading and browsing.

Five Keys to Investigative Searching

When it comes right down to it, there aren't a lot of techniques involved in investigative searching. As opposed to speculative searching--what we tend to do most of the time--investigative searching involves clear clues and places to look for information about credibility.

In speculative searching, we're not sure where to look (Google is the default) and what keywords we need to retrieve the information we are looking for.

The goals of investigation are clear: find and evaluate the author, find and evaluate the publisher, find and interpret the meaning of the date, find evidence and evaluate it for accuracy, objectivity and external verification. You don't need to collect all this information unless the stakes for using information are high (like your job depends on it or you'll be punished if you plagiarize or violate fair use). Most of the time, checking a few facts is enough.

It may be helpful to think about investigative search techniques as depending on careful reading and four other keys. Nothing takes the place of reading thoughtfully. Still, the tendency of students is to go as fast as possible, thereby overlooking important information and clues about credibility. I'd say that's the biggest problem and why students trust false information.

The rest of the problem, as I see it, is that students aren't taught some basic techniques. These are the four keys I'm referring to:


Queries [Enter Key]

Using a search engine to track down missing information and check its credibility is the most powerful tool in the investigator’s kit. Depending on the information you need, the database you search will vary. For most queries, Google is preferred. But if you need to look up the registered owner of a website—to find the publisher or someone to contact—that requires a different database: whois.net. If you need to find a list of pages that link to the page you are investigating, then Yahoo is preferred, as it returns more information than Google with the link: command.

There are keyword queries, when you need to check the facts about a person, a publisher or something an author wrote in a web page. If you can’t find information to back up an author’s claim, that lends no support to its credibility. Sometimes, you’ll find that other people have information that contradicts the author you are investigating. This is all part of triangulating: finding information from multiple sources that agrees. When that happens, there’s a better chance the information can be trusted.

There are also string queries. This is when you copy a portion of text and search for the exact phrase in a database. You may place quotes around a passage, but it’s not necessary. Quotes work best around a first and last name or a short phrase. By searching for a string, you will often find other instances of the article or person you are investigating. The other instance may contain additional information you need. You may also discover that the passage you are investigating was plagiarized: copied word for word without being cited.


Truncation [Backspace]

The URL is an important source of information. It can reveal the publisher’s name or whether the site is self-published by the author. Truncation is a good way to navigate toward the root of the site you are investigating. Not all the information you need may be found on a web page, but a directory page may contain important clues such as an author’s name, a publisher or the date an article was written. You truncate by removing parts of the URL with the backspace or delete key, starting from the right and stopping when you reach a folder marker (/).


Browsing [Left Click]

Browsing is a special kind of reading: paying attention to hyperlinks. Normally when you browse you are looking for words or images that stand out. On a web page, browsing is defined by the links you click. Think of successful browsing as a game of HOT and COLD. You are trying to use links to get you closer to information you need, though you’re not exactly sure where that information is hiding. Whenever you click a link you have to scan or read the new page to discover whether you are getting hotter or colder. If you seem to be getting colder, go back and try a different link or technique. If you are getting hotter you will discover keywords and clues that you can use in your investigation.



Page Information [Right Click]

If you are using a browser that provides page information (Firefox does this) you can right-click on a web page to bring up a menu that includes Page Information. Depending on how the web page was coded, information about the last time the page was updated may be provided. If the last updated information is ‘now’ then the coding on the web page doesn’t allow this information to be shown. Knowing the last update of a page can be helpful in determining the age of the material.



This is obviously only an introduction to the techniques, but with them you should be able to solve this challenge:

Who is the author of the Sellafield Zoo? http://www.brookview.karoo.net/Sellafield_Zoo/

Challenge Level: Intermediate (Don't forget careful reading!)

Tracking down the Loose Ends

Does this ever happen to you? You've located a relevant resource that you want to cite, but when you look for the name of the author--or maybe the date of publication--it's not there. Now what do you do?

Without information about the author, you have to rely on the reputation of the publisher. Without the publication date, you risk using information that is no longer current. The standard style guidelines (APA, MLA, etc.) allow for citing works without an author and/or a date, but this is risky. You really should try to find the missing information to be sure your source and content is credible.

Investigative searching is your best bet to track down elusive information. Start with the page you want to cite. Scour it for clues. An author's name or date is not always at the top or bottom of the page. Like most crime scene investigations, you don't want to look elsewhere until you have to. Careful reading is required.

If the page you are on turns up nothing, then you have to expand your search. Now the url and links on the page become important. Try truncating the url to navigate to pages closer to the root of the site. In the process you may find a directory that lists articles, including the one you want, including author and/or date information. Links on the page may do the same thing. Don't leave the site unless you have to. Again, look carefully for clues.

Following links (browsing) is a particularly challenging form of searching. If you get more than one link beyond a place where you detected any relevance to your search, back up and try another promising path.

If the site has a search engine, try entering the name of the article or any significant keywords from it. You may be surprised to find other references to the article on the site.

If you reach the point where the site is no help at all, conduct a broader search using a major search engine (Yahoo, Google, etc.). Query the name of the article or significant keywords from it. Articles often appear in more than one location on the Internet. You could find reviews or references to your loose ends.

If you've tried these options and still don't have a name or a date, decide if you should cite the work without them. Is the information good enough to stand on its own? What would be the consequences if this information were false?

To test your skills at tracking down an elusive author, try this challenge:

5 Essential SEO Techniques (Article is halfway down the page)

Whom would you cite? Look for the answer in my next post!

If you'd like an opportunity to sharpen your investigative skills, a new section of Web Site Investigator (WSI) starts August 11. > More information

Embedded Evidence, External Evidence

Over the weekend I created two new tutorial resources for the Website Investigator series (WSI): Accuracy and Evidence. In addition to knowing who is the author and/or publisher and when it was written or published, finding embedded evidence and external evidence can be very important in verifying the credibility of the source and the content.

In the simplest terms, credibility depends on source and content. Information about the author and publisher helps to define the source--where did these ideas originate? Is the author recognized as an expert? Does this publisher submit works to careful review before posting them? Embedded and external evidence helps to define the content--how are words used (signs of objectivity or bias)? When was this written? Who links to it? What do experts say about the content? A questionable source may produce brilliant content and a trusted author may produce flawed content--so it's important to check both before accepting information at face value.

As educators know, the majority of students today tend to accept information at face value. Somehow, merely finding information feels sufficient. Investigating it is unimportant.

To encourage investigation, students need to be shown and practice a few basic techniques. These are not hard to learn and don't take much time. Compared to searching (which I now call speculative searching--when you don't know exactly what words to search with and where to look), investigative searching is much more precise: the keywords are clues embedded in the information and the places (databases) to search are well-defined.

Think of embedded evidence as clues in the text, the url and metadata. These clues can be used to investigate the accuracy of information and often lead to external sources that have already done an evaluation.

The new Accuracy tutorial focuses on three areas: Finding powerful clues embedded in a Web page, checking the evidence by doing a secondary search and triangulating, checking what three different sources have to say about the information.

The companion Evidence tutorial emphasizes using queries to find external evidence, checking whether pages that link to the information support it or contradict it and triangulating information sources (examples that are different from the Accuracy module).

These tutorials are geared for middle schoolers through adults. There's increasing demand for similar activities aimed at elementary grade students, and that's my next project.