Thursday, September 30, 2010

Performance Testing

I finished evaluating the results from an assessment/tutorial package we hosted for Northwestern University's Center for Talent Development.

Middle School and High School students enrolled in summer programs were expected to complete a three-hour self-paced "course" that included a pretest, interactive tutorials and a posttest. We offered something similar the summer before, but that program took eight hours to complete. I was concerned that by cutting over 60% of the practice time that students wouldn't improve as much as they did the previous summer.

I'm happy to say my concerns did not materialize. Instead, students improved even more. Targeted instruction was sufficient for the majority of students to improve their ability to find and evaluate information online.

I'm most pleased with the assessments. It's difficult to write test items that isolate search and evaluation skills. By making these performance items, students have to demonstrate competency. For example, here's one of the retired questions from Information Investigator:
Enter the query in the search box that will return a list of pages that link to this investment page: http://greenrgy.org/investment/index.html
Very few students can do this, mainly because they have not been shown how to do it. Approximately 3% of students in middle school and 8% of students in high school succeeded with this task.  After being shown a few examples and given some additional practice scenarios, these same students improved their success rates to 34% and 36%, respectively. It's still far short of perfect, but 1 out of 3 is a lot better than 1 out of 12.

The example above requires the use of the link: command. Knowing when, where and why to use this is not all that simple. Actual searching doesn't prescribe what search techniques are needed. You may have to try several. The more tools you have the better, but you still don't have a clear-cut guide when and how to apply them. This is why more than a 3-hour self-guided experience is needed.

I would really like to see more integration of information fluency in middle school and high school curricula. Sorry to say, it probably won't happen without making information fluency a testing priority. But when the time comes, there are tests that are ready, including ones like Information Investigator.

Besides Northwestern, there are few other schools that are trying the Information Investigator approach. If you'd like more information about it, just write.

Can you answer the test item above?  What's the correct search query? And where would you search?

No comments: