Believing that the ability to locate, access, and evaluate information is essential to college success, academic libraries have widely adopted information literacy as the framework for the instructional programs. At the same time, universities and colleges are being held to increasing levels of responsibility for showing that students are learning and that resources are used effectively. Assessment has become a key initiative both for programmatic improvement and to fulfill obligations for accountability.
Development of the SAILS Tests
The Project for Standardized Assessment of Information Literacy Skills (SAILS) is a Kent State University initiative to develop an instrument for standardized assessment of information literacy skills. From the outset, we envisioned a standardized tool that is valid and reliable, contains items not specific to a particular institution or library, is easily administered, and provides for both external and internal comparisons. With such a tool, we have the ability to measure information literacy skills, gather national data, provide norms, and compare information literacy measures with other indicators of student achievement.
Guiding Questions
- How do we know if students are information literate?
- What are students’ entry skills upon admission to the college or university?
- Is there a significant change in skill and knowledge levels from the freshman year to graduation?
- How does the library contribute to information literacy?
- Does information literacy make a difference to student’s academic success and retention?
Tests
We applied systematic instructional design to develop test questions, using item response theory (IRT) as the measurement model, and thus created a bank of test questions of varying difficulty levels to measure information literacy. The test items are based on the ACRL Information Literacy Competency Standards for Higher Education. Use of these standards maximizes the ability of the instrument to be used by a wide variety of academic institutions for internal and external benchmarking.
Project SAILS developed two tests, one that gives cohort-level scores and one that gives individual scores. A third testing option is Build Your Own Test for which the test manager chooses specific questions from the test question bank.
Reaction
The overwhelming and enthusiastic response to Project SAILS signaled that libraries were and still are looking for tools that can help determine the effectiveness of their information literacy programs. It is an example of libraries working collaboratively to create useful, standardized tools to benefit the larger library community.
Development Milestones
2001
Carolyn Radcliff, Julie Gedeon, and Lisa O’Connor present, “Assessing Information Literacy Skills: Developing a Standardized Instrument for Institutional and Longitudinal Measurement” at the 10th ACRL conference in Denver, Colorado.
2002
Kent State University is awarded a $252,418 grant from the Institute of Museum and Library Services to develop an instrument for programmatic-level assessment of information literacy skills. Project SAILS aims to measure information literacy skills, gather national data, provide norms, and compare information literacy measures with other indicators of student achievement.
2003
The Association of Research Libraries partners with Kent State University and supports Project SAILS as one of the ARL New Measures Initiatives.
Phase I of SAILS pilot program is conducted at eight participating institutions with 4,600 students taking the test. The SAILS team meets with Phase I schools in Toronto to present results from the data analysis and receive feedback on the reports. It also gives participants the opportunity for in-person discussion with one another and the research team.
Project SAILS skill sets, regroupings of the ACRL Information Literacy Standards, are introduced to the Phase I participants and are well-received. Representatives from the Phase II participating libraries meet with the SAILS Team, who provides timelines and training for test administration.
2004
Phase II is completed with participation by more than 10,000 students at 34 institutions. Representatives from the Phase II institutions meet with the SAILS team during ALA to discuss results contained in the institution reports. Schools share how they intended to use the reports on their campuses.
2005
Development Phase III is completed with 27,400 students at 69 institutions. The SAILS team meets with representatives from the Phase III institutions during ALA. A panel discussion follows and participants hear how other schools are using their results.
A customer interface to SAILS is created and the programs used to run the instrument, analysis, and report functions are retooled. No test administration takes place during the 2005-2006 academic year. In November, the redesigned SAILS website is unveiled.
2006
The SAILS team conducts a training workshop for schools wishing to learn more about the new customer interface. The same workshop discusses timelines and upcoming developments.
SAILS is put into production.
2007 – 2011
Librarians and other staff at Kent State University work with dozens of colleges and universities as they implement SAILS.
2012
Kent State University issues an exclusive license to Carrick Enterprises, Inc. to carry on the operation of Project SAILS.
2012 – 2016
An international version of the SAILS cohort test is created. The SAILS tests are improved to comply with best practices and standards for accessibility. The Build Your Own Test (BYOT) version of SAILS is implemented.
2016
ACRL replaces the Standards upon which the SAILS test questions are based with the Framework for Information Literacy. Librarians and other educators convey their need to continue using the SAILS tests and Carrick Enterprises responds by pledging to maintain and upgrade Project SAILS.
2018
The Project SAILS web site, tools, and reports undergo major renovation and modernization.