Learn About: Criteria used to evaluate print and Internet information resources, differences between print and Internet resources, characteristics of scholarly vs. popular periodicals, and the scholarly publication cycle.
Evaluating information sources is a important part of the research process. Not all information is reliable or true, nor will all information be suitable for your paper or project. Print and Internet sources vary widely in their authority, accuracy, objectivity, currency, and coverage. Users must be able to critically evaluate the appropriateness of all types of information sources prior to relying on the information.
The Internet, especially the World Wide Web, has surpassed most libraries in the quantity of information it makes available. However, the Web has not surpassed libraries in the overall quality of information it makes available. Traditionally, a main component of library collections has been print (paper) materials. Today, however, many online resources are being added to supplement collections, replace printed (paper) items, or improve access. Although online sources are accessible via the Internet, many originated in paper form and follow the same publication criteria. Therefore the quality of print and online information sources are similar and will be considered the same in this discussion. A look at a few characteristics of print and Internet sources will identify major quality distinctions between print and Internet information sources.
Print Sources vs. World Wide Web
- Quality standards of printed materials are controlled through a system of checks and balances imposed by peer review, editors, publishers, and librarians, all of whom manage and control access to printed information. This assures that published materials have been through some form of critical review and evaluation, preventing informal, poorly designed, difficult-to-use and otherwise problematic materials from getting into the hands of users.
- In academic and other research libraries, most books and periodicals are a product of the scholarly communication system. This system ensures that authors present information in an orderly and logical manner appropriate to the topic.
- Printed information in books and periodicals follows established linear formats for logical and effective organization.
- Materials in printed form are stable. Once in print, information remains fixed for all time. New editions and revisions often are published, but these are separate and distinct physical entities that can be placed side by side with the originals.
World Wide Web
- On the web, anyone can, with no supervision or review at all, put up a web page.
- On the Web, there is no systematic monitoring of much of what appears, except, of course, for articles published in the online forms of otherwise reputable scholarly journals and books. Biases, hidden agendas, distorted perspectives, commercial promotions, inaccuracies, and so on are not monitored.
- There is no standard format for web sites and documents. Web pages exhibit fewer clues regarding their origins and authoritativeness than print sources. Important information, such as dates, author(s), and references are not always easy to locate. While a reader can easily note this information in a book or periodical article, the web user must often search through several pages, if the information is provided at all.
- Internet sources are also not stable. Web documents can be changed easily. And once changed, the original is gone forever unless a specific effort is made to preserve it. In fact, many Web documents are intentionally designed to change as necessary, and with automatic changes as with manual changes, the original disappears.
- Web resources use hypertext links and need not be organized in any linear fashion. One can easily be led astray and distracted from the topic at hand. But, of course, one can also be led to additional information of value.
- The changing nature of the web and web documents create major problems with the stability of information and with links between different units of information. Dead or broken and links on the Web are common and others just disappear or are not updated.
See Online Subscription Databases vs. Web Sites to learn how to quickly differentiate between subscription (library-selected) and Internet sources.
For print sources, quality control is sought through critical evaluation during the publication process. However, on the web, anyone with access to the Internet can publish. Web pages are easy to create with little or no training. And there is no overriding organization or governing body ensuring the validity of web page content. There is a good deal of high-quality information on the web, but there is also much that is of questionable quality. Do not assume that information on the web is more current or accurate. Each web page will have to be examined critically.
It is the user's responsibility to evaluate information sources, in print and on the web, that they find during the research process before using it in a paper or presentation.
Five Criteria for Evaluating Resources: AAOCC
For this brief introduction to evaluating sources in LS101, we will use a list of five critical criteria. You might want to remember AAOCC (Authority, Accuracy, Objectivity, Currency, and Coverage), if for no other reason than you might be asked to list these criteria and describe them briefly. The same basic questions should be asked of all information sources: books, journal articles, web pages, blogs, videos, sound recordings and e-books.
- Who is the author or creator (who is responsible for the intellectual content) and what are his or her credentials? Is there any indication of the author's education, other publications, professional affiliations or experience?
- Is there a note or paragraph in the back of the book or on the jacket (cover, jewel case, or supplementary brochure) describing the author's credentials?
- Is the author's e-mail address, postal address or phone number provided?
- Has the author been cited in other bibliographies?
- Sometimes information about an author is available in sources other than the document at hand. Instructors assigning research topics might focus on a particular author. Many authors can be looked up in such resources as:
- Who's Who in America
- Current Biography
- The International Who's Who
- Who's Who in Science and Engineering
- American Men & Women of Science
- Who's Who in the World
- Dictionary of Scientific Biography
- Look for additional information in directories of Professional Associations or Biography Index (FirstSearch)
Noted resources are accessible via the Databases by Title page.
For Web Sites:
- Be sure to distinguish between the author of the information and, if separate, the Webmaster who put it up.
- In the case of Web material provided by committees, organizations, businesses, or government agencies (rather than individuals), similar questions concerning the authority of these bodies need to be asked. Be sure to consider whether information provided by corporate bodies is likely to be objective, factual and carefully researched or whether it is biased toward the particular objectives of those bodies or the causes, movements or agendas they support.
- Analysis of the URL sometimes provides an indication of identity of the web site sponsor.
- Look for an "about us" or "FAQ" (frequently asked questions) page.
- Is the information provided specific?
- For research on any topic dealing with things and events in the real world, accuracy is, obviously, of highest importance. Data and information must be based on observations, measurements, analyses, interpretations and conclusions. In the arts and humanities, where imagination is the primary creative force, accuracy is still important in recording names, dates and places from which creative works, ideas, and opinions originated.
- In all cases, all information should be verifiable. Are conclusions based on research or actual figures that can be checked in other sources?
- Are methods of scientific research explained in such a way that it could be reproduced?
- Are sources of information listed in foot/end notes, bibliographies, or lists of references? How reliable are the cited sources?
- Are critical reviews available (for books, films, literature, music, art)? Check resources such as:
- Humanities Full-Text,Social Sciences Full-Text, MLA Bibliography, Reader's Guide to Periodical Literature
- Academic Search Premier
- Gale Literature Resource Center
Noted resources are accessible via the Databases by Title page. Check with a librarian for other useful databases.
- Does the article appear in a scholarly journal that is peer-reviewed? Review the tips for identifying Scholarly vs. Popular Periodicals.
For Web Sites:
- High-quality writing, including good format, grammar, spelling and punctuation, can enhance the appearance of accuracy and bolster a reader's confidence in the accuracy and reliability of a Web document. It is easy however, to produce a website that appears "professional;" that's a good start, but not sufficient evidence to conclude that the information provided is accurate. Use the other criteria as well.
- Authors often have their own agendas, for example to sell products, influence legislation or capture converts. There probably is no absolute objectivity upon which everyone could agree. When using any information resource, you must decide whether the information is sufficiently objective for your purpose or whether it is biased. Of course a highly biased presentation can be included in scholarly research as long as that bias is described and weighed against alternative views or interpretations.
- Is there any advertising or solicitation for donations associated with the source? This financial support may skew the subject coverage by the publication.
- Does the author provide more than one point of view?
- Does the writing use inflammatory or biased language?
- Currency is especially important in the sciences where new developments occur frequently.
- In the arts and humanities, currency needs to be judged as appropriate. In some cases, a study written years ago may be essential to understanding.
- Consider whether or not the timeliness of the information will affect its usefulness.
- In all cases, there should be some indication of the date of the material. If research results are given, consider not only the date of the publication but also when the research was actually conducted.
For Web Sites:
- Obviously it is important for information found on the web to be up-to-date. However, its appearance on the web is not a guarantee.
- There should be some indication of the date of the material, as in the "last updated" statement at the end of many Web documents. Be aware that the "Last updated" date of the web page may differ from the date of the content. This may mean checking three dates, the date the page was last updated or posted to the web, the date of publication, and the date of the research or statistics used.
- Decide whether the information source adequately covers the topic. Documents may cover only part of the topic, and you may need more sources to have a more thorough understanding.
- Consider how coverage from one source compares with coverage by other sources.
- Look for a statement describing the purpose or coverage of the source and consider if the information is in-depth enough for your needs.
- Does the information source leave questions unanswered (ask the "five W's and H" to check: who, what, when, where, why and how)?
Evaluating Information Found on the Internet (optional) from Johns Hopkins University
The phrase “critical thinking skills” is often heard in business circles or seen listed in job requirements and MBA program descriptions. However, it’s not always clear what it actually means. True critical thinking involves an intervention in one’s own thought process in order to efficiently solve a problem. Unfortunately the administrative demands on today’s educators don’t leave much time to teach this process; as a result, there are an enormous amount of people in our workforce who lack this understanding.
What Exactly Is Critical Thinking?
Whenever any of us approach a problem, we bring biases to the table, often unintentionally. Prior experiences, cultural influences, assumptions about knowledge on the subject, or public opinion all play into our thought process, whether we’re aware of it or not. The challenge in critical thinking lies in first becoming aware of those biases, and then in stepping outside of them to clearly reason your way through a problem. Successful critical thinkers make better business decisions because the process allows them to gather more information, collaborate with others and evaluate a business decision with objectivity.
For example, a new solution to an old problem may be expressed during a workplace meeting. People who are naturally resistant to change may not exercise critical thinking skills, and instead respond that “We’ve always done it that way, why change it now?” Instead of shooting down a new idea without giving it any thought, the application of critical thinking could result in a more effective way of doing business. Perhaps the marketplace has changed, or new data has been made available that suggests a different direction. Successful companies are ones that take a process apart, examine its components carefully, and gather relevant information. This collaborative process encourages creative thinking and oftentimes results in very effective problem-solving.
There are several schools of thought that detail core steps in the critical thinking process. Each of them leads to intellectual analysis of the information at hand, identifies areas that require more research, and finally indicates a course of action that best solves the problem. Successful critical thinkers generally share the following characteristics:
- Open-minded. Acceptance of new ideas, even with their inherent biases, is crucial to this process. Not everyone approaches a problem with the same experience or knowledge, but that doesn’t mean their ideas are not valuable. The ability to accept that our idea may have been wrong or incompletely thought out is an extension of this open-mindedness.
- Think logically. Applying critical thinking requires that criteria must be defined for a problem’s components. Using precisely defined criteria to measure information allows for a more objective evaluation of data, removing biases and setting a standard to which all stakeholders must adhere. Replacing emotional barriers with logic can help you spot flaws in your processes that you may not have otherwise.
- Reasonable. The best decision-making involves arguments from multiple angles, including negative ones. Using carefully researched data to entertain all possible outcomes requires an unbiased approach to the information. Informed decisions are based on sound reasoning of all aspects of the problem.
- Collaborative. Loyalty to “our” idea is a human trait, but stepping outside of our own frame of reference requires conscious thought. By working with a group of individuals, each of whom has their own biases and knowledge levels, new ideas can be exposed. Good critical thinkers welcome the opportunity to make the right decision, versus inflexibly insisting on a particular solution.
How Is Critical Thinking Relevant to Business?
Effective management skills include the ability to think critically, and making the right decision under pressure is what defines successful businesspeople. Managers and staff must weigh all possible solutions; this can be time-consuming and require involving many people in the decision, but ultimately it leads to better choices. Some examples of critical thinking applied in the workplace follow.
Innovation creates successful business products, and being closed off to new ideas automatically stifles innovation. Opening up to a variety of solutions can help you create new options for your customers.
Let’s say a publisher of textbooks is informed by its sales team that educators want better options for creating exams. A manager resistant to new ideas, technology or expense may insist the company continue to provide the printed exams it always has. A critical-thinking manager instead may take the time to explore providing new, digital exam-building tools. In the first scenario, the company risks losing market share to competitors who provide its customers with better tools; in the latter, responding to direct customer requests with new offerings keeps the company competitive in a dynamic market.
Critical thinking makes it far more likely that you can create a range of products to suit your customer’s needs. Using the same example, a critical-thinking manager at the textbook publisher not only takes the time to investigate options, but is comfortable taking the problem to colleagues across other departments. The collaborative nature of this process generates ideas from individuals who might not have otherwise been involved in the decision-making process. Ultimately, the company may discover that there are cost-effective ways to offer customers choices among several digital and print exam-building tools. The critical thinking process can easily generate multiple solutions borne out of one question.
In another example, applying the critical thinking process to product development may allow for a more polished product. A company that markets to legal professionals, recognizing that their customers are required to maintain continuing education credits, decides to create an online continuing education delivery tool.
The team member who first suggested the idea is heavily invested in the product, having dreamed it up and spent long hours developing it. Launching such a product without exposing it to a critical thinking process would be unwise; namely because the original developer may be too emotionally involved to spot potential flaws in their proposal.
A lengthier process that allows colleagues to test the product can reveal glitches or inconsistencies that deserve to be addressed ahead of time. The tool may get to market later and require more funding to develop, but will ultimately stand as a better product, which in turn could solidify the company’s relationship with their customer base.
Marketing professionals especially benefit from critical thinking. A product’s packaging, message and advertising is most successful when targeted at a specific demographic. Because marketing also relies on an emotional reaction from customers, it is absolutely crucial that multiple voices and viewpoints are brought to the table. Applied critical thinking skills also drive research and preparation. Take focus groups; when properly incorporated into product development, these groups can provide invaluable feedback – feedback that could alter the course of development altogether. And while the collaborative process takes longer and costs more – focus groups, for instance, can eat up a lot of time – the findings will bring about a highly targeted, highly effective marketing campaign.