Searching the Web--Part 3 Web Search Tools Authored by Pamela Eyerdam and Ken Burhanna The text only versions provide a linear presentation of the Hands-On flash modules.  The Hands-On modules often incorporate images and interactive presentations to help demonstrate and support the text.  These images and interactive features are lost in the text only format.  Users can supplement their reading of the text versions by going onto the World Wide Web and visiting the web sites and doing the searches that the text indicates.  This is most easily done by printing the text version and using it as a guide. After completing this module you will be familiar with and know when to use: 1. Search Engines and Meta Search Engines 2. Subject Directories 3. The Invisible or Deep Web Introduction The World Wide Web is comprised of millions of pages of content.  Search engines and subject directories help people find and access these pages. No two of these search tools are alike and sometimes you may need to use multiple tools to do a good search. And as always, everything, including the search tools themselves, requires constant and vigilant evaluation. 1. Search Engines & Meta Search Engines You are probably familiar with at least one search engine.  Google is a popular search engine. A search engine is a searchable database of web pages compiled by computer programs (called robots) with very little human oversight. Search engines look for exact word matches for keywords entered by searchers. Search engines only index web pages that are publicly accessible and not submerged within the tables of other databases. Alta Vista is another popular search engine. Any one search engine only covers a percentage of the WWW.  No search engine can cover all of the web. Because search engines use very little human input, their indexing is rudimentary and searches can return a great many irrelevant results. Dogpile is an example of a meta search engine. A meta search engine searches  a number of search engines in response to a single inquiry.  For example Dogpile searches the search engines About, Ask Jeeves, FAST, FindWhat, Google, LookSmart, Overture and many others and then returns the results from these all at once. In theory meta search engines sound awesome, but in practice they often can overlook important web pages from some search engines and overall results can be disappointing. For more examples of search engines and meta search engines visit the University Library's Virtual Ref page. When to Use a Search Engine In general search engines are most helpful when you have a good idea of what you're looking for but are not sure of where to start.  If, however, you know where the information should be (i.e. on a government web site), you should locate the appropriate subject directory and start there. Additionally, search engines are good when you: - have a narrow topic to research and you've exhausted more authoritative resources at the library. - want to find a specific web site or very specific or obscure piece of information. - want to search for specific types of documents, like photographs, foreign languages, or PDFs.                                                                                     2. Subject Directories Subject directories are another popular tool for finding information on the web.  Yahoo! is one you have probably heard of and used before. A subject directory is a collection of links to Internet resources compiled by site creators or reviewers and organized into subject categories.  Subject directories are usually searchable. Subject directories differ from search engines in some key ways: 1. Sites in subject directories are reviewed and selected by humans.  This can mean (but not always) that the sites have been carefully evaluated and the information is up to date. 2. Often web sites are organized in hierarchical categories similar to the subject headings used in library catalogs.  Directory builders also often include annotations to describe the sites in their directories. 3. Subject directories do not include the text of documents from web pages.  So unlike search engines, subject directories do not help you match keywords as they appear in the original text. Users need to become familiar with the subject vocabulary of the directories they use. It can be helpful to differentiate between two types of subject directories: 1. Commercial portals are business ventures designed to generate advertising income and heavy traffic from the general public.  These sites are often built around a central subject directory.  Yahoo! and LookSmart are examples of commercial portals with directories.  These sites usually include a number of other services (sometimes search engines).  More on portals Portals Portals are sites that offer both search engines and subject directories and often a collection of other services.  Most portals are designed for specific purposes such as the University Library's Subject Portals.  These portals aggregate a variety of information services on a single page that is organized by functional headings.  These portals are designed in conjunction with CSU faculty to support and supplement their curriculum. 2. The 2nd type of subject directory is the academic or professional directory.  These subject directories are usually created by librarians and subject experts and are often affiliated with universities and research institutions. The creators of these subject directories apply high standards and regularly update their sites.  No advertising or income is generated from these sites. INFOMINE , from the University of California, is a good example of a academic directory. For more examples of subject directories visit the University Library's Virtual Ref page. When to Use a Subject Directory A subject directory is generally good when you know where or what category the information you need is located.  For example, if you need information on universities that offer distance education, you might check Yahoo's heading for Education. Subject directories are also good when you: - have a broad, general topic to research and you have exhausted more    authoritative resources at the library. - are looking for expert recommended web sites related to your topic. - want to avoid the many irrelevant web pages that search engines can    return.                                                                                         3. The Invisible or Deep Web We mentioned earlier that search engines can't index the information contained in databases, even if the database if publicly accessible.  The information in these deep or invisible resources comprise the majority of the invisible web.  Also, dynamic content, like news items and job postings are part of the invisible web, at least until the search engines find them and update. What Information is the Invisible Web Good at Providing? Information stored in databases is likely to be part of the Invisible Web: listings in phone books, people finders, patents, laws, digital exhibits, articles, data, and documents contained in professional indexes accessible through the University Library. For example, the content inside of ProQuest Digital Dissertations is invisible to search engines. The invisible web also includes information that is new and dynamically changing:  news stories, stock quotes, job listings, pricing and availability of airline tickets and hotel rooms. Google's News page provides access to fast breaking news that goes unseen to search engines. How Do You Find Information on the Invisible Web? While the content of the invisible web is hard to find, most of these databases have a front door or a gate that you can search for.  Once you find the front door (just a web page really), then you need to search the database. Three methods to find the invisible web: 1. General subject directories:  You might have to keep your eyes open but you can find the web pages for databases in subject directories.  Infomine for example is a good resource for this. 2. Use a search engine by searching a subject term and the word "database". 3. Use a specialized subject directory of databases.  One of the best for academic content is Direct Search . Exercise - Which Type of Web Tool Should You Use Question 1 You're looking for emergency medical transportation guidelines for nurses in Canada.  After having no luck searching the Scholar Catalog and various periodical indexes, you think you might try the web.  Which type of tool would you start with? A. Search Engines B. Subject Directory C. Use a Subject Directory to find data on the Invisible Web Your topic is very specific and narrow.  A search engine like Google would be the best place to start. Question 2 You're trying to price airline tickets to Honolulu for next Christmas.  Where would you start? A. Search Engine B. Subject Directory C. Use a search engine to find a web site with a database of airline rates and schedules (invisible web). By searching "airline tickets" on a search engine like AltaVista, you would find a good database with prices and schedules. Question 3 You're looking for product information on desktop computers.  You'd like to compare several manufacturers.  Where would you start? A. Search Engines B. Subject Directory C. Use a Subject Directory to find a database of desktop computers (invisible web). Your topic is a broad one.  A subject directory like Yahoo! will have a category for computers with links to several manufacturers. This concludes Web Search Tools