Information Literacy Modules: Evaluation of Search Results

Once an initial search is conducted, several points need to be considered: amount of results, relevancy of results, and effectiveness of search strategy.

Amount of Result:

How many records are retrieved is important. Too many or too few can provide useful insights to the searching process. One question that is difficult to define is, how much is too much or too few?

Narrowing

Too many results indicate two situations. Either there is too much information on the subject because it is too broad and needs to be narrowed to a manageable focus, or the search was not specific enough to provide the necessary information.

• If the subject is too broad, review the records to find topics within the greater subject to narrow on. Then apply those new terms using various search limiters.
• If there are many records retrieved that are not on topic, the search needs to be more focused. Look at the records to find the controlled vocabulary the system is using for the topic, and incorporate those terms into the search.

Consider the following scenario:

You decide to write a paper on the use of Social Media in society.

Let’s begin with keyword search on a potential topic. We will look at all formats available here at SJR State.

As can be seen here, our first search retrieved 9062 results. This is too broad, so let’s look at some of the results and see what we can learn.

In working though some of the titles, you discover one that begins to provide some good information that could be used to help narrow your initial focus. One area to examine is Summary and the next is Contents.  These areas may contain terms that could help to identify areas of interest in the greater subject of Social Media.

The last area to focus on is Subjects. This is where you find the controlled vocabulary used to describe the item. As previously stated, the controlled vocabulary in the Catalog is based on Library of Congress Subject Headings.

So after looking through various areas of the record, you decide to try to incorporate some of the subjects listed. Social Media is one subject. Another area that may useful is the concept of Social Aspects. It is a sub-heading with two different main heading that could provide good information. In thinking about Boolean operators, proximity, and nesting, you decide to construct this search string:

“Social Media” AND “Social Aspects” AND (Internet OR Communication)

This can be searched on the advanced search screen, under Option 2. The difference is 88 results. A much more manageable amount.

 

Broadening

Too few results could indicate that the wrong search terms were selected, or that the search was too narrow.

• If there are only a few results, but at least one of them is on topic, examine the record. Look for the controlled vocabulary and use that to continue searching. Many times this will provide more relevant results. It may also provide more related terms that can help to expand on the topic.
• If the results are too specific, reexamine how the search was structured and revise it. Try different configurations, or simplify the search by looking into the main subject and seeing what narrower subtopics are available.

Consider the Following scenario:

You are interested in researching the topic of string theory. After some thought you form this search string:

Multiverse resonance “string theory”

It has no results in the catalog. What could be wrong? It could be too narrow. Let’s try to adjust the search.

In broadening your search you decide to go with the main topic of string theory, still in quotation marks for proximity, but change to a Subject search to be more specific. This search produced 15 results. It is a start, so you decide examine a few titles and see what the controlled vocabulary is.

When looking at a title that is obviously on String Theory, you discover that the Library of Congress Subject Heading for that subject is actually String Models.

Now you can take that information and begin to work with your search terms again to structure a useful search string.

 

Conclusion of Amount of Results

How much is too much or too few? This remains very difficult to define. It really depends on the topic and the scope of the paper being written. For a five-page paper, five to ten sources from a Library catalog search may be adequate. For a twenty-page paper, that may not be enough. The same holds true for large amounts of results. For more than 500 results, the topic may be too broad, and so needs to be revised. The results will need to be evaluated to determine if the search needs to be broadened or narrowed.

Relevancy of Results: Regardless of the amount of results, how relevant they are remains important.  Each record will need to be evaluated to determine if it will be useful or not. Begin with the titles. Titles that look completely off topic usually are. If it is not clear by the title what the item is about, examine the record to see listed contents, abstracts, and the controlled vocabulary (subjects) assigned by the system. Other areas to consider are the source of the item (e.g. publisher) and the date of publication. When evaluating the results, always look for other terminology that could be used to fine tune the search. We will look more at evaluating sources in the module Evaluating Sources

Effectiveness of Search Strategy: This should be examined at every stage of searching. Did the search retrieve a large number of results, but the subjects are not what was expected? How the search was conducted will need to be modified. Terminology may need to be changed and/or reconfigured using the various Search Strategy methods. Sometimes complex topics may require separate search strategies to find information on all aspects. It is not always the case that the sources found will include everything your topic requires. When this happens, it can become necessary to search for the various components of your topic separately. Researching is a process, and part of that process is restructuring and revising search strategies. It can take time to find the best resources for a paper.