Of our four search categories, validity was the most difficult search concept, largely because of inconsistent terminology used among authors, indexers, and investigators. Most authors did not seem familiar with the controlled vocabulary in the field of validity. In-depth comparison of five studies indexed in all four databases supported this result. For instance, Kristiansson et al.  did not use a validity term in either title or abstract and did not provide Author Keywords pertaining to validity. Yet indexers from MEDLINE and MANTIS recognized this study to address validity and assigned a validity term, e.g., Predictive Value of Tests in MEDLINE and Reproducibility of Results in MANTIS. On the other hand, one of these studies used validity terms three times in the abstract, but indexers from MEDLINE and CINAHL did not recognize the validity component of the study . Thus, authors seem to under-represent the validity aspects of their studies, while database indexing shows inconsistent awareness of validity component.
Authors frequently used "uncontrolled" terminology in titles and abstracts instead of the corresponding terminology that is part of controlled vocabulary in the databases of interest to their studies. For example, authors chose the term "accuracy" to represent validity but "accuracy" is not an indexing term in MEDLINE and MANTIS. One suggestion for MEDLINE and MANTIS database vendors would be to augment their controlled vocabulary to specifically include terms for validity, e.g., validity and accuracy, since these are terms preferred by authors and investigators.
Currently, the controlled vocabulary for validity in MEDLINE and MANTIS is limited and indexers use Reproducibility of Results as both a reliability and a validity indexing term. However, if the term is properly understood by authors and researchers to represent reliability rather than validity, they may not consider using Reproducibility of Results to search for validity studies. Unfortunately, automatic term mapping is not available in MEDLINE for the search term validity. Database vendors might consider applying Reproducibility of Results to reliability only and augmenting their thesaurus with commonly used validity terms.
Presently, Predictive Value of Tests, Reference Standards, Sensitivity and Specificity are some of the controlled vocabulary available in MEDLINE and MANTIS for indexing validity studies. In comparison to these MeSH descriptors, CINAHL subject headings cover a wide range of validity terms, including Validity, Concurrent Validity, Consensual Validity,Construct Validity, Content Validity,Criteria-related Validity,Qualitative Validity,Discriminant Validity, External Validity,Face Validity, Internal Validity,Predictive Validity, and Validation Studies. Further investigation might explain why only the subject headings Validation Studies and Criteria-related Validity were used to index the 13 studies in CINAHL on validity that were examined in this project.
This study shows that CINAHL indexers did not use all the validity terms available in their thesaurus. For example, Haas et al. used Construct Validity in the title, but the CINAHL indexer did not assign Construct Validity as a subject heading for that study . In the study by Sandmark et al., validity appeared in the title, yet CINAHL did not assign a subject heading for validity . In spite of the relatively large array of validity terms in the CINAHL thesaurus, validity indexing is under-utilized in this database.
In the entire database of MANTIS, the controlled vocabulary Sensitivity and Specificity was indexed only 24 times. None of these citations were in our subject area even though the term was used in six titles or abstracts. Thus, similar to CINAHL, indexers in MANTIS did not use their available controlled vocabulary as required for optimal search result. Searching for validity studies becomes complicated further as authors and investigators do not seem to have a common understanding of the types of validity. Consequently, indexers are at a loss which validity terms to use.
Fortunately, using validity assessment terms was productive because authors and indexers used these terms more often than just validity terms. Pain Measurement and Radiography showed relatively consistent use by authors and indexers.
The search category for procedural terms was relatively uncomplicated. Palpation and Manipulation were useful palpation procedural search terms. In addition, the controlled vocabulary term Physical Examination should be included in search strategies on spinal diagnostic procedures in order to maximize search results. Procedural terms in manual medicine seemed quite well represented in the databases that we studied.
Like validity, the search concept of neuromuscular dysfunction presented with challenges. For example, half of the 22 neuromusculoskeletal dysfunction terms provided by the investigators for the search strategies were not contained in the controlled vocabulary. Of the 11 controlled terms, only Back Pain and Low Back Pain were used with consistent frequency by authors and indexers, suggesting a need for standard terminology and definitions in the field of neuromusculoskeletal dysfunction with which to search the literature. Four of the five studies that were indexed in all four databases addressed low back pain or neck pain, and one study examined thoracic end-play. This admittedly small sample of studies would suggest that back and neck pain represent major areas of investigation in neuromuscular dysfunction. End-play was not recognized as a neuromuscular dysfunction by the investigator of this project. Indexing in all four databases did not include any neuromuscular dysfunction term for this study (see Table 6) which would suggest that end-play is not a recognized dysfunction term in manual medicine. Based on this few studies, some neuromuscular dysfunction terms, like low back pain and neck pain, seem clearly defined and utilized equally by authors and indexers. Other dysfunction terms, though, might be understood by experts but not by indexers, leading to an apparent under-representation of some neuromuscular dysfunction terms.
Current Status of Standard Glossaries in Manual Medicine
In 1975 the National Institutes of Health sponsored a multidisciplinary research conference on status of spinal manipulative therapy . The orthopedic, osteopathic, and chiropractic representatives reported on the understanding and use of the terminology in each respective profession. There was no consensus within or amongst professions at that time.
Searching the Internet for orthopedic terminology for professionals, we located many glossaries designed to facilitate patient comprehension of orthopedic terms. For example, the American Academy of Orthopaedic Surgeons provides a glossary of orthopedic diagnostic tests online which seems to be designed for patients . At this web site, we did not identify a glossary of standard terminology for orthopedics.
The osteopathic profession developed a standardized Glossary of Osteopathic Terminology in 1981 that has been updated annually ever since . The online version is available at the AOAnet Yearbook and Directory . The osteopathic literature database OSTMED went online October 1, 2002. For standard indexing practice, the database uses the NLM MeSH terms and about 70 terms that are uniquely osteopathic, called Osteopathic Subject Headings (OSH). To further assist in accessing the osteopathic literature database, Manipulation, Osteopathic became a MeSH term in 2002 .
The Chiropractic Library Consortium (CLIBCON) published the first list of chiropractic thesaurus (CHIROSH) in 1979 to improve access to the chiropractic literature. The consortium produced the first version of the Index to Chiropractic Literature in print in 1980 and added the online version in 1985. The standard indexing practice is based on NLM MeSH and CHIROSH . Chiropractic librarians but not the chiropractic practitioners use this CHIROSH thesaurus. Presently there does not seem to be a universally accepted glossary of terms in the chiropractic profession .
In recent years, PubMED MEDLINE articles that were submitted by publishers do not contain MeSH vocabulary. This increasingly common situation contributes to the data retrieval problems that we have presented here and furthers the argument for authors to use controlled vocabulary or commonly understood terminology in the titles and abstracts of their publications in order to facilitate retrieval of their studies by key terms.
In the field of CAM, semantic indexing might be more useful than analytic indexing because keyword indexing is generated by computer programs that pool words or phrases used by authors at least twice. For instance, if the author mentioned subluxation at least twice in the title or abstract, the term will be indexed regardless of professional agreement about the definition of the term.
Strengths and Limitations of the Controlled Vocabulary and Keywords
In our study, the limitations of semantic indexing in Web of Science produced the lowest yield of relevant studies. While Web of Science indexed 26/38 studies, only a few of these had been retrieved by the search strategy. Instead, we arrived at the 26 studies by individually verifying the 38 studies that we knew to exist by using author name and title words.
Of the 26 studies indexed in Web of Science, nine had been generated with both Author Keywords and Keywords Plus, and six studies with Author Keywords only. Web of Science did not generate any keywords for 11/26 studies. If investigators relied on searching by Keyword Plus, the search strategy might miss many potentially relevant studies. At the same time, if investigators are not familiar with the terms used by authors, a search strategy might miss relevant studies as well.
In Web of Science, Author Keywords have only been available for search since 1991. This limits users' ability to find relevant literature prior to 1991. A search of earlier years has to rely on title word searches. It is likely because of this limitation, we found relatively few Author Keywords or Keywords Plus for our four search concepts.
For databases that utilize analytic indexing, like MEDLINE, the NLM Medical Subject Headings Section staff continually revises and updates the MeSH vocabulary. While there were major changes in the 2002 NLM MeSH thesaurus for CAM, the current 2003 edition added only one CAM therapeutic modality. It would be helpful if investigators, information specialists and CAM practitioners become more active in contributing to the selection process of CAM descriptors in NLM MeSH and other database descriptors. Individual subscribers and subscribing institutions as well, have significant influence on the modifications and updating processes of databases.
The NLM Web site has a page for suggesting MeSH changes with a one or two-sentence statement, the reason for the change, and one or two authoritative citations . Since the NLM Medical Subject Headings are the standard controlled vocabulary for the allied health and biomedicine, investigators and authors in the fields of osteopathic medicine, chiropractic and physical medicine should take the opportunity to establish consistent terminology in the field of validity of spinal palpation for neuromuscular dysfunction.
We recognize the difficulties of adding new descriptors and updating the existing controlled vocabulary because searching for a new concept requires a dynamic thesaurus while searching retrospectively requires stability and continuity . Fortunately, the National Library of Medicine implements changes to the MeSH thesaurus in MEDLINE retrospectively.