Pilot Survey

2002: Pilot Survey Development

The initial research team for the project worked closely with a Technical Advisory Panel (TAP) to design the original College Students' Beliefs and Values (CSBV) survey. Our basic approach to this very complex assessment task might be characterized as one of “informed consensus.” That is, by informing ourselves about the considerable literature in this field, by consulting regularly with expert researchers in this field (our TAP), and by engaging in an ongoing dialogue to share what we were learning, we could best assure that we would be able to develop a meaningful and useful set of measures of students’ spiritual and religious qualities. Accordingly, in the year prior to the beginning of instrument development the research team read a good deal of relevant literature on spirituality, participated in a graduate seminar on spirituality in higher education, and took part in a number of brainstorming sessions concerning the meaning and assessment of spirituality.

The process of survey development began with an exploration of various definitions of “spirituality” proposed by scholars in business, education, health, psychology, sociology, and other fields. Because a number of psychologists and measurement specialists during the past decade have also attempted to develop measures of “spirituality” and “religiousness,” this critical body of work was reviewed as well. Although the research team’s evaluation of these measurement tools indicated that they contain a number of interesting and potentially useful items, no single instrument appeared to be well-suited to the purposes of this project. Among the limitations inherent in many of these instruments are the following:

• “Spirituality” is often equated with traditional religious practice and beliefs. Questions, for example, often assume (either explicitly or implicitly) that the respondent embraces a monotheistic/Judeo-Christian belief system
• No distinction is made between one’s “spirituality” and one’s theological perspective.
• No distinction is made between “inner” and “outer” manifestations of spirituality, that is, between spiritual attitudes/beliefs/perspectives and spiritual action or behavior.

In developing the new survey instrument the research team thus sought to design a set of questions that would satisfy the following requirements:

• No assumptions would be made about the student’s religious/spiritual perspective (or lack thereof). All students—regardless of their particular theological/metaphysical perspective or belief system—should be able to respond in a meaningful way.
• References to “God,” “Supreme Being,” or similar constructs would be held to a minimum, and respondents would be permitted to specify what such a concept means to them (including an option to reject the concept).
• Both spiritual beliefs/perspectives and spiritual practices/behaviors would be covered, although the use of specific denominational or sectarian terminology would be avoided (e.g., “sacred texts” would be used instead of “Bible” or “Koran.”)
• The instrument would accommodate those who define their spirituality primarily in terms of conventional religious beliefs and practices as well as those who define their spirituality in other ways.

One key resource that the research team relied heavily on in developing the CSBV was Hill and Hood’s (1999) comprehensive analysis of 125 different scales that had been developed in this area of research. The team examined every item in every scale and also attempted to identify any measurement problems associated with many of these instruments: ceiling effects, social desirability, response set, and lack of precision in defining the constructs that each scale purports to measure. This preliminary work resulted in the identification of twelve content areas or “domains” to be considered in designing items and scales to measure spirituality and religiousness:

• Spiritual/religious outlook/orientation/worldview
• Spiritual well-being
• Spiritual/religious behavior/practice
• Self-assessments (of religiousness, spirituality and related traits) 
• Compassionate behavior
• Sense of connectedness to others and the world
• Spiritual quest
• Spiritual/mystical experiences
• Facilitators/inhibitors of spiritual development
• Theological/metaphysical beliefs
• Attitudes toward religion/spirituality
• Religious affiliation/identity

Using these domains as a framework, the team developed a large number of potential survey items. In addition to modifying many of the items developed by earlier investigators, the team also created a number of new items. Throughout this process, TAP members and the research team served as “judges” in finalizing the relevant domains and selecting the most appropriate items for each domain. Because most domains had more items than needed, decisions concerning which items to include were made primarily on the basis of inter-judge agreement. Finally, after incorporating detailed feedback from the TAP, the research team prepared a draft pilot survey instrument. These items were initially combined into a questionnaire called the 2003 College Students’ Beliefs and Values (CSBV) Survey.

2003: Pilot Survey Administration

The four-page College Students’ Beliefs and Values (CSBV) pilot questionnaire was administered to 3,700 college juniors attending a diverse sample of 46 baccalaureate colleges and universities in Spring 2003. The institutional sample was designed to insure diversity with respect to institutional type (colleges and universities), control (public, private-nonsectarian, Roman Catholic, Protestant, Evangelical) and selectivity level. For the purposes of creating a longitudinal sample, these students were selected because they had already participated in the 2000 Cooperative Institutional Research Program (CIRP) annual Survey of Entering Freshmen three years earlier. Although institutions were selected randomly within CIRP stratification cells, sampling was necessarily disproportionate within cells in order to maximize institutional heterogeneity within the sample. Sixty-five institutions were initially invited to participate with the aim of achieving a sample of 50; forty-six eventually agreed to participate.

The final CSBV pilot survey included approximately 150 items having to do with spirituality and religion, 50-60 other items covering students’ activities and achievements since entering college (e.g., participation in student organizations, college GPA), and posttests on selected items from the freshman questionnaire that these same 3,700 students had completed three years earlier in fall 2000 (e.g., importance of “helping others in difficulty”).

For information about requesting use of the 2003 pilot survey data, please visit the UCLA Higher Education Research Institute’s data access page.