Tải bản đầy đủ
Instrument 7. A: A Large-Scale Survey Using Rating Scales

Instrument 7. A: A Large-Scale Survey Using Rating Scales

Tải bản đầy đủ

162

Designing and Constructing Instruments for Social Research and Evaluation

procedures and a concern that policies under a previous administration had led to
demoralization among the public workforce. In the cover letter to public employees,
the governor articulated the purpose of the survey: “As I have stated repeatedly
since my inauguration, I am keenly interested in your concerns about our State
government in general and your personal experience as an employee. You are the
backbone of Virginia’s government, and as such you know what works well and
what areas need improvement. In order to make meaningful change, I need your
cooperation” (Office of the Governor, Commonwealth of Virginia, 1998).
The Virginia State Employee Survey, 1998, is a two-page survey form, printed
on one sheet of paper, front and back. The title is centered and printed in capital
letters at the top of the page. This is followed by a statement on anonymity and
then by instructions for completing the survey. The first section of the survey consists of eighty-two items using a Likert format that includes the response choice
don’t know. Each item is numbered. Each item is unidimensional, that is, presents
only one concept. It is evident that multiple items are being used to measure a
topic. For example, items 2, 10, 11, 18, 31, and 39 address the issue of promotion
opportunities. A number of related questions obtain information about pay, benefits, training, and workplace conditions. The number 8 was probably assigned
to don’t know responses, in preparation for statistical analysis where values greater
than presented in the response set are used for alternative responses, such as a
don’t know response and unknown or missing values.
This employee survey consists of items that measure attitudes and beliefs. Items
use many qualifying terms, such as: “I generally feel . . .,” “I frequently feel . . .,” “I am
usually . . .,” and “I sometimes . . .” The terminology also features many words that
describe a state of being, such as confident, satisfied, appreciated, respected, and valued.
Although the primary format is rating scales, four open-ended questions are
included, items 83 to 86, and respondents are encouraged to expand on their
answers on a separate sheet of paper if needed. The last section consists of fourteen demographic items, which are clearly labeled as such. These items employ an
alternative response set format, which we will discuss in Chapter Nine. Finally, it
is worth noting that to fit all these items on two 8½-by-11-inch pages, the original
type size was quite small.
The Virginia State Employee Survey is a job satisfaction survey and is a good
example of the use of rating scales to measure opinions and beliefs. The purpose
of the instrument is clearly defined in the cover letter, and the findings of the survey have been made available through a state agency Web site. Item construction
conforms to the guidelines presented in this chapter.
The survey was mailed out to 143,377 state employees, and 45,598 completed
surveys were returned. With this response rate the results cannot be generalized
to the entire population of state employees; however, as the developers of the

c07.indd 162

7/9/07 11:49:38 PM

The Structure and Format of Selection Items

163

instrument note, “the large number of respondents, the general similarity of
the respondents’ demographics with demographics known for classified employees, and the patterns of response all lend credence to the results” (“Employees
Respond,” 1999, p. 2).
The fact that the instrument was developed by a survey lab located in a public
university lends credibility to the instrument design. However, public documents (for
example, the narrative summary for this survey) do not indicate how the instrument was
pretested and what steps were taken to assess instrument validity and reliability.
INSTRUMENT 7.A: LARGE-SCALE EMPLOYEE
SATISFACTION SURVEY.
THIS IS AN ANONYMOUS QUESTIONNAIRE. DO NOT PUT YOUR NAME
ON THIS DOCUMENT.

c07.indd 163

AGREE

STRONGLY
AGREE

1

2

3

4

8

1

2

3

4

8

1
1

2
2

3
3

4
4

8
8

1
1
1

2
2
2

3
3
3

4
4
4

8
8
8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

DON’T
KNOW

DISAGREE

1. State policies and procedures for employees
makes sense to me.
2. Employees are usually promoted based on
performance.
3. I am confident in the leadership at my agency.
4. If I do a good job, I think I should get
more money.
5. I am satisfied with my retirement plan.
6. It is important to me to feel appreciated at work.
7. Employees are usually respected as individuals at
my agency.
8. Fear of losing benefits has kept me from looking
for another job.
9. I generally feel informed about changes that
affect me.
10. Hard work is usually rewarded in my agency.

STRONGLY
DISAGREE

For each of the following statements, please indicate if you agree or disagree by circling the
appropriate number. This survey will be used to better understand the views of the State’s
work force. It is very important that you give answers that describe current conditions. If you
work in a university, please consider your university as the “agency.”

7/9/07 11:49:38 PM

164

Designing and Constructing Instruments for Social Research and Evaluation

11. I am aware of promotion opportunities.
12. Overall, I believe my employee evaluation system
is fair.
13. I am paid adequately for my responsibilities.
14. I am proud to be a State employee.
15. I am satisfied with my health insurance plan.
16. I am usually treated with respect at
my workplace.
17. Employees have equal access to job-related
training opportunities.
18. I have the opportunity to progress within the
State system.
19. I have adequate technology to do my job.
20. There is generally no discrimination shown at my
agency.
21. I am paid at an appropriate level for
my qualifications.
22. I feel I am well prepared to do my job.
23. I frequently feel stress in my job.
24. I believe that rewards are given fairly
where I work.
25. I get the training I need to do my job well.
26. I know people in my agency that don’t do their
share of the work.
27. The relationship between management and
employees is good.
28. My supervisor takes a personal interest in helping
me get ahead at my job.
29. There are people to whom I can go for help
when I have work-related problems.
30. I can leave work to take care of personal
matters if I need to.
31. I have the opportunity to learn skills that will
improve my chances for promotion.
32. I usually have the equipment I need to do
my job well.
33. My agency offers flextime or alternative
schedules.
34. You have to know the right people to get ahead
in the State system.
35. I know where to go for information related
to benefits.

c07.indd 164

1
1

2
2

3
3

4
4

8
8

1
1
1
1

2
2
2
2

3
3
3
3

4
4
4
4

8
8
8
8

1

2

3

4

8

1

2

3

4

8

1
1

2
2

3
3

4
4

8
8

1

2

3

4

8

1
1
1

2
2
2

3
3
3

4
4
4

8
8
8

1
1

2
2

3
3

4
4

8
8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

7/9/07 11:49:38 PM

The Structure and Format of Selection Items

36. I know where to go to get the information that
I need to do my job.
37. I think the State’s Fraud, Waste and Abuse Hotline
saves the State money.
38. I receive enough recognition for the work
that I do.
39. I have the opportunity to advance within
my agency.
40. I sometimes doubt the truth about what
management tells me.
41. I think of my benefits as part of my earnings.
42. I understand the services offered by the State
Employees’ Assistance Program.
43. I usually hear about important changes through
rumors rather than management
communication.
44. I would recommend my agency as a good place
to work.
45. I would like more flexibility in my benefit options.
46. I feel I am valued at work.
47. Computers are a good way for the State to get
information to me.
48. If I do a good job I have a better chance of
getting ahead.
49. I understand how my retirement benefits are
calculated.
50. It is easy to get answers to questions about
personnel policies.
51. It’s safe to say what I think at my job.
52. It takes too long to hire someone when
a position becomes vacant.
53. My agency offers enough training opportunities.
54. I usually know in plenty of time when important
things happen.
55. My job description matches my job duties.
56. I would recommend the State as a good place
to work.
57. My job is interesting to me.
58. My work-related concerns are generally handled
to my satisfaction.
59. People generally feel appreciated where I work.
60. My agency has a genuine concern for safety.

c07.indd 165

165

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1
1

2
2

3
3

4
4

8
8

1

2

3

4

8

1

2

3

4

8

1
1
1

2
2
2

3
3
3

4
4
4

8
8
8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1
1

2
2

3
3

4
4

8
8

1
1

2
2

3
3

4
4

8
8

1
1

2
2

3
3

4
4

8
8

1
1

2
2

3
3

4
4

8
8

1
1

2
2

3
3

4
4

8
8

7/9/07 11:49:39 PM

166

Designing and Constructing Instruments for Social Research and Evaluation

61. My agency is a place where individuals with
disabilities can work comfortably.
62. I am told if I have done a particularly good job.
63. Policies and procedures are usually applied
equally to all employees.
64. Procedures necessary to do my job often involve
unnecessary steps.
65. The benefits package that I receive is not
as good as most available in the private sector.
66. The current pay system has a positive affect on
employee productivity.
67. The grievance process available to me is a fair
way to resolve disputes between employees and
management.
68. Personnel procedures make it difficult to get
rid of poorly performing employees.
69. The information that I have received about
employee benefits is easy to understand.
70. The meetings that I have with my co-workers
and supervisor help me to get my job done.
71. I am paid appropriately for the work-related
experience that I have.
72. The State’s Employee Suggestion Program
saves the State money.
73. The State should be more flexible in
personnel matters.
74. I like my benefits package.
75. The State’s communications keep me
up-to-date.
76. The State provides a way for me to get
confidential help when I have personal problems
that affect my work.
77. My agency offers flexible working arrangements.
78. When discussing problems or complaints,
I feel that I am treated seriously.
79. I worry about losing my job.
80. There is too much paperwork involved in doing
my job.
81. I get a feeling of personal satisfaction from
my work.
82. I read most of the State’s newsletters that
I receive.

c07.indd 166

1

2

3

4

8

1
1

2
2

3
3

4
4

8
8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1

2

3

4

8

1
1

2
2

3
3

4
4

8
8

1

2

3

4

8

1
1

2
2

3
3

4
4

8
8

1
1

2
2

3
3

4
4

8
8

1

2

3

4

8

1

2

3

4

8

7/9/07 11:49:39 PM

The Structure and Format of Selection Items

167

Please use additional paper for the following questions if necessary.
83. What do you think is the most important action that should be taken to improve the
quality of work life for State government employees?

84. What do you think is the best thing about working for the State?

85. What do you think is the worst thing about working for the State?

86. What is one thing that can be done to make State government run more
efficiently?

DEMOGRAPHIC INFORMATION: The information you provide below will be grouped
with answers received from other employees and will NOT be used to identify you.
Smaller agencies of 25 or fewer employees automatically will be grouped to maintain
confidentiality.
87. Type of employee:
❑ Wage (check one):
❑ Adjunct faculty
❑ Hourly, not student
❑ Student employee/
Grad Assist.
❑ Salary/Contract
88. Do you work full or part-time?
❑ Full-time
❑ Part-time

c07.indd 167

89. Total years you’ve worked for the
State
❑ Less than 2 years
❑ 2–4 years
❑ 5–9 years
❑ 10–14
❑ 15–20
❑ More than 20
90. Zip code at work: ______________

7/9/07 11:49:39 PM

168

Designing and Constructing Instruments for Social Research and Evaluation

91. My usual work schedule:
❑ Days
❑ Evenings
❑ Nights
❑ Rotating schedule
❑ Weekends only
92. Do you supervise staff at your
workplace?
❑ Yes
❑ No
93. What is the highest level of education
that you completed?
❑ Less than high school
❑ High school or GED equivalent
❑ Associate Degree
❑ College Degree
❑ Master’s Degree or higher
94. Your State grade level
❑ Grades 1–3
❑ Grades 4–8
❑ Grades 9–12
❑ Grade 13 and above
❑ Not a graded position
❑ Faculty/Adjunct
❑ Don’t Know

96. Agency Code: (please refer to yellow
code sheet enclosed):
____ ____ ____
97. Your age:
❑ Under 25
❑ 25–34 years
❑ 35–44 years
❑ 45–54 years
❑ 55 or over
98. Do you have a disability?
❑ Yes
❑ No
99. Your race:
❑ African-American
❑ Asian, Pacific Islander
❑ American Indian
❑ Hispanic
❑ White 0
❑ Other
100. Your sex
❑ Female
❑ Male
THANK YOU FOR YOUR HELP!

95. Your annual State pay
❑ Under $10,000
❑ $10–19,999
❑ $20–29,999
❑ $30–39,999
❑ $40–49,999
❑ $50,000 or more

Instrument 7.B: An Assessment Instrument
Using Graphic Scales
The Brief Situational Confidence Questionnaire (Sobell, 1999) assesses an individual’s ability to resist use of alcohol under a variety of situations, such as when

c07.indd 168

7/9/07 11:49:39 PM

The Structure and Format of Selection Items

169

stressed, when urges appear, or during social activities. Each of the eight items
presents the respondent with a statement beginning with “I feel,” a description
of the situation, and the instruction to place an X on the line to indicate his or
her level of resistance. To facilitate accurate marking of the graphic scale, the
questionnaire includes an example illustrating how to rate each item. Each item
uses the same anchors: not at all confident and 100% totally confident.
With a graphic scale it is difficult to assign a number to the location where the
respondent places his or her mark. Therefore this instrument does not produce a
total score for the eight items. Instead, a practitioner, such as a substance abuse
counselor, uses the marked scales to determine the individual’s situations of most
resistance and least resistance to using alcohol. This information can then be
incorporated into the individual’s treatment process.

INSTRUMENT 7.B: BRIEF SITUATIONAL
CONFIDENCE QUESTIONNAIRE.
Brief Situational Confidence Questionnaire
Name: _________________________________ Date: ___________________
Listed below are eight types of situations in which some people experience an alcohol
or drug problem. Imagine yourself as you are right now in each of the following types
of situations. Indicate on the scale provided how confident you are right now that
you will be able to resist drinking heavily or resist the urge to use your primary drug
in each situation by placing an “X” along the line, from 0% “Not at all confident” to
100% “Totally confident” as in the example below.
I feel …
X
0%
Not at all confident

100%
Totally confident

Right now I would be able to resist the urge to drink heavily or use my primary
drug in situations involving . . .
1. UNPLEASANT EMOTIONS (e.g., If I were depressed about things in general; if
everything were going badly for me).
I feel . . .
0%
Not at all confident

c07.indd 169

100%
Totally confident

7/9/07 11:49:40 PM