Tải bản đầy đủ
Instrument 13. A: Behavioral Rating Scale

Instrument 13. A: Behavioral Rating Scale

Tải bản đầy đủ

Administering the Instrument

331

EXAMPLE
1. Hyper Vigilant
0
Not Present:
Behavior was not
observed.

1
Mild: Has
expressed distrust
and need to
watch others;
tends to scan the
environment.

2
Moderate: Intermittent periods
of watching
others or the
environment
to the extent that
the individual is
not attending to
immediate tasks.
May express
belief that
others are
plotting against
him/her.

3
Severe: Watching is pervasive
and becomes the
primary task to
the extent that
attention to other
tasks is compromised. May
associate everyday activities with
plots of harm.
May not want to
interact with
others due to
these fears.

INSTRUMENT 13.A: BEHAVIORAL RATING SCALE.
Child and Adolescent Inpatient Behavioral Rating Scale
INSTRUCTIONS:
This instrument may be used as either a pretreatment/post-treatment measure or as
the basis for repeated (weekly, daily, shift-to-shift, or hourly) observations.

Pretreatment/Post-Treatment Measure
Information to complete this instrument is based on the youngster’s behaviors
during the first 72 hours after admission (pretreatment) and within 72 hours prior
to the youngster’s planned discharge (post-treatment). Information to complete the
rating scale is based on:
• Direct observation
• Interview and interactions with the youngster
• Chart notes and verbal feedback from other members of the treatment team
The entire form (all items) should be completed. If the behaviors are not observed,
use “Not Present” to rate the item. The primary means of comparing pretreatment
and post-treatment scores is through the process of “eye-balling” to determine where
responses have changed over time. Items can also be graphed and the differences
observed through visual comparison.

c13.indd 331

7/9/07 11:53:35 PM

332

Designing and Constructing Instruments for Social Research and Evaluation

Daily Behavioral Observation—Clinical Indicators
Behaviors that the treatment team wishes to monitor should be identified at the time of
admission and/or during the treatment-planning meeting. Typically, the team will identify those specific behaviors that are related to treatment problems and objectives. Items
65 and 66 provide space to add behaviors which are not included on the checklist, but
which the treatment team may chose to monitor using the same scale. Typically, one,
two, or three items should suffice for each treatment problem and objective.
Depending on the behavior, the treatment team should determine the frequency of
observation; for example, observations could be made hourly or twice a shift. In this
way, the behavioral rating scale can be used in conjunction with behavior analysis and
the assessment of treatment effects.
The purpose of the rating system is to provide data on which to base continuing
assessments of the youngster’s response to treatment. The instrument does not replace
the need for analysis of the data. For example, if a pattern of behavior is detected, this
may suggest that additional information is required, such as a situational analysis to
determine factors that may elicit the behaviors. Finally, the utility of the data can also
be enhanced by using the data in conjunction with other sources of information, such
as anecdotal reports and the results of other assessment measures.

Anxiety

Mild Moderate Severe



1. Hyper vigilant

0

1

2

3



2. Difficulty settling at night

0

1

2

3



3. Repetitive behaviors

0

1

2

3



4. Nightmares/flashbacks

0

1

2

3



5. Low startle threshold

0

1

2

3



6. Panic attacks

0

1

2

3



7. Grandiose

0

1

2

3

Depression

c13.indd 332

Not
Present

Not
Present

Mild Moderate Severe



8. Withdrawn

0

1

2

3



9. Sad affect

0

1

2

3



10. Flat affect

0

1

2

3



11. Crying spells

0

1

2

3



12. Tired/loss of energy

0

1

2

3

7/9/07 11:53:35 PM

Administering the Instrument



13. Negative self-statements



14. Physical complaints

0

1

2

3



15. Irritable

0

1

2

3



16. Self-harmful statements

0

1

2

3



17. Self-injurious behavior

0

1

2

3

Communication Problems

Not
Present

0

1

2

3

Mild Moderate Severe



18. Loud/shouting



19. Under-productive speech

0

1

2

3



20. Incoherent speech

0

1

2

3



21. Pressured speech

0

1

2

3



22. Disorganized speech

0

1

2

3



23. Echolalia

0

1

2

3

Psycho-Motor Activity

0

Not
Present

1

2

3

Mild Moderate Severe



24. Dizziness and/or difficulty standing

0

1

2

3



25. Exaggerated mannerisms

0

1

2

3



26. Stereotypical movements

0

1

2

3



27. Perseveration

0

1

2

3



28. Tremors and tics

0

1

2

3



29. Psychomotor retardation

0

1

2

3



30. Clumsiness

0

1

2

3

Attention Problems/Hyperactive

c13.indd 333

333

Not
Present

Mild Moderate Severe



31. Difficulty staying on task

0

1

2

3



32. Difficulty following directions

0

1

2

3



33. Distracted by external stimuli

0

1

2

3



34. Distracted by internal stimuli

0

1

2

3



35. Fidgets/restless

0

1

2

3



36. Hyper-kinetic

0

1

2

3

7/9/07 11:53:36 PM

334

Designing and Constructing Instruments for Social Research and Evaluation

Conduct Problems/
Disruptive Behaviors

Mild Moderate Severe



37. Cursing

0

1

2

3



38. Argumentative

0

1

2

3



39. Frustration/tantrums

0

1

2

3

0

1

2

3

❑ 40. Disobedient
❑ 41. Does not accept responsibility

0

1

2

3

❑ 42. Rude

0

1

2

3

❑ 43. Manipulates others

0

1

2

3



44. Lies

0

1

2

3



45. Verbally threatens

0

1

2

3



46. Physically intimating

0

1

2

3



47. Aggressive toward objects

0

1

2

3



48. Aggressive toward people

0

1

2

3



49. Demands must be met
immediately

0

1

2

3



50. Passively defiant

0

1

2

3

Social Skills

c13.indd 334

Not
Present

Not
Present

Mild Moderate Severe



51. Touches others when/where
they don’t want

0

1

2

3



52. Teases others

0

1

2

3



53. Does not maintain appropriate
social distance

0

1

2

3



54. Engages in attention seeking
behaviors

0

1

2

3



55. Interrupts or intrudes

0

1

2

3



56. Difficulty waiting one’s turn

0

1

2

3



57. Difficulty picking up social cues

0

1

2

3



58. Sexually inappropriate—directed
toward self

0

1

2

3



59. Sexually inappropriate—directed
toward others

0

1

2

3

7/9/07 11:53:36 PM

Administering the Instrument

335



60. Difficulty maintaining personal
hygiene

0

1

2

3



61. Incontinence (including
bedwetting)

0

1

2

3



62. Bowel management problems

0

1

2

3

Eating Habits


63.
Breakfast
Lunch
Dinner
Snack

Ate most of
this meal

Skipped most
of this meal

Picky about
what he/she ate

Overeats
or gorges

1
1
1
1

2
2
2
2

3
3
3
3

4
4
4
4

Sleeping Habits


64.

1

2

Sleeps thru the
Difficulty falling
night w/o incident
asleep

Other Behaviors (specify)

3

4

Awakens
early

Restless
sleeper

Not Present

Mild

Moderate

Severe



65.

0

1

2

3



66.

0

1

2

3

Endnote
1. A concern with providing clarification for one respondent but not another is that it may
influence the recipient’s response and thus raise questions about the validity and reliability of the administration process and the results obtained. If you believe this could be a
problem in your study, you might use another typical approach: when a respondent has
a question about an item, the interviewer will repeat the item but will not provide additional
information, even if asked.

c13.indd 335

7/9/07 11:53:36 PM

336

Designing and Constructing Instruments for Social Research and Evaluation

Key Concepts and Terms

c13.indd 336

cluster sample

informed consent

purposive sample

confidence interval

interval recording

random digit dialing

confidence level

margin of error

random sample

convenience sample

nonprobability sample

sampling

discrete categorization

nonresponse bias

site selection

duration

observer training

stratified sample

frequency measures

probability sample

vignette

7/9/07 11:53:36 PM

Y
CHAPTER FOURTEEN

COMPUTERS AND INSTRUMENT
CONSTRUCTION

In this chapter we will
• Examine how computers can assist in designing and organizing an instrument.
• Identify factors to address when administering an instrument through computer
technology, including the Internet and e-mail.
Traditionally, artists have painted on a variety of surfaces—canvas, wood,
and special papers. In recent years, however, artists have turned to computers to
create their works, often with software so sophisticated that they have had
to learn entirely different ways of managing the new medium. Similarly, computers are increasingly being used by researchers in the design and administration of questionnaires. During the past decade personal computer (PC) software
has made it possible to use PCs in instrument design and development, and
the Internet has opened up new approaches for delivering the instrument to the
user. In this chapter we will examine how computers can aid you in instrument
construction and how e-mail and the Internet can facilitate administration. As
with any approach, you should be aware of the pros and cons prior to deciding
to use computer-based approaches.
The first computers, or calculators, were designed to analyze the results of
survey data, specifically census data. The 1890 census was tallied using a computer
that read punch cards (cards carrying numerical codes in the form of holes
337

c14.indd 337

7/10/07 12:43:54 AM

338

Designing and Constructing Instruments for Social Research and Evaluation

punched in each card) for data entry. With this equipment, calculation of the
census took only six weeks, as compared to the seven and a half years it took
to manually tally the 1880 census! Fifty years later, the first electronic computers continued to use punch cards for data entry. These early computers were
massive and yet had much less computational power than contemporary hardware (Kroenke, 1984). The early 1980s saw the advent of the personal computer.
However, the computer operator had to write, or hire someone to write, his or
her own software, which limited the PC’s use. The breakthrough came when
prewritten software was developed to address specific needs, such as word processing, accounting, and graphic design. In the late 1980s and early 1990s, software
designers began to develop applications for survey research. The first applications
operated on mainframe computers. Today, however, an individual can purchase
software that can operate on a personal computer with the same capabilities
to design, administer, and analyze surveys as the software used by large survey
research organizations.
We are discussing software here, near the end of this book, for a very important reason. Although survey software can assist you in the design of your instrument and the analysis of your data, it cannot replace your personal knowledge
about what constitutes an effective instrument or questionnaire item. At best, it
can take a poorly worded question and give it a polished appearance. However,
as instrument designer, you are still responsible for the quality of the individual
items and the organization and appearance of your questionnaire.
Computer software can do three things to make the process of instrument
construction more efficient. First, it can assist in the construction of individual
items and particularly in their formatting. Second, it can facilitate questionnaire
design, such as the organization of items and the final overall appearance. Third,
some (not all) software helps you perform data entry and analysis. Typically,
either this last feature is a component of the software or the software allows you to
export the data to an application whose primary purpose is statistical analysis. In
the next section we will examine how computer software can assist you with item
construction and organization of the questionnaire; issues related to administering
Web surveys will be addressed in the last section.

Item Construction and Questionnaire Organization
Item Construction
Computer software can assist you in formatting individual items and designing the
final appearance they will take. The first step is to articulate the item (following
the steps outlined in Part Two of this book). You can then chose an item format,

c14.indd 338

7/10/07 12:43:54 AM

Computers and Instrument Construction

339

such as Likert scale, numerical scale, rank order, or open-ended sentence, from
a menu of styles. With this menu you can also indicate the number of response
choices you want, and select such options as making no opinion a response choice.
Most survey software includes word processing features such as spelling and grammar checkers as well. Once you have selected all your options, the software will
add the question to the instrument and prompt you to go on to the next item.
Typically, survey software includes a variety of templates from which you can
select the physical appearance of each item. For example, you can select a graphic
for recording a response from a menu with various boxes (❑ or ) and circles (❍
or ), where boxes are used to mark all-that-apply options and circles are used
to make a selection. Although this can be done with word processing software as
well, a ready-made template saves time, and once you have established the format
you want to use, the software can apply that format to all subsequent items.

Layout and Design of the Questionnaire
Another advantage of using survey software is the ease of constructing an
instrument that is appealing in appearance. Templates facilitate arrangements that
are easy to read and follow. In the following example, items are easy to read
because the stem has been separated from the response set, and response choices
are easy to differentiate because they are separated by lines and shading. Although
this can be accomplished with word processing software, here again, using preset templates is more efficient. In addition, some survey software will allow you
to print item directions in a different color or to apply background shading in
colors.

Item

c14.indd 339

Strongly
Agree

Agree

No
Opinion

Disagree

Strongly
Disagree

Members of Congress should be
limited to serving three terms.











Raises to members of Congress
should be limited to no more
than 2% of their salary each year.











I believe that my Congressman/
woman adequately represents
my needs.











7/10/07 12:43:55 AM

340

Designing and Constructing Instruments for Social Research and Evaluation

Special Considerations for Web Instruments
When using a paper-and-pencil document, the user can see the instrument in its
entirety at once. However, the amount of information the Web-page user sees is
dependent on the display size and resolution the user selects for his or her computer screen. For this reason, what users see may vary from monitor to monitor. If
the instrument is no more than a page of written text, you may want to place the
entire instrument on one Web page, allowing the respondent to scroll through it.
This can be important when the items are related or the respondent needs to refer
back to a prior item or response choice. Alternatively, many software programs
allow you to break your questionnaire into sections so that the user completes
just a few items per page (screen) without having to scroll. Schonlau, Fricker, and
Elliott (2001) sum up the issues: “Excessive scrolling can become a burden to
respondents and lengthy Web pages can give the impression that the survey is too
long to complete, both of which have the potential to negatively impact response
rates. . . . Also, there is some evidence that using only a single screen or a few
screens for short surveys minimizes respondent ‘abandonment’ (starting but not
completing a survey) whereas using a single screen and forcing the respondent to
scroll down in long surveys increases abandonment” (p. 42).
Pretesting will of course assist you in determining how many questions to
place on one screen as well as how to organize them. Issues that pretesters may
remark on are the size of the font used, the arrangement of the item(s) on the
screen, the use of background colors and graphics to highlight information, and
also equipment considerations, such as the size of the monitor.
To assist users in moving through multiscreen Web instruments, and to provide feedback about their rate of progress, some software programs allow you to
include a counter indicating how many pages or items have been completed and
how many more are left to finish. The counter can simply indicate the number of
items or can include text such as, “This is item #17, there are eight more items to
complete in this section.” With long questionnaires a downside of counters can
be that the respondent sees there are still many pages to complete and gives up.
One way to address this possibility is to use software that allows the user to log
off and then log back on at a later time, picking up where he or she left off. This
method usually requires assigning each user a unique password that the software
can track. Consequently, you may want to look for this feature when purchasing
survey software.
Survey software can also create links that support skip items and directions.
Rather than manually navigating the instrument, the respondent can click a yes or
no, or if or then, box and be automatically transferred to the next appropriate question. Another automated process can eliminate missed items; if the respondent

c14.indd 340

7/10/07 12:43:55 AM

Computers and Instrument Construction

341

does not rate an item, the software will not allow him or her to move on to
the next item. The downside to this feature is that respondents may become
frustrated and log off without completing the questionnaire: “In our view
respondents should not be forced to provide an answer before moving on. Sometimes respondents have legitimate reasons for objecting to providing and answer
and may, in fact, be unable to provide the answer to some questions” (Dillman,
Tortora, & Bowker, 1998, p. 11).
Schonlau et al. (2001) point out yet another advantage to Web surveys—
the designer can easily use color and graphics. Color is helpful for setting off
important information. You can use different colors as backgrounds for different
sections or highlight text with more than one (but not too many) colors. Graphics can include symbols and pictures, although you should be cautious in using
pictures or photos as they can be distracting and increase the time it takes to
download a page.
Data Collection and Scoring
Suppose you have constructed a survey that will be sent out to a sample of one
hundred respondents and it contains forty items. Depending on your response
rate, you could end up with about four thousand pieces of information that you
will need to enter into a database for analysis. One of the primary benefits of
survey software is that you can perform data collection and scoring very efficiently
through scanning or on-line administration. If you have administered a paperand-pencil instrument, a scanner can transfer the data to a computer. An optical
reader, for example, can scan thirty to sixty answer sheets a minute, transferring
the data to the database section of the software. A less expensive and more timeconsuming method (although still less time consuming than manual entry) is to
use a flatbed scanner, which might scan three to four answer sheets a minute.
Regardless of the scanner used, if the answer sheets are neatly and completely
filled out, this approach should also reduce the number of input errors.
Increasingly, instruments are being designed so that they can be administered
over the Internet. The respondent can check boxes or circles with the touch of a
computer mouse, and each entry is automatically entered into a database. This
increases reliability by avoiding the transfer step of scanning or manual entry and
thus reducing data entry errors.
More Pros and Cons of Using Survey Software
One of the primary advantages of using survey software is that you will be
purchasing an application dedicated to your task. As we have pointed out,

c14.indd 341

7/10/07 12:43:55 AM