This is an electronic version of the post-print (i.e. final draft post-refereeing) of a paper published as McGill, P., Hughes, D., Teer, K. and Rye, L. (2001) Variability in staff reports of the frequency of challenging behaviour. Research in Developmental Disabilities, 22(3), 221-231. https://doi.org/10.1016/S0891-4222(01)00069-5

© 2001. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/

Running head: STAFF REPORTS OF CHALLENGING BEHAVIOR
4,969 words
Variability in Staff Reports of the Frequency of Challenging Behavior

Peter McGill
David Hughes
Kerry Teer 
Lynne Rye

University of Kent at Canterbury, Canterbury, U.K.

This research was funded by a grant from the NHS National Research and Development Programme for People with Physical and Complex Disabilities. We are very grateful to the people with developmental disabilities, staff members and services that gave their consent and cooperation for the conducting of this study. David Hughes is now at Cumbria Care Services. Kerry Teer is now at Department of Psychological Medicine, University of Glasgow. Lynne Rye is now at East Kent Community NHS Trust and the Tizard Centre. Correspondence concerning this article should be addressed to Peter McGill, Tizard Centre, University of Kent at Canterbury, Canterbury, Kent, U.K. CT2 7LZ (E-mail: P.McGill@tizard.ukc.ac.uk).

Abstract

Information was collected from 88 staff about their observations of the occurrence and frequency of challenging behaviors in 22 individuals with developmental disabilities with whom they worked. Staff reports suggested considerable variability in challenging behavior in the presence of different staff and, from day to day, in the presence of the same staff. Variability was greater for stereotyped than for aggressive/destructive behavior, and for more frequent behavior. Managers reported generally less challenging behavior than their staff. The validity of the findings were discussed and their implications for research and practice considered.

Variability in Staff Reports of the Frequency of Challenging Behavior

Introduction

Natural variability in the occurrence of challenging behavior is more common than not. Such variability has been explained as a result of variations in the occurrence of setting events (Carr, Reeve, & Magito-McLaughlin, 1996; Gardner, Cole, Davidson, & Karan, 1986), antecedent events that evoke challenging behavior (McGill, 1999; Smith & Iwata, 1997), the application of reinforcement contingencies (Iwata et al., 1994b), and biological factors (Carr, 1994). Such variability has, therefore, been of particular interest to those responsible for assessing challenging behavior, and this process of assessment can essentially be seen as one of hypothesizing the factors responsible for such variability either indirectly through questionnaires (e.g., Durand & Crimmins, 1988; Matson, Bamburg, Cherry, & Paclawskyj, 1999) or interviews (e.g., O'Neill, 1997), or more directly through observation (e.g., Emerson, Thompson, Reeves, Henderson, & Robertson, 1995). The most complex approaches to such assessment have relied not on naturally occurring variability but on the imposed variability (Sidman, 1960) that arises when an individual is exposed to specially designed settings intended as analogs to the natural environment (Iwata, Dorsey, Slifer, Bauman, & Richman, 1994a).

This pragmatic approach to variability may have diverted attention from the phenomenon itself. Thus, in general, we can conclude very little about the range of variability that is likely to be found in an individual’s behavior, about the extent to which variability can be uniformly expected across individuals, or about its relationship with the topography of challenging behavior. Such knowledge (which, in part, might be derived from a re-analysis of previously published studies containing single case data) is important for a number of reasons. First, an interest in variability naturally leads to an interest in why challenging behavior sometimes occurs at high rates/intensities and sometimes at low or zero rates/intensities, and to the investigation of factors correlated with this variation. Of course, a good functional assessment might be expected to provide this information in respect of an individual, but cannot provide this information across individuals. Knowledge about the variability of the challenging behavior of many individuals is required before we can identify correlated factors of potentially general relevance (cf., Iwata et al., 1994b). Second, information about variability may help to promote a parametric analysis of its correlated factors, drawing attention to those factors that should receive most effort both in intervening with individuals and larger-scale prevention. Third, information about variability may direct attention to the development of interventions that seek to increase the occurrence of the factors associated with low rates of challenging behavior. Such approaches have been described only rarely in the literature (e.g.., Touchette, MacDonald, & Langer, 1985) but may avoid some of the implementation (Hastings & Remington, 1993), maintenance (Ager, 1991) and generalization (Stokes & Baer, 1977) difficulties frequently reported with more traditional behavioral interventions. Fourth, information about variability may promote a view of challenging behavior as partly under the control of the services provided to the person (Emerson, McGill, & Mansell, 1994). That is, the demonstration of variability may constitute a demonstration of both the service’s success (at times of low or zero occurrence) and the service’s failure (at times of high occurrence) that directs attention to the scope for, and the nature of, improvements in the service’s performance (cf., Gilbert, 1978).

Previous studies have suggested that one particular location of variability may be the people supporting or caring for the individual. Wing and Gould (1978) reported that parents described their children with developmental disabilities/autism as displaying more stereotyped and aggressive or difficult behavior than the children’s teachers or child care workers. They concluded that there were real differences in the children’s behavior in the presence of different informants. Holmes, Shah and Wing (1982), in a study of residents in a developmental disability hospital, found similar differences in the reports of nurses and day care workers and reached a similar conclusion, also noting that genuine differences in individual behavior could be found within the same setting. Differences within the same setting were, indeed, reported by the multiple raters used by Rojahn (1984) in his study of self-injurious behavior in an institution and by Touchette, MacDonald and Langer (1985), whose data included an individual recorded as displaying self-injury at high rates only when one member of staff was on duty. Touchette et al’s study included a reversal in which staff changed their shifts, with the rate of self-injury changing in parallel. None of these studies have set out to specifically investigate variability in challenging behavior so that the generality of conclusions drawn from them may be questioned. The phenomenon they have identified, however, is not specific to developmental disability (see, for example, Graham & Rutter, 1970) and is consistent with the established notion of situation specificity. This literature provides grounds, therefore, for supposing that challenging behavior may commonly vary across the presence of different staff or caregivers.

In the study described here, variability in the reported occurrence and frequency of challenging behavior in the presence of different staff was investigated directly, with a view to estimating the extent of variability, the extent to which variability can be uniformly expected across individuals, and its relationship with the topography of challenging behavior. Additionally, relationships with the status, role and gender of reporters were investigated. 

Method

Settings

Data were gathered in 18 residential service settings, 13 provided by private agencies, 3 by voluntary organizations, 1 by the NHS and 1 by a Local Education Authority (a residential school). Settings varied widely in size having on average 10 clients (range: 2-27) and 14 staff (range: 5-26). Some settings were parts of larger units (e.g., a bungalow in a village community) while some were completely independent of other services.

Participants

Data were gathered on 22 people with developmental disabilities - 13 male and 9 female. The average age of participants was 30 years (range: 13-54). Three participants were of Black British or other ethnic origin with all others being White British. Participants were described by residential home managers as having mild developmental disabilities in 4 cases, moderate developmental disabilities in 5 cases, severe developmental disabilities in 11 cases and profound developmental disabilities in 2 cases. Sixteen of the participants had additional disabilities including epilepsy, hearing impairment, physical disability, social impairment, psychiatric disorder or significant chronic illness. Nineteen of the group used at least minimal spoken communication with three having no speech. Participants were identified in conjunction with residential home managers as meeting the following three criteria: having a developmental disability; displaying at least one challenging behavior (not necessarily of a serious kind) that, at least sometimes, occurred daily or more often; not currently (or in the near future) undergoing significant life changes such as changing accommodation, starting a new treatment or changing medication.

Respondents

The respondents were four members of staff working with each participant including, in all cases, the residential home manager. Managers were asked to identify the members of staff in whose presence the participant’s most frequent challenging behavior was “most likely” and  “least likely” and one further member of staff whose presence “made no difference”. These sets of three staff were then selected as further respondents. In seven cases, however, managers reported that there was no variation in the target behavior across staff, and in nine cases managers were only able to identify staff in some but not all of the three categories. In such cases additional or all respondents were selected at random (with the proviso that all must have worked with the participant for at least one month) so that there was a set of four respondents for each participant. In nine cases respondents included participants’ key workers.

Respondents were, therefore, the 22 home managers and 66 additional staff. The latter were mainly unqualified or non-professionally qualified. On average they had worked in the settings for over two years (range: two months-nine years) and most also had previous experience of working with people with developmental disabilities. Their average age was 34 years (range: 18-63), 23 were men and 43 were women, and 62 were of white British origin (2 describing their ethnicity as “white other” and 2 as “other”). Less extensive demographic information was collected on the 9 male and 13 female managers.

Measures

As part of a series of related studies also reported elsewhere (McGill, Teer, Rye, & Hughes, submitted for publication), all respondents completed, by interview with one of the authors, the Behavior Problems Inventory (BPI) (Rojahn, 1992, updated January 1994). The BPI is a 29-item inventory requiring respondents to rate the frequency with which they had observed each item on a 7-point scale from “never” to “more than hourly”. The items include 15 topographies of self-injurious behavior (e.g., “self-biting”), 9 topographies of aggressive/destructive behavior (e.g., “hitting others with hand or other body part”) and 5 topographies of stereotyped behavior (e.g., “rhythmic rocking of the head, trunk, and/or extremities”). The BPI, or its predecessors, has been used in a number of studies of challenging behavior (Dura, Mulick, & Rasnake, 1987; Rojahn, 1984; Rojahn, 1986; Rojahn, Polster, Mulick, & Wisneiwski, 1989; Sturmey, Fink, & Sevin, 1993; Sturmey, Sevin, & Williams, 1995). All respondents, other than home managers, were also asked about their personal experience of the minimum and maximum frequency of the challenging behavior identified previously as being the most frequent displayed by the participant. 

Reliability

In five interviews with staff or home managers using the BPI, a second researcher independently coded the interviewee’s responses. Percentage exact agreement was 98.75%. In three interviews with staff, a second researcher independently coded reports of the minimum, average and maximum frequency of the participant’s most frequent challenging behavior. Percentage exact agreement was 100%.

In sixteen cases the BPI was re-administered by interview to the same staff/home managers with a median gap of 71 days (range 30-120 days). Overall (summing all relevant responses) percentage agreement for occurrence was 80% (self-injury items: 83%, aggression items: 76%, stereotypy items: 81%) and for frequency 65% (self-injury items: 71%, aggression items: 56%, stereotypy items: 64%). Overall chance corrected agreement for occurrence was calculated using kappa (Cohen, 1960) and was 0.60 (self-injury items: 0.60, aggression items: 0.51, stereotypy items: 0.62). Overall chance corrected agreement for frequency was calculated using weighted kappa (Cohen, 1968) and was 0.60 (self-injury items: 0.60, aggression items: 0.53, stereotypy items: 0.59). 

Results

On the 88 completed BPIs respondents reported an average of 10.3 (range: 1-22) problem behavior topographies including 4.0 self-injurious behaviors (range: 0-12), 4.4 aggressive/destructive behaviors (range: 0-9) and 1.8 stereotyped behaviors (range: 0-5). Frequency ratings were converted to numerical scores for each item (from 0 = “never” to 6 = “more than hourly”). The average total frequency score was 31.8 (range: 4-89), 11.6 for self-injury (range: 0-42), 12.6 for aggressive/destructive behavior (range: 0-36) and 7.6 for stereotyped behavior (range: 0-20). Occurrence and frequency scores were highly correlated (Total: r = 0.90, df=86, p0.001; SIB, r = 0.92, df=86, p0.001; Aggression, r = 0.87, df=86, p0.001; Stereotypy: r = 0.91, df=86, p0.001). Accordingly, all further analyses were conducted only for frequency scores but can be assumed to apply also to the reported occurrence of problem behaviors. Mean self-injury scores across raters were significantly correlated with mean stereotypy scores (r = 0.61, df=20, p0.01) but not with mean aggression scores. Aggression and stereotypy scores were not significantly correlated. The scores of different raters were also correlated. Correlations, averaged across the six possible pairs of raters were 0.66 (df=20, p0.001) for total scores, 0.69 (df=20, p0.001) for self-injury, 0.73 (df=20, p0.001) for aggressive/destructive behavior, and 0.52 (df=20, p0.05) for stereotyped behavior. 

The following summary scores were available for each rater from their completion of the BPI:
•	Total frequency – the total score from the ratings of all 29 items on a scale from 0-6 (possible range: 0-174)
•	Self-injurious behavior frequency – the total score from the ratings of the 15 self-injurious behavior items on a scale from 0-6 (possible range: 0-90)
•	Aggressive/destructive behavior frequency – the total score from the ratings of the 9 aggressive/destructive behavior items on a scale from 0-6 (possible range: 0-54)
•	Stereotyped behavior frequency – the total score from the ratings of the 5 stereotyped behavior items on a scale from 0-6 (possible range: 0-30)

These scores were calculated for each BPI completed. For each participant, four of each score were available as each participant was rated by four respondents on the BPI. A table was constructed for each of the above scores displaying the scores of each of the 4 respondents for the 22 participants. The data in these tables were manipulated to create, for each score, four groups: the highest scores (Rank 1), the second highest scores (Rank 2), the third highest scores (Rank 3) and the lowest scores (Rank 4). Table 1 shows the existence of wide differences across staff in their ratings of the frequency of all categories of problem behavior. On average the highest rating was more than 2.6 times as great as the lowest rating of the same participant’s behavior (Rank 1:Rank 4 ratios: SIB, 2.6; Aggression, 1.9; Stereotypy, 3.4). Variation between raters was greatest for stereotyped behavior and least (though still substantial) for aggressive/destructive behavior. The amount of variation was, however, significantly correlated with BPI scores, that is there was more variation between raters the higher their mean score. Accordingly, relative deviations (after Repp, Karsh, Deitz, & Singh, 1992) (i.e., (a-b)/b)) from the mean (b) of the deviation (a) from the mean for each score were calculated. Relative deviations of SIB, aggression and stereotypy scores were compared by one-way analysis of variance and found to be significantly different (F=6.01, df=2,261, p0.01). Post-hoc comparison showed that deviations of stereotyped behavior frequency scores were significantly greater than those for aggressive behavior (Mean difference=0.23, df=174, p0.01). That is, variability amongst scores was related to topography with stereotyped behavior scores more variable than aggressive behavior scores.
____________________
Insert Table 1 about here
____________________

There was also considerable variation across participants in the variability of their BPI scores. SIB scores had a mean relative deviation, across participants, of 56% (range: 0-90%), aggressive behavior scores had a mean relative deviation of 72% (range: 10-97%), stereotyped behavior scores had a mean relative deviation of 56% (range: 0-87%). Despite these scores allowing, by their calculation, for differences in BPI frequency scores, they were significantly correlated with the latter – for SIB, r = 0.55 (df=20, p0.05), for aggressive/destructive behavior, r = 0.65 (df=20, p0.01), and for stereotyped behavior, r = 0.66 (df=20, p0.01). SIB, aggressive and stereotyped behavior mean relative deviation scores were not significantly correlated across participants. That is, the amount of reported variability in the behavior of a particular participant appeared to depend on the behavior rather than the participant and to be greater the more problem behavior they were reported to display.

Staff also reported considerable daily variations in the daily rate of the specific target behavior selected as the most frequent displayed by each participant. Staff reported a mean minimum of 2.67 (range: 0-17.33) and a mean maximum of 13.80 (range: 2.67-52.67). Minimum and maximum rates were not significantly correlated. On average, therefore, the maximum daily rate reported was more than five times the minimum daily rate. There were considerable variations in the minimum and maximum daily rates reported by different staff working with the same individual. Average minimum rates (rank-ordered by rater) were 3.4, 1.8 and 1.2, that is the minimum daily rates of target behaviors varied by a factor of 2.8 across raters. Average maximum rates were 26.55, 9.85 and 6.20, that is the maximum daily rates of target behaviors varied by a factor of 4.3 across raters. The highest maximum rate reported was, on average, more than 22 times the lowest minimum rate suggesting that, for these frequent target behaviors, most of which occurred at least daily, the reported rate depending on the day and the reporter could vary by a factor of, on average, more than 22. 

Managers’ scores on the BPI were compared with mean staff scores (see Table 2) using paired samples t tests. Managers generally rated the frequency of problem behavior lower, with the exception of aggressive/destructive behavior. Manager-staff differences were statistically significant for the frequency of stereotyped behavior. For the nine participants where their keyworker was one of the raters, keyworker scores were compared with other staff (excluding manager) scores. No significant differences or trends towards significance were found. For the 17 participants where at least one rater (excluding managers) was male and one female, male and female scores were compared. Female scores were higher for aggression and lower for self-injury and stereotypy but none of the differences were statistically significant. 
____________________
Insert Table 2 about here
____________________

Discussion

Conventionally, differences between staff in their reports of challenging behavior have been treated as a matter of reliability. The central proposition of this paper has been that such differences may reflect valid (rather than unreliable) differences in staff experience. The main findings of the study can be summarized as follows. Staff reports of the occurrence and frequency of challenging behaviors displayed by people with developmental disabilities were highly correlated but widely different in absolute level. That is, some staff reported much more frequent challenging behavior than others. This was particularly the case for stereotyped behavior. Although no significant differences were found between the reports of male and female staff, or keyworkers, managers were found to report generally less challenging (especially stereotyped) behavior than their staff. Variation in staff reports was mirrored in considerable variation across participants in the amount of variability reported. Such variability did not seem, however, to be a consistent characteristic of the participant but to vary by the type and frequency of their challenging behavior. In particular, more variability was reported for higher frequency challenging behavior. Staff also reported considerable day-to-day variability (from “minimum” to “maximum”) in the frequency of challenging behavior. Combining such reports across staff suggested that the frequent challenging behaviors reported might vary by a factor of 22 across time and staff.

The central issue in the interpretation of this study’s findings is whether the results reflect a lack of reliability in staff reports of challenging behavior or real variability in the behavior of people with developmental disabilities. In reaching a view on this issue a number of factors should be considered. First, the test-retest reliability data reported would normally be regarded as acceptable and are consistent with data previously reported for the BPI. Second, despite this, the data suffer inevitably from the general weaknesses of all self-report data. Data, for example, may be biased by respondents typically working with participants for different lengths of time, or in different situations, or at different times of day. More objective evidence (e.g., from direct observation) of patterns of naturally occurring variability is clearly required. Third, the results found are consistent with previous literature in identifying differences in the reported occurrence of challenging behavior across staff who have different functions. It is no surprise that managers, who would usually have less interaction with participants (Burgio, Engel, Hawkins, McCormick, & Scheve, 1990), should report generally less frequent challenging behavior. That this trend should vary somewhat for aggressive/destructive behavior is also not surprising given that such behavior is more likely to provoke comment from staff and, indeed, be “heard” by the manager even if out of sight. Fourth, other of the results found are open to ready interpretation. Stereotyped behavior, reported to be more variable than aggressive behavior, was also relatively more frequent (frequency score per topography of 4.2 vs. 2.9). Behaviors reported to be more frequent were found to be more variable so that the finding in respect of stereotyped behavior may be related to its frequency. More frequent behaviors are likely to be more variable for a number of reasons. The behavior that occurs, say 12 times a day, will inevitably occur in a greater variety of contexts than the behavior that occurs, say twice a day. The opportunities for variation in the contexts evoking the more frequent behavior are more frequent both because there are likely to be a greater number of contextual variables involved and because low frequency behavior is more likely to be tied to specific (invariant) contexts. High frequency behavior also creates more opportunities for shaping by the effects that it produces on the social and physical environment. These effects may result in short-term increases (reinforcement) or decreases (extinction or punishment) in the frequency of the behavior. 

These kinds of explanations, of course, beg the question of, if the variability is real, what produces it? As noted in the introduction to this paper, variability in challenging behavior has typically been explained as a result of variations in the occurrence of setting events, antecedent events that evoke challenging behavior, the application of reinforcement contingencies, and biological factors. Biological factors seem unlikely to be involved in variability between staff (except in the unusual situation where staff work at different times of the day) though they may well contribute to the day-to-day variability in the presence of the same staff that is also reported. Variability across staff, therefore, seems likely to be related to differences in the patterns of interactions between different staff and the same participant that have functional significance in setting the occasion for, establishing motivation for, signaling the availability of reinforcement for, and reinforcing, challenging behavior. To firmly establish the validity of the variability reported here, relationships with such events will clearly have to be investigated.

As Sidman put it, “only by exposing and tracking down major sources of variability can true generality be achieved” (Sidman, 1960, p.160). The general application of research on challenging behavior to the everyday lives of people with developmental disabilities has, say some (e.g., Holburn, 1997), not been as extensive as it might have been. It is suggested here that one reason for this may be the lack of systematic attention given to major sources of variability in challenging behavior, such as those represented in the differential occurrence of such behavior across staff. While the events proximally responsible (antecedents, reinforcers etc) for such variation are unlikely to be greatly different to those already identified, elucidation and control of the naturally occurring circumstances in which they arise may result in a more ecologically valid, and effective, approach to the prevention and management of challenging behavior.

References

Ager, A. (1991). Effecting sustainable change in client behaviour: The role of the behavioural analysis of service environments. In B. Remington (Ed.) The challenge of severe mental handicap: A behaviour analytic approach (pp. 343-368). Chichester, England: Wiley.
Burgio, L. D., Engel, B. T., Hawkins, A., McCormick, K., & Scheve, A. (1990). A descriptive analysis of nursing staff behaviors in a teaching nursing home: Differences among NAs, LPNs, and RNs. The Gerontologist, 30, 107-112. 
Carr, E. G. (1994). Emerging themes in the functional analysis of problem behavior. Journal of Applied Behavior Analysis, 27, 393-399. 
Carr, E. G., Reeve, C. E., & Magito-McLaughlin, D. (1996). Contextual influences on problem behavior in people with developmental disabilities. In L. K. Koegel, R. L. Koegel, & G. Dunlap (Eds.), Positive behavioral support: Including people with difficult behavior in the community (pp. 403-423). Baltimore: Paul H. Brookes.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46. 
Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213-220. 
Dura, J. R., Mulick, J. A., & Rasnake, L. K. (1987). Prevalence of stereotypy among institutionalized nonambulatory profoundly mentally retarded people. American Journal of Mental Deficiency, 91, 548-549. 
Durand, V. M., & Crimmins, D. B. (1988). Identifying the variables maintaining self-injurious behavior. Journal of Autism and Developmental Disorders, 18, 99-117. 
Emerson, E., McGill, P., & Mansell, J. (Ed.). (1994).  Severe learning disabilities and challenging behaviours: Designing high quality services. London:   Chapman & Hall.
Emerson, E., Thompson, S., Reeves, D., Henderson, D., & Robertson, J. (1995). Descriptive analysis of multiple response topographies of challenging behavior across two settings. Research in Developmental Disabilities, 16, 301-329. 
Gardner, W. I., Cole, C. L., Davidson, D. P., & Karan, O. C. (1986). Reducing aggression in individuals with developmental disabilities: An expanded stimulus control, assessment, and intervention model. Education and Training in Mental Retardation, 21, 3-12. 
Gilbert, T. F. (1978). Human competence: Engineering worthy performance. New York: McGraw-Hill.  
Graham, P., & Rutter, M. (1970). The selection of children with psychiatric disorder. In M. Rutter, J. Tizard, & K. Whitmore (Eds.), Education, health and behaviour (pp. 147-177). London: Longman.
Hastings, R., & Remington, B. (1993). "Is there anything on...why 'good' behavioural programmes fail?"  A brief review. Clinical Psychology Forum, 9-11. 
Holburn, S. (1997). A renaissance in residential behavior analysis? A historical perspective and a better way to help people with challenging behavior. The Behavior Analyst, 20, 61-85. 
Holmes, N., Shah, A., & Wing, L. (1982). The Disability Assessment Schedule: A brief screening device for use with the mentally retarded. Psychological Medicine, 12, 879-890. 
Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., & Richman, G. S. (1994a). Toward a functional analysis of self-injury. Journal of Applied Behavior Analysis, 27, 197-209. (Reprinted from Analysis and Intervention in Developmental Disabilities, 2, 3-20, 1982). 
Iwata, B. A., Pace, G. M., Dorsey, M. F., Zarcone, J. R., Vollmer, T. R., Smith, R. G., Rodgers, T. A., Lerman, D. C., Shore, B. A., Mazaleski, J. L., Goh, H. L., Cowdery, G. E., Kalsher, M. J., McCosh, K. C., & Willis, K. D. (1994b). The functions of self-injurious behavior: An experimental-epidemiological analysis. Journal of Applied Behavior Analysis, 27, 215-240. 
Matson, J. L., Bamburg, J. W., Cherry, K. E., & Paclawskyj, T. R. (1999). A validity study on the Questions About Behavioral Function (QABF) scale: Predicting treatment success for self-injury, aggression, and stereotypies. Research in Developmental Disabilities, 20, 163-175. 
McGill, P. (1999). Establishing operations: Implications for the assessment, treatment and prevention of problem behavior. Journal of Applied Behavior Analysis, 32, 393-418. 
McGill, P., Teer, K., Rye, L., & Hughes, D. (submitted for publication). Staff reports of setting events associated with challenging behaviour. 
O'Neill, R. E. (1997). Functional assessment and program development for problem behavior: A practical handbook (2nd ed.). Pacific Groves, CA: Brooks/Cole.  
Repp, A. C., Karsh, K. G., Deitz, D. E. D., & Singh, N. N. (1992). A study of the homeostatic level of stereotypic and other motor movements of persons with mental handicaps. Journal of Intellectual Disability Research, 36, 61-75. 
Rojahn, J. (1984). Self-injurious behavior in institutionalized, severely/profoundly retarded adults - prevalence data and staff agreement. Journal of Behavioral Assessment, 6, 13-27. 
Rojahn, J. (1986). Self-injurious and stereotypic behavior of noninstitutionalized mentally retarded people: Prevalence and classification. American Journal of Mental Deficiency, 91, 268-276. 
Rojahn, J. (1992, updated January 1994). Behavior Problems Inventory - A Prospectus. Nisonger Center for Mental Retardation and Developmental Disabilities, The Ohio State University.  
Rojahn, J., Polster, L. M., Mulick, J. A., & Wisneiwski, J. J. (1989). Reliability of the Behavior Problems Inventory. Journal of the Multihandicapped Person, 2, 283-293. 
Sidman, M. (1960). Tactics of Scientific Research. New York: Basic Books.  
Smith, R. G., & Iwata, B. A. (1997). Antecedent influences on behavior disorders. Journal of Applied Behavior Analysis, 30, 343-375. 
Stokes, T. F., & Baer, D. M. (1977). An implicit technology of generalization. Journal of Applied Behavior Analysis, 16, 349-369. 
Sturmey, P., Fink, C., & Sevin, J. A. (1993). The Behavior Problem Inventory: A replication and extention of its psychometric properties. Journal of Developmental and Physical Disabilities, 5, 327-336. 
Sturmey, P., Sevin, J. A., & Williams, D. E. (1995). The Behavior Problem Inventory: a further replication of its factor structure. Journal of Intellectual Disability Research, 39, 353-356. 
Touchette, P. E., MacDonald, R. F., & Langer, S. N. (1985). A scatter plot for identifying stimulus control of problem behavior. Journal of Applied Behavior Analysis, 18, 343-351. 
Wing, L., & Gould, J. (1978). Systematic recording of behaviour and skills of retarded and psychotic children. Journal of Autism and Childhood Schizophrenia, 8, 79-97. 

Table 1 
Rank ordered total frequency scores on the BPI

	Rank 1 mean 	Rank 2 mean	Rank 3 mean	Rank 4 mean	Overall mean	

All items	
43.36	
34.27	
28.00	
21.59	
31.81	
Self-injurious behavior	17.50	12.73	9.55	6.64	11.60	
Aggressive/destructive behavior	17.14	13.09	11.14	8.91	12.57	
Stereotyped behavior	11.91	8.68	6.45	3.50	7.64	

Table 2 
Manager and staff total frequency scores on the BPI

	Manager mean 	Overall staff mean	t value (managers vs. overall staff)	

All items	
28.41	
32.94	
-1.77	
Self-injurious behavior	9.45	12.32	-1.94	
Aggressive/destructive behavior	12.77	12.50	0.28	
Stereotyped behavior	6.18	8.12	-2.09*	
*p0.05, df=21