A Preliminary Investigation of a Tool to Measure BCBA Supervisory Behaviors

Introduction

As the demand for Board Certified Behavior Analysts (BCBAs) continues to rise, so does the need for competent supervision in the field. Effective supervision is critical for shaping skilled behavior analysts and ensuring ethical and evidence-based practices. However, many newly certified BCBAs struggle to provide high-quality supervision due to a lack of structured guidance.

While guidelines exist, there is limited empirical research on how to assess and improve supervisory behaviors systematically. This gap has led researchers Alyssa R. McElroy, Jessica E. Van Stratton, and Heather M. McGee (2024) to investigate a tool that may help measure BCBA supervisory behaviors: the Operant Supervisory Taxonomy and Index (OSTI).

This blog post will explore the importance of effective BCBA supervision, introduce the OSTI, and summarize the key findings from their research.

The Need for Effective BCBA Supervision

BCBAs play a vital role in designing and overseeing behavior-analytic programs, but quality supervision is necessary to maintain industry standards. Poor supervision can lead to:

1. Ethical violations and ineffective service delivery
2. Inconsistent training of new analysts
3. Increased stress and burnout among supervisees

Research shows that well-structured supervision improves professional competence and adherence to ethical guidelines. However, measuring supervisory effectiveness remains a challenge. Data-driven approaches, like the OSTI, offer a way to assess and refine supervisory behaviors.

What is the Operant Supervisory Taxonomy and Index (OSTI)?

The OSTI, originally developed by Komaki (1986, 1998), is a framework designed to categorize and assess supervisory behaviors. It has been applied across various professional settings, including:

• Medical insurance firms
• Police departments
• Construction sites
• Theaters
• Competitive sports teams

This study represents the first attempt to apply the OSTI in BCBA supervision. By categorizing different types of supervisory behaviors, the OSTI provides a structured way to analyze and improve supervision quality.

Categorizing BCBA Supervisory Behaviors with the OSTI

The OSTI breaks down supervision into eight key behavior categories:

1. Performance Consequences – Providing feedback and reinforcement to supervisees
2. Performance Monitors – Observing and monitoring supervisee performance
3. Performance Antecedents – Offering instructions, prompts, and expectations
4. Own Performance – Supervisors reviewing their performance and self-reflecting
5. Not Communicating – Listening without actively engaging in discussion
6. Work-Related – Discussing relevant supervision-related topics
7. Not Work-Related – Engaging in rapport-building or personal conversation
8. Solitary – Preparing materials or working independently without direct interaction with supervisees

By categorizing these behaviors, the OSTI helps supervisors identify where they are spending their time and how to improve their impact.

Applying the OSTI to BCBA Supervision: Research Findings

Study Methodology

The study by McElroy et al. (2024) aimed to observe and categorize behaviors of BCBA supervisors to assess the feasibility of using the OSTI in behavior analysis supervision. The methodology included:

1. Participants – Masters-level students in a behavior analysis program
2. Observation Settings – Virtual supervision meetings
3. Data Collection – Partial-interval recording of supervisory behaviors
4. Behavioral Coding – Assigning observed behaviors to OSTI categories
5. Interobserver Agreement (IOA) – Ensuring data reliability through multiple independent observers

Observations were conducted over multiple sessions to capture both structured and informal supervisory interactions.

Key Research Findings

1. Performance Monitoring and Consequences Were Common

   • Supervisors frequently provided feedback and reinforcement (Performance Consequences).
   • They also spent significant time observing supervisee performance (Performance Monitors).

2. Performance Antecedents Were Less Frequent

   • While some supervisors provided clear instructions and expectations, this behavior was observed less often than monitoring and feedback.

3. Time Spent on Work-Related vs. Non-Work-Related Behavior Varied

• Some supervisors engaged in significant non-work discussions, which may impact productive supervision time.

4. Supervisors Rarely Engaged in Self-Reflection (Own Performance)
   • Few supervisors reviewed their performance or discussed improvements for their own supervisory skills.

These findings highlight areas where BCBAs can improve their supervision, such as increasing structured instruction and self-reflection while balancing work-related conversations.

Implications for BCBA Supervisors

The study’s findings suggest several key takeaways for behavior analysts who provide supervision:

• Enhance Active Supervision – Prioritize direct feedback and structured guidance to help supervisees develop competence.
• Maximize Work-Related Discussions – Minimize non-work interactions to ensure that supervision time is as productive as possible.
• Engage in Self-Reflection – Supervisors should periodically assess their own styles and identify areas for improvement.

Applying the OSTI may help supervisors quantify and refine their behaviors, contributing to better training experiences for new behavior analysts.

Conclusion

Quality supervision supports the long-term success of behavior analysts and ensures effective service delivery. The study by McElroy et al. (2024) provides a first step in systematically measuring BCBA supervision using the OSTI. Their findings suggest that data-driven approaches to supervision can highlight areas for improvement and create stronger, more competent professionals in the field.

To read the full study, visit: https://doi.org/10.1007/s40617-023-00849-2.

By incorporating structured supervision methods, BCBAs can contribute to a field that is both ethical and effective, shaping the future of behavior analysis. What steps can you take to improve your own supervisory practices?

AI Research Assistant

See Full Bio