Court observation is not the ONLY way to gather hearing quality data. Again, this depends largely on the questions that you have and the resources that you have to gather data. Here are some other ideas for gathering data on the quality of court hearings.

Focus groups. Focus groups involve pulling together small groups of individuals for a discussion. Focus groups are a good way to understand perspectives of professionals or families involved in the system related to the quality of hearings. Focus groups are moderately resource intensive, depending on how many you hold. A couple of ways that focus groups might be most helpful.

• Identifying issues for further exploration. A focus group can identify what people think works well in hearings as well as where people think there are opportunities for improvement. From this, tools can be developed to better understand the extent of an issue (e.g., how often something is occurring in practice). This focus group could come before other types of data collection.
• Determining why an issue exists. Focus groups could also be useful after other types of data collection to supplement their findings. For example, a focus group could be used after court observation to better understand why practice looked like it did. If judges are not making appropriate findings on the record, the focus group could talk with judges to identify what the barriers are to making the right findings.

Surveys. Surveys are similar to focus groups. They are used to ask individuals their perspective on a topic. The advantage of surveys is that they can reach a large number of people with very minimal resources, particularly if using a web-based platform for delivery instead of paper and pencil. Surveys can be used with either professionals or families to learn more. Here are some thoughts about how surveys might be most effective with each group.

• Surveying Professionals. Professionals can be asked open ended questions about strengths and challenges of holding a high-quality hearing (similar to focus groups). Surveys could also ask individuals to respond to scale items. For example, professionals might be asked how often a specific practice (e.g., discussing child well-being at hearings) occurs and told to use a rating scale of 0=Never to 4= Always/Almost Always).
• Surveying Families. Surveys of parents and youth can be used to explore the perspective of the families in response to the court hearing. Most commonly, post court surveys have asked parents about their experiences in court. They have been asked to rate their agreement on a scale to statements such as “I was treated with respect,” “I felt part of the decision-making process,” or “I understood what happened in court today.” Parents can provide a unique perspective on hearing quality as they are able to identify what the hearing looked like to them and how they understood it.

Case File Review. Every child welfare case has a case file that includes important court information such as the petition on the case, orders from each hearing, and reports to the court. A structured review of this information can provide important information about the quality of hearings. Case file review is a resource intensive process, depending on the number of cases to review and the amount of information you want to gather. Case file review could be particularly helpful in gathering data on the following constructs:

• Presence of parties with representation. Parties are typically documented for each hearing, making it easy to identify how frequently each party (e.g., parents, attorneys) attends the hearings across the life of the case.
• Competent legal representation. Case files often include an appointment date for attorneys so it is possible to determine if attorneys are appointed prior the first hearing on the case. From the case file, you can also gather motion practice of attorneys.
• Appropriate and detailed findings on the record. Case files typically include the orders and findings generated from each hearing. From there, you can capture which findings are being made at each hearing as well as how detailed the findings might be.

Self-Assessment. A final way to gather hearing quality data might be through a self-assessment process. This process might require very few resources. A self-assessment process could include legal professionals coming together to evaluate their own performance on a given set of hearing quality constructs. The New York Child Welfare Court Improvement Program's Toolkit includes an example self-assessment tool. Professionals could gather and answer these questions together, coming to agreement on how well they are doing on different dimensions (e.g., parties present, breadth and depth of discussion).

New York Child Welfare Court Improvement Program released a Toolkit for Assessing Quality of Permanency Hearings. The toolkit is publicly available for download at the link provided.