Measuring Success
In Evaluating Your Program – What Do You Need to Know?, Elizabeth Miller, M.D., Ph.D., Assistant Professor in Pediatrics, U.C. Davis School of Medicine provides the following guidance:
- Determine your key areas of interest
The evaluation design for an RHY/DV provider partnership will vary depending on the key outcomes of interest. An evaluation may want to focus initially on service providers’ (including street outreach workers) knowledge, attitudes, and behaviors, e.g., How have the RHY program and DV agency’s providers changed in their level of knowledge of abusive behaviors among RHY, their level of confidence in assessing for partner violence, and their experiences with handling positive disclosures?
- Pay attention to risk and other factors such as literacy
Another strategy for basic data collection that is minimal risk to youth is to conduct an anonymous needs assessment survey. On-line survey software can facilitate such data collection, where youth utilizing an RHY program can be asked to complete an anonymous survey on-line after they have completed their intake where there is absolutely no way for providers to know who entered what information….Another key consideration is the low literacy levels of RHY: some researchers have argued for doing face to face interviews to address low literacy and comprehension barriers, while others have utilized computer-assisted audio technology (ACASI).
To assess whether the incorporation of a DV curriculum into RHY programming has had measurable impact in changing youth knowledge, attitudes, and behaviors, another strategy to protect the anonymity of subjects is to have each participant create a secret code which only they will know that they enter on both pre and post tests, which allows for the survey information to be linked (whether there was individual level change) without anyone being able to discern the identity of any participant.
- Under the definitions of “Mature Minors” and consent
Depending on the level of confidential services provided in the context of an RHY program, the participant may be receiving services (related to pregnancy, sexual health, substance abuse) as ‘mature minors.’ In this instance, ‘mature minors’ may be able to consent for their own participation in a study, if obtaining parental consent would breach confidentiality and potentially result in the minor refusing all care. This concept of ‘mature minor’ varies state to state, and different research ethics boards are likely to interpret the definition of a ‘mature minor’ differently.
- When more extensive data is sought
More intensive data collection (including in-depth interviews) may require adult caregiver consent (especially if they are wards of the state), and providers are advised to consult with their local research ethics board (which may be a local academic medical center) prior to collecting data that is in addition to the basic information collected for service provision.
