RUNNING HEAD: EVALUATION PLAN
Evaluation Plan:
“Power through Choices Life Skills Training” program, is a self- efficacy based intervention to empower adolescent girls living in foster care and independent living programs to make the right choices involving risky sex. Participants who receive this curriculum will increase their sexuality expectations, attitudes, perceptions and self-esteem toward sexual responsibility. Participants will gain the use of practice making healthy sex choices related to various lifestyles and adhering to those choices. The girls who take this curriculum will develop more positive social sexuality expectations (i.e. social pressures to have sex,), develop more positive self sexuality to have sex, (i.e. personal attitude about having sex and increase personal receptiveness, (i.e., likelihood of diseased transmitted and consequences of sexual activity. The prevalence of sexual activity and the frequency of contraceptives used among early adolescent girls will also be examined, (Graves, Senter, Workman & Mackey, 2011, p. 463).
The purpose of this study is to investigate the effectiveness of a comprehensive sex education program among females in out of home placement. This program is tailored to meet their needs concerning safer sex. A safer sex intervention stresses condom use and concerned health issues relating to risky sex. Safer sex interventions can reduce HIV, STD’s, and may be especially effective with sexually experience adolescents that may have longer lasting effects. The program design calls for adopting a group of teen girls in foster care and then sticking with them for one year. This will be implemented by program providers including group home counselors to identify the sequence of changes among participants that would occur as a result of the program’s activity. This design is a randomized control trail with 3, 6 and 12 month follow ups.
The mission and goals of this intervention is to provide youths in out of home care with specific skills and information to help them avoid risky sexual behavior and reduce the incidence of adolescent pregnancy, HIV and other STD’s. The goals of the curriculum focuses on recognizing and making choices related to sexual behavior, finding and using local resources and developing effective communication skills. Through role plays and other interacting activities participants will increase their awareness practicing reproductive health choices. It emphasizes the importance of building skills related to effective contraceptive use, risk-reduction techniques and provides numerous opportunities for practicing these skills, (Becker& Barth, 2000, p 277).
Target Population:
The populations of interest in this study are adolescent girls in state care between the ages of 14 to 19 year’s old residing in independent living homes throughout Baltimore City. Four focus groups comprised of youth in residential care were conducted across Baltimore City. The independent living homes include Bert’s Too, Challengers, Juneau and Bethel. The means age of the enrollees was 16.8 years old.
Youth in state custody were recruited by word of mouth, through case workers,
E-mail and flyers. The sample for this study was selected from a total population of 85 girls. The population was 45% African Americans, 25% Caucasians and 15% Hispanics. Sixty youth were randomly selected (using the table of random numbers) and randomly assigned to two groups of 30 each. Participants were informed of confidentiality and given a number unconnected to their name. Those who were selected to be in the experimental group met once per week for 10 weeks during 90 minutes sessions. The control group did not receive any treatment but received the initial questionnaire relating to risky sexual behaviors.
Participant’s views of the program:
During the course of the program youths were concerned about teen pregnancy and supported a comprehensive approach to sex education. They wanted to discuss sexual feelings as well as personal relationships. More than 20% of the youth reported an unplanned pregnancy, contacting an STD or being sexually active and never or rarely using protection. The characteristics of these teens include an intense need for affection, the absence of a supportive family and a desire to possess something of their own.
An initial needs assessment was used in the first session of the curriculum’s pilot program so that introductions, questions, concerns and competencies of the youths could be addressed. The youths were regarded as the “experts” and their feedback resulted in direct changes to the curriculum. They provided important suggestions throughout the process of the intervention. Moreover, they were encouraged to discuss the needs of youth in out of home care. The two major themes included issues relating to self improvement and the impact of choices on an individual’s future through skill building activities in each of the 10 – 90 minute sessions, (Becker& Barth, 2001).
During the second session participants were introduced to some of the characters in the curriculum. The activity was entitled, “Designing My Saturday Night”. After receiving a photograph associated with the character of Frederic, participants learned that he is a 17 year old boy who wants to become a teacher and is a senior in high school. Frederic has had sex with four different girlfriends, has always used a condom, was sexually abused and currently lives in a group home. Participants also learn that Frederic smokes marijuana, and drinks sometimes, goes to the clinic for an HIV test every six months and has always tested negative, but has never been tested for STI’s. The facilitator than describes Frederic relationship with his new girlfriend Tanya and asks the youths to act as the experts in helping Frederic and Tanya to ‘design a safer Saturday night.” The facilitator asks the youths to outline the steps that Frederic and Tanya need to take to ensure a safer Saturday night, emphasizing the importance of planning for safer sex. This activity focuses on the advantages and the disadvantages of various methods of contraceptives as well as the importance of considering personal lifestyle habits when choosing contraceptive methods.
Use of the curriculum’s characters affords participants the opportunity to act as the experts in assisting the characters in the decision making process relating to sexual activity and sexual relationships. Participants separate into small working groups. Each group is given the opportunity to act as the expert. Each group has a chance to share their views around planning safer sex and using contraceptive methods. During the course of the program residents gather more promptly at the start of the session. Group dynamics also changed with increase verbal engagement and self disclosure by withdrawn teens and decreased verbal confrontation and disruptive behavior by outspoken ones. Difficulty in terminating the session also increased with time, (Becker & Barth, 2001, pp. 276-278).
Staff:
Program evaluation staff and local providers collaborated through monthly meetings to promote the program design. The local providers also participated in training sessions. This increased their understanding and skills related to the program evaluation methods. A major weakness of the program involved local providers who were resistant to program evaluation. For example, they did not understand the purpose and value of evaluation. Program staff perceived evaluation as criticism or judgment of their skills and performance rather than a technique to improve their program. In addition, entry into these systems for the purpose of implementing the curriculum was quite challenging. Attempts to train the local providers to develop and implement their own evaluations were unsuccessful because of limited funding which requires staff to implement the components of the evaluation themselves. This was perceived by staff as extra busy work, (Taylor, Powell, Rossing, & Geran, 1998).
Strategies for overcoming the obstacles included scheduled meetings with providers at group homes and independent living homes to request permission to implement the curriculum. Scheduled meetings were held with the local providers and staff to determine the most effective manner in which the material could be included in their existing program.
Instruments:
All participants were measured at the end of the 10 week program, including a 3 month and one year follow up. At the end of the 10 week sessions, a self report measure was developed using a participatory evaluation approach with a focal interest on the curriculum’s impact related to attitudes, perceptions, and communication about sexual responsibility. The measures included a 25-item Likert-type scale with responses related on a 5-point scale (1=strongly agree through 5= strongly disagree), (Graves, Sentner, Workman, & Mackey, 2011).
The 3 month follow up included a customer survey questionnaire which measured the effects of participation on items related to safe sex, personal relationships, self- esteem, setting goals for the future and increasing decision making skills. Each participant received a stipend of $35.00 after returning the second questionnaire and $25.00 in returning the third questionnaire. The results demonstrated that what they learned will enhance their commitment and ability to practice safer sex. Post-test results demonstrated that the youth understood the curriculum’s message and that abstinence is the only 100% safe method of protection from pregnancy, STD’s, HIV and that “safer sex takes planning”. The youths completed pre-and post-test. At the end of the year, a 24-item Self Assessment Questionnaire will be mailed to each participant to elicit feedback concerning their views of the program and to measure the differences in responses across time. (Graves, Senter, Workman & Mackey, 2011).
Mythology:
The sample size estimated is two groups and an alpha level of .85. A total sample size of 85 participants is recommended and indicates that our sample size is adequate. Participants were randomly assigned to an experimental group (n=45) verses the control group, (n= 40).
Demographic information, including age, race, ethnicity, and family structure was also assessed. Independent samples t test and chi squares difference analysis included that there were not a significance difference in the age of the participants, (Graves, Senter, Workman & Mackey, 2011).
Summative Evaluation:
Program staff worked with the four agencies to develop site specific projects relating to the logic model. A chart was implemented in each of the four independent living homes specifying the short term goals. Program providers including group home counselors were asked to identify the sequence of changes among participants that would occur as a result of the program’s activity. In addition, this increased the youth’s concepts toward achieving the goals, ( Graves, Sentner &, Mackey; 2011).A sample of the logic model is shown in tables 1 and 2.
Short Term Goals/Logic Model:
1. Improve the ability to identify and create strategies to deal with risky situations. Reduce risk taking behaviors.
2. Increase skills related to communication techniques through assistance, refusal techniques, goal setting and decision making by setting boundaries and limits. Establish future aspirations toward achieving goals for the future.
3. Increase knowledge of healthy relationships. Establish and maintain healthy relation ships.
Creating a logic model for each participating site was fundamental in directing the development of the evaluation plan and procedures. Program administrators and staff recognized the benefits of using the logic model as marketing tool to a variety of audiences in the community including stakeholders’. Meetings were held bimonthly with program providers to implement the logical model. Net working with providers and stakeholders during bimonthly meeting will allow the program to seek future funding and may increase funding for additional prevention programs in the community.
The logical model/ long term goals:
1. To collaborate with stakeholders who are willing to sponsor the program. The logic model was used to assist the program in seeking funding and future funding with potential community partners and stakeholders.
2. Evaluate and assess whether the program is effective as intended to the target population. The logic model was used to identify common curricula markers that were directly related to the outcomes of interest, youth’s knowledge, attitudes intentions and behaviors that could be measured by evaluation instruments.
3. To examine the social needs and problems of the participants. Program providers were asked to identify the sequence of changes among participants that would occur as a result of the program’s activity.
During the final stages of testing 45 youths participated in the evaluation process. The youths completed a pre-and post-test (38% matched the pre- and post-test); the decreased number of youths attributed to both attrition and late program entry. In addition, classes were observed to learn teaching styles and student participation.
The youth surveyed stated that what they learned will enhance their commitment and ability to practice safer sex. Post-test results demonstrated that the youth understood the curriculum’s message that “safer sex takes planning”. The youths also reported that following completion of the program, they felt increased control over their lives and were less likely to engage in unprotected sex than at pretest. Satisfaction survey results showed that 43% of the youths surveyed thought it would be easier to practice safer sex after participating in the program and 42% of the youths surveyed rated the overall program as being very good or excellent, (Fluhr, Oman, Allen, Lanphier & Mc Leroy, 2004).
To measure the effectiveness of pregnancy prevention and safe sex programs, all the evaluations reviewed here used a true fidelity experimental design in which the behavior of participating adolescents (the treatment group) was compared with the behavior of similar adolescents not exposed to the intervention (the control group).
For example, the evaluations of reducing risky sexual behaviors and choosing safe sex both used designs in which the experimental and the control group were randomly assigned to received either the program curriculum or no treatment. The evaluation then compared the behavior of youth attending the program during the pretest and posttest. Follow ups were limited to 1 year but results with outcomes such as practicing safe sex, condom use, birth control and reduced number of partners were consistently positive as demonstrated on the post-test, (Becker & Barth, 2001).
Logic Model
Name: Paula Goodman
Title: Evaluation Plan___________________________________________
| | |Short Term Goals | |
| | |(Objectives/ |Long Term Goals |
|Antecedents |Inputs |Proximal Outcomes) |(Distal Outcomes) |
| | | | |
|Lack of social support. |To provide an educational |Increase knowledge of Healthy |Collaborate with stakeholders who are |
|High rates of sexual risk |growth group for teen age girl |relationships – Human growth & development|willing to sponsor the program. |
|taking behaviors. |residing in independent living |consequences of early sexual involvement. |Evaluate and assess whether the program|
|Less likely to have access to |homes. | |is effective as intended in relations |
|sex education and family |To provide youth in out of home|Improve ability to identify and create |to the target population. |
|planning. |care with specific skills and |strategies to deal with risky situations. |To examine the social needs and |
| |information to help them to |Increase skills related to |problems of participants. |
| |avoid high risk sexual behavior|communication-Techniques “Assistance” & | |
| |and reduce the risk of an |refusal techniques goal setting & decision| |
| |unintended pregnancy. |making setting boundaries & limits. | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
References
Becker, M.G; & Barth, R.P,. (2000).
“Power through Choices: The Development of a
Sexuality Education Curriculum for Youth in Out of Home Care”; Child Welfare
League of America
Fluhr, J.D; Oman, F.R; Allan, R.A; Lanphier, M.G; & McLeory.K.R; (2004),
“A Collaborative Approach to Program Evaluation of Community Based Teen
Pregnancy Prevention Projects”; , retrieved on
May 4, 2011
Graves, K.N.; Sentner, A.; Mackey.W.; (2011), “ Building Positive Life Skills the Smart
Girls Way”; Health Promotion Practice; http://hpp.Sagepub.com, Retrieved on
May 1, 2011.