Oklahoma Parents as Teachers annual program Evaluation |
Previous | 1 of 3 | Next |
|
small (250x250 max)
medium (500x500 max)
Large
Extra Large
large ( > 500x500)
Full Resolution
|
This page
All
|
E 1800.3 P228a 2008/09 e.l Oklahoma Parents As Teachers Annual Program Evaluation December 17,2009 Oklahoma Technical Assistance Center 123 E. Broadway Cushing, OK 74023 www.otac.info DlAC alternate paths to learning Table of Contents Executive Summary . Program Description 1 Program and Participation Data 2 Figure 1. Geographical distribution of OPAT programs 3 Table 1. Program funding levels, requirements, and costs 5 Program Evaluation Strategies 5 Participants 7 Table 2. RaciallEthnic distribution of children, OPAT participants, and general student population 7 Figure 2. Number of risk factors 9 OPAT Services 9 Personal Visits 10 Table 3. Total number of personal visits completed per month 11 Parent Group Meetings 12 Developmental and Health Screening Services 12 Outcomes 14 Analysis 1 15 Analysis 2 16 Table 4. Pre-post ASQ rating of children who participated in OPAT, 2008-09 17 OPAT Staff Surveys 18 Parent Educator Survey 18 Program Coordinator Survey 22 Survey Summary 26 Table 5. Survey items by year 27 Goal Achievement: Individual Programs 28 Table 6. Goal achievement data by program 29 Appendix Table AI. Program reporting OSDE recommendation letter Parent Knowledge Questionnaire OPAT Coordinator Survey OPAT Parent Educator Survey Program database Oklahoma Parents as Teachers Program Evaluation Executive Summary 2008-09 Program Information and Grantees The Oklahoma Parents As Teachers (OPAT) grant program funded 79 grants to programs serving families of young children in 94 school districts. Total state funding for OPAT programs ($2,045,709) was unchanged compared to previous years. The size of the award was dependent on the size (Average Daily Membership) of the applicant. In addition to funds allocated to school districts, two public housing authorities in Oklahoma City and Tulsa each received $65,695. OPAT parent educators completed a total of33,775 personal visits with 4,388 families and 5,027 children during the 2008-09 funding year. The average program cost per family was $439; average program cost per child was $384. Table 1. Program funding levels, requirements, and costs. School District Grant Number Original Optional Median Cost Median Cost (ADM) Amount Awarded Minimum Minimum Per Child Per Child Monthly Visits Monthly Visits 2007-08 2008-09 500 $13,500 31 25 22 $365 $422 1,000 $21,000 34 40 36 $365 $385 3,000 $35,000 7 65 58 $323 $402 10,000 $42,500 1 78 70 $278 $332 10,000 $48,500 3 90 81 $404 $357 18,000 $63,000 1 120 108 $290 $274 30,000 $84,000 2 160 144 $321 $374 Characteristics of Families and Children Served • The ethnic and racial composition of families served was fairly similar to the Oklahoma population of school children as a whole. • Twenty-nine percent of participating families enrolled in the program before the child was born. • Twenty-two percent of participants were teen parents (under the age of20 at time of delivery). • More than half the participants (51.2%) were classified as "low income or unemployed." • A disability was documented for 3.2 percent of children. • Children born prematurely or classified as "low birth weight" babies accounted for 9.3 percent of participants. Almost a quarter of children (22.1%) had mothers who did not complete high school. OklahomTaechnicAalssistancCeenter Page i • In 15.0 percent of OPAT families, a language other than English was spoken in the home. • Programs reported that 9.6 percent of families had additional risk factors. • More than half of the participants had at least one at-risk factor (61.7%) and almost a quarter of the participants (22.4%) had three or more risk factors. Developmental and Health Screening OPA T grant programs are required to provide health screening services to all of the children enrolled. According to data submitted by the grantees, 91.1 percent of children received health screening. The percentage of OPA T children age 19-35 months who have received appropriate immunizations continued to increase. The average in 2005-06 was 87.7 compared to 90.8 percent in 2006-07, 91.3 percent in 2007-08 and 93.9 percent in 2008-09. OPAT's efforts are reflected in the increase in immunization rates across Oklahoma for all children. In 2005, Oklahoma ranked 44th in the nation in child immunization rates compared to 15th in 2009. OPA T programs provided developmental screening through the use of the Ages and Stages Questionnaires (ASQ). Parent Educators administered the ASQ to children who were older than four months of age and within their first two months of participation in OPAT; and again in each year of participation. If the results of the ASQ indicated developmental delays, the child was referred to appropriate public and private services. This year, 84.9 percent of the participants were screened by 0 PAT and another 11.9 percent were excused. Only 3.2 percent of the children served through OPA T should have received screening but did not. The most commonly reported reason for not screening a child was because the family had moved from the district. Outcomes The ASQ was administered in order to identify children who are at-risk. The assessment covered five key developmental areas: communication, gross motor, fine motor, problem solving, and personal-social skills. Pre-post ASQ scores were used to determine the extent to which children who participated in the OPAT program, who were initially identified as at-risk, moved to the not at-risk group. Two separate statistical analyses showed significant, positive changes in total scores and children's at-risk classification after participation in OPAT. Children were more likely to be classified as at risk on their pre-program ASQ than on their latest ASQ. Children enrolled in OPA T showed a reliable gain in cognitive, language, social, and motor development. Oklahoma Technical Assistance Center Page ii Oklahoma Parents As Teachers (OPAT) Annual Program Evaluation School Year 2008-09 Submitted to Sandy Garrett, State Superintendent of Public Instruction Oklahoma State Department of Education Prepared by the Oklahoma Technical Assistance Center Program Description The mission of the Oklahoma Parents As Teachers (OPAT) is to provide the information, support, and encouragement that families need to help their children develop optimally during the crucial, early years of life. This voluntary program provided services to families with children from prenatal to 36 months old and was based on the philosophy that parents are their children's first and most important teachers. The OPAT program is designed to enhance the development of young children, enabling them to learn, grow, and develop to realize their full potential. OPAT is affiliated with the nationally validated Parents As Teachers (PAT) program that originated in 1981 as a pilot project in Missouri. The pilot program was based on research showing that greater family involvement in children's learning is critical to school success. Based upon the pilot program's success, the program was validated by the United States Department of Education's Program Effectiveness Panel. Since that time, PAT has been listed as an exemplary program by four other national organizations. The Centers for Disease Control and Prevention recognized PAT as a noteworthy prevention program and effective strategy for preventing child maltreatment. The Strengthening America's Families Project, in collaboration with the Center for Substance Abuse Prevention and the Office of Juvenile Justice and Delinquency Prevention, designated PAT as a model family-strengthening program. The national center has continued to conduct Oklahoma Technical Assistance Center 2009 Page 1 research, develop curriculum, and expand the program. There are now more than 3,000 Parents As Teachers programs covering all 50 states, United States territories, and seven additional countries ranging from Canada to China. The goal of the Oklahoma Parents As Teachers program is to strengthen the capacity of parents to be effective first teachers. Parents As Teachers offers a complete, research-based curriculum entitled Born to Learn. OPAT is also designed to foster an early partnership between home and school, enabling parents to take a more active role during their children's formal years of schooling. The program is designed to maximize a child's overall development during the first three years of life, lay the foundation for school success, and minimize developmental problems that interfere with the child's learning. Other intended benefits of the program include early identification of developmental delays or sensory problems (e.g., hearing, vision) and a reduction in the number of children who require remedial and special education programs. Program and Participation Data The number of OPAT programs increased from 76 last year to 81 this year. This includes two public housing authorities (Oklahoma City and Tulsa) who also received funding. Eleven grants were awarded to school districts that partnered with at least one other neighboring district. Eight of the multi-district grantees consisted of two partner districts, two consisted of three partner districts and the award to Avant also included the districts of Bowring, Osage Hills, and South Coffeyville. In total, 94 Oklahoma communities were served by OPAT programs. The majority of the participating districts were located in rural areas or small towns (Figure 1). Although programs were located across the state (with the exception of the panhandle), more were located in the eastern half where the greater proportion of the population resides. Oklahoma Technical Assistance Center 2009 Page 2 Figure 1. Geographical distribution ofOPAT programs 2008-09 **OPA.T Procrarrs Public HoJsing Prcgram * * * * : * * ** * *** * * * * * * * * ** * *~***>i **!* 1 * ** * * * * * * * * * * * The amount of funding appropriated for OPAT programs by the Oklahoma Legislature in 2008-09 ($2,045,709) remained unchanged from the previous fiscal year. The Oklahoma State Department of Education let a total of$I,927,814 in grant funds. (The funds awarded to school districts totaled $1,796,500 and the public housing authorities in Oklahoma City and Tulsa each received $65,657.) Additionally, Oklahoma Technical Assistance Center received $16,334 to provide program evaluation services. The remaining funds ($101,561) were used by the Oklahoma State Department of Education to provide technical assistance to grantees. School districts applied for grant funds based on their community size. This was determined by the Average Daily Membership (ADM) of the applicant district. Table 1 lists the number of grants by community size. The table also lists program requirements and the median cost per child, disaggregated by funding levels. Depending on their ADM, school districts received grant awards ranging from $13,500 to $84,000. In 2008-09, programs had the option to reduce the number of monthly visits by ten percent in order to compensate for the increase in transportation costs the projects experienced. The revised monthly minimum visits as well as the original minimums are presented in Table 1. More than three fourths of the grants (82.2%) were awarded to small school districts with an ADM of 1,000 or less. The state's two largest urban areas, Oklahoma City and Tulsa, received the largest awards. Grantees with ADM between 10,000 and 18,000 were located in suburban communities. In addition to the funding levels designated in law, the State Department of Education added an additional funding level to accommodate the needs of the Midwest City - Del City school district. The target was set at 70 rather than 78 monthly visits for the Midwest City - Del City program in exchange for a grant award of $42,500. This modification was made when the district experienced difficulties in meeting its monthly visit requirements. Oklahoma Technical Assistance Center 2009 Page 4 This year the programs reported that they served 4,388 families with their OPAT funds. Additionally, three of the grantees, Caney Valley-Copan (7) Dewey (6), and Enid (130) reported that they served families with other funding sources such as district funds or community resources. In total, OPAT programs reported that they served 5,027 children during the 2008-09 $439. Table 1 reports the median cost per child for 2007-08 and 2008-09. The cost per child program year. This resulted in a program cost per child of$384 and an average family cost of increased substantially in each of the community sizes this year. This was primarily the result of the ten percent reduction in the number of visits required for programs. The costs ranged from $274 to $422 per child. Table 1. Program funding levels, requirements, and costs. School District Grant Number Original Optional Median Cost Median Cost (ADM) Amount Awarded Minimum Minimum Per Child Per Child Monthly Visits Monthly Visits 2007-08 2008-09 500 $13,500 31 25 22 $365 $422 1,000 $21,000 34 40 36 $365 $385 3,000 $35,000 7 65 58 $323 $402 10,000 $42,500 1 78 70 $278 $332 10,000 $48,500 3 90 81 $404 $357 18,000 $63,000 1 120 108 $290 $274 30,000 $84,000 2 160 144 $321 $374 Program Evaluation Strategies Since 1991, the Oklahoma Technical Assistance Center (OTAC) has conducted a yearly evaluation of the OPAT programs. This independent evaluation of the OPAT program has included a number of components, relying on both quantitative and qualitative information. In Oklahoma Technical Assistance Center 2009 Page 5 each of these annual evaluations, the OPAT program has demonstrated its effectiveness in meeting the overall program goals. The primary focus of the program evaluation has remained consistent over the years; although, the content and methods used in the program evaluation have been revised each year in order to address specific areas of interest or concern. The evaluation relies on quantifiable data. These include monthly program service reports, pre- and post-assessments of children's development, surveys of service providers, and biannual assessments of changes in parent knowledge attributed to the program. A set of instruments for collecting relevant data from OPAT programs was developed in consultation with State Department of Education Early Childhood staff and OPAT program coordinators. The set of instruments includes the following: • an electronic project service reportingform, completed each month by local OPAT staff, used to record the number and type of project services provided to each participating family, demographic information, and pre-post assessment data; • a Parent Knowledge Questionnaire, designed by the PAT National Center, for evaluating the degree of change in parent knowledge of child development and child safety. This instrument is administered every even-numbered year; • the Ages and Stages Questionnaires, a commercially-produced standardized instrument for assessing child development in comparison with normative standards; and • two OPAT staff surveys, one for Parent Educators and one for Program Coordinators, designed to collect information on program services, common practices, exemplary practices, program costs, and future needs. Oklahoma Technical Assistance Center 2009 Page 6 One program, Quinton, failed to provide monthly data on the children and families Table AI, located in the Appendix, lists each of the funded programs in 2008-09 and the data that was submitted by that program. served throughout the year. This program was not included in the evaluation process. Participants OPAT' s emphasis on early intervention was apparent in the large number of parents who were enrolled in the program prior to the birth of their child. More than a quarter of this year's participants (29.2%) were enrolled in the program before the child was born, most between the second and seventh months of pregnancy. Additionally, 73.4 percent ofOPAT children were comparison was that, when given the option, a substantial proportion of OPAT parents preferred enrolled before their first birthday. Table 2 presents the racial and ethnic distribution of OPAT participants and Oklahoma school children. Because of the form in which the data was reported, it was difficult to compare the OPAT distribution to the state distribution. The most interesting finding from this to categorize their families as multiracial. Table 2. Racial/Ethnic distribution of children, Oklahoma Parents As Teachers participants and general student population. Group OPAT Children All Oklahoma School Children African-American/Black 5.8% 11% American Indian/Alaskan Native 12.4% 19% Asian/Pacific Islander 0.8% 2% CaucasianlWhite 58.7% 58% Hispanic/Latino 10.1% 10% Multiracial 11.0% (Not an SDE category) Not Reported 1.1% -- Oklahoma Technical Assistance Center 2009 Page 7 According to the authorizing legislation, OPAT programs were required to place a high priority on serving children considered to be at risk. The "at-risk" characteristics of 2008-09 participants are summarized as follows: • Teen parents accounted for 21.8 percent of those served. • More than half the participants (51.2%) were classified as "low income or unemployed." • In 15.0 percent of OPAT families, a language other than English was spoken in the home. • A substantial proportion of children (3.2%) had a documented disability. • Children who were born prematurely or were classified as "low birth weight" babies accounted for 9.3 percent of all participants. • A great deal of research has noted that the educational attainment ofa child's mother is one of the best predictors of the educational achievement of the child. A study by the RAND Corporation (Lara-Cinisomo, et aI., 2004) reported that "...the two factors associated most strongly with school readiness are (l) the educational attainment of mothers and (2) neighborhood poverty." The data indicated that 22.1 percent of the children served had mothers who did not complete high school. Note that this does not include the teen mothers who were still in high school. • Programs reported that 9.6 percent of families had additional risk factors - these lower-incidence factors defined by the National PAT Center were combined into one category to ease the reporting burden on the grantees. Oklahoma Technical Assistance Center 2009 Page 8 Figure 2. Number of Risk Factors As Figure 2 presents, 61.7 percent of the children had at least one of these risk factors. Almost a quarter of the participants (22.4%) had three or more factors. As in previous years, children of mothers who did not graduate high school were most likely to have multiple risk factors, 97.7 percent had at least two factors and 79 percent had three or more factors. 2,000 OPAT,200S'()9 1,SOD ••.•. c: •a•. 'u "f 1,000 ~ Q••. SOD ~ ~ is.3s%1 0 .00 1.00 2.00 3.00 4.00 5.00 6.00 Factors OPAT Services OPAT staff provided three direct services as part of the Parents As Teachers model; personal visits, parent group meetings, and developmental and health screenings. Parent educators visited families in a location preferred by the parent, usually the home, once each month. During this meeting, the parent educator presented age-appropriate child development and parenting information, helped parents learn to observe their children, and addressed parents' concerns. At least one meeting was devoted to developmental screening. Parents were also Oklahoma Technical Assistance Center 2009 Page 9 invited to attend group meetings where they shared information about child development and parenting. Personal Visits The minimum number of visits the program staff were expected to complete each month was determined by the size of the award. The number of required personal visits ranged from 22 per month (for programs that received $13,500) to 144 visits (for those that received $84,000). To be eligible for continued funding, OPAT programs must meet their minimum monthly requirements within an evaluation paradigm that sets out which visits are "countable." For example, programs may count only one visit per month per child. In the case of high-risk children, local OPAT staff may decide to conduct extra visits with the parent and child, as recommended by the PAT National Center. These visits are accounted for in the database, but are not included in the determination of whether the grantees met their monthly minimum requirements. Table 3 presents the number of visits completed in each month of the program year. The number of visits that OPAT completed was fairly stable throughout the year. The two lowest figures were in the start-up month (August) and the last month of the program (June). The number of visits completed in August and June were the result of the policy allowing programs to select their program beginning and ending dates based on community calendars. All programs were encouraged to operate for ten months of the year; if a program began its program year in the first week of August, it could conclude its program year at the end of May. Many programs began their year in congruence with school district enrollment in mid-August and operated their programs until mid-June. Additionally, new programs started later, due to the fact that they had to hire parent educators and arrange for training and certification before permitting them to work Oklahoma Technical Assistance Center 2009 Page 10 as OPAT parent educators. In total, OPAT staff conducted a total of 33,775 personal visits during the 2008-09 school year. Table 3. Total number of personal visits completed per month, all programs. Aug Sep Oct Nav Dec Jan Feb Mar Apr May Jun Total 2,316 3,275 3,406 3,409 3,274 3,345 3,484 3,441 3,445 3,449 931 33,775 In addition to personal visits usually conducted in the home, the parent educators also conducted consultations with teen parents. These sessions were usually held in the teen's school. Consultations with parenting teens followed the same format as personal visits but were conducted with the parent only - circumstances prevented a full visit with both parent and child. Teen consulting visits were often the best way to ensure that the teen parent consistently participated in the Born to Learn curriculum. This year, 1,158 consulting visits were held with teen parents. The average length of enrollment in the program for a family in 2008-09 was 7.21 months. More than a third of all participants (39.3%) were enrolled for the entire program year (ten months or more). Factors that impacted the length of enrollment in the program included: • OPAT has an open entry/open exit enrollment policy. • Participation in the program was voluntary. • Parents could enroll their child at any point during the year and could leave the program at any time. • According to the authorizing legislation, children may be served until they reach 36 months of age. Once they reach this age, they may no longer be enrolled in OPAT and OPAT funds may not be used for continued services. Oklahoma Technical Assistance Center 2009 Page 11 The average number of personal visits for all children, regardless of number of months enrolled, was 6.72. The mean number of visits for those children in the program seven or more months was 8.61 suggesting that the program staff focused on completing the monthly visits. Parent Group Meetings The programs relied on schools, early childhood centers, community centers, churches, libraries, and parks as locations for parent group meetings. The group meetings gave parents an opportunity to learn from and support each other, observe their children with other children, and practice new skills. All of the programs conducted parent group meetings in this year. The programs held a total of 1,402 meetings and averaged 17.31 meetings per program, exceeding the program goal of one meeting per month. The program hosted by the Tulsa School district provided the most parent meeting opportunities (101) and the EI Reno program conducted the fewest (3). Attendance at meetings ranged widely, depending on the community size. As a group, the mean number of parents in attendance was 8.07 regardless of program size. Developmental and Health Screening Services The Ages and Stages Questionnaires (ASQ) was used by the OPAT programs to provide developmental screening. Training in the appropriate use of this instrument was provided by the Oklahoma State Department of Education at the beginning of the program year. Qualified parent educators were responsible for its administration. Children older than four months of age were to be screened within the first two months of their participation in OPAT. (The ASQ cannot be administered until children are at least four months of age.) Children who continue in the program must be re-screened each year. Children who exhibit developmental delays were referred to appropriate public and private service providers such as Sooner Start, county health departments, or pediatricians. Oklahoma Technical Assistance Center 2009 Page 12 The goal of the program is to screen 100 percent of eligible children each year. As noted above, children younger than four months of age cannot be screened. Some children who receive annual developmental assessments from other agencies (e.g., Sooner Start) do not need to be screened by OPAT. Some participants leave the program before their children can be screened. The degree to which each OPAT grantee met screening goals is an important factor in the SDE's decision as to whether or not to continue to fund the program. Program-by-program statistics on the number of children screened, excused, and unscreened may be found in the last section of the report, Goal Achievement. Overall, OPAT programs appeared to have met their goal of screening all children. This year, 84.9 percent of the participants were screened by OPAT and another 11.9 percent were excused for reasons cited above (e.g., age, screened by another agency). Only 3.2 percent of the children served through OPAT should have received screening but did not. Most programs reported the reasons that children were not screened. In many cases, the family moved from the districts just prior to the screening process. In addition to the developmental screenings, OPAT programs are also required to provide health screening services to all of the children enrolled. According to data submitted by the grantees, 91.1 percent of children received health screening. Program-by-program statistics are included in the Goal Achievement section. According to OPAT reports, the percentage of OPAT children age 19-35 months who have received appropriate immunizations continued to increase. The average in 2005-06 was 87.7 compared to 90.8 percent in 2006-07, 91.3 percent in 2007-08 and 93.9 percent in 2008-09. OPAT's efforts are reflected in the increase in immunization rates across Oklahoma for all children. In 2005, Oklahoma ranked 44thin the nation in child immunization rates compared to Oklahoma Technical Assistance Center 2009 Page 13 __ 1 Determining the effectiveness of the OPAT program on the cognitive development of 15th in 2009 (Oklahoma Department of Health, Disease and Prevention Services. Oklahoma Immunization Update, 2009). Outcomes young children is a principal goal of the program evaluation. As noted previously, outcomes for children are evaluated through annual administration of the Ages & Stages Questionnaires (Bricker & Squires, 1999), a low-cost method for reliably assessing children for developmental delays. The emphasis of the instrument is on detecting delays and potential delays; thus, it does not assess the full range of cognitive ability. ASQ pre-test and post-test scores were available for 2,433 children, permitting statistical analysis of changes in participants' developmental risk levels. Children are tested only once per year and many children had not yet completed the three-year program. Their pre-tests and post-tests were less than three years apart. Therefore, the results should be considered as an Within two months of entry into the OPAT program, a pre-program ASQ is administered underestimate of the true effects of the program. to each child. 1 In subsequent years, an age-appropriate form of the ASQ is administered annually. The average number of months between the initial ASQ and the most recent ASQ was 14.81 months - approximately 40 percent of the length of the OPAT program. These results, then, are a conservative estimate of the effects of the program. Two sets of statistical analyses were conducted: (1) an analysis of the ASQ scale scores and (2) an analysis to determine whether there was a change in the proportion of children who scored in the "at risk" range on the ASQ. The first analysis was conducted principally to help lThe ASQ cannot be administered until the child is four months old. Parents who enter OPAT before the birth of their child may not have the "pre-test" administered until their child is four months old. Oklahoma Technical Assistance Center 2009 Page 14 Analysis 1. An initial set of t-tests was conducted on the ASQ scale scores. The ASQ understand the properties of ASQ scores over time. The second analysis was the more important one - the analysis designed to answer the research question: "Is participation in OPAT related to changes in the at-risk levels of children?" The results of all statistical tests indicated that the OPAT program resulted in reduced risk levels for participating children. scale scores are derived scores that theoretically remain fairly constant unless a child's normative standing changes - that is, unless there are changes in a child's developmental progress when compared to other children. We would, therefore, expect no change unless OPAT participants progressed at afaster rate than the norm group.' This set of tests was exploratory only, as the ASQ scores lack sufficient variance for powerful statistical analyses due to their limited scale. The instrument was designed to detect developmental delays and does not test the limits of children's development. In other words, most children score near the top and there is limited room for them to improve. For this preliminary look at changes in ASQ scores, the subtest scores were combined into a "total score" and a t-test was conducted.' The mean initial score (pre) was 52.96; the 9.225,p = .000], indicating a reliable positive change in ASQ scores for OPAT participants. An mean second (post) score was 54.17. The difference was statistically significant [t(2420) = effect size of .19 standard deviations was calculated on these scaled scores - the typical OPAT participant's second score gained one-fifth of a standard deviations between pre-test and post-test. An effect size of one-third of a standard deviation (.33) is generally accepted as a 2Although these scores range from 5 to 70, the scoring procedure always results in a multiple of five, resulting in a 14-point rather than a 70-point scale. 3An omnibus significance test is much preferred over the analysis described here. The nature of the ASQ and its scores, however, mitigated against a multivariate analysis with at least five dependent variables - there is simply not enough variance to partition. The t-test analysis was conducted for exploratory purposes only, to facilitate an understanding of the amount of change in ASQ scale scores. Oklahoma Technical Assistance Center 2009 Page 15 substantial educational intervention; an effect size such as the one observed in this analysis (.19) is regarded as important. Approximately half of the children tested were enrolled in the program for 12 months or more. The effect size (.25) was greater for these children. It should be emphasized that the ASQ does not test the full range of cognitive ability; because of the limited range of scores, we would not expect a large effect size. Statistically significant differences in children's first and most recent ASQ scores were noted on the Communication, Gross Motor, Fine Motor, and Personal Social subtests. As in previous years, only the Problem-Solving subtest did not produce positive gains. The average change in scores on these four subtests ranged from 1.15 scale points to 2.11. Analysis 2. The purpose of this analysis was to determine if participation in OPAT reduced the child's risk of school failure. This procedure was designed as a part of a longitudinal study that was a part of the original evaluation plan for OPAT.4 The study was designed so that, once OPAT participants were school age, the principal analysis would determine whether children who initially scored in the At Risk range on their pre-program ASQ demonstrated school achievement levels within the normal range. The analysis provided a preliminary look at OPAT's effects on reducing risk. Statistical tests were conducted to determine whether the children showed a reduction in the number of subtest scale scores in the At Risk range. For the purposes of this analysis, a child was classified as "at risk" if he or she had at least one subtest score in the At Risk range. A two-way chi square test was conducted on children's risk status on the initial measure and the most recent measure. Four situations were possible: 4Because of program attrition and lack of funding, the longitudinal study was abandoned. Oklahoma Technical Assistance Center 2009 Page 16 • the child was classified At Risk on both the initial measure (pre) and on the most recent measure (post), • the child was At Risk on the pre-ASQ but not on the post (positive change), • the child was Not At Risk on the pre, but was on the post (negative change), and • the child was Not At Risk on either administration of the ASQ. It is important to note that children face more at-risk factors as they age. A number of indicators of developmental delays cannot be assessed until children are old enough to begin creeping, crawling, walking, and talking. Because of this factor, we always expect some number of children to initially score as Not At Risk and later score in the At Risk range. The analysis included 2,409 children. Table 4 shows the number of children who were in each of these categories: • 61 children were classified At Risk on both the pre-ASQ and the post-ASQ, • 209 children were At Risk on the pre-ASQ but not on the post (positive change), • 128 children were Not At Risk on the pre-ASQ but were At Risk on the post (negative change), and • 2,011 were Not At Risk on either administration of the ASQ. Table 4. Pre-post ASQ rating of children who participated in OPAT, 2008-09. Most Recent ASQ Not At Risk At Risk Initial Not At Risk 2,011 128 ASQ At Risk 209 61 Because ASQ scores are not perfectly accurate, there is always some error in the instrument's classification of children as "at risk." To determine whether the number of children who changed categories was statistically significant, a chi square analysis was conducted. The Oklahoma Technical Assistance Center 2009 Page 17 results indicated that children were more likely to be classified as At Risk on the initial ASQ than on their most recent ASQ (X2 = 91.47,p =.000). On the initial ASQ, 11.21 percent of children had at least one At Risk score. On the most recent ASQ, only 7.85 percent scored in the At Risk range on any subtest. Of children with at least 12 months between pre-test and post-test, 7.37 percent scored in the At Risk range. Of children who had at least 24 months between pre-and post-test, 4.87 percent scored in the At Risk range. The longer children participated in the program, the less likely they were to be classified as At Risk. OPAT Staff Surveys The Parent Educator Survey and the Program Coordinator Survey were designed to provide more insight into the roles of the local OPAT staff. Both instruments were administered at the OPAT Spring Conference to all that attended. For those programs that did not have program representatives at the spring conference, surveys were mailed directly to the program staff. Parent Educator Survey The Parent Educator Survey consisted of 12 questions. The first 11 questions asked parent educators to choose from a list of provided responses. The last question asked parent educators to describe their best practices. Of the 164 parent educators who were reported to have served in the programs this year, 117 (71%) completed the Parent Educator Survey this year. The 117 respondents represented 69 programs. The following section lists participant responses to each of the survey questions. Oklahoma Technical Assistance Center 2009 Page 18 1. How many hours per week do you work as a parent educator? Response Percent of Respondents 1 to 10 hours 12.8% 11 to 20 hours 38.5% 21 to 30 hours 30.8% 31 to 40 hours 14.5% More than 40 hours 3.4% 2. How many hours a week are worked outside the typical work day (9-5, Monday - Friday)? Response Percent of Respondents 1 to 10 hours 86.7% 11 to 20 hours 11.5% 21 to 30 hours 1.8% 31 to 40 hours 0.0% More than 40 hours 0;0% 3. In addition to being a parent educator, are you a certified teacher? Response Percent of Respondents Yes 44.3% No 55.7% 4. If yes, are you currently employed as a full-time teacher (in addition to your OPAT work)? Response Percent of Respondents Yes 20.5% No 79.5% Oklahoma Technical Assistance Center 2009 Page 19 5. How many college courses have you taken in Early Childhood? Response Percent of Respondents None 14.2% 1-2 9.7% 3-5 19.5% 6-8 11.5% 9 or more 45.1% 6. What is the level of education you have completed? Response Percent of Respondents Child Development Associate (this is not a degree, but requires 120 clock hours of training; respondents frequently write it in, so it was included as a response) 6.0% Associate's Degree (A.A.) 14.7% BA or B.S. 53.4% MA or M.S. 19.8% High School 6.0% 7. If you have a college degree, in what field? Response Percent of Respondents Early Childhood 21.3% Elementary Education 31.9% Childhood DevelopmenUFamily Relations 8.5% Social Work 9.6% Other 28.7% Oklahoma Technical Assistance Center 2009 Page 20 Response Percent of Respondents Less than 12 5.1% 12-24 2.6% 25-48 0.9% 49-60 12.8% More than 60 3.4% No response 75.2% 8. If you do not have a college degree, how many college credit hours have you completed? 9. How many years of service do you have in OPAT? Response Percent of Respondents 0-5 years 47.0% 6-10 years 23.1% 11-15 years 15.4% more than 15 years 14.5% 10. How many personal visits are you expected to complete each month? Response Percent of Respondents 1-15 17.9% 16-25 35.9% 26-40 37.6% 41-50 6.8% more than 50 1.7% Oklahoma Technical Assistance Center 2009 Page 21 Response Percent of Respondents 1-15 19.7% 16-25 33.3% 26-40 37.6% 41-50 7.7% more than 50 1.7% 11. How many personal visits do you usually complete each month? The final question on the Parent Educator Survey asked respondents to describe the best practices of their programs. Forty-one parent educators (35%) completed this question. Many of the parent educators gave specific details that described the relationship between the program and partners including descriptions of the goods and services provided by local resources. The parent educators cited the benefits of partnering with the school district and often noted the efforts of the school administration, counselor, alternative school teachers, and community education staff. Many also listed specific programs or materials they used that interested and motivated parent participation. Programs that had been in existence for several years reported that program parents were the best source of program recruitment. One parent educator wrote, Thefamilies that receive our services make our program successful! Word of mouth really increases our enrollment possibilities. Program Coordinator Survey The Program Coordinator Survey was completed by 62 of the 81 program coordinators. The survey consisted of 11 questions, the first 10 questions were forced-choice questions and the eleventh question asked coordinators, What resources or training would benefit you most? The following section lists participant responses to each of the survey questions. Oklahoma Technical Assistance Center 2009 Page 22 1. Where is your program office or headquarters housed? Response Percent of Respondents Elementary School 41.3% Early Childhood Center 22.2% County Health Department 1.6% Non-profit agency 1.6% Special Services building 7.9% Other 25.4% 2. Where do you hold the majority of your parent meetings? Response Percent of Respondents Elementary School 45.9% Early Childhood Center 24.6% County Health Department 4.9% Non-profit agency 1.6% Special Services building 3.3% Other 19.7% 3. What steps did you take to recruit at-risk families? Mark all that apply. Response Percent of Respondents Collaborate with Sooner Start, Dept. of Human Services, and 88.9% Health Dept. Contact alternative education schools for teen parents 76.2% Distribute brochures at public places 84.1% Work with Title I schools or public housing apartments 52.4% OS/GYNs, pediatricians, and hospitals 39.7% Local press coverage or internet 44.4% Word of mouth 93.7% Other 25.4% Oklahoma Technical Assistance Center 2009 Page 23 Response Percent of Respondents Collaborate with Sooner Start, Dept. of Human Services, and 25.8% Health Dept. Contact alternative schools for teen parents 11.3% Distribute brochures at public places 6.5% Work with Title I schools or public housing apartments 14.5% OB/GYNs, pediatricians, and hospitals 1.6% Local press coverage or internet 0.0% Word of mouth 35.5% Other 4.8% 4. Of the steps marked in item three, which one is the primary recruitment method for at-risk families? Mark only one. 5. How much are your parent educators paid per hour? Response Percent of Respondents Less than $9.00 0.0% $9.01 - $12.00 26.2% $12.01 - $15.00 42.6% $15.01 - $18.00 16.4% $18.01 or other 14.8% 6. How many families were typically on waiting list for services this year? Response Percent of Respondents Our program did not have a waiting list 63.9% 5 or fewer families 24.6% 6-10 families 3.3% more than 10 families 8.2% Oklahoma Technical Assistance Center 2009 Page 24 Response Percent of Respondents Yes 10.0% No 90.0% 7. Did your program receive any additional funds this year? 8. If so, how many children were served with the additional funds? Response Percent of Respondents Less than 20 4.8% 20 to 35 3.2% 36 to 50 0.0% 51 to 65 0.0% more than 65 1.6% No response 90.5% 9. Indicate the outside agencies or organizations that your program collaborates with. Mark all that apply. Response Percent of Respondents Head Start 56.6% Sooner Start 79.4% County Health Department 65.1% Dept. of Human Services 41.3% Public Libraries 58.7% Health Clinics 38.1% Civic Organizations 39.7% Other 41.3% Oklahoma Technical Assistance Center 2009 Page 25 10. Choose which auxiliary services you offer. Mark all that apply. Response Percent of Respondents Drop in and Play 52.4% Toy/book lending library 82.5% Newsletter 79.4% Reading/Story time 36.5% Clothing/supply closet 38.1% Other 26.6% 11. What resources or training would benefit you most? Sixteen of the program coordinators responded to this question. Representative responses included: • I think all Parent Educators should have training in car seat safety. • Trainingfor families with special needs children. • How to support a parent who has lost a child. • Spanish-speaking families - how to interpret and work with them. • Resources for funding to provide books for children and supplies for projects. • Always group meeting activities ... how to encourage participation. Survey Summary Each year, evaluation staff review the instruments and data collection procedures and make appropriate revisions to improve the quality of the information collected and to address specific program issues and concerns. These yearly revisions do not allow for a complete, multi-year comparison of survey results; however, those that remained consistent are presented in Table 5. Oklahoma Technical Assistance Center 2009 Page 26 Table 5. Survey items by year. Question 2004·05 2005·06 2006·07 2007·08 2008·09 Parent Educators - Degree status of B.A.lB.S. or more? 76.4% 71.1% 67.5% 71.3% 79.2% Parent Educators - Six or more years of service in PAT? 42.4% 40.9% 45.8% 51.5% 53.0% Parent Educators - Work more than 20 hours per week? 46.2% 46.3% 44.2% 55.3% 48.7% Program - Waiting list? 42.9% 40.4% 41.4% 52.2% 36.1% The results from the parent educator and coordinator surveys suggested that OPAT programs have stable, well-educated staff in the field. More than half of the parent educators surveyed (53.0%) indicated that they had been working in an OPAT program for six or more years and more than three-fourths have at least a B.A.IB.S. degree. The data in Table 5 also indicated that approximately half of the parent educators are working more than half-time. The percentage of parent educators working more than 20 hours a week decreased from 55.3 percent last year to 48.7 percent this year. A substantial change was noted in the percentage of programs that had waiting lists. Only 36.1 percent of the project coordinators reported they had families on a waiting list, the lowest percentage in the last five years. Over a third of the parent educators indicated that they made between 26-40 personal visits per month (37.6%). Parent educators that made between 16-25 visits accounted for 33.3 percent of the group. Most of the parent educators indicated that they met the expectation set forth by the program requirements. The OSDE set the goal for projects to meet their number of minimum monthly visits at least eight months out of the program year. Factors such as late start up and winter weather sometimes caused programs difficulty in meeting the minimum each month. This year, 65.5 percent of the programs either met or exceeded this goal. Almost all of the programs indicated that they made efforts to recruit at-risk families such as contacting local agencies, physicians, and alternative schools. Word of mouth was the Oklahoma Technical Assistance Center 2009 Page 27 most frequently reported recruiting method for programs (35.5%) followed by collaboration with Sooner Start, Department of Human Services, and the Health Department. Elementary schools continued to be the most popular place to house the programs (41.3%) and to hold parent meetings (45.9%). Goal Achievement: Individual Programs Table 6 presents program information including whether each program met goals for personal visits and developmental screening. The table lists the number of visits each program was required to complete each month and the number of months each program met that standard. A column is also included that lists the total number of teen consultations (teen consultations are not included in the number of personal visits). Programs that served a substantial number of teens may not show that they attained their monthly minimum until teen consultations are taken into account. Information on developmental screening is presented on the right side of Table 6. It shows the number of children screened during the 2008-09 school year, the number of children "excused" from the screening requirement because they were too young (less than four months of age) or had not received at least two visits, the number of children who should have been screened but were not. Finally, the percentage of eligible children who were screened is listed. Oklahoma Technical Assistance Center 2009 Page 28 Table 6. 'Goal achievemenfd~ia by program."":)!;"" " ': ,:,~;;,,~ '!if ~~; ,,;,,% ~~ !,'\ii",,' !'",%l '!!;1~'!" ",'I' li!i t~, DistrictN,a~e ",',' ,,~ , Ij;,'tfof .. ,* :Total #;of" H, " ,,>' '," ~~f:~"' R,i 'i' Total #';/ 1*';Monthly . ,~#'of " ,~::lof>" percentage ~~;"'''f "i' , Months'~;~ i\0,~~' " Minimu'm'j ',,! I,een 'ir ,;,_Child~en,J;;,Ii" ~~~ O',\lli; Ii";' , 9hildren" Children~ ~;Screenedi !f~~;;: #Children' j!~,¥isits ~~, Minimum,'!' Consultation :i~~r~,~n~d;, '.":"riot ,,'Excused ~ 1:1:: ,',\\i,,~, 1S~rVe~~: ~!S1t~l~'~~~ ,iJ,;'ii, ," ,~; ",!; ;;S,~!ee-ried, from ,; i,: ',,"",;,!! i:'~~~~;~i:~~~~Jii;jg'", =:», ,,(,it S¢reening jill'ill "(i', =~:'l1 ,; " ",' 1(, ,l! ,,'f' @' f,y!! ·'.,f,!':!'; -"ii . 'f " '" , -" '" ' , Ada 44 36 8 0 43 0 1 100% Altus 48 36 2 1 26 13 9 73% Anadarko 67 36 10 0 61 2 4 97% Ardmore 85 58 8 0 80 1 4 99% Avant, Bowring, Osage Hills, South 44 22 9 0 39 0 5 100% Coffeyville Bartlesville 122 58 10 69 94 8 20 93% Bethany 57 36 10 0 53 0 4 100% Binger-Oney, Gracemont 34 22 10 0 32 0 2 100% Bixby' 80 58 5 0 67 2 11 98% Bristow 40 36 0 0 30 0 10 100% Broken Arrow 136 81 9 4 130 0 6 100% Caney Valley, Copan 60 36 10 0 52 0 8 100% Chouteau-Mazie 59 36 10 0 56 0 3 100% Claremore 57 36 8 0 48 3 6 95% Coalgate 15 22 0 0 11 4 0 73% Comanche 37 22 10 0 26 3 8 92% Dewey 60 36 10 0 52 2 6 97% Durant 55 36 9 38 45 2 8 96% EI Reno' 15 36 0 43 12 2 1 87% Enid 218 58 11 8 165 22 31 90% Fairview 24 22 3 0 14 8 2 67% Fort Gibson' 25 36 0 0 20 1 4 96% Geary, Maple 24 22 1 17 20 2 2 92% Glenpool 59 36 9 0 54 3 2 95% Grove, South Rock Creek 41 22 10 0 39 1 1 98% Guthrie 87 58 7 0 74 1 12 99% Haworth 28 22 1 0 20 4 4 86% Heavener 27 22 9 0 24 0 3 100% Hobart 38 22 8 0 29 2 7 95% Hominy 43 22 9 0 34 0 9 100% Hugo 56 36 10 0 54 0 2 100% Hulbert' 13 22 0 8 0 9 4 31% Idabel 54 36 4 2 46 5 3 91% Jenks 199 58 10 67 150 13 36 93% Kingfisher 39 36 0 12 35 0 4 100% Little Axe 34 22 10 4 30 0 4 100% Oklahoma Technical Assistance Center 2009 Page 29 Table 6. Goal achievement data by program. District Name Total # Monthly #of Total # of #of #of #of Percentage of Minimum I" Months Teen Children Children Children Screened Children Visits Minimum Consultation Screened not Excused Served Visits Met Screened from Screening Locust Grove 53 36 10 174 45 3 5 94% Maryetta 34 22 9 0 29 1 4 97% McAlester 61 36 10 0 49 1 11 98% Meeker 21 22 0 9 20 1 0 95% Midwest City-Del City 128 70 8 4 110 0 18 100% Minco 33 22 7 0 27 0 6 100% Morrison 33 22 9 0 28 0 5 100% Muldrow 52 36 8 0 41 0 11 100% Newkirk 46 22 9 0 37 0 9 100% Noble 56 36 10 0 52 0 4 100% Norman 100 81 0 275 64 10 26 90% Okay 30 22 10 92 28 0 2 100% Oklahoma City Housinc 122 60 10 0 115 0 7 100% Oklahoma City State 184 144 9 2 171 0 13 100% Pawhuska 39 22 9 0 35 0 4 100% Perkins-Tvron 58 36 9 0 57 1 0 98% Perry 48 36 4 0 42 0 6 100% Pioneer-Pleasant Vale" 20 22 0 0 15 0 5 100% Pocola, Arkoma 54 36 7 23 48 0 6 100% Porum 24 22 1 0 20 1 3 96% Poteau 72 36 10 138 63 0 9 100% Pryor 58 36 10 0 56 0 2 100% Putnam City 230 108 10 23 214 0 16 100% Quinton 8 22 0 0 0 8 0 0% Salina 30"?- . 22 10 0 24 0 6 100% Sand Sprincs 52 36 10 0 39 4 9 92% Sapulpa 41 36 2 0 31 8 2 80% Shawnee 75 58 8 0 47 2 26 97% Shidler, Frontier 23 22 0 3 19 1 3 96% Skiatook 39 36 1 0 36 0 3 100% Sulphur 58 36 8 0 50 0 8 100% Swink: Fort Towson, Forest Grove 30 22 7 0 26 0 4 100% Tahlequah 77 36 10 7 65 0 12 100% Tecumseh 63 36 8 5 51 1 11 98% Tulsa Housinq 93 60 11 1 79 0 14 100% Tulsa State 287 144 10 0 250 0 37 100% Union 146 81 10 79 134 0 12 100% Oklahoma Technical Assistance Center 2009 Page 30 Table 6. Goal achievement data by program. District Name Total # Monthly #of Total # of #of #of #of Percentage of Minimum Months Teen Children Children Children Screened Children Visits Minimum Consultation Screened not Excused ?, I""Served I,' " Visits M~l Screened Scrferoemning " Verdiqris 46 36 2 0 43 0 3 100% Vian 34 22 9 0 27 0 7 100% Watts, Peavine 32 22 9 19 28 1 3 97% Wellston 38 22 9 0 35 0 3 100% Westville 40 36 0 0 31 5 4 88% Woodland, Kaw City, Wynona 50 22 9 0 49 0 1 100% Woodward 53 36 5 1 42 0 11 100% Wriqht City, Glover 32 22 9 0 29 0 3 100% * New programs Oklahoma Technical Assistance Center 2009 Page 31 -------_\ APPENDIX Oklahoma Technical Assistance Center 2009 Table A1. Program Reporting Dis0'0trict ';':~t~!;:,;1"t:i':~~."F;>~c:!ren~t~,I'.~jP~:ro",g~.ra.s,m~··f' M:o:nth~ly ,~' ~"r ,<~'t,,,, ~ ' . " ',:", 'Educator Coordirm1:or' ReR,ort ,"i? '.ii'\\,',,i: !p'!,,>,! ,.,.: Survey k Survey ',~l " #4' Ada 1 1 X Altus X Anadarko 3 1 X Ardmore 1 1 Avant, Bowring, Osage Hills, South Coffeyville 2 1 X Bartlesville 2 1 X Bethany 1 1 X Binger-Oney, Gracemont 1 1 X Bixby* 2 1 X Bristow 1 1 X Broken Arrow 4 1 X Caney Valley, Copan X Chouteau-Mazie 1 1 X Claremore 1 1 X Coalgate X Comanche X Dewey X Durant 1 X EI Reno* 2 1 X Enid X Fairview X Fort Gibson* 1 1 X Geary, Maple 1 1 X Glenpool 2 1 X Grove, South Rock Creek 1 1 X Guthrie 3 1 X Haworth 1 1 X Heavener 1 1 X Hobart 1 1 X Hominy 1 X Hugo 1 1 X Hulbert* 1 1 X Idabel 1 1 X Jenks 5 1 X Kingfisher 1 1 X Little Axe 1 1 X Locust Grove X Maryetta 1 1 X McAlester 2 1 X Meeker 1 1 X Oklahoma Technical Assistance Center 2009 Table A1. Program Reporting "\',' ,j, ?i~:'" ,'~ ,';~;~'8l"": ~'" ," :r" , '" I,; Program -", District - , Parent Monthly - , "Educator Coordinator 1~1,Report ,v'", ", ".;;;; y ':~,,) :'@';\ii" "i~~;, ';~%t",;", "i\'! Survey Survey Midwest City-Del City 2 X Minco 1 1 X Morrison 2 1 X Muldrow 1 1 X Newkirk 1 X Noble 1 1 X Norman 4 1 X Okay 1 1 X Oklahoma City 6 1 X Pawhuska X Perkins-Tryon 1 1 X Perry 1 1 X Pioneer-Pleasant Vale* 1 1 X Pocola, Arkoma 1 1 X Porum 1 1 X Poteau 1 1 X Pryor 1 1 X Putnam City 5 1 X Quinton 1 1 Salina 1 1 X Sand Springs 2 1 X Sapulpa 1 X Shawnee 2 1 X Shidler, Frontier 1 X Skiatook 1 1 X Sulphur 4 2 X Swink,* Fort Towson, Forest Grove 1 1 X Tahlequah 2 1 X Tecumseh 1 1 X Tulsa 10 1 X Union 1 1 X Verdigris 1 1 X Vian 1 1 X Watts, Peavine 1 1 X Wellston 1 X Westville 2 1 X Woodland, Kaw City, Wynona X Woodward 1 1 X WriQht City, Glover 1 1 X Oklahoma Technical Assistance Center 2009 • -~ OTAC Oklahoma Technical Assistance Center 123 E. Broadway Phone Cushing, Oklahoma 74023 918.225.1882 July 13, 2009 Dr. Ramona Paul Assistant State Superintendent Oklahoma State Department of Education 2500 North Lincoln Boulevard Oklahoma City, OK 73105 Dear Dr. Paul, We have compiled and analyzed the data submitted by the 81 Oklahoma Parents as Teachers (OPAT) grantees throughout the school year. We have worked with each of the programs to ensure timely and consistent reporting of activities. This year, only one of the 81 programs, Quinton, did not provide a full year of data. The Quinton site reported that they operated a program this year and monthly data was provided for August through the month of November. Multiple attempts by OTAC staff to acquire information for the remaining months were unsuccessful. Attached is a table that lists the number of months that each program met its monthly minimum requirements and the percentage of children who received developmental screening. We have preliminary totals through June. We have not thoroughly reviewed the data and contacted local programs to resolve any discrepancies, so the numbers reported here are preliminary. The 81 programs served a total of 5,050 children this year. The number of separate families served was 4,379. They completed 33,739 personal visits - these are personal visits only; the total does not include teen consultation visits. A total of 1,397 parent group meetings were reported with a total attendance of 11,320 parents - an average of 17.46 meetings per grantee and an average attendance of 8.10 parents per meeting. The 81 programs served fewer children and families compared to last year. The 78 programs funded in 2007-08 served a total of 4,650 families representing 5,355 children. Services to these families included 34,751 personal visits and 1,349 parent group meetings. In total 11,376 parents attended these meetings. The grantees averaged 17.9 meetings during the year ~ith a mean 8.43 parents in attendance. A decrease was noted in the number of children and families served this year compared to 2007-08 even though three more programs were funded. This may be due to the SDE's decision to lower the minimum number of required families and visits by 10 percent. This reduction in program requirements was made in response to the increase in travel costs that affected all programs, but, particularly those in small, rural areas. The remainder of this report accounts for the programs in one of three listings: Oklahoma Technical Assistance Center 2009 (1) Recommended for continued funding: the program met the minimum requirements and screened all eligible children. (2) Recommended for continued funding with reservations: the program data suggested issues that prevented full program compliance. (3) Not recommended for continued funding: the program suffered implementation issues that would require a plan of improvement. (1) Recommended for continued funding: Ada Avant, Bowring, Osage Hills, South Coffeyville Bethany Binger-Oney, Gracemont Broken Arrow Caney Valley, Copan Chouteau-Mazie Grove, South Rock Creek Heavener Hominy Hugo Mid-Del Minco Morrison Muldrow Newkirk Noble Okay Oklahoma City Oklahoma City Housing Pawhuska Perkins- Tyron Poteau Pryor Putnam City Salina Sulphur Swink, Fort Towson, Forest Grove Tahlequah Tecumseh Tulsa Union Vian Wellston Woodland, Kaw City, Wynona Wright City, Glover (2) Recommended with reservations: Oklahoma Technical Assistance Center 2009 • Altus is recommended with reservations for the third year in a row. They met the minimum service requirements in two of the months and screened 73 percent of eligible children. The transient nature of the population in Altus continues to be an obstacle for the program. This year, they experienced a 36 percent turnover rate in families served. • Bixby was a new program this year. They met their minimum for five months and screened 98 percent of the children. • Bristow, like last year, did not meet the service minimum in anyone month, although they screened 100 percent of their children. They routinely provide two-thirds of the required visits for this size of grant. A grant award that requires a lower participation level may be appropriate. • Coalgate did not meet the service requirement in any month this year. Twenty-two visits per month were required for Coalgate's funding level; the greatest number of visits completed in a month was only 13. The program screened 94 percent of eligible children. • EI Reno was a first-year program. Less than one-third of the required number of visits were completed; 87 percent of eligible children were screened. • Fort Gibson was a new program this year and did not receive funds until later in the year. On average, they completed only half of the required monthly visits although their number of visits increased during the year. They screened 96 percent of eligible children. • Geary-Maple met the service requirement for one month. They averaged half of the required minimum visits per month. They screened 92 percent of eligible children. This program provided services for 11 months. • Haworth participated in a cooperative with Wright City in years past. Now, as a single site, the program met its service requirement in only one month. Examination of records indicated that they lacked only one or two visits each month to meet the minimum expectancies. The program screened 86 percent of eligible participants. • Hulbert was a new program this year. Their number of visits increased during the year although they did not meet their service requirement in any month. Only 31 percent of eligible children were screened. • Idabel had a new parent educator/program coordinator this year that resulted in a delay in full implementation of the program during the first semester. They met the service minimum in four of the months (spring semester) and screened 91 percent of eligible children. • Kingfisher is recommended with reservations for the fourth year in a row. The minimum service requirement was not met in any month this year. A full 100 percent of eligible children were screened. A grant award that requires a lower participation level may be appropriate. • Meeker showed improvement this year in increasing the number of monthly visits, although the number was slightly below the monthly minimums. The program screened 95 percent of eligible children. • Norman is recommended with reservations for the second year. The program screened 90 percent of eligible children. The number of monthly visits remained substantially below required minimums. A grant award that requires a lower participation level may be appropriate. Oklahoma Technical Assistance Center 2009 • Perry met their minimum in four months and barely missed the minimum in the remaining months. All of the eligible children were screened. • Pioneer-Pleasant Vale, a new program, showed growth in the number of visits during the year. The most visits completed in anyone month were 19, short of their goal of 22. All eligible children were screened. • Porum screened 96 percent of eligible children. Service requirements were met in two months. The parent educator experienced health problems that affected services during the year. • Shidler, Frontier operated as a member of a cooperative in 2007-08. This year, they did not meet their month minimum in any month. Records indicated that 96 percent of the eligible children were screened. • Skiatook restructured the program this year with the arrival of a new parent educator. Improvement was noted in the number of required visits; however, the minimum was not met until June. All eligible children were screened. • Tulsa Housing met their minimum in three of the months. The program appeared to have difficulty in completing visits during the winter months although number of visits completed was near their requirement of 54. A full 100 percent of eligible children were screened. • Verdigris screened all eligible children. The minimum was met two months during the year and fell short only one or two visits in the remaining months. • Westville did not meet the required minimum in any month and was substantially short in meeting the requirements. Only 86 percent of eligible children were screened. A grant award that requires a lower participation level may be appropriate. • Woodward met the minimum in five months and was slightly low in the remaining months. The program screened 100 percent of eligible children. (2.a) The following programs are also recommended with reservations. These programs met their monthly targets but failed to screen all eligible children. These programs screened at least 90 percent of eligible children. Anadarko Ardmore Bartlesville Claremore Comanche Dewey Durant Enid Glenpool Guthrie Hobart Jenks Little Axe Locust Grove Maryetta McAlester Pocola, Arkoma Sand Springs Oklahoma Technical Assistance Center 2009 Shawnee Watts, Peavine • Fairview has experienced difficulties for the last three years in meeting the service requirements. This year, the minimum requirement was met for three months. Like last year, only two-thirds of the eligible children were screened. The director reported that other children have been screened; however, no records were provided of this activity. • Quinton has experienced multiple difficulties over the last two years. In 2007-08, it failed to meet its targets in any month. This year, data was only reported for four months. Only eight visits were reported per month. Numerous attempts to contact the program coordinator and site superintendent did not result in the submission of program records. • Sapulpa is not recommended for funding for a second year in a row. This is the fourth year the program has not met the minimum service requirements. In 2008- 09, minimum requirements were met in two of the months. This is also the fourth year that the program has failed to screen all of the eligible children; 83 percent of children were screened this year. (3) Programs not recommended for continued funding [depending upon the circumstances (e.g., first-year programs), the SDE may require programs to submit a plan of improvement in order to receive continued funding]: Please contact me if you have any questions or concerns. Sincerely, Dr. Kathy McKean, Director Oklahoma Technical Assistance Center 2009
Object Description
Description
Title | Oklahoma Parents as Teachers annual program Evaluation |
OkDocs Class# | E1800.3 P228a 2008/09 |
Digital Format | PDF, Adobe Reader required |
ODL electronic copy | Deposited by agency in print; scanned by Oklahoma Department of Libraries 7/2011 |
Rights and Permissions | This Oklahoma state government publication is provided for educational purposes under U.S. copyright law. Other usage requires permission of copyright holders. |
Language | English |
Full text | E 1800.3 P228a 2008/09 e.l Oklahoma Parents As Teachers Annual Program Evaluation December 17,2009 Oklahoma Technical Assistance Center 123 E. Broadway Cushing, OK 74023 www.otac.info DlAC alternate paths to learning Table of Contents Executive Summary . Program Description 1 Program and Participation Data 2 Figure 1. Geographical distribution of OPAT programs 3 Table 1. Program funding levels, requirements, and costs 5 Program Evaluation Strategies 5 Participants 7 Table 2. RaciallEthnic distribution of children, OPAT participants, and general student population 7 Figure 2. Number of risk factors 9 OPAT Services 9 Personal Visits 10 Table 3. Total number of personal visits completed per month 11 Parent Group Meetings 12 Developmental and Health Screening Services 12 Outcomes 14 Analysis 1 15 Analysis 2 16 Table 4. Pre-post ASQ rating of children who participated in OPAT, 2008-09 17 OPAT Staff Surveys 18 Parent Educator Survey 18 Program Coordinator Survey 22 Survey Summary 26 Table 5. Survey items by year 27 Goal Achievement: Individual Programs 28 Table 6. Goal achievement data by program 29 Appendix Table AI. Program reporting OSDE recommendation letter Parent Knowledge Questionnaire OPAT Coordinator Survey OPAT Parent Educator Survey Program database Oklahoma Parents as Teachers Program Evaluation Executive Summary 2008-09 Program Information and Grantees The Oklahoma Parents As Teachers (OPAT) grant program funded 79 grants to programs serving families of young children in 94 school districts. Total state funding for OPAT programs ($2,045,709) was unchanged compared to previous years. The size of the award was dependent on the size (Average Daily Membership) of the applicant. In addition to funds allocated to school districts, two public housing authorities in Oklahoma City and Tulsa each received $65,695. OPAT parent educators completed a total of33,775 personal visits with 4,388 families and 5,027 children during the 2008-09 funding year. The average program cost per family was $439; average program cost per child was $384. Table 1. Program funding levels, requirements, and costs. School District Grant Number Original Optional Median Cost Median Cost (ADM) Amount Awarded Minimum Minimum Per Child Per Child Monthly Visits Monthly Visits 2007-08 2008-09 500 $13,500 31 25 22 $365 $422 1,000 $21,000 34 40 36 $365 $385 3,000 $35,000 7 65 58 $323 $402 10,000 $42,500 1 78 70 $278 $332 10,000 $48,500 3 90 81 $404 $357 18,000 $63,000 1 120 108 $290 $274 30,000 $84,000 2 160 144 $321 $374 Characteristics of Families and Children Served • The ethnic and racial composition of families served was fairly similar to the Oklahoma population of school children as a whole. • Twenty-nine percent of participating families enrolled in the program before the child was born. • Twenty-two percent of participants were teen parents (under the age of20 at time of delivery). • More than half the participants (51.2%) were classified as "low income or unemployed." • A disability was documented for 3.2 percent of children. • Children born prematurely or classified as "low birth weight" babies accounted for 9.3 percent of participants. Almost a quarter of children (22.1%) had mothers who did not complete high school. OklahomTaechnicAalssistancCeenter Page i • In 15.0 percent of OPAT families, a language other than English was spoken in the home. • Programs reported that 9.6 percent of families had additional risk factors. • More than half of the participants had at least one at-risk factor (61.7%) and almost a quarter of the participants (22.4%) had three or more risk factors. Developmental and Health Screening OPA T grant programs are required to provide health screening services to all of the children enrolled. According to data submitted by the grantees, 91.1 percent of children received health screening. The percentage of OPA T children age 19-35 months who have received appropriate immunizations continued to increase. The average in 2005-06 was 87.7 compared to 90.8 percent in 2006-07, 91.3 percent in 2007-08 and 93.9 percent in 2008-09. OPAT's efforts are reflected in the increase in immunization rates across Oklahoma for all children. In 2005, Oklahoma ranked 44th in the nation in child immunization rates compared to 15th in 2009. OPA T programs provided developmental screening through the use of the Ages and Stages Questionnaires (ASQ). Parent Educators administered the ASQ to children who were older than four months of age and within their first two months of participation in OPAT; and again in each year of participation. If the results of the ASQ indicated developmental delays, the child was referred to appropriate public and private services. This year, 84.9 percent of the participants were screened by 0 PAT and another 11.9 percent were excused. Only 3.2 percent of the children served through OPA T should have received screening but did not. The most commonly reported reason for not screening a child was because the family had moved from the district. Outcomes The ASQ was administered in order to identify children who are at-risk. The assessment covered five key developmental areas: communication, gross motor, fine motor, problem solving, and personal-social skills. Pre-post ASQ scores were used to determine the extent to which children who participated in the OPAT program, who were initially identified as at-risk, moved to the not at-risk group. Two separate statistical analyses showed significant, positive changes in total scores and children's at-risk classification after participation in OPAT. Children were more likely to be classified as at risk on their pre-program ASQ than on their latest ASQ. Children enrolled in OPA T showed a reliable gain in cognitive, language, social, and motor development. Oklahoma Technical Assistance Center Page ii Oklahoma Parents As Teachers (OPAT) Annual Program Evaluation School Year 2008-09 Submitted to Sandy Garrett, State Superintendent of Public Instruction Oklahoma State Department of Education Prepared by the Oklahoma Technical Assistance Center Program Description The mission of the Oklahoma Parents As Teachers (OPAT) is to provide the information, support, and encouragement that families need to help their children develop optimally during the crucial, early years of life. This voluntary program provided services to families with children from prenatal to 36 months old and was based on the philosophy that parents are their children's first and most important teachers. The OPAT program is designed to enhance the development of young children, enabling them to learn, grow, and develop to realize their full potential. OPAT is affiliated with the nationally validated Parents As Teachers (PAT) program that originated in 1981 as a pilot project in Missouri. The pilot program was based on research showing that greater family involvement in children's learning is critical to school success. Based upon the pilot program's success, the program was validated by the United States Department of Education's Program Effectiveness Panel. Since that time, PAT has been listed as an exemplary program by four other national organizations. The Centers for Disease Control and Prevention recognized PAT as a noteworthy prevention program and effective strategy for preventing child maltreatment. The Strengthening America's Families Project, in collaboration with the Center for Substance Abuse Prevention and the Office of Juvenile Justice and Delinquency Prevention, designated PAT as a model family-strengthening program. The national center has continued to conduct Oklahoma Technical Assistance Center 2009 Page 1 research, develop curriculum, and expand the program. There are now more than 3,000 Parents As Teachers programs covering all 50 states, United States territories, and seven additional countries ranging from Canada to China. The goal of the Oklahoma Parents As Teachers program is to strengthen the capacity of parents to be effective first teachers. Parents As Teachers offers a complete, research-based curriculum entitled Born to Learn. OPAT is also designed to foster an early partnership between home and school, enabling parents to take a more active role during their children's formal years of schooling. The program is designed to maximize a child's overall development during the first three years of life, lay the foundation for school success, and minimize developmental problems that interfere with the child's learning. Other intended benefits of the program include early identification of developmental delays or sensory problems (e.g., hearing, vision) and a reduction in the number of children who require remedial and special education programs. Program and Participation Data The number of OPAT programs increased from 76 last year to 81 this year. This includes two public housing authorities (Oklahoma City and Tulsa) who also received funding. Eleven grants were awarded to school districts that partnered with at least one other neighboring district. Eight of the multi-district grantees consisted of two partner districts, two consisted of three partner districts and the award to Avant also included the districts of Bowring, Osage Hills, and South Coffeyville. In total, 94 Oklahoma communities were served by OPAT programs. The majority of the participating districts were located in rural areas or small towns (Figure 1). Although programs were located across the state (with the exception of the panhandle), more were located in the eastern half where the greater proportion of the population resides. Oklahoma Technical Assistance Center 2009 Page 2 Figure 1. Geographical distribution ofOPAT programs 2008-09 **OPA.T Procrarrs Public HoJsing Prcgram * * * * : * * ** * *** * * * * * * * * ** * *~***>i **!* 1 * ** * * * * * * * * * * * The amount of funding appropriated for OPAT programs by the Oklahoma Legislature in 2008-09 ($2,045,709) remained unchanged from the previous fiscal year. The Oklahoma State Department of Education let a total of$I,927,814 in grant funds. (The funds awarded to school districts totaled $1,796,500 and the public housing authorities in Oklahoma City and Tulsa each received $65,657.) Additionally, Oklahoma Technical Assistance Center received $16,334 to provide program evaluation services. The remaining funds ($101,561) were used by the Oklahoma State Department of Education to provide technical assistance to grantees. School districts applied for grant funds based on their community size. This was determined by the Average Daily Membership (ADM) of the applicant district. Table 1 lists the number of grants by community size. The table also lists program requirements and the median cost per child, disaggregated by funding levels. Depending on their ADM, school districts received grant awards ranging from $13,500 to $84,000. In 2008-09, programs had the option to reduce the number of monthly visits by ten percent in order to compensate for the increase in transportation costs the projects experienced. The revised monthly minimum visits as well as the original minimums are presented in Table 1. More than three fourths of the grants (82.2%) were awarded to small school districts with an ADM of 1,000 or less. The state's two largest urban areas, Oklahoma City and Tulsa, received the largest awards. Grantees with ADM between 10,000 and 18,000 were located in suburban communities. In addition to the funding levels designated in law, the State Department of Education added an additional funding level to accommodate the needs of the Midwest City - Del City school district. The target was set at 70 rather than 78 monthly visits for the Midwest City - Del City program in exchange for a grant award of $42,500. This modification was made when the district experienced difficulties in meeting its monthly visit requirements. Oklahoma Technical Assistance Center 2009 Page 4 This year the programs reported that they served 4,388 families with their OPAT funds. Additionally, three of the grantees, Caney Valley-Copan (7) Dewey (6), and Enid (130) reported that they served families with other funding sources such as district funds or community resources. In total, OPAT programs reported that they served 5,027 children during the 2008-09 $439. Table 1 reports the median cost per child for 2007-08 and 2008-09. The cost per child program year. This resulted in a program cost per child of$384 and an average family cost of increased substantially in each of the community sizes this year. This was primarily the result of the ten percent reduction in the number of visits required for programs. The costs ranged from $274 to $422 per child. Table 1. Program funding levels, requirements, and costs. School District Grant Number Original Optional Median Cost Median Cost (ADM) Amount Awarded Minimum Minimum Per Child Per Child Monthly Visits Monthly Visits 2007-08 2008-09 500 $13,500 31 25 22 $365 $422 1,000 $21,000 34 40 36 $365 $385 3,000 $35,000 7 65 58 $323 $402 10,000 $42,500 1 78 70 $278 $332 10,000 $48,500 3 90 81 $404 $357 18,000 $63,000 1 120 108 $290 $274 30,000 $84,000 2 160 144 $321 $374 Program Evaluation Strategies Since 1991, the Oklahoma Technical Assistance Center (OTAC) has conducted a yearly evaluation of the OPAT programs. This independent evaluation of the OPAT program has included a number of components, relying on both quantitative and qualitative information. In Oklahoma Technical Assistance Center 2009 Page 5 each of these annual evaluations, the OPAT program has demonstrated its effectiveness in meeting the overall program goals. The primary focus of the program evaluation has remained consistent over the years; although, the content and methods used in the program evaluation have been revised each year in order to address specific areas of interest or concern. The evaluation relies on quantifiable data. These include monthly program service reports, pre- and post-assessments of children's development, surveys of service providers, and biannual assessments of changes in parent knowledge attributed to the program. A set of instruments for collecting relevant data from OPAT programs was developed in consultation with State Department of Education Early Childhood staff and OPAT program coordinators. The set of instruments includes the following: • an electronic project service reportingform, completed each month by local OPAT staff, used to record the number and type of project services provided to each participating family, demographic information, and pre-post assessment data; • a Parent Knowledge Questionnaire, designed by the PAT National Center, for evaluating the degree of change in parent knowledge of child development and child safety. This instrument is administered every even-numbered year; • the Ages and Stages Questionnaires, a commercially-produced standardized instrument for assessing child development in comparison with normative standards; and • two OPAT staff surveys, one for Parent Educators and one for Program Coordinators, designed to collect information on program services, common practices, exemplary practices, program costs, and future needs. Oklahoma Technical Assistance Center 2009 Page 6 One program, Quinton, failed to provide monthly data on the children and families Table AI, located in the Appendix, lists each of the funded programs in 2008-09 and the data that was submitted by that program. served throughout the year. This program was not included in the evaluation process. Participants OPAT' s emphasis on early intervention was apparent in the large number of parents who were enrolled in the program prior to the birth of their child. More than a quarter of this year's participants (29.2%) were enrolled in the program before the child was born, most between the second and seventh months of pregnancy. Additionally, 73.4 percent ofOPAT children were comparison was that, when given the option, a substantial proportion of OPAT parents preferred enrolled before their first birthday. Table 2 presents the racial and ethnic distribution of OPAT participants and Oklahoma school children. Because of the form in which the data was reported, it was difficult to compare the OPAT distribution to the state distribution. The most interesting finding from this to categorize their families as multiracial. Table 2. Racial/Ethnic distribution of children, Oklahoma Parents As Teachers participants and general student population. Group OPAT Children All Oklahoma School Children African-American/Black 5.8% 11% American Indian/Alaskan Native 12.4% 19% Asian/Pacific Islander 0.8% 2% CaucasianlWhite 58.7% 58% Hispanic/Latino 10.1% 10% Multiracial 11.0% (Not an SDE category) Not Reported 1.1% -- Oklahoma Technical Assistance Center 2009 Page 7 According to the authorizing legislation, OPAT programs were required to place a high priority on serving children considered to be at risk. The "at-risk" characteristics of 2008-09 participants are summarized as follows: • Teen parents accounted for 21.8 percent of those served. • More than half the participants (51.2%) were classified as "low income or unemployed." • In 15.0 percent of OPAT families, a language other than English was spoken in the home. • A substantial proportion of children (3.2%) had a documented disability. • Children who were born prematurely or were classified as "low birth weight" babies accounted for 9.3 percent of all participants. • A great deal of research has noted that the educational attainment ofa child's mother is one of the best predictors of the educational achievement of the child. A study by the RAND Corporation (Lara-Cinisomo, et aI., 2004) reported that "...the two factors associated most strongly with school readiness are (l) the educational attainment of mothers and (2) neighborhood poverty." The data indicated that 22.1 percent of the children served had mothers who did not complete high school. Note that this does not include the teen mothers who were still in high school. • Programs reported that 9.6 percent of families had additional risk factors - these lower-incidence factors defined by the National PAT Center were combined into one category to ease the reporting burden on the grantees. Oklahoma Technical Assistance Center 2009 Page 8 Figure 2. Number of Risk Factors As Figure 2 presents, 61.7 percent of the children had at least one of these risk factors. Almost a quarter of the participants (22.4%) had three or more factors. As in previous years, children of mothers who did not graduate high school were most likely to have multiple risk factors, 97.7 percent had at least two factors and 79 percent had three or more factors. 2,000 OPAT,200S'()9 1,SOD ••.•. c: •a•. 'u "f 1,000 ~ Q••. SOD ~ ~ is.3s%1 0 .00 1.00 2.00 3.00 4.00 5.00 6.00 Factors OPAT Services OPAT staff provided three direct services as part of the Parents As Teachers model; personal visits, parent group meetings, and developmental and health screenings. Parent educators visited families in a location preferred by the parent, usually the home, once each month. During this meeting, the parent educator presented age-appropriate child development and parenting information, helped parents learn to observe their children, and addressed parents' concerns. At least one meeting was devoted to developmental screening. Parents were also Oklahoma Technical Assistance Center 2009 Page 9 invited to attend group meetings where they shared information about child development and parenting. Personal Visits The minimum number of visits the program staff were expected to complete each month was determined by the size of the award. The number of required personal visits ranged from 22 per month (for programs that received $13,500) to 144 visits (for those that received $84,000). To be eligible for continued funding, OPAT programs must meet their minimum monthly requirements within an evaluation paradigm that sets out which visits are "countable." For example, programs may count only one visit per month per child. In the case of high-risk children, local OPAT staff may decide to conduct extra visits with the parent and child, as recommended by the PAT National Center. These visits are accounted for in the database, but are not included in the determination of whether the grantees met their monthly minimum requirements. Table 3 presents the number of visits completed in each month of the program year. The number of visits that OPAT completed was fairly stable throughout the year. The two lowest figures were in the start-up month (August) and the last month of the program (June). The number of visits completed in August and June were the result of the policy allowing programs to select their program beginning and ending dates based on community calendars. All programs were encouraged to operate for ten months of the year; if a program began its program year in the first week of August, it could conclude its program year at the end of May. Many programs began their year in congruence with school district enrollment in mid-August and operated their programs until mid-June. Additionally, new programs started later, due to the fact that they had to hire parent educators and arrange for training and certification before permitting them to work Oklahoma Technical Assistance Center 2009 Page 10 as OPAT parent educators. In total, OPAT staff conducted a total of 33,775 personal visits during the 2008-09 school year. Table 3. Total number of personal visits completed per month, all programs. Aug Sep Oct Nav Dec Jan Feb Mar Apr May Jun Total 2,316 3,275 3,406 3,409 3,274 3,345 3,484 3,441 3,445 3,449 931 33,775 In addition to personal visits usually conducted in the home, the parent educators also conducted consultations with teen parents. These sessions were usually held in the teen's school. Consultations with parenting teens followed the same format as personal visits but were conducted with the parent only - circumstances prevented a full visit with both parent and child. Teen consulting visits were often the best way to ensure that the teen parent consistently participated in the Born to Learn curriculum. This year, 1,158 consulting visits were held with teen parents. The average length of enrollment in the program for a family in 2008-09 was 7.21 months. More than a third of all participants (39.3%) were enrolled for the entire program year (ten months or more). Factors that impacted the length of enrollment in the program included: • OPAT has an open entry/open exit enrollment policy. • Participation in the program was voluntary. • Parents could enroll their child at any point during the year and could leave the program at any time. • According to the authorizing legislation, children may be served until they reach 36 months of age. Once they reach this age, they may no longer be enrolled in OPAT and OPAT funds may not be used for continued services. Oklahoma Technical Assistance Center 2009 Page 11 The average number of personal visits for all children, regardless of number of months enrolled, was 6.72. The mean number of visits for those children in the program seven or more months was 8.61 suggesting that the program staff focused on completing the monthly visits. Parent Group Meetings The programs relied on schools, early childhood centers, community centers, churches, libraries, and parks as locations for parent group meetings. The group meetings gave parents an opportunity to learn from and support each other, observe their children with other children, and practice new skills. All of the programs conducted parent group meetings in this year. The programs held a total of 1,402 meetings and averaged 17.31 meetings per program, exceeding the program goal of one meeting per month. The program hosted by the Tulsa School district provided the most parent meeting opportunities (101) and the EI Reno program conducted the fewest (3). Attendance at meetings ranged widely, depending on the community size. As a group, the mean number of parents in attendance was 8.07 regardless of program size. Developmental and Health Screening Services The Ages and Stages Questionnaires (ASQ) was used by the OPAT programs to provide developmental screening. Training in the appropriate use of this instrument was provided by the Oklahoma State Department of Education at the beginning of the program year. Qualified parent educators were responsible for its administration. Children older than four months of age were to be screened within the first two months of their participation in OPAT. (The ASQ cannot be administered until children are at least four months of age.) Children who continue in the program must be re-screened each year. Children who exhibit developmental delays were referred to appropriate public and private service providers such as Sooner Start, county health departments, or pediatricians. Oklahoma Technical Assistance Center 2009 Page 12 The goal of the program is to screen 100 percent of eligible children each year. As noted above, children younger than four months of age cannot be screened. Some children who receive annual developmental assessments from other agencies (e.g., Sooner Start) do not need to be screened by OPAT. Some participants leave the program before their children can be screened. The degree to which each OPAT grantee met screening goals is an important factor in the SDE's decision as to whether or not to continue to fund the program. Program-by-program statistics on the number of children screened, excused, and unscreened may be found in the last section of the report, Goal Achievement. Overall, OPAT programs appeared to have met their goal of screening all children. This year, 84.9 percent of the participants were screened by OPAT and another 11.9 percent were excused for reasons cited above (e.g., age, screened by another agency). Only 3.2 percent of the children served through OPAT should have received screening but did not. Most programs reported the reasons that children were not screened. In many cases, the family moved from the districts just prior to the screening process. In addition to the developmental screenings, OPAT programs are also required to provide health screening services to all of the children enrolled. According to data submitted by the grantees, 91.1 percent of children received health screening. Program-by-program statistics are included in the Goal Achievement section. According to OPAT reports, the percentage of OPAT children age 19-35 months who have received appropriate immunizations continued to increase. The average in 2005-06 was 87.7 compared to 90.8 percent in 2006-07, 91.3 percent in 2007-08 and 93.9 percent in 2008-09. OPAT's efforts are reflected in the increase in immunization rates across Oklahoma for all children. In 2005, Oklahoma ranked 44thin the nation in child immunization rates compared to Oklahoma Technical Assistance Center 2009 Page 13 __ 1 Determining the effectiveness of the OPAT program on the cognitive development of 15th in 2009 (Oklahoma Department of Health, Disease and Prevention Services. Oklahoma Immunization Update, 2009). Outcomes young children is a principal goal of the program evaluation. As noted previously, outcomes for children are evaluated through annual administration of the Ages & Stages Questionnaires (Bricker & Squires, 1999), a low-cost method for reliably assessing children for developmental delays. The emphasis of the instrument is on detecting delays and potential delays; thus, it does not assess the full range of cognitive ability. ASQ pre-test and post-test scores were available for 2,433 children, permitting statistical analysis of changes in participants' developmental risk levels. Children are tested only once per year and many children had not yet completed the three-year program. Their pre-tests and post-tests were less than three years apart. Therefore, the results should be considered as an Within two months of entry into the OPAT program, a pre-program ASQ is administered underestimate of the true effects of the program. to each child. 1 In subsequent years, an age-appropriate form of the ASQ is administered annually. The average number of months between the initial ASQ and the most recent ASQ was 14.81 months - approximately 40 percent of the length of the OPAT program. These results, then, are a conservative estimate of the effects of the program. Two sets of statistical analyses were conducted: (1) an analysis of the ASQ scale scores and (2) an analysis to determine whether there was a change in the proportion of children who scored in the "at risk" range on the ASQ. The first analysis was conducted principally to help lThe ASQ cannot be administered until the child is four months old. Parents who enter OPAT before the birth of their child may not have the "pre-test" administered until their child is four months old. Oklahoma Technical Assistance Center 2009 Page 14 Analysis 1. An initial set of t-tests was conducted on the ASQ scale scores. The ASQ understand the properties of ASQ scores over time. The second analysis was the more important one - the analysis designed to answer the research question: "Is participation in OPAT related to changes in the at-risk levels of children?" The results of all statistical tests indicated that the OPAT program resulted in reduced risk levels for participating children. scale scores are derived scores that theoretically remain fairly constant unless a child's normative standing changes - that is, unless there are changes in a child's developmental progress when compared to other children. We would, therefore, expect no change unless OPAT participants progressed at afaster rate than the norm group.' This set of tests was exploratory only, as the ASQ scores lack sufficient variance for powerful statistical analyses due to their limited scale. The instrument was designed to detect developmental delays and does not test the limits of children's development. In other words, most children score near the top and there is limited room for them to improve. For this preliminary look at changes in ASQ scores, the subtest scores were combined into a "total score" and a t-test was conducted.' The mean initial score (pre) was 52.96; the 9.225,p = .000], indicating a reliable positive change in ASQ scores for OPAT participants. An mean second (post) score was 54.17. The difference was statistically significant [t(2420) = effect size of .19 standard deviations was calculated on these scaled scores - the typical OPAT participant's second score gained one-fifth of a standard deviations between pre-test and post-test. An effect size of one-third of a standard deviation (.33) is generally accepted as a 2Although these scores range from 5 to 70, the scoring procedure always results in a multiple of five, resulting in a 14-point rather than a 70-point scale. 3An omnibus significance test is much preferred over the analysis described here. The nature of the ASQ and its scores, however, mitigated against a multivariate analysis with at least five dependent variables - there is simply not enough variance to partition. The t-test analysis was conducted for exploratory purposes only, to facilitate an understanding of the amount of change in ASQ scale scores. Oklahoma Technical Assistance Center 2009 Page 15 substantial educational intervention; an effect size such as the one observed in this analysis (.19) is regarded as important. Approximately half of the children tested were enrolled in the program for 12 months or more. The effect size (.25) was greater for these children. It should be emphasized that the ASQ does not test the full range of cognitive ability; because of the limited range of scores, we would not expect a large effect size. Statistically significant differences in children's first and most recent ASQ scores were noted on the Communication, Gross Motor, Fine Motor, and Personal Social subtests. As in previous years, only the Problem-Solving subtest did not produce positive gains. The average change in scores on these four subtests ranged from 1.15 scale points to 2.11. Analysis 2. The purpose of this analysis was to determine if participation in OPAT reduced the child's risk of school failure. This procedure was designed as a part of a longitudinal study that was a part of the original evaluation plan for OPAT.4 The study was designed so that, once OPAT participants were school age, the principal analysis would determine whether children who initially scored in the At Risk range on their pre-program ASQ demonstrated school achievement levels within the normal range. The analysis provided a preliminary look at OPAT's effects on reducing risk. Statistical tests were conducted to determine whether the children showed a reduction in the number of subtest scale scores in the At Risk range. For the purposes of this analysis, a child was classified as "at risk" if he or she had at least one subtest score in the At Risk range. A two-way chi square test was conducted on children's risk status on the initial measure and the most recent measure. Four situations were possible: 4Because of program attrition and lack of funding, the longitudinal study was abandoned. Oklahoma Technical Assistance Center 2009 Page 16 • the child was classified At Risk on both the initial measure (pre) and on the most recent measure (post), • the child was At Risk on the pre-ASQ but not on the post (positive change), • the child was Not At Risk on the pre, but was on the post (negative change), and • the child was Not At Risk on either administration of the ASQ. It is important to note that children face more at-risk factors as they age. A number of indicators of developmental delays cannot be assessed until children are old enough to begin creeping, crawling, walking, and talking. Because of this factor, we always expect some number of children to initially score as Not At Risk and later score in the At Risk range. The analysis included 2,409 children. Table 4 shows the number of children who were in each of these categories: • 61 children were classified At Risk on both the pre-ASQ and the post-ASQ, • 209 children were At Risk on the pre-ASQ but not on the post (positive change), • 128 children were Not At Risk on the pre-ASQ but were At Risk on the post (negative change), and • 2,011 were Not At Risk on either administration of the ASQ. Table 4. Pre-post ASQ rating of children who participated in OPAT, 2008-09. Most Recent ASQ Not At Risk At Risk Initial Not At Risk 2,011 128 ASQ At Risk 209 61 Because ASQ scores are not perfectly accurate, there is always some error in the instrument's classification of children as "at risk." To determine whether the number of children who changed categories was statistically significant, a chi square analysis was conducted. The Oklahoma Technical Assistance Center 2009 Page 17 results indicated that children were more likely to be classified as At Risk on the initial ASQ than on their most recent ASQ (X2 = 91.47,p =.000). On the initial ASQ, 11.21 percent of children had at least one At Risk score. On the most recent ASQ, only 7.85 percent scored in the At Risk range on any subtest. Of children with at least 12 months between pre-test and post-test, 7.37 percent scored in the At Risk range. Of children who had at least 24 months between pre-and post-test, 4.87 percent scored in the At Risk range. The longer children participated in the program, the less likely they were to be classified as At Risk. OPAT Staff Surveys The Parent Educator Survey and the Program Coordinator Survey were designed to provide more insight into the roles of the local OPAT staff. Both instruments were administered at the OPAT Spring Conference to all that attended. For those programs that did not have program representatives at the spring conference, surveys were mailed directly to the program staff. Parent Educator Survey The Parent Educator Survey consisted of 12 questions. The first 11 questions asked parent educators to choose from a list of provided responses. The last question asked parent educators to describe their best practices. Of the 164 parent educators who were reported to have served in the programs this year, 117 (71%) completed the Parent Educator Survey this year. The 117 respondents represented 69 programs. The following section lists participant responses to each of the survey questions. Oklahoma Technical Assistance Center 2009 Page 18 1. How many hours per week do you work as a parent educator? Response Percent of Respondents 1 to 10 hours 12.8% 11 to 20 hours 38.5% 21 to 30 hours 30.8% 31 to 40 hours 14.5% More than 40 hours 3.4% 2. How many hours a week are worked outside the typical work day (9-5, Monday - Friday)? Response Percent of Respondents 1 to 10 hours 86.7% 11 to 20 hours 11.5% 21 to 30 hours 1.8% 31 to 40 hours 0.0% More than 40 hours 0;0% 3. In addition to being a parent educator, are you a certified teacher? Response Percent of Respondents Yes 44.3% No 55.7% 4. If yes, are you currently employed as a full-time teacher (in addition to your OPAT work)? Response Percent of Respondents Yes 20.5% No 79.5% Oklahoma Technical Assistance Center 2009 Page 19 5. How many college courses have you taken in Early Childhood? Response Percent of Respondents None 14.2% 1-2 9.7% 3-5 19.5% 6-8 11.5% 9 or more 45.1% 6. What is the level of education you have completed? Response Percent of Respondents Child Development Associate (this is not a degree, but requires 120 clock hours of training; respondents frequently write it in, so it was included as a response) 6.0% Associate's Degree (A.A.) 14.7% BA or B.S. 53.4% MA or M.S. 19.8% High School 6.0% 7. If you have a college degree, in what field? Response Percent of Respondents Early Childhood 21.3% Elementary Education 31.9% Childhood DevelopmenUFamily Relations 8.5% Social Work 9.6% Other 28.7% Oklahoma Technical Assistance Center 2009 Page 20 Response Percent of Respondents Less than 12 5.1% 12-24 2.6% 25-48 0.9% 49-60 12.8% More than 60 3.4% No response 75.2% 8. If you do not have a college degree, how many college credit hours have you completed? 9. How many years of service do you have in OPAT? Response Percent of Respondents 0-5 years 47.0% 6-10 years 23.1% 11-15 years 15.4% more than 15 years 14.5% 10. How many personal visits are you expected to complete each month? Response Percent of Respondents 1-15 17.9% 16-25 35.9% 26-40 37.6% 41-50 6.8% more than 50 1.7% Oklahoma Technical Assistance Center 2009 Page 21 Response Percent of Respondents 1-15 19.7% 16-25 33.3% 26-40 37.6% 41-50 7.7% more than 50 1.7% 11. How many personal visits do you usually complete each month? The final question on the Parent Educator Survey asked respondents to describe the best practices of their programs. Forty-one parent educators (35%) completed this question. Many of the parent educators gave specific details that described the relationship between the program and partners including descriptions of the goods and services provided by local resources. The parent educators cited the benefits of partnering with the school district and often noted the efforts of the school administration, counselor, alternative school teachers, and community education staff. Many also listed specific programs or materials they used that interested and motivated parent participation. Programs that had been in existence for several years reported that program parents were the best source of program recruitment. One parent educator wrote, Thefamilies that receive our services make our program successful! Word of mouth really increases our enrollment possibilities. Program Coordinator Survey The Program Coordinator Survey was completed by 62 of the 81 program coordinators. The survey consisted of 11 questions, the first 10 questions were forced-choice questions and the eleventh question asked coordinators, What resources or training would benefit you most? The following section lists participant responses to each of the survey questions. Oklahoma Technical Assistance Center 2009 Page 22 1. Where is your program office or headquarters housed? Response Percent of Respondents Elementary School 41.3% Early Childhood Center 22.2% County Health Department 1.6% Non-profit agency 1.6% Special Services building 7.9% Other 25.4% 2. Where do you hold the majority of your parent meetings? Response Percent of Respondents Elementary School 45.9% Early Childhood Center 24.6% County Health Department 4.9% Non-profit agency 1.6% Special Services building 3.3% Other 19.7% 3. What steps did you take to recruit at-risk families? Mark all that apply. Response Percent of Respondents Collaborate with Sooner Start, Dept. of Human Services, and 88.9% Health Dept. Contact alternative education schools for teen parents 76.2% Distribute brochures at public places 84.1% Work with Title I schools or public housing apartments 52.4% OS/GYNs, pediatricians, and hospitals 39.7% Local press coverage or internet 44.4% Word of mouth 93.7% Other 25.4% Oklahoma Technical Assistance Center 2009 Page 23 Response Percent of Respondents Collaborate with Sooner Start, Dept. of Human Services, and 25.8% Health Dept. Contact alternative schools for teen parents 11.3% Distribute brochures at public places 6.5% Work with Title I schools or public housing apartments 14.5% OB/GYNs, pediatricians, and hospitals 1.6% Local press coverage or internet 0.0% Word of mouth 35.5% Other 4.8% 4. Of the steps marked in item three, which one is the primary recruitment method for at-risk families? Mark only one. 5. How much are your parent educators paid per hour? Response Percent of Respondents Less than $9.00 0.0% $9.01 - $12.00 26.2% $12.01 - $15.00 42.6% $15.01 - $18.00 16.4% $18.01 or other 14.8% 6. How many families were typically on waiting list for services this year? Response Percent of Respondents Our program did not have a waiting list 63.9% 5 or fewer families 24.6% 6-10 families 3.3% more than 10 families 8.2% Oklahoma Technical Assistance Center 2009 Page 24 Response Percent of Respondents Yes 10.0% No 90.0% 7. Did your program receive any additional funds this year? 8. If so, how many children were served with the additional funds? Response Percent of Respondents Less than 20 4.8% 20 to 35 3.2% 36 to 50 0.0% 51 to 65 0.0% more than 65 1.6% No response 90.5% 9. Indicate the outside agencies or organizations that your program collaborates with. Mark all that apply. Response Percent of Respondents Head Start 56.6% Sooner Start 79.4% County Health Department 65.1% Dept. of Human Services 41.3% Public Libraries 58.7% Health Clinics 38.1% Civic Organizations 39.7% Other 41.3% Oklahoma Technical Assistance Center 2009 Page 25 10. Choose which auxiliary services you offer. Mark all that apply. Response Percent of Respondents Drop in and Play 52.4% Toy/book lending library 82.5% Newsletter 79.4% Reading/Story time 36.5% Clothing/supply closet 38.1% Other 26.6% 11. What resources or training would benefit you most? Sixteen of the program coordinators responded to this question. Representative responses included: • I think all Parent Educators should have training in car seat safety. • Trainingfor families with special needs children. • How to support a parent who has lost a child. • Spanish-speaking families - how to interpret and work with them. • Resources for funding to provide books for children and supplies for projects. • Always group meeting activities ... how to encourage participation. Survey Summary Each year, evaluation staff review the instruments and data collection procedures and make appropriate revisions to improve the quality of the information collected and to address specific program issues and concerns. These yearly revisions do not allow for a complete, multi-year comparison of survey results; however, those that remained consistent are presented in Table 5. Oklahoma Technical Assistance Center 2009 Page 26 Table 5. Survey items by year. Question 2004·05 2005·06 2006·07 2007·08 2008·09 Parent Educators - Degree status of B.A.lB.S. or more? 76.4% 71.1% 67.5% 71.3% 79.2% Parent Educators - Six or more years of service in PAT? 42.4% 40.9% 45.8% 51.5% 53.0% Parent Educators - Work more than 20 hours per week? 46.2% 46.3% 44.2% 55.3% 48.7% Program - Waiting list? 42.9% 40.4% 41.4% 52.2% 36.1% The results from the parent educator and coordinator surveys suggested that OPAT programs have stable, well-educated staff in the field. More than half of the parent educators surveyed (53.0%) indicated that they had been working in an OPAT program for six or more years and more than three-fourths have at least a B.A.IB.S. degree. The data in Table 5 also indicated that approximately half of the parent educators are working more than half-time. The percentage of parent educators working more than 20 hours a week decreased from 55.3 percent last year to 48.7 percent this year. A substantial change was noted in the percentage of programs that had waiting lists. Only 36.1 percent of the project coordinators reported they had families on a waiting list, the lowest percentage in the last five years. Over a third of the parent educators indicated that they made between 26-40 personal visits per month (37.6%). Parent educators that made between 16-25 visits accounted for 33.3 percent of the group. Most of the parent educators indicated that they met the expectation set forth by the program requirements. The OSDE set the goal for projects to meet their number of minimum monthly visits at least eight months out of the program year. Factors such as late start up and winter weather sometimes caused programs difficulty in meeting the minimum each month. This year, 65.5 percent of the programs either met or exceeded this goal. Almost all of the programs indicated that they made efforts to recruit at-risk families such as contacting local agencies, physicians, and alternative schools. Word of mouth was the Oklahoma Technical Assistance Center 2009 Page 27 most frequently reported recruiting method for programs (35.5%) followed by collaboration with Sooner Start, Department of Human Services, and the Health Department. Elementary schools continued to be the most popular place to house the programs (41.3%) and to hold parent meetings (45.9%). Goal Achievement: Individual Programs Table 6 presents program information including whether each program met goals for personal visits and developmental screening. The table lists the number of visits each program was required to complete each month and the number of months each program met that standard. A column is also included that lists the total number of teen consultations (teen consultations are not included in the number of personal visits). Programs that served a substantial number of teens may not show that they attained their monthly minimum until teen consultations are taken into account. Information on developmental screening is presented on the right side of Table 6. It shows the number of children screened during the 2008-09 school year, the number of children "excused" from the screening requirement because they were too young (less than four months of age) or had not received at least two visits, the number of children who should have been screened but were not. Finally, the percentage of eligible children who were screened is listed. Oklahoma Technical Assistance Center 2009 Page 28 Table 6. 'Goal achievemenfd~ia by program."":)!;"" " ': ,:,~;;,,~ '!if ~~; ,,;,,% ~~ !,'\ii",,' !'",%l '!!;1~'!" ",'I' li!i t~, DistrictN,a~e ",',' ,,~ , Ij;,'tfof .. ,* :Total #;of" H, " ,,>' '," ~~f:~"' R,i 'i' Total #';/ 1*';Monthly . ,~#'of " ,~::lof>" percentage ~~;"'''f "i' , Months'~;~ i\0,~~' " Minimu'm'j ',,! I,een 'ir ,;,_Child~en,J;;,Ii" ~~~ O',\lli; Ii";' , 9hildren" Children~ ~;Screenedi !f~~;;: #Children' j!~,¥isits ~~, Minimum,'!' Consultation :i~~r~,~n~d;, '.":"riot ,,'Excused ~ 1:1:: ,',\\i,,~, 1S~rVe~~: ~!S1t~l~'~~~ ,iJ,;'ii, ," ,~; ",!; ;;S,~!ee-ried, from ,; i,: ',,"",;,!! i:'~~~~;~i:~~~~Jii;jg'", =:», ,,(,it S¢reening jill'ill "(i', =~:'l1 ,; " ",' 1(, ,l! ,,'f' @' f,y!! ·'.,f,!':!'; -"ii . 'f " '" , -" '" ' , Ada 44 36 8 0 43 0 1 100% Altus 48 36 2 1 26 13 9 73% Anadarko 67 36 10 0 61 2 4 97% Ardmore 85 58 8 0 80 1 4 99% Avant, Bowring, Osage Hills, South 44 22 9 0 39 0 5 100% Coffeyville Bartlesville 122 58 10 69 94 8 20 93% Bethany 57 36 10 0 53 0 4 100% Binger-Oney, Gracemont 34 22 10 0 32 0 2 100% Bixby' 80 58 5 0 67 2 11 98% Bristow 40 36 0 0 30 0 10 100% Broken Arrow 136 81 9 4 130 0 6 100% Caney Valley, Copan 60 36 10 0 52 0 8 100% Chouteau-Mazie 59 36 10 0 56 0 3 100% Claremore 57 36 8 0 48 3 6 95% Coalgate 15 22 0 0 11 4 0 73% Comanche 37 22 10 0 26 3 8 92% Dewey 60 36 10 0 52 2 6 97% Durant 55 36 9 38 45 2 8 96% EI Reno' 15 36 0 43 12 2 1 87% Enid 218 58 11 8 165 22 31 90% Fairview 24 22 3 0 14 8 2 67% Fort Gibson' 25 36 0 0 20 1 4 96% Geary, Maple 24 22 1 17 20 2 2 92% Glenpool 59 36 9 0 54 3 2 95% Grove, South Rock Creek 41 22 10 0 39 1 1 98% Guthrie 87 58 7 0 74 1 12 99% Haworth 28 22 1 0 20 4 4 86% Heavener 27 22 9 0 24 0 3 100% Hobart 38 22 8 0 29 2 7 95% Hominy 43 22 9 0 34 0 9 100% Hugo 56 36 10 0 54 0 2 100% Hulbert' 13 22 0 8 0 9 4 31% Idabel 54 36 4 2 46 5 3 91% Jenks 199 58 10 67 150 13 36 93% Kingfisher 39 36 0 12 35 0 4 100% Little Axe 34 22 10 4 30 0 4 100% Oklahoma Technical Assistance Center 2009 Page 29 Table 6. Goal achievement data by program. District Name Total # Monthly #of Total # of #of #of #of Percentage of Minimum I" Months Teen Children Children Children Screened Children Visits Minimum Consultation Screened not Excused Served Visits Met Screened from Screening Locust Grove 53 36 10 174 45 3 5 94% Maryetta 34 22 9 0 29 1 4 97% McAlester 61 36 10 0 49 1 11 98% Meeker 21 22 0 9 20 1 0 95% Midwest City-Del City 128 70 8 4 110 0 18 100% Minco 33 22 7 0 27 0 6 100% Morrison 33 22 9 0 28 0 5 100% Muldrow 52 36 8 0 41 0 11 100% Newkirk 46 22 9 0 37 0 9 100% Noble 56 36 10 0 52 0 4 100% Norman 100 81 0 275 64 10 26 90% Okay 30 22 10 92 28 0 2 100% Oklahoma City Housinc 122 60 10 0 115 0 7 100% Oklahoma City State 184 144 9 2 171 0 13 100% Pawhuska 39 22 9 0 35 0 4 100% Perkins-Tvron 58 36 9 0 57 1 0 98% Perry 48 36 4 0 42 0 6 100% Pioneer-Pleasant Vale" 20 22 0 0 15 0 5 100% Pocola, Arkoma 54 36 7 23 48 0 6 100% Porum 24 22 1 0 20 1 3 96% Poteau 72 36 10 138 63 0 9 100% Pryor 58 36 10 0 56 0 2 100% Putnam City 230 108 10 23 214 0 16 100% Quinton 8 22 0 0 0 8 0 0% Salina 30"?- . 22 10 0 24 0 6 100% Sand Sprincs 52 36 10 0 39 4 9 92% Sapulpa 41 36 2 0 31 8 2 80% Shawnee 75 58 8 0 47 2 26 97% Shidler, Frontier 23 22 0 3 19 1 3 96% Skiatook 39 36 1 0 36 0 3 100% Sulphur 58 36 8 0 50 0 8 100% Swink: Fort Towson, Forest Grove 30 22 7 0 26 0 4 100% Tahlequah 77 36 10 7 65 0 12 100% Tecumseh 63 36 8 5 51 1 11 98% Tulsa Housinq 93 60 11 1 79 0 14 100% Tulsa State 287 144 10 0 250 0 37 100% Union 146 81 10 79 134 0 12 100% Oklahoma Technical Assistance Center 2009 Page 30 Table 6. Goal achievement data by program. District Name Total # Monthly #of Total # of #of #of #of Percentage of Minimum Months Teen Children Children Children Screened Children Visits Minimum Consultation Screened not Excused ?, I""Served I,' " Visits M~l Screened Scrferoemning " Verdiqris 46 36 2 0 43 0 3 100% Vian 34 22 9 0 27 0 7 100% Watts, Peavine 32 22 9 19 28 1 3 97% Wellston 38 22 9 0 35 0 3 100% Westville 40 36 0 0 31 5 4 88% Woodland, Kaw City, Wynona 50 22 9 0 49 0 1 100% Woodward 53 36 5 1 42 0 11 100% Wriqht City, Glover 32 22 9 0 29 0 3 100% * New programs Oklahoma Technical Assistance Center 2009 Page 31 -------_\ APPENDIX Oklahoma Technical Assistance Center 2009 Table A1. Program Reporting Dis0'0trict ';':~t~!;:,;1"t:i':~~."F;>~c:!ren~t~,I'.~jP~:ro",g~.ra.s,m~··f' M:o:nth~ly ,~' ~"r ,<~'t,,,, ~ ' . " ',:", 'Educator Coordirm1:or' ReR,ort ,"i? '.ii'\\,',,i: !p'!,,>,! ,.,.: Survey k Survey ',~l " #4' Ada 1 1 X Altus X Anadarko 3 1 X Ardmore 1 1 Avant, Bowring, Osage Hills, South Coffeyville 2 1 X Bartlesville 2 1 X Bethany 1 1 X Binger-Oney, Gracemont 1 1 X Bixby* 2 1 X Bristow 1 1 X Broken Arrow 4 1 X Caney Valley, Copan X Chouteau-Mazie 1 1 X Claremore 1 1 X Coalgate X Comanche X Dewey X Durant 1 X EI Reno* 2 1 X Enid X Fairview X Fort Gibson* 1 1 X Geary, Maple 1 1 X Glenpool 2 1 X Grove, South Rock Creek 1 1 X Guthrie 3 1 X Haworth 1 1 X Heavener 1 1 X Hobart 1 1 X Hominy 1 X Hugo 1 1 X Hulbert* 1 1 X Idabel 1 1 X Jenks 5 1 X Kingfisher 1 1 X Little Axe 1 1 X Locust Grove X Maryetta 1 1 X McAlester 2 1 X Meeker 1 1 X Oklahoma Technical Assistance Center 2009 Table A1. Program Reporting "\',' ,j, ?i~:'" ,'~ ,';~;~'8l"": ~'" ," :r" , '" I,; Program -", District - , Parent Monthly - , "Educator Coordinator 1~1,Report ,v'", ", ".;;;; y ':~,,) :'@';\ii" "i~~;, ';~%t",;", "i\'! Survey Survey Midwest City-Del City 2 X Minco 1 1 X Morrison 2 1 X Muldrow 1 1 X Newkirk 1 X Noble 1 1 X Norman 4 1 X Okay 1 1 X Oklahoma City 6 1 X Pawhuska X Perkins-Tryon 1 1 X Perry 1 1 X Pioneer-Pleasant Vale* 1 1 X Pocola, Arkoma 1 1 X Porum 1 1 X Poteau 1 1 X Pryor 1 1 X Putnam City 5 1 X Quinton 1 1 Salina 1 1 X Sand Springs 2 1 X Sapulpa 1 X Shawnee 2 1 X Shidler, Frontier 1 X Skiatook 1 1 X Sulphur 4 2 X Swink,* Fort Towson, Forest Grove 1 1 X Tahlequah 2 1 X Tecumseh 1 1 X Tulsa 10 1 X Union 1 1 X Verdigris 1 1 X Vian 1 1 X Watts, Peavine 1 1 X Wellston 1 X Westville 2 1 X Woodland, Kaw City, Wynona X Woodward 1 1 X WriQht City, Glover 1 1 X Oklahoma Technical Assistance Center 2009 • -~ OTAC Oklahoma Technical Assistance Center 123 E. Broadway Phone Cushing, Oklahoma 74023 918.225.1882 July 13, 2009 Dr. Ramona Paul Assistant State Superintendent Oklahoma State Department of Education 2500 North Lincoln Boulevard Oklahoma City, OK 73105 Dear Dr. Paul, We have compiled and analyzed the data submitted by the 81 Oklahoma Parents as Teachers (OPAT) grantees throughout the school year. We have worked with each of the programs to ensure timely and consistent reporting of activities. This year, only one of the 81 programs, Quinton, did not provide a full year of data. The Quinton site reported that they operated a program this year and monthly data was provided for August through the month of November. Multiple attempts by OTAC staff to acquire information for the remaining months were unsuccessful. Attached is a table that lists the number of months that each program met its monthly minimum requirements and the percentage of children who received developmental screening. We have preliminary totals through June. We have not thoroughly reviewed the data and contacted local programs to resolve any discrepancies, so the numbers reported here are preliminary. The 81 programs served a total of 5,050 children this year. The number of separate families served was 4,379. They completed 33,739 personal visits - these are personal visits only; the total does not include teen consultation visits. A total of 1,397 parent group meetings were reported with a total attendance of 11,320 parents - an average of 17.46 meetings per grantee and an average attendance of 8.10 parents per meeting. The 81 programs served fewer children and families compared to last year. The 78 programs funded in 2007-08 served a total of 4,650 families representing 5,355 children. Services to these families included 34,751 personal visits and 1,349 parent group meetings. In total 11,376 parents attended these meetings. The grantees averaged 17.9 meetings during the year ~ith a mean 8.43 parents in attendance. A decrease was noted in the number of children and families served this year compared to 2007-08 even though three more programs were funded. This may be due to the SDE's decision to lower the minimum number of required families and visits by 10 percent. This reduction in program requirements was made in response to the increase in travel costs that affected all programs, but, particularly those in small, rural areas. The remainder of this report accounts for the programs in one of three listings: Oklahoma Technical Assistance Center 2009 (1) Recommended for continued funding: the program met the minimum requirements and screened all eligible children. (2) Recommended for continued funding with reservations: the program data suggested issues that prevented full program compliance. (3) Not recommended for continued funding: the program suffered implementation issues that would require a plan of improvement. (1) Recommended for continued funding: Ada Avant, Bowring, Osage Hills, South Coffeyville Bethany Binger-Oney, Gracemont Broken Arrow Caney Valley, Copan Chouteau-Mazie Grove, South Rock Creek Heavener Hominy Hugo Mid-Del Minco Morrison Muldrow Newkirk Noble Okay Oklahoma City Oklahoma City Housing Pawhuska Perkins- Tyron Poteau Pryor Putnam City Salina Sulphur Swink, Fort Towson, Forest Grove Tahlequah Tecumseh Tulsa Union Vian Wellston Woodland, Kaw City, Wynona Wright City, Glover (2) Recommended with reservations: Oklahoma Technical Assistance Center 2009 • Altus is recommended with reservations for the third year in a row. They met the minimum service requirements in two of the months and screened 73 percent of eligible children. The transient nature of the population in Altus continues to be an obstacle for the program. This year, they experienced a 36 percent turnover rate in families served. • Bixby was a new program this year. They met their minimum for five months and screened 98 percent of the children. • Bristow, like last year, did not meet the service minimum in anyone month, although they screened 100 percent of their children. They routinely provide two-thirds of the required visits for this size of grant. A grant award that requires a lower participation level may be appropriate. • Coalgate did not meet the service requirement in any month this year. Twenty-two visits per month were required for Coalgate's funding level; the greatest number of visits completed in a month was only 13. The program screened 94 percent of eligible children. • EI Reno was a first-year program. Less than one-third of the required number of visits were completed; 87 percent of eligible children were screened. • Fort Gibson was a new program this year and did not receive funds until later in the year. On average, they completed only half of the required monthly visits although their number of visits increased during the year. They screened 96 percent of eligible children. • Geary-Maple met the service requirement for one month. They averaged half of the required minimum visits per month. They screened 92 percent of eligible children. This program provided services for 11 months. • Haworth participated in a cooperative with Wright City in years past. Now, as a single site, the program met its service requirement in only one month. Examination of records indicated that they lacked only one or two visits each month to meet the minimum expectancies. The program screened 86 percent of eligible participants. • Hulbert was a new program this year. Their number of visits increased during the year although they did not meet their service requirement in any month. Only 31 percent of eligible children were screened. • Idabel had a new parent educator/program coordinator this year that resulted in a delay in full implementation of the program during the first semester. They met the service minimum in four of the months (spring semester) and screened 91 percent of eligible children. • Kingfisher is recommended with reservations for the fourth year in a row. The minimum service requirement was not met in any month this year. A full 100 percent of eligible children were screened. A grant award that requires a lower participation level may be appropriate. • Meeker showed improvement this year in increasing the number of monthly visits, although the number was slightly below the monthly minimums. The program screened 95 percent of eligible children. • Norman is recommended with reservations for the second year. The program screened 90 percent of eligible children. The number of monthly visits remained substantially below required minimums. A grant award that requires a lower participation level may be appropriate. Oklahoma Technical Assistance Center 2009 • Perry met their minimum in four months and barely missed the minimum in the remaining months. All of the eligible children were screened. • Pioneer-Pleasant Vale, a new program, showed growth in the number of visits during the year. The most visits completed in anyone month were 19, short of their goal of 22. All eligible children were screened. • Porum screened 96 percent of eligible children. Service requirements were met in two months. The parent educator experienced health problems that affected services during the year. • Shidler, Frontier operated as a member of a cooperative in 2007-08. This year, they did not meet their month minimum in any month. Records indicated that 96 percent of the eligible children were screened. • Skiatook restructured the program this year with the arrival of a new parent educator. Improvement was noted in the number of required visits; however, the minimum was not met until June. All eligible children were screened. • Tulsa Housing met their minimum in three of the months. The program appeared to have difficulty in completing visits during the winter months although number of visits completed was near their requirement of 54. A full 100 percent of eligible children were screened. • Verdigris screened all eligible children. The minimum was met two months during the year and fell short only one or two visits in the remaining months. • Westville did not meet the required minimum in any month and was substantially short in meeting the requirements. Only 86 percent of eligible children were screened. A grant award that requires a lower participation level may be appropriate. • Woodward met the minimum in five months and was slightly low in the remaining months. The program screened 100 percent of eligible children. (2.a) The following programs are also recommended with reservations. These programs met their monthly targets but failed to screen all eligible children. These programs screened at least 90 percent of eligible children. Anadarko Ardmore Bartlesville Claremore Comanche Dewey Durant Enid Glenpool Guthrie Hobart Jenks Little Axe Locust Grove Maryetta McAlester Pocola, Arkoma Sand Springs Oklahoma Technical Assistance Center 2009 Shawnee Watts, Peavine • Fairview has experienced difficulties for the last three years in meeting the service requirements. This year, the minimum requirement was met for three months. Like last year, only two-thirds of the eligible children were screened. The director reported that other children have been screened; however, no records were provided of this activity. • Quinton has experienced multiple difficulties over the last two years. In 2007-08, it failed to meet its targets in any month. This year, data was only reported for four months. Only eight visits were reported per month. Numerous attempts to contact the program coordinator and site superintendent did not result in the submission of program records. • Sapulpa is not recommended for funding for a second year in a row. This is the fourth year the program has not met the minimum service requirements. In 2008- 09, minimum requirements were met in two of the months. This is also the fourth year that the program has failed to screen all of the eligible children; 83 percent of children were screened this year. (3) Programs not recommended for continued funding [depending upon the circumstances (e.g., first-year programs), the SDE may require programs to submit a plan of improvement in order to receive continued funding]: Please contact me if you have any questions or concerns. Sincerely, Dr. Kathy McKean, Director Oklahoma Technical Assistance Center 2009 |
Date created | 2011-07-12 |
Date modified | 2011-10-28 |
Tags
Add tags for Oklahoma Parents as Teachers annual program Evaluation