ECHO child behavioral health survey for SoonerCare Choice : executive summary and technical specifications.

              Oklahoma Health Care Authority
ECHO® Child Behavioral Health Survey
for SoonerCare Choice
Executive Summary and
Technical Specifications
Report for Contract: State Fiscal Year 2010
Data Reviewed: December 1, 2008 – November 30, 2009
Submitted: June 2010
Submitted by:
APS Healthcare
4545 North Lincoln Boulevard
Suite 24
Oklahoma City, Oklahoma 73105
(405) 556-9700
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 2
ECHO® Child Behavioral Health Survey for SoonerCare Choice
Executive Summary
The Oklahoma Health Care Authority (OHCA) is the state’s single agency
responsible for administering Medicaid. The managed care component of this
program is known as SoonerCare Choice, which operated under a partially capitated
case management system during the first half of state fiscal year (SFY) 2009. A
patient-centered medical home model was implemented January 1, 2009. In order to
evaluate service satisfaction, the OHCA contracted with APS Healthcare (APS) to
survey children enrolled in SoonerCare Choice who accessed outpatient behavioral
health services between December 1, 2008, and November 30, 2009. This survey is
conducted annually alternating between the adult and child populations.
Comparing the 2008 survey and the 2010 survey, results indicated fairly high
levels of satisfaction holding steady across an array of 21 quality measures. One
measure showed a statistically significant difference between 2008 and 2010:
• The composite measure “As far as you know did anyone your child saw for
counseling or treatment share information with others that should have been
kept private?” had a significant decrease of members who reported “yes” from
6.7% in 2008 to 2.8% in 2010.
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 3
ECHO® Child Behavioral Health Survey for SoonerCare Choice
Technical Specifications
Consumer satisfaction surveys are an important source of information to
consumers, purchasers of health care, health plans and program administrators. By
responding to satisfaction surveys, individuals provide valuable information regarding
access to care, use of services, and satisfaction with the care they have received. The
practice of surveying the general population of managed care members about their overall
experience with their health plan has extended to surveying special populations. This
special population survey detailed responses of members enrolled in SoonerCare Choice
who received outpatient behavioral health services through the SoonerCare program.
The Oklahoma Health Care Authority (OHCA) is the state’s single agency
responsible for administering Medicaid. The managed care component of this program is
known as SoonerCare Choice, which operated under a partially capitated case
management system during the first half of the state fiscal year (SFY) 2009. A patient-centered
medical home model was implemented January 1, 2009. Behavioral health
services are available to SoonerCare Choice members. In order to evaluate service
satisfaction, the OHCA contracted with APS Healthcare to survey members enrolled in
SoonerCare Choice who accessed outpatient behavioral health services between
December 1, 2008, and November 30, 2009. This technical report includes information
on the survey methodology, findings, and summary information on the respondents’
satisfaction with behavioral health services received.
The OHCA annually administers the Experience of Care and Health Outcomes
(ECHO) survey, version 3.0, to measure members’ satisfaction with behavior health
services. The methodology for this survey was based on the Consumer Assessment of
Healthcare Providers Systems (CAHPS®). The CAHPS surveys, administration protocol
and survey analysis were developed by Harvard, RAND and the Research Triangle
Institute. The ECHO survey was developed by behavioral health consumers, clinicians
and behavioral health policy experts, including representatives from the National
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 4
Committee for Quality Assurance’s (NCQA) Behavioral Health Measurement Advisory
Panel, the Center for Mental Health Services, and the National Alliance for the Mentally
Ill. On alternating years the OHCA surveys adults about their experiences with
behavioral health treatment and parents/guardians about their children’s behavioral health
care. For SFY 2010 the survey focused on services provided to children; comparisons
were made with results from SFY 2008, the last year during which SoonerCare Choice
children were measured by the ECHO survey.
Method
The Child ECHO Health Plan survey 3.0 questionnaire (shown in Appendix A)
was administered by The Myers Group (TMG) between December 1, 2009, and March
31, 2010. The survey covered several aspects of behavioral health services, including:
• Access to care
• Receiving care without long waits
• Communication with clinicians
• Family involvement in care
• Perceived improvement in functioning
• Patient’s rights
• Experiences with the health plan
The questionnaire also asked respondents to give overall ratings of the counseling
or treatment they received and SoonerCare Choice.
Sampling
Because it was not feasible to survey the entire SoonerCare Choice population,
recognized sampling techniques were used to obtain information from a limited number
of members. This information was used to estimate the consumer satisfaction of the
SoonerCare Choice population as a whole. Sampling and fielding of the survey was
conducted by subcontract with a nationally certified CAHPS survey firm, TMG.
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 5
Sampling for this survey followed CAHPS 3.0 protocols. APS provided TMG
with a list of eligible members for sample selection. Each member included in the
eligibility list met the following criteria:
• 17 years of age or younger as of November 30, 2008;
• Enrolled in the SoonerCare Choice program as of November 30, 2009 and
• Continuously enrolled in SoonerCare Choice for 12 months between December 1,
2008, and November 30, 2009. (Continuous enrollment was defined as having no
more than one 45-day break in enrollment during the year.)
In addition, APS limited the pool of members to those who had paid claims for
outpatient behavioral health services between December 1, 2008, and November 30,
2009, in hopes of improving response rates. In accordance with CAHPS
recommendations and to reduce the burden on respondents, TMG randomly selected only
one member from a household for the sample. The ECHO Survey and Reporting Kit 3.0
standards indicate that a minimum of 411 completed surveys were needed for a valid
administration. Estimating a response rate of 40% and adjusting 15% for the
underreporting of behavioral health services received, the recommended sample was
1,183 members. The total number of SoonerCare Choice members eligible for the survey
was 17,202; TMG selected a random sample of 2,000 members.
Data Collection
The CAHPS survey methodology allowed data to be collected by mail, telephone
interview, or a combination of mail and telephone. The OHCA and APS agreed to
conduct the survey using mail, and incorporated telephone interviewing only when the
targeted response rate was not received. The survey process had six outreach
interventions that included mailings of the survey packet, reminder postcards, and phone
follow-up with bilingual interviewers.
After the sample was selected, address lists were processed using the U.S. Postal
Service’s CASS Certified ZIP + 4 Coding Software, which verified the zip code for each
address. Each packet included a questionnaire and a letter that explained the purpose and
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 6
the importance of the survey. To encourage participation, the packet also contained a
postage-paid business reply envelope. Reminder postcards served to thank individuals
who had responded to the survey and to remind others to complete their questionnaires.
Members selected for the sample who did not respond to the first survey and/or postcard
mailings were mailed a second survey. All correspondence included a toll-free number
that members could use to contact TMG with questions regarding the survey or to
complete the survey over the telephone. Each information letter contained a statement
written in Spanish asking the member to call the toll-free number to take the survey by
phone with a Spanish interpreter. If needed, follow-up calls were then initiated with
bilingual interviewers.
TMG developed a database to track the status of members selected for the sample
at each stage of the survey protocol. The database identified members who had not
returned the survey and needed subsequent mailings. The database also indicated the date
that a member responded to the survey, refused to participate in the survey, or was
determined to be ineligible for the survey. Members were determined to be ineligible for
the survey if they no longer qualified for SoonerCare or had moved to a another state.
The criteria for determining a complete ECHO survey was taken from Article X
of the CAHPS Survey and Reporting Kit entitled, “Determining a Complete ECHO
Questionnaire.” According to those criteria, if 9 of 18 key items were appropriately
answered, the survey was considered complete. Only surveys that met these criteria were
included in the analysis.
Data Coding and Data Entry
TMG recorded the responses to the questionnaires in a database specifically
designed for this purpose. The data entry program permitted the entry of only those
responses that were within the accepted range for each specific item; for example, if a
rating was supposed to be on a scale of 0 to 10, a response of 12 could not be entered.
APS performed a data clean-up process prior to the actual analysis to detect any
additional out-of-range values and response inconsistencies. Typically, inconsistencies
occurred when respondents did not follow the skip pattern of question groups. Whenever
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 7
feasible, APS recoded the items to conform to the questionnaire skip patterns; for
example, if one response indicated the person did not receive forms to fill out, then any
subsequent responses to questions about the ease of completing the forms were discarded.
However, when the intent of the respondent could not be determined, the analyst recoded
the item as missing.
Data Analysis
All analyses and calculations in the study were performed by APS using SAS
Version 9.1. Frequencies were computed for all the items on the survey. Due to the
lengthy results for this type of descriptive statistics, the frequency distributions are
reported in Appendix B rather than in the results section.
The main analysis compared this year’s survey results with those from the
previous administration of the survey, which was in SFY 2008. This analysis produced
three types of results: global ratings, composite measures, and individual items.
Global ratings, which used a scale of 0 to 10, measured the respondents’
assessment of their health plan and the quality of the care received. APS computed the
means for each global rating and used t-tests to compare means from the SFY 2008 and
SFY 2010 SoonerCare Choice surveys. The two global ratings compared were:
• Overall rating of treatment or counseling received by the child
• Overall rating of the health plan.
Composite measures combined the responses to questions that were closely
related to each other and provided more comprehensive and meaningful results than
comparing each item separately. APS computed the mean scores for each composite and
used t-tests to compare means from the SFY 2008 and SFY 2010 SoonerCare Choice
surveys. The five composite measures were compared were:
• Experiences in getting treatment quickly
• Experiences with how well clinicians communicate
• Experiences in getting treatment and information from the plan
• Perceived improvement from treatment
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 8
• Availability of professional help and support
Individual items provided meaningful results concerning specific issues. For items
with multiple responses, APS computed the mean scores for each individual item and
used t-tests to compare results from the SFY 2008 and SFY 2010 SoonerCare Choice
surveys. For items with dichotomous “yes/no” responses, APS computed chi-squared
tests to compare results. The individual items that could be compared were:
• Appointment waiting time
• Amount child was helped by treatment
• Informed about medication side effects
• Discussed goals of treatment
• Discussed treatment options
• Informed about condition management
• Informed about patient rights
• Ability to refuse treatment options
• Health information privacy
• Responsiveness to cultural needs
• Reasons for seeking treatment
Results
Table 1 shows the response by method for the current year’s survey and the
previous administration of the survey in SFY 2008.
Table 1. Completed Survey Response by Method and Year
Completed Surveys SFY 2010 Survey SFY 2008 Survey
Mailing 519 433
Phone Follow-Up 114 114
Total Completed Surveys 633 547
The study evaluated each returned questionnaire to determine if it met the CAHPS
definition for a completed survey. To be considered complete, a questionnaire must have
had appropriate responses to at least nine of 18 key questions identified by CAHPS. APS
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 9
coded returned questionnaires that did not pass the completion criteria as non-respondents
and excluded them from the analysis. During SFY 2010, 380 surveys were
deemed ineligible due to not meeting the criteria for completion.
Table 2 shows the number of ineligible survey respondents by disposition.
Table 2. Number of ineligible surveys for SFY 2010
Ineligible Disposition N %
Did not meet criteria for completion 380 100%
Total Ineligibles 380 100%
The adjusted response rate, expressed as a percentage, is the number of completed
surveys divided by the number of eligible surveys.
Number of completed surveys
Number of eligible surveys
x 100 =
Adjusted
response rate
(Total completed surveys) 633
(Sample) –(Ineligible)
=
2000 – 380
x 100 = 39.1%
The adjusted response rates for the current SoonerCare Choice survey was 39.1%; the
adjusted response rates for the previous SoonerCare Choice survey was 32.0%. The
difference in response rates probably is attributable to the decision in SFY 2010 to limit
the sampling frame to participants who had a paid claim with a behavioral health provider
during the sample timeframe.
Demographics
Respondents were asked their race and ethnicity. Figure 1 shows the results.
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 10
Figure 1. ECHO Respondents’ Racial Identity
Caucasian
61.3%
American
Indian
10.4%
Multiracial
/ Other
18.0%
Asian
0.2%
African
American
10.2%
Results showed that the majority of participants (61.3%) were Caucasian. African
Americans and American Indians were equally represented (10% each), and the
remaining members reported another race or multiple races. In response to a separate
question about ethnicity, 9.9% of respondents reported to be Hispanic or Latino.
Global Ratings
In Figures 2 and 3, the overall ratings are shown for the following:
• Overall rating of treatment or counseling received
• Overall rating of the health plan providing behavioral health services
The means for the global rating items for the two years are shown in bar graphs. The
table below each graph lists the number of respondents analyzed, the means, and the p-value
associated with the statistical comparison between the two years, using a t-test. A
p-value less than .05 was considered statistically significant. That is, if the p-value was
less than .05, the two means were statistically different from each other; otherwise, the
two means were not statistically different.*
* APS used SAS PROC TTEST to compute the significance of difference between years. Reported p-values
are from pooled tests with equal variances assumed: no items failed a test of equal variances.
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 11
Overall Rating of Treatment or Counseling Received
• Using any number from 0 to 10, where 0 is the worst counseling or treatment
possible and 10 is the best counseling or treatment possible, what number would
you use to rate all your child’s counseling or treatment in the last 12 months?
Figure 2. Overall Rating of Treatment or Counseling Received
7.83
7.84
0 1 2 3 4 5 6 7 8 9 10
2008
2010
Mean
Figure 2 demonstrates that SoonerCare Choice members remained fairly satisfied with
the counseling and treatment received in 2010 compared with the ratings in 2008. The
slight increase was not statistically significant.
Year N Mean p-value
2010 620 7.84
2008 287 7.83
.9371
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 12
Overall Rating for Health Plan on Providing Behavioral Health Services
• Using any number from 0 to 10, where 0 is the worst health plan possible and 10
is the best health plan possible, what number would you use to rate your child’s
health plan for counseling or treatment?
Figure 3. Overall Rating for Health Plan on Providing Behavioral Health Services
8.19
8.20
0 1 2 3 4 5 6 7 8 9 10
2008
2010
Mean
Year N Mean p-value
2010 616 8.20
2008 349 8.19
.9910
Figure 3 shows that SoonerCare Choice members were satisfied with the provision of
behavioral health services in 2010. The slight increase from 2008 was not statistically
significant.
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 13
Composite Measures
In Figures 4 through 8, the following composite measures are provided:
• Experiences in getting treatment quickly
• Experiences with how well clinicians communicate
• Experiences in getting treatment and information from the plan
• Perceived improvement from treatment
• Availability of help and support
The percentage in each category for the composite measures is shown in the bar
graph. The table below each graph lists the number of respondents analyzed, and the
mean and the p-value associated with the statistical comparison between the two years.
The smallest categories are combined for display purposes only, so that percentages can
be shown for each category in the graph. See footnotes below each table on how recoding
was performed.
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 14
Getting Treatment Quickly
• In the last 12 months, how often did you get the professional counseling your
child needed on the phone?
• In the last 12 months, when your child needed counseling or treatment right
away, how often did he or she see someone as soon as you wanted?
• In the last 12 months, not counting times your child needed counseling or
treatment right away, how often did your child get an appointment for counseling
or treatment as soon as you wanted?
Figure 4. Getting Treatment Quickly
31.50%
32.71%
24.64%
23.50%
43.86%
43.79%
0% 20% 40% 60% 80% 100%
2008
2010
Never/Sometimes Usually Always
Year N Mean* p-value
2010 549 3.00
2008 294 3.01
.3195
* Responses of “Never” and “Sometimes” were recoded to 1, responses of “Usually” were recoded to 2 and
responses of “Always” were recoded to 3.
Figure 4 shows that 67.3% of the respondents in 2010 reported usually or always
receiving treatment quickly. This was a decrease from 2008 when 68.5% of respondents
reported usually or always receiving treatment quickly. The mean difference was not
statistically significant.
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 15
How Well Clinicians Communicate
• In the last 12 months, how often did the people your child saw for counseling or
treatment listen carefully to you?
• In the last 12 months, how often did the people your child saw for counseling or
treatment explain things in a way you could understand?
• In the last 12 months, how often did the people your child saw for counseling or
treatment show respect for what you had to say?
• In the last 12 months, how often did the people your child saw for counseling or
treatment spend enough time with you?
• In the last 12 months, how often were you involved as much as you wanted in your
child’s counseling or treatment?
Figure 5. How Well Clinicians Communicate
11.63% 19.56% 68.81%
11.54% 21.04% 67.42%
0% 20% 40% 60% 80% 100%
2008
2010
Never/Sometimes Usually Always
Year N Mean* p-value
2010 633 3.53
2008 290 3.55
.8788
* Responses of “Never” and “Sometimes” were recoded to 1, responses of “Usually” were recoded to 2 and
responses of “Always” were recoded to 3.
Figure 5 shows that 88% of members felt that clinicians usually or always communicated
well during 2010, the same as 2008. The mean difference was not significant.
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 16
Access to Treatment and Information from the Health Plan
• Since your child joined this health plan, how much of a problem, if any, was it to
get someone for your child you are happy with?
• In the last 12 months, how much of a problem, if any, were delays in counseling
or treatment while you waited for approval from your child’s health plan?
• In the last 12 months, how much of a problem, if any, was it to get the counseling
or treatment you thought your child needed?
• In the last 12 months, how much of a problem, if any, was it to find or understand
information (about counseling or treatment)?
• In the last 12 months, how much of a problem, if any, was it to get the help you
needed for your child when you called the health plan’s customer service?
• In the last 12 months, how much of a problem, if any, did you have with
paperwork for your child’s health plan?
Figure 6. Access to Treatment and Information from the Health Plan
24.25%
23.89%
66.56%
65.99%
9.18%
10.12%
0% 20% 40% 60% 80% 100%
2008
2010
Big Problem Small Problem Not a Problem
Year N Mean p-value
2010 618 2.56
2008 351 2.57
.6891
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 17
Figure 6 demonstrates that 89.9% of respondents indicated no problem with access to
treatment and information in 2010, compared with 90.8% in 2008. The mean difference
in average ratings was not statistically significant.
Perceived Improvement from Treatment
• Compared to 12 months ago, how would you rate your child’s ability to deal with
daily problems now?
• Compared to 12 months ago, how would you rate your child’s ability to deal with
social situations now?
• Compared to 12 months ago, how would you rate your child’s ability to
accomplish the things he or she wants to do now?
• Compared to 12 months ago, how would you rate your child’s problems or
symptoms now?
Figure 7. Perceived Improvement from Treatment
31.63%
32.81%
42.66%
38.35%
20.42%
22.43%
5.3%
6.4%
0% 20% 40% 60% 80% 100%
2008
2010
Much Better A Little Better About the Same A Little/Much Worse
Year N Mean* p-value
2010 627 2.04
2008 359 2.01
.6213
* Responses of “Much better” were recoded to 1, responses of “A little better” were recoded to 2, responses
of “About the same” were recoded to 3, responses of “A little worse” and “Much worse” were recoded to 4.
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 18
Figure 7 shows that 71.2% of respondents in 2010 believed their child was “much better”
or “a little better”, compared with 74.3% in 2008. The mean difference was not
statistically significant.
Availability of Help and Support
• In the last 12 months, how often did your family get the professional help you
wanted for your child?
• In the last 12 months, how often did you feel your child had someone to talk to for
counseling or treatment when he or she was troubled?
Figure 8. Availability of Help and Support
19.41%
17.06%
26.05%
24.66%
54.55%
58.28%
0% 20% 40% 60% 80% 100%
2008
2010
Never/Sometimes Usually Always
Year N Mean* p-value
2010 627 3.36
2008 288 3.31
.4424
* Responses of “Never” and “Sometimes” were recoded to 1, responses of “Usually” were recoded to 2 and
responses of “Always” were recoded to 3.
Figure 8 shows that 82.9% of respondents in 2010 believed their child “usually” or
“always” had professional help and support available, compared with 80.6% in 2008. The
difference of means was not statistically significant.
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 19
Single Item Measures
Figures 9 and 10, and Table 2 compare results for the following single item
measures:
• Appointment waiting time
• Amount child was helped by treatment
• Informed about medication side effects
• Discussed goals of treatment
• Discussed treatment options
• Informed about condition management
• Informed about patient rights
• Ability to refuse treatment options
• Health information privacy
• Responsiveness to cultural needs
• Reasons for seeking treatment
Table 2 contains the percentage who answered “yes” to yes/no questions in 2008 and
2010, as well as the p-value for a statistical chi-squared test of the difference between
years. Differences with p-values of .05 or less are considered statistically significant.
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 20
Appointment Waiting Time
• In the last 12 months, how often was your child seen within 15 minutes of his or
her appointment?
Figure 9. Appointment Waiting Time
23.71%
25.81%
33.33%
29.68%
42.96%
44.52%
0% 20% 40% 60% 80% 100%
2008
2010
Never/Sometimes Usually Always
Year N Mean* p-value
2010 620 3.09
2008 291 3.10
.8197
* Responses of “Never” and “Sometimes” were recoded to 1, responses of “Usually” were recoded to 2 and
responses of “Always” were recoded to 3.
Figure 9 shows that 74.2% of respondents in 2010 said their child “usually” or “always”
was seen within 15 minutes, compared with 76.3% in 2008. The difference of means was
not statistically significant.
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 21
How Much Your Child was Helped
• In the last 12 months, how much was your child helped by the counseling or
treatment he or she got?
Figure 10. How Much Child was Helped
20.96%
18.81%
28.90%
32.15%
50.14%
49.04%
0% 20% 40% 60% 80% 100%
2008
2010
Not at all/A little Somewhat A lot
Year N Mean* p-value
2010 622 3.25
2008 353 3.23
.6856
* Responses of “Not at all” and “A little” were recoded to 1, responses of “Somewhat” were recoded to 2
and responses of “A lot” were recoded to 3.
Figure 10 shows that 81.2% of respondents in 2010 said their child was helped
“somewhat” or “a lot”, compared with 79.0% in 2008. The difference of means was not
statistically significant.
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 22
Table 3. Percent Responding “Yes” to Single-Item Measures
Item
SFY
2008
SFY
2010 p-value
Were you told what side effects of those medicines to
watch for?
80.30% 80.57% .9337
Were the goals of your child’s counseling or treatment
discussed completely with you?
89.55% 88.22% .5563
Were you given information about different kinds of
counseling or treatment that are available for your child?
62.11% 64.25% .5331
Were you given as much information as you wanted about
what you could do to manage your child’s condition?
73.17% 74.15% .7546
Were you given information about your child’s rights as a
patient?
88.61% 90.06% .5077
Did you feel you could refuse a specific type of medicine
or treatment for your child?
87.06% 87.68% .7936
As far as you know did anyone your child saw for
counseling or treatment share information with others that
should have been kept private?
6.67% 2.78% .0058*
Was the care your child received responsive to those
(cultural) needs?
75.00% 76.92% .8908
Was any of your child’s counseling or treatment for
problems related to ADHD or other behavior problems?
77.90% 81.73% .1481
Was any of your child’s counseling or treatment for family
problems or mental or emotional illness?
66.76% 67.91% .7146
Was any of your child’s counseling or treatment for
autism or other developmental problems?
20.00% 22.30% .4044
Was any of your child’s counseling or treatment for help
with alcohol use or drug use?
5.16% 4.21% .4958
* Statistically significant
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 23
Discussion
Comparing the SFY 2008 survey and the SFY 2010 survey, results indicated
relatively high levels of satisfaction holding steady across an array of 21 quality
measures. Although the majority of the differences were not statistically significant,
many of the quality measures showed slight improvements. These results continue to
reflect a positive upward trend occurring in regards to member satisfaction with the
behavioral health services for children.
One measure showed a statistically significant difference between SFY 2008 and
SFY 2010. The single-item measure “As far as you know did anyone your child saw for
counseling or treatment share information with others that should have been kept
private?” reflected a significant decrease in the percent of members who responded “yes”
from 6.7% in SFY 2008 to 2.8% in SFY 2010. This may indicate that members perceive
providers being more cautious about complying with HIPAA regulations regarding
disclosure of personal health information.
The SFY 2010 sampling methodology limited the sample pool to members who
had a paid behavioral health claim. This method may have contributed to an increase in
the adjusted response rate, which was 22.2% higher than the SFY 2008 rates. APS
recommends the OHCA continue to follow this sampling methodology for future surveys.
Appendix A
ECHO® Child Survey
Fiscal Year 2010
June 2010 Page 28
Appendix B
Below are the responses by survey question. There were 633 total respondents with
completed surveys. Frequencies in the tables reflect the number of people responding to
each item; percentages in the tables reflect the number of respondents to the particular
question.
Q1 In the last 12 months, did your child get counseling,
treatment or medicine for any of these reasons?
Q1 Frequency Percent
Yes 633 100.00
Q2 In the last 12 months, did you call someone to get
professional counseling on the phone for your child?
Q2 Frequency Percent
Yes 222 35.86
No 397 64.14
Total 619 100.00
Q3 In the last 12 months, how often did you get the
professional counseling your child needed on the phone?
Q3 Frequency Percent
Never 47 22.38
Sometimes 80 38.10
Usually 28 13.33
Always 55 26.19
Total 210 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 29
Q4 In the last 12 months, did your child need counseling or
treatment right away?
Q4 Frequency Percent
Yes 254 40.25
No 377 59.75
Total 631 100.00
Q5 In the last 12 months, when your child needed
counseling or treatment right away, how often did he or she
see someone as soon as you wanted?
Q5 Frequency Percent
Never 20 7.97
Sometimes 33 13.15
Usually 70 27.89
Always 128 51.00
Total 251 100.00
Q6 In the last 12 months, not counting times your child
needed counseling or treatment right away, did you make
any appointments for your child for counseling or
treatment?
Q6 Frequency Percent
Yes 510 81.21
No 118 18.79
Total 628 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 30
Q7 In the last 12 months, not counting times your child
needed counseling or treatment right away, how often did
your child get an appointment for counseling or treatment as
soon as you wanted?
Q7 Frequency Percent
Never 13 2.59
Sometimes 70 13.94
Usually 147 29.28
Always 272 54.18
Total 502 100.00
Q8 In the last 12 months, how many times did your child go
to an emergency room or crisis center to get counseling or
treatment?
Q8 Frequency Percent
None 521 82.70
1 time 59 9.37
2 times 34 5.40
3 or more times 16 2.54
Total 630 100.00
Q9 In the last 12 months (not counting emergency rooms or
crisis centers), how many times did your child get
counseling, treatment or medicine in your home or at an
office, clinic, or other treatment program?
Q9 Frequency Percent
None 0 0.00
1 to 10 times 278 43.92
11 to 20 times 171 27.01
21 or more times 184 29.07
Total 633 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 31
Q10 In the last 12 months, how many times did your child
get counseling or treatment in your home?
Q10 Frequency Percent
None 478 75.87
1 to 10 times 74 11.75
11 to 20 times 37 5.87
21 or more times 41 6.51
Total 630 100.00
Q11 In the last 12 months, how often was your child seen
within 15 minutes of his or her appointment?
Q11 Frequency Percent
Never 62 10.00
Sometimes 98 15.81
Usually 184 29.68
Always 276 44.52
Total 620 100.00
Q12 In the last 12 months, how often did the people your
child saw for counseling or treatment listen carefully to you?
Q12 Frequency Percent
Never 17 2.71
Sometimes 56 8.93
Usually 140 22.33
Always 414 66.03
Total 627 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 32
Q13 In the last 12 months, how often did the people your
child saw for counseling or treatment explain things in a way
you could understand?
Q13 Frequency Percent
Never 14 2.23
Sometimes 40 6.36
Usually 143 22.73
Always 432 68.68
Total 629 100.00
Q14 In the last 12 months, how often did the people your
child saw for counseling or treatment show respect for what
you had to say?
Q14 Frequency Percent
Never 16 2.55
Sometimes 45 7.18
Usually 124 19.78
Always 442 70.49
Total 627 100.00
Q15 In the last 12 months, how often did the people your
child saw for counseling or treatment spend enough time
with you?
Q15 Frequency Percent
Never 24 3.83
Sometimes 79 12.62
Usually 151 24.12
Always 372 59.42
Total 626 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 33
Q16 In the last 12 months, did your child take any
prescription medicines as part of his or her treatment?
Q16 Frequency Percent
Yes 457 72.42
No 174 27.58
Total 631 100.00
Q17 In the last 12 months, were you told what side effects
of those medicines to watch for?
Q17 Frequency Percent
Yes 365 80.57
No 88 19.43
Total 453 100.00
Q18 In the last 12 months, how often were you involved as
much as you wanted in your child’s counseling or
treatment?
Q18 Frequency Percent
Never 18 2.86
Sometimes 53 8.43
Usually 102 16.22
Always 456 72.50
Total 629 100.00
Q19 In the last 12 months, were the goals of your child’s
counseling or treatment discussed completely with you?
Q19 Frequency Percent
Yes 554 88.22
No 74 11.78
Total 628 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 34
Q20 In the last 12 months, how often did your family get the
professional help you wanted for your child?
Q20 Frequency Percent
Never 32 5.16
Sometimes 63 10.16
Usually 151 24.35
Always 374 60.32
Total 620 100.00
Q21 In the last 12 months, how often did you feel your child
had someone to talk to for counseling or treatment when he
or she was troubled?
Q21 Frequency Percent
Never 36 5.83
Sometimes 80 12.97
Usually 154 24.96
Always 347 56.24
Total 617 100.00
Q22 In the last 12 months, were you given information
about different kinds of counseling or treatment that are
available for your child?
Q22 Frequency Percent
Yes 399 64.25
No 222 35.75
Total 621 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 35
Q23 In the last 12 months, were you given as much
information as you wanted about what you could do to
manage your child’s condition?
Q23 Frequency Percent
Yes 459 74.15
No 160 25.85
Total 619 100.00
Q24 In the last 12 months, were you given information
about your child’s rights as a patient?
Q24 Frequency Percent
Yes 562 90.06
No 62 9.94
Total 624 100.00
Q25 In the last 12 months, did you feel you could refuse a
specific type of medicine or treatment for your child?
Q25 Frequency Percent
Yes 541 87.68
No 76 12.32
Total 617 100.00
Q26 In the last 12 months, as far as you know did anyone
your child saw for counseling or treatment share
information with others that should have been kept private?
Q26 Frequency Percent
Yes 17 2.78
No 594 97.22
Total 611 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 36
Q27 Does your child’s language, race, religion, ethnic
background or culture make any difference in the kind of
counseling or treatment he or she needs?
Q27 Frequency Percent
Yes 40 6.53
No 573 93.47
Total 613 100.00
Q28 In the last 12 months, was the care your child received
responsive to those needs?
Q28 Frequency Percent
Yes 30 76.92
No 9 23.08
Total 39 100.00
Q29 Using any number from 0 to 10, what number would
you use to rate all your child’s counseling or treatment in
the last 12 months?
Q29 Frequency Percent
0 Worst 6 0.97
1 6 0.97
2 8 1.29
3 22 3.55
4 19 3.06
5 38 6.13
6 36 5.81
7 71 11.45
8 130 20.97
9 87 14.03
10 Best 197 31.77
Total 620 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 37
Q30 In the last 12 months, how much was your child helped
by the counseling or treatment he or she got?
Q30 Frequency Percent
Not at all 32 5.14
A little 85 13.67
Somewhat 200 32.15
A lot 305 49.04
Total 622 100.00
Q31 In general, how would you rate your child’s overall
mental health now?
Q31 Frequency Percent
Excellent 54 8.68
Very Good 133 21.38
Good 255 41.00
Fair 147 23.63
Poor 33 5.31
Total 622 100.00
Q32 Compared to 12 months ago, how would you rate your
child’s ability to deal with daily problems now?
Q32 Frequency Percent
Much better 226 36.16
A little better 236 37.76
About the same 125 20.00
A little worse 22 3.52
Much worse 16 2.56
Total 625 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 38
Q33 Compared to 12 months ago, how would you rate your
child’s ability to deal with social situations now?
Q33 Frequency Percent
Much better 192 30.77
A little better 233 37.34
About the same 156 25.00
A little worse 29 4.65
Much worse 14 2.24
Total 624 100.00
Q34 Compared to 12 months ago, how would you rate your
child’s ability to accomplish the things he or she wants to do
now?
Q34 Frequency Percent
Much better 207 33.33
A little better 236 38.00
About the same 151 24.32
A little worse 21 3.38
Much worse 6 0.97
Total 621 100.00
Q35 Compared to 12 months ago, how would you rate your
child’s problems or symptoms now?
Q35 Frequency Percent
Much better 193 30.98
A little better 251 40.29
About the same 127 20.39
A little worse 41 6.58
Much worse 11 1.77
Total 623 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 39
Q36 Our records show that your child is now in
SoonerCare Choice. Is that right?
Q36 Frequency Percent
Yes 618 99.36
No 4 0.64
Total 622 100.00
Q38 How many months or years in a row has your child
been in this health plan?
Q38 Frequency Percent
Less than 1 year 22 3.62
1-2 years 48 7.91
2-5 years 155 25.54
5 or more years 382 62.93
Total 607 100.00
Q39 How much of the counseling or treatment your child
got in the last 12 months was paid for by his or her health
plan?
Q39 Frequency Percent
All of it 565 91.57
Most of it 26 4.21
Some of it 16 2.59
None of it 10 1.62
Total 617 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 40
Q40 Were you told about other ways to get counseling,
treatment, or medicine for your child?
Q40 Frequency Percent
Yes 205 33.39
No 409 66.61
Total 614 100.00
Q41 When your child joined this health plan or at any time
since then, did your child get someone new for counseling or
treatment?
Q41 Frequency Percent
Yes 294 48.04
No 318 51.96
Total 612 100.00
Q42 Since your child joined this health plan, how much of a
problem, if any, was it to get someone for your child you are
happy with?
Q42 Frequency Percent
A big problem 44 15.22
A small problem 80 27.68
Not a problem 165 57.09
Total 289 100.00
Q43 In the last 12 months, did you need approval from
your child’s health plan for any counseling or treatment?
Q43 Frequency Percent
Yes 238 39.21
No 369 60.79
Total 607 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 41
Q44 In the last 12 months, how much of a problem, if any,
were delays in counseling or treatment while you waited for
approval from your child’s health plan?
Q44 Frequency Percent
A big problem 24 10.30
A small problem 65 27.90
Not a problem 144 61.80
Total 233 100.00
Q45 In the last 12 months, how much of a problem, if any,
was it to get the counseling or treatment you thought your
child needed?
Q45 Frequency Percent
A big problem 48 7.87
A small problem 97 15.90
Not a problem 465 76.23
Total 610 100.00
Q46 In the last 12 months, did you look for any information
about counseling or treatment from your child’s health plan
in written materials or on the Internet?
Q46 Frequency Percent
Yes 192 31.22
No 423 68.78
Total 615 100.00
Q47 In the last 12 months, how much of a problem, if any,
was it to find or understand this information?
Q47 Frequency Percent
A big problem 23 12.04
A small problem 55 28.80
Not a problem 113 59.16
Total 191 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 42
Q48 In the last 12 months, did you call the health plans
customer service to get information or help about
counseling or treatment for your child?
Q48 Frequency Percent
Yes 80 13.09
No 531 86.91
Total 611 100.00
Q49 In the last 12 months, how much of a problem, if any,
was it to get the help you needed for your child when you
called the health plans customer service?
Q49 Frequency Percent
A big problem 9 11.54
A small problem 22 28.21
Not a problem 47 60.26
Total 78 100.00
Q50 In the last 12 months, did you have to fill out any
paperwork about counseling or treatment for your child’s
health plan?
Q50 Frequency Percent
Yes 298 49.09
No 309 50.91
Total 607 100.00
Q51 In the last 12 months, how much of a problem, if any,
did you have with paperwork for your child’s health plan?
Q51 Frequency Percent
A big problem 11 3.72
A small problem 44 14.86
Not a problem 241 81.42
Total 296 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 43
Q52 Using any number from 0 to 10, what number would
you use to rate your child’s health plan for counseling or
treatment?
Q52 Frequency Percent
0 Worst 7 1.14
1 6 0.97
2 1 0.16
3 14 2.27
4 11 1.79
5 37 6.01
6 24 3.90
7 62 10.06
8 117 18.99
9 114 18.51
10 Best 223 36.20
Total 616 100.00
Q53 In the last 12 months, was any of your child’s
counseling or treatment for problems related to ADHD or
other behavior problems?
Q53 Frequency Percent
Yes 510 81.73
No 114 18.27
Total 624 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 44
Q54 In the last 12 months, was any of your child’s
counseling or treatment for family problems or mental or
emotional illness?
Q54 Frequency Percent
Yes 419 67.91
No 198 32.09
Total 617 100.00
Q55 In the last 12 months, was any of your child’s
counseling or treatment for autism or other developmental
problems?
Q55 Frequency Percent
Yes 136 22.30
No 474 77.70
Total 610 100.00
Q56 In the last 12 months, was any of your child’s
counseling or treatment for help with alcohol use or drug
use?
Q56 Frequency Percent
Yes 26 4.21
No 592 95.79
Total 618 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 45
Q57 In general, how would you rate your child’s overall
health now?
Q57 Frequency Percent
Excellent 105 16.88
Very Good 213 34.24
Good 219 35.21
Fair 74 11.90
Poor 11 1.77
Total 622 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 46
Q58 What is your child’s age now?
Q58 Frequency Percent
0 3 0.48
1 2 0.32
2 0 0.00
3 4 0.64
4 7 1.12
5 35 5.59
6 35 5.59
7 36 5.75
8 57 9.11
9 62 9.90
10 62 9.90
11 53 8.47
12 58 9.27
13 52 8.31
14 42 6.71
15 62 9.90
16 40 6.39
17 16 2.56
Total 626 100.00
Q59 Is your child male or female?
Q59 Frequency Percent
Male 387 62.22
Female 235 37.78
Total 622 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 47
* Respondents who selected more than one race were grouped in the
multiracial / other category.
Q60 Is your child of Hispanic or Latino origin or descent?
Q60 Frequency Percent
Yes, Hispanic or Latino 61 9.85
No, not Hispanic or Latino 558 90.15
Total 619 100.00
Q61 What is your child's race?
Q61 Frequency Percent
White 379 61.33
American Indian 64 10.36
Black or African American 63 10.19
Asian 1 0.16
Multiple or Other* 111 17.96
Total 618 100.00
Q62 What is your age now?
Q62 Frequency Percent
18-24 10 1.62
25-34 226 36.69
35-44 206 33.44
45-54 108 17.53
55-64 42 6.82
65-74 22 3.57
75 or older 2 0.32
Total 616 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 48
Q63 Are you male or female?
Q63 Frequency Percent
Male 49 7.87
Female 574 92.13
Total 623 100.00
Q64 What is the highest grade or level of school that you
have completed?
Q64 Frequency Percent
8th grade or less 14 2.30
Some high school 79 12.97
HS grad or GED 229 37.60
Some college 233 38.26
College grad 37 6.08
More than 4yr college 17 2.79
Total 609 100.00
Q65 How are you related to the child?
Q65 Frequency Percent
Mother or Father 496 79.87
Grandparent 75 12.08
Aunt or Uncle 11 1.77
Other relative 4 0.64
Legal Guardian 35 5.64
Total 621 100.00
ECHO® Child SoonerCare Choice Survey
Fiscal Year 2010
June 2010 Page 49
Q66 Did someone help you complete this survey?
Q66 Frequency Percent
Yes 15 2.94
No 496 97.06
Total 511 100.00
* Respondents could have selected more than one response.
Respondents must have answered “Yes” to Q66 to respond on Q67.
Q67 How did that person help you?
(Mark all that apply) Frequency
Read the questions to me 7
Wrote down the answers I gave 6
Answered the questions for me 3
Translated the questions into my language 1
Helped in some other way 5
Total * 15