DEMONSTRATED PLANNING FOR AND DELIVERY OF EFFECTIVE AND SUFFICIENT CONSUMER SERVICES
COMMUNITY SERVICE REVIEWS OF ADULTS
REQUIREMENTS OF THE EXIT CRITERIA ORDER
Exit Criterion #3 requires that DMH conduct annual consumer reviews of adults (age 18 and over), using a stratified random sample of individuals who have received services within the DMH system with a sample size sufficient to provide statistical levels of confidence. The annual reviews collect data through a combination of consumer and family interviews, record review, staff interviews, caregiver interviews and document reviews. The annual reviews are conducted by an independent review team.
The Exit Criteria Order requires that the annual consumer review of adults cover seven (7) domains:
- Community Living
- Health
- Meaningful Activity (work, education, etc.)
- Social Network
- Income Assessment and Planning
- Treatment and Support Services; and
- Service System Capacity
The Court Required Performance Level is an aggregate score of 80% for positive system performance for the adults who are sampled and reviewed.
IMPLEMENTATION ACTIVITIES
To begin the process of meeting the requirements of the Exit Criteria Order, the Dixon Court Monitor retained Human Systems and Outcomes, Inc. (HSO) to assist in developing the adult review protocol. HSO is an organization with extensive experience in qualitative child service review processes used in monitoring services in class action litigation in numerous states across the country. Representatives from the provider community and DMH worked with HSO to develop the adult review protocol.
The Adult Review Protocol [PDF] was developed, tested, revised, and then used to create a baseline for subsequent measurement of progress. The adult review protocol is scored using a Roll-Up Sheet [PDF] that evaluates the following three main areas:
- Consumer Status (overall consumer status)
- Recent Progress
- Current Practice Performance (overall system performance)
Since the initial review, the design of the sampling process, training of reviewers, supervision of data collection, and analysis of data for the adult community service reviews have been conducted by HSO, under contract with the Dixon Court Monitor. HSO was contracted by the Dixon Court Monitor and worked as staff to the monitor in conducting the reviews. Logistical preparation and organization of the on-site case review activities was completed by Consumer Action Network (CAN).
Review Year | Findings |
2003 Review |
The initial review was completed in May 2003 with a sample of 28 adults. Reviews were conducted by a team of external reviewers and DMH staff trained on the protocol. Logistical support was provided by CAN. HSO issued a 2003 Report [PDF] on the results of the 2003 review which was discussed in the Court Monitor’s July 2003 report.
The results for the 2003 review had 75% of the adults with overall status ratings in the acceptable range. Overall system performance was acceptable for 54% of the adults. |
2004 Review |
The 2004 review had a larger target sample of 54 adults. 41 adults participated in the review. Review activities were completed in April 2004. Reviews were conducted by a team of external reviewers and DMH staff trained on the protocol. Logistical support was provided by CAN. HSO issued a 2004 Report [PDF] on the results of the 2004 review which was discussed in the Court Monitor’s July 2004 report.
The results for the 2004 review had 54% of the adults with overall acceptable status ratings. Overall system performance was acceptable for 39% of the adults. |
2005 Review |
The 2005 review had a target sample of 54 adults. 51 adults participated in the review. Review activities were completed in April 2005. Reviews were conducted by a team of external reviewers and DMH staff trained on the protocol. Logistical support was provided by CAN. HSO issued a 2005 Report [PDF] on the results of the 2005 review which was discussed in the Court Monitor’s July 2005 report.
The results for the 2005 review had 67% of the adults with overall acceptable status ratings. Overall system performance was acceptable for 57% of the adults. |
2006 Review |
The 2006 review had a target sample of 54 adults. 51 adults participated in the review. Review activities were completed in April 2006. Reviews were conducted by a team of external reviewers and DMH staff trained on the protocol. Logistical support was provided by CAN. HSO issued a 2006 Report [PDF] on the results of the 2006 review which was discussed in the Court Monitor’s July 2006 report.
The results for the 2006 review had 65% of the adults with overall acceptable status ratings. Overall system performance was acceptable for 69% of the adults. |
2007 Review |
The 2007 review had a sample of 52 adults. Review activities were conducted in April 2007. Reviews were conducted by a team of external reviewers and DMH staff trained on the protocol. Logistical support was provided by CAN. HSO issued a 2007 Report [PDF] on the results of the 2007 review which was discussed in the Court Monitor’s July 2007 report.
The results of the 2007 review had 75% of the adults with overall acceptable status ratings. Overall system performance was found acceptable for 80% of the adults. |
2008 Review |
In a letter dated August 24, 2007 [PDF], DMH proposed changes to the review process that were agreed upon by the Dixon Court Monitor in a letter dated October 25, 2007 [PDF]. These changes were intended to further refine the case review and scoring process. |
2009 Review |
The 2009 review had a sample size of 88 adults. Review activities were conducted in May by a team of external reviewers and DMH staff with logistical support from CAN. The 2009 Report [PDF] issued by HSO showed 74% of the adults with overall acceptable status, and the overall system performance was found acceptable for 70% of the adults. The 2009 Report was discussed in the Court Monitor’s July 2009 report. |
2010 Review |
The 2010 review had a sample size of 85 adults. Review activities were conducted in May by a team of external reviewers and DMH staff with logistical support from CAN. The 2010 Report [PDF] issued by HSO showed 80% of the adults with overall acceptable status, and the overall system performance was found acceptable by 77% of the adults. The 2010 Report was discussed in the Court Monitor’s July 2010 report. |
2011 Review |
The 2011 review had a sample size of 78 adults. Review activities were conducted in February by a team of external reviewers and DMH staff with logistical support from the Far Southeast Family Strengthening Collaborative Inc., The 2011 Report [PDF] issued by HSO showed 80% of the adults with overall acceptable status, and the overall system performance was found acceptable for 78% of the adults. The 2011 Report was discussed in the Court Monitor’s June 2011 report. |