Tag Archives: Individuals with Disabilities Education Act

The Perils of Confusing Performance Measurement with Program Evaluation

Posted on by

A group of researchers recently published a paper critiquing the child outcomes performance indicator for Part C and Part B 619. They also presented some of their thoughts in a recent webinar sponsored by the Association of University Centers on Disabilities (AUCD). The researchers’ critique is based on several faulty assumptions and consequently unfairly discredits the system for measuring child outcomes and the use of the data. Let’s look at our concerns with their critique.

First, the authors have confused performance measurement with program evaluation.

Their primary argument is that the child outcomes measurement requirement produces misleading information because it is based on a flawed evaluation design. The researchers’ critique wrongly assumes that the child outcomes indicator is designed as an evaluation. The child outcomes measurement is not a program evaluation; it is one performance indicator embedded within a larger performance measurement system that is required by the Individuals with Disabilities Education Act (IDEA). States report on a number of performance indicators that address compliance with federal regulations and program results. As such, these indicators yield information that supports program improvement and ongoing monitoring of program performance. Performance measurement systems are common in both the public (for example, Maternal and Child Health) and the private sector (for example, the Pew framework for home visiting). The Office of Special Education Programs (OSEP) implemented the child outcomes indicator in response to the Government Performance and Results Act which requires all federal agencies report on results being achieved by their programs. OSEP also uses the child outcomes indicator data to monitor states on results achieved, consistent with the strong emphasis in IDEA to improve results for children with disabilities.

The Government Accounting Office has produced a succinct summary that highlights some of the differences between the performance measurement and program evaluation. Performance measurement refers to ongoing monitoring and reporting of program accomplishments. Performance measures may address program activities, services and products, or results. The OSEP child outcomes indicator is a performance measure that addresses results. Examples of other results performance measures are teen pregnancy rates, percentage of babies born at low birth weight, 3rd grade reading scores, and high school graduation rates. In contrast, program evaluationsare periodic or one time studies usually conducted by experts external to the program and involve a more in depth look at a program’s performance. Impact evaluations are a particular type of program evaluation that determine the effect of a program by comparing the outcomes of program participation to what would have happened had the program not been provided.

Performance Measurement Compared to Program Evaluation

Feature Performance Measurement Program Evaluation
Data collected on a regular basis, e.g.,  annually Yes No
Usually conducted by experts to answer a specific question at a single point in time No Yes
Provides information about a program’s performance relative to targets or goals Yes Possibly
Provides ongoing information for program improvement Yes No
Can conclude unequivocally that the results observed were caused by the program No Yes, if well designed impact evaluation
Typically quite costly No Yes

A major difference between measuring outcomes in a performance measure system versus a program evaluation is that a well-designed impact evaluation is able to conclude unequivocally that the results observed were caused by the program. Performance measures cannot rule out alternative explanations for the results observed. Nevertheless, performance measurement data can be used for a variety of purposes including accountability, monitoring performance, and program improvement. Data on performance measures such as the Part C and Part B Section 619 child outcomes indicator can be used to track performance compared to a target or to compare results from one year to the next within programs or states. They can be used to identify state or local programs that could benefit from additional support to achieve better results. Comparing outcomes across states or programs should be done with an awareness that they might serve different population which could contribute to different outcomes. The solution to this is not to conclude that results data are useless or misleading but rather to interpret the results alongside other critical pieces of information such as the performance of children at entry to the program or the nature of the services received. Two of OSEP’s technical assistance centers, the Center for IDEA Early Childhood Data Systems (DaSy) and the Early Childhood Technical Assistance Center (ECTA, have developed a variety of resources to support states in analyzing child outcomes data including looking at outcomes for subgroups to further understand what is contributing to the results observed. Just like tracking 3rd grade reading scores or the percentage of infants who are low birth weight, there is tremendous value in knowing how young children with disabilities are doing across programs and year after year.

Second, the authors incorrectly maintain that children who did not receive Part C services would show the same results on the child outcomes indicator as children who did.

The researchers’ claim that the results states are reporting to OSEP would be achieved even if no services had been provided rests on a flawed analysis of the ECLS-B data, a longitudinal study of children born in 2001. For their analysis, the authors identify a group of 24 months olds in the data set who they label as “Part C eligible children who did not receive Part C services.” These children

  • Received a low score on a shortened version of the Bayley Scales of Infant Development (27 items) administered at 9 months of age by a field data collector; and
  • Were reported by a parent when the child was 24 months old as not having received services to help with the child’s special needs.

Few would argue that the determination of eligibility for Part C could be replicated by a 27-item assessment administered by someone unfamiliar with infants and toddlers with disabilities. Furthermore, data from the National Early Intervention Longitudinal Study show that very few children are identified as eligible for Part C based on developmental delay at 9 months of age. The first problem with the analysis is assuming all of these children would have been Part C eligible. The second problem is that it is impossible in this data set to reliably identify which children did and did not receive Part C services. Parents were asked a series of questions about services in general; they were not asked about Part C services. As we and others who have worked with national data collections have learned, parents are not good reporters of program participation for a variety of reasons. The only way to confirm participation in Part C services is to verify program participation which the study did not do. Given that children who received Part C services cannot be identified in the ECLS-B data, no one should be making conclusions about Part C participation based on this data set.

The authors also argue that a measurement phenomenon called “regression to the mean” explains why Part C and Part B 619 children showed improved performance after program participation. In essence this argument says that improvements seen in the functioning of the children are not real changes but are actually due to measurement error. One can acknowledge the reality of errors in assessment results but to maintain that measurement error is the sole or even a major explanation for the progress shown by children in Part C and Part B 619 programs is absurd.

Moving Forward

State Part C and 619 programs are required by IDEA to report on multiple performance indicators including child outcomes as part of a larger performance measurement system. The child outcomes indicator was developed with extensive stakeholder input in order to maximize its utility to local programs, state agencies, and the federal government. The process of building the infrastructure needed to collect and use child outcomes data has been complex which is why states have been working on it for over ten years. State agencies continue to identify and implement strategies for improving the data collection and use of the data. We know that the data collection processes are not perfect and more work needs to be undertaken to address data quality and other concerns. Building a national system for measuring the outcomes for young children with disabilities receiving IDEA services is a long-term undertaking that requires ongoing effort to make the process better. Disparaging the performance indicator and the data reported by states based on incorrect assumptions and flawed analyses is not productive. Instead, the field needs to collectively engage in ongoing dialogue around critical issues of data quality, data analysis, and appropriate use of the data based on an informed understanding of what the child outcomes indicator is and is not. Part C and Part B 619 state agencies and OSEP are on the forefront of collecting and using early childhood outcomes data to improve programs – which is exactly what performance measurement is intended to do.

Source: DaSy: The Center for IDEA Early Childhood Data Systems

Available at: http://dasycenter.org/the-perils-of-confusing-performance-measurement-with-program-evaluation/ 

Joint Statement on Collaboration and Coordination of the MIECHV and IDEA Part C Programs

Posted on by

January 19, 2017

Creating a high-quality system of services and supports for infants and toddlers with disabilities and their families.

The purpose of this joint statement from the U.S. Departments of Education (ED) and Health and Human Services (HHS) (the Departments), is to set a vision for stronger partnerships, collaboration, and coordination between awardees of the Maternal, Infant, and Early Childhood Home Visiting Program (MIECHV) and the Individuals with Disabilities Education Act, Part C Program (IDEA Part C Program). Specifically, this joint statement provides recommendations to states, territories, and tribal entities to identify and enhance opportunities for collaboration and coordination between MIECHV and the IDEA Part C Program.

Effective collaboration and coordination across MIECHV and the IDEA Part C Program can create a high-quality system of services and supports for infants and toddlers with disabilities and their families. It is the position of both Departments that all infants and toddlers and their families should have access to coordinated, comprehensive services that support overall health, development, and wellness. This joint ED and HHS statement aims to advance this position by:

  • Providing an overview of the MIECHV and the IDEA Part C Programs;
  • Emphasizing the potential for collaboration and coordination between MIECHV awardees and the IDEA Part C State programs;
  • Highlighting existing opportunities for partnerships between MIECHV awardees and the IDEA Part C State programs; and
  • Providing recommendations to states, territories, tribal entities, and local programs for identifying and increasing opportunities for collaboration and coordination.

Download:

Joint Policy Statement on Collaboration and Coordination of the Maternal, Infant, and Early Childhood Home Visiting and the Individuals with Disabilities Education Act Part C Programs (PDF, 1.0MB)

Follow us on Twitter and see tweet about this joint statement here! https://twitter.com/ED_Sped_Rehab/status/822090143721025536

Dear Colleague Letter Regarding Least Restrictive Environments

Posted on by

1/9/2017

Dear Colleague:

We are writing to reaffirm the position of the U.S. Department of Education (ED or Department) that all young children with disabilities should have access to inclusive high-quality early childhood programs where they are provided with individualized and appropriate supports to enable them to meet high expectations. Over the last few years, States and communities have made progress in expanding early learning opportunities for young children, with all but four States investing in free public preschool programs.1 The Federal government, while aligning with the movement of States, has led several efforts to increase access to and the quality of early childhood programs, such as the Preschool Development Grants and expansion of Head Start. States have focused on improving the quality of early learning programs, including the development of early learning program standards and incorporating these into Quality Rating and Improvement Systems (QRIS).2

In September 2015, ED and the U.S. Department of Health and Human Services (HHS) issued a policy statement on promoting inclusion in early childhood programs to set a vision on this issue and provide recommendations to States, local educational agencies (LEAs), schools, and public and private early childhood programs.3 Despite the expansion of early childhood programs, there has not yet been a proportionate expansion of inclusive early learning opportunities for young children with disabilities. Given this concern and the ED-HHS policy statement on early childhood inclusion, the Office of Special Education Programs (OSEP) is updating the February 29, 2012, Dear Colleague Letter (DCL) to reaffirm our commitment to inclusive preschool education programs for children with disabilities and to reiterate that the least restrictive environment (LRE) requirements in section 612(a)(5) of the Individuals with Disabilities Education Act (IDEA or Act) are fully applicable to the placement of preschool children with disabilities.4 This DCL supersedes the 2012 OSEP DCL and includes additional information on the reporting of educational environments data for preschool children with disabilities and the use of IDEA Part B funds to provide special education and related services to preschool children with disabilities.

The LRE requirements have existed since passage of the Education for all Handicapped Children Act (EHA) in 1975 and are a fundamental element of our nation’s policy for educating students with disabilities (the Education of the Handicapped Act was renamed the IDEA in 1990). These requirements reflect the IDEA’s strong preference for educating students with disabilities in regular classes with appropriate aids and supports. Under section 612(a)(5) of the IDEA, to the maximum extent appropriate, children with disabilities, including children in public or private institutions or other care facilities, must be educated with children who are not disabled. Further, special classes, separate schooling, or other removal of children with disabilities from the regular educational environment occurs only when the nature or severity of the disability of a child is such that education in regular classes with the use of supplementary aids and services cannot be achieved satisfactorily.

MORE…

1 Walter N. Ridley Lecture: Pre-Kindergarten Access and Quality are Essential for Children’s Growth and Development (November 2, 2016), available at: http://www.ed.gov/news/speeches/walter-n-ridley-lecture-pre-kindergarten-access-and-quality-are-essential-childrens-growth-and-development. For more detailed but less recent information on State investments in public preschool see: Barnett, W.S., Friedman-Krauss, A., Gomez, R.E., Squires, J.H., Clarke Brown, K., Weisenfeld, G.G., & Horowitz, M. (2016). The state of preschool 2015: State preschool yearbook. New Brunswick, NJ: National Institute for Early Education Research.

2 QRIS statewide systems are implemented in over half of the States and others are developing such systems. ED and the of Department of Health and Human Services have supported States in further developing such systems under Race to the Top-Early Learning Challenge and the Child Care Development Fund. For more information see: https://qrisguide.acf.hhs.gov/index.cfm?do=qrisabout.

3 See U.S. Departments of Education and Health and Human Services Policy Letter on the Inclusion of Children with Disabilities in Early Childhood Programs (September 14, 2015), available at: http://www2.ed.gov/policy/speced/guid/earlylearning/joint-statement-full-text.pdf.

4 Although not discussed here, other Federal laws apply to preschool-aged children with disabilities as well. These laws include section 504 of the Rehabilitation Act of 1973, as amended (Section 504) and Title II of the Americans with Disabilities Act of 1990, as amended (ADA). The Department’s Office for Civil Rights (OCR) enforces Section 504 and pursuant to a delegation by the Attorney General of the United States, OCR shares (with the U.S. Department of Justice and HHS) in the enforcement of Title II of the ADA in the education context. HHS has Title II jurisdiction over public preschools. 35 CFR §35.190(b)(3). Section 504 prohibits discrimination on the basis of disability in programs or activities that receive Federal financial assistance from the Department. 29 U.S.C. § 794, 34 CFR §104.4(a). Section 104.38 of the Department’s Section 504 regulations specify that recipients of Federal financial assistance from the Department that provide preschool education may not on the basis of disability exclude qualified persons with disabilities, and must take into account the needs of these persons in determining the aid, benefits, or services to be provided. 34 CFR §104.38. Title II prohibits discrimination on the basis of disability by public entities, including public schools, regardless of whether they receive Federal financial assistance. 42 U.S.C. §§ 12131-12134, 28 CFR Part 35 (Title II). Additionally, as applicable, entities providing preschool education must comply with the nondiscrimination requirements set forth in Title III of the ADA that prohibit discrimination on the basis of disability in places of public accommodation, including businesses and nonprofit agencies that serve the public. The U.S. Department of Justice enforces Title III of the ADA. 42 U.S.C. §§ 12181-12189, 28 CFR Part 36 (Title III).

Source: Office of Special Education Programs, U.S. Department of Education

Available at: https://www2.ed.gov/policy/speced/guid/idea/memosdcltrs/preschool-lre-dcl-1-10-17.pdf 

FACT SHEET: Equity in IDEA | U.S. Department of Education

Posted on by

12/12/16

The U.S. Department of Education today made available to the public final regulations under Part B of the Individuals with Disabilities Education Act (IDEA), aimed at promoting equity by targeting widespread disparities in the treatment of students of color with disabilities. The regulations will address a number of issues related to significant disproportionality in the identification, placement, and discipline of students with disabilities based on race or ethnicity. The Department is also releasing a new Dear Colleague Letter addressing racial discrimination.

“Children with disabilities are often disproportionately and unfairly suspended and expelled from school and educated in classrooms separate from their peers,” said U.S. Secretary of Education John B. King Jr. “Children of color with disabilities are overrepresented within the special education population, and the contrast in how frequently they are disciplined is even starker.”

King added, “Today’s new regulations and supporting documents provide the necessary guidance and support to school districts and build upon the work from public education advocates and local leaders who believe, like we do, that we need to address racial and ethnic disparities in special education. This important step forward is about ensuring the right services get to the right students in the right way.”

In order to address those inequities, IDEA requires states to identify districts with “significant disproportionality” in special education—that is, when districts identify, place in more restrictive settings, or discipline children from any racial or ethnic group at markedly higher rates than their peers.

Children of color—particularly African-American and American Indian youth—are identified as students with disabilities at substantially higher rates than their peers. It is critical to ensure that overrepresentation is not the result of misidentification, including both over- and under-identification, which can interfere with a school’s ability to provide children with the appropriate educational services required by law. It is equally important to ensure that all children who are suspected of having a disability are evaluated and, as appropriate, receive needed special education and related services in the most appropriate setting and with the most appropriate discipline strategies employed.

Source: U.S. Department of Education

Available at: https://www.ed.gov/news/press-releases/fact-sheet-equity-idea?utm_content=&utm_medium=email&utm_name=&utm_source=govdelivery&utm_term=

Assistance to States for the Education of Children With Disabilities; Preschool Grants for Children With Disabilities

Posted on by

12/19/2016

The Secretary amends the regulations under Part B of the Individuals with Disabilities Education Act (IDEA) governing the Assistance to States for the Education of Children with Disabilities program and the Preschool Grants for Children with Disabilities program. With the goal of promoting equity under IDEA, the regulations will establish a standard methodology States must use to determine whether significant disproportionality based on race and ethnicity is occurring in the State and in its local educational agencies (LEAs); clarify that States must address significant disproportionality in the incidence, duration, and type of disciplinary actions, including suspensions and expulsions, using the same statutory remedies required to address significant disproportionality in the identification and placement of children with disabilities; clarify requirements for the review and revision of policies, practices, and procedures when significant disproportionality is found; and require that LEAs identify and address the factors contributing to significant disproportionality as part of comprehensive coordinated early intervening services (comprehensive CEIS) and allow these services for children from age 3 through grade 12, with and without disabilities.

Source: Federal Register

Available at: https://www.federalregister.gov/documents/2016/12/19/2016-30190/assistance-to-states-for-the-education-of-children-with-disabilities-preschool-grants-for-children

Thirty-sixth Annual Report to Congress on the Implementation of the Individuals with Disabilities Education Act, Parts B and C.

Posted on by

1/2015

This is the 36th Annual Report to Congress on the Implementation of the Individuals with Disabilities Education Act, 2014. Section 664(d) of the Individuals with Disabilities Education Act (IDEA), as reauthorized in 2004, requires that the Department of Education report annually on the progress made toward the provision of a free appropriate public education to all children with disabilities and the provision of early intervention services to infants and toddlers with disabilities.The 36th Annual Report to Congress on the Implementation of the Individuals with Disabilities Education Act, 2016 describes our nation’s progress in:

  • providing a free appropriate public education (FAPE) for all children with disabilities,
  • ensuring that the rights of children with disabilities and their parents are protected,
  • assisting states and localities in providing for the education of all children with disabilities, and
  • assessing the effectiveness of efforts to educate children with disabilities.

The report focuses on the children and students with disabilities being served under IDEA, Part C or B, nationally and at the state level.

Source: U.S. Department of Education

Available at: http://www2.ed.gov/about/reports/annual/osep/2014/parts-b-c/index.html

2013 Annual Reports to Congress on the Implementation of the Individuals with Disabilities Education Act

Posted on by

8/2014

The 35th Annual Report to Congress on the Implementation of the Individuals with Disabilities Education Act, 2013 describes our nation’s progress in:

  1. providing a free appropriate public education FAPE for all children with disabilities,
  2. ensuring that the rights of children with disabilities and their parents are protected,
  3. assisting states and localities in providing for the education of all children with disabilities, and
  4. assessing the effectiveness of efforts to educate children with disabilities.

The report focuses on the children and students with disabilities being served under IDEA, Part C or B, nationally and at the state level.

Source: U.S. Department of Education

Available at: http://www2.ed.gov/about/reports/annual/osep/2013/index.html

Outcomes for Children Served Through IDEA’s Early Childhood Programs: 2011–12

Posted on by

9/2013

In 2011-12, children with delays or disabilities who received services under the Individuals with Disabilities Education Act (IDEA) showed greater than expected developmental progress. Many children exited the program functioning within age expectations, and most made progress.

States’ Part C and Part B preschool programs report data annually on three outcomes:
1. Social relationships, which includes getting along with other children and relating well with adults
2. Use of knowledge and skills, which refers to thinking, reasoning, problem solving, and early literacy and math skills
3. Taking action to meet needs, which includes feeding, dressing, self-care, and following rules related to health and safety.

Source: The Early Childhood Outcomes Center

Available at: http://www.ectacenter.org/~pdfs/eco/OutcomesforChildren-FFY2011.pdf

State Performance Plan (SPP) and Annual Performance Report (APR) Forms

Posted on by

7/2013

As required by law, the Department has issued annual determination letters regarding states’ implementation of theIndividuals with Disabilities Education Act (IDEA).  Each state was evaluated on key indicators under Part B (ages 3 through 21) and Part C (infants through age 2) and placed into one of four categories: meets requirements, needs assistance, needs intervention, and needs substantial intervention.  Most states fell into the top categories; 38 states met requirements for Part B, and 37 states and Puerto Rico met requirements for Part C.  No states were in needs substantial intervention.  The IDEA identifies specific technical assistance or enforcement actions that the agency must undertake for states that do not meet requirements.

Source: U.S. Department of Education

Available at: http://www2.ed.gov/fund/data/report/idea/sppapr.html

Applications for New Awards; Technical Assistance and Dissemination To Improve Services and Results for Children With Disabilities and the Safe and Drug-Free Schools and Communities Program-National Technical Assistance Center on Positive Behavioral Interventions and Supports

Posted on by

Deadline: 8/19/2013

Purpose of Programs: The purpose of the Technical Assistance and Dissemination To Improve Services and Results for Children With Disabilities program is to promote academic achievement and to improve results for children with disabilities by providing technical assistance (TA), supporting model demonstration projects, disseminating useful information, and implementing activities that are supported by scientifically based research.

The Safe and Drug-Free Schools and Communities program provides support to State educational agencies (SEAs) for a variety of drug-abuse- and violence-prevention activities focused primarily on school-age youths.

Priorities: This notice includes two absolute priorities. In accordance with 34 CFR 75.105(b)(2)(v), absolute priority 1 is from allowable activities specified or otherwise authorized in the Individuals With Disabilities Education Act (IDEA) (see sections 663 and 681(d) of the IDEA, 20 U.S.C. 1463 and 1481(d)). We are establishing absolute priority 2 under the authority in section 4121 of the Elementary and Secondary Education Act of 1965, as amended, and in accordance with section 437(d)(1) of the General Education Provisions Act (GEPA) (20 U.S.C. 7131; 20 U.S.C. 1232(d)(1).

Absolute Priorities: These priorities are absolute priorities. Under 34 CFR 75.105(c)(3), we consider only applications that meet these priorities.

Source: Federal Register

Available at: https://www.federalregister.gov/articles/2013/07/05/2013-16191/applications-for-new-awards-technical-assistance-and-dissemination-to-improve-services-and-results