Employment and Training Administration
Washington, D. C. 20210


Perf. Standards




June 30, 1998















Office of Regional Management




Job Training Partnership Act (JTPA) Title II and Title III Performance Standards for PYs 1998 and 1999

  1. Purpose. To transmit additional guidance on JTPA Performance Standards pursuant to Training and Employment Guidance Letter No. 2-97, dated February 19, 1998.

  2. References.

    1. Job Training Partnership Act (JTPA) 29 USC 1501, et seq, as amended.

    2. JTPA Regulations, 20 CFR Parts 626-629 and 631, published in the Federal Register on September 2, 1994.

    3. Training and Employment Guidance Letter (TEGL) No. 2-97, dated February 19, 1998.

    4. TEGL No. 2-95, dated August 10, 1995.

    5. TEGL No. 1-94, dated August 31, 1994.

    6. Training and Employment Information Notice (TEIN) No. 5-93, Change 1, dated June 23, 1994, and Change 2, dated January 24, 1997.

    7. TEGL 8-97, dated April 23, 1998.

  3. Background. This Guidance Letter provides additional information on the Secretary's required performance measures and the Secretary's implementing instructions for performance standards for Program Years (PYs) 1998 and 1999 (July 1, 1998-June 30, 1999; July 1, 1999-June 30, 2000). Sec. 106 of JTPA, as amended, directs the Secretary to establish performance standards for adult, youth, and dislocated orker programs. These standards may be updated every 2 years based on the most recent JTPA program experience, as well as program emphases and goals established by the Department of Labor. The Secretary also issues instructions for implementing these standards, and issues parameters for States to follow in adjusting the Secretary's standards for service delivery areas (SDAs) and substate areas (SSAs).

    It should be noted that many States have requested and are operating under waivers granted under the FY 1997 and 1998 Department of Labor Appropriations Acts, and that some or all of those waivers may take precedence over some of the provisions in this Guidance Letter. After consideration of the requirements in this Guidance Letter, States may wish to consider requesting additional waivers and/or modifying existing waivers.

    To assist the Department in assessing the JTPA operational environment and the constraints likely to influence State and local performance during the reference period, a Performance Standards Work Group was convened in March 1998. The Workgroup had representatives from State and local JTPA programs; and public interest groups, including the National Governors' Association, the U.S. Conference of Mayors, the National Association of Counties, and the National Council on the Aging. This Guidance Letter largely incorporates the Workgroup's recommendations.

  4. Highlights of Changes from PY's 1996 and 1997. Following is a summary of major changes from PY 1996 and 1997 policy:

  5. Performance Management Goals for PY's 1998 and 1999. The GPRA requires federal programs to develop quantifiable measures of outcomes, which JTPA has been doing for many years. It also requires a commitment to continuous performance improvement in the outcomes for those measures. Continuous performance improvement is a new challenge for some programs in the JTPA system, but it is a challenge that JTPA is well-positioned to accomplish through its performance standards system.

    In JTPA, performance standards have for many years marked the dividing line between minimally acceptable levels of program outcomes and unsatisfactory performance. State policy can also use performance standards to set a different, higher dividing line between rewardable levels of performance and satisfactory levels that do not fully contribute to the system's accomplishment of its performance goals. Policies that set higher levels of performance and performance improvement to qualify for incentive funds than the lower levels required to avoid sanctions have already been adopted in some States. The monetary incentives provided under JTPA have been and will continue to be strong motivators of organizational performance. Incentive funds can be used to motivate the system to meet the new requirements posed by GPRA while affording States continued flexibility in developing policies to meet these new challenges. The Employment and Training Administration (ETA) challenges all States to design their States' JTPA incentive policies to promote higher levels of performance and performance improvement.

    Departmental goals for JTPA's performance management system serving disadvantaged adults and youth and dislocated workers were submitted to Congress in September 1997 in the ETA Strategic Plan for FY 1997-2002. The Strategic Plan commits to continuous performance improvement each and every year of the 5 years covered. Those goals have since been revised in the FY 1999 Annual Performance Plan (see TEGL 8-97) to support the Secretary's vision, facilitate increased coordination and foster greater cohesion within ETA and the Department. To achieve its GPRA goals for PY 1998/FY 1999 and its mission, ETA has established for JTPA the following three strategic goals along with associated outcome objectives and performance targets:


    A Prepared Workforce: Enhance opportunities for America's Workforce.


    A Secure Workforce: Promote the economic security of workers and families.


        * Protect worker benefits.

        * Provide worker retraining.


        * Under JTPA Title III for dislocated workers, 74% of program terminees will be employed at an average wage replacement rate of 93% at termination and 76% will be employed one quarter after program exit at an average wage replacement rate of 97%.


    Quality Workplaces: Foster quality workplaces that are safe, healthy, and fair.


        * [None directly related to JTPA performance standards]

    These performance targets merit special attention because they directly relate to the subject of this Guidance Letter: the Secretary's JTPA Title II and Title III performance measures, national numerical standards for these measures, incentive award criteria, and sanction requirements.

    States have the authority to establish additional standards which reflect State goals, to develop the definitions of passing and failing individual standards, and to develop the specific approach to determining incentive awards. Data to support additional non-cost measures will continue to be reported, and States may use these measures or others in making State incentive award determinations for the up to 25% of incentive funds permitted for additional State standards under Section 106(e). Data on costs, together with program performance data, will provide critical information for State monitoring and fiscal oversight, and assist States in measuring returns on their human resource investments.

  6. Secretary's National Standards for PY's 1998 and 1999. The Secretary's performance measures and national standards for Title II-A, Title II-C, section 204(d) Older Worker programs, and Title III [all of section 302(c)(1) State activities, and sections 302(c)(2) and 302(d) substate area activities] are as follows:

    PY 1998 and 1999 Performance Standards Title II-A

    Adult Follow-up Employment Rate*: 60%
    Adult Weekly Earnings at Follow-up*: $289
    Welfare Follow-up Employment Rate*: 52%
    Welfare Weekly Earnings at Follow-up*: $255
    Adult Pilot Sustained Employment Rate* (optional): Baseline TBD
    Adult Pilot Sustained Quarterly Earnings*: (optional) Baseline TBD
    Welfare Pilot Sustained Employment Rate* (optional): Baseline TBD
    Welfare Pilot Sustained Quarterly Earnings* (optional): Baseline TBD

    Title II-C

    Youth Entered Employment Rate*:: 45%
    Youth Employability Enhancement Rate*: 40%
    Youth Positive Termination Rate* (optional): 72%

    Section 204(d) Older Worker Programs

    Entered Employment Rate: 56%
    Average Hourly Wage at Placement: $6.10

    Title III

    Entered Employment Rate: 73%
    Follow-Up Employment Rate (optional): 72%
    Average Wage Replacement Rate at Termination (optional): 91%
    Average Wage Replacement Rate at Follow-Up (optional): 93%
    Average Wage at Placement (optional): $9.32

    *Indicates Secretary's required "core" measures that are subject to Title II incentives and sanctions.

    The four Title II-A adult and welfare follow-up measures will continue to be calculated based on outcomes data for individuals who terminate during the first three quarters of the program year and the last quarter of the previous program year. The four optional pilot Title II-A measures based on wage record data will be calculated on data for individuals who terminate in the program year.

  7. Explanation of Performance Standards Levels. The Title II-A and II-C numerical standards were derived from PY 1995 and 1996 program data aggregated from the Standardized Program Information Report (SPIR). This approach for setting numerical levels is expected to produce local standards that are similar in difficulty to what SDAs/SSAs have experienced in the recent past. Past practice was to set numerical levels on non-earnings measures where at least 75% of SDAs can be expected to meet the standard, and to set levels for earnings measures where at least 60% of SDAs can be expected to meet the standard. The higher level was used for earnings measures to account for expected inflation in wage rates.

    Welfare reform and the new Welfare-to-Work programs are expected to have continuing substantial impacts on JTPA Title II-A program outcomes. Because welfare clients make up one of the most significant hard-to-serve groups included in the adult measures, the effects of welfare reform are reflected in both the adult measures and the welfare measures. Changes in the mix of hard-to-serve characteristics of welfare recipients served in JTPA, the emphasis on immediate placement at any wage, and the "earnings disregard" practice in many States are impacting the types of services provided welfare clients and the outcomes achieved. This appears to be slowing the rate of gain in welfare clients' earnings and impacting the employment retention rate. Therefore, national levels for the Title II-A measures impacted by welfare reform were moderated for this performance standards cycle.

    The actual levels of performance to be achieved by the JTPA system nationwide in PY's 1998 and 1999 are expected to be higher than the levels of the departure points, all other things being equal. The departure points are viewed as levels of minimally acceptable performance for local programs, and are therefore set at minimums ranging from the 20th to the 35th percentiles. The JTPA system's national average performance can be expected to approximate the 50th percentile of performance, all other things being equal. These distinctions are critical in any discussion of whether the national departure points are sufficient for JTPA to attain its GPRA performance goals.

  8. Title II-A Adult Measures. The Adult Follow-up Weekly Earnings level was set at the 35th percentile, the level at which 65 percent of SDAs would be expected to meet or exceed the standard based on past experience. Past experience indicates that inflation would have the effect of raising above 65 percent the number of SDAs that meet or exceed the earnings standards. The Welfare Follow-Up Weekly Earnings level was continued at the 25th percentile; it had been moderated in the previous 2-year cycle to account for expected impacts of welfare reform. The level for the Welfare Follow-Up Employment Rate was set at the 20th percentile because feedback from local programs indicates that increasingly hard-to-serve welfare recipients required by welfare reform to take immediate employment are changing jobs and employers within the initial 13 week period. The level of the Adult Follow-Up Employment Rate was continued at the 25th percentile level, which results in a modest increase of 1 percentage point over the previous level and is believed to be an achievable yet challenging goal for Title II-A.

  9. Title II-C Youth Measures. Because Title II-C funding was significantly scaled back several years ago, many smaller local areas are experiencing difficulty running programs serving both out-of-school youth and in-school youth, which typically have different expected outcomes. The optional Youth Positive Termination Rate adapts to the changing youth program mix by permitting positive outcomes from both strategies to be combined in one measure. The departure point for the Youth Positive Termination Rate was set at the 25th percentile.

    The youth employability enhancement outcome has been clarified to include youth who go on for postsecondary education. The definition of "Entered Non-Title Training" now includes youth who go on to postsecondary education for at least one academic quarter. They can be counted as a positive termination for either the Youth Employability Enhancement Rate or the Youth Positive Termination Rate. Attachment 2 contains more complete guidance on youth employability enhancement definitions.

  10. Title III Measures. Similar to the Title II-A and Title II-C standards, the required Title III standard for PY 1998 and 1999 was derived from PY 1995 and 1996 program data aggregated from the SPIR. This standard is also set at a level that approximately 75 percent of the substate areas can be expected to meet or exceed. There are no core measures for Title III because incentives and sanctions are not required in Title III programs.

    There are four additional optional measures for Title III. The Average Wage at Placement has been optional in previous years, and will continue to be optional. The national departure point for this measure was set at a level that approximately 65 percent of the substate areas can be expected to meet or exceed.

    The other three optional Title III measures, Follow-Up Employment Rate, Average Wage Replacement at Termination, and Average Wage Replacement at Follow-Up are newly introduced this performance standards cycle, although they have been used to track Title III performance goals under GPRA. National departure points for survey-based measures were set at the 25th percentile of performance, levels which about 75 percent of substate areas can be expected to meet or exceed.

    States may choose to use wage record data instead of telephone surveys to track postprogram employment and earnings in Title III. Instructions on reporting Title III outcomes and computing performance will be issued separately at a later date.

    Adjustment models and departure points will not be available for setting local wage-based follow up standards in PY l998. In future performance standards cycles, after developmental research is completed, ETA will provide adjustment models and departure points for the full complement of Title III performance standards measures whether they are based on surveys or UI wage records.

    For consistency in performance measurement, all Title III measures have been defined to exclude terminees who were called back or remained with their layoff employers (see Attachment 1).

  11. Section 204(d) Older Worker Program Performance Standards.  Performance standards levels for the Section 204(d) Older Workers program were set at the 25th percentile of performance for both the Entered Employment Rate and the Average Wage at Placement. Programs operated under section 204(d) are State programs even though they may be managed by various local entities. Therefore, performance standards will be applied to each State's entire Section 204(d) Older Worker program. Unlike the adult and youth programs under Title II-A/C, however, no incentive awards or sanctions are associated with these standards.

  12. Title II-A Optional Pilot Measures. The four Title II-A wage record-based measures may be substituted for the four follow-up measures. The State must specify that the four required core measures for Title II-A be either all of the required telephone follow-up measures or all of the optional pilot wage record-based measures. Of the State's specified core measures, at least one must be an adult employment measure, one must be an adult earnings measure, one must be a welfare employment measure, and one must be a welfare earnings measure.

    For States that opt for pilot wage record-based post-program measures, there will be certain requirements that States must meet to implement this new approach to measuring employment retention. These requirements are necessary to allow ETA to manage the transitional dual reporting system that will result when some States adopt wage-based retention measures while other States continue with telephone survey follow up. The requirements are outlined in Attachment 6.

    One requirement is that States choosing wage records as their data source during the next 2-year performance standards cycle need to ensure that they will be able to manage their local programs and fully comply with federal reporting requirements. Also, ETA plans to develop a mathematical algorithm to help explain variations in outcome levels between the two measurement systems. Therefore, it will be necessary for States to provide a 5 percent statewide sample of follow-up survey data for PY 1998 (and/or PY 1999, if applicable). Because the pilot retention measures reference the second full quarter after termination, and because there is a further delay of two to three additional quarters to obtain the wage record data, performance information for making incentive awards and imposing sanctions will not be available for a considerable period of time. States will be required to develop alternative incentive and sanctions policies to accommodate this lag time.

    The optional wage record-based employment retention measures will be pilot-tested during PYs 1998 and 1999; therefore, the Department has established baseline goals instead of national departure points for measuring PYs 1998 and 1999 performance. In performance reports to the public, the Department plans to identify separately performance based on pilot measures and estimates of comparable national performance on follow-up measures.

    ETA will issue implementing guidance for the pilot wage record-based sustained employment and earnings measures, including optional adjustment models, in a future directive. The Department will also develop a technical assistance plan during this pilot test period to provide assistance in the use of wage records for performance measurement, and at the same time, inform users about the implications for local performance results when using these pilot measures.

  13. Optional Youth Positive Termination Rate. States may choose to adopt the single core Youth Positive Termination Rate to take the place of the two core measures Youth Entered Employment Rate and Youth Employability Enhancement Rate.

  14. Implementing Provisions. The following implementing requirements must be followed:

    1. Required Standards. For Titles II-A and II-C, States are required to set, for each SDA, a numerical performance standard for each of the core Secretary's measures; for the Older Worker program, States are required to set numerical Entered Employment Rate and Average Wage at Placement standards for programs operated under Section 204(d); for Title III, States are required to set for each substate area a numerical performance standard for the Entered Employment Rate; for Title III Governor's Reserve programs, States are required to set a statewide standard; and for Title III, States are also strongly encouraged to adopt and set numerical levels for all of the optional GPRA measures for substate programs and for Governor's Reserve programs.

    2. Setting the Standards. Consistent with provisions in JTPA, States are required to adjust the Secretary's performance standards to reflect local area circumstances (Section 106(d)). Such adjustments apply to Title II-A, Title II-C, Section 204(d) and Title III programs, and must conform to the Secretary's parameters described below:

      1. Procedures must be:

        • Responsive to the intent of the Act,

        • Consistently applied among the SDAs/SSAs,

        • Objective and equitable throughout the State,

        • In conformance with widely accepted statistical criteria;

      2. Source data must be:

        • Of public use quality,

        • Available upon request;

      3. Results must be:

        • Documented,

        • Reproducible; and

      4. Adjustment factors must be limited to:

        • Economic factors,

        • Labor market conditions,

        • Geographic factors,

        • Characteristics of the population to be served,

        • Demonstrated difficulties in serving thepopulation, and

        • Type of services to be provided.

        The Department offers States an optional adjustment methodology that conforms both to these parameters and to the requirement in section 106(d). This methodology covers Title II-A, Title II-C, Section 204(d), and Title III programs and will be provided to States in a future directive. Should the State choose to use an alternate methodology, or make adjustments not addressed by the Departmental model, it must conform to the parameter criteria and be documented in the Governor's Coordination and Special Services Plan for the program year to which it applies.

        The State Job Training Coordinating Council and, where appropriate, the State Human Resources Investment Council must have an opportunity to consider adjustments to the Secretary's standards and to recommend variations. To determine whether an SDA has met or exceeded a performance standard, States must use actual end-of-year program data to recalculate the performance standards.

      5. /strong> Performance Standards Definitions. States must calculate the performance of their SDAs, SSAs, and section 204(d) programs according to the definitions included in the Attachments.

      6. Titles II-A and II-C Incentive Policies. ETA encourages States to develop incentive policies that encourage higher levels of performance, sustained high performance and performance innovation for local areas to receive incentive awards. (As discussed later, States should also develop parallel sanction policies that distinguish between unacceptable levels and those that are acceptable, but not rewardable.)

        States are to develop and implement policies and procedures for awarding incentive grants in accordance with Section 106(b)(7). As the basis for making incentive awards, the State must use all and cannot "zero weight" any of the Secretary's core measures. At least 75 percent of the funds set aside for performance incentives must be related to these measures and, if applicable, the out-of-school youth and employer-assisted benefits criteria, in accordance with section 106(b)(7)(E). Up to 25 percent of the funds set aside may be used to reward performance on additional State standards, excluding cost standards. The following criteria are required in States' incentive award policies:

        1. Standard for Service to Hard-to-Serve. A Secretary's standard for service to the hard-to-serve, as required by section 106(b)(7)(B) of JTPA, has been established in the form of a stand-alone eligibility criterion ("gate") for incentive awards.

          In order for an SDA to be eligible to receive ANY incentive award, at least 65 percent of BOTH the SDA's (a) Title II-A AND (b) Title II-C (in-school and out-of-school youth combined) participants receiving training and/or other services beyond objective assessment must be hard-to-serve. The definitions of hard-to-serve are to be consistent with the definitions in sections 203(b), 263(b), and 263(d) of the Act.

        2. Amount of Incentive Award. For those SDAs that successfully "pass through" the gate, (in addition to any funds set aside for State standards) the amount of the incentive award for SDAs exceeding the Secretary's performance standards will be determined by the State's policy in conformance with DOL requirements.

        3. Exceeding All Core Standards. SDAs that pass through the "gate" and exceed all of the Secretary's Titles II-A and II-C core standards by the State's definition of exceeding standards must receive an incentive award.

        4. Additional State Measures. States also may select additional non-cost measures, such as increased service to hard-to-serve participants, to include in incentive policies.

        5. Cost Standards Not Allowed. Cost standards cannot be used for incentive award purposes. However, States are reminded of the integral role of financial reviews in program management. States are encouraged to explore ways of relating overall costs of job training to more direct measures of long-term employment, earnings and reductions in welfare.

        6. Adjustments to Incentive Awards. Incentive policies may include adjustments to incentive award amounts based upon factors such as grant size, additional services to the hard-to-serve, intensity of service, and expenditure level.

        7. Excluding Pilot Projects. In PYs 1998 and 1999, States will continue to have the authority to exclude pilot projects serving "hard-to-serve" individuals funded from the 5 percent incentive fund set-aside in computing their standards and actual performance. States and SDAs are encouraged to use such funds to develop or replicate model programs serving out-of-school youth.

          NOTE:  For those SDAs in which "incentive projects" are indistinguishable from those that provide general training, these programs would not be considered exempt from performance standards.

        8. Incentives Using Pilot Measures. States using optional pilot wage record-based performance measures must develop incentive policies that also take into account the lag time in availability of wage record earnings data.

        9. Additional Incentive Criteria. States are encouraged to include in their incentive policies criteria relating to: 1) attaining levels of performance and performance improvement that assist the JTPA system to attain its annual and strategic performance goals under the GPRA; 2) programs successfully serving out-of-school youth, and 3) placement in jobs providing employer-assisted benefits. Although successful programs for out-of-school youth remain the cornerstone of out-of-school incentives, SDAs will still be expected to exceed the 50 percent minimum service level to be rewarded nder that criterion. More information is included in the Attachments.

      7. Titles II-A and II-C Technical Assistance and Sanction Policies. ETA encourages States to develop sanction policies that treat performance standards as minimal levels of acceptable performance. (States should also develop parallel incentive policies that encourage higher levels of performance and performance improvement for local areas to receive incentive awards. These policies should distinguish levels of performance that are rewardable from those that are acceptable but not rewardable.)

        Determination of an SDA's failure to meet standards and the consequent imposition of technical assistance and reorganization requirements under section 106(j) will be based only on the Secretary's Title II-A and Title II-C core measures. "Meeting Performance Standards" overall is defined as meeting more than half of the core standards. Conversely, overall "failure to meet performance standards" is defined as failing half or more of the core standards. Definitions for meeting and failing individual standards will be established by States.

        Regardless of whether the State opts for five or six core standards (four adult standards and either one or two youth standards), by these definitions overall failure to meet performance standards means failing three or more core standards. Further, meeting performance standards overall means meeting or exceeding at least four out of six core standards or at least three out of five core performance standards.

        Overall failure to meet performance standards as defined above for the first year precludes an SDA from receiving any incentive awards and requires States to provide technical assistance to the underperforming SDA.

        Overall failure to meet performance standards as defined above in the second consecutive year precludes an SDA from receiving any incentive award and requires States to impose a reorganization plan.

        Furthermore, in the past, the overall effect of States' performance standards adjustment policies, including adjustments beyond the models, may have sustained or encouraged levels of performance outcomes lower than they otherwise might have been in the absence of Governor's adjustments. In some States, the effect of Governor's adjustments across the board may have been to lower all the local programs' performance standards levels below the level of minimally acceptable performance established by the national departure points. States also have the option to adopt additional adjustments that raise the levels higher than they otherwise would be, but that has not often happened in the program's experience.

        Reducing standards in this way has the disadvantage of reducing the performance levels at which SDAs qualify for incentive awards in many states, and may possibly encourage and sustain lower levels of performance for all SDAs. ETA strongly encourages States that set policies that lower the national departure points across the board in the State to also adopt more challenging levels to determine eligibility for incentive awards.

  15. Performance Status Summary Reports. Section 106(j)(3) requires each State to report to the Secretary, not later than 90 days after the end of each program year, the actual performance and performance standards for each SDA within the State. Within the same time frame, technical assistance plans developed by the State are required for each SDA failing to meet performance standards overall for the first year. The 90-day time frame also applies to the imposition of a reorganization plan, which is mandatory when an SDA fails to meet performance standards overall for a second consecutive year. Specific procedures for the formal performance standards report and required State action were provided in Training and Employment Guidance Letter 2-95, dated August 10, 1995.

    However, in addition to the formal annual process, there should be ongoing oversight of SDA performance and continuous technical assistance and capacity-building aimed at addressing areas where program performance can be improved.

    In the future, the submission of quarterly performance reports, in addition to annual reports, may be necessary in order to effectively demonstrate progress toward annual and strategic GPRA goals. While no quarterly submission is currently required in order to document actual performance, States should be aware of this possibility.

  16. Action Required. The following actions are required of States:

    1. Distribution. States must distribute this Guidance Letter to all officials within the State who need such information to implement the performance standards policies and requirements for PY 1998 and 1999.

    2. State Plan Development. States must specify in the GCSSP their incentive award policy under section 202(c)(1)(B) and 202(c)(3)(A) and policy for imposition of sanctions under section 106(j). As the timing of this issuance may have precluded States from submitting this information with their PY 1998 and 1999 GCSSP, if necessary, States should submit a GCSSP amendment containing complete information on Title II incentive and sanction policies no later than August 31, 1998. (Such amendments are not required for Title III plans.)

  17. Inquiries. Questions concerning this issuance may be directed to Valerie Lloyd at (202) 219-5487, ext. 107.

  18. Attachments.

    1. Definitions for Performance Standards

    2. Youth Employability Enhancement Definitions

    3. Rewarding Model Programs for Out-of-School Youth

    4. Rewarding Placements in Jobs Providing Employer-Assisted Benefits

    5. Incentive Policies that Promote Continuous Performance Improvement

    6. Guidelines for Substitution of Wage Record Post-Program Measures in Lieu of Survey Follow-Up in JTPA Performance Standards for Program Years 1998 and 1999