DIRECTIVE	:	TRAINING AND EMPLOYMENT INFORMATION NOTICE NO. 13-00
TO	:	ALL STATE WORKFORCE LIAISONS ALL STATE EMPLOYMENT SECURITY AGENCIES ALL STATE WORKER ADJUSTMENT LIAISONS ALL ONE-STOP CAREER CENTER SYSTEM LEADS
FROM	:	Wendy L. McConnell for LENITA JACOBS-SIMMONS Deputy Assistant Secretary
SUBJECT	:	Consultation Paper on Labor Exchange Performance Measurement System

1. Purpose. To notify all appropriate workforce development system agencies of the labor exchange performance measurement system that the Employment and Training Administration (ETA), U.S. Department of Labor (DOL) proposes to implement for the public labor exchange during program year (PY) 2001. This system will consist of three components: (1) a set of labor exchange performance measures, (2) reporting requirements for labor exchange services and performance outcomes, and (3) procedures for establishing performance goals that State agencies and ETA can use in assuring the delivery of high quality labor exchange services. This notice conveys ETA's current position on the framework for the labor exchange performance measurement system in order to facilitate planning for implementation beginning July 1, 2001.

2. Background. Under sections 3(a), 3(c)(2), 7(b), 10(c), 13(a), and 15(e)(2)(l) of the Wagner-Peyser Act (29 U.S.C. 49), ETA initiated the development of a performance measurement system for the public labor exchange in early 2000. ETA formed workgroup in collaboration with the Interstate Conference of Employment Security Agencies (ICESA) - now the National Association of State Workforce Agencies (NASWA) - consisting of representatives from fifteen State agencies, ICESA, the Veterans' Employment and Training Service (VETS), and the ETA Regional and National Offices. The workgroup met three times during 2000 to develop recommendations for a labor exchange performance measurement system.

   Based on recommendations the workgroup developed during its first two meetings, ETA published a set of five labor exchange performance measures in the Federal Register at 65 FR 49708-14, (August 14, 2000). These measures were: employer customer satisfaction; job seeker customer satisfaction; employment rate; entered employment rate; and employment retention rate at six months. ETA also published a framework for establishing expected levels of performance for each of these measures./P>

   During its third meeting, the workgroup reviewed and analyzed all comments received on the Federal Register notice. The review and analysis led to the proposed final set of performance measures presented in this document. The workgroup also provided substantial input that will lead to revised ETA 9002 Reports as well as a revised ETA Handbook No. 406 . Finally, the workgroup recommended that the procedures for establishing expected levels of performance be revised, based on the comments received.

3. Description. Based on consideration of the comments received on the proposed labor exchange performance measurement system published in the Federal Register on August 14, 2000, and the recommendations of the labor exchange performance measurement system workgroup, ETA proposes to establish a labor exchange performance measurement system for implementation beginning July 1, 2001. This TEIN conveys the major concepts of the new performance measurement system and provides an overview of the proposed final performance measures.

   The labor exchange performance measurement system will apply to public labor exchange services provided as part of the One-Stop delivery systems of the States. This includes labor exchange services provided to job seekers and employers under the Wagner-Peyser Act, and to veterans by Disabled Veterans' Outreach Program (DVOP) and Local Veterans' Employment Representative (LVER) staff under VETS programs, as specified in Title 38, U.S.C. Other publicly-funded labor exchange services may be included in the labor exchange performance measurement system at the discretion of individual States.

   The labor exchange performance measurement system will apply to job seekers who are registered applicants with the public labor exchange and to employers who receive substantial service involving personal contact with One-Stop staff. Registered applicants will be defined as all job-seeking customers who complete registration with the labor exchange. Required elements of registration will include: name, contact information, Social Security Number, demographic information, employment status, and date of registration. Applicants may be registered upon contacting the labor exchange through the One-Stop delivery system or as required by State law or policy; however, applicants receiving staff-assisted services funded under the Wagner-Peyser Act must be registered. Job seekers who seek self-services or facilitated self-help services also may be registered. A job-seeking customer will be counted as a registered applicant only once during a registration year, defined as the calendar quarter during which registration occurs (registration quarter) and the subsequent three quarters. Registered applicants who receive services during the fourth quarter after the registration quarter should be re-registered. Such applicants would then begin a new registration year.

   a)  Labor Exchange Performance Measures

   Four labor exchange performance measures are proposed for the public labor exchange:

   • Job Seeker Entered Employment Rate

   • Job Seeker Employment Retention Rate at Six Months

   • ob Seeker Customer Satisfaction

   • Employer Customer Satisfaction

     1)  Job Seeker Entered Employment Rate (JSEER):

       # Entered Employment with a New Employer
     

     JSEER = ------------------------------------------------
     
       # Registered Applicants - # Employed or Re-employed with Same Employer

     Elements of the measure are defined below:

     Entered Employment with a New Employer:  The number of registered applicants who, in the first or second quarter following the registration quarter, earned wages from a new or different employer than that from which the registered applicant earned wages in the quarter prior to registration.

     Registered Applicants:  All job-seeking customers who registered or re-registered with the labor exchange during any of the previous four calendar quarters.

     Employed or Re-employed with Same Employer:  Those applicants whose wages earned in the first and second quarter following registration were exclusively with the same employer from which wages were earned in the quarter prior to registration.

     ETA is proposing to establish this measure for the labor exchange in response to comments received on the employment rate and entered employment rate measures initially proposed in the August 14, 2000 Federal Register Notice. The newly-defined Job Seeker Entered Employment Rate combines concepts from the two measures initially proposed, and accounts for the successful employment outcomes of both incumbent workers and individuals who were unemployed at the time of registration who enter new employment after registering with the labor exchange.

     According to this measure, a successful employment outcome will be recorded for a job seeker who enters employment with a new employer, whether or not the job seeker was employed at the time of registration. This will be determined by comparing the employer identification numbers (EIN) of registered applicant's employers prior to and following registration based on information contained in the unemployment insurance (UI) wage record database, the State Directory of New Hires (SDNH) database, or other available records. An unsuccessful outcome will be recorded for a job seeker who does not enter employment with a new employer during the reporting period. Job seekers who remain employed exclusively with the same employer during the measurement period will be excluded from measurement.

     2)  Job Seeker Employment Retention Rate at Six Months (JSERR):

       # Retained Employment Two Quarters after Entered Employment with a New Employer (age 19 and over)
     

     JSERR = ------------------------------------------------
     
         # Entered Employment with a New Employer (age 19 and over)

     Elements of the measure are defined below:

     Retained Employment Two Quarters after Entered Employment with a New Employer (age 19 and over):  The number of registered applicants age 19 and older at the time of registration who earned wages in the second quarter following the quarter in which they Entered Employment with a New Employer.

     Entered Employment with New Employer (age 19 and over):   The number of registered applicants age 19 and older at the time of registration who, in the first or second quarter following the registration quarter, earned wages from a new or different employer than that from which the registered applicant earned wages in the quarter prior to registration.

     ETA is proposing to establish this measure for the labor exchange based on comments received on the employment retention rate at six months measure initially proposed in the August 14, 2000 Federal Register notice. The retention measure proposed earlier has been modified to make the measurement cohort for employment retention consistent with the job seekers for whom a successful outcome was recorded in the job seeker entered employment rate measure. Further, job seekers who are under age 19 at the time of registration are now excluded from the measure. Job seekers under age 19 are excluded from the measure because there is a high likelihood of their being students and not seeking to work year round.

     According to this measure, a successful employment retention outcome is recorded for job seekers, age 19 and over, who were determined to have entered employment according to the job seeker entered employment rate measure, and who were found through wage record matching to be employed in the second quarter following the quarter in which they first were determined to have entered employment. A successful outcome for retention is recorded for employment identified through wage record matching with any employer.

     3)  Job Seeker Customer Satisfaction

     ETA is proposing to establish a job seeker customer satisfaction measure for the public labor exchange that mirrors the Workforce Investment Act of 1998 (WIA) participant customer satisfaction survey and uses the American Customer Satisfaction Index

     (ACSI) methodology. State agencies will be able to exercise discretion in how they administer the survey, so long as the ACSI methodology is followed. Possibilities might range from surveying a population of registered applicants in a distinct survey to coordinating the job seeker customer satisfaction survey with the WIA participant customer satisfaction survey or any customer satisfaction survey that might be administered by VETS or another One-Stop partner program. In all cases, sufficient sample sizes must be maintained to provide statistically valid results (technical requirements for sample sizes, response rates, and confidence intervals will be published in a future notice). ETA supports the concept of common measurement techniques for services provided as part of the One-Stop delivery system and intends to provide States with the broadest opportunity to coordinate surveys of One-Stop customers' satisfaction.

     4)  Employer Customer Satisfaction

     The public labor exchange will adopt the results of the ACSI survey administered under WIA to measure employer satisfaction with One-Stop services as an indicator of employers' satisfaction with labor exchange services. Accordingly, States are encouraged to conduct one survey of employers to measure their satisfaction with One-Stop employer services to meet both the WIA and the public labor exchange employer customer satisfaction measurement requirements.

     ETA proposes to adopt the WIA employer customer satisfaction measure for the labor exchange because the employer population from which the sample is drawn for the WIA survey consists of employers who received a substantial service involving personal contact with One-Stop staff - to include labor exchange services. Using a common measure to obtain information on employer customer satisfaction for both WIA and the labor exchange supports the integration of the labor exchange into the One-Stop delivery system. It also emphasizes the importance of providing high quality services to employers, a focus of the One-Stop delivery system. This measure is limited in scope to the population of employers who receive services involving personal contact with One-Stop staff.

   b)  Reporting

   State agencies will report data on labor exchange services provided to job seekers, and on job openings employers list with the labor exchange, based on information contained in administrative records. State agencies are required to collect and maintain information to support labor exchange program reporting under OMB No. 1205-0001, Work Application/Job Order Recordkeeping for job seekers registered with the labor exchange and for employers placing job orders. ETA is revising the elements of services and outcomes required to be reported by category of job seeker to better reflect the services provided by the modern labor exchange, and to meet current Federal data collection requirements. ETA also is revising the elements required to be reported about job openings employers list with the labor exchange to be consistent with the Occupational Information Network - Standard Occupational Classification (O*NET-SOC) system and the North American Industry Classification System (NAICS). State agencies will need to transition to using the new classification systems for identifying job openings during PY 2001.

   ETA also proposes to require State agencies to report employment outcomes of job seekers, including veterans, as well as the job seeker and employer customer satisfaction scores. Because employment outcome information is dependent on wage record data, it will not be available at the same time as information on the number of job seekers who are registered applicants or who receive labor exchange services. Therefore, outcome information for job seekers and veterans will be collected on separate reports from information on services. The chart below describes the information that will be collected on each of the proposed ETA 9002 reports. Detailed instructions and reporting specifications for labor exchange services to job seekers, performance outcomes, and job openings listed with the labor exchange will be published by ETA in a future notice in the Federal Register.

   Report Page	Description
   ETA 9002 A	Services to Job Seekers
   ETA 9002 B	Services to Veterans
   ETA 9002 C	Performance Outcomes - Job Seekers
   ETA 9002 D	Performance Outcomes - Veterans
   ETA 9002 E	Job Openings Received by Occupation (O*NET-SOC) and Industry (NAICS)

   The employment outcomes of job-seeking applicants registered with the labor exchange will be derived by matching the Social Security Numbers of registered applicants with employment information contained in State databases, including the UI wage record database, the SDNH database, or any other records the State agency may have access to that reliably indicate entry into employment. ETA anticipates that State agencies will have the option of using the wage record interchange system (WRIS) to obtain UI wage record information from other States as this system is implemented.

   Concurrent with ETA's proposed modifications to the ETA 9002 reports, VETS will propose a revised VETS 200 report and instructions that will be consistent with ETA's reports. ETA and VETS are coordinating their respective performance measurement systems to assure that labor exchange services, including services to veterans by DVOPs and LVERs, are measured and reported in a consistent manner.

   ETA plans to implement a rolling four-quarter reporting period for the labor exchange performance measurement system. For the job seeker and veterans reports, cohorts of registered applicants will be identified according to their quarter of registration. Reports will include the most current data available for each reporting element for applicants registering in any of four consecutive quarters. States will submit information on the Services to Job Seekers (ETA 9002 A) and Services to Veterans (ETA 9002 B) reports forty-five days following the completion of each calendar quarter. The reports will include information on the number of new and renewed registered applicants and the services provided to them during the four quarter reporting period. Data on performance outcomes, to be reported on the Performance Outcomes: Job Seekers (ETA 9002 C) and Performance Outcomes: Veterans (ETA 9002 D) reports, often will not be available until several quarters after a job seeker has registered with the labor exchange. States will report information on the ETA 9002 C and D reports forty-five days following the completion of the quarter in which outcome data become available for four consecutive cohorts of registered applicants. As outcome data will become available for different performance measures at different times, the cohorts of registered applicants for which data are being reported will be identified on the reports next to each performance measure. (Note: during transition to the new reporting system, it may be necessary to report less than four consecutive quarters of data until the system is fully implemented).

   This system of reporting will provide program managers with the most current information available for program management, while also allowing for performance outcome information to be paired with information on services for purposes of evaluation, once complete information on a particular cohort becomes available. The rolling four quarter reporting period will provide program managers and other users of labor exchange information with the most current data available for a one-year period of time, during any quarter of the year. This system of reporting also will diminish the impact of seasonal variations in the reports on services or outcomes by always including each of the four calendar quarters in any particular reporting period. Once complete information for the four cohorts registering during a particular program year becomes available, final program year reports may then be assembled. This information will then be used to assess a State's success in meeting its performance goals.

   c)Performance Goals

   Performance goals for each of the four labor exchange performance measures will be incorporated into State Strategic Five-Year Plans for Title I of the Workforce Investment Act of 1998 and the Wagner-Peyser Act or State Five-Year Unified Plans, as is currently done for WIA-Title I performance goals, through a modification to the plan. Based on comments received from the August 14, 2000 Federal Register notice, ETA is exploring the possibility of developing an econometric model that would account for State and regional economic conditions and the socioeconomic characteristics of applicants to use in facilitating the process of adjusting State levels of performance on the job seeker entered employment rate and job seeker employment retention rate at six months measures. Details of the goal-setting procedure and possible econometric model will be published in a future notice.

   As employment outcome information derived from UI wage records or the SDNH database for the job seeker entered employment rate and job seeker employment retention rate at six months measures will not become available for four to six quarters after implementation of the new system, ETA is proposing to use the PY 2001 transition year to develop baseline data. The baseline would be constructed by applying the labor exchange performance measures to job seekers who registered with the labor exchange during PY 2000. This baseline would then be considered in the establishment of performance goals for future program years.

4. Implementation. Following the dissemination of this TEIN, ETA will be publishing the proposed final labor exchange performance measures as a notice in the Federal Register. Also, ETA will be publishing the proposed ETA 9002/VETS 200 Reports and the proposed ETA Handbook No. 406 in the Federal Register for comment. Details on the transition to, and implementation of, the new system, including the use of NAICS and O*NET - SOC codes, will be published in a separate notice.

   ETA intends to provide State agencies with technical assistance and training on a variety of areas associated with implementation of the new labor exchange performance measurement system.

5. Action Required. State agencies are requested to distribute copies of this TEIN to all relevant workforce development partners including Employment Service directors and Information Technology directors.

6. Inquiries. Questions and comments concerning this TEIN should be directed to the appropriate Regional Office.