Attachment A UIPL 07-99

BPC MEASURES PROJECT EVALUATION

The Benefits Payment Control Measures Project funded three States -- West Virginia, Utah, and California -- to test 10 measures proposed to supplement or replace the two measures used for BPC in the Quality Appraisal system.The indicators are in the general categories of:

• establishment effectiveness;

• collection effectiveness; and

• aging of receivables.

Each SESA gathered data during the project for two quarters and transmitted it to the National Office. Some of the data are identical to those in the ETA-227 and ETA-5159 Reports. Also included are data that are similar but arrayed differently, i.e., the ETA-227 groups data into the categories of State UI and UCFE/UCX; however, for this project data has been grouped into the categories of intrastate and interstate, with UCFE and UCX included within these two groupings.

Upon receipt from the SESAs, raw data was entered into the National Office data base where calculations were made to obtain the percentages for each measure. The results were returned to the SESAs for evaluation. (See Attachment B for report data for one quarter.) Each SESA was then asked to comment on the following:

• the feasibility of collecting the data,

• the usefulness of the data,

• the desirability of reporting data separately for interstate claims, UCFE claims, and UCX claims,

• the desirability of revising the intervals used to report data for the measures involving time lapse and case aging, and

• other recommendations/suggested measures.

1.  Measures Tested

  a.   Establishment Effectiveness. Two measures examine the effectiveness of establishing overpayments, one using overpayment cases and the other using overpaid dollars:

The test States commented that these two measures could provide valuable trend information over the long term (trend analyses), but they don't provide immediately useful program information for SESA managers. They did not favor these as performance measures.

These measures differ only in that one is based on cases and the other on dollars. The PEG believed that the most emphasis should be placed upon dollars and recommended measure number ² for UI PERFORMS. (See the discussion of the proposal in the UIPL.)

  b.   Collection Effectiveness. This grouping consists of six measures for the effectiveness of collecting and/or otherwise disposing of (waiver, write-off) overpayment cases or dollars.

The only one proposed for inclusion into UI PERFORMS is measure number ³:

The test States indicated that measures ³ and ⁸ could be useful management tools. The other measures (⁴,⁵, ⁶,and ⁷)received unfavorable comments.

The PEG considered waivers and write-offs and concluded that it would not be appropriate to measure either because there are positive and negative aspects of both, i.e., it is good for SESAs to utilize waivers in the interest of equity and write-offs to purge uncollectible debt from the books, but it would not be good to encourage use of either in lieu of collecting overpayments that should be recovered. Thus, measures ⁴ ,⁵,⁶,and ⁷ are not recommended.

A substantial portion of overpayment dollars recovered are accomplished via offset. By measuring cases closed, as proposed in measure ⁸, two overpayments established for $800 would show "equal" performance (nothing accomplished) if $750 had been recovered for one while $0 had been recovered for the other. To avoid placing undue emphasis on collecting the last few dollars outstanding for some cases while not giving attention to larger amounts collectible for other cases, it was concluded that this measure should not be used as an indicator of performance.

See the UIPL for discussion of the recommendation to recommend a modification of measure number ³ for UI PERFORMS.

  c.   Aging. Two measures were field tested that record the age of outstanding overpayments -- one of cases and the other of dollars. They are not recommended for inclusion in UI PERFORMS.

The test States commented that these are useful and valuable measures and would make good monitoring tools.

The PEG agreed that these data could be useful but believed that they would be mostly limited to internal SESA management of operations.

2.  SESA Comments on Other Aspects of Testing.

  a.   Category for UCFE/UCX. Although UCFE/UCX is a separate category for the ETA-227 Report, it was not for this project. All three of the test States indicated that it is desirable to retain separate breakouts for UCFE/UCX.

The Department of Labor has the responsibility to monitor the Federal UI programs, and this requires the ability to identify and measure performance of UCFE and UCX claims. Therefore, this category must be retained for the ETA-227 reporting. The PEG acknowledged this but recommended that performance be measured in aggregate for UI PERFORMS.

  b.   Category for Interstate. Interstate is not currently broken out as a separate category on the ETA-227 Report; however, for this project, interstate was reported separately. The recommendations from the test States varied -- from "needed" to "helpful" to &quopt;not needed".

The discussion at the PEG meeting led to the conclusion that it would be beneficial for management purposes to have this information reported in a separate category, and SESAs are encouraged to track this internally. However, to be consistent with other measures in UI PERFORMS, it was recommended that only aggregate totals be measured.

  c.   Intervals for Time Lapse and Case Aging. Three of the tested measures involved counts of days to record time lapse and case aging. The intervals used in the test differed from those currently reported for the ETA-227, including an increase in the number of categories in the test from six to nine. The categories are displayed in the following table:

Test results regarding reporting intervals were inconclusive. Therefore, at this point, the PEG is not recommending alteration of the existing format.

  d.   Other Comments. The test States were invited to recommend other performance indicators that might be considered and also to make any other pertinent comments related to the project and the evaluation of performance in the area of BPC. Comments received were:

(1)  The percentages of achievement are not really useful without Desired Levels of Achievement (DLAs) for individual measurements that allow us to assess whether we are exceeding, meeting, or failing performance expectations. Establishing DLA standards based on a National average or the average of a group of states with similar laws and regulations appears to be the most equitable.

  UI PERFORMS has established a system with two tiers or categories of performance objectives. Tier I measures represent key aspects of UI activities with federal, system-wide criteria distinguishing adequate from sub-par performance that must be addressed in every planning cycle. Tier II measures represent activities for which SESA management and Regional Administrators negotiate annually State-specific performance goals.

(2)  We (test State) would like to see a measurement that compares dollars recovered vs. dollars spent.

Although SESAs may track "dollars spent", the National Office does not do so because the States have bottom line authority on staffing for BPC. The PEG gave consideration to a similar concept -- overpaid dollars established compared to staff positions allocated and overpaid dollars recovered compared to staff positions allocated. The consensus was that these could be useful internal SESA management tools but would not be recommended for UI PERFORMS.