Community Development Block Grant (CDBG) Program - Youth Services

Community Development Block Grant (CDBG) Program - Youth Services is sponsored by City of Milwaukee Community Development Grants Administration (CDGA). The City of Milwaukee CDGA, on behalf of the Milwaukee Youth Council, accepts applications for Youth Services projects supported by CDBG funding. Eligible projects focus on Youth Homelessness Intervention and Youth Suicide Prevention & Intervention.

# Milwaukee’s Community Development Block Grant # Connecting Youth Services and Economic Development Nina Carlson Julie Fitzgerald Kevin Murphy Bethany Shore Prepared for the City of Milwaukee Budget and Management Division, Department of Administration Workshop in Public Affairs, Domestic Issues Public Affairs 869 Spring 2007 Figures & Tables....................................................................................................

iv Foreword ................................................................................................................. v Acknowledgments................................................................................................. vii Executive Summary ...............................................................................................

ix Introduction............................................................................................................. 1 Status Quo: Rationale for CDGA-Sponsored Youth Services Grants ................ 2 Research Question #1:Is CDGA Youth Services Funding Used Effectively?....................................................................................................

4 Measurement Criteria.......................................................................................... 4 Reporting and Monitoring Systems Quality ....................................................... 4 Recommendations for Improving CDGA Grantee Performance Measurement ...............................................................................

10 Goal Achievement ............................................................................................ 14 Research Question #2: How Can Milwaukee Link Youth Services Programming to Economic Development? .................................

20 Introduction....................................................................................................... 20 Effectiveness of Youth Services Programs for Economic Outcomes............... 20 Analysis of Milwaukee Data.............................................................................

24 Recommendation: Take a Broader Perspective on Economic Development ... 27 Conclusion ............................................................................................................ 28 References.............................................................................................................

30 Appendix A: Grant Application and Award Process........................................... 37 Appendix B: CDGA Monitoring Procedures ...................................................... 39 Appendix C: Selected Current CDGA Grantee Outcome Indicator Measuresand Data Sources, and Corresponding Suggested Outcome Indicator Measures and Data Sources, by Service Provision Category ......................................................

41 Appendix D: Recent Trends in CDBG Funding.................................................. 44 Appendix E: Oakland, California........................................................................ 45 Appendix F: Youth Services Programming Categories......................................

46 Appendix G: List of 2006 Grantees, Amount Received and Number of Youth Served ........................................................ 47 Appendix H: Successful Intensive Youth Programs ........................................... 51 iv Figure 1: 2006 CDGA Services Allocations by Type of Program ......................

16 Figure 2: Per Youth Expenditures by Program Type .......................................... 25 Figure 3: Total CDBG Appropriations in Real Dollars, 2003-2008: Steady Decline in Available Funds ...................................................... 44 Table 1: Carnegie Youth Characteristics and Corresponding CDGA Program Categories..................................................................

15 Table 2: CDGA-Sponsored Youth Programming by Category.......................... 15 Table 3: Summary of Recommendations ........................................................... 29 v This report, which analyzes the delivery of Community Development Block Grant (CDBG) resources in the area of Youth Services in the City of Milwaukee, is the product of a collaboration between the Robert M.

La Follette School of Public Affairs at the University of Wisconsin–Madison, the City of Milwaukee Budget and Management Division, Department of Administration and City of Milwaukee’s Community Development Grants Administration (CDGA).

Our objective is to provide graduate students at La Follette the opportunity to improve their policy analysis skills while contributing to the capacity of the City of Milwaukee to effectively provide public services. The La Follette School offers a two-year graduate program leading to a master’s degree in public affairs.

Students study policy analysis and public management, and pursue a concentration in a public policy area of their choice. They spend the first year and a half taking courses that provide them with the tools needed to analyze public policies. The authors of this report are all enrolled in Public Affairs 869, Workshop in Public Affairs, Domestic Issues.

Although acquiring a set of policy analysis skills is important, there is no substitute for doing policy analysis as a means of learning policy analysis. Public Affairs 869 provides graduate students that opportunity. The students were assigned to one of several project teams.

Two additional groups worked with the City of Milwaukee Budget and Management Division. Other teams worked with the Wisconsin Joint Legislative Council or the Wisconsin departments of Revenue, Administration, or Health and Family Services.

The topic of this report—evaluating the effectiveness of Milwaukee’s use of Community Development Block Grant funding for youth programming— was chosen by Mark Nicolini, the budget director of the City of Milwaukee, in consultation with his staff. This report does not provide the final word on the complex issues the authors address.

The graduate student authors are, after all, generally new to policy analysis, and the topic they have addressed is complex. Nevertheless, much has been accomplished, and I trust that the students have learned a great deal, and that Mayor Barrett and the staff of the City’s Budget and Management Division will profit from the evaluation.

Like many large cities, Milwaukee faces an environment with consistent demand for CDBG services, but limited and declining resources to provide such programs. Given these constraints, the CDGA has to consider how to make the most of funding. This report dealt with this issue in two ways.

First, it evaluated the provi-sion of current services. Through analysis of CDGA records, site visits, and analysis of social science research, the team concluded that the Youth Services spending is vi generally targeted in areas that have a proven connection to positive youth out-comes.

However, in anticipation of increasing demands for accountability in this area, the research team suggested specific ways to improve the quality of perfor-mance reporting. Second, the research team examined whether more Youth Services resources could be better spent on economic development programs.

Through an analysis of current spending and a review of relevant social science research, the research team found that many traditional economic development programs, such as job training, do not have a strong track record in generating positive economic outcomes. Other forms of youth programming such as educa-tional and recreational programs appear to have stronger connections with econo-mic outcomes.

This suggests no pressing need for new types of programming and that the CDGA adopt a broader understanding of economic development to incorporate much current programming.

This report would not have been possible without the support and encouragement of Budget Director Mark Nicolini and Eric Pearson, who served as the project coordinator for the Budget and Management Division and solicited ideas for policy analysis from the Budget Office staff and coordinated the efforts of staff in support of the project.

CDGA staff Steven Mahan, Hettie White, and Matthew Balistrieri provided practical guidance in helping the authors understand how CDGB funding was allocated. The report also benefited greatly from the active support of the staff of the La Follette School. Terry Shelton, the La Follette outreach director, along with Kari Reynolds, Mary Mead, and Gregory Lynch, contributed logistic and practical support for the project.

Karen Faster, La Follette publications director, edited the report and shouldered the task of producing the final bound document. I am very grateful to Wilbur R. Voigt whose generous gift to the La Follette School supports the La Follette School public affairs workshop projects.

With his support, we are able to finance the production of the final reports, plus other expenses associated with the projects. By involving La Follette students in one of the tough issues faced by the City of Milwaukee, I hope the students not only have learned a great deal about doing policy analysis but also have gained an appreciation of the complexities and challenges facing city government in Wisconsin and elsewhere.

I also hope that this report will contribute to the work of the Division of Budget and Management and the CDGA in their efforts to provide services to the young citizens of Milwaukee. Donald Moynihan May 1, 2007 vii We would like to express our sincere gratitude to all those who contributed significant time and effort to this project.

This includes Steven Mahan, Hettie White, and Matthew Balistrieri of the City of Milwaukee’s Community Development Grants Administration, Department of Administration; Eric Pearson of the City of Milwaukee’s Budget and Management Division, Department of Administration; and Professor Donald Moynihan of the Robert M. La Follette School of Public Affairs at the University of Wisconsin-Madison.

We would also like to thank Karen Faster, publications director at the La Follette School of Public Affairs, for her support and assistance. viii ix Milwaukee’s youth face a multitude of barriers to adequate development. Poor educational achievement and high rates of poverty, drug use, violence, and teen pregnancy all forecast a troubled future for Milwaukee’s youth and for the city at large.

To combat these negative influences on youth development, the City of Milwaukee’s Community Development Grants Administration (CDGA) allocates almost $2 million each year in federal Community Development Block Grant funding to community organizations that provide programming for youth.

These Youth Services grants provide recreational, educational, civic engagement, job training, and crime prevention programming to Milwaukee’s poorest young people. The Milwaukee Budget Office asked us to review CDGA’s Youth Services grant process to determine the effectiveness of Youth Services grants and to explore the possibility of better linking Youth Services programming to economic develop-ment.

Based on extensive literature reviews, analysis of CDGA files, interviews with CDGA staff, and site visits to three Youth Services grantees, we have concluded that: 1. though CDGA’s Youth Services grantees provide quality programming, the agency should adjust its reporting and monitoring systems to yield better performance data to reflect these efforts, and 2.

a shift of more money toward youth job training and placement would not result in stronger economic development outcomes relative to other uses of those resources. To answer our first research question— Is CDGA Youth Services funding used effectively? —we considered two measurement criteria: the quality of CDGA reporting and monitoring systems and goal achievement.

In the course of our ana-lysis, we realized that our ability to fully determine the effectiveness of Youth Services grants depended on the nature and quality of the performance data required by CDGA and provided by grantees.

Our review of CDGA’s reporting and monitoring systems revealed varying levels of grantee knowledge regarding the basics of performance management; CDGA reporting requirements that, though consistent with federal requirements, could be improved; and a need for closer communication between CDGA and grantees in developing, tracking, and reporting program outcomes.

As a result, we recommend: • CDGA should host a mandatory annual grantee training session on developing and measuring valid program outcomes, • CDGA should strengthen its reporting requirements by manda-ting that grantees track and report all benefits they claim to produce, and • CDGA should implement a more interactive monitoring system to assist grantees struggling with CDGA reporting requirements.

xIf implemented, we believe these recommendations, which we consider to be administratively feasible, would yield higher quality performance data that CDGA could use to make more informed funding decisions. Despite the aforementioned limitations of CDGA’s reporting and monitoring system, we used existing grantee performance data and social science literature to consider goal achievement.

First, we broke down by program category the 36 organizations that received grants in 2006. The five program categories included education, crime prevention/personal development, civic engagement, economic development, and recreation. In terms of program type, CDGA funds a dispropor-tionately high number of crime prevention/personal development programs, with education and economic development programs receiving the fewest grants.

In terms of funding, recreation programs receive a disproportionately large share (44 percent) of Youth Services funding, with the other four categories receiving roughly equal shares.

Despite these imbalances, on the whole we believe CDGA funding allocations effectively target the barriers to adequate youth development in Milwaukee and they contribute to achieving CDGA’s stated goals of reducing crime, increasing economic opportunity, and improving quality of life. The vast majority of funding supports activities that social science research shows to produce positive outcomes.

To answer our second research question— Can Youth Services programming be better linked to economic development? —we consulted social science research regarding the economic benefits of different types of youth programming. We also outlined the tradeoffs associated with shifting more Youth Services funding to such traditional economic development programming as job training and placement for youth.

Our research suggests that, though CDGA may feel pressure from the U.S. Department of Housing and Urban Development to emphasize economic development as a funding priority, redirecting more Youth Services funding toward job training and placement would be misdirected, because existing research on such programs has found that these programs offer only marginal economic benefits compared to other program types.

Instead, we recommend CDGA employ a broader definition of economic development, one that encompasses all types of programs that have empirically demonstrable effects on positive economic activity. Such programs include education programming—especially programs that intervene early—and after-school recreational and community service programs that foster positive habits and attitudes that are rewarded later in the labor market.

Our analysis of alternative funding strategies indicates that CDGA’s current funding strategy is likely to yield the greatest long-term economic benefits for the city of Milwaukee and its youth. 1 Each year, the Milwaukee Community Development Grants Administration (CDGA) receives federal funding from the U.S. Department of Housing and Urban Development’s (HUD) Community Development Block Grant (CDBG) program.

CDGA uses its CDBG funds in efforts to: • increase property values • increase economic vitality, and • improve quality of life CDGA sets aside approximately 10 percent of its CDBG dollars to fund Youth Services programming aimed at combating the challenges facing Milwaukee’s youth population.

These challenges include high rates of poverty, poor educa-tional achievement, and high rates of risky, anti-social behaviors such as violence, drug use and unsafe sexual activity. CDGA contracts with community organizations that provide youth services. Through a Request for Proposals (RFP) process, CDGA reviews grant applications and makes funding recommendations to Milwaukee’s Common Council.

The Common Council makes all final funding decisions. In 2006, the Common Council awarded 36 grants totaling $1. 7 million to 27 Youth Services providers (Commun-ity Development Grants Administration, City of Milwaukee, 2006e).

1 For detailed information on the grant application and award process, see Appendix A. The Milwaukee Budget Office asked our team to explore the current Youth Services funding process, to determine the effectiveness of Youth Services grants, and to research the possibility of linking Youth Services programming to economic development. Based on Milwaukee’s requests, this report will seek to answer two critical research questions.

First, are CDGA’s Youth Services grants effective? More specifically, are grantees meeting CDGA goals and addressing the aforementioned barriers to positive youth development? Also, does CDGA possess the reporting and monitoring systems needed to accurately gauge a grant’s effectiveness?

Second, we consider whether—and if so, how—Youth Services grants can be better linked with economic and workforce development. We will explore which types of youth programming provide a link between youth and economic development, suggest ways CDGA could encourage such program-ming, and outline the tradeoffs of shifting more funding to economic and workforce development programming for youth.

1 Some grantees have multiple locations and reported their outcomes in one report. For instance, six locations of the Boys and Girls Club received grants, but all locations pooled their information and submitted one year-end narrative report. We evaluated the Boys and Girls club as a single grantee.

However, in other cases, an organization that received several grants for different locations reported separate outcomes for each location, such as the YMCA. In these cases, we evaluate each location as its own entity. 2 To conduct our analysis, we reviewed CDGA files, interviewed staff and conducted site visits.

Prior to our research, most grantee performance files existed only on paper. Our analysis aggregated and put into electronic format much of the descriptive performance data collected by CDGA. To perform our analyses we used CDGA data from 2006.

CDGA defines youth as Milwaukee citizens from ages 5 to 25 (personal com-munication, February 14, 2007). This broad definition of youth, coupled with a similarly wide variety of potential youth programs, results in an extremely broad definition of Youth Services. This reality will be important to consider when exploring methods of linking Youth Services to economic development.

HUD defines economic development as “creating and maintaining jobs” (U.S. Department of Housing and Urban Development, 2006). While accurate, this definition is limited in its use for classifying the types of programs that can foster positive economic activity. We will explore a broader definition of economic development later in the analysis.

## Status Quo: Rationale for CDGA-Sponsored Youth Services Grants A multitude of barriers to adequate youth development exist today in Milwaukee. According to Milwaukee’s 3-5 Year Consolidated Plan (Community Development Grants Administration, City of Milwaukee, 2006a), the City has identified a number of youth issues that need attention.

These issues include education, teen pregnancy, substance abuse, violence, juvenile delinquency, unemployment, and environmental conditions. The following statistics demonstrate the rationale for government-sponsored youth programming in a high-poverty city such as Milwaukee. Twenty-one percent of Milwaukee families earned incomes below the federal pov-erty line in 2005, more than double the national average (U.S. Census Bureau, n.

d.) Thirty-eight percent of Milwaukee children lived below the poverty line, which is also more than double the national average (DeNavas-Walt, Proctor, & Lee, 2006). According to data collected from the Wisconsin Department of Public Instruction, Milwaukee Public Schools students fall well below the state average on a number of important education indicators.

On the 2005 Wisconsin Concepts and Know-ledge Exam, the percentage of Milwaukee Public Schools eighth-graders scoring “minimal” and “basic” was two times the state average, and only 12 percent of these students scored “advanced,” compared to 40 percent statewide (Wisconsin Department of Public Instruction, 2005). Such substantial disparities in academic achievement persist throughout high school.

For example, relative to other 3Wisconsin school districts, far fewer Milwaukee juniors and seniors take the ACT, and those that do score four points lower than their Wisconsin peers. 2In addition, Milwaukee Public Schools has five times as many habitually truant students and expels more than twice as many students compared to other districts statewide (Wisconsin Department of Public Instruction, n. d.)

Finally, Milwaukee students participate in extra-curricular activities at just half the rate of other Wisconsin high school students. In addition to the poor education indicators, Milwaukee high school students exhibit high rates of risky behaviors.

Sixty percent have engaged in sexual intercourse, 52 percent have smoked marijuana, 37 percent and 19 percent have consumed alcohol and carried a weapon, respectively, in the last month, and 17 percent have contemplated suicide in the last year (Center for Urban Initiatives and Research—University of Wisconsin-Milwaukee, 2003a).

The sexual inter-course, weapon, and marijuana indicators all exceed the national average (Centers for Disease Control and Prevention. , U.S. Department of Health and Human Services, 2004). Milwaukee middle school students also engage in risky behavior at high rates.

Twenty-nine percent have engaged in sexual intercourse, 30 percent have carried a weapon, 24 percent have contemplated suicide, 49 percent have consumed alcohol at least once, and 21 percent have tried marijuana (Center for Urban Initiatives and Research – University of Wisconsin-Milwaukee, 2003b).

To combat these impediments to youth development, CDGA offers grants to organizations that provide educational, recreational, civic engagement, job training, and crime prevention programs. Research questions 1 and 2 discuss these programs in detail. Research question 1 also outlines the current performance reporting mechanism in place.

2 Considering that the best possible score on the ACT is a 36, the 4-point difference between the Milwaukee Public Schools and the state average is substantial. 4 # Research Question #1:Is CDGA Youth Services Funding Used Effectively?

Since Milwaukee devotes nearly $2 million of its CDBG funding to Youth Services, city officials want to determine whether CDGA-sponsored youth programs target barriers to youth development and whether they meet the City’s goals to reduce crime, increase economic vitality, and improve quality of life.

To answer the first research question, our team developed two criteria—goal achievement and quality of reporting and monitoring systems—by which we can evaluate the success of CDGA grants. We examine the existing data to identify links and disconnects between current programming and CDGA goals. We also explore whether CDGA data reporting and monitoring systems could be improved so that these systems yield more meaningful data.

Reporting and Monitoring Systems Quality We reviewed CDGA’s reporting and monitoring systems to determine whether: 1) grantees provide CDGA with valid measures of intended outcomes in their performance measurement reports; 2) the grantee provides quality data from a reliable source when reporting its outcome(s); and 3) CDGA penalizes grantees who do not comply with CDGA reporting requirements.

To judge goal achievement, we coupled academic research on the benefits of youth programming with data available through CDGA’s reporting and monitoring systems. We measured goal achievement by whether: 1) CDGA allocation decisions align with the challenges facing Milwaukee youth and CDGA’s goals of reducing crime, increasing economic vitality, and improving quality of life; and 2) the grantee met its goal for number of youth served.

## Reporting and Monitoring Systems Quality In this section, we provide a description of the data that CDGA requires of its grantees, the data that CDGA must submit to HUD, and CDGA’s monitoring processes. This allows us to assess the effectiveness of CDGA’s current reporting and monitoring processes.

5 Data CDGA Requires of Grantees Once a youth services organization secures a grant, CDGA requires it to submit a number of reports each year. These reports include: • Outcome & Activity Measurement Workplan; • Monthly Activity Reports; • Six-Month Performance Measurement Report; and • Year-End Performance Measurement Report.

The Outcome & Activity Measurement Workplan, submitted at the beginning of each year, must outline all program outcomes that the grantee plans to measure during the year. The grantee must identify outcome measures, a data source, and a data collection method. CDGA requires grantees to track and provide data for at least one program outcome.

These tracking requirements constitute an outcome-based reporting method (Community Development Grants Admini-stration, City of Milwaukee, 2006c). Grantee outcomes and CDGA outcome tracking requirements are the focus of Research Question 1. The second required report, the Monthly Activity Report, documents the projected and actual number of youth served in the month.

CDGA requires supporting documentation, such as attendance sheets, to substantiate program attendance. By the end of the year, each grantee should have submitted 12 monthly activity reports to CDGA (Community Development Grants Administration, City of Milwaukee, 2006b).

The next required report is the Six-Month Performance Measurement Report; this report should contain a mid-year activity summary to document the number of youth served in the first half of the year and should present at least one outcome measurement. In the six-month report, grantees must provide a narrative of program activities and outcomes.

As part of this narrative, grantees must explain the connection between the measured outcome and CDGA goals (Community Development Grants Administration, City of Milwaukee, 2006d). The final report requirement for grantees is the Year End Performance Measure-ment Report.

This report is identical to the six-month report, but it documents the entire year’s activities and outcomes (Community Development Grants Admini-stration, City of Milwaukee, 2006f). Data HUD Requires of CDGA As a condition of receiving federal funds, CDGA must submit compliance reports to HUD, which requires a demographic breakout of youth served that includes race, gender, and income.

It also requires the total amount of funding given to each grantee. CDGA submits this data to HUD using an online database, known as the Integrated Disbursement & Information System. 6In addition to these, CDGA submits an annual narrative report, called the Consoli-dated Annual Performance and Evaluation Report.

This report documents the ways in which Milwaukee grantees meet short- and long-term HUD and CDGA goals. 3 In all, CDGA collects and submits more data than HUD requires. HUD has recently instituted new performance measures for municipalities that receive federal funds.

HUD does not require individual agencies to track multiple outcomes; rather HUD requires that municipalities report their overall progress in meeting the municipality’s chosen long-term outcomes. To meet this goal, CDGA requires that agencies track a minimum of one key outcome.

CDGA aggregates outcomes of all funded activities (regardless of category) to determine the effectiveness in meeting long-term outcomes to reduce crime, increase property values, increase economic vitality and improve quality of life.

CDGA monitors compliance with each grantee’s contractual agreement and sponsors voluntary training sessions conducted by the Milwaukee Nonprofit Center, which provides technical assistance, such as training on management, financial operations, and board and staff development.

CDGA staff use the six-month and year-end reports submitted by grantees to deter-mine: 1) whether the program met its goals for youth served and 2) how closely the program activities and outcomes comport with CDGA goals. CDGA references these reports when organizations request funding again in subsequent years. As a condition of payment, agencies are required to submit monthly financial and programmatic reports.

CDGA grants-monitoring staff review these reports to determine that submitted costs are eligible and that the funded activity is being performed to a satisfactory level. The City Comptroller’s office conducts annual financial audits of funded groups and monitors the timeliness of expenditures. In addition to financial monitoring, CDGA staff maintain contact with agencies and provide technical assistance to groups where needed.

Informal and formal site visits are conducted to ensure compliance with program requirements. Risk assess-ments and in-house desk audits are performed annually of all funded agencies. When CDGA grant monitors conduct site visits, they comply with HUD require-ments (U.S. Department of Housing and Urban Development, n.

d.) , that include: • File review, verification, and documentation of performance data; • Interviews with key staff, program clients, and subcontractors; • Physical inspections (if appropriate); and, • Entrance and exit conferences with program staff. For more detailed information on CDGA monitoring procedures, see Appendix B.

3 HUD requires jurisdictions receiving CDBG grants to submit five-year consolidated plans, which detail long-term goals, as well as Annual Action Reports, which break the five-year plan into annual incre-ments. Therefore, Milwaukee’s consolidated plan contains goals for 2005-2009 and the 2007 action plan.

7 Best Practices in Reporting and Monitoring Systems Quality Before analyzing Milwaukee’s current reporting and monitoring systems, we first conducted a literature review in the area of performance measurement. In doing so, we identified best practices in the field and adopted these best practices as a benchmark against which we could compare CDGA grantee performance mea-surement.

The main areas of interest for our literature review were distinguishing among components of the measurement process and the usage of quality data sources. The text below outlines our benchmark for best practices in each of these areas; we will compare Milwaukee’s current reporting and monitoring systems quality against these criteria. Our benchmarks and resulting recommendations do not reflect HUD requirements.

Rather they summarize best practices that the performance measurement literature discusses. As stated earlier, CDGA already exceeds HUD reporting requirements. There are four main steps to effective measurement.

The first is to track program inputs. Inputs refer to the resources devoted to a program, such as funding or staff time (Urban Institute, 2003). We did not focus our analysis on program inputs, but impact data can be used to determine whether inputs are efficient, for instance whether the staff time expended is proportional to the outcome achieved.

The second step in measuring a program is to track organizational outputs. Outputs are the direct results of the funded program, and include things such as number of participants served, number of community service hours completed, or number of people attending an event (United Way of America, 1996). Outputs measure volume of results, rather than the quality or impact of results.

It can be difficult to distinguish between outputs and outcomes, which are the next step in performance measurement. Our analysis will show that CDGA grantees confuse outputs and outcomes regularly when they report program results to the CDGA. The third and fourth steps in the program measurement process relate to measur-ing program outcomes.

Outcomes , as opposed to outputs, are the benefits partici-pants receive as a result of their program participation. There are two measurable stages of outcomes. The first stage is the intermediate outcome.

Intermediate out-comes are benefits participants receive, but which lead eventually to a larger or more general benefit; intermediate outcomes are not the end goal of the program. Completing a job training program can be considered an intermediate outcome, because it shows that participants have not only attended and that they received the information and skills the program taught. The final measurement stage is to track end outcomes.

In contrast to intermediate outcomes, an end outcome is the benefit to or consequence for participants as a result of the intermediate outcome. Continuing with the job training example, obtaining a job as a result of complet-ing job training would constitute an end outcome . Other examples of end out-comes include demonstrated changes in skill levels, behaviors, attitudes, or values (United Way of America, 1996).

8To accurately report the intermediate or end outcomes caused by a program, organizations must verify their claims using quality data sources.

According to the academic literature on data reporting, the following are considered quality data sources (Thomas, 2005): • program records and statistics • client questionnaires and surveys • trained observer ratings • qualitative measures Examples of how CDGA can use these data sources appear in the following pages and in Appendix C.

Analysis of Milwaukee Data and Comparison to Best Practices To compare Milwaukee’s current performance measurement system to the best practices outlined above, our team first conducted a qualitative analysis of current CDGA grantee outcome indicators and data sources. Grantees provide this information in the Outcome & Activity Measurement Workplan that they submit to CDGA at the beginning of each grant year.

These plans are designed to show which outcomes the group will measure over the course of the year. Next, we compared the reported measures and sources against the current best practices standards.

Where an outcome measure or data source did not meet the standards of good outcome measurement, we suggested a possible measurement or source alternative, which would bring the organization into compliance with the stan-dards of good outcome measurement. These suggestions represent only one way in which the organization might validly measure and track outcomes, but we endeavored to make suggestions that were feasible to implement.

In many cases, this meant that we suggested surveys of program participants to assess attitudes or behaviors. With survey data, there will always be questions of reliability, but this approach is preferable to simple participation measures and is significantly easier to implement than more sophisticated approaches to performance measurement.

Representative examples from each Milwaukee service provision category are presented in Appendix C. In summary, of the 30 distinct organizations funded by the CDGA in 2006 (some organizations reported all locations as one), 20 of these groups are providing at least one valid intermediate or end outcome in their outcome work plan. The