Arkansas National Guard Youth Challenge Program

Arkansas National Guard Youth Challenge Program is sponsored by U.S. Department of Defense (DoD). Provides funding for programs aimed at at-risk youth in Arkansas.

Defense Management: Actions Are Needed to Improve the Management and Oversight of the National Guard Youth Challenge Program Defense Management: Actions Are Needed to Improve the Management and Oversight of the National Guard Youth Challenge Program The fiscal year 1993 National Defense Authorization Act established the National Guard Youth Challenge Program as a pilot program to evaluate the effectiveness of providing military based training to improve the life skills of high school dropouts.

The Assistant Secretary of Defense for Reserve Affairs, under the authority of the Under Secretary of Defense for Personnel and Readiness, is responsible for overall policy for the program. The National Guard Bureau (NGB) provides direct management and oversight. In 1998, Congress permanently authorized the program and began decreasing the federal cost share until it reached its current level of 60 percent in 2001.

Conference Report 108-767 directed GAO to review the program. Specifically, GAO reviewed (1) historical trends of the program; (2) the extent of analyses performed to determine program costs and the need to adjust the federal and state cost share; and (3) NGB oversight of the program. GAO is also providing information on Reserve Affairs' and states' efforts to obtain funding from alternative sources.

-------------------------Indexing Terms------------------------- TITLE: Defense Management: Actions Are Needed to Improve the Management and Oversight of the National Guard Youth Challenge Education or training costs National Guard Youth Challenge Program ****************************************************************** ** This file contains an ASCII representation of the text of a ** ** No attempt has been made to display graphic images, although ** ** figure captions are reproduced.

Tables are included, but ** ** may not resemble those in the printed version. ** ** Please see the PDF (Portable Document Format) file, when ** ** available, for a complete electronic file of the printed ** ** document's contents.

** ****************************************************************** Defense Management Defense Management Defense Management ement Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Defense Management Challenge Program Expenditures and Participation Have Increased and NGB Has Reported Positive Performance Outcomes 9 Reserve Affairs and NGB Have Not Analyzed Data to Determine Actual Program Costs or the Need to Adjust the Federal and State Cost Share 18 The National Guard Bureau Lacks a Complete Framework for Oversight of the Challenge Program, Making It Difficult to Measure Program Effectiveness 21 Reserve Affairs Lacks a Formal Approach for Securing Alternative Funding Support for the Challenge Program, While Some Participating States Have Obtained Alternative Funding Support as a Result of Their Efforts 25 Recommendations for Executive Action 29 Agency Comments and Our Evaluation 30 Appendix I Scope and Methodology 33 Appendix II National Guard Youth Challenge Program Sites 36 Appendix III Comments from the Department of Defense 37 Appendix IV GAO Contact and Staff Acknowledgment 40 Table 1: Hypothetical Examples Showing How Changing the Cost Basis or Cost-Share Ratio Would Affect Yearly Challenge Program Funding Amounts 19 Table 2: States Selected for Site Visits or Interviews 34 Figure 1: Annual Expenditures for the National Guard Youth Challenge Figure 2: Number of National Guard Youth Challenge Program Sites by Figure 3: Youth Participation in National Guard Youth Challenge Program 13 Figure 4: Challenge Program Sites by 2004 Graduation Results 14 Figure 5: Number of GEDs Earned by Challenge Students 16 Figure 6: Post-Residential Placement Data, 2003-2005 17 protection in the United States.

It may be reproduced and distributed in its entirety without further permission from GAO.

However, because this DMARS Data Management and Reporting System DOD Department of Defense DOJ Department of Justice DOL Department of Labor ED Department of Education GED general educational development NGB National Guard Bureau United States Government Accountability Office November 29, 2005 November 29, 2005 The Honorable John Warner Chairman The Honorable Carl Levin Ranking Minority Member Committee on Armed Services United States Senate The Honorable John Warner Chairman The Honorable Carl Levin Ranking Minority Member Committee on Armed Services United States Senate The Honorable Duncan L.

Hunter Chairman The Honorable Ike Skelton Ranking Minority Member Committee on Armed Services House of Representatives The Honorable Duncan L.

Hunter Chairman The Honorable Ike Skelton Ranking Minority Member Committee on Armed Services House of Representatives After Operation Desert Storm and the end of the Cold War, Congress began to reevaluate the focus of U.S. military activities and proposed using some military assets and training to help address critical domestic challenges such as drugs, poverty, and unemployment.

In particular, some policymakers saw an opportunity to use the military's capabilities to address domestic needs by providing role models for youth and training and educational opportunities for the disadvantaged.

Section 1091 of the National Defense Authorization Act for Fiscal Year 1993 established the National Guard Youth Challenge Program (Challenge Program) as a pilot program to evaluate the feasibility and effectiveness of military based training to improve the life skills and employability of high school dropouts.

After the program was permanently authorized in 1998, Congress began limiting the share of state operating costs covered by the federal government through the Department of Defense (DOD), which initially paid for 100 percent of these costs. DOD now pays 60 percent of state operating costs, and states provide 40 percent.

In addition, the authorizing legislation for the Challenge Program allows the Secretary of Defense to use nondefense funding sources in support of the program. After Operation Desert Storm and the end of the Cold War, Congress began to reevaluate the focus of U.S. military activities and proposed using some military assets and training to help address critical domestic challenges such as drugs, poverty, and unemployment.

In particular, some policymakers saw an opportunity to use the military's capabilities to address domestic needs by providing role models for youth and training and educational opportunities for the disadvantaged.

Section 1091 of the National Defense Authorization Act for Fiscal Year 1993 established the National Guard Youth Challenge Program (Challenge Program) as a pilot program to evaluate the feasibility and effectiveness of military based training to improve the life skills and employability of high school dropouts.

After the program was permanently authorized in 1998, Congress began limiting the share of state operating costs covered by the federal government through the Department of Defense (DOD), which initially paid for 100 percent of these costs. DOD now pays 60 percent of state operating costs, and states provide 40 percent.

In addition, the authorizing legislation for the Challenge Program allows the Secretary of Defense to use nondefense funding sources in support of the program.

The Assistant Secretary of Defense for Reserve Affairs, under the authority of the Under Secretary of Defense for Personnel and Readiness, is responsible for management oversight of the Challenge Program, including policy and funding issues, while the National Guard Bureau (NGB) is responsible for the administration of the program, including daily management and oversight.

For example, NGB maintains frequent contacts with state programs and conducts program evaluations through a The Assistant Secretary of Defense for Reserve Affairs, under the authority of the Under Secretary of Defense for Personnel and Readiness, is responsible for management oversight of the Challenge Program, including policy and funding issues, while the National Guard Bureau (NGB) is responsible for the administration of the program, including daily management and oversight.

For example, NGB maintains frequent contacts with state programs and conducts program evaluations through a contractor, while Reserve Affairs prepares the program budget based on input from NGB and ensures that states are able to provide their required share.

To participate in the program, states must sign a cooperative agreement with NGB that lays out roles and responsibilities among the states and NGB, and also provides guidance on the content of the program. Since 1993, NGB has used a cost of $14,000 per student as a basis for providing funds to state Conference Report 108-767 directed the Comptroller General to conduct a study of the Challenge Program.

1 Specifically, we examined: (1) historical trends of the Challenge Program, including program expenditures, participation, and performance; (2) the extent to which Reserve Affairs and NGB have determined actual program costs and the consequent need to adjust the federal and state cost share; and (3) the extent to which NGB has provided oversight of the program.

We also determined the extent to which Reserve Affairs and participating states have made an effort to obtain alternative funding support for the Challenge Program.

In performing our work, we interviewed officials from the Office of the Assistant Secretary of Defense for Reserve Affairs; NGB; the Departments of Education, Labor, and Justice; the contractor that monitors and evaluates state programs; the National Guard Youth Foundation; and selected states.

We also reviewed the program's annual reports and other documents provided by Reserve Affairs and NGB as well as the data management system used by states to collect Challenge Program student information and track individual and program activities. We concluded that the data were sufficiently reliable for our purposes.

We analyzed program evaluations, resource management reviews, and audits, in addition to reviewing existing policies and procedures for managing the program. We conducted a survey of all 29 established programs to collect information on their budgets and expenditures, federal and state funding levels, each state's ability to fund their individual program, and the availability of funding from other sources.

We also visited or contacted eight Challenge Program sites in seven states and asked them about any alternative sources of funding for their programs. A detailed description of our scope and methodology is presented in appendix I. We conducted our work from January 2005 to October 2005 in accordance with generally accepted government 1Conference Report 108-767 accompanied the Ronald W.

Reagan National Defense Authorization Act for Fiscal Year 2005, Pub. L. No. 108-375 According to NGB, program expenditures and state participation in the Challenge Program have increased since the program began, and the program has achieved positive performance outcomes.

Between fiscal years 1998 and 2004, total expenditures for the Challenge Program, including funds spent to cover the federal and state cost shares and federal management expenses, have increased from about $63 million to about $107 million. For fiscal year 2004, for example, NGB spent approximately $61. 6 million for the federal share of the program and $5.

8 million for NGB management costs, while states contributed approximately $40. 5 million. Ten states participated in the first year of the pilot program in 1993; and today there are 29 Challenge Program sites in 24 states and Puerto Rico.

Additionally, 9 states remain on a waiting list to start new programs, and several states have expressed an interest in expanding their existing programs.

Some common performance outcomes that NGB reports include the number of graduates earning a general educational development credential, changes in program students' scores on standardized math and reading tests, and the number of community service hours performed by Challenge Program students. For example, the Challenge Program reported that 70 percent of graduates in 2004 earned a general education development credential.

Another performance outcome that NGB reports is the number of placements of program graduates in continuing education, the military, or the labor force at the time they complete their 12-month post-residential phase. For example, in 2004, 3,698 graduates were placed in one of these categories by the end of the post-residential phase of the program.

Although Reserve Affairs and NGB have expressed concern about the current program funding level and have suggested increasing both the cost basis used to determine funding needs and the federal cost share, we found that neither Reserve Affairs nor NGB has performed analyses to support the need for such changes.

Good budget practices, included in the Office of Management and Budget's Federal Financial Accounting Standards, state that agencies should determine actual costs of their activities on a regular basis and that reliable cost information is crucial for effective management of government operations.

Since 1993, NGB has used a cost of $14,000 per student as the basis for determining the amount of funds needed to cover program operating costs. However, due to cost variations between states, program officials that we surveyed reported that they actually spent between $9,300 and $31,031 per graduate.

According to NGB, all 29 programs are providing the services required by the cooperative agreements, and several states have added program enhancements such as field trips or vocational classes. However, some of these states reported that they reduced nonrequired services to stay within their budgets.

For example, they implemented staff pay and hiring freezes, eliminated the student stipend, and eliminated some enrichment activities such as field trips and vocational classes.

In addition, Reserve Affairs, NGB, and participating states have suggested that the cost-share ratio be changed from its current 60 percent federal share to a 75 percent federal share because they believe that the current required 40 percent state share is sometimes difficult for states to meet.

However, neither Reserve Affairs nor NGB has compiled or analyzed data on actual program costs, states' financial situations, or the impact of adjusting the federal and state cost share.

Reserve Affairs has asked the National Guard Bureau to determine a new funding formula for the program based on individual state needs, but, at the time of our review, NGB had not yet done so and Reserve Affairs has not given NGB a deadline for completion. Without better cost and financial information, DOD cannot justify future funding requests or a change in the cost share ratio.

Although NGB has several mechanisms in place for overseeing the Challenge Program, it lacks a complete oversight framework, making it difficult to measure the effectiveness of the program and to adequately address audit and review findings.

NGB conducts several oversight activities, including informally communicating with state program directors and having an outside evaluator conduct yearly operational evaluations and biennial resource management reviews of the Challenge Program, and these reports have prompted some changes to the program. However, until recently, NGB did not have a formal mechanism for tracking the findings of these reports.

In response to our review, NGB recently implemented a mechanism for tracking these findings. As part of its oversight activities, NGB does require each state program to report certain performance outcome measures, such as academic test scores. However, it does not require states to establish performance goals in these areas.

A complete oversight framework includes performance goals and measures against which to objectively measure performance as well as a mechanism for tracking findings of audits or reviews and responding to those findings. Without clear and agreed upon performance goals, there is no objective yardstick against which to fully measure program performance and effectiveness and thereby assess DOD's return on investment.

In addition, Standards for Internal Control in the Federal Government states that agencies need to ensure that the findings of audits and reviews are promptly resolved. Under the cooperative agreements governing the Challenge Program, United States property and fiscal officers in each state are required to conduct full audits of the individual Challenge Programs at least once every three years.

However, according to property and fiscal officers that we spoke with, all of these audits have not been conducted as required, due to the low priority placed on these audits and lack of staff for the property and fiscal officer. If these audits are not conducted, it may be difficult to ensure that federal interests are adequately protected.

Copies of these audits are not currently being provided to program managers at NGB because, according to the Chief of Property and Fiscal Affairs at NGB, there is no specific requirement to do so. If the National Guard does not see the audits, it cannot ensure that the findings of these audits are promptly resolved.

The authorizing legislation for the Challenge Program allows the Secretary of Defense to use nondefense funding sources in support of the program, and allows states to supplement program funds from other resources. However, Reserve Affairs has not adopted a formal strategy for pursuing nondefense funding, while some states have been successful in securing alternative funding support for their programs.

Reserve Affairs has primarily adopted informal strategies to contact agencies outside DOD to inform them about the Challenge Program and seek opportunities for partnerships, but has not been successful in securing funds from these agencies.

We found that because Reserve Affairs has not made a formal business case to request funds from these nondefense agencies, these agencies are unable to determine whether or not they are specifically able to fund the Challenge Program. Officials from some federal agencies told us that that their agencies have general authority to provide funds to other programs if those programs are consistent with their agency's interests.

However, officials at the Department of Justice, the Department of Labor, and the Department of Education stated that Reserve Affairs needed to present more specific information to them before they could determine whether funds could be provided to the Challenge Program.

According to those officials, at a minimum, any such request should contain the amount of funding sought and a sufficiently detailed description of the program to allow the agency receiving the request to determine whether it would be an appropriate use of funds. Until Reserve Affairs makes a more formal request for funding, other agencies will be unable to determine if they can provide funds for the Challenge Program.

At the state level, some states we contacted had made efforts to obtain alternative funding support to enhance or maintain their program's operations.

For example, some state programs organized into charter schools to obtain funding from local school districts, applied for National School Lunch Program funding, obtained grants from nonprofit organizations, and received additional funding from the state government beyond the required match. Some states reported obstacles in securing supplemental funding, such as the prohibition against using Challenge funds to hire grant writers.

We are making recommendations designed to improve DOD's management and oversight of the Challenge Program and to strengthen efforts in obtaining alternative funding support for the program. In commenting on a draft of this report, DOD concurred with our recommendations. DOD's comments and our evaluation of them appear later in this report.

The National Guard Youth Challenge Program is a 17-month program that serves at-risk youth at 29 sites in 24 states and Puerto Rico. The purpose of the program is to improve the education, life skills, and employment potential of students by providing military-based training, supervised work experience, and knowledge in eight core program components.

2 Students must be 16 to 18 years old, drug-free, unemployed, high school dropouts, and not in trouble with the law. NGB reports that more than 59,000 students have been graduated from the Challenge Program since it began as a pilot in 1993. The program was authorized by 32 U.S.C.

S:509 on a permanent basis in fiscal year 1998, at which time states were to begin paying a share of operating costs. Challenge Program Structure Each Challenge Program site operates two residential classes per year, one of which begins in January and the other around July. 3 A typical graduation goal is 100 students per class, or 200 per year, although several programs graduate more students.

In 2004, for example, Illinois graduated almost 800 students in 2004, and Louisiana's three sites combined graduated more than 950 students. The residential phase of the program runs 22 weeks and includes a 2-week Pre-Challenge phase. During Pre-Challenge, applicants are assessed for their ability and motivation to complete the remaining 20 weeks of the residential program.

Those who successfully complete Pre-Challenge are then formally enrolled in the Challenge Program in numbers that equal each program's graduation target plus normal program attrition rate. In the residential phase, students receive military-based training and supervised work experience.

Additionally, each state develops a curriculum that incorporates the eight core components and the tasks, conditions, and standards that students must complete to demonstrate progress in those components. 4 Each student must receive a score of at least 80 percent on each core component to be graduated from the program.

2 The eight core program components are: leadership/followership, job skills, responsible citizenship, health and hygiene, service to the community, academic excellence, life-coping skills, and physical fitness. 3 NGB reports on site activities by NGB class numbers.

The first class under the pilot program, NGB-1, began in July 1993, and NGB-25 started in During the 12-month post-residential phase, individuals who have successfully completed the residential phase are involved in placement activities, which include employment, education, volunteer activities, or any combination of the three or military service.

The graduates work with adult mentors who were matched with them during the residential phase. These mentors provide guidance and support to the graduates and are required to contact the youths twice each month at a minimum. Program staff use the written, post-residential action plan that each student prepares and updates during the residential phase to monitor placement activities.

Mentors also use this plan during their interactions with graduates. The Challenge Program reports youth placement activities at the end of the 12-month follow-up period. To further assess the long-term impact of the program, NGB has contracted with AOC Solutions to conduct a retrospective longitudinal study of program graduates as well as students who did not complete the program.

Prior to 1998, the federal government, through DOD, completely funded the Challenge Program. In fiscal year 1998, Congress began requiring states to provide a minimum of 25 percent of their programs' operating costs. The state cost share increased 5 percent each year until fiscal year 2001, when it reached the current funding requirement of 40 percent.

5 Although some states had provided more funds than required in the past, program funding each year is now determined by the 40 percent share, which is based on $14,000 for each youth targeted for graduation. In addition to the federal and state funds used to operate the program sites, DOD also provides funds for NGB management expenses such as program evaluations, contractor-provided training, and travel for training and workshops.

These NGB program management costs are not subject to the federal/state 60/40 4A task is a clearly defined and measurable activity accomplished by an individual. Condition describes the training situation, environment, or field conditions under which the cadet must perform the task. The standard establishes criteria for how well a task must be performed.

Each state submits a budget to NGB that is based on that state's target for number of graduates. Since the program's inception, the funding provided by NGB has been based on a cost per student of $14,000. 6 For example, if a state has a target of 100 students per class (200 per year) to graduate, the estimated program costs would be $2.

8 million. The federal contribution, or 60 percent of the total, would be $1. 68 million, while the state contribution would be $1.

12 million. To receive federal funding, a state must certify that it has sufficient funds to provide its 40 percent share. State funds can be composed of cash, noncash supplies, services, or a combination of these sources.

States are allowed to provide additional funding (over and above the 40 percent share) to the program from sources such as individual and corporate donations, additional moneys from the state general fund or other state revenue sources, or other federal funding.

Some Challenge Program sites, for example, operate as alternative schools and are reimbursed by their state education agencies for portions of their program Management, Oversight, and Support Reserve Affairs, under the authority of the Under Secretary of Defense for Personnel and Readiness, is responsible for preparing the annual budget and reviewing state budgets and funding certifications.

Reserve Affairs is to monitor program compliance with DOD policy, issue supplemental policy guidance, and submit the Challenge Program annual report to Congress. NGB provides day-to-day administration and oversight of the Challenge Program, issuing regulations, and submitting budgets and annual report drafts to Reserve Affairs. NGB has contracted with AOC Solutions to assist with the oversight of the Challenge Program.

AOC Solutions performs the annual operational evaluations and the biennial resource management reviews. This contractor also pulls together the program information for the annual report and maintains and oversees the Data Management and Reporting System (DMARS), which is used to collect student data and report on individual and program activities.

NGB has also contracted with Dare Mighty Things to provide training and technical assistance to Challenge Program staff. 7 Finally, NGB has United States property and fiscal officers in each state who are responsible for receiving and accounting for all federal Challenge Program funds and property under control of the National Guard in that state.

The property and fiscal officers are also responsible for ensuring that federal funds are properly obligated and expended. 6Funding allocations to the states have historically been based on target enrollments; that is, the maximum number of students per class that are funded annually in a program's budget.

Beginning with fiscal year 2004, funding allocations are now based on target graduation, which is the maximum number of students identified in a program's budget for participation in and graduation from the program. According to NGB officials, although allocations were originally based on a site's target enrollment, this figure was treated as a graduation goal.

These numbers NGB enters into cooperative agreements with governors of states approved to participate in the Challenge Program. The cooperative agreements describe the responsibilities of the states and NGB as well as the funding, costs, and regulations for operating National Guard Youth Programs.

8 The cooperative agreements also define the eight core components and provide guidance on how to run the residential and post-residential phases and other aspects of the Challenge Program. Each Challenge Program state is also required to submit state plans and budget estimates for their state. These state plans must include details on the state's procedures and be consistent with overall program guidance provided by DOD.

For example, state plans include information on application and selection procedures, staffing and staff training, and a Challenge Program Expenditures and Participation Have Increased and NGB Has Reported Positive Performance Outcomes According to NGB, Challenge Program expenditures and state participation have increased since the program began, and the program has achieved positive program performance outcomes over time.

Since the program's inception, total expenditures have increased from about $63 million to about $107 million per year. The number of states participating in the Challenge Program has also increased, and several states have expressed interest in adding a program or expanding existing ones. Challenge sites must account for their activities throughout the year, and NGB has reported positive performance outcomes over time.

7Contracts with AOC Solutions and Dare Mighty Things were competitively awarded for a 1-year base period with four 1-year options. 8The Master Youth Programs Cooperative Agreement covers two youth programs: the National Guard Youth Challenge Program and the National Guard STARBASE Program. The STARBASE Program was established to improve the knowledge and skills of at-risk youth in math, science and technology.

Program Expenditures Have Increased Over Time NGB reports that overall federal expenditures for the Challenge Program have increased over time, but states have also increased their expenditures since the program was permanently authorized in fiscal year 1998.

Between fiscal years 1998 and 2004, total expenditures for the Challenge Program, including funds spent to cover the federal and state cost shares and federal management expenses, have increased from about $63 million to about $107 million. For fiscal year 2004, for example, NGB expenses included $61. 6 million for the federal cost share and $5.

8 million for NGB management costs, while states contributed approximately $40. 5 million. 9 In addition, in 2000 and 2001, the Challenge Program received $5 million and $7,483,500 respectively from the Department of Justice.

Reserve Affairs stated that the primary use of these funds has been to start new Challenge Program sites. Since 2001, four programs were established using these funds and three programs' operations were maintained in 2002. In total, approximately $5.

97 million remain unspent in a nonexpiring account. Officials at Reserve Affairs and NGB told us that these funds remain unspent because no new Challenge Programs have started. According to these officials, new programs have not been established because state governments have not committed the required 40 percent match.

(See fig. 1 for total program expenditures from fiscal year 1998 to 2004, broken down by federal and state cost share and NGB 9At the time of our review, final expenditure reports for fiscal year 2005 Figure 1: Annual Expenditures for the National Guard Youth Challenge Note: Federal cost shares for fiscal years 2001 and 2002 include $6. 5 million from Department of Justice.

State cost share amounts are the required state matches only and do not include any additional funding or in-kind support the states might have provided. The Number of States Participating in the Challenge Program Has Increased When the pilot program began in 1993, there were 10 Challenge Program sites in 10 states. The program has now grown to 29 sites in 24 states and Puerto Rico.

10 In fiscal year 2005, Wyoming received funds to start up a program site. According to NGB, Wyoming will begin its first class in January 2006. In addition to those states currently operating Challenge Program sites, there are also nine states that have expressed interest in establishing new programs.

11 For example, according to NGB officials, representatives from Washington and Indiana National Guard units have visited some existing program sites and are in the process of developing state programs. Other states are interested in expanding their programs to serve more youth at existing sites or to open new locations.

On the other hand, for various reasons including difficulty meeting the state match requirement, lack of state support, and substandard facilities, four states have discontinued their Challenge Programs. Connecticut, a pilot program state,