Last month, the Government Accountability Office (GAO) released results from its investigation into the effectiveness of the Department of Defense (DoD) Pilot Mentor-Protégé Program. In accordance with the National Defense Authorization Act (NDAA) for 2016, GAO reviewed DoD’s procedures for approving mentor-protégé agreements, its performance measures for the program, and the differences between its program and the new All Small Mentor Protégé Program established by the Small Business Administration (SBA).
Based on its findings, GAO recommended that DoD conduct periodic reviews of its mentor-protégé approvals, and that it establish performance goals and related measures to assess the program’s success. As the only agency permitted to retain a mentor-protégé program separate from that of the SBA, and the only agency that receives federal funding ($28.3 million in 2016) to reimburse qualifying mentors, review and incorporation of GAO’s recommendations are particularly important for the continuation of this “pilot” program.
In its study, GAO reviewed DoD procedures for processing mentor-protégé applications and how it measures the program’s success. GAO’s audit spanned from January 2016 to April 2017 and resulted in two specific recommendations. GAO also analyzed a randomly selected sample of 44 of the 78 active mentor-protégé agreements to verify their compliance with the mentor-protégé application criteria.
First, GAO found that DoD lacks proper controls to provide reasonable assurance that the approved mentor-protégé agreements include the required elements enumerated in DoD policies and regulations. Mentor-protégé agreements are submitted to the DoD Office of Small Business Programs (OSBP) and reviewed by the director of small business programs for each of the DoD components. However, DoD regulations and procedures have not established a standard process that DoD components should follow in approving mentor-protégé agreements. Instead, each of the DoD components – the Air Force, the Army and the Navy – has published individualized guidance on mentor-protégé approval.
In reviewing the sample of active mentor-protégé agreements, GAO found that approximately 27% did not contain the elements required of all agreements – irrespective of which component the entities were applying for. According to GAO, 25% of agreements omitted signatures from the mentor and protégé, 9% were missing the protégé’s NAICS code, and 7% didn’t include start and end dates for the agreement. GAO concluded that the discrepancies resulted from inadequate procedures for approving mentor-protégé agreements and from the lack of guidance in how to review mentor-protégé proposals. While the components are given a checklist for reviewing the proposals, the OSBP does not mandate that they abide by the checklist to avoid micromanagement and give the components flexibility in creating their own checklist.
GAO highlighted that DoD is required to abide by federal control standards, which requires that the OSBP oversees the pilot program. However, under DoD’s current protocols, the OSBP does not review the components’ procedures for evaluating mentor-protégé proposals and therefore cannot ensure that the program is serving its intended purposes.
In conjunction with the OSBP’s insufficient oversight, GAO recommended that DoD establish better performance measures and goals to fully assess the program’s success. GAO explained that evaluating performance of the program requires comparing actual performance against planned or expected results. GAO recognized that DoD collects and creates annual reports on protégé revenues, employment levels, and prime and subcontracts awards, but the collected information should be weighed against established goals for these factors. Similarly, GAO urged DoD to collect information not only on the increase in participation in small business contracts, but also on how the program enhances each small business’s capabilities in carrying out contracts awarded under the program. Hence, according to GAO, not only does DoD not properly access the data it has available, it is also not fully assessing all of the program’s intended benefits.
Upon receiving a draft of the report, DoD concurred with GAO’s recommendations and has drafted a standard operating procedure (SOP) to internalize GAO’s suggestions. According to DoD, the SOP revises the checklist for reviewing mentor-protégé agreements and requires component managers to review the packages before submitting them for further review. While DoD has chosen not to adopt the same performance goals used for other small businesses, it is working on establishing its own performance criteria.
The report, as well as DoD’s response to GAO recommendations and critiques, suggests that DoD will work to improve its review of mentor-protégé proposals, standardize how the various components implement the program, require more information from mentors and protégés to evaluate the program’s benefits, and set goals for measuring the program’s success. Given the findings, mentors and protégés should carefully review their proposals prior to submitting them to the DoD, verifying that the agreement contains all proper signatures, dates, and a prediction of its success in improving the protégé’s contracting abilities.
According to DoD’s own estimates, it would take approximately two years to develop and establish baselines for measures to calculate the program’s effectiveness. GAO warns that absent reform in evaluating the program’s success, Congress may be reluctant in reauthorizing the program, which has been consistently extended since its introduction in 1990. Finally, another option for the future of the program is implementing reforms that harmonize the DoD program with the provisions of the SBA All Small program, changes that the agencies anticipate to address in July 2017. A summary of the differences in the programs can be found in our earlier post, here.