On May 17, 1999, PricewaterhouseCoopers (PwC) issued a spiral-bound report with a four-color cover, containing 2 pages of text and 1 illustrative diagram. This spiffy document entitled, Executive Summary: Performance Measurement Study of Americas Promises Commitments, is available for free to anyone and was announced with press release fanfare. The arrogance of this document is breathtaking.
In a time when all nonprofits are being held to the standard of impact measurement, this document trumpets out activity counts as if doing something is equivalent to accomplishing something. The PwC report coincides with the news that Americas Promise (AP) has decided to remain in business as a full organization past its original year 2000 sunset date. So one can only assume that the attitude at AP is that everyone is so awed with the Emperor, no one will remark on his lack of clothes.
For those of you who have not yet seen the report, it is simple to summarize (clearly there must be a longer report somewhere, but this is the material widely disseminated.) PwC notes from the outset that data are presented as reported by the commitment makers (the underlining is theirs). In other words, the only source of data is the companies and large agencies that made commitments in the first place. Not one recipient of services or donations was asked to validate the claims -- neither nonprofit or public agencies nor, naturally, children themselves. Furthermore, there is a complete abandonment of any focus on volunteering. No one is assessing whether volunteering has increased, strengthened, or made a difference. This PwC report discusses the number of delivery sites, not service providers (i.e., volunteers).
Here are the published results of the study. (I am giving you every figure in the Executive Summary.) Out of 441 Commitments made to AP, only 91 commitment-makers reported, claiming to have provided service at 14,000 Reported Delivery Sites and -- take a deep breath for this one -- claiming 10,354,634 Children Reached. Yes, thats ten million, three hundred fifty-four thousand, six hundred thirty-four. (And I thought the number was 10, 354, 672!) The total number of children under the age of 17 in the United States as of January 1999, according to the U.S. Census Bureau Web site, is 50,906,000. So, according to AP and affirmed by PwC, AP has reached 20% of the total youth population. At that rate, you and I ought to be able to walk out to any street corner and yell out will any young person who has been reached by Americas Promise please identify yourself? -- and get 1 in 5 positive responses!!
Now, let's take these statistics further. Are we supposed to extrapolate that, if 20% of the commitment makers (the % of those that reported) served 20% of youth, did 100% serve the entire youth population?
The dollar value figures reported by only 47 commitment-makers (out of 441), show 22,000 Reported Delivery Sites and $209,501,335 Total Dollar Value. This figure ostensibly includes gifts of goods and money as well as the value of volunteer services (the calculation method used is not described.) Does any of this make sense to anybody? Does this mean if all 441 had reported, the figure would be larger than the gross national product?
But here is the best part. PwC uses the two text pages of the report to explain its methodology and, in one of only four section headings, remarks on Data Integrity. In these all-important few paragraphs, PwC states: ...However, there are several factors that are not addressed in this review that could influence the number of children reached and the dollar value of all commitments. These factors include: Duplication; Inaccurate data; Inadequate data. So what else is left? In the old days, when accounting firms focused on impartial audits rather than on vague consulting, no CPA would have released a report based on guess work. This is not a clean review and no one should quote the data without PwCs caveats. Yet in America's Promise's press release announcing the report located at (no longer available) http://www.americaspromise.org/rtn2_frame.htm , AP freely throws around the numbers with no caveats. (By the way, I'd love to refer you to this "Executive Summary" on the Internet but couldn't find any links to it on AP or PwC's sites.)
PwC notes that the commitment reporting requirements currently in place were not designed to produce information that could be easily audited (page 2). Yet it persists in calling the exercise a performance measurement study. I clearly remember Colin Powell being interviewed by Ted Koppel on Nightline during the Presidents Summit. Koppel asked Powell how AP would make sure that these commitment-makers actually follow through on what they have promised. Powell responded --with stunning naivete -- that when American business leaders say theyll do something, they will.
The real mystery is why there has been such public silence surrounding the inadequacies of Americas Promise. In most other circumstances, a report such as this one would have been loudly challenged or, at least, received some scrutiny. And PricewaterhouseCoopers might have been embarrassed enough to refuse to issue a questionable report with its good name on the cover. Is all of this just another example of the big lie? Make statements so grandiose that no one could imagine anyone would fabricate them?
During the past two years, I have been interviewed about AP by about a dozen reporters from various publications, some of general interest and some in the nonprofit field. In every case the reporter told me: Its a relief to find someone willing to be quoted. Everyone else insists on speaking off the record.
I understand that many people can tell wonderful stories about specific, local results of AP's efforts. However, the purpose of this Hot Topic is not to share success or horror stories, rather to ask you the following questions:
- Even if these extreme figures are accepted at face value, can anyone demonstrate that children "reached" were "helped"?
- At a time when real service programs are being held to such high accountability standards for relatively small amounts of money, how can a multi-million dollar organization get away with so little scrutiny?
- Why are so many people afraid to go public with negative opinions widely voiced behind the scenes?
- What exactly is the risk of questioning Colin Powells leadership or why he is requiring megabucks for his presentations to conferences?
- What consequences -- real or imagined -- do people anticipate if they protest the squandering of so much money, media attention, and even good intentions while the genuine needs of youth remain unmet?
Receive an update when the next "News and Tips" is posted!