CASPER
New member
INSIDE WASHINGTON: NASA safety survey a mystery
--------------------------------------------------------------------------------
Years after thousands of pilots told NASA about their in-flight safety experiences and NASA shut down the survey without divulging any findings, the pilots' views remain a mystery.
The Government Accountability Office, the investigative arm of Congress, said the unprecedented $11 million telephone survey was sound in its design but plagued by resistance from federal aviation regulators and shortcomings in its implementation. NASA cut off the interviews in 2004 and chose not to analyze the results.
The results, which include pilots' experiences with bird strikes and other safety events, are so complicated they couldn't be analyzed without more costly and extensive research, including adjstments to make sure the pilots interviewed actually represented the nation's pilots overall, said the GAO report.
At this point, the survey has outlived its usefulness to air travel today, the GAO said.
The saga of what was formally called the National Aviation Operations Monitoring Service demonstrates how taxpayer money can buy a lot of government activity and end up without much to show for it.
Congress asked the GAO to review and analyze NASA's project last year after the space agency tried to keep its results secret. The interviews ran from 2001 to 2004.
"This was a well-designed project that failed because it was executed without proper agency oversight and didn't have the support it needed from its primary customer — the FAA," said Rep. Bart Gordon, D-Tenn., chairman of the Science and Technology Committee, which requested the GAO study.
Rep. Jerry Costello, D-Ill., chairman of the House Transportation Committee's aviation subcommittee, said it was "a perfect example of how good ideas and resources are wasted due to poor planning."
NASA began meticulous preparation as far back as 1997 to create a scientific survey for tracking air safety problems and accident precursors. A NASA contractor conducted 25,000 interviews with airline pilots and 5,000 with private pilots, asking them dozens of questions about safety incidents they encountered.
NASA had hoped the FAA would accept the survey to complement other government monitoring. But FAA officials challenged its validity when preliminary results indicated bird strikes and other events occurred more frequently than FAA's tracking systems showed. They contended the pilots' answers were subjective.
The GAO said it was not surprising that pilots reported hitting birds more often because aviation researchers have estimated 80 percent of bird strikes are not reported to the FAA's voluntary system. A US Airways jet crash landed in the Hudson River on Jan. 15 after a collision with large birds.
The GAO concluded that interviewing pilots was a valid way to identify air-safety issues. If revived, the survey would need clearer planning, sampling and analysis, and collaboration with the FAA and industry, as well as cost-benefit review to see if it was relevant, the report said.
The NASA survey came to light in 2007 when the AP disclosed that NASA had axed it and was refusing to release the results on grounds that the information was sensitive and could hurt the airlines' image and profits, or somehow identify pilots who were promised anonymity.
Former NASA Administrator Michael Griffin told Congress the project was poorly managed, didn't meet the space agency's standards and was not meaningful to air travelers.
Lawmakers said air travelers should know what the pilots reported.
Pulling conclusions from the survey results now, as Congress asked, would require complicated adjustments, the GAO said. For instance, the interviewees likely did not represent U.S. pilots at large because NASA used a public registry that withheld pilot names on request, resulting in unknown bias.
In light of costs to address such technical complexities, Congress changed its directive for the GAO to analyze the information. The agency instead focused on the methodology and ways to improve any future survey.
--------------------------------------------------------------------------------
Years after thousands of pilots told NASA about their in-flight safety experiences and NASA shut down the survey without divulging any findings, the pilots' views remain a mystery.
The Government Accountability Office, the investigative arm of Congress, said the unprecedented $11 million telephone survey was sound in its design but plagued by resistance from federal aviation regulators and shortcomings in its implementation. NASA cut off the interviews in 2004 and chose not to analyze the results.
The results, which include pilots' experiences with bird strikes and other safety events, are so complicated they couldn't be analyzed without more costly and extensive research, including adjstments to make sure the pilots interviewed actually represented the nation's pilots overall, said the GAO report.
At this point, the survey has outlived its usefulness to air travel today, the GAO said.
The saga of what was formally called the National Aviation Operations Monitoring Service demonstrates how taxpayer money can buy a lot of government activity and end up without much to show for it.
Congress asked the GAO to review and analyze NASA's project last year after the space agency tried to keep its results secret. The interviews ran from 2001 to 2004.
"This was a well-designed project that failed because it was executed without proper agency oversight and didn't have the support it needed from its primary customer — the FAA," said Rep. Bart Gordon, D-Tenn., chairman of the Science and Technology Committee, which requested the GAO study.
Rep. Jerry Costello, D-Ill., chairman of the House Transportation Committee's aviation subcommittee, said it was "a perfect example of how good ideas and resources are wasted due to poor planning."
NASA began meticulous preparation as far back as 1997 to create a scientific survey for tracking air safety problems and accident precursors. A NASA contractor conducted 25,000 interviews with airline pilots and 5,000 with private pilots, asking them dozens of questions about safety incidents they encountered.
NASA had hoped the FAA would accept the survey to complement other government monitoring. But FAA officials challenged its validity when preliminary results indicated bird strikes and other events occurred more frequently than FAA's tracking systems showed. They contended the pilots' answers were subjective.
The GAO said it was not surprising that pilots reported hitting birds more often because aviation researchers have estimated 80 percent of bird strikes are not reported to the FAA's voluntary system. A US Airways jet crash landed in the Hudson River on Jan. 15 after a collision with large birds.
The GAO concluded that interviewing pilots was a valid way to identify air-safety issues. If revived, the survey would need clearer planning, sampling and analysis, and collaboration with the FAA and industry, as well as cost-benefit review to see if it was relevant, the report said.
The NASA survey came to light in 2007 when the AP disclosed that NASA had axed it and was refusing to release the results on grounds that the information was sensitive and could hurt the airlines' image and profits, or somehow identify pilots who were promised anonymity.
Former NASA Administrator Michael Griffin told Congress the project was poorly managed, didn't meet the space agency's standards and was not meaningful to air travelers.
Lawmakers said air travelers should know what the pilots reported.
Pulling conclusions from the survey results now, as Congress asked, would require complicated adjustments, the GAO said. For instance, the interviewees likely did not represent U.S. pilots at large because NASA used a public registry that withheld pilot names on request, resulting in unknown bias.
In light of costs to address such technical complexities, Congress changed its directive for the GAO to analyze the information. The agency instead focused on the methodology and ways to improve any future survey.
