Google pushes federal governent sites to expand searchable content

Published 26 October 2006

As much as 40 percent of government content is unavailable to search users; penetrating the “deep Web” is key; an OMB order requires agency cooperation, but the deadline has passed and Google is lending a hand to get the program back on track

Fresh off its acquisition of YouTube, and still stinging from criticism over its plan to digitally reproduce the content of the world’s largest libraries, Google is now going after government information. U.S. agency information, being presumably more accurate than other sources of data, is highly coveted by users, but search engines are not able to index “deep Web” areas of many government Web sites — forty percent by some estimates. Information housed at the Environmental Protection Agency’s Regulations.gov Web site, for instance, which allows users to view government regulations and post comments on proposed agency rules, is not searchable.

If you were a business owner and found out you were potentially subject to a new regulation that you wanted to find out more information on, it may be difficult to find this information using a search engine like Google,” J.L. Needham, a strategic partner development manager at Google said. “The problem is that search engines are unable to crawl the full text of many government agencies’ databases.” A 2005 Office of Management and Budget order required agencies to configure data in a searchable format by 1 September 2006, but many are behind in drafting the neccesary rules. Google, eager to get its hands on this vast trove, is now meeting with agency officials to see what can be done.

-read more in Daniel Pulliam’s GovExec report