The most popular and comprehensive Open Source ECM platform
Let us know more
Google: Structured Data is Hard
We talk a lot about how hard it is to unlock the data stored in spreadsheets, powerpoints, and text documents — unstructured data. Generally the assumption has been that searching for data stored in structured database tables is easy and well understood and it is unstructured data that is hard. When you’re on the inside and have direct access to the database tables that may be the case, but when you’re on the outside looking in, it isn’t that easy.
Google’s Alon Halevy commented that Google has had a hard time searching web pages that are database-driven. Google search can search web pages that are artifacts representing structured data in databases, but they have no access to the underlying database.
Google estimates that there are really only about 5 million static web pages out there, but for every static web page there are 500 dynamic database-driven ones. Crawling through the dynamic pages to find various permutations to get a good snapshot of the underlying data isn’t an easy problem.
Google calls these dynamic pages, the “Deep Web” or “vertical searching”. Halevy now heads the Deep Web project at Google. Yahoo has a similar project underway that makes use of a technology called SearchMonkey.
