{"id":3013,"date":"2012-06-05T07:00:09","date_gmt":"2012-06-05T15:00:09","guid":{"rendered":"http:\/\/www.formtek.com\/blog\/?p=3013"},"modified":"2012-06-05T07:00:09","modified_gmt":"2012-06-05T15:00:09","slug":"big-data-unlocking-meaning-from-unstructureddark-data","status":"publish","type":"post","link":"https:\/\/formtek.com\/blog\/big-data-unlocking-meaning-from-unstructureddark-data\/","title":{"rendered":"Big Data: Unlocking Meaning from Unstructured\/Dark Data"},"content":{"rendered":"<div id=\"_mcePaste\">Interest in both Big Data and analytics has dramatically increased over the last couple of years. \u00a0Some indication of their popularity can be seen using Google Trends, where we see large increases in the number of searches that were made for both of these terms.<\/div>\n<div><a href=\"http:\/\/formtek.com\/blog\/wp-content\/uploads\/2012\/05\/bigdata1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-3015\" title=\"bigdata\" src=\"http:\/\/formtek.com\/blog\/wp-content\/uploads\/2012\/05\/bigdata1.png\" alt=\"\" width=\"647\" height=\"244\" srcset=\"https:\/\/formtek.com\/blog\/wp-content\/uploads\/2012\/05\/bigdata1.png 647w, https:\/\/formtek.com\/blog\/wp-content\/uploads\/2012\/05\/bigdata1-300x113.png 300w\" sizes=\"auto, (max-width: 647px) 100vw, 647px\" \/><\/a><\/div>\n<div><\/div>\n<div><a href=\"http:\/\/formtek.com\/blog\/wp-content\/uploads\/2012\/05\/analytics.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-3016\" title=\"analytics\" src=\"http:\/\/formtek.com\/blog\/wp-content\/uploads\/2012\/05\/analytics.png\" alt=\"\" width=\"647\" height=\"242\" srcset=\"https:\/\/formtek.com\/blog\/wp-content\/uploads\/2012\/05\/analytics.png 647w, https:\/\/formtek.com\/blog\/wp-content\/uploads\/2012\/05\/analytics-300x112.png 300w\" sizes=\"auto, (max-width: 647px) 100vw, 647px\" \/><\/a><\/div>\n<div><\/div>\n<div>Now another related term is coming into more frequent use. \u00a0It is &#8216;<strong>dark data<\/strong>&#8216;. \u00a0It&#8217;s becoming a synonym for &#8216;unstructured data&#8217; &#8212; information that either does not have a pre-defined data model and\/or does not fit well into relational tables. \u00a0 The value of Dark Data is \u00a0locked up in a way so that it isn&#8217;t readily available for use by analytics.<\/div>\n<div><\/div>\n<div>Dark Data is a term picked up on by Lucid Imagination with the introduction of their<a title=\"Unstructured Dark Data\" href=\"http:\/\/www.marketwatch.com\/story\/lucid-imagination-combines-search-analytics-and-big-data-to-tackle-the-problem-of-dark-data-2012-05-09-82390\" target=\"_blank\"> LucidWorks Big Data platform<\/a>, a combination of Lucene\/SOLR full text search on a Haddop-backed Big-Data-size platform capable of processing massive amounts of data.<\/div>\n<div><a href=\"http:\/\/formtek.com\/blog\/wp-content\/uploads\/2012\/05\/darkdata.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-3017\" title=\"darkdata\" src=\"http:\/\/formtek.com\/blog\/wp-content\/uploads\/2012\/05\/darkdata.png\" alt=\"\" width=\"647\" height=\"239\" srcset=\"https:\/\/formtek.com\/blog\/wp-content\/uploads\/2012\/05\/darkdata.png 647w, https:\/\/formtek.com\/blog\/wp-content\/uploads\/2012\/05\/darkdata-300x110.png 300w\" sizes=\"auto, (max-width: 647px) 100vw, 647px\" \/><\/a><\/div>\n<div><\/div>\n<div><\/div>\n<div class=\"lightsocial_container\"><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/digg.com\/submit?url=https%3A%2F%2Fformtek.com%2Fblog%2Fbig-data-unlocking-meaning-from-unstructureddark-data%2F&amp;title=\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/digg.png\" alt=\"Digg This\" title=\"Digg This\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.reddit.com\/submit?url=https%3A%2F%2Fformtek.com%2Fblog%2Fbig-data-unlocking-meaning-from-unstructureddark-data%2F&amp;title=\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/reddit.png\" alt=\"Reddit This\" title=\"Reddit This\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.stumbleupon.com\/submit?url=https%3A%2F%2Fformtek.com%2Fblog%2Fbig-data-unlocking-meaning-from-unstructureddark-data%2F&amp;title=\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/stumbleupon.png\" alt=\"Stumble Now!\" title=\"Stumble Now!\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/buzz.yahoo.com\/buzz?targetUrl=https%3A%2F%2Fformtek.com%2Fblog%2Fbig-data-unlocking-meaning-from-unstructureddark-data%2F&amp;headline=\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/yahoo_buzz.png\" alt=\"Buzz This\" title=\"Buzz This\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.dzone.com\/links\/add.html?title=&amp;url=https%3A%2F%2Fformtek.com%2Fblog%2Fbig-data-unlocking-meaning-from-unstructureddark-data%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/dzone.png\" alt=\"Vote on DZone\" title=\"Vote on DZone\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.facebook.com\/sharer.php?t=&amp;u=https%3A%2F%2Fformtek.com%2Fblog%2Fbig-data-unlocking-meaning-from-unstructureddark-data%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/facebook.png\" alt=\"Share on Facebook\" title=\"Share on Facebook\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/delicious.com\/save?title=&amp;url=https%3A%2F%2Fformtek.com%2Fblog%2Fbig-data-unlocking-meaning-from-unstructureddark-data%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/delicious.png\" alt=\"Bookmark this on Delicious\" title=\"Bookmark this on Delicious\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.dotnetkicks.com\/kick\/?title=&amp;url=https%3A%2F%2Fformtek.com%2Fblog%2Fbig-data-unlocking-meaning-from-unstructureddark-data%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/dotnetkicks.png\" alt=\"Kick It on DotNetKicks.com\" title=\"Kick It on DotNetKicks.com\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/dotnetshoutout.com\/Submit?title=&amp;url=https%3A%2F%2Fformtek.com%2Fblog%2Fbig-data-unlocking-meaning-from-unstructureddark-data%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/dotnetshoutout.png\" alt=\"Shout it\" title=\"Shout it\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.linkedin.com\/shareArticle?mini=true&amp;url=https%3A%2F%2Fformtek.com%2Fblog%2Fbig-data-unlocking-meaning-from-unstructureddark-data%2F&amp;title=&amp;summary=&amp;source=\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/linkedin.png\" alt=\"Share on LinkedIn\" title=\"Share on LinkedIn\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.technorati.com\/faves?add=https%3A%2F%2Fformtek.com%2Fblog%2Fbig-data-unlocking-meaning-from-unstructureddark-data%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/technorati.png\" alt=\"Bookmark this on Technorati\" title=\"Bookmark this on Technorati\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/twitter.com\/home?status=Reading+https%3A%2F%2Fformtek.com%2Fblog%2Fbig-data-unlocking-meaning-from-unstructureddark-data%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/twitter.png\" alt=\"Post on Twitter\" title=\"Post on Twitter\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.google.com\/buzz\/post?url=https%3A%2F%2Fformtek.com%2Fblog%2Fbig-data-unlocking-meaning-from-unstructureddark-data%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/google_buzz.png\" alt=\"Google Buzz (aka. Google Reader)\" title=\"Google Buzz (aka. Google Reader)\" \/><\/a><\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>Interest in both Big Data and analytics has dramatically increased over the last couple of years. \u00a0Some indication of their popularity can be seen using Google Trends, where we see large increases in the number of searches that were made<span class=\"ellipsis\">&hellip;<\/span><\/p>\n<div class=\"read-more\"><a href=\"https:\/\/formtek.com\/blog\/big-data-unlocking-meaning-from-unstructureddark-data\/\">Read more &#8250;<\/a><\/div>\n<p><!-- end of .read-more --><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,35],"tags":[],"class_list":["post-3013","post","type-post","status-publish","format-standard","hentry","category-big-data","category-structured-data"],"_links":{"self":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/3013","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/comments?post=3013"}],"version-history":[{"count":0,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/3013\/revisions"}],"wp:attachment":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/media?parent=3013"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/categories?post=3013"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/tags?post=3013"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}