{"id":1404,"date":"2010-07-30T07:00:33","date_gmt":"2010-07-30T15:00:33","guid":{"rendered":"http:\/\/www.formtek.com\/blog\/?p=1404"},"modified":"2010-07-30T07:00:33","modified_gmt":"2010-07-30T15:00:33","slug":"big-data-science-experiments-with-cloud-computing","status":"publish","type":"post","link":"https:\/\/formtek.com\/blog\/big-data-science-experiments-with-cloud-computing\/","title":{"rendered":"Big Data: Science Experiments with Cloud Computing"},"content":{"rendered":"

The Department of Energy is asking the question whether cloud computing is useful for large-scale science simulation projects. \u00a0In the past the DoE and DoD have been leaders in large-scale modeling and simulation projects using some of the world’s biggest supercomputers. \u00a0Now they’re looking to see what sort of bang-for-the-buck they’ll be able to get from massive server farms used for cloud computing.<\/p>\n

The DoE testbed project is called Magellan<\/a>,\u00a0named\u00a0\u00a0after the Portuguese explorer Ferdinand Magellan, the first explorer to circumnavigate the earth. \u00a0Magellan\u00a0is a cloud environment built <\/a>on IBM\u2019s iDataplex chassis and based on\u00a01,440 Intel Nehalem quad-core processors (5,760 cores total) running Linux. \u00a0The total computer performance is on the order of 100 teraflop\/s. \u00a0 Scientists hope to be able to compare how a cloud environment stacks up with standard supercomputers in practice.<\/p>\n

The program hopes to be able to determine which types of scientific applications are best suited for cloud computing, what potential security risks are involved, and how well the cloud can hold up under very intensive data-rich calculations. \u00a0To test this, the project will be running some big-data-type scientific problems using tools like Hadoop (Map\/Reduce). \u00a0 \u00a0The kinds of problems that they envision tackling include looking at large \u00a0scientific datasets in biology, climate change, and physics. \u00a0The project is being run under DoE funding (with dollars obtained from the federal stimulus spending package<\/a> — a two-year $32 million project) at Lawrence Berkeley National Labs.<\/p>\n

It sounds great, but early results have been disappointing<\/a>. \u00a0Some of the early tests with the Magellan cloud involved weather simulation. \u00a0This type of simulation used a type of programming method called Message Passing Interface (MPI). \u00a0MPI is used extensively in weather forecasting and chemistry interaction calculations. \u00a0But results showed that there were significant slowdowns with MPI on Magellan, up to as much as 10 times slower than what had been expected.<\/p>\n

The study also intends to look to see if there are ways in which the public cloud could also be involved in some phases of the calculations.<\/p>\n

\"Digg<\/a><\/div>
\"Reddit<\/a><\/div>
\"Stumble<\/a><\/div>
\"Buzz<\/a><\/div>
\"Vote<\/a><\/div>
\"Share<\/a><\/div>
\"Bookmark<\/a><\/div>
\"Kick<\/a><\/div>
\"Shout<\/a><\/div>
\"Share<\/a><\/div>
\"Bookmark<\/a><\/div>
\"Post<\/a><\/div>
\"Google<\/a><\/div><\/div>","protected":false},"excerpt":{"rendered":"

The Department of Energy is asking the question whether cloud computing is useful for large-scale science simulation projects. \u00a0In the past the DoE and DoD have been leaders in large-scale modeling and simulation projects using some of the world’s biggest…<\/span><\/p>\n

Read more ›<\/a><\/div>\n

<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"class_list":["post-1404","post","type-post","status-publish","format-standard","hentry","category-cloud-computing"],"_links":{"self":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/1404","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/comments?post=1404"}],"version-history":[{"count":0,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/1404\/revisions"}],"wp:attachment":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/media?parent=1404"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/categories?post=1404"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/tags?post=1404"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}