The most popular and comprehensive Open Source ECM platform
Let us know more
Big Data: Science Experiments with Cloud Computing
The Department of Energy is asking the question whether cloud computing is useful for large-scale science simulation projects. In the past the DoE and DoD have been leaders in large-scale modeling and simulation projects using some of the world’s biggest supercomputers. Now they’re looking to see what sort of bang-for-the-buck they’ll be able to get from massive server farms used for cloud computing.
The DoE testbed project is called Magellan, named after the Portuguese explorer Ferdinand Magellan, the first explorer to circumnavigate the earth. Magellan is a cloud environment built on IBM’s iDataplex chassis and based on 1,440 Intel Nehalem quad-core processors (5,760 cores total) running Linux. The total computer performance is on the order of 100 teraflop/s. Scientists hope to be able to compare how a cloud environment stacks up with standard supercomputers in practice.
The program hopes to be able to determine which types of scientific applications are best suited for cloud computing, what potential security risks are involved, and how well the cloud can hold up under very intensive data-rich calculations. To test this, the project will be running some big-data-type scientific problems using tools like Hadoop (Map/Reduce). The kinds of problems that they envision tackling include looking at large scientific datasets in biology, climate change, and physics. The project is being run under DoE funding (with dollars obtained from the federal stimulus spending package — a two-year $32 million project) at Lawrence Berkeley National Labs.
It sounds great, but early results have been disappointing. Some of the early tests with the Magellan cloud involved weather simulation. This type of simulation used a type of programming method called Message Passing Interface (MPI). MPI is used extensively in weather forecasting and chemistry interaction calculations. But results showed that there were significant slowdowns with MPI on Magellan, up to as much as 10 times slower than what had been expected.
The study also intends to look to see if there are ways in which the public cloud could also be involved in some phases of the calculations.
