Scientists have processed 400 gigabytes of data a second as they tested data pipelines for the Square Kilometre Array (SKA) telescope.

Researchers from ICRAR in Perth, Oak Ridge National Laboratory in the US and Shanghai Astronomical Observatory in China used the world's most powerful supercomputer - Summit - to process simulated observations of the early universe ahead of the radio telescope being built in Western Australia and South Africa.

The data rate achieved was the equivalent of more than 1,600 hours of standard definition YouTube videos every second.

Professor Andreas Wicenec, the director of Data Intensive Astronomy at the International Centre for Radio Astronomy Research (ICRAR), said it was the first time radio astronomy data has been processed on this scale. "Until now, we had no idea if we could take an algorithm designed for processing data coming from today's radio telescopes and apply it to something a thousand times bigger," he said.

"Completing this test successfully tells us we'll be able to deal with the data deluge of the SKA when it comes online in the next decade. But, the fact that we need the world's biggest supercomputer to run this test successfully shows the SKA's needs exist at the very edge of what today's supercomputers are capable of delivering."

The billion-dollar SKA is one of the world's largest science projects, with the low frequency part of the telescope set to have more than 130,000 antennas in the project's initial phase, generating around 550 gigabytes of data every second.

Summit is located at the US Department of Energy's Oak Ridge National Laboratory in Tennessee. It is the world's most powerful scientific supercomputer, with a peak performance of 200,000 trillion calculations per second.

Oak Ridge National Laboratory software engineer and researcher Dr. Ruonan Wang, a former ICRAR PhD student, said the huge volume of data used for the SKA test run meant the data had to be generated on the machine itself. "We used a sophisticated software simulator written by scientists at the University of Oxford, and gave it a cosmological model and the array configuration of the telescope so it could generate data as it would come from the telescope observing the sky," he said.

"Usually this simulator runs on just a single computer, generating only a small fraction of what the SKA would produce. So we used another piece of software written by ICRAR, called the Data Activated Flow Graph Engine (DALiuGE), to distribute one of these simulators to each of the 27,648 graphics processing units that make up Summit. We also used the Adaptable IO System (ADIOS), developed at the Oak Ridge National Laboratory, to resolve a bottleneck caused by trying to process so much data at the same time."

The test run used a cosmological simulation of the early universe at a time known as the Epoch of Reionisation, when the first stars and galaxies formed and became visible.

Professor Tao An of the Shanghai Astronomical Observatory said the data was first averaged down to a size 36 times smaller. "The averaged data was then used to produce an image cube of a kind that can be analysed by astronomers," he said. "Finally, the image cube was sent to Perth, simulating the complete data flow from the telescope to the end-users."

Construction of the SKA is expected to begin in 2021.

Related Links
International Centre For Radio Astronomy
Space Technology News - Applications and Research

Tweet

Thanks for being here; We need your help. The SpaceDaily news network continues to grow but revenues have never been harder to maintain. With the rise of Ad Blockers, and Facebook - our traditional revenue sources via quality network advertising continues to decline. And unlike so many other news sites, we don't have a paywall - with those annoying usernames and passwords. Our news coverage takes time and effort to publish 365 days a year. If you find our news sites informative and useful then please consider becoming a regular supporter or for now make a one off contribution.
SpaceDaily Contributor $5 Billed Once credit card or paypal		SpaceDaily Monthly Supporter $5 Billed Monthly paypal only

China data centres set to consume more power than Australia: report
Beijing (AFP) Sept 9, 2019
China's data centres - the backbone of the country's fast-growing digital economy - are on track to guzzle more electricity by 2023 than all of Australia last year, said a Greenpeace report published Monday. While China leads the world in renewable energy capacity, data centres in the country remain dependent on coal. Nearly three-quarters of the electricity used by data centres was sourced from coal in 2018, according to research by Greenpeace East Asia and the North China Electric Power Univ ... read more