Skip to content
Suchen Sie nach Produkten und Lösungen von InterSystems, Karrieremöglichkeiten und mehr.

Crunching Data to Map the Milky Way

European Space Agency (ESA)

Eduardo Anglada of the European Space Agency (ESA) and the European Space Astronomy Centre is currently working on the Gaia mission to map one billion objects in the Milky Way. Anglada and Jordi Calvera Sagué of InterSystems explained how much data is involved in categorizing a billion space objects, and the requirements of the technology that supports this extra-terrestrial task.

The mission of the European Space Agency (ESA) is to explore the universe and it has a fleet of 10 craft out in space to do so. The Gaia space mission began in 1993 and the actual spacecraft was launched in 2013. Its aim is to create a precise 3D map of 10% of the Milky Way, and in doing so characterize up to 1 billion objects.

Eduardo Anglada
Eduardo Anglada

Eduardo Anglada, computer analyst and grid engineer at ESA, explained that by ‘characterize’ he meant that he and his team are aiming to chart the position, velocities, physical parameters, and temperature of those objects.

They also want to answer questions relating to the metallicity and age of the objects, as well as whether they are binary systems (with two stars close together). The process will reveal the composition, formation, and evolution of the galaxy.

As one might imagine, the numbers are staggering. The satellite,  which costs almost €700 million, is between 700,000 kilometers and 1.5 million kilometers away from Earth. The stakes are high, as the hardware, the software, and the security of the data all have to be impeccable because at that distance, it is impossible to repair or refuel the satellite. Essentially, the data that is downloaded from this satellite is irreplaceable.

The camera onboard the craft is 938 million pixels and has 106 charge coupled devices (CCDs), the type of image sensor used in most digital cameras. “This is one of the biggest cameras ever built,” said Anglada. The satellite takes six hours to do a full rotation on its axis and sends down between 45 and 100 gigabytes per day, having taken around 70 million images daily. “Once the data is on Earth we have 24 hours to analyze it.”

The data from the camera is downloaded daily and is received by three antennae in Spain, Australia, and Argentina. With the different antennae, ESA have a ‘follow the sun’ way of working so that the satellite is constantly tracked by an antenna in one of those countries. “If we lose it, it’s a disaster. In a few hours, it can be many, many kilometers from where it is supposed to be and it can be very difficult to track and find it again,” Anglada explained.

Eduardo Anglada, European Space Agency (ESA), and European Space Astronomy Centre

From those locations, the data is sent to the mission operation center in Germany where it is checked to see if there are any problems with the satellite, and fix them if they are present. Then the compressed data is sent to ESA in Madrid where the observations are decompressed, calibrated, and stored in a cache database.

Then it is sent to the data specialist centers, for among other things, simulations and object, photometric, and spectroscopic processing. It then goes back to Madrid, the location of the central database.

The system was set up in a hub and spokes model so that all researchers could access the data in order to fulfill the requirements. The spoke locations are dotted around Europe including Cambridge, Turin, and Toulouse.

There are four stages to the data processing; daily and cyclic operations, the main database, calibration activities and payload commanding, and finally development.

Anglada said that the daily database is quite big now. “It is almost 30 terabytes. It’s a single instance. We have a big server with 1.5 terabytes of RAM and seven terabytes of solid state disk.”

The cyclic operations are completed every four to six months. In that time the data centers have to finish their processes of the data and start sending it round to other data centers in order for them to refine the results.

InterSystems Caché is part of the main database and the daily processing. The space agency got in touch with Jordi Calvera Sagué, regional managing director at InterSystems, to say that it faced the challenge of inserting and processing a large volume of data in a short period of time. ESA provided the company with some dummy data and the software engineers did a configuration of the database cache in three days. “We changed the architecture after the proof of concept, but it was relatively quick and just did the configuration,” he said.

In addition to InterSystems Caché, ESA also uses Aspera for distribution across the data processing centers and Atlassian Jira for bug tracking. The ESA team comprises 400 people in Europe and 26 in ESAC with different areas of expertise from calibration to management to daily download.

So what was the attraction to InterSystems? Anglada said that it is extremely reliable and the company offers comprehensive support. “In the last 11 months we have had no problems at all. I am a very technical person, I am part of the daily team and InterSystems has the best support that we have had by far.” Anglada added that it is very easy to get in touch with the account manager and find out if the tool can feasibly meet a new requirement. “It is not so common that a company, once the project is mature, wants to continue working with you like that.” He added that that Caché is so robust that daily checks can be completed in minutes.

As previously mentioned, there are some big numbers involved in this project and for Anglada, one is particularly impressive. “We have analyzed more than one trillion observations. It is quite a figure and it is thanks to Caché. It handles 30 terabytes of database without problems.” InterSystems has even increased the shelf life of the project. Calvera Sagué said that the satellite is so optimized that the mission can keep going for two to three years longer than anticipated.

The results of this mission were published in a catalog for the first time in 2016 and mapped 300 light years, just part of the Milky Way. The second release April 2018 has mapped about 8,000 light years. The Gaia team was able to document the brightness of 1.7 billion stars and the surface temperature of 1.4 billion.

The level of detail meant that there were some surprising discoveries. “Having the velocities meant we could study the kinematics of the galaxy. For example the Milky Way clashed with another galaxy many years ago and we have seen the last stages of how both galaxies are merging. It has been known for years but so far nobody could measure it. With this data it has been possible.”

Furthermore, last year, the first interstellar comet, Oumuamua, was detected. “It is the first object detected in the Solar System which is not part of the solar system. This comet was travelling through space for about 6,000 years and it came into the Solar System so fast that it wasn’t trapped by the Sun but the Sun’s gravity changed its trajectory.”

The third release is likely to take place in the first half of 2021, and the date of the final release, which will consist of full astrometric, photometric, and radial-velocity catalogs, is yet to be decided. Anglada said the astronomy community is extremely happy with the results of this mission so far and the average number of scientific articles citing the catalog is three or four per day “which is amazing for science.”

“Everything is quite material now,” said Anglada. “Each time a scientist realizes that we can measure something new, new ideas for the corresponding development takes place. With this catalog the scientific community will be able to study the physics they want about these stars for the next 30 or 40 years.”

Read the complete article

Andere Ressourcen, die Ihnen gefallen könnten.

09 Oct 2024
Entmystifizierung von Vector Embeddings: Erfahren Sie, was sie sind, wie sie funktionieren und warum sie für KI so wichtig sind.
09 Oct 2024
Produktdatenblatt
InterSystems Data Fabric Studio ist ein wesentliches Element einer modernen Datenstrategie für Unternehmen der Finanzdienstleistungsbranche, um Wachstum zu ermöglichen, die Geschäftsberichterstattung zu optimieren, Risiken zu reduzieren und die Einhaltung von Vorschriften zu unterstützen.
26 Sep 2024
Erfahren Sie, wie Generative KI im Gesundheitswesen die Patientenversorgung, Diagnose und Arzneimittelentwicklung revolutioniert. Erfahren Sie mehr über Anwendungen, Vorteile und ethische Überlegungen.
23 Sep 2024
Alle Welt redet über KI, doch sich dem als Krankenhaus zu nähern, wird vielerorts als schwierig empfunden. In der E-HEALTH-COM Videosprechstunde erläutert Julian Grundt von InterSystems nicht nur, wie KI-Anwendungen ein Krankenhaus medizinisch und organisatorisch voranbringen können. Er zeigt auch den Weg dorthin auf – der gar nicht so viel Personal bindet, wie manche denken.
20 Sep 2024
Entdecken Sie, wie das Datenbank-Sharding die Leistung und Skalierbarkeit verbessert. Lernen Sie die Vorteile, Methoden und Best Practices für die Implementierung dieser leistungsstarken Datenverteilungstechnik in Ihrem Unternehmen kennen.
18 Sep 2024
Erfahren Sie, was eine relationale Datenbank ist, was ihre wichtigsten Merkmale sind und warum sie für Ihr Unternehmen so wichtig ist. Erfahren Sie mehr über die Vorteile, beliebte Systeme und den Vergleich mit anderen Datenbanken
17 Sep 2024
Bloor Spotlight
Data Fabric ist eine flexiblere Datenarchitektur, die ohne den Aufbau weiterer Datensilos auskommt und stattdessen einen dynamischen Zugriff auf die Daten vor Ort in den operativen Quellsystemen ermöglicht.
17 Sep 2024
Bloor Research
Ein Bloor Gold Mutable Award 2024 Von Andy Hayler - Practice Leader, Daten als Vermögenswert InterSystems und Data FabricsWarum sollte Sie das interessieren?
17 Sep 2024
Bloor Market Update
Ein Blick auf den aktuellen Stand des Data-Fabric-Marktes von Bloor Research
04 Sep 2024
Künstliche Intelligenz im Gesundheitswesen
Die Medizin ist mit ihrer Fülle an Diagnose- und Behandlungsdaten prädestiniert für KI-Anwendungen. Drei Anwendungsfälle zeigen, wo sich KI heute im administrativen klinischen Bereich einsetzen lässt.

Machen Sie den nächsten Schritt

Wir würden gerne mit Ihnen reden. Füllen Sie einige Angaben aus und wir melden uns bei Ihnen.
*Erforderliche Felder
Highlighted fields are required
*Erforderliche Felder
Highlighted fields are required
** Durch die Auswahl von "Ja" erklären Sie sich damit einverstanden, dass wir Sie für Neuigkeiten, Updates und andere Marketingzwecke in Bezug auf bestehende und zukünftige Produkte und Veranstaltungen von InterSystems kontaktieren. Darüber hinaus erklären Sie sich damit einverstanden, dass Ihre geschäftlichen Kontaktinformationen in unsere CRM-Lösung eingegeben werden, die in den Vereinigten Staaten gehostet wird, aber im Einklang mit den geltenden Datenschutzgesetzen gepflegt wird.