Climate researchers involve big cloud providers in big data challenges

The ability of the cloud to process large amounts of data using machine learning algorithms is attracting more researchers from the world of data-intensive climate science.

“This is a game changer,” said Duncan Watson Paris, a postdoctoral fellow at Oxford University. He uses Amazon Web Services cloud-based machine learning algorithms to better understand how aerosols such as soot and sulphate from cargo ships change real clouds.

And that change hasn’t been overlooked by big three cloud providers. AWS and others offer subscription-based remote data storage and online tools, and researchers say that instead of setting up and maintaining their own hardware, it could be an affordable alternative. Stated.

With the additional computing power of the cloud, researchers can identify patterns and machine learning designed to extract insights from vast amounts of climate data, such as seawater temperatures, rainfall patterns, and decades of satellites. You can easily execute the algorithm. image.

Werner Vogels, Chief Technology Officer of Inc, said: “Datasets are getting bigger and bigger, so machine learning is starting to play a more important role in finding patterns in the data.”

Last year, Oxford researchers began using AWS machine learning tools to investigate the effects of aerosols on clouds. Certain aerosols are earth-cooled and can offset some of the effects of greenhouse gases.

“We’re not doing much well,” Watson-Paris said. Understanding how aerosols change clouds may help mitigate the effects of climate change, he added.

Researchers then screened petabytes of satellite imagery to identify so-called “ship trajectories,” which are clouds that appear bright because they are exposed to aerosols from the exhaust fumes emitted by the ship. I trained.

Previously, the observation was made by statistical analysis of only a selected number of images. “We would have manually sat there and labeled these trucks,” Watson Paris said.

The next step is to use machine learning algorithms to help researchers understand whether the relationship between aerosols and clouds is causal rather than statistical. Fifteen PhD students across Europe began working with climate scientists and machine learning experts in September to try to understand it.

Duncan Watson Paris, a postdoctoral fellow at Oxford University.


Duncan Watson-Paris

Simon Mingay, vice president of research at Gartner, said climate researchers are following the bigger trend of academic institutions and businesses moving to the cloud.

“Sustainability, climate mitigation, and climate adaptation are incredibly data-intensive. You’re talking about large amounts of data, and you’re talking about large and complex datasets,” he said. It was. “Of course, the cloud will play an increasingly important role.”

Some cloud providers have programs that directly address environmental research.

Microsoft Corp.

For example, the AI ​​for Earth program provides grants and technical assistance from Azure, the cloud division, for such projects.

According to Amazon’s Vogels, AWS uses machine learning services to support start-ups and nonprofits in dozens of climate-related projects, from soil data analysis and marine conservation efforts to forest conservation. The number is increasing.

Broadly speaking, with tools like Amazon SageMaker launched in 2017, software developers in all industries can build, train, and use machine learning models faster and with less technical complexity than ever before. You will be able to do it.

“We’re trying to democratize machine learning to make sure it’s available to everyone, not just in the realm of data scientists,” Vogels said.

Access to advanced machine learning tools is one of the main benefits of using the cloud, said Sid-Ahmed Boukabara, principal scientist at the National Oceanic and Atmospheric Administration’s Satellite Applications and Research Center.

In October, NOAA’s satellite information service announced a partnership with Google to study ways to use cloud-based machine learning tools to provide more accurate weather forecasts and monitor extreme weather events such as hurricanes.

NOAA collects terabytes of environmental data, including satellite images, daily. “I’m very interested in the cloud and the AI ​​technology that comes with it because of the big data challenges,” says Boukabara.

There are problems with huge amounts of data, and several techniques are used to address them. The first is to “thin out,” or reduce the amount of data, to perform the analysis on NOAA’s proprietary high-performance computing hardware.

However, Google Cloud could help NOAA analyze data using advanced machine learning algorithms designed to process more data and run on specific AI-based hardware. Yes, he said. He added that algorithms that analyze data in the cloud could help make more accurate forecasts within a 1-kilometer radius compared to a 10-kilometer radius.

“AI helps add value to what our system is currently producing,” he said.

Write to Sara Castellanos at

Copyright © 2020 DowJones & Company, Inc. all rights reserved. 87990cbe856818d5eddac44c7b1cdeb8

Climate researchers involve big cloud providers in big data challenges

Source link Climate researchers involve big cloud providers in big data challenges

Back to top button