Cities-Weather-S3-Snowflake-Slack-notif-ETL-by-Airflow-on-EC2

Public

This is my second AWS Cloud ETL project. This data pipeline orchestration uses Apache Airflow on AWS EC2. It demonstrates how to build an ETL data pipeline that would extract data (JSON) from the OpenWeatherMap API, transform it, dump it as CSV in S3 bucket, then copy it to destination tables in Snowflake DW and send Slack notification.

apache-airflow aws-ec2 business-intelligence dags data-warehousing etl-pipeline openweathermap-api orchestration python3 slack-webhook

Creat：2025-03-12T06:18:23

Update：2025-04-11T02:21:16

Stars

Stars Increase

Related projects

AutoGPT

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

177399

5个月前

+44today

Stable Diffusion Webui

Stable Diffusion web UI

155093

1年前

+38today

Transformers

Hot

bert

? Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

147733

2年前

+59today

Frp

expose

A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.

96761

7个月前

+45today

Supabase

Hot

The open source Firebase alternative. Supabase gives you a dedicated Postgres database to build your web, mobile, and AI applications.

86419

4个月前

+52today

Playwright

Hot

automation

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.

75467

4个月前

+65today

Deep Live Cam

real time face swap and one-click video deepfake with only a single image

72183

5个月前

+30today

Grafana

alerting

The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.

69261

4个月前

+22today