Setting up email notifications in airflow – Step by Step learning

Apache Airflow, a robust workflow management platform, comes equipped with an array of tools to keep users informed about the state of their tasks and DAGs. One particularly effective method of notification is through email alerts. In this article, we’ll walk through the steps to set up email notifications in Airflow and create an example DAG that sends an email upon task failure.

Why use email notifications in airflow?

Prompt Alerting: Immediate notifications help users quickly address any failures or issues.

Detailed Reporting: Emails can contain logs, error messages, or custom messages that provide context to the issue.

Versatility: Suitable for teams of all sizes, from solo developers to large organizations.

Setting Up email in Airflow:

1. Configure SMTP:

Airflow uses SMTP (Simple Mail Transfer Protocol) to send emails. First, you’ll need to configure SMTP settings in your airflow.cfg file.

Here’s a general setup using Gmail’s SMTP server as an example:

[smtp]
smtp_host = smtp.freshers.in
smtp_starttls = True
smtp_ssl = False
smtp_user = sachin@freshers.in
smtp_password = your_password
smtp_port = 587
smtp_mail_from = sachin@freshers.in

2. Set up email on failure:

By default, Airflow sends emails to owners of the DAGs when a task fails. To enable this:

Set the email_on_failure parameter to True in the default_args of your DAG.
Provide a list of email addresses to the email parameter.

3. Set up email on retrying:

If you want to receive an email when Airflow retries a task:

Set the email_on_retry parameter to True in the default_args of your DAG.

Example DAG with email notifications:

Let’s create a DAG named freshers_in_learning_email_alerts that sends an email when a task fails:

from datetime import datetime, timedelta
from airflow import DAG
from airflow.operators.dummy_operator import DummyOperator
from airflow.operators.python_operator import PythonOperator
def failure_function():
    raise ValueError("This function is designed to fail!")
default_args = {
    'owner': 'airflow',
    'start_date': datetime(2023, 1, 1),
    'retries': 1,
    'retry_delay': timedelta(minutes=5),
    'email': ['rakesh@<code class="language-bash">freshers.in'], 'email_on_failure': True, 'email_on_retry': False } dag = DAG('freshers_in_learning_email_alerts', default_args=default_args, description='A DAG to demonstrate email alerts', schedule_interval=timedelta(days=1), catchup=False) start = DummyOperator(task_id='start_task', dag=dag) end = DummyOperator(task_id='end_task', dag=dag) fail_task = PythonOperator(task_id='fail_task', python_callable=failure_function, dag=dag) start >> fail_task >> end

In this DAG, the fail_task is designed to fail. When it does, Airflow sends an email to recipient@example.com, alerting them of the failure.

Read more on Airflow here : Airflow