Airflow dags not getting refreshed/updating. How to do it manually?

Apache Airflow

After creating a DAG in Airflow, we anticipate its immediate refresh. However, even after refreshing the Airflow UI, the latest changes or newly added DAGs may not be immediately visible. Typically, Airflow refreshes DAGs every 30 seconds, thus we expect to observe the latest modifications within this timeframe. The common scenario where the dags are not getting refreshed may be due to some error in the dag python code. Another scenario , is if there are duplicate id ( same dag name for two files, can occour when you copy a code to create a new one ) .  To get the dags refreshed manually you can do the following. Log on to the server where your airflow is installed( If its a virtual environment , go to that path )  and perform the following steps. 

source workspace/virtualenv/bin/activate
export AIRFLOW_HOME
export AIRFLOW_HOME=~/workspace/airflow
cd $AIRFLOW_HOME
airflow dags list

The above will list all the dags in that server path. 

No if you want to get the error list as well then you have to give -v  in the last command above. To also retrieve a list of errors, you need to append -v to the final command mentioned above

airflow dags list -v

 

Author: user

Leave a Reply