Thursday, 26 September 2019


Yarn - Long Running Jobs Alerts for Hive TEZ , Application Containers

Hadoop / Hive / Yarn / Applications Long running Jobs alert script

Currently at a client where their are some long running jobs in hive that hog the resources , You can see the logs in the application history manager or yarn etc. But we needed an alert mechanism. So we wrote this python script to detect any over-runs , it will spit out a file with the relevant records that are over running . The script can be run with any method really.Airflow . ODI , Cron Job just add a step to send a mail using xmail or what ever suits you.

We use Airflow and if the file produced contains any records we send an email to the Admins to see whats going on , Alternatively you could always issue a yarn application kill command.

The script is within our Github Python.

Over and Out

Shahed and Krishna


Write comments
27 January 2022 at 16:58 delete

Sands Casino: 50 FREE Spins No Deposit Code
Get 50 Free Spins No Deposit Code on 바카라사이트 a $500 Bonus! Play 메리트카지노 Slots at Sands Casino. Play over 2000 Slots, Table Games 샌즈카지노 and Live Dealer Games!


About Authors

Shahed Munir

Krishna Udathu

Shahed and Krishna are Oracle / Big Data Experts