Sunday, 24 November 2019


GCP Google BigQuery Cost Reporting - Control your costs

Setting up the BigQuery audit logs export within GCS Monitor Your BigQuery Costs in GCP Google Cloud Platform Instructions G...

Friday, 22 November 2019


GCS HADOOP/Hive to Google Big Query Migrate Data Quickly ORC Format

GCS Dataproc Hadoop(Hive) to GCS Big Query (BQ) Quick Transfer ORC File Formats Software Used Google Dataproc - Google&#...

Friday, 1 November 2019


Apache Airflow Check Previous Run Status for a DAG

Apache Airflow Check Previous Run Status for a DAG The Problem We encountered a scenario where if any previous DAG run fails the n...

Thursday, 26 September 2019


Yarn - Long Running Jobs Alerts for Hive TEZ , Application Containers

Hadoop / Hive / Yarn / Applications Long running Jobs alert script Currently at a client where their are some long running jobs...

Thursday, 29 August 2019


HADOOP Balancing the cluster

Hadoop Balancing the Cluster Balancing an hadoop cluster i feel is very important and the nodes should have a deviation of no more tha...

Wednesday, 6 March 2019


Apache Airflow 1.10.2 Maintenance Dags

Apache Airflow 1.10 / 1.10.2 + Maintenance Dags (Logs) We have noticed on Airflow 1.10.+ , The maintenance dags to manage datab...
About Authors

Shahed Munir

Krishna Udathu

Shahed and Krishna are Oracle / Big Data Experts