GCP Google BigQuery Cost Reporting - Control your costs

GCP Google BigQuery Cost Reporting - Control your costs

Setting up the BigQuery audit logs export within GCS Monitor Your BigQuery Costs in GCP Google Cloud Platform Instructions G...

GCS HADOOP/Hive to Google Big Query Migrate Data Quickly ORC Format

GCS HADOOP/Hive to Google Big Query Migrate Data Quickly ORC Format

GCS Dataproc Hadoop(Hive) to GCS Big Query (BQ) Quick Transfer ORC File Formats Software Used Google Dataproc - Google&#...

Apache Airflow Check Previous Run Status for a DAG

Apache Airflow Check Previous Run Status for a DAG

Apache Airflow Check Previous Run Status for a DAG The Problem We encountered a scenario where if any previous DAG run fails the n...

Yarn - Long Running Jobs Alerts for Hive TEZ , Application Containers

Yarn - Long Running Jobs Alerts for Hive TEZ , Application Containers

Hadoop / Hive / Yarn / Applications Long running Jobs alert script Currently at a client where their are some long running jobs...

HADOOP Balancing the cluster

HADOOP Balancing the cluster

Hadoop Balancing the Cluster Balancing an hadoop cluster i feel is very important and the nodes should have a deviation of no more tha...

Apache Airflow 1.10.2 Maintenance Dags

Apache Airflow 1.10.2 Maintenance Dags

Apache Airflow 1.10 / 1.10.2 + Maintenance Dags (Logs) We have noticed on Airflow 1.10.+ , The maintenance dags to manage datab...