Translate

Sunday, 24 November 2019

Shahed

GCP Google BigQuery Cost Reporting - Control your costs

Setting up the BigQuery audit logs export within GCS Monitor Your BigQuery Costs in GCP Google Cloud Platform Instructions G...

Friday, 22 November 2019

Shahed

GCS HADOOP/Hive to Google Big Query Migrate Data Quickly ORC Format

GCS Dataproc Hadoop(Hive) to GCS Big Query (BQ) Quick Transfer ORC File Formats Software Used Google Dataproc - Google&#...

Friday, 1 November 2019

Krishna

Apache Airflow Check Previous Run Status for a DAG

Apache Airflow Check Previous Run Status for a DAG The Problem We encountered a scenario where if any previous DAG run fails the n...

Thursday, 26 September 2019

Shahed

Yarn - Long Running Jobs Alerts for Hive TEZ , Application Containers

Hadoop / Hive / Yarn / Applications Long running Jobs alert script Currently at a client where their are some long running jobs...

Thursday, 29 August 2019

Shahed

HADOOP Balancing the cluster

Hadoop Balancing the Cluster Balancing an hadoop cluster i feel is very important and the nodes should have a deviation of no more tha...

Wednesday, 6 March 2019

Shahed

Apache Airflow 1.10.2 Maintenance Dags

Apache Airflow 1.10 / 1.10.2 + Maintenance Dags (Logs) We have noticed on Airflow 1.10.+ , The maintenance dags to manage datab...

Friday, 16 November 2018

Shahed
Google Drive API Google Sheets Extract to CSV with SERVICE KEY.JSON Recent Client had some Google Sheets stored on their Goo...

Thursday, 8 November 2018

Shahed

Hive HPLSQL setup on Google DataProc

Hive HPLSQL setup on Google DataProc Google Dataproc Hadoop and Hive Hive Version : 2.3.2 (Version Supports Tez Engine) ...

Tuesday, 6 November 2018

Shahed

PowerBI Custom Direct Query ODBC Connector

PowerBI Custom Direct Query ODBC Connector At DELIVERBI we have various challenges presented to us and a client of ours us...

Tuesday, 30 October 2018

Shahed

Hive Tez vs Presto 12 Billion Records and 250 Columns Performance

Hive Tez vs Presto as a query Engine & Performance 12 BILLION ROWS AND a 250 Column Table with Dimensions.  Here at DE...
About Authors

Shahed Munir

Krishna Udathu

Shahed and Krishna are Oracle / Big Data Experts