Hive HPLSQL setup on Google DataProc

Hive HPLSQL setup on Google DataProc

Google Dataproc Hadoop and Hive

Hive Version : 2.3.2 (Version Supports Tez Engine)

More Information on hplsql

http://www.hplsql.org/

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=59690156

As always we are trying to implement some functionality and its not working with a security setup on Hive aswell as the Tez execution engine etc.

Here are the Steps to get it going.

To enable HPLSQL . Cant find any documentation on this for google dataproc anywhere !!

As we have security etc setup on hive you have to edit the hplsql-site.xml to connect to the thriftserver2 for hive and to do this you need to follow the steps below

Find the following File

/usr/lib/hive/lib/hive-hplsql-2.3.2.jar

Copy the file to a folder and unjar it 

jar -xvf hive-hplsql-2.3.2.jar

vi the file hplsql-site.xml (You Will see it when you unjar the above file)

Ammend the hiveconn2 connection and add username and password etc

I have also ammended the config so i can run querys on tez within my specific DB. Its all quite straightforward.

<property>

  <name>hplsql.conn.hive2conn</name>  <value>org.apache.hive.jdbc.HiveDriver;jdbc:hive2://localhost:10000;myuser;mypassword</value>

  <description>HiveServer2 JDBC connection</description>

</property>

<property>

  <name>hplsql.conn.init.hive2conn</name>

  <value>

     set hive.execution.engine=tez;

     use myspecificdb;

  </value>

  <description>Statements for execute after connection to the database</description>

</property>

Remove the jar file currently in folder and run the jar command within the folder you are editing the hplsql-site.xml file

jar cvf hive-hplsql-2.3.2.jar *

Copy the jar file back to the original location

cp hive-hplsql-2.3.2.jar /usr/lib/hive/lib/.

Command to test

/usr/lib/hive/bin# ./hplsql -e "select * from mydb.mytablename"

Power to the future of Big Data @ DELIVERBI

We actually Know Big Data !!

Cheers

Shahed Munir & Krishna Udathu