Shell command executes hive scripts (hive interaction)

Hive execution method

There are three ways to execute hql commands in Hive:

1. Direct execution of CLI mode
2. Call hive -e as a string to execute (-S to enable silence, remove "OK", "Time taken")
3. As a separate file, execute by calling hive –f or hive –i through shell

Method 1

Type“hive”，start uphiveofcliInteractive mode。SetAll environment settings parameters can be viewed，And can be reset。Other commands such as，
    Use database        Select a library
    quit/exit   quitHiveofInteractive mode 
    set –v  showHive中of所有变量
    set &lt;key&gt;=&lt;value&gt;       Set parameters
    Execute localshell ：!&lt;cmd&gt;       Interactive mode下可执行shellOrder，For example（ChecklinuxFile list in the root directory："!ls -l /;"）
    操作云Order：dfs &lt; command&gt;        Interactive mode下直接操作hadoopOrder如 dfs fs –ls
    HqlStatement       Execute a query and output to standard output
    add [FILE|JAR|ARCHIVE] &lt;value&gt; [&lt;value&gt;]*       Add a file to the resource list
    list FILE       列出所有已经添加of资源

Method 2

HqlAs a string inshellExecute in script，like
    hive -e "use ${database};select * from tb"
 The query results can be exported directly to the local file (the default delimiter is \t):
     hive -e "select * from tb" &gt;

If you need to view the execution steps, add it before the command

set –x

In addition, in shell scripts, there are two ways to define strings:

1) Directly define the string object: sql=”String”

2) Through the command definition: sql=$(cat <<endtag string endtag) method, the shell script for executing the hql command is as follows:

####### execute hive ######
sql=$(cat <<!EOF

USE pmp;
set =queue3;

drop table if exists people_targeted_delivery;
create table people_targeted_delivery
( special_tag_id int,
  cnt bigint
);

INSERT OVERWRITE LOCAL DIRECTORY '$cur_path/people_targeted_delivery'
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' 
select special_tag_id,count(1) 
from t_pmp_special_user_tags
group by special_tag_id;

!EOF)
############  execute begin   ###########
echo $sql
$HIVE_HOME/bin/hive -e "$sql"

exitCode=$?
if [ $exitCode -ne 0 ];then
         echo "[ERROR] hive execute failed!"
         exit $exitCode
fi

Method Three

Save the hql statement as a separate file, with no limit on the suffix name, and you can use .q or .hql as the identifier:
A. This file is executed in cli mode using the source command, such as: source ./
B, execute commands in the shell, such as: hive -f

Hive specifies the pre-execution file command "hive –i" (or initialization file)

Order：hive -i 
existhivestart upcliBefore，Execute the specified file first（）中的Order。
That is to say，允许用户existclistart up时预先执行一个指定document，for example，There are some common environment parameter settings，频繁执行的Order，可以Add toexist初始化document中，for example，
    Certain parameter settings
        set =queue3;
        SET =14;
    Add toudfdocument
        add JAR ./;
    set upHiveLog level 
        hive -hiveconf =INFO;

This is the article about shell command execution hive script (hive interaction). For more relevant shell command execution hive content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!