hadoop 集群安装sqoop数据传输工具

sqoop工具   大数据  

一、环境

  • sqoop是Apache顶级项目,主要用来在Hadoop和关系数据库中传递数据。通过sqoop,我们可以方便的将数据从关系数据库导入到HDFS,或者将数据从HDFS导出到关系数据库。
  • CentOS 6.6
  • sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz

安装

  • 解压
tar zxf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz -C /home/hadoop/src/  
chown hadoop.hadoop /home/hadoop -R  
cd /home/hadoop/src/  
ln -s sqoop-1.4.6.bin__hadoop-2.0.4-alpha sqoop  
  • 配置环境变量
cd src/sqoop/conf && cp sqoop-env-template.sh sqoop-env.sh  
vim sqoop-env.sh  
-------------------------------------------------------
export HADOOP_COMMON_HOME=/home/hadoop/src/hadoop-2.7.2/  
export HADOOP_MAPRED_HOME=/home/hadoop/src/hadoop-2.7.2/  
export HBASE_HOME=/home/hadoop/src/hbase-1.2.2  
export HIVE_HOME=/home/hadoop/src/apache-hive-1.2.1-bin  
export HCAT_HOME=/home/hadoop/src/apache-hive-1.2.1-bin  
export ZOOCFGDIR=/home/hadoop/src/zookeeper-3.4.8  
export ZOOKEEPER_HOME=/home/hadoop/src/zookeeper-3.4.8/  
export ZOOKEEPER_HOME=/home/hadoop/src/zookeeper-3.4.8/  
--------------------------------------------------------

测试sqoop

  • 执行 sqoop version
[hadoop@master bin]$ ./sqoop version
16/09/09 12:10:53 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6  
Sqoop 1.4.6  
git commit id c0c5a81723759fa575844a0a1eae8f510fa32c25  
Compiled by root on Mon Apr 27 14:38:36 CST 2015  
  • 如果出现如下报错
Warning: /home/hadoop/src/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../accumulo does not exist! Accumulo imports will fail.  
  • 修改configure-sqoop,注释以下几行
#if [ ! -d "${ACCUMULO_HOME}" ]; then
#  echo "Warning: $ACCUMULO_HOME does not exist! Accumulo imports will fail."
#  echo 'Please set $ACCUMULO_HOME to the root of your Accumulo installation.'
#fi