Sqoop Consume Teiid
Downloading and installing Sqoop
Sqoop can import/export data from Database, like mysql, sql server, etc.
Installing Sqoop
Acturally, the installation is quite easy.
Step.1 Download
You can get Sqoop from http://sqoop.apache.org. Compatible Sqoop version is needed to match you Hadoop version.
$ wget http://apache.fayea.com/sqoop/1.4.6/sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
$ tar -xvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
Step.2 Configure
Just set environment variable, that’s all.
Edit /etc/profile, root user is needed.
export SQOOP_HOME=/home/userName/sqoop-1.4.6.bin__hadoop-2.0.4-alpha
export PATH = $SQOOP_HOME/bin:$PATH
After edit
$ source /etc/profile
Sqoop consume Teiid
This quickstart demonstrates how to import data from teiid to HDFS by Sqoop.
- VDB: Portfolio
System requirements
- Java 1.7+
- Teiid Server
- Hadoop
- Sqoop
NOTE: This example relies upon the vdb-datafederation example and that it needs to be deployed prior to running this example. Therefore, read the vdb-datafederation’s README.md and follow its directions before continuing.
Sqoop Demonstrations
1. Copy teiid-jdbc to $SQOOP_HOME/lib
You need download Teiid JDBC Driver from http://teiid.jboss.org/downloads/ and copy it to $SQOOP_HOME/lib
cp teiid-9.0.0.Final-jdbc.jar $SQOOP_HOME/lib
2. Import data to HDFS
$ sqoop import --connect jdbc:teiid:Portfolio@mm://127.0.0.1:31000 --driver org.teiid.jdbc.TeiidDriver --username odataUser --password password1! --table product
3. Check HDFS
Now, you can verify that whether the data from VDB is imported to HDFS.
- Verify HDFS
$ hadoop dfs -ls /user/username/product
- Verify Data
$ hadoop dfs -get /user/username/product ./result
$ cat result/part-m-*