DATAx安装

准备工作

Linux

  • JDK(1.8以上,推荐1.8)

  • Python(推荐Python2.6.X),我使用的是 2.7.10(启动命令:python2.7,由于同时安装了python3)

  • 原因:因为后面执行datax.py的时候,里面的python的print会执行不了,导致运行不成功,会提示你print语法要加括号,python2中加不加都行 python3中必须要加,否则报语法错

  • Apache Maven 3.x (Compile DataX)

下载:http://datax-opensource.oss-cn-hangzhou.aliyuncs.com/datax.tar.gz

下载后直接tar -zxvf datax.tar.gz 解压即可。

配置

第一步、创建创业的配置文件(json格式)

可以通过命令查看配置模板: python datax.py -r -w
gBSroq.md.jpg

以mysql为例

根据模板配置json文件(myjob.json)如下:

{
    "job": {
        "content": [
            {
                "reader": {
                    "name": "mysqlreader", 
                    "parameter": {
                        "column": [
							"myid",
                            "name",
				"mypart"
						], 
                        "connection": [
                            {
                                "jdbcUrl": ["jdbc:mysql://127.0.0.1:3306/test"], 
                                "table": ["b"]
                            }
                        ], 
                        "password": "*****", 
                        "username": "root"
                    }
                }, 
                "writer": {
                    "name": "mysqlwriter", 
                    "parameter": {
                        "column": [
                        "myID",
                        "name",
			"part"
						], 
                        "connection": [
                            {
                                "jdbcUrl": "jdbc:mysql://127.0.0.1:3306/test", 
                                "table": ["b_copy"]
                            }
                        ], 
                        "password": "123456", 
                        "username": "root"
                    }
                }
            }
        ], 
        "setting": {
            "speed": {
                "channel": "1"
            }
        }
    }
}

datax 使用mysql转oracle脚本

{
    "job": {
        "setting": {
            "speed": {
                "channel": 5
            }
        },
        "content": [
            {
                "reader": {
                    "name": "mysqlreader",
                    "parameter": {
                        "username": "system",
                        "password": "****",
                        "column": ["*"],
                        "connection": [
                           {
                              "table": ["datax_test"],
                              "jdbcUrl": ["jdbc:mysql://ip:3306/vision"]
                           }
                        ]
                    }
                },
                "writer": {
                    "name": "oraclewriter",
                    "parameter": {
                        "username": "vision",
                        "password": "***",
                        "column": ["*"],
                        "connection": [
                            {
                               "jdbcUrl": "jdbc:oracle:thin:@ip:1521:bol",
                               "table": ["datax_test"]
                            }
                        ]
                      }
                    }
                }
        ]
    }
}

启动数据同步

命令:python /data/soft/datax/bin/datax.py xxx.json
程序会执行脚本进行数据同步,最后显示同步结果
gB9NrQ.md.png

Q.E.D.

知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议