datax保姆级同步mysql库至clickhouse
下载datax github地址:https://github.com/alibaba/DataX。
·
一:
下载datax github地址:https://github.com/alibaba/DataX
二
三
需要一个json文本配置如下:
{
"job": {
"setting": {
"speed": {
"channel": 3
},
"errorLimit": {
"record": 0,
"percentage": 0.02
}
},
"content": [
{
"reader": {
"name": "mysqlreader",
"parameter": {
"username": "mysql账号",
"password": "密码",
"column": [
"查询字段",
"nickname"
],
"connection": [
{
"querySql": [
"mysql查询语句"
],
"jdbcUrl": [
"mysql链接"
]
}
]
}
},
"writer": {
"name": "clickhousewriter",
"parameter": {
"username": "clickhouse地址",
"password": "密码",
"column": ["需要同步的字段", "nick_name", "avatar", "phone", "province", "city", "ip_addr", "gender", "create_time", "dept_id"],
"connection": [
{
"jdbcUrl": "jdbc:clickhouse://链接",
"table": ["member_property"]
}
],
"preSql": [],
"postSql": [],
"batchSize": 65536,
"batchByteSize": 134217728,
"dryRun": false,
"writeMode": "insert"
}
}
}
]
}
}
四
解压datax,进入bin目录打开cmd,运行json配置文件
执行命令:python datax.py C:\xxx\xxx\xxx\xxx.json
至此

魔乐社区(Modelers.cn) 是一个中立、公益的人工智能社区,提供人工智能工具、模型、数据的托管、展示与应用协同服务,为人工智能开发及爱好者搭建开放的学习交流平台。社区通过理事会方式运作,由全产业链共同建设、共同运营、共同享有,推动国产AI生态繁荣发展。
更多推荐
所有评论(0)