一:

下载datax github地址:https://github.com/alibaba/DataX
在这里插入图片描述

电脑安装python环境:
https://blog.csdn.net/qq_21552167/article/details/115113260?ops_request_misc=%257B%2522request%255Fid%2522%253A%2522168238800916800186595888%2522%252C%2522scm%2522%253A%252220140713.130102334…%2522%257D&request_id=168238800916800186595888&biz_id=0&utm_medium=distribute.pc_search_result.none-task-blog-2allbaidu_landing_v2~default-2-115113260-null-null.142v86wechat,239v2insert_chatgpt&utm_term=win10%E5%AE%89%E8%A3%85pathon&spm=1018.2226.3001.4187

需要一个json文本配置如下:

{
    "job": {
        "setting": {
            "speed": {
                 "channel": 3
            },
            "errorLimit": {
                "record": 0,
                "percentage": 0.02
            }
        },
        "content": [
            {
                "reader": {
                    "name": "mysqlreader",
                    "parameter": {
                        "username": "mysql账号",
                        "password": "密码",
                        "column": [
                            "查询字段",
							"nickname"
                        ],
                        "connection": [
                            {
                                "querySql": [
                                  "mysql查询语句"
                                ],
                                "jdbcUrl": [
     "mysql链接"
                                ]
                            }
                        ]
                    }
                },
               "writer": {
                    "name": "clickhousewriter",
                    "parameter": {
                        "username": "clickhouse地址",
                        "password": "密码",
                        "column": ["需要同步的字段", "nick_name", "avatar", "phone", "province", "city", "ip_addr", "gender", "create_time", "dept_id"],
                        "connection": [
                            {
                                "jdbcUrl": "jdbc:clickhouse://链接",
                                "table": ["member_property"]
                            }
                        ],
                        "preSql": [],
                        "postSql": [],

                        "batchSize": 65536,
                        "batchByteSize": 134217728,
                        "dryRun": false,
                        "writeMode": "insert"
                    }
                }
            }
        ]
    }
}

解压datax,进入bin目录打开cmd,运行json配置文件
执行命令:python datax.py C:\xxx\xxx\xxx\xxx.json
在这里插入图片描述

至此

Logo

魔乐社区(Modelers.cn) 是一个中立、公益的人工智能社区,提供人工智能工具、模型、数据的托管、展示与应用协同服务,为人工智能开发及爱好者搭建开放的学习交流平台。社区通过理事会方式运作,由全产业链共同建设、共同运营、共同享有,推动国产AI生态繁荣发展。

更多推荐