总体架构图


                        Unix Socket
   MongoDB dump                               CDC Core



       ES dump
                       Unix Socket

接⼊ cdc 必要的条件
1. Source 为各个 dump ⼯作程序（即⼆进制⽂件）          ，由 cdc core 在收到 xpanel 下发任务
   时，根据任务类型启动对应⼆进制⽂件⼯作
2. Source 程序，需要⽀持—workDir, --serverUnixFile 启动参数，如果还有其他参数需要提
   前 提 供 给 CDC 。 例 如 ./redis_dump – workDir=/abc/xxx               --
   serverUnixFile=./redis_dump.sock
3. Source 程序启动后，根据 serverUnixFile ⽂件，启动 unix socket，并监听该⽂件
4. CDC Core 和各 Source 程序之间是通过 unix socket 通信的，具体通信协议如下：
   4.1 socket 包格式
       ---------------------------
       |message_len | message       |
       ---------------------------
   4.2 message 为 json 格式内容，具体如下
       {“type”:msg_type(整数), “data”:””}

      msg_type 类型为
      enum class MessageType : uint32_t {
        M_ReqInitCfg,        -- source 启动时，传⼊参数
        M_ReqDumpJob,       -- dump 任务，添加/删除任务。具体参数
        M_RspDumpJob,       -- dump 任务执⾏情况，
        M_RspRowMsg,        -- source 获取具体数据，上报给 CDC Core
        M_RspStateMsg       -- source 上报 dump 状态给 CDC Core
      }

   具体 MesssageType 类型详细对应 data 中数据
   Ø M_ReqInitCfg，{“local_ip“:””, …}
   Ø M_ReqDumpJob {“url”:””, “pos”:””,…}
   Ø M_RspDumpJob {“error_code”:”0“, “error_msg”:”xxx”}, 如果添加/删除 dump 任务
      成功，在 error_code=0,如果失败⾮ 0 值，error_msg 为具体原因
   Ø M_RspRowMsg 具体格式有 source 程序定义，例如
     第⼀版本：
            {"event_type":"INSERT","db_name":"postgres_$$_test","table_name":"t2","sql":"
      ","isDdl":"0","gtid":"05817685-5ba6-11ef-9aef-0800277f54bd:80762","xid":"5-
      1724487143-415115","original_commit_timestamp":"172448715747
            0540","immediate_commit_timestamp":"1724487158115955","original_server_
       version":"80026","immediate_server_version":"80026","data":[{"id":"98354","name":"
       'XXXX'","age":"22","address":"'GGGGGGGG'","salary":"7289"},{"id":"98355","name":"'
       SSSS'","age":"11","
               address":"'JJJJJJJJ'","salary":"8662"},{"id":"98356","name":"'RRRR'","age":"26","a
       ddress":"'TTTTTTTT'","salary":"8660"},{"id":"98357","name":"'XXXX'","age":"14","addr
       ess":"'OOOOOOOO'","salary":"9269"},{"id":"98358","name":"'SSSS'","age":"2","addr
       ess":"'H
               HHHHHHH'","salary":"137"},{"id":"98359","name":"'YYYY'","age":"30","address":
       "'IIIIIIII'","salary":"4534"},{"id":"98360","name":"'CCCC'","age":"24","address":"'HHHH
       HHHH'","salary":"3669"},{"id":"98361","name":"'SSSS'","age":"31","address":"'CCCCC
       CCC'","s
               alary":"5582"},{"id":"98362","name":"'MMMM'","age":"10","address":"'HHHHHH
       HH'","salary":"1679"},{"id":"98363","name":"'PPPP'","age":"22","address":"'BBBBBBBB
       '","salary":"2476"},{"id":"98364","name":"'JJJJ'","age":"27","address":"'DDDDDDDD'","
       salary":"244
               9"},{"id":"98365","name":"'EEEE'","age":"24","address":"'EEEEEEEE'","salary":"824
       8"},{"id":"98366","name":"'SSSS'","age":"16","address":"'WWWWWWWW'","salary":"
       6453"},{"id":"98367","name":"'HHHH'","age":"5","address":"'DDDDDDDD'","salary":"
       3281"},{"id":"9
               8368","name":"'VVVV'","age":"13","address":"'SSSSSSSS'","salary":"4896"},{"id":"
       98369","name":"'GGGG'","age":"27","address":"'BBBBBBBB'","salary":"6842"},{"id":"9
       8370","name":"'BBBB'","age":"12","address":"'WWWWWWWW'","salary":"6121"},{"id
       ":"98371","name
               ":"'ZZZZ'","age":"32","address":"'WWWWWWWW'","salary":"3023"},{"id":"9837
       2","name":"'RRRR'","age":"35","address":"'OOOOOOOO'","salary":"7784"},{"id":"983
       73","name":"'QQQQ'","age":"26","address":"'QQQQQQQQ'","salary":"3358"},{"id":"9
       8374","name":"'EEEE'",
               "age":"16","address":"'[[[[[[[['","salary":"4183"},{"id":"98375","name":"'PPPP'","ag
       e":"3","address":"'TTTTTTTT'","salary":"3810"},{"id":"98376","name":"'MMMM'","age"
       :"16","address":"'XXXXXXXX'","salary":"6700"},{"id":"98377","name":"'FFFF'","age":"3
       3","
       address":"'JJJJJJJJ'","salary":"2293"},{"id":"98378","name":"'HHHH'","age":"7","addres
       s":"'WWWWWWWW'","salary":"2096"}],"old":""}
       第⼆版本：
              改成字符串流⽅式，减少 json serialize/deserialize 系统开销

字段名              格式                                               说明
db_name          db 名⻓度(⼩端) + db 名，如果没有给⻓                         库名
                 度为 0
tb_name          tb 名⻓度(⼩端) + tb 名                                表名
gtid
sql
column_num       uint32                                           列的数量
Column_names     多个，每个都是字符串⻓度 + 字符串内                              列名
                容
column_state    uint32， 如果不需要直接写 0                   列属性
row_num         uint32                               ⾏数量
data            [len + str, len+str, len + str, …]   具      体     数     据   ，
                [len + str, len+str, len + str, …]   insert/update/delete
                [len + str, len+str, len + str, …]
old             [len + str, len+str, len + str, …]   如果为 update 类型，为更新
                [len + str, len+str, len + str, …]   前值
                [len + str, len+str, len + str, …]


      Ø   M_RspStateMsg，source 程序定义，⼀般上报 dump 状态等。








1. 类型：使用`byte(protocol.MRsprowmsg)`作为单字节类型标识
2. 多少行：使用`uint32`类型的`rowCount`表示行数
3. gtid：按`len+str`格式发送`mongoData.Id`
4. binlog_file：按`len+str`格式发送`[]byte{0}`（二进制0，不是字符串"0"）
5. 数据类型：13(insert)/15(delete)/14(update)
6. db_name：按`len+str`格式发送数据库名
7. table_name：按`len+str`格式发送表名
8. gtid：再次按`len+str`格式发送`mongoData.Id`
9. sql：按`len+str`格式发送`[]byte{0}`
10. 列数量：发送`uint32`类型的列数（固定为3）
11. 列名：按`len+str`格式顺序发送所有列名
12. 列属性：为每列发送`uint32(0)`的属性值
13. 行数：发送`uint32`类型的行数（固定为1）
14. 具体行内容：按格式发送每行的实际数据
