mypipe部署

git clone https://github.com/mardambey/mypipe.git vi mypipe-runner/src/main/scala/mypipe/runner/PipeRunner.scala 55行 //shutdown() 注释 使用的主配置文件 mypipe-runner/src/main/resources/application.conf ，修改kafka和zookper地址 mypipe-runner/target/scala-2.11/classes/application.conf (编译后)

数据库 过滤表 配置文件 mypipe-api/src/main/resources/reference.conf mypipe-api/target/scala-2.11/classes/reference.conf(编译后) quit-on-empty-mutation-commit-failure = false flush-interval-seconds = 1 include-event-condition = “”” (db == “newtable” && table == “lip”) “”“

创建用于订阅数据库的账户： GRANT REPLICATION SLAVE, REPLICATION CLIENT ON . TO ‘mypipe’@’%’ IDENTIFIED BY ‘mypipe’; GRANT ALL PRIVILEGES ON . TO ‘mypipe’@’%’; GRANT RELOAD ON . TO ‘mypipe’@’%’;

在用户目录下新增文件 用阿里云的源 vi .sbt/repositories [repositories] local osc: http://maven.aliyun.com/nexus/content/groups/public/ typesafe: http://repo.typesafe.com/typesafe/ivy-releases/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/artifact.[ext], bootOnly sonatype-oss-releases maven-central sonatype-oss-snapshots

修改编译前文件后 编译 ./sbt clean package 启动命令 ./sbt “project runner” “runMain mypipe.runner.PipeRunner”

kafka配置

kafka的配置文件编辑 vi consumer.properties zookeeper.connect=172.18.0.1:2181 vi server.properties zookeeper.connect=172.18.0.1:2181

启动 kafka bin/kafka-server-start.sh config/server.properties 或者 bin/kafka-server-start.sh -daemon config/server.properties 后台启动

mypipe后台方法 crtl + z jobs bg n