1

我已经设法在一个 ec2 实例上部署了 ksqldb 服务器，并使用 msk kafka 作为 kafka 服务器，docker-compose并且它的工作正常（能够使用 python KSQL API 客户端连接并访问主题和 msk 中的流）。

version: '2'

services:
  ksqldb-server:
    image: confluentinc/ksqldb-server:0.14.0
    hostname: ksqldb-server
    container_name: ksqldb-server
    ports:
      - "8088:8088"
    environment:
      KSQL_LISTENERS: http://0.0.0.0:8088
      KSQL_BOOTSTRAP_SERVERS: b-1.ksql-kafka-tesing.boqxnyhyk.c2.kafka.ap-south-1.amazonaws.com:9092,b-2.ksql-kafka-tesing.byuuttoqxnk.c2.kafka.ap-south-1.amazonaws.com:9092,b-3.ksql-kafka-tesing.boqxyytuytutnk.c2.kafka.ap-south-1.amazonaws.com:9092
      KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
      KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true"

但是，当我使用正确的集群和任务定义在 AWS FARGET 上部署下面的 docker 映像时，它会在 cloudwatch 日志上提供以下错误日志

2021-01-19T21:16:50.425+05:30   ssl.truststore.type = JKS

2021-01-19T21:16:50.425+05:30   (io.confluent.ksql.util.KsqlConfig:354)

2021-01-19T21:16:53.417+05:30   [2021-01-19 15:46:53,413] ERROR Failed to start KSQL (io.confluent.ksql.rest.server.KsqlServerMain:68)

2021-01-19T21:16:53.417+05:30   java.lang.RuntimeException: Failed to get Kafka cluster information

2021-01-19T21:16:53.417+05:30   at io.confluent.ksql.services.KafkaClusterUtil.getKafkaClusterId(KafkaClusterUtil.java:107)

2021-01-19T21:16:53.417+05:30   at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:605)

2021-01-19T21:16:53.417+05:30   at io.confluent.ksql.rest.server.KsqlServerMain.createExecutable(KsqlServerMain.java:161)

2021-01-19T21:16:53.417+05:30   at io.confluent.ksql.rest.server.KsqlServerMain.main(KsqlServerMain.java:61)

2021-01-19T21:16:53.417+05:30   Caused by: org.apache.kafka.common.config.ConfigException: Missing required configuration "bootstrap.servers" which has no default value.

2021-01-19T21:16:53.417+05:30   at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:478)

2021-01-19T21:16:53.417+05:30   at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:468)

2021-01-19T21:16:53.417+05:30   at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:108)

2021-01-19T21:16:53.417+05:30   at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:142)

2021-01-19T21:16:53.417+05:30   at org.apache.kafka.clients.admin.AdminClientConfig.<init>(AdminClientConfig.java:233)

2021-01-19T21:16:53.417+05:30   at org.apache.kafka.clients.admin.Admin.create(Admin.java:74)

2021-01-19T21:16:53.417+05:30   at org.apache.kafka.streams.processor.internals.DefaultKafkaClientSupplier.getAdmin(DefaultKafkaClientSupplier.java:41)

2021-01-19T21:16:53.417+05:30   at io.confluent.ksql.services.ServiceContextFactory.lambda$create$1(ServiceContextFactory.java:58)

2021-01-19T21:16:53.417+05:30   at com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:164)

2021-01-19T21:16:53.417+05:30   at io.confluent.ksql.services.DefaultServiceContext$MemoizedSupplier.get(DefaultServiceContext.java:176)

2021-01-19T21:16:53.417+05:30   at io.confluent.ksql.services.DefaultServiceContext.getAdminClient(DefaultServiceContext.java:116)

2021-01-19T21:16:53.417+05:30   at io.confluent.ksql.services.LazyServiceContext.getAdminClient(LazyServiceContext.java:33)

2021-01-19T21:16:53.417+05:30   at io.confluent.ksql.services.KafkaClusterUtil.getKafkaClusterId(KafkaClusterUtil.java:102)

我还在farget容器中启用了端口8083、8088、80，同时在容器中添加了docker镜像。

无法找出为什么相同的 docker compose 映像没有在 farget 上运行但在 ec2 机器上运行而没有错误。

有人可以帮我吗？我对 ksql 和 aws farget 也完全陌生。提前致谢

PS：我已按照本教程在 Farget 上部署 Docker 映像 使用 ECR 和 ECS 在Farget 上部署 Docker 映像

4

1 回答 1

0

docker-compose 中的那些环境变量没有从容器中的图像中读取。在farget部署中添加容器时必须特别提到环境变量和值。

完美地工作。

于 2021-01-20T11:25:54.463 回答