redis - How can I write a tuple in to redis as well as cassandra using trident topology

Question

I am writing a Trident topology to process stream of data from Kafka and feed in to Redis and Cassandra. I am able to write the data in to Cassandra. Now I would like to write the same data in to Redis.

Is there a way to duplicate the tuples and branch it in to 2 flow where one goes in to Redis and another goes in to Cassandra?

Answer 1

对于 Trident，您可以像这样使用 smth：

TridentTopology topology = new TridentTopology();
Stream stream = topology.newStream("MySpout", spout);
stream.partitionPersist(...); // to Redis
stream.partitionPersist(...); // to Cassandra

所以它会将数据从您的流中并行保存到两个数据库中。

但是，我也认为是否应该在单个拓扑中完成这种并行的事情，或者如果从同一主题读取两个不同的拓扑是一个更好的主意。想象一下 Cassandra 集群出现故障。如果有两种拓扑，您仍然可以继续将数据保存到 Redis。但是，如果只有一个拓扑，那么每个未能进入 Cassandra 的元组很可能会导致 FailedException 来触发重放，并且每个后续重放元组都将不必要地再次将元组保存到 Redis。