请帮助我了解我错过了什么。我在带有LIMIT和ORDER BY DESC子句的SELECT上看到一个集群节点的奇怪行为:
SELECT cid FROM test_cf WHERE uid = 0x50236b6de695baa1140004bf ORDER BY tuuid DESC LIMIT 1000;
追踪(仅部分):
…<br> 发送 REQUEST_RESPONSE 消息到 /10.0.25.56 [MessagingService-Outgoing-/10.0.25.56] | 2016-02-29 22:17:25.117000 | 10.0.23.15 | 7862
将 REQUEST_RESPONSE 消息发送到 /10.0.25.56 [MessagingService-Outgoing-/10.0.25.56] | 2016-02-29 22:17:25.136000 | 10.0.25.57 | 6283
将 REQUEST_RESPONSE 消息发送到 /10.0.25.56 [MessagingService-Outgoing-/10.0.25.56] | 2016-02-29 22:17:38.568000 | 10.0.24.51 | 457931
…
10.0.25.56 - 协调节点
10.0.23.15, 10.0.24.51 , 10.0.25.57 - 有数据的节点
协调器比其他节点晚 13 秒从 10.0.24.51 获得响应!为什么这样?我该如何解决?
分区键(uid = 0x50236b6de695baa1140004bf)的行数约为 300。
如果我们使用ORDER BY ASC(我们的聚类顺序)或LIMIT值小于此分区键的行数,一切都很好。
Cassandra (v2.2.5) 集群包含 25 个节点。每个节点拥有大约 400Gb 的数据。
集群放置在 AWS 中。节点均匀分布在 VPC 中的 3 个子网中。节点的实例类型为 c3.4xlarge(16 个 CPU 内核,30GB RAM)。我们使用 EBS 支持的存储(1TB GP SSD)。
键空间 RF 等于 3。
列族:
CREATE TABLE test_cf (
uid blob,
tuuid timeuuid,
cid text,
cuid blob,
PRIMARY KEY (uid, tuuid)
) WITH CLUSTERING ORDER BY (tuuid ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction ={'class':'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
AND compression ={'sstable_compression':'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 86400
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
节点工具 gcstats (10.0.25.57):
Interval (ms) Max GC Elapsed (ms)Total GC Elapsed (ms)Stdev GC Elapsed (ms) GC Reclaimed (MB) Collections Direct Memory Bytes
1208504 368 4559 73 553798792712 58 305691840
节点工具 gcstats (10.0.23.15):
Interval (ms) Max GC Elapsed (ms)Total GC Elapsed (ms)Stdev GC Elapsed (ms) GC Reclaimed (MB) Collections Direct Memory Bytes
1445602 369 3120 57 381929718000 38 277907601
节点工具 gcstats (10.0.24.51):
Interval (ms) Max GC Elapsed (ms)Total GC Elapsed (ms)Stdev GC Elapsed (ms) GC Reclaimed (MB) Collections Direct Memory Bytes
1174966 397 4137 69 1900387479552 45 304448986