我看到 MySql 查询出现无法解释的性能问题。
数据是一个 MySql InnoDB 表，包含 385 万行项目到项目的相关数据。
For item "item_i", another item "also_i" was ordered by "count_i" people.
CREATE TABLE `hl_also2sm` (
`item_i` int(10) unsigned NOT NULL DEFAULT '0',
`also_i` int(10) unsigned NOT NULL DEFAULT '0',
`count_i` int(10) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`item_i`,`also_i`),
KEY `count_i` (`count_i`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
通过获取项目列表、查找相关项目并返回 MySql 查询运行的大致时间来完成示例关联。
// Javascript in NodeJS with MySql, on Debian Linux
var sql = require('./routes/sqlpool'); // connects to DB
var cmd = util.promisify(sql.cmd); // Promise of raw MySql command function
async function inquiry(NumberOfItems){
// generate random list of items to perform correlation against
var rtn = await cmd(`select DISTINCT item_i from hl_also2sm order by RAND() limit ${NumberOfItems}`);
var items_l = rtn.map((h)=>{return h.item_i});
var ts = Date.now();
// get top 50 correlated items
var c = `select also_i,COUNT(*) as cnt,SUM(count_i) as sum from hl_also2sm
where item_i IN (${items_l.join(",")})
AND also_i NOT IN (${items_l.join(",")})
group by also_i
order by cnt DESC,sum DESC limit 50`;
await cmd(c);
var MilliSeconds = Date.now()-ts;
return MilliSeconds;
};
在一系列项目上进行测试
async function inquiries(){
for (items=200;items<3000;items+=200) {
var Data = [];
for (var i=0;i<10;i++) {
Data.push(await inquiry(items));
}
Data.sort();
console.log(`${items} items - min:${Data[0]} max:${Data[9]}`);
}
结果是
200 items - min:315 max:331
400 items - min:1214 max:1235
600 items - min:2669 max:2718
800 items - min:4796 max:4823
1000 items - min:6872 max:7006
1200 items - min:134 max:154
1400 items - min:147 max:169
1600 items - min:162 max:198
1800 items - min:190 max:212
2000 items - min:210 max:244
2200 items - min:237 max:258
2400 items - min:248 max:293
2600 items - min:263 max:302
2800 items - min:292 max:322
这是非常令人费解的。
为什么 2000 个项目比 1000 个项目快 25 倍以上？
1000 项选择 EXPLAIN 是
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
| 1 | SIMPLE | hl_also2sm | index | PRIMARY | count_i | 4 | NULL | 4043135 | Using where; Using index; Using temporary; Using filesort |
2000 年选择 EXPLAIN 是
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
| 1 | SIMPLE | hl_also2sm | range | PRIMARY | PRIMARY | 4 | NULL | 758326 | Using where; Using temporary; Using filesort |
我跑了很多次，每次都产生相似的结果。
是的，我的许多用户通过网页浏览、评论、查看图片或订购对数以千计的商品表现出了兴趣。我想为他们制作一个好的“你可能也喜欢”。
问题总结
select also_i,
COUNT(*) as cnt,
SUM(count_i) as sum
from hl_also2sm
where item_i IN (...) -- Varying the number of IN items
AND also_i NOT IN (...) -- Varying the number of IN items
group by also_i
order by cnt DESC, sum DESC
limit 50
对于IN列表中 <= 1K 的项目，查询使用KEY(count_i)运行速度较慢。
对于列表中 > 1K 的项目IN，查询会进行表扫描并运行得更快。
为什么？？