2

我正在设计一个非常专业的使用模式的桌子。该表将以有限的流量连续记录 - 每秒约 25 条记录，然后我每天晚上运行一个大查询以提取大量数据。

我的表创建脚本目前如下所示：

SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

IF NOT EXISTS (select * from sysobjects where name='records' and xtype='U')
CREATE TABLE [dbo].[records](
    [TripID] varchar(255) NOT NULL,
    [RecordTimeUTC] datetime2(0) NOT NULL,
    [TimeOfDaySeconds] [int] NOT NULL,
    [T0Latitude] [float] NOT NULL,
    [T0Longitude] [float] NOT NULL,
    [T1Latitude] [float] NULL,
    [T1Longitude] [float] NULL,
    [T2Latitude] [float] NULL,
    [T2Longitude] [float] NULL,
    [T3Latitude] [float] NULL,
    [T3Longitude] [float] NULL,
    [T4Latitude] [float] NULL,
    [T4Longitude] [float] NULL,
    [T5Latitude] [float] NULL,
    [T5Longitude] [float] NULL,
    [VehicleID] [int] NULL,
    [ID] [int] IDENTITY(1,1) NOT NULL PRIMARY KEY
) ON [PRIMARY]
GO

IF NOT EXISTS (select * from sys.indexes where name='TripIDRecordTimeIndex' and object_id = OBJECT_ID('dbo.records'))
    CREATE INDEX TripIDRecordTimeIndex ON records (TripID, RecordTimeUTC desc)
GO

IF NOT EXISTS (select * from sys.indexes where name='TripIDIndex' and object_id = OBJECT_ID('dbo.records'))
    CREATE INDEX TripIDIndex ON records (TripID)
GO

IF NOT EXISTS (select * from sys.indexes where name='RecordTimeUTCIndex' and object_id = OBJECT_ID('dbo.records'))
    CREATE INDEX RecordTimeUTCIndex ON records (RecordTimeUTC desc)
GO

IF NOT EXISTS (select  * from sys.objects where name like 'UniqueConstraint2' and parent_object_id = OBJECT_ID('dbo.records'))
    ALTER TABLE [dbo].[records] ADD CONSTRAINT UniqueConstraint2 UNIQUE(VehicleID, RecordTimeUTC desc);
GO

IF NOT EXISTS (select * from sys.indexes where name='VehicleIDIndex' and object_id = OBJECT_ID('dbo.records'))
    CREATE INDEX VehicleIDIndex ON records (VehicleID)
GO

我的表中目前有大约 6000 万条记录，大小不到 50 GB。

提取数据的查询非常耗时。目前需要一个多小时。我不确定是我的表设计还是查询设计是根本原因（尽管可能两者兼而有之）。

我需要为我指定的一组 TripID 中的每个 TripID 提取最新的 X 个项目。大约有 10k 个不同的 ID，我通常想查询其中的一半。X 在它们之间也有所不同，所以我目前最好的查询方法是生成一个看起来有点像这样的脚本：

SELECT rs.* FROM (SELECT *, ROW_NUMBER() over (Partition BY TripID ORDER BY RecordTimeUTC DESC ) AS Rank FROM records where TripID in (20141000,20441000,30011022,30011021,30011008,30012029,30012028,30012027,30011007,30011019,30011018,30012026,30012025,30012024,30011017,30011016,30012023,30012022,30011015,30011014,30012021,30012020,30011013,30011012,30013000,30013001,30013019,30013009,30011011,30011010,30011009,30013008,30013007,30012010,30012009,30013005,30013004,30013003,30012014,30012019,30013021,30013020,30011006,30011004,30012018,30012017,30012016,30013006,30011003,30011002,30012015,30012013,30013013,30013002,30011001,30011000,30011020,30012012,30012011,30011005,30011030,30012001,30012008,30012007,30011029,30011028,30012006,30012005,30011031,30011027,30012004,30012003,30011026,30011025,30011024,30012002,30012000,30012031,30011023,30012030,30015005,30016006,30016013,30016012,30014020,30014019,30014018,30016011,30016010,30014017,30014016,30016009,30016008,30014015,30014013,30014012,30016005,30016004,30016003,30014010,30014009,30016002,30016001,30014008,30014007,30016000,30016007,30014006,30014005,30014004,30014003,30014002,30014001,30014000,30014023,30014014,30015012,30015004,30015003,30013018,30013017,30015002,30015001,30013016,30013015,30013014,30015000,30015013,30015011,30013012,30013011,30015010,30015009,30013010,30014011,30015008,30015007,30014022,30014021,30015006,33651001,33661006)) rs WHERE Rank <= 690
 UNION 
SELECT rs.* FROM (SELECT *, ROW_NUMBER() over (Partition BY TripID ORDER BY RecordTimeUTC DESC ) AS Rank FROM records where TripID in (20431003,20431002,20431001,20432003,20432002,20432001,30221001,33861002,33861003)) rs WHERE Rank <= 855
 UNION 
SELECT rs.* FROM (SELECT *, ROW_NUMBER() over (Partition BY TripID ORDER BY RecordTimeUTC DESC ) AS Rank FROM records where TripID in (20171029,20171030,20002002,26122001)) rs WHERE Rank <= 45
 UNION 
...

（上面的查询在第一个列表中返回每个行程的 690 个实例，在第二个列表中返回 855 个实例，在第三个中返回 45 个，依此类推。查询比这大得多 - 这只是它的一个片段。总共我提取 10-15 百万行）

如前所述，我的表现很糟糕。是云的事吗？它是设计的东西吗？我应该使用聚集索引吗？（对 TripID 进行了尝试，但情况更糟）。我可以以某种方式改进我的查询吗？例如，为每个 ID 提取相同数量的实例，然后过滤？

我注意到我有几个额外的索引可能不会在我的查询中使用。我只是尝试添加更多，因为插入性能不是问题。计划是在我的查询中使用 TripIDRecordTimeIndex。

即使将 Azure SQL 中的数据计划扩展到 S7（800 DPU），我也无法快速运行。任何反馈表示赞赏。

编辑：我最近将 TripID 从 int 更改为 varchar(255) - 这会破坏我的表现吗？

Edit2：执行计划： 执行计划截图

下载完整执行计划的链接

Edit3：发现在我查询的 TripID 周围添加引号 ('') 可以极大地提高性能！

Edit4：我添加了 TheGameiswar 提出的索引 - 区别在于白天和黑夜！谢谢！附上新的执行计划。

新的执行计划

4

1 回答 1

2 
SELECT rs.* FROM (SELECT *, 
ROW_NUMBER() over (Partition BY TripID ORDER BY RecordTimeUTC DESC ) 
AS Rank FROM records where TripID in (20141000,20441000,30011022,30011021,30011008,30012029,30012028,30012027,30011007,30011019,30011018, 30012026,30012025.....)) rs WHERE Rank <= 690

您拥有的索引对于查询的以下部分没有用...

SELECT *, 
ROW_NUMBER() over (Partition BY TripID ORDER BY RecordTimeUTC DESC ) 
AS Rank FROM records where TripID in

我会创建一个如下所示的索引

create index nci_sometst on table (tripid,recorddatetime)
include(<remaining columsn you are selecting>)

上面的查询可以帮助获取tripID的IN部分的记录，但是你正在使用派生表计算排名，如果你的内部查询结果集很大，这可能没有多大帮助。

我可能会将其放入临时表中并在排名上创建索引，因此这对其他联合查询也很有帮助。

还查看了您的执行计划，我可以看到您多次扫描同一个表并每次读取很多行

在此处输入图像描述

即使没有数据类型转换警告，您的查询也不会使用您有效拥有的任何索引

于 2017-11-20T15:42:13.887 回答