2

我们正在对我们公司的一些 Haskell 代码进行基准测试，我们刚刚遇到了一个非常奇怪的案例。这是一个代码，它对同一事物进行了 2 次基准测试。前者使用Criterion.env为所有测试创建一次的 an，后者为每个测试创建 env。这是唯一的区别，但是为每个长凳创建环境的那个，运行速度快了 5 倍。

有谁知道什么会导致它？最小的例子：

module Main where

import Prelude
import Control.Monad
import qualified Data.Vector.Storable.Mutable as Vector
import qualified Data.Vector.Storable         as Vector
import           Data.Vector.Storable         (Vector)
import           Criterion.Main


testf :: Int -> Vector Int -> IO (Vector Int)
testf !i !v = do
    mv <- Vector.unsafeThaw v
    let go j = do
          x <- if j == 0 then return 0 else Vector.unsafeRead mv (j - 1)
          Vector.unsafeWrite mv j (x+1)
          when (j < i - 1) $ go (j + 1)
    go 0
    Vector.unsafeFreeze mv

mkVec :: Int -> IO (Vector Int)
mkVec !i = Vector.unsafeFreeze =<< Vector.new (i + 1)

main :: IO ()
main = do
    defaultMain
        [ bgroup "env per all runs"
            $ (\(i :: Int) -> env (mkVec (10 ^ i))
            $ \v -> bench ("10e" ++ show i)
            $ nfIO (testf (10 ^ i) v))  <$> [7..8]

        , bgroup "env per every run"
            $ (\(i :: Int) -> bench ("10e" ++ show i)
            $ perRunEnv (mkVec (10 ^ i))
            $ (testf (10 ^ i)))  <$> [7..8]
        ]

和结果：

benchmarking env per all runs/10e7
time                 17.34 ms   (17.20 ms .. 17.41 ms)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 17.43 ms   (17.34 ms .. 17.67 ms)
std dev              321.5 μs   (142.1 μs .. 591.3 μs)

benchmarking env per all runs/10e8
time                 173.5 ms   (173.2 ms .. 173.8 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 173.8 ms   (173.6 ms .. 174.0 ms)
std dev              279.5 μs   (194.9 μs .. 355.6 μs)
variance introduced by outliers: 12% (moderately inflated)

benchmarking env per every run/10e7
time                 4.289 ms   (1.807 ms .. 5.771 ms)
                     0.924 R²   (0.696 R² .. 1.000 R²)
mean                 8.903 ms   (5.752 ms .. 14.20 ms)
std dev              5.029 ms   (249.0 μs .. 6.244 ms)
variance introduced by outliers: 79% (severely inflated)

benchmarking env per every run/10e8
time                 53.76 ms   (30.23 ms .. 98.51 ms)
                     0.940 R²   (0.920 R² .. 1.000 R²)
mean                 102.9 ms   (68.67 ms .. 127.1 ms)
std dev              36.55 ms   (0.0 s .. 41.99 ms)
variance introduced by outliers: 73% (severely inflated)
4

1 回答 1

3

与您可能是同事的问题一样，我无法重现此问题。注意我使用的是 GHC 8.2.2 和标准 1.3.0.0

benchmarking env per all runs/10e7
time                 18.94 ms   (18.71 ms .. 19.22 ms)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 19.59 ms   (19.39 ms .. 19.99 ms)
std dev              618.3 μs   (379.7 μs .. 952.8 μs)

benchmarking env per all runs/10e8
time                 192.0 ms   (189.5 ms .. 194.9 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 191.8 ms   (190.3 ms .. 193.1 ms)
std dev              1.778 ms   (1.088 ms .. 2.457 ms)
variance introduced by outliers: 14% (moderately inflated)

benchmarking env per every run/10e7
time                 18.97 ms   (18.38 ms .. 19.62 ms)
                     0.999 R²   (0.996 R² .. 1.000 R²)
mean                 18.98 ms   (18.83 ms .. 19.35 ms)
std dev              298.8 μs   (25.39 μs .. 391.3 μs)
variance introduced by outliers: 14% (moderately inflated)

benchmarking env per every run/10e8
time                 194.0 ms   (182.0 ms .. 211.6 ms)
                     0.999 R²   (0.997 R² .. 1.000 R²)
mean                 192.0 ms   (189.0 ms .. 193.9 ms)
std dev              2.850 ms   (0.0 s .. 3.261 ms)
variance introduced by outliers: 19% (moderately inflated)

./p  18.32s user 1.16s system 99% cpu 19.531 total
% ghc-pkg list criterion
...
    criterion-1.3.0.0

我注意到我的标准版本是在@jberryman 指出的错误修复后上传的，所以也许这就是区别。

于 2018-02-08T02:48:44.177 回答