4

Let say I have a video from a drive recorder. I want to construct the recorded scene's points cloud using structure from motion technique. First I need to track some points.

Which algorithm can yield a better result? By using the sparse optical flow (Kanade-Lucas-Tomasi tracker) or the dense optical flow (Farneback)? I have experimented a bit but cannot really decide. Each one of them has their own strengths and weaknesses.

The ultimate target is to get the points cloud of the recorded cars in the scene. By using the sparse optical flow, I can track the interesting points of the cars. But it would be quite unpredictable. One solution is to make some kind of grid in the image, and force the tracker to track one interesting point in each of the grid. But I think this would be quite hard.

By using the dense flow, I can get the movement of every pixel, but the problem is, it cannot really detect the motion of cars that have only little motion. Also, I have doubt that the flow of every pixel yielded by the algorithm would be that accurate. Plus, with this, I believe I can only get the pixels movement between two frames only (unlike by using the sparse optical flow in which I can get multiple coordinates of the same interesting point along time t)

4

1 回答 1

3

您的标题表示 SFM，其中包括姿势估计，

跟踪只是第一步（匹配），如果你想要视频中的点云（非常艰巨的任务），我首先会想到的是捆绑调整，它也适用于 MVE，

尽管如此，对于视频我们可以做更多，因为帧彼此太接近，我们可以使用更快的算法，例如（光流），/而不是匹配 SIFT/，并从中提取F矩阵，然后：

E = 1/K * F * K

回到你原来的问题，什么更好：

1) 密集光流，或

2）稀疏一个。

显然你是离线工作，所以速度并不重要，但我会推荐稀疏的，

更新

对于 3d 重建，密集可能看起来更有吸引力，但正如你所说，它很少健壮，所以你可以使用稀疏但添加尽可能多的点，使其成为半密集，

我不能说出一些可以做到这一点的方法，比如mono-slam或orb-slam

最终更新

正如我之前写的那样使用半密集，但 SFM 总是假设静态对象（没有运动），否则它将永远无法工作。

实际上，使用图像中的所有像素从未用于 3d 重建（不是直接方法），并且总是 SIFT 被称赞为特征检测和匹配的方式，.. 最近所有像素都用于不同类型的校准，例如在以下方法中：直接稀疏里程计和称为直接方法的LSD

于 2017-05-09T06:59:45.153 回答