hadoop - Hadoop获取日期之间的时间差

Question

我在hadoop中苦苦挣扎

由于我的映射器，我得到了关注

KeyValue1, 2014-02-01 20:42:00
KeyValue1, 2014-02-01 20:45:12
KeyValue1, 2014-05-01 10:35:02
KeyValue2, 2014-03-01 01:45:12
KeyValue2, 2014-03-01 02:08:18
KeyValue3, 2014-02-01 20:45:12
KeyValue4, 2015-02-01 05:45:12
KeyValue4, 2013-02-01 10:45:12

继续..

归根结底，我想要这个；

 KeyValue1, TimeDifference(first occurrence - last occurrence)
 KeyValue2, TimeDifference(first occurrence - last occurrence)
 KeyValue3, -occured once-
 KeyValue4, TimeDifference(first occurrence - last occurrence)

任何输入都受到高度赞赏。干杯

Answer 1

有多种方法。我会建议使用复合键。创建自定义 Partitioner、KeyComparator 和 GroupComparator。然后在减速器一侧，您可以简单地选择第一行和最后一行并有所不同。