0

我有一个具有两个功能的 Native Impala UDF (Cpp) 这两个功能是相互补充的。

String myUDF(BigInt)
BigInt myUDFReverso(String)

myUDF("myInput")给出一些输出，什么时候myUDFReverso(myUDF("myInput"))应该回馈myInput

当我在这样的镶木地板上运行 impala 查询时，

select column1,myUDF(column1),length(myUDF(column1)),myUDFreverso(myUDF(column1)) from my_parquet_table order by column1 LIMIT 10;

输出随机为 NULL。

输出在第一次运行时说，

+------------+----------------------+------------------------+-------------------------------------+
| column1    | myDB.myUDF(column1)  | length(myUDF(column1)) | myDB.myUDFReverso(myUDF(column1))   |
+------------+----------------------+------------------------+-------------------------------------+
| 27011991   | 1.0.128.9            | 9                      | 27011991                            |
| 27011991   | 1.0.128.9            | 9                      | NULL                                |
| 14022013   | 1.0.131.239          | 11                     | NULL                                |
| 14022013   | 1.0.131.239          | 11                     | NULL                                |
| 14022013   | 1.0.131.239          | 11                     | NULL                                |
| 14022013   | 1.0.131.239          | 11                     | NULL                                |
| 14022013   | 1.0.131.239          | 11                     | NULL                                |
| 14022013   | 1.0.131.239          | 11                     | NULL                                |
| 14022013   | 1.0.131.239          | 11                     | 14022013                            |
| 14022013   | 1.0.131.239          | 11                     | NULL                                |
+------------+----------------------+------------------------+-------------------------------------+

并假设在第二次运行时，

+------------+----------------------+------------------------+-------------------------------------+
| column1    | myDB.myUDF(column1)  | length(myUDF(column1)) | myDB.myUDFReverso(myUDF(column1))   |
+------------+----------------------+------------------------+-------------------------------------+
| 27011991   | 1.0.128.9            | 9                      | 27011991                            |
| 27011991   | 1.0.128.9            | 9                      | NULL                                |
| 14022013   | 1.0.131.239          | 11                     | NULL                                | 
| 14022013   | 1.0.131.239          | 11                     | NULL                                |
| 14022013   | 1.0.131.239          | 11                     | NULL                                |
| 14022013   | 1.0.131.239          | 11                     | 14022013                            |
| 14022013   | 1.0.131.239          | 11                     | 14022013                            |
| 14022013   | 1.0.131.239          | 11                     | NULL                                |
| 14022013   | 1.0.131.239          | 11                     | 14022013                            |
| 14022013   | 1.0.131.239          | 11                     | NULL                                |
+------------+----------------------+------------------------+-------------------------------------+

有时它也会为所有行提供正确的值。

我已经在Impala v1.2.4和v2.1上测试过这 是什么原因？一些内存问题？

编辑1：

BigIntVal myUDF(FunctionContext* context, const StringVal& myInput)
{
  if (myInput.is_null) return BigIntVal::null();

  unsigned int temp_op= 0;
  unsigned long result= 0;
  uint8_t *p;
  char c= '.';

  p=myInput.ptr;

  while (*p != '\0')
  {
    c= *p++;
    int digit= c*2;

    if (digit >= 22 && digit <= 31)
    {
      if ((temp_op= temp_op * 10 - digit) > 493)
      {
        return BigIntVal::null();
      }
    }
    else if (c == '.')
    {
      result= (result << 8) + (unsigned long) temp_op;
      temp_op= 0;
    }
    else
    {
      return BigIntVal::null();
    }
  }

  return BigIntVal((result << 8) + (unsigned long) temp_op);
}

In .h file the macro lowerbytify is defined as 

#define lowerbytify(T,A)        { *(T)= (char)((A));\
                                  *((T)+1)= (char)(((A) >> 8));\
                                  *((T)+2)= (char)(((A) >> 16));\
                                  *((T)+3)= (char)(((A) >> 24)); }

StringVal myUDFReverso(FunctionContext* context, const BigIntVal& origMyInput)
{
  if (origMyInput.is_null)
   return StringVal::null(); 

  int64_t myInput=origMyInput.val;
  char myInputArr[16];
  unsigned int l=0;        

  unsigned char temp[8];
  lowerbytify(temp, myInput);

  char calc[4];
  calc[3]= '.';

  for (unsigned char *p= temp + 4; p-- > temp;)
  {
    unsigned int c= *p;
    unsigned int n1, n2;
    n1= c / 100;
    c-= n1 * 100;
    n2= c / 10;
    c-= n2 * 10;
    calc[0]= (char) n1 + '0';
    calc[1]= (char) n2 + '0';
    calc[2]= (char) c + '0';
    unsigned int length= (n1 ? 4 : (n2 ? 3 : 2));
    unsigned int point= (p <= temp) ? 1 : 0;

    char * begin = &calc[4-length];

    for(int step = length - point;step>0;step--,l++,begin++)
    {
        myInputArr[l]=*begin;
    }
   }

   myInputArr[l]='\0';

   StringVal result(context,l);
   memcpy(result.ptr, myInputArr,l);

    return result;
}
4

1 回答 1

1

我认为您不能假设该字符串以空值结尾。您应该使用StringVal::len来迭代字符而不是while (*p != '\0'). 另外，我建议使用impala-udf-samples github 中的 UDF 测试框架编写一些单元测试，请参阅此示例。

于 2015-05-08T16:46:00.457 回答