3

跟随如何在数据集中存储自定义对象？并尝试为数据帧注册我自己的 kryo 编码器，但我遇到了一个问题Schema for type com.esri.core.geometry.Envelope is not supported

有一个函数可以将字符串（WKT）解析为几何对象，例如：

def mapWKTToEnvelope(wkt: String): Envelope = {
    val envBound = new Envelope()
    val spatialReference = SpatialReference.create(4326)
    // Parse the WKT String into a Geometry Object
    val ogcObj = OGCGeometry.fromText(wkt)
    ogcObj.setSpatialReference(spatialReference)
    ogcObj.getEsriGeometry.queryEnvelope(envBound)
    envBound
  }

这适用于 UDF，例如：

implicit val envelopeEncoder: Encoder[Envelope] = Encoders.kryo[Envelope]
val ST_Envelope = udf((wkt: String) => mapWKTToEnvelope(wkt))

但是，UDF 将编译但会引发以下运行时错误：

[error] Exception in thread "main" java.lang.UnsupportedOperationException: Schema for type com.esri.core.geometry.Envelope is not supported
[error]         at org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:733)
[error]         at org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:671)
[error]         at org.apache.spark.sql.functions$.udf(functions.scala:3076)

编辑

然而

val first = df[(String, String)].first
val envBound = new Envelope()
val ogcObj = OGCGeometry.fromText(first._1)
ogcObj.setSpatialReference(spatialReference)
ogcObj.getEsriGeometry.queryEnvelope(envBound)
spark.createDataset(Seq((envBound)))(envelopeEncoder)

工作得很好：

root
 |-- value: binary (nullable = true)
+--------------------+
|               value|
+--------------------+
|[01 00 63 6F 6D 2...|
+--------------------+

我怎样才能让它在 UDF 中也能工作

4

0 回答 0