1

当我尝试在 ElasticSearch 设置中禁用动态映射时，我遇到了错误。我正在使用 ElasticSearch 1.7 版本进行实施。

堆栈跟踪 ：

8151 [main] WARN org.apache.hadoop.mapred.YarnChild - Exception running child : org.elasticsearch.hadoop.rest.EsHadoopInvalidRequest: Found unrecoverable error [10.74.51.71:9200] returned Not Found(404) - [TypeMissingException[[test_2017051222] type[[vehicle, trying to auto create mapping, but dynamic mapping is disabled]] missing]]; Bailing out..
    at org.elasticsearch.hadoop.rest.RestClient.retryFailedEntries(RestClient.java:207)
    at org.elasticsearch.hadoop.rest.RestClient.bulk(RestClient.java:170)
    at org.elasticsearch.hadoop.rest.RestRepository.tryFlush(RestRepository.java:225)
    at org.elasticsearch.hadoop.rest.RestRepository.flush(RestRepository.java:248)
    at org.elasticsearch.hadoop.rest.RestRepository.doWriteToIndex(RestRepository.java:187)
    at org.elasticsearch.hadoop.rest.RestRepository.writeToIndex(RestRepository.java:163)
    at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:151)
    at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:566)
    at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
    at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
    at org.apache.hadoop.mapreduce.Reducer.reduce(Reducer.java:150)
    at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
    at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:635)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

设置片段：

  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1,
    "index.query.default_field":"test",
    "index.refresh_interval" : "5s",
    "index.mapper.dynamic": false ,
    "analysis": {
         "filter": {
            "ngram_filter": {
               "type": "ngram",
               "min_gram": 2,
               "max_gram": 18,
               "token_chars": [
                  "letter",
                  "digit"
               ]
            }
         },
         "analyzer": {
            "ngram_analyzer": {
               "type": "custom",
               "tokenizer": "standard",
               "filter": [
                  "lowercase",
                  "ngram_filter"
               ]
            }
         }
      }
  }

我看到在 ES 端点的设置中禁用了动态映射，但作业失败。我有一个 avro json 映射文件和 es json 映射文件，其中 avro json 映射文件是超集，而 es json 映射文件是子集。我不希望超集映射文件中的所有字段都反映在 ES 索引上，而只转储子映射文件中的字段。我做错了还是有其他方法可以做到。

谢谢。

4

1 回答 1

0

那是因为你已经设置 "index.mapper.dynamic": false了这意味着新类型不会在没有首先声明的情况下自动创建。

您想要做的是"dynamic": "false"在您的类型的映射中设置。 PUT /test_index { "mappings": { "test_type": { "dynamic": "false" } } } 更多信息： https ://www.elastic.co/guide/en/elasticsearch/guide/1.x/dynamic-mapping.html

例子：

  1. 运行映射 PUT /my_index { "mappings": { "testing": { "dynamic": "false", "properties": { "field1": { "type": "string", "index": "analyzed" } } } } }

  2. testing以类型 索引文档 POST /my_index/testing/1 { "field1":"demo", "field99":"anotherDemo" }

  3. 的回应GET /my_index/testing/_mapping { "my_index": { "mappings": { "testing": { "dynamic": "false", "properties": { "field1": { "type": "string" } } } } } }

如您所见，field99 没有映射。

于 2017-05-14T09:18:15.127 回答