0

我在运行 hadoop 节点时遇到错误

start-dfs.cmd

我的配置：

<property>

 <name>dfs.replication</name>

 <value>1</value>


</property>



<property>

<name>dfs.namenode.name.dir</name>

<value>/hadoop/data/namenode</value>


</property>


<property>

<name>dfs.datanode.data.dir</name>

<value>C:/hadoop/data/datanode</value>


</property>



  <property>
      <name>dfs.permissions</name>
      <value>false</value>
   </property>

错误：

 2021-05-25 21:20:34,259 INFO checker.ThrottledAsyncChecker: Scheduling a check for [DISK]file:/C:/hadoop/data/datanode
    2021-05-25 21:20:34,299 WARN checker.StorageLocationChecker: Exception checking StorageLocation [DISK]file:/C:/hadoop/data/datanode
    java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$POSIX.stat(Ljava/lang/String;)Lorg/apache/hadoop/io/nativeio/NativeIO$POSIX$Stat;
            at org.apache.hadoop.io.nativeio.NativeIO$POSIX.stat(Native Method)
            at org.apache.hadoop.io.nativeio.NativeIO$POSIX.getStat(NativeIO.java:608)
            at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfoByNativeIO(RawLocalFileSystem.java:823)
            at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:737)
            at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.getPermission(RawLocalFileSystem.java:705)
            at org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:233)
            at org.apache.hadoop.util.DiskChecker.checkDirInternal(DiskChecker.java:141)
            at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:116)
            at org.apache.hadoop.hdfs.server.datanode.StorageLocation.check(StorageLocation.java:239)
            at org.apache.hadoop.hdfs.server.datanode.StorageLocation.check(StorageLocation.java:52)
            at org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker$1.call(ThrottledAsyncChecker.java:142)
            at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
            at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
            at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)
    2021-05-25 21:20:34,307 ERROR datanode.DataNode: Exception in secureMain
    org.apache.hadoop.util.DiskChecker$DiskErrorException: Too many failed volumes - current valid volumes: 0, volumes configured: 1, volumes failed: 1, volume failures tolerated: 0
            at org.apache.hadoop.hdfs.server.datanode.checker.StorageLocationChecker.check(StorageLocationChecker.java:231)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2806)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2721)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2763)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2907)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2931)
    2021-05-25 21:20:34,307 INFO util.ExitUtil: Exiting with status 1: org.apache.hadoop.util.DiskChecker$DiskErrorException: Too many failed volumes - current valid volumes: 0, volumes configured: 1, volumes failed: 1, volume failures tolerated: 0
    2021-05-25 21:20:34,315 INFO datanode.DataNode: SHUTDOWN_MSG:
    /************************************************************
    SHUTDOWN_MSG: Shutting down DataNode at DESKTOP-TQPMKPM/172.18.160.1
    ************************************************************/

我尝试了一堆不同的配置，比如为节点故障创建最小阈值

我还在stackoverflow上尝试了无数看起来与此类似的解决方案，但我无法让它工作！

4

0 回答 0