Couldn’t deploy Yarn session cluster
错误信息
flink on yarn启动yarn-session时, 出现yarn-session无法部署错误
并且提示2.1G的虚拟内存使用了2.2GB内存.
...org.apache.flink.client.deployment.ClusterDeploymentException: `Couldn't deploy Yarn session cluster`at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:381)at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:548)at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$5(FlinkYarnSessionCli.java:785)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:422)at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:785)Caused by: `org.apache.flink.yarn.YarnClusterDescriptor$YarnDeploymentException: The YARN application unexpectedly switched to state FAILED during deployment. `Diagnostics from YARN: Application application_1617573631507_0006 failed 1 times due to AM Container for appattempt_1617573631507_0006_000001 exited with exitCode: -103For more detailed output, check application tracking page:http://node02:8088/cluster/app/application_1617573631507_0006Then, click on links to logs of each attempt.Diagnostics: Container [pid=72909,containerID=container_1617573631507_0006_01_000001] is running beyond virtual memory limits. `Current usage: 167.3 MB of 1 GB physical memory used; 2.2 GB of 2.1 GB virtual memory used. Killing container.`...
问题原因
Yarn容器虚拟内存率不足, 因为是用的虚拟机, 资源相对不足, yarn的虚拟内存比较小.导致启动后yarn创建applicationMaster时内存不足
解决方法
可以先尝试配置yarn, 不检测内存[ryxiong@node01 hadoop-2.7.2]$ vim etc/hadoop/yarn-site.xml
添加如下配置
<!-- 关闭yarn内存检查 --><property><name>yarn.nodemanager.pmem-check-enabled</name><value>false</value></property><property><name>yarn.nodemanager.vmem-check-enabled</name><value>false</value></property>
如果方式1, 还无法解决问题
添加如下配置, 提高yarn容器虚拟内存率
<!-- 提高内存虚拟化率 --><property><name>yarn.nodemanager.vmem-pmem-ratio</name><value>3.0</value></property>
问题解决
