Linux服务器EDAC CE memory read error |
您所在的位置：网站首页 › 内存edca报错 › Linux服务器EDAC CE memory read error |
|
之前在大数据集群中，有一台服务器的CPU占用总是莫名其妙飙高，就算执行简单任务也会耗费很长时间，且reboot不能解决问题。检查了各种可能的问题之后，最终在查看dmesg命令的设备信息时，发现大量如下的日志： ...... [1180532.573917] EDAC MC1: 128 CE memory read error on CPU_SrcID#1_Ha#0_Chan#1_DIMM#1 (channel:1 slot:1 page:0x2b67afa offset:0xb40 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0092 socket:1 ha:0 channel_mask:2 rank:5) [1180532.573925] EDAC MC1: 147 CE memory read error on CPU_SrcID#1_Ha#0_Chan#1_DIMM#1 (channel:1 slot:1 page:0x30b6e9a offset:0x540 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0092 socket:1 ha:0 channel_mask:2 rank:5) [1180532.573930] EDAC MC1: 236 CE memory read error on CPU_SrcID#1_Ha#0_Chan#0_DIMM#1 (channel:0 slot:1 page:0x236e7bb offset:0xd00 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0092 socket:1 ha:0 channel_mask:1 rank:5) [1180532.573935] EDAC MC1: 189 CE memory read error on CPU_SrcID#1_Ha#0_Chan#1_DIMM#1 (channel:1 slot:1 page:0x365669d offset:0xf40 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0092 socket:1 ha:0 channel_mask:2 rank:5) [1180532.573940] EDAC MC1: 255 CE memory read error on CPU_SrcID#1_Ha#0_Chan#1_DIMM#1 (channel:1 slot:1 page:0x3d2953a offset:0x540 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0092 socket:1 ha:0 channel_mask:2 rank:5) [1180532.573944] EDAC MC1: 177 CE memory read error on CPU_SrcID#1_Ha#0_Chan#1_DIMM#1 (channel:1 slot:1 page:0x212b93c offset:0x540 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0092 socket:1 ha:0 channel_mask:2 rank:5) [1180532.573949] EDAC MC1: 136 CE memory read error on CPU_SrcID#1_Ha#0_Chan#1_DIMM#1 (channel:1 slot:1 page:0x31d099d offset:0xb40 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0092 socket:1 ha:0 channel_mask:2 rank:5) [1180532.573953] EDAC MC1: 88 CE memory read error on CPU_SrcID#1_Ha#0_Chan#0_DIMM#1 (channel:0 slot:1 page:0x228e5be offset:0x100 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0092 socket:1 ha:0 channel_mask:1 rank:5) [1180532.573958] EDAC MC1: 111 CE memory read error on CPU_SrcID#1_Ha#0_Chan#0_DIMM#1 (channel:0 slot:1 page:0x34e5e7b offset:0xf00 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0092 socket:1 ha:0 channel_mask:1 rank:5) [1180532.573962] EDAC MC1: 149 CE memory read error on CPU_SrcID#1_Ha#0_Chan#0_DIMM#1 (channel:0 slot:1 page:0x314bb9f offset:0xd00 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0092 socket:1 ha:0 channel_mask:1 rank:5) [1180532.611035] EDAC sbridge MC1: HANDLING MCE MEMORY ERROR [1180532.611045] EDAC sbridge MC1: HANDLING MCE MEMORY ERROR [1180533.460534] EDAC sbridge MC1: HANDLING MCE MEMORY ERROR [1180533.460544] EDAC sbridge MC1: HANDLING MCE MEMORY ERROR [1180533.547047] EDAC sbridge MC1: HANDLING MCE MEMORY ERROR ...... EDAC即error detection and correction（错误检测与纠正），是Linux系统内部的机制。在上面的日志中，可以清楚地看出是内存读错误。MC即memory controller（内存控制器）。CE则代表correctable error，是ECC内存中可以纠正的错误，相对地还有UE（uncorrectable error）。 为了摸清是哪些内存出了问题，找出所有内存的CE计数： ~ grep "[0-9]" /sys/devices/system/edac/mc/mc*/csrow*/ch*_ce_count /sys/devices/system/edac/mc/mc0/csrow0/ch0_ce_count:0 /sys/devices/system/edac/mc/mc0/csrow0/ch1_ce_count:0 /sys/devices/system/edac/mc/mc0/csrow0/ch2_ce_count:0 /sys/devices/system/edac/mc/mc0/csrow0/ch3_ce_count:0 /sys/devices/system/edac/mc/mc0/csrow1/ch0_ce_count:0 /sys/devices/system/edac/mc/mc0/csrow1/ch1_ce_count:0 /sys/devices/system/edac/mc/mc0/csrow1/ch2_ce_count:0 /sys/devices/system/edac/mc/mc0/csrow1/ch3_ce_count:0 /sys/devices/system/edac/mc/mc1/csrow0/ch0_ce_count:0 /sys/devices/system/edac/mc/mc1/csrow0/ch1_ce_count:0 /sys/devices/system/edac/mc/mc1/csrow0/ch2_ce_count:0 /sys/devices/system/edac/mc/mc1/csrow0/ch3_ce_count:0 /sys/devices/system/edac/mc/mc1/csrow1/ch0_ce_count:198276530 /sys/devices/system/edac/mc/mc1/csrow1/ch1_ce_count:233266714 /sys/devices/system/edac/mc/mc1/csrow1/ch2_ce_count:0 /sys/devices/system/edac/mc/mc1/csrow1/ch3_ce_count:0 可见是mc1/csrow1/ch0和mc1/csrow1/ch1两个通道上的内存出了问题。mc、csrow（chip-select row）与ch之间的关系类似下图： mc0 ch0 ch1 ============================== csrow0 | DIMM_A0 | DIMM_B0 | csrow1 | DIMM_A0 | DIMM_B0 | ============================== mc1 ch0 ch1 ============================== csrow2 | DIMM_A1 | DIMM_B1 | csrow3 | DIMM_A1 | DIMM_B1 | ============================== 通过dmidecode -t memory命令，可以查看每个DIMM（也就是常说的内存条）对应的位置。然后根据映射关系，去服务器上将出现问题的两根内存条更换，问题就解决了。 作者：LittleMagic 链接：https://www.jianshu.com/p/f1fd08176ca0 来源：简书 著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。 |
今日新闻 |
点击排行 |
|
推荐新闻 |
图片新闻 |
|
专题文章 |
| CopyRight 2018-2019 实验室设备网 版权所有 win10的实时保护怎么永久关闭 |