{"id":41,"date":"2008-10-16T09:27:04","date_gmt":"2008-10-16T13:27:04","guid":{"rendered":"http:\/\/www.cs.toronto.edu\/~jhancock\/wlog\/?p=41"},"modified":"2008-10-16T09:27:04","modified_gmt":"2008-10-16T13:27:04","slug":"swap-file-problems-in-centos-rocks-cluster","status":"publish","type":"post","link":"http:\/\/www.cs.toronto.edu\/~jhancock\/wlog\/?p=41","title":{"rendered":"Swap file problems in CentOS (Rocks Cluster)"},"content":{"rendered":"<p>We&#8217;ve been experiencing an interesting problem on our cluster nodes which causes them to freeze up.\u00a0 It appears to be related to the way the linux kernel in CentOS deals with memory allocation requests.\u00a0 The issue is caused by the swap partition on\u00a0a machine filling completely, which freezes the system.\u00a0 Any attempt to start a new process hangs, waiting for space to become available from the swap (which it never does).\u00a0 There are several ways of trying to deal with this.\u00a0 The first is to use oomkiller, a process that will detect when the memory limit is going to be reached and kill a process it decides can be sacrificed for the greater good. <a href=\"http:\/\/www.win.tue.nl\/~aeb\/linux\/lk\/lk-9.html\">this<\/a> is a good description of the memory issues and how to test for them.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We&#8217;ve been experiencing an interesting problem on our cluster nodes which causes them to freeze up.\u00a0 It appears to be related to the way the linux kernel in CentOS deals with memory allocation requests.\u00a0 The issue is caused by the swap partition on\u00a0a machine filling completely, which freezes the system.\u00a0 Any attempt to start a [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[19,25,26,24],"_links":{"self":[{"href":"http:\/\/www.cs.toronto.edu\/~jhancock\/wlog\/index.php?rest_route=\/wp\/v2\/posts\/41"}],"collection":[{"href":"http:\/\/www.cs.toronto.edu\/~jhancock\/wlog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.cs.toronto.edu\/~jhancock\/wlog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.cs.toronto.edu\/~jhancock\/wlog\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/www.cs.toronto.edu\/~jhancock\/wlog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=41"}],"version-history":[{"count":2,"href":"http:\/\/www.cs.toronto.edu\/~jhancock\/wlog\/index.php?rest_route=\/wp\/v2\/posts\/41\/revisions"}],"predecessor-version":[{"id":49,"href":"http:\/\/www.cs.toronto.edu\/~jhancock\/wlog\/index.php?rest_route=\/wp\/v2\/posts\/41\/revisions\/49"}],"wp:attachment":[{"href":"http:\/\/www.cs.toronto.edu\/~jhancock\/wlog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=41"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.cs.toronto.edu\/~jhancock\/wlog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=41"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.cs.toronto.edu\/~jhancock\/wlog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=41"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}