Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720648] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720652] php cpuset=/ mems_allowed=0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720655] Pid: 9173, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720656] Call Trace: Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720663] [] dump_header+0x91/0xe0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720665] [] oom_kill_process+0x85/0xb0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720668] [] out_of_memory+0xfa/0x220 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720671] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720674] [] alloc_pages_current+0xb6/0x120 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720677] [] __page_cache_alloc+0xb7/0xd0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720679] [] filemap_fault+0x212/0x3c0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720682] [] __do_fault+0x72/0x550 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720685] [] handle_pte_fault+0xfa/0x200 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720688] [] ? xen_pmd_val+0xe/0x10 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720691] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720693] [] handle_mm_fault+0x269/0x370 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720696] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720698] [] ? check_events+0x12/0x20 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720702] [] do_page_fault+0x184/0x550 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720704] [] ? do_brk+0x21f/0x320 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720706] [] ? xen_clts+0x8d/0x190 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720709] [] ? __schedule+0x3cc/0x6f0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720712] [] page_fault+0x25/0x30 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720714] Mem-Info: Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720715] Node 0 DMA per-cpu: Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720717] CPU 0: hi: 0, btch: 1 usd: 0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720718] CPU 1: hi: 0, btch: 1 usd: 0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720719] Node 0 DMA32 per-cpu: Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720721] CPU 0: hi: 186, btch: 31 usd: 30 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720722] CPU 1: hi: 186, btch: 31 usd: 53 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720725] active_anon:932617 inactive_anon:1393 isolated_anon:0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720726] active_file:0 inactive_file:0 isolated_file:0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720726] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720727] free:4029 slab_reclaimable:2703 slab_unreclaimable:2697 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720728] mapped:1339 shmem:1405 pagetables:3164 bounce:0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720729] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720736] lowmem_reserve[]: 0 3772 3772 3772 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720739] Node 0 DMA32 free:7780kB min:7848kB low:9808kB high:11772kB active_anon:3730468kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5356kB shmem:5620kB slab_reclaimable:10812kB slab_unreclaimable:10788kB kernel_stack:784kB pagetables:12656kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:116 all_unreclaimable? yes Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720745] lowmem_reserve[]: 0 0 0 0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720748] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720754] Node 0 DMA32: 1433*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7780kB Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720761] 1434 total pagecache pages Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720762] 0 pages in swap cache Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720763] Swap cache stats: add 0, delete 0, find 0/0 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720764] Free swap = 0kB Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720765] Total swap = 0kB Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726142] 985072 pages RAM Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726144] 24998 pages reserved Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726145] 3792 pages shared Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726147] 953274 pages non-shared Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726148] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726157] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726161] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726164] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726167] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726170] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726174] [ 664] 0 664 1815 123 1 0 0 dhclient3 Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726177] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726180] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726184] [ 790] 101 790 63477 545 0 0 0 rsyslogd Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726187] [ 827] 0 827 1081 36 1 0 0 acpid Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726190] [ 831] 0 831 4226 41 0 0 0 atd Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726194] [ 975] 0 975 6276 80 0 0 0 master Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726197] [ 983] 108 983 6834 106 0 0 0 qmgr Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726200] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726203] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726206] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726209] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726212] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726216] [15657] 0 15657 4777 60 0 0 0 cron Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726219] [ 654] 1012 654 70287 5737 1 0 0 php Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726222] [ 657] 1012 657 69242 6257 0 0 0 php Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726225] [ 658] 1012 658 73718 10819 0 0 0 php Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726228] [ 678] 1012 678 70152 5559 0 0 0 php Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726231] [13468] 0 13468 89584 2524 0 0 0 php5-fpm Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726234] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726237] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726240] [ 9173] 1012 9173 956614 893396 0 0 0 php Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726243] [25666] 1012 25666 68618 5633 0 0 0 php Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726247] [15959] 108 15959 6792 69 0 0 0 pickup Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726250] [20529] 108 20529 6795 70 0 0 0 trivial-rewrite Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726253] [20530] 108 20530 11107 159 0 0 0 smtp Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726256] [20626] 108 20626 6818 74 0 0 0 cleanup Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726259] [20630] 108 20630 11107 157 0 0 0 smtp Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726262] [20634] 108 20634 11107 157 0 0 0 smtp Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726265] [20638] 108 20638 11107 142 1 0 0 smtp Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726268] [20642] 108 20642 11107 158 1 0 0 smtp Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726271] [20648] 108 20648 6792 69 0 0 0 scache Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726274] Out of memory: Kill process 9173 (php) score 932 or sacrifice child Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726286] Killed process 9173 (php) total-vm:3826456kB, anon-rss:3573576kB, file-rss:8kB Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004159] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004164] php cpuset=/ mems_allowed=0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004167] Pid: 7531, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004169] Call Trace: Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004177] [] dump_header+0x91/0xe0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004180] [] oom_kill_process+0x85/0xb0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004183] [] out_of_memory+0xfa/0x220 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004187] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004193] [] ? noalloc_get_block_write+0x30/0x30 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004196] [] alloc_pages_current+0xb6/0x120 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004201] [] __page_cache_alloc+0xb7/0xd0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004204] [] filemap_fault+0x212/0x3c0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004208] [] __do_fault+0x72/0x550 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004211] [] handle_pte_fault+0xfa/0x200 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004215] [] ? xen_pmd_val+0xe/0x10 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004218] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004222] [] handle_mm_fault+0x269/0x370 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004225] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004228] [] ? check_events+0x12/0x20 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004232] [] do_page_fault+0x184/0x550 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004235] [] ? xen_mc_flush+0xb2/0x1c0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004238] [] ? xen_clts+0x8d/0x190 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004242] [] page_fault+0x25/0x30 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004244] Mem-Info: Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004245] Node 0 DMA per-cpu: Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004248] CPU 0: hi: 0, btch: 1 usd: 0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004250] CPU 1: hi: 0, btch: 1 usd: 0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004251] Node 0 DMA32 per-cpu: Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004253] CPU 0: hi: 186, btch: 31 usd: 0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004255] CPU 1: hi: 186, btch: 31 usd: 0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004258] active_anon:933071 inactive_anon:1393 isolated_anon:0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004259] active_file:0 inactive_file:0 isolated_file:0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004260] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004261] free:4048 slab_reclaimable:2519 slab_unreclaimable:2627 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004262] mapped:1367 shmem:1405 pagetables:3058 bounce:0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004264] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004273] lowmem_reserve[]: 0 3772 3772 3772 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004277] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732284kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5468kB shmem:5620kB slab_reclaimable:10076kB slab_unreclaimable:10508kB kernel_stack:712kB pagetables:12232kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004285] lowmem_reserve[]: 0 0 0 0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004288] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004297] Node 0 DMA32: 886*4kB 183*8kB 38*16kB 9*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004307] 1467 total pagecache pages Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004309] 0 pages in swap cache Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004310] Swap cache stats: add 0, delete 0, find 0/0 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004312] Free swap = 0kB Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004313] Total swap = 0kB Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009822] 985072 pages RAM Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009824] 24998 pages reserved Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009825] 3789 pages shared Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009826] 953329 pages non-shared Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009828] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009837] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009840] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009844] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009847] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009850] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009853] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009857] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009860] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009864] [ 790] 101 790 63477 545 0 0 0 rsyslogd Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009867] [ 827] 0 827 1081 36 1 0 0 acpid Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009870] [ 831] 0 831 4226 41 0 0 0 atd Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009874] [ 975] 0 975 6276 80 1 0 0 master Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009877] [ 983] 108 983 6834 106 0 0 0 qmgr Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009880] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009883] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009886] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009889] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009892] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009895] [15657] 0 15657 4777 60 1 0 0 cron Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009898] [ 654] 1012 654 70287 5737 1 0 0 php Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009901] [ 657] 1012 657 69242 6257 1 0 0 php Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009904] [ 658] 1012 658 73718 10820 0 0 0 php Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009907] [ 678] 1012 678 70152 5559 1 0 0 php Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009911] [13468] 0 13468 89584 2540 1 0 0 php5-fpm Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009914] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009917] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009920] [25666] 1012 25666 68618 5632 0 0 0 php Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009923] [20652] 1012 20652 952535 889294 1 0 0 php Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009926] [ 7531] 1012 7531 68580 5604 0 0 0 php Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009929] Out of memory: Kill process 20652 (php) score 928 or sacrifice child Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009939] Killed process 20652 (php) total-vm:3810140kB, anon-rss:3557176kB, file-rss:0kB Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116021] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116025] ntpd cpuset=/ mems_allowed=0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116028] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116030] Call Trace: Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116037] [] dump_header+0x91/0xe0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116040] [] oom_kill_process+0x85/0xb0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116042] [] out_of_memory+0xfa/0x220 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116045] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116048] [] alloc_pages_current+0xb6/0x120 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116054] [] __page_cache_alloc+0xb7/0xd0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116070] [] filemap_fault+0x212/0x3c0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116074] [] __do_fault+0x72/0x550 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116077] [] handle_pte_fault+0xfa/0x200 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116081] [] ? xen_pmd_val+0xe/0x10 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116085] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116088] [] handle_mm_fault+0x269/0x370 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116092] [] do_page_fault+0x184/0x550 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116096] [] page_fault+0x25/0x30 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116098] Mem-Info: Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116100] Node 0 DMA per-cpu: Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116102] CPU 0: hi: 0, btch: 1 usd: 0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116104] CPU 1: hi: 0, btch: 1 usd: 0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116106] Node 0 DMA32 per-cpu: Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116108] CPU 0: hi: 186, btch: 31 usd: 61 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116110] CPU 1: hi: 186, btch: 31 usd: 31 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116114] active_anon:932765 inactive_anon:1393 isolated_anon:0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116115] active_file:0 inactive_file:1 isolated_file:8 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116115] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116116] free:4036 slab_reclaimable:2736 slab_unreclaimable:2654 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116117] mapped:1364 shmem:1405 pagetables:3075 bounce:0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116120] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116128] lowmem_reserve[]: 0 3772 3772 3772 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116132] Node 0 DMA32 free:7808kB min:7848kB low:9808kB high:11772kB active_anon:3731060kB inactive_anon:5572kB active_file:0kB inactive_file:4kB unevictable:0kB isolated(anon):0kB isolated(file):32kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5456kB shmem:5620kB slab_reclaimable:10944kB slab_unreclaimable:10616kB kernel_stack:728kB pagetables:12300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:85 all_unreclaimable? yes Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116140] lowmem_reserve[]: 0 0 0 0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116144] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116154] Node 0 DMA32: 1236*4kB 94*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7808kB Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116164] 1434 total pagecache pages Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116166] 0 pages in swap cache Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116167] Swap cache stats: add 0, delete 0, find 0/0 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116169] Free swap = 0kB Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116170] Total swap = 0kB Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121630] 985072 pages RAM Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121631] 24998 pages reserved Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121632] 3773 pages shared Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121633] 953279 pages non-shared Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121635] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121643] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121645] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121648] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121650] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121652] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121654] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121657] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121659] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121662] [ 790] 101 790 63477 545 0 0 0 rsyslogd Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121665] [ 827] 0 827 1081 36 1 0 0 acpid Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121667] [ 831] 0 831 4226 41 0 0 0 atd Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121669] [ 975] 0 975 6276 80 0 0 0 master Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121672] [ 983] 108 983 6834 106 0 0 0 qmgr Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121674] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121676] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121678] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121681] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121683] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121685] [15657] 0 15657 4777 60 0 0 0 cron Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121687] [ 654] 1012 654 70287 5737 0 0 0 php Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121689] [ 657] 1012 657 69242 6257 0 0 0 php Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121691] [ 658] 1012 658 73718 10820 0 0 0 php Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121694] [ 678] 1012 678 70152 5559 1 0 0 php Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121696] [13468] 0 13468 89584 2533 1 0 0 php5-fpm Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121698] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121700] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121703] [25666] 1012 25666 68618 5633 0 0 0 php Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121705] [ 7595] 1012 7595 952140 888909 1 0 0 php Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121707] [ 5418] 108 5418 6792 69 1 0 0 pickup Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121710] [18897] 1012 18897 68580 5605 0 0 0 php Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121712] Out of memory: Kill process 7595 (php) score 927 or sacrifice child Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121721] Killed process 7595 (php) total-vm:3808560kB, anon-rss:3555636kB, file-rss:0kB Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956202] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956206] php cpuset=/ mems_allowed=0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956209] Pid: 3519, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956210] Call Trace: Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956217] [] dump_header+0x91/0xe0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956220] [] oom_kill_process+0x85/0xb0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956222] [] out_of_memory+0xfa/0x220 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956225] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956228] [] alloc_pages_current+0xb6/0x120 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956231] [] __page_cache_alloc+0xb7/0xd0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956233] [] filemap_fault+0x212/0x3c0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956236] [] __do_fault+0x72/0x550 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956239] [] handle_pte_fault+0xfa/0x200 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956242] [] ? xen_pmd_val+0xe/0x10 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956245] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956247] [] handle_mm_fault+0x269/0x370 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956250] [] ? find_vma+0x1/0x70 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956253] [] do_page_fault+0x184/0x550 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956256] [] ? vfs_write+0x110/0x180 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956259] [] page_fault+0x25/0x30 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956261] Mem-Info: Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956262] Node 0 DMA per-cpu: Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956264] CPU 0: hi: 0, btch: 1 usd: 0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956265] CPU 1: hi: 0, btch: 1 usd: 0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956266] Node 0 DMA32 per-cpu: Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956268] CPU 0: hi: 186, btch: 31 usd: 35 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956269] CPU 1: hi: 186, btch: 31 usd: 36 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956272] active_anon:932814 inactive_anon:1393 isolated_anon:0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956273] active_file:21 inactive_file:10 isolated_file:0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956273] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956274] free:4044 slab_reclaimable:2714 slab_unreclaimable:2678 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956275] mapped:1360 shmem:1405 pagetables:3072 bounce:0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956276] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956283] lowmem_reserve[]: 0 3772 3772 3772 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956286] Node 0 DMA32 free:7840kB min:7848kB low:9808kB high:11772kB active_anon:3731256kB inactive_anon:5572kB active_file:84kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5440kB shmem:5620kB slab_reclaimable:10856kB slab_unreclaimable:10712kB kernel_stack:728kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:239 all_unreclaimable? yes Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956292] lowmem_reserve[]: 0 0 0 0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956295] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956302] Node 0 DMA32: 1281*4kB 75*8kB 2*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7836kB Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956308] 1425 total pagecache pages Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956310] 0 pages in swap cache Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956311] Swap cache stats: add 0, delete 0, find 0/0 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956312] Free swap = 0kB Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956313] Total swap = 0kB Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961780] 985072 pages RAM Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961783] 24998 pages reserved Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961784] 3774 pages shared Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961785] 953297 pages non-shared Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961787] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961796] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961799] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961802] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961805] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961809] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961812] [ 664] 0 664 1815 123 1 0 0 dhclient3 Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961815] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961819] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961822] [ 790] 101 790 63477 569 0 0 0 rsyslogd Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961825] [ 827] 0 827 1081 36 1 0 0 acpid Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961828] [ 831] 0 831 4226 41 0 0 0 atd Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961832] [ 975] 0 975 6276 80 1 0 0 master Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961835] [ 983] 108 983 6834 106 0 0 0 qmgr Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961838] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961841] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961844] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961847] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961850] [ 1242] 107 1242 9445 163 0 0 0 ntpd Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961854] [15657] 0 15657 4777 60 0 0 0 cron Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961857] [ 654] 1012 654 70287 5737 1 0 0 php Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961860] [ 657] 1012 657 69242 6257 1 0 0 php Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961863] [ 658] 1012 658 73718 10820 0 0 0 php Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961866] [ 678] 1012 678 70152 5559 0 0 0 php Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961869] [13468] 0 13468 89584 2508 0 0 0 php5-fpm Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961872] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961875] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961878] [25666] 1012 25666 68618 5646 1 0 0 php Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961881] [18961] 1012 18961 952167 888945 0 0 0 php Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961885] [28937] 108 28937 6792 69 0 0 0 pickup Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961888] [ 3519] 1012 3519 68580 5604 0 0 0 php Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961891] Out of memory: Kill process 18961 (php) score 927 or sacrifice child Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961901] Killed process 18961 (php) total-vm:3808668kB, anon-rss:3555780kB, file-rss:0kB Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116002] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116007] ntpd cpuset=/ mems_allowed=0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116010] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116012] Call Trace: Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116020] [] dump_header+0x91/0xe0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116024] [] oom_kill_process+0x85/0xb0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116027] [] out_of_memory+0xfa/0x220 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116031] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116035] [] alloc_pages_current+0xb6/0x120 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116040] [] __page_cache_alloc+0xb7/0xd0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116043] [] filemap_fault+0x212/0x3c0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116047] [] __do_fault+0x72/0x550 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116052] [] handle_pte_fault+0xfa/0x200 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116068] [] ? xen_pmd_val+0xe/0x10 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116071] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116075] [] handle_mm_fault+0x269/0x370 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116079] [] do_page_fault+0x184/0x550 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116083] [] ? pvclock_clocksource_read+0x55/0xf0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116087] [] page_fault+0x25/0x30 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116089] Mem-Info: Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116091] Node 0 DMA per-cpu: Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116094] CPU 0: hi: 0, btch: 1 usd: 0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116096] CPU 1: hi: 0, btch: 1 usd: 0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116097] Node 0 DMA32 per-cpu: Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116099] CPU 0: hi: 186, btch: 31 usd: 83 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116101] CPU 1: hi: 186, btch: 31 usd: 86 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116105] active_anon:931913 inactive_anon:1393 isolated_anon:0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116106] active_file:29 inactive_file:7 isolated_file:20 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116107] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116108] free:4048 slab_reclaimable:2861 slab_unreclaimable:2758 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116109] mapped:1427 shmem:1405 pagetables:3466 bounce:0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116111] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116119] lowmem_reserve[]: 0 3772 3772 3772 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116123] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3727652kB inactive_anon:5572kB active_file:116kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):80kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5708kB shmem:5620kB slab_reclaimable:11444kB slab_unreclaimable:11032kB kernel_stack:840kB pagetables:13864kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:313 all_unreclaimable? yes Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116132] lowmem_reserve[]: 0 0 0 0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116135] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116145] Node 0 DMA32: 1459*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7884kB Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116155] 1458 total pagecache pages Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116157] 0 pages in swap cache Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116158] Swap cache stats: add 0, delete 0, find 0/0 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116160] Free swap = 0kB Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116161] Total swap = 0kB Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121699] 985072 pages RAM Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121700] 24998 pages reserved Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121701] 3781 pages shared Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121702] 953140 pages non-shared Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121704] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121711] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121714] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121716] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121718] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121721] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121723] [ 664] 0 664 1815 123 1 0 0 dhclient3 Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121725] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121728] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121731] [ 790] 101 790 63477 569 0 0 0 rsyslogd Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121733] [ 827] 0 827 1081 36 1 0 0 acpid Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121736] [ 831] 0 831 4226 41 0 0 0 atd Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121738] [ 975] 0 975 6276 80 1 0 0 master Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121740] [ 983] 108 983 6834 106 0 0 0 qmgr Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121742] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121745] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121747] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121749] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121751] [ 1242] 107 1242 9445 163 0 0 0 ntpd Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121753] [15657] 0 15657 4777 60 0 0 0 cron Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121756] [ 654] 1012 654 70287 5737 1 0 0 php Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121758] [ 657] 1012 657 69242 6258 1 0 0 php Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121760] [ 658] 1012 658 73718 10820 1 0 0 php Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121762] [ 678] 1012 678 70152 5559 0 0 0 php Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121764] [13468] 0 13468 89584 2493 0 0 0 php5-fpm Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121767] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121769] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121771] [25666] 1012 25666 68618 5657 0 0 0 php Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121774] [23848] 1012 23848 943905 880691 0 0 0 php Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121776] [10738] 108 10738 6792 70 0 0 0 pickup Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121778] [19236] 108 19236 6818 73 0 0 0 cleanup Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121781] [19237] 108 19237 6795 71 0 0 0 trivial-rewrite Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121783] [19238] 108 19238 11107 159 1 0 0 smtp Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121785] [19245] 108 19245 11107 158 0 0 0 smtp Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121787] [19249] 108 19249 11107 158 1 0 0 smtp Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121789] [19494] 1012 19494 71025 8073 1 0 0 php Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121791] [19591] 1012 19591 1099 25 1 0 0 sh Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121793] [19592] 1012 19592 1099 24 1 0 0 sh Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121795] [19593] 1012 19593 64043 1403 0 0 0 php Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121798] [19594] 1012 19594 64040 1401 1 0 0 php Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121800] [19595] 1012 19595 1099 25 1 0 0 sh Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121802] [19596] 1012 19596 63985 1367 1 0 0 php Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121804] Out of memory: Kill process 23848 (php) score 919 or sacrifice child Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121814] Killed process 23848 (php) total-vm:3775620kB, anon-rss:3522764kB, file-rss:0kB Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868215] git invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868218] git cpuset=/ mems_allowed=0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868221] Pid: 31369, comm: git Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868223] Call Trace: Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868230] [] dump_header+0x91/0xe0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868232] [] oom_kill_process+0x85/0xb0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868234] [] out_of_memory+0xfa/0x220 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868237] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868240] [] alloc_pages_current+0xb6/0x120 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868244] [] __page_cache_alloc+0xb7/0xd0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868246] [] filemap_fault+0x212/0x3c0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868249] [] __do_fault+0x72/0x550 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868253] [] ? error_exit+0x2a/0x60 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868256] [] ? retint_restore_args+0x5/0x6 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868258] [] ? __do_fault+0x1/0x550 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868260] [] handle_pte_fault+0xfa/0x200 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868264] [] ? xen_pmd_val+0xe/0x10 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868266] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868269] [] handle_mm_fault+0x269/0x370 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868271] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868273] [] ? check_events+0x12/0x20 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868276] [] do_page_fault+0x184/0x550 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868278] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868281] [] ? putname+0x35/0x50 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868284] [] ? release_tgcred.isra.4+0x30/0x30 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868287] [] ? xen_restore_fl_direct_reloc+0x4/0x4 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868290] [] ? __call_rcu+0x9d/0x170 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868293] [] ? mntput_no_expire+0x30/0xf0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868295] [] ? call_rcu_sched+0x15/0x20 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868297] [] ? __put_cred+0x3f/0x50 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868300] [] ? sys_faccessat+0x114/0x1e0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868302] [] page_fault+0x25/0x30 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868304] Mem-Info: Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868305] Node 0 DMA per-cpu: Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868307] CPU 0: hi: 0, btch: 1 usd: 0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868308] CPU 1: hi: 0, btch: 1 usd: 0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868309] Node 0 DMA32 per-cpu: Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868311] CPU 0: hi: 186, btch: 31 usd: 30 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868312] CPU 1: hi: 186, btch: 31 usd: 27 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868315] active_anon:931953 inactive_anon:1393 isolated_anon:0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868316] active_file:0 inactive_file:21 isolated_file:0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868316] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868317] free:4029 slab_reclaimable:3105 slab_unreclaimable:2754 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868318] mapped:1359 shmem:1405 pagetables:3253 bounce:0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868319] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868326] lowmem_reserve[]: 0 3772 3772 3772 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868329] Node 0 DMA32 free:7780kB min:7848kB low:9808kB high:11772kB active_anon:3727812kB inactive_anon:5572kB active_file:0kB inactive_file:84kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5436kB shmem:5620kB slab_reclaimable:12420kB slab_unreclaimable:11016kB kernel_stack:808kB pagetables:13012kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? yes Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868335] lowmem_reserve[]: 0 0 0 0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868338] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868344] Node 0 DMA32: 1419*4kB 3*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7780kB Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868351] 1457 total pagecache pages Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868352] 0 pages in swap cache Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868353] Swap cache stats: add 0, delete 0, find 0/0 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868355] Free swap = 0kB Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868356] Total swap = 0kB Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873783] 985072 pages RAM Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873785] 24998 pages reserved Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873787] 3805 pages shared Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873788] 953284 pages non-shared Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873789] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873798] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873802] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873805] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873808] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873811] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873814] [ 664] 0 664 1815 123 1 0 0 dhclient3 Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873818] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873821] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873825] [ 790] 101 790 63477 569 0 0 0 rsyslogd Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873828] [ 827] 0 827 1081 36 1 0 0 acpid Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873831] [ 831] 0 831 4226 41 0 0 0 atd Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873834] [ 975] 0 975 6276 80 0 0 0 master Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873837] [ 983] 108 983 6834 106 1 0 0 qmgr Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873840] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873844] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873847] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873850] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873853] [ 1242] 107 1242 9445 163 0 0 0 ntpd Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873856] [15657] 0 15657 4777 60 0 0 0 cron Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873859] [ 654] 1012 654 70287 5738 1 0 0 php Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873863] [ 657] 1012 657 69242 6258 1 0 0 php Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873866] [ 658] 1012 658 73782 10849 1 0 0 php Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873869] [ 678] 1012 678 70152 5563 1 0 0 php Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873872] [13468] 0 13468 89584 2510 0 0 0 php5-fpm Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873875] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873878] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873881] [22638] 1012 22638 68618 5647 0 0 0 php Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873884] [11638] 1012 11638 950168 886834 0 0 0 php Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873887] [20820] 108 20820 6792 69 0 0 0 pickup Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873891] [26906] 108 26906 6818 73 0 0 0 cleanup Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873894] [26907] 108 26907 6795 71 0 0 0 trivial-rewrite Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873897] [26908] 108 26908 11107 161 1 0 0 smtp Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873900] [26916] 108 26916 11107 159 0 0 0 smtp Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873903] [28599] 108 28599 6792 70 0 0 0 scache Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873906] [28614] 108 28614 11108 160 1 0 0 smtp Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873910] [28623] 108 28623 11108 159 0 0 0 smtp Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873913] [31365] 1012 31365 1099 24 1 0 0 sh Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873916] [31366] 1012 31366 68825 5847 1 0 0 php Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873919] [31368] 1012 31368 1099 26 1 0 0 sh Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873922] [31369] 1012 31369 5065 64 1 0 0 git Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873924] Out of memory: Kill process 11638 (php) score 925 or sacrifice child Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873935] Killed process 11638 (php) total-vm:3800672kB, anon-rss:3547336kB, file-rss:0kB Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268130] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268135] php cpuset=/ mems_allowed=0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268138] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268140] Call Trace: Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268148] [] dump_header+0x91/0xe0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268151] [] oom_kill_process+0x85/0xb0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268154] [] out_of_memory+0xfa/0x220 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268158] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268162] [] alloc_pages_current+0xb6/0x120 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268166] [] __page_cache_alloc+0xb7/0xd0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268169] [] filemap_fault+0x212/0x3c0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268173] [] __do_fault+0x72/0x550 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268177] [] handle_pte_fault+0xfa/0x200 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268181] [] ? xen_pmd_val+0xe/0x10 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268184] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268187] [] handle_mm_fault+0x269/0x370 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268191] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268194] [] ? check_events+0x12/0x20 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268198] [] do_page_fault+0x184/0x550 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268202] [] ? do_nanosleep+0xa4/0xd0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268205] [] ? __switch_to+0x288/0x360 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268209] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268212] [] ? update_rmtp+0x70/0x70 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268214] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268217] [] page_fault+0x25/0x30 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268219] Mem-Info: Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268220] Node 0 DMA per-cpu: Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268222] CPU 0: hi: 0, btch: 1 usd: 0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268223] CPU 1: hi: 0, btch: 1 usd: 0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268224] Node 0 DMA32 per-cpu: Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268226] CPU 0: hi: 186, btch: 31 usd: 0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268227] CPU 1: hi: 186, btch: 31 usd: 0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268230] active_anon:932800 inactive_anon:1393 isolated_anon:0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268230] active_file:15 inactive_file:24 isolated_file:0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268231] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268232] free:4048 slab_reclaimable:2595 slab_unreclaimable:2719 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268233] mapped:1386 shmem:1405 pagetables:3138 bounce:0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268234] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268241] lowmem_reserve[]: 0 3772 3772 3772 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268244] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731200kB inactive_anon:5572kB active_file:60kB inactive_file:96kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5544kB shmem:5620kB slab_reclaimable:10380kB slab_unreclaimable:10876kB kernel_stack:736kB pagetables:12552kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:6338 all_unreclaimable? yes Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268250] lowmem_reserve[]: 0 0 0 0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268253] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268260] Node 0 DMA32: 1456*4kB 10*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7968kB Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268266] 1445 total pagecache pages Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268267] 0 pages in swap cache Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268269] Swap cache stats: add 0, delete 0, find 0/0 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268270] Free swap = 0kB Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268271] Total swap = 0kB Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273575] 985072 pages RAM Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273578] 24998 pages reserved Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273579] 3802 pages shared Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273580] 953306 pages non-shared Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273581] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273591] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273594] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273597] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273600] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273603] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273607] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273610] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273613] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273617] [ 790] 101 790 63477 569 0 0 0 rsyslogd Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273621] [ 827] 0 827 1081 36 1 0 0 acpid Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273624] [ 831] 0 831 4226 41 0 0 0 atd Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273627] [ 975] 0 975 6276 80 0 0 0 master Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273631] [ 983] 108 983 6834 106 1 0 0 qmgr Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273634] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273637] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273640] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273643] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273646] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273649] [15657] 0 15657 4777 60 0 0 0 cron Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273652] [ 654] 1012 654 70287 5738 1 0 0 php Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273655] [ 657] 1012 657 69242 6257 0 0 0 php Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273658] [ 658] 1012 658 73782 10849 1 0 0 php Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273661] [ 678] 1012 678 70152 5562 1 0 0 php Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273664] [13468] 0 13468 89584 2535 1 0 0 php5-fpm Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273668] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273671] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273674] [22638] 1012 22638 68618 5647 1 0 0 php Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273677] [24070] 108 24070 6792 70 1 0 0 pickup Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273680] [10322] 1012 10322 946534 883279 1 0 0 php Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273685] [16803] 1012 16803 68580 5605 1 0 0 php Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273688] [16820] 1012 16820 68580 5604 1 0 0 php Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273691] Out of memory: Kill process 10322 (php) score 921 or sacrifice child Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273702] Killed process 10322 (php) total-vm:3786136kB, anon-rss:3533028kB, file-rss:88kB Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908326] rsyslogd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908331] rsyslogd cpuset=/ mems_allowed=0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908334] Pid: 790, comm: rsyslogd Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908336] Call Trace: Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908344] [] dump_header+0x91/0xe0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908347] [] oom_kill_process+0x85/0xb0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908350] [] out_of_memory+0xfa/0x220 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908354] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908361] [] ? noalloc_get_block_write+0x30/0x30 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908372] [] alloc_pages_current+0xb6/0x120 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908378] [] __page_cache_alloc+0xb7/0xd0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908381] [] filemap_fault+0x212/0x3c0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908385] [] __do_fault+0x72/0x550 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908388] [] handle_pte_fault+0xfa/0x200 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908391] [] ? xen_pmd_val+0xe/0x10 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908394] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908398] [] handle_mm_fault+0x269/0x370 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908403] [] ? error_exit+0x2a/0x60 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908406] [] ? retint_restore_args+0x5/0x6 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908409] [] do_page_fault+0x184/0x550 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908414] [] ? ktime_get_ts+0xad/0xe0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908418] [] ? poll_select_copy_remaining+0xed/0x140 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908421] [] ? sys_select+0xcd/0x100 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908424] [] page_fault+0x25/0x30 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908427] Mem-Info: Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908428] Node 0 DMA per-cpu: Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908431] CPU 0: hi: 0, btch: 1 usd: 0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908433] CPU 1: hi: 0, btch: 1 usd: 0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908436] Node 0 DMA32 per-cpu: Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908438] CPU 0: hi: 186, btch: 31 usd: 0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908440] CPU 1: hi: 186, btch: 31 usd: 4 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908444] active_anon:932453 inactive_anon:1393 isolated_anon:0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908445] active_file:0 inactive_file:17 isolated_file:5 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908446] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908447] free:4048 slab_reclaimable:2953 slab_unreclaimable:2690 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908448] mapped:1366 shmem:1405 pagetables:3170 bounce:0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908450] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908458] lowmem_reserve[]: 0 3772 3772 3772 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908462] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729812kB inactive_anon:5572kB active_file:0kB inactive_file:68kB unevictable:0kB isolated(anon):0kB isolated(file):20kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:11812kB slab_unreclaimable:10760kB kernel_stack:744kB pagetables:12680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:111 all_unreclaimable? yes Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908471] lowmem_reserve[]: 0 0 0 0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908474] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908484] Node 0 DMA32: 1448*4kB 13*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7960kB Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908493] 1443 total pagecache pages Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908494] 0 pages in swap cache Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908496] Swap cache stats: add 0, delete 0, find 0/0 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908498] Free swap = 0kB Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908499] Total swap = 0kB Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913975] 985072 pages RAM Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913977] 24998 pages reserved Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913978] 3777 pages shared Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913980] 953328 pages non-shared Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913982] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913990] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913994] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913996] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913999] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914002] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914005] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914008] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914012] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914015] [ 790] 101 790 63477 583 0 0 0 rsyslogd Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914018] [ 827] 0 827 1081 36 1 0 0 acpid Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914021] [ 831] 0 831 4226 41 0 0 0 atd Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914024] [ 975] 0 975 6276 80 0 0 0 master Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914027] [ 983] 108 983 6834 106 1 0 0 qmgr Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914030] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914033] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914036] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914039] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914042] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914046] [15657] 0 15657 4777 60 1 0 0 cron Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914049] [ 654] 1012 654 70287 5737 1 0 0 php Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914052] [ 657] 1012 657 69242 6258 0 0 0 php Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914055] [ 658] 1012 658 73782 10849 0 0 0 php Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914058] [ 678] 1012 678 70152 5562 1 0 0 php Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914061] [13468] 0 13468 89584 2504 1 0 0 php5-fpm Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914064] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914067] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914070] [22638] 1012 22638 68682 5692 1 0 0 php Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914073] [23445] 1012 23445 945905 882582 1 0 0 php Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914076] [31951] 108 31951 6792 69 1 0 0 pickup Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914079] [ 8726] 1012 8726 68580 5603 1 0 0 php Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914082] [ 8728] 1012 8728 1099 24 0 0 0 sh Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914085] [ 8729] 1012 8729 68825 5871 1 0 0 php Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914088] Out of memory: Kill process 23445 (php) score 921 or sacrifice child Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914099] Killed process 23445 (php) total-vm:3783620kB, anon-rss:3530328kB, file-rss:0kB Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942840] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942844] php cpuset=/ mems_allowed=0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942846] Pid: 18876, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942848] Call Trace: Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942855] [] dump_header+0x91/0xe0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942857] [] oom_kill_process+0x85/0xb0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942860] [] out_of_memory+0xfa/0x220 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942863] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942866] [] alloc_pages_current+0xb6/0x120 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942870] [] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942873] [] __page_cache_alloc+0xb7/0xd0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942875] [] filemap_fault+0x212/0x3c0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942878] [] __do_fault+0x72/0x550 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942881] [] ? load_TLS_descriptor+0x42/0xd0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942884] [] handle_pte_fault+0xfa/0x200 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942886] [] ? xen_pmd_val+0xe/0x10 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942888] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942891] [] handle_mm_fault+0x269/0x370 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942895] [] ? lock_hrtimer_base.isra.25+0x30/0x60 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942897] [] do_page_fault+0x184/0x550 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942900] [] ? do_nanosleep+0xa4/0xd0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942903] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942906] [] ? update_rmtp+0x70/0x70 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942908] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942911] [] page_fault+0x25/0x30 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942913] Mem-Info: Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942914] Node 0 DMA per-cpu: Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942916] CPU 0: hi: 0, btch: 1 usd: 0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942918] CPU 1: hi: 0, btch: 1 usd: 0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942919] Node 0 DMA32 per-cpu: Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942920] CPU 0: hi: 186, btch: 31 usd: 50 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942922] CPU 1: hi: 186, btch: 31 usd: 37 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942924] active_anon:932466 inactive_anon:1393 isolated_anon:0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942925] active_file:43 inactive_file:6 isolated_file:12 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942926] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942926] free:4034 slab_reclaimable:2934 slab_unreclaimable:2628 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942927] mapped:1342 shmem:1405 pagetables:3160 bounce:0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942929] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942935] lowmem_reserve[]: 0 3772 3772 3772 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942938] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3729864kB inactive_anon:5572kB active_file:172kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):48kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5368kB shmem:5620kB slab_reclaimable:11736kB slab_unreclaimable:10512kB kernel_stack:760kB pagetables:12640kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:381 all_unreclaimable? yes Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942945] lowmem_reserve[]: 0 0 0 0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942947] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942954] Node 0 DMA32: 1428*4kB 3*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7800kB Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942961] 1418 total pagecache pages Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942963] 0 pages in swap cache Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942964] Swap cache stats: add 0, delete 0, find 0/0 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942965] Free swap = 0kB Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942966] Total swap = 0kB Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948128] 985072 pages RAM Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948130] 24998 pages reserved Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948131] 3775 pages shared Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948133] 953285 pages non-shared Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948134] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948144] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948148] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948151] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948154] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948157] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948161] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948164] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948167] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948171] [ 790] 101 790 63477 583 1 0 0 rsyslogd Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948174] [ 827] 0 827 1081 36 1 0 0 acpid Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948178] [ 831] 0 831 4226 41 0 0 0 atd Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948181] [ 975] 0 975 6276 80 0 0 0 master Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948184] [ 983] 108 983 6834 106 0 0 0 qmgr Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948187] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948190] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948193] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948196] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948199] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948203] [15657] 0 15657 4777 60 1 0 0 cron Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948206] [ 654] 1012 654 70287 5737 1 0 0 php Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948209] [ 657] 1012 657 69242 6258 1 0 0 php Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948212] [ 658] 1012 658 76918 14029 1 0 0 php Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948215] [ 678] 1012 678 70152 5562 0 0 0 php Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948218] [13468] 0 13468 89584 2508 0 0 0 php5-fpm Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948221] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948224] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948227] [22638] 1012 22638 68682 5692 1 0 0 php Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948230] [16072] 1012 16072 948043 884849 0 0 0 php Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948234] [ 4842] 108 4842 6792 69 0 0 0 pickup Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948237] [18805] 108 18805 6818 73 0 0 0 cleanup Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948240] [18806] 108 18806 6795 70 1 0 0 trivial-rewrite Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948243] [18807] 108 18807 11107 158 0 0 0 smtp Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948246] [18811] 108 18811 11107 158 0 0 0 smtp Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948249] [18876] 1012 18876 68580 5604 1 0 0 php Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948252] Out of memory: Kill process 16072 (php) score 923 or sacrifice child Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948264] Killed process 16072 (php) total-vm:3792172kB, anon-rss:3539396kB, file-rss:0kB Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249004] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249009] php cpuset=/ mems_allowed=0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249012] Pid: 13997, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249014] Call Trace: Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249021] [] dump_header+0x91/0xe0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249023] [] oom_kill_process+0x85/0xb0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249025] [] out_of_memory+0xfa/0x220 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249028] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249032] [] alloc_pages_current+0xb6/0x120 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249035] [] __page_cache_alloc+0xb7/0xd0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249037] [] filemap_fault+0x212/0x3c0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249040] [] __do_fault+0x72/0x550 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249043] [] handle_pte_fault+0xfa/0x200 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249046] [] ? xen_pmd_val+0xe/0x10 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249048] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249051] [] handle_mm_fault+0x269/0x370 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249054] [] do_page_fault+0x184/0x550 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249058] [] ? sys_newlstat+0x2a/0x40 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249061] [] page_fault+0x25/0x30 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249062] Mem-Info: Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249064] Node 0 DMA per-cpu: Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249065] CPU 0: hi: 0, btch: 1 usd: 0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249067] CPU 1: hi: 0, btch: 1 usd: 0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249068] Node 0 DMA32 per-cpu: Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249070] CPU 0: hi: 186, btch: 31 usd: 30 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249071] CPU 1: hi: 186, btch: 31 usd: 157 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249073] active_anon:932573 inactive_anon:1393 isolated_anon:0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249074] active_file:12 inactive_file:13 isolated_file:0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249075] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249075] free:4036 slab_reclaimable:2701 slab_unreclaimable:2605 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249076] mapped:1362 shmem:1405 pagetables:3228 bounce:0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249078] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249084] lowmem_reserve[]: 0 3772 3772 3772 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249087] Node 0 DMA32 free:7808kB min:7848kB low:9808kB high:11772kB active_anon:3730292kB inactive_anon:5572kB active_file:48kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5620kB slab_reclaimable:10804kB slab_unreclaimable:10420kB kernel_stack:776kB pagetables:12912kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:329 all_unreclaimable? yes Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249094] lowmem_reserve[]: 0 0 0 0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249096] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249103] Node 0 DMA32: 1064*4kB 188*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7808kB Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249110] 1461 total pagecache pages Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249111] 0 pages in swap cache Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249112] Swap cache stats: add 0, delete 0, find 0/0 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249113] Free swap = 0kB Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249114] Total swap = 0kB Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254324] 985072 pages RAM Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254326] 24998 pages reserved Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254327] 3793 pages shared Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254328] 953169 pages non-shared Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254330] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254339] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254343] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254346] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254349] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254352] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254356] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254359] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254362] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254366] [ 790] 101 790 63477 583 0 0 0 rsyslogd Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254369] [ 827] 0 827 1081 36 1 0 0 acpid Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254372] [ 831] 0 831 4226 41 0 0 0 atd Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254376] [ 975] 0 975 6276 80 0 0 0 master Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254379] [ 983] 108 983 6834 106 0 0 0 qmgr Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254382] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254385] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254388] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254391] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254394] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254397] [15657] 0 15657 4777 60 0 0 0 cron Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254401] [ 654] 1012 654 70287 5738 0 0 0 php Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254404] [ 657] 1012 657 69242 6258 0 0 0 php Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254407] [ 658] 1012 658 76918 14029 0 0 0 php Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254410] [ 678] 1012 678 70152 5562 0 0 0 php Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254413] [13468] 0 13468 89584 2539 0 0 0 php5-fpm Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254416] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254419] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254422] [22638] 1012 22638 68682 5692 1 0 0 php Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254426] [19154] 1012 19154 950704 887465 1 0 0 php Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254428] [24159] 108 24159 6792 69 0 0 0 pickup Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254432] [13866] 108 13866 6818 73 0 0 0 cleanup Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254435] [13867] 108 13867 6795 70 0 0 0 trivial-rewrite Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254438] [13868] 108 13868 11107 157 1 0 0 smtp Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254442] [13994] 1012 13994 1099 24 0 0 0 sh Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254444] [13995] 1012 13995 1099 26 0 0 0 sh Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254448] [13996] 1012 13996 64213 1627 1 0 0 php Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254450] [13997] 1012 13997 64213 1581 0 0 0 php Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254453] Out of memory: Kill process 19154 (php) score 926 or sacrifice child Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254465] Killed process 19154 (php) total-vm:3802816kB, anon-rss:3549860kB, file-rss:0kB Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164121] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164125] php cpuset=/ mems_allowed=0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164129] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164131] Call Trace: Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164139] [] dump_header+0x91/0xe0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164142] [] oom_kill_process+0x85/0xb0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164145] [] out_of_memory+0xfa/0x220 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164148] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164153] [] ? noalloc_get_block_write+0x30/0x30 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164155] [] alloc_pages_current+0xb6/0x120 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164159] [] __page_cache_alloc+0xb7/0xd0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164161] [] filemap_fault+0x212/0x3c0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164164] [] __do_fault+0x72/0x550 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164167] [] handle_pte_fault+0xfa/0x200 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164170] [] ? xen_pmd_val+0xe/0x10 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164172] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164175] [] handle_mm_fault+0x269/0x370 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164177] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164180] [] ? check_events+0x12/0x20 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164183] [] do_page_fault+0x184/0x550 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164186] [] ? do_nanosleep+0xa4/0xd0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164190] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164193] [] ? update_rmtp+0x70/0x70 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164196] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164198] [] page_fault+0x25/0x30 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164200] Mem-Info: Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164201] Node 0 DMA per-cpu: Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164203] CPU 0: hi: 0, btch: 1 usd: 0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164204] CPU 1: hi: 0, btch: 1 usd: 0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164205] Node 0 DMA32 per-cpu: Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164207] CPU 0: hi: 186, btch: 31 usd: 0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164208] CPU 1: hi: 186, btch: 31 usd: 0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164211] active_anon:932667 inactive_anon:1393 isolated_anon:0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164212] active_file:0 inactive_file:0 isolated_file:0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164212] unevictable:0 dirty:10 writeback:0 unstable:0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164213] free:4048 slab_reclaimable:2816 slab_unreclaimable:2700 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164214] mapped:1375 shmem:1405 pagetables:3051 bounce:0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164215] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164222] lowmem_reserve[]: 0 3772 3772 3772 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164225] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730668kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:40kB writeback:0kB mapped:5500kB shmem:5620kB slab_reclaimable:11264kB slab_unreclaimable:10800kB kernel_stack:728kB pagetables:12204kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164231] lowmem_reserve[]: 0 0 0 0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164234] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164241] Node 0 DMA32: 1469*4kB 2*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7940kB Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164247] 1432 total pagecache pages Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164248] 0 pages in swap cache Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164249] Swap cache stats: add 0, delete 0, find 0/0 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164251] Free swap = 0kB Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164251] Total swap = 0kB Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169387] 985072 pages RAM Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169390] 24998 pages reserved Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169391] 3800 pages shared Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169392] 953316 pages non-shared Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169394] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169403] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169406] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169409] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169412] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169416] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169419] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169422] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169425] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169429] [ 790] 101 790 63477 583 1 0 0 rsyslogd Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169433] [ 827] 0 827 1081 36 1 0 0 acpid Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169436] [ 831] 0 831 4226 41 0 0 0 atd Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169439] [ 975] 0 975 6276 80 0 0 0 master Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169442] [ 983] 108 983 6834 106 0 0 0 qmgr Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169445] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169448] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169451] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169455] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169458] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169461] [15657] 0 15657 4777 60 0 0 0 cron Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169464] [ 654] 1012 654 70287 5738 1 0 0 php Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169467] [ 657] 1012 657 69242 6257 0 0 0 php Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169470] [ 658] 1012 658 135801 72207 1 0 0 php Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169473] [ 678] 1012 678 70152 5562 1 0 0 php Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169476] [13468] 0 13468 89584 2524 0 0 0 php5-fpm Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169479] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169482] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169485] [22638] 1012 22638 68682 5692 1 0 0 php Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169489] [27214] 1012 27214 890524 827357 1 0 0 php Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169492] [ 2211] 108 2211 6792 69 0 0 0 pickup Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169495] [ 6395] 1012 6395 68580 5605 1 0 0 php Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169498] Out of memory: Kill process 27214 (php) score 863 or sacrifice child Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169509] Killed process 27214 (php) total-vm:3562096kB, anon-rss:3309428kB, file-rss:0kB Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805400] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805405] php cpuset=/ mems_allowed=0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805408] Pid: 6731, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805410] Call Trace: Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805418] [] dump_header+0x91/0xe0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805421] [] oom_kill_process+0x85/0xb0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805424] [] out_of_memory+0xfa/0x220 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805428] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805434] [] ? noalloc_get_block_write+0x30/0x30 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805438] [] alloc_pages_current+0xb6/0x120 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805442] [] __page_cache_alloc+0xb7/0xd0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805445] [] filemap_fault+0x212/0x3c0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805449] [] __do_fault+0x72/0x550 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805453] [] ? lru_cache_add_lru+0x2d/0x50 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805456] [] handle_pte_fault+0xfa/0x200 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805460] [] ? xen_pmd_val+0xe/0x10 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805463] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805467] [] handle_mm_fault+0x269/0x370 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805472] [] do_page_fault+0x184/0x550 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805479] [] ? do_brk+0x21f/0x320 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805483] [] ? __schedule+0x3cc/0x6f0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805486] [] page_fault+0x25/0x30 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805489] Mem-Info: Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805490] Node 0 DMA per-cpu: Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805492] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805494] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805495] Node 0 DMA32 per-cpu: Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805498] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805499] CPU 1: hi: 186, btch: 31 usd: 0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805503] active_anon:932829 inactive_anon:1393 isolated_anon:0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805504] active_file:2 inactive_file:0 isolated_file:0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805505] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805506] free:4048 slab_reclaimable:2746 slab_unreclaimable:2648 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805507] mapped:1374 shmem:1405 pagetables:3071 bounce:0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805509] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805517] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805521] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731316kB inactive_anon:5572kB active_file:8kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5496kB shmem:5620kB slab_reclaimable:10984kB slab_unreclaimable:10592kB kernel_stack:728kB pagetables:12284kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:24 all_unreclaimable? yes Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805529] lowmem_reserve[]: 0 0 0 0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805533] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805542] Node 0 DMA32: 1451*4kB 6*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7900kB Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805552] 1419 total pagecache pages Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805553] 0 pages in swap cache Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805555] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805557] Free swap = 0kB Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805558] Total swap = 0kB Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810902] 985072 pages RAM Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810905] 24998 pages reserved Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810906] 3788 pages shared Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810907] 953326 pages non-shared Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810909] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810918] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810921] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810924] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810927] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810930] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810934] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810937] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810941] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810944] [ 790] 101 790 63477 583 0 0 0 rsyslogd Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810948] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810951] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810954] [ 975] 0 975 6276 80 1 0 0 master Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810957] [ 983] 108 983 6834 106 1 0 0 qmgr Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810960] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810963] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810967] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810970] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810973] [ 1242] 107 1242 9445 163 0 0 0 ntpd Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810976] [15657] 0 15657 4777 60 0 0 0 cron Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810980] [ 654] 1012 654 70287 5738 0 0 0 php Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810982] [ 657] 1012 657 69242 6258 0 0 0 php Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810985] [ 658] 1012 658 135801 72207 0 0 0 php Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810988] [ 678] 1012 678 70152 5566 0 0 0 php Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810992] [13468] 0 13468 89584 2509 0 0 0 php5-fpm Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810995] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810998] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.811001] [ 6731] 1012 6731 890805 827560 1 0 0 php Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.811004] [ 2882] 108 2882 6792 70 1 0 0 pickup Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.811007] [ 8112] 1012 8112 68610 5635 1 0 0 php Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.811010] [ 9092] 1012 9092 68580 5605 1 0 0 php Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.811013] Out of memory: Kill process 6731 (php) score 863 or sacrifice child Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.811023] Killed process 6731 (php) total-vm:3563220kB, anon-rss:3310240kB, file-rss:0kB Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212160] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212164] php cpuset=/ mems_allowed=0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212167] Pid: 10012, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212169] Call Trace: Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212175] [] dump_header+0x91/0xe0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212178] [] oom_kill_process+0x85/0xb0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212180] [] out_of_memory+0xfa/0x220 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212183] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212186] [] alloc_pages_vma+0xb3/0x190 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212190] [] do_anonymous_page.isra.38+0x7c/0x2f0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212192] [] handle_pte_fault+0x1e1/0x200 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212195] [] ? xen_pmd_val+0xe/0x10 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212198] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212200] [] handle_mm_fault+0x269/0x370 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212203] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212205] [] ? check_events+0x12/0x20 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212208] [] do_page_fault+0x184/0x550 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212211] [] ? do_brk+0x21f/0x320 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212214] [] ? __schedule+0x3cc/0x6f0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212216] [] page_fault+0x25/0x30 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212218] Mem-Info: Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212219] Node 0 DMA per-cpu: Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212221] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212223] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212224] Node 0 DMA32 per-cpu: Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212226] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212227] CPU 1: hi: 186, btch: 31 usd: 0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212229] active_anon:932397 inactive_anon:1393 isolated_anon:0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212230] active_file:0 inactive_file:0 isolated_file:0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212231] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212231] free:4048 slab_reclaimable:2921 slab_unreclaimable:2714 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212232] mapped:1384 shmem:1405 pagetables:3211 bounce:0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212234] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212240] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212243] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729588kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5536kB shmem:5620kB slab_reclaimable:11684kB slab_unreclaimable:10856kB kernel_stack:784kB pagetables:12844kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212250] lowmem_reserve[]: 0 0 0 0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212252] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212259] Node 0 DMA32: 1462*4kB 7*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212265] 1437 total pagecache pages Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212267] 0 pages in swap cache Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212268] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212269] Free swap = 0kB Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212270] Total swap = 0kB Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217372] 985072 pages RAM Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217375] 24998 pages reserved Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217376] 3810 pages shared Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217377] 953296 pages non-shared Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217379] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217388] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217392] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217395] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217398] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217401] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217404] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217408] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217411] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217415] [ 790] 101 790 63477 599 0 0 0 rsyslogd Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217418] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217421] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217424] [ 975] 0 975 6276 80 0 0 0 master Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217427] [ 983] 108 983 6834 106 0 0 0 qmgr Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217430] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217434] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217437] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217440] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217443] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217446] [15657] 0 15657 4777 60 0 0 0 cron Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217449] [ 654] 1012 654 70287 5738 0 0 0 php Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217452] [ 657] 1012 657 69242 6257 0 0 0 php Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217455] [ 658] 1012 658 135801 72207 0 0 0 php Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217458] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217462] [13468] 0 13468 89584 2494 0 0 0 php5-fpm Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217465] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217468] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217471] [ 8112] 1012 8112 68610 5642 0 0 0 php Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217474] [10012] 1012 10012 883193 820220 0 0 0 php Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217477] [ 9949] 108 9949 6792 69 0 0 0 scache Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217480] [10950] 108 10950 6818 72 0 0 0 cleanup Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217483] [10951] 108 10951 6795 71 0 0 0 trivial-rewrite Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217486] [10952] 108 10952 11107 161 0 0 0 smtp Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217490] [10956] 108 10956 11107 161 0 0 0 smtp Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217493] [10960] 108 10960 11107 160 0 0 0 smtp Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217496] [11402] 108 11402 11107 160 0 0 0 smtp Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217499] [11418] 108 11418 11108 159 0 0 0 smtp Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217502] [11732] 1012 11732 74525 11582 0 0 0 php Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217505] Out of memory: Kill process 10012 (php) score 856 or sacrifice child Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217516] Killed process 10012 (php) total-vm:3532772kB, anon-rss:3280792kB, file-rss:88kB Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500118] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500122] php cpuset=/ mems_allowed=0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500124] Pid: 4382, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500126] Call Trace: Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500133] [] dump_header+0x91/0xe0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500135] [] oom_kill_process+0x85/0xb0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500137] [] out_of_memory+0xfa/0x220 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500140] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500143] [] alloc_pages_current+0xb6/0x120 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500147] [] __page_cache_alloc+0xb7/0xd0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500149] [] filemap_fault+0x212/0x3c0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500152] [] __do_fault+0x72/0x550 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500155] [] ? lru_cache_add_lru+0x2d/0x50 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500157] [] handle_pte_fault+0xfa/0x200 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500161] [] ? xen_pmd_val+0xe/0x10 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500163] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500166] [] handle_mm_fault+0x269/0x370 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500169] [] do_page_fault+0x184/0x550 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500172] [] ? do_brk+0x21f/0x320 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500175] [] ? __schedule+0x3cc/0x6f0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500177] [] page_fault+0x25/0x30 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500179] Mem-Info: Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500180] Node 0 DMA per-cpu: Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500182] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500184] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500185] Node 0 DMA32 per-cpu: Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500186] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500187] CPU 1: hi: 186, btch: 31 usd: 0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500190] active_anon:931818 inactive_anon:1393 isolated_anon:0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500191] active_file:0 inactive_file:0 isolated_file:0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500191] unevictable:0 dirty:11 writeback:0 unstable:0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500192] free:4048 slab_reclaimable:3320 slab_unreclaimable:2783 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500193] mapped:1395 shmem:1405 pagetables:3291 bounce:0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500194] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500201] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500204] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3727272kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:44kB writeback:0kB mapped:5580kB shmem:5620kB slab_reclaimable:13280kB slab_unreclaimable:11132kB kernel_stack:808kB pagetables:13164kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1 all_unreclaimable? yes Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500211] lowmem_reserve[]: 0 0 0 0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500213] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500220] Node 0 DMA32: 1441*4kB 6*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7860kB Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500226] 1443 total pagecache pages Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500228] 0 pages in swap cache Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500229] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500230] Free swap = 0kB Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500231] Total swap = 0kB Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505412] 985072 pages RAM Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505414] 24998 pages reserved Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505415] 4084 pages shared Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505416] 953309 pages non-shared Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505418] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505427] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505430] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505433] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505436] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505439] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505442] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505445] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505449] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505452] [ 790] 101 790 63477 599 0 0 0 rsyslogd Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505455] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505457] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505460] [ 975] 0 975 6276 88 1 0 0 master Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505463] [ 983] 108 983 6834 106 1 0 0 qmgr Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505466] [ 1024] 0 1024 3309 39 0 0 0 mdadm Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505468] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505471] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505474] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505476] [ 1242] 107 1242 9445 166 1 0 0 ntpd Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505479] [15657] 0 15657 4777 61 0 0 0 cron Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505482] [ 654] 1012 654 70287 5768 1 0 0 php Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505484] [ 657] 1012 657 69242 6282 1 0 0 php Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505487] [ 658] 1012 658 135033 71460 1 0 0 php Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505490] [ 678] 1012 678 70152 5567 1 0 0 php Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505494] [13468] 0 13468 89584 2524 1 0 0 php5-fpm Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505497] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505501] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505504] [ 8112] 1012 8112 68610 5677 1 0 0 php Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505507] [ 4382] 1012 4382 889105 825985 0 0 0 php Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505511] [ 2671] 108 2671 6792 70 0 0 0 pickup Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505514] [ 5855] 108 5855 6818 73 0 0 0 cleanup Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505517] [ 5856] 108 5856 6795 70 1 0 0 trivial-rewrite Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505519] [ 5857] 108 5857 11107 167 1 0 0 smtp Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505522] [ 5861] 108 5861 11107 169 1 0 0 smtp Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505526] [ 7595] 108 7595 11107 166 0 0 0 smtp Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505528] [ 7608] 108 7608 6792 69 0 0 0 scache Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505531] [ 7624] 108 7624 11108 167 0 0 0 smtp Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505534] [ 7630] 108 7630 11108 166 0 0 0 smtp Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505537] [ 7637] 108 7637 11108 168 0 0 0 smtp Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505541] [ 7641] 108 7641 11108 167 1 0 0 smtp Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505544] [ 7825] 1012 7825 68580 5629 1 0 0 php Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505548] Out of memory: Kill process 4382 (php) score 862 or sacrifice child Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505559] Killed process 4382 (php) total-vm:3556420kB, anon-rss:3303804kB, file-rss:124kB Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809443] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809447] php5-fpm cpuset=/ mems_allowed=0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809450] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809452] Call Trace: Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809459] [] dump_header+0x91/0xe0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809461] [] oom_kill_process+0x85/0xb0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809464] [] out_of_memory+0xfa/0x220 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809466] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809470] [] alloc_pages_current+0xb6/0x120 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809473] [] __page_cache_alloc+0xb7/0xd0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809476] [] filemap_fault+0x212/0x3c0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809478] [] __do_fault+0x72/0x550 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809482] [] ? __switch_to+0x138/0x360 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809485] [] handle_pte_fault+0xfa/0x200 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809488] [] ? xen_pmd_val+0xe/0x10 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809490] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809493] [] handle_mm_fault+0x269/0x370 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809495] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809498] [] ? check_events+0x12/0x20 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809501] [] do_page_fault+0x184/0x550 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809504] [] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809508] [] ? ep_poll+0x1cc/0x380 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809511] [] ? pvclock_clocksource_read+0x55/0xf0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809514] [] ? xen_clocksource_read+0x20/0x30 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809516] [] ? xen_clocksource_get_cycles+0x9/0x10 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809520] [] ? ktime_get_ts+0xad/0xe0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809522] [] page_fault+0x25/0x30 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809524] Mem-Info: Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809525] Node 0 DMA per-cpu: Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809527] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809528] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809529] Node 0 DMA32 per-cpu: Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809531] CPU 0: hi: 186, btch: 31 usd: 33 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809532] CPU 1: hi: 186, btch: 31 usd: 27 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809535] active_anon:932826 inactive_anon:1393 isolated_anon:0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809536] active_file:6 inactive_file:0 isolated_file:0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809536] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809537] free:4040 slab_reclaimable:2425 slab_unreclaimable:2633 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809538] mapped:1375 shmem:1405 pagetables:3298 bounce:0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809539] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809546] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809549] Node 0 DMA32 free:7824kB min:7848kB low:9808kB high:11772kB active_anon:3731304kB inactive_anon:5572kB active_file:24kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5500kB shmem:5620kB slab_reclaimable:9700kB slab_unreclaimable:10532kB kernel_stack:776kB pagetables:13192kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:275 all_unreclaimable? yes Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809555] lowmem_reserve[]: 0 0 0 0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809558] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809564] Node 0 DMA32: 1008*4kB 218*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7824kB Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809571] 1410 total pagecache pages Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809572] 0 pages in swap cache Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809573] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809574] Free swap = 0kB Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809575] Total swap = 0kB Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814948] 985072 pages RAM Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814950] 24998 pages reserved Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814952] 3775 pages shared Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814953] 953304 pages non-shared Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814955] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814963] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814966] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814970] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814973] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814976] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814979] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814982] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814986] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814989] [ 790] 101 790 63477 599 1 0 0 rsyslogd Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814993] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814996] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814999] [ 975] 0 975 6276 79 1 0 0 master Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815002] [ 983] 108 983 6834 106 1 0 0 qmgr Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815005] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815008] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815011] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815014] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815018] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815021] [15657] 0 15657 4777 59 1 0 0 cron Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815024] [ 654] 1012 654 70287 5743 1 0 0 php Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815027] [ 657] 1012 657 69242 6258 1 0 0 php Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815030] [ 658] 1012 658 135033 71439 1 0 0 php Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815033] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815036] [13468] 0 13468 89584 2528 0 0 0 php5-fpm Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815039] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815042] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815045] [ 8112] 1012 8112 68610 5652 1 0 0 php Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815049] [ 2671] 108 2671 6792 76 0 0 0 pickup Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815052] [ 7608] 108 7608 6792 69 1 0 0 scache Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815055] [ 8743] 1012 8743 884611 821520 1 0 0 php Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815058] [ 9477] 1012 9477 68580 5604 1 0 0 php Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815062] [ 9558] 1012 9558 1099 25 1 0 0 sh Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815065] [ 9559] 1012 9559 68825 5871 1 0 0 php Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815068] [ 9565] 1012 9565 1099 26 1 0 0 sh Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815071] [ 9566] 1012 9566 14040 63 1 0 0 git Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815074] [ 9567] 1012 9567 43910 713 1 0 0 php Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815076] Out of memory: Kill process 8743 (php) score 857 or sacrifice child Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815088] Killed process 8743 (php) total-vm:3538444kB, anon-rss:3286080kB, file-rss:0kB Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024285] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024289] php cpuset=/ mems_allowed=0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024292] Pid: 12196, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024294] Call Trace: Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024301] [] dump_header+0x91/0xe0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024303] [] oom_kill_process+0x85/0xb0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024305] [] out_of_memory+0xfa/0x220 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024308] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024312] [] alloc_pages_current+0xb6/0x120 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024315] [] __page_cache_alloc+0xb7/0xd0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024317] [] filemap_fault+0x212/0x3c0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024320] [] __do_fault+0x72/0x550 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024325] [] ? xen_hypervisor_callback+0x1b/0x20 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024327] [] handle_pte_fault+0xfa/0x200 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024331] [] ? xen_pmd_val+0xe/0x10 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024333] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024335] [] handle_mm_fault+0x269/0x370 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024338] [] do_page_fault+0x184/0x550 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024341] [] ? do_wait+0x12c/0x260 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024344] [] ? sys_wait4+0xab/0xf0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024347] [] page_fault+0x25/0x30 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024349] Mem-Info: Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024350] Node 0 DMA per-cpu: Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024352] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024353] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024354] Node 0 DMA32 per-cpu: Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024356] CPU 0: hi: 186, btch: 31 usd: 30 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024357] CPU 1: hi: 186, btch: 31 usd: 57 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024360] active_anon:932473 inactive_anon:1393 isolated_anon:0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024360] active_file:1 inactive_file:0 isolated_file:0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024361] unevictable:0 dirty:0 writeback:11 unstable:0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024362] free:4018 slab_reclaimable:2750 slab_unreclaimable:2649 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024362] mapped:1356 shmem:1405 pagetables:3311 bounce:0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024364] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024370] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024373] Node 0 DMA32 free:7736kB min:7848kB low:9808kB high:11772kB active_anon:3729892kB inactive_anon:5572kB active_file:4kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:44kB mapped:5424kB shmem:5620kB slab_reclaimable:11000kB slab_unreclaimable:10596kB kernel_stack:784kB pagetables:13244kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:105 all_unreclaimable? yes Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024380] lowmem_reserve[]: 0 0 0 0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024383] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024389] Node 0 DMA32: 1375*4kB 26*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7756kB Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024396] 1442 total pagecache pages Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024397] 0 pages in swap cache Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024398] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024399] Free swap = 0kB Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024400] Total swap = 0kB Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029662] 985072 pages RAM Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029664] 24998 pages reserved Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029666] 3790 pages shared Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029667] 953278 pages non-shared Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029669] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029677] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029681] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029684] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029687] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029690] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029694] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029697] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029700] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029704] [ 790] 101 790 63477 599 0 0 0 rsyslogd Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029707] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029711] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029714] [ 975] 0 975 6276 80 0 0 0 master Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029717] [ 983] 108 983 6834 106 1 0 0 qmgr Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029720] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029723] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029726] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029729] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029732] [ 1242] 107 1242 9445 163 0 0 0 ntpd Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029736] [15657] 0 15657 4777 60 0 0 0 cron Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029739] [ 654] 1012 654 70287 5743 1 0 0 php Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029742] [ 657] 1012 657 69242 6258 0 0 0 php Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029745] [ 658] 1012 658 135033 71439 0 0 0 php Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029748] [ 678] 1012 678 70152 5566 0 0 0 php Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029751] [13468] 0 13468 89584 2512 0 0 0 php5-fpm Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029754] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029757] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029760] [ 8112] 1012 8112 68610 5653 1 0 0 php Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029763] [13068] 1012 13068 879395 816196 1 0 0 php Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029767] [ 7952] 108 7952 6792 70 1 0 0 pickup Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029770] [12163] 1012 12163 68580 5604 0 0 0 php Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029773] [12195] 1012 12195 1099 25 0 0 0 sh Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029776] [12196] 1012 12196 68825 5870 0 0 0 php Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029779] [12202] 1012 12202 1099 25 0 0 0 sh Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029782] [12203] 1012 12203 4984 64 1 0 0 git Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029785] [12204] 1012 12204 68499 5551 0 0 0 php Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029788] [12206] 1012 12206 1099 26 0 0 0 sh Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029791] [12207] 1012 12207 10901 175 0 0 0 ssh Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029796] Out of memory: Kill process 13068 (php) score 851 or sacrifice child Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029807] Killed process 13068 (php) total-vm:3517580kB, anon-rss:3264784kB, file-rss:0kB Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272139] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272143] php cpuset=/ mems_allowed=0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272147] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272149] Call Trace: Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272156] [] dump_header+0x91/0xe0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272159] [] oom_kill_process+0x85/0xb0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272163] [] out_of_memory+0xfa/0x220 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272166] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272170] [] alloc_pages_current+0xb6/0x120 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272175] [] __page_cache_alloc+0xb7/0xd0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272177] [] filemap_fault+0x212/0x3c0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272181] [] __do_fault+0x72/0x550 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272186] [] ? error_exit+0x2a/0x60 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272189] [] ? retint_restore_args+0x5/0x6 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272193] [] handle_pte_fault+0xfa/0x200 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272197] [] ? xen_pmd_val+0xe/0x10 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272200] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272203] [] handle_mm_fault+0x269/0x370 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272206] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272210] [] ? check_events+0x12/0x20 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272213] [] do_page_fault+0x184/0x550 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272215] [] ? xen_mc_flush+0xb2/0x1c0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272217] [] ? xen_clts+0x8d/0x190 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272219] [] page_fault+0x25/0x30 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272221] Mem-Info: Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272222] Node 0 DMA per-cpu: Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272224] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272225] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272226] Node 0 DMA32 per-cpu: Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272228] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272229] CPU 1: hi: 186, btch: 31 usd: 0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272232] active_anon:932347 inactive_anon:1393 isolated_anon:0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272233] active_file:0 inactive_file:11 isolated_file:0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272233] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272234] free:4048 slab_reclaimable:3045 slab_unreclaimable:2706 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272235] mapped:1354 shmem:1405 pagetables:3131 bounce:0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272236] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272242] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272245] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729388kB inactive_anon:5572kB active_file:0kB inactive_file:44kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5620kB slab_reclaimable:12180kB slab_unreclaimable:10824kB kernel_stack:752kB pagetables:12524kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10139 all_unreclaimable? yes Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272252] lowmem_reserve[]: 0 0 0 0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272254] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272261] Node 0 DMA32: 1467*4kB 8*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8012kB Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272268] 1470 total pagecache pages Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272269] 0 pages in swap cache Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272270] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272271] Free swap = 0kB Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272272] Total swap = 0kB Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277415] 985072 pages RAM Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277417] 24998 pages reserved Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277418] 3796 pages shared Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277419] 953299 pages non-shared Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277421] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277430] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277433] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277436] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277439] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277443] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277446] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277449] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277453] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277457] [ 790] 101 790 63477 599 0 0 0 rsyslogd Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277460] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277463] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277466] [ 975] 0 975 6276 80 0 0 0 master Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277469] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277472] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277475] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277478] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277481] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277484] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277488] [15657] 0 15657 4777 60 0 0 0 cron Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277491] [ 654] 1012 654 70287 5743 1 0 0 php Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277494] [ 657] 1012 657 69242 6257 0 0 0 php Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277497] [ 658] 1012 658 135033 71439 0 0 0 php Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277500] [ 678] 1012 678 70152 5566 0 0 0 php Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277503] [13468] 0 13468 89584 2504 0 0 0 php5-fpm Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277506] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277509] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277513] [ 8112] 1012 8112 68610 5660 0 0 0 php Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277516] [31572] 108 31572 6792 70 1 0 0 pickup Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277519] [ 9497] 1012 9497 890579 827488 1 0 0 php Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277522] [ 9595] 108 9595 6818 73 0 0 0 cleanup Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277526] [ 9596] 108 9596 6795 70 0 0 0 trivial-rewrite Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277529] [ 9597] 108 9597 11107 158 1 0 0 smtp Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277532] [10121] 1012 10121 68580 5604 0 0 0 php Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277535] Out of memory: Kill process 9497 (php) score 863 or sacrifice child Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277545] Killed process 9497 (php) total-vm:3562316kB, anon-rss:3309928kB, file-rss:24kB Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918815] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918820] php cpuset=/ mems_allowed=0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918823] Pid: 11880, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918825] Call Trace: Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918833] [] dump_header+0x91/0xe0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918836] [] oom_kill_process+0x85/0xb0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918839] [] out_of_memory+0xfa/0x220 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918843] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918848] [] ? error_exit+0x2a/0x60 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918852] [] alloc_pages_current+0xb6/0x120 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918857] [] __page_cache_alloc+0xb7/0xd0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918860] [] filemap_fault+0x212/0x3c0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918864] [] __do_fault+0x72/0x550 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918868] [] ? xen_hypervisor_callback+0x1b/0x20 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918872] [] handle_pte_fault+0xfa/0x200 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918875] [] ? xen_pmd_val+0xe/0x10 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918879] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918886] [] handle_mm_fault+0x269/0x370 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918890] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918893] [] ? check_events+0x12/0x20 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918897] [] do_page_fault+0x184/0x550 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918900] [] ? do_nanosleep+0xa4/0xd0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918905] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918909] [] ? update_rmtp+0x70/0x70 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918913] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918916] [] page_fault+0x25/0x30 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918918] Mem-Info: Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918920] Node 0 DMA per-cpu: Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918923] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918924] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918925] Node 0 DMA32 per-cpu: Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918928] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918929] CPU 1: hi: 186, btch: 31 usd: 11 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918933] active_anon:932645 inactive_anon:1393 isolated_anon:0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918934] active_file:0 inactive_file:8 isolated_file:0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918935] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918936] free:4093 slab_reclaimable:2594 slab_unreclaimable:2695 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918937] mapped:1358 shmem:1405 pagetables:3187 bounce:0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918939] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918947] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918951] Node 0 DMA32 free:8036kB min:7848kB low:9808kB high:11772kB active_anon:3730580kB inactive_anon:5572kB active_file:0kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:10376kB slab_unreclaimable:10780kB kernel_stack:768kB pagetables:12748kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918960] lowmem_reserve[]: 0 0 0 0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918963] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918973] Node 0 DMA32: 787*4kB 105*8kB 109*16kB 8*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8036kB Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918982] 1445 total pagecache pages Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918984] 0 pages in swap cache Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918985] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918987] Free swap = 0kB Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918988] Total swap = 0kB Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926194] 985072 pages RAM Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926196] 24998 pages reserved Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926198] 3787 pages shared Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926199] 953332 pages non-shared Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926201] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926209] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926212] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926215] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926218] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926221] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926224] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926226] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926229] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926233] [ 790] 101 790 63477 606 0 0 0 rsyslogd Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926236] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926239] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926242] [ 975] 0 975 6276 80 1 0 0 master Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926245] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926248] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926251] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926254] [ 1051] 0 1051 201627 844 0 0 0 collectd Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926257] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926260] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926264] [15657] 0 15657 4777 59 0 0 0 cron Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926266] [ 654] 1012 654 70351 5745 1 0 0 php Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926269] [ 657] 1012 657 69242 6257 1 0 0 php Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926272] [ 658] 1012 658 135033 71439 1 0 0 php Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926275] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926278] [13468] 0 13468 89584 2511 0 0 0 php5-fpm Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926280] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926283] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926286] [ 8112] 1012 8112 68610 5659 1 0 0 php Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926289] [31572] 108 31572 6792 76 0 0 0 pickup Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926293] [10396] 1012 10396 888569 825488 1 0 0 php Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926296] [11776] 108 11776 6818 73 0 0 0 cleanup Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926299] [11777] 108 11777 6795 69 1 0 0 trivial-rewrite Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926302] [11778] 108 11778 11107 159 0 0 0 smtp Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926305] [11782] 108 11782 11107 158 0 0 0 smtp Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926308] [11786] 108 11786 11107 159 0 0 0 smtp Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926311] [11880] 1012 11880 70571 7579 0 0 0 php Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926314] Out of memory: Kill process 10396 (php) score 861 or sacrifice child Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926327] Killed process 10396 (php) total-vm:3554276kB, anon-rss:3301952kB, file-rss:0kB Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116023] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116028] ntpd cpuset=/ mems_allowed=0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116031] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116033] Call Trace: Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116041] [] dump_header+0x91/0xe0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116045] [] oom_kill_process+0x85/0xb0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116048] [] out_of_memory+0xfa/0x220 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116066] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116071] [] alloc_pages_current+0xb6/0x120 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116076] [] __page_cache_alloc+0xb7/0xd0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116079] [] filemap_fault+0x212/0x3c0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116083] [] __do_fault+0x72/0x550 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116086] [] handle_pte_fault+0xfa/0x200 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116090] [] ? xen_pmd_val+0xe/0x10 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116093] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116096] [] handle_mm_fault+0x269/0x370 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116101] [] do_page_fault+0x184/0x550 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116105] [] ? pvclock_clocksource_read+0x55/0xf0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116109] [] page_fault+0x25/0x30 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116111] Mem-Info: Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116113] Node 0 DMA per-cpu: Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116116] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116118] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116119] Node 0 DMA32 per-cpu: Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116122] CPU 0: hi: 186, btch: 31 usd: 89 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116124] CPU 1: hi: 186, btch: 31 usd: 10 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116128] active_anon:932500 inactive_anon:1393 isolated_anon:0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116129] active_file:23 inactive_file:35 isolated_file:12 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116130] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116131] free:4043 slab_reclaimable:2625 slab_unreclaimable:2645 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116132] mapped:1390 shmem:1405 pagetables:3279 bounce:0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116134] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116142] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116146] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3730000kB inactive_anon:5572kB active_file:92kB inactive_file:140kB unevictable:0kB isolated(anon):0kB isolated(file):48kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5560kB shmem:5620kB slab_reclaimable:10500kB slab_unreclaimable:10580kB kernel_stack:768kB pagetables:13116kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:194 all_unreclaimable? yes Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116155] lowmem_reserve[]: 0 0 0 0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116158] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116169] Node 0 DMA32: 1103*4kB 163*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7796kB Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116179] 1490 total pagecache pages Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116180] 0 pages in swap cache Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116182] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116184] Free swap = 0kB Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116185] Total swap = 0kB Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121619] 985072 pages RAM Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121621] 24998 pages reserved Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121622] 3802 pages shared Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121623] 953219 pages non-shared Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121624] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121632] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121635] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121637] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121639] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121641] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121644] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121646] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121649] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121652] [ 790] 101 790 63477 606 1 0 0 rsyslogd Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121655] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121657] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121659] [ 975] 0 975 6276 80 1 0 0 master Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121662] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121664] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121666] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121668] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121671] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121673] [ 1242] 107 1242 9445 163 0 0 0 ntpd Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121675] [15657] 0 15657 4777 60 0 0 0 cron Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121677] [ 654] 1012 654 70351 5745 0 0 0 php Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121679] [ 657] 1012 657 69242 6258 0 0 0 php Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121682] [ 658] 1012 658 135033 71439 0 0 0 php Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121684] [ 678] 1012 678 70152 5566 0 0 0 php Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121686] [13468] 0 13468 89584 2529 1 0 0 php5-fpm Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121688] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121690] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121693] [ 8112] 1012 8112 68610 5662 1 0 0 php Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121695] [12047] 1012 12047 880712 817658 0 0 0 php Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121697] [ 970] 108 970 6792 69 0 0 0 pickup Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121700] [ 3659] 1012 3659 68580 5610 0 0 0 php Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121702] [ 3841] 1012 3841 1099 25 0 0 0 sh Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121704] [ 3842] 1012 3842 68825 5872 0 0 0 php Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121706] [ 3848] 1012 3848 1099 25 0 0 0 sh Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121708] [ 3849] 1012 3849 4776 57 0 0 0 git Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121710] [ 3850] 1012 3850 67197 4289 1 0 0 php Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121712] Out of memory: Kill process 12047 (php) score 853 or sacrifice child Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121722] Killed process 12047 (php) total-vm:3522848kB, anon-rss:3270632kB, file-rss:0kB Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660613] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660617] php cpuset=/ mems_allowed=0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660620] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660621] Call Trace: Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660629] [] dump_header+0x91/0xe0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660631] [] oom_kill_process+0x85/0xb0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660633] [] out_of_memory+0xfa/0x220 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660636] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660639] [] alloc_pages_current+0xb6/0x120 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660643] [] __page_cache_alloc+0xb7/0xd0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660645] [] filemap_fault+0x212/0x3c0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660648] [] __do_fault+0x72/0x550 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660652] [] ? error_exit+0x2a/0x60 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660654] [] ? retint_restore_args+0x5/0x6 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660657] [] handle_pte_fault+0xfa/0x200 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660660] [] ? xen_pmd_val+0xe/0x10 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660662] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660665] [] handle_mm_fault+0x269/0x370 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660668] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660670] [] ? check_events+0x12/0x20 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660673] [] do_page_fault+0x184/0x550 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660676] [] ? security_file_permission+0x93/0xb0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660679] [] ? rw_verify_area+0x61/0xf0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660682] [] ? vfs_read+0xb0/0x180 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660684] [] page_fault+0x25/0x30 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660686] Mem-Info: Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660687] Node 0 DMA per-cpu: Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660689] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660690] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660691] Node 0 DMA32 per-cpu: Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660693] CPU 0: hi: 186, btch: 31 usd: 4 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660694] CPU 1: hi: 186, btch: 31 usd: 57 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660697] active_anon:932670 inactive_anon:1393 isolated_anon:0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660698] active_file:10 inactive_file:16 isolated_file:0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660698] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660699] free:4034 slab_reclaimable:2701 slab_unreclaimable:2666 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660700] mapped:1364 shmem:1405 pagetables:3170 bounce:0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660701] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660708] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660711] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3730680kB inactive_anon:5572kB active_file:40kB inactive_file:64kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5456kB shmem:5620kB slab_reclaimable:10804kB slab_unreclaimable:10664kB kernel_stack:744kB pagetables:12680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:188 all_unreclaimable? yes Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660717] lowmem_reserve[]: 0 0 0 0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660720] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660727] Node 0 DMA32: 1438*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7800kB Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660733] 1470 total pagecache pages Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660734] 0 pages in swap cache Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660735] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660737] Free swap = 0kB Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660737] Total swap = 0kB Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666155] 985072 pages RAM Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666158] 24998 pages reserved Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666159] 3800 pages shared Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666160] 953288 pages non-shared Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666162] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666171] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666175] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666178] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666181] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666184] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666188] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666191] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666194] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666198] [ 790] 101 790 63477 606 0 0 0 rsyslogd Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666201] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666204] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666208] [ 975] 0 975 6276 80 1 0 0 master Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666211] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666214] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666217] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666220] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666223] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666226] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666229] [15657] 0 15657 4777 60 1 0 0 cron Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666233] [ 654] 1012 654 70351 5745 1 0 0 php Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666236] [ 657] 1012 657 69242 6258 0 0 0 php Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666239] [ 658] 1012 658 135033 71439 0 0 0 php Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666242] [ 678] 1012 678 70152 5566 0 0 0 php Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666245] [13468] 0 13468 89584 2539 0 0 0 php5-fpm Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666248] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666251] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666254] [ 8112] 1012 8112 68610 5663 0 0 0 php Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666257] [ 3905] 1012 3905 885361 822230 1 0 0 php Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666260] [29427] 108 29427 6792 69 0 0 0 pickup Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666263] [14893] 1012 14893 68580 5604 0 0 0 php Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666266] [15016] 1012 15016 1099 25 0 0 0 sh Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666270] [15017] 1012 15017 68825 5869 1 0 0 php Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666272] Out of memory: Kill process 3905 (php) score 858 or sacrifice child Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666284] Killed process 3905 (php) total-vm:3541444kB, anon-rss:3288920kB, file-rss:0kB Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912139] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912144] php cpuset=/ mems_allowed=0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912147] Pid: 15078, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912149] Call Trace: Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912157] [] dump_header+0x91/0xe0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912160] [] oom_kill_process+0x85/0xb0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912162] [] out_of_memory+0xfa/0x220 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912165] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912169] [] alloc_pages_vma+0xb3/0x190 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912172] [] do_anonymous_page.isra.38+0x7c/0x2f0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912174] [] handle_pte_fault+0x1e1/0x200 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912177] [] ? xen_pmd_val+0xe/0x10 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912180] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912182] [] handle_mm_fault+0x269/0x370 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912185] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912187] [] ? check_events+0x12/0x20 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912191] [] do_page_fault+0x184/0x550 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912193] [] ? xen_mc_flush+0xb2/0x1c0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912195] [] ? xen_clts+0x8d/0x190 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912198] [] ? __schedule+0x3cc/0x6f0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912200] [] page_fault+0x25/0x30 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912202] Mem-Info: Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912203] Node 0 DMA per-cpu: Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912205] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912207] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912208] Node 0 DMA32 per-cpu: Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912209] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912211] CPU 1: hi: 186, btch: 31 usd: 0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912213] active_anon:933170 inactive_anon:1393 isolated_anon:0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912214] active_file:13 inactive_file:15 isolated_file:0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912215] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912215] free:4048 slab_reclaimable:2435 slab_unreclaimable:2629 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912216] mapped:1358 shmem:1405 pagetables:3076 bounce:0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912217] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912224] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912227] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732680kB inactive_anon:5572kB active_file:52kB inactive_file:60kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:9740kB slab_unreclaimable:10516kB kernel_stack:728kB pagetables:12304kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:14791 all_unreclaimable? yes Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912234] lowmem_reserve[]: 0 0 0 0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912236] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912243] Node 0 DMA32: 799*4kB 166*8kB 72*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7820kB Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912249] 1463 total pagecache pages Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912251] 0 pages in swap cache Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912252] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912253] Free swap = 0kB Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912254] Total swap = 0kB Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917547] 985072 pages RAM Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917549] 24998 pages reserved Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917550] 3793 pages shared Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917551] 953355 pages non-shared Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917553] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917563] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917566] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917570] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917573] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917576] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917579] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917582] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917586] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917590] [ 790] 101 790 63477 606 1 0 0 rsyslogd Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917593] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917596] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917599] [ 975] 0 975 6276 79 0 0 0 master Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917602] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917605] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917609] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917612] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917615] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917618] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917621] [15657] 0 15657 4777 59 1 0 0 cron Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917624] [ 654] 1012 654 70351 5745 1 0 0 php Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917628] [ 657] 1012 657 69242 6257 1 0 0 php Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917631] [ 658] 1012 658 135033 71438 1 0 0 php Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917634] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917637] [13468] 0 13468 89584 2536 0 0 0 php5-fpm Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917640] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917643] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917646] [ 8112] 1012 8112 68610 5663 1 0 0 php Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917649] [29427] 108 29427 6792 69 0 0 0 pickup Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917652] [15078] 1012 15078 891627 828609 0 0 0 php Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917656] [17814] 1012 17814 68580 5604 1 0 0 php Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917658] Out of memory: Kill process 15078 (php) score 864 or sacrifice child Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917669] Killed process 15078 (php) total-vm:3566508kB, anon-rss:3314348kB, file-rss:88kB Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916137] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916142] php cpuset=/ mems_allowed=0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916145] Pid: 17913, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916148] Call Trace: Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916155] [] dump_header+0x91/0xe0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916158] [] oom_kill_process+0x85/0xb0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916161] [] out_of_memory+0xfa/0x220 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916165] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916169] [] alloc_pages_current+0xb6/0x120 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916174] [] __page_cache_alloc+0xb7/0xd0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916177] [] filemap_fault+0x212/0x3c0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916181] [] __do_fault+0x72/0x550 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916185] [] ? lru_cache_add_lru+0x2d/0x50 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916188] [] handle_pte_fault+0xfa/0x200 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916192] [] ? xen_pmd_val+0xe/0x10 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916195] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916198] [] handle_mm_fault+0x269/0x370 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916201] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916204] [] ? check_events+0x12/0x20 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916209] [] do_page_fault+0x184/0x550 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916211] [] ? xen_mc_flush+0xb2/0x1c0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916214] [] ? xen_clts+0x8d/0x190 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916218] [] ? __schedule+0x3cc/0x6f0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916221] [] page_fault+0x25/0x30 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916224] Mem-Info: Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916225] Node 0 DMA per-cpu: Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916228] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916230] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916231] Node 0 DMA32 per-cpu: Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916234] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916235] CPU 1: hi: 186, btch: 31 usd: 0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916239] active_anon:932922 inactive_anon:1393 isolated_anon:0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916240] active_file:0 inactive_file:19 isolated_file:0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916241] unevictable:0 dirty:9 writeback:0 unstable:0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916242] free:4045 slab_reclaimable:2697 slab_unreclaimable:2625 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916242] mapped:1364 shmem:1405 pagetables:3075 bounce:0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916244] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916252] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916255] Node 0 DMA32 free:7844kB min:7848kB low:9808kB high:11772kB active_anon:3731688kB inactive_anon:5572kB active_file:0kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:36kB writeback:0kB mapped:5456kB shmem:5620kB slab_reclaimable:10788kB slab_unreclaimable:10500kB kernel_stack:728kB pagetables:12300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:47623 all_unreclaimable? yes Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916262] lowmem_reserve[]: 0 0 0 0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916264] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916271] Node 0 DMA32: 952*4kB 234*8kB 15*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7968kB Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916278] 1436 total pagecache pages Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916279] 0 pages in swap cache Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916280] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916281] Free swap = 0kB Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916282] Total swap = 0kB Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921609] 985072 pages RAM Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921611] 24998 pages reserved Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921613] 3781 pages shared Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921614] 953334 pages non-shared Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921616] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921624] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921628] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921631] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921635] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921638] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921641] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921644] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921647] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921652] [ 790] 101 790 63477 606 0 0 0 rsyslogd Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921655] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921658] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921661] [ 975] 0 975 6276 79 1 0 0 master Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921664] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921667] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921670] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921674] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921677] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921680] [ 1242] 107 1242 9445 163 0 0 0 ntpd Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921683] [15657] 0 15657 4777 60 0 0 0 cron Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921686] [ 654] 1012 654 70351 5746 0 0 0 php Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921689] [ 657] 1012 657 69242 6258 0 0 0 php Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921692] [ 658] 1012 658 134393 70799 0 0 0 php Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921695] [ 678] 1012 678 70152 5566 0 0 0 php Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921698] [13468] 0 13468 89584 2521 0 0 0 php5-fpm Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921701] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921704] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921707] [ 8112] 1012 8112 68610 5663 1 0 0 php Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921710] [17913] 1012 17913 892089 828972 1 0 0 php Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921714] [30131] 108 30131 6792 69 1 0 0 pickup Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921717] [ 7975] 1012 7975 68580 5605 0 0 0 php Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921720] Out of memory: Kill process 17913 (php) score 865 or sacrifice child Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921730] Killed process 17913 (php) total-vm:3568356kB, anon-rss:3315888kB, file-rss:0kB Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600908] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600912] php cpuset=/ mems_allowed=0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600915] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600916] Call Trace: Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600924] [] dump_header+0x91/0xe0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600926] [] oom_kill_process+0x85/0xb0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600928] [] out_of_memory+0xfa/0x220 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600931] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600934] [] alloc_pages_current+0xb6/0x120 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600938] [] __page_cache_alloc+0xb7/0xd0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600940] [] filemap_fault+0x212/0x3c0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600943] [] __do_fault+0x72/0x550 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600948] [] ? error_exit+0x2a/0x60 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600950] [] ? retint_restore_args+0x5/0x6 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600953] [] handle_pte_fault+0xfa/0x200 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600956] [] ? xen_pmd_val+0xe/0x10 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600958] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600961] [] handle_mm_fault+0x269/0x370 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600963] [] do_page_fault+0x184/0x550 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600966] [] ? do_nanosleep+0xa4/0xd0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600969] [] ? __switch_to+0x288/0x360 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600973] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600976] [] ? update_rmtp+0x70/0x70 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600979] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600981] [] page_fault+0x25/0x30 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600983] Mem-Info: Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600984] Node 0 DMA per-cpu: Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600986] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600987] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600988] Node 0 DMA32 per-cpu: Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600990] CPU 0: hi: 186, btch: 31 usd: 30 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600991] CPU 1: hi: 186, btch: 31 usd: 27 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600994] active_anon:932953 inactive_anon:1393 isolated_anon:0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600995] active_file:6 inactive_file:10 isolated_file:0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600995] unevictable:0 dirty:22 writeback:0 unstable:0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600996] free:4035 slab_reclaimable:2618 slab_unreclaimable:2645 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600997] mapped:1357 shmem:1405 pagetables:3048 bounce:0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600998] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601005] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601008] Node 0 DMA32 free:7804kB min:7848kB low:9808kB high:11772kB active_anon:3731812kB inactive_anon:5572kB active_file:24kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:88kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:10472kB slab_unreclaimable:10580kB kernel_stack:720kB pagetables:12192kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:153 all_unreclaimable? yes Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601014] lowmem_reserve[]: 0 0 0 0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601017] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601023] Node 0 DMA32: 871*4kB 233*8kB 25*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7860kB Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601030] 1401 total pagecache pages Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601031] 0 pages in swap cache Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601033] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601034] Free swap = 0kB Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601035] Total swap = 0kB Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606230] 985072 pages RAM Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606233] 24998 pages reserved Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606234] 3788 pages shared Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606235] 953292 pages non-shared Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606237] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606246] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606250] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606253] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606256] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606259] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606262] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606266] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606269] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606273] [ 790] 101 790 63477 606 0 0 0 rsyslogd Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606276] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606279] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606282] [ 975] 0 975 6276 80 1 0 0 master Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606285] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606288] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606291] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606294] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606297] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606301] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606304] [15657] 0 15657 4777 60 0 0 0 cron Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606307] [ 654] 1012 654 70351 5745 1 0 0 php Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606310] [ 657] 1012 657 69242 6257 0 0 0 php Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606313] [ 658] 1012 658 134393 70799 1 0 0 php Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606316] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606319] [13468] 0 13468 89584 2506 1 0 0 php5-fpm Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606322] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606325] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606328] [ 8112] 1012 8112 68610 5663 0 0 0 php Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606331] [ 8100] 1012 8100 892207 829060 1 0 0 php Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606335] [24435] 108 24435 6792 69 0 0 0 pickup Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606338] [26729] 1012 26729 68580 5605 1 0 0 php Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606340] Out of memory: Kill process 8100 (php) score 865 or sacrifice child Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606351] Killed process 8100 (php) total-vm:3568828kB, anon-rss:3316240kB, file-rss:0kB Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532149] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532153] php cpuset=/ mems_allowed=0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532156] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532158] Call Trace: Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532165] [] dump_header+0x91/0xe0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532167] [] oom_kill_process+0x85/0xb0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532169] [] out_of_memory+0xfa/0x220 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532172] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532177] [] ? noalloc_get_block_write+0x30/0x30 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532180] [] alloc_pages_current+0xb6/0x120 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532183] [] __page_cache_alloc+0xb7/0xd0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532185] [] filemap_fault+0x212/0x3c0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532188] [] __do_fault+0x72/0x550 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532192] [] ? __pollwait+0xf0/0xf0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532194] [] handle_pte_fault+0xfa/0x200 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532197] [] ? xen_pmd_val+0xe/0x10 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532199] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532202] [] handle_mm_fault+0x269/0x370 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532205] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532207] [] ? check_events+0x12/0x20 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532210] [] do_page_fault+0x184/0x550 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532214] [] ? ktime_get_ts+0xad/0xe0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532216] [] ? poll_select_copy_remaining+0xed/0x140 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532219] [] ? sys_select+0xcd/0x100 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532222] [] page_fault+0x25/0x30 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532223] Mem-Info: Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532224] Node 0 DMA per-cpu: Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532226] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532228] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532229] Node 0 DMA32 per-cpu: Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532230] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532232] CPU 1: hi: 186, btch: 31 usd: 0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532234] active_anon:932969 inactive_anon:1393 isolated_anon:0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532235] active_file:0 inactive_file:0 isolated_file:0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532235] unevictable:0 dirty:22 writeback:0 unstable:0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532236] free:4048 slab_reclaimable:2558 slab_unreclaimable:2612 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532237] mapped:1377 shmem:1405 pagetables:3070 bounce:0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532238] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532245] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532248] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731876kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:88kB writeback:0kB mapped:5508kB shmem:5620kB slab_reclaimable:10232kB slab_unreclaimable:10448kB kernel_stack:728kB pagetables:12280kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532254] lowmem_reserve[]: 0 0 0 0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532257] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532264] Node 0 DMA32: 754*4kB 171*8kB 73*16kB 8*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532270] 1467 total pagecache pages Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532272] 0 pages in swap cache Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532273] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532274] Free swap = 0kB Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532275] Total swap = 0kB Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537452] 985072 pages RAM Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537455] 24998 pages reserved Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537456] 3799 pages shared Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537457] 953339 pages non-shared Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537459] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537468] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537471] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537474] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537478] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537481] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537484] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537487] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537491] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537495] [ 790] 101 790 63477 606 1 0 0 rsyslogd Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537498] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537501] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537505] [ 975] 0 975 6276 79 1 0 0 master Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537508] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537511] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537514] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537517] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537520] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537523] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537526] [15657] 0 15657 4777 59 1 0 0 cron Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537530] [ 654] 1012 654 70351 5745 1 0 0 php Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537533] [ 657] 1012 657 69242 6258 0 0 0 php Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537536] [ 658] 1012 658 134393 70799 0 0 0 php Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537539] [ 678] 1012 678 70152 5566 0 0 0 php Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537542] [13468] 0 13468 89584 2521 0 0 0 php5-fpm Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537545] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537548] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537551] [ 8112] 1012 8112 68610 5664 1 0 0 php Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537554] [24435] 108 24435 6792 75 0 0 0 pickup Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537558] [26872] 1012 26872 892202 829071 0 0 0 php Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537561] [28787] 1012 28787 68580 5604 0 0 0 php Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537563] Out of memory: Kill process 26872 (php) score 865 or sacrifice child Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537574] Killed process 26872 (php) total-vm:3568808kB, anon-rss:3316284kB, file-rss:0kB Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032154] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032158] php cpuset=/ mems_allowed=0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032161] Pid: 28908, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032163] Call Trace: Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032169] [] dump_header+0x91/0xe0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032172] [] oom_kill_process+0x85/0xb0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032174] [] out_of_memory+0xfa/0x220 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032177] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032180] [] alloc_pages_current+0xb6/0x120 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032183] [] __page_cache_alloc+0xb7/0xd0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032185] [] filemap_fault+0x212/0x3c0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032188] [] __do_fault+0x72/0x550 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032192] [] ? lru_cache_add_lru+0x2d/0x50 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032194] [] handle_pte_fault+0xfa/0x200 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032197] [] ? xen_pmd_val+0xe/0x10 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032200] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032202] [] handle_mm_fault+0x269/0x370 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032205] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032207] [] ? check_events+0x12/0x20 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032210] [] do_page_fault+0x184/0x550 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032212] [] ? xen_mc_flush+0xb2/0x1c0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032214] [] ? xen_clts+0x8d/0x190 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032217] [] ? __schedule+0x3cc/0x6f0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032220] [] page_fault+0x25/0x30 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032222] Mem-Info: Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032223] Node 0 DMA per-cpu: Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032225] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032226] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032227] Node 0 DMA32 per-cpu: Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032229] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032230] CPU 1: hi: 186, btch: 31 usd: 0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032233] active_anon:933180 inactive_anon:1393 isolated_anon:0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032233] active_file:0 inactive_file:0 isolated_file:0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032234] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032235] free:4048 slab_reclaimable:2456 slab_unreclaimable:2582 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032235] mapped:1351 shmem:1405 pagetables:3075 bounce:0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032237] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032243] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032246] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732720kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5404kB shmem:5620kB slab_reclaimable:9824kB slab_unreclaimable:10328kB kernel_stack:728kB pagetables:12300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032253] lowmem_reserve[]: 0 0 0 0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032255] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032262] Node 0 DMA32: 870*4kB 211*8kB 15*16kB 10*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7904kB Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032269] 1461 total pagecache pages Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032270] 0 pages in swap cache Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032271] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032272] Free swap = 0kB Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032273] Total swap = 0kB Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037497] 985072 pages RAM Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037500] 24998 pages reserved Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037501] 3784 pages shared Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037502] 953341 pages non-shared Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037504] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037513] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037517] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037520] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037523] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037526] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037530] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037533] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037536] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037540] [ 790] 101 790 63477 610 1 0 0 rsyslogd Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037544] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037547] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037550] [ 975] 0 975 6276 80 1 0 0 master Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037553] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037556] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037560] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037563] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037566] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037569] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037572] [15657] 0 15657 4777 60 1 0 0 cron Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037575] [ 654] 1012 654 70351 5745 0 0 0 php Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037579] [ 657] 1012 657 69242 6257 1 0 0 php Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037582] [ 658] 1012 658 134393 70799 1 0 0 php Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037585] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037588] [13468] 0 13468 89584 2506 1 0 0 php5-fpm Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037591] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037594] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037597] [ 8112] 1012 8112 68610 5685 1 0 0 php Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037600] [28908] 1012 28908 892404 829245 1 0 0 php Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037603] [ 4158] 108 4158 6792 69 0 0 0 pickup Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037607] [11385] 1012 11385 68580 5605 1 0 0 php Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037610] Out of memory: Kill process 28908 (php) score 865 or sacrifice child Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037620] Killed process 28908 (php) total-vm:3569616kB, anon-rss:3316980kB, file-rss:0kB Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156493] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156498] php cpuset=/ mems_allowed=0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156502] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156504] Call Trace: Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156512] [] dump_header+0x91/0xe0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156515] [] oom_kill_process+0x85/0xb0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156518] [] out_of_memory+0xfa/0x220 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156522] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156527] [] alloc_pages_current+0xb6/0x120 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156531] [] __page_cache_alloc+0xb7/0xd0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156534] [] filemap_fault+0x212/0x3c0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156537] [] __do_fault+0x72/0x550 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156540] [] ? __pollwait+0xf0/0xf0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156543] [] handle_pte_fault+0xfa/0x200 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156546] [] ? xen_pmd_val+0xe/0x10 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156548] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156551] [] handle_mm_fault+0x269/0x370 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156554] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156556] [] ? check_events+0x12/0x20 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156560] [] do_page_fault+0x184/0x550 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156563] [] ? ktime_get_ts+0xad/0xe0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156566] [] ? poll_select_copy_remaining+0xed/0x140 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156568] [] ? sys_select+0xcd/0x100 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156571] [] page_fault+0x25/0x30 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156573] Mem-Info: Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156574] Node 0 DMA per-cpu: Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156576] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156577] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156578] Node 0 DMA32 per-cpu: Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156580] CPU 0: hi: 186, btch: 31 usd: 46 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156582] CPU 1: hi: 186, btch: 31 usd: 30 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156585] active_anon:933102 inactive_anon:1393 isolated_anon:0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156586] active_file:8 inactive_file:0 isolated_file:0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156587] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156588] free:4026 slab_reclaimable:2440 slab_unreclaimable:2627 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156589] mapped:1376 shmem:1405 pagetables:3057 bounce:0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156591] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156598] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156601] Node 0 DMA32 free:7768kB min:7848kB low:9808kB high:11772kB active_anon:3732408kB inactive_anon:5572kB active_file:32kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5504kB shmem:5620kB slab_reclaimable:9760kB slab_unreclaimable:10508kB kernel_stack:728kB pagetables:12228kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:173 all_unreclaimable? yes Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156607] lowmem_reserve[]: 0 0 0 0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156610] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156617] Node 0 DMA32: 916*4kB 219*8kB 15*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7768kB Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156623] 1412 total pagecache pages Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156625] 0 pages in swap cache Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156626] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156627] Free swap = 0kB Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156628] Total swap = 0kB Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161964] 985072 pages RAM Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161966] 24998 pages reserved Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161968] 3776 pages shared Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161969] 953284 pages non-shared Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161971] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161979] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161983] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161986] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161989] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161992] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161996] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161999] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162002] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162006] [ 790] 101 790 63477 610 1 0 0 rsyslogd Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162009] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162012] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162015] [ 975] 0 975 6276 79 0 0 0 master Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162018] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162022] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162025] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162028] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162031] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162034] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162037] [15657] 0 15657 4777 59 1 0 0 cron Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162040] [ 654] 1012 654 70351 5745 1 0 0 php Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162043] [ 657] 1012 657 69242 6257 0 0 0 php Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162047] [ 658] 1012 658 134393 70799 0 0 0 php Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162050] [ 678] 1012 678 70152 5566 0 0 0 php Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162053] [13468] 0 13468 89584 2538 0 0 0 php5-fpm Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162056] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162059] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162062] [ 8112] 1012 8112 68610 5684 0 0 0 php Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162065] [ 4158] 108 4158 6792 69 0 0 0 pickup Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162069] [11505] 1012 11505 892239 829168 1 0 0 php Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162072] [12244] 1012 12244 68580 5604 0 0 0 php Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162074] Out of memory: Kill process 11505 (php) score 865 or sacrifice child Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162086] Killed process 11505 (php) total-vm:3568956kB, anon-rss:3316672kB, file-rss:0kB Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316428] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316432] php cpuset=/ mems_allowed=0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316435] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316436] Call Trace: Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316443] [] dump_header+0x91/0xe0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316445] [] oom_kill_process+0x85/0xb0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316448] [] out_of_memory+0xfa/0x220 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316450] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316454] [] alloc_pages_current+0xb6/0x120 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316457] [] __page_cache_alloc+0xb7/0xd0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316459] [] filemap_fault+0x212/0x3c0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316462] [] __do_fault+0x72/0x550 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316466] [] ? error_exit+0x2a/0x60 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316469] [] ? retint_restore_args+0x5/0x6 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316471] [] handle_pte_fault+0xfa/0x200 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316475] [] ? xen_pmd_val+0xe/0x10 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316477] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316480] [] handle_mm_fault+0x269/0x370 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316482] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316485] [] ? check_events+0x12/0x20 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316487] [] do_page_fault+0x184/0x550 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316490] [] ? ktime_get_ts+0xad/0xe0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316494] [] ? poll_select_copy_remaining+0xed/0x140 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316496] [] ? sys_select+0xcd/0x100 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316499] [] page_fault+0x25/0x30 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316500] Mem-Info: Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316502] Node 0 DMA per-cpu: Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316504] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316505] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316506] Node 0 DMA32 per-cpu: Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316508] CPU 0: hi: 186, btch: 31 usd: 30 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316509] CPU 1: hi: 186, btch: 31 usd: 76 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316512] active_anon:932683 inactive_anon:1393 isolated_anon:0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316512] active_file:0 inactive_file:0 isolated_file:0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316513] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316514] free:4036 slab_reclaimable:2826 slab_unreclaimable:2610 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316514] mapped:1357 shmem:1405 pagetables:3073 bounce:0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316516] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316522] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316525] Node 0 DMA32 free:7808kB min:7848kB low:9808kB high:11772kB active_anon:3730732kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:11304kB slab_unreclaimable:10440kB kernel_stack:728kB pagetables:12292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:128 all_unreclaimable? yes Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316532] lowmem_reserve[]: 0 0 0 0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316534] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316541] Node 0 DMA32: 1381*4kB 21*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7740kB Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316548] 1436 total pagecache pages Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316549] 0 pages in swap cache Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316550] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316551] Free swap = 0kB Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316552] Total swap = 0kB Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321859] 985072 pages RAM Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321862] 24998 pages reserved Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321863] 3787 pages shared Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321864] 953273 pages non-shared Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321866] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321875] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321878] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321882] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321885] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321888] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321891] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321895] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321898] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321902] [ 790] 101 790 63477 610 0 0 0 rsyslogd Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321906] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321909] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321912] [ 975] 0 975 6276 80 1 0 0 master Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321915] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321918] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321921] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321925] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321928] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321931] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321934] [15657] 0 15657 4777 60 0 0 0 cron Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321937] [ 654] 1012 654 70351 5745 1 0 0 php Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321940] [ 657] 1012 657 69242 6257 0 0 0 php Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321943] [ 658] 1012 658 134393 70799 0 0 0 php Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321946] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321950] [13468] 0 13468 89584 2539 0 0 0 php5-fpm Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321953] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321955] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321959] [ 8112] 1012 8112 68610 5685 1 0 0 php Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321962] [12340] 1012 12340 891860 828744 0 0 0 php Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321965] [20087] 108 20087 6792 69 0 0 0 pickup Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321969] [25981] 1012 25981 68580 5604 1 0 0 php Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321971] Out of memory: Kill process 12340 (php) score 864 or sacrifice child Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321982] Killed process 12340 (php) total-vm:3567440kB, anon-rss:3314976kB, file-rss:0kB Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.863981] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.863986] php cpuset=/ mems_allowed=0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.863989] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.863991] Call Trace: Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.863999] [] dump_header+0x91/0xe0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864002] [] oom_kill_process+0x85/0xb0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864005] [] out_of_memory+0xfa/0x220 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864009] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864015] [] ? noalloc_get_block_write+0x30/0x30 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864019] [] alloc_pages_current+0xb6/0x120 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864023] [] __page_cache_alloc+0xb7/0xd0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864026] [] filemap_fault+0x212/0x3c0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864030] [] __do_fault+0x72/0x550 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864033] [] handle_pte_fault+0xfa/0x200 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864037] [] ? xen_pmd_val+0xe/0x10 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864040] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864043] [] handle_mm_fault+0x269/0x370 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864047] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864052] [] ? check_events+0x12/0x20 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864067] [] do_page_fault+0x184/0x550 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864070] [] ? xen_mc_flush+0xb2/0x1c0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864072] [] ? xen_clts+0x8d/0x190 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864077] [] page_fault+0x25/0x30 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864079] Mem-Info: Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864081] Node 0 DMA per-cpu: Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864083] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864085] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864086] Node 0 DMA32 per-cpu: Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864089] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864090] CPU 1: hi: 186, btch: 31 usd: 30 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864094] active_anon:933012 inactive_anon:1393 isolated_anon:0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864095] active_file:0 inactive_file:22 isolated_file:0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864096] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864097] free:4043 slab_reclaimable:2580 slab_unreclaimable:2606 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864098] mapped:1352 shmem:1405 pagetables:3072 bounce:0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864100] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864108] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864112] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3732048kB inactive_anon:5572kB active_file:0kB inactive_file:88kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:10320kB slab_unreclaimable:10424kB kernel_stack:728kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10 all_unreclaimable? yes Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864121] lowmem_reserve[]: 0 0 0 0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864124] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864134] Node 0 DMA32: 810*4kB 160*8kB 32*16kB 22*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7848kB Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864144] 1427 total pagecache pages Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864145] 0 pages in swap cache Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864147] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864157] Free swap = 0kB Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864158] Total swap = 0kB Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869957] 985072 pages RAM Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869960] 24998 pages reserved Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869961] 3787 pages shared Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869962] 953324 pages non-shared Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869964] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869972] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869976] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869979] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869982] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869985] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869988] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869991] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869994] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869998] [ 790] 101 790 63477 610 0 0 0 rsyslogd Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870001] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870003] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870006] [ 975] 0 975 6276 79 0 0 0 master Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870009] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870012] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870015] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870018] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870021] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870025] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870028] [15657] 0 15657 4777 59 0 0 0 cron Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870031] [ 654] 1012 654 70351 5745 0 0 0 php Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870034] [ 657] 1012 657 69242 6257 0 0 0 php Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870037] [ 658] 1012 658 134393 70799 1 0 0 php Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870040] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870044] [13468] 0 13468 89584 2506 0 0 0 php5-fpm Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870047] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870049] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870053] [ 8112] 1012 8112 68610 5684 0 0 0 php Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870056] [20087] 108 20087 6792 69 1 0 0 pickup Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870059] [26079] 1012 26079 892139 829074 1 0 0 php Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870061] [28333] 1012 28333 68580 5603 1 0 0 php Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870064] Out of memory: Kill process 26079 (php) score 865 or sacrifice child Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870075] Killed process 26079 (php) total-vm:3568556kB, anon-rss:3316296kB, file-rss:0kB Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544136] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544140] php cpuset=/ mems_allowed=0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544144] Pid: 29039, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544146] Call Trace: Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544153] [] dump_header+0x91/0xe0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544157] [] oom_kill_process+0x85/0xb0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544160] [] out_of_memory+0xfa/0x220 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544163] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544168] [] alloc_pages_current+0xb6/0x120 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544172] [] __page_cache_alloc+0xb7/0xd0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544175] [] filemap_fault+0x212/0x3c0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544178] [] __do_fault+0x72/0x550 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544183] [] ? lru_cache_add_lru+0x2d/0x50 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544185] [] handle_pte_fault+0xfa/0x200 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544188] [] ? xen_pmd_val+0xe/0x10 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544191] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544193] [] handle_mm_fault+0x269/0x370 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544196] [] ? find_vma+0x30/0x70 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544199] [] do_page_fault+0x184/0x550 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544201] [] ? xen_mc_flush+0xb2/0x1c0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544203] [] ? xen_clts+0x8d/0x190 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544206] [] ? __schedule+0x3cc/0x6f0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544209] [] page_fault+0x25/0x30 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544210] Mem-Info: Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544211] Node 0 DMA per-cpu: Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544213] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544215] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544216] Node 0 DMA32 per-cpu: Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544218] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544219] CPU 1: hi: 186, btch: 31 usd: 0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544221] active_anon:932418 inactive_anon:1393 isolated_anon:20 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544222] active_file:13 inactive_file:14 isolated_file:0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544223] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544223] free:4048 slab_reclaimable:2817 slab_unreclaimable:2708 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544224] mapped:1340 shmem:1405 pagetables:3227 bounce:0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544226] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544233] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544236] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729672kB inactive_anon:5572kB active_file:52kB inactive_file:56kB unevictable:0kB isolated(anon):80kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5360kB shmem:5620kB slab_reclaimable:11268kB slab_unreclaimable:10832kB kernel_stack:784kB pagetables:12908kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10988 all_unreclaimable? yes Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544242] lowmem_reserve[]: 0 0 0 0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544245] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544251] Node 0 DMA32: 1480*4kB 6*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8016kB Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544258] 1448 total pagecache pages Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544259] 0 pages in swap cache Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544260] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544261] Free swap = 0kB Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544262] Total swap = 0kB Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549504] 985072 pages RAM Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549506] 24998 pages reserved Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549508] 3791 pages shared Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549509] 953301 pages non-shared Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549511] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549519] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549522] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549526] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549529] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549532] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549535] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549539] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549542] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549546] [ 790] 101 790 63477 610 1 0 0 rsyslogd Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549549] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549552] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549555] [ 975] 0 975 6276 80 0 0 0 master Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549558] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549561] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549564] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549567] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549570] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549574] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549577] [15657] 0 15657 4777 60 0 0 0 cron Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549580] [ 654] 1012 654 70351 5745 1 0 0 php Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549583] [ 657] 1012 657 69242 6258 0 0 0 php Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549586] [ 658] 1012 658 134393 70799 0 0 0 php Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549589] [ 678] 1012 678 70152 5566 0 0 0 php Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549592] [13468] 0 13468 89584 2517 0 0 0 php5-fpm Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549595] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549598] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549601] [ 8112] 1012 8112 68610 5685 1 0 0 php Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549605] [29039] 1012 29039 885137 822059 1 0 0 php Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549608] [ 2228] 108 2228 6792 69 1 0 0 pickup Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549611] [ 4421] 108 4421 6818 74 1 0 0 cleanup Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549614] [ 4422] 108 4422 6795 71 0 0 0 trivial-rewrite Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549617] [ 4423] 108 4423 11107 158 1 0 0 smtp Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549620] [ 4757] 1012 4757 68580 5606 1 0 0 php Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549623] [ 4849] 1012 4849 1099 24 0 0 0 sh Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549626] [ 4850] 1012 4850 68982 6013 0 0 0 php Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549629] [ 4873] 1012 4873 1099 25 1 0 0 sh Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549632] [ 4876] 1012 4876 4798 69 0 0 0 git Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549635] Out of memory: Kill process 29039 (php) score 858 or sacrifice child Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549646] Killed process 29039 (php) total-vm:3540548kB, anon-rss:3288236kB, file-rss:0kB Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003469] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003473] php cpuset=/ mems_allowed=0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003476] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003477] Call Trace: Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003485] [] dump_header+0x91/0xe0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003487] [] oom_kill_process+0x85/0xb0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003489] [] out_of_memory+0xfa/0x220 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003492] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003495] [] alloc_pages_current+0xb6/0x120 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003499] [] __page_cache_alloc+0xb7/0xd0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003501] [] filemap_fault+0x212/0x3c0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003504] [] __do_fault+0x72/0x550 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003507] [] ? load_TLS_descriptor+0x42/0xd0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003510] [] handle_pte_fault+0xfa/0x200 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003512] [] ? xen_pmd_val+0xe/0x10 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003514] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003517] [] handle_mm_fault+0x269/0x370 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003521] [] ? lock_hrtimer_base.isra.25+0x30/0x60 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003524] [] do_page_fault+0x184/0x550 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003528] [] ? do_nanosleep+0xa4/0xd0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003531] [] ? __switch_to+0x288/0x360 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003534] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003537] [] ? update_rmtp+0x70/0x70 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003539] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003542] [] page_fault+0x25/0x30 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003543] Mem-Info: Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003545] Node 0 DMA per-cpu: Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003547] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003548] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003549] Node 0 DMA32 per-cpu: Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003551] CPU 0: hi: 186, btch: 31 usd: 61 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003552] CPU 1: hi: 186, btch: 31 usd: 0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003555] active_anon:932982 inactive_anon:1393 isolated_anon:0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003556] active_file:40 inactive_file:0 isolated_file:0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003557] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003557] free:4022 slab_reclaimable:2444 slab_unreclaimable:2626 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003558] mapped:1356 shmem:1405 pagetables:3170 bounce:0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003560] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003566] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003569] Node 0 DMA32 free:7752kB min:7848kB low:9808kB high:11772kB active_anon:3731928kB inactive_anon:5572kB active_file:160kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:9776kB slab_unreclaimable:10504kB kernel_stack:760kB pagetables:12680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:154 all_unreclaimable? yes Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003576] lowmem_reserve[]: 0 0 0 0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003578] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003585] Node 0 DMA32: 1426*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7752kB Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003591] 1435 total pagecache pages Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003593] 0 pages in swap cache Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003594] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003595] Free swap = 0kB Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003596] Total swap = 0kB Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008774] 985072 pages RAM Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008777] 24998 pages reserved Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008778] 3792 pages shared Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008779] 953306 pages non-shared Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008781] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008790] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008793] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008797] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008800] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008803] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008806] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008810] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008813] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008817] [ 790] 101 790 63477 610 0 0 0 rsyslogd Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008820] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008823] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008827] [ 975] 0 975 6276 80 0 0 0 master Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008830] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008833] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008836] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008839] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008842] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008845] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008849] [15657] 0 15657 4777 59 0 0 0 cron Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008852] [ 654] 1012 654 70351 5746 0 0 0 php Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008855] [ 657] 1012 657 69242 6257 1 0 0 php Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008858] [ 658] 1012 658 134393 70799 1 0 0 php Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008861] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008865] [13468] 0 13468 89584 2520 1 0 0 php5-fpm Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008868] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008871] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008874] [ 5841] 1012 5841 68618 5641 1 0 0 php Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008877] [11561] 108 11561 6792 70 1 0 0 pickup Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008880] [12406] 1012 12406 882237 819092 1 0 0 php Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008883] [19463] 108 19463 6795 71 1 0 0 trivial-rewrite Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008886] [19954] 1012 19954 78096 15119 1 0 0 php Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008889] [20129] 108 20129 6818 74 1 0 0 cleanup Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008893] [20130] 108 20130 11107 159 0 0 0 smtp Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008896] [20131] 1012 20131 12484 151 0 0 0 php Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008898] Out of memory: Kill process 12406 (php) score 854 or sacrifice child Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008910] Killed process 12406 (php) total-vm:3528948kB, anon-rss:3276368kB, file-rss:0kB Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284122] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284126] php cpuset=/ mems_allowed=0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284129] Pid: 20163, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284131] Call Trace: Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284138] [] dump_header+0x91/0xe0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284140] [] oom_kill_process+0x85/0xb0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284142] [] out_of_memory+0xfa/0x220 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284145] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284148] [] alloc_pages_current+0xb6/0x120 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284152] [] __page_cache_alloc+0xb7/0xd0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284154] [] filemap_fault+0x212/0x3c0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284157] [] ? pte_mfn_to_pfn+0x8d/0x110 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284160] [] __do_fault+0x72/0x550 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284163] [] ? lru_cache_add_lru+0x2d/0x50 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284166] [] handle_pte_fault+0xfa/0x200 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284168] [] ? xen_pmd_val+0xe/0x10 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284170] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284173] [] handle_mm_fault+0x269/0x370 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284175] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284178] [] ? check_events+0x12/0x20 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284181] [] do_page_fault+0x184/0x550 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284183] [] ? xen_mc_flush+0xb2/0x1c0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284185] [] ? xen_clts+0x8d/0x190 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284188] [] ? __schedule+0x3cc/0x6f0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284191] [] page_fault+0x25/0x30 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284192] Mem-Info: Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284194] Node 0 DMA per-cpu: Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284196] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284197] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284198] Node 0 DMA32 per-cpu: Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284200] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284201] CPU 1: hi: 186, btch: 31 usd: 0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284204] active_anon:932816 inactive_anon:1393 isolated_anon:0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284204] active_file:21 inactive_file:22 isolated_file:0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284205] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284206] free:4048 slab_reclaimable:2543 slab_unreclaimable:2654 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284206] mapped:1392 shmem:1405 pagetables:3213 bounce:0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284208] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284214] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284217] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731264kB inactive_anon:5572kB active_file:84kB inactive_file:88kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5568kB shmem:5620kB slab_reclaimable:10172kB slab_unreclaimable:10616kB kernel_stack:776kB pagetables:12852kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2079 all_unreclaimable? yes Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284224] lowmem_reserve[]: 0 0 0 0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284226] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284233] Node 0 DMA32: 973*4kB 213*8kB 15*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7884kB Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284240] 1469 total pagecache pages Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284241] 0 pages in swap cache Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284242] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284243] Free swap = 0kB Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284244] Total swap = 0kB Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289638] 985072 pages RAM Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289640] 24998 pages reserved Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289641] 3802 pages shared Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289642] 953322 pages non-shared Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289644] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289653] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289657] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289660] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289663] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289666] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289670] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289673] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289676] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289680] [ 790] 101 790 63477 610 0 0 0 rsyslogd Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289683] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289687] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289690] [ 975] 0 975 6276 80 0 0 0 master Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289693] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289696] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289699] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289702] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289705] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289709] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289712] [15657] 0 15657 4777 59 0 0 0 cron Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289715] [ 654] 1012 654 70351 5746 1 0 0 php Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289718] [ 657] 1012 657 69242 6258 0 0 0 php Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289721] [ 658] 1012 658 134393 70799 0 0 0 php Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289724] [ 678] 1012 678 70152 5566 0 0 0 php Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289727] [13468] 0 13468 89584 2517 0 0 0 php5-fpm Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289731] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289733] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289737] [ 5841] 1012 5841 68618 5645 1 0 0 php Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289740] [11561] 108 11561 6792 76 1 0 0 pickup Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289743] [20163] 1012 20163 891143 828169 0 0 0 php Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289746] [20731] 108 20731 6818 74 1 0 0 cleanup Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289749] [20732] 108 20732 6795 71 0 0 0 trivial-rewrite Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289752] [20733] 108 20733 11107 158 1 0 0 smtp Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289755] [21016] 108 21016 11107 158 1 0 0 smtp Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289758] [21020] 108 21020 11107 159 0 0 0 smtp Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289761] [21024] 108 21024 11107 157 1 0 0 smtp Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289764] [21148] 1012 21148 68580 5603 1 0 0 php Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289767] Out of memory: Kill process 20163 (php) score 864 or sacrifice child Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289778] Killed process 20163 (php) total-vm:3564572kB, anon-rss:3312588kB, file-rss:88kB Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506553] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506556] php cpuset=/ mems_allowed=0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506559] Pid: 21225, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506561] Call Trace: Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506568] [] dump_header+0x91/0xe0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506570] [] oom_kill_process+0x85/0xb0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506573] [] out_of_memory+0xfa/0x220 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506575] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506579] [] alloc_pages_current+0xb6/0x120 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506582] [] __page_cache_alloc+0xb7/0xd0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506584] [] filemap_fault+0x212/0x3c0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506587] [] __do_fault+0x72/0x550 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506590] [] ? lru_cache_add_lru+0x2d/0x50 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506593] [] handle_pte_fault+0xfa/0x200 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506596] [] ? xen_pmd_val+0xe/0x10 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506598] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506601] [] handle_mm_fault+0x269/0x370 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506603] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506606] [] ? check_events+0x12/0x20 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506609] [] do_page_fault+0x184/0x550 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506611] [] ? do_brk+0x21f/0x320 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506615] [] ? __schedule+0x3cc/0x6f0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506618] [] page_fault+0x25/0x30 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506619] Mem-Info: Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506620] Node 0 DMA per-cpu: Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506622] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506623] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506625] Node 0 DMA32 per-cpu: Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506626] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506628] CPU 1: hi: 186, btch: 31 usd: 14 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506630] active_anon:932900 inactive_anon:1393 isolated_anon:0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506631] active_file:0 inactive_file:2 isolated_file:0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506632] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506632] free:4048 slab_reclaimable:2722 slab_unreclaimable:2631 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506633] mapped:1350 shmem:1405 pagetables:3054 bounce:0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506634] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506641] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506644] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731600kB inactive_anon:5572kB active_file:0kB inactive_file:8kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5400kB shmem:5620kB slab_reclaimable:10888kB slab_unreclaimable:10524kB kernel_stack:728kB pagetables:12216kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:41 all_unreclaimable? yes Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506650] lowmem_reserve[]: 0 0 0 0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506653] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506660] Node 0 DMA32: 1420*4kB 6*8kB 7*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7888kB Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506666] 1419 total pagecache pages Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506667] 0 pages in swap cache Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506669] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506670] Free swap = 0kB Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506671] Total swap = 0kB Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511815] 985072 pages RAM Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511817] 24998 pages reserved Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511819] 3788 pages shared Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511820] 953328 pages non-shared Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511822] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511831] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511835] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511838] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511841] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511844] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511847] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511851] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511854] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511858] [ 790] 101 790 63477 615 1 0 0 rsyslogd Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511862] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511865] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511868] [ 975] 0 975 6276 80 1 0 0 master Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511871] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511875] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511878] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511881] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511884] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511887] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511890] [15657] 0 15657 4777 60 1 0 0 cron Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511894] [ 654] 1012 654 70351 5747 1 0 0 php Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511897] [ 657] 1012 657 69242 6258 1 0 0 php Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511900] [ 658] 1012 658 134393 70799 1 0 0 php Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511903] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511906] [13468] 0 13468 89584 2502 1 0 0 php5-fpm Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511909] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511912] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511915] [ 5841] 1012 5841 68618 5647 1 0 0 php Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511918] [21225] 1012 21225 891926 828997 0 0 0 php Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511921] [ 757] 108 757 6792 70 1 0 0 pickup Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511925] [ 5308] 1012 5308 68580 5606 1 0 0 php Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511928] Out of memory: Kill process 21225 (php) score 865 or sacrifice child Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511939] Killed process 21225 (php) total-vm:3567704kB, anon-rss:3315964kB, file-rss:24kB Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244161] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244166] ntpd cpuset=/ mems_allowed=0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244169] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244171] Call Trace: Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244180] [] dump_header+0x91/0xe0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244183] [] oom_kill_process+0x85/0xb0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244186] [] out_of_memory+0xfa/0x220 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244190] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244196] [] ? noalloc_get_block_write+0x30/0x30 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244199] [] alloc_pages_current+0xb6/0x120 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244204] [] __page_cache_alloc+0xb7/0xd0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244206] [] filemap_fault+0x212/0x3c0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244210] [] __do_fault+0x72/0x550 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244213] [] handle_pte_fault+0xfa/0x200 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244217] [] ? xen_pmd_val+0xe/0x10 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244220] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244223] [] handle_mm_fault+0x269/0x370 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244228] [] do_page_fault+0x184/0x550 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244232] [] ? pvclock_clocksource_read+0x55/0xf0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244237] [] page_fault+0x25/0x30 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244239] Mem-Info: Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244240] Node 0 DMA per-cpu: Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244243] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244245] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244246] Node 0 DMA32 per-cpu: Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244248] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244250] CPU 1: hi: 186, btch: 31 usd: 27 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244254] active_anon:932972 inactive_anon:1393 isolated_anon:0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244255] active_file:0 inactive_file:21 isolated_file:0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244256] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244257] free:4048 slab_reclaimable:2549 slab_unreclaimable:2638 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244257] mapped:1361 shmem:1405 pagetables:3072 bounce:0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244260] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244269] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244273] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731888kB inactive_anon:5572kB active_file:0kB inactive_file:84kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5620kB slab_reclaimable:10196kB slab_unreclaimable:10552kB kernel_stack:752kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:34 all_unreclaimable? yes Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244281] lowmem_reserve[]: 0 0 0 0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244285] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244293] Node 0 DMA32: 1426*4kB 11*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244301] 1456 total pagecache pages Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244302] 0 pages in swap cache Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244303] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244304] Free swap = 0kB Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244305] Total swap = 0kB Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249906] 985072 pages RAM Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249909] 24998 pages reserved Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249910] 3786 pages shared Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249911] 953321 pages non-shared Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249913] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249922] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249925] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249929] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249932] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249935] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249938] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249942] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249945] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249949] [ 790] 101 790 63477 615 1 0 0 rsyslogd Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249952] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249955] [ 831] 0 831 4226 41 1 0 0 atd Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249959] [ 975] 0 975 6276 80 0 0 0 master Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249962] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249965] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249968] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249971] [ 1051] 0 1051 201629 847 1 0 0 collectd Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249974] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249977] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249980] [15657] 0 15657 4777 59 1 0 0 cron Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249984] [ 654] 1012 654 70351 5747 0 0 0 php Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249987] [ 657] 1012 657 69242 6257 0 0 0 php Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249990] [ 658] 1012 658 134393 70799 1 0 0 php Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249993] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249996] [13468] 0 13468 89584 2539 1 0 0 php5-fpm Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249999] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250002] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250005] [ 5841] 1012 5841 68618 5647 1 0 0 php Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250008] [ 757] 108 757 6792 76 0 0 0 pickup Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250012] [12910] 1012 12910 897434 834204 1 0 0 php Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250015] [13494] 108 13494 6818 73 0 0 0 cleanup Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250018] [13495] 108 13495 6795 69 0 0 0 trivial-rewrite Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250021] [13496] 108 13496 11107 158 0 0 0 smtp Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250024] [13582] 108 13582 11107 159 1 0 0 smtp Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250027] Out of memory: Kill process 12910 (php) score 870 or sacrifice child Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250039] Killed process 12910 (php) total-vm:3589736kB, anon-rss:3336816kB, file-rss:0kB Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825357] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825362] php5-fpm cpuset=/ mems_allowed=0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825364] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825366] Call Trace: Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825373] [] dump_header+0x91/0xe0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825376] [] oom_kill_process+0x85/0xb0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825378] [] out_of_memory+0xfa/0x220 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825381] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825384] [] alloc_pages_current+0xb6/0x120 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825387] [] __page_cache_alloc+0xb7/0xd0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825389] [] filemap_fault+0x212/0x3c0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825392] [] __do_fault+0x72/0x550 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825396] [] ? __switch_to+0x138/0x360 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825399] [] ? set_next_entity+0xad/0xd0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825402] [] handle_pte_fault+0xfa/0x200 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825405] [] ? xen_pmd_val+0xe/0x10 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825407] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825410] [] handle_mm_fault+0x269/0x370 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825413] [] do_page_fault+0x184/0x550 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825416] [] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825421] [] ? ep_poll+0x1cc/0x380 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825424] [] ? try_to_wake_up+0x200/0x200 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825427] [] ? fput+0xe/0x30 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825429] [] page_fault+0x25/0x30 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825431] Mem-Info: Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825432] Node 0 DMA per-cpu: Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825434] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825436] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825437] Node 0 DMA32 per-cpu: Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825438] CPU 0: hi: 186, btch: 31 usd: 44 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825440] CPU 1: hi: 186, btch: 31 usd: 35 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825442] active_anon:932734 inactive_anon:1393 isolated_anon:0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825443] active_file:0 inactive_file:9 isolated_file:4 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825444] unevictable:0 dirty:1 writeback:0 unstable:0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825444] free:4032 slab_reclaimable:2349 slab_unreclaimable:2641 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825445] mapped:1351 shmem:1405 pagetables:3379 bounce:0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825446] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825453] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825456] Node 0 DMA32 free:7792kB min:7848kB low:9808kB high:11772kB active_anon:3730936kB inactive_anon:5572kB active_file:0kB inactive_file:36kB unevictable:0kB isolated(anon):0kB isolated(file):16kB present:3862656kB mlocked:0kB dirty:4kB writeback:0kB mapped:5404kB shmem:5620kB slab_reclaimable:9396kB slab_unreclaimable:10564kB kernel_stack:824kB pagetables:13516kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:177 all_unreclaimable? yes Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825463] lowmem_reserve[]: 0 0 0 0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825465] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825472] Node 0 DMA32: 724*4kB 288*8kB 29*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7712kB Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825479] 1456 total pagecache pages Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825480] 0 pages in swap cache Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825481] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825482] Free swap = 0kB Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825483] Total swap = 0kB Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830579] 985072 pages RAM Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830582] 24998 pages reserved Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830583] 3786 pages shared Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830584] 953299 pages non-shared Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830586] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830595] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830599] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830602] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830605] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830608] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830611] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830615] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830618] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830622] [ 790] 101 790 63477 615 1 0 0 rsyslogd Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830625] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830628] [ 831] 0 831 4226 41 1 0 0 atd Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830631] [ 975] 0 975 6276 79 0 0 0 master Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830634] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830638] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830641] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830644] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830647] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830650] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830653] [15657] 0 15657 4777 59 0 0 0 cron Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830657] [ 654] 1012 654 70351 5746 1 0 0 php Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830660] [ 657] 1012 657 69242 6258 1 0 0 php Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830663] [ 658] 1012 658 134393 70799 0 0 0 php Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830666] [ 678] 1012 678 70152 5566 0 0 0 php Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830669] [13468] 0 13468 89584 2524 0 0 0 php5-fpm Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830672] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830675] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830678] [ 5841] 1012 5841 68618 5646 1 0 0 php Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830681] [ 757] 108 757 6792 76 0 0 0 pickup Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830685] [13494] 108 13494 6818 81 0 0 0 cleanup Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830688] [13495] 108 13495 6795 73 0 0 0 trivial-rewrite Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830691] [13496] 108 13496 11107 157 1 0 0 smtp Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830694] [13582] 108 13582 11107 158 1 0 0 smtp Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830697] [14021] 1012 14021 647943 584764 1 0 0 php Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830700] [14319] 1012 14319 300584 237475 1 0 0 php Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830703] [14408] 1012 14408 68580 5605 1 0 0 php Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830706] [14410] 1012 14410 1099 24 0 0 0 sh Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830709] [14411] 1012 14411 68982 6015 1 0 0 php Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830712] [14439] 1012 14439 1099 26 1 0 0 sh Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830715] [14440] 1012 14440 4254 39 1 0 0 git Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830718] Out of memory: Kill process 14021 (php) score 610 or sacrifice child Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830729] Killed process 14021 (php) total-vm:2591772kB, anon-rss:2339056kB, file-rss:0kB Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077037] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077041] php cpuset=/ mems_allowed=0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077044] Pid: 14408, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077045] Call Trace: Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077053] [] dump_header+0x91/0xe0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077055] [] oom_kill_process+0x85/0xb0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077057] [] out_of_memory+0xfa/0x220 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077060] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077063] [] alloc_pages_current+0xb6/0x120 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077068] [] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077071] [] __page_cache_alloc+0xb7/0xd0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077073] [] filemap_fault+0x212/0x3c0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077076] [] __do_fault+0x72/0x550 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077079] [] ? load_TLS_descriptor+0x42/0xd0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077082] [] handle_pte_fault+0xfa/0x200 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077084] [] ? xen_pmd_val+0xe/0x10 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077086] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077089] [] handle_mm_fault+0x269/0x370 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077093] [] ? lock_hrtimer_base.isra.25+0x30/0x60 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077096] [] do_page_fault+0x184/0x550 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077099] [] ? do_nanosleep+0xa4/0xd0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077101] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077104] [] ? update_rmtp+0x70/0x70 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077107] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077109] [] page_fault+0x25/0x30 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077111] Mem-Info: Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077112] Node 0 DMA per-cpu: Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077114] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077116] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077117] Node 0 DMA32 per-cpu: Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077119] CPU 0: hi: 186, btch: 31 usd: 30 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077120] CPU 1: hi: 186, btch: 31 usd: 82 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077122] active_anon:933055 inactive_anon:1393 isolated_anon:0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077123] active_file:17 inactive_file:32 isolated_file:0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077124] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077125] free:4036 slab_reclaimable:2240 slab_unreclaimable:2637 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077125] mapped:1363 shmem:1405 pagetables:3254 bounce:0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077127] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077133] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077136] Node 0 DMA32 free:7808kB min:7848kB low:9808kB high:11772kB active_anon:3732220kB inactive_anon:5572kB active_file:68kB inactive_file:128kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5452kB shmem:5620kB slab_reclaimable:8960kB slab_unreclaimable:10548kB kernel_stack:768kB pagetables:13016kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:197 all_unreclaimable? yes Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077143] lowmem_reserve[]: 0 0 0 0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077145] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077152] Node 0 DMA32: 241*4kB 464*8kB 68*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7812kB Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077159] 1437 total pagecache pages Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077160] 0 pages in swap cache Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077161] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077162] Free swap = 0kB Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077163] Total swap = 0kB Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082359] 985072 pages RAM Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082362] 24998 pages reserved Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082363] 3789 pages shared Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082364] 953242 pages non-shared Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082366] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082375] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082378] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082382] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082385] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082388] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082391] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082395] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082398] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082402] [ 790] 101 790 63477 615 1 0 0 rsyslogd Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082406] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082409] [ 831] 0 831 4226 41 1 0 0 atd Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082412] [ 975] 0 975 6276 79 1 0 0 master Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082415] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082418] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082421] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082424] [ 1051] 0 1051 201626 844 0 0 0 collectd Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082427] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082431] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082434] [15657] 0 15657 4777 59 0 0 0 cron Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082437] [ 654] 1012 654 70351 5746 1 0 0 php Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082440] [ 657] 1012 657 69242 6257 0 0 0 php Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082443] [ 658] 1012 658 134393 70799 0 0 0 php Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082446] [ 678] 1012 678 70152 5566 0 0 0 php Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082449] [13468] 0 13468 89584 2539 0 0 0 php5-fpm Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082452] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082455] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082458] [ 5841] 1012 5841 68618 5646 1 0 0 php Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082461] [ 757] 108 757 6792 76 1 0 0 pickup Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082465] [13494] 108 13494 6818 81 0 0 0 cleanup Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082468] [13495] 108 13495 6795 73 1 0 0 trivial-rewrite Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082471] [13496] 108 13496 11107 157 1 0 0 smtp Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082474] [13582] 108 13582 11107 158 0 0 0 smtp Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082477] [14319] 1012 14319 886155 823070 1 0 0 php Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082480] [14408] 1012 14408 68580 5605 0 0 0 php Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082483] [14441] 1012 14441 68580 5603 0 0 0 php Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082486] Out of memory: Kill process 14319 (php) score 859 or sacrifice child Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082497] Killed process 14319 (php) total-vm:3544620kB, anon-rss:3292280kB, file-rss:0kB Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048666] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048670] php cpuset=/ mems_allowed=0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048672] Pid: 32277, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048674] Call Trace: Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048681] [] dump_header+0x91/0xe0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048683] [] oom_kill_process+0x85/0xb0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048685] [] out_of_memory+0xfa/0x220 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048688] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048691] [] alloc_pages_current+0xb6/0x120 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048695] [] __page_cache_alloc+0xb7/0xd0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048697] [] filemap_fault+0x212/0x3c0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048700] [] __do_fault+0x72/0x550 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048703] [] ? lru_cache_add_lru+0x2d/0x50 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048705] [] handle_pte_fault+0xfa/0x200 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048709] [] ? xen_pmd_val+0xe/0x10 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048711] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048713] [] handle_mm_fault+0x269/0x370 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048716] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048718] [] ? check_events+0x12/0x20 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048722] [] do_page_fault+0x184/0x550 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048724] [] ? do_brk+0x21f/0x320 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048727] [] ? __schedule+0x3cc/0x6f0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048730] [] page_fault+0x25/0x30 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048732] Mem-Info: Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048733] Node 0 DMA per-cpu: Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048735] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048736] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048737] Node 0 DMA32 per-cpu: Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048739] CPU 0: hi: 186, btch: 31 usd: 30 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048740] CPU 1: hi: 186, btch: 31 usd: 26 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048743] active_anon:932511 inactive_anon:1393 isolated_anon:0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048743] active_file:0 inactive_file:0 isolated_file:0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048744] unevictable:0 dirty:16 writeback:0 unstable:0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048745] free:4034 slab_reclaimable:2806 slab_unreclaimable:2724 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048745] mapped:1357 shmem:1405 pagetables:3173 bounce:0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048747] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048753] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048756] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3730044kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:64kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:11224kB slab_unreclaimable:10896kB kernel_stack:768kB pagetables:12692kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:111 all_unreclaimable? yes Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048763] lowmem_reserve[]: 0 0 0 0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048765] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048772] Node 0 DMA32: 1428*4kB 2*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7824kB Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048779] 1439 total pagecache pages Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048780] 0 pages in swap cache Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048781] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048782] Free swap = 0kB Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048783] Total swap = 0kB Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054023] 985072 pages RAM Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054025] 24998 pages reserved Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054026] 3783 pages shared Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054027] 953301 pages non-shared Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054029] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054038] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054042] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054045] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054048] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054051] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054054] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054058] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054061] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054065] [ 790] 101 790 63477 615 0 0 0 rsyslogd Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054068] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054071] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054075] [ 975] 0 975 6276 80 1 0 0 master Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054078] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054081] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054084] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054087] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054090] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054093] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054097] [15657] 0 15657 4777 60 0 0 0 cron Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054100] [ 654] 1012 654 70351 5748 1 0 0 php Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054103] [ 657] 1012 657 69242 6258 1 0 0 php Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054106] [ 658] 1012 658 134393 70799 1 0 0 php Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054109] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054113] [13468] 0 13468 89584 2524 1 0 0 php5-fpm Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054116] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054119] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054122] [ 5841] 1012 5841 68618 5647 1 0 0 php Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054125] [32277] 1012 32277 886676 823439 0 0 0 php Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054128] [13160] 108 13160 6792 70 1 0 0 pickup Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054131] [22662] 108 22662 6818 73 1 0 0 cleanup Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054134] [22663] 108 22663 6795 71 1 0 0 trivial-rewrite Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054137] [22664] 108 22664 11107 158 0 0 0 smtp Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054140] [22672] 108 22672 11107 159 1 0 0 smtp Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054143] [22676] 108 22676 11107 159 1 0 0 smtp Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054147] [24093] 1012 24093 73115 10147 1 0 0 php Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054149] Out of memory: Kill process 32277 (php) score 859 or sacrifice child Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054161] Killed process 32277 (php) total-vm:3546704kB, anon-rss:3293756kB, file-rss:0kB Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824204] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824209] php5-fpm cpuset=/ mems_allowed=0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824213] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824215] Call Trace: Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824223] [] dump_header+0x91/0xe0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824226] [] oom_kill_process+0x85/0xb0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824230] [] out_of_memory+0xfa/0x220 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824233] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824238] [] alloc_pages_current+0xb6/0x120 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824242] [] __page_cache_alloc+0xb7/0xd0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824245] [] filemap_fault+0x212/0x3c0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824249] [] __do_fault+0x72/0x550 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824253] [] ? __switch_to+0x138/0x360 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824257] [] handle_pte_fault+0xfa/0x200 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824261] [] ? xen_pmd_val+0xe/0x10 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824264] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824267] [] handle_mm_fault+0x269/0x370 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824271] [] do_page_fault+0x184/0x550 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824275] [] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824280] [] ? ep_poll+0x1cc/0x380 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824285] [] ? try_to_wake_up+0x200/0x200 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824288] [] ? fget+0x1d/0x90 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824292] [] page_fault+0x25/0x30 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824294] Mem-Info: Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824295] Node 0 DMA per-cpu: Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824298] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824300] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824301] Node 0 DMA32 per-cpu: Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824304] CPU 0: hi: 186, btch: 31 usd: 112 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824305] CPU 1: hi: 186, btch: 31 usd: 0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824309] active_anon:932989 inactive_anon:1393 isolated_anon:0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824310] active_file:0 inactive_file:27 isolated_file:0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824311] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824312] free:4048 slab_reclaimable:2489 slab_unreclaimable:2615 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824313] mapped:1352 shmem:1405 pagetables:3078 bounce:0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824315] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824323] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824327] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731956kB inactive_anon:5572kB active_file:0kB inactive_file:108kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:9956kB slab_unreclaimable:10460kB kernel_stack:728kB pagetables:12312kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:295 all_unreclaimable? yes Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824335] lowmem_reserve[]: 0 0 0 0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824339] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824348] Node 0 DMA32: 802*4kB 183*8kB 55*16kB 6*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824358] 1432 total pagecache pages Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824360] 0 pages in swap cache Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824362] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824363] Free swap = 0kB Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824364] Total swap = 0kB Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831789] 985072 pages RAM Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831791] 24998 pages reserved Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831792] 3783 pages shared Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831793] 953242 pages non-shared Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831795] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831804] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831808] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831811] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831814] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831817] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831820] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831823] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831827] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831831] [ 790] 101 790 63477 615 0 0 0 rsyslogd Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831834] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831837] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831840] [ 975] 0 975 6276 80 0 0 0 master Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831843] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831846] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831849] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831852] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831855] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831859] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831862] [15657] 0 15657 4777 59 1 0 0 cron Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831865] [ 654] 1012 654 70351 5747 0 0 0 php Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831868] [ 657] 1012 657 69242 6258 1 0 0 php Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831871] [ 658] 1012 658 134393 70799 1 0 0 php Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831874] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831877] [13468] 0 13468 89584 2509 1 0 0 php5-fpm Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831880] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831883] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831886] [ 5841] 1012 5841 68618 5647 1 0 0 php Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831890] [13160] 108 13160 6792 76 1 0 0 pickup Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831893] [24274] 1012 24274 892212 829073 0 0 0 php Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831896] [31890] 1012 31890 68580 5603 1 0 0 php Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831899] Out of memory: Kill process 24274 (php) score 865 or sacrifice child Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831911] Killed process 24274 (php) total-vm:3568848kB, anon-rss:3316292kB, file-rss:0kB Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636089] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636093] php cpuset=/ mems_allowed=0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636096] Pid: 32048, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636097] Call Trace: Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636105] [] dump_header+0x91/0xe0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636107] [] oom_kill_process+0x85/0xb0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636109] [] out_of_memory+0xfa/0x220 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636112] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636117] [] ? noalloc_get_block_write+0x30/0x30 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636120] [] alloc_pages_current+0xb6/0x120 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636123] [] __page_cache_alloc+0xb7/0xd0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636126] [] filemap_fault+0x212/0x3c0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636129] [] __do_fault+0x72/0x550 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636131] [] ? lru_cache_add_lru+0x2d/0x50 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636134] [] handle_pte_fault+0xfa/0x200 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636137] [] ? xen_pmd_val+0xe/0x10 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636139] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636142] [] handle_mm_fault+0x269/0x370 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636145] [] do_page_fault+0x184/0x550 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636147] [] ? xen_mc_flush+0xb2/0x1c0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636149] [] ? xen_clts+0x8d/0x190 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636152] [] ? __schedule+0x3cc/0x6f0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636155] [] page_fault+0x25/0x30 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636157] Mem-Info: Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636158] Node 0 DMA per-cpu: Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636160] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636161] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636162] Node 0 DMA32 per-cpu: Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636164] CPU 0: hi: 186, btch: 31 usd: 0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636165] CPU 1: hi: 186, btch: 31 usd: 0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636168] active_anon:933255 inactive_anon:1393 isolated_anon:0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636168] active_file:0 inactive_file:0 isolated_file:0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636169] unevictable:0 dirty:7 writeback:0 unstable:0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636170] free:4048 slab_reclaimable:2346 slab_unreclaimable:2621 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636170] mapped:1386 shmem:1405 pagetables:3069 bounce:0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636172] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636179] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636182] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3733020kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:28kB writeback:0kB mapped:5544kB shmem:5620kB slab_reclaimable:9384kB slab_unreclaimable:10484kB kernel_stack:784kB pagetables:12276kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636188] lowmem_reserve[]: 0 0 0 0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636190] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636197] Node 0 DMA32: 604*4kB 104*8kB 104*16kB 28*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636204] 1445 total pagecache pages Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636205] 0 pages in swap cache Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636207] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636208] Free swap = 0kB Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636209] Total swap = 0kB Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641763] 985072 pages RAM Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641766] 24998 pages reserved Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641767] 3796 pages shared Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641768] 953342 pages non-shared Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641770] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641779] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641782] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641785] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641788] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641791] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641795] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641798] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641801] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641806] [ 790] 101 790 63477 615 1 0 0 rsyslogd Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641809] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641812] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641815] [ 975] 0 975 6276 79 1 0 0 master Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641818] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641822] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641825] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641828] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641831] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641834] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641837] [15657] 0 15657 4777 59 1 0 0 cron Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641841] [ 654] 1012 654 70351 5751 1 0 0 php Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641844] [ 657] 1012 657 69242 6258 1 0 0 php Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641847] [ 658] 1012 658 134393 70799 1 0 0 php Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641850] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641853] [13468] 0 13468 89584 2490 1 0 0 php5-fpm Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641856] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641859] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641863] [ 5841] 1012 5841 68618 5646 1 0 0 php Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641866] [13160] 108 13160 6792 76 1 0 0 pickup Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641869] [32048] 1012 32048 892364 829326 0 0 0 php Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641872] [32111] 1012 32111 68580 5606 1 0 0 php Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641875] Out of memory: Kill process 32048 (php) score 865 or sacrifice child Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641886] Killed process 32048 (php) total-vm:3569456kB, anon-rss:3317304kB, file-rss:0kB Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225034] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225039] php cpuset=/ mems_allowed=0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225042] Pid: 18795, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225043] Call Trace: Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225050] [] dump_header+0x91/0xe0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225053] [] oom_kill_process+0x85/0xb0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225055] [] out_of_memory+0xfa/0x220 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225058] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225063] [] ? error_exit+0x2a/0x60 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225065] [] alloc_pages_current+0xb6/0x120 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225069] [] __page_cache_alloc+0xb7/0xd0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225071] [] filemap_fault+0x212/0x3c0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225074] [] __do_fault+0x72/0x550 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225077] [] handle_pte_fault+0xfa/0x200 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225080] [] ? xen_pmd_val+0xe/0x10 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225082] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225085] [] handle_mm_fault+0x269/0x370 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225087] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225090] [] ? check_events+0x12/0x20 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225092] [] do_page_fault+0x184/0x550 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225094] [] ? do_brk+0x21f/0x320 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225096] [] ? xen_clts+0x8d/0x190 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225099] [] ? __schedule+0x3cc/0x6f0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225102] [] page_fault+0x25/0x30 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225104] Mem-Info: Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225105] Node 0 DMA per-cpu: Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225107] CPU 0: hi: 0, btch: 1 usd: 0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225108] CPU 1: hi: 0, btch: 1 usd: 0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225109] Node 0 DMA32 per-cpu: Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225111] CPU 0: hi: 186, btch: 31 usd: 30 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225112] CPU 1: hi: 186, btch: 31 usd: 30 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225114] active_anon:932295 inactive_anon:1393 isolated_anon:0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225115] active_file:22 inactive_file:0 isolated_file:0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225116] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225116] free:4038 slab_reclaimable:2915 slab_unreclaimable:2716 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225117] mapped:1371 shmem:1405 pagetables:3303 bounce:0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225119] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225125] lowmem_reserve[]: 0 3772 3772 3772 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225128] Node 0 DMA32 free:7816kB min:7848kB low:9808kB high:11772kB active_anon:3729180kB inactive_anon:5572kB active_file:88kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5484kB shmem:5620kB slab_reclaimable:11660kB slab_unreclaimable:10864kB kernel_stack:784kB pagetables:13212kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:264 all_unreclaimable? yes Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225135] lowmem_reserve[]: 0 0 0 0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225137] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225144] Node 0 DMA32: 1454*4kB 1*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7904kB Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225151] 1421 total pagecache pages Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225152] 0 pages in swap cache Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225153] Swap cache stats: add 0, delete 0, find 0/0 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225154] Free swap = 0kB Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225155] Total swap = 0kB Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230491] 985072 pages RAM Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230493] 24998 pages reserved Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230495] 3785 pages shared Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230496] 953273 pages non-shared Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230497] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230507] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230511] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230514] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230517] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230520] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230524] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230527] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230530] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230535] [ 790] 101 790 63477 615 1 0 0 rsyslogd Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230538] [ 827] 0 827 1081 36 1 0 0 acpid Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230541] [ 831] 0 831 4226 41 0 0 0 atd Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230544] [ 975] 0 975 6276 80 1 0 0 master Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230547] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230550] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230553] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230556] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230560] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230563] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230566] [15657] 0 15657 4777 60 0 0 0 cron Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230569] [ 654] 1012 654 70351 5750 1 0 0 php Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230572] [ 657] 1012 657 69242 6258 1 0 0 php Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230575] [ 658] 1012 658 134393 70799 1 0 0 php Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230578] [ 678] 1012 678 70152 5566 1 0 0 php Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230582] [13468] 0 13468 89584 2527 1 0 0 php5-fpm Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230585] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230588] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230591] [ 5841] 1012 5841 68618 5647 0 0 0 php Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230594] [ 1165] 108 1165 6792 70 0 0 0 pickup Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230597] [18795] 1012 18795 877707 814659 0 0 0 php Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230600] [19509] 1012 19509 70570 7579 1 0 0 php Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230603] [19625] 1012 19625 1099 25 1 0 0 sh Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230607] [19626] 1012 19626 68825 5871 1 0 0 php Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230610] [19632] 1012 19632 1099 25 1 0 0 sh Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230613] [19633] 1012 19633 4700 74 1 0 0 git Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230616] [19634] 1012 19634 68499 5552 1 0 0 php Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230619] [19636] 1012 19636 1099 25 1 0 0 sh Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230622] [19637] 1012 19637 10901 176 1 0 0 ssh Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230625] Out of memory: Kill process 18795 (php) score 850 or sacrifice child Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230636] Killed process 18795 (php) total-vm:3510828kB, anon-rss:3258636kB, file-rss:0kB Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231093] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231098] php cpuset=/ mems_allowed=0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231102] Pid: 24206, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231104] Call Trace: Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231113] [] dump_header+0x91/0xe0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231116] [] oom_kill_process+0x85/0xb0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231119] [] out_of_memory+0xfa/0x220 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231124] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231128] [] alloc_pages_current+0xb6/0x120 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231133] [] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231139] [] __page_cache_alloc+0xb7/0xd0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231142] [] filemap_fault+0x212/0x3c0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231146] [] __do_fault+0x72/0x550 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231149] [] ? load_TLS_descriptor+0x42/0xd0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231153] [] handle_pte_fault+0xfa/0x200 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231157] [] ? xen_pmd_val+0xe/0x10 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231160] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231164] [] handle_mm_fault+0x269/0x370 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231169] [] ? lock_hrtimer_base.isra.25+0x30/0x60 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231172] [] do_page_fault+0x184/0x550 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231176] [] ? do_nanosleep+0xa4/0xd0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231180] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231183] [] ? update_rmtp+0x70/0x70 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231186] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231190] [] page_fault+0x25/0x30 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231192] Mem-Info: Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231193] Node 0 DMA per-cpu: Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231196] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231198] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231200] Node 0 DMA32 per-cpu: Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231201] CPU 0: hi: 186, btch: 31 usd: 121 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231203] CPU 1: hi: 186, btch: 31 usd: 26 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231205] active_anon:932109 inactive_anon:1393 isolated_anon:0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231206] active_file:4 inactive_file:19 isolated_file:21 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231207] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231207] free:4043 slab_reclaimable:2896 slab_unreclaimable:2700 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231208] mapped:1396 shmem:1405 pagetables:3354 bounce:0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231210] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231216] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231219] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3728436kB inactive_anon:5572kB active_file:16kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):84kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5584kB shmem:5620kB slab_reclaimable:11584kB slab_unreclaimable:10800kB kernel_stack:808kB pagetables:13416kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:235 all_unreclaimable? yes Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231226] lowmem_reserve[]: 0 0 0 0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231229] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231235] Node 0 DMA32: 1455*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7868kB Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231242] 1454 total pagecache pages Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231243] 0 pages in swap cache Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231244] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231245] Free swap = 0kB Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231246] Total swap = 0kB Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236409] 985072 pages RAM Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236411] 24998 pages reserved Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236413] 3794 pages shared Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236414] 953183 pages non-shared Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236416] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236425] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236429] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236432] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236435] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236439] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236442] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236445] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236449] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236452] [ 790] 101 790 63477 620 0 0 0 rsyslogd Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236456] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236459] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236462] [ 975] 0 975 6276 80 1 0 0 master Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236465] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236468] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236472] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236475] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236478] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236481] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236485] [15657] 0 15657 4777 60 0 0 0 cron Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236488] [ 654] 1012 654 70351 5750 1 0 0 php Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236491] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236494] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236497] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236500] [13468] 0 13468 89584 2499 0 0 0 php5-fpm Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236503] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236506] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236509] [ 5841] 1012 5841 68618 5647 1 0 0 php Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236512] [19719] 1012 19719 886446 823222 0 0 0 php Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236516] [14792] 108 14792 6792 69 0 0 0 pickup Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236519] [23944] 108 23944 6818 74 0 0 0 cleanup Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236522] [23945] 108 23945 6795 70 1 0 0 trivial-rewrite Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236525] [23946] 108 23946 11107 158 0 0 0 smtp Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236528] [23950] 108 23950 11107 157 1 0 0 smtp Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236531] [23954] 108 23954 11107 158 0 0 0 smtp Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236534] [23958] 108 23958 11107 158 1 0 0 smtp Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236537] [23962] 108 23962 11107 159 1 0 0 smtp Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236540] [23966] 108 23966 6792 70 1 0 0 scache Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236543] [24206] 1012 24206 68580 5605 1 0 0 php Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236546] [24343] 1012 24343 1099 25 1 0 0 sh Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236550] [24344] 1012 24344 66354 3941 0 0 0 php Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236552] Out of memory: Kill process 19719 (php) score 859 or sacrifice child Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236564] Killed process 19719 (php) total-vm:3545784kB, anon-rss:3292888kB, file-rss:0kB Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260107] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260111] php cpuset=/ mems_allowed=0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260114] Pid: 24470, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260116] Call Trace: Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260122] [] dump_header+0x91/0xe0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260124] [] oom_kill_process+0x85/0xb0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260126] [] out_of_memory+0xfa/0x220 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260129] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260134] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260137] [] alloc_pages_current+0xb6/0x120 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260140] [] __page_cache_alloc+0xb7/0xd0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260143] [] filemap_fault+0x212/0x3c0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260146] [] __do_fault+0x72/0x550 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260150] [] ? error_exit+0x2a/0x60 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260152] [] handle_pte_fault+0xfa/0x200 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260155] [] ? xen_pmd_val+0xe/0x10 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260157] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260160] [] handle_mm_fault+0x269/0x370 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260163] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260165] [] ? check_events+0x12/0x20 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260167] [] do_page_fault+0x184/0x550 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260169] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260171] [] ? xen_clts+0x8d/0x190 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260174] [] ? __schedule+0x3cc/0x6f0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260177] [] page_fault+0x25/0x30 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260178] Mem-Info: Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260180] Node 0 DMA per-cpu: Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260181] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260183] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260184] Node 0 DMA32 per-cpu: Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260185] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260187] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260189] active_anon:932884 inactive_anon:1393 isolated_anon:0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260190] active_file:0 inactive_file:0 isolated_file:0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260191] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260191] free:4048 slab_reclaimable:2704 slab_unreclaimable:2611 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260192] mapped:1321 shmem:1405 pagetables:3076 bounce:0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260193] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260200] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260203] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731536kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5284kB shmem:5620kB slab_reclaimable:10816kB slab_unreclaimable:10444kB kernel_stack:728kB pagetables:12304kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260209] lowmem_reserve[]: 0 0 0 0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260212] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260218] Node 0 DMA32: 1222*4kB 107*8kB 2*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260225] 1461 total pagecache pages Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260226] 0 pages in swap cache Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260227] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260229] Free swap = 0kB Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260229] Total swap = 0kB Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265385] 985072 pages RAM Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265388] 24998 pages reserved Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265389] 3802 pages shared Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265390] 953335 pages non-shared Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265392] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265401] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265404] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265408] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265411] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265414] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265417] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265420] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265424] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265428] [ 790] 101 790 63477 620 0 0 0 rsyslogd Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265431] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265434] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265437] [ 975] 0 975 6276 80 0 0 0 master Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265440] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265443] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265446] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265449] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265453] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265456] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265459] [15657] 0 15657 4777 60 0 0 0 cron Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265462] [ 654] 1012 654 70351 5750 1 0 0 php Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265465] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265468] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265471] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265474] [13468] 0 13468 89584 2536 1 0 0 php5-fpm Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265477] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265480] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265483] [ 5841] 1012 5841 68618 5647 1 0 0 php Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265487] [14792] 108 14792 6792 75 1 0 0 pickup Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265490] [24470] 1012 24470 892106 828963 0 0 0 php Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265493] [31276] 1012 31276 68580 5604 1 0 0 php Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265496] Out of memory: Kill process 24470 (php) score 865 or sacrifice child Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265507] Killed process 24470 (php) total-vm:3568424kB, anon-rss:3315852kB, file-rss:0kB Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240238] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240245] php cpuset=/ mems_allowed=0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240248] Pid: 15998, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240250] Call Trace: Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240257] [] dump_header+0x91/0xe0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240259] [] oom_kill_process+0x85/0xb0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240261] [] out_of_memory+0xfa/0x220 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240264] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240269] [] ? error_exit+0x2a/0x60 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240272] [] alloc_pages_current+0xb6/0x120 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240275] [] __page_cache_alloc+0xb7/0xd0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240277] [] filemap_fault+0x212/0x3c0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240280] [] __do_fault+0x72/0x550 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240283] [] ? lru_cache_add_lru+0x2d/0x50 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240286] [] handle_pte_fault+0xfa/0x200 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240289] [] ? xen_pmd_val+0xe/0x10 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240291] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240294] [] handle_mm_fault+0x269/0x370 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240296] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240299] [] ? check_events+0x12/0x20 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240301] [] do_page_fault+0x184/0x550 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240303] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240305] [] ? xen_clts+0x8d/0x190 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240308] [] page_fault+0x25/0x30 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240309] Mem-Info: Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240311] Node 0 DMA per-cpu: Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240313] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240314] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240315] Node 0 DMA32 per-cpu: Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240317] CPU 0: hi: 186, btch: 31 usd: 30 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240318] CPU 1: hi: 186, btch: 31 usd: 29 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240321] active_anon:932821 inactive_anon:1393 isolated_anon:0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240321] active_file:0 inactive_file:42 isolated_file:0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240322] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240323] free:4019 slab_reclaimable:2807 slab_unreclaimable:2638 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240323] mapped:1360 shmem:1405 pagetables:2979 bounce:0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240325] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240331] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240334] Node 0 DMA32 free:7740kB min:7848kB low:9808kB high:11772kB active_anon:3731284kB inactive_anon:5572kB active_file:0kB inactive_file:168kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5440kB shmem:5620kB slab_reclaimable:11228kB slab_unreclaimable:10552kB kernel_stack:728kB pagetables:11916kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? yes Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240340] lowmem_reserve[]: 0 0 0 0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240343] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240350] Node 0 DMA32: 1411*4kB 4*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7740kB Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240356] 1497 total pagecache pages Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240358] 0 pages in swap cache Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240359] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240360] Free swap = 0kB Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240361] Total swap = 0kB Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245603] 985072 pages RAM Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245605] 24998 pages reserved Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245606] 3802 pages shared Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245608] 953306 pages non-shared Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245609] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245618] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245622] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245625] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245628] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245631] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245635] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245638] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245641] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245645] [ 790] 101 790 63477 617 0 0 0 rsyslogd Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245648] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245651] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245655] [ 975] 0 975 6276 80 1 0 0 master Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245657] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245660] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245664] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245667] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245670] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245673] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245676] [15657] 0 15657 4777 60 0 0 0 cron Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245680] [ 654] 1012 654 70351 5757 1 0 0 php Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245683] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245686] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245689] [ 678] 1012 678 70152 5567 1 0 0 php Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245692] [13468] 0 13468 89584 2521 1 0 0 php5-fpm Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245695] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245698] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245701] [ 5841] 1012 5841 68618 5649 1 0 0 php Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245704] [14792] 108 14792 6792 75 0 0 0 pickup Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245707] [15998] 1012 15998 897548 834502 0 0 0 php Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245710] Out of memory: Kill process 15998 (php) score 870 or sacrifice child Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245721] Killed process 15998 (php) total-vm:3590192kB, anon-rss:3338008kB, file-rss:0kB Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162741] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162746] php cpuset=/ mems_allowed=0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162749] Pid: 31567, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162751] Call Trace: Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162759] [] dump_header+0x91/0xe0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162762] [] oom_kill_process+0x85/0xb0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162765] [] out_of_memory+0xfa/0x220 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162769] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162773] [] alloc_pages_current+0xb6/0x120 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162778] [] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162783] [] __page_cache_alloc+0xb7/0xd0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162786] [] filemap_fault+0x212/0x3c0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162789] [] __do_fault+0x72/0x550 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162793] [] ? load_TLS_descriptor+0x42/0xd0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162796] [] handle_pte_fault+0xfa/0x200 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162799] [] ? xen_pmd_val+0xe/0x10 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162802] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162806] [] handle_mm_fault+0x269/0x370 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162811] [] ? lock_hrtimer_base.isra.25+0x30/0x60 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162814] [] do_page_fault+0x184/0x550 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162818] [] ? do_nanosleep+0xa4/0xd0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162821] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162825] [] ? update_rmtp+0x70/0x70 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162828] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162832] [] page_fault+0x25/0x30 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162834] Mem-Info: Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162835] Node 0 DMA per-cpu: Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162838] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162840] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162841] Node 0 DMA32 per-cpu: Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162844] CPU 0: hi: 186, btch: 31 usd: 82 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162845] CPU 1: hi: 186, btch: 31 usd: 30 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162849] active_anon:932069 inactive_anon:1393 isolated_anon:0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162850] active_file:14 inactive_file:3 isolated_file:25 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162851] unevictable:0 dirty:2 writeback:0 unstable:0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162852] free:4043 slab_reclaimable:2858 slab_unreclaimable:2732 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162853] mapped:1402 shmem:1405 pagetables:3385 bounce:0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162855] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162863] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162867] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3728276kB inactive_anon:5572kB active_file:56kB inactive_file:12kB unevictable:0kB isolated(anon):0kB isolated(file):100kB present:3862656kB mlocked:0kB dirty:8kB writeback:0kB mapped:5608kB shmem:5620kB slab_reclaimable:11432kB slab_unreclaimable:10928kB kernel_stack:816kB pagetables:13540kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:208 all_unreclaimable? yes Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162875] lowmem_reserve[]: 0 0 0 0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162878] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162885] Node 0 DMA32: 1447*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7836kB Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162891] 1436 total pagecache pages Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162893] 0 pages in swap cache Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162894] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162895] Free swap = 0kB Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162896] Total swap = 0kB Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168057] 985072 pages RAM Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168059] 24998 pages reserved Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168060] 3817 pages shared Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168061] 953215 pages non-shared Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168063] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168072] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168075] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168079] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168082] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168085] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168088] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168092] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168095] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168099] [ 790] 101 790 63477 617 0 0 0 rsyslogd Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168102] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168105] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168108] [ 975] 0 975 6276 80 1 0 0 master Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168111] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168114] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168118] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168121] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168124] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168127] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168130] [15657] 0 15657 4777 60 0 0 0 cron Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168134] [ 654] 1012 654 70351 5757 1 0 0 php Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168137] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168140] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168143] [ 678] 1012 678 70152 5567 0 0 0 php Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168146] [13468] 0 13468 89584 2506 1 0 0 php5-fpm Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168149] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168152] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168155] [21570] 1012 21570 884073 820966 1 0 0 php Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168158] [23281] 108 23281 6792 70 1 0 0 pickup Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168161] [ 2686] 1012 2686 68618 5646 1 0 0 php Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168165] [30252] 108 30252 6818 73 0 0 0 cleanup Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168168] [30253] 108 30253 6795 71 0 0 0 trivial-rewrite Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168171] [30254] 108 30254 11107 159 1 0 0 smtp Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168174] [30258] 108 30258 11107 158 0 0 0 smtp Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168177] [30262] 108 30262 11107 158 1 0 0 smtp Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168180] [31567] 1012 31567 68580 5604 1 0 0 php Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168183] [31837] 1012 31837 1099 24 1 0 0 sh Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168186] [31839] 1012 31839 68825 5868 1 0 0 php Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168189] [31851] 1012 31851 1099 25 1 0 0 sh Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168192] [31852] 1012 31852 4655 58 1 0 0 git Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168195] [31853] 1012 31853 43811 625 0 0 0 php Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168198] Out of memory: Kill process 21570 (php) score 856 or sacrifice child Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168209] Killed process 21570 (php) total-vm:3536292kB, anon-rss:3283760kB, file-rss:104kB Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424139] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424144] php cpuset=/ mems_allowed=0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424147] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424149] Call Trace: Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424157] [] dump_header+0x91/0xe0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424160] [] oom_kill_process+0x85/0xb0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424163] [] out_of_memory+0xfa/0x220 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424166] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424169] [] alloc_pages_current+0xb6/0x120 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424173] [] __page_cache_alloc+0xb7/0xd0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424175] [] filemap_fault+0x212/0x3c0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424178] [] __do_fault+0x72/0x550 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424180] [] handle_pte_fault+0xfa/0x200 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424184] [] ? xen_pmd_val+0xe/0x10 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424186] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424189] [] handle_mm_fault+0x269/0x370 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424191] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424194] [] ? check_events+0x12/0x20 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424197] [] do_page_fault+0x184/0x550 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424201] [] ? vfs_write+0x110/0x180 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424204] [] page_fault+0x25/0x30 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424205] Mem-Info: Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424207] Node 0 DMA per-cpu: Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424209] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424210] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424211] Node 0 DMA32 per-cpu: Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424213] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424214] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424216] active_anon:932401 inactive_anon:1393 isolated_anon:0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424217] active_file:34 inactive_file:40 isolated_file:0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424218] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424218] free:4048 slab_reclaimable:2988 slab_unreclaimable:2679 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424219] mapped:1381 shmem:1405 pagetables:3143 bounce:0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424221] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424227] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424230] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729604kB inactive_anon:5572kB active_file:136kB inactive_file:160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5524kB shmem:5620kB slab_reclaimable:11952kB slab_unreclaimable:10716kB kernel_stack:760kB pagetables:12572kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:28808 all_unreclaimable? yes Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424237] lowmem_reserve[]: 0 0 0 0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424239] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424246] Node 0 DMA32: 1454*4kB 14*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7992kB Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424253] 1484 total pagecache pages Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424254] 0 pages in swap cache Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424255] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424256] Free swap = 0kB Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424257] Total swap = 0kB Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429439] 985072 pages RAM Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429441] 24998 pages reserved Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429442] 3824 pages shared Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429443] 953277 pages non-shared Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429445] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429454] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429458] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429461] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429464] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429467] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429471] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429474] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429478] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429482] [ 790] 101 790 63477 620 1 0 0 rsyslogd Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429485] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429488] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429491] [ 975] 0 975 6276 80 1 0 0 master Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429495] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429498] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429501] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429504] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429507] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429510] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429514] [15657] 0 15657 4777 60 0 0 0 cron Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429517] [ 654] 1012 654 70351 5758 1 0 0 php Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429520] [ 657] 1012 657 69242 6257 0 0 0 php Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429523] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429526] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429529] [13468] 0 13468 89584 2491 1 0 0 php5-fpm Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429532] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429535] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429538] [ 2686] 1012 2686 68618 5646 0 0 0 php Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429542] [11415] 1012 11415 891137 828027 1 0 0 php Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429545] [19313] 108 19313 6792 70 0 0 0 pickup Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429548] [23459] 108 23459 6818 73 0 0 0 cleanup Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429551] [23460] 108 23460 6795 71 0 0 0 trivial-rewrite Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429555] [23461] 108 23461 11107 143 1 0 0 smtp Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429558] [23465] 108 23465 11107 158 1 0 0 smtp Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429561] [24144] 1012 24144 68580 5605 1 0 0 php Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429564] Out of memory: Kill process 11415 (php) score 864 or sacrifice child Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429575] Killed process 11415 (php) total-vm:3564548kB, anon-rss:3312084kB, file-rss:24kB Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336304] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336308] php cpuset=/ mems_allowed=0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336311] Pid: 24495, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336313] Call Trace: Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336320] [] dump_header+0x91/0xe0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336322] [] oom_kill_process+0x85/0xb0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336324] [] out_of_memory+0xfa/0x220 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336327] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336330] [] alloc_pages_current+0xb6/0x120 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336334] [] __page_cache_alloc+0xb7/0xd0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336336] [] filemap_fault+0x212/0x3c0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336339] [] __do_fault+0x72/0x550 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336342] [] ? lru_cache_add_lru+0x2d/0x50 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336344] [] handle_pte_fault+0xfa/0x200 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336347] [] ? xen_pmd_val+0xe/0x10 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336349] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336352] [] handle_mm_fault+0x269/0x370 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336355] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336357] [] ? check_events+0x12/0x20 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336360] [] do_page_fault+0x184/0x550 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336363] [] ? do_brk+0x21f/0x320 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336366] [] page_fault+0x25/0x30 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336368] Mem-Info: Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336369] Node 0 DMA per-cpu: Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336371] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336372] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336373] Node 0 DMA32 per-cpu: Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336375] CPU 0: hi: 186, btch: 31 usd: 30 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336376] CPU 1: hi: 186, btch: 31 usd: 25 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336379] active_anon:932097 inactive_anon:1393 isolated_anon:0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336379] active_file:0 inactive_file:13 isolated_file:0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336380] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336381] free:4021 slab_reclaimable:2824 slab_unreclaimable:2731 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336381] mapped:1385 shmem:1405 pagetables:3444 bounce:0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336383] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336389] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336392] Node 0 DMA32 free:7748kB min:7848kB low:9808kB high:11772kB active_anon:3728388kB inactive_anon:5572kB active_file:0kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5540kB shmem:5620kB slab_reclaimable:11296kB slab_unreclaimable:10924kB kernel_stack:840kB pagetables:13776kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:205 all_unreclaimable? yes Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336399] lowmem_reserve[]: 0 0 0 0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336401] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336408] Node 0 DMA32: 1356*4kB 42*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7840kB Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336415] 1463 total pagecache pages Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336416] 0 pages in swap cache Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336417] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336418] Free swap = 0kB Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336419] Total swap = 0kB Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341656] 985072 pages RAM Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341659] 24998 pages reserved Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341660] 3794 pages shared Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341661] 953278 pages non-shared Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341663] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341672] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341675] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341678] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341681] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341685] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341688] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341691] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341695] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341699] [ 790] 101 790 63477 623 0 0 0 rsyslogd Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341702] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341705] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341709] [ 975] 0 975 6276 80 0 0 0 master Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341712] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341715] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341718] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341721] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341724] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341728] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341731] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341734] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341737] [ 657] 1012 657 69242 6258 0 0 0 php Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341740] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341743] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341746] [13468] 0 13468 89584 2531 0 0 0 php5-fpm Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341749] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341752] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341756] [ 2686] 1012 2686 68618 5646 1 0 0 php Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341759] [19313] 108 19313 6792 76 0 0 0 pickup Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341762] [24495] 1012 24495 878725 815583 0 0 0 php Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341765] [16057] 108 16057 6795 71 0 0 0 trivial-rewrite Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341768] [16066] 108 16066 6792 73 0 0 0 scache Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341771] [17771] 108 17771 6818 73 1 0 0 cleanup Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341774] [17772] 108 17772 11107 158 0 0 0 smtp Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341777] [17776] 108 17776 11107 159 1 0 0 smtp Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341781] [17780] 108 17780 11107 159 1 0 0 smtp Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341784] [17784] 108 17784 11107 159 0 0 0 smtp Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341786] [19044] 1012 19044 68580 5603 1 0 0 php Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341790] [19260] 1012 19260 1099 25 1 0 0 sh Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341793] [19261] 1012 19261 68825 5871 1 0 0 php Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341795] [19272] 1012 19272 1099 25 1 0 0 sh Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341799] [19274] 1012 19274 4651 62 0 0 0 git Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341801] [19276] 1012 19276 68499 5552 1 0 0 php Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341804] [19280] 1012 19280 1099 24 1 0 0 sh Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341807] [19282] 1012 19282 10901 176 1 0 0 ssh Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341810] Out of memory: Kill process 24495 (php) score 851 or sacrifice child Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341820] Killed process 24495 (php) total-vm:3514900kB, anon-rss:3262332kB, file-rss:0kB Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437353] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437358] php5-fpm cpuset=/ mems_allowed=0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437361] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437363] Call Trace: Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437372] [] dump_header+0x91/0xe0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437375] [] oom_kill_process+0x85/0xb0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437377] [] out_of_memory+0xfa/0x220 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437380] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437383] [] alloc_pages_current+0xb6/0x120 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437386] [] __page_cache_alloc+0xb7/0xd0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437388] [] filemap_fault+0x212/0x3c0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437391] [] __do_fault+0x72/0x550 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437395] [] ? __switch_to+0x138/0x360 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437398] [] ? set_next_entity+0xad/0xd0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437400] [] handle_pte_fault+0xfa/0x200 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437403] [] ? xen_pmd_val+0xe/0x10 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437406] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437408] [] handle_mm_fault+0x269/0x370 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437412] [] do_page_fault+0x184/0x550 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437415] [] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437419] [] ? ep_poll+0x1cc/0x380 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437422] [] ? try_to_wake_up+0x200/0x200 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437425] [] page_fault+0x25/0x30 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437427] Mem-Info: Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437428] Node 0 DMA per-cpu: Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437430] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437431] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437433] Node 0 DMA32 per-cpu: Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437434] CPU 0: hi: 186, btch: 31 usd: 38 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437436] CPU 1: hi: 186, btch: 31 usd: 99 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437438] active_anon:931603 inactive_anon:1393 isolated_anon:0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437439] active_file:0 inactive_file:0 isolated_file:20 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437440] unevictable:0 dirty:47 writeback:0 unstable:0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437440] free:4048 slab_reclaimable:2644 slab_unreclaimable:2773 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437441] mapped:1373 shmem:1405 pagetables:3872 bounce:0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437443] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437449] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437452] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3726412kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):80kB present:3862656kB mlocked:0kB dirty:188kB writeback:0kB mapped:5492kB shmem:5620kB slab_reclaimable:10576kB slab_unreclaimable:11092kB kernel_stack:896kB pagetables:15488kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:206 all_unreclaimable? yes Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437459] lowmem_reserve[]: 0 0 0 0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437461] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437468] Node 0 DMA32: 514*4kB 169*8kB 23*16kB 20*32kB 14*64kB 4*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7872kB Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437475] 1437 total pagecache pages Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437476] 0 pages in swap cache Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437477] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437479] Free swap = 0kB Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437479] Total swap = 0kB Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442619] 985072 pages RAM Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442622] 24998 pages reserved Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442623] 3783 pages shared Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442624] 953191 pages non-shared Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442626] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442635] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442638] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442642] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442645] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442648] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442651] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442654] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442658] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442662] [ 790] 101 790 63477 623 1 0 0 rsyslogd Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442665] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442668] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442671] [ 975] 0 975 6276 79 0 0 0 master Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442675] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442678] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442681] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442684] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442687] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442690] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442693] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442697] [ 654] 1012 654 70351 5759 0 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442700] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442703] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442706] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442709] [13468] 0 13468 89584 2511 0 0 0 php5-fpm Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442712] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442715] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442719] [ 2686] 1012 2686 68618 5645 1 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442722] [19313] 108 19313 6792 76 0 0 0 pickup Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442725] [16057] 108 16057 6795 71 0 0 0 trivial-rewrite Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442728] [17771] 108 17771 6818 73 1 0 0 cleanup Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442732] [17772] 108 17772 11107 158 0 0 0 smtp Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442735] [17776] 108 17776 11107 159 1 0 0 smtp Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442738] [17780] 108 17780 11107 159 1 0 0 smtp Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442741] [17784] 108 17784 11107 159 0 0 0 smtp Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442744] [19323] 1012 19323 664126 601020 0 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442747] [19396] 1012 19396 263044 199931 0 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442750] [19619] 1012 19619 72647 9704 1 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442753] [19631] 1012 19631 1099 25 0 0 0 sh Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442756] [19632] 1012 19632 68825 5873 1 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442759] [19633] 1012 19633 1099 25 0 0 0 sh Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442762] [19634] 1012 19634 68825 5872 1 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442765] [19635] 1012 19635 1099 25 0 0 0 sh Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442768] [19637] 1012 19637 68825 5871 0 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442771] [19644] 1012 19644 1099 26 0 0 0 sh Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442774] [19645] 1012 19645 4655 58 0 0 0 git Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442777] [19646] 1012 19646 65507 3090 1 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442780] [19655] 1012 19655 1099 26 1 0 0 sh Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442783] [19656] 1012 19656 14040 63 1 0 0 git Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442785] [19658] 1012 19658 31450 417 1 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442789] [19659] 1012 19659 1099 25 1 0 0 sh Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442792] [19660] 1012 19660 4649 58 0 0 0 git Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442795] [19661] 1012 19661 23521 347 0 0 0 php Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442797] Out of memory: Kill process 19323 (php) score 627 or sacrifice child Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442808] Killed process 19323 (php) total-vm:2656504kB, anon-rss:2404080kB, file-rss:0kB Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856151] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856155] php cpuset=/ mems_allowed=0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856159] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856161] Call Trace: Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856168] [] dump_header+0x91/0xe0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856172] [] oom_kill_process+0x85/0xb0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856175] [] out_of_memory+0xfa/0x220 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856179] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856184] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856188] [] alloc_pages_current+0xb6/0x120 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856193] [] __page_cache_alloc+0xb7/0xd0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856196] [] filemap_fault+0x212/0x3c0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856200] [] __do_fault+0x72/0x550 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856204] [] ? __pollwait+0xf0/0xf0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856207] [] handle_pte_fault+0xfa/0x200 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856211] [] ? xen_pmd_val+0xe/0x10 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856214] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856217] [] handle_mm_fault+0x269/0x370 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856221] [] do_page_fault+0x184/0x550 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856225] [] ? ktime_get_ts+0xad/0xe0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856229] [] ? poll_select_copy_remaining+0xed/0x140 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856232] [] ? sys_select+0xcd/0x100 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856235] [] page_fault+0x25/0x30 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856237] Mem-Info: Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856238] Node 0 DMA per-cpu: Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856240] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856242] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856243] Node 0 DMA32 per-cpu: Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856244] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856245] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856248] active_anon:932629 inactive_anon:1393 isolated_anon:0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856249] active_file:9 inactive_file:11 isolated_file:0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856249] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856250] free:4048 slab_reclaimable:2581 slab_unreclaimable:2674 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856251] mapped:1354 shmem:1405 pagetables:3305 bounce:0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856252] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856259] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856262] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730516kB inactive_anon:5572kB active_file:36kB inactive_file:44kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5620kB slab_reclaimable:10324kB slab_unreclaimable:10696kB kernel_stack:784kB pagetables:13220kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:8675 all_unreclaimable? yes Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856268] lowmem_reserve[]: 0 0 0 0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856271] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856277] Node 0 DMA32: 756*4kB 272*8kB 40*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8016kB Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856284] 1426 total pagecache pages Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856285] 0 pages in swap cache Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856287] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856288] Free swap = 0kB Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856289] Total swap = 0kB Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861464] 985072 pages RAM Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861466] 24998 pages reserved Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861467] 3801 pages shared Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861468] 953289 pages non-shared Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861471] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861479] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861482] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861485] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861488] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861492] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861495] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861498] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861501] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861505] [ 790] 101 790 63477 623 1 0 0 rsyslogd Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861509] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861512] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861515] [ 975] 0 975 6276 79 0 0 0 master Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861518] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861521] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861524] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861527] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861530] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861534] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861537] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861540] [ 654] 1012 654 70351 5762 1 0 0 php Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861543] [ 657] 1012 657 69242 6257 0 0 0 php Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861546] [ 658] 1012 658 134393 70798 0 0 0 php Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861549] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861552] [13468] 0 13468 89584 2502 0 0 0 php5-fpm Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861555] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861558] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861561] [ 2686] 1012 2686 68618 5645 1 0 0 php Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861565] [19313] 108 19313 6792 76 0 0 0 pickup Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861568] [16057] 108 16057 6795 75 1 0 0 trivial-rewrite Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861571] [17771] 108 17771 6818 81 1 0 0 cleanup Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861574] [17772] 108 17772 11107 157 1 0 0 smtp Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861577] [17776] 108 17776 11107 158 0 0 0 smtp Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861580] [17780] 108 17780 11107 158 1 0 0 smtp Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861583] [17784] 108 17784 11107 159 0 0 0 smtp Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861587] [19396] 1012 19396 875562 812467 0 0 0 php Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861590] [19619] 1012 19619 73702 10752 1 0 0 php Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861593] [19761] 1012 19761 73267 10297 1 0 0 php Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861595] Out of memory: Kill process 19396 (php) score 848 or sacrifice child Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861607] Killed process 19396 (php) total-vm:3502248kB, anon-rss:3249868kB, file-rss:0kB Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496281] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496285] php cpuset=/ mems_allowed=0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496288] Pid: 3247, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496289] Call Trace: Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496296] [] dump_header+0x91/0xe0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496299] [] oom_kill_process+0x85/0xb0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496301] [] out_of_memory+0xfa/0x220 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496304] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496307] [] alloc_pages_current+0xb6/0x120 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496310] [] __page_cache_alloc+0xb7/0xd0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496312] [] filemap_fault+0x212/0x3c0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496315] [] __do_fault+0x72/0x550 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496318] [] handle_pte_fault+0xfa/0x200 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496321] [] ? xen_pmd_val+0xe/0x10 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496323] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496326] [] handle_mm_fault+0x269/0x370 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496329] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496331] [] ? check_events+0x12/0x20 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496334] [] do_page_fault+0x184/0x550 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496336] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496338] [] ? xen_clts+0x8d/0x190 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496341] [] ? __schedule+0x3cc/0x6f0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496344] [] page_fault+0x25/0x30 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496345] Mem-Info: Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496347] Node 0 DMA per-cpu: Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496349] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496350] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496351] Node 0 DMA32 per-cpu: Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496353] CPU 0: hi: 186, btch: 31 usd: 30 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496354] CPU 1: hi: 186, btch: 31 usd: 31 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496356] active_anon:932673 inactive_anon:1393 isolated_anon:0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496357] active_file:0 inactive_file:0 isolated_file:0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496358] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496358] free:4044 slab_reclaimable:2874 slab_unreclaimable:2641 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496359] mapped:1363 shmem:1405 pagetables:3073 bounce:0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496361] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496367] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496370] Node 0 DMA32 free:7840kB min:7848kB low:9808kB high:11772kB active_anon:3730692kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5452kB shmem:5620kB slab_reclaimable:11496kB slab_unreclaimable:10564kB kernel_stack:768kB pagetables:12292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:75 all_unreclaimable? yes Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496377] lowmem_reserve[]: 0 0 0 0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496379] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496386] Node 0 DMA32: 1434*4kB 7*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7840kB Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496392] 1429 total pagecache pages Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496394] 0 pages in swap cache Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496395] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496396] Free swap = 0kB Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496397] Total swap = 0kB Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501595] 985072 pages RAM Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501597] 24998 pages reserved Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501599] 3784 pages shared Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501600] 953292 pages non-shared Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501602] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501610] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501614] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501617] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501620] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501623] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501627] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501630] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501633] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501638] [ 790] 101 790 63477 623 1 0 0 rsyslogd Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501641] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501644] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501647] [ 975] 0 975 6276 80 0 0 0 master Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501650] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501653] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501657] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501660] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501663] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501666] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501669] [15657] 0 15657 4777 60 1 0 0 cron Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501672] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501675] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501679] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501682] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501685] [13468] 0 13468 89584 2526 1 0 0 php5-fpm Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501688] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501691] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501694] [ 2686] 1012 2686 68618 5646 1 0 0 php Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501697] [30422] 108 30422 6792 70 0 0 0 pickup Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501700] [ 3247] 1012 3247 896947 833822 0 0 0 php Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501704] [24042] 108 24042 6792 70 0 0 0 scache Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501707] [26329] 108 26329 6818 72 0 0 0 cleanup Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501710] [26330] 108 26330 6795 69 1 0 0 trivial-rewrite Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501713] [26331] 108 26331 11107 158 1 0 0 smtp Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501716] [26335] 108 26335 11107 158 1 0 0 smtp Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501719] Out of memory: Kill process 3247 (php) score 870 or sacrifice child Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501729] Killed process 3247 (php) total-vm:3587788kB, anon-rss:3335288kB, file-rss:0kB Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824128] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824132] php cpuset=/ mems_allowed=0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824136] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824138] Call Trace: Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824145] [] dump_header+0x91/0xe0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824148] [] oom_kill_process+0x85/0xb0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824151] [] out_of_memory+0xfa/0x220 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824155] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824161] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824165] [] alloc_pages_current+0xb6/0x120 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824169] [] __page_cache_alloc+0xb7/0xd0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824172] [] filemap_fault+0x212/0x3c0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824175] [] __do_fault+0x72/0x550 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824179] [] ? error_exit+0x2a/0x60 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824182] [] ? xen_hypervisor_callback+0x1b/0x20 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824185] [] handle_pte_fault+0xfa/0x200 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824188] [] ? xen_pmd_val+0xe/0x10 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824190] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824193] [] handle_mm_fault+0x269/0x370 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824196] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824198] [] ? check_events+0x12/0x20 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824200] [] do_page_fault+0x184/0x550 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824204] [] ? security_file_permission+0x93/0xb0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824207] [] ? rw_verify_area+0x61/0xf0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824209] [] ? vfs_read+0xb0/0x180 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824211] [] page_fault+0x25/0x30 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824213] Mem-Info: Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824214] Node 0 DMA per-cpu: Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824216] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824218] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824219] Node 0 DMA32 per-cpu: Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824220] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824222] CPU 1: hi: 186, btch: 31 usd: 1 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824224] active_anon:932702 inactive_anon:1393 isolated_anon:0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824225] active_file:15 inactive_file:18 isolated_file:8 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824226] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824226] free:4047 slab_reclaimable:2694 slab_unreclaimable:2644 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824227] mapped:1356 shmem:1405 pagetables:3170 bounce:0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824228] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824235] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824238] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3730808kB inactive_anon:5572kB active_file:60kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):32kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:10776kB slab_unreclaimable:10576kB kernel_stack:760kB pagetables:12680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:932 all_unreclaimable? yes Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824244] lowmem_reserve[]: 0 0 0 0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824247] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824253] Node 0 DMA32: 1479*4kB 5*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8004kB Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824260] 1439 total pagecache pages Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824261] 0 pages in swap cache Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824262] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824264] Free swap = 0kB Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824264] Total swap = 0kB Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829519] 985072 pages RAM Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829522] 24998 pages reserved Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829523] 3778 pages shared Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829525] 953317 pages non-shared Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829526] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829534] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829538] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829541] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829544] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829547] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829551] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829554] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829557] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829562] [ 790] 101 790 63477 623 0 0 0 rsyslogd Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829565] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829568] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829571] [ 975] 0 975 6276 80 0 0 0 master Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829574] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829577] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829580] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829584] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829587] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829590] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829593] [15657] 0 15657 4777 60 1 0 0 cron Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829596] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829599] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829603] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829606] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829609] [13468] 0 13468 89584 2531 1 0 0 php5-fpm Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829612] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829615] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829618] [ 2686] 1012 2686 68618 5647 1 0 0 php Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829621] [29283] 108 29283 6792 69 1 0 0 pickup Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829624] [ 9277] 1012 9277 891230 828089 0 0 0 php Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829628] [19999] 1012 19999 68580 5604 1 0 0 php Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829631] [20179] 1012 20179 1099 24 1 0 0 sh Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829634] [20180] 1012 20180 28546 368 0 0 0 php Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829637] [20181] 1012 20181 1099 25 1 0 0 sh Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829640] [20182] 1012 20182 12611 262 1 0 0 php Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829642] Out of memory: Kill process 9277 (php) score 864 or sacrifice child Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829653] Killed process 9277 (php) total-vm:3564920kB, anon-rss:3312356kB, file-rss:0kB Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288402] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288406] php cpuset=/ mems_allowed=0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288409] Pid: 3903, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288410] Call Trace: Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288418] [] dump_header+0x91/0xe0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288420] [] oom_kill_process+0x85/0xb0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288422] [] out_of_memory+0xfa/0x220 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288425] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288428] [] alloc_pages_current+0xb6/0x120 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288432] [] __page_cache_alloc+0xb7/0xd0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288434] [] filemap_fault+0x212/0x3c0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288437] [] __do_fault+0x72/0x550 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288440] [] handle_pte_fault+0xfa/0x200 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288443] [] ? xen_pmd_val+0xe/0x10 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288445] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288448] [] handle_mm_fault+0x269/0x370 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288451] [] do_page_fault+0x184/0x550 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288453] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288455] [] ? xen_clts+0x8d/0x190 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288458] [] ? __schedule+0x3cc/0x6f0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288461] [] page_fault+0x25/0x30 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288463] Mem-Info: Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288464] Node 0 DMA per-cpu: Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288466] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288467] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288468] Node 0 DMA32 per-cpu: Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288470] CPU 0: hi: 186, btch: 31 usd: 13 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288471] CPU 1: hi: 186, btch: 31 usd: 54 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288474] active_anon:932501 inactive_anon:1393 isolated_anon:0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288474] active_file:0 inactive_file:53 isolated_file:0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288475] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288476] free:4018 slab_reclaimable:2626 slab_unreclaimable:2649 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288476] mapped:1381 shmem:1405 pagetables:3354 bounce:0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288478] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288484] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288487] Node 0 DMA32 free:7736kB min:7848kB low:9808kB high:11772kB active_anon:3730004kB inactive_anon:5572kB active_file:0kB inactive_file:212kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5524kB shmem:5620kB slab_reclaimable:10504kB slab_unreclaimable:10596kB kernel_stack:808kB pagetables:13416kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:419 all_unreclaimable? yes Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288494] lowmem_reserve[]: 0 0 0 0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288496] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288503] Node 0 DMA32: 1308*4kB 57*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7736kB Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288510] 1438 total pagecache pages Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288511] 0 pages in swap cache Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288512] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288513] Free swap = 0kB Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288514] Total swap = 0kB Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294147] 985072 pages RAM Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294150] 24998 pages reserved Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294151] 3775 pages shared Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294152] 953315 pages non-shared Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294154] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294163] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294166] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294170] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294173] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294176] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294179] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294182] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294186] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294190] [ 790] 101 790 63477 623 1 0 0 rsyslogd Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294193] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294196] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294199] [ 975] 0 975 6276 80 0 0 0 master Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294202] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294205] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294208] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294212] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294215] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294218] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294221] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294224] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294227] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294230] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294234] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294237] [13468] 0 13468 89584 2536 0 0 0 php5-fpm Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294240] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294243] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294246] [ 2686] 1012 2686 68618 5647 1 0 0 php Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294249] [ 3903] 1012 3903 886275 823228 0 0 0 php Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294252] [ 4486] 108 4486 6818 74 0 0 0 cleanup Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294255] [ 4487] 108 4487 6795 70 1 0 0 trivial-rewrite Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294258] [ 4488] 108 4488 11107 158 0 0 0 smtp Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294262] [ 4492] 108 4492 11107 159 1 0 0 smtp Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294265] [ 4496] 108 4496 11107 158 0 0 0 smtp Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294268] [ 4501] 108 4501 11107 142 0 0 0 smtp Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294271] [ 4505] 108 4505 6792 70 0 0 0 pickup Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294274] [ 4506] 108 4506 11107 159 0 0 0 smtp Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294277] [ 4510] 108 4510 6792 70 1 0 0 scache Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294280] [ 5491] 1012 5491 68580 5604 0 0 0 php Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294283] [ 5725] 1012 5725 1099 25 1 0 0 sh Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294286] [ 5726] 1012 5726 67296 4334 1 0 0 php Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294289] Out of memory: Kill process 3903 (php) score 859 or sacrifice child Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294299] Killed process 3903 (php) total-vm:3545100kB, anon-rss:3292912kB, file-rss:0kB Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924413] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924417] php cpuset=/ mems_allowed=0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924420] Pid: 5880, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924421] Call Trace: Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924429] [] dump_header+0x91/0xe0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924431] [] oom_kill_process+0x85/0xb0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924433] [] out_of_memory+0xfa/0x220 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924436] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924439] [] alloc_pages_current+0xb6/0x120 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924443] [] __page_cache_alloc+0xb7/0xd0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924445] [] filemap_fault+0x212/0x3c0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924448] [] __do_fault+0x72/0x550 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924452] [] ? error_exit+0x2a/0x60 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924455] [] handle_pte_fault+0xfa/0x200 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924458] [] ? xen_pmd_val+0xe/0x10 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924460] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924463] [] handle_mm_fault+0x269/0x370 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924465] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924468] [] ? check_events+0x12/0x20 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924470] [] do_page_fault+0x184/0x550 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924472] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924474] [] ? xen_clts+0x8d/0x190 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924477] [] ? __schedule+0x3cc/0x6f0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924479] [] page_fault+0x25/0x30 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924481] Mem-Info: Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924482] Node 0 DMA per-cpu: Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924484] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924486] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924487] Node 0 DMA32 per-cpu: Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924488] CPU 0: hi: 186, btch: 31 usd: 30 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924490] CPU 1: hi: 186, btch: 31 usd: 58 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924492] active_anon:932901 inactive_anon:1393 isolated_anon:0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924493] active_file:0 inactive_file:0 isolated_file:0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924494] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924494] free:4044 slab_reclaimable:2363 slab_unreclaimable:2633 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924495] mapped:1361 shmem:1405 pagetables:3193 bounce:0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924496] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924503] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924506] Node 0 DMA32 free:7840kB min:7848kB low:9808kB high:11772kB active_anon:3731604kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5620kB slab_reclaimable:9452kB slab_unreclaimable:10532kB kernel_stack:752kB pagetables:12772kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:72 all_unreclaimable? yes Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924512] lowmem_reserve[]: 0 0 0 0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924515] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924522] Node 0 DMA32: 824*4kB 222*8kB 39*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7840kB Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924528] 1416 total pagecache pages Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924530] 0 pages in swap cache Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924531] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924532] Free swap = 0kB Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924533] Total swap = 0kB Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929646] 985072 pages RAM Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929649] 24998 pages reserved Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929650] 3792 pages shared Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929651] 953258 pages non-shared Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929653] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929662] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929665] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929669] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929672] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929675] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929678] [ 664] 0 664 1815 123 0 0 0 dhclient3 Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929682] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929685] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929690] [ 790] 101 790 63477 623 1 0 0 rsyslogd Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929693] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929696] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929699] [ 975] 0 975 6276 79 1 0 0 master Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929702] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929706] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929709] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929712] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929715] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929718] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929721] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929725] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929728] [ 657] 1012 657 69242 6258 0 0 0 php Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929731] [ 658] 1012 658 134393 70798 1 0 0 php Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929734] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929737] [13468] 0 13468 89584 2503 0 0 0 php5-fpm Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929740] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929743] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929746] [ 2686] 1012 2686 68618 5646 1 0 0 php Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929750] [ 4505] 108 4505 6792 70 0 0 0 pickup Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929753] [ 4510] 108 4510 6792 70 1 0 0 scache Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929756] [ 5880] 1012 5880 886027 822972 0 0 0 php Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929759] [ 6615] 1012 6615 68580 5604 1 0 0 php Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929762] [ 6802] 1012 6802 1099 25 0 0 0 sh Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929765] [ 6803] 1012 6803 68913 5897 1 0 0 php Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929768] Out of memory: Kill process 5880 (php) score 858 or sacrifice child Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929780] Killed process 5880 (php) total-vm:3544108kB, anon-rss:3291888kB, file-rss:0kB Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412124] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412128] php cpuset=/ mems_allowed=0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412131] Pid: 4819, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412132] Call Trace: Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412139] [] dump_header+0x91/0xe0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412141] [] oom_kill_process+0x85/0xb0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412143] [] out_of_memory+0xfa/0x220 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412146] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412151] [] ? error_exit+0x2a/0x60 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412153] [] alloc_pages_current+0xb6/0x120 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412157] [] __page_cache_alloc+0xb7/0xd0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412159] [] filemap_fault+0x212/0x3c0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412162] [] __do_fault+0x72/0x550 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412164] [] ? error_exit+0x2a/0x60 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412167] [] ? retint_restore_args+0x5/0x6 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412169] [] handle_pte_fault+0xfa/0x200 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412172] [] ? xen_pmd_val+0xe/0x10 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412175] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412177] [] handle_mm_fault+0x269/0x370 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412180] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412182] [] ? check_events+0x12/0x20 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412185] [] do_page_fault+0x184/0x550 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412187] [] ? do_nanosleep+0xa4/0xd0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412191] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412194] [] ? update_rmtp+0x70/0x70 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412197] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412199] [] page_fault+0x25/0x30 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412201] Mem-Info: Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412202] Node 0 DMA per-cpu: Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412204] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412205] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412207] Node 0 DMA32 per-cpu: Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412208] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412209] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412212] active_anon:932627 inactive_anon:1393 isolated_anon:0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412213] active_file:14 inactive_file:14 isolated_file:0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412213] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412214] free:4048 slab_reclaimable:2821 slab_unreclaimable:2630 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412215] mapped:1368 shmem:1405 pagetables:3155 bounce:0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412216] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412223] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412226] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730508kB inactive_anon:5572kB active_file:56kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5472kB shmem:5620kB slab_reclaimable:11284kB slab_unreclaimable:10520kB kernel_stack:760kB pagetables:12620kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:15932 all_unreclaimable? yes Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412232] lowmem_reserve[]: 0 0 0 0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412234] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412241] Node 0 DMA32: 1452*4kB 6*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7936kB Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412248] 1457 total pagecache pages Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412249] 0 pages in swap cache Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412250] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412251] Free swap = 0kB Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412252] Total swap = 0kB Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417466] 985072 pages RAM Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417468] 24998 pages reserved Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417469] 3806 pages shared Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417470] 953307 pages non-shared Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417472] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417481] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417485] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417488] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417491] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417495] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417498] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417501] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417505] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417509] [ 790] 101 790 63477 626 1 0 0 rsyslogd Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417512] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417515] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417519] [ 975] 0 975 6276 80 1 0 0 master Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417522] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417525] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417528] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417531] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417534] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417537] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417540] [15657] 0 15657 4777 60 0 0 0 cron Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417544] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417547] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417550] [ 658] 1012 658 134393 70798 0 0 0 php Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417553] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417556] [13468] 0 13468 89584 2511 1 0 0 php5-fpm Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417559] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417562] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417565] [ 2686] 1012 2686 68618 5647 0 0 0 php Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417569] [ 2329] 108 2329 6818 73 1 0 0 cleanup Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417572] [ 2330] 108 2330 6795 71 0 0 0 trivial-rewrite Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417575] [ 2331] 108 2331 11107 159 0 0 0 smtp Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417578] [ 3728] 108 3728 6792 70 0 0 0 pickup Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417582] [ 3729] 108 3729 11107 141 1 0 0 smtp Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417585] [ 3856] 1012 3856 891340 828238 0 0 0 php Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417588] [ 4819] 1012 4819 68580 5604 1 0 0 php Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417591] Out of memory: Kill process 3856 (php) score 864 or sacrifice child Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417601] Killed process 3856 (php) total-vm:3565360kB, anon-rss:3312868kB, file-rss:84kB Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372283] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372287] php cpuset=/ mems_allowed=0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372290] Pid: 5244, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372292] Call Trace: Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372299] [] dump_header+0x91/0xe0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372301] [] oom_kill_process+0x85/0xb0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372303] [] out_of_memory+0xfa/0x220 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372306] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372309] [] alloc_pages_current+0xb6/0x120 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372313] [] __page_cache_alloc+0xb7/0xd0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372315] [] filemap_fault+0x212/0x3c0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372318] [] __do_fault+0x72/0x550 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372321] [] ? lru_cache_add_lru+0x2d/0x50 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372323] [] handle_pte_fault+0xfa/0x200 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372327] [] ? xen_pmd_val+0xe/0x10 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372329] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372331] [] handle_mm_fault+0x269/0x370 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372334] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372336] [] ? check_events+0x12/0x20 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372340] [] do_page_fault+0x184/0x550 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372342] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372344] [] ? xen_clts+0x8d/0x190 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372347] [] ? __schedule+0x3cc/0x6f0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372349] [] page_fault+0x25/0x30 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372351] Mem-Info: Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372352] Node 0 DMA per-cpu: Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372354] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372355] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372357] Node 0 DMA32 per-cpu: Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372358] CPU 0: hi: 186, btch: 31 usd: 30 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372359] CPU 1: hi: 186, btch: 31 usd: 27 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372362] active_anon:932189 inactive_anon:1393 isolated_anon:0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372363] active_file:0 inactive_file:0 isolated_file:0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372363] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372364] free:4038 slab_reclaimable:2955 slab_unreclaimable:2685 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372365] mapped:1365 shmem:1405 pagetables:3352 bounce:0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372366] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372373] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372376] Node 0 DMA32 free:7816kB min:7848kB low:9808kB high:11772kB active_anon:3728756kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5460kB shmem:5620kB slab_reclaimable:11820kB slab_unreclaimable:10740kB kernel_stack:848kB pagetables:13408kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:74 all_unreclaimable? yes Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372382] lowmem_reserve[]: 0 0 0 0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372385] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372391] Node 0 DMA32: 1442*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7816kB Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372398] 1423 total pagecache pages Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372399] 0 pages in swap cache Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372400] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372401] Free swap = 0kB Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372402] Total swap = 0kB Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377495] 985072 pages RAM Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377498] 24998 pages reserved Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377499] 3808 pages shared Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377500] 953267 pages non-shared Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377502] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377511] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377514] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377517] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377520] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377523] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377526] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377528] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377532] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377535] [ 790] 101 790 63477 626 0 0 0 rsyslogd Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377538] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377541] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377544] [ 975] 0 975 6276 80 1 0 0 master Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377547] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377550] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377553] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377556] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377559] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377562] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377566] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377569] [ 654] 1012 654 70351 5759 0 0 0 php Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377572] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377575] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377578] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377581] [13468] 0 13468 89584 2540 0 0 0 php5-fpm Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377584] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377587] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377590] [ 2686] 1012 2686 68618 5647 1 0 0 php Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377594] [ 3728] 108 3728 6792 76 0 0 0 pickup Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377597] [ 5244] 1012 5244 889997 826857 0 0 0 php Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377599] [ 8313] 0 8313 8460 79 1 0 0 cron Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377602] [ 8316] 0 8316 1099 26 1 0 0 sh Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377605] [ 8317] 0 8317 1074 26 0 0 0 run-parts Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377608] [ 8321] 0 8321 1099 35 0 0 0 apt Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377611] [ 8362] 0 8362 1469 24 0 0 0 sleep Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377615] [29567] 108 29567 6818 72 0 0 0 cleanup Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377618] [29568] 108 29568 6795 70 1 0 0 trivial-rewrite Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377621] [29569] 108 29569 11107 159 1 0 0 smtp Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377624] [29576] 108 29576 11076 150 1 0 0 smtp Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377627] [29591] 108 29591 11076 143 0 0 0 smtp Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377630] [29600] 108 29600 11107 158 0 0 0 smtp Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377633] [29605] 108 29605 11107 157 0 0 0 smtp Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377636] [29739] 108 29739 11108 160 0 0 0 smtp Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377639] [29740] 108 29740 6792 69 1 0 0 scache Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377642] [29780] 108 29780 6800 69 1 0 0 bounce Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377645] [29901] 1012 29901 68580 5604 0 0 0 php Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377648] Out of memory: Kill process 5244 (php) score 863 or sacrifice child Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377659] Killed process 5244 (php) total-vm:3559988kB, anon-rss:3307428kB, file-rss:0kB Jul 15 06:48:57 ip-10-61-174-46 kernel: [1397833.528163] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528167] php cpuset=/ mems_allowed=0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528170] Pid: 30258, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528172] Call Trace: Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528178] [] dump_header+0x91/0xe0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528181] [] oom_kill_process+0x85/0xb0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528183] [] out_of_memory+0xfa/0x220 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528186] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528189] [] alloc_pages_current+0xb6/0x120 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528192] [] __page_cache_alloc+0xb7/0xd0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528194] [] filemap_fault+0x212/0x3c0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528197] [] __do_fault+0x72/0x550 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528200] [] handle_pte_fault+0xfa/0x200 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528203] [] ? xen_pmd_val+0xe/0x10 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528205] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528208] [] handle_mm_fault+0x269/0x370 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528211] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528213] [] ? check_events+0x12/0x20 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528216] [] do_page_fault+0x184/0x550 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528219] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528220] [] ? xen_clts+0x8d/0x190 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528223] [] ? __schedule+0x3cc/0x6f0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528226] [] page_fault+0x25/0x30 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528228] Mem-Info: Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528229] Node 0 DMA per-cpu: Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528231] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528232] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528233] Node 0 DMA32 per-cpu: Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528235] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528236] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528239] active_anon:932703 inactive_anon:1393 isolated_anon:0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528239] active_file:0 inactive_file:0 isolated_file:0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528240] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528240] free:4048 slab_reclaimable:2554 slab_unreclaimable:2643 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528241] mapped:1374 shmem:1405 pagetables:3277 bounce:0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528243] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528249] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528252] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730812kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5496kB shmem:5620kB slab_reclaimable:10216kB slab_unreclaimable:10572kB kernel_stack:824kB pagetables:13108kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528259] lowmem_reserve[]: 0 0 0 0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528261] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528268] Node 0 DMA32: 883*4kB 102*8kB 24*16kB 32*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7932kB Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528275] 1439 total pagecache pages Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528276] 0 pages in swap cache Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528277] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528278] Free swap = 0kB Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528279] Total swap = 0kB Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533462] 985072 pages RAM Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533464] 24998 pages reserved Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533465] 3823 pages shared Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533466] 953285 pages non-shared Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533469] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533477] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533481] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533483] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533486] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533489] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533492] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533495] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533498] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533501] [ 790] 101 790 63477 626 0 0 0 rsyslogd Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533504] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533507] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533511] [ 975] 0 975 6276 79 1 0 0 master Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533514] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533517] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533520] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533523] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533526] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533529] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533533] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533536] [ 654] 1012 654 70351 5760 0 0 0 php Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533539] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533541] [ 658] 1012 658 134393 70798 1 0 0 php Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533545] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533548] [13468] 0 13468 89584 2525 0 0 0 php5-fpm Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533551] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533554] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533557] [ 2686] 1012 2686 68618 5646 1 0 0 php Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533561] [ 3728] 108 3728 6792 76 0 0 0 pickup Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533564] [ 8313] 0 8313 8460 79 1 0 0 cron Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533567] [ 8316] 0 8316 1099 26 1 0 0 sh Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533570] [ 8317] 0 8317 1074 26 0 0 0 run-parts Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533573] [ 8321] 0 8321 1099 35 0 0 0 apt Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533576] [ 8362] 0 8362 1469 24 0 0 0 sleep Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533579] [29569] 108 29569 11107 159 0 0 0 smtp Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533582] [29576] 108 29576 11076 150 0 0 0 smtp Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533585] [29600] 108 29600 11107 158 0 0 0 smtp Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533588] [29605] 108 29605 11107 157 0 0 0 smtp Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533591] [29739] 108 29739 11108 160 0 0 0 smtp Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533593] [29740] 108 29740 6792 69 0 0 0 scache Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533596] [29780] 108 29780 6800 69 1 0 0 bounce Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533599] [30258] 1012 30258 890764 827677 0 0 0 php Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533602] [31138] 1012 31138 68580 5605 0 0 0 php Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533605] Out of memory: Kill process 30258 (php) score 863 or sacrifice child Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533616] Killed process 30258 (php) total-vm:3563056kB, anon-rss:3310664kB, file-rss:44kB Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668144] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668149] php cpuset=/ mems_allowed=0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668153] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668154] Call Trace: Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668161] [] dump_header+0x91/0xe0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668164] [] oom_kill_process+0x85/0xb0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668166] [] out_of_memory+0xfa/0x220 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668169] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668172] [] alloc_pages_current+0xb6/0x120 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668175] [] __page_cache_alloc+0xb7/0xd0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668177] [] filemap_fault+0x212/0x3c0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668181] [] __do_fault+0x72/0x550 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668185] [] ? error_exit+0x2a/0x60 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668188] [] ? xen_hypervisor_callback+0x1b/0x20 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668191] [] handle_pte_fault+0xfa/0x200 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668194] [] ? xen_pmd_val+0xe/0x10 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668196] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668199] [] handle_mm_fault+0x269/0x370 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668201] [] ? load_TLS_descriptor+0x42/0xd0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668203] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668205] [] do_page_fault+0x184/0x550 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668209] [] ? __switch_to+0x138/0x360 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668212] [] ? finish_task_switch+0x53/0xf0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668215] [] ? __schedule+0x3cc/0x6f0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668217] [] page_fault+0x25/0x30 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668219] Mem-Info: Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668221] Node 0 DMA per-cpu: Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668222] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668224] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668225] Node 0 DMA32 per-cpu: Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668226] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668228] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668230] active_anon:932238 inactive_anon:1393 isolated_anon:0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668231] active_file:11 inactive_file:6 isolated_file:0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668232] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668232] free:4048 slab_reclaimable:2867 slab_unreclaimable:2692 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668233] mapped:1362 shmem:1405 pagetables:3380 bounce:0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668234] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668241] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668244] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3728952kB inactive_anon:5572kB active_file:44kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5620kB slab_reclaimable:11468kB slab_unreclaimable:10768kB kernel_stack:816kB pagetables:13520kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2646 all_unreclaimable? yes Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668250] lowmem_reserve[]: 0 0 0 0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668253] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668259] Node 0 DMA32: 1470*4kB 9*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8000kB Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668266] 1407 total pagecache pages Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668267] 0 pages in swap cache Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668268] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668269] Free swap = 0kB Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668270] Total swap = 0kB Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673569] 985072 pages RAM Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673572] 24998 pages reserved Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673573] 3775 pages shared Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673574] 953315 pages non-shared Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673576] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673585] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673588] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673591] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673594] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673598] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673601] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673604] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673608] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673612] [ 790] 101 790 63477 626 1 0 0 rsyslogd Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673615] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673618] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673621] [ 975] 0 975 6276 80 0 0 0 master Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673624] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673627] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673631] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673634] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673637] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673640] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673643] [15657] 0 15657 4777 60 0 0 0 cron Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673647] [ 654] 1012 654 70351 5759 0 0 0 php Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673650] [ 657] 1012 657 69242 6258 0 0 0 php Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673653] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673656] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673659] [13468] 0 13468 89584 2539 0 0 0 php5-fpm Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673662] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673665] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673668] [ 2686] 1012 2686 68618 5647 0 0 0 php Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673671] [17863] 108 17863 6792 69 1 0 0 pickup Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673675] [ 5425] 1012 5425 879249 816299 1 0 0 php Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673678] [14040] 108 14040 6818 74 0 0 0 cleanup Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673681] [14041] 108 14041 6795 74 0 0 0 trivial-rewrite Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673684] [14042] 108 14042 11107 158 1 0 0 smtp Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673687] [14046] 108 14046 11107 159 1 0 0 smtp Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673690] [14079] 1012 14079 68580 5603 0 0 0 php Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673693] [14233] 1012 14233 1099 24 0 0 0 sh Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673696] [14234] 1012 14234 68825 5867 1 0 0 php Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673700] [14241] 1012 14241 1099 25 0 0 0 sh Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673703] [14242] 1012 14242 4724 58 0 0 0 git Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673705] [14243] 1012 14243 68499 5552 0 0 0 php Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673709] [14245] 1012 14245 1099 24 1 0 0 sh Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673711] Out of memory: Kill process 5425 (php) score 852 or sacrifice child Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673722] Killed process 5425 (php) total-vm:3516996kB, anon-rss:3265196kB, file-rss:0kB Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116011] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116015] ntpd cpuset=/ mems_allowed=0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116018] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116020] Call Trace: Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116027] [] dump_header+0x91/0xe0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116029] [] oom_kill_process+0x85/0xb0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116032] [] out_of_memory+0xfa/0x220 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116034] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116038] [] alloc_pages_current+0xb6/0x120 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116041] [] __page_cache_alloc+0xb7/0xd0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116043] [] filemap_fault+0x212/0x3c0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116046] [] __do_fault+0x72/0x550 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116048] [] handle_pte_fault+0xfa/0x200 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116053] [] ? xen_pmd_val+0xe/0x10 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116070] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116074] [] handle_mm_fault+0x269/0x370 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116078] [] do_page_fault+0x184/0x550 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116082] [] ? pvclock_clocksource_read+0x55/0xf0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116086] [] page_fault+0x25/0x30 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116088] Mem-Info: Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116090] Node 0 DMA per-cpu: Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116092] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116094] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116096] Node 0 DMA32 per-cpu: Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116098] CPU 0: hi: 186, btch: 31 usd: 68 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116100] CPU 1: hi: 186, btch: 31 usd: 4 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116104] active_anon:932722 inactive_anon:1393 isolated_anon:0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116105] active_file:0 inactive_file:4 isolated_file:12 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116106] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116106] free:4047 slab_reclaimable:2602 slab_unreclaimable:2663 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116107] mapped:1357 shmem:1405 pagetables:3160 bounce:0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116109] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116118] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116122] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3730888kB inactive_anon:5572kB active_file:0kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):48kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:10408kB slab_unreclaimable:10652kB kernel_stack:768kB pagetables:12640kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:216 all_unreclaimable? yes Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116130] lowmem_reserve[]: 0 0 0 0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116134] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116144] Node 0 DMA32: 939*4kB 132*8kB 54*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7852kB Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116154] 1446 total pagecache pages Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116156] 0 pages in swap cache Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116157] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116159] Free swap = 0kB Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116160] Total swap = 0kB Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121438] 985072 pages RAM Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121440] 24998 pages reserved Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121442] 3783 pages shared Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121443] 953278 pages non-shared Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121444] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121454] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121457] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121459] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121462] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121465] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121468] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121471] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121474] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121477] [ 790] 101 790 63477 626 0 0 0 rsyslogd Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121480] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121482] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121484] [ 975] 0 975 6276 79 0 0 0 master Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121486] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121488] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121490] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121493] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121495] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121497] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121499] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121502] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121504] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121506] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121508] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121511] [13468] 0 13468 89584 2499 0 0 0 php5-fpm Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121513] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121515] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121517] [ 2686] 1012 2686 68618 5646 0 0 0 php Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121520] [17863] 108 17863 6792 75 0 0 0 pickup Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121522] [14040] 108 14040 6818 82 0 0 0 cleanup Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121524] [14041] 108 14041 6795 74 0 0 0 trivial-rewrite Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121526] [14042] 108 14042 11107 157 1 0 0 smtp Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121528] [14046] 108 14046 11107 158 1 0 0 smtp Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121531] [14388] 1012 14388 891510 828312 1 0 0 php Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121533] [14917] 1012 14917 68580 5604 1 0 0 php Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121535] Out of memory: Kill process 14388 (php) score 864 or sacrifice child Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121544] Killed process 14388 (php) total-vm:3566040kB, anon-rss:3313248kB, file-rss:0kB Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900393] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900397] php cpuset=/ mems_allowed=0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900400] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900402] Call Trace: Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900409] [] dump_header+0x91/0xe0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900411] [] oom_kill_process+0x85/0xb0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900413] [] out_of_memory+0xfa/0x220 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900416] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900419] [] alloc_pages_current+0xb6/0x120 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900423] [] __page_cache_alloc+0xb7/0xd0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900425] [] filemap_fault+0x212/0x3c0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900428] [] __do_fault+0x72/0x550 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900431] [] ? load_TLS_descriptor+0x42/0xd0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900434] [] handle_pte_fault+0xfa/0x200 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900436] [] ? xen_pmd_val+0xe/0x10 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900438] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900441] [] handle_mm_fault+0x269/0x370 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900445] [] ? lock_hrtimer_base.isra.25+0x30/0x60 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900448] [] do_page_fault+0x184/0x550 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900452] [] ? do_nanosleep+0xa4/0xd0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900455] [] ? __switch_to+0x288/0x360 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900458] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900460] [] ? update_rmtp+0x70/0x70 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900463] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900466] [] page_fault+0x25/0x30 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900467] Mem-Info: Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900468] Node 0 DMA per-cpu: Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900470] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900472] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900473] Node 0 DMA32 per-cpu: Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900474] CPU 0: hi: 186, btch: 31 usd: 34 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900476] CPU 1: hi: 186, btch: 31 usd: 62 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900478] active_anon:932856 inactive_anon:1393 isolated_anon:0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900479] active_file:0 inactive_file:10 isolated_file:0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900480] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900480] free:4034 slab_reclaimable:2547 slab_unreclaimable:2617 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900481] mapped:1381 shmem:1405 pagetables:3156 bounce:0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900483] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900489] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900492] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3731424kB inactive_anon:5572kB active_file:0kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5524kB shmem:5620kB slab_reclaimable:10188kB slab_unreclaimable:10468kB kernel_stack:760kB pagetables:12624kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:304 all_unreclaimable? yes Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900499] lowmem_reserve[]: 0 0 0 0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900501] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900508] Node 0 DMA32: 886*4kB 261*8kB 8*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7808kB Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900515] 1428 total pagecache pages Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900516] 0 pages in swap cache Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900517] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900518] Free swap = 0kB Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900519] Total swap = 0kB Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905706] 985072 pages RAM Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905708] 24998 pages reserved Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905710] 3787 pages shared Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905711] 953262 pages non-shared Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905713] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905721] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905725] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905728] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905731] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905734] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905738] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905741] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905745] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905749] [ 790] 101 790 63477 626 0 0 0 rsyslogd Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905752] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905755] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905758] [ 975] 0 975 6276 80 0 0 0 master Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905761] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905764] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905767] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905770] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905773] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905777] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905780] [15657] 0 15657 4777 59 1 0 0 cron Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905783] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905786] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905789] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905792] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905796] [13468] 0 13468 89584 2528 0 0 0 php5-fpm Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905799] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905802] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905805] [ 2686] 1012 2686 68618 5656 1 0 0 php Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905808] [17863] 108 17863 6792 75 1 0 0 pickup Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905812] [15141] 1012 15141 891631 828443 0 0 0 php Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905815] [23486] 108 23486 6818 74 0 0 0 cleanup Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905819] [23487] 108 23487 6795 71 0 0 0 trivial-rewrite Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905822] [23488] 108 23488 11107 158 0 0 0 smtp Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905825] [23501] 108 23501 11107 157 1 0 0 smtp Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905828] [24632] 1012 24632 68580 5598 1 0 0 php Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905831] Out of memory: Kill process 15141 (php) score 864 or sacrifice child Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905841] Killed process 15141 (php) total-vm:3566524kB, anon-rss:3313772kB, file-rss:0kB Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766336] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766340] php cpuset=/ mems_allowed=0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766343] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766344] Call Trace: Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766351] [] dump_header+0x91/0xe0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766354] [] oom_kill_process+0x85/0xb0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766356] [] out_of_memory+0xfa/0x220 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766359] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766364] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766367] [] alloc_pages_current+0xb6/0x120 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766370] [] __page_cache_alloc+0xb7/0xd0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766372] [] filemap_fault+0x212/0x3c0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766375] [] __do_fault+0x72/0x550 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766378] [] handle_pte_fault+0xfa/0x200 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766381] [] ? xen_pmd_val+0xe/0x10 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766383] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766385] [] handle_mm_fault+0x269/0x370 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766388] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766390] [] ? check_events+0x12/0x20 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766394] [] do_page_fault+0x184/0x550 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766397] [] ? do_nanosleep+0xa4/0xd0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766400] [] ? __switch_to+0x288/0x360 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766404] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766407] [] ? update_rmtp+0x70/0x70 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766410] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766413] [] page_fault+0x25/0x30 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766414] Mem-Info: Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766415] Node 0 DMA per-cpu: Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766417] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766419] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766420] Node 0 DMA32 per-cpu: Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766421] CPU 0: hi: 186, btch: 31 usd: 30 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766423] CPU 1: hi: 186, btch: 31 usd: 31 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766425] active_anon:932864 inactive_anon:1393 isolated_anon:0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766426] active_file:11 inactive_file:10 isolated_file:0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766427] unevictable:0 dirty:17 writeback:0 unstable:0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766427] free:4018 slab_reclaimable:2604 slab_unreclaimable:2629 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766428] mapped:1353 shmem:1405 pagetables:3172 bounce:0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766430] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766436] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766439] Node 0 DMA32 free:7736kB min:7848kB low:9808kB high:11772kB active_anon:3731456kB inactive_anon:5572kB active_file:44kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:68kB writeback:0kB mapped:5412kB shmem:5620kB slab_reclaimable:10416kB slab_unreclaimable:10516kB kernel_stack:768kB pagetables:12688kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:175 all_unreclaimable? yes Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766446] lowmem_reserve[]: 0 0 0 0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766448] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766455] Node 0 DMA32: 980*4kB 167*8kB 27*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7736kB Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766462] 1417 total pagecache pages Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766463] 0 pages in swap cache Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766464] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766465] Free swap = 0kB Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766466] Total swap = 0kB Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771650] 985072 pages RAM Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771652] 24998 pages reserved Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771653] 3784 pages shared Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771655] 953317 pages non-shared Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771657] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771665] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771669] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771672] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771675] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771678] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771682] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771685] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771688] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771692] [ 790] 101 790 63477 626 0 0 0 rsyslogd Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771695] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771698] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771702] [ 975] 0 975 6276 79 0 0 0 master Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771705] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771708] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771711] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771714] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771717] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771720] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771723] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771727] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771729] [ 657] 1012 657 69242 6257 0 0 0 php Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771732] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771735] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771738] [13468] 0 13468 89584 2513 1 0 0 php5-fpm Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771741] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771744] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771748] [ 2686] 1012 2686 68618 5656 1 0 0 php Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771751] [17863] 108 17863 6792 75 0 0 0 pickup Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771755] [23486] 108 23486 6818 82 0 0 0 cleanup Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771758] [23487] 108 23487 6795 75 1 0 0 trivial-rewrite Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771761] [23488] 108 23488 11107 158 1 0 0 smtp Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771764] [23501] 108 23501 11107 156 1 0 0 smtp Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771767] [24632] 1012 24632 886918 823825 0 0 0 php Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771770] [24675] 108 24675 11107 159 1 0 0 smtp Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771774] [24676] 1012 24676 73006 10075 1 0 0 php Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771776] Out of memory: Kill process 24632 (php) score 859 or sacrifice child Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771788] Killed process 24632 (php) total-vm:3547672kB, anon-rss:3295300kB, file-rss:0kB Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936144] php invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0, oom_score_adj=0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936148] php cpuset=/ mems_allowed=0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936150] Pid: 13526, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936152] Call Trace: Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936160] [] dump_header+0x91/0xe0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936162] [] oom_kill_process+0x85/0xb0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936164] [] out_of_memory+0xfa/0x220 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936167] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936170] [] ? unlock_page+0x2a/0x40 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936174] [] dup_task_struct+0x71/0x1d0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936176] [] copy_process.part.18+0x6e/0xea0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936180] [] ? do_page_fault+0x288/0x550 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936182] [] copy_process+0x77/0x80 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936184] [] do_fork+0xfa/0x340 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936187] [] ? _raw_spin_lock+0xe/0x20 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936190] [] ? fd_install+0x61/0x80 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936193] [] ? do_pipe_flags+0xc3/0x120 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936196] [] sys_clone+0x28/0x30 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936199] [] stub_clone+0x13/0x20 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936202] [] ? system_call_fastpath+0x16/0x1b Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936203] Mem-Info: Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936205] Node 0 DMA per-cpu: Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936207] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936208] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936209] Node 0 DMA32 per-cpu: Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936211] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936212] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936215] active_anon:932997 inactive_anon:1393 isolated_anon:0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936215] active_file:28 inactive_file:1 isolated_file:0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936216] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936217] free:4199 slab_reclaimable:2311 slab_unreclaimable:2645 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936217] mapped:1349 shmem:1405 pagetables:3145 bounce:0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936219] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936226] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936228] Node 0 DMA32 free:8460kB min:7848kB low:9808kB high:11772kB active_anon:3731988kB inactive_anon:5572kB active_file:112kB inactive_file:4kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5396kB shmem:5620kB slab_reclaimable:9244kB slab_unreclaimable:10580kB kernel_stack:760kB pagetables:12580kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1636 all_unreclaimable? yes Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936235] lowmem_reserve[]: 0 0 0 0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936237] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936244] Node 0 DMA32: 1479*4kB 66*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8492kB Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936251] 1439 total pagecache pages Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936252] 0 pages in swap cache Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936253] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936254] Free swap = 0kB Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936255] Total swap = 0kB Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941531] 985072 pages RAM Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941533] 24998 pages reserved Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941534] 3774 pages shared Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941535] 953200 pages non-shared Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941537] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941546] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941549] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941553] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941556] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941559] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941562] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941565] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941569] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941573] [ 790] 101 790 63477 629 0 0 0 rsyslogd Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941576] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941579] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941583] [ 975] 0 975 6276 80 0 0 0 master Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941586] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941589] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941593] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941596] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941600] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941603] [ 1242] 107 1242 9445 163 0 0 0 ntpd Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941606] [15657] 0 15657 4777 60 1 0 0 cron Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941609] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941612] [ 657] 1012 657 69242 6258 0 0 0 php Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941616] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941618] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941622] [13468] 0 13468 89584 2533 1 0 0 php5-fpm Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941625] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941627] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941631] [ 2686] 1012 2686 68618 5661 1 0 0 php Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941634] [17863] 108 17863 6792 75 1 0 0 pickup Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941638] [ 1278] 1012 1278 891690 828463 1 0 0 php Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941641] [13216] 108 13216 6818 74 0 0 0 cleanup Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941644] [13217] 108 13217 6795 69 0 0 0 trivial-rewrite Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941647] [13218] 108 13218 11107 159 0 0 0 smtp Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941650] [13525] 1012 13525 1099 24 0 0 0 sh Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941653] [13526] 1012 13526 68825 5849 0 0 0 php Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941656] Out of memory: Kill process 1278 (php) score 864 or sacrifice child Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941667] Killed process 1278 (php) total-vm:3566760kB, anon-rss:3313852kB, file-rss:0kB Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904152] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904156] php cpuset=/ mems_allowed=0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904159] Pid: 15682, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904161] Call Trace: Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904169] [] dump_header+0x91/0xe0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904172] [] oom_kill_process+0x85/0xb0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904175] [] out_of_memory+0xfa/0x220 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904180] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904186] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904189] [] alloc_pages_current+0xb6/0x120 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904193] [] __page_cache_alloc+0xb7/0xd0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904195] [] filemap_fault+0x212/0x3c0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904198] [] __do_fault+0x72/0x550 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904201] [] ? load_TLS_descriptor+0x42/0xd0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904204] [] handle_pte_fault+0xfa/0x200 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904206] [] ? xen_pmd_val+0xe/0x10 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904208] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904211] [] handle_mm_fault+0x269/0x370 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904213] [] ? find_vma+0x1/0x70 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904216] [] do_page_fault+0x184/0x550 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904219] [] ? do_nanosleep+0xa4/0xd0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904224] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904226] [] ? update_rmtp+0x70/0x70 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904229] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904232] [] page_fault+0x25/0x30 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904233] Mem-Info: Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904235] Node 0 DMA per-cpu: Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904237] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904238] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904239] Node 0 DMA32 per-cpu: Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904241] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904242] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904245] active_anon:932944 inactive_anon:1393 isolated_anon:0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904245] active_file:25 inactive_file:32 isolated_file:0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904246] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904247] free:4047 slab_reclaimable:2571 slab_unreclaimable:2602 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904247] mapped:1357 shmem:1405 pagetables:3121 bounce:0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904249] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904255] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904258] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3731776kB inactive_anon:5572kB active_file:100kB inactive_file:128kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:10284kB slab_unreclaimable:10408kB kernel_stack:744kB pagetables:12484kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:22664 all_unreclaimable? yes Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904265] lowmem_reserve[]: 0 0 0 0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904267] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904274] Node 0 DMA32: 936*4kB 166*8kB 42*16kB 4*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7984kB Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904281] 1443 total pagecache pages Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904282] 0 pages in swap cache Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904283] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904284] Free swap = 0kB Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904285] Total swap = 0kB Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909558] 985072 pages RAM Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909561] 24998 pages reserved Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909562] 3774 pages shared Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909563] 953329 pages non-shared Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909565] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909573] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909577] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909580] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909583] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909586] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909589] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909593] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909596] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909600] [ 790] 101 790 63477 629 0 0 0 rsyslogd Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909603] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909606] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909609] [ 975] 0 975 6276 80 0 0 0 master Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909612] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909616] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909619] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909622] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909625] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909628] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909631] [15657] 0 15657 4777 59 1 0 0 cron Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909634] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909638] [ 657] 1012 657 69242 6257 0 0 0 php Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909641] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909644] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909647] [13468] 0 13468 89584 2507 0 0 0 php5-fpm Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909650] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909653] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909656] [ 2686] 1012 2686 68618 5660 0 0 0 php Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909660] [13618] 1012 13618 891847 828753 1 0 0 php Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909663] [15399] 108 15399 6818 73 0 0 0 cleanup Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909666] [15400] 108 15400 6795 71 0 0 0 trivial-rewrite Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909669] [15401] 108 15401 11107 158 1 0 0 smtp Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909672] [15682] 1012 15682 68580 5604 0 0 0 php Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909675] Out of memory: Kill process 13618 (php) score 865 or sacrifice child Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909686] Killed process 13618 (php) total-vm:3567388kB, anon-rss:3315012kB, file-rss:0kB Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756114] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756118] php cpuset=/ mems_allowed=0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756121] Pid: 15940, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756123] Call Trace: Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756130] [] dump_header+0x91/0xe0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756132] [] oom_kill_process+0x85/0xb0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756135] [] out_of_memory+0xfa/0x220 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756138] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756141] [] alloc_pages_current+0xb6/0x120 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756144] [] __page_cache_alloc+0xb7/0xd0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756147] [] filemap_fault+0x212/0x3c0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756150] [] ? pte_mfn_to_pfn+0x8d/0x110 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756153] [] __do_fault+0x72/0x550 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756156] [] ? lru_cache_add_lru+0x2d/0x50 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756159] [] handle_pte_fault+0xfa/0x200 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756161] [] ? xen_pmd_val+0xe/0x10 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756163] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756166] [] handle_mm_fault+0x269/0x370 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756169] [] do_page_fault+0x184/0x550 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756172] [] ? do_brk+0x21f/0x320 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756174] [] ? xen_clts+0x8d/0x190 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756177] [] ? __schedule+0x3cc/0x6f0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756180] [] page_fault+0x25/0x30 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756181] Mem-Info: Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756183] Node 0 DMA per-cpu: Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756185] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756186] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756187] Node 0 DMA32 per-cpu: Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756189] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756190] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756193] active_anon:933016 inactive_anon:1393 isolated_anon:0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756193] active_file:8 inactive_file:16 isolated_file:0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756194] unevictable:0 dirty:9 writeback:0 unstable:0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756195] free:4048 slab_reclaimable:2480 slab_unreclaimable:2627 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756195] mapped:1369 shmem:1405 pagetables:3154 bounce:0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756197] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756204] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756207] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732064kB inactive_anon:5572kB active_file:32kB inactive_file:64kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:36kB writeback:0kB mapped:5476kB shmem:5620kB slab_reclaimable:9920kB slab_unreclaimable:10508kB kernel_stack:760kB pagetables:12616kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:8199 all_unreclaimable? yes Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756213] lowmem_reserve[]: 0 0 0 0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756216] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756222] Node 0 DMA32: 901*4kB 207*8kB 30*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7884kB Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756229] 1463 total pagecache pages Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756231] 0 pages in swap cache Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756232] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756233] Free swap = 0kB Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756234] Total swap = 0kB Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761587] 985072 pages RAM Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761590] 24998 pages reserved Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761591] 3801 pages shared Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761592] 953325 pages non-shared Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761594] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761602] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761606] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761609] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761612] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761615] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761618] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761622] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761625] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761629] [ 790] 101 790 63477 629 0 0 0 rsyslogd Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761632] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761635] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761638] [ 975] 0 975 6276 79 0 0 0 master Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761641] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761645] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761648] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761651] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761654] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761657] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761660] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761664] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761667] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761670] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761673] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761676] [13468] 0 13468 89584 2492 1 0 0 php5-fpm Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761679] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761682] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761685] [ 2686] 1012 2686 68618 5660 1 0 0 php Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761689] [15399] 108 15399 6818 81 0 0 0 cleanup Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761692] [15400] 108 15400 6795 75 0 0 0 trivial-rewrite Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761695] [15401] 108 15401 11107 158 0 0 0 smtp Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761698] [15940] 1012 15940 889708 826718 0 0 0 php Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761701] [16174] 108 16174 6792 70 1 0 0 pickup Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761705] [16185] 108 16185 11107 158 0 0 0 smtp Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761708] [17499] 1012 17499 70448 7489 1 0 0 php Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761711] Out of memory: Kill process 15940 (php) score 862 or sacrifice child Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761722] Killed process 15940 (php) total-vm:3558832kB, anon-rss:3306768kB, file-rss:104kB Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116013] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116017] ntpd cpuset=/ mems_allowed=0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116020] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116022] Call Trace: Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116029] [] dump_header+0x91/0xe0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116031] [] oom_kill_process+0x85/0xb0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116033] [] out_of_memory+0xfa/0x220 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116036] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116039] [] alloc_pages_current+0xb6/0x120 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116043] [] __page_cache_alloc+0xb7/0xd0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116045] [] filemap_fault+0x212/0x3c0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116048] [] __do_fault+0x72/0x550 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116052] [] handle_pte_fault+0xfa/0x200 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116067] [] ? xen_pmd_val+0xe/0x10 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116070] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116074] [] handle_mm_fault+0x269/0x370 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116078] [] do_page_fault+0x184/0x550 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116082] [] ? pvclock_clocksource_read+0x55/0xf0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116086] [] page_fault+0x25/0x30 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116088] Mem-Info: Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116090] Node 0 DMA per-cpu: Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116092] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116094] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116096] Node 0 DMA32 per-cpu: Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116098] CPU 0: hi: 186, btch: 31 usd: 106 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116100] CPU 1: hi: 186, btch: 31 usd: 43 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116104] active_anon:932770 inactive_anon:1393 isolated_anon:0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116105] active_file:9 inactive_file:18 isolated_file:20 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116106] unevictable:0 dirty:2 writeback:0 unstable:0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116107] free:4044 slab_reclaimable:2440 slab_unreclaimable:2585 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116108] mapped:1358 shmem:1405 pagetables:3253 bounce:0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116110] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116118] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116122] Node 0 DMA32 free:7840kB min:7848kB low:9808kB high:11772kB active_anon:3731080kB inactive_anon:5572kB active_file:36kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):80kB present:3862656kB mlocked:0kB dirty:8kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:9760kB slab_unreclaimable:10340kB kernel_stack:792kB pagetables:13012kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:209 all_unreclaimable? yes Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116131] lowmem_reserve[]: 0 0 0 0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116135] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116145] Node 0 DMA32: 687*4kB 212*8kB 66*16kB 8*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7868kB Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116155] 1478 total pagecache pages Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116156] 0 pages in swap cache Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116158] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116160] Free swap = 0kB Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116161] Total swap = 0kB Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121457] 985072 pages RAM Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121459] 24998 pages reserved Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121460] 3788 pages shared Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121461] 953189 pages non-shared Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121462] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121470] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121472] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121475] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121477] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121479] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121481] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121484] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121487] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121490] [ 790] 101 790 63477 629 0 0 0 rsyslogd Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121492] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121494] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121497] [ 975] 0 975 6276 80 1 0 0 master Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121499] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121501] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121503] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121505] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121508] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121510] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121512] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121514] [ 654] 1012 654 70351 5759 0 0 0 php Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121516] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121519] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121521] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121523] [13468] 0 13468 89584 2526 1 0 0 php5-fpm Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121525] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121527] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121530] [ 2686] 1012 2686 68618 5660 1 0 0 php Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121533] [16174] 108 16174 6792 76 1 0 0 pickup Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121535] [17806] 1012 17806 888533 825457 0 0 0 php Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121537] [20624] 108 20624 6818 74 0 0 0 cleanup Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121539] [20625] 108 20625 6795 70 1 0 0 trivial-rewrite Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121542] [20626] 108 20626 11107 158 0 0 0 smtp Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121544] [20872] 1012 20872 1099 25 1 0 0 sh Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121546] [20873] 1012 20873 68825 5870 1 0 0 php Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121548] [20879] 1012 20879 1099 26 1 0 0 sh Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121550] [20880] 1012 20880 4676 57 1 0 0 git Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121552] [20881] 1012 20881 65117 2699 0 0 0 php Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121554] Out of memory: Kill process 17806 (php) score 861 or sacrifice child Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121564] Killed process 17806 (php) total-vm:3554132kB, anon-rss:3301828kB, file-rss:0kB Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592343] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592347] php cpuset=/ mems_allowed=0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592350] Pid: 32189, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592352] Call Trace: Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592359] [] dump_header+0x91/0xe0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592361] [] oom_kill_process+0x85/0xb0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592363] [] out_of_memory+0xfa/0x220 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592366] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592370] [] alloc_pages_current+0xb6/0x120 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592373] [] __page_cache_alloc+0xb7/0xd0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592375] [] filemap_fault+0x212/0x3c0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592378] [] __do_fault+0x72/0x550 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592381] [] handle_pte_fault+0xfa/0x200 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592384] [] ? xen_pmd_val+0xe/0x10 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592386] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592389] [] handle_mm_fault+0x269/0x370 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592393] [] do_page_fault+0x184/0x550 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592395] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592397] [] ? xen_clts+0x8d/0x190 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592400] [] page_fault+0x25/0x30 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592401] Mem-Info: Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592403] Node 0 DMA per-cpu: Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592404] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592406] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592407] Node 0 DMA32 per-cpu: Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592409] CPU 0: hi: 186, btch: 31 usd: 30 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592410] CPU 1: hi: 186, btch: 31 usd: 50 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592413] active_anon:932885 inactive_anon:1393 isolated_anon:0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592413] active_file:28 inactive_file:6 isolated_file:0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592414] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592415] free:4034 slab_reclaimable:2642 slab_unreclaimable:2587 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592415] mapped:1359 shmem:1405 pagetables:3113 bounce:0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592417] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592423] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592426] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3731540kB inactive_anon:5572kB active_file:112kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5436kB shmem:5620kB slab_reclaimable:10568kB slab_unreclaimable:10348kB kernel_stack:728kB pagetables:12452kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:280 all_unreclaimable? yes Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592433] lowmem_reserve[]: 0 0 0 0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592435] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592442] Node 0 DMA32: 803*4kB 278*8kB 17*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7756kB Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592449] 1426 total pagecache pages Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592450] 0 pages in swap cache Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592451] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592452] Free swap = 0kB Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592453] Total swap = 0kB Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597797] 985072 pages RAM Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597799] 24998 pages reserved Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597800] 3785 pages shared Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597801] 953295 pages non-shared Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597803] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597812] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597816] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597819] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597822] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597825] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597828] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597832] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597835] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597839] [ 790] 101 790 63477 629 0 0 0 rsyslogd Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597842] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597845] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597849] [ 975] 0 975 6276 79 1 0 0 master Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597852] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597855] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597858] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597861] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597864] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597867] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597871] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597874] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597877] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597880] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597883] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597886] [13468] 0 13468 89584 2527 1 0 0 php5-fpm Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597889] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597892] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597895] [ 2686] 1012 2686 68618 5661 1 0 0 php Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597899] [16174] 108 16174 6792 76 1 0 0 pickup Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597902] [20999] 1012 20999 891667 828552 0 0 0 php Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597905] [32036] 1012 32036 68580 5604 1 0 0 php Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597908] [32188] 1012 32188 1099 25 1 0 0 sh Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597911] [32189] 1012 32189 31456 379 0 0 0 php Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597914] Out of memory: Kill process 20999 (php) score 864 or sacrifice child Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597925] Killed process 20999 (php) total-vm:3566668kB, anon-rss:3314208kB, file-rss:0kB Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080111] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080115] php cpuset=/ mems_allowed=0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080117] Pid: 32264, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080119] Call Trace: Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080126] [] dump_header+0x91/0xe0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080128] [] oom_kill_process+0x85/0xb0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080130] [] out_of_memory+0xfa/0x220 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080133] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080137] [] alloc_pages_vma+0xb3/0x190 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080140] [] do_anonymous_page.isra.38+0x7c/0x2f0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080142] [] handle_pte_fault+0x1e1/0x200 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080146] [] ? xen_pmd_val+0xe/0x10 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080148] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080150] [] handle_mm_fault+0x269/0x370 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080154] [] do_page_fault+0x184/0x550 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080156] [] ? do_brk+0x21f/0x320 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080158] [] ? xen_clts+0x8d/0x190 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080161] [] ? __schedule+0x3cc/0x6f0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080164] [] page_fault+0x25/0x30 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080166] Mem-Info: Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080167] Node 0 DMA per-cpu: Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080169] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080170] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080171] Node 0 DMA32 per-cpu: Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080173] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080174] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080176] active_anon:933123 inactive_anon:1393 isolated_anon:0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080177] active_file:0 inactive_file:0 isolated_file:0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080178] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080178] free:4048 slab_reclaimable:2378 slab_unreclaimable:2583 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080179] mapped:1364 shmem:1405 pagetables:3171 bounce:0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080181] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080187] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080190] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732492kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5456kB shmem:5620kB slab_reclaimable:9512kB slab_unreclaimable:10332kB kernel_stack:760kB pagetables:12684kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080197] lowmem_reserve[]: 0 0 0 0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080199] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080206] Node 0 DMA32: 1189*4kB 132*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7924kB Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080212] 1458 total pagecache pages Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080214] 0 pages in swap cache Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080215] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080216] Free swap = 0kB Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080217] Total swap = 0kB Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085737] 985072 pages RAM Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085739] 24998 pages reserved Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085740] 3802 pages shared Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085741] 953317 pages non-shared Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085743] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085752] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085755] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085758] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085761] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085765] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085768] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085771] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085774] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085779] [ 790] 101 790 63477 629 1 0 0 rsyslogd Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085782] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085785] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085788] [ 975] 0 975 6276 80 0 0 0 master Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085791] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085794] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085797] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085800] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085804] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085807] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085810] [15657] 0 15657 4777 60 0 0 0 cron Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085813] [ 654] 1012 654 70351 5760 0 0 0 php Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085816] [ 657] 1012 657 69242 6258 0 0 0 php Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085819] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085822] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085825] [13468] 0 13468 89584 2512 1 0 0 php5-fpm Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085828] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085831] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085834] [ 2686] 1012 2686 68618 5661 0 0 0 php Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085837] [16174] 108 16174 6792 76 1 0 0 pickup Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085841] [32264] 1012 32264 886530 823287 0 0 0 php Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085844] [29752] 1012 29752 68580 5606 1 0 0 php Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085847] [30013] 1012 30013 1099 25 0 0 0 sh Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085850] [30014] 1012 30014 68825 5858 0 0 0 php Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085853] Out of memory: Kill process 32264 (php) score 859 or sacrifice child Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085864] Killed process 32264 (php) total-vm:3546120kB, anon-rss:3293136kB, file-rss:12kB Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704120] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704124] php cpuset=/ mems_allowed=0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704126] Pid: 31539, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704128] Call Trace: Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704135] [] dump_header+0x91/0xe0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704137] [] oom_kill_process+0x85/0xb0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704139] [] out_of_memory+0xfa/0x220 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704142] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704146] [] alloc_pages_vma+0xb3/0x190 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704149] [] do_anonymous_page.isra.38+0x7c/0x2f0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704151] [] handle_pte_fault+0x1e1/0x200 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704154] [] ? xen_pmd_val+0xe/0x10 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704157] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704159] [] handle_mm_fault+0x269/0x370 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704163] [] do_page_fault+0x184/0x550 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704165] [] ? do_brk+0x21f/0x320 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704167] [] ? xen_clts+0x8d/0x190 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704170] [] ? __schedule+0x3cc/0x6f0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704173] [] page_fault+0x25/0x30 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704174] Mem-Info: Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704176] Node 0 DMA per-cpu: Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704178] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704179] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704180] Node 0 DMA32 per-cpu: Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704182] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704183] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704185] active_anon:932991 inactive_anon:1393 isolated_anon:0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704186] active_file:28 inactive_file:30 isolated_file:0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704187] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704188] free:4048 slab_reclaimable:2594 slab_unreclaimable:2605 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704188] mapped:1386 shmem:1405 pagetables:3072 bounce:0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704190] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704196] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704199] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731964kB inactive_anon:5572kB active_file:112kB inactive_file:120kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5544kB shmem:5620kB slab_reclaimable:10376kB slab_unreclaimable:10420kB kernel_stack:744kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:6986 all_unreclaimable? yes Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704206] lowmem_reserve[]: 0 0 0 0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704208] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704215] Node 0 DMA32: 1058*4kB 182*8kB 5*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7912kB Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704222] 1479 total pagecache pages Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704223] 0 pages in swap cache Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704224] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704225] Free swap = 0kB Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704226] Total swap = 0kB Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709427] 985072 pages RAM Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709429] 24998 pages reserved Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709430] 3808 pages shared Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709431] 953315 pages non-shared Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709433] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709442] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709445] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709448] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709451] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709455] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709458] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709461] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709464] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709469] [ 790] 101 790 63477 629 1 0 0 rsyslogd Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709472] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709475] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709478] [ 975] 0 975 6276 79 0 0 0 master Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709481] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709484] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709487] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709490] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709493] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709497] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709500] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709503] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709506] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709509] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709512] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709516] [13468] 0 13468 89584 2497 1 0 0 php5-fpm Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709519] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709521] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709525] [ 2686] 1012 2686 68618 5660 1 0 0 php Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709528] [16174] 108 16174 6792 76 1 0 0 pickup Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709532] [31539] 1012 31539 892179 829048 0 0 0 php Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709535] [32253] 1012 32253 68580 5604 1 0 0 php Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709538] Out of memory: Kill process 31539 (php) score 865 or sacrifice child Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709549] Killed process 31539 (php) total-vm:3568716kB, anon-rss:3316068kB, file-rss:124kB Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348596] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348600] php cpuset=/ mems_allowed=0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348603] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348604] Call Trace: Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348612] [] dump_header+0x91/0xe0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348614] [] oom_kill_process+0x85/0xb0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348616] [] out_of_memory+0xfa/0x220 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348619] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348624] [] ? error_exit+0x2a/0x60 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348626] [] alloc_pages_current+0xb6/0x120 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348630] [] __page_cache_alloc+0xb7/0xd0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348633] [] filemap_fault+0x212/0x3c0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348636] [] __do_fault+0x72/0x550 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348639] [] handle_pte_fault+0xfa/0x200 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348643] [] ? xen_pmd_val+0xe/0x10 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348646] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348649] [] handle_mm_fault+0x269/0x370 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348651] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348654] [] ? check_events+0x12/0x20 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348656] [] do_page_fault+0x184/0x550 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348659] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348660] [] ? xen_clts+0x8d/0x190 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348663] [] page_fault+0x25/0x30 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348665] Mem-Info: Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348666] Node 0 DMA per-cpu: Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348668] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348669] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348671] Node 0 DMA32 per-cpu: Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348672] CPU 0: hi: 186, btch: 31 usd: 30 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348673] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348676] active_anon:932989 inactive_anon:1393 isolated_anon:0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348677] active_file:6 inactive_file:8 isolated_file:0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348677] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348678] free:4047 slab_reclaimable:2660 slab_unreclaimable:2639 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348679] mapped:1367 shmem:1405 pagetables:3055 bounce:0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348680] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348687] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348690] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3731956kB inactive_anon:5572kB active_file:24kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5468kB shmem:5620kB slab_reclaimable:10640kB slab_unreclaimable:10556kB kernel_stack:728kB pagetables:12220kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:90 all_unreclaimable? yes Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348696] lowmem_reserve[]: 0 0 0 0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348699] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348705] Node 0 DMA32: 867*4kB 252*8kB 18*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7852kB Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348712] 1444 total pagecache pages Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348713] 0 pages in swap cache Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348715] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348716] Free swap = 0kB Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348716] Total swap = 0kB Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353960] 985072 pages RAM Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353963] 24998 pages reserved Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353964] 3779 pages shared Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353966] 953330 pages non-shared Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353967] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353976] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353979] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353982] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353985] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353989] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353992] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353995] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353998] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354003] [ 790] 101 790 63477 629 0 0 0 rsyslogd Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354006] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354009] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354012] [ 975] 0 975 6276 79 0 0 0 master Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354015] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354018] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354021] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354024] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354027] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354031] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354034] [15657] 0 15657 4777 59 1 0 0 cron Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354037] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354040] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354043] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354046] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354049] [13468] 0 13468 89584 2512 0 0 0 php5-fpm Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354052] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354055] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354059] [ 2686] 1012 2686 68618 5661 0 0 0 php Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354062] [16174] 108 16174 6792 76 1 0 0 pickup Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354066] [32442] 1012 32442 892209 829031 0 0 0 php Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354069] [ 6550] 1012 6550 68580 5603 0 0 0 php Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354071] Out of memory: Kill process 32442 (php) score 865 or sacrifice child Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354082] Killed process 32442 (php) total-vm:3568836kB, anon-rss:3316124kB, file-rss:0kB Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204511] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204515] php cpuset=/ mems_allowed=0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204519] Pid: 7885, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204521] Call Trace: Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204529] [] dump_header+0x91/0xe0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204533] [] oom_kill_process+0x85/0xb0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204535] [] out_of_memory+0xfa/0x220 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204539] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204544] [] alloc_pages_current+0xb6/0x120 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204550] [] __page_cache_alloc+0xb7/0xd0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204553] [] filemap_fault+0x212/0x3c0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204557] [] __do_fault+0x72/0x550 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204562] [] ? error_exit+0x2a/0x60 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204565] [] ? retint_restore_args+0x5/0x6 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204569] [] ? pte_mfn_to_pfn+0x3a/0x110 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204572] [] handle_pte_fault+0xfa/0x200 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204575] [] ? xen_pmd_val+0xe/0x10 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204578] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204581] [] handle_mm_fault+0x269/0x370 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204590] [] do_page_fault+0x184/0x550 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204594] [] ? ktime_get_ts+0xad/0xe0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204599] [] ? poll_select_copy_remaining+0xed/0x140 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204603] [] ? sys_select+0xcd/0x100 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204606] [] page_fault+0x25/0x30 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204608] Mem-Info: Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204610] Node 0 DMA per-cpu: Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204612] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204614] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204616] Node 0 DMA32 per-cpu: Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204618] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204620] CPU 1: hi: 186, btch: 31 usd: 30 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204624] active_anon:932943 inactive_anon:1393 isolated_anon:0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204625] active_file:18 inactive_file:41 isolated_file:1 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204626] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204626] free:4033 slab_reclaimable:2308 slab_unreclaimable:2628 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204627] mapped:1328 shmem:1405 pagetables:3300 bounce:0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204630] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204638] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204642] Node 0 DMA32 free:7796kB min:7848kB low:9808kB high:11772kB active_anon:3731772kB inactive_anon:5572kB active_file:72kB inactive_file:164kB unevictable:0kB isolated(anon):0kB isolated(file):4kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5312kB shmem:5620kB slab_reclaimable:9232kB slab_unreclaimable:10512kB kernel_stack:776kB pagetables:13200kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204651] lowmem_reserve[]: 0 0 0 0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204654] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204664] Node 0 DMA32: 815*4kB 229*8kB 33*16kB 2*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7796kB Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204673] 1503 total pagecache pages Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204675] 0 pages in swap cache Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204676] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204678] Free swap = 0kB Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204679] Total swap = 0kB Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209953] 985072 pages RAM Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209955] 24998 pages reserved Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209957] 3795 pages shared Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209958] 953322 pages non-shared Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209960] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209969] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209972] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209975] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209978] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209982] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209985] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209988] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209992] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209996] [ 790] 101 790 63477 630 0 0 0 rsyslogd Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209999] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210002] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210005] [ 975] 0 975 6276 79 1 0 0 master Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210008] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210012] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210015] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210018] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210021] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210024] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210027] [15657] 0 15657 4777 59 1 0 0 cron Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210030] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210033] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210036] [ 658] 1012 658 134393 70798 1 0 0 php Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210039] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210042] [13468] 0 13468 89584 2533 0 0 0 php5-fpm Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210045] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210048] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210052] [ 2686] 1012 2686 68618 5660 0 0 0 php Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210056] [ 6708] 1012 6708 880466 817334 1 0 0 php Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210059] [ 7822] 1012 7822 68580 5604 0 0 0 php Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210062] [ 7884] 1012 7884 1099 24 0 0 0 sh Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210065] [ 7885] 1012 7885 68825 5867 0 0 0 php Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210068] [ 7891] 1012 7891 1099 27 1 0 0 sh Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210071] [ 7892] 1012 7892 4655 57 1 0 0 git Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210074] [ 7893] 1012 7893 68499 5552 1 0 0 php Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210077] [ 7895] 1012 7895 1099 25 1 0 0 sh Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210080] [ 7896] 1012 7896 10901 165 1 0 0 ssh Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210082] Out of memory: Kill process 6708 (php) score 853 or sacrifice child Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210093] Killed process 6708 (php) total-vm:3521864kB, anon-rss:3269336kB, file-rss:0kB Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271088] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271093] php cpuset=/ mems_allowed=0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271096] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271098] Call Trace: Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271105] [] dump_header+0x91/0xe0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271107] [] oom_kill_process+0x85/0xb0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271110] [] out_of_memory+0xfa/0x220 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271112] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271116] [] alloc_pages_current+0xb6/0x120 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271119] [] __page_cache_alloc+0xb7/0xd0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271121] [] filemap_fault+0x212/0x3c0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271125] [] ? __pollwait+0xf0/0xf0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271128] [] __do_fault+0x72/0x550 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271130] [] ? __pollwait+0xf0/0xf0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271133] [] handle_pte_fault+0xfa/0x200 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271136] [] ? xen_pmd_val+0xe/0x10 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271138] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271141] [] handle_mm_fault+0x269/0x370 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271144] [] do_page_fault+0x184/0x550 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271147] [] ? ktime_get_ts+0xad/0xe0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271150] [] ? poll_select_copy_remaining+0xed/0x140 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271152] [] ? sys_select+0xcd/0x100 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271155] [] page_fault+0x25/0x30 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271157] Mem-Info: Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271158] Node 0 DMA per-cpu: Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271160] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271162] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271163] Node 0 DMA32 per-cpu: Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271164] CPU 0: hi: 186, btch: 31 usd: 30 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271166] CPU 1: hi: 186, btch: 31 usd: 65 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271168] active_anon:932737 inactive_anon:1393 isolated_anon:0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271169] active_file:7 inactive_file:10 isolated_file:0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271170] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271171] free:4045 slab_reclaimable:2771 slab_unreclaimable:2607 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271171] mapped:1390 shmem:1405 pagetables:3081 bounce:0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271173] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271179] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271182] Node 0 DMA32 free:7844kB min:7848kB low:9808kB high:11772kB active_anon:3730948kB inactive_anon:5572kB active_file:28kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5560kB shmem:5620kB slab_reclaimable:11084kB slab_unreclaimable:10428kB kernel_stack:728kB pagetables:12324kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:115 all_unreclaimable? yes Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271189] lowmem_reserve[]: 0 0 0 0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271191] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271198] Node 0 DMA32: 737*4kB 170*8kB 39*16kB 17*32kB 5*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7844kB Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271205] 1442 total pagecache pages Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271206] 0 pages in swap cache Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271207] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271208] Free swap = 0kB Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271209] Total swap = 0kB Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276317] 985072 pages RAM Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276320] 24998 pages reserved Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276321] 3802 pages shared Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276322] 953243 pages non-shared Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276324] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276333] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276336] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276339] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276342] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276346] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276349] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276352] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276355] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276359] [ 790] 101 790 63477 630 0 0 0 rsyslogd Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276362] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276365] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276369] [ 975] 0 975 6276 79 0 0 0 master Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276372] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276375] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276378] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276381] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276384] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276387] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276391] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276394] [ 654] 1012 654 70351 5760 0 0 0 php Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276397] [ 657] 1012 657 69242 6258 0 0 0 php Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276400] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276403] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276406] [13468] 0 13468 89584 2518 0 0 0 php5-fpm Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276409] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276412] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276415] [ 2686] 1012 2686 68618 5660 0 0 0 php Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276419] [ 7987] 1012 7987 891955 828798 0 0 0 php Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276422] [ 8098] 108 8098 6792 70 0 0 0 pickup Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276425] [12061] 1012 12061 68580 5605 0 0 0 php Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276428] Out of memory: Kill process 7987 (php) score 865 or sacrifice child Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276439] Killed process 7987 (php) total-vm:3567820kB, anon-rss:3315192kB, file-rss:0kB Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784122] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784127] php cpuset=/ mems_allowed=0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784130] Pid: 13559, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784132] Call Trace: Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784140] [] dump_header+0x91/0xe0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784142] [] oom_kill_process+0x85/0xb0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784144] [] out_of_memory+0xfa/0x220 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784147] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784150] [] alloc_pages_current+0xb6/0x120 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784154] [] __page_cache_alloc+0xb7/0xd0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784156] [] filemap_fault+0x212/0x3c0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784159] [] __do_fault+0x72/0x550 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784161] [] handle_pte_fault+0xfa/0x200 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784165] [] ? xen_pmd_val+0xe/0x10 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784167] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784169] [] handle_mm_fault+0x269/0x370 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784172] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784175] [] ? check_events+0x12/0x20 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784178] [] do_page_fault+0x184/0x550 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784180] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784182] [] ? xen_clts+0x8d/0x190 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784185] [] page_fault+0x25/0x30 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784187] Mem-Info: Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784188] Node 0 DMA per-cpu: Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784190] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784191] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784192] Node 0 DMA32 per-cpu: Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784194] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784195] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784198] active_anon:932774 inactive_anon:1393 isolated_anon:0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784199] active_file:36 inactive_file:40 isolated_file:0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784199] unevictable:0 dirty:18 writeback:0 unstable:0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784200] free:4048 slab_reclaimable:2568 slab_unreclaimable:2619 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784201] mapped:1404 shmem:1405 pagetables:3242 bounce:0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784202] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784209] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784212] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731096kB inactive_anon:5572kB active_file:144kB inactive_file:160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:72kB writeback:0kB mapped:5616kB shmem:5620kB slab_reclaimable:10272kB slab_unreclaimable:10476kB kernel_stack:760kB pagetables:12968kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:20044 all_unreclaimable? yes Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784218] lowmem_reserve[]: 0 0 0 0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784221] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784227] Node 0 DMA32: 893*4kB 145*8kB 26*16kB 22*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8028kB Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784234] 1471 total pagecache pages Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784235] 0 pages in swap cache Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784237] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784238] Free swap = 0kB Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784239] Total swap = 0kB Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789415] 985072 pages RAM Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789417] 24998 pages reserved Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789418] 3822 pages shared Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789419] 953276 pages non-shared Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789421] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789430] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789434] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789437] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789440] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789443] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789446] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789449] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789453] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789457] [ 790] 101 790 63477 630 1 0 0 rsyslogd Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789460] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789463] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789466] [ 975] 0 975 6276 79 1 0 0 master Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789469] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789473] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789476] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789479] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789482] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789485] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789488] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789491] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789494] [ 657] 1012 657 69242 6258 0 0 0 php Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789497] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789500] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789504] [13468] 0 13468 89584 2523 0 0 0 php5-fpm Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789507] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789510] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789513] [ 2686] 1012 2686 68618 5660 1 0 0 php Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789517] [ 8098] 108 8098 6792 70 0 0 0 pickup Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789520] [12322] 1012 12322 883667 820550 1 0 0 php Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789523] [13463] 1012 13463 68580 5604 1 0 0 php Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789526] [13558] 1012 13558 1099 24 0 0 0 sh Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789529] [13559] 1012 13559 66679 4300 0 0 0 php Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789532] [13560] 1012 13560 1099 25 0 0 0 sh Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789535] [13561] 1012 13561 66419 3969 1 0 0 php Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789538] Out of memory: Kill process 12322 (php) score 856 or sacrifice child Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789549] Killed process 12322 (php) total-vm:3534668kB, anon-rss:3282200kB, file-rss:0kB Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026285] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026289] php cpuset=/ mems_allowed=0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026292] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026294] Call Trace: Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026301] [] dump_header+0x91/0xe0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026303] [] oom_kill_process+0x85/0xb0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026305] [] out_of_memory+0xfa/0x220 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026308] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026311] [] alloc_pages_current+0xb6/0x120 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026315] [] __page_cache_alloc+0xb7/0xd0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026317] [] filemap_fault+0x212/0x3c0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026320] [] __do_fault+0x72/0x550 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026323] [] ? __pollwait+0xf0/0xf0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026325] [] handle_pte_fault+0xfa/0x200 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026329] [] ? xen_pmd_val+0xe/0x10 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026331] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026334] [] handle_mm_fault+0x269/0x370 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026337] [] do_page_fault+0x184/0x550 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026340] [] ? ktime_get_ts+0xad/0xe0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026343] [] ? poll_select_copy_remaining+0xed/0x140 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026345] [] ? sys_select+0xcd/0x100 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026348] [] page_fault+0x25/0x30 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026350] Mem-Info: Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026351] Node 0 DMA per-cpu: Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026353] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026354] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026355] Node 0 DMA32 per-cpu: Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026357] CPU 0: hi: 186, btch: 31 usd: 4 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026358] CPU 1: hi: 186, btch: 31 usd: 61 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026361] active_anon:933352 inactive_anon:1393 isolated_anon:0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026362] active_file:4 inactive_file:0 isolated_file:0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026363] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026363] free:4047 slab_reclaimable:2328 slab_unreclaimable:2577 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026364] mapped:1373 shmem:1405 pagetables:3069 bounce:0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026366] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026372] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026375] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3733408kB inactive_anon:5572kB active_file:16kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5492kB shmem:5620kB slab_reclaimable:9312kB slab_unreclaimable:10308kB kernel_stack:728kB pagetables:12276kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:125 all_unreclaimable? yes Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026381] lowmem_reserve[]: 0 0 0 0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026384] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026391] Node 0 DMA32: 709*4kB 155*8kB 34*16kB 31*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7852kB Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026397] 1426 total pagecache pages Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026398] 0 pages in swap cache Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026400] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026401] Free swap = 0kB Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026402] Total swap = 0kB Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031646] 985072 pages RAM Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031648] 24998 pages reserved Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031649] 3772 pages shared Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031650] 953301 pages non-shared Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031652] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031661] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031665] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031668] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031671] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031674] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031677] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031681] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031684] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031688] [ 790] 101 790 63477 630 0 0 0 rsyslogd Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031691] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031695] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031698] [ 975] 0 975 6276 79 0 0 0 master Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031701] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031704] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031707] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031710] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031713] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031716] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031720] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031723] [ 654] 1012 654 70351 5759 0 0 0 php Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031726] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031729] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031732] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031735] [13468] 0 13468 89584 2508 0 0 0 php5-fpm Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031738] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031741] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031744] [ 2686] 1012 2686 68618 5660 0 0 0 php Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031748] [ 8098] 108 8098 6792 76 0 0 0 pickup Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031751] [13620] 1012 13620 892499 829403 0 0 0 php Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031754] [14209] 1012 14209 68580 5605 0 0 0 php Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031757] Out of memory: Kill process 13620 (php) score 865 or sacrifice child Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031768] Killed process 13620 (php) total-vm:3569996kB, anon-rss:3317612kB, file-rss:0kB Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562647] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562654] php cpuset=/ mems_allowed=0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562657] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562660] Call Trace: Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562668] [] dump_header+0x91/0xe0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562671] [] oom_kill_process+0x85/0xb0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562675] [] out_of_memory+0xfa/0x220 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562679] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562682] [] alloc_pages_current+0xb6/0x120 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562687] [] __page_cache_alloc+0xb7/0xd0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562690] [] filemap_fault+0x212/0x3c0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562694] [] ? __pollwait+0xf0/0xf0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562697] [] __do_fault+0x72/0x550 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562700] [] ? __pollwait+0xf0/0xf0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562704] [] handle_pte_fault+0xfa/0x200 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562708] [] ? xen_pmd_val+0xe/0x10 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562711] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562713] [] handle_mm_fault+0x269/0x370 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562717] [] do_page_fault+0x184/0x550 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562720] [] ? ktime_get_ts+0xad/0xe0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562723] [] ? poll_select_copy_remaining+0xed/0x140 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562725] [] ? sys_select+0xcd/0x100 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562728] [] ? hypercall_page+0x22a/0x1000 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562731] [] page_fault+0x25/0x30 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562733] Mem-Info: Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562735] Node 0 DMA per-cpu: Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562736] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562738] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562739] Node 0 DMA32 per-cpu: Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562741] CPU 0: hi: 186, btch: 31 usd: 79 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562742] CPU 1: hi: 186, btch: 31 usd: 40 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562745] active_anon:932720 inactive_anon:1393 isolated_anon:0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562745] active_file:0 inactive_file:2 isolated_file:31 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562746] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562747] free:4045 slab_reclaimable:2497 slab_unreclaimable:2632 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562747] mapped:1365 shmem:1405 pagetables:3256 bounce:0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562749] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562755] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562758] Node 0 DMA32 free:7844kB min:7848kB low:9808kB high:11772kB active_anon:3730880kB inactive_anon:5572kB active_file:0kB inactive_file:8kB unevictable:0kB isolated(anon):0kB isolated(file):124kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5460kB shmem:5620kB slab_reclaimable:9988kB slab_unreclaimable:10528kB kernel_stack:744kB pagetables:13024kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:225 all_unreclaimable? yes Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562765] lowmem_reserve[]: 0 0 0 0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562767] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562774] Node 0 DMA32: 951*4kB 225*8kB 10*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7844kB Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562781] 1453 total pagecache pages Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562782] 0 pages in swap cache Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562783] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562784] Free swap = 0kB Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562785] Total swap = 0kB Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568076] 985072 pages RAM Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568078] 24998 pages reserved Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568079] 3781 pages shared Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568080] 953235 pages non-shared Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568082] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568091] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568094] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568097] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568100] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568104] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568107] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568110] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568114] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568117] [ 790] 101 790 63477 630 0 0 0 rsyslogd Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568121] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568124] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568127] [ 975] 0 975 6276 79 0 0 0 master Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568130] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568133] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568136] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568139] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568142] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568146] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568149] [15657] 0 15657 4777 60 1 0 0 cron Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568152] [ 654] 1012 654 70351 5759 0 0 0 php Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568155] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568158] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568161] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568164] [13468] 0 13468 89584 2532 0 0 0 php5-fpm Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568167] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568170] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568173] [ 2686] 1012 2686 68618 5661 1 0 0 php Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568177] [ 8098] 108 8098 6792 76 1 0 0 pickup Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568180] [14344] 1012 14344 885550 822379 0 0 0 php Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568184] [28739] 1012 28739 68580 5604 1 0 0 php Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568187] [29002] 1012 29002 1099 25 1 0 0 sh Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568190] [29003] 1012 29003 68825 5870 1 0 0 php Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568193] [29009] 1012 29009 1099 26 0 0 0 sh Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568196] [29010] 1012 29010 4736 65 0 0 0 git Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568199] [29011] 1012 29011 31456 384 0 0 0 php Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568202] Out of memory: Kill process 14344 (php) score 858 or sacrifice child Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568213] Killed process 14344 (php) total-vm:3542200kB, anon-rss:3289516kB, file-rss:0kB Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684731] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684736] php cpuset=/ mems_allowed=0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684739] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684741] Call Trace: Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684749] [] dump_header+0x91/0xe0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684752] [] oom_kill_process+0x85/0xb0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684755] [] out_of_memory+0xfa/0x220 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684759] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684765] [] ? error_exit+0x2a/0x60 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684768] [] alloc_pages_current+0xb6/0x120 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684773] [] __page_cache_alloc+0xb7/0xd0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684776] [] filemap_fault+0x212/0x3c0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684780] [] __do_fault+0x72/0x550 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684784] [] ? __pollwait+0xf0/0xf0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684787] [] handle_pte_fault+0xfa/0x200 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684792] [] ? xen_pmd_val+0xe/0x10 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684795] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684799] [] handle_mm_fault+0x269/0x370 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684803] [] do_page_fault+0x184/0x550 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684807] [] ? ktime_get_ts+0xad/0xe0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684810] [] ? poll_select_copy_remaining+0xed/0x140 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684813] [] ? sys_select+0xcd/0x100 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684817] [] page_fault+0x25/0x30 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684819] Mem-Info: Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684821] Node 0 DMA per-cpu: Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684823] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684825] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684826] Node 0 DMA32 per-cpu: Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684829] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684831] CPU 1: hi: 186, btch: 31 usd: 95 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684834] active_anon:933063 inactive_anon:1393 isolated_anon:0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684835] active_file:0 inactive_file:0 isolated_file:0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684836] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684837] free:4028 slab_reclaimable:2445 slab_unreclaimable:2644 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684838] mapped:1356 shmem:1405 pagetables:3065 bounce:0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684840] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684848] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684852] Node 0 DMA32 free:7776kB min:7848kB low:9808kB high:11772kB active_anon:3732252kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:9780kB slab_unreclaimable:10576kB kernel_stack:728kB pagetables:12260kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:358 all_unreclaimable? yes Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684861] lowmem_reserve[]: 0 0 0 0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684864] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684872] Node 0 DMA32: 902*4kB 199*8kB 25*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7776kB Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684878] 1406 total pagecache pages Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684880] 0 pages in swap cache Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684881] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684882] Free swap = 0kB Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684883] Total swap = 0kB Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690259] 985072 pages RAM Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690262] 24998 pages reserved Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690263] 3775 pages shared Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690264] 953287 pages non-shared Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690266] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690274] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690278] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690281] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690284] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690288] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690291] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690294] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690297] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690301] [ 790] 101 790 63477 630 0 0 0 rsyslogd Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690305] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690308] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690311] [ 975] 0 975 6276 79 0 0 0 master Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690314] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690317] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690320] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690324] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690327] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690330] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690333] [15657] 0 15657 4777 59 1 0 0 cron Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690336] [ 654] 1012 654 70351 5759 0 0 0 php Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690339] [ 657] 1012 657 69242 6258 0 0 0 php Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690342] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690345] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690348] [13468] 0 13468 89584 2517 0 0 0 php5-fpm Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690351] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690354] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690358] [ 2686] 1012 2686 68618 5660 1 0 0 php Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690361] [ 8098] 108 8098 6792 76 1 0 0 pickup Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690365] [29034] 1012 29034 892243 829130 1 0 0 php Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690368] [30598] 1012 30598 68580 5605 0 0 0 php Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690371] Out of memory: Kill process 29034 (php) score 865 or sacrifice child Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690382] Killed process 29034 (php) total-vm:3568972kB, anon-rss:3316520kB, file-rss:0kB Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056231] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056235] php cpuset=/ mems_allowed=0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056238] Pid: 848, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056240] Call Trace: Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056247] [] dump_header+0x91/0xe0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056249] [] oom_kill_process+0x85/0xb0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056251] [] out_of_memory+0xfa/0x220 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056254] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056257] [] alloc_pages_current+0xb6/0x120 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056261] [] __page_cache_alloc+0xb7/0xd0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056263] [] filemap_fault+0x212/0x3c0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056266] [] __do_fault+0x72/0x550 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056269] [] ? mntput_no_expire+0x30/0xf0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056272] [] handle_pte_fault+0xfa/0x200 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056275] [] ? xen_pmd_val+0xe/0x10 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056277] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056280] [] handle_mm_fault+0x269/0x370 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056283] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056285] [] ? check_events+0x12/0x20 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056288] [] do_page_fault+0x184/0x550 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056292] [] ? sys_newfstat+0x2a/0x40 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056295] [] page_fault+0x25/0x30 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056297] Mem-Info: Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056298] Node 0 DMA per-cpu: Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056300] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056301] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056303] Node 0 DMA32 per-cpu: Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056304] CPU 0: hi: 186, btch: 31 usd: 33 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056306] CPU 1: hi: 186, btch: 31 usd: 62 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056308] active_anon:933063 inactive_anon:1393 isolated_anon:0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056309] active_file:37 inactive_file:0 isolated_file:0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056310] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056310] free:4028 slab_reclaimable:2372 slab_unreclaimable:2599 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056311] mapped:1342 shmem:1405 pagetables:3166 bounce:0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056312] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056319] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056322] Node 0 DMA32 free:7776kB min:7848kB low:9808kB high:11772kB active_anon:3732252kB inactive_anon:5572kB active_file:148kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5368kB shmem:5620kB slab_reclaimable:9488kB slab_unreclaimable:10396kB kernel_stack:752kB pagetables:12664kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:164 all_unreclaimable? yes Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056329] lowmem_reserve[]: 0 0 0 0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056331] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056338] Node 0 DMA32: 866*4kB 231*8kB 18*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7776kB Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056344] 1427 total pagecache pages Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056346] 0 pages in swap cache Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056347] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056348] Free swap = 0kB Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056349] Total swap = 0kB Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061569] 985072 pages RAM Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061572] 24998 pages reserved Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061573] 3793 pages shared Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061574] 953266 pages non-shared Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061576] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061585] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061589] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061592] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061595] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061598] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061601] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061605] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061608] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061612] [ 790] 101 790 63477 630 0 0 0 rsyslogd Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061615] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061618] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061622] [ 975] 0 975 6276 79 1 0 0 master Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061625] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061628] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061631] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061634] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061637] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061640] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061643] [15657] 0 15657 4777 59 1 0 0 cron Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061647] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061650] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061653] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061656] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061659] [13468] 0 13468 89584 2502 0 0 0 php5-fpm Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061662] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061665] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061668] [ 2686] 1012 2686 68618 5660 0 0 0 php Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061672] [ 8098] 108 8098 6792 76 0 0 0 pickup Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061675] [30806] 1012 30806 888684 825558 0 0 0 php Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061678] [ 710] 1012 710 68580 5605 1 0 0 php Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061681] [ 847] 1012 847 1099 25 1 0 0 sh Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061684] [ 848] 1012 848 65963 3547 1 0 0 php Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061687] Out of memory: Kill process 30806 (php) score 861 or sacrifice child Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061699] Killed process 30806 (php) total-vm:3554736kB, anon-rss:3302232kB, file-rss:0kB Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876165] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876168] php cpuset=/ mems_allowed=0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876171] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876173] Call Trace: Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876181] [] dump_header+0x91/0xe0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876184] [] oom_kill_process+0x85/0xb0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876187] [] out_of_memory+0xfa/0x220 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876190] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876197] [] ? error_exit+0x2a/0x60 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876205] [] alloc_pages_current+0xb6/0x120 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876209] [] __page_cache_alloc+0xb7/0xd0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876212] [] filemap_fault+0x212/0x3c0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876216] [] __do_fault+0x72/0x550 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876219] [] handle_pte_fault+0xfa/0x200 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876223] [] ? xen_pmd_val+0xe/0x10 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876226] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876229] [] handle_mm_fault+0x269/0x370 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876233] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876236] [] ? check_events+0x12/0x20 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876239] [] do_page_fault+0x184/0x550 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876242] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876247] [] ? xen_clts+0x8d/0x190 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876251] [] page_fault+0x25/0x30 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876255] Mem-Info: Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876256] Node 0 DMA per-cpu: Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876259] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876261] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876262] Node 0 DMA32 per-cpu: Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876265] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876267] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876270] active_anon:933073 inactive_anon:1393 isolated_anon:0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876271] active_file:3 inactive_file:17 isolated_file:0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876272] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876273] free:4048 slab_reclaimable:2516 slab_unreclaimable:2578 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876274] mapped:1343 shmem:1405 pagetables:3072 bounce:0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876276] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876284] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876288] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732292kB inactive_anon:5572kB active_file:12kB inactive_file:68kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5372kB shmem:5620kB slab_reclaimable:10064kB slab_unreclaimable:10312kB kernel_stack:728kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:24227 all_unreclaimable? yes Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876297] lowmem_reserve[]: 0 0 0 0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876300] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876309] Node 0 DMA32: 672*4kB 168*8kB 58*16kB 24*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7904kB Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876318] 1437 total pagecache pages Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876320] 0 pages in swap cache Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876322] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876323] Free swap = 0kB Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876325] Total swap = 0kB Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882161] 985072 pages RAM Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882163] 24998 pages reserved Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882165] 3799 pages shared Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882166] 953331 pages non-shared Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882168] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882176] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882180] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882183] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882186] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882189] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882193] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882196] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882199] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882203] [ 790] 101 790 63477 630 1 0 0 rsyslogd Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882207] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882210] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882213] [ 975] 0 975 6276 79 0 0 0 master Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882216] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882222] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882225] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882228] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882231] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882234] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882237] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882240] [ 654] 1012 654 70351 5759 0 0 0 php Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882243] [ 657] 1012 657 69242 6257 0 0 0 php Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882246] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882249] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882252] [13468] 0 13468 89584 2525 1 0 0 php5-fpm Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882256] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882259] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882262] [ 2686] 1012 2686 68618 5660 1 0 0 php Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882266] [ 8098] 108 8098 6792 76 1 0 0 pickup Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882269] [ 880] 1012 880 892207 829116 0 0 0 php Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882272] [ 3125] 1012 3125 68580 5603 0 0 0 php Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882275] Out of memory: Kill process 880 (php) score 865 or sacrifice child Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882289] Killed process 880 (php) total-vm:3568828kB, anon-rss:3316376kB, file-rss:88kB Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008505] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008510] php cpuset=/ mems_allowed=0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008512] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008514] Call Trace: Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008521] [] dump_header+0x91/0xe0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008523] [] oom_kill_process+0x85/0xb0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008526] [] out_of_memory+0xfa/0x220 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008529] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008532] [] alloc_pages_current+0xb6/0x120 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008536] [] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008540] [] __page_cache_alloc+0xb7/0xd0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008542] [] filemap_fault+0x212/0x3c0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008545] [] __do_fault+0x72/0x550 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008548] [] ? load_TLS_descriptor+0x42/0xd0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008551] [] handle_pte_fault+0xfa/0x200 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008553] [] ? xen_pmd_val+0xe/0x10 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008556] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008558] [] handle_mm_fault+0x269/0x370 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008561] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008563] [] ? check_events+0x12/0x20 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008565] [] do_page_fault+0x184/0x550 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008568] [] ? do_nanosleep+0xa4/0xd0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008573] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008575] [] ? update_rmtp+0x70/0x70 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008578] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008581] [] page_fault+0x25/0x30 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008582] Mem-Info: Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008583] Node 0 DMA per-cpu: Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008585] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008587] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008588] Node 0 DMA32 per-cpu: Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008589] CPU 0: hi: 186, btch: 31 usd: 39 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008591] CPU 1: hi: 186, btch: 31 usd: 30 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008593] active_anon:933083 inactive_anon:1393 isolated_anon:0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008594] active_file:0 inactive_file:0 isolated_file:0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008594] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008595] free:4044 slab_reclaimable:2517 slab_unreclaimable:2575 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008596] mapped:1355 shmem:1405 pagetables:3077 bounce:0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008597] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008604] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008607] Node 0 DMA32 free:7840kB min:7848kB low:9808kB high:11772kB active_anon:3732332kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5620kB slab_reclaimable:10068kB slab_unreclaimable:10300kB kernel_stack:728kB pagetables:12308kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:85 all_unreclaimable? yes Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008613] lowmem_reserve[]: 0 0 0 0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008616] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008623] Node 0 DMA32: 890*4kB 193*8kB 33*16kB 5*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7840kB Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008630] 1465 total pagecache pages Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008631] 0 pages in swap cache Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008632] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008633] Free swap = 0kB Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008634] Total swap = 0kB Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013938] 985072 pages RAM Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013940] 24998 pages reserved Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013941] 3783 pages shared Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013943] 953289 pages non-shared Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013945] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013954] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013957] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013960] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013963] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013967] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013970] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013973] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013977] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013981] [ 790] 101 790 63477 630 1 0 0 rsyslogd Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013984] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013987] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013991] [ 975] 0 975 6276 79 0 0 0 master Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013994] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013997] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014000] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014003] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014006] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014009] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014013] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014016] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014019] [ 657] 1012 657 69242 6257 0 0 0 php Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014022] [ 658] 1012 658 134393 70798 1 0 0 php Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014025] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014028] [13468] 0 13468 89584 2541 0 0 0 php5-fpm Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014031] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014034] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014037] [ 2686] 1012 2686 68618 5660 1 0 0 php Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014041] [ 8098] 108 8098 6792 76 1 0 0 pickup Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014044] [ 3257] 1012 3257 892243 829129 0 0 0 php Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014047] [ 5015] 1012 5015 68580 5603 0 0 0 php Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014050] Out of memory: Kill process 3257 (php) score 865 or sacrifice child Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014061] Killed process 3257 (php) total-vm:3568972kB, anon-rss:3316516kB, file-rss:0kB Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120140] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120144] php cpuset=/ mems_allowed=0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120147] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120148] Call Trace: Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120155] [] dump_header+0x91/0xe0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120157] [] oom_kill_process+0x85/0xb0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120159] [] out_of_memory+0xfa/0x220 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120162] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120167] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120170] [] alloc_pages_current+0xb6/0x120 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120173] [] __page_cache_alloc+0xb7/0xd0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120175] [] filemap_fault+0x212/0x3c0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120178] [] __do_fault+0x72/0x550 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120181] [] ? load_TLS_descriptor+0x42/0xd0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120184] [] handle_pte_fault+0xfa/0x200 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120186] [] ? xen_pmd_val+0xe/0x10 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120188] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120191] [] handle_mm_fault+0x269/0x370 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120193] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120196] [] ? check_events+0x12/0x20 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120199] [] do_page_fault+0x184/0x550 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120202] [] ? do_nanosleep+0xa4/0xd0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120205] [] ? __switch_to+0x288/0x360 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120210] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120213] [] ? update_rmtp+0x70/0x70 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120215] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120218] [] page_fault+0x25/0x30 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120220] Mem-Info: Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120221] Node 0 DMA per-cpu: Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120223] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120224] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120225] Node 0 DMA32 per-cpu: Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120227] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120228] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120231] active_anon:933072 inactive_anon:1393 isolated_anon:0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120232] active_file:0 inactive_file:0 isolated_file:0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120232] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120233] free:4048 slab_reclaimable:2576 slab_unreclaimable:2573 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120234] mapped:1366 shmem:1405 pagetables:3073 bounce:0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120235] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120242] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120245] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732288kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:10304kB slab_unreclaimable:10292kB kernel_stack:728kB pagetables:12292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120251] lowmem_reserve[]: 0 0 0 0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120253] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120260] Node 0 DMA32: 986*4kB 221*8kB 3*16kB 0*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7872kB Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120267] 1477 total pagecache pages Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120268] 0 pages in swap cache Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120269] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120270] Free swap = 0kB Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120271] Total swap = 0kB Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125675] 985072 pages RAM Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125678] 24998 pages reserved Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125679] 3817 pages shared Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125680] 953319 pages non-shared Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125682] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125690] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125694] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125697] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125700] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125703] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125706] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125710] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125713] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125717] [ 790] 101 790 63477 630 0 0 0 rsyslogd Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125720] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125724] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125727] [ 975] 0 975 6276 79 1 0 0 master Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125730] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125733] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125736] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125739] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125742] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125745] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125749] [15657] 0 15657 4777 60 1 0 0 cron Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125752] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125755] [ 657] 1012 657 69242 6257 0 0 0 php Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125758] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125761] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125764] [13468] 0 13468 89584 2522 1 0 0 php5-fpm Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125767] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125770] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125773] [ 2686] 1012 2686 68618 5661 1 0 0 php Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125777] [ 8098] 108 8098 6792 76 0 0 0 pickup Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125780] [ 5182] 1012 5182 892243 829130 0 0 0 php Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125783] [14326] 1012 14326 68580 5604 1 0 0 php Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125786] Out of memory: Kill process 5182 (php) score 865 or sacrifice child Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125798] Killed process 5182 (php) total-vm:3568972kB, anon-rss:3316520kB, file-rss:0kB Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012471] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012475] php cpuset=/ mems_allowed=0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012478] Pid: 14571, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012480] Call Trace: Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012487] [] dump_header+0x91/0xe0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012489] [] oom_kill_process+0x85/0xb0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012491] [] out_of_memory+0xfa/0x220 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012494] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012499] [] ? error_exit+0x2a/0x60 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012502] [] alloc_pages_current+0xb6/0x120 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012505] [] __page_cache_alloc+0xb7/0xd0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012507] [] filemap_fault+0x212/0x3c0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012510] [] __do_fault+0x72/0x550 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012513] [] ? error_exit+0x2a/0x60 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012515] [] ? retint_restore_args+0x5/0x6 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012519] [] ? pte_mfn_to_pfn+0x67/0x110 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012521] [] handle_pte_fault+0xfa/0x200 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012523] [] ? xen_pmd_val+0xe/0x10 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012526] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012528] [] handle_mm_fault+0x269/0x370 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012531] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012533] [] ? check_events+0x12/0x20 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012535] [] do_page_fault+0x184/0x550 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012538] [] ? do_brk+0x21f/0x320 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012540] [] ? xen_clts+0x8d/0x190 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012543] [] ? __schedule+0x3cc/0x6f0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012545] [] page_fault+0x25/0x30 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012547] Mem-Info: Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012548] Node 0 DMA per-cpu: Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012550] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012551] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012552] Node 0 DMA32 per-cpu: Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012554] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012557] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012559] active_anon:933405 inactive_anon:1393 isolated_anon:0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012560] active_file:20 inactive_file:10 isolated_file:0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012561] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012561] free:4048 slab_reclaimable:2263 slab_unreclaimable:2580 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012562] mapped:1358 shmem:1405 pagetables:3068 bounce:0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012564] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012570] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012573] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3733620kB inactive_anon:5572kB active_file:80kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:9052kB slab_unreclaimable:10320kB kernel_stack:728kB pagetables:12272kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:192 all_unreclaimable? yes Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012580] lowmem_reserve[]: 0 0 0 0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012582] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012589] Node 0 DMA32: 551*4kB 289*8kB 70*16kB 10*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8004kB Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012596] 1430 total pagecache pages Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012597] 0 pages in swap cache Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012598] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012599] Free swap = 0kB Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012600] Total swap = 0kB Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017770] 985072 pages RAM Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017772] 24998 pages reserved Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017774] 3775 pages shared Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017775] 953325 pages non-shared Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017777] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017786] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017789] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017792] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017795] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017798] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017802] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017805] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017809] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017813] [ 790] 101 790 63477 630 1 0 0 rsyslogd Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017816] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017819] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017822] [ 975] 0 975 6276 79 1 0 0 master Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017825] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017828] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017831] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017834] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017837] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017840] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017844] [15657] 0 15657 4777 60 1 0 0 cron Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017847] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017850] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017853] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017856] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017859] [13468] 0 13468 89584 2507 1 0 0 php5-fpm Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017862] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017865] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017869] [ 2686] 1012 2686 68618 5661 1 0 0 php Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017872] [ 8098] 108 8098 6792 76 0 0 0 pickup Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017876] [14571] 1012 14571 892563 829462 0 0 0 php Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017879] [14704] 1012 14704 68580 5604 1 0 0 php Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017881] Out of memory: Kill process 14571 (php) score 865 or sacrifice child Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017892] Killed process 14571 (php) total-vm:3570252kB, anon-rss:3317848kB, file-rss:0kB Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584177] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584181] php cpuset=/ mems_allowed=0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584184] Pid: 13231, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584186] Call Trace: Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584193] [] dump_header+0x91/0xe0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584195] [] oom_kill_process+0x85/0xb0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584198] [] out_of_memory+0xfa/0x220 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584200] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584204] [] alloc_pages_current+0xb6/0x120 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584208] [] __page_cache_alloc+0xb7/0xd0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584210] [] filemap_fault+0x212/0x3c0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584213] [] __do_fault+0x72/0x550 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584215] [] ? do_wp_page+0x2e8/0x740 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584219] [] ? pte_mfn_to_pfn+0x8d/0x110 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584221] [] handle_pte_fault+0xfa/0x200 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584223] [] ? xen_pmd_val+0xe/0x10 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584226] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584228] [] handle_mm_fault+0x269/0x370 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584232] [] ? check_preempt_wakeup+0x15b/0x230 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584236] [] do_page_fault+0x184/0x550 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584239] [] ? ktime_get_ts+0xad/0xe0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584242] [] ? poll_select_copy_remaining+0xed/0x140 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584245] [] ? sys_select+0xcd/0x100 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584248] [] page_fault+0x25/0x30 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584250] Mem-Info: Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584251] Node 0 DMA per-cpu: Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584253] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584254] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584255] Node 0 DMA32 per-cpu: Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584257] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584258] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584261] active_anon:932762 inactive_anon:1393 isolated_anon:0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584261] active_file:30 inactive_file:32 isolated_file:0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584262] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584263] free:4048 slab_reclaimable:2605 slab_unreclaimable:2611 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584263] mapped:1369 shmem:1405 pagetables:3219 bounce:0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584265] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584272] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584275] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731048kB inactive_anon:5572kB active_file:120kB inactive_file:128kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5476kB shmem:5620kB slab_reclaimable:10420kB slab_unreclaimable:10444kB kernel_stack:768kB pagetables:12876kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:14048 all_unreclaimable? yes Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584281] lowmem_reserve[]: 0 0 0 0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584284] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584290] Node 0 DMA32: 843*4kB 155*8kB 48*16kB 8*32kB 4*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7940kB Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584297] 1515 total pagecache pages Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584298] 0 pages in swap cache Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584300] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584301] Free swap = 0kB Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584302] Total swap = 0kB Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589859] 985072 pages RAM Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589862] 24998 pages reserved Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589863] 3829 pages shared Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589865] 953287 pages non-shared Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589867] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589875] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589879] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589882] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589885] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589888] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589891] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589894] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589898] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589902] [ 790] 101 790 63477 630 0 0 0 rsyslogd Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589905] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589908] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589912] [ 975] 0 975 6276 79 0 0 0 master Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589915] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589918] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589921] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589924] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589927] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589930] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589934] [15657] 0 15657 4777 60 0 0 0 cron Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589937] [ 654] 1012 654 70351 5759 0 0 0 php Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589940] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589943] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589946] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589949] [13468] 0 13468 89584 2519 0 0 0 php5-fpm Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589952] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589955] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589958] [ 2686] 1012 2686 68618 5661 1 0 0 php Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589962] [ 2077] 108 2077 6792 69 1 0 0 pickup Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589965] [ 6968] 1012 6968 885843 822684 0 0 0 php Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589968] [13000] 1012 13000 68580 5603 0 0 0 php Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589971] [13230] 1012 13230 1099 25 1 0 0 sh Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589974] [13231] 1012 13231 68825 5872 1 0 0 php Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589977] [13237] 1012 13237 1099 25 1 0 0 sh Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589980] [13238] 1012 13238 14040 63 1 0 0 git Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589983] [13239] 1012 13239 12484 152 0 0 0 php Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589985] Out of memory: Kill process 6968 (php) score 858 or sacrifice child Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589996] Killed process 6968 (php) total-vm:3543372kB, anon-rss:3290628kB, file-rss:108kB Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548117] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548121] php cpuset=/ mems_allowed=0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548124] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548126] Call Trace: Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548133] [] dump_header+0x91/0xe0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548135] [] oom_kill_process+0x85/0xb0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548137] [] out_of_memory+0xfa/0x220 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548140] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548145] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548148] [] alloc_pages_current+0xb6/0x120 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548152] [] __page_cache_alloc+0xb7/0xd0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548154] [] filemap_fault+0x212/0x3c0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548157] [] __do_fault+0x72/0x550 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548161] [] ? error_exit+0x2a/0x60 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548163] [] ? retint_restore_args+0x5/0x6 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548166] [] handle_pte_fault+0xfa/0x200 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548169] [] ? xen_pmd_val+0xe/0x10 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548171] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548174] [] handle_mm_fault+0x269/0x370 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548176] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548179] [] ? check_events+0x12/0x20 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548181] [] do_page_fault+0x184/0x550 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548185] [] ? security_file_permission+0x93/0xb0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548188] [] ? rw_verify_area+0x61/0xf0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548190] [] ? vfs_read+0xb0/0x180 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548192] [] page_fault+0x25/0x30 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548194] Mem-Info: Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548195] Node 0 DMA per-cpu: Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548197] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548198] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548199] Node 0 DMA32 per-cpu: Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548201] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548202] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548205] active_anon:933026 inactive_anon:1393 isolated_anon:0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548206] active_file:0 inactive_file:0 isolated_file:0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548206] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548207] free:4048 slab_reclaimable:2495 slab_unreclaimable:2619 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548208] mapped:1381 shmem:1405 pagetables:3053 bounce:0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548209] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548216] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548219] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732104kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5524kB shmem:5620kB slab_reclaimable:9980kB slab_unreclaimable:10476kB kernel_stack:728kB pagetables:12212kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548225] lowmem_reserve[]: 0 0 0 0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548228] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548234] Node 0 DMA32: 796*4kB 210*8kB 49*16kB 5*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548241] 1448 total pagecache pages Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548242] 0 pages in swap cache Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548244] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548245] Free swap = 0kB Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548246] Total swap = 0kB Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553451] 985072 pages RAM Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553453] 24998 pages reserved Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553454] 3795 pages shared Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553456] 953344 pages non-shared Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553457] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553466] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553470] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553473] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553476] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553479] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553483] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553486] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553489] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553493] [ 790] 101 790 63477 631 1 0 0 rsyslogd Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553496] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553499] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553503] [ 975] 0 975 6276 79 0 0 0 master Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553506] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553509] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553512] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553515] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553518] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553521] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553524] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553528] [ 654] 1012 654 70351 5759 0 0 0 php Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553530] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553533] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553536] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553539] [13468] 0 13468 89584 2504 0 0 0 php5-fpm Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553543] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553546] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553549] [ 2686] 1012 2686 68618 5660 1 0 0 php Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553552] [ 2077] 108 2077 6792 69 1 0 0 pickup Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553556] [13283] 1012 13283 892243 829097 0 0 0 php Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553559] [14187] 1012 14187 68580 5604 0 0 0 php Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553562] Out of memory: Kill process 13283 (php) score 865 or sacrifice child Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553572] Killed process 13283 (php) total-vm:3568972kB, anon-rss:3316388kB, file-rss:0kB Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032169] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032174] php cpuset=/ mems_allowed=0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032177] Pid: 14401, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032179] Call Trace: Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032186] [] dump_header+0x91/0xe0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032189] [] oom_kill_process+0x85/0xb0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032191] [] out_of_memory+0xfa/0x220 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032194] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032199] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032202] [] alloc_pages_current+0xb6/0x120 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032206] [] __page_cache_alloc+0xb7/0xd0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032208] [] filemap_fault+0x212/0x3c0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032211] [] __do_fault+0x72/0x550 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032214] [] ? lru_cache_add_lru+0x2d/0x50 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032216] [] handle_pte_fault+0xfa/0x200 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032219] [] ? xen_pmd_val+0xe/0x10 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032221] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032224] [] handle_mm_fault+0x269/0x370 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032227] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032229] [] ? check_events+0x12/0x20 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032232] [] do_page_fault+0x184/0x550 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032235] [] ? do_brk+0x21f/0x320 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032237] [] ? xen_clts+0x8d/0x190 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032240] [] ? __schedule+0x3cc/0x6f0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032242] [] page_fault+0x25/0x30 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032244] Mem-Info: Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032245] Node 0 DMA per-cpu: Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032247] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032248] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032250] Node 0 DMA32 per-cpu: Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032251] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032253] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032255] active_anon:933043 inactive_anon:1393 isolated_anon:0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032256] active_file:0 inactive_file:0 isolated_file:0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032257] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032257] free:4048 slab_reclaimable:2517 slab_unreclaimable:2581 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032258] mapped:1362 shmem:1405 pagetables:3071 bounce:0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032259] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032266] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032269] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732172kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5620kB slab_reclaimable:10068kB slab_unreclaimable:10324kB kernel_stack:728kB pagetables:12284kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032275] lowmem_reserve[]: 0 0 0 0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032278] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032285] Node 0 DMA32: 860*4kB 156*8kB 48*16kB 11*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032291] 1438 total pagecache pages Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032292] 0 pages in swap cache Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032294] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032295] Free swap = 0kB Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032296] Total swap = 0kB Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037521] 985072 pages RAM Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037524] 24998 pages reserved Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037525] 3816 pages shared Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037526] 953309 pages non-shared Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037528] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037537] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037540] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037543] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037546] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037549] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037553] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037556] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037559] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037563] [ 790] 101 790 63477 631 0 0 0 rsyslogd Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037567] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037570] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037573] [ 975] 0 975 6276 79 0 0 0 master Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037576] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037579] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037582] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037585] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037588] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037591] [ 1242] 107 1242 9445 163 0 0 0 ntpd Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037595] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037597] [ 654] 1012 654 70351 5759 1 0 0 php Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037600] [ 657] 1012 657 69242 6260 1 0 0 php Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037603] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037606] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037609] [13468] 0 13468 89584 2533 0 0 0 php5-fpm Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037612] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037615] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037618] [ 2686] 1012 2686 68618 5660 1 0 0 php Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037622] [ 2077] 108 2077 6792 70 1 0 0 pickup Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037625] [14401] 1012 14401 892243 829095 0 0 0 php Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037628] [17552] 1012 17552 68580 5606 0 0 0 php Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037631] Out of memory: Kill process 14401 (php) score 865 or sacrifice child Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037642] Killed process 14401 (php) total-vm:3568972kB, anon-rss:3316380kB, file-rss:0kB Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840149] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840153] php cpuset=/ mems_allowed=0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840155] Pid: 17625, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840157] Call Trace: Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840163] [] dump_header+0x91/0xe0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840166] [] oom_kill_process+0x85/0xb0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840168] [] out_of_memory+0xfa/0x220 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840171] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840176] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840178] [] alloc_pages_current+0xb6/0x120 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840182] [] __page_cache_alloc+0xb7/0xd0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840184] [] filemap_fault+0x212/0x3c0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840187] [] __do_fault+0x72/0x550 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840190] [] handle_pte_fault+0xfa/0x200 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840193] [] ? xen_pmd_val+0xe/0x10 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840195] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840198] [] handle_mm_fault+0x269/0x370 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840200] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840203] [] ? check_events+0x12/0x20 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840206] [] do_page_fault+0x184/0x550 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840208] [] ? do_brk+0x21f/0x320 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840210] [] ? xen_clts+0x8d/0x190 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840214] [] page_fault+0x25/0x30 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840215] Mem-Info: Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840216] Node 0 DMA per-cpu: Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840218] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840220] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840221] Node 0 DMA32 per-cpu: Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840223] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840224] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840226] active_anon:932922 inactive_anon:1393 isolated_anon:0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840227] active_file:0 inactive_file:4 isolated_file:0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840228] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840228] free:4048 slab_reclaimable:2512 slab_unreclaimable:2611 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840229] mapped:1361 shmem:1405 pagetables:3169 bounce:0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840231] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840237] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840240] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731688kB inactive_anon:5572kB active_file:0kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5620kB slab_reclaimable:10048kB slab_unreclaimable:10444kB kernel_stack:744kB pagetables:12676kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2697 all_unreclaimable? yes Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840247] lowmem_reserve[]: 0 0 0 0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840249] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840256] Node 0 DMA32: 811*4kB 165*8kB 57*16kB 14*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7972kB Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840262] 1460 total pagecache pages Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840264] 0 pages in swap cache Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840265] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840266] Free swap = 0kB Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840267] Total swap = 0kB Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845437] 985072 pages RAM Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845439] 24998 pages reserved Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845440] 3787 pages shared Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845441] 953319 pages non-shared Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845443] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845452] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845456] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845459] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845462] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845465] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845468] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845472] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845475] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845479] [ 790] 101 790 63477 631 1 0 0 rsyslogd Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845482] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845485] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845488] [ 975] 0 975 6276 79 1 0 0 master Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845491] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845494] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845498] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845501] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845504] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845507] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845510] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845513] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845516] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845519] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845522] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845525] [13468] 0 13468 89584 2503 0 0 0 php5-fpm Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845529] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845532] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845535] [ 2686] 1012 2686 68618 5660 1 0 0 php Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845538] [ 2077] 108 2077 6792 70 0 0 0 pickup Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845542] [17625] 1012 17625 889363 826275 0 0 0 php Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845545] [19378] 1012 19378 68580 5605 1 0 0 php Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845548] [19607] 1012 19607 1099 24 1 0 0 sh Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845551] [19608] 1012 19608 65118 2673 1 0 0 php Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845553] Out of memory: Kill process 17625 (php) score 862 or sacrifice child Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845564] Killed process 17625 (php) total-vm:3557452kB, anon-rss:3305100kB, file-rss:0kB Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668140] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668144] php cpuset=/ mems_allowed=0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668147] Pid: 19640, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668149] Call Trace: Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668156] [] dump_header+0x91/0xe0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668158] [] oom_kill_process+0x85/0xb0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668160] [] out_of_memory+0xfa/0x220 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668163] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668166] [] alloc_pages_current+0xb6/0x120 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668170] [] __page_cache_alloc+0xb7/0xd0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668172] [] filemap_fault+0x212/0x3c0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668175] [] __do_fault+0x72/0x550 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668178] [] ? lru_cache_add_lru+0x2d/0x50 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668180] [] handle_pte_fault+0xfa/0x200 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668184] [] ? xen_pmd_val+0xe/0x10 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668186] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668188] [] handle_mm_fault+0x269/0x370 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668192] [] do_page_fault+0x184/0x550 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668194] [] ? do_brk+0x21f/0x320 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668196] [] ? xen_clts+0x8d/0x190 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668199] [] ? __schedule+0x3cc/0x6f0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668202] [] page_fault+0x25/0x30 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668203] Mem-Info: Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668205] Node 0 DMA per-cpu: Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668207] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668208] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668209] Node 0 DMA32 per-cpu: Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668211] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668212] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668215] active_anon:932940 inactive_anon:1393 isolated_anon:0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668215] active_file:0 inactive_file:13 isolated_file:0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668216] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668216] free:4048 slab_reclaimable:2417 slab_unreclaimable:2591 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668217] mapped:1336 shmem:1405 pagetables:3262 bounce:0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668219] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668225] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668228] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731760kB inactive_anon:5572kB active_file:0kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5344kB shmem:5620kB slab_reclaimable:9668kB slab_unreclaimable:10364kB kernel_stack:736kB pagetables:13048kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:848 all_unreclaimable? yes Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668235] lowmem_reserve[]: 0 0 0 0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668237] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668244] Node 0 DMA32: 826*4kB 293*8kB 16*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668251] 1413 total pagecache pages Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668252] 0 pages in swap cache Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668253] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668254] Free swap = 0kB Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668255] Total swap = 0kB Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673486] 985072 pages RAM Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673489] 24998 pages reserved Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673490] 3777 pages shared Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673491] 953331 pages non-shared Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673493] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673501] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673505] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673508] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673511] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673514] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673518] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673521] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673524] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673528] [ 790] 101 790 63477 631 1 0 0 rsyslogd Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673531] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673534] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673538] [ 975] 0 975 6276 79 0 0 0 master Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673541] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673544] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673547] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673550] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673553] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673557] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673560] [15657] 0 15657 4777 60 1 0 0 cron Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673563] [ 654] 1012 654 70351 5761 1 0 0 php Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673566] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673569] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673573] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673575] [13468] 0 13468 89584 2541 1 0 0 php5-fpm Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673578] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673581] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673585] [ 2686] 1012 2686 68618 5661 1 0 0 php Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673588] [ 2077] 108 2077 6792 70 1 0 0 pickup Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673591] [19640] 1012 19640 885676 822541 0 0 0 php Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673595] [ 2845] 1012 2845 68580 5605 1 0 0 php Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673598] [ 2997] 1012 2997 1099 24 1 0 0 sh Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673601] [ 2998] 1012 2998 68825 5870 1 0 0 php Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673604] [ 3004] 1012 3004 1099 25 1 0 0 sh Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673607] [ 3005] 1012 3005 4703 67 1 0 0 git Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673610] [ 3006] 1012 3006 39759 454 0 0 0 php Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673613] Out of memory: Kill process 19640 (php) score 858 or sacrifice child Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673624] Killed process 19640 (php) total-vm:3542704kB, anon-rss:3290164kB, file-rss:0kB Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844128] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844132] php5-fpm cpuset=/ mems_allowed=0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844135] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844136] Call Trace: Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844143] [] dump_header+0x91/0xe0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844146] [] oom_kill_process+0x85/0xb0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844148] [] out_of_memory+0xfa/0x220 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844152] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844155] [] alloc_pages_current+0xb6/0x120 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844159] [] __page_cache_alloc+0xb7/0xd0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844162] [] filemap_fault+0x212/0x3c0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844165] [] __do_fault+0x72/0x550 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844168] [] ? __switch_to+0x138/0x360 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844172] [] ? _raw_spin_lock+0xe/0x20 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844174] [] handle_pte_fault+0xfa/0x200 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844177] [] ? xen_pmd_val+0xe/0x10 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844180] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844182] [] handle_mm_fault+0x269/0x370 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844185] [] do_page_fault+0x184/0x550 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844187] [] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844191] [] ? ep_poll+0x1cc/0x380 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844195] [] ? try_to_wake_up+0x200/0x200 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844199] [] ? fget+0x30/0x90 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844201] [] page_fault+0x25/0x30 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844203] Mem-Info: Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844204] Node 0 DMA per-cpu: Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844206] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844207] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844208] Node 0 DMA32 per-cpu: Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844210] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844211] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844214] active_anon:933169 inactive_anon:1393 isolated_anon:0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844214] active_file:10 inactive_file:7 isolated_file:0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844215] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844216] free:4048 slab_reclaimable:2422 slab_unreclaimable:2587 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844216] mapped:1377 shmem:1405 pagetables:3072 bounce:0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844218] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844224] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844227] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732676kB inactive_anon:5572kB active_file:40kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5508kB shmem:5620kB slab_reclaimable:9688kB slab_unreclaimable:10348kB kernel_stack:728kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:12675 all_unreclaimable? yes Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844234] lowmem_reserve[]: 0 0 0 0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844236] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844243] Node 0 DMA32: 741*4kB 172*8kB 44*16kB 23*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7956kB Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844249] 1449 total pagecache pages Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844251] 0 pages in swap cache Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844252] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844253] Free swap = 0kB Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844254] Total swap = 0kB Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849454] 985072 pages RAM Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849457] 24998 pages reserved Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849458] 3775 pages shared Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849459] 953337 pages non-shared Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849461] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849470] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849473] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849476] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849480] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849483] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849486] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849489] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849493] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849496] [ 790] 101 790 63477 631 1 0 0 rsyslogd Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849500] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849503] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849506] [ 975] 0 975 6276 79 0 0 0 master Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849509] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849512] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849515] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849518] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849522] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849525] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849528] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849531] [ 654] 1012 654 70351 5760 0 0 0 php Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849534] [ 657] 1012 657 69242 6257 0 0 0 php Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849537] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849540] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849543] [13468] 0 13468 89584 2526 1 0 0 php5-fpm Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849546] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849549] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849553] [ 2686] 1012 2686 68618 5660 1 0 0 php Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849556] [ 2077] 108 2077 6792 70 1 0 0 pickup Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849560] [ 3130] 1012 3130 892307 829200 0 0 0 php Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849562] [ 3712] 1012 3712 68580 5604 1 0 0 php Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849565] Out of memory: Kill process 3130 (php) score 865 or sacrifice child Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849576] Killed process 3130 (php) total-vm:3569228kB, anon-rss:3316796kB, file-rss:4kB Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404148] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404153] php cpuset=/ mems_allowed=0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404155] Pid: 3884, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404157] Call Trace: Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404164] [] dump_header+0x91/0xe0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404166] [] oom_kill_process+0x85/0xb0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404168] [] out_of_memory+0xfa/0x220 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404171] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404174] [] alloc_pages_current+0xb6/0x120 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404178] [] __page_cache_alloc+0xb7/0xd0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404180] [] filemap_fault+0x212/0x3c0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404183] [] __do_fault+0x72/0x550 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404186] [] ? lru_cache_add_lru+0x2d/0x50 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404188] [] handle_pte_fault+0xfa/0x200 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404191] [] ? xen_pmd_val+0xe/0x10 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404194] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404196] [] handle_mm_fault+0x269/0x370 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404199] [] do_page_fault+0x184/0x550 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404202] [] ? do_brk+0x21f/0x320 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404204] [] ? xen_clts+0x8d/0x190 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404207] [] ? __schedule+0x3cc/0x6f0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404210] [] page_fault+0x25/0x30 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404211] Mem-Info: Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404212] Node 0 DMA per-cpu: Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404214] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404215] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404217] Node 0 DMA32 per-cpu: Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404218] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404219] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404222] active_anon:933160 inactive_anon:1393 isolated_anon:0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404223] active_file:0 inactive_file:0 isolated_file:0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404223] unevictable:0 dirty:0 writeback:37 unstable:0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404224] free:4048 slab_reclaimable:2503 slab_unreclaimable:2587 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404225] mapped:1333 shmem:1405 pagetables:3074 bounce:0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404226] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404233] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404236] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732640kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:148kB mapped:5332kB shmem:5620kB slab_reclaimable:10012kB slab_unreclaimable:10348kB kernel_stack:736kB pagetables:12296kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404242] lowmem_reserve[]: 0 0 0 0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404245] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404252] Node 0 DMA32: 1057*4kB 195*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7884kB Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404258] 1436 total pagecache pages Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404260] 0 pages in swap cache Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404261] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404262] Free swap = 0kB Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404263] Total swap = 0kB Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409418] 985072 pages RAM Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409421] 24998 pages reserved Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409422] 3790 pages shared Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409423] 953339 pages non-shared Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409425] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409433] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409437] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409440] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409443] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409446] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409450] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409453] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409456] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409461] [ 790] 101 790 63477 631 1 0 0 rsyslogd Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409464] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409467] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409470] [ 975] 0 975 6276 79 0 0 0 master Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409473] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409476] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409479] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409482] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409486] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409489] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409492] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409495] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409498] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409501] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409505] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409508] [13468] 0 13468 89584 2511 1 0 0 php5-fpm Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409511] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409514] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409517] [ 2686] 1012 2686 68618 5660 0 0 0 php Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409521] [ 2077] 108 2077 6792 70 0 0 0 pickup Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409524] [ 3884] 1012 3884 892371 829200 1 0 0 php Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409527] [ 5195] 1012 5195 68580 5605 1 0 0 php Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409530] Out of memory: Kill process 3884 (php) score 865 or sacrifice child Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409542] Killed process 3884 (php) total-vm:3569484kB, anon-rss:3316800kB, file-rss:0kB Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817823] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817826] php cpuset=/ mems_allowed=0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817829] Pid: 5352, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817831] Call Trace: Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817838] [] dump_header+0x91/0xe0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817840] [] oom_kill_process+0x85/0xb0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817842] [] out_of_memory+0xfa/0x220 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817846] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817849] [] alloc_pages_current+0xb6/0x120 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817852] [] __page_cache_alloc+0xb7/0xd0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817854] [] filemap_fault+0x212/0x3c0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817857] [] __do_fault+0x72/0x550 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817860] [] ? lru_cache_add_lru+0x2d/0x50 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817863] [] handle_pte_fault+0xfa/0x200 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817866] [] ? xen_pmd_val+0xe/0x10 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817868] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817871] [] handle_mm_fault+0x269/0x370 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817873] [] ? handle_mm_fault+0x1/0x370 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817876] [] do_page_fault+0x184/0x550 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817879] [] ? do_brk+0x21f/0x320 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817882] [] ? __schedule+0x6b1/0x6f0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817884] [] page_fault+0x25/0x30 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817886] Mem-Info: Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817887] Node 0 DMA per-cpu: Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817889] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817890] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817891] Node 0 DMA32 per-cpu: Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817893] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817894] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817897] active_anon:933229 inactive_anon:1393 isolated_anon:0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817898] active_file:15 inactive_file:10 isolated_file:0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817898] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817899] free:4048 slab_reclaimable:2468 slab_unreclaimable:2541 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817900] mapped:1371 shmem:1405 pagetables:3076 bounce:0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817901] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817908] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817911] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732916kB inactive_anon:5572kB active_file:60kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5484kB shmem:5620kB slab_reclaimable:9872kB slab_unreclaimable:10164kB kernel_stack:728kB pagetables:12304kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:157 all_unreclaimable? yes Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817917] lowmem_reserve[]: 0 0 0 0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817920] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817926] Node 0 DMA32: 799*4kB 170*8kB 63*16kB 11*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7964kB Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817933] 1441 total pagecache pages Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817934] 0 pages in swap cache Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817935] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817936] Free swap = 0kB Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817937] Total swap = 0kB Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823149] 985072 pages RAM Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823151] 24998 pages reserved Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823152] 3773 pages shared Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823154] 953337 pages non-shared Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823155] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823164] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823168] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823171] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823174] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823177] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823180] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823184] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823187] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823191] [ 790] 101 790 63477 631 0 0 0 rsyslogd Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823194] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823197] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823200] [ 975] 0 975 6276 79 0 0 0 master Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823203] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823207] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823210] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823213] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823216] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823219] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823222] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823226] [ 654] 1012 654 70351 5761 1 0 0 php Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823229] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823232] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823235] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823238] [13468] 0 13468 89584 2496 1 0 0 php5-fpm Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823241] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823244] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823248] [ 2686] 1012 2686 68618 5661 1 0 0 php Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823251] [ 2077] 108 2077 6792 70 0 0 0 pickup Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823254] [ 5352] 1012 5352 892371 829235 0 0 0 php Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823258] [12198] 1012 12198 68580 5606 1 0 0 php Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823260] Out of memory: Kill process 5352 (php) score 865 or sacrifice child Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823272] Killed process 5352 (php) total-vm:3569484kB, anon-rss:3316940kB, file-rss:0kB Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916135] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916139] php cpuset=/ mems_allowed=0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916143] Pid: 12322, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916145] Call Trace: Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916153] [] dump_header+0x91/0xe0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916156] [] oom_kill_process+0x85/0xb0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916160] [] out_of_memory+0xfa/0x220 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916163] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916167] [] alloc_pages_current+0xb6/0x120 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916172] [] __page_cache_alloc+0xb7/0xd0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916175] [] filemap_fault+0x212/0x3c0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916179] [] __do_fault+0x72/0x550 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916184] [] ? error_exit+0x2a/0x60 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916187] [] ? retint_restore_args+0x5/0x6 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916190] [] handle_pte_fault+0xfa/0x200 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916194] [] ? xen_pmd_val+0xe/0x10 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916196] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916198] [] handle_mm_fault+0x269/0x370 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916201] [] do_page_fault+0x184/0x550 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916204] [] ? do_brk+0x21f/0x320 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916206] [] ? xen_clts+0x8d/0x190 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916208] [] ? __schedule+0x3cc/0x6f0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916211] [] page_fault+0x25/0x30 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916213] Mem-Info: Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916214] Node 0 DMA per-cpu: Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916216] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916217] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916218] Node 0 DMA32 per-cpu: Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916220] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916221] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916224] active_anon:932617 inactive_anon:1393 isolated_anon:0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916224] active_file:0 inactive_file:0 isolated_file:0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916225] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916226] free:4031 slab_reclaimable:2346 slab_unreclaimable:2617 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916226] mapped:1366 shmem:1405 pagetables:3553 bounce:0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916228] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916235] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916237] Node 0 DMA32 free:7788kB min:7848kB low:9808kB high:11772kB active_anon:3730468kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:9384kB slab_unreclaimable:10468kB kernel_stack:840kB pagetables:14212kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916244] lowmem_reserve[]: 0 0 0 0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916246] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916253] Node 0 DMA32: 627*4kB 201*8kB 47*16kB 6*32kB 1*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 7812kB Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916260] 1471 total pagecache pages Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916261] 0 pages in swap cache Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916262] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916263] Free swap = 0kB Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916264] Total swap = 0kB Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921526] 985072 pages RAM Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921528] 24998 pages reserved Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921529] 3789 pages shared Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921530] 953345 pages non-shared Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921532] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921541] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921545] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921548] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921551] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921554] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921557] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921561] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921564] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921568] [ 790] 101 790 63477 631 0 0 0 rsyslogd Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921571] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921574] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921577] [ 975] 0 975 6276 79 0 0 0 master Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921580] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921584] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921587] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921591] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921594] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921598] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921601] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921604] [ 654] 1012 654 70351 5761 1 0 0 php Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921607] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921610] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921613] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921616] [13468] 0 13468 89584 2531 1 0 0 php5-fpm Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921619] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921622] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921625] [ 2686] 1012 2686 68618 5660 1 0 0 php Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921629] [ 2077] 108 2077 6792 70 0 0 0 pickup Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921632] [12322] 1012 12322 868370 805176 0 0 0 php Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921635] [12428] 1012 12428 68580 5605 1 0 0 php Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921638] [12570] 1012 12570 1099 24 1 0 0 sh Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921641] [12571] 1012 12571 1099 25 1 0 0 sh Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921645] [12572] 1012 12572 68825 5870 1 0 0 php Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921648] [12573] 1012 12573 68825 5872 0 0 0 php Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921651] [12584] 1012 12584 1099 26 1 0 0 sh Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921654] [12585] 1012 12585 1099 25 1 0 0 sh Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921657] [12586] 1012 12586 4675 60 1 0 0 git Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921660] [12587] 1012 12587 4676 58 1 0 0 git Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921663] [12588] 1012 12588 68499 5552 1 0 0 php Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921666] [12589] 1012 12589 68499 5551 0 0 0 php Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921669] [12592] 1012 12592 1099 25 1 0 0 sh Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921672] [12593] 1012 12593 10901 176 1 0 0 ssh Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921675] [12594] 1012 12594 1099 24 0 0 0 sh Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921678] [12595] 1012 12595 10901 175 1 0 0 ssh Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921680] Out of memory: Kill process 12322 (php) score 840 or sacrifice child Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921692] Killed process 12322 (php) total-vm:3473480kB, anon-rss:3220704kB, file-rss:0kB Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271343] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271347] php5-fpm cpuset=/ mems_allowed=0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271350] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271351] Call Trace: Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271358] [] dump_header+0x91/0xe0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271361] [] oom_kill_process+0x85/0xb0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271363] [] out_of_memory+0xfa/0x220 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271366] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271369] [] alloc_pages_current+0xb6/0x120 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271373] [] __page_cache_alloc+0xb7/0xd0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271375] [] filemap_fault+0x212/0x3c0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271378] [] __do_fault+0x72/0x550 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271382] [] ? __switch_to+0x138/0x360 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271384] [] handle_pte_fault+0xfa/0x200 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271387] [] ? xen_pmd_val+0xe/0x10 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271390] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271392] [] handle_mm_fault+0x269/0x370 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271395] [] do_page_fault+0x184/0x550 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271398] [] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271403] [] ? ep_poll+0x1cc/0x380 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271407] [] ? try_to_wake_up+0x200/0x200 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271410] [] page_fault+0x25/0x30 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271411] Mem-Info: Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271413] Node 0 DMA per-cpu: Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271414] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271416] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271417] Node 0 DMA32 per-cpu: Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271419] CPU 0: hi: 186, btch: 31 usd: 83 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271420] CPU 1: hi: 186, btch: 31 usd: 14 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271423] active_anon:933249 inactive_anon:1393 isolated_anon:0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271423] active_file:24 inactive_file:22 isolated_file:8 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271424] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271425] free:4029 slab_reclaimable:2361 slab_unreclaimable:2549 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271425] mapped:1356 shmem:1405 pagetables:3075 bounce:0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271427] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271434] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271437] Node 0 DMA32 free:7780kB min:7848kB low:9808kB high:11772kB active_anon:3732996kB inactive_anon:5572kB active_file:96kB inactive_file:88kB unevictable:0kB isolated(anon):0kB isolated(file):32kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:9444kB slab_unreclaimable:10196kB kernel_stack:728kB pagetables:12300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:411 all_unreclaimable? yes Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271443] lowmem_reserve[]: 0 0 0 0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271446] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271452] Node 0 DMA32: 455*4kB 97*8kB 146*16kB 21*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7780kB Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271459] 1430 total pagecache pages Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271460] 0 pages in swap cache Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271462] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271463] Free swap = 0kB Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271464] Total swap = 0kB Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276632] 985072 pages RAM Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276634] 24998 pages reserved Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276635] 3779 pages shared Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276636] 953280 pages non-shared Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276638] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276647] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276651] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276654] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276657] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276660] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276663] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276667] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276670] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276674] [ 790] 101 790 63477 631 0 0 0 rsyslogd Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276677] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276680] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276684] [ 975] 0 975 6276 79 1 0 0 master Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276687] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276690] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276693] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276696] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276700] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276703] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276706] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276709] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276712] [ 657] 1012 657 69242 6257 0 0 0 php Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276715] [ 658] 1012 658 134393 70798 1 0 0 php Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276718] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276721] [13468] 0 13468 89584 2527 0 0 0 php5-fpm Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276724] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276727] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276731] [ 2686] 1012 2686 68618 5660 1 0 0 php Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276734] [ 2077] 108 2077 6792 70 0 0 0 pickup Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276738] [12691] 1012 12691 892435 829298 1 0 0 php Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276741] [12693] 1012 12693 68580 5604 0 0 0 php Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276744] Out of memory: Kill process 12691 (php) score 865 or sacrifice child Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276755] Killed process 12691 (php) total-vm:3569740kB, anon-rss:3317192kB, file-rss:0kB Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504129] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504133] php cpuset=/ mems_allowed=0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504136] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504138] Call Trace: Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504145] [] dump_header+0x91/0xe0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504147] [] oom_kill_process+0x85/0xb0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504149] [] out_of_memory+0xfa/0x220 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504152] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504157] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504160] [] alloc_pages_current+0xb6/0x120 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504164] [] __page_cache_alloc+0xb7/0xd0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504166] [] filemap_fault+0x212/0x3c0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504169] [] __do_fault+0x72/0x550 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504171] [] handle_pte_fault+0xfa/0x200 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504175] [] ? xen_pmd_val+0xe/0x10 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504177] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504179] [] handle_mm_fault+0x269/0x370 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504182] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504184] [] ? check_events+0x12/0x20 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504188] [] do_page_fault+0x184/0x550 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504191] [] ? ktime_get_ts+0xad/0xe0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504195] [] ? poll_select_copy_remaining+0xed/0x140 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504197] [] ? sys_select+0xcd/0x100 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504200] [] page_fault+0x25/0x30 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504202] Mem-Info: Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504203] Node 0 DMA per-cpu: Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504205] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504206] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504207] Node 0 DMA32 per-cpu: Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504209] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504210] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504212] active_anon:932892 inactive_anon:1393 isolated_anon:0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504213] active_file:0 inactive_file:0 isolated_file:0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504214] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504214] free:4048 slab_reclaimable:2685 slab_unreclaimable:2565 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504215] mapped:1345 shmem:1405 pagetables:3084 bounce:0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504217] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504223] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504226] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731568kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5380kB shmem:5620kB slab_reclaimable:10740kB slab_unreclaimable:10260kB kernel_stack:728kB pagetables:12336kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504233] lowmem_reserve[]: 0 0 0 0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504235] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504242] Node 0 DMA32: 929*4kB 251*8kB 10*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7932kB Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504249] 1486 total pagecache pages Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504250] 0 pages in swap cache Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504251] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504252] Free swap = 0kB Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504253] Total swap = 0kB Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509337] 985072 pages RAM Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509340] 24998 pages reserved Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509341] 3817 pages shared Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509342] 953303 pages non-shared Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509344] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509352] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509356] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509359] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509362] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509365] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509368] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509372] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509375] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509379] [ 790] 101 790 63477 631 0 0 0 rsyslogd Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509382] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509385] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509388] [ 975] 0 975 6276 80 0 0 0 master Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509392] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509395] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509398] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509401] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509404] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509407] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509410] [15657] 0 15657 4777 59 1 0 0 cron Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509414] [ 654] 1012 654 70351 5769 0 0 0 php Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509417] [ 657] 1012 657 69242 6258 0 0 0 php Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509420] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509423] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509426] [13468] 0 13468 89584 2535 0 0 0 php5-fpm Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509429] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509432] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509435] [ 2686] 1012 2686 68618 5662 0 0 0 php Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509439] [20084] 108 20084 6792 70 0 0 0 pickup Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509442] [ 462] 1012 462 892051 828964 1 0 0 php Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509445] [ 951] 1012 951 68580 5604 1 0 0 php Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509448] Out of memory: Kill process 462 (php) score 865 or sacrifice child Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509459] Killed process 462 (php) total-vm:3568204kB, anon-rss:3315856kB, file-rss:0kB Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628387] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628392] php cpuset=/ mems_allowed=0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628396] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628406] Call Trace: Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628414] [] dump_header+0x91/0xe0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628418] [] oom_kill_process+0x85/0xb0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628421] [] out_of_memory+0xfa/0x220 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628424] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628430] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628434] [] alloc_pages_current+0xb6/0x120 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628438] [] __page_cache_alloc+0xb7/0xd0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628441] [] filemap_fault+0x212/0x3c0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628445] [] __do_fault+0x72/0x550 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628448] [] handle_pte_fault+0xfa/0x200 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628452] [] ? xen_pmd_val+0xe/0x10 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628455] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628458] [] handle_mm_fault+0x269/0x370 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628463] [] do_page_fault+0x184/0x550 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628467] [] ? security_file_permission+0x93/0xb0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628470] [] ? rw_verify_area+0x4b/0xf0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628472] [] ? rw_verify_area+0x61/0xf0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628474] [] ? vfs_read+0xb0/0x180 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628477] [] page_fault+0x25/0x30 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628478] Mem-Info: Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628480] Node 0 DMA per-cpu: Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628482] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628483] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628484] Node 0 DMA32 per-cpu: Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628486] CPU 0: hi: 186, btch: 31 usd: 30 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628487] CPU 1: hi: 186, btch: 31 usd: 33 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628490] active_anon:933101 inactive_anon:1393 isolated_anon:0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628490] active_file:6 inactive_file:25 isolated_file:0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628491] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628492] free:4031 slab_reclaimable:2502 slab_unreclaimable:2574 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628492] mapped:1366 shmem:1405 pagetables:3082 bounce:0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628494] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628500] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628503] Node 0 DMA32 free:7788kB min:7848kB low:9808kB high:11772kB active_anon:3732404kB inactive_anon:5572kB active_file:24kB inactive_file:100kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:10008kB slab_unreclaimable:10296kB kernel_stack:728kB pagetables:12328kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:216 all_unreclaimable? yes Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628510] lowmem_reserve[]: 0 0 0 0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628513] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628519] Node 0 DMA32: 985*4kB 207*8kB 9*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7820kB Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628526] 1424 total pagecache pages Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628527] 0 pages in swap cache Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628528] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628530] Free swap = 0kB Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628530] Total swap = 0kB Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633725] 985072 pages RAM Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633728] 24998 pages reserved Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633729] 3783 pages shared Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633730] 953300 pages non-shared Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633733] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633741] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633744] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633748] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633751] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633754] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633757] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633760] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633764] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633768] [ 790] 101 790 63477 631 0 0 0 rsyslogd Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633771] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633774] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633777] [ 975] 0 975 6276 79 0 0 0 master Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633780] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633783] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633786] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633789] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633793] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633796] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633799] [15657] 0 15657 4777 60 0 0 0 cron Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633802] [ 654] 1012 654 70351 5760 0 0 0 php Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633805] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633808] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633811] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633814] [13468] 0 13468 89584 2528 1 0 0 php5-fpm Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633817] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633820] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633823] [ 2686] 1012 2686 68618 5662 1 0 0 php Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633827] [20084] 108 20084 6792 76 1 0 0 pickup Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633831] [ 1153] 1012 1153 892264 829168 1 0 0 php Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633834] [ 9004] 1012 9004 68580 5604 1 0 0 php Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633837] Out of memory: Kill process 1153 (php) score 865 or sacrifice child Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633847] Killed process 1153 (php) total-vm:3569056kB, anon-rss:3316672kB, file-rss:0kB Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444106] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444110] php cpuset=/ mems_allowed=0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444112] Pid: 9148, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444114] Call Trace: Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444121] [] dump_header+0x91/0xe0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444124] [] oom_kill_process+0x85/0xb0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444126] [] out_of_memory+0xfa/0x220 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444129] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444132] [] alloc_pages_current+0xb6/0x120 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444135] [] __page_cache_alloc+0xb7/0xd0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444137] [] filemap_fault+0x212/0x3c0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444141] [] __do_fault+0x72/0x550 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444143] [] handle_pte_fault+0xfa/0x200 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444146] [] ? xen_pmd_val+0xe/0x10 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444149] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444151] [] handle_mm_fault+0x269/0x370 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444154] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444156] [] ? check_events+0x12/0x20 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444160] [] do_page_fault+0x184/0x550 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444162] [] ? do_brk+0x21f/0x320 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444164] [] ? xen_clts+0x8d/0x190 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444167] [] ? __schedule+0x3cc/0x6f0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444170] [] page_fault+0x25/0x30 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444172] Mem-Info: Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444173] Node 0 DMA per-cpu: Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444175] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444176] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444177] Node 0 DMA32 per-cpu: Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444179] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444180] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444183] active_anon:933238 inactive_anon:1393 isolated_anon:0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444183] active_file:35 inactive_file:35 isolated_file:0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444184] unevictable:0 dirty:9 writeback:0 unstable:0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444185] free:4048 slab_reclaimable:2466 slab_unreclaimable:2560 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444185] mapped:1371 shmem:1405 pagetables:3063 bounce:0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444187] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444193] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444196] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732952kB inactive_anon:5572kB active_file:140kB inactive_file:140kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:36kB writeback:0kB mapped:5484kB shmem:5620kB slab_reclaimable:9864kB slab_unreclaimable:10240kB kernel_stack:728kB pagetables:12252kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:7267 all_unreclaimable? yes Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444203] lowmem_reserve[]: 0 0 0 0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444205] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444212] Node 0 DMA32: 964*4kB 230*8kB 7*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444219] 1443 total pagecache pages Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444220] 0 pages in swap cache Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444221] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444222] Free swap = 0kB Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444223] Total swap = 0kB Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449577] 985072 pages RAM Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449580] 24998 pages reserved Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449581] 3784 pages shared Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449582] 953329 pages non-shared Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449584] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449593] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449597] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449600] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449603] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449607] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449610] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449613] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449617] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449621] [ 790] 101 790 63477 631 0 0 0 rsyslogd Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449624] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449627] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449630] [ 975] 0 975 6276 79 0 0 0 master Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449633] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449637] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449640] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449643] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449646] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449649] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449652] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449655] [ 654] 1012 654 70351 5760 0 0 0 php Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449658] [ 657] 1012 657 69242 6258 0 0 0 php Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449662] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449665] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449668] [13468] 0 13468 89584 2513 0 0 0 php5-fpm Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449671] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449674] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449677] [ 2686] 1012 2686 68618 5661 1 0 0 php Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449681] [20084] 108 20084 6792 76 0 0 0 pickup Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449684] [ 9148] 1012 9148 892435 829253 0 0 0 php Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449687] [10439] 1012 10439 68580 5604 0 0 0 php Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449689] Out of memory: Kill process 9148 (php) score 865 or sacrifice child Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449701] Killed process 9148 (php) total-vm:3569740kB, anon-rss:3317012kB, file-rss:0kB Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780128] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780133] php cpuset=/ mems_allowed=0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780135] Pid: 10533, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780137] Call Trace: Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780144] [] dump_header+0x91/0xe0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780146] [] oom_kill_process+0x85/0xb0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780148] [] out_of_memory+0xfa/0x220 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780152] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780155] [] alloc_pages_current+0xb6/0x120 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780158] [] __page_cache_alloc+0xb7/0xd0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780160] [] filemap_fault+0x212/0x3c0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780163] [] __do_fault+0x72/0x550 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780166] [] ? lru_cache_add_lru+0x2d/0x50 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780169] [] handle_pte_fault+0xfa/0x200 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780172] [] ? xen_pmd_val+0xe/0x10 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780174] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780177] [] handle_mm_fault+0x269/0x370 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780180] [] do_page_fault+0x184/0x550 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780182] [] ? do_brk+0x21f/0x320 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780184] [] ? xen_clts+0x8d/0x190 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780187] [] page_fault+0x25/0x30 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780189] Mem-Info: Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780191] Node 0 DMA per-cpu: Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780192] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780194] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780195] Node 0 DMA32 per-cpu: Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780196] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780198] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780200] active_anon:933306 inactive_anon:1393 isolated_anon:0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780201] active_file:0 inactive_file:0 isolated_file:0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780202] unevictable:0 dirty:20 writeback:0 unstable:0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780202] free:4048 slab_reclaimable:2369 slab_unreclaimable:2566 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780203] mapped:1368 shmem:1405 pagetables:3057 bounce:0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780205] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780211] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780214] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3733224kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:80kB writeback:0kB mapped:5472kB shmem:5620kB slab_reclaimable:9476kB slab_unreclaimable:10264kB kernel_stack:760kB pagetables:12228kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780221] lowmem_reserve[]: 0 0 0 0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780223] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780230] Node 0 DMA32: 645*4kB 116*8kB 84*16kB 24*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7860kB Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780237] 1437 total pagecache pages Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780238] 0 pages in swap cache Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780239] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780240] Free swap = 0kB Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780241] Total swap = 0kB Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785487] 985072 pages RAM Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785489] 24998 pages reserved Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785490] 3803 pages shared Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785491] 953335 pages non-shared Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785493] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785502] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785505] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785508] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785511] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785514] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785518] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785521] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785524] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785528] [ 790] 101 790 63477 631 0 0 0 rsyslogd Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785531] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785534] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785538] [ 975] 0 975 6276 79 0 0 0 master Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785541] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785544] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785547] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785550] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785553] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785556] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785560] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785563] [ 654] 1012 654 70351 5761 1 0 0 php Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785566] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785569] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785572] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785575] [13468] 0 13468 89584 2498 1 0 0 php5-fpm Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785578] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785581] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785585] [ 2686] 1012 2686 68618 5661 1 0 0 php Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785588] [20084] 108 20084 6792 76 0 0 0 pickup Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785591] [10533] 1012 10533 892499 829351 0 0 0 php Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785594] [10686] 1012 10686 68580 5606 1 0 0 php Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785597] Out of memory: Kill process 10533 (php) score 865 or sacrifice child Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785608] Killed process 10533 (php) total-vm:3569996kB, anon-rss:3317404kB, file-rss:0kB Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904122] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904127] php cpuset=/ mems_allowed=0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904130] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904132] Call Trace: Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904140] [] dump_header+0x91/0xe0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904142] [] oom_kill_process+0x85/0xb0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904144] [] out_of_memory+0xfa/0x220 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904147] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904152] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904155] [] alloc_pages_current+0xb6/0x120 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904159] [] __page_cache_alloc+0xb7/0xd0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904161] [] filemap_fault+0x212/0x3c0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904164] [] __do_fault+0x72/0x550 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904167] [] ? load_TLS_descriptor+0x42/0xd0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904169] [] handle_pte_fault+0xfa/0x200 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904172] [] ? xen_pmd_val+0xe/0x10 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904174] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904176] [] handle_mm_fault+0x269/0x370 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904181] [] ? lock_hrtimer_base.isra.25+0x30/0x60 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904184] [] do_page_fault+0x184/0x550 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904187] [] ? do_nanosleep+0xa4/0xd0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904190] [] ? hrtimer_nanosleep+0xb8/0x180 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904193] [] ? update_rmtp+0x70/0x70 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904195] [] ? hrtimer_start_range_ns+0x14/0x20 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904198] [] page_fault+0x25/0x30 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904200] Mem-Info: Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904201] Node 0 DMA per-cpu: Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904203] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904204] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904205] Node 0 DMA32 per-cpu: Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904207] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904208] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904210] active_anon:932999 inactive_anon:1393 isolated_anon:0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904211] active_file:0 inactive_file:7 isolated_file:0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904212] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904213] free:4048 slab_reclaimable:2498 slab_unreclaimable:2582 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904213] mapped:1354 shmem:1405 pagetables:3168 bounce:0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904215] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904221] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904224] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731996kB inactive_anon:5572kB active_file:0kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5620kB slab_reclaimable:9992kB slab_unreclaimable:10328kB kernel_stack:744kB pagetables:12672kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:697 all_unreclaimable? yes Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904231] lowmem_reserve[]: 0 0 0 0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904233] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904240] Node 0 DMA32: 692*4kB 114*8kB 87*16kB 26*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904247] 1426 total pagecache pages Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904248] 0 pages in swap cache Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904249] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904251] Free swap = 0kB Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904251] Total swap = 0kB Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909464] 985072 pages RAM Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909466] 24998 pages reserved Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909467] 3784 pages shared Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909468] 953331 pages non-shared Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909470] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909479] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909482] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909486] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909489] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909492] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909495] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909498] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909502] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909506] [ 790] 101 790 63477 631 1 0 0 rsyslogd Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909509] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909512] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909515] [ 975] 0 975 6276 79 1 0 0 master Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909518] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909521] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909525] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909528] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909531] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909534] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909537] [15657] 0 15657 4777 59 1 0 0 cron Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909540] [ 654] 1012 654 70351 5761 0 0 0 php Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909543] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909546] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909549] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909552] [13468] 0 13468 89584 2516 0 0 0 php5-fpm Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909555] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909558] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909562] [ 2686] 1012 2686 68618 5662 0 0 0 php Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909565] [20084] 108 20084 6792 76 0 0 0 pickup Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909568] [10810] 1012 10810 889364 826282 0 0 0 php Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909572] [20060] 1012 20060 68580 5604 1 0 0 php Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909575] [20154] 1012 20154 1099 25 1 0 0 sh Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909578] [20155] 1012 20155 65183 2725 1 0 0 php Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909581] Out of memory: Kill process 10810 (php) score 862 or sacrifice child Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909592] Killed process 10810 (php) total-vm:3557456kB, anon-rss:3305128kB, file-rss:0kB Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016129] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016134] php cpuset=/ mems_allowed=0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016137] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016139] Call Trace: Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016147] [] dump_header+0x91/0xe0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016151] [] oom_kill_process+0x85/0xb0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016154] [] out_of_memory+0xfa/0x220 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016157] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016163] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016166] [] alloc_pages_current+0xb6/0x120 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016170] [] __page_cache_alloc+0xb7/0xd0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016172] [] filemap_fault+0x212/0x3c0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016175] [] __do_fault+0x72/0x550 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016179] [] ? error_exit+0x2a/0x60 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016182] [] ? retint_restore_args+0x5/0x6 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016185] [] ? pte_mfn_to_pfn+0x81/0x110 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016187] [] handle_pte_fault+0xfa/0x200 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016189] [] ? xen_pmd_val+0xe/0x10 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016192] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016194] [] handle_mm_fault+0x269/0x370 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016197] [] do_page_fault+0x184/0x550 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016201] [] ? ktime_get_ts+0xad/0xe0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016204] [] ? poll_select_copy_remaining+0xed/0x140 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016207] [] ? sys_select+0xcd/0x100 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016209] [] page_fault+0x25/0x30 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016211] Mem-Info: Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016212] Node 0 DMA per-cpu: Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016214] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016215] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016216] Node 0 DMA32 per-cpu: Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016218] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016219] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016222] active_anon:933256 inactive_anon:1393 isolated_anon:0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016223] active_file:0 inactive_file:0 isolated_file:0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016223] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016224] free:4048 slab_reclaimable:2391 slab_unreclaimable:2569 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016225] mapped:1369 shmem:1405 pagetables:3077 bounce:0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016226] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016233] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016236] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3733024kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5476kB shmem:5620kB slab_reclaimable:9564kB slab_unreclaimable:10276kB kernel_stack:728kB pagetables:12308kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10 all_unreclaimable? yes Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016242] lowmem_reserve[]: 0 0 0 0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016245] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016252] Node 0 DMA32: 694*4kB 191*8kB 58*16kB 14*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016258] 1440 total pagecache pages Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016260] 0 pages in swap cache Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016261] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016262] Free swap = 0kB Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016263] Total swap = 0kB Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021434] 985072 pages RAM Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021437] 24998 pages reserved Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021438] 3795 pages shared Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021439] 953334 pages non-shared Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021441] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021449] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021453] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021456] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021459] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021462] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021466] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021469] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021472] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021477] [ 790] 101 790 63477 631 1 0 0 rsyslogd Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021480] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021483] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021486] [ 975] 0 975 6276 79 1 0 0 master Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021489] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021492] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021495] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021498] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021501] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021505] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021508] [15657] 0 15657 4777 59 1 0 0 cron Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021511] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021514] [ 657] 1012 657 69242 6257 0 0 0 php Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021517] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021520] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021523] [13468] 0 13468 89584 2535 0 0 0 php5-fpm Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021526] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021529] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021532] [ 2686] 1012 2686 68618 5661 1 0 0 php Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021535] [20084] 108 20084 6792 76 0 0 0 pickup Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021539] [20218] 1012 20218 892435 829287 0 0 0 php Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021542] [20388] 1012 20388 68580 5604 0 0 0 php Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021545] Out of memory: Kill process 20218 (php) score 865 or sacrifice child Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021555] Killed process 20218 (php) total-vm:3569740kB, anon-rss:3317148kB, file-rss:0kB Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664109] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664113] php cpuset=/ mems_allowed=0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664116] Pid: 20594, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664118] Call Trace: Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664125] [] dump_header+0x91/0xe0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664127] [] oom_kill_process+0x85/0xb0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664129] [] out_of_memory+0xfa/0x220 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664132] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664137] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664140] [] alloc_pages_current+0xb6/0x120 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664144] [] __page_cache_alloc+0xb7/0xd0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664146] [] filemap_fault+0x212/0x3c0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664149] [] __do_fault+0x72/0x550 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664153] [] ? error_exit+0x2a/0x60 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664155] [] ? retint_restore_args+0x5/0x6 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664157] [] ? __do_fault+0x1/0x550 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664160] [] handle_pte_fault+0xfa/0x200 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664163] [] ? xen_pmd_val+0xe/0x10 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664165] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664168] [] handle_mm_fault+0x269/0x370 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664171] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664173] [] ? check_events+0x12/0x20 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664175] [] do_page_fault+0x184/0x550 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664177] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664179] [] ? xen_clts+0x8d/0x190 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664182] [] ? __schedule+0x3cc/0x6f0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664185] [] page_fault+0x25/0x30 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664186] Mem-Info: Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664188] Node 0 DMA per-cpu: Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664189] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664191] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664192] Node 0 DMA32 per-cpu: Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664194] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664195] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664197] active_anon:933084 inactive_anon:1393 isolated_anon:0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664198] active_file:0 inactive_file:0 isolated_file:0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664199] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664199] free:4048 slab_reclaimable:2557 slab_unreclaimable:2565 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664200] mapped:1370 shmem:1405 pagetables:3056 bounce:0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664201] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664208] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664211] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732336kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5480kB shmem:5620kB slab_reclaimable:10228kB slab_unreclaimable:10260kB kernel_stack:728kB pagetables:12224kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664217] lowmem_reserve[]: 0 0 0 0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664220] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664227] Node 0 DMA32: 721*4kB 244*8kB 54*16kB 8*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8004kB Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664233] 1474 total pagecache pages Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664235] 0 pages in swap cache Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664236] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664237] Free swap = 0kB Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664238] Total swap = 0kB Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669423] 985072 pages RAM Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669426] 24998 pages reserved Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669428] 3792 pages shared Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669429] 953299 pages non-shared Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669431] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669439] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669443] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669446] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669449] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669452] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669456] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669459] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669462] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669466] [ 790] 101 790 63477 631 0 0 0 rsyslogd Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669470] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669473] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669476] [ 975] 0 975 6276 80 0 0 0 master Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669479] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669482] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669485] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669488] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669491] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669494] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669498] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669501] [ 654] 1012 654 70351 5760 0 0 0 php Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669504] [ 657] 1012 657 69242 6257 0 0 0 php Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669507] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669510] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669513] [13468] 0 13468 89584 2507 0 0 0 php5-fpm Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669516] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669519] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669522] [ 2686] 1012 2686 68618 5661 0 0 0 php Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669526] [20594] 1012 20594 892331 829148 0 0 0 php Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669529] [20831] 108 20831 6792 70 0 0 0 pickup Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669532] [21604] 1012 21604 68580 5605 1 0 0 php Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669534] Out of memory: Kill process 20594 (php) score 865 or sacrifice child Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669546] Killed process 20594 (php) total-vm:3569324kB, anon-rss:3316592kB, file-rss:0kB Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904121] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904126] php5-fpm cpuset=/ mems_allowed=0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904130] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904132] Call Trace: Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904140] [] dump_header+0x91/0xe0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904143] [] oom_kill_process+0x85/0xb0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904146] [] out_of_memory+0xfa/0x220 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904150] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904156] [] ? noalloc_get_block_write+0x30/0x30 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904160] [] alloc_pages_current+0xb6/0x120 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904164] [] __page_cache_alloc+0xb7/0xd0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904167] [] filemap_fault+0x212/0x3c0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904171] [] __do_fault+0x72/0x550 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904175] [] ? __switch_to+0x138/0x360 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904179] [] handle_pte_fault+0xfa/0x200 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904182] [] ? xen_pmd_val+0xe/0x10 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904185] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904188] [] handle_mm_fault+0x269/0x370 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904191] [] do_page_fault+0x184/0x550 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904195] [] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904199] [] ? ep_poll+0x1cc/0x380 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904203] [] ? try_to_wake_up+0x200/0x200 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904206] [] page_fault+0x25/0x30 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904207] Mem-Info: Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904209] Node 0 DMA per-cpu: Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904211] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904212] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904213] Node 0 DMA32 per-cpu: Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904215] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904216] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904219] active_anon:933249 inactive_anon:1393 isolated_anon:0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904219] active_file:13 inactive_file:19 isolated_file:0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904220] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904221] free:4047 slab_reclaimable:2452 slab_unreclaimable:2525 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904221] mapped:1366 shmem:1405 pagetables:3070 bounce:0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904223] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904229] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904232] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3732996kB inactive_anon:5572kB active_file:52kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:9808kB slab_unreclaimable:10100kB kernel_stack:728kB pagetables:12280kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2284 all_unreclaimable? yes Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904239] lowmem_reserve[]: 0 0 0 0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904241] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904248] Node 0 DMA32: 825*4kB 239*8kB 35*16kB 3*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7980kB Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904255] 1453 total pagecache pages Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904256] 0 pages in swap cache Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904258] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904259] Free swap = 0kB Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904259] Total swap = 0kB Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909689] 985072 pages RAM Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909692] 24998 pages reserved Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909693] 3787 pages shared Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909694] 953322 pages non-shared Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909696] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909705] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909708] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909712] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909715] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909718] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909721] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909724] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909728] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909732] [ 790] 101 790 63477 631 0 0 0 rsyslogd Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909735] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909738] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909742] [ 975] 0 975 6276 79 0 0 0 master Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909744] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909747] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909751] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909754] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909757] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909760] [ 1242] 107 1242 9445 164 0 0 0 ntpd Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909763] [15657] 0 15657 4777 60 0 0 0 cron Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909766] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909770] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909773] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909776] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909779] [13468] 0 13468 89584 2539 0 0 0 php5-fpm Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909782] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909785] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909788] [ 2686] 1012 2686 68618 5661 0 0 0 php Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909792] [20831] 108 20831 6792 76 0 0 0 pickup Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909795] [21701] 1012 21701 892371 829262 1 0 0 php Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909798] [22850] 1012 22850 68580 5603 0 0 0 php Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909801] Out of memory: Kill process 21701 (php) score 865 or sacrifice child Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909813] Killed process 21701 (php) total-vm:3569484kB, anon-rss:3317048kB, file-rss:0kB Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532129] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532134] php cpuset=/ mems_allowed=0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532138] Pid: 23032, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532140] Call Trace: Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532148] [] dump_header+0x91/0xe0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532151] [] oom_kill_process+0x85/0xb0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532154] [] out_of_memory+0xfa/0x220 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532157] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532162] [] alloc_pages_current+0xb6/0x120 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532169] [] __page_cache_alloc+0xb7/0xd0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532172] [] filemap_fault+0x212/0x3c0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532176] [] __do_fault+0x72/0x550 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532180] [] ? lru_cache_add_lru+0x2d/0x50 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532183] [] handle_pte_fault+0xfa/0x200 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532187] [] ? xen_pmd_val+0xe/0x10 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532190] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532193] [] handle_mm_fault+0x269/0x370 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532197] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532200] [] ? check_events+0x12/0x20 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532204] [] do_page_fault+0x184/0x550 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532207] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532210] [] ? xen_clts+0x8d/0x190 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532213] [] ? __schedule+0x3cc/0x6f0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532217] [] page_fault+0x25/0x30 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532221] Mem-Info: Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532222] Node 0 DMA per-cpu: Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532225] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532227] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532228] Node 0 DMA32 per-cpu: Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532231] CPU 0: hi: 186, btch: 31 usd: 0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532232] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532236] active_anon:933240 inactive_anon:1393 isolated_anon:0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532237] active_file:0 inactive_file:0 isolated_file:0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532238] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532239] free:4048 slab_reclaimable:2408 slab_unreclaimable:2534 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532240] mapped:1348 shmem:1405 pagetables:3071 bounce:0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532242] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532250] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532253] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732960kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5392kB shmem:5620kB slab_reclaimable:9632kB slab_unreclaimable:10136kB kernel_stack:720kB pagetables:12284kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532262] lowmem_reserve[]: 0 0 0 0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532265] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532276] Node 0 DMA32: 803*4kB 189*8kB 49*16kB 4*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7876kB Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532286] 1458 total pagecache pages Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532288] 0 pages in swap cache Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532289] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532291] Free swap = 0kB Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532292] Total swap = 0kB Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539490] 985072 pages RAM Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539493] 24998 pages reserved Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539494] 3787 pages shared Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539495] 953346 pages non-shared Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539496] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539505] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539509] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539512] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539514] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539517] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539520] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539523] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539526] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539530] [ 790] 101 790 63477 631 0 0 0 rsyslogd Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539533] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539536] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539539] [ 975] 0 975 6276 79 1 0 0 master Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539541] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539544] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539547] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539550] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539553] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539556] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539559] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539565] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539568] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539570] [ 658] 1012 658 134393 70798 1 0 0 php Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539573] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539576] [13468] 0 13468 89584 2530 1 0 0 php5-fpm Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539579] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539582] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539586] [ 2686] 1012 2686 68618 5661 0 0 0 php Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539590] [20831] 108 20831 6792 76 0 0 0 pickup Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539593] [23032] 1012 23032 892399 829260 0 0 0 php Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539596] [23920] 1012 23920 68580 5604 1 0 0 php Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539599] Out of memory: Kill process 23032 (php) score 865 or sacrifice child Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539610] Killed process 23032 (php) total-vm:3569596kB, anon-rss:3317028kB, file-rss:12kB Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682835] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682840] php5-fpm cpuset=/ mems_allowed=0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682843] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682845] Call Trace: Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682853] [] dump_header+0x91/0xe0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682855] [] oom_kill_process+0x85/0xb0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682857] [] out_of_memory+0xfa/0x220 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682860] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682863] [] alloc_pages_current+0xb6/0x120 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682867] [] __page_cache_alloc+0xb7/0xd0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682869] [] filemap_fault+0x212/0x3c0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682872] [] __do_fault+0x72/0x550 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682876] [] ? __switch_to+0x138/0x360 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682880] [] ? _raw_spin_lock+0xe/0x20 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682882] [] handle_pte_fault+0xfa/0x200 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682885] [] ? xen_pmd_val+0xe/0x10 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682887] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682890] [] handle_mm_fault+0x269/0x370 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682892] [] do_page_fault+0x184/0x550 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682894] [] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682899] [] ? ep_poll+0x1cc/0x380 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682903] [] ? try_to_wake_up+0x200/0x200 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682905] [] page_fault+0x25/0x30 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682907] Mem-Info: Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682908] Node 0 DMA per-cpu: Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682910] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682912] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682913] Node 0 DMA32 per-cpu: Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682914] CPU 0: hi: 186, btch: 31 usd: 95 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682916] CPU 1: hi: 186, btch: 31 usd: 12 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682918] active_anon:932785 inactive_anon:1393 isolated_anon:0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682919] active_file:0 inactive_file:4 isolated_file:14 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682920] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682920] free:4026 slab_reclaimable:2624 slab_unreclaimable:2577 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682921] mapped:1352 shmem:1405 pagetables:3238 bounce:0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682923] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682929] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682932] Node 0 DMA32 free:7768kB min:7848kB low:9808kB high:11772kB active_anon:3731140kB inactive_anon:5572kB active_file:0kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):56kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:10496kB slab_unreclaimable:10308kB kernel_stack:760kB pagetables:12952kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:155 all_unreclaimable? yes Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682939] lowmem_reserve[]: 0 0 0 0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682941] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682948] Node 0 DMA32: 810*4kB 228*8kB 42*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7784kB Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682955] 1431 total pagecache pages Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682956] 0 pages in swap cache Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682957] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682958] Free swap = 0kB Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682959] Total swap = 0kB Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688076] 985072 pages RAM Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688078] 24998 pages reserved Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688079] 3779 pages shared Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688080] 953263 pages non-shared Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688082] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688091] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688095] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688098] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688101] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688104] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688108] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688111] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688114] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688118] [ 790] 101 790 63477 631 1 0 0 rsyslogd Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688121] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688124] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688128] [ 975] 0 975 6276 80 1 0 0 master Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688131] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688134] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688137] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688140] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688143] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688146] [ 1242] 107 1242 9445 162 0 0 0 ntpd Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688150] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688153] [ 654] 1012 654 70351 5761 0 0 0 php Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688156] [ 657] 1012 657 69242 6258 0 0 0 php Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688159] [ 658] 1012 658 134393 70799 0 0 0 php Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688162] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688165] [13468] 0 13468 89584 2515 0 0 0 php5-fpm Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688168] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688171] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688174] [ 2686] 1012 2686 68618 5661 1 0 0 php Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688178] [20831] 108 20831 6792 76 1 0 0 pickup Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688181] [23985] 1012 23985 891029 827945 0 0 0 php Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688184] [25060] 108 25060 6818 73 1 0 0 cleanup Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688187] [25061] 108 25061 6795 71 0 0 0 trivial-rewrite Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688190] [25062] 108 25062 11107 159 0 0 0 smtp Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688193] [25066] 108 25066 11107 158 1 0 0 smtp Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688196] [25593] 1012 25593 68580 5603 1 0 0 php Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688199] [25685] 1012 25685 1099 25 0 0 0 sh Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688202] [25686] 1012 25686 32613 425 0 0 0 php Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688205] Out of memory: Kill process 23985 (php) score 864 or sacrifice child Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688217] Killed process 23985 (php) total-vm:3564116kB, anon-rss:3311760kB, file-rss:20kB Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133939] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133943] php cpuset=/ mems_allowed=0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133947] Pid: 25730, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133949] Call Trace: Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133956] [] dump_header+0x91/0xe0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133960] [] oom_kill_process+0x85/0xb0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133963] [] out_of_memory+0xfa/0x220 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133966] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133970] [] alloc_pages_current+0xb6/0x120 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133974] [] __page_cache_alloc+0xb7/0xd0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133977] [] filemap_fault+0x212/0x3c0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133981] [] __do_fault+0x72/0x550 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133986] [] ? error_exit+0x2a/0x60 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133991] [] ? xen_hypervisor_callback+0x1b/0x20 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133994] [] handle_pte_fault+0xfa/0x200 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133998] [] ? xen_pmd_val+0xe/0x10 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134001] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134004] [] handle_mm_fault+0x269/0x370 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134007] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134011] [] ? check_events+0x12/0x20 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134014] [] do_page_fault+0x184/0x550 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134017] [] ? xen_mc_flush+0xb2/0x1c0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134020] [] ? xen_clts+0x8d/0x190 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134023] [] ? __schedule+0x3cc/0x6f0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134027] [] page_fault+0x25/0x30 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134029] Mem-Info: Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134030] Node 0 DMA per-cpu: Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134033] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134034] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134036] Node 0 DMA32 per-cpu: Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134038] CPU 0: hi: 186, btch: 31 usd: 49 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134040] CPU 1: hi: 186, btch: 31 usd: 0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134044] active_anon:933020 inactive_anon:1393 isolated_anon:0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134045] active_file:7 inactive_file:0 isolated_file:28 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134046] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134046] free:4048 slab_reclaimable:2610 slab_unreclaimable:2538 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134047] mapped:1355 shmem:1405 pagetables:3071 bounce:0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134050] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134060] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134064] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732080kB inactive_anon:5572kB active_file:28kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):112kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5620kB slab_reclaimable:10440kB slab_unreclaimable:10152kB kernel_stack:728kB pagetables:12284kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:169 all_unreclaimable? yes Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134073] lowmem_reserve[]: 0 0 0 0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134076] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134085] Node 0 DMA32: 795*4kB 217*8kB 42*16kB 6*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7892kB Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134094] 1423 total pagecache pages Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134096] 0 pages in swap cache Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134098] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134099] Free swap = 0kB Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134101] Total swap = 0kB Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139473] 985072 pages RAM Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139476] 24998 pages reserved Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139477] 3772 pages shared Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139478] 953288 pages non-shared Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139480] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139488] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139492] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139495] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139498] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139501] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139504] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139508] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139511] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139516] [ 790] 101 790 63477 631 1 0 0 rsyslogd Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139519] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139522] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139526] [ 975] 0 975 6276 80 0 0 0 master Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139529] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139532] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139535] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139538] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139541] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139544] [ 1242] 107 1242 9445 162 1 0 0 ntpd Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139547] [15657] 0 15657 4777 60 0 0 0 cron Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139550] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139553] [ 657] 1012 657 69242 6257 1 0 0 php Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139556] [ 658] 1012 658 134393 70799 1 0 0 php Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139559] [ 678] 1012 678 70152 5566 0 0 0 php Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139563] [13468] 0 13468 89584 2527 0 0 0 php5-fpm Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139566] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139569] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139572] [ 2686] 1012 2686 68618 5662 0 0 0 php Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139576] [20831] 108 20831 6792 76 1 0 0 pickup Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139579] [25730] 1012 25730 892179 829077 1 0 0 php Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139582] [32114] 1012 32114 68580 5604 1 0 0 php Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139585] Out of memory: Kill process 25730 (php) score 865 or sacrifice child Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139596] Killed process 25730 (php) total-vm:3568716kB, anon-rss:3316308kB, file-rss:0kB Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889405] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889409] php cpuset=/ mems_allowed=0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889412] Pid: 32149, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889413] Call Trace: Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889421] [] dump_header+0x91/0xe0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889423] [] oom_kill_process+0x85/0xb0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889425] [] out_of_memory+0xfa/0x220 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889428] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889431] [] alloc_pages_current+0xb6/0x120 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889435] [] __page_cache_alloc+0xb7/0xd0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889437] [] filemap_fault+0x212/0x3c0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889440] [] __do_fault+0x72/0x550 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889444] [] ? error_exit+0x2a/0x60 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889447] [] ? retint_restore_args+0x5/0x6 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889449] [] ? __do_fault+0x1/0x550 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889451] [] handle_pte_fault+0xfa/0x200 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889455] [] ? xen_pmd_val+0xe/0x10 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889457] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889459] [] handle_mm_fault+0x269/0x370 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889462] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889464] [] ? check_events+0x12/0x20 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889467] [] do_page_fault+0x184/0x550 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889469] [] ? do_brk+0x21f/0x320 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889472] [] ? __schedule+0x3cc/0x6f0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889474] [] page_fault+0x25/0x30 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889476] Mem-Info: Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889477] Node 0 DMA per-cpu: Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889479] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889480] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889481] Node 0 DMA32 per-cpu: Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889483] CPU 0: hi: 186, btch: 31 usd: 3 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889484] CPU 1: hi: 186, btch: 31 usd: 133 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889487] active_anon:932739 inactive_anon:1393 isolated_anon:0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889488] active_file:0 inactive_file:0 isolated_file:0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889488] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889489] free:4045 slab_reclaimable:2808 slab_unreclaimable:2583 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889490] mapped:1370 shmem:1405 pagetables:3071 bounce:0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889491] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889498] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889501] Node 0 DMA32 free:7844kB min:7848kB low:9808kB high:11772kB active_anon:3730956kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5480kB shmem:5620kB slab_reclaimable:11232kB slab_unreclaimable:10332kB kernel_stack:728kB pagetables:12284kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:70 all_unreclaimable? yes Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889507] lowmem_reserve[]: 0 0 0 0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889510] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889516] Node 0 DMA32: 1089*4kB 170*8kB 9*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7908kB Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889523] 1439 total pagecache pages Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889524] 0 pages in swap cache Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889525] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889526] Free swap = 0kB Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889527] Total swap = 0kB Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894896] 985072 pages RAM Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894898] 24998 pages reserved Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894899] 3790 pages shared Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894900] 953199 pages non-shared Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894902] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894911] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894915] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894918] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894921] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894924] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894928] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894931] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894934] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894938] [ 790] 101 790 63477 631 1 0 0 rsyslogd Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894941] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894944] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894948] [ 975] 0 975 6276 80 0 0 0 master Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894951] [ 983] 108 983 6834 107 0 0 0 qmgr Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894954] [ 1024] 0 1024 3309 38 1 0 0 mdadm Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894957] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894960] [ 1051] 0 1051 201625 844 0 0 0 collectd Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894963] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894967] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894970] [15657] 0 15657 4777 60 0 0 0 cron Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894973] [ 654] 1012 654 70351 5760 1 0 0 php Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894976] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894979] [ 658] 1012 658 134457 70863 1 0 0 php Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894982] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894985] [13468] 0 13468 89584 2512 1 0 0 php5-fpm Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894988] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894991] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894995] [ 2686] 1012 2686 68618 5664 0 0 0 php Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894998] [20831] 108 20831 6792 76 1 0 0 pickup Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.895001] [32149] 1012 32149 891891 828714 0 0 0 php Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.895004] [15244] 1012 15244 68580 5605 1 0 0 php Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.895007] Out of memory: Kill process 32149 (php) score 864 or sacrifice child Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.895019] Killed process 32149 (php) total-vm:3567564kB, anon-rss:3314856kB, file-rss:0kB Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108304] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108307] php cpuset=/ mems_allowed=0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108310] Pid: 16233, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108312] Call Trace: Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108319] [] dump_header+0x91/0xe0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108322] [] oom_kill_process+0x85/0xb0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108324] [] out_of_memory+0xfa/0x220 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108327] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108330] [] alloc_pages_current+0xb6/0x120 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108334] [] __page_cache_alloc+0xb7/0xd0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108336] [] filemap_fault+0x212/0x3c0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108339] [] __do_fault+0x72/0x550 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108343] [] ? error_exit+0x2a/0x60 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108346] [] ? retint_restore_args+0x5/0x6 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108348] [] handle_pte_fault+0xfa/0x200 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108351] [] ? xen_pmd_val+0xe/0x10 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108354] [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108356] [] handle_mm_fault+0x269/0x370 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108359] [] ? xen_force_evtchn_callback+0xd/0x10 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108361] [] ? check_events+0x12/0x20 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108364] [] do_page_fault+0x184/0x550 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108367] [] ? do_brk+0x21f/0x320 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108369] [] ? xen_clts+0x8d/0x190 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108371] [] ? __schedule+0x3cc/0x6f0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108374] [] page_fault+0x25/0x30 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108376] Mem-Info: Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108377] Node 0 DMA per-cpu: Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108379] CPU 0: hi: 0, btch: 1 usd: 0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108380] CPU 1: hi: 0, btch: 1 usd: 0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108381] Node 0 DMA32 per-cpu: Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108383] CPU 0: hi: 186, btch: 31 usd: 30 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108384] CPU 1: hi: 186, btch: 31 usd: 14 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108387] active_anon:933098 inactive_anon:1393 isolated_anon:0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108387] active_file:0 inactive_file:0 isolated_file:0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108388] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108389] free:4041 slab_reclaimable:2566 slab_unreclaimable:2549 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108389] mapped:1365 shmem:1405 pagetables:3061 bounce:0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108391] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108397] lowmem_reserve[]: 0 3772 3772 3772 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108400] Node 0 DMA32 free:7828kB min:7848kB low:9808kB high:11772kB active_anon:3732392kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5460kB shmem:5620kB slab_reclaimable:10264kB slab_unreclaimable:10196kB kernel_stack:728kB pagetables:12244kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:14 all_unreclaimable? yes Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108407] lowmem_reserve[]: 0 0 0 0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108409] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108416] Node 0 DMA32: 793*4kB 216*8kB 43*16kB 6*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7828kB Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108422] 1421 total pagecache pages Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108424] 0 pages in swap cache Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108425] Swap cache stats: add 0, delete 0, find 0/0 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108426] Free swap = 0kB Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108427] Total swap = 0kB Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113621] 985072 pages RAM Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113623] 24998 pages reserved Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113624] 3790 pages shared Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113626] 953311 pages non-shared Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113627] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113636] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113640] [ 477] 0 477 5393 119 1 -17 -1000 udevd Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113643] [ 530] 0 530 5363 105 1 -17 -1000 udevd Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113646] [ 531] 0 531 5363 103 1 -17 -1000 udevd Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113649] [ 607] 0 607 3796 46 0 0 0 upstart-socket- Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113652] [ 664] 0 664 1815 124 0 0 0 dhclient3 Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113655] [ 765] 0 765 12508 153 1 -17 -1000 sshd Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113659] [ 778] 102 778 5954 71 0 0 0 dbus-daemon Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113663] [ 790] 101 790 63477 631 1 0 0 rsyslogd Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113666] [ 827] 0 827 1081 36 1 0 0 acpid Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113669] [ 831] 0 831 4226 41 0 0 0 atd Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113673] [ 975] 0 975 6276 79 1 0 0 master Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113676] [ 983] 108 983 6834 107 1 0 0 qmgr Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113679] [ 1024] 0 1024 3309 38 0 0 0 mdadm Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113682] [ 1049] 0 1049 1072 23 1 0 0 collectdmon Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113685] [ 1051] 0 1051 201625 844 1 0 0 collectd Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113688] [ 1088] 0 1088 3625 40 1 0 0 getty Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113691] [ 1242] 107 1242 9445 163 1 0 0 ntpd Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113695] [15657] 0 15657 4777 59 0 0 0 cron Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113698] [ 654] 1012 654 70351 5761 1 0 0 php Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113701] [ 657] 1012 657 69242 6258 1 0 0 php Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113704] [ 658] 1012 658 134457 70863 1 0 0 php Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113707] [ 678] 1012 678 70152 5566 1 0 0 php Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113710] [13468] 0 13468 89584 2497 1 0 0 php5-fpm Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113713] [13470] 33 13470 89584 1189 0 0 0 php5-fpm Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113716] [13471] 33 13471 89584 1189 0 0 0 php5-fpm Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113719] [ 2686] 1012 2686 68618 5663 1 0 0 php Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113722] [20831] 108 20831 6792 76 0 0 0 pickup Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113726] [16233] 1012 16233 892243 829105 0 0 0 php Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113729] [16553] 1012 16553 68580 5605 1 0 0 php Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113732] Out of memory: Kill process 16233 (php) score 865 or sacrifice child Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113743] Killed process 16233 (php) total-vm:3568972kB, anon-rss:3316388kB, file-rss:32kB Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261526] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261530] php cpuset=/ mems_allowed=0 Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261533] Pid: 23428, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261534] Call Trace: Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261542] [] dump_header+0x91/0xe0 Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261544] [] oom_kill_process+0x85/0xb0 Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261546] [] out_of_memory+0xfa/0x220 Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261549] [] __alloc_pages_nodemask+0x8c3/0x8e0 Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261552] [