Linux over commit heuristic

问题描述 投票:3回答:1

来自内核文档的过度提交文章提到过度提交模式0基于启发式提交处理。它没有概述所涉及的启发式。

有人可以阐明实际的启发式是什么吗?任何与内核源相关的链接也都有效!

linux memory-management linux-kernel
1个回答
8
投票

实际上,过度使用会计的内核文档有一些细节:https://www.kernel.org/doc/Documentation/vm/overcommit-accounting

Linux内核支持以下过度使用处理模式

0 - 启发式过度使用处理。

地址空间的明显过度使用被拒绝。用于典型系统。它确保严重的疯狂分配失败,同时允许过度使用以减少交换使用。 root允许在此模式下分配更多内存。这是默认值。

还有Documentation/sysctl/vm.txt

overcommit_memory:此值包含一个启用内存过量使用的标志。 当此标志为0时,内核尝试估计当用户空间请求更多内存时剩余的可用内存量...

有关更多信息,请参阅Documentation / vm / overcommit-accounting和mm / mmap.c :: __ vm_enough_memory()。

另外,man 5 proc

/proc/sys/vm/overcommit_memory此文件包含内核虚拟内存记帐模式。价值观是:

                0: heuristic overcommit (this is the default)
                1: always overcommit, never check
                2: always check, never overcommit

在模式0中,不检查mmap(2)MAP_NORESERVE的调用,并且默认检查非常弱,导致获得进程“OOM-killed”的风险。

因此,启发式会禁用非常大的分配,但有时应用程序可能会分配比系统中物理内存大小更多的虚拟内存,如果它不使用全部内存。使用MAP_NORESERVE的mmapable内存量可能会更高。

设置为“过度使用策略是通过sysctl`vm.overcommit_memory'设置的”,因此我们可以在源代码中找到它的实现方式:http://lxr.free-electrons.com/ident?v=4.4;i=sysctl_overcommit_memory,在line 112 of mm/mmap.c中定义

  112 int sysctl_overcommit_memory __read_mostly = OVERCOMMIT_GUESS;  /* heuristic overcommit */

和常量OVERCOMMIT_GUESS(在linux/mman.h中定义)实际上只是在used中的line 170 of mm/mmap.c,这是启发式的实现:

138 /*
139  * Check that a process has enough memory to allocate a new virtual
140  * mapping. 0 means there is enough memory for the allocation to
141  * succeed and -ENOMEM implies there is not.
142  *
143  * We currently support three overcommit policies, which are set via the
144  * vm.overcommit_memory sysctl.  See Documentation/vm/overcommit-accounting
145  *
146  * Strict overcommit modes added 2002 Feb 26 by Alan Cox.
147  * Additional code 2002 Jul 20 by Robert Love.
148  *
149  * cap_sys_admin is 1 if the process has admin privileges, 0 otherwise.
150  *
151  * Note this is a helper function intended to be used by LSMs which
152  * wish to use this logic.
153  */
154 int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
...
170         if (sysctl_overcommit_memory == OVERCOMMIT_GUESS) {
171                 free = global_page_state(NR_FREE_PAGES);
172                 free += global_page_state(NR_FILE_PAGES);
173 
174                 /*
175                  * shmem pages shouldn't be counted as free in this
176                  * case, they can't be purged, only swapped out, and
177                  * that won't affect the overall amount of available
178                  * memory in the system.
179                  */
180                 free -= global_page_state(NR_SHMEM);
181 
182                 free += get_nr_swap_pages();
183 
184                 /*
185                  * Any slabs which are created with the
186                  * SLAB_RECLAIM_ACCOUNT flag claim to have contents
187                  * which are reclaimable, under pressure.  The dentry
188                  * cache and most inode caches should fall into this
189                  */
190                 free += global_page_state(NR_SLAB_RECLAIMABLE);
191 
192                 /*
193                  * Leave reserved pages. The pages are not for anonymous pages.
194                  */
195                 if (free <= totalreserve_pages)
196                         goto error;
197                 else
198                         free -= totalreserve_pages;
199 
200                 /*
201                  * Reserve some for root
202                  */
203                 if (!cap_sys_admin)
204                         free -= sysctl_admin_reserve_kbytes >> (PAGE_SHIFT - 10);
205 
206                 if (free > pages)
207                         return 0;
208 
209                 goto error;
210         }

因此,启发式是估计现在使用多少物理内存页面的方法(free),当处理更多内存的请求时(应用程序请求pages页面)。

始终启用overcommit(“1”),此函数始终返回0(“此请求有足够的内存”)

164         /*
165          * Sometimes we want to use more memory than we have
166          */
167         if (sysctl_overcommit_memory == OVERCOMMIT_ALWAYS)
168                 return 0;

如果没有此默认启发式,在模式“2”中,内核将尝试将所请求的pages页面计入新的Committed_AS(来自/proc/meminfo):

162         vm_acct_memory(pages);
...

is actually只增加了vm_committed_as - __percpu_counter_add(&vm_committed_as, pages, vm_committed_as_batch);

212         allowed = vm_commit_limit();

一些神奇的is here:

401 /*
402  * Committed memory limit enforced when OVERCOMMIT_NEVER policy is used
403  */
404 unsigned long vm_commit_limit(void)
405 {
406         unsigned long allowed;
407 
408         if (sysctl_overcommit_kbytes)
409                 allowed = sysctl_overcommit_kbytes >> (PAGE_SHIFT - 10);
410         else
411                 allowed = ((totalram_pages - hugetlb_total_pages())
412                            * sysctl_overcommit_ratio / 100);
413         allowed += total_swap_pages;
414 
415         return allowed;
416 }
417 

因此,allowedvm.overcommit_kbytes sysctl中设置为千字节,或者以物理RAM的百分比设置为vm.overcommit_ratio,加上交换大小。

213         /*
214          * Reserve some for root
215          */
216         if (!cap_sys_admin)
217                 allowed -= sysctl_admin_reserve_kbytes >> (PAGE_SHIFT - 10);

仅为root提供一些内存(对于健康人来说,Page_shift是12,page_shift-10只是从kbytes到页数的转换)。

218 
219         /*
220          * Don't let a single process grow so big a user can't recover
221          */
222         if (mm) {
223                 reserve = sysctl_user_reserve_kbytes >> (PAGE_SHIFT - 10);
224                 allowed -= min_t(long, mm->total_vm / 32, reserve);
225         }
226 
227         if (percpu_counter_read_positive(&vm_committed_as) < allowed)
228                 return 0;

如果在对请求进行计费后,所有用户空间仍然提交的内存量少于允许,则分配它。在其他情况下,拒绝请求(并取消计算请求)。

229 error:
230         vm_unacct_memory(pages);
231 
232         return -ENOMEM;

换句话说,总结在“Linux内核。关于Linux内核的一些评论”,2003-02-01作者:Andries Brouwer,9。Memory,9.6 Overcommit和OOM - https://www.win.tue.nl/~aeb/linux/lk/lk-9.html

走向正确的方向

从2.5.30开始,价值是:

  • 0(默认):和以前一样:猜测过度承诺是多少合理,
  • 1:永远不要拒绝任何malloc(),
  • 2:准确地说是过度使用 - 永远不要提交比交换空间大的虚拟地址空间加上物理内存的分数overcommit_ratio

因此“2”是在请求之后使用的存储量的精确计算,并且“0”是启发式估计。

© www.soinside.com 2019 - 2024. All rights reserved.