History

mm: Don't count the stack guard page towards RLIMIT_STACK

Commit fee7e49d4514 ("mm: propagate error from stack expansion even for guard page") made sure that we return the error properly for stack growth conditions. It also theorized that counting the guard page towards the stack limit might break something, but also said "Let's see if anybody notices". Somebody did notice. Apparently android-x86 sets the stack limit very close to the limit indeed, and including the guard page in the rlimit check causes the android 'zygote' process problems. So this adds the (fairly trivial) code to make the stack rlimit check be against the actual real stack size, rather than the size of the vma that includes the guard page.

Linus Torvalds committed 11 years ago

Commit 690eac51 parent 4850d371 file · +5 −2Browse files

Unified · Split

mm/mmap.c+5−2
@@ -2099,14 +2099,17 @@ static int acct_stack_growth(struct vm_area_struct *vma, unsigned long size, uns · 2098 unmodified lines
20992099 {
21002100 struct mm_struct *mm = vma->vm_mm;
21012101 struct rlimit *rlim = current->signal->rlim;
2102− unsigned long new_start;
2102+ unsigned long new_start, actual_size;
21032103
21042104 /* address space limit tests */
21052105 if (!may_expand_vm(mm, grow))
21062106 return -ENOMEM;
21072107
21082108 /* Stack limit test */
2109− if (size > ACCESS_ONCE(rlim[RLIMIT_STACK].rlim_cur))
2109+ actual_size = size;
2110+ if (size && (vma->vm_flags & (VM_GROWSUP | VM_GROWSDOWN)))
2111+ actual_size -= PAGE_SIZE;
2112+ if (actual_size > ACCESS_ONCE(rlim[RLIMIT_STACK].rlim_cur))
21102113 return -ENOMEM;
21112114
21122115 /* mlock limit tests */