History

mm, thp: do not allow thp faults to avoid cpuset restrictions

The page allocator relies on __GFP_WAIT to determine if ALLOC_CPUSET should be set in allocflags. ALLOC_CPUSET controls if a page allocation should be restricted only to the set of allowed cpuset mems. Transparent hugepages clears __GFP_WAIT when defrag is disabled to prevent the fault path from using memory compaction or direct reclaim. Thus, it is unfairly able to allocate outside of its cpuset mems restriction as a side-effect. This patch ensures that ALLOC_CPUSET is only cleared when the gfp mask is truly GFP_ATOMIC by verifying it is also not a thp allocation.

David Rientjes committed 12 years ago

Commit b104a351 parent f6789591 file · +8 −8Browse files

Unified · Split

mm/page_alloc.c+8−8
@@ -2447,7 +2447,7 @@ static inline int · 2446 unmodified lines
2447gfp_to_alloc_flags(gfp_t gfp_mask)2447gfp_to_alloc_flags(gfp_t gfp_mask)
2448{2448{
2449 int alloc_flags = ALLOC_WMARK_MIN | ALLOC_CPUSET;2449 int alloc_flags = ALLOC_WMARK_MIN | ALLOC_CPUSET;
2450 const gfp_t wait = gfp_mask & __GFP_WAIT;2450 const bool atomic = !(gfp_mask & (__GFP_WAIT | __GFP_NO_KSWAPD));
2451 2451
2452 /* __GFP_HIGH is assumed to be the same as ALLOC_HIGH to save a branch. */2452 /* __GFP_HIGH is assumed to be the same as ALLOC_HIGH to save a branch. */
2453 BUILD_BUG_ON(__GFP_HIGH != (__force gfp_t) ALLOC_HIGH);2453 BUILD_BUG_ON(__GFP_HIGH != (__force gfp_t) ALLOC_HIGH);
@@ -2456,20 +2456,20 @@ gfp_to_alloc_flags(gfp_t gfp_mask) · 2 unmodified lines
2456 * The caller may dip into page reserves a bit more if the caller2456 * The caller may dip into page reserves a bit more if the caller
2457 * cannot run direct reclaim, or if the caller has realtime scheduling2457 * cannot run direct reclaim, or if the caller has realtime scheduling
2458 * policy or is asking for __GFP_HIGH memory. GFP_ATOMIC requests will2458 * policy or is asking for __GFP_HIGH memory. GFP_ATOMIC requests will
2459 * set both ALLOC_HARDER (!wait) and ALLOC_HIGH (__GFP_HIGH).2459 * set both ALLOC_HARDER (atomic == true) and ALLOC_HIGH (__GFP_HIGH).
2460 */2460 */
2461 alloc_flags |= (__force int) (gfp_mask & __GFP_HIGH);2461 alloc_flags |= (__force int) (gfp_mask & __GFP_HIGH);
2462 2462
2463 if (!wait) {2463 if (atomic) {
2464 /*2464 /*
2465 * Not worth trying to allocate harder for2465 * Not worth trying to allocate harder for __GFP_NOMEMALLOC even
2466 * __GFP_NOMEMALLOC even if it can't schedule.2466 * if it can't schedule.
2467 */2467 */
2468 if (!(gfp_mask & __GFP_NOMEMALLOC))2468 if (!(gfp_mask & __GFP_NOMEMALLOC))
2469 alloc_flags |= ALLOC_HARDER;2469 alloc_flags |= ALLOC_HARDER;
2470 /*2470 /*
2471 * Ignore cpuset if GFP_ATOMIC (!wait) rather than fail alloc.2471 * Ignore cpuset mems for GFP_ATOMIC rather than fail, see the
2472 * See also cpuset_zone_allowed() comment in kernel/cpuset.c.2472 * comment for __cpuset_node_allowed_softwall().
2473 */2473 */
2474 alloc_flags &= ~ALLOC_CPUSET;2474 alloc_flags &= ~ALLOC_CPUSET;
2475 } else if (unlikely(rt_task(current)) && !in_interrupt())2475 } else if (unlikely(rt_task(current)) && !in_interrupt())

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t