Pull hotfixes from Andrew Morton: "Minor things, mainly - mailmap updates, MAINTAINERS updates, etc. Fixes for this merge window: - fix for a damon boot hang, from SeongJae - fix for a kfence warning splat, from Jason Donenfeld - fix for zero-pfn pinning, from Alex Williamson - fix for fallocate hole punch clearing, from Mike Kravetz Fixes for previous releases: - fix for a performance regression, from Marcelo - fix for a hwpoisining BUG from zhenwei pi" * tag 'mm-hotfixes-stable-2022-06-26' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: mailmap: add entry for Christian Marangi mm/memory-failure: disable unpoison once hw error happens hugetlbfs: zero partial pages during fallocate hole punch mm: memcontrol: reference to tools/cgroup/memcg_slabinfo.py mm: re-allow pinning of zero pfns mm/kfence: select random number before taking raw lock MAINTAINERS: add maillist information for LoongArch MAINTAINERS: update MM tree references MAINTAINERS: update Abel Vesa's email MAINTAINERS: add MEMORY HOT(UN)PLUG section and add David as reviewer MAINTAINERS: add Miaohe Lin as a memory-failure reviewer mailmap: add alias for jarkko@profian.com mm/damon/reclaim: schedule 'damon_reclaim_timer' only after 'system_wq' is initialized kthread: make it clear that kthread_create_on_node() might be terminated by any fatal signal mm: lru_cache_disable: use synchronize_rcu_expedited mm/page_isolation.c: fix one kernel-doc comment
Unified · Split
| @@ -10,6 +10,8 @@ · 9 unmodified lines | |||
| 10 | # Please keep this list dictionary sorted. | 10 | # Please keep this list dictionary sorted. |
| 11 | # | 11 | # |
| 12 | Aaron Durbin <adurbin@google.com> | 12 | Aaron Durbin <adurbin@google.com> |
| 13 | Abel Vesa <abelvesa@kernel.org> <abel.vesa@nxp.com> | ||
| 14 | Abel Vesa <abelvesa@kernel.org> <abelvesa@gmail.com> | ||
| 13 | Abhinav Kumar <quic_abhinavk@quicinc.com> <abhinavk@codeaurora.org> | 15 | Abhinav Kumar <quic_abhinavk@quicinc.com> <abhinavk@codeaurora.org> |
| 14 | Adam Oldham <oldhamca@gmail.com> | 16 | Adam Oldham <oldhamca@gmail.com> |
| 15 | Adam Radford <aradford@gmail.com> | 17 | Adam Radford <aradford@gmail.com> |
| @@ -85,6 +87,7 @@ Christian Borntraeger <borntraeger@linux.ibm.com> <borntrae@de.ibm.com> · 69 unmodified lines | |||
| 85 | Christian Brauner <brauner@kernel.org> <christian@brauner.io> | 87 | Christian Brauner <brauner@kernel.org> <christian@brauner.io> |
| 86 | Christian Brauner <brauner@kernel.org> <christian.brauner@canonical.com> | 88 | Christian Brauner <brauner@kernel.org> <christian.brauner@canonical.com> |
| 87 | Christian Brauner <brauner@kernel.org> <christian.brauner@ubuntu.com> | 89 | Christian Brauner <brauner@kernel.org> <christian.brauner@ubuntu.com> |
| 90 | Christian Marangi <ansuelsmth@gmail.com> | ||
| 88 | Christophe Ricard <christophe.ricard@gmail.com> | 91 | Christophe Ricard <christophe.ricard@gmail.com> |
| 89 | Christoph Hellwig <hch@lst.de> | 92 | Christoph Hellwig <hch@lst.de> |
| 90 | Colin Ian King <colin.king@intel.com> <colin.king@canonical.com> | 93 | Colin Ian King <colin.king@intel.com> <colin.king@canonical.com> |
| @@ -165,6 +168,7 @@ Jan Glauber <jan.glauber@gmail.com> <jang@de.ibm.com> · 74 unmodified lines | |||
| 165 | Jan Glauber <jan.glauber@gmail.com> <jang@linux.vnet.ibm.com> | 168 | Jan Glauber <jan.glauber@gmail.com> <jang@linux.vnet.ibm.com> |
| 166 | Jan Glauber <jan.glauber@gmail.com> <jglauber@cavium.com> | 169 | Jan Glauber <jan.glauber@gmail.com> <jglauber@cavium.com> |
| 167 | Jarkko Sakkinen <jarkko@kernel.org> <jarkko.sakkinen@linux.intel.com> | 170 | Jarkko Sakkinen <jarkko@kernel.org> <jarkko.sakkinen@linux.intel.com> |
| 171 | Jarkko Sakkinen <jarkko@kernel.org> <jarkko@profian.com> | ||
| 168 | Jason Gunthorpe <jgg@ziepe.ca> <jgg@mellanox.com> | 172 | Jason Gunthorpe <jgg@ziepe.ca> <jgg@mellanox.com> |
| 169 | Jason Gunthorpe <jgg@ziepe.ca> <jgg@nvidia.com> | 173 | Jason Gunthorpe <jgg@ziepe.ca> <jgg@nvidia.com> |
| 170 | Jason Gunthorpe <jgg@ziepe.ca> <jgunthorpe@obsidianresearch.com> | 174 | Jason Gunthorpe <jgg@ziepe.ca> <jgunthorpe@obsidianresearch.com> |
| @@ -120,7 +120,8 @@ Testing · 119 unmodified lines | |||
| 120 | unpoison-pfn | 120 | unpoison-pfn |
| 121 | Software-unpoison page at PFN echoed into this file. This way | 121 | Software-unpoison page at PFN echoed into this file. This way |
| 122 | a page can be reused again. This only works for Linux | 122 | a page can be reused again. This only works for Linux |
| 123 | injected failures, not for real memory failures. | 123 | injected failures, not for real memory failures. Once any hardware |
| 124 | memory failure happens, this feature is disabled. | ||
| 124 | 125 | ||
| 125 | Note these injection interfaces are not stable and might change between | 126 | Note these injection interfaces are not stable and might change between |
| 126 | kernel versions | 127 | kernel versions |
| @@ -9152,6 +9152,7 @@ F: drivers/media/platform/st/sti/hva · 9151 unmodified lines | |||
| 9152 | 9152 | ||
| 9153 | HWPOISON MEMORY FAILURE HANDLING | 9153 | HWPOISON MEMORY FAILURE HANDLING |
| 9154 | M: Naoya Horiguchi <naoya.horiguchi@nec.com> | 9154 | M: Naoya Horiguchi <naoya.horiguchi@nec.com> |
| 9155 | R: Miaohe Lin <linmiaohe@huawei.com> | ||
| 9155 | L: linux-mm@kvack.org | 9156 | L: linux-mm@kvack.org |
| 9156 | S: Maintained | 9157 | S: Maintained |
| 9157 | F: mm/hwpoison-inject.c | 9158 | F: mm/hwpoison-inject.c |
| @@ -11623,6 +11624,7 @@ F: drivers/gpu/drm/bridge/lontium-lt8912b.c · 2465 unmodified lines | |||
| 11623 | LOONGARCH | 11624 | LOONGARCH |
| 11624 | M: Huacai Chen <chenhuacai@kernel.org> | 11625 | M: Huacai Chen <chenhuacai@kernel.org> |
| 11625 | R: WANG Xuerui <kernel@xen0n.name> | 11626 | R: WANG Xuerui <kernel@xen0n.name> |
| 11627 | L: loongarch@lists.linux.dev | ||
| 11626 | S: Maintained | 11628 | S: Maintained |
| 11627 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git | 11629 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git |
| 11628 | F: arch/loongarch/ | 11630 | F: arch/loongarch/ |
| @@ -12879,9 +12881,8 @@ M: Andrew Morton <akpm@linux-foundation.org> · 1250 unmodified lines | |||
| 12879 | L: linux-mm@kvack.org | 12881 | L: linux-mm@kvack.org |
| 12880 | S: Maintained | 12882 | S: Maintained |
| 12881 | W: http://www.linux-mm.org | 12883 | W: http://www.linux-mm.org |
| 12882 | T: quilt https://ozlabs.org/~akpm/mmotm/ | 12884 | T: git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm |
| 12883 | T: quilt https://ozlabs.org/~akpm/mmots/ | 12885 | T: quilt git://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new |
| 12884 | T: git git://github.com/hnaz/linux-mm.git | ||
| 12885 | F: include/linux/gfp.h | 12886 | F: include/linux/gfp.h |
| 12886 | F: include/linux/memory_hotplug.h | 12887 | F: include/linux/memory_hotplug.h |
| 12887 | F: include/linux/mm.h | 12888 | F: include/linux/mm.h |
| @@ -12891,6 +12892,18 @@ F: include/linux/vmalloc.h · 3 unmodified lines | |||
| 12891 | F: mm/ | 12892 | F: mm/ |
| 12892 | F: tools/testing/selftests/vm/ | 12893 | F: tools/testing/selftests/vm/ |
| 12893 | 12894 | ||
| 12895 | MEMORY HOT(UN)PLUG | ||
| 12896 | M: David Hildenbrand <david@redhat.com> | ||
| 12897 | M: Oscar Salvador <osalvador@suse.de> | ||
| 12898 | L: linux-mm@kvack.org | ||
| 12899 | S: Maintained | ||
| 12900 | F: Documentation/admin-guide/mm/memory-hotplug.rst | ||
| 12901 | F: Documentation/core-api/memory-hotplug.rst | ||
| 12902 | F: drivers/base/memory.c | ||
| 12903 | F: include/linux/memory_hotplug.h | ||
| 12904 | F: mm/memory_hotplug.c | ||
| 12905 | F: tools/testing/selftests/memory-hotplug/ | ||
| 12906 | |||
| 12894 | MEMORY TECHNOLOGY DEVICES (MTD) | 12907 | MEMORY TECHNOLOGY DEVICES (MTD) |
| 12895 | M: Miquel Raynal <miquel.raynal@bootlin.com> | 12908 | M: Miquel Raynal <miquel.raynal@bootlin.com> |
| 12896 | M: Richard Weinberger <richard@nod.at> | 12909 | M: Richard Weinberger <richard@nod.at> |
| @@ -14295,7 +14308,7 @@ F: drivers/iio/gyro/fxas21002c_i2c.c · 1398 unmodified lines | |||
| 14295 | F: drivers/iio/gyro/fxas21002c_spi.c | 14308 | F: drivers/iio/gyro/fxas21002c_spi.c |
| 14296 | 14309 | ||
| 14297 | NXP i.MX CLOCK DRIVERS | 14310 | NXP i.MX CLOCK DRIVERS |
| 14298 | M: Abel Vesa <abel.vesa@nxp.com> | 14311 | M: Abel Vesa <abelvesa@kernel.org> |
| 14299 | L: linux-clk@vger.kernel.org | 14312 | L: linux-clk@vger.kernel.org |
| 14300 | L: linux-imx@nxp.com | 14313 | L: linux-imx@nxp.com |
| 14301 | S: Maintained | 14314 | S: Maintained |
| @@ -558,7 +558,7 @@ static ssize_t hard_offline_page_store(struct device *dev, · 557 unmodified lines | |||
| 558 | if (kstrtoull(buf, 0, &pfn) < 0) | 558 | if (kstrtoull(buf, 0, &pfn) < 0) |
| 559 | return -EINVAL; | 559 | return -EINVAL; |
| 560 | pfn >>= PAGE_SHIFT; | 560 | pfn >>= PAGE_SHIFT; |
| 561 | ret = memory_failure(pfn, 0); | 561 | ret = memory_failure(pfn, MF_SW_SIMULATED); |
| 562 | if (ret == -EOPNOTSUPP) | 562 | if (ret == -EOPNOTSUPP) |
| 563 | ret = 0; | 563 | ret = 0; |
| 564 | return ret ? ret : count; | 564 | return ret ? ret : count; |
| @@ -600,41 +600,79 @@ static void hugetlb_vmtruncate(struct inode *inode, loff_t offset) · 599 unmodified lines | |||
| 600 | remove_inode_hugepages(inode, offset, LLONG_MAX); | 600 | remove_inode_hugepages(inode, offset, LLONG_MAX); |
| 601 | } | 601 | } |
| 602 | 602 | ||
| 603 | static void hugetlbfs_zero_partial_page(struct hstate *h, | ||
| 604 | struct address_space *mapping, | ||
| 605 | loff_t start, | ||
| 606 | loff_t end) | ||
| 607 | { | ||
| 608 | pgoff_t idx = start >> huge_page_shift(h); | ||
| 609 | struct folio *folio; | ||
| 610 | |||
| 611 | folio = filemap_lock_folio(mapping, idx); | ||
| 612 | if (!folio) | ||
| 613 | return; | ||
| 614 | |||
| 615 | start = start & ~huge_page_mask(h); | ||
| 616 | end = end & ~huge_page_mask(h); | ||
| 617 | if (!end) | ||
| 618 | end = huge_page_size(h); | ||
| 619 | |||
| 620 | folio_zero_segment(folio, (size_t)start, (size_t)end); | ||
| 621 | |||
| 622 | folio_unlock(folio); | ||
| 623 | folio_put(folio); | ||
| 624 | } | ||
| 625 | |||
| 603 | static long hugetlbfs_punch_hole(struct inode *inode, loff_t offset, loff_t len) | 626 | static long hugetlbfs_punch_hole(struct inode *inode, loff_t offset, loff_t len) |
| 604 | { | 627 | { |
| 628 | struct hugetlbfs_inode_info *info = HUGETLBFS_I(inode); | ||
| 629 | struct address_space *mapping = inode->i_mapping; | ||
| 605 | struct hstate *h = hstate_inode(inode); | 630 | struct hstate *h = hstate_inode(inode); |
| 606 | loff_t hpage_size = huge_page_size(h); | 631 | loff_t hpage_size = huge_page_size(h); |
| 607 | loff_t hole_start, hole_end; | 632 | loff_t hole_start, hole_end; |
| 608 | 633 | ||
| 609 | /* | 634 | /* |
| 610 | * For hole punch round up the beginning offset of the hole and | 635 | * hole_start and hole_end indicate the full pages within the hole. |
| 611 | * round down the end. | ||
| 612 | */ | 636 | */ |
| 613 | hole_start = round_up(offset, hpage_size); | 637 | hole_start = round_up(offset, hpage_size); |
| 614 | hole_end = round_down(offset + len, hpage_size); | 638 | hole_end = round_down(offset + len, hpage_size); |
| 615 | 639 | ||
| 616 | if (hole_end > hole_start) { | 640 | inode_lock(inode); |
| 617 | struct address_space *mapping = inode->i_mapping; | ||
| 618 | struct hugetlbfs_inode_info *info = HUGETLBFS_I(inode); | ||
| 619 | 641 | ||
| 620 | inode_lock(inode); | 642 | /* protected by i_rwsem */ |
| 643 | if (info->seals & (F_SEAL_WRITE | F_SEAL_FUTURE_WRITE)) { | ||
| 644 | inode_unlock(inode); | ||
| 645 | return -EPERM; | ||
| 646 | } | ||
| 621 | 647 | ||
| 622 | /* protected by i_rwsem */ | 648 | i_mmap_lock_write(mapping); |
| 623 | if (info->seals & (F_SEAL_WRITE | F_SEAL_FUTURE_WRITE)) { | 649 | |
| 624 | inode_unlock(inode); | 650 | /* If range starts before first full page, zero partial page. */ |
| 625 | return -EPERM; | 651 | if (offset < hole_start) |
| 626 | } | 652 | hugetlbfs_zero_partial_page(h, mapping, |
| 653 | offset, min(offset + len, hole_start)); | ||
| 627 | 654 | ||
| 628 | i_mmap_lock_write(mapping); | 655 | /* Unmap users of full pages in the hole. */ |
| 656 | if (hole_end > hole_start) { | ||
| 629 | if (!RB_EMPTY_ROOT(&mapping->i_mmap.rb_root)) | 657 | if (!RB_EMPTY_ROOT(&mapping->i_mmap.rb_root)) |
| 630 | hugetlb_vmdelete_list(&mapping->i_mmap, | 658 | hugetlb_vmdelete_list(&mapping->i_mmap, |
| 631 | hole_start >> PAGE_SHIFT, | 659 | hole_start >> PAGE_SHIFT, |
| 632 | hole_end >> PAGE_SHIFT, 0); | 660 | hole_end >> PAGE_SHIFT, 0); |
| 633 | i_mmap_unlock_write(mapping); | ||
| 634 | remove_inode_hugepages(inode, hole_start, hole_end); | ||
| 635 | inode_unlock(inode); | ||
| 636 | } | 661 | } |
| 637 | 662 | ||
| 663 | /* If range extends beyond last full page, zero partial page. */ | ||
| 664 | if ((offset + len) > hole_end && (offset + len) > hole_start) | ||
| 665 | hugetlbfs_zero_partial_page(h, mapping, | ||
| 666 | hole_end, offset + len); | ||
| 667 | |||
| 668 | i_mmap_unlock_write(mapping); | ||
| 669 | |||
| 670 | /* Remove full pages from the file. */ | ||
| 671 | if (hole_end > hole_start) | ||
| 672 | remove_inode_hugepages(inode, hole_start, hole_end); | ||
| 673 | |||
| 674 | inode_unlock(inode); | ||
| 675 | |||
| 638 | return 0; | 676 | return 0; |
| 639 | } | 677 | } |
| 640 | 678 | ||
| @@ -1600,7 +1600,7 @@ static inline bool is_pinnable_page(struct page *page) · 1599 unmodified lines | |||
| 1600 | if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE) | 1600 | if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE) |
| 1601 | return false; | 1601 | return false; |
| 1602 | #endif | 1602 | #endif |
| 1603 | return !(is_zone_movable_page(page) || is_zero_pfn(page_to_pfn(page))); | 1603 | return !is_zone_movable_page(page) || is_zero_pfn(page_to_pfn(page)); |
| 1604 | } | 1604 | } |
| 1605 | #else | 1605 | #else |
| 1606 | static inline bool is_pinnable_page(struct page *page) | 1606 | static inline bool is_pinnable_page(struct page *page) |
| @@ -3232,6 +3232,7 @@ enum mf_flags { · 1625 unmodified lines | |||
| 3232 | MF_MUST_KILL = 1 << 2, | 3232 | MF_MUST_KILL = 1 << 2, |
| 3233 | MF_SOFT_OFFLINE = 1 << 3, | 3233 | MF_SOFT_OFFLINE = 1 << 3, |
| 3234 | MF_UNPOISON = 1 << 4, | 3234 | MF_UNPOISON = 1 << 4, |
| 3235 | MF_SW_SIMULATED = 1 << 5, | ||
| 3235 | }; | 3236 | }; |
| 3236 | extern int memory_failure(unsigned long pfn, int flags); | 3237 | extern int memory_failure(unsigned long pfn, int flags); |
| 3237 | extern void memory_failure_queue(unsigned long pfn, int flags); | 3238 | extern void memory_failure_queue(unsigned long pfn, int flags); |
| @@ -340,7 +340,7 @@ static int kthread(void *_create) · 339 unmodified lines | |||
| 340 | 340 | ||
| 341 | self = to_kthread(current); | 341 | self = to_kthread(current); |
| 342 | 342 | ||
| 343 | /* If user was SIGKILLed, I release the structure. */ | 343 | /* Release the structure when caller killed by a fatal signal. */ |
| 344 | done = xchg(&create->done, NULL); | 344 | done = xchg(&create->done, NULL); |
| 345 | if (!done) { | 345 | if (!done) { |
| 346 | kfree(create); | 346 | kfree(create); |
| @@ -398,7 +398,7 @@ static void create_kthread(struct kthread_create_info *create) · 51 unmodified lines | |||
| 398 | /* We want our own signal handler (we take no signals by default). */ | 398 | /* We want our own signal handler (we take no signals by default). */ |
| 399 | pid = kernel_thread(kthread, create, CLONE_FS | CLONE_FILES | SIGCHLD); | 399 | pid = kernel_thread(kthread, create, CLONE_FS | CLONE_FILES | SIGCHLD); |
| 400 | if (pid < 0) { | 400 | if (pid < 0) { |
| 401 | /* If user was SIGKILLed, I release the structure. */ | 401 | /* Release the structure when caller killed by a fatal signal. */ |
| 402 | struct completion *done = xchg(&create->done, NULL); | 402 | struct completion *done = xchg(&create->done, NULL); |
| 403 | 403 | ||
| 404 | if (!done) { | 404 | if (!done) { |
| @@ -440,9 +440,9 @@ struct task_struct *__kthread_create_on_node(int (*threadfn)(void *data), · 35 unmodified lines | |||
| 440 | */ | 440 | */ |
| 441 | if (unlikely(wait_for_completion_killable(&done))) { | 441 | if (unlikely(wait_for_completion_killable(&done))) { |
| 442 | /* | 442 | /* |
| 443 | * If I was SIGKILLed before kthreadd (or new kernel thread) | 443 | * If I was killed by a fatal signal before kthreadd (or new |
| 444 | * calls complete(), leave the cleanup of this structure to | 444 | * kernel thread) calls complete(), leave the cleanup of this |
| 445 | * that thread. | 445 | * structure to that thread. |
| 446 | */ | 446 | */ |
| 447 | if (xchg(&create->done, NULL)) | 447 | if (xchg(&create->done, NULL)) |
| 448 | return ERR_PTR(-EINTR); | 448 | return ERR_PTR(-EINTR); |
| @@ -876,7 +876,7 @@ fail_task: · 427 unmodified lines | |||
| 876 | * | 876 | * |
| 877 | * Returns a pointer to the allocated worker on success, ERR_PTR(-ENOMEM) | 877 | * Returns a pointer to the allocated worker on success, ERR_PTR(-ENOMEM) |
| 878 | * when the needed structures could not get allocated, and ERR_PTR(-EINTR) | 878 | * when the needed structures could not get allocated, and ERR_PTR(-EINTR) |
| 879 | * when the worker was SIGKILLed. | 879 | * when the caller was killed by a fatal signal. |
| 880 | */ | 880 | */ |
| 881 | struct kthread_worker * | 881 | struct kthread_worker * |
| 882 | kthread_create_worker(unsigned int flags, const char namefmt[], ...) | 882 | kthread_create_worker(unsigned int flags, const char namefmt[], ...) |
| @@ -925,7 +925,7 @@ EXPORT_SYMBOL(kthread_create_worker); · 42 unmodified lines | |||
| 925 | * Return: | 925 | * Return: |
| 926 | * The pointer to the allocated worker on success, ERR_PTR(-ENOMEM) | 926 | * The pointer to the allocated worker on success, ERR_PTR(-ENOMEM) |
| 927 | * when the needed structures could not get allocated, and ERR_PTR(-EINTR) | 927 | * when the needed structures could not get allocated, and ERR_PTR(-EINTR) |
| 928 | * when the worker was SIGKILLed. | 928 | * when the caller was killed by a fatal signal. |
| 929 | */ | 929 | */ |
| 930 | struct kthread_worker * | 930 | struct kthread_worker * |
| 931 | kthread_create_worker_on_cpu(int cpu, unsigned int flags, | 931 | kthread_create_worker_on_cpu(int cpu, unsigned int flags, |
| @@ -374,6 +374,8 @@ static void damon_reclaim_timer_fn(struct work_struct *work) · 373 unmodified lines | |||
| 374 | } | 374 | } |
| 375 | static DECLARE_DELAYED_WORK(damon_reclaim_timer, damon_reclaim_timer_fn); | 375 | static DECLARE_DELAYED_WORK(damon_reclaim_timer, damon_reclaim_timer_fn); |
| 376 | 376 | ||
| 377 | static bool damon_reclaim_initialized; | ||
| 378 | |||
| 377 | static int enabled_store(const char *val, | 379 | static int enabled_store(const char *val, |
| 378 | const struct kernel_param *kp) | 380 | const struct kernel_param *kp) |
| 379 | { | 381 | { |
| @@ -382,6 +384,10 @@ static int enabled_store(const char *val, · 2 unmodified lines | |||
| 382 | if (rc < 0) | 384 | if (rc < 0) |
| 383 | return rc; | 385 | return rc; |
| 384 | 386 | ||
| 387 | /* system_wq might not initialized yet */ | ||
| 388 | if (!damon_reclaim_initialized) | ||
| 389 | return rc; | ||
| 390 | |||
| 385 | if (enabled) | 391 | if (enabled) |
| 386 | schedule_delayed_work(&damon_reclaim_timer, 0); | 392 | schedule_delayed_work(&damon_reclaim_timer, 0); |
| 387 | 393 | ||
| @@ -449,6 +455,8 @@ static int __init damon_reclaim_init(void) · 61 unmodified lines | |||
| 449 | damon_add_target(ctx, target); | 455 | damon_add_target(ctx, target); |
| 450 | 456 | ||
| 451 | schedule_delayed_work(&damon_reclaim_timer, 0); | 457 | schedule_delayed_work(&damon_reclaim_timer, 0); |
| 458 | |||
| 459 | damon_reclaim_initialized = true; | ||
| 452 | return 0; | 460 | return 0; |
| 453 | } | 461 | } |
| 454 | 462 | ||
| @@ -48,7 +48,7 @@ static int hwpoison_inject(void *data, u64 val) · 47 unmodified lines | |||
| 48 | 48 | ||
| 49 | inject: | 49 | inject: |
| 50 | pr_info("Injecting memory failure at pfn %#lx\n", pfn); | 50 | pr_info("Injecting memory failure at pfn %#lx\n", pfn); |
| 51 | err = memory_failure(pfn, 0); | 51 | err = memory_failure(pfn, MF_SW_SIMULATED); |
| 52 | return (err == -EOPNOTSUPP) ? 0 : err; | 52 | return (err == -EOPNOTSUPP) ? 0 : err; |
| 53 | } | 53 | } |
| 54 | 54 | ||
| @@ -360,6 +360,9 @@ static void *kfence_guarded_alloc(struct kmem_cache *cache, size_t size, gfp_t g · 359 unmodified lines | |||
| 360 | unsigned long flags; | 360 | unsigned long flags; |
| 361 | struct slab *slab; | 361 | struct slab *slab; |
| 362 | void *addr; | 362 | void *addr; |
| 363 | const bool random_right_allocate = prandom_u32_max(2); | ||
| 364 | const bool random_fault = CONFIG_KFENCE_STRESS_TEST_FAULTS && | ||
| 365 | !prandom_u32_max(CONFIG_KFENCE_STRESS_TEST_FAULTS); | ||
| 363 | 366 | ||
| 364 | /* Try to obtain a free object. */ | 367 | /* Try to obtain a free object. */ |
| 365 | raw_spin_lock_irqsave(&kfence_freelist_lock, flags); | 368 | raw_spin_lock_irqsave(&kfence_freelist_lock, flags); |
| @@ -404,7 +407,7 @@ static void *kfence_guarded_alloc(struct kmem_cache *cache, size_t size, gfp_t g · 38 unmodified lines | |||
| 404 | * is that the out-of-bounds accesses detected are deterministic for | 407 | * is that the out-of-bounds accesses detected are deterministic for |
| 405 | * such allocations. | 408 | * such allocations. |
| 406 | */ | 409 | */ |
| 407 | if (prandom_u32_max(2)) { | 410 | if (random_right_allocate) { |
| 408 | /* Allocate on the "right" side, re-calculate address. */ | 411 | /* Allocate on the "right" side, re-calculate address. */ |
| 409 | meta->addr += PAGE_SIZE - size; | 412 | meta->addr += PAGE_SIZE - size; |
| 410 | meta->addr = ALIGN_DOWN(meta->addr, cache->align); | 413 | meta->addr = ALIGN_DOWN(meta->addr, cache->align); |
| @@ -444,7 +447,7 @@ static void *kfence_guarded_alloc(struct kmem_cache *cache, size_t size, gfp_t g · 33 unmodified lines | |||
| 444 | if (cache->ctor) | 447 | if (cache->ctor) |
| 445 | cache->ctor(addr); | 448 | cache->ctor(addr); |
| 446 | 449 | ||
| 447 | if (CONFIG_KFENCE_STRESS_TEST_FAULTS && !prandom_u32_max(CONFIG_KFENCE_STRESS_TEST_FAULTS)) | 450 | if (random_fault) |
| 448 | kfence_protect(meta->addr); /* Random "faults" by protecting the object. */ | 451 | kfence_protect(meta->addr); /* Random "faults" by protecting the object. */ |
| 449 | 452 | ||
| 450 | atomic_long_inc(&counters[KFENCE_COUNTER_ALLOCATED]); | 453 | atomic_long_inc(&counters[KFENCE_COUNTER_ALLOCATED]); |
| @@ -1112,7 +1112,7 @@ static int madvise_inject_error(int behavior, · 1111 unmodified lines | |||
| 1112 | } else { | 1112 | } else { |
| 1113 | pr_info("Injecting memory failure for pfn %#lx at process virtual address %#lx\n", | 1113 | pr_info("Injecting memory failure for pfn %#lx at process virtual address %#lx\n", |
| 1114 | pfn, start); | 1114 | pfn, start); |
| 1115 | ret = memory_failure(pfn, MF_COUNT_INCREASED); | 1115 | ret = memory_failure(pfn, MF_COUNT_INCREASED | MF_SW_SIMULATED); |
| 1116 | if (ret == -EOPNOTSUPP) | 1116 | if (ret == -EOPNOTSUPP) |
| 1117 | ret = 0; | 1117 | ret = 0; |
| 1118 | } | 1118 | } |
| @@ -4859,7 +4859,7 @@ static int mem_cgroup_slab_show(struct seq_file *m, void *p) · 4858 unmodified lines | |||
| 4859 | { | 4859 | { |
| 4860 | /* | 4860 | /* |
| 4861 | * Deprecated. | 4861 | * Deprecated. |
| 4862 | * Please, take a look at tools/cgroup/slabinfo.py . | 4862 | * Please, take a look at tools/cgroup/memcg_slabinfo.py . |
| 4863 | */ | 4863 | */ |
| 4864 | return 0; | 4864 | return 0; |
| 4865 | } | 4865 | } |
| @@ -69,6 +69,8 @@ int sysctl_memory_failure_recovery __read_mostly = 1; · 68 unmodified lines | |||
| 69 | 69 | ||
| 70 | atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); | 70 | atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); |
| 71 | 71 | ||
| 72 | static bool hw_memory_failure __read_mostly = false; | ||
| 73 | |||
| 72 | static bool __page_handle_poison(struct page *page) | 74 | static bool __page_handle_poison(struct page *page) |
| 73 | { | 75 | { |
| 74 | int ret; | 76 | int ret; |
| @@ -1768,6 +1770,9 @@ int memory_failure(unsigned long pfn, int flags) · 1693 unmodified lines | |||
| 1768 | 1770 | ||
| 1769 | mutex_lock(&mf_mutex); | 1771 | mutex_lock(&mf_mutex); |
| 1770 | 1772 | ||
| 1773 | if (!(flags & MF_SW_SIMULATED)) | ||
| 1774 | hw_memory_failure = true; | ||
| 1775 | |||
| 1771 | p = pfn_to_online_page(pfn); | 1776 | p = pfn_to_online_page(pfn); |
| 1772 | if (!p) { | 1777 | if (!p) { |
| 1773 | res = arch_memory_failure(pfn, flags); | 1778 | res = arch_memory_failure(pfn, flags); |
| @@ -2103,6 +2108,13 @@ int unpoison_memory(unsigned long pfn) · 329 unmodified lines | |||
| 2103 | 2108 | ||
| 2104 | mutex_lock(&mf_mutex); | 2109 | mutex_lock(&mf_mutex); |
| 2105 | 2110 | ||
| 2111 | if (hw_memory_failure) { | ||
| 2112 | unpoison_pr_info("Unpoison: Disabled after HW memory failure %#lx\n", | ||
| 2113 | pfn, &unpoison_rs); | ||
| 2114 | ret = -EOPNOTSUPP; | ||
| 2115 | goto unlock_mutex; | ||
| 2116 | } | ||
| 2117 | |||
| 2106 | if (!PageHWPoison(p)) { | 2118 | if (!PageHWPoison(p)) { |
| 2107 | unpoison_pr_info("Unpoison: Page was already unpoisoned %#lx\n", | 2119 | unpoison_pr_info("Unpoison: Page was already unpoisoned %#lx\n", |
| 2108 | pfn, &unpoison_rs); | 2120 | pfn, &unpoison_rs); |
| @@ -286,6 +286,8 @@ __first_valid_page(unsigned long pfn, unsigned long nr_pages) · 285 unmodified lines | |||
| 286 | * @flags: isolation flags | 286 | * @flags: isolation flags |
| 287 | * @gfp_flags: GFP flags used for migrating pages | 287 | * @gfp_flags: GFP flags used for migrating pages |
| 288 | * @isolate_before: isolate the pageblock before the boundary_pfn | 288 | * @isolate_before: isolate the pageblock before the boundary_pfn |
| 289 | * @skip_isolation: the flag to skip the pageblock isolation in second | ||
| 290 | * isolate_single_pageblock() | ||
| 289 | * | 291 | * |
| 290 | * Free and in-use pages can be as big as MAX_ORDER-1 and contain more than one | 292 | * Free and in-use pages can be as big as MAX_ORDER-1 and contain more than one |
| 291 | * pageblock. When not all pageblocks within a page are isolated at the same | 293 | * pageblock. When not all pageblocks within a page are isolated at the same |
| @@ -881,7 +881,7 @@ void lru_cache_disable(void) · 880 unmodified lines | |||
| 881 | * lru_disable_count = 0 will have exited the critical | 881 | * lru_disable_count = 0 will have exited the critical |
| 882 | * section when synchronize_rcu() returns. | 882 | * section when synchronize_rcu() returns. |
| 883 | */ | 883 | */ |
| 884 | synchronize_rcu(); | 884 | synchronize_rcu_expedited(); |
| 885 | #ifdef CONFIG_SMP | 885 | #ifdef CONFIG_SMP |
| 886 | __lru_add_drain_all(true); | 886 | __lru_add_drain_all(true); |
| 887 | #else | 887 | #else |