History

Merge tag 'f2fs-for-6.12-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs

Pull f2fs updates from Jaegeuk Kim: "The main changes include converting major IO paths to use folio, and adding various knobs to control GC more flexibly for Zoned devices. In addition, there are several patches to address corner cases of atomic file operations and better support for file pinning on zoned device. Enhancement: - add knobs to tune foreground/background GCs for Zoned devices - convert IO paths to use folio - reduce expensive checkpoint trigger frequency - allow F2FS_IPU_NOCACHE for pinned file - forcibly migrate to secure space for zoned device file pinning - get rid of buffer_head use - add write priority option based on zone UFS - get rid of online repair on corrupted directory Bug fixes: - fix to don't panic system for no free segment fault injection - fix to don't set SB_RDONLY in f2fs_handle_critical_error() - avoid unused block when dio write in LFS mode - compress: don't redirty sparse cluster during {,de}compress - check discard support for conventional zones - atomic: prevent atomic file from being dirtied before commit - atomic: fix to check atomic_file in f2fs ioctl interfaces - atomic: fix to forbid dio in atomic_file - atomic: fix to truncate pagecache before on-disk metadata truncation - atomic: create COW inode from parent dentry - atomic: fix to avoid racing w/ GC - atomic: require FMODE_WRITE for atomic write ioctls - fix to wait page writeback before setting gcing flag - fix to avoid racing in between read and OPU dio write, dio completion - fix several potential integer overflows in file offsets and dir_block_index - fix to avoid use-after-free in f2fs_stop_gc_thread() As usual, there are several code clean-ups and refactorings" * tag 'f2fs-for-6.12-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs: (60 commits) f2fs: allow F2FS_IPU_NOCACHE for pinned file f2fs: forcibly migrate to secure space for zoned device file pinning f2fs: remove unused parameters f2fs: fix to don't panic system for no free segment fault injection f2fs: fix to don't set SB_RDONLY in f2fs_handle_critical_error() f2fs: add valid block ratio not to do excessive GC for one time GC f2fs: create gc_no_zoned_gc_percent and gc_boost_zoned_gc_percent f2fs: do FG_GC when GC boosting is required for zoned devices f2fs: increase BG GC migration window granularity when boosted for zoned devices f2fs: add reserved_segments sysfs node f2fs: introduce migration_window_granularity f2fs: make BG GC more aggressive for zoned devices f2fs: avoid unused block when dio write in LFS mode f2fs: fix to check atomic_file in f2fs ioctl interfaces f2fs: get rid of online repaire on corrupted directory f2fs: prevent atomic file from being dirtied before commit f2fs: get rid of page->index f2fs: convert read_node_page() to use folio f2fs: convert __write_node_page() to use folio f2fs: convert f2fs_write_data_page() to use folio ...

Linus Torvalds committed 2 years ago

Commit 79952bd2 parents fa8380a ae87b9c23 files · +798 −463Browse files

Unified · Split

23 files
  1. Documentation/ABI/testing/sysfs-fs-f2fs +56 −0
  2. fs/f2fs/checkpoint.c +9 −8
  3. fs/f2fs/compress.c +43 −20
  4. fs/f2fs/data.c +92 −72
  5. fs/f2fs/debug.c +1 −1
  6. fs/f2fs/dir.c +5 −3
  7. fs/f2fs/extent_cache.c +2 −2
  8. fs/f2fs/f2fs.h +91 −57
  9. fs/f2fs/file.c +129 −70
  10. fs/f2fs/gc.c +86 −27
  11. fs/f2fs/gc.h +29 −0
  12. fs/f2fs/inline.c +15 −16
  13. fs/f2fs/inode.c +7 −2
  14. fs/f2fs/namei.c +0 −68
  15. fs/f2fs/node.c +24 −22
  16. fs/f2fs/segment.c +56 −16
  17. fs/f2fs/segment.h +3 −2
  18. fs/f2fs/super.c +74 −45
  19. fs/f2fs/sysfs.c +57 −25
  20. fs/f2fs/verity.c +3 −2
  21. fs/f2fs/xattr.c +12 −2
  22. include/linux/f2fs_fs.h +2 −2
  23. include/trace/events/f2fs.h +2 −1
Documentation/ABI/testing/sysfs-fs-f2fs+56−0
@@ -579,6 +579,12 @@ Description: When ATGC is on, it controls age threshold to bypass GCing young · 578 unmodified lines
579 candidates whose age is not beyond the threshold, by default it was579 candidates whose age is not beyond the threshold, by default it was
580 initialized as 604800 seconds (equals to 7 days).580 initialized as 604800 seconds (equals to 7 days).
581 581
582What: /sys/fs/f2fs/<disk>/atgc_enabled
583Date: Feb 2024
584Contact: "Jinbao Liu" <liujinbao1@xiaomi.com>
585Description: It represents whether ATGC is on or off. The value is 1 which
586 indicates that ATGC is on, and 0 indicates that it is off.
587
582What: /sys/fs/f2fs/<disk>/gc_reclaimed_segments588What: /sys/fs/f2fs/<disk>/gc_reclaimed_segments
583Date: July 2021589Date: July 2021
584Contact: "Daeho Jeong" <daehojeong@google.com>590Contact: "Daeho Jeong" <daehojeong@google.com>
@@ -763,3 +769,53 @@ Date: November 2023 · 178 unmodified lines
763Contact: "Chao Yu" <chao@kernel.org>769Contact: "Chao Yu" <chao@kernel.org>
764Description: It controls to enable/disable IO aware feature for background discard.770Description: It controls to enable/disable IO aware feature for background discard.
765 By default, the value is 1 which indicates IO aware is on.771 By default, the value is 1 which indicates IO aware is on.
772
773What: /sys/fs/f2fs/<disk>/blkzone_alloc_policy
774Date: July 2024
775Contact: "Yuanhong Liao" <liaoyuanhong@vivo.com>
776Description: The zone UFS we are currently using consists of two parts:
777 conventional zones and sequential zones. It can be used to control which part
778 to prioritize for writes, with a default value of 0.
779
780 ======================== =========================================
781 value description
782 blkzone_alloc_policy = 0 Prioritize writing to sequential zones
783 blkzone_alloc_policy = 1 Only allow writing to sequential zones
784 blkzone_alloc_policy = 2 Prioritize writing to conventional zones
785 ======================== =========================================
786
787What: /sys/fs/f2fs/<disk>/migration_window_granularity
788Date: September 2024
789Contact: "Daeho Jeong" <daehojeong@google.com>
790Description: Controls migration window granularity of garbage collection on large
791 section. it can control the scanning window granularity for GC migration
792 in a unit of segment, while migration_granularity controls the number
793 of segments which can be migrated at the same turn.
794
795What: /sys/fs/f2fs/<disk>/reserved_segments
796Date: September 2024
797Contact: "Daeho Jeong" <daehojeong@google.com>
798Description: In order to fine tune GC behavior, we can control the number of
799 reserved segments.
800
801What: /sys/fs/f2fs/<disk>/gc_no_zoned_gc_percent
802Date: September 2024
803Contact: "Daeho Jeong" <daehojeong@google.com>
804Description: If the percentage of free sections over total sections is above this
805 number, F2FS do not garbage collection for zoned devices through the
806 background GC thread. the default number is "60".
807
808What: /sys/fs/f2fs/<disk>/gc_boost_zoned_gc_percent
809Date: September 2024
810Contact: "Daeho Jeong" <daehojeong@google.com>
811Description: If the percentage of free sections over total sections is under this
812 number, F2FS boosts garbage collection for zoned devices through the
813 background GC thread. the default number is "25".
814
815What: /sys/fs/f2fs/<disk>/gc_valid_thresh_ratio
816Date: September 2024
817Contact: "Daeho Jeong" <daehojeong@google.com>
818Description: It controls the valid block ratio threshold not to trigger excessive GC
819 for zoned deivces. The initial value of it is 95(%). F2FS will stop the
820 background GC thread from intiating GC for sections having valid blocks
821 exceeding the ratio.
fs/f2fs/checkpoint.c+9−8
@@ -99,7 +99,7 @@ repeat: · 98 unmodified lines
99 }99 }
100 100
101 if (unlikely(!PageUptodate(page))) {101 if (unlikely(!PageUptodate(page))) {
102 f2fs_handle_page_eio(sbi, page->index, META);102 f2fs_handle_page_eio(sbi, page_folio(page), META);
103 f2fs_put_page(page, 1);103 f2fs_put_page(page, 1);
104 return ERR_PTR(-EIO);104 return ERR_PTR(-EIO);
105 }105 }
@@ -345,30 +345,31 @@ static int __f2fs_write_meta_page(struct page *page, · 239 unmodified lines
345 enum iostat_type io_type)345 enum iostat_type io_type)
346{346{
347 struct f2fs_sb_info *sbi = F2FS_P_SB(page);347 struct f2fs_sb_info *sbi = F2FS_P_SB(page);
348 struct folio *folio = page_folio(page);
348 349
349 trace_f2fs_writepage(page_folio(page), META);350 trace_f2fs_writepage(folio, META);
350 351
351 if (unlikely(f2fs_cp_error(sbi))) {352 if (unlikely(f2fs_cp_error(sbi))) {
352 if (is_sbi_flag_set(sbi, SBI_IS_CLOSE)) {353 if (is_sbi_flag_set(sbi, SBI_IS_CLOSE)) {
353 ClearPageUptodate(page);354 folio_clear_uptodate(folio);
354 dec_page_count(sbi, F2FS_DIRTY_META);355 dec_page_count(sbi, F2FS_DIRTY_META);
355 unlock_page(page);356 folio_unlock(folio);
356 return 0;357 return 0;
357 }358 }
358 goto redirty_out;359 goto redirty_out;
359 }360 }
360 if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING)))361 if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING)))
361 goto redirty_out;362 goto redirty_out;
362 if (wbc->for_reclaim && page->index < GET_SUM_BLOCK(sbi, 0))363 if (wbc->for_reclaim && folio->index < GET_SUM_BLOCK(sbi, 0))
363 goto redirty_out;364 goto redirty_out;
364 365
365 f2fs_do_write_meta_page(sbi, page, io_type);366 f2fs_do_write_meta_page(sbi, folio, io_type);
366 dec_page_count(sbi, F2FS_DIRTY_META);367 dec_page_count(sbi, F2FS_DIRTY_META);
367 368
368 if (wbc->for_reclaim)369 if (wbc->for_reclaim)
369 f2fs_submit_merged_write_cond(sbi, NULL, page, 0, META);370 f2fs_submit_merged_write_cond(sbi, NULL, page, 0, META);
370 371
371 unlock_page(page);372 folio_unlock(folio);
372 373
373 if (unlikely(f2fs_cp_error(sbi)))374 if (unlikely(f2fs_cp_error(sbi)))
374 f2fs_submit_merged_write(sbi, META);375 f2fs_submit_merged_write(sbi, META);
@@ -1551,7 +1552,7 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc) · 1176 unmodified lines
1551 blk = start_blk + BLKS_PER_SEG(sbi) - nm_i->nat_bits_blocks;1552 blk = start_blk + BLKS_PER_SEG(sbi) - nm_i->nat_bits_blocks;
1552 for (i = 0; i < nm_i->nat_bits_blocks; i++)1553 for (i = 0; i < nm_i->nat_bits_blocks; i++)
1553 f2fs_update_meta_page(sbi, nm_i->nat_bits +1554 f2fs_update_meta_page(sbi, nm_i->nat_bits +
1554 (i << F2FS_BLKSIZE_BITS), blk + i);1555 F2FS_BLK_TO_BYTES(i), blk + i);
1555 }1556 }
1556 1557
1557 /* write out checkpoint buffer at block 0 */1558 /* write out checkpoint buffer at block 0 */
fs/f2fs/compress.c+43−20
@@ -90,11 +90,13 @@ bool f2fs_is_compressed_page(struct page *page) · 89 unmodified lines
90static void f2fs_set_compressed_page(struct page *page,90static void f2fs_set_compressed_page(struct page *page,
91 struct inode *inode, pgoff_t index, void *data)91 struct inode *inode, pgoff_t index, void *data)
92{92{
93 attach_page_private(page, (void *)data);93 struct folio *folio = page_folio(page);
94
95 folio_attach_private(folio, (void *)data);
94 96
95 /* i_crypto_info and iv index */97 /* i_crypto_info and iv index */
96 page->index = index;98 folio->index = index;
97 page->mapping = inode->i_mapping;99 folio->mapping = inode->i_mapping;
98}100}
99 101
100static void f2fs_drop_rpages(struct compress_ctx *cc, int len, bool unlock)102static void f2fs_drop_rpages(struct compress_ctx *cc, int len, bool unlock)
@@ -160,17 +162,17 @@ void f2fs_destroy_compress_ctx(struct compress_ctx *cc, bool reuse) · 59 unmodified lines
160 cc->cluster_idx = NULL_CLUSTER;162 cc->cluster_idx = NULL_CLUSTER;
161}163}
162 164
163void f2fs_compress_ctx_add_page(struct compress_ctx *cc, struct page *page)165void f2fs_compress_ctx_add_page(struct compress_ctx *cc, struct folio *folio)
164{166{
165 unsigned int cluster_ofs;167 unsigned int cluster_ofs;
166 168
167 if (!f2fs_cluster_can_merge_page(cc, page->index))169 if (!f2fs_cluster_can_merge_page(cc, folio->index))
168 f2fs_bug_on(F2FS_I_SB(cc->inode), 1);170 f2fs_bug_on(F2FS_I_SB(cc->inode), 1);
169 171
170 cluster_ofs = offset_in_cluster(cc, page->index);172 cluster_ofs = offset_in_cluster(cc, folio->index);
171 cc->rpages[cluster_ofs] = page;173 cc->rpages[cluster_ofs] = folio_page(folio, 0);
172 cc->nr_rpages++;174 cc->nr_rpages++;
173 cc->cluster_idx = cluster_idx(cc, page->index);175 cc->cluster_idx = cluster_idx(cc, folio->index);
174}176}
175 177
176#ifdef CONFIG_F2FS_FS_LZO178#ifdef CONFIG_F2FS_FS_LZO
@@ -879,7 +881,7 @@ static bool cluster_has_invalid_data(struct compress_ctx *cc) · 702 unmodified lines
879 f2fs_bug_on(F2FS_I_SB(cc->inode), !page);881 f2fs_bug_on(F2FS_I_SB(cc->inode), !page);
880 882
881 /* beyond EOF */883 /* beyond EOF */
882 if (page->index >= nr_pages)884 if (page_folio(page)->index >= nr_pages)
883 return true;885 return true;
884 }886 }
885 return false;887 return false;
@@ -945,7 +947,7 @@ static int __f2fs_get_cluster_blocks(struct inode *inode, · 59 unmodified lines
945 unsigned int cluster_size = F2FS_I(inode)->i_cluster_size;947 unsigned int cluster_size = F2FS_I(inode)->i_cluster_size;
946 int count, i;948 int count, i;
947 949
948 for (i = 1, count = 1; i < cluster_size; i++) {950 for (i = 0, count = 0; i < cluster_size; i++) {
949 block_t blkaddr = data_blkaddr(dn->inode, dn->node_page,951 block_t blkaddr = data_blkaddr(dn->inode, dn->node_page,
950 dn->ofs_in_node + i);952 dn->ofs_in_node + i);
951 953
@@ -956,8 +958,8 @@ static int __f2fs_get_cluster_blocks(struct inode *inode, · 4 unmodified lines
956 return count;958 return count;
957}959}
958 960
959static int __f2fs_cluster_blocks(struct inode *inode,961static int __f2fs_cluster_blocks(struct inode *inode, unsigned int cluster_idx,
960 unsigned int cluster_idx, bool compr_blks)962 enum cluster_check_type type)
961{963{
962 struct dnode_of_data dn;964 struct dnode_of_data dn;
963 unsigned int start_idx = cluster_idx <<965 unsigned int start_idx = cluster_idx <<
@@ -978,10 +980,12 @@ static int __f2fs_cluster_blocks(struct inode *inode, · 14 unmodified lines
978 }980 }
979 981
980 if (dn.data_blkaddr == COMPRESS_ADDR) {982 if (dn.data_blkaddr == COMPRESS_ADDR) {
981 if (compr_blks)983 if (type == CLUSTER_COMPR_BLKS)
982 ret = __f2fs_get_cluster_blocks(inode, &dn);984 ret = 1 + __f2fs_get_cluster_blocks(inode, &dn);
983 else985 else if (type == CLUSTER_IS_COMPR)
984 ret = 1;986 ret = 1;
987 } else if (type == CLUSTER_RAW_BLKS) {
988 ret = __f2fs_get_cluster_blocks(inode, &dn);
985 }989 }
986fail:990fail:
987 f2fs_put_dnode(&dn);991 f2fs_put_dnode(&dn);
@@ -991,7 +995,16 @@ fail: · 3 unmodified lines
991/* return # of compressed blocks in compressed cluster */995/* return # of compressed blocks in compressed cluster */
992static int f2fs_compressed_blocks(struct compress_ctx *cc)996static int f2fs_compressed_blocks(struct compress_ctx *cc)
993{997{
994 return __f2fs_cluster_blocks(cc->inode, cc->cluster_idx, true);998 return __f2fs_cluster_blocks(cc->inode, cc->cluster_idx,
999 CLUSTER_COMPR_BLKS);
1000}
1001
1002/* return # of raw blocks in non-compressed cluster */
1003static int f2fs_decompressed_blocks(struct inode *inode,
1004 unsigned int cluster_idx)
1005{
1006 return __f2fs_cluster_blocks(inode, cluster_idx,
1007 CLUSTER_RAW_BLKS);
995}1008}
996 1009
997/* return whether cluster is compressed one or not */1010/* return whether cluster is compressed one or not */
@@ -999,7 +1012,16 @@ int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index) · 1 unmodified line
999{1012{
1000 return __f2fs_cluster_blocks(inode,1013 return __f2fs_cluster_blocks(inode,
1001 index >> F2FS_I(inode)->i_log_cluster_size,1014 index >> F2FS_I(inode)->i_log_cluster_size,
1002 false);1015 CLUSTER_IS_COMPR);
1016}
1017
1018/* return whether cluster contains non raw blocks or not */
1019bool f2fs_is_sparse_cluster(struct inode *inode, pgoff_t index)
1020{
1021 unsigned int cluster_idx = index >> F2FS_I(inode)->i_log_cluster_size;
1022
1023 return f2fs_decompressed_blocks(inode, cluster_idx) !=
1024 F2FS_I(inode)->i_cluster_size;
1003}1025}
1004 1026
1005static bool cluster_may_compress(struct compress_ctx *cc)1027static bool cluster_may_compress(struct compress_ctx *cc)
@@ -1093,7 +1115,7 @@ retry: · 87 unmodified lines
1093 if (PageUptodate(page))1115 if (PageUptodate(page))
1094 f2fs_put_page(page, 1);1116 f2fs_put_page(page, 1);
1095 else1117 else
1096 f2fs_compress_ctx_add_page(cc, page);1118 f2fs_compress_ctx_add_page(cc, page_folio(page));
1097 }1119 }
1098 1120
1099 if (!f2fs_cluster_is_empty(cc)) {1121 if (!f2fs_cluster_is_empty(cc)) {
@@ -1123,7 +1145,7 @@ retry: · 23 unmodified lines
1123 }1145 }
1124 1146
1125 f2fs_wait_on_page_writeback(page, DATA, true, true);1147 f2fs_wait_on_page_writeback(page, DATA, true, true);
1126 f2fs_compress_ctx_add_page(cc, page);1148 f2fs_compress_ctx_add_page(cc, page_folio(page));
1127 1149
1128 if (!PageUptodate(page)) {1150 if (!PageUptodate(page)) {
1129release_and_retry:1151release_and_retry:
@@ -1523,7 +1545,8 @@ continue_unlock: · 393 unmodified lines
1523 if (!clear_page_dirty_for_io(cc->rpages[i]))1545 if (!clear_page_dirty_for_io(cc->rpages[i]))
1524 goto continue_unlock;1546 goto continue_unlock;
1525 1547
1526 ret = f2fs_write_single_data_page(cc->rpages[i], &submitted,1548 ret = f2fs_write_single_data_page(page_folio(cc->rpages[i]),
1549 &submitted,
1527 NULL, NULL, wbc, io_type,1550 NULL, NULL, wbc, io_type,
1528 compr_blocks, false);1551 compr_blocks, false);
1529 if (ret) {1552 if (ret) {
fs/f2fs/data.c+92−72
@@ -7,7 +7,6 @@ · 6 unmodified lines
7 */7 */
8#include <linux/fs.h>8#include <linux/fs.h>
9#include <linux/f2fs_fs.h>9#include <linux/f2fs_fs.h>
10#include <linux/buffer_head.h>
11#include <linux/sched/mm.h>10#include <linux/sched/mm.h>
12#include <linux/mpage.h>11#include <linux/mpage.h>
13#include <linux/writeback.h>12#include <linux/writeback.h>
@@ -355,7 +354,7 @@ static void f2fs_write_end_io(struct bio *bio) · 341 unmodified lines
355 }354 }
356 355
357 f2fs_bug_on(sbi, page->mapping == NODE_MAPPING(sbi) &&356 f2fs_bug_on(sbi, page->mapping == NODE_MAPPING(sbi) &&
358 page->index != nid_of_node(page));357 page_folio(page)->index != nid_of_node(page));
359 358
360 dec_page_count(sbi, type);359 dec_page_count(sbi, type);
361 if (f2fs_in_warm_node_list(sbi, page))360 if (f2fs_in_warm_node_list(sbi, page))
@@ -704,7 +703,7 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio) · 342 unmodified lines
704 bio = __bio_alloc(fio, 1);703 bio = __bio_alloc(fio, 1);
705 704
706 f2fs_set_bio_crypt_ctx(bio, fio->page->mapping->host,705 f2fs_set_bio_crypt_ctx(bio, fio->page->mapping->host,
707 fio->page->index, fio, GFP_NOIO);706 page_folio(fio->page)->index, fio, GFP_NOIO);
708 707
709 if (bio_add_page(bio, page, PAGE_SIZE, 0) < PAGE_SIZE) {708 if (bio_add_page(bio, page, PAGE_SIZE, 0) < PAGE_SIZE) {
710 bio_put(bio);709 bio_put(bio);
@@ -803,7 +802,7 @@ static int add_ipu_page(struct f2fs_io_info *fio, struct bio **bio, · 92 unmodified lines
803 fio->new_blkaddr));802 fio->new_blkaddr));
804 if (f2fs_crypt_mergeable_bio(*bio,803 if (f2fs_crypt_mergeable_bio(*bio,
805 fio->page->mapping->host,804 fio->page->mapping->host,
806 fio->page->index, fio) &&805 page_folio(fio->page)->index, fio) &&
807 bio_add_page(*bio, page, PAGE_SIZE, 0) ==806 bio_add_page(*bio, page, PAGE_SIZE, 0) ==
808 PAGE_SIZE) {807 PAGE_SIZE) {
809 ret = 0;808 ret = 0;
@@ -903,7 +902,7 @@ alloc_new: · 93 unmodified lines
903 if (!bio) {902 if (!bio) {
904 bio = __bio_alloc(fio, BIO_MAX_VECS);903 bio = __bio_alloc(fio, BIO_MAX_VECS);
905 f2fs_set_bio_crypt_ctx(bio, fio->page->mapping->host,904 f2fs_set_bio_crypt_ctx(bio, fio->page->mapping->host,
906 fio->page->index, fio, GFP_NOIO);905 page_folio(fio->page)->index, fio, GFP_NOIO);
907 906
908 add_bio_entry(fio->sbi, bio, page, fio->temp);907 add_bio_entry(fio->sbi, bio, page, fio->temp);
909 } else {908 } else {
@@ -996,13 +995,13 @@ next: · 86 unmodified lines
996 (!io_is_mergeable(sbi, io->bio, io, fio, io->last_block_in_bio,995 (!io_is_mergeable(sbi, io->bio, io, fio, io->last_block_in_bio,
997 fio->new_blkaddr) ||996 fio->new_blkaddr) ||
998 !f2fs_crypt_mergeable_bio(io->bio, fio->page->mapping->host,997 !f2fs_crypt_mergeable_bio(io->bio, fio->page->mapping->host,
999 bio_page->index, fio)))998 page_folio(bio_page)->index, fio)))
1000 __submit_merged_bio(io);999 __submit_merged_bio(io);
1001alloc_new:1000alloc_new:
1002 if (io->bio == NULL) {1001 if (io->bio == NULL) {
1003 io->bio = __bio_alloc(fio, BIO_MAX_VECS);1002 io->bio = __bio_alloc(fio, BIO_MAX_VECS);
1004 f2fs_set_bio_crypt_ctx(io->bio, fio->page->mapping->host,1003 f2fs_set_bio_crypt_ctx(io->bio, fio->page->mapping->host,
1005 bio_page->index, fio, GFP_NOIO);1004 page_folio(bio_page)->index, fio, GFP_NOIO);
1006 io->fio = *fio;1005 io->fio = *fio;
1007 }1006 }
1008 1007
@@ -1087,7 +1086,7 @@ static struct bio *f2fs_grab_read_bio(struct inode *inode, block_t blkaddr, · 78 unmodified lines
1087}1086}
1088 1087
1089/* This can handle encryption stuffs */1088/* This can handle encryption stuffs */
1090static int f2fs_submit_page_read(struct inode *inode, struct page *page,1089static int f2fs_submit_page_read(struct inode *inode, struct folio *folio,
1091 block_t blkaddr, blk_opf_t op_flags,1090 block_t blkaddr, blk_opf_t op_flags,
1092 bool for_write)1091 bool for_write)
1093{1092{
@@ -1095,14 +1094,14 @@ static int f2fs_submit_page_read(struct inode *inode, struct page *page, · 1 unmodified line
1095 struct bio *bio;1094 struct bio *bio;
1096 1095
1097 bio = f2fs_grab_read_bio(inode, blkaddr, 1, op_flags,1096 bio = f2fs_grab_read_bio(inode, blkaddr, 1, op_flags,
1098 page->index, for_write);1097 folio->index, for_write);
1099 if (IS_ERR(bio))1098 if (IS_ERR(bio))
1100 return PTR_ERR(bio);1099 return PTR_ERR(bio);
1101 1100
1102 /* wait for GCed page writeback via META_MAPPING */1101 /* wait for GCed page writeback via META_MAPPING */
1103 f2fs_wait_on_block_writeback(inode, blkaddr);1102 f2fs_wait_on_block_writeback(inode, blkaddr);
1104 1103
1105 if (bio_add_page(bio, page, PAGE_SIZE, 0) < PAGE_SIZE) {1104 if (!bio_add_folio(bio, folio, PAGE_SIZE, 0)) {
1106 iostat_update_and_unbind_ctx(bio);1105 iostat_update_and_unbind_ctx(bio);
1107 if (bio->bi_private)1106 if (bio->bi_private)
1108 mempool_free(bio->bi_private, bio_post_read_ctx_pool);1107 mempool_free(bio->bi_private, bio_post_read_ctx_pool);
@@ -1270,7 +1269,7 @@ got_it: · 161 unmodified lines
1270 return page;1269 return page;
1271 }1270 }
1272 1271
1273 err = f2fs_submit_page_read(inode, page, dn.data_blkaddr,1272 err = f2fs_submit_page_read(inode, page_folio(page), dn.data_blkaddr,
1274 op_flags, for_write);1273 op_flags, for_write);
1275 if (err)1274 if (err)
1276 goto put_err;1275 goto put_err;
@@ -1713,6 +1712,14 @@ skip: · 436 unmodified lines
1713 dn.ofs_in_node = end_offset;1712 dn.ofs_in_node = end_offset;
1714 }1713 }
1715 1714
1715 if (flag == F2FS_GET_BLOCK_DIO && f2fs_lfs_mode(sbi) &&
1716 map->m_may_create) {
1717 /* the next block to be allocated may not be contiguous. */
1718 if (GET_SEGOFF_FROM_SEG0(sbi, blkaddr) % BLKS_PER_SEC(sbi) ==
1719 CAP_BLKS_PER_SEC(sbi) - 1)
1720 goto sync_out;
1721 }
1722
1716 if (pgofs >= end)1723 if (pgofs >= end)
1717 goto sync_out;1724 goto sync_out;
1718 else if (dn.ofs_in_node < end_offset)1725 else if (dn.ofs_in_node < end_offset)
@@ -1939,7 +1946,7 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, · 220 unmodified lines
1939 1946
1940 inode_lock_shared(inode);1947 inode_lock_shared(inode);
1941 1948
1942 maxbytes = max_file_blocks(inode) << F2FS_BLKSIZE_BITS;1949 maxbytes = F2FS_BLK_TO_BYTES(max_file_blocks(inode));
1943 if (start > maxbytes) {1950 if (start > maxbytes) {
1944 ret = -EFBIG;1951 ret = -EFBIG;
1945 goto out;1952 goto out;
@@ -2064,7 +2071,7 @@ out: · 118 unmodified lines
2064static inline loff_t f2fs_readpage_limit(struct inode *inode)2071static inline loff_t f2fs_readpage_limit(struct inode *inode)
2065{2072{
2066 if (IS_ENABLED(CONFIG_FS_VERITY) && IS_VERITY(inode))2073 if (IS_ENABLED(CONFIG_FS_VERITY) && IS_VERITY(inode))
2067 return inode->i_sb->s_maxbytes;2074 return F2FS_BLK_TO_BYTES(max_file_blocks(inode));
2068 2075
2069 return i_size_read(inode);2076 return i_size_read(inode);
2070}2077}
@@ -2208,19 +2215,22 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret, · 137 unmodified lines
2208 /* get rid of pages beyond EOF */2215 /* get rid of pages beyond EOF */
2209 for (i = 0; i < cc->cluster_size; i++) {2216 for (i = 0; i < cc->cluster_size; i++) {
2210 struct page *page = cc->rpages[i];2217 struct page *page = cc->rpages[i];
2218 struct folio *folio;
2211 2219
2212 if (!page)2220 if (!page)
2213 continue;2221 continue;
2214 if ((sector_t)page->index >= last_block_in_file) {2222
2215 zero_user_segment(page, 0, PAGE_SIZE);2223 folio = page_folio(page);
2216 if (!PageUptodate(page))2224 if ((sector_t)folio->index >= last_block_in_file) {
2217 SetPageUptodate(page);2225 folio_zero_segment(folio, 0, folio_size(folio));
2218 } else if (!PageUptodate(page)) {2226 if (!folio_test_uptodate(folio))
2227 folio_mark_uptodate(folio);
2228 } else if (!folio_test_uptodate(folio)) {
2219 continue;2229 continue;
2220 }2230 }
2221 unlock_page(page);2231 folio_unlock(folio);
2222 if (for_write)2232 if (for_write)
2223 put_page(page);2233 folio_put(folio);
2224 cc->rpages[i] = NULL;2234 cc->rpages[i] = NULL;
2225 cc->nr_rpages--;2235 cc->nr_rpages--;
2226 }2236 }
@@ -2280,7 +2290,7 @@ skip_reading_dnode: · 53 unmodified lines
2280 }2290 }
2281 2291
2282 for (i = 0; i < cc->nr_cpages; i++) {2292 for (i = 0; i < cc->nr_cpages; i++) {
2283 struct page *page = dic->cpages[i];2293 struct folio *folio = page_folio(dic->cpages[i]);
2284 block_t blkaddr;2294 block_t blkaddr;
2285 struct bio_post_read_ctx *ctx;2295 struct bio_post_read_ctx *ctx;
2286 2296
@@ -2290,7 +2300,8 @@ skip_reading_dnode: · 3 unmodified lines
2290 2300
2291 f2fs_wait_on_block_writeback(inode, blkaddr);2301 f2fs_wait_on_block_writeback(inode, blkaddr);
2292 2302
2293 if (f2fs_load_compressed_page(sbi, page, blkaddr)) {2303 if (f2fs_load_compressed_page(sbi, folio_page(folio, 0),
2304 blkaddr)) {
2294 if (atomic_dec_and_test(&dic->remaining_pages)) {2305 if (atomic_dec_and_test(&dic->remaining_pages)) {
2295 f2fs_decompress_cluster(dic, true);2306 f2fs_decompress_cluster(dic, true);
2296 break;2307 break;
@@ -2300,7 +2311,7 @@ skip_reading_dnode: · 3 unmodified lines
2300 2311
2301 if (bio && (!page_is_mergeable(sbi, bio,2312 if (bio && (!page_is_mergeable(sbi, bio,
2302 *last_block_in_bio, blkaddr) ||2313 *last_block_in_bio, blkaddr) ||
2303 !f2fs_crypt_mergeable_bio(bio, inode, page->index, NULL))) {2314 !f2fs_crypt_mergeable_bio(bio, inode, folio->index, NULL))) {
2304submit_and_realloc:2315submit_and_realloc:
2305 f2fs_submit_read_bio(sbi, bio, DATA);2316 f2fs_submit_read_bio(sbi, bio, DATA);
2306 bio = NULL;2317 bio = NULL;
@@ -2309,7 +2320,7 @@ submit_and_realloc: · 2 unmodified lines
2309 if (!bio) {2320 if (!bio) {
2310 bio = f2fs_grab_read_bio(inode, blkaddr, nr_pages,2321 bio = f2fs_grab_read_bio(inode, blkaddr, nr_pages,
2311 f2fs_ra_op_flags(rac),2322 f2fs_ra_op_flags(rac),
2312 page->index, for_write);2323 folio->index, for_write);
2313 if (IS_ERR(bio)) {2324 if (IS_ERR(bio)) {
2314 ret = PTR_ERR(bio);2325 ret = PTR_ERR(bio);
2315 f2fs_decompress_end_io(dic, ret, true);2326 f2fs_decompress_end_io(dic, ret, true);
@@ -2319,7 +2330,7 @@ submit_and_realloc: · 3 unmodified lines
2319 }2330 }
2320 }2331 }
2321 2332
2322 if (bio_add_page(bio, page, blocksize, 0) < blocksize)2333 if (!bio_add_folio(bio, folio, blocksize, 0))
2323 goto submit_and_realloc;2334 goto submit_and_realloc;
2324 2335
2325 ctx = get_post_read_ctx(bio);2336 ctx = get_post_read_ctx(bio);
@@ -2430,7 +2441,7 @@ static int f2fs_mpage_readpages(struct inode *inode, · 104 unmodified lines
2430 if (ret)2441 if (ret)
2431 goto set_error_page;2442 goto set_error_page;
2432 2443
2433 f2fs_compress_ctx_add_page(&cc, &folio->page);2444 f2fs_compress_ctx_add_page(&cc, folio);
2434 2445
2435 goto next_page;2446 goto next_page;
2436read_single_page:2447read_single_page:
@@ -2645,21 +2656,24 @@ static inline bool need_inplace_update(struct f2fs_io_info *fio) · 208 unmodified lines
2645 2656
2646int f2fs_do_write_data_page(struct f2fs_io_info *fio)2657int f2fs_do_write_data_page(struct f2fs_io_info *fio)
2647{2658{
2648 struct page *page = fio->page;2659 struct folio *folio = page_folio(fio->page);
2649 struct inode *inode = page->mapping->host;2660 struct inode *inode = folio->mapping->host;
2650 struct dnode_of_data dn;2661 struct dnode_of_data dn;
2651 struct node_info ni;2662 struct node_info ni;
2652 bool ipu_force = false;2663 bool ipu_force = false;
2664 bool atomic_commit;
2653 int err = 0;2665 int err = 0;
2654 2666
2655 /* Use COW inode to make dnode_of_data for atomic write */2667 /* Use COW inode to make dnode_of_data for atomic write */
2656 if (f2fs_is_atomic_file(inode))2668 atomic_commit = f2fs_is_atomic_file(inode) &&
2669 page_private_atomic(folio_page(folio, 0));
2670 if (atomic_commit)
2657 set_new_dnode(&dn, F2FS_I(inode)->cow_inode, NULL, NULL, 0);2671 set_new_dnode(&dn, F2FS_I(inode)->cow_inode, NULL, NULL, 0);
2658 else2672 else
2659 set_new_dnode(&dn, inode, NULL, NULL, 0);2673 set_new_dnode(&dn, inode, NULL, NULL, 0);
2660 2674
2661 if (need_inplace_update(fio) &&2675 if (need_inplace_update(fio) &&
2662 f2fs_lookup_read_extent_cache_block(inode, page->index,2676 f2fs_lookup_read_extent_cache_block(inode, folio->index,
2663 &fio->old_blkaddr)) {2677 &fio->old_blkaddr)) {
2664 if (!f2fs_is_valid_blkaddr(fio->sbi, fio->old_blkaddr,2678 if (!f2fs_is_valid_blkaddr(fio->sbi, fio->old_blkaddr,
2665 DATA_GENERIC_ENHANCE))2679 DATA_GENERIC_ENHANCE))
@@ -2674,7 +2688,7 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio) · 8 unmodified lines
2674 if (fio->need_lock == LOCK_REQ && !f2fs_trylock_op(fio->sbi))2688 if (fio->need_lock == LOCK_REQ && !f2fs_trylock_op(fio->sbi))
2675 return -EAGAIN;2689 return -EAGAIN;
2676 2690
2677 err = f2fs_get_dnode_of_data(&dn, page->index, LOOKUP_NODE);2691 err = f2fs_get_dnode_of_data(&dn, folio->index, LOOKUP_NODE);
2678 if (err)2692 if (err)
2679 goto out;2693 goto out;
2680 2694
@@ -2682,8 +2696,8 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio) · 1 unmodified line
2682 2696
2683 /* This page is already truncated */2697 /* This page is already truncated */
2684 if (fio->old_blkaddr == NULL_ADDR) {2698 if (fio->old_blkaddr == NULL_ADDR) {
2685 ClearPageUptodate(page);2699 folio_clear_uptodate(folio);
2686 clear_page_private_gcing(page);2700 clear_page_private_gcing(folio_page(folio, 0));
2687 goto out_writepage;2701 goto out_writepage;
2688 }2702 }
2689got_it:2703got_it:
@@ -2709,7 +2723,7 @@ got_it: · 19 unmodified lines
2709 if (err)2723 if (err)
2710 goto out_writepage;2724 goto out_writepage;
2711 2725
2712 set_page_writeback(page);2726 folio_start_writeback(folio);
2713 f2fs_put_dnode(&dn);2727 f2fs_put_dnode(&dn);
2714 if (fio->need_lock == LOCK_REQ)2728 if (fio->need_lock == LOCK_REQ)
2715 f2fs_unlock_op(fio->sbi);2729 f2fs_unlock_op(fio->sbi);
@@ -2717,11 +2731,11 @@ got_it: · 1 unmodified line
2717 if (err) {2731 if (err) {
2718 if (fscrypt_inode_uses_fs_layer_crypto(inode))2732 if (fscrypt_inode_uses_fs_layer_crypto(inode))
2719 fscrypt_finalize_bounce_page(&fio->encrypted_page);2733 fscrypt_finalize_bounce_page(&fio->encrypted_page);
2720 end_page_writeback(page);2734 folio_end_writeback(folio);
2721 } else {2735 } else {
2722 set_inode_flag(inode, FI_UPDATE_WRITE);2736 set_inode_flag(inode, FI_UPDATE_WRITE);
2723 }2737 }
2724 trace_f2fs_do_write_data_page(page_folio(page), IPU);2738 trace_f2fs_do_write_data_page(folio, IPU);
2725 return err;2739 return err;
2726 }2740 }
2727 2741
@@ -2743,15 +2757,17 @@ got_it: · 15 unmodified lines
2743 if (err)2757 if (err)
2744 goto out_writepage;2758 goto out_writepage;
2745 2759
2746 set_page_writeback(page);2760 folio_start_writeback(folio);
2747 2761
2748 if (fio->compr_blocks && fio->old_blkaddr == COMPRESS_ADDR)2762 if (fio->compr_blocks && fio->old_blkaddr == COMPRESS_ADDR)
2749 f2fs_i_compr_blocks_update(inode, fio->compr_blocks - 1, false);2763 f2fs_i_compr_blocks_update(inode, fio->compr_blocks - 1, false);
2750 2764
2751 /* LFS mode write path */2765 /* LFS mode write path */
2752 f2fs_outplace_write_data(&dn, fio);2766 f2fs_outplace_write_data(&dn, fio);
2753 trace_f2fs_do_write_data_page(page_folio(page), OPU);2767 trace_f2fs_do_write_data_page(folio, OPU);
2754 set_inode_flag(inode, FI_APPEND_WRITE);2768 set_inode_flag(inode, FI_APPEND_WRITE);
2769 if (atomic_commit)
2770 clear_page_private_atomic(folio_page(folio, 0));
2755out_writepage:2771out_writepage:
2756 f2fs_put_dnode(&dn);2772 f2fs_put_dnode(&dn);
2757out:2773out:
@@ -2760,7 +2776,7 @@ out: · 2 unmodified lines
2760 return err;2776 return err;
2761}2777}
2762 2778
2763int f2fs_write_single_data_page(struct page *page, int *submitted,2779int f2fs_write_single_data_page(struct folio *folio, int *submitted,
2764 struct bio **bio,2780 struct bio **bio,
2765 sector_t *last_block,2781 sector_t *last_block,
2766 struct writeback_control *wbc,2782 struct writeback_control *wbc,
@@ -2768,12 +2784,13 @@ int f2fs_write_single_data_page(struct page *page, int *submitted, · 1 unmodified line
2768 int compr_blocks,2784 int compr_blocks,
2769 bool allow_balance)2785 bool allow_balance)
2770{2786{
2771 struct inode *inode = page->mapping->host;2787 struct inode *inode = folio->mapping->host;
2788 struct page *page = folio_page(folio, 0);
2772 struct f2fs_sb_info *sbi = F2FS_I_SB(inode);2789 struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
2773 loff_t i_size = i_size_read(inode);2790 loff_t i_size = i_size_read(inode);
2774 const pgoff_t end_index = ((unsigned long long)i_size)2791 const pgoff_t end_index = ((unsigned long long)i_size)
2775 >> PAGE_SHIFT;2792 >> PAGE_SHIFT;
2776 loff_t psize = (loff_t)(page->index + 1) << PAGE_SHIFT;2793 loff_t psize = (loff_t)(folio->index + 1) << PAGE_SHIFT;
2777 unsigned offset = 0;2794 unsigned offset = 0;
2778 bool need_balance_fs = false;2795 bool need_balance_fs = false;
2779 bool quota_inode = IS_NOQUOTA(inode);2796 bool quota_inode = IS_NOQUOTA(inode);
@@ -2797,11 +2814,11 @@ int f2fs_write_single_data_page(struct page *page, int *submitted, · 17 unmodified lines
2797 .last_block = last_block,2814 .last_block = last_block,
2798 };2815 };
2799 2816
2800 trace_f2fs_writepage(page_folio(page), DATA);2817 trace_f2fs_writepage(folio, DATA);
2801 2818
2802 /* we should bypass data pages to proceed the kworker jobs */2819 /* we should bypass data pages to proceed the kworker jobs */
2803 if (unlikely(f2fs_cp_error(sbi))) {2820 if (unlikely(f2fs_cp_error(sbi))) {
2804 mapping_set_error(page->mapping, -EIO);2821 mapping_set_error(folio->mapping, -EIO);
2805 /*2822 /*
2806 * don't drop any dirty dentry pages for keeping lastest2823 * don't drop any dirty dentry pages for keeping lastest
2807 * directory structure.2824 * directory structure.
@@ -2819,7 +2836,7 @@ int f2fs_write_single_data_page(struct page *page, int *submitted, · 11 unmodified lines
2819 if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING)))2836 if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING)))
2820 goto redirty_out;2837 goto redirty_out;
2821 2838
2822 if (page->index < end_index ||2839 if (folio->index < end_index ||
2823 f2fs_verity_in_progress(inode) ||2840 f2fs_verity_in_progress(inode) ||
2824 compr_blocks)2841 compr_blocks)
2825 goto write;2842 goto write;
@@ -2829,10 +2846,10 @@ int f2fs_write_single_data_page(struct page *page, int *submitted, · 3 unmodified lines
2829 * this page does not have to be written to disk.2846 * this page does not have to be written to disk.
2830 */2847 */
2831 offset = i_size & (PAGE_SIZE - 1);2848 offset = i_size & (PAGE_SIZE - 1);
2832 if ((page->index >= end_index + 1) || !offset)2849 if ((folio->index >= end_index + 1) || !offset)
2833 goto out;2850 goto out;
2834 2851
2835 zero_user_segment(page, offset, PAGE_SIZE);2852 folio_zero_segment(folio, offset, folio_size(folio));
2836write:2853write:
2837 /* Dentry/quota blocks are controlled by checkpoint */2854 /* Dentry/quota blocks are controlled by checkpoint */
2838 if (S_ISDIR(inode->i_mode) || quota_inode) {2855 if (S_ISDIR(inode->i_mode) || quota_inode) {
@@ -2862,7 +2879,7 @@ write: · 23 unmodified lines
2862 2879
2863 err = -EAGAIN;2880 err = -EAGAIN;
2864 if (f2fs_has_inline_data(inode)) {2881 if (f2fs_has_inline_data(inode)) {
2865 err = f2fs_write_inline_data(inode, page);2882 err = f2fs_write_inline_data(inode, folio);
2866 if (!err)2883 if (!err)
2867 goto out;2884 goto out;
2868 }2885 }
@@ -2892,7 +2909,7 @@ done: · 23 unmodified lines
2892out:2909out:
2893 inode_dec_dirty_pages(inode);2910 inode_dec_dirty_pages(inode);
2894 if (err) {2911 if (err) {
2895 ClearPageUptodate(page);2912 folio_clear_uptodate(folio);
2896 clear_page_private_gcing(page);2913 clear_page_private_gcing(page);
2897 }2914 }
2898 2915
@@ -2902,7 +2919,7 @@ out: · 3 unmodified lines
2902 f2fs_remove_dirty_inode(inode);2919 f2fs_remove_dirty_inode(inode);
2903 submitted = NULL;2920 submitted = NULL;
2904 }2921 }
2905 unlock_page(page);2922 folio_unlock(folio);
2906 if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) &&2923 if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) &&
2907 !F2FS_I(inode)->wb_task && allow_balance)2924 !F2FS_I(inode)->wb_task && allow_balance)
2908 f2fs_balance_fs(sbi, need_balance_fs);2925 f2fs_balance_fs(sbi, need_balance_fs);
@@ -2920,7 +2937,7 @@ out: · 11 unmodified lines
2920 return 0;2937 return 0;
2921 2938
2922redirty_out:2939redirty_out:
2923 redirty_page_for_writepage(wbc, page);2940 folio_redirty_for_writepage(wbc, folio);
2924 /*2941 /*
2925 * pageout() in MM translates EAGAIN, so calls handle_write_error()2942 * pageout() in MM translates EAGAIN, so calls handle_write_error()
2926 * -> mapping_set_error() -> set_bit(AS_EIO, ...).2943 * -> mapping_set_error() -> set_bit(AS_EIO, ...).
@@ -2929,29 +2946,30 @@ redirty_out: · 2 unmodified lines
2929 */2946 */
2930 if (!err || wbc->for_reclaim)2947 if (!err || wbc->for_reclaim)
2931 return AOP_WRITEPAGE_ACTIVATE;2948 return AOP_WRITEPAGE_ACTIVATE;
2932 unlock_page(page);2949 folio_unlock(folio);
2933 return err;2950 return err;
2934}2951}
2935 2952
2936static int f2fs_write_data_page(struct page *page,2953static int f2fs_write_data_page(struct page *page,
2937 struct writeback_control *wbc)2954 struct writeback_control *wbc)
2938{2955{
2956 struct folio *folio = page_folio(page);
2939#ifdef CONFIG_F2FS_FS_COMPRESSION2957#ifdef CONFIG_F2FS_FS_COMPRESSION
2940 struct inode *inode = page->mapping->host;2958 struct inode *inode = folio->mapping->host;
2941 2959
2942 if (unlikely(f2fs_cp_error(F2FS_I_SB(inode))))2960 if (unlikely(f2fs_cp_error(F2FS_I_SB(inode))))
2943 goto out;2961 goto out;
2944 2962
2945 if (f2fs_compressed_file(inode)) {2963 if (f2fs_compressed_file(inode)) {
2946 if (f2fs_is_compressed_cluster(inode, page->index)) {2964 if (f2fs_is_compressed_cluster(inode, folio->index)) {
2947 redirty_page_for_writepage(wbc, page);2965 folio_redirty_for_writepage(wbc, folio);
2948 return AOP_WRITEPAGE_ACTIVATE;2966 return AOP_WRITEPAGE_ACTIVATE;
2949 }2967 }
2950 }2968 }
2951out:2969out:
2952#endif2970#endif
2953 2971
2954 return f2fs_write_single_data_page(page, NULL, NULL, NULL,2972 return f2fs_write_single_data_page(folio, NULL, NULL, NULL,
2955 wbc, FS_DATA_IO, 0, true);2973 wbc, FS_DATA_IO, 0, true);
2956}2974}
2957 2975
@@ -3157,11 +3175,11 @@ continue_unlock: · 199 unmodified lines
3157#ifdef CONFIG_F2FS_FS_COMPRESSION3175#ifdef CONFIG_F2FS_FS_COMPRESSION
3158 if (f2fs_compressed_file(inode)) {3176 if (f2fs_compressed_file(inode)) {
3159 folio_get(folio);3177 folio_get(folio);
3160 f2fs_compress_ctx_add_page(&cc, &folio->page);3178 f2fs_compress_ctx_add_page(&cc, folio);
3161 continue;3179 continue;
3162 }3180 }
3163#endif3181#endif
3164 ret = f2fs_write_single_data_page(&folio->page,3182 ret = f2fs_write_single_data_page(folio,
3165 &submitted, &bio, &last_block,3183 &submitted, &bio, &last_block,
3166 wbc, io_type, 0, true);3184 wbc, io_type, 0, true);
3167 if (ret == AOP_WRITEPAGE_ACTIVATE)3185 if (ret == AOP_WRITEPAGE_ACTIVATE)
@@ -3369,11 +3387,11 @@ void f2fs_write_failed(struct inode *inode, loff_t to) · 201 unmodified lines
3369}3387}
3370 3388
3371static int prepare_write_begin(struct f2fs_sb_info *sbi,3389static int prepare_write_begin(struct f2fs_sb_info *sbi,
3372 struct page *page, loff_t pos, unsigned len,3390 struct folio *folio, loff_t pos, unsigned int len,
3373 block_t *blk_addr, bool *node_changed)3391 block_t *blk_addr, bool *node_changed)
3374{3392{
3375 struct inode *inode = page->mapping->host;3393 struct inode *inode = folio->mapping->host;
3376 pgoff_t index = page->index;3394 pgoff_t index = folio->index;
3377 struct dnode_of_data dn;3395 struct dnode_of_data dn;
3378 struct page *ipage;3396 struct page *ipage;
3379 bool locked = false;3397 bool locked = false;
@@ -3410,13 +3428,13 @@ restart: · 30 unmodified lines
3410 3428
3411 if (f2fs_has_inline_data(inode)) {3429 if (f2fs_has_inline_data(inode)) {
3412 if (pos + len <= MAX_INLINE_DATA(inode)) {3430 if (pos + len <= MAX_INLINE_DATA(inode)) {
3413 f2fs_do_read_inline_data(page_folio(page), ipage);3431 f2fs_do_read_inline_data(folio, ipage);
3414 set_inode_flag(inode, FI_DATA_EXIST);3432 set_inode_flag(inode, FI_DATA_EXIST);
3415 if (inode->i_nlink)3433 if (inode->i_nlink)
3416 set_page_private_inline(ipage);3434 set_page_private_inline(ipage);
3417 goto out;3435 goto out;
3418 }3436 }
3419 err = f2fs_convert_inline_page(&dn, page);3437 err = f2fs_convert_inline_page(&dn, folio_page(folio, 0));
3420 if (err || dn.data_blkaddr != NULL_ADDR)3438 if (err || dn.data_blkaddr != NULL_ADDR)
3421 goto out;3439 goto out;
3422 }3440 }
@@ -3509,12 +3527,12 @@ unlock_out: · 86 unmodified lines
3509}3527}
3510 3528
3511static int prepare_atomic_write_begin(struct f2fs_sb_info *sbi,3529static int prepare_atomic_write_begin(struct f2fs_sb_info *sbi,
3512 struct page *page, loff_t pos, unsigned int len,3530 struct folio *folio, loff_t pos, unsigned int len,
3513 block_t *blk_addr, bool *node_changed, bool *use_cow)3531 block_t *blk_addr, bool *node_changed, bool *use_cow)
3514{3532{
3515 struct inode *inode = page->mapping->host;3533 struct inode *inode = folio->mapping->host;
3516 struct inode *cow_inode = F2FS_I(inode)->cow_inode;3534 struct inode *cow_inode = F2FS_I(inode)->cow_inode;
3517 pgoff_t index = page->index;3535 pgoff_t index = folio->index;
3518 int err = 0;3536 int err = 0;
3519 block_t ori_blk_addr = NULL_ADDR;3537 block_t ori_blk_addr = NULL_ADDR;
3520 3538
@@ -3620,10 +3638,10 @@ repeat: · 99 unmodified lines
3620 *foliop = folio;3638 *foliop = folio;
3621 3639
3622 if (f2fs_is_atomic_file(inode))3640 if (f2fs_is_atomic_file(inode))
3623 err = prepare_atomic_write_begin(sbi, &folio->page, pos, len,3641 err = prepare_atomic_write_begin(sbi, folio, pos, len,
3624 &blkaddr, &need_balance, &use_cow);3642 &blkaddr, &need_balance, &use_cow);
3625 else3643 else
3626 err = prepare_write_begin(sbi, &folio->page, pos, len,3644 err = prepare_write_begin(sbi, folio, pos, len,
3627 &blkaddr, &need_balance);3645 &blkaddr, &need_balance);
3628 if (err)3646 if (err)
3629 goto put_folio;3647 goto put_folio;
@@ -3648,7 +3666,7 @@ repeat: · 18 unmodified lines
3648 3666
3649 if (!(pos & (PAGE_SIZE - 1)) && (pos + len) >= i_size_read(inode) &&3667 if (!(pos & (PAGE_SIZE - 1)) && (pos + len) >= i_size_read(inode) &&
3650 !f2fs_verity_in_progress(inode)) {3668 !f2fs_verity_in_progress(inode)) {
3651 folio_zero_segment(folio, len, PAGE_SIZE);3669 folio_zero_segment(folio, len, folio_size(folio));
3652 return 0;3670 return 0;
3653 }3671 }
3654 3672
@@ -3662,8 +3680,8 @@ repeat: · 7 unmodified lines
3662 goto put_folio;3680 goto put_folio;
3663 }3681 }
3664 err = f2fs_submit_page_read(use_cow ?3682 err = f2fs_submit_page_read(use_cow ?
3665 F2FS_I(inode)->cow_inode : inode, &folio->page,3683 F2FS_I(inode)->cow_inode : inode,
3666 blkaddr, 0, true);3684 folio, blkaddr, 0, true);
3667 if (err)3685 if (err)
3668 goto put_folio;3686 goto put_folio;
3669 3687
@@ -3727,6 +3745,9 @@ static int f2fs_write_end(struct file *file, · 57 unmodified lines
3727 3745
3728 folio_mark_dirty(folio);3746 folio_mark_dirty(folio);
3729 3747
3748 if (f2fs_is_atomic_file(inode))
3749 set_page_private_atomic(folio_page(folio, 0));
3750
3730 if (pos + copied > i_size_read(inode) &&3751 if (pos + copied > i_size_read(inode) &&
3731 !f2fs_verity_in_progress(inode)) {3752 !f2fs_verity_in_progress(inode)) {
3732 f2fs_i_size_write(inode, pos + copied);3753 f2fs_i_size_write(inode, pos + copied);
@@ -4117,9 +4138,8 @@ const struct address_space_operations f2fs_dblock_aops = { · 384 unmodified lines
4117 .swap_deactivate = f2fs_swap_deactivate,4138 .swap_deactivate = f2fs_swap_deactivate,
4118};4139};
4119 4140
4120void f2fs_clear_page_cache_dirty_tag(struct page *page)4141void f2fs_clear_page_cache_dirty_tag(struct folio *folio)
4121{4142{
4122 struct folio *folio = page_folio(page);
4123 struct address_space *mapping = folio->mapping;4143 struct address_space *mapping = folio->mapping;
4124 unsigned long flags;4144 unsigned long flags;
4125 4145
fs/f2fs/debug.c+1−1
@@ -275,7 +275,7 @@ static void update_mem_info(struct f2fs_sb_info *sbi) · 274 unmodified lines
275 /* build nm */275 /* build nm */
276 si->base_mem += sizeof(struct f2fs_nm_info);276 si->base_mem += sizeof(struct f2fs_nm_info);
277 si->base_mem += __bitmap_size(sbi, NAT_BITMAP);277 si->base_mem += __bitmap_size(sbi, NAT_BITMAP);
278 si->base_mem += (NM_I(sbi)->nat_bits_blocks << F2FS_BLKSIZE_BITS);278 si->base_mem += F2FS_BLK_TO_BYTES(NM_I(sbi)->nat_bits_blocks);
279 si->base_mem += NM_I(sbi)->nat_blocks *279 si->base_mem += NM_I(sbi)->nat_blocks *
280 f2fs_bitmap_size(NAT_ENTRY_PER_BLOCK);280 f2fs_bitmap_size(NAT_ENTRY_PER_BLOCK);
281 si->base_mem += NM_I(sbi)->nat_blocks / 8;281 si->base_mem += NM_I(sbi)->nat_blocks / 8;
fs/f2fs/dir.c+5−3
@@ -166,7 +166,8 @@ static unsigned long dir_block_index(unsigned int level, · 165 unmodified lines
166 unsigned long bidx = 0;166 unsigned long bidx = 0;
167 167
168 for (i = 0; i < level; i++)168 for (i = 0; i < level; i++)
169 bidx += dir_buckets(i, dir_level) * bucket_blocks(i);169 bidx += mul_u32_u32(dir_buckets(i, dir_level),
170 bucket_blocks(i));
170 bidx += idx * bucket_blocks(level);171 bidx += idx * bucket_blocks(level);
171 return bidx;172 return bidx;
172}173}
@@ -841,6 +842,7 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, struct page *page, · 668 unmodified lines
841 struct f2fs_dentry_block *dentry_blk;842 struct f2fs_dentry_block *dentry_blk;
842 unsigned int bit_pos;843 unsigned int bit_pos;
843 int slots = GET_DENTRY_SLOTS(le16_to_cpu(dentry->name_len));844 int slots = GET_DENTRY_SLOTS(le16_to_cpu(dentry->name_len));
845 pgoff_t index = page_folio(page)->index;
844 int i;846 int i;
845 847
846 f2fs_update_time(F2FS_I_SB(dir), REQ_TIME);848 f2fs_update_time(F2FS_I_SB(dir), REQ_TIME);
@@ -866,8 +868,8 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, struct page *page, · 19 unmodified lines
866 set_page_dirty(page);868 set_page_dirty(page);
867 869
868 if (bit_pos == NR_DENTRY_IN_BLOCK &&870 if (bit_pos == NR_DENTRY_IN_BLOCK &&
869 !f2fs_truncate_hole(dir, page->index, page->index + 1)) {871 !f2fs_truncate_hole(dir, index, index + 1)) {
870 f2fs_clear_page_cache_dirty_tag(page);872 f2fs_clear_page_cache_dirty_tag(page_folio(page));
871 clear_page_dirty_for_io(page);873 clear_page_dirty_for_io(page);
872 ClearPageUptodate(page);874 ClearPageUptodate(page);
873 clear_page_private_all(page);875 clear_page_private_all(page);
fs/f2fs/extent_cache.c+2−2
@@ -366,7 +366,7 @@ static unsigned int __free_extent_tree(struct f2fs_sb_info *sbi, · 365 unmodified lines
366static void __drop_largest_extent(struct extent_tree *et,366static void __drop_largest_extent(struct extent_tree *et,
367 pgoff_t fofs, unsigned int len)367 pgoff_t fofs, unsigned int len)
368{368{
369 if (fofs < et->largest.fofs + et->largest.len &&369 if (fofs < (pgoff_t)et->largest.fofs + et->largest.len &&
370 fofs + len > et->largest.fofs) {370 fofs + len > et->largest.fofs) {
371 et->largest.len = 0;371 et->largest.len = 0;
372 et->largest_updated = true;372 et->largest_updated = true;
@@ -456,7 +456,7 @@ static bool __lookup_extent_tree(struct inode *inode, pgoff_t pgofs, · 83 unmodified lines
456 456
457 if (type == EX_READ &&457 if (type == EX_READ &&
458 et->largest.fofs <= pgofs &&458 et->largest.fofs <= pgofs &&
459 et->largest.fofs + et->largest.len > pgofs) {459 (pgoff_t)et->largest.fofs + et->largest.len > pgofs) {
460 *ei = et->largest;460 *ei = et->largest;
461 ret = true;461 ret = true;
462 stat_inc_largest_node_hit(sbi);462 stat_inc_largest_node_hit(sbi);
fs/f2fs/f2fs.h+91−57
@@ -11,7 +11,6 @@ · 10 unmodified lines
11#include <linux/uio.h>11#include <linux/uio.h>
12#include <linux/types.h>12#include <linux/types.h>
13#include <linux/page-flags.h>13#include <linux/page-flags.h>
14#include <linux/buffer_head.h>
15#include <linux/slab.h>14#include <linux/slab.h>
16#include <linux/crc32.h>15#include <linux/crc32.h>
17#include <linux/magic.h>16#include <linux/magic.h>
@@ -134,6 +133,12 @@ typedef u32 nid_t; · 116 unmodified lines
134 133
135#define COMPRESS_EXT_NUM 16134#define COMPRESS_EXT_NUM 16
136 135
136enum blkzone_allocation_policy {
137 BLKZONE_ALLOC_PRIOR_SEQ, /* Prioritize writing to sequential zones */
138 BLKZONE_ALLOC_ONLY_SEQ, /* Only allow writing to sequential zones */
139 BLKZONE_ALLOC_PRIOR_CONV, /* Prioritize writing to conventional zones */
140};
141
137/*142/*
138 * An implementation of an rwsem that is explicitly unfair to readers. This143 * An implementation of an rwsem that is explicitly unfair to readers. This
139 * prevents priority inversion when a low-priority reader acquires the read lock144 * prevents priority inversion when a low-priority reader acquires the read lock
@@ -285,6 +290,7 @@ enum { · 145 unmodified lines
285 APPEND_INO, /* for append ino list */290 APPEND_INO, /* for append ino list */
286 UPDATE_INO, /* for update ino list */291 UPDATE_INO, /* for update ino list */
287 TRANS_DIR_INO, /* for transactions dir ino list */292 TRANS_DIR_INO, /* for transactions dir ino list */
293 XATTR_DIR_INO, /* for xattr updated dir ino list */
288 FLUSH_INO, /* for multiple device flushing */294 FLUSH_INO, /* for multiple device flushing */
289 MAX_INO_ENTRY, /* max. list */295 MAX_INO_ENTRY, /* max. list */
290};296};
@@ -784,7 +790,6 @@ enum { · 493 unmodified lines
784 FI_NEED_IPU, /* used for ipu per file */790 FI_NEED_IPU, /* used for ipu per file */
785 FI_ATOMIC_FILE, /* indicate atomic file */791 FI_ATOMIC_FILE, /* indicate atomic file */
786 FI_DATA_EXIST, /* indicate data exists */792 FI_DATA_EXIST, /* indicate data exists */
787 FI_INLINE_DOTS, /* indicate inline dot dentries */
788 FI_SKIP_WRITES, /* should skip data page writeback */793 FI_SKIP_WRITES, /* should skip data page writeback */
789 FI_OPU_WRITE, /* used for opu per file */794 FI_OPU_WRITE, /* used for opu per file */
790 FI_DIRTY_FILE, /* indicate regular/symlink has dirty pages */795 FI_DIRTY_FILE, /* indicate regular/symlink has dirty pages */
@@ -802,6 +807,7 @@ enum { · 11 unmodified lines
802 FI_ALIGNED_WRITE, /* enable aligned write */807 FI_ALIGNED_WRITE, /* enable aligned write */
803 FI_COW_FILE, /* indicate COW file */808 FI_COW_FILE, /* indicate COW file */
804 FI_ATOMIC_COMMITTED, /* indicate atomic commit completed except disk sync */809 FI_ATOMIC_COMMITTED, /* indicate atomic commit completed except disk sync */
810 FI_ATOMIC_DIRTIED, /* indicate atomic file is dirtied */
805 FI_ATOMIC_REPLACE, /* indicate atomic replace */811 FI_ATOMIC_REPLACE, /* indicate atomic replace */
806 FI_OPENED_FILE, /* indicate file has been opened */812 FI_OPENED_FILE, /* indicate file has been opened */
807 FI_MAX, /* max flag, never be used */813 FI_MAX, /* max flag, never be used */
@@ -1155,6 +1161,7 @@ enum cp_reason_type { · 347 unmodified lines
1155 CP_FASTBOOT_MODE,1161 CP_FASTBOOT_MODE,
1156 CP_SPEC_LOG_NUM,1162 CP_SPEC_LOG_NUM,
1157 CP_RECOVER_DIR,1163 CP_RECOVER_DIR,
1164 CP_XATTR_DIR,
1158};1165};
1159 1166
1160enum iostat_type {1167enum iostat_type {
@@ -1293,6 +1300,7 @@ struct f2fs_gc_control { · 132 unmodified lines
1293 bool no_bg_gc; /* check the space and stop bg_gc */1300 bool no_bg_gc; /* check the space and stop bg_gc */
1294 bool should_migrate_blocks; /* should migrate blocks */1301 bool should_migrate_blocks; /* should migrate blocks */
1295 bool err_gc_skipped; /* return EAGAIN if GC skipped */1302 bool err_gc_skipped; /* return EAGAIN if GC skipped */
1303 bool one_time; /* require one time GC in one migration unit */
1296 unsigned int nr_free_secs; /* # of free sections to do GC */1304 unsigned int nr_free_secs; /* # of free sections to do GC */
1297};1305};
1298 1306
@@ -1418,7 +1426,8 @@ static inline void f2fs_clear_bit(unsigned int nr, char *addr); · 119 unmodified lines
1418 * bit 1 PAGE_PRIVATE_ONGOING_MIGRATION1426 * bit 1 PAGE_PRIVATE_ONGOING_MIGRATION
1419 * bit 2 PAGE_PRIVATE_INLINE_INODE1427 * bit 2 PAGE_PRIVATE_INLINE_INODE
1420 * bit 3 PAGE_PRIVATE_REF_RESOURCE1428 * bit 3 PAGE_PRIVATE_REF_RESOURCE
1421 * bit 4- f2fs private data1429 * bit 4 PAGE_PRIVATE_ATOMIC_WRITE
1430 * bit 5- f2fs private data
1422 *1431 *
1423 * Layout B: lowest bit should be 01432 * Layout B: lowest bit should be 0
1424 * page.private is a wrapped pointer.1433 * page.private is a wrapped pointer.
@@ -1428,6 +1437,7 @@ enum { · 3 unmodified lines
1428 PAGE_PRIVATE_ONGOING_MIGRATION, /* data page which is on-going migrating */1437 PAGE_PRIVATE_ONGOING_MIGRATION, /* data page which is on-going migrating */
1429 PAGE_PRIVATE_INLINE_INODE, /* inode page contains inline data */1438 PAGE_PRIVATE_INLINE_INODE, /* inode page contains inline data */
1430 PAGE_PRIVATE_REF_RESOURCE, /* dirty page has referenced resources */1439 PAGE_PRIVATE_REF_RESOURCE, /* dirty page has referenced resources */
1440 PAGE_PRIVATE_ATOMIC_WRITE, /* data page from atomic write path */
1431 PAGE_PRIVATE_MAX1441 PAGE_PRIVATE_MAX
1432};1442};
1433 1443
@@ -1559,6 +1569,8 @@ struct f2fs_sb_info { · 125 unmodified lines
1559#ifdef CONFIG_BLK_DEV_ZONED1569#ifdef CONFIG_BLK_DEV_ZONED
1560 unsigned int blocks_per_blkz; /* F2FS blocks per zone */1570 unsigned int blocks_per_blkz; /* F2FS blocks per zone */
1561 unsigned int max_open_zones; /* max open zone resources of the zoned device */1571 unsigned int max_open_zones; /* max open zone resources of the zoned device */
1572 /* For adjust the priority writing position of data in zone UFS */
1573 unsigned int blkzone_alloc_policy;
1562#endif1574#endif
1563 1575
1564 /* for node-related operations */1576 /* for node-related operations */
@@ -1685,6 +1697,8 @@ struct f2fs_sb_info { · 120 unmodified lines
1685 unsigned int max_victim_search;1697 unsigned int max_victim_search;
1686 /* migration granularity of garbage collection, unit: segment */1698 /* migration granularity of garbage collection, unit: segment */
1687 unsigned int migration_granularity;1699 unsigned int migration_granularity;
1700 /* migration window granularity of garbage collection, unit: segment */
1701 unsigned int migration_window_granularity;
1688 1702
1689 /*1703 /*
1690 * for stat information.1704 * for stat information.
@@ -1994,6 +2008,16 @@ static inline struct f2fs_super_block *F2FS_RAW_SUPER(struct f2fs_sb_info *sbi) · 303 unmodified lines
1994 return (struct f2fs_super_block *)(sbi->raw_super);2008 return (struct f2fs_super_block *)(sbi->raw_super);
1995}2009}
1996 2010
2011static inline struct f2fs_super_block *F2FS_SUPER_BLOCK(struct folio *folio,
2012 pgoff_t index)
2013{
2014 pgoff_t idx_in_folio = index % (1 << folio_order(folio));
2015
2016 return (struct f2fs_super_block *)
2017 (page_address(folio_page(folio, idx_in_folio)) +
2018 F2FS_SUPER_OFFSET);
2019}
2020
1997static inline struct f2fs_checkpoint *F2FS_CKPT(struct f2fs_sb_info *sbi)2021static inline struct f2fs_checkpoint *F2FS_CKPT(struct f2fs_sb_info *sbi)
1998{2022{
1999 return (struct f2fs_checkpoint *)(sbi->ckpt);2023 return (struct f2fs_checkpoint *)(sbi->ckpt);
@@ -2396,14 +2420,17 @@ static inline void clear_page_private_##name(struct page *page) \ · 396 unmodified lines
2396PAGE_PRIVATE_GET_FUNC(nonpointer, NOT_POINTER);2420PAGE_PRIVATE_GET_FUNC(nonpointer, NOT_POINTER);
2397PAGE_PRIVATE_GET_FUNC(inline, INLINE_INODE);2421PAGE_PRIVATE_GET_FUNC(inline, INLINE_INODE);
2398PAGE_PRIVATE_GET_FUNC(gcing, ONGOING_MIGRATION);2422PAGE_PRIVATE_GET_FUNC(gcing, ONGOING_MIGRATION);
2423PAGE_PRIVATE_GET_FUNC(atomic, ATOMIC_WRITE);
2399 2424
2400PAGE_PRIVATE_SET_FUNC(reference, REF_RESOURCE);2425PAGE_PRIVATE_SET_FUNC(reference, REF_RESOURCE);
2401PAGE_PRIVATE_SET_FUNC(inline, INLINE_INODE);2426PAGE_PRIVATE_SET_FUNC(inline, INLINE_INODE);
2402PAGE_PRIVATE_SET_FUNC(gcing, ONGOING_MIGRATION);2427PAGE_PRIVATE_SET_FUNC(gcing, ONGOING_MIGRATION);
2428PAGE_PRIVATE_SET_FUNC(atomic, ATOMIC_WRITE);
2403 2429
2404PAGE_PRIVATE_CLEAR_FUNC(reference, REF_RESOURCE);2430PAGE_PRIVATE_CLEAR_FUNC(reference, REF_RESOURCE);
2405PAGE_PRIVATE_CLEAR_FUNC(inline, INLINE_INODE);2431PAGE_PRIVATE_CLEAR_FUNC(inline, INLINE_INODE);
2406PAGE_PRIVATE_CLEAR_FUNC(gcing, ONGOING_MIGRATION);2432PAGE_PRIVATE_CLEAR_FUNC(gcing, ONGOING_MIGRATION);
2433PAGE_PRIVATE_CLEAR_FUNC(atomic, ATOMIC_WRITE);
2407 2434
2408static inline unsigned long get_page_private_data(struct page *page)2435static inline unsigned long get_page_private_data(struct page *page)
2409{2436{
@@ -2435,6 +2462,7 @@ static inline void clear_page_private_all(struct page *page) · 25 unmodified lines
2435 clear_page_private_reference(page);2462 clear_page_private_reference(page);
2436 clear_page_private_gcing(page);2463 clear_page_private_gcing(page);
2437 clear_page_private_inline(page);2464 clear_page_private_inline(page);
2465 clear_page_private_atomic(page);
2438 2466
2439 f2fs_bug_on(F2FS_P_SB(page), page_private(page));2467 f2fs_bug_on(F2FS_P_SB(page), page_private(page));
2440}2468}
@@ -2854,13 +2882,26 @@ static inline bool is_inflight_io(struct f2fs_sb_info *sbi, int type) · 413 unmodified lines
2854 return false;2882 return false;
2855}2883}
2856 2884
2885static inline bool is_inflight_read_io(struct f2fs_sb_info *sbi)
2886{
2887 return get_pages(sbi, F2FS_RD_DATA) || get_pages(sbi, F2FS_DIO_READ);
2888}
2889
2857static inline bool is_idle(struct f2fs_sb_info *sbi, int type)2890static inline bool is_idle(struct f2fs_sb_info *sbi, int type)
2858{2891{
2892 bool zoned_gc = (type == GC_TIME &&
2893 F2FS_HAS_FEATURE(sbi, F2FS_FEATURE_BLKZONED));
2894
2859 if (sbi->gc_mode == GC_URGENT_HIGH)2895 if (sbi->gc_mode == GC_URGENT_HIGH)
2860 return true;2896 return true;
2861 2897
2862 if (is_inflight_io(sbi, type))2898 if (zoned_gc) {
2863 return false;2899 if (is_inflight_read_io(sbi))
2900 return false;
2901 } else {
2902 if (is_inflight_io(sbi, type))
2903 return false;
2904 }
2864 2905
2865 if (sbi->gc_mode == GC_URGENT_MID)2906 if (sbi->gc_mode == GC_URGENT_MID)
2866 return true;2907 return true;
@@ -2869,6 +2910,9 @@ static inline bool is_idle(struct f2fs_sb_info *sbi, int type) · 2 unmodified lines
2869 (type == DISCARD_TIME || type == GC_TIME))2910 (type == DISCARD_TIME || type == GC_TIME))
2870 return true;2911 return true;
2871 2912
2913 if (zoned_gc)
2914 return true;
2915
2872 return f2fs_time_over(sbi, type);2916 return f2fs_time_over(sbi, type);
2873}2917}
2874 2918
@@ -2900,26 +2944,27 @@ static inline __le32 *blkaddr_in_node(struct f2fs_node *node) · 25 unmodified lines
2900}2944}
2901 2945
2902static inline int f2fs_has_extra_attr(struct inode *inode);2946static inline int f2fs_has_extra_attr(struct inode *inode);
2903static inline block_t data_blkaddr(struct inode *inode,2947static inline unsigned int get_dnode_base(struct inode *inode,
2904 struct page *node_page, unsigned int offset)2948 struct page *node_page)
2905{2949{
2906 struct f2fs_node *raw_node;2950 if (!IS_INODE(node_page))
2907 __le32 *addr_array;2951 return 0;
2908 int base = 0;
2909 bool is_inode = IS_INODE(node_page);
2910 2952
2911 raw_node = F2FS_NODE(node_page);2953 return inode ? get_extra_isize(inode) :
2954 offset_in_addr(&F2FS_NODE(node_page)->i);
2955}
2912 2956
2913 if (is_inode) {2957static inline __le32 *get_dnode_addr(struct inode *inode,
2914 if (!inode)2958 struct page *node_page)
2915 /* from GC path only */2959{
2916 base = offset_in_addr(&raw_node->i);2960 return blkaddr_in_node(F2FS_NODE(node_page)) +
2917 else if (f2fs_has_extra_attr(inode))2961 get_dnode_base(inode, node_page);
2918 base = get_extra_isize(inode);2962}
2919 }
2920 2963
2921 addr_array = blkaddr_in_node(raw_node);2964static inline block_t data_blkaddr(struct inode *inode,
2922 return le32_to_cpu(addr_array[base + offset]);2965 struct page *node_page, unsigned int offset)
2966{
2967 return le32_to_cpu(*(get_dnode_addr(inode, node_page) + offset));
2923}2968}
2924 2969
2925static inline block_t f2fs_data_blkaddr(struct dnode_of_data *dn)2970static inline block_t f2fs_data_blkaddr(struct dnode_of_data *dn)
@@ -3038,10 +3083,8 @@ static inline void __mark_inode_dirty_flag(struct inode *inode, · 112 unmodified lines
3038 return;3083 return;
3039 fallthrough;3084 fallthrough;
3040 case FI_DATA_EXIST:3085 case FI_DATA_EXIST:
3041 case FI_INLINE_DOTS:
3042 case FI_PIN_FILE:3086 case FI_PIN_FILE:
3043 case FI_COMPRESS_RELEASED:3087 case FI_COMPRESS_RELEASED:
3044 case FI_ATOMIC_COMMITTED:
3045 f2fs_mark_inode_dirty_sync(inode, true);3088 f2fs_mark_inode_dirty_sync(inode, true);
3046 }3089 }
3047}3090}
@@ -3163,8 +3206,6 @@ static inline void get_inline_info(struct inode *inode, struct f2fs_inode *ri) · 115 unmodified lines
3163 set_bit(FI_INLINE_DENTRY, fi->flags);3206 set_bit(FI_INLINE_DENTRY, fi->flags);
3164 if (ri->i_inline & F2FS_DATA_EXIST)3207 if (ri->i_inline & F2FS_DATA_EXIST)
3165 set_bit(FI_DATA_EXIST, fi->flags);3208 set_bit(FI_DATA_EXIST, fi->flags);
3166 if (ri->i_inline & F2FS_INLINE_DOTS)
3167 set_bit(FI_INLINE_DOTS, fi->flags);
3168 if (ri->i_inline & F2FS_EXTRA_ATTR)3209 if (ri->i_inline & F2FS_EXTRA_ATTR)
3169 set_bit(FI_EXTRA_ATTR, fi->flags);3210 set_bit(FI_EXTRA_ATTR, fi->flags);
3170 if (ri->i_inline & F2FS_PIN_FILE)3211 if (ri->i_inline & F2FS_PIN_FILE)
@@ -3185,8 +3226,6 @@ static inline void set_raw_inline(struct inode *inode, struct f2fs_inode *ri) · 14 unmodified lines
3185 ri->i_inline |= F2FS_INLINE_DENTRY;3226 ri->i_inline |= F2FS_INLINE_DENTRY;
3186 if (is_inode_flag_set(inode, FI_DATA_EXIST))3227 if (is_inode_flag_set(inode, FI_DATA_EXIST))
3187 ri->i_inline |= F2FS_DATA_EXIST;3228 ri->i_inline |= F2FS_DATA_EXIST;
3188 if (is_inode_flag_set(inode, FI_INLINE_DOTS))
3189 ri->i_inline |= F2FS_INLINE_DOTS;
3190 if (is_inode_flag_set(inode, FI_EXTRA_ATTR))3229 if (is_inode_flag_set(inode, FI_EXTRA_ATTR))
3191 ri->i_inline |= F2FS_EXTRA_ATTR;3230 ri->i_inline |= F2FS_EXTRA_ATTR;
3192 if (is_inode_flag_set(inode, FI_PIN_FILE))3231 if (is_inode_flag_set(inode, FI_PIN_FILE))
@@ -3267,11 +3306,6 @@ static inline int f2fs_exist_data(struct inode *inode) · 74 unmodified lines
3267 return is_inode_flag_set(inode, FI_DATA_EXIST);3306 return is_inode_flag_set(inode, FI_DATA_EXIST);
3268}3307}
3269 3308
3270static inline int f2fs_has_inline_dots(struct inode *inode)
3271{
3272 return is_inode_flag_set(inode, FI_INLINE_DOTS);
3273}
3274
3275static inline int f2fs_is_mmap_file(struct inode *inode)3309static inline int f2fs_is_mmap_file(struct inode *inode)
3276{3310{
3277 return is_inode_flag_set(inode, FI_MMAP_FILE);3311 return is_inode_flag_set(inode, FI_MMAP_FILE);
@@ -3292,8 +3326,6 @@ static inline bool f2fs_is_cow_file(struct inode *inode) · 14 unmodified lines
3292 return is_inode_flag_set(inode, FI_COW_FILE);3326 return is_inode_flag_set(inode, FI_COW_FILE);
3293}3327}
3294 3328
3295static inline __le32 *get_dnode_addr(struct inode *inode,
3296 struct page *node_page);
3297static inline void *inline_data_addr(struct inode *inode, struct page *page)3329static inline void *inline_data_addr(struct inode *inode, struct page *page)
3298{3330{
3299 __le32 *addr = get_dnode_addr(inode, page);3331 __le32 *addr = get_dnode_addr(inode, page);
@@ -3432,17 +3464,6 @@ static inline int get_inline_xattr_addrs(struct inode *inode) · 132 unmodified lines
3432 return F2FS_I(inode)->i_inline_xattr_size;3464 return F2FS_I(inode)->i_inline_xattr_size;
3433}3465}
3434 3466
3435static inline __le32 *get_dnode_addr(struct inode *inode,
3436 struct page *node_page)
3437{
3438 int base = 0;
3439
3440 if (IS_INODE(node_page) && f2fs_has_extra_attr(inode))
3441 base = get_extra_isize(inode);
3442
3443 return blkaddr_in_node(F2FS_NODE(node_page)) + base;
3444}
3445
3446#define f2fs_get_inode_mode(i) \3467#define f2fs_get_inode_mode(i) \
3447 ((is_inode_flag_set(i, FI_ACL_MODE)) ? \3468 ((is_inode_flag_set(i, FI_ACL_MODE)) ? \
3448 (F2FS_I(i)->i_acl_mode) : ((i)->i_mode))3469 (F2FS_I(i)->i_acl_mode) : ((i)->i_mode))
@@ -3495,7 +3516,7 @@ int f2fs_setattr(struct mnt_idmap *idmap, struct dentry *dentry, · 46 unmodified lines
3495int f2fs_truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end);3516int f2fs_truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end);
3496void f2fs_truncate_data_blocks_range(struct dnode_of_data *dn, int count);3517void f2fs_truncate_data_blocks_range(struct dnode_of_data *dn, int count);
3497int f2fs_do_shutdown(struct f2fs_sb_info *sbi, unsigned int flag,3518int f2fs_do_shutdown(struct f2fs_sb_info *sbi, unsigned int flag,
3498 bool readonly);3519 bool readonly, bool need_lock);
3499int f2fs_precache_extents(struct inode *inode);3520int f2fs_precache_extents(struct inode *inode);
3500int f2fs_fileattr_get(struct dentry *dentry, struct fileattr *fa);3521int f2fs_fileattr_get(struct dentry *dentry, struct fileattr *fa);
3501int f2fs_fileattr_set(struct mnt_idmap *idmap,3522int f2fs_fileattr_set(struct mnt_idmap *idmap,
@@ -3719,7 +3740,7 @@ bool f2fs_exist_trim_candidates(struct f2fs_sb_info *sbi, · 217 unmodified lines
3719struct page *f2fs_get_sum_page(struct f2fs_sb_info *sbi, unsigned int segno);3740struct page *f2fs_get_sum_page(struct f2fs_sb_info *sbi, unsigned int segno);
3720void f2fs_update_meta_page(struct f2fs_sb_info *sbi, void *src,3741void f2fs_update_meta_page(struct f2fs_sb_info *sbi, void *src,
3721 block_t blk_addr);3742 block_t blk_addr);
3722void f2fs_do_write_meta_page(struct f2fs_sb_info *sbi, struct page *page,3743void f2fs_do_write_meta_page(struct f2fs_sb_info *sbi, struct folio *folio,
3723 enum iostat_type io_type);3744 enum iostat_type io_type);
3724void f2fs_do_write_node_page(unsigned int nid, struct f2fs_io_info *fio);3745void f2fs_do_write_node_page(unsigned int nid, struct f2fs_io_info *fio);
3725void f2fs_outplace_write_data(struct dnode_of_data *dn,3746void f2fs_outplace_write_data(struct dnode_of_data *dn,
@@ -3759,8 +3780,7 @@ void f2fs_destroy_segment_manager_caches(void); · 33 unmodified lines
3759int f2fs_rw_hint_to_seg_type(struct f2fs_sb_info *sbi, enum rw_hint hint);3780int f2fs_rw_hint_to_seg_type(struct f2fs_sb_info *sbi, enum rw_hint hint);
3760enum rw_hint f2fs_io_type_to_rw_hint(struct f2fs_sb_info *sbi,3781enum rw_hint f2fs_io_type_to_rw_hint(struct f2fs_sb_info *sbi,
3761 enum page_type type, enum temp_type temp);3782 enum page_type type, enum temp_type temp);
3762unsigned int f2fs_usable_segs_in_sec(struct f2fs_sb_info *sbi,3783unsigned int f2fs_usable_segs_in_sec(struct f2fs_sb_info *sbi);
3763 unsigned int segno);
3764unsigned int f2fs_usable_blks_in_seg(struct f2fs_sb_info *sbi,3784unsigned int f2fs_usable_blks_in_seg(struct f2fs_sb_info *sbi,
3765 unsigned int segno);3785 unsigned int segno);
3766 3786
@@ -3868,7 +3888,7 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, · 101 unmodified lines
3868int f2fs_encrypt_one_page(struct f2fs_io_info *fio);3888int f2fs_encrypt_one_page(struct f2fs_io_info *fio);
3869bool f2fs_should_update_inplace(struct inode *inode, struct f2fs_io_info *fio);3889bool f2fs_should_update_inplace(struct inode *inode, struct f2fs_io_info *fio);
3870bool f2fs_should_update_outplace(struct inode *inode, struct f2fs_io_info *fio);3890bool f2fs_should_update_outplace(struct inode *inode, struct f2fs_io_info *fio);
3871int f2fs_write_single_data_page(struct page *page, int *submitted,3891int f2fs_write_single_data_page(struct folio *folio, int *submitted,
3872 struct bio **bio, sector_t *last_block,3892 struct bio **bio, sector_t *last_block,
3873 struct writeback_control *wbc,3893 struct writeback_control *wbc,
3874 enum iostat_type io_type,3894 enum iostat_type io_type,
@@ -3877,7 +3897,7 @@ void f2fs_write_failed(struct inode *inode, loff_t to); · 2 unmodified lines
3877void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length);3897void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length);
3878bool f2fs_release_folio(struct folio *folio, gfp_t wait);3898bool f2fs_release_folio(struct folio *folio, gfp_t wait);
3879bool f2fs_overwrite_io(struct inode *inode, loff_t pos, size_t len);3899bool f2fs_overwrite_io(struct inode *inode, loff_t pos, size_t len);
3880void f2fs_clear_page_cache_dirty_tag(struct page *page);3900void f2fs_clear_page_cache_dirty_tag(struct folio *folio);
3881int f2fs_init_post_read_processing(void);3901int f2fs_init_post_read_processing(void);
3882void f2fs_destroy_post_read_processing(void);3902void f2fs_destroy_post_read_processing(void);
3883int f2fs_init_post_read_wq(struct f2fs_sb_info *sbi);3903int f2fs_init_post_read_wq(struct f2fs_sb_info *sbi);
@@ -3901,7 +3921,7 @@ void f2fs_destroy_garbage_collection_cache(void); · 17 unmodified lines
3901/* victim selection function for cleaning and SSR */3921/* victim selection function for cleaning and SSR */
3902int f2fs_get_victim(struct f2fs_sb_info *sbi, unsigned int *result,3922int f2fs_get_victim(struct f2fs_sb_info *sbi, unsigned int *result,
3903 int gc_type, int type, char alloc_mode,3923 int gc_type, int type, char alloc_mode,
3904 unsigned long long age);3924 unsigned long long age, bool one_time);
3905 3925
3906/*3926/*
3907 * recovery.c3927 * recovery.c
@@ -3987,7 +4007,7 @@ static inline struct f2fs_stat_info *F2FS_STAT(struct f2fs_sb_info *sbi) · 79 unmodified lines
3987 4007
3988#define stat_inc_cp_call_count(sbi, foreground) \4008#define stat_inc_cp_call_count(sbi, foreground) \
3989 atomic_inc(&sbi->cp_call_count[(foreground)])4009 atomic_inc(&sbi->cp_call_count[(foreground)])
3990#define stat_inc_cp_count(si) (F2FS_STAT(sbi)->cp_count++)4010#define stat_inc_cp_count(sbi) (F2FS_STAT(sbi)->cp_count++)
3991#define stat_io_skip_bggc_count(sbi) ((sbi)->io_skip_bggc++)4011#define stat_io_skip_bggc_count(sbi) ((sbi)->io_skip_bggc++)
3992#define stat_other_skip_bggc_count(sbi) ((sbi)->other_skip_bggc++)4012#define stat_other_skip_bggc_count(sbi) ((sbi)->other_skip_bggc++)
3993#define stat_inc_dirty_inode(sbi, type) ((sbi)->ndirty_inode[type]++)4013#define stat_inc_dirty_inode(sbi, type) ((sbi)->ndirty_inode[type]++)
@@ -4172,7 +4192,7 @@ int f2fs_read_inline_data(struct inode *inode, struct folio *folio); · 178 unmodified lines
4172int f2fs_convert_inline_page(struct dnode_of_data *dn, struct page *page);4192int f2fs_convert_inline_page(struct dnode_of_data *dn, struct page *page);
4173int f2fs_convert_inline_inode(struct inode *inode);4193int f2fs_convert_inline_inode(struct inode *inode);
4174int f2fs_try_convert_inline_dir(struct inode *dir, struct dentry *dentry);4194int f2fs_try_convert_inline_dir(struct inode *dir, struct dentry *dentry);
4175int f2fs_write_inline_data(struct inode *inode, struct page *page);4195int f2fs_write_inline_data(struct inode *inode, struct folio *folio);
4176int f2fs_recover_inline_data(struct inode *inode, struct page *npage);4196int f2fs_recover_inline_data(struct inode *inode, struct page *npage);
4177struct f2fs_dir_entry *f2fs_find_in_inline_dir(struct inode *dir,4197struct f2fs_dir_entry *f2fs_find_in_inline_dir(struct inode *dir,
4178 const struct f2fs_filename *fname,4198 const struct f2fs_filename *fname,
@@ -4289,6 +4309,11 @@ static inline bool f2fs_meta_inode_gc_required(struct inode *inode) · 110 unmodified lines
4289 * compress.c4309 * compress.c
4290 */4310 */
4291#ifdef CONFIG_F2FS_FS_COMPRESSION4311#ifdef CONFIG_F2FS_FS_COMPRESSION
4312enum cluster_check_type {
4313 CLUSTER_IS_COMPR, /* check only if compressed cluster */
4314 CLUSTER_COMPR_BLKS, /* return # of compressed blocks in a cluster */
4315 CLUSTER_RAW_BLKS /* return # of raw blocks in a cluster */
4316};
4292bool f2fs_is_compressed_page(struct page *page);4317bool f2fs_is_compressed_page(struct page *page);
4293struct page *f2fs_compress_control_page(struct page *page);4318struct page *f2fs_compress_control_page(struct page *page);
4294int f2fs_prepare_compress_overwrite(struct inode *inode,4319int f2fs_prepare_compress_overwrite(struct inode *inode,
@@ -4309,12 +4334,13 @@ bool f2fs_cluster_can_merge_page(struct compress_ctx *cc, pgoff_t index); · 14 unmodified lines
4309bool f2fs_all_cluster_page_ready(struct compress_ctx *cc, struct page **pages,4334bool f2fs_all_cluster_page_ready(struct compress_ctx *cc, struct page **pages,
4310 int index, int nr_pages, bool uptodate);4335 int index, int nr_pages, bool uptodate);
4311bool f2fs_sanity_check_cluster(struct dnode_of_data *dn);4336bool f2fs_sanity_check_cluster(struct dnode_of_data *dn);
4312void f2fs_compress_ctx_add_page(struct compress_ctx *cc, struct page *page);4337void f2fs_compress_ctx_add_page(struct compress_ctx *cc, struct folio *folio);
4313int f2fs_write_multi_pages(struct compress_ctx *cc,4338int f2fs_write_multi_pages(struct compress_ctx *cc,
4314 int *submitted,4339 int *submitted,
4315 struct writeback_control *wbc,4340 struct writeback_control *wbc,
4316 enum iostat_type io_type);4341 enum iostat_type io_type);
4317int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index);4342int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index);
4343bool f2fs_is_sparse_cluster(struct inode *inode, pgoff_t index);
4318void f2fs_update_read_extent_tree_range_compressed(struct inode *inode,4344void f2fs_update_read_extent_tree_range_compressed(struct inode *inode,
4319 pgoff_t fofs, block_t blkaddr,4345 pgoff_t fofs, block_t blkaddr,
4320 unsigned int llen, unsigned int c_len);4346 unsigned int llen, unsigned int c_len);
@@ -4401,6 +4427,12 @@ static inline bool f2fs_load_compressed_page(struct f2fs_sb_info *sbi, · 80 unmodified lines
4401static inline void f2fs_invalidate_compress_pages(struct f2fs_sb_info *sbi,4427static inline void f2fs_invalidate_compress_pages(struct f2fs_sb_info *sbi,
4402 nid_t ino) { }4428 nid_t ino) { }
4403#define inc_compr_inode_stat(inode) do { } while (0)4429#define inc_compr_inode_stat(inode) do { } while (0)
4430static inline int f2fs_is_compressed_cluster(
4431 struct inode *inode,
4432 pgoff_t index) { return 0; }
4433static inline bool f2fs_is_sparse_cluster(
4434 struct inode *inode,
4435 pgoff_t index) { return true; }
4404static inline void f2fs_update_read_extent_tree_range_compressed(4436static inline void f2fs_update_read_extent_tree_range_compressed(
4405 struct inode *inode,4437 struct inode *inode,
4406 pgoff_t fofs, block_t blkaddr,4438 pgoff_t fofs, block_t blkaddr,
@@ -4653,9 +4685,11 @@ static inline void f2fs_io_schedule_timeout(long timeout) · 246 unmodified lines
4653 io_schedule_timeout(timeout);4685 io_schedule_timeout(timeout);
4654}4686}
4655 4687
4656static inline void f2fs_handle_page_eio(struct f2fs_sb_info *sbi, pgoff_t ofs,4688static inline void f2fs_handle_page_eio(struct f2fs_sb_info *sbi,
4657 enum page_type type)4689 struct folio *folio, enum page_type type)
4658{4690{
4691 pgoff_t ofs = folio->index;
4692
4659 if (unlikely(f2fs_cp_error(sbi)))4693 if (unlikely(f2fs_cp_error(sbi)))
4660 return;4694 return;
4661 4695
fs/f2fs/file.c+129−70
@@ -8,7 +8,6 @@ · 7 unmodified lines
8#include <linux/fs.h>8#include <linux/fs.h>
9#include <linux/f2fs_fs.h>9#include <linux/f2fs_fs.h>
10#include <linux/stat.h>10#include <linux/stat.h>
11#include <linux/buffer_head.h>
12#include <linux/writeback.h>11#include <linux/writeback.h>
13#include <linux/blkdev.h>12#include <linux/blkdev.h>
14#include <linux/falloc.h>13#include <linux/falloc.h>
@@ -54,7 +53,7 @@ static vm_fault_t f2fs_filemap_fault(struct vm_fault *vmf) · 39 unmodified lines
54 53
55static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)54static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
56{55{
57 struct page *page = vmf->page;56 struct folio *folio = page_folio(vmf->page);
58 struct inode *inode = file_inode(vmf->vma->vm_file);57 struct inode *inode = file_inode(vmf->vma->vm_file);
59 struct f2fs_sb_info *sbi = F2FS_I_SB(inode);58 struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
60 struct dnode_of_data dn;59 struct dnode_of_data dn;
@@ -86,7 +85,7 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf) · 25 unmodified lines
86 85
87#ifdef CONFIG_F2FS_FS_COMPRESSION86#ifdef CONFIG_F2FS_FS_COMPRESSION
88 if (f2fs_compressed_file(inode)) {87 if (f2fs_compressed_file(inode)) {
89 int ret = f2fs_is_compressed_cluster(inode, page->index);88 int ret = f2fs_is_compressed_cluster(inode, folio->index);
90 89
91 if (ret < 0) {90 if (ret < 0) {
92 err = ret;91 err = ret;
@@ -106,11 +105,11 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf) · 13 unmodified lines
106 105
107 file_update_time(vmf->vma->vm_file);106 file_update_time(vmf->vma->vm_file);
108 filemap_invalidate_lock_shared(inode->i_mapping);107 filemap_invalidate_lock_shared(inode->i_mapping);
109 lock_page(page);108 folio_lock(folio);
110 if (unlikely(page->mapping != inode->i_mapping ||109 if (unlikely(folio->mapping != inode->i_mapping ||
111 page_offset(page) > i_size_read(inode) ||110 folio_pos(folio) > i_size_read(inode) ||
112 !PageUptodate(page))) {111 !folio_test_uptodate(folio))) {
113 unlock_page(page);112 folio_unlock(folio);
114 err = -EFAULT;113 err = -EFAULT;
115 goto out_sem;114 goto out_sem;
116 }115 }
@@ -118,9 +117,9 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf) · 1 unmodified line
118 set_new_dnode(&dn, inode, NULL, NULL, 0);117 set_new_dnode(&dn, inode, NULL, NULL, 0);
119 if (need_alloc) {118 if (need_alloc) {
120 /* block allocation */119 /* block allocation */
121 err = f2fs_get_block_locked(&dn, page->index);120 err = f2fs_get_block_locked(&dn, folio->index);
122 } else {121 } else {
123 err = f2fs_get_dnode_of_data(&dn, page->index, LOOKUP_NODE);122 err = f2fs_get_dnode_of_data(&dn, folio->index, LOOKUP_NODE);
124 f2fs_put_dnode(&dn);123 f2fs_put_dnode(&dn);
125 if (f2fs_is_pinned_file(inode) &&124 if (f2fs_is_pinned_file(inode) &&
126 !__is_valid_data_blkaddr(dn.data_blkaddr))125 !__is_valid_data_blkaddr(dn.data_blkaddr))
@@ -128,11 +127,11 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf) · 1 unmodified line
128 }127 }
129 128
130 if (err) {129 if (err) {
131 unlock_page(page);130 folio_unlock(folio);
132 goto out_sem;131 goto out_sem;
133 }132 }
134 133
135 f2fs_wait_on_page_writeback(page, DATA, false, true);134 f2fs_wait_on_page_writeback(folio_page(folio, 0), DATA, false, true);
136 135
137 /* wait for GCed page writeback via META_MAPPING */136 /* wait for GCed page writeback via META_MAPPING */
138 f2fs_wait_on_block_writeback(inode, dn.data_blkaddr);137 f2fs_wait_on_block_writeback(inode, dn.data_blkaddr);
@@ -140,18 +139,18 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf) · 1 unmodified line
140 /*139 /*
141 * check to see if the page is mapped already (no holes)140 * check to see if the page is mapped already (no holes)
142 */141 */
143 if (PageMappedToDisk(page))142 if (folio_test_mappedtodisk(folio))
144 goto out_sem;143 goto out_sem;
145 144
146 /* page is wholly or partially inside EOF */145 /* page is wholly or partially inside EOF */
147 if (((loff_t)(page->index + 1) << PAGE_SHIFT) >146 if (((loff_t)(folio->index + 1) << PAGE_SHIFT) >
148 i_size_read(inode)) {147 i_size_read(inode)) {
149 loff_t offset;148 loff_t offset;
150 149
151 offset = i_size_read(inode) & ~PAGE_MASK;150 offset = i_size_read(inode) & ~PAGE_MASK;
152 zero_user_segment(page, offset, PAGE_SIZE);151 folio_zero_segment(folio, offset, folio_size(folio));
153 }152 }
154 set_page_dirty(page);153 folio_mark_dirty(folio);
155 154
156 f2fs_update_iostat(sbi, inode, APP_MAPPED_IO, F2FS_BLKSIZE);155 f2fs_update_iostat(sbi, inode, APP_MAPPED_IO, F2FS_BLKSIZE);
157 f2fs_update_time(sbi, REQ_TIME);156 f2fs_update_time(sbi, REQ_TIME);
@@ -163,7 +162,7 @@ out_sem: · 5 unmodified lines
163out:162out:
164 ret = vmf_fs_error(err);163 ret = vmf_fs_error(err);
165 164
166 trace_f2fs_vm_page_mkwrite(inode, page->index, vmf->vma->vm_flags, ret);165 trace_f2fs_vm_page_mkwrite(inode, folio->index, vmf->vma->vm_flags, ret);
167 return ret;166 return ret;
168}167}
169 168
@@ -218,6 +217,9 @@ static inline enum cp_reason_type need_do_checkpoint(struct inode *inode) · 48 unmodified lines
218 f2fs_exist_written_data(sbi, F2FS_I(inode)->i_pino,217 f2fs_exist_written_data(sbi, F2FS_I(inode)->i_pino,
219 TRANS_DIR_INO))218 TRANS_DIR_INO))
220 cp_reason = CP_RECOVER_DIR;219 cp_reason = CP_RECOVER_DIR;
220 else if (f2fs_exist_written_data(sbi, F2FS_I(inode)->i_pino,
221 XATTR_DIR_INO))
222 cp_reason = CP_XATTR_DIR;
221 223
222 return cp_reason;224 return cp_reason;
223}225}
@@ -373,8 +375,7 @@ sync_nodes: · 149 unmodified lines
373 f2fs_remove_ino_entry(sbi, ino, APPEND_INO);375 f2fs_remove_ino_entry(sbi, ino, APPEND_INO);
374 clear_inode_flag(inode, FI_APPEND_WRITE);376 clear_inode_flag(inode, FI_APPEND_WRITE);
375flush_out:377flush_out:
376 if ((!atomic && F2FS_OPTION(sbi).fsync_mode != FSYNC_MODE_NOBARRIER) ||378 if (!atomic && F2FS_OPTION(sbi).fsync_mode != FSYNC_MODE_NOBARRIER)
377 (atomic && !test_opt(sbi, NOBARRIER) && f2fs_sb_has_blkzoned(sbi)))
378 ret = f2fs_issue_flush(sbi, inode->i_ino);379 ret = f2fs_issue_flush(sbi, inode->i_ino);
379 if (!ret) {380 if (!ret) {
380 f2fs_remove_ino_entry(sbi, ino, UPDATE_INO);381 f2fs_remove_ino_entry(sbi, ino, UPDATE_INO);
@@ -431,7 +432,7 @@ static bool __found_offset(struct address_space *mapping, · 50 unmodified lines
431static loff_t f2fs_seek_block(struct file *file, loff_t offset, int whence)432static loff_t f2fs_seek_block(struct file *file, loff_t offset, int whence)
432{433{
433 struct inode *inode = file->f_mapping->host;434 struct inode *inode = file->f_mapping->host;
434 loff_t maxbytes = inode->i_sb->s_maxbytes;435 loff_t maxbytes = F2FS_BLK_TO_BYTES(max_file_blocks(inode));
435 struct dnode_of_data dn;436 struct dnode_of_data dn;
436 pgoff_t pgofs, end_offset;437 pgoff_t pgofs, end_offset;
437 loff_t data_ofs = offset;438 loff_t data_ofs = offset;
@@ -513,10 +514,7 @@ fail: · 75 unmodified lines
513static loff_t f2fs_llseek(struct file *file, loff_t offset, int whence)514static loff_t f2fs_llseek(struct file *file, loff_t offset, int whence)
514{515{
515 struct inode *inode = file->f_mapping->host;516 struct inode *inode = file->f_mapping->host;
516 loff_t maxbytes = inode->i_sb->s_maxbytes;517 loff_t maxbytes = F2FS_BLK_TO_BYTES(max_file_blocks(inode));
517
518 if (f2fs_compressed_file(inode))
519 maxbytes = max_file_blocks(inode) << F2FS_BLKSIZE_BITS;
520 518
521 switch (whence) {519 switch (whence) {
522 case SEEK_SET:520 case SEEK_SET:
@@ -1052,6 +1050,13 @@ int f2fs_setattr(struct mnt_idmap *idmap, struct dentry *dentry, · 529 unmodified lines
1052 return err;1050 return err;
1053 }1051 }
1054 1052
1053 /*
1054 * wait for inflight dio, blocks should be removed after
1055 * IO completion.
1056 */
1057 if (attr->ia_size < old_size)
1058 inode_dio_wait(inode);
1059
1055 f2fs_down_write(&fi->i_gc_rwsem[WRITE]);1060 f2fs_down_write(&fi->i_gc_rwsem[WRITE]);
1056 filemap_invalidate_lock(inode->i_mapping);1061 filemap_invalidate_lock(inode->i_mapping);
1057 1062
@@ -1888,6 +1893,12 @@ static long f2fs_fallocate(struct file *file, int mode, · 830 unmodified lines
1888 if (ret)1893 if (ret)
1889 goto out;1894 goto out;
1890 1895
1896 /*
1897 * wait for inflight dio, blocks should be removed after IO
1898 * completion.
1899 */
1900 inode_dio_wait(inode);
1901
1891 if (mode & FALLOC_FL_PUNCH_HOLE) {1902 if (mode & FALLOC_FL_PUNCH_HOLE) {
1892 if (offset >= inode->i_size)1903 if (offset >= inode->i_size)
1893 goto out;1904 goto out;
@@ -2116,10 +2127,12 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate) · 222 unmodified lines
2116 struct mnt_idmap *idmap = file_mnt_idmap(filp);2127 struct mnt_idmap *idmap = file_mnt_idmap(filp);
2117 struct f2fs_inode_info *fi = F2FS_I(inode);2128 struct f2fs_inode_info *fi = F2FS_I(inode);
2118 struct f2fs_sb_info *sbi = F2FS_I_SB(inode);2129 struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
2119 struct inode *pinode;
2120 loff_t isize;2130 loff_t isize;
2121 int ret;2131 int ret;
2122 2132
2133 if (!(filp->f_mode & FMODE_WRITE))
2134 return -EBADF;
2135
2123 if (!inode_owner_or_capable(idmap, inode))2136 if (!inode_owner_or_capable(idmap, inode))
2124 return -EACCES;2137 return -EACCES;
2125 2138
@@ -2149,6 +2162,7 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate) · 23 unmodified lines
2149 goto out;2162 goto out;
2150 2163
2151 f2fs_down_write(&fi->i_gc_rwsem[WRITE]);2164 f2fs_down_write(&fi->i_gc_rwsem[WRITE]);
2165 f2fs_down_write(&fi->i_gc_rwsem[READ]);
2152 2166
2153 /*2167 /*
2154 * Should wait end_io to count F2FS_WB_CP_DATA correctly by2168 * Should wait end_io to count F2FS_WB_CP_DATA correctly by
@@ -2158,27 +2172,18 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate) · 3 unmodified lines
2158 f2fs_warn(sbi, "Unexpected flush for atomic writes: ino=%lu, npages=%u",2172 f2fs_warn(sbi, "Unexpected flush for atomic writes: ino=%lu, npages=%u",
2159 inode->i_ino, get_dirty_pages(inode));2173 inode->i_ino, get_dirty_pages(inode));
2160 ret = filemap_write_and_wait_range(inode->i_mapping, 0, LLONG_MAX);2174 ret = filemap_write_and_wait_range(inode->i_mapping, 0, LLONG_MAX);
2161 if (ret) {2175 if (ret)
2162 f2fs_up_write(&fi->i_gc_rwsem[WRITE]);2176 goto out_unlock;
2163 goto out;
2164 }
2165 2177
2166 /* Check if the inode already has a COW inode */2178 /* Check if the inode already has a COW inode */
2167 if (fi->cow_inode == NULL) {2179 if (fi->cow_inode == NULL) {
2168 /* Create a COW inode for atomic write */2180 /* Create a COW inode for atomic write */
2169 pinode = f2fs_iget(inode->i_sb, fi->i_pino);2181 struct dentry *dentry = file_dentry(filp);
2170 if (IS_ERR(pinode)) {2182 struct inode *dir = d_inode(dentry->d_parent);
2171 f2fs_up_write(&fi->i_gc_rwsem[WRITE]);
2172 ret = PTR_ERR(pinode);
2173 goto out;
2174 }
2175 2183
2176 ret = f2fs_get_tmpfile(idmap, pinode, &fi->cow_inode);2184 ret = f2fs_get_tmpfile(idmap, dir, &fi->cow_inode);
2177 iput(pinode);2185 if (ret)
2178 if (ret) {2186 goto out_unlock;
2179 f2fs_up_write(&fi->i_gc_rwsem[WRITE]);
2180 goto out;
2181 }
2182 2187
2183 set_inode_flag(fi->cow_inode, FI_COW_FILE);2188 set_inode_flag(fi->cow_inode, FI_COW_FILE);
2184 clear_inode_flag(fi->cow_inode, FI_INLINE_DATA);2189 clear_inode_flag(fi->cow_inode, FI_INLINE_DATA);
@@ -2187,11 +2192,13 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate) · 2 unmodified lines
2187 F2FS_I(fi->cow_inode)->atomic_inode = inode;2192 F2FS_I(fi->cow_inode)->atomic_inode = inode;
2188 } else {2193 } else {
2189 /* Reuse the already created COW inode */2194 /* Reuse the already created COW inode */
2195 f2fs_bug_on(sbi, get_dirty_pages(fi->cow_inode));
2196
2197 invalidate_mapping_pages(fi->cow_inode->i_mapping, 0, -1);
2198
2190 ret = f2fs_do_truncate_blocks(fi->cow_inode, 0, true);2199 ret = f2fs_do_truncate_blocks(fi->cow_inode, 0, true);
2191 if (ret) {2200 if (ret)
2192 f2fs_up_write(&fi->i_gc_rwsem[WRITE]);2201 goto out_unlock;
2193 goto out;
2194 }
2195 }2202 }
2196 2203
2197 f2fs_write_inode(inode, NULL);2204 f2fs_write_inode(inode, NULL);
@@ -2210,7 +2217,11 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate) · 12 unmodified lines
2210 }2217 }
2211 f2fs_i_size_write(fi->cow_inode, isize);2218 f2fs_i_size_write(fi->cow_inode, isize);
2212 2219
2220out_unlock:
2221 f2fs_up_write(&fi->i_gc_rwsem[READ]);
2213 f2fs_up_write(&fi->i_gc_rwsem[WRITE]);2222 f2fs_up_write(&fi->i_gc_rwsem[WRITE]);
2223 if (ret)
2224 goto out;
2214 2225
2215 f2fs_update_time(sbi, REQ_TIME);2226 f2fs_update_time(sbi, REQ_TIME);
2216 fi->atomic_write_task = current;2227 fi->atomic_write_task = current;
@@ -2228,6 +2239,9 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) · 11 unmodified lines
2228 struct mnt_idmap *idmap = file_mnt_idmap(filp);2239 struct mnt_idmap *idmap = file_mnt_idmap(filp);
2229 int ret;2240 int ret;
2230 2241
2242 if (!(filp->f_mode & FMODE_WRITE))
2243 return -EBADF;
2244
2231 if (!inode_owner_or_capable(idmap, inode))2245 if (!inode_owner_or_capable(idmap, inode))
2232 return -EACCES;2246 return -EACCES;
2233 2247
@@ -2260,6 +2274,9 @@ static int f2fs_ioc_abort_atomic_write(struct file *filp) · 26 unmodified lines
2260 struct mnt_idmap *idmap = file_mnt_idmap(filp);2274 struct mnt_idmap *idmap = file_mnt_idmap(filp);
2261 int ret;2275 int ret;
2262 2276
2277 if (!(filp->f_mode & FMODE_WRITE))
2278 return -EBADF;
2279
2263 if (!inode_owner_or_capable(idmap, inode))2280 if (!inode_owner_or_capable(idmap, inode))
2264 return -EACCES;2281 return -EACCES;
2265 2282
@@ -2279,7 +2296,7 @@ static int f2fs_ioc_abort_atomic_write(struct file *filp) · 13 unmodified lines
2279}2296}
2280 2297
2281int f2fs_do_shutdown(struct f2fs_sb_info *sbi, unsigned int flag,2298int f2fs_do_shutdown(struct f2fs_sb_info *sbi, unsigned int flag,
2282 bool readonly)2299 bool readonly, bool need_lock)
2283{2300{
2284 struct super_block *sb = sbi->sb;2301 struct super_block *sb = sbi->sb;
2285 int ret = 0;2302 int ret = 0;
@@ -2326,12 +2343,19 @@ int f2fs_do_shutdown(struct f2fs_sb_info *sbi, unsigned int flag, · 40 unmodified lines
2326 if (readonly)2343 if (readonly)
2327 goto out;2344 goto out;
2328 2345
2346 /* grab sb->s_umount to avoid racing w/ remount() */
2347 if (need_lock)
2348 down_read(&sbi->sb->s_umount);
2349
2329 f2fs_stop_gc_thread(sbi);2350 f2fs_stop_gc_thread(sbi);
2330 f2fs_stop_discard_thread(sbi);2351 f2fs_stop_discard_thread(sbi);
2331 2352
2332 f2fs_drop_discard_cmd(sbi);2353 f2fs_drop_discard_cmd(sbi);
2333 clear_opt(sbi, DISCARD);2354 clear_opt(sbi, DISCARD);
2334 2355
2356 if (need_lock)
2357 up_read(&sbi->sb->s_umount);
2358
2335 f2fs_update_time(sbi, REQ_TIME);2359 f2fs_update_time(sbi, REQ_TIME);
2336out:2360out:
2337 2361
@@ -2368,7 +2392,7 @@ static int f2fs_ioc_shutdown(struct file *filp, unsigned long arg) · 30 unmodified lines
2368 }2392 }
2369 }2393 }
2370 2394
2371 ret = f2fs_do_shutdown(sbi, in, readonly);2395 ret = f2fs_do_shutdown(sbi, in, readonly, true);
2372 2396
2373 if (need_drop)2397 if (need_drop)
2374 mnt_drop_write_file(filp);2398 mnt_drop_write_file(filp);
@@ -2686,7 +2710,8 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi, · 311 unmodified lines
2686 (range->start + range->len) >> PAGE_SHIFT,2710 (range->start + range->len) >> PAGE_SHIFT,
2687 DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE));2711 DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE));
2688 2712
2689 if (is_inode_flag_set(inode, FI_COMPRESS_RELEASED)) {2713 if (is_inode_flag_set(inode, FI_COMPRESS_RELEASED) ||
2714 f2fs_is_atomic_file(inode)) {
2690 err = -EINVAL;2715 err = -EINVAL;
2691 goto unlock_out;2716 goto unlock_out;
2692 }2717 }
@@ -2710,7 +2735,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi, · 17 unmodified lines
2710 * block addresses are continuous.2735 * block addresses are continuous.
2711 */2736 */
2712 if (f2fs_lookup_read_extent_cache(inode, pg_start, &ei)) {2737 if (f2fs_lookup_read_extent_cache(inode, pg_start, &ei)) {
2713 if (ei.fofs + ei.len >= pg_end)2738 if ((pgoff_t)ei.fofs + ei.len >= pg_end)
2714 goto out;2739 goto out;
2715 }2740 }
2716 2741
@@ -2793,6 +2818,8 @@ do_map: · 76 unmodified lines
2793 goto clear_out;2818 goto clear_out;
2794 }2819 }
2795 2820
2821 f2fs_wait_on_page_writeback(page, DATA, true, true);
2822
2796 set_page_dirty(page);2823 set_page_dirty(page);
2797 set_page_private_gcing(page);2824 set_page_private_gcing(page);
2798 f2fs_put_page(page, 1);2825 f2fs_put_page(page, 1);
@@ -2917,6 +2944,11 @@ static int f2fs_move_file_range(struct file *file_in, loff_t pos_in, · 118 unmodified lines
2917 goto out_unlock;2944 goto out_unlock;
2918 }2945 }
2919 2946
2947 if (f2fs_is_atomic_file(src) || f2fs_is_atomic_file(dst)) {
2948 ret = -EINVAL;
2949 goto out_unlock;
2950 }
2951
2920 ret = -EINVAL;2952 ret = -EINVAL;
2921 if (pos_in + len > src->i_size || pos_in + len < pos_in)2953 if (pos_in + len > src->i_size || pos_in + len < pos_in)
2922 goto out_unlock;2954 goto out_unlock;
@@ -2968,9 +3000,9 @@ static int f2fs_move_file_range(struct file *file_in, loff_t pos_in, · 45 unmodified lines
2968 }3000 }
2969 3001
2970 f2fs_lock_op(sbi);3002 f2fs_lock_op(sbi);
2971 ret = __exchange_data_block(src, dst, pos_in >> F2FS_BLKSIZE_BITS,3003 ret = __exchange_data_block(src, dst, F2FS_BYTES_TO_BLK(pos_in),
2972 pos_out >> F2FS_BLKSIZE_BITS,3004 F2FS_BYTES_TO_BLK(pos_out),
2973 len >> F2FS_BLKSIZE_BITS, false);3005 F2FS_BYTES_TO_BLK(len), false);
2974 3006
2975 if (!ret) {3007 if (!ret) {
2976 if (dst_max_i_size)3008 if (dst_max_i_size)
@@ -3300,6 +3332,11 @@ static int f2fs_ioc_set_pin_file(struct file *filp, unsigned long arg) · 323 unmodified lines
3300 3332
3301 inode_lock(inode);3333 inode_lock(inode);
3302 3334
3335 if (f2fs_is_atomic_file(inode)) {
3336 ret = -EINVAL;
3337 goto out;
3338 }
3339
3303 if (!pin) {3340 if (!pin) {
3304 clear_inode_flag(inode, FI_PIN_FILE);3341 clear_inode_flag(inode, FI_PIN_FILE);
3305 f2fs_i_gc_failures_write(inode, 0);3342 f2fs_i_gc_failures_write(inode, 0);
@@ -4193,6 +4230,8 @@ static int redirty_blocks(struct inode *inode, pgoff_t page_idx, int len) · 887 unmodified lines
4193 /* It will never fail, when page has pinned above */4230 /* It will never fail, when page has pinned above */
4194 f2fs_bug_on(F2FS_I_SB(inode), !page);4231 f2fs_bug_on(F2FS_I_SB(inode), !page);
4195 4232
4233 f2fs_wait_on_page_writeback(page, DATA, true, true);
4234
4196 set_page_dirty(page);4235 set_page_dirty(page);
4197 set_page_private_gcing(page);4236 set_page_private_gcing(page);
4198 f2fs_put_page(page, 1);4237 f2fs_put_page(page, 1);
@@ -4207,9 +4246,8 @@ static int f2fs_ioc_decompress_file(struct file *filp) · 8 unmodified lines
4207 struct inode *inode = file_inode(filp);4246 struct inode *inode = file_inode(filp);
4208 struct f2fs_sb_info *sbi = F2FS_I_SB(inode);4247 struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
4209 struct f2fs_inode_info *fi = F2FS_I(inode);4248 struct f2fs_inode_info *fi = F2FS_I(inode);
4210 pgoff_t page_idx = 0, last_idx;4249 pgoff_t page_idx = 0, last_idx, cluster_idx;
4211 int cluster_size = fi->i_cluster_size;4250 int ret;
4212 int count, ret;
4213 4251
4214 if (!f2fs_sb_has_compression(sbi) ||4252 if (!f2fs_sb_has_compression(sbi) ||
4215 F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)4253 F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)
@@ -4244,10 +4282,15 @@ static int f2fs_ioc_decompress_file(struct file *filp) · 28 unmodified lines
4244 goto out;4282 goto out;
4245 4283
4246 last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);4284 last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
4285 last_idx >>= fi->i_log_cluster_size;
4247 4286
4248 count = last_idx - page_idx;4287 for (cluster_idx = 0; cluster_idx < last_idx; cluster_idx++) {
4249 while (count && count >= cluster_size) {4288 page_idx = cluster_idx << fi->i_log_cluster_size;
4250 ret = redirty_blocks(inode, page_idx, cluster_size);4289
4290 if (!f2fs_is_compressed_cluster(inode, page_idx))
4291 continue;
4292
4293 ret = redirty_blocks(inode, page_idx, fi->i_cluster_size);
4251 if (ret < 0)4294 if (ret < 0)
4252 break;4295 break;
4253 4296
@@ -4257,9 +4300,6 @@ static int f2fs_ioc_decompress_file(struct file *filp) · 3 unmodified lines
4257 break;4300 break;
4258 }4301 }
4259 4302
4260 count -= cluster_size;
4261 page_idx += cluster_size;
4262
4263 cond_resched();4303 cond_resched();
4264 if (fatal_signal_pending(current)) {4304 if (fatal_signal_pending(current)) {
4265 ret = -EINTR;4305 ret = -EINTR;
@@ -4286,9 +4326,9 @@ static int f2fs_ioc_compress_file(struct file *filp) · 20 unmodified lines
4286{4326{
4287 struct inode *inode = file_inode(filp);4327 struct inode *inode = file_inode(filp);
4288 struct f2fs_sb_info *sbi = F2FS_I_SB(inode);4328 struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
4289 pgoff_t page_idx = 0, last_idx;4329 struct f2fs_inode_info *fi = F2FS_I(inode);
4290 int cluster_size = F2FS_I(inode)->i_cluster_size;4330 pgoff_t page_idx = 0, last_idx, cluster_idx;
4291 int count, ret;4331 int ret;
4292 4332
4293 if (!f2fs_sb_has_compression(sbi) ||4333 if (!f2fs_sb_has_compression(sbi) ||
4294 F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)4334 F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)
@@ -4322,10 +4362,15 @@ static int f2fs_ioc_compress_file(struct file *filp) · 27 unmodified lines
4322 set_inode_flag(inode, FI_ENABLE_COMPRESS);4362 set_inode_flag(inode, FI_ENABLE_COMPRESS);
4323 4363
4324 last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);4364 last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
4365 last_idx >>= fi->i_log_cluster_size;
4366
4367 for (cluster_idx = 0; cluster_idx < last_idx; cluster_idx++) {
4368 page_idx = cluster_idx << fi->i_log_cluster_size;
4369
4370 if (f2fs_is_sparse_cluster(inode, page_idx))
4371 continue;
4325 4372
4326 count = last_idx - page_idx;4373 ret = redirty_blocks(inode, page_idx, fi->i_cluster_size);
4327 while (count && count >= cluster_size) {
4328 ret = redirty_blocks(inode, page_idx, cluster_size);
4329 if (ret < 0)4374 if (ret < 0)
4330 break;4375 break;
4331 4376
@@ -4335,9 +4380,6 @@ static int f2fs_ioc_compress_file(struct file *filp) · 3 unmodified lines
4335 break;4380 break;
4336 }4381 }
4337 4382
4338 count -= cluster_size;
4339 page_idx += cluster_size;
4340
4341 cond_resched();4383 cond_resched();
4342 if (fatal_signal_pending(current)) {4384 if (fatal_signal_pending(current)) {
4343 ret = -EINTR;4385 ret = -EINTR;
@@ -4538,6 +4580,13 @@ static ssize_t f2fs_dio_read_iter(struct kiocb *iocb, struct iov_iter *to) · 194 unmodified lines
4538 f2fs_down_read(&fi->i_gc_rwsem[READ]);4580 f2fs_down_read(&fi->i_gc_rwsem[READ]);
4539 }4581 }
4540 4582
4583 /* dio is not compatible w/ atomic file */
4584 if (f2fs_is_atomic_file(inode)) {
4585 f2fs_up_read(&fi->i_gc_rwsem[READ]);
4586 ret = -EOPNOTSUPP;
4587 goto out;
4588 }
4589
4541 /*4590 /*
4542 * We have to use __iomap_dio_rw() and iomap_dio_complete() instead of4591 * We have to use __iomap_dio_rw() and iomap_dio_complete() instead of
4543 * the higher-level function iomap_dio_rw() in order to ensure that the4592 * the higher-level function iomap_dio_rw() in order to ensure that the
@@ -4597,6 +4646,10 @@ static ssize_t f2fs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) · 53 unmodified lines
4597 f2fs_trace_rw_file_path(iocb->ki_filp, iocb->ki_pos,4646 f2fs_trace_rw_file_path(iocb->ki_filp, iocb->ki_pos,
4598 iov_iter_count(to), READ);4647 iov_iter_count(to), READ);
4599 4648
4649 /* In LFS mode, if there is inflight dio, wait for its completion */
4650 if (f2fs_lfs_mode(F2FS_I_SB(inode)))
4651 inode_dio_wait(inode);
4652
4600 if (f2fs_should_use_dio(inode, iocb, to)) {4653 if (f2fs_should_use_dio(inode, iocb, to)) {
4601 ret = f2fs_dio_read_iter(iocb, to);4654 ret = f2fs_dio_read_iter(iocb, to);
4602 } else {4655 } else {
@@ -4949,6 +5002,12 @@ static ssize_t f2fs_file_write_iter(struct kiocb *iocb, struct iov_iter *from) · 346 unmodified lines
4949 /* Determine whether we will do a direct write or a buffered write. */5002 /* Determine whether we will do a direct write or a buffered write. */
4950 dio = f2fs_should_use_dio(inode, iocb, from);5003 dio = f2fs_should_use_dio(inode, iocb, from);
4951 5004
5005 /* dio is not compatible w/ atomic write */
5006 if (dio && f2fs_is_atomic_file(inode)) {
5007 ret = -EOPNOTSUPP;
5008 goto out_unlock;
5009 }
5010
4952 /* Possibly preallocate the blocks for the write. */5011 /* Possibly preallocate the blocks for the write. */
4953 target_size = iocb->ki_pos + iov_iter_count(from);5012 target_size = iocb->ki_pos + iov_iter_count(from);
4954 preallocated = f2fs_preallocate_blocks(iocb, from, dio);5013 preallocated = f2fs_preallocate_blocks(iocb, from, dio);
fs/f2fs/gc.c+86−27
@@ -81,6 +81,8 @@ static int gc_thread_func(void *data) · 80 unmodified lines
81 continue;81 continue;
82 }82 }
83 83
84 gc_control.one_time = false;
85
84 /*86 /*
85 * [GC triggering condition]87 * [GC triggering condition]
86 * 0. GC is not conducted currently.88 * 0. GC is not conducted currently.
@@ -116,15 +118,30 @@ static int gc_thread_func(void *data) · 29 unmodified lines
116 goto next;118 goto next;
117 }119 }
118 120
119 if (has_enough_invalid_blocks(sbi))121 if (f2fs_sb_has_blkzoned(sbi)) {
122 if (has_enough_free_blocks(sbi,
123 gc_th->no_zoned_gc_percent)) {
124 wait_ms = gc_th->no_gc_sleep_time;
125 f2fs_up_write(&sbi->gc_lock);
126 goto next;
127 }
128 if (wait_ms == gc_th->no_gc_sleep_time)
129 wait_ms = gc_th->max_sleep_time;
130 }
131
132 if (need_to_boost_gc(sbi)) {
120 decrease_sleep_time(gc_th, &wait_ms);133 decrease_sleep_time(gc_th, &wait_ms);
121 else134 if (f2fs_sb_has_blkzoned(sbi))
135 gc_control.one_time = true;
136 } else {
122 increase_sleep_time(gc_th, &wait_ms);137 increase_sleep_time(gc_th, &wait_ms);
138 }
123do_gc:139do_gc:
124 stat_inc_gc_call_count(sbi, foreground ?140 stat_inc_gc_call_count(sbi, foreground ?
125 FOREGROUND : BACKGROUND);141 FOREGROUND : BACKGROUND);
126 142
127 sync_mode = F2FS_OPTION(sbi).bggc_mode == BGGC_MODE_SYNC;143 sync_mode = (F2FS_OPTION(sbi).bggc_mode == BGGC_MODE_SYNC) ||
144 gc_control.one_time;
128 145
129 /* foreground GC was been triggered via f2fs_balance_fs() */146 /* foreground GC was been triggered via f2fs_balance_fs() */
130 if (foreground)147 if (foreground)
@@ -179,9 +196,21 @@ int f2fs_start_gc_thread(struct f2fs_sb_info *sbi) · 48 unmodified lines
179 return -ENOMEM;196 return -ENOMEM;
180 197
181 gc_th->urgent_sleep_time = DEF_GC_THREAD_URGENT_SLEEP_TIME;198 gc_th->urgent_sleep_time = DEF_GC_THREAD_URGENT_SLEEP_TIME;
182 gc_th->min_sleep_time = DEF_GC_THREAD_MIN_SLEEP_TIME;199 gc_th->valid_thresh_ratio = DEF_GC_THREAD_VALID_THRESH_RATIO;
183 gc_th->max_sleep_time = DEF_GC_THREAD_MAX_SLEEP_TIME;200
184 gc_th->no_gc_sleep_time = DEF_GC_THREAD_NOGC_SLEEP_TIME;201 if (f2fs_sb_has_blkzoned(sbi)) {
202 gc_th->min_sleep_time = DEF_GC_THREAD_MIN_SLEEP_TIME_ZONED;
203 gc_th->max_sleep_time = DEF_GC_THREAD_MAX_SLEEP_TIME_ZONED;
204 gc_th->no_gc_sleep_time = DEF_GC_THREAD_NOGC_SLEEP_TIME_ZONED;
205 gc_th->no_zoned_gc_percent = LIMIT_NO_ZONED_GC;
206 gc_th->boost_zoned_gc_percent = LIMIT_BOOST_ZONED_GC;
207 } else {
208 gc_th->min_sleep_time = DEF_GC_THREAD_MIN_SLEEP_TIME;
209 gc_th->max_sleep_time = DEF_GC_THREAD_MAX_SLEEP_TIME;
210 gc_th->no_gc_sleep_time = DEF_GC_THREAD_NOGC_SLEEP_TIME;
211 gc_th->no_zoned_gc_percent = 0;
212 gc_th->boost_zoned_gc_percent = 0;
213 }
185 214
186 gc_th->gc_wake = false;215 gc_th->gc_wake = false;
187 216
@@ -339,7 +368,7 @@ static unsigned int get_cb_cost(struct f2fs_sb_info *sbi, unsigned int segno) · 151 unmodified lines
339 unsigned char age = 0;368 unsigned char age = 0;
340 unsigned char u;369 unsigned char u;
341 unsigned int i;370 unsigned int i;
342 unsigned int usable_segs_per_sec = f2fs_usable_segs_in_sec(sbi, segno);371 unsigned int usable_segs_per_sec = f2fs_usable_segs_in_sec(sbi);
343 372
344 for (i = 0; i < usable_segs_per_sec; i++)373 for (i = 0; i < usable_segs_per_sec; i++)
345 mtime += get_seg_entry(sbi, start + i)->mtime;374 mtime += get_seg_entry(sbi, start + i)->mtime;
@@ -368,6 +397,11 @@ static inline unsigned int get_gc_cost(struct f2fs_sb_info *sbi, · 22 unmodified lines
368 if (p->alloc_mode == SSR)397 if (p->alloc_mode == SSR)
369 return get_seg_entry(sbi, segno)->ckpt_valid_blocks;398 return get_seg_entry(sbi, segno)->ckpt_valid_blocks;
370 399
400 if (p->one_time_gc && (get_valid_blocks(sbi, segno, true) >=
401 CAP_BLKS_PER_SEC(sbi) * sbi->gc_thread->valid_thresh_ratio /
402 100))
403 return UINT_MAX;
404
371 /* alloc_mode == LFS */405 /* alloc_mode == LFS */
372 if (p->gc_mode == GC_GREEDY)406 if (p->gc_mode == GC_GREEDY)
373 return get_valid_blocks(sbi, segno, true);407 return get_valid_blocks(sbi, segno, true);
@@ -742,7 +776,7 @@ static int f2fs_gc_pinned_control(struct inode *inode, int gc_type, · 368 unmodified lines
742 */776 */
743int f2fs_get_victim(struct f2fs_sb_info *sbi, unsigned int *result,777int f2fs_get_victim(struct f2fs_sb_info *sbi, unsigned int *result,
744 int gc_type, int type, char alloc_mode,778 int gc_type, int type, char alloc_mode,
745 unsigned long long age)779 unsigned long long age, bool one_time)
746{780{
747 struct dirty_seglist_info *dirty_i = DIRTY_I(sbi);781 struct dirty_seglist_info *dirty_i = DIRTY_I(sbi);
748 struct sit_info *sm = SIT_I(sbi);782 struct sit_info *sm = SIT_I(sbi);
@@ -759,6 +793,7 @@ int f2fs_get_victim(struct f2fs_sb_info *sbi, unsigned int *result, · 10 unmodified lines
759 p.alloc_mode = alloc_mode;793 p.alloc_mode = alloc_mode;
760 p.age = age;794 p.age = age;
761 p.age_threshold = sbi->am.age_threshold;795 p.age_threshold = sbi->am.age_threshold;
796 p.one_time_gc = one_time;
762 797
763retry:798retry:
764 select_policy(sbi, gc_type, type, &p);799 select_policy(sbi, gc_type, type, &p);
@@ -1670,13 +1705,14 @@ next_step: · 905 unmodified lines
1670}1705}
1671 1706
1672static int __get_victim(struct f2fs_sb_info *sbi, unsigned int *victim,1707static int __get_victim(struct f2fs_sb_info *sbi, unsigned int *victim,
1673 int gc_type)1708 int gc_type, bool one_time)
1674{1709{
1675 struct sit_info *sit_i = SIT_I(sbi);1710 struct sit_info *sit_i = SIT_I(sbi);
1676 int ret;1711 int ret;
1677 1712
1678 down_write(&sit_i->sentry_lock);1713 down_write(&sit_i->sentry_lock);
1679 ret = f2fs_get_victim(sbi, victim, gc_type, NO_CHECK_TYPE, LFS, 0);1714 ret = f2fs_get_victim(sbi, victim, gc_type, NO_CHECK_TYPE,
1715 LFS, 0, one_time);
1680 up_write(&sit_i->sentry_lock);1716 up_write(&sit_i->sentry_lock);
1681 return ret;1717 return ret;
1682}1718}
@@ -1684,30 +1720,49 @@ static int __get_victim(struct f2fs_sb_info *sbi, unsigned int *victim, · 1 unmodified line
1684static int do_garbage_collect(struct f2fs_sb_info *sbi,1720static int do_garbage_collect(struct f2fs_sb_info *sbi,
1685 unsigned int start_segno,1721 unsigned int start_segno,
1686 struct gc_inode_list *gc_list, int gc_type,1722 struct gc_inode_list *gc_list, int gc_type,
1687 bool force_migrate)1723 bool force_migrate, bool one_time)
1688{1724{
1689 struct page *sum_page;1725 struct page *sum_page;
1690 struct f2fs_summary_block *sum;1726 struct f2fs_summary_block *sum;
1691 struct blk_plug plug;1727 struct blk_plug plug;
1692 unsigned int segno = start_segno;1728 unsigned int segno = start_segno;
1693 unsigned int end_segno = start_segno + SEGS_PER_SEC(sbi);1729 unsigned int end_segno = start_segno + SEGS_PER_SEC(sbi);
1730 unsigned int sec_end_segno;
1694 int seg_freed = 0, migrated = 0;1731 int seg_freed = 0, migrated = 0;
1695 unsigned char type = IS_DATASEG(get_seg_entry(sbi, segno)->type) ?1732 unsigned char type = IS_DATASEG(get_seg_entry(sbi, segno)->type) ?
1696 SUM_TYPE_DATA : SUM_TYPE_NODE;1733 SUM_TYPE_DATA : SUM_TYPE_NODE;
1697 unsigned char data_type = (type == SUM_TYPE_DATA) ? DATA : NODE;1734 unsigned char data_type = (type == SUM_TYPE_DATA) ? DATA : NODE;
1698 int submitted = 0;1735 int submitted = 0;
1699 1736
1700 if (__is_large_section(sbi))1737 if (__is_large_section(sbi)) {
1701 end_segno = rounddown(end_segno, SEGS_PER_SEC(sbi));1738 sec_end_segno = rounddown(end_segno, SEGS_PER_SEC(sbi));
1702 1739
1703 /*1740 /*
1704 * zone-capacity can be less than zone-size in zoned devices,1741 * zone-capacity can be less than zone-size in zoned devices,
1705 * resulting in less than expected usable segments in the zone,1742 * resulting in less than expected usable segments in the zone,
1706 * calculate the end segno in the zone which can be garbage collected1743 * calculate the end segno in the zone which can be garbage
1707 */1744 * collected
1708 if (f2fs_sb_has_blkzoned(sbi))1745 */
1709 end_segno -= SEGS_PER_SEC(sbi) -1746 if (f2fs_sb_has_blkzoned(sbi))
1710 f2fs_usable_segs_in_sec(sbi, segno);1747 sec_end_segno -= SEGS_PER_SEC(sbi) -
1748 f2fs_usable_segs_in_sec(sbi);
1749
1750 if (gc_type == BG_GC || one_time) {
1751 unsigned int window_granularity =
1752 sbi->migration_window_granularity;
1753
1754 if (f2fs_sb_has_blkzoned(sbi) &&
1755 !has_enough_free_blocks(sbi,
1756 sbi->gc_thread->boost_zoned_gc_percent))
1757 window_granularity *=
1758 BOOST_GC_MULTIPLE;
1759
1760 end_segno = start_segno + window_granularity;
1761 }
1762
1763 if (end_segno > sec_end_segno)
1764 end_segno = sec_end_segno;
1765 }
1711 1766
1712 sanity_check_seg_type(sbi, get_seg_entry(sbi, segno)->type);1767 sanity_check_seg_type(sbi, get_seg_entry(sbi, segno)->type);
1713 1768
@@ -1786,7 +1841,8 @@ freed: · 72 unmodified lines
1786 1841
1787 if (__is_large_section(sbi))1842 if (__is_large_section(sbi))
1788 sbi->next_victim_seg[gc_type] =1843 sbi->next_victim_seg[gc_type] =
1789 (segno + 1 < end_segno) ? segno + 1 : NULL_SEGNO;1844 (segno + 1 < sec_end_segno) ?
1845 segno + 1 : NULL_SEGNO;
1790skip:1846skip:
1791 f2fs_put_page(sum_page, 0);1847 f2fs_put_page(sum_page, 0);
1792 }1848 }
@@ -1863,7 +1919,7 @@ gc_more: · 70 unmodified lines
1863 goto stop;1919 goto stop;
1864 }1920 }
1865retry:1921retry:
1866 ret = __get_victim(sbi, &segno, gc_type);1922 ret = __get_victim(sbi, &segno, gc_type, gc_control->one_time);
1867 if (ret) {1923 if (ret) {
1868 /* allow to search victim from sections has pinned data */1924 /* allow to search victim from sections has pinned data */
1869 if (ret == -ENODATA && gc_type == FG_GC &&1925 if (ret == -ENODATA && gc_type == FG_GC &&
@@ -1875,17 +1931,21 @@ retry: · 5 unmodified lines
1875 }1931 }
1876 1932
1877 seg_freed = do_garbage_collect(sbi, segno, &gc_list, gc_type,1933 seg_freed = do_garbage_collect(sbi, segno, &gc_list, gc_type,
1878 gc_control->should_migrate_blocks);1934 gc_control->should_migrate_blocks,
1935 gc_control->one_time);
1879 if (seg_freed < 0)1936 if (seg_freed < 0)
1880 goto stop;1937 goto stop;
1881 1938
1882 total_freed += seg_freed;1939 total_freed += seg_freed;
1883 1940
1884 if (seg_freed == f2fs_usable_segs_in_sec(sbi, segno)) {1941 if (seg_freed == f2fs_usable_segs_in_sec(sbi)) {
1885 sec_freed++;1942 sec_freed++;
1886 total_sec_freed++;1943 total_sec_freed++;
1887 }1944 }
1888 1945
1946 if (gc_control->one_time)
1947 goto stop;
1948
1889 if (gc_type == FG_GC) {1949 if (gc_type == FG_GC) {
1890 sbi->cur_victim_sec = NULL_SEGNO;1950 sbi->cur_victim_sec = NULL_SEGNO;
1891 1951
@@ -2010,8 +2070,7 @@ int f2fs_gc_range(struct f2fs_sb_info *sbi, · 118 unmodified lines
2010 .iroot = RADIX_TREE_INIT(gc_list.iroot, GFP_NOFS),2070 .iroot = RADIX_TREE_INIT(gc_list.iroot, GFP_NOFS),
2011 };2071 };
2012 2072
2013 do_garbage_collect(sbi, segno, &gc_list, FG_GC,2073 do_garbage_collect(sbi, segno, &gc_list, FG_GC, true, false);
2014 dry_run_sections == 0);
2015 put_gc_inode(&gc_list);2074 put_gc_inode(&gc_list);
2016 2075
2017 if (!dry_run && get_valid_blocks(sbi, segno, true))2076 if (!dry_run && get_valid_blocks(sbi, segno, true))
fs/f2fs/gc.h+29−0
@@ -15,16 +15,27 @@ · 14 unmodified lines
15#define DEF_GC_THREAD_MAX_SLEEP_TIME 6000015#define DEF_GC_THREAD_MAX_SLEEP_TIME 60000
16#define DEF_GC_THREAD_NOGC_SLEEP_TIME 300000 /* wait 5 min */16#define DEF_GC_THREAD_NOGC_SLEEP_TIME 300000 /* wait 5 min */
17 17
18/* GC sleep parameters for zoned deivces */
19#define DEF_GC_THREAD_MIN_SLEEP_TIME_ZONED 10
20#define DEF_GC_THREAD_MAX_SLEEP_TIME_ZONED 20
21#define DEF_GC_THREAD_NOGC_SLEEP_TIME_ZONED 60000
22
18/* choose candidates from sections which has age of more than 7 days */23/* choose candidates from sections which has age of more than 7 days */
19#define DEF_GC_THREAD_AGE_THRESHOLD (60 * 60 * 24 * 7)24#define DEF_GC_THREAD_AGE_THRESHOLD (60 * 60 * 24 * 7)
20#define DEF_GC_THREAD_CANDIDATE_RATIO 20 /* select 20% oldest sections as candidates */25#define DEF_GC_THREAD_CANDIDATE_RATIO 20 /* select 20% oldest sections as candidates */
21#define DEF_GC_THREAD_MAX_CANDIDATE_COUNT 10 /* select at most 10 sections as candidates */26#define DEF_GC_THREAD_MAX_CANDIDATE_COUNT 10 /* select at most 10 sections as candidates */
22#define DEF_GC_THREAD_AGE_WEIGHT 60 /* age weight */27#define DEF_GC_THREAD_AGE_WEIGHT 60 /* age weight */
28#define DEF_GC_THREAD_VALID_THRESH_RATIO 95 /* do not GC over 95% valid block ratio for one time GC */
23#define DEFAULT_ACCURACY_CLASS 10000 /* accuracy class */29#define DEFAULT_ACCURACY_CLASS 10000 /* accuracy class */
24 30
25#define LIMIT_INVALID_BLOCK 40 /* percentage over total user space */31#define LIMIT_INVALID_BLOCK 40 /* percentage over total user space */
26#define LIMIT_FREE_BLOCK 40 /* percentage over invalid + free space */32#define LIMIT_FREE_BLOCK 40 /* percentage over invalid + free space */
27 33
34#define LIMIT_NO_ZONED_GC 60 /* percentage over total user space of no gc for zoned devices */
35#define LIMIT_BOOST_ZONED_GC 25 /* percentage over total user space of boosted gc for zoned devices */
36#define DEF_MIGRATION_WINDOW_GRANULARITY_ZONED 3
37#define BOOST_GC_MULTIPLE 5
38
28#define DEF_GC_FAILED_PINNED_FILES 204839#define DEF_GC_FAILED_PINNED_FILES 2048
29#define MAX_GC_FAILED_PINNED_FILES USHRT_MAX40#define MAX_GC_FAILED_PINNED_FILES USHRT_MAX
30 41
@@ -51,6 +62,11 @@ struct f2fs_gc_kthread { · 20 unmodified lines
51 * caller of f2fs_balance_fs()62 * caller of f2fs_balance_fs()
52 * will wait on this wait queue.63 * will wait on this wait queue.
53 */64 */
65
66 /* for gc control for zoned devices */
67 unsigned int no_zoned_gc_percent;
68 unsigned int boost_zoned_gc_percent;
69 unsigned int valid_thresh_ratio;
54};70};
55 71
56struct gc_inode_list {72struct gc_inode_list {
@@ -152,6 +168,12 @@ static inline void decrease_sleep_time(struct f2fs_gc_kthread *gc_th, · 95 unmodified lines
152 *wait -= min_time;168 *wait -= min_time;
153}169}
154 170
171static inline bool has_enough_free_blocks(struct f2fs_sb_info *sbi,
172 unsigned int limit_perc)
173{
174 return free_sections(sbi) > ((sbi->total_sections * limit_perc) / 100);
175}
176
155static inline bool has_enough_invalid_blocks(struct f2fs_sb_info *sbi)177static inline bool has_enough_invalid_blocks(struct f2fs_sb_info *sbi)
156{178{
157 block_t user_block_count = sbi->user_block_count;179 block_t user_block_count = sbi->user_block_count;
@@ -167,3 +189,10 @@ static inline bool has_enough_invalid_blocks(struct f2fs_sb_info *sbi) · 9 unmodified lines
167 free_user_blocks(sbi) <189 free_user_blocks(sbi) <
168 limit_free_user_blocks(invalid_user_blocks));190 limit_free_user_blocks(invalid_user_blocks));
169}191}
192
193static inline bool need_to_boost_gc(struct f2fs_sb_info *sbi)
194{
195 if (f2fs_sb_has_blkzoned(sbi))
196 return !has_enough_free_blocks(sbi, LIMIT_BOOST_ZONED_GC);
197 return has_enough_invalid_blocks(sbi);
198}
fs/f2fs/inline.c+15−16
@@ -260,35 +260,34 @@ out: · 259 unmodified lines
260 return err;260 return err;
261}261}
262 262
263int f2fs_write_inline_data(struct inode *inode, struct page *page)263int f2fs_write_inline_data(struct inode *inode, struct folio *folio)
264{264{
265 struct dnode_of_data dn;265 struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
266 int err;266 struct page *ipage;
267 267
268 set_new_dnode(&dn, inode, NULL, NULL, 0);268 ipage = f2fs_get_node_page(sbi, inode->i_ino);
269 err = f2fs_get_dnode_of_data(&dn, 0, LOOKUP_NODE);269 if (IS_ERR(ipage))
270 if (err)270 return PTR_ERR(ipage);
271 return err;
272 271
273 if (!f2fs_has_inline_data(inode)) {272 if (!f2fs_has_inline_data(inode)) {
274 f2fs_put_dnode(&dn);273 f2fs_put_page(ipage, 1);
275 return -EAGAIN;274 return -EAGAIN;
276 }275 }
277 276
278 f2fs_bug_on(F2FS_I_SB(inode), page->index);277 f2fs_bug_on(F2FS_I_SB(inode), folio->index);
279 278
280 f2fs_wait_on_page_writeback(dn.inode_page, NODE, true, true);279 f2fs_wait_on_page_writeback(ipage, NODE, true, true);
281 memcpy_from_page(inline_data_addr(inode, dn.inode_page),280 memcpy_from_folio(inline_data_addr(inode, ipage),
282 page, 0, MAX_INLINE_DATA(inode));281 folio, 0, MAX_INLINE_DATA(inode));
283 set_page_dirty(dn.inode_page);282 set_page_dirty(ipage);
284 283
285 f2fs_clear_page_cache_dirty_tag(page);284 f2fs_clear_page_cache_dirty_tag(folio);
286 285
287 set_inode_flag(inode, FI_APPEND_WRITE);286 set_inode_flag(inode, FI_APPEND_WRITE);
288 set_inode_flag(inode, FI_DATA_EXIST);287 set_inode_flag(inode, FI_DATA_EXIST);
289 288
290 clear_page_private_inline(dn.inode_page);289 clear_page_private_inline(ipage);
291 f2fs_put_dnode(&dn);290 f2fs_put_page(ipage, 1);
292 return 0;291 return 0;
293}292}
294 293
fs/f2fs/inode.c+7−2
@@ -7,7 +7,6 @@ · 6 unmodified lines
7 */7 */
8#include <linux/fs.h>8#include <linux/fs.h>
9#include <linux/f2fs_fs.h>9#include <linux/f2fs_fs.h>
10#include <linux/buffer_head.h>
11#include <linux/writeback.h>10#include <linux/writeback.h>
12#include <linux/sched/mm.h>11#include <linux/sched/mm.h>
13#include <linux/lz4.h>12#include <linux/lz4.h>
@@ -35,6 +34,11 @@ void f2fs_mark_inode_dirty_sync(struct inode *inode, bool sync) · 21 unmodified lines
35 if (f2fs_inode_dirtied(inode, sync))34 if (f2fs_inode_dirtied(inode, sync))
36 return;35 return;
37 36
37 if (f2fs_is_atomic_file(inode)) {
38 set_inode_flag(inode, FI_ATOMIC_DIRTIED);
39 return;
40 }
41
38 mark_inode_dirty_sync(inode);42 mark_inode_dirty_sync(inode);
39}43}
40 44
@@ -175,7 +179,8 @@ bool f2fs_inode_chksum_verify(struct f2fs_sb_info *sbi, struct page *page) · 134 unmodified lines
175 179
176 if (provided != calculated)180 if (provided != calculated)
177 f2fs_warn(sbi, "checksum invalid, nid = %lu, ino_of_node = %x, %x vs. %x",181 f2fs_warn(sbi, "checksum invalid, nid = %lu, ino_of_node = %x, %x vs. %x",
178 page->index, ino_of_node(page), provided, calculated);182 page_folio(page)->index, ino_of_node(page),
183 provided, calculated);
179 184
180 return provided == calculated;185 return provided == calculated;
181}186}
fs/f2fs/namei.c+0−68
@@ -457,62 +457,6 @@ struct dentry *f2fs_get_parent(struct dentry *child) · 456 unmodified lines
457 return d_obtain_alias(f2fs_iget(child->d_sb, ino));457 return d_obtain_alias(f2fs_iget(child->d_sb, ino));
458}458}
459 459
460static int __recover_dot_dentries(struct inode *dir, nid_t pino)
461{
462 struct f2fs_sb_info *sbi = F2FS_I_SB(dir);
463 struct qstr dot = QSTR_INIT(".", 1);
464 struct f2fs_dir_entry *de;
465 struct page *page;
466 int err = 0;
467
468 if (f2fs_readonly(sbi->sb)) {
469 f2fs_info(sbi, "skip recovering inline_dots inode (ino:%lu, pino:%u) in readonly mountpoint",
470 dir->i_ino, pino);
471 return 0;
472 }
473
474 if (!S_ISDIR(dir->i_mode)) {
475 f2fs_err(sbi, "inconsistent inode status, skip recovering inline_dots inode (ino:%lu, i_mode:%u, pino:%u)",
476 dir->i_ino, dir->i_mode, pino);
477 set_sbi_flag(sbi, SBI_NEED_FSCK);
478 return -ENOTDIR;
479 }
480
481 err = f2fs_dquot_initialize(dir);
482 if (err)
483 return err;
484
485 f2fs_balance_fs(sbi, true);
486
487 f2fs_lock_op(sbi);
488
489 de = f2fs_find_entry(dir, &dot, &page);
490 if (de) {
491 f2fs_put_page(page, 0);
492 } else if (IS_ERR(page)) {
493 err = PTR_ERR(page);
494 goto out;
495 } else {
496 err = f2fs_do_add_link(dir, &dot, NULL, dir->i_ino, S_IFDIR);
497 if (err)
498 goto out;
499 }
500
501 de = f2fs_find_entry(dir, &dotdot_name, &page);
502 if (de)
503 f2fs_put_page(page, 0);
504 else if (IS_ERR(page))
505 err = PTR_ERR(page);
506 else
507 err = f2fs_do_add_link(dir, &dotdot_name, NULL, pino, S_IFDIR);
508out:
509 if (!err)
510 clear_inode_flag(dir, FI_INLINE_DOTS);
511
512 f2fs_unlock_op(sbi);
513 return err;
514}
515
516static struct dentry *f2fs_lookup(struct inode *dir, struct dentry *dentry,460static struct dentry *f2fs_lookup(struct inode *dir, struct dentry *dentry,
517 unsigned int flags)461 unsigned int flags)
518{462{
@@ -522,7 +466,6 @@ static struct dentry *f2fs_lookup(struct inode *dir, struct dentry *dentry, · 3 unmodified lines
522 struct dentry *new;466 struct dentry *new;
523 nid_t ino = -1;467 nid_t ino = -1;
524 int err = 0;468 int err = 0;
525 unsigned int root_ino = F2FS_ROOT_INO(F2FS_I_SB(dir));
526 struct f2fs_filename fname;469 struct f2fs_filename fname;
527 470
528 trace_f2fs_lookup_start(dir, dentry, flags);471 trace_f2fs_lookup_start(dir, dentry, flags);
@@ -558,17 +501,6 @@ static struct dentry *f2fs_lookup(struct inode *dir, struct dentry *dentry, · 29 unmodified lines
558 goto out;501 goto out;
559 }502 }
560 503
561 if ((dir->i_ino == root_ino) && f2fs_has_inline_dots(dir)) {
562 err = __recover_dot_dentries(dir, root_ino);
563 if (err)
564 goto out_iput;
565 }
566
567 if (f2fs_has_inline_dots(inode)) {
568 err = __recover_dot_dentries(inode, dir->i_ino);
569 if (err)
570 goto out_iput;
571 }
572 if (IS_ENCRYPTED(dir) &&504 if (IS_ENCRYPTED(dir) &&
573 (S_ISDIR(inode->i_mode) || S_ISLNK(inode->i_mode)) &&505 (S_ISDIR(inode->i_mode) || S_ISLNK(inode->i_mode)) &&
574 !fscrypt_has_permitted_context(dir, inode)) {506 !fscrypt_has_permitted_context(dir, inode)) {
fs/f2fs/node.c+24−22
@@ -20,7 +20,7 @@ · 19 unmodified lines
20#include "iostat.h"20#include "iostat.h"
21#include <trace/events/f2fs.h>21#include <trace/events/f2fs.h>
22 22
23#define on_f2fs_build_free_nids(nmi) mutex_is_locked(&(nm_i)->build_lock)23#define on_f2fs_build_free_nids(nm_i) mutex_is_locked(&(nm_i)->build_lock)
24 24
25static struct kmem_cache *nat_entry_slab;25static struct kmem_cache *nat_entry_slab;
26static struct kmem_cache *free_nid_slab;26static struct kmem_cache *free_nid_slab;
@@ -123,7 +123,7 @@ bool f2fs_available_free_memory(struct f2fs_sb_info *sbi, int type) · 96 unmodified lines
123static void clear_node_page_dirty(struct page *page)123static void clear_node_page_dirty(struct page *page)
124{124{
125 if (PageDirty(page)) {125 if (PageDirty(page)) {
126 f2fs_clear_page_cache_dirty_tag(page);126 f2fs_clear_page_cache_dirty_tag(page_folio(page));
127 clear_page_dirty_for_io(page);127 clear_page_dirty_for_io(page);
128 dec_page_count(F2FS_P_SB(page), F2FS_DIRTY_NODES);128 dec_page_count(F2FS_P_SB(page), F2FS_DIRTY_NODES);
129 }129 }
@@ -919,7 +919,7 @@ static int truncate_node(struct dnode_of_data *dn) · 789 unmodified lines
919 clear_node_page_dirty(dn->node_page);919 clear_node_page_dirty(dn->node_page);
920 set_sbi_flag(sbi, SBI_IS_DIRTY);920 set_sbi_flag(sbi, SBI_IS_DIRTY);
921 921
922 index = dn->node_page->index;922 index = page_folio(dn->node_page)->index;
923 f2fs_put_page(dn->node_page, 1);923 f2fs_put_page(dn->node_page, 1);
924 924
925 invalidate_mapping_pages(NODE_MAPPING(sbi),925 invalidate_mapping_pages(NODE_MAPPING(sbi),
@@ -1369,6 +1369,7 @@ fail: · 443 unmodified lines
1369 */1369 */
1370static int read_node_page(struct page *page, blk_opf_t op_flags)1370static int read_node_page(struct page *page, blk_opf_t op_flags)
1371{1371{
1372 struct folio *folio = page_folio(page);
1372 struct f2fs_sb_info *sbi = F2FS_P_SB(page);1373 struct f2fs_sb_info *sbi = F2FS_P_SB(page);
1373 struct node_info ni;1374 struct node_info ni;
1374 struct f2fs_io_info fio = {1375 struct f2fs_io_info fio = {
@@ -1381,21 +1382,21 @@ static int read_node_page(struct page *page, blk_opf_t op_flags) · 6 unmodified lines
1381 };1382 };
1382 int err;1383 int err;
1383 1384
1384 if (PageUptodate(page)) {1385 if (folio_test_uptodate(folio)) {
1385 if (!f2fs_inode_chksum_verify(sbi, page)) {1386 if (!f2fs_inode_chksum_verify(sbi, page)) {
1386 ClearPageUptodate(page);1387 folio_clear_uptodate(folio);
1387 return -EFSBADCRC;1388 return -EFSBADCRC;
1388 }1389 }
1389 return LOCKED_PAGE;1390 return LOCKED_PAGE;
1390 }1391 }
1391 1392
1392 err = f2fs_get_node_info(sbi, page->index, &ni, false);1393 err = f2fs_get_node_info(sbi, folio->index, &ni, false);
1393 if (err)1394 if (err)
1394 return err;1395 return err;
1395 1396
1396 /* NEW_ADDR can be seen, after cp_error drops some dirty node pages */1397 /* NEW_ADDR can be seen, after cp_error drops some dirty node pages */
1397 if (unlikely(ni.blk_addr == NULL_ADDR || ni.blk_addr == NEW_ADDR)) {1398 if (unlikely(ni.blk_addr == NULL_ADDR || ni.blk_addr == NEW_ADDR)) {
1398 ClearPageUptodate(page);1399 folio_clear_uptodate(folio);
1399 return -ENOENT;1400 return -ENOENT;
1400 }1401 }
1401 1402
@@ -1492,7 +1493,7 @@ out_err: · 90 unmodified lines
1492out_put_err:1493out_put_err:
1493 /* ENOENT comes from read_node_page which is not an error. */1494 /* ENOENT comes from read_node_page which is not an error. */
1494 if (err != -ENOENT)1495 if (err != -ENOENT)
1495 f2fs_handle_page_eio(sbi, page->index, NODE);1496 f2fs_handle_page_eio(sbi, page_folio(page), NODE);
1496 f2fs_put_page(page, 1);1497 f2fs_put_page(page, 1);
1497 return ERR_PTR(err);1498 return ERR_PTR(err);
1498}1499}
@@ -1535,7 +1536,7 @@ static void flush_inline_data(struct f2fs_sb_info *sbi, nid_t ino) · 36 unmodified lines
1535 if (!clear_page_dirty_for_io(page))1536 if (!clear_page_dirty_for_io(page))
1536 goto page_out;1537 goto page_out;
1537 1538
1538 ret = f2fs_write_inline_data(inode, page);1539 ret = f2fs_write_inline_data(inode, page_folio(page));
1539 inode_dec_dirty_pages(inode);1540 inode_dec_dirty_pages(inode);
1540 f2fs_remove_dirty_inode(inode);1541 f2fs_remove_dirty_inode(inode);
1541 if (ret)1542 if (ret)
@@ -1608,6 +1609,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, · 66 unmodified lines
1608 enum iostat_type io_type, unsigned int *seq_id)1609 enum iostat_type io_type, unsigned int *seq_id)
1609{1610{
1610 struct f2fs_sb_info *sbi = F2FS_P_SB(page);1611 struct f2fs_sb_info *sbi = F2FS_P_SB(page);
1612 struct folio *folio = page_folio(page);
1611 nid_t nid;1613 nid_t nid;
1612 struct node_info ni;1614 struct node_info ni;
1613 struct f2fs_io_info fio = {1615 struct f2fs_io_info fio = {
@@ -1624,15 +1626,15 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, · 10 unmodified lines
1624 };1626 };
1625 unsigned int seq;1627 unsigned int seq;
1626 1628
1627 trace_f2fs_writepage(page_folio(page), NODE);1629 trace_f2fs_writepage(folio, NODE);
1628 1630
1629 if (unlikely(f2fs_cp_error(sbi))) {1631 if (unlikely(f2fs_cp_error(sbi))) {
1630 /* keep node pages in remount-ro mode */1632 /* keep node pages in remount-ro mode */
1631 if (F2FS_OPTION(sbi).errors == MOUNT_ERRORS_READONLY)1633 if (F2FS_OPTION(sbi).errors == MOUNT_ERRORS_READONLY)
1632 goto redirty_out;1634 goto redirty_out;
1633 ClearPageUptodate(page);1635 folio_clear_uptodate(folio);
1634 dec_page_count(sbi, F2FS_DIRTY_NODES);1636 dec_page_count(sbi, F2FS_DIRTY_NODES);
1635 unlock_page(page);1637 folio_unlock(folio);
1636 return 0;1638 return 0;
1637 }1639 }
1638 1640
@@ -1646,7 +1648,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, · 7 unmodified lines
1646 1648
1647 /* get old block addr of this node page */1649 /* get old block addr of this node page */
1648 nid = nid_of_node(page);1650 nid = nid_of_node(page);
1649 f2fs_bug_on(sbi, page->index != nid);1651 f2fs_bug_on(sbi, folio->index != nid);
1650 1652
1651 if (f2fs_get_node_info(sbi, nid, &ni, !do_balance))1653 if (f2fs_get_node_info(sbi, nid, &ni, !do_balance))
1652 goto redirty_out;1654 goto redirty_out;
@@ -1660,10 +1662,10 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, · 7 unmodified lines
1660 1662
1661 /* This page is already truncated */1663 /* This page is already truncated */
1662 if (unlikely(ni.blk_addr == NULL_ADDR)) {1664 if (unlikely(ni.blk_addr == NULL_ADDR)) {
1663 ClearPageUptodate(page);1665 folio_clear_uptodate(folio);
1664 dec_page_count(sbi, F2FS_DIRTY_NODES);1666 dec_page_count(sbi, F2FS_DIRTY_NODES);
1665 f2fs_up_read(&sbi->node_write);1667 f2fs_up_read(&sbi->node_write);
1666 unlock_page(page);1668 folio_unlock(folio);
1667 return 0;1669 return 0;
1668 }1670 }
1669 1671
@@ -1674,7 +1676,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, · 4 unmodified lines
1674 goto redirty_out;1676 goto redirty_out;
1675 }1677 }
1676 1678
1677 if (atomic && !test_opt(sbi, NOBARRIER) && !f2fs_sb_has_blkzoned(sbi))1679 if (atomic && !test_opt(sbi, NOBARRIER))
1678 fio.op_flags |= REQ_PREFLUSH | REQ_FUA;1680 fio.op_flags |= REQ_PREFLUSH | REQ_FUA;
1679 1681
1680 /* should add to global list before clearing PAGECACHE status */1682 /* should add to global list before clearing PAGECACHE status */
@@ -1684,7 +1686,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, · 3 unmodified lines
1684 *seq_id = seq;1686 *seq_id = seq;
1685 }1687 }
1686 1688
1687 set_page_writeback(page);1689 folio_start_writeback(folio);
1688 1690
1689 fio.old_blkaddr = ni.blk_addr;1691 fio.old_blkaddr = ni.blk_addr;
1690 f2fs_do_write_node_page(nid, &fio);1692 f2fs_do_write_node_page(nid, &fio);
@@ -1697,7 +1699,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, · 6 unmodified lines
1697 submitted = NULL;1699 submitted = NULL;
1698 }1700 }
1699 1701
1700 unlock_page(page);1702 folio_unlock(folio);
1701 1703
1702 if (unlikely(f2fs_cp_error(sbi))) {1704 if (unlikely(f2fs_cp_error(sbi))) {
1703 f2fs_submit_merged_write(sbi, NODE);1705 f2fs_submit_merged_write(sbi, NODE);
@@ -1711,7 +1713,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, · 7 unmodified lines
1711 return 0;1713 return 0;
1712 1714
1713redirty_out:1715redirty_out:
1714 redirty_page_for_writepage(wbc, page);1716 folio_redirty_for_writepage(wbc, folio);
1715 return AOP_WRITEPAGE_ACTIVATE;1717 return AOP_WRITEPAGE_ACTIVATE;
1716}1718}
1717 1719
@@ -1867,7 +1869,7 @@ continue_unlock: · 149 unmodified lines
1867 }1869 }
1868 if (!ret && atomic && !marked) {1870 if (!ret && atomic && !marked) {
1869 f2fs_debug(sbi, "Retry to write fsync mark: ino=%u, idx=%lx",1871 f2fs_debug(sbi, "Retry to write fsync mark: ino=%u, idx=%lx",
1870 ino, last_page->index);1872 ino, page_folio(last_page)->index);
1871 lock_page(last_page);1873 lock_page(last_page);
1872 f2fs_wait_on_page_writeback(last_page, NODE, true, true);1874 f2fs_wait_on_page_writeback(last_page, NODE, true, true);
1873 set_page_dirty(last_page);1875 set_page_dirty(last_page);
@@ -3166,7 +3168,7 @@ static int __get_nat_bitmaps(struct f2fs_sb_info *sbi) · 1292 unmodified lines
3166 3168
3167 nm_i->nat_bits_blocks = F2FS_BLK_ALIGN((nat_bits_bytes << 1) + 8);3169 nm_i->nat_bits_blocks = F2FS_BLK_ALIGN((nat_bits_bytes << 1) + 8);
3168 nm_i->nat_bits = f2fs_kvzalloc(sbi,3170 nm_i->nat_bits = f2fs_kvzalloc(sbi,
3169 nm_i->nat_bits_blocks << F2FS_BLKSIZE_BITS, GFP_KERNEL);3171 F2FS_BLK_TO_BYTES(nm_i->nat_bits_blocks), GFP_KERNEL);
3170 if (!nm_i->nat_bits)3172 if (!nm_i->nat_bits)
3171 return -ENOMEM;3173 return -ENOMEM;
3172 3174
@@ -3185,7 +3187,7 @@ static int __get_nat_bitmaps(struct f2fs_sb_info *sbi) · 12 unmodified lines
3185 if (IS_ERR(page))3187 if (IS_ERR(page))
3186 return PTR_ERR(page);3188 return PTR_ERR(page);
3187 3189
3188 memcpy(nm_i->nat_bits + (i << F2FS_BLKSIZE_BITS),3190 memcpy(nm_i->nat_bits + F2FS_BLK_TO_BYTES(i),
3189 page_address(page), F2FS_BLKSIZE);3191 page_address(page), F2FS_BLKSIZE);
3190 f2fs_put_page(page, 1);3192 f2fs_put_page(page, 1);
3191 }3193 }
fs/f2fs/segment.c+56−16
@@ -199,6 +199,10 @@ void f2fs_abort_atomic_write(struct inode *inode, bool clean) · 198 unmodified lines
199 clear_inode_flag(inode, FI_ATOMIC_COMMITTED);199 clear_inode_flag(inode, FI_ATOMIC_COMMITTED);
200 clear_inode_flag(inode, FI_ATOMIC_REPLACE);200 clear_inode_flag(inode, FI_ATOMIC_REPLACE);
201 clear_inode_flag(inode, FI_ATOMIC_FILE);201 clear_inode_flag(inode, FI_ATOMIC_FILE);
202 if (is_inode_flag_set(inode, FI_ATOMIC_DIRTIED)) {
203 clear_inode_flag(inode, FI_ATOMIC_DIRTIED);
204 f2fs_mark_inode_dirty_sync(inode, true);
205 }
202 stat_dec_atomic_inode(inode);206 stat_dec_atomic_inode(inode);
203 207
204 F2FS_I(inode)->atomic_write_task = NULL;208 F2FS_I(inode)->atomic_write_task = NULL;
@@ -366,6 +370,10 @@ out: · 161 unmodified lines
366 } else {370 } else {
367 sbi->committed_atomic_block += fi->atomic_write_cnt;371 sbi->committed_atomic_block += fi->atomic_write_cnt;
368 set_inode_flag(inode, FI_ATOMIC_COMMITTED);372 set_inode_flag(inode, FI_ATOMIC_COMMITTED);
373 if (is_inode_flag_set(inode, FI_ATOMIC_DIRTIED)) {
374 clear_inode_flag(inode, FI_ATOMIC_DIRTIED);
375 f2fs_mark_inode_dirty_sync(inode, true);
376 }
369 }377 }
370 378
371 __complete_revoke_list(inode, &revoke_list, ret ? true : false);379 __complete_revoke_list(inode, &revoke_list, ret ? true : false);
@@ -1282,6 +1290,13 @@ static int __submit_discard_cmd(struct f2fs_sb_info *sbi, · 910 unmodified lines
1282 wait_list, issued);1290 wait_list, issued);
1283 return 0;1291 return 0;
1284 }1292 }
1293
1294 /*
1295 * Issue discard for conventional zones only if the device
1296 * supports discard.
1297 */
1298 if (!bdev_max_discard_sectors(bdev))
1299 return -EOPNOTSUPP;
1285 }1300 }
1286#endif1301#endif
1287 1302
@@ -2686,22 +2701,47 @@ static int get_new_segment(struct f2fs_sb_info *sbi, · 1398 unmodified lines
2686 goto got_it;2701 goto got_it;
2687 }2702 }
2688 2703
2704#ifdef CONFIG_BLK_DEV_ZONED
2689 /*2705 /*
2690 * If we format f2fs on zoned storage, let's try to get pinned sections2706 * If we format f2fs on zoned storage, let's try to get pinned sections
2691 * from beginning of the storage, which should be a conventional one.2707 * from beginning of the storage, which should be a conventional one.
2692 */2708 */
2693 if (f2fs_sb_has_blkzoned(sbi)) {2709 if (f2fs_sb_has_blkzoned(sbi)) {
2694 segno = pinning ? 0 : max(first_zoned_segno(sbi), *newseg);2710 /* Prioritize writing to conventional zones */
2711 if (sbi->blkzone_alloc_policy == BLKZONE_ALLOC_PRIOR_CONV || pinning)
2712 segno = 0;
2713 else
2714 segno = max(first_zoned_segno(sbi), *newseg);
2695 hint = GET_SEC_FROM_SEG(sbi, segno);2715 hint = GET_SEC_FROM_SEG(sbi, segno);
2696 }2716 }
2717#endif
2697 2718
2698find_other_zone:2719find_other_zone:
2699 secno = find_next_zero_bit(free_i->free_secmap, MAIN_SECS(sbi), hint);2720 secno = find_next_zero_bit(free_i->free_secmap, MAIN_SECS(sbi), hint);
2721
2722#ifdef CONFIG_BLK_DEV_ZONED
2723 if (secno >= MAIN_SECS(sbi) && f2fs_sb_has_blkzoned(sbi)) {
2724 /* Write only to sequential zones */
2725 if (sbi->blkzone_alloc_policy == BLKZONE_ALLOC_ONLY_SEQ) {
2726 hint = GET_SEC_FROM_SEG(sbi, first_zoned_segno(sbi));
2727 secno = find_next_zero_bit(free_i->free_secmap, MAIN_SECS(sbi), hint);
2728 } else
2729 secno = find_first_zero_bit(free_i->free_secmap,
2730 MAIN_SECS(sbi));
2731 if (secno >= MAIN_SECS(sbi)) {
2732 ret = -ENOSPC;
2733 f2fs_bug_on(sbi, 1);
2734 goto out_unlock;
2735 }
2736 }
2737#endif
2738
2700 if (secno >= MAIN_SECS(sbi)) {2739 if (secno >= MAIN_SECS(sbi)) {
2701 secno = find_first_zero_bit(free_i->free_secmap,2740 secno = find_first_zero_bit(free_i->free_secmap,
2702 MAIN_SECS(sbi));2741 MAIN_SECS(sbi));
2703 if (secno >= MAIN_SECS(sbi)) {2742 if (secno >= MAIN_SECS(sbi)) {
2704 ret = -ENOSPC;2743 ret = -ENOSPC;
2744 f2fs_bug_on(sbi, 1);
2705 goto out_unlock;2745 goto out_unlock;
2706 }2746 }
2707 }2747 }
@@ -2743,10 +2783,8 @@ got_it: · 35 unmodified lines
2743out_unlock:2783out_unlock:
2744 spin_unlock(&free_i->segmap_lock);2784 spin_unlock(&free_i->segmap_lock);
2745 2785
2746 if (ret == -ENOSPC) {2786 if (ret == -ENOSPC)
2747 f2fs_stop_checkpoint(sbi, false, STOP_CP_REASON_NO_SEGMENT);2787 f2fs_stop_checkpoint(sbi, false, STOP_CP_REASON_NO_SEGMENT);
2748 f2fs_bug_on(sbi, 1);
2749 }
2750 return ret;2788 return ret;
2751}2789}
2752 2790
@@ -3052,7 +3090,8 @@ static int get_ssr_segment(struct f2fs_sb_info *sbi, int type, · 299 unmodified lines
3052 sanity_check_seg_type(sbi, seg_type);3090 sanity_check_seg_type(sbi, seg_type);
3053 3091
3054 /* f2fs_need_SSR() already forces to do this */3092 /* f2fs_need_SSR() already forces to do this */
3055 if (!f2fs_get_victim(sbi, &segno, BG_GC, seg_type, alloc_mode, age)) {3093 if (!f2fs_get_victim(sbi, &segno, BG_GC, seg_type,
3094 alloc_mode, age, false)) {
3056 curseg->next_segno = segno;3095 curseg->next_segno = segno;
3057 return 1;3096 return 1;
3058 }3097 }
@@ -3079,7 +3118,8 @@ static int get_ssr_segment(struct f2fs_sb_info *sbi, int type, · 20 unmodified lines
3079 for (; cnt-- > 0; reversed ? i-- : i++) {3118 for (; cnt-- > 0; reversed ? i-- : i++) {
3080 if (i == seg_type)3119 if (i == seg_type)
3081 continue;3120 continue;
3082 if (!f2fs_get_victim(sbi, &segno, BG_GC, i, alloc_mode, age)) {3121 if (!f2fs_get_victim(sbi, &segno, BG_GC, i,
3122 alloc_mode, age, false)) {
3083 curseg->next_segno = segno;3123 curseg->next_segno = segno;
3084 return 1;3124 return 1;
3085 }3125 }
@@ -3522,7 +3562,8 @@ static int __get_segment_type_6(struct f2fs_io_info *fio) · 436 unmodified lines
3522 if (file_is_cold(inode) || f2fs_need_compress_data(inode))3562 if (file_is_cold(inode) || f2fs_need_compress_data(inode))
3523 return CURSEG_COLD_DATA;3563 return CURSEG_COLD_DATA;
3524 3564
3525 type = __get_age_segment_type(inode, fio->page->index);3565 type = __get_age_segment_type(inode,
3566 page_folio(fio->page)->index);
3526 if (type != NO_CHECK_TYPE)3567 if (type != NO_CHECK_TYPE)
3527 return type;3568 return type;
3528 3569
@@ -3781,7 +3822,7 @@ out: · 252 unmodified lines
3781 f2fs_up_read(&fio->sbi->io_order_lock);3822 f2fs_up_read(&fio->sbi->io_order_lock);
3782}3823}
3783 3824
3784void f2fs_do_write_meta_page(struct f2fs_sb_info *sbi, struct page *page,3825void f2fs_do_write_meta_page(struct f2fs_sb_info *sbi, struct folio *folio,
3785 enum iostat_type io_type)3826 enum iostat_type io_type)
3786{3827{
3787 struct f2fs_io_info fio = {3828 struct f2fs_io_info fio = {
@@ -3790,20 +3831,20 @@ void f2fs_do_write_meta_page(struct f2fs_sb_info *sbi, struct page *page, · 2 unmodified lines
3790 .temp = HOT,3831 .temp = HOT,
3791 .op = REQ_OP_WRITE,3832 .op = REQ_OP_WRITE,
3792 .op_flags = REQ_SYNC | REQ_META | REQ_PRIO,3833 .op_flags = REQ_SYNC | REQ_META | REQ_PRIO,
3793 .old_blkaddr = page->index,3834 .old_blkaddr = folio->index,
3794 .new_blkaddr = page->index,3835 .new_blkaddr = folio->index,
3795 .page = page,3836 .page = folio_page(folio, 0),
3796 .encrypted_page = NULL,3837 .encrypted_page = NULL,
3797 .in_list = 0,3838 .in_list = 0,
3798 };3839 };
3799 3840
3800 if (unlikely(page->index >= MAIN_BLKADDR(sbi)))3841 if (unlikely(folio->index >= MAIN_BLKADDR(sbi)))
3801 fio.op_flags &= ~REQ_META;3842 fio.op_flags &= ~REQ_META;
3802 3843
3803 set_page_writeback(page);3844 folio_start_writeback(folio);
3804 f2fs_submit_page_write(&fio);3845 f2fs_submit_page_write(&fio);
3805 3846
3806 stat_inc_meta_count(sbi, page->index);3847 stat_inc_meta_count(sbi, folio->index);
3807 f2fs_update_iostat(sbi, NULL, io_type, F2FS_BLKSIZE);3848 f2fs_update_iostat(sbi, NULL, io_type, F2FS_BLKSIZE);
3808}3849}
3809 3850
@@ -5381,8 +5422,7 @@ unsigned int f2fs_usable_blks_in_seg(struct f2fs_sb_info *sbi, · 1571 unmodified lines
5381 return BLKS_PER_SEG(sbi);5422 return BLKS_PER_SEG(sbi);
5382}5423}
5383 5424
5384unsigned int f2fs_usable_segs_in_sec(struct f2fs_sb_info *sbi,5425unsigned int f2fs_usable_segs_in_sec(struct f2fs_sb_info *sbi)
5385 unsigned int segno)
5386{5426{
5387 if (f2fs_sb_has_blkzoned(sbi))5427 if (f2fs_sb_has_blkzoned(sbi))
5388 return CAP_SEGS_PER_SEC(sbi);5428 return CAP_SEGS_PER_SEC(sbi);
fs/f2fs/segment.h+3−2
@@ -188,6 +188,7 @@ struct victim_sel_policy { · 187 unmodified lines
188 unsigned int min_segno; /* segment # having min. cost */188 unsigned int min_segno; /* segment # having min. cost */
189 unsigned long long age; /* mtime of GCed section*/189 unsigned long long age; /* mtime of GCed section*/
190 unsigned long long age_threshold;/* age threshold */190 unsigned long long age_threshold;/* age threshold */
191 bool one_time_gc; /* one time GC */
191};192};
192 193
193struct seg_entry {194struct seg_entry {
@@ -430,7 +431,7 @@ static inline void __set_free(struct f2fs_sb_info *sbi, unsigned int segno) · 236 unmodified lines
430 unsigned int secno = GET_SEC_FROM_SEG(sbi, segno);431 unsigned int secno = GET_SEC_FROM_SEG(sbi, segno);
431 unsigned int start_segno = GET_SEG_FROM_SEC(sbi, secno);432 unsigned int start_segno = GET_SEG_FROM_SEC(sbi, secno);
432 unsigned int next;433 unsigned int next;
433 unsigned int usable_segs = f2fs_usable_segs_in_sec(sbi, segno);434 unsigned int usable_segs = f2fs_usable_segs_in_sec(sbi);
434 435
435 spin_lock(&free_i->segmap_lock);436 spin_lock(&free_i->segmap_lock);
436 clear_bit(segno, free_i->free_segmap);437 clear_bit(segno, free_i->free_segmap);
@@ -464,7 +465,7 @@ static inline void __set_test_and_free(struct f2fs_sb_info *sbi, · 27 unmodified lines
464 unsigned int secno = GET_SEC_FROM_SEG(sbi, segno);465 unsigned int secno = GET_SEC_FROM_SEG(sbi, segno);
465 unsigned int start_segno = GET_SEG_FROM_SEC(sbi, secno);466 unsigned int start_segno = GET_SEG_FROM_SEC(sbi, secno);
466 unsigned int next;467 unsigned int next;
467 unsigned int usable_segs = f2fs_usable_segs_in_sec(sbi, segno);468 unsigned int usable_segs = f2fs_usable_segs_in_sec(sbi);
468 469
469 spin_lock(&free_i->segmap_lock);470 spin_lock(&free_i->segmap_lock);
470 if (test_and_clear_bit(segno, free_i->free_segmap)) {471 if (test_and_clear_bit(segno, free_i->free_segmap)) {
fs/f2fs/super.c+74−45
@@ -11,7 +11,6 @@ · 10 unmodified lines
11#include <linux/fs_context.h>11#include <linux/fs_context.h>
12#include <linux/sched/mm.h>12#include <linux/sched/mm.h>
13#include <linux/statfs.h>13#include <linux/statfs.h>
14#include <linux/buffer_head.h>
15#include <linux/kthread.h>14#include <linux/kthread.h>
16#include <linux/parser.h>15#include <linux/parser.h>
17#include <linux/mount.h>16#include <linux/mount.h>
@@ -707,6 +706,11 @@ static int parse_options(struct super_block *sb, char *options, bool is_remount) · 689 unmodified lines
707 if (!strcmp(name, "on")) {706 if (!strcmp(name, "on")) {
708 F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_ON;707 F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_ON;
709 } else if (!strcmp(name, "off")) {708 } else if (!strcmp(name, "off")) {
709 if (f2fs_sb_has_blkzoned(sbi)) {
710 f2fs_warn(sbi, "zoned devices need bggc");
711 kfree(name);
712 return -EINVAL;
713 }
710 F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_OFF;714 F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_OFF;
711 } else if (!strcmp(name, "sync")) {715 } else if (!strcmp(name, "sync")) {
712 F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_SYNC;716 F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_SYNC;
@@ -2561,7 +2565,7 @@ restore_opts: · 1848 unmodified lines
2561 2565
2562static void f2fs_shutdown(struct super_block *sb)2566static void f2fs_shutdown(struct super_block *sb)
2563{2567{
2564 f2fs_do_shutdown(F2FS_SB(sb), F2FS_GOING_DOWN_NOSYNC, false);2568 f2fs_do_shutdown(F2FS_SB(sb), F2FS_GOING_DOWN_NOSYNC, false, false);
2565}2569}
2566 2570
2567#ifdef CONFIG_QUOTA2571#ifdef CONFIG_QUOTA
@@ -3318,29 +3322,47 @@ loff_t max_file_blocks(struct inode *inode) · 750 unmodified lines
3318 * fit within U32_MAX + 1 data units.3322 * fit within U32_MAX + 1 data units.
3319 */3323 */
3320 3324
3321 result = min(result, (((loff_t)U32_MAX + 1) * 4096) >> F2FS_BLKSIZE_BITS);3325 result = min(result, F2FS_BYTES_TO_BLK(((loff_t)U32_MAX + 1) * 4096));
3322 3326
3323 return result;3327 return result;
3324}3328}
3325 3329
3326static int __f2fs_commit_super(struct buffer_head *bh,3330static int __f2fs_commit_super(struct f2fs_sb_info *sbi, struct folio *folio,
3327 struct f2fs_super_block *super)3331 pgoff_t index, bool update)
3328{3332{
3329 lock_buffer(bh);3333 struct bio *bio;
3330 if (super)
3331 memcpy(bh->b_data + F2FS_SUPER_OFFSET, super, sizeof(*super));
3332 set_buffer_dirty(bh);
3333 unlock_buffer(bh);
3334
3335 /* it's rare case, we can do fua all the time */3334 /* it's rare case, we can do fua all the time */
3336 return __sync_dirty_buffer(bh, REQ_SYNC | REQ_PREFLUSH | REQ_FUA);3335 blk_opf_t opf = REQ_OP_WRITE | REQ_SYNC | REQ_PREFLUSH | REQ_FUA;
3336 int ret;
3337
3338 folio_lock(folio);
3339 folio_wait_writeback(folio);
3340 if (update)
3341 memcpy(F2FS_SUPER_BLOCK(folio, index), F2FS_RAW_SUPER(sbi),
3342 sizeof(struct f2fs_super_block));
3343 folio_mark_dirty(folio);
3344 folio_clear_dirty_for_io(folio);
3345 folio_start_writeback(folio);
3346 folio_unlock(folio);
3347
3348 bio = bio_alloc(sbi->sb->s_bdev, 1, opf, GFP_NOFS);
3349
3350 /* it doesn't need to set crypto context for superblock update */
3351 bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(folio_index(folio));
3352
3353 if (!bio_add_folio(bio, folio, folio_size(folio), 0))
3354 f2fs_bug_on(sbi, 1);
3355
3356 ret = submit_bio_wait(bio);
3357 folio_end_writeback(folio);
3358
3359 return ret;
3337}3360}
3338 3361
3339static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi,3362static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi,
3340 struct buffer_head *bh)3363 struct folio *folio, pgoff_t index)
3341{3364{
3342 struct f2fs_super_block *raw_super = (struct f2fs_super_block *)3365 struct f2fs_super_block *raw_super = F2FS_SUPER_BLOCK(folio, index);
3343 (bh->b_data + F2FS_SUPER_OFFSET);
3344 struct super_block *sb = sbi->sb;3366 struct super_block *sb = sbi->sb;
3345 u32 segment0_blkaddr = le32_to_cpu(raw_super->segment0_blkaddr);3367 u32 segment0_blkaddr = le32_to_cpu(raw_super->segment0_blkaddr);
3346 u32 cp_blkaddr = le32_to_cpu(raw_super->cp_blkaddr);3368 u32 cp_blkaddr = le32_to_cpu(raw_super->cp_blkaddr);
@@ -3356,9 +3378,9 @@ static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi, · 9 unmodified lines
3356 u32 segment_count = le32_to_cpu(raw_super->segment_count);3378 u32 segment_count = le32_to_cpu(raw_super->segment_count);
3357 u32 log_blocks_per_seg = le32_to_cpu(raw_super->log_blocks_per_seg);3379 u32 log_blocks_per_seg = le32_to_cpu(raw_super->log_blocks_per_seg);
3358 u64 main_end_blkaddr = main_blkaddr +3380 u64 main_end_blkaddr = main_blkaddr +
3359 (segment_count_main << log_blocks_per_seg);3381 ((u64)segment_count_main << log_blocks_per_seg);
3360 u64 seg_end_blkaddr = segment0_blkaddr +3382 u64 seg_end_blkaddr = segment0_blkaddr +
3361 (segment_count << log_blocks_per_seg);3383 ((u64)segment_count << log_blocks_per_seg);
3362 3384
3363 if (segment0_blkaddr != cp_blkaddr) {3385 if (segment0_blkaddr != cp_blkaddr) {
3364 f2fs_info(sbi, "Mismatch start address, segment0(%u) cp_blkaddr(%u)",3386 f2fs_info(sbi, "Mismatch start address, segment0(%u) cp_blkaddr(%u)",
@@ -3415,7 +3437,7 @@ static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi, · 50 unmodified lines
3415 set_sbi_flag(sbi, SBI_NEED_SB_WRITE);3437 set_sbi_flag(sbi, SBI_NEED_SB_WRITE);
3416 res = "internally";3438 res = "internally";
3417 } else {3439 } else {
3418 err = __f2fs_commit_super(bh, NULL);3440 err = __f2fs_commit_super(sbi, folio, index, false);
3419 res = err ? "failed" : "done";3441 res = err ? "failed" : "done";
3420 }3442 }
3421 f2fs_info(sbi, "Fix alignment : %s, start(%u) end(%llu) block(%u)",3443 f2fs_info(sbi, "Fix alignment : %s, start(%u) end(%llu) block(%u)",
@@ -3428,12 +3450,11 @@ static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi, · 6 unmodified lines
3428}3450}
3429 3451
3430static int sanity_check_raw_super(struct f2fs_sb_info *sbi,3452static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
3431 struct buffer_head *bh)3453 struct folio *folio, pgoff_t index)
3432{3454{
3433 block_t segment_count, segs_per_sec, secs_per_zone, segment_count_main;3455 block_t segment_count, segs_per_sec, secs_per_zone, segment_count_main;
3434 block_t total_sections, blocks_per_seg;3456 block_t total_sections, blocks_per_seg;
3435 struct f2fs_super_block *raw_super = (struct f2fs_super_block *)3457 struct f2fs_super_block *raw_super = F2FS_SUPER_BLOCK(folio, index);
3436 (bh->b_data + F2FS_SUPER_OFFSET);
3437 size_t crc_offset = 0;3458 size_t crc_offset = 0;
3438 __u32 crc = 0;3459 __u32 crc = 0;
3439 3460
@@ -3591,7 +3612,7 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi, · 151 unmodified lines
3591 }3612 }
3592 3613
3593 /* check CP/SIT/NAT/SSA/MAIN_AREA area boundary */3614 /* check CP/SIT/NAT/SSA/MAIN_AREA area boundary */
3594 if (sanity_check_area_boundary(sbi, bh))3615 if (sanity_check_area_boundary(sbi, folio, index))
3595 return -EFSCORRUPTED;3616 return -EFSCORRUPTED;
3596 3617
3597 return 0;3618 return 0;
@@ -3786,6 +3807,8 @@ static void init_sb_info(struct f2fs_sb_info *sbi) · 188 unmodified lines
3786 sbi->next_victim_seg[FG_GC] = NULL_SEGNO;3807 sbi->next_victim_seg[FG_GC] = NULL_SEGNO;
3787 sbi->max_victim_search = DEF_MAX_VICTIM_SEARCH;3808 sbi->max_victim_search = DEF_MAX_VICTIM_SEARCH;
3788 sbi->migration_granularity = SEGS_PER_SEC(sbi);3809 sbi->migration_granularity = SEGS_PER_SEC(sbi);
3810 sbi->migration_window_granularity = f2fs_sb_has_blkzoned(sbi) ?
3811 DEF_MIGRATION_WINDOW_GRANULARITY_ZONED : SEGS_PER_SEC(sbi);
3789 sbi->seq_file_ra_mul = MIN_RA_MUL;3812 sbi->seq_file_ra_mul = MIN_RA_MUL;
3790 sbi->max_fragment_chunk = DEF_FRAGMENT_SIZE;3813 sbi->max_fragment_chunk = DEF_FRAGMENT_SIZE;
3791 sbi->max_fragment_hole = DEF_FRAGMENT_SIZE;3814 sbi->max_fragment_hole = DEF_FRAGMENT_SIZE;
@@ -3938,7 +3961,7 @@ static int read_raw_super_block(struct f2fs_sb_info *sbi, · 146 unmodified lines
3938{3961{
3939 struct super_block *sb = sbi->sb;3962 struct super_block *sb = sbi->sb;
3940 int block;3963 int block;
3941 struct buffer_head *bh;3964 struct folio *folio;
3942 struct f2fs_super_block *super;3965 struct f2fs_super_block *super;
3943 int err = 0;3966 int err = 0;
3944 3967
@@ -3947,32 +3970,32 @@ static int read_raw_super_block(struct f2fs_sb_info *sbi, · 2 unmodified lines
3947 return -ENOMEM;3970 return -ENOMEM;
3948 3971
3949 for (block = 0; block < 2; block++) {3972 for (block = 0; block < 2; block++) {
3950 bh = sb_bread(sb, block);3973 folio = read_mapping_folio(sb->s_bdev->bd_mapping, block, NULL);
3951 if (!bh) {3974 if (IS_ERR(folio)) {
3952 f2fs_err(sbi, "Unable to read %dth superblock",3975 f2fs_err(sbi, "Unable to read %dth superblock",
3953 block + 1);3976 block + 1);
3954 err = -EIO;3977 err = PTR_ERR(folio);
3955 *recovery = 1;3978 *recovery = 1;
3956 continue;3979 continue;
3957 }3980 }
3958 3981
3959 /* sanity checking of raw super */3982 /* sanity checking of raw super */
3960 err = sanity_check_raw_super(sbi, bh);3983 err = sanity_check_raw_super(sbi, folio, block);
3961 if (err) {3984 if (err) {
3962 f2fs_err(sbi, "Can't find valid F2FS filesystem in %dth superblock",3985 f2fs_err(sbi, "Can't find valid F2FS filesystem in %dth superblock",
3963 block + 1);3986 block + 1);
3964 brelse(bh);3987 folio_put(folio);
3965 *recovery = 1;3988 *recovery = 1;
3966 continue;3989 continue;
3967 }3990 }
3968 3991
3969 if (!*raw_super) {3992 if (!*raw_super) {
3970 memcpy(super, bh->b_data + F2FS_SUPER_OFFSET,3993 memcpy(super, F2FS_SUPER_BLOCK(folio, block),
3971 sizeof(*super));3994 sizeof(*super));
3972 *valid_super_block = block;3995 *valid_super_block = block;
3973 *raw_super = super;3996 *raw_super = super;
3974 }3997 }
3975 brelse(bh);3998 folio_put(folio);
3976 }3999 }
3977 4000
3978 /* No valid superblock */4001 /* No valid superblock */
@@ -3986,7 +4009,8 @@ static int read_raw_super_block(struct f2fs_sb_info *sbi, · 7 unmodified lines
3986 4009
3987int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover)4010int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover)
3988{4011{
3989 struct buffer_head *bh;4012 struct folio *folio;
4013 pgoff_t index;
3990 __u32 crc = 0;4014 __u32 crc = 0;
3991 int err;4015 int err;
3992 4016
@@ -4004,22 +4028,24 @@ int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover) · 11 unmodified lines
4004 }4028 }
4005 4029
4006 /* write back-up superblock first */4030 /* write back-up superblock first */
4007 bh = sb_bread(sbi->sb, sbi->valid_super_block ? 0 : 1);4031 index = sbi->valid_super_block ? 0 : 1;
4008 if (!bh)4032 folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping, index, NULL);
4009 return -EIO;4033 if (IS_ERR(folio))
4010 err = __f2fs_commit_super(bh, F2FS_RAW_SUPER(sbi));4034 return PTR_ERR(folio);
4011 brelse(bh);4035 err = __f2fs_commit_super(sbi, folio, index, true);
4036 folio_put(folio);
4012 4037
4013 /* if we are in recovery path, skip writing valid superblock */4038 /* if we are in recovery path, skip writing valid superblock */
4014 if (recover || err)4039 if (recover || err)
4015 return err;4040 return err;
4016 4041
4017 /* write current valid superblock */4042 /* write current valid superblock */
4018 bh = sb_bread(sbi->sb, sbi->valid_super_block);4043 index = sbi->valid_super_block;
4019 if (!bh)4044 folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping, index, NULL);
4020 return -EIO;4045 if (IS_ERR(folio))
4021 err = __f2fs_commit_super(bh, F2FS_RAW_SUPER(sbi));4046 return PTR_ERR(folio);
4022 brelse(bh);4047 err = __f2fs_commit_super(sbi, folio, index, true);
4048 folio_put(folio);
4023 return err;4049 return err;
4024}4050}
4025 4051
@@ -4173,12 +4199,14 @@ void f2fs_handle_critical_error(struct f2fs_sb_info *sbi, unsigned char reason, · 147 unmodified lines
4173 }4199 }
4174 4200
4175 f2fs_warn(sbi, "Remounting filesystem read-only");4201 f2fs_warn(sbi, "Remounting filesystem read-only");
4202
4176 /*4203 /*
4177 * Make sure updated value of ->s_mount_flags will be visible before4204 * We have already set CP_ERROR_FLAG flag to stop all updates
4178 * ->s_flags update4205 * to filesystem, so it doesn't need to set SB_RDONLY flag here
4206 * because the flag should be set covered w/ sb->s_umount semaphore
4207 * via remount procedure, otherwise, it will confuse code like
4208 * freeze_super() which will lead to deadlocks and other problems.
4179 */4209 */
4180 smp_wmb();
4181 sb->s_flags |= SB_RDONLY;
4182}4210}
4183 4211
4184static void f2fs_record_error_work(struct work_struct *work)4212static void f2fs_record_error_work(struct work_struct *work)
@@ -4219,6 +4247,7 @@ static int f2fs_scan_devices(struct f2fs_sb_info *sbi) · 34 unmodified lines
4219 sbi->aligned_blksize = true;4247 sbi->aligned_blksize = true;
4220#ifdef CONFIG_BLK_DEV_ZONED4248#ifdef CONFIG_BLK_DEV_ZONED
4221 sbi->max_open_zones = UINT_MAX;4249 sbi->max_open_zones = UINT_MAX;
4250 sbi->blkzone_alloc_policy = BLKZONE_ALLOC_PRIOR_SEQ;
4222#endif4251#endif
4223 4252
4224 for (i = 0; i < max_devices; i++) {4253 for (i = 0; i < max_devices; i++) {
fs/f2fs/sysfs.c+57−25
@@ -170,6 +170,12 @@ static ssize_t undiscard_blks_show(struct f2fs_attr *a, · 169 unmodified lines
170 SM_I(sbi)->dcc_info->undiscard_blks);170 SM_I(sbi)->dcc_info->undiscard_blks);
171}171}
172 172
173static ssize_t atgc_enabled_show(struct f2fs_attr *a,
174 struct f2fs_sb_info *sbi, char *buf)
175{
176 return sysfs_emit(buf, "%d\n", sbi->am.atgc_enabled ? 1 : 0);
177}
178
173static ssize_t gc_mode_show(struct f2fs_attr *a,179static ssize_t gc_mode_show(struct f2fs_attr *a,
174 struct f2fs_sb_info *sbi, char *buf)180 struct f2fs_sb_info *sbi, char *buf)
175{181{
@@ -182,50 +188,50 @@ static ssize_t features_show(struct f2fs_attr *a, · 6 unmodified lines
182 int len = 0;188 int len = 0;
183 189
184 if (f2fs_sb_has_encrypt(sbi))190 if (f2fs_sb_has_encrypt(sbi))
185 len += scnprintf(buf, PAGE_SIZE - len, "%s",191 len += sysfs_emit_at(buf, len, "%s",
186 "encryption");192 "encryption");
187 if (f2fs_sb_has_blkzoned(sbi))193 if (f2fs_sb_has_blkzoned(sbi))
188 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",194 len += sysfs_emit_at(buf, len, "%s%s",
189 len ? ", " : "", "blkzoned");195 len ? ", " : "", "blkzoned");
190 if (f2fs_sb_has_extra_attr(sbi))196 if (f2fs_sb_has_extra_attr(sbi))
191 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",197 len += sysfs_emit_at(buf, len, "%s%s",
192 len ? ", " : "", "extra_attr");198 len ? ", " : "", "extra_attr");
193 if (f2fs_sb_has_project_quota(sbi))199 if (f2fs_sb_has_project_quota(sbi))
194 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",200 len += sysfs_emit_at(buf, len, "%s%s",
195 len ? ", " : "", "projquota");201 len ? ", " : "", "projquota");
196 if (f2fs_sb_has_inode_chksum(sbi))202 if (f2fs_sb_has_inode_chksum(sbi))
197 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",203 len += sysfs_emit_at(buf, len, "%s%s",
198 len ? ", " : "", "inode_checksum");204 len ? ", " : "", "inode_checksum");
199 if (f2fs_sb_has_flexible_inline_xattr(sbi))205 if (f2fs_sb_has_flexible_inline_xattr(sbi))
200 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",206 len += sysfs_emit_at(buf, len, "%s%s",
201 len ? ", " : "", "flexible_inline_xattr");207 len ? ", " : "", "flexible_inline_xattr");
202 if (f2fs_sb_has_quota_ino(sbi))208 if (f2fs_sb_has_quota_ino(sbi))
203 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",209 len += sysfs_emit_at(buf, len, "%s%s",
204 len ? ", " : "", "quota_ino");210 len ? ", " : "", "quota_ino");
205 if (f2fs_sb_has_inode_crtime(sbi))211 if (f2fs_sb_has_inode_crtime(sbi))
206 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",212 len += sysfs_emit_at(buf, len, "%s%s",
207 len ? ", " : "", "inode_crtime");213 len ? ", " : "", "inode_crtime");
208 if (f2fs_sb_has_lost_found(sbi))214 if (f2fs_sb_has_lost_found(sbi))
209 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",215 len += sysfs_emit_at(buf, len, "%s%s",
210 len ? ", " : "", "lost_found");216 len ? ", " : "", "lost_found");
211 if (f2fs_sb_has_verity(sbi))217 if (f2fs_sb_has_verity(sbi))
212 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",218 len += sysfs_emit_at(buf, len, "%s%s",
213 len ? ", " : "", "verity");219 len ? ", " : "", "verity");
214 if (f2fs_sb_has_sb_chksum(sbi))220 if (f2fs_sb_has_sb_chksum(sbi))
215 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",221 len += sysfs_emit_at(buf, len, "%s%s",
216 len ? ", " : "", "sb_checksum");222 len ? ", " : "", "sb_checksum");
217 if (f2fs_sb_has_casefold(sbi))223 if (f2fs_sb_has_casefold(sbi))
218 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",224 len += sysfs_emit_at(buf, len, "%s%s",
219 len ? ", " : "", "casefold");225 len ? ", " : "", "casefold");
220 if (f2fs_sb_has_readonly(sbi))226 if (f2fs_sb_has_readonly(sbi))
221 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",227 len += sysfs_emit_at(buf, len, "%s%s",
222 len ? ", " : "", "readonly");228 len ? ", " : "", "readonly");
223 if (f2fs_sb_has_compression(sbi))229 if (f2fs_sb_has_compression(sbi))
224 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",230 len += sysfs_emit_at(buf, len, "%s%s",
225 len ? ", " : "", "compression");231 len ? ", " : "", "compression");
226 len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s",232 len += sysfs_emit_at(buf, len, "%s%s",
227 len ? ", " : "", "pin_file");233 len ? ", " : "", "pin_file");
228 len += scnprintf(buf + len, PAGE_SIZE - len, "\n");234 len += sysfs_emit_at(buf, len, "\n");
229 return len;235 return len;
230}236}
231 237
@@ -323,17 +329,14 @@ static ssize_t f2fs_sbi_show(struct f2fs_attr *a, · 91 unmodified lines
323 int hot_count = sbi->raw_super->hot_ext_count;329 int hot_count = sbi->raw_super->hot_ext_count;
324 int len = 0, i;330 int len = 0, i;
325 331
326 len += scnprintf(buf + len, PAGE_SIZE - len,332 len += sysfs_emit_at(buf, len, "cold file extension:\n");
327 "cold file extension:\n");
328 for (i = 0; i < cold_count; i++)333 for (i = 0; i < cold_count; i++)
329 len += scnprintf(buf + len, PAGE_SIZE - len, "%s\n",334 len += sysfs_emit_at(buf, len, "%s\n", extlist[i]);
330 extlist[i]);
331 335
332 len += scnprintf(buf + len, PAGE_SIZE - len,336 len += sysfs_emit_at(buf, len, "hot file extension:\n");
333 "hot file extension:\n");
334 for (i = cold_count; i < cold_count + hot_count; i++)337 for (i = cold_count; i < cold_count + hot_count; i++)
335 len += scnprintf(buf + len, PAGE_SIZE - len, "%s\n",338 len += sysfs_emit_at(buf, len, "%s\n", extlist[i]);
336 extlist[i]);339
337 return len;340 return len;
338 }341 }
339 342
@@ -561,6 +564,11 @@ out: · 221 unmodified lines
561 return -EINVAL;564 return -EINVAL;
562 }565 }
563 566
567 if (!strcmp(a->attr.name, "migration_window_granularity")) {
568 if (t == 0 || t > SEGS_PER_SEC(sbi))
569 return -EINVAL;
570 }
571
564 if (!strcmp(a->attr.name, "gc_urgent")) {572 if (!strcmp(a->attr.name, "gc_urgent")) {
565 if (t == 0) {573 if (t == 0) {
566 sbi->gc_mode = GC_NORMAL;574 sbi->gc_mode = GC_NORMAL;
@@ -627,6 +635,15 @@ out: · 60 unmodified lines
627 }635 }
628#endif636#endif
629 637
638#ifdef CONFIG_BLK_DEV_ZONED
639 if (!strcmp(a->attr.name, "blkzone_alloc_policy")) {
640 if (t < BLKZONE_ALLOC_PRIOR_SEQ || t > BLKZONE_ALLOC_PRIOR_CONV)
641 return -EINVAL;
642 sbi->blkzone_alloc_policy = t;
643 return count;
644 }
645#endif
646
630#ifdef CONFIG_F2FS_FS_COMPRESSION647#ifdef CONFIG_F2FS_FS_COMPRESSION
631 if (!strcmp(a->attr.name, "compr_written_block") ||648 if (!strcmp(a->attr.name, "compr_written_block") ||
632 !strcmp(a->attr.name, "compr_saved_block")) {649 !strcmp(a->attr.name, "compr_saved_block")) {
@@ -775,7 +792,8 @@ out: · 142 unmodified lines
775 if (!strcmp(a->attr.name, "ipu_policy")) {792 if (!strcmp(a->attr.name, "ipu_policy")) {
776 if (t >= BIT(F2FS_IPU_MAX))793 if (t >= BIT(F2FS_IPU_MAX))
777 return -EINVAL;794 return -EINVAL;
778 if (t && f2fs_lfs_mode(sbi))795 /* allow F2FS_IPU_NOCACHE only for IPU in the pinned file */
796 if (f2fs_lfs_mode(sbi) && (t & ~BIT(F2FS_IPU_NOCACHE)))
779 return -EINVAL;797 return -EINVAL;
780 SM_I(sbi)->ipu_policy = (unsigned int)t;798 SM_I(sbi)->ipu_policy = (unsigned int)t;
781 return count;799 return count;
@@ -960,6 +978,9 @@ GC_THREAD_RW_ATTR(gc_urgent_sleep_time, urgent_sleep_time); · 178 unmodified lines
960GC_THREAD_RW_ATTR(gc_min_sleep_time, min_sleep_time);978GC_THREAD_RW_ATTR(gc_min_sleep_time, min_sleep_time);
961GC_THREAD_RW_ATTR(gc_max_sleep_time, max_sleep_time);979GC_THREAD_RW_ATTR(gc_max_sleep_time, max_sleep_time);
962GC_THREAD_RW_ATTR(gc_no_gc_sleep_time, no_gc_sleep_time);980GC_THREAD_RW_ATTR(gc_no_gc_sleep_time, no_gc_sleep_time);
981GC_THREAD_RW_ATTR(gc_no_zoned_gc_percent, no_zoned_gc_percent);
982GC_THREAD_RW_ATTR(gc_boost_zoned_gc_percent, boost_zoned_gc_percent);
983GC_THREAD_RW_ATTR(gc_valid_thresh_ratio, valid_thresh_ratio);
963 984
964/* SM_INFO ATTR */985/* SM_INFO ATTR */
965SM_INFO_RW_ATTR(reclaim_segments, rec_prefree_segments);986SM_INFO_RW_ATTR(reclaim_segments, rec_prefree_segments);
@@ -969,6 +990,7 @@ SM_INFO_GENERAL_RW_ATTR(min_fsync_blocks); · 3 unmodified lines
969SM_INFO_GENERAL_RW_ATTR(min_seq_blocks);990SM_INFO_GENERAL_RW_ATTR(min_seq_blocks);
970SM_INFO_GENERAL_RW_ATTR(min_hot_blocks);991SM_INFO_GENERAL_RW_ATTR(min_hot_blocks);
971SM_INFO_GENERAL_RW_ATTR(min_ssr_sections);992SM_INFO_GENERAL_RW_ATTR(min_ssr_sections);
993SM_INFO_GENERAL_RW_ATTR(reserved_segments);
972 994
973/* DCC_INFO ATTR */995/* DCC_INFO ATTR */
974DCC_INFO_RW_ATTR(max_small_discards, max_discards);996DCC_INFO_RW_ATTR(max_small_discards, max_discards);
@@ -1001,6 +1023,7 @@ F2FS_SBI_RW_ATTR(gc_pin_file_thresh, gc_pin_file_threshold); · 26 unmodified lines
1001F2FS_SBI_RW_ATTR(gc_reclaimed_segments, gc_reclaimed_segs);1023F2FS_SBI_RW_ATTR(gc_reclaimed_segments, gc_reclaimed_segs);
1002F2FS_SBI_GENERAL_RW_ATTR(max_victim_search);1024F2FS_SBI_GENERAL_RW_ATTR(max_victim_search);
1003F2FS_SBI_GENERAL_RW_ATTR(migration_granularity);1025F2FS_SBI_GENERAL_RW_ATTR(migration_granularity);
1026F2FS_SBI_GENERAL_RW_ATTR(migration_window_granularity);
1004F2FS_SBI_GENERAL_RW_ATTR(dir_level);1027F2FS_SBI_GENERAL_RW_ATTR(dir_level);
1005#ifdef CONFIG_F2FS_IOSTAT1028#ifdef CONFIG_F2FS_IOSTAT
1006F2FS_SBI_GENERAL_RW_ATTR(iostat_enable);1029F2FS_SBI_GENERAL_RW_ATTR(iostat_enable);
@@ -1033,6 +1056,7 @@ F2FS_SBI_GENERAL_RW_ATTR(warm_data_age_threshold); · 26 unmodified lines
1033F2FS_SBI_GENERAL_RW_ATTR(last_age_weight);1056F2FS_SBI_GENERAL_RW_ATTR(last_age_weight);
1034#ifdef CONFIG_BLK_DEV_ZONED1057#ifdef CONFIG_BLK_DEV_ZONED
1035F2FS_SBI_GENERAL_RO_ATTR(unusable_blocks_per_sec);1058F2FS_SBI_GENERAL_RO_ATTR(unusable_blocks_per_sec);
1059F2FS_SBI_GENERAL_RW_ATTR(blkzone_alloc_policy);
1036#endif1060#endif
1037 1061
1038/* STAT_INFO ATTR */1062/* STAT_INFO ATTR */
@@ -1072,6 +1096,7 @@ F2FS_GENERAL_RO_ATTR(encoding); · 33 unmodified lines
1072F2FS_GENERAL_RO_ATTR(mounted_time_sec);1096F2FS_GENERAL_RO_ATTR(mounted_time_sec);
1073F2FS_GENERAL_RO_ATTR(main_blkaddr);1097F2FS_GENERAL_RO_ATTR(main_blkaddr);
1074F2FS_GENERAL_RO_ATTR(pending_discard);1098F2FS_GENERAL_RO_ATTR(pending_discard);
1099F2FS_GENERAL_RO_ATTR(atgc_enabled);
1075F2FS_GENERAL_RO_ATTR(gc_mode);1100F2FS_GENERAL_RO_ATTR(gc_mode);
1076#ifdef CONFIG_F2FS_STAT_FS1101#ifdef CONFIG_F2FS_STAT_FS
1077F2FS_GENERAL_RO_ATTR(moved_blocks_background);1102F2FS_GENERAL_RO_ATTR(moved_blocks_background);
@@ -1116,6 +1141,9 @@ static struct attribute *f2fs_attrs[] = { · 38 unmodified lines
1116 ATTR_LIST(gc_min_sleep_time),1141 ATTR_LIST(gc_min_sleep_time),
1117 ATTR_LIST(gc_max_sleep_time),1142 ATTR_LIST(gc_max_sleep_time),
1118 ATTR_LIST(gc_no_gc_sleep_time),1143 ATTR_LIST(gc_no_gc_sleep_time),
1144 ATTR_LIST(gc_no_zoned_gc_percent),
1145 ATTR_LIST(gc_boost_zoned_gc_percent),
1146 ATTR_LIST(gc_valid_thresh_ratio),
1119 ATTR_LIST(gc_idle),1147 ATTR_LIST(gc_idle),
1120 ATTR_LIST(gc_urgent),1148 ATTR_LIST(gc_urgent),
1121 ATTR_LIST(reclaim_segments),1149 ATTR_LIST(reclaim_segments),
@@ -1138,8 +1166,10 @@ static struct attribute *f2fs_attrs[] = { · 16 unmodified lines
1138 ATTR_LIST(min_seq_blocks),1166 ATTR_LIST(min_seq_blocks),
1139 ATTR_LIST(min_hot_blocks),1167 ATTR_LIST(min_hot_blocks),
1140 ATTR_LIST(min_ssr_sections),1168 ATTR_LIST(min_ssr_sections),
1169 ATTR_LIST(reserved_segments),
1141 ATTR_LIST(max_victim_search),1170 ATTR_LIST(max_victim_search),
1142 ATTR_LIST(migration_granularity),1171 ATTR_LIST(migration_granularity),
1172 ATTR_LIST(migration_window_granularity),
1143 ATTR_LIST(dir_level),1173 ATTR_LIST(dir_level),
1144 ATTR_LIST(ram_thresh),1174 ATTR_LIST(ram_thresh),
1145 ATTR_LIST(ra_nid_pages),1175 ATTR_LIST(ra_nid_pages),
@@ -1187,6 +1217,7 @@ static struct attribute *f2fs_attrs[] = { · 41 unmodified lines
1187#endif1217#endif
1188#ifdef CONFIG_BLK_DEV_ZONED1218#ifdef CONFIG_BLK_DEV_ZONED
1189 ATTR_LIST(unusable_blocks_per_sec),1219 ATTR_LIST(unusable_blocks_per_sec),
1220 ATTR_LIST(blkzone_alloc_policy),
1190#endif1221#endif
1191#ifdef CONFIG_F2FS_FS_COMPRESSION1222#ifdef CONFIG_F2FS_FS_COMPRESSION
1192 ATTR_LIST(compr_written_block),1223 ATTR_LIST(compr_written_block),
@@ -1200,6 +1231,7 @@ static struct attribute *f2fs_attrs[] = { · 7 unmodified lines
1200 ATTR_LIST(atgc_candidate_count),1231 ATTR_LIST(atgc_candidate_count),
1201 ATTR_LIST(atgc_age_weight),1232 ATTR_LIST(atgc_age_weight),
1202 ATTR_LIST(atgc_age_threshold),1233 ATTR_LIST(atgc_age_threshold),
1234 ATTR_LIST(atgc_enabled),
1203 ATTR_LIST(seq_file_ra_mul),1235 ATTR_LIST(seq_file_ra_mul),
1204 ATTR_LIST(gc_segment_mode),1236 ATTR_LIST(gc_segment_mode),
1205 ATTR_LIST(gc_reclaimed_segments),1237 ATTR_LIST(gc_reclaimed_segments),
fs/f2fs/verity.c+3−2
@@ -74,7 +74,7 @@ static int pagecache_write(struct inode *inode, const void *buf, size_t count, · 73 unmodified lines
74 struct address_space *mapping = inode->i_mapping;74 struct address_space *mapping = inode->i_mapping;
75 const struct address_space_operations *aops = mapping->a_ops;75 const struct address_space_operations *aops = mapping->a_ops;
76 76
77 if (pos + count > inode->i_sb->s_maxbytes)77 if (pos + count > F2FS_BLK_TO_BYTES(max_file_blocks(inode)))
78 return -EFBIG;78 return -EFBIG;
79 79
80 while (count) {80 while (count) {
@@ -237,7 +237,8 @@ static int f2fs_get_verity_descriptor(struct inode *inode, void *buf, · 156 unmodified lines
237 pos = le64_to_cpu(dloc.pos);237 pos = le64_to_cpu(dloc.pos);
238 238
239 /* Get the descriptor */239 /* Get the descriptor */
240 if (pos + size < pos || pos + size > inode->i_sb->s_maxbytes ||240 if (pos + size < pos ||
241 pos + size > F2FS_BLK_TO_BYTES(max_file_blocks(inode)) ||
241 pos < f2fs_verity_metadata_pos(inode) || size > INT_MAX) {242 pos < f2fs_verity_metadata_pos(inode) || size > INT_MAX) {
242 f2fs_warn(F2FS_I_SB(inode), "invalid verity xattr");243 f2fs_warn(F2FS_I_SB(inode), "invalid verity xattr");
243 f2fs_handle_error(F2FS_I_SB(inode),244 f2fs_handle_error(F2FS_I_SB(inode),
fs/f2fs/xattr.c+12−2
@@ -629,6 +629,7 @@ static int __f2fs_setxattr(struct inode *inode, int index, · 628 unmodified lines
629 const char *name, const void *value, size_t size,629 const char *name, const void *value, size_t size,
630 struct page *ipage, int flags)630 struct page *ipage, int flags)
631{631{
632 struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
632 struct f2fs_xattr_entry *here, *last;633 struct f2fs_xattr_entry *here, *last;
633 void *base_addr, *last_base_addr;634 void *base_addr, *last_base_addr;
634 int found, newsize;635 int found, newsize;
@@ -772,9 +773,18 @@ retry: · 137 unmodified lines
772 if (index == F2FS_XATTR_INDEX_ENCRYPTION &&773 if (index == F2FS_XATTR_INDEX_ENCRYPTION &&
773 !strcmp(name, F2FS_XATTR_NAME_ENCRYPTION_CONTEXT))774 !strcmp(name, F2FS_XATTR_NAME_ENCRYPTION_CONTEXT))
774 f2fs_set_encrypted_inode(inode);775 f2fs_set_encrypted_inode(inode);
775 if (S_ISDIR(inode->i_mode))
776 set_sbi_flag(F2FS_I_SB(inode), SBI_NEED_CP);
777 776
777 if (!S_ISDIR(inode->i_mode))
778 goto same;
779 /*
780 * In restrict mode, fsync() always try to trigger checkpoint for all
781 * metadata consistency, in other mode, it triggers checkpoint when
782 * parent's xattr metadata was updated.
783 */
784 if (F2FS_OPTION(sbi).fsync_mode == FSYNC_MODE_STRICT)
785 set_sbi_flag(sbi, SBI_NEED_CP);
786 else
787 f2fs_add_ino_entry(sbi, inode->i_ino, XATTR_DIR_INO);
778same:788same:
779 if (is_inode_flag_set(inode, FI_ACL_MODE)) {789 if (is_inode_flag_set(inode, FI_ACL_MODE)) {
780 inode->i_mode = F2FS_I(inode)->i_acl_mode;790 inode->i_mode = F2FS_I(inode)->i_acl_mode;
include/linux/f2fs_fs.h+2−2
@@ -19,7 +19,6 @@ · 18 unmodified lines
19#define F2FS_BLKSIZE_BITS PAGE_SHIFT /* bits for F2FS_BLKSIZE */19#define F2FS_BLKSIZE_BITS PAGE_SHIFT /* bits for F2FS_BLKSIZE */
20#define F2FS_MAX_EXTENSION 64 /* # of extension entries */20#define F2FS_MAX_EXTENSION 64 /* # of extension entries */
21#define F2FS_EXTENSION_LEN 8 /* max size of extension */21#define F2FS_EXTENSION_LEN 8 /* max size of extension */
22#define F2FS_BLK_ALIGN(x) (((x) + F2FS_BLKSIZE - 1) >> F2FS_BLKSIZE_BITS)
23 22
24#define NULL_ADDR ((block_t)0) /* used as block_t addresses */23#define NULL_ADDR ((block_t)0) /* used as block_t addresses */
25#define NEW_ADDR ((block_t)-1) /* used as block_t addresses */24#define NEW_ADDR ((block_t)-1) /* used as block_t addresses */
@@ -28,6 +27,7 @@ · 2 unmodified lines
28#define F2FS_BYTES_TO_BLK(bytes) ((bytes) >> F2FS_BLKSIZE_BITS)27#define F2FS_BYTES_TO_BLK(bytes) ((bytes) >> F2FS_BLKSIZE_BITS)
29#define F2FS_BLK_TO_BYTES(blk) ((blk) << F2FS_BLKSIZE_BITS)28#define F2FS_BLK_TO_BYTES(blk) ((blk) << F2FS_BLKSIZE_BITS)
30#define F2FS_BLK_END_BYTES(blk) (F2FS_BLK_TO_BYTES(blk + 1) - 1)29#define F2FS_BLK_END_BYTES(blk) (F2FS_BLK_TO_BYTES(blk + 1) - 1)
30#define F2FS_BLK_ALIGN(x) (F2FS_BYTES_TO_BLK((x) + F2FS_BLKSIZE - 1))
31 31
32/* 0, 1(node nid), 2(meta nid) are reserved node id */32/* 0, 1(node nid), 2(meta nid) are reserved node id */
33#define F2FS_RESERVED_NODE_NUM 333#define F2FS_RESERVED_NODE_NUM 3
@@ -278,7 +278,7 @@ struct node_footer { · 244 unmodified lines
278#define F2FS_INLINE_DATA 0x02 /* file inline data flag */278#define F2FS_INLINE_DATA 0x02 /* file inline data flag */
279#define F2FS_INLINE_DENTRY 0x04 /* file inline dentry flag */279#define F2FS_INLINE_DENTRY 0x04 /* file inline dentry flag */
280#define F2FS_DATA_EXIST 0x08 /* file inline data exist flag */280#define F2FS_DATA_EXIST 0x08 /* file inline data exist flag */
281#define F2FS_INLINE_DOTS 0x10 /* file having implicit dot dentries */281#define F2FS_INLINE_DOTS 0x10 /* file having implicit dot dentries (obsolete) */
282#define F2FS_EXTRA_ATTR 0x20 /* file having extra attribute */282#define F2FS_EXTRA_ATTR 0x20 /* file having extra attribute */
283#define F2FS_PIN_FILE 0x40 /* file should not be gced */283#define F2FS_PIN_FILE 0x40 /* file should not be gced */
284#define F2FS_COMPRESS_RELEASED 0x80 /* file released compressed blocks */284#define F2FS_COMPRESS_RELEASED 0x80 /* file released compressed blocks */
include/trace/events/f2fs.h+2−1
@@ -139,7 +139,8 @@ TRACE_DEFINE_ENUM(EX_BLOCK_AGE); · 138 unmodified lines
139 { CP_NODE_NEED_CP, "node needs cp" }, \139 { CP_NODE_NEED_CP, "node needs cp" }, \
140 { CP_FASTBOOT_MODE, "fastboot mode" }, \140 { CP_FASTBOOT_MODE, "fastboot mode" }, \
141 { CP_SPEC_LOG_NUM, "log type is 2" }, \141 { CP_SPEC_LOG_NUM, "log type is 2" }, \
142 { CP_RECOVER_DIR, "dir needs recovery" })142 { CP_RECOVER_DIR, "dir needs recovery" }, \
143 { CP_XATTR_DIR, "dir's xattr updated" })
143 144
144#define show_shutdown_mode(type) \145#define show_shutdown_mode(type) \
145 __print_symbolic(type, \146 __print_symbolic(type, \