History

Merge tag 'erofs-for-5.16-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs

Pull erofs updates from Gao Xiang: "There are some new features available for this cycle. Firstly, EROFS LZMA algorithm support, specifically called MicroLZMA, is available as an option for embedded devices, LiveCDs and/or as the secondary auxiliary compression algorithm besides the primary algorithm in one file. In order to better support the LZMA fixed-sized output compression, especially for 4KiB pcluster size (which has lowest memory pressure thus useful for memory-sensitive scenarios), Lasse introduced a new LZMA header/container format called MicroLZMA to minimize the original LZMA1 header (for example, we don't need to waste 4-byte dictionary size and another 8-byte uncompressed size, which can be calculated by fs directly, for each pcluster) and enable EROFS fixed-sized output compression. Note that MicroLZMA can also be later used by other things in addition to EROFS too where wasting minimal amount of space for headers is important and it can be only compiled by enabling XZ_DEC_MICROLZMA. MicroLZMA has been supported by the latest upstream XZ embedded [1] & XZ utils [2], apply the latest related XZ embedded upstream patches by the XZ author Lasse here. Secondly, multiple device is also supported in this cycle, which is designed for multi-layer container images. By working together with inter-layer data deduplication and compression, we can achieve the next high-performance container image solution. Our team will announce the new Nydus container image service [3] implementation with new RAFS v6 (EROFS-compatible) format in Open Source Summit 2021 China [4] soon. Besides, the secondary compression head support and readmore decompression strategy are also included in this cycle. There are also some minor bugfixes and cleanups, as always. Summary: - support multiple devices for multi-layer container images; - support the secondary compression head; - support readmore decompression strategy; - support new LZMA algorithm (specifically called MicroLZMA); - some bugfixes & cleanups" * tag 'erofs-for-5.16-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs: erofs: don't trigger WARN() when decompression fails erofs: get rid of ->lru usage erofs: lzma compression support erofs: rename some generic methods in decompressor lib/xz, lib/decompress_unxz.c: Fix spelling in comments lib/xz: Add MicroLZMA decoder lib/xz: Move s->lzma.len = 0 initialization to lzma_reset() lib/xz: Validate the value before assigning it to an enum variable lib/xz: Avoid overlapping memcpy() with invalid input with in-place decompression erofs: introduce readmore decompression strategy erofs: introduce the secondary compression head erofs: get compression algorithms directly on mapping erofs: add multiple device support erofs: decouple basic mount options from fs_context erofs: remove the fast path of per-CPU buffer decompression

Linus Torvalds committed 5 years ago

Commit 67a135b2 parents cd3e8ea a0961f325 files · +1281 −320Browse files

Unified · Split

25 files
  1. Documentation/filesystems/erofs.rst +8 −4
  2. fs/erofs/Kconfig +31 −9
  3. fs/erofs/Makefile +1 −0
  4. fs/erofs/compress.h +19 −9
  5. fs/erofs/data.c +60 −13
  6. fs/erofs/decompressor.c +44 −95
  7. fs/erofs/decompressor_lzma.c +290 −0
  8. fs/erofs/erofs_fs.h +50 −23
  9. fs/erofs/inode.c +1 −1
  10. fs/erofs/internal.h +94 −11
  11. fs/erofs/pcpubuf.c +3 −3
  12. fs/erofs/super.c +188 −43
  13. fs/erofs/utils.c +14 −5
  14. fs/erofs/xattr.c +2 −2
  15. fs/erofs/zdata.c +120 −55
  16. fs/erofs/zdata.h +0 −7
  17. fs/erofs/zmap.c +42 −23
  18. include/linux/xz.h +106 −0
  19. include/trace/events/erofs.h +1 −1
  20. lib/decompress_unxz.c +5 −5
  21. lib/xz/Kconfig +13 −0
  22. lib/xz/xz_dec_lzma2.c +175 −7
  23. lib/xz/xz_dec_stream.c +3 −3
  24. lib/xz/xz_dec_syms.c +8 −1
  25. lib/xz/xz_private.h +3 −0
Documentation/filesystems/erofs.rst+8−4
@@ -19,9 +19,10 @@ It is designed as a better filesystem solution for the following scenarios: · 18 unmodified lines
1919 immutable and bit-for-bit identical to the official golden image for
2020 their releases due to security and other considerations and
2121
22− - hope to save some extra storage space with guaranteed end-to-end performance
23− by using reduced metadata and transparent file compression, especially
24− for those embedded devices with limited memory (ex, smartphone);
22+ - hope to minimize extra storage space with guaranteed end-to-end performance
23+ by using compact layout, transparent file compression and direct access,
24+ especially for those embedded devices with limited memory and high-density
25+ hosts with numerous containers;
2526
2627 Here is the main features of EROFS:
2728
@@ -51,7 +52,9 @@ Here is the main features of EROFS: · 23 unmodified lines
5152 - Support POSIX.1e ACLs by using xattrs;
5253
5354 - Support transparent data compression as an option:
54− LZ4 algorithm with the fixed-sized output compression for high performance.
55+ LZ4 algorithm with the fixed-sized output compression for high performance;
56+
57+ - Multiple device support for multi-layer container images.
5558
5659 The following git tree provides the file system user-space tools under
5760 development (ex, formatting tool mkfs.erofs):
@@ -87,6 +90,7 @@ cache_strategy=%s Select a strategy for cached decompression from now on: · 29 unmodified lines
8790 dax={always,never} Use direct access (no page cache). See
8891 Documentation/filesystems/dax.rst.
8992 dax A legacy option which is an alias for ``dax=always``.
93+device=%s Specify a path to an extra device to be used together.
9094 =================== =========================================================
9195
9296 On-disk details
fs/erofs/Kconfig+31−9
@@ -6,16 +6,22 @@ config EROFS_FS · 5 unmodified lines
66 select FS_IOMAP
77 select LIBCRC32C
88 help
9− EROFS (Enhanced Read-Only File System) is a lightweight
10− read-only file system with modern designs (eg. page-sized
11− blocks, inline xattrs/data, etc.) for scenarios which need
12− high-performance read-only requirements, e.g. Android OS
13− for mobile phones and LIVECDs.
9+ EROFS (Enhanced Read-Only File System) is a lightweight read-only
10+ file system with modern designs (e.g. no buffer heads, inline
11+ xattrs/data, chunk-based deduplication, multiple devices, etc.) for
12+ scenarios which need high-performance read-only solutions, e.g.
13+ smartphones with Android OS, LiveCDs and high-density hosts with
14+ numerous containers;
1415
15− It also provides fixed-sized output compression support,
16− which improves storage density, keeps relatively higher
17− compression ratios, which is more useful to achieve high
18− performance for embedded devices with limited memory.
16+ It also provides fixed-sized output compression support in order to
17+ improve storage density as well as keep relatively higher compression
18+ ratios and implements in-place decompression to reuse the file page
19+ for compressed data temporarily with proper strategies, which is
20+ quite useful to ensure guaranteed end-to-end runtime decompression
21+ performance under extremely memory pressure without extra cost.
22+
23+ See the documentation at <file:Documentation/filesystems/erofs.rst>
24+ for more details.
1925
2026 If unsure, say N.
2127
@@ -76,3 +82,19 @@ config EROFS_FS_ZIP · 54 unmodified lines
7682 Enable fixed-sized output compression for EROFS.
7783
7884 If you don't want to enable compression feature, say N.
85+
86+config EROFS_FS_ZIP_LZMA
87+ bool "EROFS LZMA compressed data support"
88+ depends on EROFS_FS_ZIP
89+ select XZ_DEC
90+ select XZ_DEC_MICROLZMA
91+ help
92+ Saying Y here includes support for reading EROFS file systems
93+ containing LZMA compressed data, specifically called microLZMA. it
94+ gives better compression ratios than the LZ4 algorithm, at the
95+ expense of more CPU overhead.
96+
97+ LZMA support is an experimental feature for now and so most file
98+ systems will be readable without selecting this option.
99+
100+ If unsure, say N.
fs/erofs/Makefile+1−0
@@ -4,3 +4,4 @@ obj-$(CONFIG_EROFS_FS) += erofs.o · 3 unmodified lines
44 erofs-objs := super.o inode.o data.o namei.o dir.o utils.o pcpubuf.o
55 erofs-$(CONFIG_EROFS_FS_XATTR) += xattr.o
66 erofs-$(CONFIG_EROFS_FS_ZIP) += decompressor.o zmap.o zdata.o
7+erofs-$(CONFIG_EROFS_FS_ZIP_LZMA) += decompressor_lzma.o
fs/erofs/compress.h+19−9
@@ -8,11 +8,6 @@ · 7 unmodified lines
88
99 #include "internal.h"
1010
11−enum {
12− Z_EROFS_COMPRESSION_SHIFTED = Z_EROFS_COMPRESSION_MAX,
13− Z_EROFS_COMPRESSION_RUNTIME_MAX
14−};
15−
1611 struct z_erofs_decompress_req {
1712 struct super_block *sb;
1813 struct page **in, **out;
@@ -25,6 +20,12 @@ struct z_erofs_decompress_req { · 6 unmodified lines
2520 bool inplace_io, partial_decoding;
2621 };
2722
23+struct z_erofs_decompressor {
24+ int (*decompress)(struct z_erofs_decompress_req *rq,
25+ struct page **pagepool);
26+ char *name;
27+};
28+
2829 /* some special page->private (unsigned long, see below) */
2930 #define Z_EROFS_SHORTLIVED_PAGE (-1UL << 2)
3031 #define Z_EROFS_PREALLOCATED_PAGE (-2UL << 2)
@@ -63,7 +64,7 @@ static inline bool z_erofs_is_shortlived_page(struct page *page) · 32 unmodified lines
6364 return true;
6465 }
6566
66−static inline bool z_erofs_put_shortlivedpage(struct list_head *pagepool,
67+static inline bool z_erofs_put_shortlivedpage(struct page **pagepool,
6768 struct page *page)
6869 {
6970 if (!z_erofs_is_shortlived_page(page))
@@ -74,13 +75,22 @@ static inline bool z_erofs_put_shortlivedpage(struct list_head *pagepool, · 4 unmodified lines
7475 put_page(page);
7576 } else {
7677 /* follow the pcluster rule above. */
77− set_page_private(page, 0);
78− list_add(&page->lru, pagepool);
78+ erofs_pagepool_add(pagepool, page);
7979 }
8080 return true;
8181 }
8282
83+#define MNGD_MAPPING(sbi) ((sbi)->managed_cache->i_mapping)
84+static inline bool erofs_page_is_managed(const struct erofs_sb_info *sbi,
85+ struct page *page)
86+{
87+ return page->mapping == MNGD_MAPPING(sbi);
88+}
89+
8390 int z_erofs_decompress(struct z_erofs_decompress_req *rq,
84− struct list_head *pagepool);
91+ struct page **pagepool);
8592
93+/* prototypes for specific algorithms */
94+int z_erofs_lzma_decompress(struct z_erofs_decompress_req *rq,
95+ struct page **pagepool);
8696 #endif
fs/erofs/data.c+60−13
@@ -89,6 +89,7 @@ static int erofs_map_blocks(struct inode *inode, · 88 unmodified lines
8989 erofs_off_t pos;
9090 int err = 0;
9191
92+ map->m_deviceid = 0;
9293 if (map->m_la >= inode->i_size) {
9394 /* leave out-of-bound access unmapped */
9495 map->m_flags = 0;
@@ -135,14 +136,8 @@ static int erofs_map_blocks(struct inode *inode, · 40 unmodified lines
135136 map->m_flags = 0;
136137 break;
137138 default:
138− /* only one device is supported for now */
139− if (idx->device_id) {
140− erofs_err(sb, "invalid device id %u @ %llu for nid %llu",
141− le16_to_cpu(idx->device_id),
142− chunknr, vi->nid);
143− err = -EFSCORRUPTED;
144− goto out_unlock;
145− }
139+ map->m_deviceid = le16_to_cpu(idx->device_id) &
140+ EROFS_SB(sb)->device_id_mask;
146141 map->m_pa = blknr_to_addr(le32_to_cpu(idx->blkaddr));
147142 map->m_flags = EROFS_MAP_MAPPED;
148143 break;
@@ -155,11 +150,55 @@ out: · 6 unmodified lines
155150 return err;
156151 }
157152
153+int erofs_map_dev(struct super_block *sb, struct erofs_map_dev *map)
154+{
155+ struct erofs_dev_context *devs = EROFS_SB(sb)->devs;
156+ struct erofs_device_info *dif;
157+ int id;
158+
159+ /* primary device by default */
160+ map->m_bdev = sb->s_bdev;
161+ map->m_daxdev = EROFS_SB(sb)->dax_dev;
162+
163+ if (map->m_deviceid) {
164+ down_read(&devs->rwsem);
165+ dif = idr_find(&devs->tree, map->m_deviceid - 1);
166+ if (!dif) {
167+ up_read(&devs->rwsem);
168+ return -ENODEV;
169+ }
170+ map->m_bdev = dif->bdev;
171+ map->m_daxdev = dif->dax_dev;
172+ up_read(&devs->rwsem);
173+ } else if (devs->extra_devices) {
174+ down_read(&devs->rwsem);
175+ idr_for_each_entry(&devs->tree, dif, id) {
176+ erofs_off_t startoff, length;
177+
178+ if (!dif->mapped_blkaddr)
179+ continue;
180+ startoff = blknr_to_addr(dif->mapped_blkaddr);
181+ length = blknr_to_addr(dif->blocks);
182+
183+ if (map->m_pa >= startoff &&
184+ map->m_pa < startoff + length) {
185+ map->m_pa -= startoff;
186+ map->m_bdev = dif->bdev;
187+ map->m_daxdev = dif->dax_dev;
188+ break;
189+ }
190+ }
191+ up_read(&devs->rwsem);
192+ }
193+ return 0;
194+}
195+
158196 static int erofs_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
159197 unsigned int flags, struct iomap *iomap, struct iomap *srcmap)
160198 {
161199 int ret;
162200 struct erofs_map_blocks map;
201+ struct erofs_map_dev mdev;
163202
164203 map.m_la = offset;
165204 map.m_llen = length;
@@ -168,8 +207,16 @@ static int erofs_iomap_begin(struct inode *inode, loff_t offset, loff_t length, · 2 unmodified lines
168207 if (ret < 0)
169208 return ret;
170209
171− iomap->bdev = inode->i_sb->s_bdev;
172− iomap->dax_dev = EROFS_I_SB(inode)->dax_dev;
210+ mdev = (struct erofs_map_dev) {
211+ .m_deviceid = map.m_deviceid,
212+ .m_pa = map.m_pa,
213+ };
214+ ret = erofs_map_dev(inode->i_sb, &mdev);
215+ if (ret)
216+ return ret;
217+
218+ iomap->bdev = mdev.m_bdev;
219+ iomap->dax_dev = mdev.m_daxdev;
173220 iomap->offset = map.m_la;
174221 iomap->length = map.m_llen;
175222 iomap->flags = 0;
@@ -188,15 +235,15 @@ static int erofs_iomap_begin(struct inode *inode, loff_t offset, loff_t length, · 12 unmodified lines
188235
189236 iomap->type = IOMAP_INLINE;
190237 ipage = erofs_get_meta_page(inode->i_sb,
191− erofs_blknr(map.m_pa));
238+ erofs_blknr(mdev.m_pa));
192239 if (IS_ERR(ipage))
193240 return PTR_ERR(ipage);
194241 iomap->inline_data = page_address(ipage) +
195− erofs_blkoff(map.m_pa);
242+ erofs_blkoff(mdev.m_pa);
196243 iomap->private = ipage;
197244 } else {
198245 iomap->type = IOMAP_MAPPED;
199− iomap->addr = map.m_pa;
246+ iomap->addr = mdev.m_pa;
200247 }
201248 return 0;
202249 }
fs/erofs/decompressor.c+44−95
@@ -16,17 +16,6 @@ · 15 unmodified lines
1616 #define LZ4_DECOMPRESS_INPLACE_MARGIN(srcsize) (((srcsize) >> 8) + 32)
1717 #endif
1818
19−struct z_erofs_decompressor {
20− /*
21− * if destpages have sparsed pages, fill them with bounce pages.
22− * it also check whether destpages indicate continuous physical memory.
23− */
24− int (*prepare_destpages)(struct z_erofs_decompress_req *rq,
25− struct list_head *pagepool);
26− int (*decompress)(struct z_erofs_decompress_req *rq, u8 *out);
27− char *name;
28−};
29−
3019 int z_erofs_load_lz4_config(struct super_block *sb,
3120 struct erofs_super_block *dsb,
3221 struct z_erofs_lz4_cfgs *lz4, int size)
@@ -63,8 +52,12 @@ int z_erofs_load_lz4_config(struct super_block *sb, · 30 unmodified lines
6352 return erofs_pcpubuf_growsize(sbi->lz4.max_pclusterblks);
6453 }
6554
66−static int z_erofs_lz4_prepare_destpages(struct z_erofs_decompress_req *rq,
67− struct list_head *pagepool)
55+/*
56+ * Fill all gaps with bounce pages if it's a sparse page list. Also check if
57+ * all physical pages are consecutive, which can be seen for moderate CR.
58+ */
59+static int z_erofs_lz4_prepare_dstpages(struct z_erofs_decompress_req *rq,
60+ struct page **pagepool)
6861 {
6962 const unsigned int nr =
7063 PAGE_ALIGN(rq->pageofs_out + rq->outputsize) >> PAGE_SHIFT;
@@ -119,7 +112,7 @@ static int z_erofs_lz4_prepare_destpages(struct z_erofs_decompress_req *rq, · 48 unmodified lines
119112 return kaddr ? 1 : 0;
120113 }
121114
122−static void *z_erofs_handle_inplace_io(struct z_erofs_decompress_req *rq,
115+static void *z_erofs_lz4_handle_inplace_io(struct z_erofs_decompress_req *rq,
123116 void *inpage, unsigned int *inputmargin, int *maptype,
124117 bool support_0padding)
125118 {
@@ -189,7 +182,8 @@ docopy: · 63 unmodified lines
189182 return src;
190183 }
191184
192−static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out)
185+static int z_erofs_lz4_decompress_mem(struct z_erofs_decompress_req *rq,
186+ u8 *out)
193187 {
194188 unsigned int inputmargin;
195189 u8 *headpage, *src;
@@ -216,8 +210,8 @@ static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out) · 20 unmodified lines
216210 }
217211
218212 rq->inputsize -= inputmargin;
219− src = z_erofs_handle_inplace_io(rq, headpage, &inputmargin, &maptype,
220− support_0padding);
213+ src = z_erofs_lz4_handle_inplace_io(rq, headpage, &inputmargin,
214+ &maptype, support_0padding);
221215 if (IS_ERR(src))
222216 return PTR_ERR(src);
223217
@@ -233,7 +227,6 @@ static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out) · 9 unmodified lines
233227 erofs_err(rq->sb, "failed to decompress %d in[%u, %u] out[%u]",
234228 ret, rq->inputsize, inputmargin, rq->outputsize);
235229
236− WARN_ON(1);
237230 print_hex_dump(KERN_DEBUG, "[ in]: ", DUMP_PREFIX_OFFSET,
238231 16, 1, src + inputmargin, rq->inputsize, true);
239232 print_hex_dump(KERN_DEBUG, "[out]: ", DUMP_PREFIX_OFFSET,
@@ -242,6 +235,8 @@ static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out) · 2 unmodified lines
242235 if (ret >= 0)
243236 memset(out + ret, 0, rq->outputsize - ret);
244237 ret = -EIO;
238+ } else {
239+ ret = 0;
245240 }
246241
247242 if (maptype == 0) {
@@ -257,86 +252,25 @@ static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out) · 9 unmodified lines
257252 return ret;
258253 }
259254
260−static struct z_erofs_decompressor decompressors[] = {
261− [Z_EROFS_COMPRESSION_SHIFTED] = {
262− .name = "shifted"
263− },
264− [Z_EROFS_COMPRESSION_LZ4] = {
265− .prepare_destpages = z_erofs_lz4_prepare_destpages,
266− .decompress = z_erofs_lz4_decompress,
267− .name = "lz4"
268− },
269−};
270−
271−static void copy_from_pcpubuf(struct page **out, const char *dst,
272− unsigned short pageofs_out,
273− unsigned int outputsize)
274−{
275− const char *end = dst + outputsize;
276− const unsigned int righthalf = PAGE_SIZE - pageofs_out;
277− const char *cur = dst - pageofs_out;
278−
279− while (cur < end) {
280− struct page *const page = *out++;
281−
282− if (page) {
283− char *buf = kmap_atomic(page);
284−
285− if (cur >= dst) {
286− memcpy(buf, cur, min_t(uint, PAGE_SIZE,
287− end - cur));
288− } else {
289− memcpy(buf + pageofs_out, cur + pageofs_out,
290− min_t(uint, righthalf, end - cur));
291− }
292− kunmap_atomic(buf);
293− }
294− cur += PAGE_SIZE;
295− }
296−}
297−
298−static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq,
299− struct list_head *pagepool)
255+static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq,
256+ struct page **pagepool)
300257 {
301258 const unsigned int nrpages_out =
302259 PAGE_ALIGN(rq->pageofs_out + rq->outputsize) >> PAGE_SHIFT;
303− const struct z_erofs_decompressor *alg = decompressors + rq->alg;
304260 unsigned int dst_maptype;
305261 void *dst;
306262 int ret;
307263
308− /* two optimized fast paths only for non bigpcluster cases yet */
309− if (rq->inputsize <= PAGE_SIZE) {
310− if (nrpages_out == 1 && !rq->inplace_io) {
311− DBG_BUGON(!*rq->out);
312− dst = kmap_atomic(*rq->out);
313− dst_maptype = 0;
314− goto dstmap_out;
315− }
316−
317− /*
318− * For the case of small output size (especially much less
319− * than PAGE_SIZE), memcpy the decompressed data rather than
320− * compressed data is preferred.
321− */
322− if (rq->outputsize <= PAGE_SIZE * 7 / 8) {
323− dst = erofs_get_pcpubuf(1);
324− if (IS_ERR(dst))
325− return PTR_ERR(dst);
326−
327− rq->inplace_io = false;
328− ret = alg->decompress(rq, dst);
329− if (!ret)
330− copy_from_pcpubuf(rq->out, dst, rq->pageofs_out,
331− rq->outputsize);
332−
333− erofs_put_pcpubuf(dst);
334− return ret;
335− }
264+ /* one optimized fast path only for non bigpcluster cases yet */
265+ if (rq->inputsize <= PAGE_SIZE && nrpages_out == 1 && !rq->inplace_io) {
266+ DBG_BUGON(!*rq->out);
267+ dst = kmap_atomic(*rq->out);
268+ dst_maptype = 0;
269+ goto dstmap_out;
336270 }
337271
338272 /* general decoding path which can be used for all cases */
339− ret = alg->prepare_destpages(rq, pagepool);
273+ ret = z_erofs_lz4_prepare_dstpages(rq, pagepool);
340274 if (ret < 0)
341275 return ret;
342276 if (ret) {
@@ -351,7 +285,7 @@ static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq, · 8 unmodified lines
351285 dst_maptype = 2;
352286
353287 dstmap_out:
354− ret = alg->decompress(rq, dst + rq->pageofs_out);
288+ ret = z_erofs_lz4_decompress_mem(rq, dst + rq->pageofs_out);
355289
356290 if (!dst_maptype)
357291 kunmap_atomic(dst);
@@ -360,8 +294,8 @@ dstmap_out: · 2 unmodified lines
360294 return ret;
361295 }
362296
363−static int z_erofs_shifted_transform(const struct z_erofs_decompress_req *rq,
364− struct list_head *pagepool)
297+static int z_erofs_shifted_transform(struct z_erofs_decompress_req *rq,
298+ struct page **pagepool)
365299 {
366300 const unsigned int nrpages_out =
367301 PAGE_ALIGN(rq->pageofs_out + rq->outputsize) >> PAGE_SHIFT;
@@ -399,10 +333,25 @@ static int z_erofs_shifted_transform(const struct z_erofs_decompress_req *rq, · 31 unmodified lines
399333 return 0;
400334 }
401335
336+static struct z_erofs_decompressor decompressors[] = {
337+ [Z_EROFS_COMPRESSION_SHIFTED] = {
338+ .decompress = z_erofs_shifted_transform,
339+ .name = "shifted"
340+ },
341+ [Z_EROFS_COMPRESSION_LZ4] = {
342+ .decompress = z_erofs_lz4_decompress,
343+ .name = "lz4"
344+ },
345+#ifdef CONFIG_EROFS_FS_ZIP_LZMA
346+ [Z_EROFS_COMPRESSION_LZMA] = {
347+ .decompress = z_erofs_lzma_decompress,
348+ .name = "lzma"
349+ },
350+#endif
351+};
352+
402353 int z_erofs_decompress(struct z_erofs_decompress_req *rq,
403− struct list_head *pagepool)
354+ struct page **pagepool)
404355 {
405− if (rq->alg == Z_EROFS_COMPRESSION_SHIFTED)
406− return z_erofs_shifted_transform(rq, pagepool);
407− return z_erofs_decompress_generic(rq, pagepool);
356+ return decompressors[rq->alg].decompress(rq, pagepool);
408357 }
fs/erofs/decompressor_lzma.c+290−0
@@ -0,0 +1,290 @@
1+// SPDX-License-Identifier: GPL-2.0-or-later
2+#include <linux/xz.h>
3+#include <linux/module.h>
4+#include "compress.h"
5+
6+struct z_erofs_lzma {
7+ struct z_erofs_lzma *next;
8+ struct xz_dec_microlzma *state;
9+ struct xz_buf buf;
10+ u8 bounce[PAGE_SIZE];
11+};
12+
13+/* considering the LZMA performance, no need to use a lockless list for now */
14+static DEFINE_SPINLOCK(z_erofs_lzma_lock);
15+static unsigned int z_erofs_lzma_max_dictsize;
16+static unsigned int z_erofs_lzma_nstrms, z_erofs_lzma_avail_strms;
17+static struct z_erofs_lzma *z_erofs_lzma_head;
18+static DECLARE_WAIT_QUEUE_HEAD(z_erofs_lzma_wq);
19+
20+module_param_named(lzma_streams, z_erofs_lzma_nstrms, uint, 0444);
21+
22+void z_erofs_lzma_exit(void)
23+{
24+ /* there should be no running fs instance */
25+ while (z_erofs_lzma_avail_strms) {
26+ struct z_erofs_lzma *strm;
27+
28+ spin_lock(&z_erofs_lzma_lock);
29+ strm = z_erofs_lzma_head;
30+ if (!strm) {
31+ spin_unlock(&z_erofs_lzma_lock);
32+ DBG_BUGON(1);
33+ return;
34+ }
35+ z_erofs_lzma_head = NULL;
36+ spin_unlock(&z_erofs_lzma_lock);
37+
38+ while (strm) {
39+ struct z_erofs_lzma *n = strm->next;
40+
41+ if (strm->state)
42+ xz_dec_microlzma_end(strm->state);
43+ kfree(strm);
44+ --z_erofs_lzma_avail_strms;
45+ strm = n;
46+ }
47+ }
48+}
49+
50+int z_erofs_lzma_init(void)
51+{
52+ unsigned int i;
53+
54+ /* by default, use # of possible CPUs instead */
55+ if (!z_erofs_lzma_nstrms)
56+ z_erofs_lzma_nstrms = num_possible_cpus();
57+
58+ for (i = 0; i < z_erofs_lzma_nstrms; ++i) {
59+ struct z_erofs_lzma *strm = kzalloc(sizeof(*strm), GFP_KERNEL);
60+
61+ if (!strm) {
62+ z_erofs_lzma_exit();
63+ return -ENOMEM;
64+ }
65+ spin_lock(&z_erofs_lzma_lock);
66+ strm->next = z_erofs_lzma_head;
67+ z_erofs_lzma_head = strm;
68+ spin_unlock(&z_erofs_lzma_lock);
69+ ++z_erofs_lzma_avail_strms;
70+ }
71+ return 0;
72+}
73+
74+int z_erofs_load_lzma_config(struct super_block *sb,
75+ struct erofs_super_block *dsb,
76+ struct z_erofs_lzma_cfgs *lzma, int size)
77+{
78+ static DEFINE_MUTEX(lzma_resize_mutex);
79+ unsigned int dict_size, i;
80+ struct z_erofs_lzma *strm, *head = NULL;
81+ int err;
82+
83+ if (!lzma || size < sizeof(struct z_erofs_lzma_cfgs)) {
84+ erofs_err(sb, "invalid lzma cfgs, size=%u", size);
85+ return -EINVAL;
86+ }
87+ if (lzma->format) {
88+ erofs_err(sb, "unidentified lzma format %x, please check kernel version",
89+ le16_to_cpu(lzma->format));
90+ return -EINVAL;
91+ }
92+ dict_size = le32_to_cpu(lzma->dict_size);
93+ if (dict_size > Z_EROFS_LZMA_MAX_DICT_SIZE || dict_size < 4096) {
94+ erofs_err(sb, "unsupported lzma dictionary size %u",
95+ dict_size);
96+ return -EINVAL;
97+ }
98+
99+ erofs_info(sb, "EXPERIMENTAL MicroLZMA in use. Use at your own risk!");
100+
101+ /* in case 2 z_erofs_load_lzma_config() race to avoid deadlock */
102+ mutex_lock(&lzma_resize_mutex);
103+
104+ if (z_erofs_lzma_max_dictsize >= dict_size) {
105+ mutex_unlock(&lzma_resize_mutex);
106+ return 0;
107+ }
108+
109+ /* 1. collect/isolate all streams for the following check */
110+ for (i = 0; i < z_erofs_lzma_avail_strms; ++i) {
111+ struct z_erofs_lzma *last;
112+
113+again:
114+ spin_lock(&z_erofs_lzma_lock);
115+ strm = z_erofs_lzma_head;
116+ if (!strm) {
117+ spin_unlock(&z_erofs_lzma_lock);
118+ wait_event(z_erofs_lzma_wq,
119+ READ_ONCE(z_erofs_lzma_head));
120+ goto again;
121+ }
122+ z_erofs_lzma_head = NULL;
123+ spin_unlock(&z_erofs_lzma_lock);
124+
125+ for (last = strm; last->next; last = last->next)
126+ ++i;
127+ last->next = head;
128+ head = strm;
129+ }
130+
131+ err = 0;
132+ /* 2. walk each isolated stream and grow max dict_size if needed */
133+ for (strm = head; strm; strm = strm->next) {
134+ if (strm->state)
135+ xz_dec_microlzma_end(strm->state);
136+ strm->state = xz_dec_microlzma_alloc(XZ_PREALLOC, dict_size);
137+ if (!strm->state)
138+ err = -ENOMEM;
139+ }
140+
141+ /* 3. push back all to the global list and update max dict_size */
142+ spin_lock(&z_erofs_lzma_lock);
143+ DBG_BUGON(z_erofs_lzma_head);
144+ z_erofs_lzma_head = head;
145+ spin_unlock(&z_erofs_lzma_lock);
146+
147+ z_erofs_lzma_max_dictsize = dict_size;
148+ mutex_unlock(&lzma_resize_mutex);
149+ return err;
150+}
151+
152+int z_erofs_lzma_decompress(struct z_erofs_decompress_req *rq,
153+ struct page **pagepool)
154+{
155+ const unsigned int nrpages_out =
156+ PAGE_ALIGN(rq->pageofs_out + rq->outputsize) >> PAGE_SHIFT;
157+ const unsigned int nrpages_in =
158+ PAGE_ALIGN(rq->inputsize) >> PAGE_SHIFT;
159+ unsigned int inputmargin, inlen, outlen, pageofs;
160+ struct z_erofs_lzma *strm;
161+ u8 *kin;
162+ bool bounced = false;
163+ int no, ni, j, err = 0;
164+
165+ /* 1. get the exact LZMA compressed size */
166+ kin = kmap(*rq->in);
167+ inputmargin = 0;
168+ while (!kin[inputmargin & ~PAGE_MASK])
169+ if (!(++inputmargin & ~PAGE_MASK))
170+ break;
171+
172+ if (inputmargin >= PAGE_SIZE) {
173+ kunmap(*rq->in);
174+ return -EFSCORRUPTED;
175+ }
176+ rq->inputsize -= inputmargin;
177+
178+ /* 2. get an available lzma context */
179+again:
180+ spin_lock(&z_erofs_lzma_lock);
181+ strm = z_erofs_lzma_head;
182+ if (!strm) {
183+ spin_unlock(&z_erofs_lzma_lock);
184+ wait_event(z_erofs_lzma_wq, READ_ONCE(z_erofs_lzma_head));
185+ goto again;
186+ }
187+ z_erofs_lzma_head = strm->next;
188+ spin_unlock(&z_erofs_lzma_lock);
189+
190+ /* 3. multi-call decompress */
191+ inlen = rq->inputsize;
192+ outlen = rq->outputsize;
193+ xz_dec_microlzma_reset(strm->state, inlen, outlen,
194+ !rq->partial_decoding);
195+ pageofs = rq->pageofs_out;
196+ strm->buf.in = kin + inputmargin;
197+ strm->buf.in_pos = 0;
198+ strm->buf.in_size = min_t(u32, inlen, PAGE_SIZE - inputmargin);
199+ inlen -= strm->buf.in_size;
200+ strm->buf.out = NULL;
201+ strm->buf.out_pos = 0;
202+ strm->buf.out_size = 0;
203+
204+ for (ni = 0, no = -1;;) {
205+ enum xz_ret xz_err;
206+
207+ if (strm->buf.out_pos == strm->buf.out_size) {
208+ if (strm->buf.out) {
209+ kunmap(rq->out[no]);
210+ strm->buf.out = NULL;
211+ }
212+
213+ if (++no >= nrpages_out || !outlen) {
214+ erofs_err(rq->sb, "decompressed buf out of bound");
215+ err = -EFSCORRUPTED;
216+ break;
217+ }
218+ strm->buf.out_pos = 0;
219+ strm->buf.out_size = min_t(u32, outlen,
220+ PAGE_SIZE - pageofs);
221+ outlen -= strm->buf.out_size;
222+ if (rq->out[no])
223+ strm->buf.out = kmap(rq->out[no]) + pageofs;
224+ pageofs = 0;
225+ } else if (strm->buf.in_pos == strm->buf.in_size) {
226+ kunmap(rq->in[ni]);
227+
228+ if (++ni >= nrpages_in || !inlen) {
229+ erofs_err(rq->sb, "compressed buf out of bound");
230+ err = -EFSCORRUPTED;
231+ break;
232+ }
233+ strm->buf.in_pos = 0;
234+ strm->buf.in_size = min_t(u32, inlen, PAGE_SIZE);
235+ inlen -= strm->buf.in_size;
236+ kin = kmap(rq->in[ni]);
237+ strm->buf.in = kin;
238+ bounced = false;
239+ }
240+
241+ /*
242+ * Handle overlapping: Use bounced buffer if the compressed
243+ * data is under processing; Otherwise, Use short-lived pages
244+ * from the on-stack pagepool where pages share with the same
245+ * request.
246+ */
247+ if (!bounced && rq->out[no] == rq->in[ni]) {
248+ memcpy(strm->bounce, strm->buf.in, strm->buf.in_size);
249+ strm->buf.in = strm->bounce;
250+ bounced = true;
251+ }
252+ for (j = ni + 1; j < nrpages_in; ++j) {
253+ struct page *tmppage;
254+
255+ if (rq->out[no] != rq->in[j])
256+ continue;
257+
258+ DBG_BUGON(erofs_page_is_managed(EROFS_SB(rq->sb),
259+ rq->in[j]));
260+ tmppage = erofs_allocpage(pagepool,
261+ GFP_KERNEL | __GFP_NOFAIL);
262+ set_page_private(tmppage, Z_EROFS_SHORTLIVED_PAGE);
263+ copy_highpage(tmppage, rq->in[j]);
264+ rq->in[j] = tmppage;
265+ }
266+ xz_err = xz_dec_microlzma_run(strm->state, &strm->buf);
267+ DBG_BUGON(strm->buf.out_pos > strm->buf.out_size);
268+ DBG_BUGON(strm->buf.in_pos > strm->buf.in_size);
269+
270+ if (xz_err != XZ_OK) {
271+ if (xz_err == XZ_STREAM_END && !outlen)
272+ break;
273+ erofs_err(rq->sb, "failed to decompress %d in[%u] out[%u]",
274+ xz_err, rq->inputsize, rq->outputsize);
275+ err = -EFSCORRUPTED;
276+ break;
277+ }
278+ }
279+ if (no < nrpages_out && strm->buf.out)
280+ kunmap(rq->in[no]);
281+ if (ni < nrpages_in)
282+ kunmap(rq->in[ni]);
283+ /* 4. push back LZMA stream context to the global list */
284+ spin_lock(&z_erofs_lzma_lock);
285+ strm->next = z_erofs_lzma_head;
286+ z_erofs_lzma_head = strm;
287+ spin_unlock(&z_erofs_lzma_lock);
288+ wake_up(&z_erofs_lzma_wq);
289+ return err;
290+}
fs/erofs/erofs_fs.h+50−23
@@ -21,14 +21,29 @@ · 20 unmodified lines
2121 #define EROFS_FEATURE_INCOMPAT_COMPR_CFGS 0x00000002
2222 #define EROFS_FEATURE_INCOMPAT_BIG_PCLUSTER 0x00000002
2323 #define EROFS_FEATURE_INCOMPAT_CHUNKED_FILE 0x00000004
24+#define EROFS_FEATURE_INCOMPAT_DEVICE_TABLE 0x00000008
25+#define EROFS_FEATURE_INCOMPAT_COMPR_HEAD2 0x00000008
2426 #define EROFS_ALL_FEATURE_INCOMPAT \
2527 (EROFS_FEATURE_INCOMPAT_LZ4_0PADDING | \
2628 EROFS_FEATURE_INCOMPAT_COMPR_CFGS | \
2729 EROFS_FEATURE_INCOMPAT_BIG_PCLUSTER | \
28− EROFS_FEATURE_INCOMPAT_CHUNKED_FILE)
30+ EROFS_FEATURE_INCOMPAT_CHUNKED_FILE | \
31+ EROFS_FEATURE_INCOMPAT_DEVICE_TABLE | \
32+ EROFS_FEATURE_INCOMPAT_COMPR_HEAD2)
2933
3034 #define EROFS_SB_EXTSLOT_SIZE 16
3135
36+struct erofs_deviceslot {
37+ union {
38+ u8 uuid[16]; /* used for device manager later */
39+ u8 userdata[64]; /* digest(sha256), etc. */
40+ } u;
41+ __le32 blocks; /* total fs blocks of this device */
42+ __le32 mapped_blkaddr; /* map starting at mapped_blkaddr */
43+ u8 reserved[56];
44+};
45+#define EROFS_DEVT_SLOT_SIZE sizeof(struct erofs_deviceslot)
46+
3247 /* erofs on-disk super block (currently 128 bytes) */
3348 struct erofs_super_block {
3449 __le32 magic; /* file system magic number */
@@ -54,7 +69,9 @@ struct erofs_super_block { · 19 unmodified lines
5469 /* customized sliding window size instead of 64k by default */
5570 __le16 lz4_max_distance;
5671 } __packed u1;
57− __u8 reserved2[42];
72+ __le16 extra_devices; /* # of devices besides the primary device */
73+ __le16 devt_slotoff; /* startoff = devt_slotoff * devt_slotsize */
74+ __u8 reserved2[38];
5875 };
5976
6077 /*
@@ -238,7 +255,7 @@ static inline unsigned int erofs_xattr_entry_size(struct erofs_xattr_entry *e) · 177 unmodified lines
238255 /* 8-byte inode chunk indexes */
239256 struct erofs_inode_chunk_index {
240257 __le16 advise; /* always 0, don't care for now */
241− __le16 device_id; /* back-end storage id, always 0 for now */
258+ __le16 device_id; /* back-end storage id (with bits masked) */
242259 __le32 blkaddr; /* start block address of this inode chunk */
243260 };
244261
@@ -247,10 +264,11 @@ struct erofs_inode_chunk_index { · 2 unmodified lines
247264
248265 /* available compression algorithm types (for h_algorithmtype) */
249266 enum {
250− Z_EROFS_COMPRESSION_LZ4 = 0,
267+ Z_EROFS_COMPRESSION_LZ4 = 0,
268+ Z_EROFS_COMPRESSION_LZMA = 1,
251269 Z_EROFS_COMPRESSION_MAX
252270 };
253−#define Z_EROFS_ALL_COMPR_ALGS (1 << (Z_EROFS_COMPRESSION_MAX - 1))
271+#define Z_EROFS_ALL_COMPR_ALGS ((1 << Z_EROFS_COMPRESSION_MAX) - 1)
254272
255273 /* 14 bytes (+ length field = 16 bytes) */
256274 struct z_erofs_lz4_cfgs {
@@ -259,6 +277,15 @@ struct z_erofs_lz4_cfgs { · 2 unmodified lines
259277 u8 reserved[10];
260278 } __packed;
261279
280+/* 14 bytes (+ length field = 16 bytes) */
281+struct z_erofs_lzma_cfgs {
282+ __le32 dict_size;
283+ __le16 format;
284+ u8 reserved[8];
285+} __packed;
286+
287+#define Z_EROFS_LZMA_MAX_DICT_SIZE (8 * Z_EROFS_PCLUSTER_MAX_SIZE)
288+
262289 /*
263290 * bit 0 : COMPACTED_2B indexes (0 - off; 1 - on)
264291 * e.g. for 4k logical cluster size, 4B if compacted 2B is off;
@@ -288,35 +315,34 @@ struct z_erofs_map_header { · 23 unmodified lines
288315 #define Z_EROFS_VLE_LEGACY_HEADER_PADDING 8
289316
290317 /*
291− * Fixed-sized output compression ondisk Logical Extent cluster type:
292− * 0 - literal (uncompressed) cluster
293− * 1 - compressed cluster (for the head logical cluster)
294− * 2 - compressed cluster (for the other logical clusters)
318+ * Fixed-sized output compression on-disk logical cluster type:
319+ * 0 - literal (uncompressed) lcluster
320+ * 1,3 - compressed lcluster (for HEAD lclusters)
321+ * 2 - compressed lcluster (for NONHEAD lclusters)
295322 *
296323 * In detail,
297− * 0 - literal (uncompressed) cluster,
324+ * 0 - literal (uncompressed) lcluster,
298325 * di_advise = 0
299− * di_clusterofs = the literal data offset of the cluster
300− * di_blkaddr = the blkaddr of the literal cluster
326+ * di_clusterofs = the literal data offset of the lcluster
327+ * di_blkaddr = the blkaddr of the literal pcluster
301328 *
302− * 1 - compressed cluster (for the head logical cluster)
303− * di_advise = 1
304− * di_clusterofs = the decompressed data offset of the cluster
305− * di_blkaddr = the blkaddr of the compressed cluster
329+ * 1,3 - compressed lcluster (for HEAD lclusters)
330+ * di_advise = 1 or 3
331+ * di_clusterofs = the decompressed data offset of the lcluster
332+ * di_blkaddr = the blkaddr of the compressed pcluster
306333 *
307− * 2 - compressed cluster (for the other logical clusters)
334+ * 2 - compressed lcluster (for NONHEAD lclusters)
308335 * di_advise = 2
309336 * di_clusterofs =
310− * the decompressed data offset in its own head cluster
311− * di_u.delta[0] = distance to its corresponding head cluster
312− * di_u.delta[1] = distance to its corresponding tail cluster
313− * (di_advise could be 0, 1 or 2)
337+ * the decompressed data offset in its own HEAD lcluster
338+ * di_u.delta[0] = distance to this HEAD lcluster
339+ * di_u.delta[1] = distance to the next HEAD lcluster
314340 */
315341 enum {
316342 Z_EROFS_VLE_CLUSTER_TYPE_PLAIN = 0,
317− Z_EROFS_VLE_CLUSTER_TYPE_HEAD = 1,
343+ Z_EROFS_VLE_CLUSTER_TYPE_HEAD1 = 1,
318344 Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD = 2,
319− Z_EROFS_VLE_CLUSTER_TYPE_RESERVED = 3,
345+ Z_EROFS_VLE_CLUSTER_TYPE_HEAD2 = 3,
320346 Z_EROFS_VLE_CLUSTER_TYPE_MAX
321347 };
322348
@@ -384,6 +410,7 @@ static inline void erofs_check_ondisk_layout_definitions(void) · 61 unmodified lines
384410 /* keep in sync between 2 index structures for better extendibility */
385411 BUILD_BUG_ON(sizeof(struct erofs_inode_chunk_index) !=
386412 sizeof(struct z_erofs_vle_decompressed_index));
413+ BUILD_BUG_ON(sizeof(struct erofs_deviceslot) != 128);
387414
388415 BUILD_BUG_ON(BIT(Z_EROFS_VLE_DI_CLUSTER_TYPE_BITS) <
389416 Z_EROFS_VLE_CLUSTER_TYPE_MAX - 1);
fs/erofs/inode.c+1−1
@@ -192,7 +192,7 @@ static struct page *erofs_read_inode(struct inode *inode, · 191 unmodified lines
192192 inode->i_atime.tv_nsec = inode->i_ctime.tv_nsec;
193193
194194 inode->i_flags &= ~S_DAX;
195− if (test_opt(&sbi->ctx, DAX_ALWAYS) && S_ISREG(inode->i_mode) &&
195+ if (test_opt(&sbi->opt, DAX_ALWAYS) && S_ISREG(inode->i_mode) &&
196196 vi->datalayout == EROFS_INODE_FLAT_PLAIN)
197197 inode->i_flags |= S_DAX;
198198 if (!nblks)
fs/erofs/internal.h+94−11
@@ -47,7 +47,16 @@ typedef u64 erofs_off_t; · 46 unmodified lines
4747 /* data type for filesystem-wide blocks number */
4848 typedef u32 erofs_blk_t;
4949
50−struct erofs_fs_context {
50+struct erofs_device_info {
51+ char *path;
52+ struct block_device *bdev;
53+ struct dax_device *dax_dev;
54+
55+ u32 blocks;
56+ u32 mapped_blkaddr;
57+};
58+
59+struct erofs_mount_opts {
5160 #ifdef CONFIG_EROFS_FS_ZIP
5261 /* current strategy of how to use managed cache */
5362 unsigned char cache_strategy;
@@ -60,6 +69,18 @@ struct erofs_fs_context { · 6 unmodified lines
6069 unsigned int mount_opt;
6170 };
6271
72+struct erofs_dev_context {
73+ struct idr tree;
74+ struct rw_semaphore rwsem;
75+
76+ unsigned int extra_devices;
77+};
78+
79+struct erofs_fs_context {
80+ struct erofs_mount_opts opt;
81+ struct erofs_dev_context *devs;
82+};
83+
6384 /* all filesystem-wide lz4 configurations */
6485 struct erofs_sb_lz4_info {
6586 /* # of pages needed for EROFS lz4 rolling decompression */
@@ -69,6 +90,7 @@ struct erofs_sb_lz4_info { · 3 unmodified lines
6990 };
7091
7192 struct erofs_sb_info {
93+ struct erofs_mount_opts opt; /* options */
7294 #ifdef CONFIG_EROFS_FS_ZIP
7395 /* list for all registered superblocks, mainly for shrinker */
7496 struct list_head list;
@@ -85,12 +107,16 @@ struct erofs_sb_info { · 10 unmodified lines
85107
86108 struct erofs_sb_lz4_info lz4;
87109 #endif /* CONFIG_EROFS_FS_ZIP */
110+ struct erofs_dev_context *devs;
88111 struct dax_device *dax_dev;
89− u32 blocks;
112+ u64 total_blocks;
113+ u32 primarydevice_blocks;
114+
90115 u32 meta_blkaddr;
91116 #ifdef CONFIG_EROFS_FS_XATTR
92117 u32 xattr_blkaddr;
93118 #endif
119+ u16 device_id_mask; /* valid bits of device id to be used */
94120
95121 /* inode slot unit size in bit shift */
96122 unsigned char islotbits;
@@ -108,8 +134,6 @@ struct erofs_sb_info { · 11 unmodified lines
108134 u8 volume_name[16]; /* volume name */
109135 u32 feature_compat;
110136 u32 feature_incompat;
111−
112− struct erofs_fs_context ctx; /* options */
113137 };
114138
115139 #define EROFS_SB(sb) ((struct erofs_sb_info *)(sb)->s_fs_info)
@@ -121,9 +145,9 @@ struct erofs_sb_info { · 5 unmodified lines
121145 #define EROFS_MOUNT_DAX_ALWAYS 0x00000040
122146 #define EROFS_MOUNT_DAX_NEVER 0x00000080
123147
124−#define clear_opt(ctx, option) ((ctx)->mount_opt &= ~EROFS_MOUNT_##option)
125−#define set_opt(ctx, option) ((ctx)->mount_opt |= EROFS_MOUNT_##option)
126−#define test_opt(ctx, option) ((ctx)->mount_opt & EROFS_MOUNT_##option)
148+#define clear_opt(opt, option) ((opt)->mount_opt &= ~EROFS_MOUNT_##option)
149+#define set_opt(opt, option) ((opt)->mount_opt |= EROFS_MOUNT_##option)
150+#define test_opt(opt, option) ((opt)->mount_opt & EROFS_MOUNT_##option)
127151
128152 enum {
129153 EROFS_ZIP_CACHE_DISABLED,
@@ -237,6 +261,7 @@ static inline bool erofs_sb_has_##name(struct erofs_sb_info *sbi) \ · 107 unmodified lines
237261 EROFS_FEATURE_FUNCS(lz4_0padding, incompat, INCOMPAT_LZ4_0PADDING)
238262 EROFS_FEATURE_FUNCS(compr_cfgs, incompat, INCOMPAT_COMPR_CFGS)
239263 EROFS_FEATURE_FUNCS(big_pcluster, incompat, INCOMPAT_BIG_PCLUSTER)
264+EROFS_FEATURE_FUNCS(device_table, incompat, INCOMPAT_DEVICE_TABLE)
240265 EROFS_FEATURE_FUNCS(sb_chksum, compat, COMPAT_SB_CHKSUM)
241266
242267 /* atomic flag definitions */
@@ -307,6 +332,19 @@ static inline unsigned int erofs_inode_datalayout(unsigned int value) · 64 unmodified lines
307332 EROFS_I_DATALAYOUT_BITS);
308333 }
309334
335+/*
336+ * Different from grab_cache_page_nowait(), reclaiming is never triggered
337+ * when allocating new pages.
338+ */
339+static inline
340+struct page *erofs_grab_cache_page_nowait(struct address_space *mapping,
341+ pgoff_t index)
342+{
343+ return pagecache_get_page(mapping, index,
344+ FGP_LOCK|FGP_CREAT|FGP_NOFS|FGP_NOWAIT,
345+ readahead_gfp_mask(mapping) & ~__GFP_RECLAIM);
346+}
347+
310348 extern const struct super_operations erofs_sops;
311349
312350 extern const struct address_space_operations erofs_raw_access_aops;
@@ -338,7 +376,7 @@ extern const struct address_space_operations z_erofs_aops; · 25 unmodified lines
338376 * of the corresponding uncompressed data in the file.
339377 */
340378 enum {
341− BH_Zipped = BH_PrivateStart,
379+ BH_Encoded = BH_PrivateStart,
342380 BH_FullMapped,
343381 };
344382
@@ -346,8 +384,8 @@ enum { · 1 unmodified line
346384 #define EROFS_MAP_MAPPED (1 << BH_Mapped)
347385 /* Located in metadata (could be copied from bd_inode) */
348386 #define EROFS_MAP_META (1 << BH_Meta)
349−/* The extent has been compressed */
350−#define EROFS_MAP_ZIPPED (1 << BH_Zipped)
387+/* The extent is encoded */
388+#define EROFS_MAP_ENCODED (1 << BH_Encoded)
351389 /* The length of extent is full */
352390 #define EROFS_MAP_FULL_MAPPED (1 << BH_FullMapped)
353391
@@ -355,6 +393,8 @@ struct erofs_map_blocks { · 1 unmodified line
355393 erofs_off_t m_pa, m_la;
356394 u64 m_plen, m_llen;
357395
396+ unsigned short m_deviceid;
397+ char m_algorithmformat;
358398 unsigned int m_flags;
359399
360400 struct page *mpage;
@@ -367,6 +407,13 @@ struct erofs_map_blocks { · 6 unmodified lines
367407 * approach instead if possible since it's more metadata lightweight.)
368408 */
369409 #define EROFS_GET_BLOCKS_FIEMAP 0x0002
410+/* Used to map the whole extent if non-negligible data is requested for LZMA */
411+#define EROFS_GET_BLOCKS_READMORE 0x0004
412+
413+enum {
414+ Z_EROFS_COMPRESSION_SHIFTED = Z_EROFS_COMPRESSION_MAX,
415+ Z_EROFS_COMPRESSION_RUNTIME_MAX
416+};
370417
371418 /* zmap.c */
372419 extern const struct iomap_ops z_erofs_iomap_report_ops;
@@ -386,9 +433,18 @@ static inline int z_erofs_map_blocks_iter(struct inode *inode, · 13 unmodified lines
386433 }
387434 #endif /* !CONFIG_EROFS_FS_ZIP */
388435
436+struct erofs_map_dev {
437+ struct block_device *m_bdev;
438+ struct dax_device *m_daxdev;
439+
440+ erofs_off_t m_pa;
441+ unsigned int m_deviceid;
442+};
443+
389444 /* data.c */
390445 extern const struct file_operations erofs_file_fops;
391446 struct page *erofs_get_meta_page(struct super_block *sb, erofs_blk_t blkaddr);
447+int erofs_map_dev(struct super_block *sb, struct erofs_map_dev *dev);
392448 int erofs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
393449 u64 start, u64 len);
394450
@@ -443,7 +499,14 @@ void erofs_pcpubuf_init(void); · 48 unmodified lines
443499 void erofs_pcpubuf_exit(void);
444500
445501 /* utils.c / zdata.c */
446−struct page *erofs_allocpage(struct list_head *pool, gfp_t gfp);
502+struct page *erofs_allocpage(struct page **pagepool, gfp_t gfp);
503+static inline void erofs_pagepool_add(struct page **pagepool,
504+ struct page *page)
505+{
506+ set_page_private(page, (unsigned long)*pagepool);
507+ *pagepool = page;
508+}
509+void erofs_release_pages(struct page **pagepool);
447510
448511 #ifdef CONFIG_EROFS_FS_ZIP
449512 int erofs_workgroup_put(struct erofs_workgroup *grp);
@@ -483,6 +546,26 @@ static inline int z_erofs_load_lz4_config(struct super_block *sb, · 33 unmodified lines
483546 }
484547 #endif /* !CONFIG_EROFS_FS_ZIP */
485548
549+#ifdef CONFIG_EROFS_FS_ZIP_LZMA
550+int z_erofs_lzma_init(void);
551+void z_erofs_lzma_exit(void);
552+int z_erofs_load_lzma_config(struct super_block *sb,
553+ struct erofs_super_block *dsb,
554+ struct z_erofs_lzma_cfgs *lzma, int size);
555+#else
556+static inline int z_erofs_lzma_init(void) { return 0; }
557+static inline int z_erofs_lzma_exit(void) { return 0; }
558+static inline int z_erofs_load_lzma_config(struct super_block *sb,
559+ struct erofs_super_block *dsb,
560+ struct z_erofs_lzma_cfgs *lzma, int size) {
561+ if (lzma) {
562+ erofs_err(sb, "lzma algorithm isn't enabled");
563+ return -EINVAL;
564+ }
565+ return 0;
566+}
567+#endif /* !CONFIG_EROFS_FS_ZIP */
568+
486569 #define EFSCORRUPTED EUCLEAN /* Filesystem is corrupted */
487570
488571 #endif /* __EROFS_INTERNAL_H */
fs/erofs/pcpubuf.c+3−3
@@ -49,7 +49,7 @@ int erofs_pcpubuf_growsize(unsigned int nrpages) · 48 unmodified lines
4949 {
5050 static DEFINE_MUTEX(pcb_resize_mutex);
5151 static unsigned int pcb_nrpages;
52− LIST_HEAD(pagepool);
52+ struct page *pagepool = NULL;
5353 int delta, cpu, ret, i;
5454
5555 mutex_lock(&pcb_resize_mutex);
@@ -102,13 +102,13 @@ int erofs_pcpubuf_growsize(unsigned int nrpages) · 46 unmodified lines
102102 vunmap(old_ptr);
103103 free_pagearray:
104104 while (i)
105− list_add(&oldpages[--i]->lru, &pagepool);
105+ erofs_pagepool_add(&pagepool, oldpages[--i]);
106106 kfree(oldpages);
107107 if (ret)
108108 break;
109109 }
110110 pcb_nrpages = nrpages;
111− put_pages_list(&pagepool);
111+ erofs_release_pages(&pagepool);
112112 out:
113113 mutex_unlock(&pcb_resize_mutex);
114114 return ret;
fs/erofs/super.c+188−43
@@ -225,6 +225,9 @@ static int erofs_load_compr_cfgs(struct super_block *sb, · 224 unmodified lines
225225 case Z_EROFS_COMPRESSION_LZ4:
226226 ret = z_erofs_load_lz4_config(sb, dsb, data, size);
227227 break;
228+ case Z_EROFS_COMPRESSION_LZMA:
229+ ret = z_erofs_load_lzma_config(sb, dsb, data, size);
230+ break;
228231 default:
229232 DBG_BUGON(1);
230233 ret = -EFAULT;
@@ -252,6 +255,79 @@ static int erofs_load_compr_cfgs(struct super_block *sb, · 21 unmodified lines
252255 }
253256 #endif
254257
258+static int erofs_init_devices(struct super_block *sb,
259+ struct erofs_super_block *dsb)
260+{
261+ struct erofs_sb_info *sbi = EROFS_SB(sb);
262+ unsigned int ondisk_extradevs;
263+ erofs_off_t pos;
264+ struct page *page = NULL;
265+ struct erofs_device_info *dif;
266+ struct erofs_deviceslot *dis;
267+ void *ptr;
268+ int id, err = 0;
269+
270+ sbi->total_blocks = sbi->primarydevice_blocks;
271+ if (!erofs_sb_has_device_table(sbi))
272+ ondisk_extradevs = 0;
273+ else
274+ ondisk_extradevs = le16_to_cpu(dsb->extra_devices);
275+
276+ if (ondisk_extradevs != sbi->devs->extra_devices) {
277+ erofs_err(sb, "extra devices don't match (ondisk %u, given %u)",
278+ ondisk_extradevs, sbi->devs->extra_devices);
279+ return -EINVAL;
280+ }
281+ if (!ondisk_extradevs)
282+ return 0;
283+
284+ sbi->device_id_mask = roundup_pow_of_two(ondisk_extradevs + 1) - 1;
285+ pos = le16_to_cpu(dsb->devt_slotoff) * EROFS_DEVT_SLOT_SIZE;
286+ down_read(&sbi->devs->rwsem);
287+ idr_for_each_entry(&sbi->devs->tree, dif, id) {
288+ erofs_blk_t blk = erofs_blknr(pos);
289+ struct block_device *bdev;
290+
291+ if (!page || page->index != blk) {
292+ if (page) {
293+ kunmap(page);
294+ unlock_page(page);
295+ put_page(page);
296+ }
297+
298+ page = erofs_get_meta_page(sb, blk);
299+ if (IS_ERR(page)) {
300+ up_read(&sbi->devs->rwsem);
301+ return PTR_ERR(page);
302+ }
303+ ptr = kmap(page);
304+ }
305+ dis = ptr + erofs_blkoff(pos);
306+
307+ bdev = blkdev_get_by_path(dif->path,
308+ FMODE_READ | FMODE_EXCL,
309+ sb->s_type);
310+ if (IS_ERR(bdev)) {
311+ err = PTR_ERR(bdev);
312+ goto err_out;
313+ }
314+ dif->bdev = bdev;
315+ dif->dax_dev = fs_dax_get_by_bdev(bdev);
316+ dif->blocks = le32_to_cpu(dis->blocks);
317+ dif->mapped_blkaddr = le32_to_cpu(dis->mapped_blkaddr);
318+ sbi->total_blocks += dif->blocks;
319+ pos += EROFS_DEVT_SLOT_SIZE;
320+ }
321+err_out:
322+ up_read(&sbi->devs->rwsem);
323+ if (page) {
324+ kunmap(page);
325+ unlock_page(page);
326+ put_page(page);
327+ }
328+ return err;
329+}
330+
255331 static int erofs_read_superblock(struct super_block *sb)
256332 {
257333 struct erofs_sb_info *sbi;
@@ -303,7 +379,7 @@ static int erofs_read_superblock(struct super_block *sb) · 45 unmodified lines
303379 sbi->sb_size);
304380 goto out;
305381 }
306− sbi->blocks = le32_to_cpu(dsb->blocks);
382+ sbi->primarydevice_blocks = le32_to_cpu(dsb->blocks);
307383 sbi->meta_blkaddr = le32_to_cpu(dsb->meta_blkaddr);
308384 #ifdef CONFIG_EROFS_FS_XATTR
309385 sbi->xattr_blkaddr = le32_to_cpu(dsb->xattr_blkaddr);
@@ -330,6 +406,11 @@ static int erofs_read_superblock(struct super_block *sb) · 20 unmodified lines
330406 ret = erofs_load_compr_cfgs(sb, dsb);
331407 else
332408 ret = z_erofs_load_lz4_config(sb, dsb, NULL, 0);
409+ if (ret < 0)
410+ goto out;
411+
412+ /* handle multiple devices */
413+ ret = erofs_init_devices(sb, dsb);
333414 out:
334415 kunmap(page);
335416 put_page(page);
@@ -340,15 +421,15 @@ out: · 4 unmodified lines
340421 static void erofs_default_options(struct erofs_fs_context *ctx)
341422 {
342423 #ifdef CONFIG_EROFS_FS_ZIP
343− ctx->cache_strategy = EROFS_ZIP_CACHE_READAROUND;
344− ctx->max_sync_decompress_pages = 3;
345− ctx->readahead_sync_decompress = false;
424+ ctx->opt.cache_strategy = EROFS_ZIP_CACHE_READAROUND;
425+ ctx->opt.max_sync_decompress_pages = 3;
426+ ctx->opt.readahead_sync_decompress = false;
346427 #endif
347428 #ifdef CONFIG_EROFS_FS_XATTR
348− set_opt(ctx, XATTR_USER);
429+ set_opt(&ctx->opt, XATTR_USER);
349430 #endif
350431 #ifdef CONFIG_EROFS_FS_POSIX_ACL
351− set_opt(ctx, POSIX_ACL);
432+ set_opt(&ctx->opt, POSIX_ACL);
352433 #endif
353434 }
354435
@@ -358,6 +439,7 @@ enum { · 3 unmodified lines
358439 Opt_cache_strategy,
359440 Opt_dax,
360441 Opt_dax_enum,
442+ Opt_device,
361443 Opt_err
362444 };
363445
@@ -381,6 +463,7 @@ static const struct fs_parameter_spec erofs_fs_parameters[] = { · 17 unmodified lines
381463 erofs_param_cache_strategy),
382464 fsparam_flag("dax", Opt_dax),
383465 fsparam_enum("dax", Opt_dax_enum, erofs_dax_param_enums),
466+ fsparam_string("device", Opt_device),
384467 {}
385468 };
386469
@@ -392,12 +475,12 @@ static bool erofs_fc_set_dax_mode(struct fs_context *fc, unsigned int mode) · 5 unmodified lines
392475 switch (mode) {
393476 case EROFS_MOUNT_DAX_ALWAYS:
394477 warnfc(fc, "DAX enabled. Warning: EXPERIMENTAL, use at your own risk");
395− set_opt(ctx, DAX_ALWAYS);
396− clear_opt(ctx, DAX_NEVER);
478+ set_opt(&ctx->opt, DAX_ALWAYS);
479+ clear_opt(&ctx->opt, DAX_NEVER);
397480 return true;
398481 case EROFS_MOUNT_DAX_NEVER:
399− set_opt(ctx, DAX_NEVER);
400− clear_opt(ctx, DAX_ALWAYS);
482+ set_opt(&ctx->opt, DAX_NEVER);
483+ clear_opt(&ctx->opt, DAX_ALWAYS);
401484 return true;
402485 default:
403486 DBG_BUGON(1);
@@ -412,9 +495,10 @@ static bool erofs_fc_set_dax_mode(struct fs_context *fc, unsigned int mode) · 8 unmodified lines
412495 static int erofs_fc_parse_param(struct fs_context *fc,
413496 struct fs_parameter *param)
414497 {
415− struct erofs_fs_context *ctx __maybe_unused = fc->fs_private;
498+ struct erofs_fs_context *ctx = fc->fs_private;
416499 struct fs_parse_result result;
417− int opt;
500+ struct erofs_device_info *dif;
501+ int opt, ret;
418502
419503 opt = fs_parse(fc, erofs_fs_parameters, param, &result);
420504 if (opt < 0)
@@ -424,9 +508,9 @@ static int erofs_fc_parse_param(struct fs_context *fc, · 3 unmodified lines
424508 case Opt_user_xattr:
425509 #ifdef CONFIG_EROFS_FS_XATTR
426510 if (result.boolean)
427− set_opt(ctx, XATTR_USER);
511+ set_opt(&ctx->opt, XATTR_USER);
428512 else
429− clear_opt(ctx, XATTR_USER);
513+ clear_opt(&ctx->opt, XATTR_USER);
430514 #else
431515 errorfc(fc, "{,no}user_xattr options not supported");
432516 #endif
@@ -434,16 +518,16 @@ static int erofs_fc_parse_param(struct fs_context *fc, · 1 unmodified line
434518 case Opt_acl:
435519 #ifdef CONFIG_EROFS_FS_POSIX_ACL
436520 if (result.boolean)
437− set_opt(ctx, POSIX_ACL);
521+ set_opt(&ctx->opt, POSIX_ACL);
438522 else
439− clear_opt(ctx, POSIX_ACL);
523+ clear_opt(&ctx->opt, POSIX_ACL);
440524 #else
441525 errorfc(fc, "{,no}acl options not supported");
442526 #endif
443527 break;
444528 case Opt_cache_strategy:
445529 #ifdef CONFIG_EROFS_FS_ZIP
446− ctx->cache_strategy = result.uint_32;
530+ ctx->opt.cache_strategy = result.uint_32;
447531 #else
448532 errorfc(fc, "compression not supported, cache_strategy ignored");
449533 #endif
@@ -456,6 +540,25 @@ static int erofs_fc_parse_param(struct fs_context *fc, · 6 unmodified lines
456540 if (!erofs_fc_set_dax_mode(fc, result.uint_32))
457541 return -EINVAL;
458542 break;
543+ case Opt_device:
544+ dif = kzalloc(sizeof(*dif), GFP_KERNEL);
545+ if (!dif)
546+ return -ENOMEM;
547+ dif->path = kstrdup(param->string, GFP_KERNEL);
548+ if (!dif->path) {
549+ kfree(dif);
550+ return -ENOMEM;
551+ }
552+ down_write(&ctx->devs->rwsem);
553+ ret = idr_alloc(&ctx->devs->tree, dif, 0, 0, GFP_KERNEL);
554+ up_write(&ctx->devs->rwsem);
555+ if (ret < 0) {
556+ kfree(dif->path);
557+ kfree(dif);
558+ return ret;
559+ }
560+ ++ctx->devs->extra_devices;
561+ break;
459562 default:
460563 return -ENOPARAM;
461564 }
@@ -540,15 +643,19 @@ static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc) · 78 unmodified lines
540643 return -ENOMEM;
541644
542645 sb->s_fs_info = sbi;
646+ sbi->opt = ctx->opt;
543647 sbi->dax_dev = fs_dax_get_by_bdev(sb->s_bdev);
648+ sbi->devs = ctx->devs;
649+ ctx->devs = NULL;
650+
544651 err = erofs_read_superblock(sb);
545652 if (err)
546653 return err;
547654
548− if (test_opt(ctx, DAX_ALWAYS) &&
655+ if (test_opt(&sbi->opt, DAX_ALWAYS) &&
549656 !dax_supported(sbi->dax_dev, sb->s_bdev, EROFS_BLKSIZ, 0, bdev_nr_sectors(sb->s_bdev))) {
550657 errorfc(fc, "DAX unsupported by block device. Turning off DAX.");
551− clear_opt(ctx, DAX_ALWAYS);
658+ clear_opt(&sbi->opt, DAX_ALWAYS);
552659 }
553660 sb->s_flags |= SB_RDONLY | SB_NOATIME;
554661 sb->s_maxbytes = MAX_LFS_FILESIZE;
@@ -557,13 +664,11 @@ static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc) · 2 unmodified lines
557664 sb->s_op = &erofs_sops;
558665 sb->s_xattr = erofs_xattr_handlers;
559666
560− if (test_opt(ctx, POSIX_ACL))
667+ if (test_opt(&sbi->opt, POSIX_ACL))
561668 sb->s_flags |= SB_POSIXACL;
562669 else
563670 sb->s_flags &= ~SB_POSIXACL;
564671
565− sbi->ctx = *ctx;
566−
567672 #ifdef CONFIG_EROFS_FS_ZIP
568673 xa_init(&sbi->managed_pslots);
569674 #endif
@@ -607,20 +712,44 @@ static int erofs_fc_reconfigure(struct fs_context *fc) · 37 unmodified lines
607712
608713 DBG_BUGON(!sb_rdonly(sb));
609714
610− if (test_opt(ctx, POSIX_ACL))
715+ if (test_opt(&ctx->opt, POSIX_ACL))
611716 fc->sb_flags |= SB_POSIXACL;
612717 else
613718 fc->sb_flags &= ~SB_POSIXACL;
614719
615− sbi->ctx = *ctx;
720+ sbi->opt = ctx->opt;
616721
617722 fc->sb_flags |= SB_RDONLY;
618723 return 0;
619724 }
620725
726+static int erofs_release_device_info(int id, void *ptr, void *data)
727+{
728+ struct erofs_device_info *dif = ptr;
729+
730+ fs_put_dax(dif->dax_dev);
731+ if (dif->bdev)
732+ blkdev_put(dif->bdev, FMODE_READ | FMODE_EXCL);
733+ kfree(dif->path);
734+ kfree(dif);
735+ return 0;
736+}
737+
738+static void erofs_free_dev_context(struct erofs_dev_context *devs)
739+{
740+ if (!devs)
741+ return;
742+ idr_for_each(&devs->tree, &erofs_release_device_info, NULL);
743+ idr_destroy(&devs->tree);
744+ kfree(devs);
745+}
746+
621747 static void erofs_fc_free(struct fs_context *fc)
622748 {
623− kfree(fc->fs_private);
749+ struct erofs_fs_context *ctx = fc->fs_private;
750+
751+ erofs_free_dev_context(ctx->devs);
752+ kfree(ctx);
624753 }
625754
626755 static const struct fs_context_operations erofs_context_ops = {
@@ -632,15 +761,21 @@ static const struct fs_context_operations erofs_context_ops = { · 5 unmodified lines
632761
633762 static int erofs_init_fs_context(struct fs_context *fc)
634763 {
635− fc->fs_private = kzalloc(sizeof(struct erofs_fs_context), GFP_KERNEL);
636− if (!fc->fs_private)
637− return -ENOMEM;
764+ struct erofs_fs_context *ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
638765
639− /* set default mount options */
640− erofs_default_options(fc->fs_private);
766+ if (!ctx)
767+ return -ENOMEM;
768+ ctx->devs = kzalloc(sizeof(struct erofs_dev_context), GFP_KERNEL);
769+ if (!ctx->devs) {
770+ kfree(ctx);
771+ return -ENOMEM;
772+ }
773+ fc->fs_private = ctx;
641774
775+ idr_init(&ctx->devs->tree);
776+ init_rwsem(&ctx->devs->rwsem);
777+ erofs_default_options(ctx);
642778 fc->ops = &erofs_context_ops;
643−
644779 return 0;
645780 }
646781
@@ -659,6 +794,8 @@ static void erofs_kill_sb(struct super_block *sb) · 12 unmodified lines
659794 sbi = EROFS_SB(sb);
660795 if (!sbi)
661796 return;
797+
798+ erofs_free_dev_context(sbi->devs);
662799 fs_put_dax(sbi->dax_dev);
663800 kfree(sbi);
664801 sb->s_fs_info = NULL;
@@ -706,6 +843,10 @@ static int __init erofs_module_init(void) · 41 unmodified lines
706843 if (err)
707844 goto shrinker_err;
708845
846+ err = z_erofs_lzma_init();
847+ if (err)
848+ goto lzma_err;
849+
709850 erofs_pcpubuf_init();
710851 err = z_erofs_init_zip_subsystem();
711852 if (err)
@@ -720,6 +861,8 @@ static int __init erofs_module_init(void) · 8 unmodified lines
720861 fs_err:
721862 z_erofs_exit_zip_subsystem();
722863 zip_err:
864+ z_erofs_lzma_exit();
865+lzma_err:
723866 erofs_exit_shrinker();
724867 shrinker_err:
725868 kmem_cache_destroy(erofs_inode_cachep);
@@ -730,11 +873,13 @@ icache_err: · 4 unmodified lines
730873 static void __exit erofs_module_exit(void)
731874 {
732875 unregister_filesystem(&erofs_fs_type);
733− z_erofs_exit_zip_subsystem();
734− erofs_exit_shrinker();
735876
736− /* Ensure all RCU free inodes are safe before cache is destroyed. */
877+ /* Ensure all RCU free inodes / pclusters are safe to be destroyed. */
737878 rcu_barrier();
879+
880+ z_erofs_exit_zip_subsystem();
881+ z_erofs_lzma_exit();
882+ erofs_exit_shrinker();
738883 kmem_cache_destroy(erofs_inode_cachep);
739884 erofs_pcpubuf_exit();
740885 }
@@ -748,7 +893,7 @@ static int erofs_statfs(struct dentry *dentry, struct kstatfs *buf) · 7 unmodified lines
748893
749894 buf->f_type = sb->s_magic;
750895 buf->f_bsize = EROFS_BLKSIZ;
751− buf->f_blocks = sbi->blocks;
896+ buf->f_blocks = sbi->total_blocks;
752897 buf->f_bfree = buf->f_bavail = 0;
753898
754899 buf->f_files = ULLONG_MAX;
@@ -763,31 +908,31 @@ static int erofs_statfs(struct dentry *dentry, struct kstatfs *buf) · 8 unmodified lines
763908 static int erofs_show_options(struct seq_file *seq, struct dentry *root)
764909 {
765910 struct erofs_sb_info *sbi = EROFS_SB(root->d_sb);
766− struct erofs_fs_context *ctx = &sbi->ctx;
911+ struct erofs_mount_opts *opt = &sbi->opt;
767912
768913 #ifdef CONFIG_EROFS_FS_XATTR
769− if (test_opt(ctx, XATTR_USER))
914+ if (test_opt(opt, XATTR_USER))
770915 seq_puts(seq, ",user_xattr");
771916 else
772917 seq_puts(seq, ",nouser_xattr");
773918 #endif
774919 #ifdef CONFIG_EROFS_FS_POSIX_ACL
775− if (test_opt(ctx, POSIX_ACL))
920+ if (test_opt(opt, POSIX_ACL))
776921 seq_puts(seq, ",acl");
777922 else
778923 seq_puts(seq, ",noacl");
779924 #endif
780925 #ifdef CONFIG_EROFS_FS_ZIP
781− if (ctx->cache_strategy == EROFS_ZIP_CACHE_DISABLED)
926+ if (opt->cache_strategy == EROFS_ZIP_CACHE_DISABLED)
782927 seq_puts(seq, ",cache_strategy=disabled");
783− else if (ctx->cache_strategy == EROFS_ZIP_CACHE_READAHEAD)
928+ else if (opt->cache_strategy == EROFS_ZIP_CACHE_READAHEAD)
784929 seq_puts(seq, ",cache_strategy=readahead");
785− else if (ctx->cache_strategy == EROFS_ZIP_CACHE_READAROUND)
930+ else if (opt->cache_strategy == EROFS_ZIP_CACHE_READAROUND)
786931 seq_puts(seq, ",cache_strategy=readaround");
787932 #endif
788− if (test_opt(ctx, DAX_ALWAYS))
933+ if (test_opt(opt, DAX_ALWAYS))
789934 seq_puts(seq, ",dax=always");
790− if (test_opt(ctx, DAX_NEVER))
935+ if (test_opt(opt, DAX_NEVER))
791936 seq_puts(seq, ",dax=never");
792937 return 0;
793938 }
fs/erofs/utils.c+14−5
@@ -6,20 +6,29 @@ · 5 unmodified lines
66 #include "internal.h"
77 #include <linux/pagevec.h>
88
9−struct page *erofs_allocpage(struct list_head *pool, gfp_t gfp)
9+struct page *erofs_allocpage(struct page **pagepool, gfp_t gfp)
1010 {
11− struct page *page;
11+ struct page *page = *pagepool;
1212
13− if (!list_empty(pool)) {
14− page = lru_to_page(pool);
13+ if (page) {
1514 DBG_BUGON(page_ref_count(page) != 1);
16− list_del(&page->lru);
15+ *pagepool = (struct page *)page_private(page);
1716 } else {
1817 page = alloc_page(gfp);
1918 }
2019 return page;
2120 }
2221
22+void erofs_release_pages(struct page **pagepool)
23+{
24+ while (*pagepool) {
25+ struct page *page = *pagepool;
26+
27+ *pagepool = (struct page *)page_private(page);
28+ put_page(page);
29+ }
30+}
31+
2332 #ifdef CONFIG_EROFS_FS_ZIP
2433 /* global shrink count (for all mounted EROFS instances) */
2534 static atomic_long_t erofs_global_shrink_cnt;
fs/erofs/xattr.c+2−2
@@ -429,7 +429,7 @@ static int shared_getxattr(struct inode *inode, struct getxattr_iter *it) · 428 unmodified lines
429429
430430 static bool erofs_xattr_user_list(struct dentry *dentry)
431431 {
432− return test_opt(&EROFS_SB(dentry->d_sb)->ctx, XATTR_USER);
432+ return test_opt(&EROFS_SB(dentry->d_sb)->opt, XATTR_USER);
433433 }
434434
435435 static bool erofs_xattr_trusted_list(struct dentry *dentry)
@@ -476,7 +476,7 @@ static int erofs_xattr_generic_get(const struct xattr_handler *handler, · 40 unmodified lines
476476
477477 switch (handler->flags) {
478478 case EROFS_XATTR_INDEX_USER:
479− if (!test_opt(&sbi->ctx, XATTR_USER))
479+ if (!test_opt(&sbi->opt, XATTR_USER))
480480 return -EOPNOTSUPP;
481481 break;
482482 case EROFS_XATTR_INDEX_TRUSTED:
fs/erofs/zdata.c+120−55
@@ -236,7 +236,7 @@ static DEFINE_MUTEX(z_pagemap_global_lock); · 235 unmodified lines
236236 static void preload_compressed_pages(struct z_erofs_collector *clt,
237237 struct address_space *mc,
238238 enum z_erofs_cache_alloctype type,
239− struct list_head *pagepool)
239+ struct page **pagepool)
240240 {
241241 struct z_erofs_pcluster *pcl = clt->pcl;
242242 bool standalone = true;
@@ -287,12 +287,10 @@ static void preload_compressed_pages(struct z_erofs_collector *clt, · 44 unmodified lines
287287 if (!cmpxchg_relaxed(pages, NULL, tagptr_cast_ptr(t)))
288288 continue;
289289
290− if (page) {
290+ if (page)
291291 put_page(page);
292− } else if (newpage) {
293− set_page_private(newpage, 0);
294− list_add(&newpage->lru, pagepool);
295− }
292+ else if (newpage)
293+ erofs_pagepool_add(pagepool, newpage);
296294 }
297295
298296 /*
@@ -476,6 +474,11 @@ static int z_erofs_register_collection(struct z_erofs_collector *clt, · 177 unmodified lines
476474 struct erofs_workgroup *grp;
477475 int err;
478476
477+ if (!(map->m_flags & EROFS_MAP_ENCODED)) {
478+ DBG_BUGON(1);
479+ return -EFSCORRUPTED;
480+ }
481+
479482 /* no available pcluster, let's allocate one */
480483 pcl = z_erofs_alloc_pcluster(map->m_plen >> PAGE_SHIFT);
481484 if (IS_ERR(pcl))
@@ -483,16 +486,11 @@ static int z_erofs_register_collection(struct z_erofs_collector *clt, · 1 unmodified line
483486
484487 atomic_set(&pcl->obj.refcount, 1);
485488 pcl->obj.index = map->m_pa >> PAGE_SHIFT;
486−
489+ pcl->algorithmformat = map->m_algorithmformat;
487490 pcl->length = (map->m_llen << Z_EROFS_PCLUSTER_LENGTH_BIT) |
488491 (map->m_flags & EROFS_MAP_FULL_MAPPED ?
489492 Z_EROFS_PCLUSTER_FULL_LENGTH : 0);
490493
491− if (map->m_flags & EROFS_MAP_ZIPPED)
492− pcl->algorithmformat = Z_EROFS_COMPRESSION_LZ4;
493− else
494− pcl->algorithmformat = Z_EROFS_COMPRESSION_SHIFTED;
495−
496494 /* new pclusters should be claimed as type 1, primary and followed */
497495 pcl->next = clt->owned_head;
498496 clt->mode = COLLECT_PRIMARY_FOLLOWED;
@@ -643,7 +641,7 @@ static bool should_alloc_managed_pages(struct z_erofs_decompress_frontend *fe, · 144 unmodified lines
643641 }
644642
645643 static int z_erofs_do_read_page(struct z_erofs_decompress_frontend *fe,
646− struct page *page, struct list_head *pagepool)
644+ struct page *page, struct page **pagepool)
647645 {
648646 struct inode *const inode = fe->inode;
649647 struct erofs_sb_info *const sbi = EROFS_I_SB(inode);
@@ -695,7 +693,7 @@ restart_now: · 45 unmodified lines
695693 goto err_out;
696694
697695 /* preload all compressed pages (maybe downgrade role if necessary) */
698− if (should_alloc_managed_pages(fe, sbi->ctx.cache_strategy, map->m_la))
696+ if (should_alloc_managed_pages(fe, sbi->opt.cache_strategy, map->m_la))
699697 cache_strategy = TRYALLOC;
700698 else
701699 cache_strategy = DONTALLOC;
@@ -796,7 +794,7 @@ static void z_erofs_decompress_kickoff(struct z_erofs_decompressqueue *io, · 94 unmodified lines
796794 /* Use workqueue and sync decompression for atomic contexts only */
797795 if (in_atomic() || irqs_disabled()) {
798796 queue_work(z_erofs_workqueue, &io->u.work);
799− sbi->ctx.readahead_sync_decompress = true;
797+ sbi->opt.readahead_sync_decompress = true;
800798 return;
801799 }
802800 z_erofs_decompressqueue_work(&io->u.work);
@@ -836,7 +834,7 @@ static void z_erofs_decompressqueue_endio(struct bio *bio) · 33 unmodified lines
836834
837835 static int z_erofs_decompress_pcluster(struct super_block *sb,
838836 struct z_erofs_pcluster *pcl,
839− struct list_head *pagepool)
837+ struct page **pagepool)
840838 {
841839 struct erofs_sb_info *const sbi = EROFS_SB(sb);
842840 struct z_erofs_pagevec_ctor ctor;
@@ -1036,7 +1034,7 @@ out: · 193 unmodified lines
10361034 }
10371035
10381036 static void z_erofs_decompress_queue(const struct z_erofs_decompressqueue *io,
1039− struct list_head *pagepool)
1037+ struct page **pagepool)
10401038 {
10411039 z_erofs_next_pcluster_t owned = io->head;
10421040
@@ -1060,18 +1058,18 @@ static void z_erofs_decompressqueue_work(struct work_struct *work) · 17 unmodified lines
10601058 {
10611059 struct z_erofs_decompressqueue *bgq =
10621060 container_of(work, struct z_erofs_decompressqueue, u.work);
1063− LIST_HEAD(pagepool);
1061+ struct page *pagepool = NULL;
10641062
10651063 DBG_BUGON(bgq->head == Z_EROFS_PCLUSTER_TAIL_CLOSED);
10661064 z_erofs_decompress_queue(bgq, &pagepool);
10671065
1068− put_pages_list(&pagepool);
1066+ erofs_release_pages(&pagepool);
10691067 kvfree(bgq);
10701068 }
10711069
10721070 static struct page *pickup_page_for_submission(struct z_erofs_pcluster *pcl,
10731071 unsigned int nr,
1074− struct list_head *pagepool,
1072+ struct page **pagepool,
10751073 struct address_space *mc,
10761074 gfp_t gfp)
10771075 {
@@ -1173,7 +1171,7 @@ repeat: · 95 unmodified lines
11731171 out_allocpage:
11741172 page = erofs_allocpage(pagepool, gfp | __GFP_NOFAIL);
11751173 if (oldpage != cmpxchg(&pcl->compressed_pages[nr], oldpage, page)) {
1176− list_add(&page->lru, pagepool);
1174+ erofs_pagepool_add(pagepool, page);
11771175 cond_resched();
11781176 goto repeat;
11791177 }
@@ -1257,7 +1255,7 @@ static void move_to_bypass_jobqueue(struct z_erofs_pcluster *pcl, · 77 unmodified lines
12571255
12581256 static void z_erofs_submit_queue(struct super_block *sb,
12591257 struct z_erofs_decompress_frontend *f,
1260− struct list_head *pagepool,
1258+ struct page **pagepool,
12611259 struct z_erofs_decompressqueue *fgq,
12621260 bool *force_fg)
12631261 {
@@ -1266,8 +1264,9 @@ static void z_erofs_submit_queue(struct super_block *sb, · 2 unmodified lines
12661264 struct z_erofs_decompressqueue *q[NR_JOBQUEUES];
12671265 void *bi_private;
12681266 z_erofs_next_pcluster_t owned_head = f->clt.owned_head;
1269− /* since bio will be NULL, no need to initialize last_index */
1267+ /* bio is NULL initially, so no need to initialize last_{index,bdev} */
12701268 pgoff_t last_index;
1269+ struct block_device *last_bdev;
12711270 unsigned int nr_bios = 0;
12721271 struct bio *bio = NULL;
12731272
@@ -1279,6 +1278,7 @@ static void z_erofs_submit_queue(struct super_block *sb, · 5 unmodified lines
12791278 q[JQ_SUBMIT]->head = owned_head;
12801279
12811280 do {
1281+ struct erofs_map_dev mdev;
12821282 struct z_erofs_pcluster *pcl;
12831283 pgoff_t cur, end;
12841284 unsigned int i = 0;
@@ -1290,7 +1290,13 @@ static void z_erofs_submit_queue(struct super_block *sb, · 5 unmodified lines
12901290
12911291 pcl = container_of(owned_head, struct z_erofs_pcluster, next);
12921292
1293− cur = pcl->obj.index;
1293+ /* no device id here, thus it will always succeed */
1294+ mdev = (struct erofs_map_dev) {
1295+ .m_pa = blknr_to_addr(pcl->obj.index),
1296+ };
1297+ (void)erofs_map_dev(sb, &mdev);
1298+
1299+ cur = erofs_blknr(mdev.m_pa);
12941300 end = cur + pcl->pclusterpages;
12951301
12961302 /* close the main owned chain at first */
@@ -1306,7 +1312,8 @@ static void z_erofs_submit_queue(struct super_block *sb, · 9 unmodified lines
13061312 if (!page)
13071313 continue;
13081314
1309− if (bio && cur != last_index + 1) {
1315+ if (bio && (cur != last_index + 1 ||
1316+ last_bdev != mdev.m_bdev)) {
13101317 submit_bio_retry:
13111318 submit_bio(bio);
13121319 bio = NULL;
@@ -1314,9 +1321,10 @@ submit_bio_retry: · 1 unmodified line
13141321
13151322 if (!bio) {
13161323 bio = bio_alloc(GFP_NOIO, BIO_MAX_VECS);
1317−
13181324 bio->bi_end_io = z_erofs_decompressqueue_endio;
1319− bio_set_dev(bio, sb->s_bdev);
1325+
1326+ bio_set_dev(bio, mdev.m_bdev);
1327+ last_bdev = mdev.m_bdev;
13201328 bio->bi_iter.bi_sector = (sector_t)cur <<
13211329 LOG_SECTORS_PER_BLOCK;
13221330 bio->bi_private = bi_private;
@@ -1355,7 +1363,7 @@ submit_bio_retry: · 32 unmodified lines
13551363
13561364 static void z_erofs_runqueue(struct super_block *sb,
13571365 struct z_erofs_decompress_frontend *f,
1358− struct list_head *pagepool, bool force_fg)
1366+ struct page **pagepool, bool force_fg)
13591367 {
13601368 struct z_erofs_decompressqueue io[NR_JOBQUEUES];
13611369
@@ -1377,18 +1385,87 @@ static void z_erofs_runqueue(struct super_block *sb, · 15 unmodified lines
13771385 z_erofs_decompress_queue(&io[JQ_SUBMIT], pagepool);
13781386 }
13791387
1388+/*
1389+ * Since partial uptodate is still unimplemented for now, we have to use
1390+ * approximate readmore strategies as a start.
1391+ */
1392+static void z_erofs_pcluster_readmore(struct z_erofs_decompress_frontend *f,
1393+ struct readahead_control *rac,
1394+ erofs_off_t end,
1395+ struct page **pagepool,
1396+ bool backmost)
1397+{
1398+ struct inode *inode = f->inode;
1399+ struct erofs_map_blocks *map = &f->map;
1400+ erofs_off_t cur;
1401+ int err;
1402+
1403+ if (backmost) {
1404+ map->m_la = end;
1405+ err = z_erofs_map_blocks_iter(inode, map,
1406+ EROFS_GET_BLOCKS_READMORE);
1407+ if (err)
1408+ return;
1409+
1410+ /* expend ra for the trailing edge if readahead */
1411+ if (rac) {
1412+ loff_t newstart = readahead_pos(rac);
1413+
1414+ cur = round_up(map->m_la + map->m_llen, PAGE_SIZE);
1415+ readahead_expand(rac, newstart, cur - newstart);
1416+ return;
1417+ }
1418+ end = round_up(end, PAGE_SIZE);
1419+ } else {
1420+ end = round_up(map->m_la, PAGE_SIZE);
1421+
1422+ if (!map->m_llen)
1423+ return;
1424+ }
1425+
1426+ cur = map->m_la + map->m_llen - 1;
1427+ while (cur >= end) {
1428+ pgoff_t index = cur >> PAGE_SHIFT;
1429+ struct page *page;
1430+
1431+ page = erofs_grab_cache_page_nowait(inode->i_mapping, index);
1432+ if (!page)
1433+ goto skip;
1434+
1435+ if (PageUptodate(page)) {
1436+ unlock_page(page);
1437+ put_page(page);
1438+ goto skip;
1439+ }
1440+
1441+ err = z_erofs_do_read_page(f, page, pagepool);
1442+ if (err)
1443+ erofs_err(inode->i_sb,
1444+ "readmore error at page %lu @ nid %llu",
1445+ index, EROFS_I(inode)->nid);
1446+ put_page(page);
1447+skip:
1448+ if (cur < PAGE_SIZE)
1449+ break;
1450+ cur = (index << PAGE_SHIFT) - 1;
1451+ }
1452+}
1453+
13801454 static int z_erofs_readpage(struct file *file, struct page *page)
13811455 {
13821456 struct inode *const inode = page->mapping->host;
13831457 struct z_erofs_decompress_frontend f = DECOMPRESS_FRONTEND_INIT(inode);
1458+ struct page *pagepool = NULL;
13841459 int err;
1385− LIST_HEAD(pagepool);
13861460
13871461 trace_erofs_readpage(page, false);
1388−
13891462 f.headoffset = (erofs_off_t)page->index << PAGE_SHIFT;
13901463
1464+ z_erofs_pcluster_readmore(&f, NULL, f.headoffset + PAGE_SIZE - 1,
1465+ &pagepool, true);
13911466 err = z_erofs_do_read_page(&f, page, &pagepool);
1467+ z_erofs_pcluster_readmore(&f, NULL, 0, &pagepool, false);
1468+
13921469 (void)z_erofs_collector_end(&f.clt);
13931470
13941471 /* if some compressed cluster ready, need submit them anyway */
@@ -1400,8 +1477,7 @@ static int z_erofs_readpage(struct file *file, struct page *page) · 5 unmodified lines
14001477 if (f.map.mpage)
14011478 put_page(f.map.mpage);
14021479
1403− /* clean up the remaining free pages */
1404− put_pages_list(&pagepool);
1480+ erofs_release_pages(&pagepool);
14051481 return err;
14061482 }
14071483
@@ -1409,29 +1485,19 @@ static void z_erofs_readahead(struct readahead_control *rac) · 1 unmodified line
14091485 {
14101486 struct inode *const inode = rac->mapping->host;
14111487 struct erofs_sb_info *const sbi = EROFS_I_SB(inode);
1412−
1413− unsigned int nr_pages = readahead_count(rac);
1414− bool sync = (sbi->ctx.readahead_sync_decompress &&
1415− nr_pages <= sbi->ctx.max_sync_decompress_pages);
14161488 struct z_erofs_decompress_frontend f = DECOMPRESS_FRONTEND_INIT(inode);
1417− struct page *page, *head = NULL;
1418− LIST_HEAD(pagepool);
1419−
1420− trace_erofs_readpages(inode, readahead_index(rac), nr_pages, false);
1489+ struct page *pagepool = NULL, *head = NULL, *page;
1490+ unsigned int nr_pages;
14211491
14221492 f.readahead = true;
14231493 f.headoffset = readahead_pos(rac);
14241494
1425− while ((page = readahead_page(rac))) {
1426− prefetchw(&page->flags);
1427−
1428− /*
1429− * A pure asynchronous readahead is indicated if
1430− * a PG_readahead marked page is hitted at first.
1431− * Let's also do asynchronous decompression for this case.
1432− */
1433− sync &= !(PageReadahead(page) && !head);
1495+ z_erofs_pcluster_readmore(&f, rac, f.headoffset +
1496+ readahead_length(rac) - 1, &pagepool, true);
1497+ nr_pages = readahead_count(rac);
1498+ trace_erofs_readpages(inode, readahead_index(rac), nr_pages, false);
14341499
1500+ while ((page = readahead_page(rac))) {
14351501 set_page_private(page, (unsigned long)head);
14361502 head = page;
14371503 }
@@ -1450,16 +1516,15 @@ static void z_erofs_readahead(struct readahead_control *rac) · 12 unmodified lines
14501516 page->index, EROFS_I(inode)->nid);
14511517 put_page(page);
14521518 }
1453−
1519+ z_erofs_pcluster_readmore(&f, rac, 0, &pagepool, false);
14541520 (void)z_erofs_collector_end(&f.clt);
14551521
1456− z_erofs_runqueue(inode->i_sb, &f, &pagepool, sync);
1457−
1522+ z_erofs_runqueue(inode->i_sb, &f, &pagepool,
1523+ sbi->opt.readahead_sync_decompress &&
1524+ nr_pages <= sbi->opt.max_sync_decompress_pages);
14581525 if (f.map.mpage)
14591526 put_page(f.map.mpage);
1460−
1461− /* clean up the remaining free pages */
1462− put_pages_list(&pagepool);
1527+ erofs_release_pages(&pagepool);
14631528 }
14641529
14651530 const struct address_space_operations z_erofs_aops = {
fs/erofs/zdata.h+0−7
@@ -94,13 +94,6 @@ struct z_erofs_decompressqueue { · 93 unmodified lines
9494 } u;
9595 };
9696
97−#define MNGD_MAPPING(sbi) ((sbi)->managed_cache->i_mapping)
98−static inline bool erofs_page_is_managed(const struct erofs_sb_info *sbi,
99− struct page *page)
100−{
101− return page->mapping == MNGD_MAPPING(sbi);
102−}
103−
10497 #define Z_EROFS_ONLINEPAGE_COUNT_BITS 2
10598 #define Z_EROFS_ONLINEPAGE_COUNT_MASK ((1 << Z_EROFS_ONLINEPAGE_COUNT_BITS) - 1)
10699 #define Z_EROFS_ONLINEPAGE_INDEX_SHIFT (Z_EROFS_ONLINEPAGE_COUNT_BITS)
fs/erofs/zmap.c+42−23
@@ -28,7 +28,7 @@ static int z_erofs_fill_inode_lazy(struct inode *inode) · 27 unmodified lines
2828 {
2929 struct erofs_inode *const vi = EROFS_I(inode);
3030 struct super_block *const sb = inode->i_sb;
31− int err;
31+ int err, headnr;
3232 erofs_off_t pos;
3333 struct page *page;
3434 void *kaddr;
@@ -68,9 +68,11 @@ static int z_erofs_fill_inode_lazy(struct inode *inode) · 33 unmodified lines
6868 vi->z_algorithmtype[0] = h->h_algorithmtype & 15;
6969 vi->z_algorithmtype[1] = h->h_algorithmtype >> 4;
7070
71− if (vi->z_algorithmtype[0] >= Z_EROFS_COMPRESSION_MAX) {
72− erofs_err(sb, "unknown compression format %u for nid %llu, please upgrade kernel",
73− vi->z_algorithmtype[0], vi->nid);
71+ headnr = 0;
72+ if (vi->z_algorithmtype[0] >= Z_EROFS_COMPRESSION_MAX ||
73+ vi->z_algorithmtype[++headnr] >= Z_EROFS_COMPRESSION_MAX) {
74+ erofs_err(sb, "unknown HEAD%u format %u for nid %llu, please upgrade kernel",
75+ headnr + 1, vi->z_algorithmtype[headnr], vi->nid);
7476 err = -EOPNOTSUPP;
7577 goto unmap_done;
7678 }
@@ -111,7 +113,7 @@ struct z_erofs_maprecorder { · 34 unmodified lines
111113
112114 unsigned long lcn;
113115 /* compression extent information gathered */
114− u8 type;
116+ u8 type, headtype;
115117 u16 clusterofs;
116118 u16 delta[2];
117119 erofs_blk_t pblk, compressedlcs;
@@ -178,7 +180,8 @@ static int legacy_load_cluster_from_disk(struct z_erofs_maprecorder *m, · 60 unmodified lines
178180 m->clusterofs = 1 << vi->z_logical_clusterbits;
179181 m->delta[0] = le16_to_cpu(di->di_u.delta[0]);
180182 if (m->delta[0] & Z_EROFS_VLE_DI_D0_CBLKCNT) {
181− if (!(vi->z_advise & Z_EROFS_ADVISE_BIG_PCLUSTER_1)) {
183+ if (!(vi->z_advise & (Z_EROFS_ADVISE_BIG_PCLUSTER_1 |
184+ Z_EROFS_ADVISE_BIG_PCLUSTER_2))) {
182185 DBG_BUGON(1);
183186 return -EFSCORRUPTED;
184187 }
@@ -189,7 +192,8 @@ static int legacy_load_cluster_from_disk(struct z_erofs_maprecorder *m, · 4 unmodified lines
189192 m->delta[1] = le16_to_cpu(di->di_u.delta[1]);
190193 break;
191194 case Z_EROFS_VLE_CLUSTER_TYPE_PLAIN:
192− case Z_EROFS_VLE_CLUSTER_TYPE_HEAD:
195+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD1:
196+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD2:
193197 m->clusterofs = le16_to_cpu(di->di_clusterofs);
194198 m->pblk = le32_to_cpu(di->di_u.blkaddr);
195199 break;
@@ -446,9 +450,9 @@ static int z_erofs_extent_lookback(struct z_erofs_maprecorder *m, · 250 unmodified lines
446450 }
447451 return z_erofs_extent_lookback(m, m->delta[0]);
448452 case Z_EROFS_VLE_CLUSTER_TYPE_PLAIN:
449− map->m_flags &= ~EROFS_MAP_ZIPPED;
450− fallthrough;
451− case Z_EROFS_VLE_CLUSTER_TYPE_HEAD:
453+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD1:
454+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD2:
455+ m->headtype = m->type;
452456 map->m_la = (lcn << lclusterbits) | m->clusterofs;
453457 break;
454458 default:
@@ -471,13 +475,18 @@ static int z_erofs_get_extent_compressedlen(struct z_erofs_maprecorder *m, · 16 unmodified lines
471475 int err;
472476
473477 DBG_BUGON(m->type != Z_EROFS_VLE_CLUSTER_TYPE_PLAIN &&
474− m->type != Z_EROFS_VLE_CLUSTER_TYPE_HEAD);
475− if (!(map->m_flags & EROFS_MAP_ZIPPED) ||
476− !(vi->z_advise & Z_EROFS_ADVISE_BIG_PCLUSTER_1)) {
478+ m->type != Z_EROFS_VLE_CLUSTER_TYPE_HEAD1 &&
479+ m->type != Z_EROFS_VLE_CLUSTER_TYPE_HEAD2);
480+ DBG_BUGON(m->type != m->headtype);
481+
482+ if (m->headtype == Z_EROFS_VLE_CLUSTER_TYPE_PLAIN ||
483+ ((m->headtype == Z_EROFS_VLE_CLUSTER_TYPE_HEAD1) &&
484+ !(vi->z_advise & Z_EROFS_ADVISE_BIG_PCLUSTER_1)) ||
485+ ((m->headtype == Z_EROFS_VLE_CLUSTER_TYPE_HEAD2) &&
486+ !(vi->z_advise & Z_EROFS_ADVISE_BIG_PCLUSTER_2))) {
477487 map->m_plen = 1 << lclusterbits;
478488 return 0;
479489 }
480−
481490 lcn = m->lcn + 1;
482491 if (m->compressedlcs)
483492 goto out;
@@ -499,7 +508,8 @@ static int z_erofs_get_extent_compressedlen(struct z_erofs_maprecorder *m, · 15 unmodified lines
499508
500509 switch (m->type) {
501510 case Z_EROFS_VLE_CLUSTER_TYPE_PLAIN:
502− case Z_EROFS_VLE_CLUSTER_TYPE_HEAD:
511+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD1:
512+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD2:
503513 /*
504514 * if the 1st NONHEAD lcluster is actually PLAIN or HEAD type
505515 * rather than CBLKCNT, it's a 1 lcluster-sized pcluster.
@@ -554,7 +564,8 @@ static int z_erofs_get_extent_decompressedlen(struct z_erofs_maprecorder *m) · 48 unmodified lines
554564 DBG_BUGON(!m->delta[1] &&
555565 m->clusterofs != 1 << lclusterbits);
556566 } else if (m->type == Z_EROFS_VLE_CLUSTER_TYPE_PLAIN ||
557− m->type == Z_EROFS_VLE_CLUSTER_TYPE_HEAD) {
567+ m->type == Z_EROFS_VLE_CLUSTER_TYPE_HEAD1 ||
568+ m->type == Z_EROFS_VLE_CLUSTER_TYPE_HEAD2) {
558569 /* go on until the next HEAD lcluster */
559570 if (lcn != headlcn)
560571 break;
@@ -609,16 +620,15 @@ int z_erofs_map_blocks_iter(struct inode *inode, · 48 unmodified lines
609620 if (err)
610621 goto unmap_out;
611622
612− map->m_flags = EROFS_MAP_ZIPPED; /* by default, compressed */
623+ map->m_flags = EROFS_MAP_MAPPED | EROFS_MAP_ENCODED;
613624 end = (m.lcn + 1ULL) << lclusterbits;
614625
615626 switch (m.type) {
616627 case Z_EROFS_VLE_CLUSTER_TYPE_PLAIN:
617− if (endoff >= m.clusterofs)
618− map->m_flags &= ~EROFS_MAP_ZIPPED;
619− fallthrough;
620− case Z_EROFS_VLE_CLUSTER_TYPE_HEAD:
628+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD1:
629+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD2:
621630 if (endoff >= m.clusterofs) {
631+ m.headtype = m.type;
622632 map->m_la = (m.lcn << lclusterbits) | m.clusterofs;
623633 break;
624634 }
@@ -650,13 +660,22 @@ int z_erofs_map_blocks_iter(struct inode *inode, · 25 unmodified lines
650660
651661 map->m_llen = end - map->m_la;
652662 map->m_pa = blknr_to_addr(m.pblk);
653− map->m_flags |= EROFS_MAP_MAPPED;
654663
655664 err = z_erofs_get_extent_compressedlen(&m, initial_lcn);
656665 if (err)
657666 goto out;
658667
659− if (flags & EROFS_GET_BLOCKS_FIEMAP) {
668+ if (m.headtype == Z_EROFS_VLE_CLUSTER_TYPE_PLAIN)
669+ map->m_algorithmformat = Z_EROFS_COMPRESSION_SHIFTED;
670+ else if (m.headtype == Z_EROFS_VLE_CLUSTER_TYPE_HEAD2)
671+ map->m_algorithmformat = vi->z_algorithmtype[1];
672+ else
673+ map->m_algorithmformat = vi->z_algorithmtype[0];
674+
675+ if ((flags & EROFS_GET_BLOCKS_FIEMAP) ||
676+ ((flags & EROFS_GET_BLOCKS_READMORE) &&
677+ map->m_algorithmformat == Z_EROFS_COMPRESSION_LZMA &&
678+ map->m_llen >= EROFS_BLKSIZ)) {
660679 err = z_erofs_get_extent_decompressedlen(&m);
661680 if (!err)
662681 map->m_flags |= EROFS_MAP_FULL_MAPPED;
include/linux/xz.h+106−0
@@ -233,6 +233,112 @@ XZ_EXTERN void xz_dec_reset(struct xz_dec *s); · 232 unmodified lines
233233 */
234234 XZ_EXTERN void xz_dec_end(struct xz_dec *s);
235235
236+/*
237+ * Decompressor for MicroLZMA, an LZMA variant with a very minimal header.
238+ * See xz_dec_microlzma_alloc() below for details.
239+ *
240+ * These functions aren't used or available in preboot code and thus aren't
241+ * marked with XZ_EXTERN. This avoids warnings about static functions that
242+ * are never defined.
243+ */
244+/**
245+ * struct xz_dec_microlzma - Opaque type to hold the MicroLZMA decoder state
246+ */
247+struct xz_dec_microlzma;
248+
249+/**
250+ * xz_dec_microlzma_alloc() - Allocate memory for the MicroLZMA decoder
251+ * @mode XZ_SINGLE or XZ_PREALLOC
252+ * @dict_size LZMA dictionary size. This must be at least 4 KiB and
253+ * at most 3 GiB.
254+ *
255+ * In contrast to xz_dec_init(), this function only allocates the memory
256+ * and remembers the dictionary size. xz_dec_microlzma_reset() must be used
257+ * before calling xz_dec_microlzma_run().
258+ *
259+ * The amount of allocated memory is a little less than 30 KiB with XZ_SINGLE.
260+ * With XZ_PREALLOC also a dictionary buffer of dict_size bytes is allocated.
261+ *
262+ * On success, xz_dec_microlzma_alloc() returns a pointer to
263+ * struct xz_dec_microlzma. If memory allocation fails or
264+ * dict_size is invalid, NULL is returned.
265+ *
266+ * The compressed format supported by this decoder is a raw LZMA stream
267+ * whose first byte (always 0x00) has been replaced with bitwise-negation
268+ * of the LZMA properties (lc/lp/pb) byte. For example, if lc/lp/pb is
269+ * 3/0/2, the first byte is 0xA2. This way the first byte can never be 0x00.
270+ * Just like with LZMA2, lc + lp <= 4 must be true. The LZMA end-of-stream
271+ * marker must not be used. The unused values are reserved for future use.
272+ * This MicroLZMA header format was created for use in EROFS but may be used
273+ * by others too.
274+ */
275+extern struct xz_dec_microlzma *xz_dec_microlzma_alloc(enum xz_mode mode,
276+ uint32_t dict_size);
277+
278+/**
279+ * xz_dec_microlzma_reset() - Reset the MicroLZMA decoder state
280+ * @s Decoder state allocated using xz_dec_microlzma_alloc()
281+ * @comp_size Compressed size of the input stream
282+ * @uncomp_size Uncompressed size of the input stream. A value smaller
283+ * than the real uncompressed size of the input stream can
284+ * be specified if uncomp_size_is_exact is set to false.
285+ * uncomp_size can never be set to a value larger than the
286+ * expected real uncompressed size because it would eventually
287+ * result in XZ_DATA_ERROR.
288+ * @uncomp_size_is_exact This is an int instead of bool to avoid
289+ * requiring stdbool.h. This should normally be set to true.
290+ * When this is set to false, error detection is weaker.
291+ */
292+extern void xz_dec_microlzma_reset(struct xz_dec_microlzma *s,
293+ uint32_t comp_size, uint32_t uncomp_size,
294+ int uncomp_size_is_exact);
295+
296+/**
297+ * xz_dec_microlzma_run() - Run the MicroLZMA decoder
298+ * @s Decoder state initialized using xz_dec_microlzma_reset()
299+ * @b: Input and output buffers
300+ *
301+ * This works similarly to xz_dec_run() with a few important differences.
302+ * Only the differences are documented here.
303+ *
304+ * The only possible return values are XZ_OK, XZ_STREAM_END, and
305+ * XZ_DATA_ERROR. This function cannot return XZ_BUF_ERROR: if no progress
306+ * is possible due to lack of input data or output space, this function will
307+ * keep returning XZ_OK. Thus, the calling code must be written so that it
308+ * will eventually provide input and output space matching (or exceeding)
309+ * comp_size and uncomp_size arguments given to xz_dec_microlzma_reset().
310+ * If the caller cannot do this (for example, if the input file is truncated
311+ * or otherwise corrupt), the caller must detect this error by itself to
312+ * avoid an infinite loop.
313+ *
314+ * If the compressed data seems to be corrupt, XZ_DATA_ERROR is returned.
315+ * This can happen also when incorrect dictionary, uncompressed, or
316+ * compressed sizes have been specified.
317+ *
318+ * With XZ_PREALLOC only: As an extra feature, b->out may be NULL to skip over
319+ * uncompressed data. This way the caller doesn't need to provide a temporary
320+ * output buffer for the bytes that will be ignored.
321+ *
322+ * With XZ_SINGLE only: In contrast to xz_dec_run(), the return value XZ_OK
323+ * is also possible and thus XZ_SINGLE is actually a limited multi-call mode.
324+ * After XZ_OK the bytes decoded so far may be read from the output buffer.
325+ * It is possible to continue decoding but the variables b->out and b->out_pos
326+ * MUST NOT be changed by the caller. Increasing the value of b->out_size is
327+ * allowed to make more output space available; one doesn't need to provide
328+ * space for the whole uncompressed data on the first call. The input buffer
329+ * may be changed normally like with XZ_PREALLOC. This way input data can be
330+ * provided from non-contiguous memory.
331+ */
332+extern enum xz_ret xz_dec_microlzma_run(struct xz_dec_microlzma *s,
333+ struct xz_buf *b);
334+
335+/**
336+ * xz_dec_microlzma_end() - Free the memory allocated for the decoder state
337+ * @s: Decoder state allocated using xz_dec_microlzma_alloc().
338+ * If s is NULL, this function does nothing.
339+ */
340+extern void xz_dec_microlzma_end(struct xz_dec_microlzma *s);
341+
236342 /*
237343 * Standalone build (userspace build or in-kernel build for boot time use)
238344 * needs a CRC32 implementation. For normal in-kernel use, kernel's own
include/trace/events/erofs.h+1−1
@@ -24,7 +24,7 @@ struct erofs_map_blocks; · 23 unmodified lines
2424 #define show_mflags(flags) __print_flags(flags, "", \
2525 { EROFS_MAP_MAPPED, "M" }, \
2626 { EROFS_MAP_META, "I" }, \
27− { EROFS_MAP_ZIPPED, "Z" })
27+ { EROFS_MAP_ENCODED, "E" })
2828
2929 TRACE_EVENT(erofs_lookup,
3030
lib/decompress_unxz.c+5−5
@@ -20,8 +20,8 @@ · 19 unmodified lines
2020 *
2121 * The worst case for in-place decompression is that the beginning of
2222 * the file is compressed extremely well, and the rest of the file is
23− * uncompressible. Thus, we must look for worst-case expansion when the
24− * compressor is encoding uncompressible data.
23+ * incompressible. Thus, we must look for worst-case expansion when the
24+ * compressor is encoding incompressible data.
2525 *
2626 * The structure of the .xz file in case of a compressed kernel is as follows.
2727 * Sizes (as bytes) of the fields are in parenthesis.
@@ -58,7 +58,7 @@ · 30 unmodified lines
5858 * uncompressed size of the payload is in practice never less than the
5959 * payload size itself. The LZMA2 format would allow uncompressed size
6060 * to be less than the payload size, but no sane compressor creates such
61− * files. LZMA2 supports storing uncompressible data in uncompressed form,
61+ * files. LZMA2 supports storing incompressible data in uncompressed form,
6262 * so there's never a need to create payloads whose uncompressed size is
6363 * smaller than the compressed size.
6464 *
@@ -167,8 +167,8 @@ · 102 unmodified lines
167167 * memeq and memzero are not used much and any remotely sane implementation
168168 * is fast enough. memcpy/memmove speed matters in multi-call mode, but
169169 * the kernel image is decompressed in single-call mode, in which only
170− * memcpy speed can matter and only if there is a lot of uncompressible data
171− * (LZMA2 stores uncompressible chunks in uncompressed form). Thus, the
170+ * memmove speed can matter and only if there is a lot of incompressible data
171+ * (LZMA2 stores incompressible chunks in uncompressed form). Thus, the
172172 * functions below should just be kept small; it's probably not worth
173173 * optimizing for speed.
174174 */
lib/xz/Kconfig+13−0
@@ -39,6 +39,19 @@ config XZ_DEC_SPARC · 38 unmodified lines
3939 default y
4040 select XZ_DEC_BCJ
4141
42+config XZ_DEC_MICROLZMA
43+ bool "MicroLZMA decoder"
44+ default n
45+ help
46+ MicroLZMA is a header format variant where the first byte
47+ of a raw LZMA stream (without the end of stream marker) has
48+ been replaced with a bitwise-negation of the lc/lp/pb
49+ properties byte. MicroLZMA was created to be used in EROFS
50+ but can be used by other things too where wasting minimal
51+ amount of space for headers is important.
52+
53+ Unless you know that you need this, say N.
54+
4255 endif
4356
4457 config XZ_DEC_BCJ
lib/xz/xz_dec_lzma2.c+175−7
@@ -248,6 +248,10 @@ struct lzma2_dec { · 247 unmodified lines
248248 * before the first LZMA chunk.
249249 */
250250 bool need_props;
251+
252+#ifdef XZ_DEC_MICROLZMA
253+ bool pedantic_microlzma;
254+#endif
251255 };
252256
253257 struct xz_dec_lzma2 {
@@ -387,7 +391,14 @@ static void dict_uncompressed(struct dictionary *dict, struct xz_buf *b, · 133 unmodified lines
387391
388392 *left -= copy_size;
389393
390− memcpy(dict->buf + dict->pos, b->in + b->in_pos, copy_size);
394+ /*
395+ * If doing in-place decompression in single-call mode and the
396+ * uncompressed size of the file is larger than the caller
397+ * thought (i.e. it is invalid input!), the buffers below may
398+ * overlap and cause undefined behavior with memcpy().
399+ * With valid inputs memcpy() would be fine here.
400+ */
401+ memmove(dict->buf + dict->pos, b->in + b->in_pos, copy_size);
391402 dict->pos += copy_size;
392403
393404 if (dict->full < dict->pos)
@@ -397,7 +408,11 @@ static void dict_uncompressed(struct dictionary *dict, struct xz_buf *b, · 3 unmodified lines
397408 if (dict->pos == dict->end)
398409 dict->pos = 0;
399410
400− memcpy(b->out + b->out_pos, b->in + b->in_pos,
411+ /*
412+ * Like above but for multi-call mode: use memmove()
413+ * to avoid undefined behavior with invalid input.
414+ */
415+ memmove(b->out + b->out_pos, b->in + b->in_pos,
401416 copy_size);
402417 }
403418
@@ -408,6 +423,12 @@ static void dict_uncompressed(struct dictionary *dict, struct xz_buf *b, · 4 unmodified lines
408423 }
409424 }
410425
426+#ifdef XZ_DEC_MICROLZMA
427+# define DICT_FLUSH_SUPPORTS_SKIPPING true
428+#else
429+# define DICT_FLUSH_SUPPORTS_SKIPPING false
430+#endif
431+
411432 /*
412433 * Flush pending data from dictionary to b->out. It is assumed that there is
413434 * enough space in b->out. This is guaranteed because caller uses dict_limit()
@@ -421,8 +442,19 @@ static uint32_t dict_flush(struct dictionary *dict, struct xz_buf *b) · 7 unmodified lines
421442 if (dict->pos == dict->end)
422443 dict->pos = 0;
423444
424− memcpy(b->out + b->out_pos, dict->buf + dict->start,
425− copy_size);
445+ /*
446+ * These buffers cannot overlap even if doing in-place
447+ * decompression because in multi-call mode dict->buf
448+ * has been allocated by us in this file; it's not
449+ * provided by the caller like in single-call mode.
450+ *
451+ * With MicroLZMA, b->out can be NULL to skip bytes that
452+ * the caller doesn't need. This cannot be done with XZ
453+ * because it would break BCJ filters.
454+ */
455+ if (!DICT_FLUSH_SUPPORTS_SKIPPING || b->out != NULL)
456+ memcpy(b->out + b->out_pos, dict->buf + dict->start,
457+ copy_size);
426458 }
427459
428460 dict->start = dict->pos;
@@ -488,7 +520,7 @@ static __always_inline void rc_normalize(struct rc_dec *rc) · 59 unmodified lines
488520 * functions so that the compiler is supposed to be able to more easily avoid
489521 * an extra branch. In this particular version of the LZMA decoder, this
490522 * doesn't seem to be a good idea (tested with GCC 3.3.6, 3.4.6, and 4.3.3
491− * on x86). Using a non-splitted version results in nicer looking code too.
523+ * on x86). Using a non-split version results in nicer looking code too.
492524 *
493525 * NOTE: This must return an int. Do not make it return a bool or the speed
494526 * of the code generated by GCC 3.x decreases 10-15 %. (GCC 4.3 doesn't care,
@@ -774,6 +806,7 @@ static void lzma_reset(struct xz_dec_lzma2 *s) · 279 unmodified lines
774806 s->lzma.rep1 = 0;
775807 s->lzma.rep2 = 0;
776808 s->lzma.rep3 = 0;
809+ s->lzma.len = 0;
777810
778811 /*
779812 * All probabilities are initialized to the same value. This hack
@@ -1157,8 +1190,6 @@ XZ_EXTERN enum xz_ret xz_dec_lzma2_reset(struct xz_dec_lzma2 *s, uint8_t props) · 377 unmodified lines
11571190 }
11581191 }
11591192
1160− s->lzma.len = 0;
1161−
11621193 s->lzma2.sequence = SEQ_CONTROL;
11631194 s->lzma2.need_dict_reset = true;
11641195
@@ -1174,3 +1205,140 @@ XZ_EXTERN void xz_dec_lzma2_end(struct xz_dec_lzma2 *s) · 9 unmodified lines
11741205
11751206 kfree(s);
11761207 }
1208+
1209+#ifdef XZ_DEC_MICROLZMA
1210+/* This is a wrapper struct to have a nice struct name in the public API. */
1211+struct xz_dec_microlzma {
1212+ struct xz_dec_lzma2 s;
1213+};
1214+
1215+enum xz_ret xz_dec_microlzma_run(struct xz_dec_microlzma *s_ptr,
1216+ struct xz_buf *b)
1217+{
1218+ struct xz_dec_lzma2 *s = &s_ptr->s;
1219+
1220+ /*
1221+ * sequence is SEQ_PROPERTIES before the first input byte,
1222+ * SEQ_LZMA_PREPARE until a total of five bytes have been read,
1223+ * and SEQ_LZMA_RUN for the rest of the input stream.
1224+ */
1225+ if (s->lzma2.sequence != SEQ_LZMA_RUN) {
1226+ if (s->lzma2.sequence == SEQ_PROPERTIES) {
1227+ /* One byte is needed for the props. */
1228+ if (b->in_pos >= b->in_size)
1229+ return XZ_OK;
1230+
1231+ /*
1232+ * Don't increment b->in_pos here. The same byte is
1233+ * also passed to rc_read_init() which will ignore it.
1234+ */
1235+ if (!lzma_props(s, ~b->in[b->in_pos]))
1236+ return XZ_DATA_ERROR;
1237+
1238+ s->lzma2.sequence = SEQ_LZMA_PREPARE;
1239+ }
1240+
1241+ /*
1242+ * xz_dec_microlzma_reset() doesn't validate the compressed
1243+ * size so we do it here. We have to limit the maximum size
1244+ * to avoid integer overflows in lzma2_lzma(). 3 GiB is a nice
1245+ * round number and much more than users of this code should
1246+ * ever need.
1247+ */
1248+ if (s->lzma2.compressed < RC_INIT_BYTES
1249+ || s->lzma2.compressed > (3U << 30))
1250+ return XZ_DATA_ERROR;
1251+
1252+ if (!rc_read_init(&s->rc, b))
1253+ return XZ_OK;
1254+
1255+ s->lzma2.compressed -= RC_INIT_BYTES;
1256+ s->lzma2.sequence = SEQ_LZMA_RUN;
1257+
1258+ dict_reset(&s->dict, b);
1259+ }
1260+
1261+ /* This is to allow increasing b->out_size between calls. */
1262+ if (DEC_IS_SINGLE(s->dict.mode))
1263+ s->dict.end = b->out_size - b->out_pos;
1264+
1265+ while (true) {
1266+ dict_limit(&s->dict, min_t(size_t, b->out_size - b->out_pos,
1267+ s->lzma2.uncompressed));
1268+
1269+ if (!lzma2_lzma(s, b))
1270+ return XZ_DATA_ERROR;
1271+
1272+ s->lzma2.uncompressed -= dict_flush(&s->dict, b);
1273+
1274+ if (s->lzma2.uncompressed == 0) {
1275+ if (s->lzma2.pedantic_microlzma) {
1276+ if (s->lzma2.compressed > 0 || s->lzma.len > 0
1277+ || !rc_is_finished(&s->rc))
1278+ return XZ_DATA_ERROR;
1279+ }
1280+
1281+ return XZ_STREAM_END;
1282+ }
1283+
1284+ if (b->out_pos == b->out_size)
1285+ return XZ_OK;
1286+
1287+ if (b->in_pos == b->in_size
1288+ && s->temp.size < s->lzma2.compressed)
1289+ return XZ_OK;
1290+ }
1291+}
1292+
1293+struct xz_dec_microlzma *xz_dec_microlzma_alloc(enum xz_mode mode,
1294+ uint32_t dict_size)
1295+{
1296+ struct xz_dec_microlzma *s;
1297+
1298+ /* Restrict dict_size to the same range as in the LZMA2 code. */
1299+ if (dict_size < 4096 || dict_size > (3U << 30))
1300+ return NULL;
1301+
1302+ s = kmalloc(sizeof(*s), GFP_KERNEL);
1303+ if (s == NULL)
1304+ return NULL;
1305+
1306+ s->s.dict.mode = mode;
1307+ s->s.dict.size = dict_size;
1308+
1309+ if (DEC_IS_MULTI(mode)) {
1310+ s->s.dict.end = dict_size;
1311+
1312+ s->s.dict.buf = vmalloc(dict_size);
1313+ if (s->s.dict.buf == NULL) {
1314+ kfree(s);
1315+ return NULL;
1316+ }
1317+ }
1318+
1319+ return s;
1320+}
1321+
1322+void xz_dec_microlzma_reset(struct xz_dec_microlzma *s, uint32_t comp_size,
1323+ uint32_t uncomp_size, int uncomp_size_is_exact)
1324+{
1325+ /*
1326+ * comp_size is validated in xz_dec_microlzma_run().
1327+ * uncomp_size can safely be anything.
1328+ */
1329+ s->s.lzma2.compressed = comp_size;
1330+ s->s.lzma2.uncompressed = uncomp_size;
1331+ s->s.lzma2.pedantic_microlzma = uncomp_size_is_exact;
1332+
1333+ s->s.lzma2.sequence = SEQ_PROPERTIES;
1334+ s->s.temp.size = 0;
1335+}
1336+
1337+void xz_dec_microlzma_end(struct xz_dec_microlzma *s)
1338+{
1339+ if (DEC_IS_MULTI(s->s.dict.mode))
1340+ vfree(s->s.dict.buf);
1341+
1342+ kfree(s);
1343+}
1344+#endif
lib/xz/xz_dec_stream.c+3−3
@@ -402,12 +402,12 @@ static enum xz_ret dec_stream_header(struct xz_dec *s) · 401 unmodified lines
402402 * we will accept other check types too, but then the check won't
403403 * be verified and a warning (XZ_UNSUPPORTED_CHECK) will be given.
404404 */
405+ if (s->temp.buf[HEADER_MAGIC_SIZE + 1] > XZ_CHECK_MAX)
406+ return XZ_OPTIONS_ERROR;
407+
405408 s->check_type = s->temp.buf[HEADER_MAGIC_SIZE + 1];
406409
407410 #ifdef XZ_DEC_ANY_CHECK
408− if (s->check_type > XZ_CHECK_MAX)
409− return XZ_OPTIONS_ERROR;
410−
411411 if (s->check_type > XZ_CHECK_CRC32)
412412 return XZ_UNSUPPORTED_CHECK;
413413 #else
lib/xz/xz_dec_syms.c+8−1
@@ -15,8 +15,15 @@ EXPORT_SYMBOL(xz_dec_reset); · 14 unmodified lines
1515 EXPORT_SYMBOL(xz_dec_run);
1616 EXPORT_SYMBOL(xz_dec_end);
1717
18+#ifdef CONFIG_XZ_DEC_MICROLZMA
19+EXPORT_SYMBOL(xz_dec_microlzma_alloc);
20+EXPORT_SYMBOL(xz_dec_microlzma_reset);
21+EXPORT_SYMBOL(xz_dec_microlzma_run);
22+EXPORT_SYMBOL(xz_dec_microlzma_end);
23+#endif
24+
1825 MODULE_DESCRIPTION("XZ decompressor");
19−MODULE_VERSION("1.0");
26+MODULE_VERSION("1.1");
2027 MODULE_AUTHOR("Lasse Collin <lasse.collin@tukaani.org> and Igor Pavlov");
2128
2229 /*
lib/xz/xz_private.h+3−0
@@ -37,6 +37,9 @@ · 36 unmodified lines
3737 # ifdef CONFIG_XZ_DEC_SPARC
3838 # define XZ_DEC_SPARC
3939 # endif
40+# ifdef CONFIG_XZ_DEC_MICROLZMA
41+# define XZ_DEC_MICROLZMA
42+# endif
4043 # define memeq(a, b, size) (memcmp(a, b, size) == 0)
4144 # define memzero(buf, size) memset(buf, 0, size)
4245 # endif

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t