History

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client

* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client: (205 commits) ceph: update for write_inode API change ceph: reset osd after relevant messages timed out ceph: fix flush_dirty_caps race with caps migration ceph: include migrating caps in issued set ceph: fix osdmap decoding when pools include (removed) snaps ceph: return EBADF if waiting for caps on closed file ceph: set osd request message front length correctly ceph: reset front len on return to msgpool; BUG on mismatched front iov ceph: fix snaptrace decoding on cap migration between mds ceph: use single osd op reply msg ceph: reset bits on connection close ceph: remove bogus mds forward warning ceph: remove fragile __map_osds optimization ceph: fix connection fault STANDBY check ceph: invalidate_authorizer without con->mutex held ceph: don't clobber write return value when using O_SYNC ceph: fix client_request_forward decoding ceph: drop messages on unregistered mds sessions; cleanup ceph: fix comments, locking in destroy_inode ceph: move dereference after NULL test ... Fix trivial conflicts in Documentation/ioctl/ioctl-number.txt

Linus Torvalds committed 16 years ago

Commit fc7f99c2 parents 0a492fd f1a3d5767 files · +28066 −0Browse files

Unified · Split

67 files
  1. Documentation/filesystems/ceph.txt +139 −0
  2. Documentation/ioctl/ioctl-number.txt +1 −0
  3. MAINTAINERS +9 −0
  4. fs/Kconfig +1 −0
  5. fs/Makefile +1 −0
  6. fs/ceph/Kconfig +27 −0
  7. fs/ceph/Makefile +39 −0
  8. fs/ceph/README +20 −0
  9. fs/ceph/addr.c +1188 −0
  10. fs/ceph/armor.c +99 −0
  11. fs/ceph/auth.c +257 −0
  12. fs/ceph/auth.h +84 −0
  13. fs/ceph/auth_none.c +121 −0
  14. fs/ceph/auth_none.h +28 −0
  15. fs/ceph/auth_x.c +656 −0
  16. fs/ceph/auth_x.h +49 −0
  17. fs/ceph/auth_x_protocol.h +90 −0
  18. fs/ceph/buffer.c +78 −0
  19. fs/ceph/buffer.h +39 −0
  20. fs/ceph/caps.c +2927 −0
  21. fs/ceph/ceph_debug.h +37 −0
  22. fs/ceph/ceph_frag.c +21 −0
  23. fs/ceph/ceph_frag.h +109 −0
  24. fs/ceph/ceph_fs.c +74 −0
  25. fs/ceph/ceph_fs.h +650 −0
  26. fs/ceph/ceph_hash.c +118 −0
  27. fs/ceph/ceph_hash.h +13 −0
  28. fs/ceph/ceph_strings.c +176 −0
  29. fs/ceph/crush/crush.c +151 −0
  30. fs/ceph/crush/crush.h +180 −0
  31. fs/ceph/crush/hash.c +149 −0
  32. fs/ceph/crush/hash.h +17 −0
  33. fs/ceph/crush/mapper.c +596 −0
  34. fs/ceph/crush/mapper.h +20 −0
  35. fs/ceph/crypto.c +408 −0
  36. fs/ceph/crypto.h +48 −0
  37. fs/ceph/debugfs.c +483 −0
  38. fs/ceph/decode.h +194 −0
  39. fs/ceph/dir.c +1220 −0
  40. fs/ceph/export.c +223 −0
  41. fs/ceph/file.c +937 −0
  42. fs/ceph/inode.c +1750 −0
  43. fs/ceph/ioctl.c +160 −0
  44. fs/ceph/ioctl.h +40 −0
  45. fs/ceph/mds_client.c +3021 −0
  46. fs/ceph/mds_client.h +335 −0
  47. fs/ceph/mdsmap.c +174 −0
  48. fs/ceph/mdsmap.h +54 −0
  49. fs/ceph/messenger.c +2240 −0
  50. fs/ceph/messenger.h +254 −0
  51. fs/ceph/mon_client.c +834 −0
  52. fs/ceph/mon_client.h +119 −0
  53. fs/ceph/msgpool.c +186 −0
  54. fs/ceph/msgpool.h +27 −0
  55. fs/ceph/msgr.h +158 −0
  56. fs/ceph/osd_client.c +1537 −0
  57. fs/ceph/osd_client.h +166 −0
  58. fs/ceph/osdmap.c +1019 −0
  59. fs/ceph/osdmap.h +125 −0
  60. fs/ceph/pagelist.c +54 −0
  61. fs/ceph/pagelist.h +54 −0
  62. fs/ceph/rados.h +374 −0
  63. fs/ceph/snap.c +904 −0
  64. fs/ceph/super.c +1030 −0
  65. fs/ceph/super.h +901 −0
  66. fs/ceph/types.h +29 −0
  67. fs/ceph/xattr.c +844 −0
Documentation/filesystems/ceph.txt+139−0
@@ -0,0 +1,139 @@
1Ceph Distributed File System
2============================
3
4Ceph is a distributed network file system designed to provide good
5performance, reliability, and scalability.
6
7Basic features include:
8
9 * POSIX semantics
10 * Seamless scaling from 1 to many thousands of nodes
11 * High availability and reliability. No single points of failure.
12 * N-way replication of data across storage nodes
13 * Fast recovery from node failures
14 * Automatic rebalancing of data on node addition/removal
15 * Easy deployment: most FS components are userspace daemons
16
17Also,
18 * Flexible snapshots (on any directory)
19 * Recursive accounting (nested files, directories, bytes)
20
21In contrast to cluster filesystems like GFS, OCFS2, and GPFS that rely
22on symmetric access by all clients to shared block devices, Ceph
23separates data and metadata management into independent server
24clusters, similar to Lustre. Unlike Lustre, however, metadata and
25storage nodes run entirely as user space daemons. Storage nodes
26utilize btrfs to store data objects, leveraging its advanced features
27(checksumming, metadata replication, etc.). File data is striped
28across storage nodes in large chunks to distribute workload and
29facilitate high throughputs. When storage nodes fail, data is
30re-replicated in a distributed fashion by the storage nodes themselves
31(with some minimal coordination from a cluster monitor), making the
32system extremely efficient and scalable.
33
34Metadata servers effectively form a large, consistent, distributed
35in-memory cache above the file namespace that is extremely scalable,
36dynamically redistributes metadata in response to workload changes,
37and can tolerate arbitrary (well, non-Byzantine) node failures. The
38metadata server takes a somewhat unconventional approach to metadata
39storage to significantly improve performance for common workloads. In
40particular, inodes with only a single link are embedded in
41directories, allowing entire directories of dentries and inodes to be
42loaded into its cache with a single I/O operation. The contents of
43extremely large directories can be fragmented and managed by
44independent metadata servers, allowing scalable concurrent access.
45
46The system offers automatic data rebalancing/migration when scaling
47from a small cluster of just a few nodes to many hundreds, without
48requiring an administrator carve the data set into static volumes or
49go through the tedious process of migrating data between servers.
50When the file system approaches full, new nodes can be easily added
51and things will "just work."
52
53Ceph includes flexible snapshot mechanism that allows a user to create
54a snapshot on any subdirectory (and its nested contents) in the
55system. Snapshot creation and deletion are as simple as 'mkdir
56.snap/foo' and 'rmdir .snap/foo'.
57
58Ceph also provides some recursive accounting on directories for nested
59files and bytes. That is, a 'getfattr -d foo' on any directory in the
60system will reveal the total number of nested regular files and
61subdirectories, and a summation of all nested file sizes. This makes
62the identification of large disk space consumers relatively quick, as
63no 'du' or similar recursive scan of the file system is required.
64
65
66Mount Syntax
67============
68
69The basic mount syntax is:
70
71 # mount -t ceph monip[:port][,monip2[:port]...]:/[subdir] mnt
72
73You only need to specify a single monitor, as the client will get the
74full list when it connects. (However, if the monitor you specify
75happens to be down, the mount won't succeed.) The port can be left
76off if the monitor is using the default. So if the monitor is at
771.2.3.4,
78
79 # mount -t ceph 1.2.3.4:/ /mnt/ceph
80
81is sufficient. If /sbin/mount.ceph is installed, a hostname can be
82used instead of an IP address.
83
84
85
86Mount Options
87=============
88
89 ip=A.B.C.D[:N]
90 Specify the IP and/or port the client should bind to locally.
91 There is normally not much reason to do this. If the IP is not
92 specified, the client's IP address is determined by looking at the
93 address it's connection to the monitor originates from.
94
95 wsize=X
96 Specify the maximum write size in bytes. By default there is no
97 maximu. Ceph will normally size writes based on the file stripe
98 size.
99
100 rsize=X
101 Specify the maximum readahead.
102
103 mount_timeout=X
104 Specify the timeout value for mount (in seconds), in the case
105 of a non-responsive Ceph file system. The default is 30
106 seconds.
107
108 rbytes
109 When stat() is called on a directory, set st_size to 'rbytes',
110 the summation of file sizes over all files nested beneath that
111 directory. This is the default.
112
113 norbytes
114 When stat() is called on a directory, set st_size to the
115 number of entries in that directory.
116
117 nocrc
118 Disable CRC32C calculation for data writes. If set, the OSD
119 must rely on TCP's error correction to detect data corruption
120 in the data payload.
121
122 noasyncreaddir
123 Disable client's use its local cache to satisfy readdir
124 requests. (This does not change correctness; the client uses
125 cached metadata only when a lease or capability ensures it is
126 valid.)
127
128
129More Information
130================
131
132For more information on Ceph, see the home page at
133 http://ceph.newdream.net/
134
135The Linux kernel client source tree is available at
136 git://ceph.newdream.net/linux-ceph-client.git
137
138and the source for the full system is at
139 git://ceph.newdream.net/ceph.git
Documentation/ioctl/ioctl-number.txt+1−0
@@ -291,6 +291,7 @@ Code Seq#(hex) Include File Comments · 290 unmodified lines
2910x92 00-0F drivers/usb/mon/mon_bin.c2910x92 00-0F drivers/usb/mon/mon_bin.c
2920x93 60-7F linux/auto_fs.h2920x93 60-7F linux/auto_fs.h
2930x94 all fs/btrfs/ioctl.h2930x94 all fs/btrfs/ioctl.h
2940x97 00-7F fs/ceph/ioctl.h Ceph file system
2940x99 00-0F 537-Addinboard driver2950x99 00-0F 537-Addinboard driver
295 <mailto:buk@buks.ipn.de>296 <mailto:buk@buks.ipn.de>
2960xA0 all linux/sdp/sdp.h Industrial Device Project2970xA0 all linux/sdp/sdp.h Industrial Device Project
MAINTAINERS+9−0
@@ -1441,6 +1441,15 @@ F: arch/powerpc/include/asm/spu*.h · 1440 unmodified lines
1441F: arch/powerpc/oprofile/*cell*1441F: arch/powerpc/oprofile/*cell*
1442F: arch/powerpc/platforms/cell/1442F: arch/powerpc/platforms/cell/
1443 1443
1444CEPH DISTRIBUTED FILE SYSTEM CLIENT
1445M: Sage Weil <sage@newdream.net>
1446L: ceph-devel@lists.sourceforge.net
1447W: http://ceph.newdream.net/
1448T: git git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client.git
1449S: Supported
1450F: Documentation/filesystems/ceph.txt
1451F: fs/ceph
1452
1444CERTIFIED WIRELESS USB (WUSB) SUBSYSTEM:1453CERTIFIED WIRELESS USB (WUSB) SUBSYSTEM:
1445M: David Vrabel <david.vrabel@csr.com>1454M: David Vrabel <david.vrabel@csr.com>
1446L: linux-usb@vger.kernel.org1455L: linux-usb@vger.kernel.org
fs/Kconfig+1−0
@@ -235,6 +235,7 @@ config NFS_COMMON · 234 unmodified lines
235 235
236source "net/sunrpc/Kconfig"236source "net/sunrpc/Kconfig"
237source "fs/smbfs/Kconfig"237source "fs/smbfs/Kconfig"
238source "fs/ceph/Kconfig"
238source "fs/cifs/Kconfig"239source "fs/cifs/Kconfig"
239source "fs/ncpfs/Kconfig"240source "fs/ncpfs/Kconfig"
240source "fs/coda/Kconfig"241source "fs/coda/Kconfig"
fs/Makefile+1−0
@@ -125,3 +125,4 @@ obj-$(CONFIG_OCFS2_FS) += ocfs2/ · 124 unmodified lines
125obj-$(CONFIG_BTRFS_FS) += btrfs/125obj-$(CONFIG_BTRFS_FS) += btrfs/
126obj-$(CONFIG_GFS2_FS) += gfs2/126obj-$(CONFIG_GFS2_FS) += gfs2/
127obj-$(CONFIG_EXOFS_FS) += exofs/127obj-$(CONFIG_EXOFS_FS) += exofs/
128obj-$(CONFIG_CEPH_FS) += ceph/
fs/ceph/Kconfig+27−0
@@ -0,0 +1,27 @@
1config CEPH_FS
2 tristate "Ceph distributed file system (EXPERIMENTAL)"
3 depends on INET && EXPERIMENTAL
4 select LIBCRC32C
5 select CONFIG_CRYPTO_AES
6 help
7 Choose Y or M here to include support for mounting the
8 experimental Ceph distributed file system. Ceph is an extremely
9 scalable file system designed to provide high performance,
10 reliable access to petabytes of storage.
11
12 More information at http://ceph.newdream.net/.
13
14 If unsure, say N.
15
16config CEPH_FS_PRETTYDEBUG
17 bool "Include file:line in ceph debug output"
18 depends on CEPH_FS
19 default n
20 help
21 If you say Y here, debug output will include a filename and
22 line to aid debugging. This icnreases kernel size and slows
23 execution slightly when debug call sites are enabled (e.g.,
24 via CONFIG_DYNAMIC_DEBUG).
25
26 If unsure, say N.
27
fs/ceph/Makefile+39−0
@@ -0,0 +1,39 @@
1#
2# Makefile for CEPH filesystem.
3#
4
5ifneq ($(KERNELRELEASE),)
6
7obj-$(CONFIG_CEPH_FS) += ceph.o
8
9ceph-objs := super.o inode.o dir.o file.o addr.o ioctl.o \
10 export.o caps.o snap.o xattr.o \
11 messenger.o msgpool.o buffer.o pagelist.o \
12 mds_client.o mdsmap.o \
13 mon_client.o \
14 osd_client.o osdmap.o crush/crush.o crush/mapper.o crush/hash.o \
15 debugfs.o \
16 auth.o auth_none.o \
17 crypto.o armor.o \
18 auth_x.o \
19 ceph_fs.o ceph_strings.o ceph_hash.o ceph_frag.o
20
21else
22#Otherwise we were called directly from the command
23# line; invoke the kernel build system.
24
25KERNELDIR ?= /lib/modules/$(shell uname -r)/build
26PWD := $(shell pwd)
27
28default: all
29
30all:
31 $(MAKE) -C $(KERNELDIR) M=$(PWD) CONFIG_CEPH_FS=m modules
32
33modules_install:
34 $(MAKE) -C $(KERNELDIR) M=$(PWD) CONFIG_CEPH_FS=m modules_install
35
36clean:
37 $(MAKE) -C $(KERNELDIR) M=$(PWD) clean
38
39endif
fs/ceph/README+20−0
@@ -0,0 +1,20 @@
1#
2# The following files are shared by (and manually synchronized
3# between) the Ceph userland and kernel client.
4#
5# userland kernel
6src/include/ceph_fs.h fs/ceph/ceph_fs.h
7src/include/ceph_fs.cc fs/ceph/ceph_fs.c
8src/include/msgr.h fs/ceph/msgr.h
9src/include/rados.h fs/ceph/rados.h
10src/include/ceph_strings.cc fs/ceph/ceph_strings.c
11src/include/ceph_frag.h fs/ceph/ceph_frag.h
12src/include/ceph_frag.cc fs/ceph/ceph_frag.c
13src/include/ceph_hash.h fs/ceph/ceph_hash.h
14src/include/ceph_hash.cc fs/ceph/ceph_hash.c
15src/crush/crush.c fs/ceph/crush/crush.c
16src/crush/crush.h fs/ceph/crush/crush.h
17src/crush/mapper.c fs/ceph/crush/mapper.c
18src/crush/mapper.h fs/ceph/crush/mapper.h
19src/crush/hash.h fs/ceph/crush/hash.h
20src/crush/hash.c fs/ceph/crush/hash.c
fs/ceph/addr.c+1188−0
@@ -0,0 +1,1188 @@
1#include "ceph_debug.h"
2
3#include <linux/backing-dev.h>
4#include <linux/fs.h>
5#include <linux/mm.h>
6#include <linux/pagemap.h>
7#include <linux/writeback.h> /* generic_writepages */
8#include <linux/pagevec.h>
9#include <linux/task_io_accounting_ops.h>
10
11#include "super.h"
12#include "osd_client.h"
13
14/*
15 * Ceph address space ops.
16 *
17 * There are a few funny things going on here.
18 *
19 * The page->private field is used to reference a struct
20 * ceph_snap_context for _every_ dirty page. This indicates which
21 * snapshot the page was logically dirtied in, and thus which snap
22 * context needs to be associated with the osd write during writeback.
23 *
24 * Similarly, struct ceph_inode_info maintains a set of counters to
25 * count dirty pages on the inode. In the absense of snapshots,
26 * i_wrbuffer_ref == i_wrbuffer_ref_head == the dirty page count.
27 *
28 * When a snapshot is taken (that is, when the client receives
29 * notification that a snapshot was taken), each inode with caps and
30 * with dirty pages (dirty pages implies there is a cap) gets a new
31 * ceph_cap_snap in the i_cap_snaps list (which is sorted in ascending
32 * order, new snaps go to the tail). The i_wrbuffer_ref_head count is
33 * moved to capsnap->dirty. (Unless a sync write is currently in
34 * progress. In that case, the capsnap is said to be "pending", new
35 * writes cannot start, and the capsnap isn't "finalized" until the
36 * write completes (or fails) and a final size/mtime for the inode for
37 * that snap can be settled upon.) i_wrbuffer_ref_head is reset to 0.
38 *
39 * On writeback, we must submit writes to the osd IN SNAP ORDER. So,
40 * we look for the first capsnap in i_cap_snaps and write out pages in
41 * that snap context _only_. Then we move on to the next capsnap,
42 * eventually reaching the "live" or "head" context (i.e., pages that
43 * are not yet snapped) and are writing the most recently dirtied
44 * pages.
45 *
46 * Invalidate and so forth must take care to ensure the dirty page
47 * accounting is preserved.
48 */
49
50#define CONGESTION_ON_THRESH(congestion_kb) (congestion_kb >> (PAGE_SHIFT-10))
51#define CONGESTION_OFF_THRESH(congestion_kb) \
52 (CONGESTION_ON_THRESH(congestion_kb) - \
53 (CONGESTION_ON_THRESH(congestion_kb) >> 2))
54
55
56
57/*
58 * Dirty a page. Optimistically adjust accounting, on the assumption
59 * that we won't race with invalidate. If we do, readjust.
60 */
61static int ceph_set_page_dirty(struct page *page)
62{
63 struct address_space *mapping = page->mapping;
64 struct inode *inode;
65 struct ceph_inode_info *ci;
66 int undo = 0;
67 struct ceph_snap_context *snapc;
68
69 if (unlikely(!mapping))
70 return !TestSetPageDirty(page);
71
72 if (TestSetPageDirty(page)) {
73 dout("%p set_page_dirty %p idx %lu -- already dirty\n",
74 mapping->host, page, page->index);
75 return 0;
76 }
77
78 inode = mapping->host;
79 ci = ceph_inode(inode);
80
81 /*
82 * Note that we're grabbing a snapc ref here without holding
83 * any locks!
84 */
85 snapc = ceph_get_snap_context(ci->i_snap_realm->cached_context);
86
87 /* dirty the head */
88 spin_lock(&inode->i_lock);
89 if (ci->i_wrbuffer_ref_head == 0)
90 ci->i_head_snapc = ceph_get_snap_context(snapc);
91 ++ci->i_wrbuffer_ref_head;
92 if (ci->i_wrbuffer_ref == 0)
93 igrab(inode);
94 ++ci->i_wrbuffer_ref;
95 dout("%p set_page_dirty %p idx %lu head %d/%d -> %d/%d "
96 "snapc %p seq %lld (%d snaps)\n",
97 mapping->host, page, page->index,
98 ci->i_wrbuffer_ref-1, ci->i_wrbuffer_ref_head-1,
99 ci->i_wrbuffer_ref, ci->i_wrbuffer_ref_head,
100 snapc, snapc->seq, snapc->num_snaps);
101 spin_unlock(&inode->i_lock);
102
103 /* now adjust page */
104 spin_lock_irq(&mapping->tree_lock);
105 if (page->mapping) { /* Race with truncate? */
106 WARN_ON_ONCE(!PageUptodate(page));
107
108 if (mapping_cap_account_dirty(mapping)) {
109 __inc_zone_page_state(page, NR_FILE_DIRTY);
110 __inc_bdi_stat(mapping->backing_dev_info,
111 BDI_RECLAIMABLE);
112 task_io_account_write(PAGE_CACHE_SIZE);
113 }
114 radix_tree_tag_set(&mapping->page_tree,
115 page_index(page), PAGECACHE_TAG_DIRTY);
116
117 /*
118 * Reference snap context in page->private. Also set
119 * PagePrivate so that we get invalidatepage callback.
120 */
121 page->private = (unsigned long)snapc;
122 SetPagePrivate(page);
123 } else {
124 dout("ANON set_page_dirty %p (raced truncate?)\n", page);
125 undo = 1;
126 }
127
128 spin_unlock_irq(&mapping->tree_lock);
129
130 if (undo)
131 /* whoops, we failed to dirty the page */
132 ceph_put_wrbuffer_cap_refs(ci, 1, snapc);
133
134 __mark_inode_dirty(mapping->host, I_DIRTY_PAGES);
135
136 BUG_ON(!PageDirty(page));
137 return 1;
138}
139
140/*
141 * If we are truncating the full page (i.e. offset == 0), adjust the
142 * dirty page counters appropriately. Only called if there is private
143 * data on the page.
144 */
145static void ceph_invalidatepage(struct page *page, unsigned long offset)
146{
147 struct inode *inode;
148 struct ceph_inode_info *ci;
149 struct ceph_snap_context *snapc = (void *)page->private;
150
151 BUG_ON(!PageLocked(page));
152 BUG_ON(!page->private);
153 BUG_ON(!PagePrivate(page));
154 BUG_ON(!page->mapping);
155
156 inode = page->mapping->host;
157
158 /*
159 * We can get non-dirty pages here due to races between
160 * set_page_dirty and truncate_complete_page; just spit out a
161 * warning, in case we end up with accounting problems later.
162 */
163 if (!PageDirty(page))
164 pr_err("%p invalidatepage %p page not dirty\n", inode, page);
165
166 if (offset == 0)
167 ClearPageChecked(page);
168
169 ci = ceph_inode(inode);
170 if (offset == 0) {
171 dout("%p invalidatepage %p idx %lu full dirty page %lu\n",
172 inode, page, page->index, offset);
173 ceph_put_wrbuffer_cap_refs(ci, 1, snapc);
174 ceph_put_snap_context(snapc);
175 page->private = 0;
176 ClearPagePrivate(page);
177 } else {
178 dout("%p invalidatepage %p idx %lu partial dirty page\n",
179 inode, page, page->index);
180 }
181}
182
183/* just a sanity check */
184static int ceph_releasepage(struct page *page, gfp_t g)
185{
186 struct inode *inode = page->mapping ? page->mapping->host : NULL;
187 dout("%p releasepage %p idx %lu\n", inode, page, page->index);
188 WARN_ON(PageDirty(page));
189 WARN_ON(page->private);
190 WARN_ON(PagePrivate(page));
191 return 0;
192}
193
194/*
195 * read a single page, without unlocking it.
196 */
197static int readpage_nounlock(struct file *filp, struct page *page)
198{
199 struct inode *inode = filp->f_dentry->d_inode;
200 struct ceph_inode_info *ci = ceph_inode(inode);
201 struct ceph_osd_client *osdc = &ceph_inode_to_client(inode)->osdc;
202 int err = 0;
203 u64 len = PAGE_CACHE_SIZE;
204
205 dout("readpage inode %p file %p page %p index %lu\n",
206 inode, filp, page, page->index);
207 err = ceph_osdc_readpages(osdc, ceph_vino(inode), &ci->i_layout,
208 page->index << PAGE_CACHE_SHIFT, &len,
209 ci->i_truncate_seq, ci->i_truncate_size,
210 &page, 1);
211 if (err == -ENOENT)
212 err = 0;
213 if (err < 0) {
214 SetPageError(page);
215 goto out;
216 } else if (err < PAGE_CACHE_SIZE) {
217 /* zero fill remainder of page */
218 zero_user_segment(page, err, PAGE_CACHE_SIZE);
219 }
220 SetPageUptodate(page);
221
222out:
223 return err < 0 ? err : 0;
224}
225
226static int ceph_readpage(struct file *filp, struct page *page)
227{
228 int r = readpage_nounlock(filp, page);
229 unlock_page(page);
230 return r;
231}
232
233/*
234 * Build a vector of contiguous pages from the provided page list.
235 */
236static struct page **page_vector_from_list(struct list_head *page_list,
237 unsigned *nr_pages)
238{
239 struct page **pages;
240 struct page *page;
241 int next_index, contig_pages = 0;
242
243 /* build page vector */
244 pages = kmalloc(sizeof(*pages) * *nr_pages, GFP_NOFS);
245 if (!pages)
246 return ERR_PTR(-ENOMEM);
247
248 BUG_ON(list_empty(page_list));
249 next_index = list_entry(page_list->prev, struct page, lru)->index;
250 list_for_each_entry_reverse(page, page_list, lru) {
251 if (page->index == next_index) {
252 dout("readpages page %d %p\n", contig_pages, page);
253 pages[contig_pages] = page;
254 contig_pages++;
255 next_index++;
256 } else {
257 break;
258 }
259 }
260 *nr_pages = contig_pages;
261 return pages;
262}
263
264/*
265 * Read multiple pages. Leave pages we don't read + unlock in page_list;
266 * the caller (VM) cleans them up.
267 */
268static int ceph_readpages(struct file *file, struct address_space *mapping,
269 struct list_head *page_list, unsigned nr_pages)
270{
271 struct inode *inode = file->f_dentry->d_inode;
272 struct ceph_inode_info *ci = ceph_inode(inode);
273 struct ceph_osd_client *osdc = &ceph_inode_to_client(inode)->osdc;
274 int rc = 0;
275 struct page **pages;
276 struct pagevec pvec;
277 loff_t offset;
278 u64 len;
279
280 dout("readpages %p file %p nr_pages %d\n",
281 inode, file, nr_pages);
282
283 pages = page_vector_from_list(page_list, &nr_pages);
284 if (IS_ERR(pages))
285 return PTR_ERR(pages);
286
287 /* guess read extent */
288 offset = pages[0]->index << PAGE_CACHE_SHIFT;
289 len = nr_pages << PAGE_CACHE_SHIFT;
290 rc = ceph_osdc_readpages(osdc, ceph_vino(inode), &ci->i_layout,
291 offset, &len,
292 ci->i_truncate_seq, ci->i_truncate_size,
293 pages, nr_pages);
294 if (rc == -ENOENT)
295 rc = 0;
296 if (rc < 0)
297 goto out;
298
299 /* set uptodate and add to lru in pagevec-sized chunks */
300 pagevec_init(&pvec, 0);
301 for (; !list_empty(page_list) && len > 0;
302 rc -= PAGE_CACHE_SIZE, len -= PAGE_CACHE_SIZE) {
303 struct page *page =
304 list_entry(page_list->prev, struct page, lru);
305
306 list_del(&page->lru);
307
308 if (rc < (int)PAGE_CACHE_SIZE) {
309 /* zero (remainder of) page */
310 int s = rc < 0 ? 0 : rc;
311 zero_user_segment(page, s, PAGE_CACHE_SIZE);
312 }
313
314 if (add_to_page_cache(page, mapping, page->index, GFP_NOFS)) {
315 page_cache_release(page);
316 dout("readpages %p add_to_page_cache failed %p\n",
317 inode, page);
318 continue;
319 }
320 dout("readpages %p adding %p idx %lu\n", inode, page,
321 page->index);
322 flush_dcache_page(page);
323 SetPageUptodate(page);
324 unlock_page(page);
325 if (pagevec_add(&pvec, page) == 0)
326 pagevec_lru_add_file(&pvec); /* add to lru */
327 }
328 pagevec_lru_add_file(&pvec);
329 rc = 0;
330
331out:
332 kfree(pages);
333 return rc;
334}
335
336/*
337 * Get ref for the oldest snapc for an inode with dirty data... that is, the
338 * only snap context we are allowed to write back.
339 *
340 * Caller holds i_lock.
341 */
342static struct ceph_snap_context *__get_oldest_context(struct inode *inode,
343 u64 *snap_size)
344{
345 struct ceph_inode_info *ci = ceph_inode(inode);
346 struct ceph_snap_context *snapc = NULL;
347 struct ceph_cap_snap *capsnap = NULL;
348
349 list_for_each_entry(capsnap, &ci->i_cap_snaps, ci_item) {
350 dout(" cap_snap %p snapc %p has %d dirty pages\n", capsnap,
351 capsnap->context, capsnap->dirty_pages);
352 if (capsnap->dirty_pages) {
353 snapc = ceph_get_snap_context(capsnap->context);
354 if (snap_size)
355 *snap_size = capsnap->size;
356 break;
357 }
358 }
359 if (!snapc && ci->i_snap_realm) {
360 snapc = ceph_get_snap_context(ci->i_snap_realm->cached_context);
361 dout(" head snapc %p has %d dirty pages\n",
362 snapc, ci->i_wrbuffer_ref_head);
363 }
364 return snapc;
365}
366
367static struct ceph_snap_context *get_oldest_context(struct inode *inode,
368 u64 *snap_size)
369{
370 struct ceph_snap_context *snapc = NULL;
371
372 spin_lock(&inode->i_lock);
373 snapc = __get_oldest_context(inode, snap_size);
374 spin_unlock(&inode->i_lock);
375 return snapc;
376}
377
378/*
379 * Write a single page, but leave the page locked.
380 *
381 * If we get a write error, set the page error bit, but still adjust the
382 * dirty page accounting (i.e., page is no longer dirty).
383 */
384static int writepage_nounlock(struct page *page, struct writeback_control *wbc)
385{
386 struct inode *inode;
387 struct ceph_inode_info *ci;
388 struct ceph_client *client;
389 struct ceph_osd_client *osdc;
390 loff_t page_off = page->index << PAGE_CACHE_SHIFT;
391 int len = PAGE_CACHE_SIZE;
392 loff_t i_size;
393 int err = 0;
394 struct ceph_snap_context *snapc;
395 u64 snap_size = 0;
396 long writeback_stat;
397
398 dout("writepage %p idx %lu\n", page, page->index);
399
400 if (!page->mapping || !page->mapping->host) {
401 dout("writepage %p - no mapping\n", page);
402 return -EFAULT;
403 }
404 inode = page->mapping->host;
405 ci = ceph_inode(inode);
406 client = ceph_inode_to_client(inode);
407 osdc = &client->osdc;
408
409 /* verify this is a writeable snap context */
410 snapc = (void *)page->private;
411 if (snapc == NULL) {
412 dout("writepage %p page %p not dirty?\n", inode, page);
413 goto out;
414 }
415 if (snapc != get_oldest_context(inode, &snap_size)) {
416 dout("writepage %p page %p snapc %p not writeable - noop\n",
417 inode, page, (void *)page->private);
418 /* we should only noop if called by kswapd */
419 WARN_ON((current->flags & PF_MEMALLOC) == 0);
420 goto out;
421 }
422
423 /* is this a partial page at end of file? */
424 if (snap_size)
425 i_size = snap_size;
426 else
427 i_size = i_size_read(inode);
428 if (i_size < page_off + len)
429 len = i_size - page_off;
430
431 dout("writepage %p page %p index %lu on %llu~%u\n",
432 inode, page, page->index, page_off, len);
433
434 writeback_stat = atomic_long_inc_return(&client->writeback_count);
435 if (writeback_stat >
436 CONGESTION_ON_THRESH(client->mount_args->congestion_kb))
437 set_bdi_congested(&client->backing_dev_info, BLK_RW_ASYNC);
438
439 set_page_writeback(page);
440 err = ceph_osdc_writepages(osdc, ceph_vino(inode),
441 &ci->i_layout, snapc,
442 page_off, len,
443 ci->i_truncate_seq, ci->i_truncate_size,
444 &inode->i_mtime,
445 &page, 1, 0, 0, true);
446 if (err < 0) {
447 dout("writepage setting page/mapping error %d %p\n", err, page);
448 SetPageError(page);
449 mapping_set_error(&inode->i_data, err);
450 if (wbc)
451 wbc->pages_skipped++;
452 } else {
453 dout("writepage cleaned page %p\n", page);
454 err = 0; /* vfs expects us to return 0 */
455 }
456 page->private = 0;
457 ClearPagePrivate(page);
458 end_page_writeback(page);
459 ceph_put_wrbuffer_cap_refs(ci, 1, snapc);
460 ceph_put_snap_context(snapc);
461out:
462 return err;
463}
464
465static int ceph_writepage(struct page *page, struct writeback_control *wbc)
466{
467 int err;
468 struct inode *inode = page->mapping->host;
469 BUG_ON(!inode);
470 igrab(inode);
471 err = writepage_nounlock(page, wbc);
472 unlock_page(page);
473 iput(inode);
474 return err;
475}
476
477
478/*
479 * lame release_pages helper. release_pages() isn't exported to
480 * modules.
481 */
482static void ceph_release_pages(struct page **pages, int num)
483{
484 struct pagevec pvec;
485 int i;
486
487 pagevec_init(&pvec, 0);
488 for (i = 0; i < num; i++) {
489 if (pagevec_add(&pvec, pages[i]) == 0)
490 pagevec_release(&pvec);
491 }
492 pagevec_release(&pvec);
493}
494
495
496/*
497 * async writeback completion handler.
498 *
499 * If we get an error, set the mapping error bit, but not the individual
500 * page error bits.
501 */
502static void writepages_finish(struct ceph_osd_request *req,
503 struct ceph_msg *msg)
504{
505 struct inode *inode = req->r_inode;
506 struct ceph_osd_reply_head *replyhead;
507 struct ceph_osd_op *op;
508 struct ceph_inode_info *ci = ceph_inode(inode);
509 unsigned wrote;
510 struct page *page;
511 int i;
512 struct ceph_snap_context *snapc = req->r_snapc;
513 struct address_space *mapping = inode->i_mapping;
514 struct writeback_control *wbc = req->r_wbc;
515 __s32 rc = -EIO;
516 u64 bytes = 0;
517 struct ceph_client *client = ceph_inode_to_client(inode);
518 long writeback_stat;
519 unsigned issued = __ceph_caps_issued(ci, NULL);
520
521 /* parse reply */
522 replyhead = msg->front.iov_base;
523 WARN_ON(le32_to_cpu(replyhead->num_ops) == 0);
524 op = (void *)(replyhead + 1);
525 rc = le32_to_cpu(replyhead->result);
526 bytes = le64_to_cpu(op->extent.length);
527
528 if (rc >= 0) {
529 /*
530 * Assume we wrote the pages we originally sent. The
531 * osd might reply with fewer pages if our writeback
532 * raced with a truncation and was adjusted at the osd,
533 * so don't believe the reply.
534 */
535 wrote = req->r_num_pages;
536 } else {
537 wrote = 0;
538 mapping_set_error(mapping, rc);
539 }
540 dout("writepages_finish %p rc %d bytes %llu wrote %d (pages)\n",
541 inode, rc, bytes, wrote);
542
543 /* clean all pages */
544 for (i = 0; i < req->r_num_pages; i++) {
545 page = req->r_pages[i];
546 BUG_ON(!page);
547 WARN_ON(!PageUptodate(page));
548
549 writeback_stat =
550 atomic_long_dec_return(&client->writeback_count);
551 if (writeback_stat <
552 CONGESTION_OFF_THRESH(client->mount_args->congestion_kb))
553 clear_bdi_congested(&client->backing_dev_info,
554 BLK_RW_ASYNC);
555
556 if (i >= wrote) {
557 dout("inode %p skipping page %p\n", inode, page);
558 wbc->pages_skipped++;
559 }
560 page->private = 0;
561 ClearPagePrivate(page);
562 ceph_put_snap_context(snapc);
563 dout("unlocking %d %p\n", i, page);
564 end_page_writeback(page);
565
566 /*
567 * We lost the cache cap, need to truncate the page before
568 * it is unlocked, otherwise we'd truncate it later in the
569 * page truncation thread, possibly losing some data that
570 * raced its way in
571 */
572 if ((issued & CEPH_CAP_FILE_CACHE) == 0)
573 generic_error_remove_page(inode->i_mapping, page);
574
575 unlock_page(page);
576 }
577 dout("%p wrote+cleaned %d pages\n", inode, wrote);
578 ceph_put_wrbuffer_cap_refs(ci, req->r_num_pages, snapc);
579
580 ceph_release_pages(req->r_pages, req->r_num_pages);
581 if (req->r_pages_from_pool)
582 mempool_free(req->r_pages,
583 ceph_client(inode->i_sb)->wb_pagevec_pool);
584 else
585 kfree(req->r_pages);
586 ceph_osdc_put_request(req);
587}
588
589/*
590 * allocate a page vec, either directly, or if necessary, via a the
591 * mempool. we avoid the mempool if we can because req->r_num_pages
592 * may be less than the maximum write size.
593 */
594static void alloc_page_vec(struct ceph_client *client,
595 struct ceph_osd_request *req)
596{
597 req->r_pages = kmalloc(sizeof(struct page *) * req->r_num_pages,
598 GFP_NOFS);
599 if (!req->r_pages) {
600 req->r_pages = mempool_alloc(client->wb_pagevec_pool, GFP_NOFS);
601 req->r_pages_from_pool = 1;
602 WARN_ON(!req->r_pages);
603 }
604}
605
606/*
607 * initiate async writeback
608 */
609static int ceph_writepages_start(struct address_space *mapping,
610 struct writeback_control *wbc)
611{
612 struct inode *inode = mapping->host;
613 struct backing_dev_info *bdi = mapping->backing_dev_info;
614 struct ceph_inode_info *ci = ceph_inode(inode);
615 struct ceph_client *client;
616 pgoff_t index, start, end;
617 int range_whole = 0;
618 int should_loop = 1;
619 pgoff_t max_pages = 0, max_pages_ever = 0;
620 struct ceph_snap_context *snapc = NULL, *last_snapc = NULL;
621 struct pagevec pvec;
622 int done = 0;
623 int rc = 0;
624 unsigned wsize = 1 << inode->i_blkbits;
625 struct ceph_osd_request *req = NULL;
626 int do_sync;
627 u64 snap_size = 0;
628
629 /*
630 * Include a 'sync' in the OSD request if this is a data
631 * integrity write (e.g., O_SYNC write or fsync()), or if our
632 * cap is being revoked.
633 */
634 do_sync = wbc->sync_mode == WB_SYNC_ALL;
635 if (ceph_caps_revoking(ci, CEPH_CAP_FILE_BUFFER))
636 do_sync = 1;
637 dout("writepages_start %p dosync=%d (mode=%s)\n",
638 inode, do_sync,
639 wbc->sync_mode == WB_SYNC_NONE ? "NONE" :
640 (wbc->sync_mode == WB_SYNC_ALL ? "ALL" : "HOLD"));
641
642 client = ceph_inode_to_client(inode);
643 if (client->mount_state == CEPH_MOUNT_SHUTDOWN) {
644 pr_warning("writepage_start %p on forced umount\n", inode);
645 return -EIO; /* we're in a forced umount, don't write! */
646 }
647 if (client->mount_args->wsize && client->mount_args->wsize < wsize)
648 wsize = client->mount_args->wsize;
649 if (wsize < PAGE_CACHE_SIZE)
650 wsize = PAGE_CACHE_SIZE;
651 max_pages_ever = wsize >> PAGE_CACHE_SHIFT;
652
653 pagevec_init(&pvec, 0);
654
655 /* ?? */
656 if (wbc->nonblocking && bdi_write_congested(bdi)) {
657 dout(" writepages congested\n");
658 wbc->encountered_congestion = 1;
659 goto out_final;
660 }
661
662 /* where to start/end? */
663 if (wbc->range_cyclic) {
664 start = mapping->writeback_index; /* Start from prev offset */
665 end = -1;
666 dout(" cyclic, start at %lu\n", start);
667 } else {
668 start = wbc->range_start >> PAGE_CACHE_SHIFT;
669 end = wbc->range_end >> PAGE_CACHE_SHIFT;
670 if (wbc->range_start == 0 && wbc->range_end == LLONG_MAX)
671 range_whole = 1;
672 should_loop = 0;
673 dout(" not cyclic, %lu to %lu\n", start, end);
674 }
675 index = start;
676
677retry:
678 /* find oldest snap context with dirty data */
679 ceph_put_snap_context(snapc);
680 snapc = get_oldest_context(inode, &snap_size);
681 if (!snapc) {
682 /* hmm, why does writepages get called when there
683 is no dirty data? */
684 dout(" no snap context with dirty data?\n");
685 goto out;
686 }
687 dout(" oldest snapc is %p seq %lld (%d snaps)\n",
688 snapc, snapc->seq, snapc->num_snaps);
689 if (last_snapc && snapc != last_snapc) {
690 /* if we switched to a newer snapc, restart our scan at the
691 * start of the original file range. */
692 dout(" snapc differs from last pass, restarting at %lu\n",
693 index);
694 index = start;
695 }
696 last_snapc = snapc;
697
698 while (!done && index <= end) {
699 unsigned i;
700 int first;
701 pgoff_t next;
702 int pvec_pages, locked_pages;
703 struct page *page;
704 int want;
705 u64 offset, len;
706 struct ceph_osd_request_head *reqhead;
707 struct ceph_osd_op *op;
708 long writeback_stat;
709
710 next = 0;
711 locked_pages = 0;
712 max_pages = max_pages_ever;
713
714get_more_pages:
715 first = -1;
716 want = min(end - index,
717 min((pgoff_t)PAGEVEC_SIZE,
718 max_pages - (pgoff_t)locked_pages) - 1)
719 + 1;
720 pvec_pages = pagevec_lookup_tag(&pvec, mapping, &index,
721 PAGECACHE_TAG_DIRTY,
722 want);
723 dout("pagevec_lookup_tag got %d\n", pvec_pages);
724 if (!pvec_pages && !locked_pages)
725 break;
726 for (i = 0; i < pvec_pages && locked_pages < max_pages; i++) {
727 page = pvec.pages[i];
728 dout("? %p idx %lu\n", page, page->index);
729 if (locked_pages == 0)
730 lock_page(page); /* first page */
731 else if (!trylock_page(page))
732 break;
733
734 /* only dirty pages, or our accounting breaks */
735 if (unlikely(!PageDirty(page)) ||
736 unlikely(page->mapping != mapping)) {
737 dout("!dirty or !mapping %p\n", page);
738 unlock_page(page);
739 break;
740 }
741 if (!wbc->range_cyclic && page->index > end) {
742 dout("end of range %p\n", page);
743 done = 1;
744 unlock_page(page);
745 break;
746 }
747 if (next && (page->index != next)) {
748 dout("not consecutive %p\n", page);
749 unlock_page(page);
750 break;
751 }
752 if (wbc->sync_mode != WB_SYNC_NONE) {
753 dout("waiting on writeback %p\n", page);
754 wait_on_page_writeback(page);
755 }
756 if ((snap_size && page_offset(page) > snap_size) ||
757 (!snap_size &&
758 page_offset(page) > i_size_read(inode))) {
759 dout("%p page eof %llu\n", page, snap_size ?
760 snap_size : i_size_read(inode));
761 done = 1;
762 unlock_page(page);
763 break;
764 }
765 if (PageWriteback(page)) {
766 dout("%p under writeback\n", page);
767 unlock_page(page);
768 break;
769 }
770
771 /* only if matching snap context */
772 if (snapc != (void *)page->private) {
773 dout("page snapc %p != oldest %p\n",
774 (void *)page->private, snapc);
775 unlock_page(page);
776 if (!locked_pages)
777 continue; /* keep looking for snap */
778 break;
779 }
780
781 if (!clear_page_dirty_for_io(page)) {
782 dout("%p !clear_page_dirty_for_io\n", page);
783 unlock_page(page);
784 break;
785 }
786
787 /* ok */
788 if (locked_pages == 0) {
789 /* prepare async write request */
790 offset = page->index << PAGE_CACHE_SHIFT;
791 len = wsize;
792 req = ceph_osdc_new_request(&client->osdc,
793 &ci->i_layout,
794 ceph_vino(inode),
795 offset, &len,
796 CEPH_OSD_OP_WRITE,
797 CEPH_OSD_FLAG_WRITE |
798 CEPH_OSD_FLAG_ONDISK,
799 snapc, do_sync,
800 ci->i_truncate_seq,
801 ci->i_truncate_size,
802 &inode->i_mtime, true, 1);
803 max_pages = req->r_num_pages;
804
805 alloc_page_vec(client, req);
806 req->r_callback = writepages_finish;
807 req->r_inode = inode;
808 req->r_wbc = wbc;
809 }
810
811 /* note position of first page in pvec */
812 if (first < 0)
813 first = i;
814 dout("%p will write page %p idx %lu\n",
815 inode, page, page->index);
816
817 writeback_stat = atomic_long_inc_return(&client->writeback_count);
818 if (writeback_stat > CONGESTION_ON_THRESH(client->mount_args->congestion_kb)) {
819 set_bdi_congested(&client->backing_dev_info, BLK_RW_ASYNC);
820 }
821
822 set_page_writeback(page);
823 req->r_pages[locked_pages] = page;
824 locked_pages++;
825 next = page->index + 1;
826 }
827
828 /* did we get anything? */
829 if (!locked_pages)
830 goto release_pvec_pages;
831 if (i) {
832 int j;
833 BUG_ON(!locked_pages || first < 0);
834
835 if (pvec_pages && i == pvec_pages &&
836 locked_pages < max_pages) {
837 dout("reached end pvec, trying for more\n");
838 pagevec_reinit(&pvec);
839 goto get_more_pages;
840 }
841
842 /* shift unused pages over in the pvec... we
843 * will need to release them below. */
844 for (j = i; j < pvec_pages; j++) {
845 dout(" pvec leftover page %p\n",
846 pvec.pages[j]);
847 pvec.pages[j-i+first] = pvec.pages[j];
848 }
849 pvec.nr -= i-first;
850 }
851
852 /* submit the write */
853 offset = req->r_pages[0]->index << PAGE_CACHE_SHIFT;
854 len = min((snap_size ? snap_size : i_size_read(inode)) - offset,
855 (u64)locked_pages << PAGE_CACHE_SHIFT);
856 dout("writepages got %d pages at %llu~%llu\n",
857 locked_pages, offset, len);
858
859 /* revise final length, page count */
860 req->r_num_pages = locked_pages;
861 reqhead = req->r_request->front.iov_base;
862 op = (void *)(reqhead + 1);
863 op->extent.length = cpu_to_le64(len);
864 op->payload_len = cpu_to_le32(len);
865 req->r_request->hdr.data_len = cpu_to_le32(len);
866
867 ceph_osdc_start_request(&client->osdc, req, true);
868 req = NULL;
869
870 /* continue? */
871 index = next;
872 wbc->nr_to_write -= locked_pages;
873 if (wbc->nr_to_write <= 0)
874 done = 1;
875
876release_pvec_pages:
877 dout("pagevec_release on %d pages (%p)\n", (int)pvec.nr,
878 pvec.nr ? pvec.pages[0] : NULL);
879 pagevec_release(&pvec);
880
881 if (locked_pages && !done)
882 goto retry;
883 }
884
885 if (should_loop && !done) {
886 /* more to do; loop back to beginning of file */
887 dout("writepages looping back to beginning of file\n");
888 should_loop = 0;
889 index = 0;
890 goto retry;
891 }
892
893 if (wbc->range_cyclic || (range_whole && wbc->nr_to_write > 0))
894 mapping->writeback_index = index;
895
896out:
897 if (req)
898 ceph_osdc_put_request(req);
899 if (rc > 0)
900 rc = 0; /* vfs expects us to return 0 */
901 ceph_put_snap_context(snapc);
902 dout("writepages done, rc = %d\n", rc);
903out_final:
904 return rc;
905}
906
907
908
909/*
910 * See if a given @snapc is either writeable, or already written.
911 */
912static int context_is_writeable_or_written(struct inode *inode,
913 struct ceph_snap_context *snapc)
914{
915 struct ceph_snap_context *oldest = get_oldest_context(inode, NULL);
916 return !oldest || snapc->seq <= oldest->seq;
917}
918
919/*
920 * We are only allowed to write into/dirty the page if the page is
921 * clean, or already dirty within the same snap context.
922 */
923static int ceph_update_writeable_page(struct file *file,
924 loff_t pos, unsigned len,
925 struct page *page)
926{
927 struct inode *inode = file->f_dentry->d_inode;
928 struct ceph_inode_info *ci = ceph_inode(inode);
929 struct ceph_mds_client *mdsc = &ceph_inode_to_client(inode)->mdsc;
930 loff_t page_off = pos & PAGE_CACHE_MASK;
931 int pos_in_page = pos & ~PAGE_CACHE_MASK;
932 int end_in_page = pos_in_page + len;
933 loff_t i_size;
934 struct ceph_snap_context *snapc;
935 int r;
936
937retry_locked:
938 /* writepages currently holds page lock, but if we change that later, */
939 wait_on_page_writeback(page);
940
941 /* check snap context */
942 BUG_ON(!ci->i_snap_realm);
943 down_read(&mdsc->snap_rwsem);
944 BUG_ON(!ci->i_snap_realm->cached_context);
945 if (page->private &&
946 (void *)page->private != ci->i_snap_realm->cached_context) {
947 /*
948 * this page is already dirty in another (older) snap
949 * context! is it writeable now?
950 */
951 snapc = get_oldest_context(inode, NULL);
952 up_read(&mdsc->snap_rwsem);
953
954 if (snapc != (void *)page->private) {
955 dout(" page %p snapc %p not current or oldest\n",
956 page, (void *)page->private);
957 /*
958 * queue for writeback, and wait for snapc to
959 * be writeable or written
960 */
961 snapc = ceph_get_snap_context((void *)page->private);
962 unlock_page(page);
963 ceph_queue_writeback(inode);
964 wait_event_interruptible(ci->i_cap_wq,
965 context_is_writeable_or_written(inode, snapc));
966 ceph_put_snap_context(snapc);
967 return -EAGAIN;
968 }
969
970 /* yay, writeable, do it now (without dropping page lock) */
971 dout(" page %p snapc %p not current, but oldest\n",
972 page, snapc);
973 if (!clear_page_dirty_for_io(page))
974 goto retry_locked;
975 r = writepage_nounlock(page, NULL);
976 if (r < 0)
977 goto fail_nosnap;
978 goto retry_locked;
979 }
980
981 if (PageUptodate(page)) {
982 dout(" page %p already uptodate\n", page);
983 return 0;
984 }
985
986 /* full page? */
987 if (pos_in_page == 0 && len == PAGE_CACHE_SIZE)
988 return 0;
989
990 /* past end of file? */
991 i_size = inode->i_size; /* caller holds i_mutex */
992
993 if (i_size + len > inode->i_sb->s_maxbytes) {
994 /* file is too big */
995 r = -EINVAL;
996 goto fail;
997 }
998
999 if (page_off >= i_size ||
1000 (pos_in_page == 0 && (pos+len) >= i_size &&
1001 end_in_page - pos_in_page != PAGE_CACHE_SIZE)) {
1002 dout(" zeroing %p 0 - %d and %d - %d\n",
1003 page, pos_in_page, end_in_page, (int)PAGE_CACHE_SIZE);
1004 zero_user_segments(page,
1005 0, pos_in_page,
1006 end_in_page, PAGE_CACHE_SIZE);
1007 return 0;
1008 }
1009
1010 /* we need to read it. */
1011 up_read(&mdsc->snap_rwsem);
1012 r = readpage_nounlock(file, page);
1013 if (r < 0)
1014 goto fail_nosnap;
1015 goto retry_locked;
1016
1017fail:
1018 up_read(&mdsc->snap_rwsem);
1019fail_nosnap:
1020 unlock_page(page);
1021 return r;
1022}
1023
1024/*
1025 * We are only allowed to write into/dirty the page if the page is
1026 * clean, or already dirty within the same snap context.
1027 */
1028static int ceph_write_begin(struct file *file, struct address_space *mapping,
1029 loff_t pos, unsigned len, unsigned flags,
1030 struct page **pagep, void **fsdata)
1031{
1032 struct inode *inode = file->f_dentry->d_inode;
1033 struct page *page;
1034 pgoff_t index = pos >> PAGE_CACHE_SHIFT;
1035 int r;
1036
1037 do {
1038 /* get a page*/
1039 page = grab_cache_page_write_begin(mapping, index, 0);
1040 if (!page)
1041 return -ENOMEM;
1042 *pagep = page;
1043
1044 dout("write_begin file %p inode %p page %p %d~%d\n", file,
1045 inode, page, (int)pos, (int)len);
1046
1047 r = ceph_update_writeable_page(file, pos, len, page);
1048 } while (r == -EAGAIN);
1049
1050 return r;
1051}
1052
1053/*
1054 * we don't do anything in here that simple_write_end doesn't do
1055 * except adjust dirty page accounting and drop read lock on
1056 * mdsc->snap_rwsem.
1057 */
1058static int ceph_write_end(struct file *file, struct address_space *mapping,
1059 loff_t pos, unsigned len, unsigned copied,
1060 struct page *page, void *fsdata)
1061{
1062 struct inode *inode = file->f_dentry->d_inode;
1063 struct ceph_client *client = ceph_inode_to_client(inode);
1064 struct ceph_mds_client *mdsc = &client->mdsc;
1065 unsigned from = pos & (PAGE_CACHE_SIZE - 1);
1066 int check_cap = 0;
1067
1068 dout("write_end file %p inode %p page %p %d~%d (%d)\n", file,
1069 inode, page, (int)pos, (int)copied, (int)len);
1070
1071 /* zero the stale part of the page if we did a short copy */
1072 if (copied < len)
1073 zero_user_segment(page, from+copied, len);
1074
1075 /* did file size increase? */
1076 /* (no need for i_size_read(); we caller holds i_mutex */
1077 if (pos+copied > inode->i_size)
1078 check_cap = ceph_inode_set_size(inode, pos+copied);
1079
1080 if (!PageUptodate(page))
1081 SetPageUptodate(page);
1082
1083 set_page_dirty(page);
1084
1085 unlock_page(page);
1086 up_read(&mdsc->snap_rwsem);
1087 page_cache_release(page);
1088
1089 if (check_cap)
1090 ceph_check_caps(ceph_inode(inode), CHECK_CAPS_AUTHONLY, NULL);
1091
1092 return copied;
1093}
1094
1095/*
1096 * we set .direct_IO to indicate direct io is supported, but since we
1097 * intercept O_DIRECT reads and writes early, this function should
1098 * never get called.
1099 */
1100static ssize_t ceph_direct_io(int rw, struct kiocb *iocb,
1101 const struct iovec *iov,
1102 loff_t pos, unsigned long nr_segs)
1103{
1104 WARN_ON(1);
1105 return -EINVAL;
1106}
1107
1108const struct address_space_operations ceph_aops = {
1109 .readpage = ceph_readpage,
1110 .readpages = ceph_readpages,
1111 .writepage = ceph_writepage,
1112 .writepages = ceph_writepages_start,
1113 .write_begin = ceph_write_begin,
1114 .write_end = ceph_write_end,
1115 .set_page_dirty = ceph_set_page_dirty,
1116 .invalidatepage = ceph_invalidatepage,
1117 .releasepage = ceph_releasepage,
1118 .direct_IO = ceph_direct_io,
1119};
1120
1121
1122/*
1123 * vm ops
1124 */
1125
1126/*
1127 * Reuse write_begin here for simplicity.
1128 */
1129static int ceph_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf)
1130{
1131 struct inode *inode = vma->vm_file->f_dentry->d_inode;
1132 struct page *page = vmf->page;
1133 struct ceph_mds_client *mdsc = &ceph_inode_to_client(inode)->mdsc;
1134 loff_t off = page->index << PAGE_CACHE_SHIFT;
1135 loff_t size, len;
1136 int ret;
1137
1138 size = i_size_read(inode);
1139 if (off + PAGE_CACHE_SIZE <= size)
1140 len = PAGE_CACHE_SIZE;
1141 else
1142 len = size & ~PAGE_CACHE_MASK;
1143
1144 dout("page_mkwrite %p %llu~%llu page %p idx %lu\n", inode,
1145 off, len, page, page->index);
1146
1147 lock_page(page);
1148
1149 ret = VM_FAULT_NOPAGE;
1150 if ((off > size) ||
1151 (page->mapping != inode->i_mapping))
1152 goto out;
1153
1154 ret = ceph_update_writeable_page(vma->vm_file, off, len, page);
1155 if (ret == 0) {
1156 /* success. we'll keep the page locked. */
1157 set_page_dirty(page);
1158 up_read(&mdsc->snap_rwsem);
1159 ret = VM_FAULT_LOCKED;
1160 } else {
1161 if (ret == -ENOMEM)
1162 ret = VM_FAULT_OOM;
1163 else
1164 ret = VM_FAULT_SIGBUS;
1165 }
1166out:
1167 dout("page_mkwrite %p %llu~%llu = %d\n", inode, off, len, ret);
1168 if (ret != VM_FAULT_LOCKED)
1169 unlock_page(page);
1170 return ret;
1171}
1172
1173static struct vm_operations_struct ceph_vmops = {
1174 .fault = filemap_fault,
1175 .page_mkwrite = ceph_page_mkwrite,
1176};
1177
1178int ceph_mmap(struct file *file, struct vm_area_struct *vma)
1179{
1180 struct address_space *mapping = file->f_mapping;
1181
1182 if (!mapping->a_ops->readpage)
1183 return -ENOEXEC;
1184 file_accessed(file);
1185 vma->vm_ops = &ceph_vmops;
1186 vma->vm_flags |= VM_CAN_NONLINEAR;
1187 return 0;
1188}
fs/ceph/armor.c+99−0
@@ -0,0 +1,99 @@
1
2#include <linux/errno.h>
3
4/*
5 * base64 encode/decode.
6 */
7
8const char *pem_key = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
9
10static int encode_bits(int c)
11{
12 return pem_key[c];
13}
14
15static int decode_bits(char c)
16{
17 if (c >= 'A' && c <= 'Z')
18 return c - 'A';
19 if (c >= 'a' && c <= 'z')
20 return c - 'a' + 26;
21 if (c >= '0' && c <= '9')
22 return c - '0' + 52;
23 if (c == '+')
24 return 62;
25 if (c == '/')
26 return 63;
27 if (c == '=')
28 return 0; /* just non-negative, please */
29 return -EINVAL;
30}
31
32int ceph_armor(char *dst, const char *src, const char *end)
33{
34 int olen = 0;
35 int line = 0;
36
37 while (src < end) {
38 unsigned char a, b, c;
39
40 a = *src++;
41 *dst++ = encode_bits(a >> 2);
42 if (src < end) {
43 b = *src++;
44 *dst++ = encode_bits(((a & 3) << 4) | (b >> 4));
45 if (src < end) {
46 c = *src++;
47 *dst++ = encode_bits(((b & 15) << 2) |
48 (c >> 6));
49 *dst++ = encode_bits(c & 63);
50 } else {
51 *dst++ = encode_bits((b & 15) << 2);
52 *dst++ = '=';
53 }
54 } else {
55 *dst++ = encode_bits(((a & 3) << 4));
56 *dst++ = '=';
57 *dst++ = '=';
58 }
59 olen += 4;
60 line += 4;
61 if (line == 64) {
62 line = 0;
63 *(dst++) = '\n';
64 olen++;
65 }
66 }
67 return olen;
68}
69
70int ceph_unarmor(char *dst, const char *src, const char *end)
71{
72 int olen = 0;
73
74 while (src < end) {
75 int a, b, c, d;
76
77 if (src < end && src[0] == '\n')
78 src++;
79 if (src + 4 > end)
80 return -EINVAL;
81 a = decode_bits(src[0]);
82 b = decode_bits(src[1]);
83 c = decode_bits(src[2]);
84 d = decode_bits(src[3]);
85 if (a < 0 || b < 0 || c < 0 || d < 0)
86 return -EINVAL;
87
88 *dst++ = (a << 2) | (b >> 4);
89 if (src[2] == '=')
90 return olen + 1;
91 *dst++ = ((b & 15) << 4) | (c >> 2);
92 if (src[3] == '=')
93 return olen + 2;
94 *dst++ = ((c & 3) << 6) | d;
95 olen += 3;
96 src += 4;
97 }
98 return olen;
99}
fs/ceph/auth.c+257−0
@@ -0,0 +1,257 @@
1#include "ceph_debug.h"
2
3#include <linux/module.h>
4#include <linux/err.h>
5
6#include "types.h"
7#include "auth_none.h"
8#include "auth_x.h"
9#include "decode.h"
10#include "super.h"
11
12#include "messenger.h"
13
14/*
15 * get protocol handler
16 */
17static u32 supported_protocols[] = {
18 CEPH_AUTH_NONE,
19 CEPH_AUTH_CEPHX
20};
21
22int ceph_auth_init_protocol(struct ceph_auth_client *ac, int protocol)
23{
24 switch (protocol) {
25 case CEPH_AUTH_NONE:
26 return ceph_auth_none_init(ac);
27 case CEPH_AUTH_CEPHX:
28 return ceph_x_init(ac);
29 default:
30 return -ENOENT;
31 }
32}
33
34/*
35 * setup, teardown.
36 */
37struct ceph_auth_client *ceph_auth_init(const char *name, const char *secret)
38{
39 struct ceph_auth_client *ac;
40 int ret;
41
42 dout("auth_init name '%s' secret '%s'\n", name, secret);
43
44 ret = -ENOMEM;
45 ac = kzalloc(sizeof(*ac), GFP_NOFS);
46 if (!ac)
47 goto out;
48
49 ac->negotiating = true;
50 if (name)
51 ac->name = name;
52 else
53 ac->name = CEPH_AUTH_NAME_DEFAULT;
54 dout("auth_init name %s secret %s\n", ac->name, secret);
55 ac->secret = secret;
56 return ac;
57
58out:
59 return ERR_PTR(ret);
60}
61
62void ceph_auth_destroy(struct ceph_auth_client *ac)
63{
64 dout("auth_destroy %p\n", ac);
65 if (ac->ops)
66 ac->ops->destroy(ac);
67 kfree(ac);
68}
69
70/*
71 * Reset occurs when reconnecting to the monitor.
72 */
73void ceph_auth_reset(struct ceph_auth_client *ac)
74{
75 dout("auth_reset %p\n", ac);
76 if (ac->ops && !ac->negotiating)
77 ac->ops->reset(ac);
78 ac->negotiating = true;
79}
80
81int ceph_entity_name_encode(const char *name, void **p, void *end)
82{
83 int len = strlen(name);
84
85 if (*p + 2*sizeof(u32) + len > end)
86 return -ERANGE;
87 ceph_encode_32(p, CEPH_ENTITY_TYPE_CLIENT);
88 ceph_encode_32(p, len);
89 ceph_encode_copy(p, name, len);
90 return 0;
91}
92
93/*
94 * Initiate protocol negotiation with monitor. Include entity name
95 * and list supported protocols.
96 */
97int ceph_auth_build_hello(struct ceph_auth_client *ac, void *buf, size_t len)
98{
99 struct ceph_mon_request_header *monhdr = buf;
100 void *p = monhdr + 1, *end = buf + len, *lenp;
101 int i, num;
102 int ret;
103
104 dout("auth_build_hello\n");
105 monhdr->have_version = 0;
106 monhdr->session_mon = cpu_to_le16(-1);
107 monhdr->session_mon_tid = 0;
108
109 ceph_encode_32(&p, 0); /* no protocol, yet */
110
111 lenp = p;
112 p += sizeof(u32);
113
114 ceph_decode_need(&p, end, 1 + sizeof(u32), bad);
115 ceph_encode_8(&p, 1);
116 num = ARRAY_SIZE(supported_protocols);
117 ceph_encode_32(&p, num);
118 ceph_decode_need(&p, end, num * sizeof(u32), bad);
119 for (i = 0; i < num; i++)
120 ceph_encode_32(&p, supported_protocols[i]);
121
122 ret = ceph_entity_name_encode(ac->name, &p, end);
123 if (ret < 0)
124 return ret;
125 ceph_decode_need(&p, end, sizeof(u64), bad);
126 ceph_encode_64(&p, ac->global_id);
127
128 ceph_encode_32(&lenp, p - lenp - sizeof(u32));
129 return p - buf;
130
131bad:
132 return -ERANGE;
133}
134
135int ceph_build_auth_request(struct ceph_auth_client *ac,
136 void *msg_buf, size_t msg_len)
137{
138 struct ceph_mon_request_header *monhdr = msg_buf;
139 void *p = monhdr + 1;
140 void *end = msg_buf + msg_len;
141 int ret;
142
143 monhdr->have_version = 0;
144 monhdr->session_mon = cpu_to_le16(-1);
145 monhdr->session_mon_tid = 0;
146
147 ceph_encode_32(&p, ac->protocol);
148
149 ret = ac->ops->build_request(ac, p + sizeof(u32), end);
150 if (ret < 0) {
151 pr_err("error %d building request\n", ret);
152 return ret;
153 }
154 dout(" built request %d bytes\n", ret);
155 ceph_encode_32(&p, ret);
156 return p + ret - msg_buf;
157}
158
159/*
160 * Handle auth message from monitor.
161 */
162int ceph_handle_auth_reply(struct ceph_auth_client *ac,
163 void *buf, size_t len,
164 void *reply_buf, size_t reply_len)
165{
166 void *p = buf;
167 void *end = buf + len;
168 int protocol;
169 s32 result;
170 u64 global_id;
171 void *payload, *payload_end;
172 int payload_len;
173 char *result_msg;
174 int result_msg_len;
175 int ret = -EINVAL;
176
177 dout("handle_auth_reply %p %p\n", p, end);
178 ceph_decode_need(&p, end, sizeof(u32) * 3 + sizeof(u64), bad);
179 protocol = ceph_decode_32(&p);
180 result = ceph_decode_32(&p);
181 global_id = ceph_decode_64(&p);
182 payload_len = ceph_decode_32(&p);
183 payload = p;
184 p += payload_len;
185 ceph_decode_need(&p, end, sizeof(u32), bad);
186 result_msg_len = ceph_decode_32(&p);
187 result_msg = p;
188 p += result_msg_len;
189 if (p != end)
190 goto bad;
191
192 dout(" result %d '%.*s' gid %llu len %d\n", result, result_msg_len,
193 result_msg, global_id, payload_len);
194
195 payload_end = payload + payload_len;
196
197 if (global_id && ac->global_id != global_id) {
198 dout(" set global_id %lld -> %lld\n", ac->global_id, global_id);
199 ac->global_id = global_id;
200 }
201
202 if (ac->negotiating) {
203 /* server does not support our protocols? */
204 if (!protocol && result < 0) {
205 ret = result;
206 goto out;
207 }
208 /* set up (new) protocol handler? */
209 if (ac->protocol && ac->protocol != protocol) {
210 ac->ops->destroy(ac);
211 ac->protocol = 0;
212 ac->ops = NULL;
213 }
214 if (ac->protocol != protocol) {
215 ret = ceph_auth_init_protocol(ac, protocol);
216 if (ret) {
217 pr_err("error %d on auth protocol %d init\n",
218 ret, protocol);
219 goto out;
220 }
221 }
222
223 ac->negotiating = false;
224 }
225
226 ret = ac->ops->handle_reply(ac, result, payload, payload_end);
227 if (ret == -EAGAIN) {
228 return ceph_build_auth_request(ac, reply_buf, reply_len);
229 } else if (ret) {
230 pr_err("authentication error %d\n", ret);
231 return ret;
232 }
233 return 0;
234
235bad:
236 pr_err("failed to decode auth msg\n");
237out:
238 return ret;
239}
240
241int ceph_build_auth(struct ceph_auth_client *ac,
242 void *msg_buf, size_t msg_len)
243{
244 if (!ac->protocol)
245 return ceph_auth_build_hello(ac, msg_buf, msg_len);
246 BUG_ON(!ac->ops);
247 if (!ac->ops->is_authenticated(ac))
248 return ceph_build_auth_request(ac, msg_buf, msg_len);
249 return 0;
250}
251
252int ceph_auth_is_authenticated(struct ceph_auth_client *ac)
253{
254 if (!ac->ops)
255 return 0;
256 return ac->ops->is_authenticated(ac);
257}
fs/ceph/auth.h+84−0
@@ -0,0 +1,84 @@
1#ifndef _FS_CEPH_AUTH_H
2#define _FS_CEPH_AUTH_H
3
4#include "types.h"
5#include "buffer.h"
6
7/*
8 * Abstract interface for communicating with the authenticate module.
9 * There is some handshake that takes place between us and the monitor
10 * to acquire the necessary keys. These are used to generate an
11 * 'authorizer' that we use when connecting to a service (mds, osd).
12 */
13
14struct ceph_auth_client;
15struct ceph_authorizer;
16
17struct ceph_auth_client_ops {
18 /*
19 * true if we are authenticated and can connect to
20 * services.
21 */
22 int (*is_authenticated)(struct ceph_auth_client *ac);
23
24 /*
25 * build requests and process replies during monitor
26 * handshake. if handle_reply returns -EAGAIN, we build
27 * another request.
28 */
29 int (*build_request)(struct ceph_auth_client *ac, void *buf, void *end);
30 int (*handle_reply)(struct ceph_auth_client *ac, int result,
31 void *buf, void *end);
32
33 /*
34 * Create authorizer for connecting to a service, and verify
35 * the response to authenticate the service.
36 */
37 int (*create_authorizer)(struct ceph_auth_client *ac, int peer_type,
38 struct ceph_authorizer **a,
39 void **buf, size_t *len,
40 void **reply_buf, size_t *reply_len);
41 int (*verify_authorizer_reply)(struct ceph_auth_client *ac,
42 struct ceph_authorizer *a, size_t len);
43 void (*destroy_authorizer)(struct ceph_auth_client *ac,
44 struct ceph_authorizer *a);
45 void (*invalidate_authorizer)(struct ceph_auth_client *ac,
46 int peer_type);
47
48 /* reset when we (re)connect to a monitor */
49 void (*reset)(struct ceph_auth_client *ac);
50
51 void (*destroy)(struct ceph_auth_client *ac);
52};
53
54struct ceph_auth_client {
55 u32 protocol; /* CEPH_AUTH_* */
56 void *private; /* for use by protocol implementation */
57 const struct ceph_auth_client_ops *ops; /* null iff protocol==0 */
58
59 bool negotiating; /* true if negotiating protocol */
60 const char *name; /* entity name */
61 u64 global_id; /* our unique id in system */
62 const char *secret; /* our secret key */
63 unsigned want_keys; /* which services we want */
64};
65
66extern struct ceph_auth_client *ceph_auth_init(const char *name,
67 const char *secret);
68extern void ceph_auth_destroy(struct ceph_auth_client *ac);
69
70extern void ceph_auth_reset(struct ceph_auth_client *ac);
71
72extern int ceph_auth_build_hello(struct ceph_auth_client *ac,
73 void *buf, size_t len);
74extern int ceph_handle_auth_reply(struct ceph_auth_client *ac,
75 void *buf, size_t len,
76 void *reply_buf, size_t reply_len);
77extern int ceph_entity_name_encode(const char *name, void **p, void *end);
78
79extern int ceph_build_auth(struct ceph_auth_client *ac,
80 void *msg_buf, size_t msg_len);
81
82extern int ceph_auth_is_authenticated(struct ceph_auth_client *ac);
83
84#endif
fs/ceph/auth_none.c+121−0
@@ -0,0 +1,121 @@
1
2#include "ceph_debug.h"
3
4#include <linux/err.h>
5#include <linux/module.h>
6#include <linux/random.h>
7
8#include "auth_none.h"
9#include "auth.h"
10#include "decode.h"
11
12static void reset(struct ceph_auth_client *ac)
13{
14 struct ceph_auth_none_info *xi = ac->private;
15
16 xi->starting = true;
17 xi->built_authorizer = false;
18}
19
20static void destroy(struct ceph_auth_client *ac)
21{
22 kfree(ac->private);
23 ac->private = NULL;
24}
25
26static int is_authenticated(struct ceph_auth_client *ac)
27{
28 struct ceph_auth_none_info *xi = ac->private;
29
30 return !xi->starting;
31}
32
33/*
34 * the generic auth code decode the global_id, and we carry no actual
35 * authenticate state, so nothing happens here.
36 */
37static int handle_reply(struct ceph_auth_client *ac, int result,
38 void *buf, void *end)
39{
40 struct ceph_auth_none_info *xi = ac->private;
41
42 xi->starting = false;
43 return result;
44}
45
46/*
47 * build an 'authorizer' with our entity_name and global_id. we can
48 * reuse a single static copy since it is identical for all services
49 * we connect to.
50 */
51static int ceph_auth_none_create_authorizer(
52 struct ceph_auth_client *ac, int peer_type,
53 struct ceph_authorizer **a,
54 void **buf, size_t *len,
55 void **reply_buf, size_t *reply_len)
56{
57 struct ceph_auth_none_info *ai = ac->private;
58 struct ceph_none_authorizer *au = &ai->au;
59 void *p, *end;
60 int ret;
61
62 if (!ai->built_authorizer) {
63 p = au->buf;
64 end = p + sizeof(au->buf);
65 ceph_encode_8(&p, 1);
66 ret = ceph_entity_name_encode(ac->name, &p, end - 8);
67 if (ret < 0)
68 goto bad;
69 ceph_decode_need(&p, end, sizeof(u64), bad2);
70 ceph_encode_64(&p, ac->global_id);
71 au->buf_len = p - (void *)au->buf;
72 ai->built_authorizer = true;
73 dout("built authorizer len %d\n", au->buf_len);
74 }
75
76 *a = (struct ceph_authorizer *)au;
77 *buf = au->buf;
78 *len = au->buf_len;
79 *reply_buf = au->reply_buf;
80 *reply_len = sizeof(au->reply_buf);
81 return 0;
82
83bad2:
84 ret = -ERANGE;
85bad:
86 return ret;
87}
88
89static void ceph_auth_none_destroy_authorizer(struct ceph_auth_client *ac,
90 struct ceph_authorizer *a)
91{
92 /* nothing to do */
93}
94
95static const struct ceph_auth_client_ops ceph_auth_none_ops = {
96 .reset = reset,
97 .destroy = destroy,
98 .is_authenticated = is_authenticated,
99 .handle_reply = handle_reply,
100 .create_authorizer = ceph_auth_none_create_authorizer,
101 .destroy_authorizer = ceph_auth_none_destroy_authorizer,
102};
103
104int ceph_auth_none_init(struct ceph_auth_client *ac)
105{
106 struct ceph_auth_none_info *xi;
107
108 dout("ceph_auth_none_init %p\n", ac);
109 xi = kzalloc(sizeof(*xi), GFP_NOFS);
110 if (!xi)
111 return -ENOMEM;
112
113 xi->starting = true;
114 xi->built_authorizer = false;
115
116 ac->protocol = CEPH_AUTH_NONE;
117 ac->private = xi;
118 ac->ops = &ceph_auth_none_ops;
119 return 0;
120}
121
fs/ceph/auth_none.h+28−0
@@ -0,0 +1,28 @@
1#ifndef _FS_CEPH_AUTH_NONE_H
2#define _FS_CEPH_AUTH_NONE_H
3
4#include "auth.h"
5
6/*
7 * null security mode.
8 *
9 * we use a single static authorizer that simply encodes our entity name
10 * and global id.
11 */
12
13struct ceph_none_authorizer {
14 char buf[128];
15 int buf_len;
16 char reply_buf[0];
17};
18
19struct ceph_auth_none_info {
20 bool starting;
21 bool built_authorizer;
22 struct ceph_none_authorizer au; /* we only need one; it's static */
23};
24
25extern int ceph_auth_none_init(struct ceph_auth_client *ac);
26
27#endif
28
fs/ceph/auth_x.c+656−0
@@ -0,0 +1,656 @@
1
2#include "ceph_debug.h"
3
4#include <linux/err.h>
5#include <linux/module.h>
6#include <linux/random.h>
7
8#include "auth_x.h"
9#include "auth_x_protocol.h"
10#include "crypto.h"
11#include "auth.h"
12#include "decode.h"
13
14struct kmem_cache *ceph_x_ticketbuf_cachep;
15
16#define TEMP_TICKET_BUF_LEN 256
17
18static void ceph_x_validate_tickets(struct ceph_auth_client *ac, int *pneed);
19
20static int ceph_x_is_authenticated(struct ceph_auth_client *ac)
21{
22 struct ceph_x_info *xi = ac->private;
23 int need;
24
25 ceph_x_validate_tickets(ac, &need);
26 dout("ceph_x_is_authenticated want=%d need=%d have=%d\n",
27 ac->want_keys, need, xi->have_keys);
28 return (ac->want_keys & xi->have_keys) == ac->want_keys;
29}
30
31static int ceph_x_encrypt(struct ceph_crypto_key *secret,
32 void *ibuf, int ilen, void *obuf, size_t olen)
33{
34 struct ceph_x_encrypt_header head = {
35 .struct_v = 1,
36 .magic = cpu_to_le64(CEPHX_ENC_MAGIC)
37 };
38 size_t len = olen - sizeof(u32);
39 int ret;
40
41 ret = ceph_encrypt2(secret, obuf + sizeof(u32), &len,
42 &head, sizeof(head), ibuf, ilen);
43 if (ret)
44 return ret;
45 ceph_encode_32(&obuf, len);
46 return len + sizeof(u32);
47}
48
49static int ceph_x_decrypt(struct ceph_crypto_key *secret,
50 void **p, void *end, void *obuf, size_t olen)
51{
52 struct ceph_x_encrypt_header head;
53 size_t head_len = sizeof(head);
54 int len, ret;
55
56 len = ceph_decode_32(p);
57 if (*p + len > end)
58 return -EINVAL;
59
60 dout("ceph_x_decrypt len %d\n", len);
61 ret = ceph_decrypt2(secret, &head, &head_len, obuf, &olen,
62 *p, len);
63 if (ret)
64 return ret;
65 if (head.struct_v != 1 || le64_to_cpu(head.magic) != CEPHX_ENC_MAGIC)
66 return -EPERM;
67 *p += len;
68 return olen;
69}
70
71/*
72 * get existing (or insert new) ticket handler
73 */
74struct ceph_x_ticket_handler *get_ticket_handler(struct ceph_auth_client *ac,
75 int service)
76{
77 struct ceph_x_ticket_handler *th;
78 struct ceph_x_info *xi = ac->private;
79 struct rb_node *parent = NULL, **p = &xi->ticket_handlers.rb_node;
80
81 while (*p) {
82 parent = *p;
83 th = rb_entry(parent, struct ceph_x_ticket_handler, node);
84 if (service < th->service)
85 p = &(*p)->rb_left;
86 else if (service > th->service)
87 p = &(*p)->rb_right;
88 else
89 return th;
90 }
91
92 /* add it */
93 th = kzalloc(sizeof(*th), GFP_NOFS);
94 if (!th)
95 return ERR_PTR(-ENOMEM);
96 th->service = service;
97 rb_link_node(&th->node, parent, p);
98 rb_insert_color(&th->node, &xi->ticket_handlers);
99 return th;
100}
101
102static void remove_ticket_handler(struct ceph_auth_client *ac,
103 struct ceph_x_ticket_handler *th)
104{
105 struct ceph_x_info *xi = ac->private;
106
107 dout("remove_ticket_handler %p %d\n", th, th->service);
108 rb_erase(&th->node, &xi->ticket_handlers);
109 ceph_crypto_key_destroy(&th->session_key);
110 if (th->ticket_blob)
111 ceph_buffer_put(th->ticket_blob);
112 kfree(th);
113}
114
115static int ceph_x_proc_ticket_reply(struct ceph_auth_client *ac,
116 struct ceph_crypto_key *secret,
117 void *buf, void *end)
118{
119 struct ceph_x_info *xi = ac->private;
120 int num;
121 void *p = buf;
122 int ret;
123 char *dbuf;
124 char *ticket_buf;
125 u8 struct_v;
126
127 dbuf = kmem_cache_alloc(ceph_x_ticketbuf_cachep, GFP_NOFS | GFP_ATOMIC);
128 if (!dbuf)
129 return -ENOMEM;
130
131 ret = -ENOMEM;
132 ticket_buf = kmem_cache_alloc(ceph_x_ticketbuf_cachep,
133 GFP_NOFS | GFP_ATOMIC);
134 if (!ticket_buf)
135 goto out_dbuf;
136
137 ceph_decode_need(&p, end, 1 + sizeof(u32), bad);
138 struct_v = ceph_decode_8(&p);
139 if (struct_v != 1)
140 goto bad;
141 num = ceph_decode_32(&p);
142 dout("%d tickets\n", num);
143 while (num--) {
144 int type;
145 u8 struct_v;
146 struct ceph_x_ticket_handler *th;
147 void *dp, *dend;
148 int dlen;
149 char is_enc;
150 struct timespec validity;
151 struct ceph_crypto_key old_key;
152 void *tp, *tpend;
153
154 ceph_decode_need(&p, end, sizeof(u32) + 1, bad);
155
156 type = ceph_decode_32(&p);
157 dout(" ticket type %d %s\n", type, ceph_entity_type_name(type));
158
159 struct_v = ceph_decode_8(&p);
160 if (struct_v != 1)
161 goto bad;
162
163 th = get_ticket_handler(ac, type);
164 if (IS_ERR(th)) {
165 ret = PTR_ERR(th);
166 goto out;
167 }
168
169 /* blob for me */
170 dlen = ceph_x_decrypt(secret, &p, end, dbuf,
171 TEMP_TICKET_BUF_LEN);
172 if (dlen <= 0) {
173 ret = dlen;
174 goto out;
175 }
176 dout(" decrypted %d bytes\n", dlen);
177 dend = dbuf + dlen;
178 dp = dbuf;
179
180 struct_v = ceph_decode_8(&dp);
181 if (struct_v != 1)
182 goto bad;
183
184 memcpy(&old_key, &th->session_key, sizeof(old_key));
185 ret = ceph_crypto_key_decode(&th->session_key, &dp, dend);
186 if (ret)
187 goto out;
188
189 ceph_decode_copy(&dp, &th->validity, sizeof(th->validity));
190 ceph_decode_timespec(&validity, &th->validity);
191 th->expires = get_seconds() + validity.tv_sec;
192 th->renew_after = th->expires - (validity.tv_sec / 4);
193 dout(" expires=%lu renew_after=%lu\n", th->expires,
194 th->renew_after);
195
196 /* ticket blob for service */
197 ceph_decode_8_safe(&p, end, is_enc, bad);
198 tp = ticket_buf;
199 if (is_enc) {
200 /* encrypted */
201 dout(" encrypted ticket\n");
202 dlen = ceph_x_decrypt(&old_key, &p, end, ticket_buf,
203 TEMP_TICKET_BUF_LEN);
204 if (dlen < 0) {
205 ret = dlen;
206 goto out;
207 }
208 dlen = ceph_decode_32(&tp);
209 } else {
210 /* unencrypted */
211 ceph_decode_32_safe(&p, end, dlen, bad);
212 ceph_decode_need(&p, end, dlen, bad);
213 ceph_decode_copy(&p, ticket_buf, dlen);
214 }
215 tpend = tp + dlen;
216 dout(" ticket blob is %d bytes\n", dlen);
217 ceph_decode_need(&tp, tpend, 1 + sizeof(u64), bad);
218 struct_v = ceph_decode_8(&tp);
219 th->secret_id = ceph_decode_64(&tp);
220 ret = ceph_decode_buffer(&th->ticket_blob, &tp, tpend);
221 if (ret)
222 goto out;
223 dout(" got ticket service %d (%s) secret_id %lld len %d\n",
224 type, ceph_entity_type_name(type), th->secret_id,
225 (int)th->ticket_blob->vec.iov_len);
226 xi->have_keys |= th->service;
227 }
228
229 ret = 0;
230out:
231 kmem_cache_free(ceph_x_ticketbuf_cachep, ticket_buf);
232out_dbuf:
233 kmem_cache_free(ceph_x_ticketbuf_cachep, dbuf);
234 return ret;
235
236bad:
237 ret = -EINVAL;
238 goto out;
239}
240
241static int ceph_x_build_authorizer(struct ceph_auth_client *ac,
242 struct ceph_x_ticket_handler *th,
243 struct ceph_x_authorizer *au)
244{
245 int len;
246 struct ceph_x_authorize_a *msg_a;
247 struct ceph_x_authorize_b msg_b;
248 void *p, *end;
249 int ret;
250 int ticket_blob_len =
251 (th->ticket_blob ? th->ticket_blob->vec.iov_len : 0);
252
253 dout("build_authorizer for %s %p\n",
254 ceph_entity_type_name(th->service), au);
255
256 len = sizeof(*msg_a) + sizeof(msg_b) + sizeof(u32) +
257 ticket_blob_len + 16;
258 dout(" need len %d\n", len);
259 if (au->buf && au->buf->alloc_len < len) {
260 ceph_buffer_put(au->buf);
261 au->buf = NULL;
262 }
263 if (!au->buf) {
264 au->buf = ceph_buffer_new(len, GFP_NOFS);
265 if (!au->buf)
266 return -ENOMEM;
267 }
268 au->service = th->service;
269
270 msg_a = au->buf->vec.iov_base;
271 msg_a->struct_v = 1;
272 msg_a->global_id = cpu_to_le64(ac->global_id);
273 msg_a->service_id = cpu_to_le32(th->service);
274 msg_a->ticket_blob.struct_v = 1;
275 msg_a->ticket_blob.secret_id = cpu_to_le64(th->secret_id);
276 msg_a->ticket_blob.blob_len = cpu_to_le32(ticket_blob_len);
277 if (ticket_blob_len) {
278 memcpy(msg_a->ticket_blob.blob, th->ticket_blob->vec.iov_base,
279 th->ticket_blob->vec.iov_len);
280 }
281 dout(" th %p secret_id %lld %lld\n", th, th->secret_id,
282 le64_to_cpu(msg_a->ticket_blob.secret_id));
283
284 p = msg_a + 1;
285 p += ticket_blob_len;
286 end = au->buf->vec.iov_base + au->buf->vec.iov_len;
287
288 get_random_bytes(&au->nonce, sizeof(au->nonce));
289 msg_b.struct_v = 1;
290 msg_b.nonce = cpu_to_le64(au->nonce);
291 ret = ceph_x_encrypt(&th->session_key, &msg_b, sizeof(msg_b),
292 p, end - p);
293 if (ret < 0)
294 goto out_buf;
295 p += ret;
296 au->buf->vec.iov_len = p - au->buf->vec.iov_base;
297 dout(" built authorizer nonce %llx len %d\n", au->nonce,
298 (int)au->buf->vec.iov_len);
299 return 0;
300
301out_buf:
302 ceph_buffer_put(au->buf);
303 au->buf = NULL;
304 return ret;
305}
306
307static int ceph_x_encode_ticket(struct ceph_x_ticket_handler *th,
308 void **p, void *end)
309{
310 ceph_decode_need(p, end, 1 + sizeof(u64), bad);
311 ceph_encode_8(p, 1);
312 ceph_encode_64(p, th->secret_id);
313 if (th->ticket_blob) {
314 const char *buf = th->ticket_blob->vec.iov_base;
315 u32 len = th->ticket_blob->vec.iov_len;
316
317 ceph_encode_32_safe(p, end, len, bad);
318 ceph_encode_copy_safe(p, end, buf, len, bad);
319 } else {
320 ceph_encode_32_safe(p, end, 0, bad);
321 }
322
323 return 0;
324bad:
325 return -ERANGE;
326}
327
328static void ceph_x_validate_tickets(struct ceph_auth_client *ac, int *pneed)
329{
330 int want = ac->want_keys;
331 struct ceph_x_info *xi = ac->private;
332 int service;
333
334 *pneed = ac->want_keys & ~(xi->have_keys);
335
336 for (service = 1; service <= want; service <<= 1) {
337 struct ceph_x_ticket_handler *th;
338
339 if (!(ac->want_keys & service))
340 continue;
341
342 if (*pneed & service)
343 continue;
344
345 th = get_ticket_handler(ac, service);
346
347 if (!th) {
348 *pneed |= service;
349 continue;
350 }
351
352 if (get_seconds() >= th->renew_after)
353 *pneed |= service;
354 if (get_seconds() >= th->expires)
355 xi->have_keys &= ~service;
356 }
357}
358
359
360static int ceph_x_build_request(struct ceph_auth_client *ac,
361 void *buf, void *end)
362{
363 struct ceph_x_info *xi = ac->private;
364 int need;
365 struct ceph_x_request_header *head = buf;
366 int ret;
367 struct ceph_x_ticket_handler *th =
368 get_ticket_handler(ac, CEPH_ENTITY_TYPE_AUTH);
369
370 ceph_x_validate_tickets(ac, &need);
371
372 dout("build_request want %x have %x need %x\n",
373 ac->want_keys, xi->have_keys, need);
374
375 if (need & CEPH_ENTITY_TYPE_AUTH) {
376 struct ceph_x_authenticate *auth = (void *)(head + 1);
377 void *p = auth + 1;
378 struct ceph_x_challenge_blob tmp;
379 char tmp_enc[40];
380 u64 *u;
381
382 if (p > end)
383 return -ERANGE;
384
385 dout(" get_auth_session_key\n");
386 head->op = cpu_to_le16(CEPHX_GET_AUTH_SESSION_KEY);
387
388 /* encrypt and hash */
389 get_random_bytes(&auth->client_challenge, sizeof(u64));
390 tmp.client_challenge = auth->client_challenge;
391 tmp.server_challenge = cpu_to_le64(xi->server_challenge);
392 ret = ceph_x_encrypt(&xi->secret, &tmp, sizeof(tmp),
393 tmp_enc, sizeof(tmp_enc));
394 if (ret < 0)
395 return ret;
396
397 auth->struct_v = 1;
398 auth->key = 0;
399 for (u = (u64 *)tmp_enc; u + 1 <= (u64 *)(tmp_enc + ret); u++)
400 auth->key ^= *u;
401 dout(" server_challenge %llx client_challenge %llx key %llx\n",
402 xi->server_challenge, le64_to_cpu(auth->client_challenge),
403 le64_to_cpu(auth->key));
404
405 /* now encode the old ticket if exists */
406 ret = ceph_x_encode_ticket(th, &p, end);
407 if (ret < 0)
408 return ret;
409
410 return p - buf;
411 }
412
413 if (need) {
414 void *p = head + 1;
415 struct ceph_x_service_ticket_request *req;
416
417 if (p > end)
418 return -ERANGE;
419 head->op = cpu_to_le16(CEPHX_GET_PRINCIPAL_SESSION_KEY);
420
421 BUG_ON(!th);
422 ret = ceph_x_build_authorizer(ac, th, &xi->auth_authorizer);
423 if (ret)
424 return ret;
425 ceph_encode_copy(&p, xi->auth_authorizer.buf->vec.iov_base,
426 xi->auth_authorizer.buf->vec.iov_len);
427
428 req = p;
429 req->keys = cpu_to_le32(need);
430 p += sizeof(*req);
431 return p - buf;
432 }
433
434 return 0;
435}
436
437static int ceph_x_handle_reply(struct ceph_auth_client *ac, int result,
438 void *buf, void *end)
439{
440 struct ceph_x_info *xi = ac->private;
441 struct ceph_x_reply_header *head = buf;
442 struct ceph_x_ticket_handler *th;
443 int len = end - buf;
444 int op;
445 int ret;
446
447 if (result)
448 return result; /* XXX hmm? */
449
450 if (xi->starting) {
451 /* it's a hello */
452 struct ceph_x_server_challenge *sc = buf;
453
454 if (len != sizeof(*sc))
455 return -EINVAL;
456 xi->server_challenge = le64_to_cpu(sc->server_challenge);
457 dout("handle_reply got server challenge %llx\n",
458 xi->server_challenge);
459 xi->starting = false;
460 xi->have_keys &= ~CEPH_ENTITY_TYPE_AUTH;
461 return -EAGAIN;
462 }
463
464 op = le32_to_cpu(head->op);
465 result = le32_to_cpu(head->result);
466 dout("handle_reply op %d result %d\n", op, result);
467 switch (op) {
468 case CEPHX_GET_AUTH_SESSION_KEY:
469 /* verify auth key */
470 ret = ceph_x_proc_ticket_reply(ac, &xi->secret,
471 buf + sizeof(*head), end);
472 break;
473
474 case CEPHX_GET_PRINCIPAL_SESSION_KEY:
475 th = get_ticket_handler(ac, CEPH_ENTITY_TYPE_AUTH);
476 BUG_ON(!th);
477 ret = ceph_x_proc_ticket_reply(ac, &th->session_key,
478 buf + sizeof(*head), end);
479 break;
480
481 default:
482 return -EINVAL;
483 }
484 if (ret)
485 return ret;
486 if (ac->want_keys == xi->have_keys)
487 return 0;
488 return -EAGAIN;
489}
490
491static int ceph_x_create_authorizer(
492 struct ceph_auth_client *ac, int peer_type,
493 struct ceph_authorizer **a,
494 void **buf, size_t *len,
495 void **reply_buf, size_t *reply_len)
496{
497 struct ceph_x_authorizer *au;
498 struct ceph_x_ticket_handler *th;
499 int ret;
500
501 th = get_ticket_handler(ac, peer_type);
502 if (IS_ERR(th))
503 return PTR_ERR(th);
504
505 au = kzalloc(sizeof(*au), GFP_NOFS);
506 if (!au)
507 return -ENOMEM;
508
509 ret = ceph_x_build_authorizer(ac, th, au);
510 if (ret) {
511 kfree(au);
512 return ret;
513 }
514
515 *a = (struct ceph_authorizer *)au;
516 *buf = au->buf->vec.iov_base;
517 *len = au->buf->vec.iov_len;
518 *reply_buf = au->reply_buf;
519 *reply_len = sizeof(au->reply_buf);
520 return 0;
521}
522
523static int ceph_x_verify_authorizer_reply(struct ceph_auth_client *ac,
524 struct ceph_authorizer *a, size_t len)
525{
526 struct ceph_x_authorizer *au = (void *)a;
527 struct ceph_x_ticket_handler *th;
528 int ret = 0;
529 struct ceph_x_authorize_reply reply;
530 void *p = au->reply_buf;
531 void *end = p + sizeof(au->reply_buf);
532
533 th = get_ticket_handler(ac, au->service);
534 if (!th)
535 return -EIO; /* hrm! */
536 ret = ceph_x_decrypt(&th->session_key, &p, end, &reply, sizeof(reply));
537 if (ret < 0)
538 return ret;
539 if (ret != sizeof(reply))
540 return -EPERM;
541
542 if (au->nonce + 1 != le64_to_cpu(reply.nonce_plus_one))
543 ret = -EPERM;
544 else
545 ret = 0;
546 dout("verify_authorizer_reply nonce %llx got %llx ret %d\n",
547 au->nonce, le64_to_cpu(reply.nonce_plus_one), ret);
548 return ret;
549}
550
551static void ceph_x_destroy_authorizer(struct ceph_auth_client *ac,
552 struct ceph_authorizer *a)
553{
554 struct ceph_x_authorizer *au = (void *)a;
555
556 ceph_buffer_put(au->buf);
557 kfree(au);
558}
559
560
561static void ceph_x_reset(struct ceph_auth_client *ac)
562{
563 struct ceph_x_info *xi = ac->private;
564
565 dout("reset\n");
566 xi->starting = true;
567 xi->server_challenge = 0;
568}
569
570static void ceph_x_destroy(struct ceph_auth_client *ac)
571{
572 struct ceph_x_info *xi = ac->private;
573 struct rb_node *p;
574
575 dout("ceph_x_destroy %p\n", ac);
576 ceph_crypto_key_destroy(&xi->secret);
577
578 while ((p = rb_first(&xi->ticket_handlers)) != NULL) {
579 struct ceph_x_ticket_handler *th =
580 rb_entry(p, struct ceph_x_ticket_handler, node);
581 remove_ticket_handler(ac, th);
582 }
583
584 kmem_cache_destroy(ceph_x_ticketbuf_cachep);
585
586 kfree(ac->private);
587 ac->private = NULL;
588}
589
590static void ceph_x_invalidate_authorizer(struct ceph_auth_client *ac,
591 int peer_type)
592{
593 struct ceph_x_ticket_handler *th;
594
595 th = get_ticket_handler(ac, peer_type);
596 if (th && !IS_ERR(th))
597 remove_ticket_handler(ac, th);
598}
599
600
601static const struct ceph_auth_client_ops ceph_x_ops = {
602 .is_authenticated = ceph_x_is_authenticated,
603 .build_request = ceph_x_build_request,
604 .handle_reply = ceph_x_handle_reply,
605 .create_authorizer = ceph_x_create_authorizer,
606 .verify_authorizer_reply = ceph_x_verify_authorizer_reply,
607 .destroy_authorizer = ceph_x_destroy_authorizer,
608 .invalidate_authorizer = ceph_x_invalidate_authorizer,
609 .reset = ceph_x_reset,
610 .destroy = ceph_x_destroy,
611};
612
613
614int ceph_x_init(struct ceph_auth_client *ac)
615{
616 struct ceph_x_info *xi;
617 int ret;
618
619 dout("ceph_x_init %p\n", ac);
620 xi = kzalloc(sizeof(*xi), GFP_NOFS);
621 if (!xi)
622 return -ENOMEM;
623
624 ret = -ENOMEM;
625 ceph_x_ticketbuf_cachep = kmem_cache_create("ceph_x_ticketbuf",
626 TEMP_TICKET_BUF_LEN, 8,
627 (SLAB_RECLAIM_ACCOUNT|SLAB_MEM_SPREAD),
628 NULL);
629 if (!ceph_x_ticketbuf_cachep)
630 goto done_nomem;
631 ret = -EINVAL;
632 if (!ac->secret) {
633 pr_err("no secret set (for auth_x protocol)\n");
634 goto done_nomem;
635 }
636
637 ret = ceph_crypto_key_unarmor(&xi->secret, ac->secret);
638 if (ret)
639 goto done_nomem;
640
641 xi->starting = true;
642 xi->ticket_handlers = RB_ROOT;
643
644 ac->protocol = CEPH_AUTH_CEPHX;
645 ac->private = xi;
646 ac->ops = &ceph_x_ops;
647 return 0;
648
649done_nomem:
650 kfree(xi);
651 if (ceph_x_ticketbuf_cachep)
652 kmem_cache_destroy(ceph_x_ticketbuf_cachep);
653 return ret;
654}
655
656
fs/ceph/auth_x.h+49−0
@@ -0,0 +1,49 @@
1#ifndef _FS_CEPH_AUTH_X_H
2#define _FS_CEPH_AUTH_X_H
3
4#include <linux/rbtree.h>
5
6#include "crypto.h"
7#include "auth.h"
8#include "auth_x_protocol.h"
9
10/*
11 * Handle ticket for a single service.
12 */
13struct ceph_x_ticket_handler {
14 struct rb_node node;
15 unsigned service;
16
17 struct ceph_crypto_key session_key;
18 struct ceph_timespec validity;
19
20 u64 secret_id;
21 struct ceph_buffer *ticket_blob;
22
23 unsigned long renew_after, expires;
24};
25
26
27struct ceph_x_authorizer {
28 struct ceph_buffer *buf;
29 unsigned service;
30 u64 nonce;
31 char reply_buf[128]; /* big enough for encrypted blob */
32};
33
34struct ceph_x_info {
35 struct ceph_crypto_key secret;
36
37 bool starting;
38 u64 server_challenge;
39
40 unsigned have_keys;
41 struct rb_root ticket_handlers;
42
43 struct ceph_x_authorizer auth_authorizer;
44};
45
46extern int ceph_x_init(struct ceph_auth_client *ac);
47
48#endif
49
fs/ceph/auth_x_protocol.h+90−0
@@ -0,0 +1,90 @@
1#ifndef __FS_CEPH_AUTH_X_PROTOCOL
2#define __FS_CEPH_AUTH_X_PROTOCOL
3
4#define CEPHX_GET_AUTH_SESSION_KEY 0x0100
5#define CEPHX_GET_PRINCIPAL_SESSION_KEY 0x0200
6#define CEPHX_GET_ROTATING_KEY 0x0400
7
8/* common bits */
9struct ceph_x_ticket_blob {
10 __u8 struct_v;
11 __le64 secret_id;
12 __le32 blob_len;
13 char blob[];
14} __attribute__ ((packed));
15
16
17/* common request/reply headers */
18struct ceph_x_request_header {
19 __le16 op;
20} __attribute__ ((packed));
21
22struct ceph_x_reply_header {
23 __le16 op;
24 __le32 result;
25} __attribute__ ((packed));
26
27
28/* authenticate handshake */
29
30/* initial hello (no reply header) */
31struct ceph_x_server_challenge {
32 __u8 struct_v;
33 __le64 server_challenge;
34} __attribute__ ((packed));
35
36struct ceph_x_authenticate {
37 __u8 struct_v;
38 __le64 client_challenge;
39 __le64 key;
40 /* ticket blob */
41} __attribute__ ((packed));
42
43struct ceph_x_service_ticket_request {
44 __u8 struct_v;
45 __le32 keys;
46} __attribute__ ((packed));
47
48struct ceph_x_challenge_blob {
49 __le64 server_challenge;
50 __le64 client_challenge;
51} __attribute__ ((packed));
52
53
54
55/* authorize handshake */
56
57/*
58 * The authorizer consists of two pieces:
59 * a - service id, ticket blob
60 * b - encrypted with session key
61 */
62struct ceph_x_authorize_a {
63 __u8 struct_v;
64 __le64 global_id;
65 __le32 service_id;
66 struct ceph_x_ticket_blob ticket_blob;
67} __attribute__ ((packed));
68
69struct ceph_x_authorize_b {
70 __u8 struct_v;
71 __le64 nonce;
72} __attribute__ ((packed));
73
74struct ceph_x_authorize_reply {
75 __u8 struct_v;
76 __le64 nonce_plus_one;
77} __attribute__ ((packed));
78
79
80/*
81 * encyption bundle
82 */
83#define CEPHX_ENC_MAGIC 0xff009cad8826aa55ull
84
85struct ceph_x_encrypt_header {
86 __u8 struct_v;
87 __le64 magic;
88} __attribute__ ((packed));
89
90#endif
fs/ceph/buffer.c+78−0
@@ -0,0 +1,78 @@
1
2#include "ceph_debug.h"
3#include "buffer.h"
4#include "decode.h"
5
6struct ceph_buffer *ceph_buffer_new(size_t len, gfp_t gfp)
7{
8 struct ceph_buffer *b;
9
10 b = kmalloc(sizeof(*b), gfp);
11 if (!b)
12 return NULL;
13
14 b->vec.iov_base = kmalloc(len, gfp | __GFP_NOWARN);
15 if (b->vec.iov_base) {
16 b->is_vmalloc = false;
17 } else {
18 b->vec.iov_base = __vmalloc(len, gfp, PAGE_KERNEL);
19 if (!b->vec.iov_base) {
20 kfree(b);
21 return NULL;
22 }
23 b->is_vmalloc = true;
24 }
25
26 kref_init(&b->kref);
27 b->alloc_len = len;
28 b->vec.iov_len = len;
29 dout("buffer_new %p\n", b);
30 return b;
31}
32
33void ceph_buffer_release(struct kref *kref)
34{
35 struct ceph_buffer *b = container_of(kref, struct ceph_buffer, kref);
36
37 dout("buffer_release %p\n", b);
38 if (b->vec.iov_base) {
39 if (b->is_vmalloc)
40 vfree(b->vec.iov_base);
41 else
42 kfree(b->vec.iov_base);
43 }
44 kfree(b);
45}
46
47int ceph_buffer_alloc(struct ceph_buffer *b, int len, gfp_t gfp)
48{
49 b->vec.iov_base = kmalloc(len, gfp | __GFP_NOWARN);
50 if (b->vec.iov_base) {
51 b->is_vmalloc = false;
52 } else {
53 b->vec.iov_base = __vmalloc(len, gfp, PAGE_KERNEL);
54 b->is_vmalloc = true;
55 }
56 if (!b->vec.iov_base)
57 return -ENOMEM;
58 b->alloc_len = len;
59 b->vec.iov_len = len;
60 return 0;
61}
62
63int ceph_decode_buffer(struct ceph_buffer **b, void **p, void *end)
64{
65 size_t len;
66
67 ceph_decode_need(p, end, sizeof(u32), bad);
68 len = ceph_decode_32(p);
69 dout("decode_buffer len %d\n", (int)len);
70 ceph_decode_need(p, end, len, bad);
71 *b = ceph_buffer_new(len, GFP_NOFS);
72 if (!*b)
73 return -ENOMEM;
74 ceph_decode_copy(p, (*b)->vec.iov_base, len);
75 return 0;
76bad:
77 return -EINVAL;
78}
fs/ceph/buffer.h+39−0
@@ -0,0 +1,39 @@
1#ifndef __FS_CEPH_BUFFER_H
2#define __FS_CEPH_BUFFER_H
3
4#include <linux/kref.h>
5#include <linux/mm.h>
6#include <linux/vmalloc.h>
7#include <linux/types.h>
8#include <linux/uio.h>
9
10/*
11 * a simple reference counted buffer.
12 *
13 * use kmalloc for small sizes (<= one page), vmalloc for larger
14 * sizes.
15 */
16struct ceph_buffer {
17 struct kref kref;
18 struct kvec vec;
19 size_t alloc_len;
20 bool is_vmalloc;
21};
22
23extern struct ceph_buffer *ceph_buffer_new(size_t len, gfp_t gfp);
24extern void ceph_buffer_release(struct kref *kref);
25
26static inline struct ceph_buffer *ceph_buffer_get(struct ceph_buffer *b)
27{
28 kref_get(&b->kref);
29 return b;
30}
31
32static inline void ceph_buffer_put(struct ceph_buffer *b)
33{
34 kref_put(&b->kref, ceph_buffer_release);
35}
36
37extern int ceph_decode_buffer(struct ceph_buffer **b, void **p, void *end);
38
39#endif
fs/ceph/caps.c+2927−0
@@ -0,0 +1,2927 @@
1#include "ceph_debug.h"
2
3#include <linux/fs.h>
4#include <linux/kernel.h>
5#include <linux/sched.h>
6#include <linux/vmalloc.h>
7#include <linux/wait.h>
8#include <linux/writeback.h>
9
10#include "super.h"
11#include "decode.h"
12#include "messenger.h"
13
14/*
15 * Capability management
16 *
17 * The Ceph metadata servers control client access to inode metadata
18 * and file data by issuing capabilities, granting clients permission
19 * to read and/or write both inode field and file data to OSDs
20 * (storage nodes). Each capability consists of a set of bits
21 * indicating which operations are allowed.
22 *
23 * If the client holds a *_SHARED cap, the client has a coherent value
24 * that can be safely read from the cached inode.
25 *
26 * In the case of a *_EXCL (exclusive) or FILE_WR capabilities, the
27 * client is allowed to change inode attributes (e.g., file size,
28 * mtime), note its dirty state in the ceph_cap, and asynchronously
29 * flush that metadata change to the MDS.
30 *
31 * In the event of a conflicting operation (perhaps by another
32 * client), the MDS will revoke the conflicting client capabilities.
33 *
34 * In order for a client to cache an inode, it must hold a capability
35 * with at least one MDS server. When inodes are released, release
36 * notifications are batched and periodically sent en masse to the MDS
37 * cluster to release server state.
38 */
39
40
41/*
42 * Generate readable cap strings for debugging output.
43 */
44#define MAX_CAP_STR 20
45static char cap_str[MAX_CAP_STR][40];
46static DEFINE_SPINLOCK(cap_str_lock);
47static int last_cap_str;
48
49static char *gcap_string(char *s, int c)
50{
51 if (c & CEPH_CAP_GSHARED)
52 *s++ = 's';
53 if (c & CEPH_CAP_GEXCL)
54 *s++ = 'x';
55 if (c & CEPH_CAP_GCACHE)
56 *s++ = 'c';
57 if (c & CEPH_CAP_GRD)
58 *s++ = 'r';
59 if (c & CEPH_CAP_GWR)
60 *s++ = 'w';
61 if (c & CEPH_CAP_GBUFFER)
62 *s++ = 'b';
63 if (c & CEPH_CAP_GLAZYIO)
64 *s++ = 'l';
65 return s;
66}
67
68const char *ceph_cap_string(int caps)
69{
70 int i;
71 char *s;
72 int c;
73
74 spin_lock(&cap_str_lock);
75 i = last_cap_str++;
76 if (last_cap_str == MAX_CAP_STR)
77 last_cap_str = 0;
78 spin_unlock(&cap_str_lock);
79
80 s = cap_str[i];
81
82 if (caps & CEPH_CAP_PIN)
83 *s++ = 'p';
84
85 c = (caps >> CEPH_CAP_SAUTH) & 3;
86 if (c) {
87 *s++ = 'A';
88 s = gcap_string(s, c);
89 }
90
91 c = (caps >> CEPH_CAP_SLINK) & 3;
92 if (c) {
93 *s++ = 'L';
94 s = gcap_string(s, c);
95 }
96
97 c = (caps >> CEPH_CAP_SXATTR) & 3;
98 if (c) {
99 *s++ = 'X';
100 s = gcap_string(s, c);
101 }
102
103 c = caps >> CEPH_CAP_SFILE;
104 if (c) {
105 *s++ = 'F';
106 s = gcap_string(s, c);
107 }
108
109 if (s == cap_str[i])
110 *s++ = '-';
111 *s = 0;
112 return cap_str[i];
113}
114
115/*
116 * Cap reservations
117 *
118 * Maintain a global pool of preallocated struct ceph_caps, referenced
119 * by struct ceph_caps_reservations. This ensures that we preallocate
120 * memory needed to successfully process an MDS response. (If an MDS
121 * sends us cap information and we fail to process it, we will have
122 * problems due to the client and MDS being out of sync.)
123 *
124 * Reservations are 'owned' by a ceph_cap_reservation context.
125 */
126static spinlock_t caps_list_lock;
127static struct list_head caps_list; /* unused (reserved or unreserved) */
128static int caps_total_count; /* total caps allocated */
129static int caps_use_count; /* in use */
130static int caps_reserve_count; /* unused, reserved */
131static int caps_avail_count; /* unused, unreserved */
132static int caps_min_count; /* keep at least this many (unreserved) */
133
134void __init ceph_caps_init(void)
135{
136 INIT_LIST_HEAD(&caps_list);
137 spin_lock_init(&caps_list_lock);
138}
139
140void ceph_caps_finalize(void)
141{
142 struct ceph_cap *cap;
143
144 spin_lock(&caps_list_lock);
145 while (!list_empty(&caps_list)) {
146 cap = list_first_entry(&caps_list, struct ceph_cap, caps_item);
147 list_del(&cap->caps_item);
148 kmem_cache_free(ceph_cap_cachep, cap);
149 }
150 caps_total_count = 0;
151 caps_avail_count = 0;
152 caps_use_count = 0;
153 caps_reserve_count = 0;
154 caps_min_count = 0;
155 spin_unlock(&caps_list_lock);
156}
157
158void ceph_adjust_min_caps(int delta)
159{
160 spin_lock(&caps_list_lock);
161 caps_min_count += delta;
162 BUG_ON(caps_min_count < 0);
163 spin_unlock(&caps_list_lock);
164}
165
166int ceph_reserve_caps(struct ceph_cap_reservation *ctx, int need)
167{
168 int i;
169 struct ceph_cap *cap;
170 int have;
171 int alloc = 0;
172 LIST_HEAD(newcaps);
173 int ret = 0;
174
175 dout("reserve caps ctx=%p need=%d\n", ctx, need);
176
177 /* first reserve any caps that are already allocated */
178 spin_lock(&caps_list_lock);
179 if (caps_avail_count >= need)
180 have = need;
181 else
182 have = caps_avail_count;
183 caps_avail_count -= have;
184 caps_reserve_count += have;
185 BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
186 caps_avail_count);
187 spin_unlock(&caps_list_lock);
188
189 for (i = have; i < need; i++) {
190 cap = kmem_cache_alloc(ceph_cap_cachep, GFP_NOFS);
191 if (!cap) {
192 ret = -ENOMEM;
193 goto out_alloc_count;
194 }
195 list_add(&cap->caps_item, &newcaps);
196 alloc++;
197 }
198 BUG_ON(have + alloc != need);
199
200 spin_lock(&caps_list_lock);
201 caps_total_count += alloc;
202 caps_reserve_count += alloc;
203 list_splice(&newcaps, &caps_list);
204
205 BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
206 caps_avail_count);
207 spin_unlock(&caps_list_lock);
208
209 ctx->count = need;
210 dout("reserve caps ctx=%p %d = %d used + %d resv + %d avail\n",
211 ctx, caps_total_count, caps_use_count, caps_reserve_count,
212 caps_avail_count);
213 return 0;
214
215out_alloc_count:
216 /* we didn't manage to reserve as much as we needed */
217 pr_warning("reserve caps ctx=%p ENOMEM need=%d got=%d\n",
218 ctx, need, have);
219 return ret;
220}
221
222int ceph_unreserve_caps(struct ceph_cap_reservation *ctx)
223{
224 dout("unreserve caps ctx=%p count=%d\n", ctx, ctx->count);
225 if (ctx->count) {
226 spin_lock(&caps_list_lock);
227 BUG_ON(caps_reserve_count < ctx->count);
228 caps_reserve_count -= ctx->count;
229 caps_avail_count += ctx->count;
230 ctx->count = 0;
231 dout("unreserve caps %d = %d used + %d resv + %d avail\n",
232 caps_total_count, caps_use_count, caps_reserve_count,
233 caps_avail_count);
234 BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
235 caps_avail_count);
236 spin_unlock(&caps_list_lock);
237 }
238 return 0;
239}
240
241static struct ceph_cap *get_cap(struct ceph_cap_reservation *ctx)
242{
243 struct ceph_cap *cap = NULL;
244
245 /* temporary, until we do something about cap import/export */
246 if (!ctx)
247 return kmem_cache_alloc(ceph_cap_cachep, GFP_NOFS);
248
249 spin_lock(&caps_list_lock);
250 dout("get_cap ctx=%p (%d) %d = %d used + %d resv + %d avail\n",
251 ctx, ctx->count, caps_total_count, caps_use_count,
252 caps_reserve_count, caps_avail_count);
253 BUG_ON(!ctx->count);
254 BUG_ON(ctx->count > caps_reserve_count);
255 BUG_ON(list_empty(&caps_list));
256
257 ctx->count--;
258 caps_reserve_count--;
259 caps_use_count++;
260
261 cap = list_first_entry(&caps_list, struct ceph_cap, caps_item);
262 list_del(&cap->caps_item);
263
264 BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
265 caps_avail_count);
266 spin_unlock(&caps_list_lock);
267 return cap;
268}
269
270void ceph_put_cap(struct ceph_cap *cap)
271{
272 spin_lock(&caps_list_lock);
273 dout("put_cap %p %d = %d used + %d resv + %d avail\n",
274 cap, caps_total_count, caps_use_count,
275 caps_reserve_count, caps_avail_count);
276 caps_use_count--;
277 /*
278 * Keep some preallocated caps around (ceph_min_count), to
279 * avoid lots of free/alloc churn.
280 */
281 if (caps_avail_count >= caps_reserve_count + caps_min_count) {
282 caps_total_count--;
283 kmem_cache_free(ceph_cap_cachep, cap);
284 } else {
285 caps_avail_count++;
286 list_add(&cap->caps_item, &caps_list);
287 }
288
289 BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
290 caps_avail_count);
291 spin_unlock(&caps_list_lock);
292}
293
294void ceph_reservation_status(struct ceph_client *client,
295 int *total, int *avail, int *used, int *reserved,
296 int *min)
297{
298 if (total)
299 *total = caps_total_count;
300 if (avail)
301 *avail = caps_avail_count;
302 if (used)
303 *used = caps_use_count;
304 if (reserved)
305 *reserved = caps_reserve_count;
306 if (min)
307 *min = caps_min_count;
308}
309
310/*
311 * Find ceph_cap for given mds, if any.
312 *
313 * Called with i_lock held.
314 */
315static struct ceph_cap *__get_cap_for_mds(struct ceph_inode_info *ci, int mds)
316{
317 struct ceph_cap *cap;
318 struct rb_node *n = ci->i_caps.rb_node;
319
320 while (n) {
321 cap = rb_entry(n, struct ceph_cap, ci_node);
322 if (mds < cap->mds)
323 n = n->rb_left;
324 else if (mds > cap->mds)
325 n = n->rb_right;
326 else
327 return cap;
328 }
329 return NULL;
330}
331
332/*
333 * Return id of any MDS with a cap, preferably FILE_WR|WRBUFFER|EXCL, else
334 * -1.
335 */
336static int __ceph_get_cap_mds(struct ceph_inode_info *ci, u32 *mseq)
337{
338 struct ceph_cap *cap;
339 int mds = -1;
340 struct rb_node *p;
341
342 /* prefer mds with WR|WRBUFFER|EXCL caps */
343 for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
344 cap = rb_entry(p, struct ceph_cap, ci_node);
345 mds = cap->mds;
346 if (mseq)
347 *mseq = cap->mseq;
348 if (cap->issued & (CEPH_CAP_FILE_WR |
349 CEPH_CAP_FILE_BUFFER |
350 CEPH_CAP_FILE_EXCL))
351 break;
352 }
353 return mds;
354}
355
356int ceph_get_cap_mds(struct inode *inode)
357{
358 int mds;
359 spin_lock(&inode->i_lock);
360 mds = __ceph_get_cap_mds(ceph_inode(inode), NULL);
361 spin_unlock(&inode->i_lock);
362 return mds;
363}
364
365/*
366 * Called under i_lock.
367 */
368static void __insert_cap_node(struct ceph_inode_info *ci,
369 struct ceph_cap *new)
370{
371 struct rb_node **p = &ci->i_caps.rb_node;
372 struct rb_node *parent = NULL;
373 struct ceph_cap *cap = NULL;
374
375 while (*p) {
376 parent = *p;
377 cap = rb_entry(parent, struct ceph_cap, ci_node);
378 if (new->mds < cap->mds)
379 p = &(*p)->rb_left;
380 else if (new->mds > cap->mds)
381 p = &(*p)->rb_right;
382 else
383 BUG();
384 }
385
386 rb_link_node(&new->ci_node, parent, p);
387 rb_insert_color(&new->ci_node, &ci->i_caps);
388}
389
390/*
391 * (re)set cap hold timeouts, which control the delayed release
392 * of unused caps back to the MDS. Should be called on cap use.
393 */
394static void __cap_set_timeouts(struct ceph_mds_client *mdsc,
395 struct ceph_inode_info *ci)
396{
397 struct ceph_mount_args *ma = mdsc->client->mount_args;
398
399 ci->i_hold_caps_min = round_jiffies(jiffies +
400 ma->caps_wanted_delay_min * HZ);
401 ci->i_hold_caps_max = round_jiffies(jiffies +
402 ma->caps_wanted_delay_max * HZ);
403 dout("__cap_set_timeouts %p min %lu max %lu\n", &ci->vfs_inode,
404 ci->i_hold_caps_min - jiffies, ci->i_hold_caps_max - jiffies);
405}
406
407/*
408 * (Re)queue cap at the end of the delayed cap release list.
409 *
410 * If I_FLUSH is set, leave the inode at the front of the list.
411 *
412 * Caller holds i_lock
413 * -> we take mdsc->cap_delay_lock
414 */
415static void __cap_delay_requeue(struct ceph_mds_client *mdsc,
416 struct ceph_inode_info *ci)
417{
418 __cap_set_timeouts(mdsc, ci);
419 dout("__cap_delay_requeue %p flags %d at %lu\n", &ci->vfs_inode,
420 ci->i_ceph_flags, ci->i_hold_caps_max);
421 if (!mdsc->stopping) {
422 spin_lock(&mdsc->cap_delay_lock);
423 if (!list_empty(&ci->i_cap_delay_list)) {
424 if (ci->i_ceph_flags & CEPH_I_FLUSH)
425 goto no_change;
426 list_del_init(&ci->i_cap_delay_list);
427 }
428 list_add_tail(&ci->i_cap_delay_list, &mdsc->cap_delay_list);
429no_change:
430 spin_unlock(&mdsc->cap_delay_lock);
431 }
432}
433
434/*
435 * Queue an inode for immediate writeback. Mark inode with I_FLUSH,
436 * indicating we should send a cap message to flush dirty metadata
437 * asap, and move to the front of the delayed cap list.
438 */
439static void __cap_delay_requeue_front(struct ceph_mds_client *mdsc,
440 struct ceph_inode_info *ci)
441{
442 dout("__cap_delay_requeue_front %p\n", &ci->vfs_inode);
443 spin_lock(&mdsc->cap_delay_lock);
444 ci->i_ceph_flags |= CEPH_I_FLUSH;
445 if (!list_empty(&ci->i_cap_delay_list))
446 list_del_init(&ci->i_cap_delay_list);
447 list_add(&ci->i_cap_delay_list, &mdsc->cap_delay_list);
448 spin_unlock(&mdsc->cap_delay_lock);
449}
450
451/*
452 * Cancel delayed work on cap.
453 *
454 * Caller must hold i_lock.
455 */
456static void __cap_delay_cancel(struct ceph_mds_client *mdsc,
457 struct ceph_inode_info *ci)
458{
459 dout("__cap_delay_cancel %p\n", &ci->vfs_inode);
460 if (list_empty(&ci->i_cap_delay_list))
461 return;
462 spin_lock(&mdsc->cap_delay_lock);
463 list_del_init(&ci->i_cap_delay_list);
464 spin_unlock(&mdsc->cap_delay_lock);
465}
466
467/*
468 * Common issue checks for add_cap, handle_cap_grant.
469 */
470static void __check_cap_issue(struct ceph_inode_info *ci, struct ceph_cap *cap,
471 unsigned issued)
472{
473 unsigned had = __ceph_caps_issued(ci, NULL);
474
475 /*
476 * Each time we receive FILE_CACHE anew, we increment
477 * i_rdcache_gen.
478 */
479 if ((issued & CEPH_CAP_FILE_CACHE) &&
480 (had & CEPH_CAP_FILE_CACHE) == 0)
481 ci->i_rdcache_gen++;
482
483 /*
484 * if we are newly issued FILE_SHARED, clear I_COMPLETE; we
485 * don't know what happened to this directory while we didn't
486 * have the cap.
487 */
488 if ((issued & CEPH_CAP_FILE_SHARED) &&
489 (had & CEPH_CAP_FILE_SHARED) == 0) {
490 ci->i_shared_gen++;
491 if (S_ISDIR(ci->vfs_inode.i_mode)) {
492 dout(" marking %p NOT complete\n", &ci->vfs_inode);
493 ci->i_ceph_flags &= ~CEPH_I_COMPLETE;
494 }
495 }
496}
497
498/*
499 * Add a capability under the given MDS session.
500 *
501 * Caller should hold session snap_rwsem (read) and s_mutex.
502 *
503 * @fmode is the open file mode, if we are opening a file, otherwise
504 * it is < 0. (This is so we can atomically add the cap and add an
505 * open file reference to it.)
506 */
507int ceph_add_cap(struct inode *inode,
508 struct ceph_mds_session *session, u64 cap_id,
509 int fmode, unsigned issued, unsigned wanted,
510 unsigned seq, unsigned mseq, u64 realmino, int flags,
511 struct ceph_cap_reservation *caps_reservation)
512{
513 struct ceph_mds_client *mdsc = &ceph_inode_to_client(inode)->mdsc;
514 struct ceph_inode_info *ci = ceph_inode(inode);
515 struct ceph_cap *new_cap = NULL;
516 struct ceph_cap *cap;
517 int mds = session->s_mds;
518 int actual_wanted;
519
520 dout("add_cap %p mds%d cap %llx %s seq %d\n", inode,
521 session->s_mds, cap_id, ceph_cap_string(issued), seq);
522
523 /*
524 * If we are opening the file, include file mode wanted bits
525 * in wanted.
526 */
527 if (fmode >= 0)
528 wanted |= ceph_caps_for_mode(fmode);
529
530retry:
531 spin_lock(&inode->i_lock);
532 cap = __get_cap_for_mds(ci, mds);
533 if (!cap) {
534 if (new_cap) {
535 cap = new_cap;
536 new_cap = NULL;
537 } else {
538 spin_unlock(&inode->i_lock);
539 new_cap = get_cap(caps_reservation);
540 if (new_cap == NULL)
541 return -ENOMEM;
542 goto retry;
543 }
544
545 cap->issued = 0;
546 cap->implemented = 0;
547 cap->mds = mds;
548 cap->mds_wanted = 0;
549
550 cap->ci = ci;
551 __insert_cap_node(ci, cap);
552
553 /* clear out old exporting info? (i.e. on cap import) */
554 if (ci->i_cap_exporting_mds == mds) {
555 ci->i_cap_exporting_issued = 0;
556 ci->i_cap_exporting_mseq = 0;
557 ci->i_cap_exporting_mds = -1;
558 }
559
560 /* add to session cap list */
561 cap->session = session;
562 spin_lock(&session->s_cap_lock);
563 list_add_tail(&cap->session_caps, &session->s_caps);
564 session->s_nr_caps++;
565 spin_unlock(&session->s_cap_lock);
566 }
567
568 if (!ci->i_snap_realm) {
569 /*
570 * add this inode to the appropriate snap realm
571 */
572 struct ceph_snap_realm *realm = ceph_lookup_snap_realm(mdsc,
573 realmino);
574 if (realm) {
575 ceph_get_snap_realm(mdsc, realm);
576 spin_lock(&realm->inodes_with_caps_lock);
577 ci->i_snap_realm = realm;
578 list_add(&ci->i_snap_realm_item,
579 &realm->inodes_with_caps);
580 spin_unlock(&realm->inodes_with_caps_lock);
581 } else {
582 pr_err("ceph_add_cap: couldn't find snap realm %llx\n",
583 realmino);
584 }
585 }
586
587 __check_cap_issue(ci, cap, issued);
588
589 /*
590 * If we are issued caps we don't want, or the mds' wanted
591 * value appears to be off, queue a check so we'll release
592 * later and/or update the mds wanted value.
593 */
594 actual_wanted = __ceph_caps_wanted(ci);
595 if ((wanted & ~actual_wanted) ||
596 (issued & ~actual_wanted & CEPH_CAP_ANY_WR)) {
597 dout(" issued %s, mds wanted %s, actual %s, queueing\n",
598 ceph_cap_string(issued), ceph_cap_string(wanted),
599 ceph_cap_string(actual_wanted));
600 __cap_delay_requeue(mdsc, ci);
601 }
602
603 if (flags & CEPH_CAP_FLAG_AUTH)
604 ci->i_auth_cap = cap;
605 else if (ci->i_auth_cap == cap)
606 ci->i_auth_cap = NULL;
607
608 dout("add_cap inode %p (%llx.%llx) cap %p %s now %s seq %d mds%d\n",
609 inode, ceph_vinop(inode), cap, ceph_cap_string(issued),
610 ceph_cap_string(issued|cap->issued), seq, mds);
611 cap->cap_id = cap_id;
612 cap->issued = issued;
613 cap->implemented |= issued;
614 cap->mds_wanted |= wanted;
615 cap->seq = seq;
616 cap->issue_seq = seq;
617 cap->mseq = mseq;
618 cap->cap_gen = session->s_cap_gen;
619
620 if (fmode >= 0)
621 __ceph_get_fmode(ci, fmode);
622 spin_unlock(&inode->i_lock);
623 wake_up(&ci->i_cap_wq);
624 return 0;
625}
626
627/*
628 * Return true if cap has not timed out and belongs to the current
629 * generation of the MDS session (i.e. has not gone 'stale' due to
630 * us losing touch with the mds).
631 */
632static int __cap_is_valid(struct ceph_cap *cap)
633{
634 unsigned long ttl;
635 u32 gen;
636
637 spin_lock(&cap->session->s_cap_lock);
638 gen = cap->session->s_cap_gen;
639 ttl = cap->session->s_cap_ttl;
640 spin_unlock(&cap->session->s_cap_lock);
641
642 if (cap->cap_gen < gen || time_after_eq(jiffies, ttl)) {
643 dout("__cap_is_valid %p cap %p issued %s "
644 "but STALE (gen %u vs %u)\n", &cap->ci->vfs_inode,
645 cap, ceph_cap_string(cap->issued), cap->cap_gen, gen);
646 return 0;
647 }
648
649 return 1;
650}
651
652/*
653 * Return set of valid cap bits issued to us. Note that caps time
654 * out, and may be invalidated in bulk if the client session times out
655 * and session->s_cap_gen is bumped.
656 */
657int __ceph_caps_issued(struct ceph_inode_info *ci, int *implemented)
658{
659 int have = ci->i_snap_caps | ci->i_cap_exporting_issued;
660 struct ceph_cap *cap;
661 struct rb_node *p;
662
663 if (implemented)
664 *implemented = 0;
665 for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
666 cap = rb_entry(p, struct ceph_cap, ci_node);
667 if (!__cap_is_valid(cap))
668 continue;
669 dout("__ceph_caps_issued %p cap %p issued %s\n",
670 &ci->vfs_inode, cap, ceph_cap_string(cap->issued));
671 have |= cap->issued;
672 if (implemented)
673 *implemented |= cap->implemented;
674 }
675 return have;
676}
677
678/*
679 * Get cap bits issued by caps other than @ocap
680 */
681int __ceph_caps_issued_other(struct ceph_inode_info *ci, struct ceph_cap *ocap)
682{
683 int have = ci->i_snap_caps;
684 struct ceph_cap *cap;
685 struct rb_node *p;
686
687 for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
688 cap = rb_entry(p, struct ceph_cap, ci_node);
689 if (cap == ocap)
690 continue;
691 if (!__cap_is_valid(cap))
692 continue;
693 have |= cap->issued;
694 }
695 return have;
696}
697
698/*
699 * Move a cap to the end of the LRU (oldest caps at list head, newest
700 * at list tail).
701 */
702static void __touch_cap(struct ceph_cap *cap)
703{
704 struct ceph_mds_session *s = cap->session;
705
706 spin_lock(&s->s_cap_lock);
707 if (s->s_cap_iterator == NULL) {
708 dout("__touch_cap %p cap %p mds%d\n", &cap->ci->vfs_inode, cap,
709 s->s_mds);
710 list_move_tail(&cap->session_caps, &s->s_caps);
711 } else {
712 dout("__touch_cap %p cap %p mds%d NOP, iterating over caps\n",
713 &cap->ci->vfs_inode, cap, s->s_mds);
714 }
715 spin_unlock(&s->s_cap_lock);
716}
717
718/*
719 * Check if we hold the given mask. If so, move the cap(s) to the
720 * front of their respective LRUs. (This is the preferred way for
721 * callers to check for caps they want.)
722 */
723int __ceph_caps_issued_mask(struct ceph_inode_info *ci, int mask, int touch)
724{
725 struct ceph_cap *cap;
726 struct rb_node *p;
727 int have = ci->i_snap_caps;
728
729 if ((have & mask) == mask) {
730 dout("__ceph_caps_issued_mask %p snap issued %s"
731 " (mask %s)\n", &ci->vfs_inode,
732 ceph_cap_string(have),
733 ceph_cap_string(mask));
734 return 1;
735 }
736
737 for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
738 cap = rb_entry(p, struct ceph_cap, ci_node);
739 if (!__cap_is_valid(cap))
740 continue;
741 if ((cap->issued & mask) == mask) {
742 dout("__ceph_caps_issued_mask %p cap %p issued %s"
743 " (mask %s)\n", &ci->vfs_inode, cap,
744 ceph_cap_string(cap->issued),
745 ceph_cap_string(mask));
746 if (touch)
747 __touch_cap(cap);
748 return 1;
749 }
750
751 /* does a combination of caps satisfy mask? */
752 have |= cap->issued;
753 if ((have & mask) == mask) {
754 dout("__ceph_caps_issued_mask %p combo issued %s"
755 " (mask %s)\n", &ci->vfs_inode,
756 ceph_cap_string(cap->issued),
757 ceph_cap_string(mask));
758 if (touch) {
759 struct rb_node *q;
760
761 /* touch this + preceeding caps */
762 __touch_cap(cap);
763 for (q = rb_first(&ci->i_caps); q != p;
764 q = rb_next(q)) {
765 cap = rb_entry(q, struct ceph_cap,
766 ci_node);
767 if (!__cap_is_valid(cap))
768 continue;
769 __touch_cap(cap);
770 }
771 }
772 return 1;
773 }
774 }
775
776 return 0;
777}
778
779/*
780 * Return true if mask caps are currently being revoked by an MDS.
781 */
782int ceph_caps_revoking(struct ceph_inode_info *ci, int mask)
783{
784 struct inode *inode = &ci->vfs_inode;
785 struct ceph_cap *cap;
786 struct rb_node *p;
787 int ret = 0;
788
789 spin_lock(&inode->i_lock);
790 for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
791 cap = rb_entry(p, struct ceph_cap, ci_node);
792 if (__cap_is_valid(cap) &&
793 (cap->implemented & ~cap->issued & mask)) {
794 ret = 1;
795 break;
796 }
797 }
798 spin_unlock(&inode->i_lock);
799 dout("ceph_caps_revoking %p %s = %d\n", inode,
800 ceph_cap_string(mask), ret);
801 return ret;
802}
803
804int __ceph_caps_used(struct ceph_inode_info *ci)
805{
806 int used = 0;
807 if (ci->i_pin_ref)
808 used |= CEPH_CAP_PIN;
809 if (ci->i_rd_ref)
810 used |= CEPH_CAP_FILE_RD;
811 if (ci->i_rdcache_ref || ci->i_rdcache_gen)
812 used |= CEPH_CAP_FILE_CACHE;
813 if (ci->i_wr_ref)
814 used |= CEPH_CAP_FILE_WR;
815 if (ci->i_wrbuffer_ref)
816 used |= CEPH_CAP_FILE_BUFFER;
817 return used;
818}
819
820/*
821 * wanted, by virtue of open file modes
822 */
823int __ceph_caps_file_wanted(struct ceph_inode_info *ci)
824{
825 int want = 0;
826 int mode;
827 for (mode = 0; mode < 4; mode++)
828 if (ci->i_nr_by_mode[mode])
829 want |= ceph_caps_for_mode(mode);
830 return want;
831}
832
833/*
834 * Return caps we have registered with the MDS(s) as 'wanted'.
835 */
836int __ceph_caps_mds_wanted(struct ceph_inode_info *ci)
837{
838 struct ceph_cap *cap;
839 struct rb_node *p;
840 int mds_wanted = 0;
841
842 for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
843 cap = rb_entry(p, struct ceph_cap, ci_node);
844 if (!__cap_is_valid(cap))
845 continue;
846 mds_wanted |= cap->mds_wanted;
847 }
848 return mds_wanted;
849}
850
851/*
852 * called under i_lock
853 */
854static int __ceph_is_any_caps(struct ceph_inode_info *ci)
855{
856 return !RB_EMPTY_ROOT(&ci->i_caps) || ci->i_cap_exporting_mds >= 0;
857}
858
859/*
860 * caller should hold i_lock.
861 * caller will not hold session s_mutex if called from destroy_inode.
862 */
863void __ceph_remove_cap(struct ceph_cap *cap)
864{
865 struct ceph_mds_session *session = cap->session;
866 struct ceph_inode_info *ci = cap->ci;
867 struct ceph_mds_client *mdsc = &ceph_client(ci->vfs_inode.i_sb)->mdsc;
868
869 dout("__ceph_remove_cap %p from %p\n", cap, &ci->vfs_inode);
870
871 /* remove from inode list */
872 rb_erase(&cap->ci_node, &ci->i_caps);
873 cap->ci = NULL;
874 if (ci->i_auth_cap == cap)
875 ci->i_auth_cap = NULL;
876
877 /* remove from session list */
878 spin_lock(&session->s_cap_lock);
879 if (session->s_cap_iterator == cap) {
880 /* not yet, we are iterating over this very cap */
881 dout("__ceph_remove_cap delaying %p removal from session %p\n",
882 cap, cap->session);
883 } else {
884 list_del_init(&cap->session_caps);
885 session->s_nr_caps--;
886 cap->session = NULL;
887 }
888 spin_unlock(&session->s_cap_lock);
889
890 if (cap->session == NULL)
891 ceph_put_cap(cap);
892
893 if (!__ceph_is_any_caps(ci) && ci->i_snap_realm) {
894 struct ceph_snap_realm *realm = ci->i_snap_realm;
895 spin_lock(&realm->inodes_with_caps_lock);
896 list_del_init(&ci->i_snap_realm_item);
897 ci->i_snap_realm_counter++;
898 ci->i_snap_realm = NULL;
899 spin_unlock(&realm->inodes_with_caps_lock);
900 ceph_put_snap_realm(mdsc, realm);
901 }
902 if (!__ceph_is_any_real_caps(ci))
903 __cap_delay_cancel(mdsc, ci);
904}
905
906/*
907 * Build and send a cap message to the given MDS.
908 *
909 * Caller should be holding s_mutex.
910 */
911static int send_cap_msg(struct ceph_mds_session *session,
912 u64 ino, u64 cid, int op,
913 int caps, int wanted, int dirty,
914 u32 seq, u64 flush_tid, u32 issue_seq, u32 mseq,
915 u64 size, u64 max_size,
916 struct timespec *mtime, struct timespec *atime,
917 u64 time_warp_seq,
918 uid_t uid, gid_t gid, mode_t mode,
919 u64 xattr_version,
920 struct ceph_buffer *xattrs_buf,
921 u64 follows)
922{
923 struct ceph_mds_caps *fc;
924 struct ceph_msg *msg;
925
926 dout("send_cap_msg %s %llx %llx caps %s wanted %s dirty %s"
927 " seq %u/%u mseq %u follows %lld size %llu/%llu"
928 " xattr_ver %llu xattr_len %d\n", ceph_cap_op_name(op),
929 cid, ino, ceph_cap_string(caps), ceph_cap_string(wanted),
930 ceph_cap_string(dirty),
931 seq, issue_seq, mseq, follows, size, max_size,
932 xattr_version, xattrs_buf ? (int)xattrs_buf->vec.iov_len : 0);
933
934 msg = ceph_msg_new(CEPH_MSG_CLIENT_CAPS, sizeof(*fc), 0, 0, NULL);
935 if (IS_ERR(msg))
936 return PTR_ERR(msg);
937
938 msg->hdr.tid = cpu_to_le64(flush_tid);
939
940 fc = msg->front.iov_base;
941 memset(fc, 0, sizeof(*fc));
942
943 fc->cap_id = cpu_to_le64(cid);
944 fc->op = cpu_to_le32(op);
945 fc->seq = cpu_to_le32(seq);
946 fc->issue_seq = cpu_to_le32(issue_seq);
947 fc->migrate_seq = cpu_to_le32(mseq);
948 fc->caps = cpu_to_le32(caps);
949 fc->wanted = cpu_to_le32(wanted);
950 fc->dirty = cpu_to_le32(dirty);
951 fc->ino = cpu_to_le64(ino);
952 fc->snap_follows = cpu_to_le64(follows);
953
954 fc->size = cpu_to_le64(size);
955 fc->max_size = cpu_to_le64(max_size);
956 if (mtime)
957 ceph_encode_timespec(&fc->mtime, mtime);
958 if (atime)
959 ceph_encode_timespec(&fc->atime, atime);
960 fc->time_warp_seq = cpu_to_le32(time_warp_seq);
961
962 fc->uid = cpu_to_le32(uid);
963 fc->gid = cpu_to_le32(gid);
964 fc->mode = cpu_to_le32(mode);
965
966 fc->xattr_version = cpu_to_le64(xattr_version);
967 if (xattrs_buf) {
968 msg->middle = ceph_buffer_get(xattrs_buf);
969 fc->xattr_len = cpu_to_le32(xattrs_buf->vec.iov_len);
970 msg->hdr.middle_len = cpu_to_le32(xattrs_buf->vec.iov_len);
971 }
972
973 ceph_con_send(&session->s_con, msg);
974 return 0;
975}
976
977/*
978 * Queue cap releases when an inode is dropped from our cache. Since
979 * inode is about to be destroyed, there is no need for i_lock.
980 */
981void ceph_queue_caps_release(struct inode *inode)
982{
983 struct ceph_inode_info *ci = ceph_inode(inode);
984 struct rb_node *p;
985
986 p = rb_first(&ci->i_caps);
987 while (p) {
988 struct ceph_cap *cap = rb_entry(p, struct ceph_cap, ci_node);
989 struct ceph_mds_session *session = cap->session;
990 struct ceph_msg *msg;
991 struct ceph_mds_cap_release *head;
992 struct ceph_mds_cap_item *item;
993
994 spin_lock(&session->s_cap_lock);
995 BUG_ON(!session->s_num_cap_releases);
996 msg = list_first_entry(&session->s_cap_releases,
997 struct ceph_msg, list_head);
998
999 dout(" adding %p release to mds%d msg %p (%d left)\n",
1000 inode, session->s_mds, msg, session->s_num_cap_releases);
1001
1002 BUG_ON(msg->front.iov_len + sizeof(*item) > PAGE_CACHE_SIZE);
1003 head = msg->front.iov_base;
1004 head->num = cpu_to_le32(le32_to_cpu(head->num) + 1);
1005 item = msg->front.iov_base + msg->front.iov_len;
1006 item->ino = cpu_to_le64(ceph_ino(inode));
1007 item->cap_id = cpu_to_le64(cap->cap_id);
1008 item->migrate_seq = cpu_to_le32(cap->mseq);
1009 item->seq = cpu_to_le32(cap->issue_seq);
1010
1011 session->s_num_cap_releases--;
1012
1013 msg->front.iov_len += sizeof(*item);
1014 if (le32_to_cpu(head->num) == CEPH_CAPS_PER_RELEASE) {
1015 dout(" release msg %p full\n", msg);
1016 list_move_tail(&msg->list_head,
1017 &session->s_cap_releases_done);
1018 } else {
1019 dout(" release msg %p at %d/%d (%d)\n", msg,
1020 (int)le32_to_cpu(head->num),
1021 (int)CEPH_CAPS_PER_RELEASE,
1022 (int)msg->front.iov_len);
1023 }
1024 spin_unlock(&session->s_cap_lock);
1025 p = rb_next(p);
1026 __ceph_remove_cap(cap);
1027 }
1028}
1029
1030/*
1031 * Send a cap msg on the given inode. Update our caps state, then
1032 * drop i_lock and send the message.
1033 *
1034 * Make note of max_size reported/requested from mds, revoked caps
1035 * that have now been implemented.
1036 *
1037 * Make half-hearted attempt ot to invalidate page cache if we are
1038 * dropping RDCACHE. Note that this will leave behind locked pages
1039 * that we'll then need to deal with elsewhere.
1040 *
1041 * Return non-zero if delayed release, or we experienced an error
1042 * such that the caller should requeue + retry later.
1043 *
1044 * called with i_lock, then drops it.
1045 * caller should hold snap_rwsem (read), s_mutex.
1046 */
1047static int __send_cap(struct ceph_mds_client *mdsc, struct ceph_cap *cap,
1048 int op, int used, int want, int retain, int flushing,
1049 unsigned *pflush_tid)
1050 __releases(cap->ci->vfs_inode->i_lock)
1051{
1052 struct ceph_inode_info *ci = cap->ci;
1053 struct inode *inode = &ci->vfs_inode;
1054 u64 cap_id = cap->cap_id;
1055 int held, revoking, dropping, keep;
1056 u64 seq, issue_seq, mseq, time_warp_seq, follows;
1057 u64 size, max_size;
1058 struct timespec mtime, atime;
1059 int wake = 0;
1060 mode_t mode;
1061 uid_t uid;
1062 gid_t gid;
1063 struct ceph_mds_session *session;
1064 u64 xattr_version = 0;
1065 int delayed = 0;
1066 u64 flush_tid = 0;
1067 int i;
1068 int ret;
1069
1070 held = cap->issued | cap->implemented;
1071 revoking = cap->implemented & ~cap->issued;
1072 retain &= ~revoking;
1073 dropping = cap->issued & ~retain;
1074
1075 dout("__send_cap %p cap %p session %p %s -> %s (revoking %s)\n",
1076 inode, cap, cap->session,
1077 ceph_cap_string(held), ceph_cap_string(held & retain),
1078 ceph_cap_string(revoking));
1079 BUG_ON((retain & CEPH_CAP_PIN) == 0);
1080
1081 session = cap->session;
1082
1083 /* don't release wanted unless we've waited a bit. */
1084 if ((ci->i_ceph_flags & CEPH_I_NODELAY) == 0 &&
1085 time_before(jiffies, ci->i_hold_caps_min)) {
1086 dout(" delaying issued %s -> %s, wanted %s -> %s on send\n",
1087 ceph_cap_string(cap->issued),
1088 ceph_cap_string(cap->issued & retain),
1089 ceph_cap_string(cap->mds_wanted),
1090 ceph_cap_string(want));
1091 want |= cap->mds_wanted;
1092 retain |= cap->issued;
1093 delayed = 1;
1094 }
1095 ci->i_ceph_flags &= ~(CEPH_I_NODELAY | CEPH_I_FLUSH);
1096
1097 cap->issued &= retain; /* drop bits we don't want */
1098 if (cap->implemented & ~cap->issued) {
1099 /*
1100 * Wake up any waiters on wanted -> needed transition.
1101 * This is due to the weird transition from buffered
1102 * to sync IO... we need to flush dirty pages _before_
1103 * allowing sync writes to avoid reordering.
1104 */
1105 wake = 1;
1106 }
1107 cap->implemented &= cap->issued | used;
1108 cap->mds_wanted = want;
1109
1110 if (flushing) {
1111 /*
1112 * assign a tid for flush operations so we can avoid
1113 * flush1 -> dirty1 -> flush2 -> flushack1 -> mark
1114 * clean type races. track latest tid for every bit
1115 * so we can handle flush AxFw, flush Fw, and have the
1116 * first ack clean Ax.
1117 */
1118 flush_tid = ++ci->i_cap_flush_last_tid;
1119 if (pflush_tid)
1120 *pflush_tid = flush_tid;
1121 dout(" cap_flush_tid %d\n", (int)flush_tid);
1122 for (i = 0; i < CEPH_CAP_BITS; i++)
1123 if (flushing & (1 << i))
1124 ci->i_cap_flush_tid[i] = flush_tid;
1125 }
1126
1127 keep = cap->implemented;
1128 seq = cap->seq;
1129 issue_seq = cap->issue_seq;
1130 mseq = cap->mseq;
1131 size = inode->i_size;
1132 ci->i_reported_size = size;
1133 max_size = ci->i_wanted_max_size;
1134 ci->i_requested_max_size = max_size;
1135 mtime = inode->i_mtime;
1136 atime = inode->i_atime;
1137 time_warp_seq = ci->i_time_warp_seq;
1138 follows = ci->i_snap_realm->cached_context->seq;
1139 uid = inode->i_uid;
1140 gid = inode->i_gid;
1141 mode = inode->i_mode;
1142
1143 if (dropping & CEPH_CAP_XATTR_EXCL) {
1144 __ceph_build_xattrs_blob(ci);
1145 xattr_version = ci->i_xattrs.version + 1;
1146 }
1147
1148 spin_unlock(&inode->i_lock);
1149
1150 ret = send_cap_msg(session, ceph_vino(inode).ino, cap_id,
1151 op, keep, want, flushing, seq, flush_tid, issue_seq, mseq,
1152 size, max_size, &mtime, &atime, time_warp_seq,
1153 uid, gid, mode,
1154 xattr_version,
1155 (flushing & CEPH_CAP_XATTR_EXCL) ? ci->i_xattrs.blob : NULL,
1156 follows);
1157 if (ret < 0) {
1158 dout("error sending cap msg, must requeue %p\n", inode);
1159 delayed = 1;
1160 }
1161
1162 if (wake)
1163 wake_up(&ci->i_cap_wq);
1164
1165 return delayed;
1166}
1167
1168/*
1169 * When a snapshot is taken, clients accumulate dirty metadata on
1170 * inodes with capabilities in ceph_cap_snaps to describe the file
1171 * state at the time the snapshot was taken. This must be flushed
1172 * asynchronously back to the MDS once sync writes complete and dirty
1173 * data is written out.
1174 *
1175 * Called under i_lock. Takes s_mutex as needed.
1176 */
1177void __ceph_flush_snaps(struct ceph_inode_info *ci,
1178 struct ceph_mds_session **psession)
1179{
1180 struct inode *inode = &ci->vfs_inode;
1181 int mds;
1182 struct ceph_cap_snap *capsnap;
1183 u32 mseq;
1184 struct ceph_mds_client *mdsc = &ceph_inode_to_client(inode)->mdsc;
1185 struct ceph_mds_session *session = NULL; /* if session != NULL, we hold
1186 session->s_mutex */
1187 u64 next_follows = 0; /* keep track of how far we've gotten through the
1188 i_cap_snaps list, and skip these entries next time
1189 around to avoid an infinite loop */
1190
1191 if (psession)
1192 session = *psession;
1193
1194 dout("__flush_snaps %p\n", inode);
1195retry:
1196 list_for_each_entry(capsnap, &ci->i_cap_snaps, ci_item) {
1197 /* avoid an infiniute loop after retry */
1198 if (capsnap->follows < next_follows)
1199 continue;
1200 /*
1201 * we need to wait for sync writes to complete and for dirty
1202 * pages to be written out.
1203 */
1204 if (capsnap->dirty_pages || capsnap->writing)
1205 continue;
1206
1207 /* pick mds, take s_mutex */
1208 mds = __ceph_get_cap_mds(ci, &mseq);
1209 if (session && session->s_mds != mds) {
1210 dout("oops, wrong session %p mutex\n", session);
1211 mutex_unlock(&session->s_mutex);
1212 ceph_put_mds_session(session);
1213 session = NULL;
1214 }
1215 if (!session) {
1216 spin_unlock(&inode->i_lock);
1217 mutex_lock(&mdsc->mutex);
1218 session = __ceph_lookup_mds_session(mdsc, mds);
1219 mutex_unlock(&mdsc->mutex);
1220 if (session) {
1221 dout("inverting session/ino locks on %p\n",
1222 session);
1223 mutex_lock(&session->s_mutex);
1224 }
1225 /*
1226 * if session == NULL, we raced against a cap
1227 * deletion. retry, and we'll get a better
1228 * @mds value next time.
1229 */
1230 spin_lock(&inode->i_lock);
1231 goto retry;
1232 }
1233
1234 capsnap->flush_tid = ++ci->i_cap_flush_last_tid;
1235 atomic_inc(&capsnap->nref);
1236 if (!list_empty(&capsnap->flushing_item))
1237 list_del_init(&capsnap->flushing_item);
1238 list_add_tail(&capsnap->flushing_item,
1239 &session->s_cap_snaps_flushing);
1240 spin_unlock(&inode->i_lock);
1241
1242 dout("flush_snaps %p cap_snap %p follows %lld size %llu\n",
1243 inode, capsnap, next_follows, capsnap->size);
1244 send_cap_msg(session, ceph_vino(inode).ino, 0,
1245 CEPH_CAP_OP_FLUSHSNAP, capsnap->issued, 0,
1246 capsnap->dirty, 0, capsnap->flush_tid, 0, mseq,
1247 capsnap->size, 0,
1248 &capsnap->mtime, &capsnap->atime,
1249 capsnap->time_warp_seq,
1250 capsnap->uid, capsnap->gid, capsnap->mode,
1251 0, NULL,
1252 capsnap->follows);
1253
1254 next_follows = capsnap->follows + 1;
1255 ceph_put_cap_snap(capsnap);
1256
1257 spin_lock(&inode->i_lock);
1258 goto retry;
1259 }
1260
1261 /* we flushed them all; remove this inode from the queue */
1262 spin_lock(&mdsc->snap_flush_lock);
1263 list_del_init(&ci->i_snap_flush_item);
1264 spin_unlock(&mdsc->snap_flush_lock);
1265
1266 if (psession)
1267 *psession = session;
1268 else if (session) {
1269 mutex_unlock(&session->s_mutex);
1270 ceph_put_mds_session(session);
1271 }
1272}
1273
1274static void ceph_flush_snaps(struct ceph_inode_info *ci)
1275{
1276 struct inode *inode = &ci->vfs_inode;
1277
1278 spin_lock(&inode->i_lock);
1279 __ceph_flush_snaps(ci, NULL);
1280 spin_unlock(&inode->i_lock);
1281}
1282
1283/*
1284 * Mark caps dirty. If inode is newly dirty, add to the global dirty
1285 * list.
1286 */
1287void __ceph_mark_dirty_caps(struct ceph_inode_info *ci, int mask)
1288{
1289 struct ceph_mds_client *mdsc = &ceph_client(ci->vfs_inode.i_sb)->mdsc;
1290 struct inode *inode = &ci->vfs_inode;
1291 int was = ci->i_dirty_caps;
1292 int dirty = 0;
1293
1294 dout("__mark_dirty_caps %p %s dirty %s -> %s\n", &ci->vfs_inode,
1295 ceph_cap_string(mask), ceph_cap_string(was),
1296 ceph_cap_string(was | mask));
1297 ci->i_dirty_caps |= mask;
1298 if (was == 0) {
1299 dout(" inode %p now dirty\n", &ci->vfs_inode);
1300 BUG_ON(!list_empty(&ci->i_dirty_item));
1301 spin_lock(&mdsc->cap_dirty_lock);
1302 list_add(&ci->i_dirty_item, &mdsc->cap_dirty);
1303 spin_unlock(&mdsc->cap_dirty_lock);
1304 if (ci->i_flushing_caps == 0) {
1305 igrab(inode);
1306 dirty |= I_DIRTY_SYNC;
1307 }
1308 }
1309 BUG_ON(list_empty(&ci->i_dirty_item));
1310 if (((was | ci->i_flushing_caps) & CEPH_CAP_FILE_BUFFER) &&
1311 (mask & CEPH_CAP_FILE_BUFFER))
1312 dirty |= I_DIRTY_DATASYNC;
1313 if (dirty)
1314 __mark_inode_dirty(inode, dirty);
1315 __cap_delay_requeue(mdsc, ci);
1316}
1317
1318/*
1319 * Add dirty inode to the flushing list. Assigned a seq number so we
1320 * can wait for caps to flush without starving.
1321 *
1322 * Called under i_lock.
1323 */
1324static int __mark_caps_flushing(struct inode *inode,
1325 struct ceph_mds_session *session)
1326{
1327 struct ceph_mds_client *mdsc = &ceph_client(inode->i_sb)->mdsc;
1328 struct ceph_inode_info *ci = ceph_inode(inode);
1329 int flushing;
1330
1331 BUG_ON(ci->i_dirty_caps == 0);
1332 BUG_ON(list_empty(&ci->i_dirty_item));
1333
1334 flushing = ci->i_dirty_caps;
1335 dout("__mark_caps_flushing flushing %s, flushing_caps %s -> %s\n",
1336 ceph_cap_string(flushing),
1337 ceph_cap_string(ci->i_flushing_caps),
1338 ceph_cap_string(ci->i_flushing_caps | flushing));
1339 ci->i_flushing_caps |= flushing;
1340 ci->i_dirty_caps = 0;
1341 dout(" inode %p now !dirty\n", inode);
1342
1343 spin_lock(&mdsc->cap_dirty_lock);
1344 list_del_init(&ci->i_dirty_item);
1345
1346 ci->i_cap_flush_seq = ++mdsc->cap_flush_seq;
1347 if (list_empty(&ci->i_flushing_item)) {
1348 list_add_tail(&ci->i_flushing_item, &session->s_cap_flushing);
1349 mdsc->num_cap_flushing++;
1350 dout(" inode %p now flushing seq %lld\n", inode,
1351 ci->i_cap_flush_seq);
1352 } else {
1353 list_move_tail(&ci->i_flushing_item, &session->s_cap_flushing);
1354 dout(" inode %p now flushing (more) seq %lld\n", inode,
1355 ci->i_cap_flush_seq);
1356 }
1357 spin_unlock(&mdsc->cap_dirty_lock);
1358
1359 return flushing;
1360}
1361
1362/*
1363 * try to invalidate mapping pages without blocking.
1364 */
1365static int mapping_is_empty(struct address_space *mapping)
1366{
1367 struct page *page = find_get_page(mapping, 0);
1368
1369 if (!page)
1370 return 1;
1371
1372 put_page(page);
1373 return 0;
1374}
1375
1376static int try_nonblocking_invalidate(struct inode *inode)
1377{
1378 struct ceph_inode_info *ci = ceph_inode(inode);
1379 u32 invalidating_gen = ci->i_rdcache_gen;
1380
1381 spin_unlock(&inode->i_lock);
1382 invalidate_mapping_pages(&inode->i_data, 0, -1);
1383 spin_lock(&inode->i_lock);
1384
1385 if (mapping_is_empty(&inode->i_data) &&
1386 invalidating_gen == ci->i_rdcache_gen) {
1387 /* success. */
1388 dout("try_nonblocking_invalidate %p success\n", inode);
1389 ci->i_rdcache_gen = 0;
1390 ci->i_rdcache_revoking = 0;
1391 return 0;
1392 }
1393 dout("try_nonblocking_invalidate %p failed\n", inode);
1394 return -1;
1395}
1396
1397/*
1398 * Swiss army knife function to examine currently used and wanted
1399 * versus held caps. Release, flush, ack revoked caps to mds as
1400 * appropriate.
1401 *
1402 * CHECK_CAPS_NODELAY - caller is delayed work and we should not delay
1403 * cap release further.
1404 * CHECK_CAPS_AUTHONLY - we should only check the auth cap
1405 * CHECK_CAPS_FLUSH - we should flush any dirty caps immediately, without
1406 * further delay.
1407 */
1408void ceph_check_caps(struct ceph_inode_info *ci, int flags,
1409 struct ceph_mds_session *session)
1410{
1411 struct ceph_client *client = ceph_inode_to_client(&ci->vfs_inode);
1412 struct ceph_mds_client *mdsc = &client->mdsc;
1413 struct inode *inode = &ci->vfs_inode;
1414 struct ceph_cap *cap;
1415 int file_wanted, used;
1416 int took_snap_rwsem = 0; /* true if mdsc->snap_rwsem held */
1417 int drop_session_lock = session ? 0 : 1;
1418 int issued, implemented, want, retain, revoking, flushing = 0;
1419 int mds = -1; /* keep track of how far we've gone through i_caps list
1420 to avoid an infinite loop on retry */
1421 struct rb_node *p;
1422 int tried_invalidate = 0;
1423 int delayed = 0, sent = 0, force_requeue = 0, num;
1424 int queue_invalidate = 0;
1425 int is_delayed = flags & CHECK_CAPS_NODELAY;
1426
1427 /* if we are unmounting, flush any unused caps immediately. */
1428 if (mdsc->stopping)
1429 is_delayed = 1;
1430
1431 spin_lock(&inode->i_lock);
1432
1433 if (ci->i_ceph_flags & CEPH_I_FLUSH)
1434 flags |= CHECK_CAPS_FLUSH;
1435
1436 /* flush snaps first time around only */
1437 if (!list_empty(&ci->i_cap_snaps))
1438 __ceph_flush_snaps(ci, &session);
1439 goto retry_locked;
1440retry:
1441 spin_lock(&inode->i_lock);
1442retry_locked:
1443 file_wanted = __ceph_caps_file_wanted(ci);
1444 used = __ceph_caps_used(ci);
1445 want = file_wanted | used;
1446 issued = __ceph_caps_issued(ci, &implemented);
1447 revoking = implemented & ~issued;
1448
1449 retain = want | CEPH_CAP_PIN;
1450 if (!mdsc->stopping && inode->i_nlink > 0) {
1451 if (want) {
1452 retain |= CEPH_CAP_ANY; /* be greedy */
1453 } else {
1454 retain |= CEPH_CAP_ANY_SHARED;
1455 /*
1456 * keep RD only if we didn't have the file open RW,
1457 * because then the mds would revoke it anyway to
1458 * journal max_size=0.
1459 */
1460 if (ci->i_max_size == 0)
1461 retain |= CEPH_CAP_ANY_RD;
1462 }
1463 }
1464
1465 dout("check_caps %p file_want %s used %s dirty %s flushing %s"
1466 " issued %s revoking %s retain %s %s%s%s\n", inode,
1467 ceph_cap_string(file_wanted),
1468 ceph_cap_string(used), ceph_cap_string(ci->i_dirty_caps),
1469 ceph_cap_string(ci->i_flushing_caps),
1470 ceph_cap_string(issued), ceph_cap_string(revoking),
1471 ceph_cap_string(retain),
1472 (flags & CHECK_CAPS_AUTHONLY) ? " AUTHONLY" : "",
1473 (flags & CHECK_CAPS_NODELAY) ? " NODELAY" : "",
1474 (flags & CHECK_CAPS_FLUSH) ? " FLUSH" : "");
1475
1476 /*
1477 * If we no longer need to hold onto old our caps, and we may
1478 * have cached pages, but don't want them, then try to invalidate.
1479 * If we fail, it's because pages are locked.... try again later.
1480 */
1481 if ((!is_delayed || mdsc->stopping) &&
1482 ci->i_wrbuffer_ref == 0 && /* no dirty pages... */
1483 ci->i_rdcache_gen && /* may have cached pages */
1484 (file_wanted == 0 || /* no open files */
1485 (revoking & CEPH_CAP_FILE_CACHE)) && /* or revoking cache */
1486 !tried_invalidate) {
1487 dout("check_caps trying to invalidate on %p\n", inode);
1488 if (try_nonblocking_invalidate(inode) < 0) {
1489 if (revoking & CEPH_CAP_FILE_CACHE) {
1490 dout("check_caps queuing invalidate\n");
1491 queue_invalidate = 1;
1492 ci->i_rdcache_revoking = ci->i_rdcache_gen;
1493 } else {
1494 dout("check_caps failed to invalidate pages\n");
1495 /* we failed to invalidate pages. check these
1496 caps again later. */
1497 force_requeue = 1;
1498 __cap_set_timeouts(mdsc, ci);
1499 }
1500 }
1501 tried_invalidate = 1;
1502 goto retry_locked;
1503 }
1504
1505 num = 0;
1506 for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
1507 cap = rb_entry(p, struct ceph_cap, ci_node);
1508 num++;
1509
1510 /* avoid looping forever */
1511 if (mds >= cap->mds ||
1512 ((flags & CHECK_CAPS_AUTHONLY) && cap != ci->i_auth_cap))
1513 continue;
1514
1515 /* NOTE: no side-effects allowed, until we take s_mutex */
1516
1517 revoking = cap->implemented & ~cap->issued;
1518 if (revoking)
1519 dout(" mds%d revoking %s\n", cap->mds,
1520 ceph_cap_string(revoking));
1521
1522 if (cap == ci->i_auth_cap &&
1523 (cap->issued & CEPH_CAP_FILE_WR)) {
1524 /* request larger max_size from MDS? */
1525 if (ci->i_wanted_max_size > ci->i_max_size &&
1526 ci->i_wanted_max_size > ci->i_requested_max_size) {
1527 dout("requesting new max_size\n");
1528 goto ack;
1529 }
1530
1531 /* approaching file_max? */
1532 if ((inode->i_size << 1) >= ci->i_max_size &&
1533 (ci->i_reported_size << 1) < ci->i_max_size) {
1534 dout("i_size approaching max_size\n");
1535 goto ack;
1536 }
1537 }
1538 /* flush anything dirty? */
1539 if (cap == ci->i_auth_cap && (flags & CHECK_CAPS_FLUSH) &&
1540 ci->i_dirty_caps) {
1541 dout("flushing dirty caps\n");
1542 goto ack;
1543 }
1544
1545 /* completed revocation? going down and there are no caps? */
1546 if (revoking && (revoking & used) == 0) {
1547 dout("completed revocation of %s\n",
1548 ceph_cap_string(cap->implemented & ~cap->issued));
1549 goto ack;
1550 }
1551
1552 /* want more caps from mds? */
1553 if (want & ~(cap->mds_wanted | cap->issued))
1554 goto ack;
1555
1556 /* things we might delay */
1557 if ((cap->issued & ~retain) == 0 &&
1558 cap->mds_wanted == want)
1559 continue; /* nope, all good */
1560
1561 if (is_delayed)
1562 goto ack;
1563
1564 /* delay? */
1565 if ((ci->i_ceph_flags & CEPH_I_NODELAY) == 0 &&
1566 time_before(jiffies, ci->i_hold_caps_max)) {
1567 dout(" delaying issued %s -> %s, wanted %s -> %s\n",
1568 ceph_cap_string(cap->issued),
1569 ceph_cap_string(cap->issued & retain),
1570 ceph_cap_string(cap->mds_wanted),
1571 ceph_cap_string(want));
1572 delayed++;
1573 continue;
1574 }
1575
1576ack:
1577 if (ci->i_ceph_flags & CEPH_I_NOFLUSH) {
1578 dout(" skipping %p I_NOFLUSH set\n", inode);
1579 continue;
1580 }
1581
1582 if (session && session != cap->session) {
1583 dout("oops, wrong session %p mutex\n", session);
1584 mutex_unlock(&session->s_mutex);
1585 session = NULL;
1586 }
1587 if (!session) {
1588 session = cap->session;
1589 if (mutex_trylock(&session->s_mutex) == 0) {
1590 dout("inverting session/ino locks on %p\n",
1591 session);
1592 spin_unlock(&inode->i_lock);
1593 if (took_snap_rwsem) {
1594 up_read(&mdsc->snap_rwsem);
1595 took_snap_rwsem = 0;
1596 }
1597 mutex_lock(&session->s_mutex);
1598 goto retry;
1599 }
1600 }
1601 /* take snap_rwsem after session mutex */
1602 if (!took_snap_rwsem) {
1603 if (down_read_trylock(&mdsc->snap_rwsem) == 0) {
1604 dout("inverting snap/in locks on %p\n",
1605 inode);
1606 spin_unlock(&inode->i_lock);
1607 down_read(&mdsc->snap_rwsem);
1608 took_snap_rwsem = 1;
1609 goto retry;
1610 }
1611 took_snap_rwsem = 1;
1612 }
1613
1614 if (cap == ci->i_auth_cap && ci->i_dirty_caps)
1615 flushing = __mark_caps_flushing(inode, session);
1616
1617 mds = cap->mds; /* remember mds, so we don't repeat */
1618 sent++;
1619
1620 /* __send_cap drops i_lock */
1621 delayed += __send_cap(mdsc, cap, CEPH_CAP_OP_UPDATE, used, want,
1622 retain, flushing, NULL);
1623 goto retry; /* retake i_lock and restart our cap scan. */
1624 }
1625
1626 /*
1627 * Reschedule delayed caps release if we delayed anything,
1628 * otherwise cancel.
1629 */
1630 if (delayed && is_delayed)
1631 force_requeue = 1; /* __send_cap delayed release; requeue */
1632 if (!delayed && !is_delayed)
1633 __cap_delay_cancel(mdsc, ci);
1634 else if (!is_delayed || force_requeue)
1635 __cap_delay_requeue(mdsc, ci);
1636
1637 spin_unlock(&inode->i_lock);
1638
1639 if (queue_invalidate)
1640 ceph_queue_invalidate(inode);
1641
1642 if (session && drop_session_lock)
1643 mutex_unlock(&session->s_mutex);
1644 if (took_snap_rwsem)
1645 up_read(&mdsc->snap_rwsem);
1646}
1647
1648/*
1649 * Try to flush dirty caps back to the auth mds.
1650 */
1651static int try_flush_caps(struct inode *inode, struct ceph_mds_session *session,
1652 unsigned *flush_tid)
1653{
1654 struct ceph_mds_client *mdsc = &ceph_client(inode->i_sb)->mdsc;
1655 struct ceph_inode_info *ci = ceph_inode(inode);
1656 int unlock_session = session ? 0 : 1;
1657 int flushing = 0;
1658
1659retry:
1660 spin_lock(&inode->i_lock);
1661 if (ci->i_ceph_flags & CEPH_I_NOFLUSH) {
1662 dout("try_flush_caps skipping %p I_NOFLUSH set\n", inode);
1663 goto out;
1664 }
1665 if (ci->i_dirty_caps && ci->i_auth_cap) {
1666 struct ceph_cap *cap = ci->i_auth_cap;
1667 int used = __ceph_caps_used(ci);
1668 int want = __ceph_caps_wanted(ci);
1669 int delayed;
1670
1671 if (!session) {
1672 spin_unlock(&inode->i_lock);
1673 session = cap->session;
1674 mutex_lock(&session->s_mutex);
1675 goto retry;
1676 }
1677 BUG_ON(session != cap->session);
1678 if (cap->session->s_state < CEPH_MDS_SESSION_OPEN)
1679 goto out;
1680
1681 flushing = __mark_caps_flushing(inode, session);
1682
1683 /* __send_cap drops i_lock */
1684 delayed = __send_cap(mdsc, cap, CEPH_CAP_OP_FLUSH, used, want,
1685 cap->issued | cap->implemented, flushing,
1686 flush_tid);
1687 if (!delayed)
1688 goto out_unlocked;
1689
1690 spin_lock(&inode->i_lock);
1691 __cap_delay_requeue(mdsc, ci);
1692 }
1693out:
1694 spin_unlock(&inode->i_lock);
1695out_unlocked:
1696 if (session && unlock_session)
1697 mutex_unlock(&session->s_mutex);
1698 return flushing;
1699}
1700
1701/*
1702 * Return true if we've flushed caps through the given flush_tid.
1703 */
1704static int caps_are_flushed(struct inode *inode, unsigned tid)
1705{
1706 struct ceph_inode_info *ci = ceph_inode(inode);
1707 int dirty, i, ret = 1;
1708
1709 spin_lock(&inode->i_lock);
1710 dirty = __ceph_caps_dirty(ci);
1711 for (i = 0; i < CEPH_CAP_BITS; i++)
1712 if ((ci->i_flushing_caps & (1 << i)) &&
1713 ci->i_cap_flush_tid[i] <= tid) {
1714 /* still flushing this bit */
1715 ret = 0;
1716 break;
1717 }
1718 spin_unlock(&inode->i_lock);
1719 return ret;
1720}
1721
1722/*
1723 * Wait on any unsafe replies for the given inode. First wait on the
1724 * newest request, and make that the upper bound. Then, if there are
1725 * more requests, keep waiting on the oldest as long as it is still older
1726 * than the original request.
1727 */
1728static void sync_write_wait(struct inode *inode)
1729{
1730 struct ceph_inode_info *ci = ceph_inode(inode);
1731 struct list_head *head = &ci->i_unsafe_writes;
1732 struct ceph_osd_request *req;
1733 u64 last_tid;
1734
1735 spin_lock(&ci->i_unsafe_lock);
1736 if (list_empty(head))
1737 goto out;
1738
1739 /* set upper bound as _last_ entry in chain */
1740 req = list_entry(head->prev, struct ceph_osd_request,
1741 r_unsafe_item);
1742 last_tid = req->r_tid;
1743
1744 do {
1745 ceph_osdc_get_request(req);
1746 spin_unlock(&ci->i_unsafe_lock);
1747 dout("sync_write_wait on tid %llu (until %llu)\n",
1748 req->r_tid, last_tid);
1749 wait_for_completion(&req->r_safe_completion);
1750 spin_lock(&ci->i_unsafe_lock);
1751 ceph_osdc_put_request(req);
1752
1753 /*
1754 * from here on look at first entry in chain, since we
1755 * only want to wait for anything older than last_tid
1756 */
1757 if (list_empty(head))
1758 break;
1759 req = list_entry(head->next, struct ceph_osd_request,
1760 r_unsafe_item);
1761 } while (req->r_tid < last_tid);
1762out:
1763 spin_unlock(&ci->i_unsafe_lock);
1764}
1765
1766int ceph_fsync(struct file *file, struct dentry *dentry, int datasync)
1767{
1768 struct inode *inode = dentry->d_inode;
1769 struct ceph_inode_info *ci = ceph_inode(inode);
1770 unsigned flush_tid;
1771 int ret;
1772 int dirty;
1773
1774 dout("fsync %p%s\n", inode, datasync ? " datasync" : "");
1775 sync_write_wait(inode);
1776
1777 ret = filemap_write_and_wait(inode->i_mapping);
1778 if (ret < 0)
1779 return ret;
1780
1781 dirty = try_flush_caps(inode, NULL, &flush_tid);
1782 dout("fsync dirty caps are %s\n", ceph_cap_string(dirty));
1783
1784 /*
1785 * only wait on non-file metadata writeback (the mds
1786 * can recover size and mtime, so we don't need to
1787 * wait for that)
1788 */
1789 if (!datasync && (dirty & ~CEPH_CAP_ANY_FILE_WR)) {
1790 dout("fsync waiting for flush_tid %u\n", flush_tid);
1791 ret = wait_event_interruptible(ci->i_cap_wq,
1792 caps_are_flushed(inode, flush_tid));
1793 }
1794
1795 dout("fsync %p%s done\n", inode, datasync ? " datasync" : "");
1796 return ret;
1797}
1798
1799/*
1800 * Flush any dirty caps back to the mds. If we aren't asked to wait,
1801 * queue inode for flush but don't do so immediately, because we can
1802 * get by with fewer MDS messages if we wait for data writeback to
1803 * complete first.
1804 */
1805int ceph_write_inode(struct inode *inode, struct writeback_control *wbc)
1806{
1807 struct ceph_inode_info *ci = ceph_inode(inode);
1808 unsigned flush_tid;
1809 int err = 0;
1810 int dirty;
1811 int wait = wbc->sync_mode == WB_SYNC_ALL;
1812
1813 dout("write_inode %p wait=%d\n", inode, wait);
1814 if (wait) {
1815 dirty = try_flush_caps(inode, NULL, &flush_tid);
1816 if (dirty)
1817 err = wait_event_interruptible(ci->i_cap_wq,
1818 caps_are_flushed(inode, flush_tid));
1819 } else {
1820 struct ceph_mds_client *mdsc = &ceph_client(inode->i_sb)->mdsc;
1821
1822 spin_lock(&inode->i_lock);
1823 if (__ceph_caps_dirty(ci))
1824 __cap_delay_requeue_front(mdsc, ci);
1825 spin_unlock(&inode->i_lock);
1826 }
1827 return err;
1828}
1829
1830/*
1831 * After a recovering MDS goes active, we need to resend any caps
1832 * we were flushing.
1833 *
1834 * Caller holds session->s_mutex.
1835 */
1836static void kick_flushing_capsnaps(struct ceph_mds_client *mdsc,
1837 struct ceph_mds_session *session)
1838{
1839 struct ceph_cap_snap *capsnap;
1840
1841 dout("kick_flushing_capsnaps mds%d\n", session->s_mds);
1842 list_for_each_entry(capsnap, &session->s_cap_snaps_flushing,
1843 flushing_item) {
1844 struct ceph_inode_info *ci = capsnap->ci;
1845 struct inode *inode = &ci->vfs_inode;
1846 struct ceph_cap *cap;
1847
1848 spin_lock(&inode->i_lock);
1849 cap = ci->i_auth_cap;
1850 if (cap && cap->session == session) {
1851 dout("kick_flushing_caps %p cap %p capsnap %p\n", inode,
1852 cap, capsnap);
1853 __ceph_flush_snaps(ci, &session);
1854 } else {
1855 pr_err("%p auth cap %p not mds%d ???\n", inode,
1856 cap, session->s_mds);
1857 spin_unlock(&inode->i_lock);
1858 }
1859 }
1860}
1861
1862void ceph_kick_flushing_caps(struct ceph_mds_client *mdsc,
1863 struct ceph_mds_session *session)
1864{
1865 struct ceph_inode_info *ci;
1866
1867 kick_flushing_capsnaps(mdsc, session);
1868
1869 dout("kick_flushing_caps mds%d\n", session->s_mds);
1870 list_for_each_entry(ci, &session->s_cap_flushing, i_flushing_item) {
1871 struct inode *inode = &ci->vfs_inode;
1872 struct ceph_cap *cap;
1873 int delayed = 0;
1874
1875 spin_lock(&inode->i_lock);
1876 cap = ci->i_auth_cap;
1877 if (cap && cap->session == session) {
1878 dout("kick_flushing_caps %p cap %p %s\n", inode,
1879 cap, ceph_cap_string(ci->i_flushing_caps));
1880 delayed = __send_cap(mdsc, cap, CEPH_CAP_OP_FLUSH,
1881 __ceph_caps_used(ci),
1882 __ceph_caps_wanted(ci),
1883 cap->issued | cap->implemented,
1884 ci->i_flushing_caps, NULL);
1885 if (delayed) {
1886 spin_lock(&inode->i_lock);
1887 __cap_delay_requeue(mdsc, ci);
1888 spin_unlock(&inode->i_lock);
1889 }
1890 } else {
1891 pr_err("%p auth cap %p not mds%d ???\n", inode,
1892 cap, session->s_mds);
1893 spin_unlock(&inode->i_lock);
1894 }
1895 }
1896}
1897
1898
1899/*
1900 * Take references to capabilities we hold, so that we don't release
1901 * them to the MDS prematurely.
1902 *
1903 * Protected by i_lock.
1904 */
1905static void __take_cap_refs(struct ceph_inode_info *ci, int got)
1906{
1907 if (got & CEPH_CAP_PIN)
1908 ci->i_pin_ref++;
1909 if (got & CEPH_CAP_FILE_RD)
1910 ci->i_rd_ref++;
1911 if (got & CEPH_CAP_FILE_CACHE)
1912 ci->i_rdcache_ref++;
1913 if (got & CEPH_CAP_FILE_WR)
1914 ci->i_wr_ref++;
1915 if (got & CEPH_CAP_FILE_BUFFER) {
1916 if (ci->i_wrbuffer_ref == 0)
1917 igrab(&ci->vfs_inode);
1918 ci->i_wrbuffer_ref++;
1919 dout("__take_cap_refs %p wrbuffer %d -> %d (?)\n",
1920 &ci->vfs_inode, ci->i_wrbuffer_ref-1, ci->i_wrbuffer_ref);
1921 }
1922}
1923
1924/*
1925 * Try to grab cap references. Specify those refs we @want, and the
1926 * minimal set we @need. Also include the larger offset we are writing
1927 * to (when applicable), and check against max_size here as well.
1928 * Note that caller is responsible for ensuring max_size increases are
1929 * requested from the MDS.
1930 */
1931static int try_get_cap_refs(struct ceph_inode_info *ci, int need, int want,
1932 int *got, loff_t endoff, int *check_max, int *err)
1933{
1934 struct inode *inode = &ci->vfs_inode;
1935 int ret = 0;
1936 int have, implemented;
1937 int file_wanted;
1938
1939 dout("get_cap_refs %p need %s want %s\n", inode,
1940 ceph_cap_string(need), ceph_cap_string(want));
1941 spin_lock(&inode->i_lock);
1942
1943 /* make sure file is actually open */
1944 file_wanted = __ceph_caps_file_wanted(ci);
1945 if ((file_wanted & need) == 0) {
1946 dout("try_get_cap_refs need %s file_wanted %s, EBADF\n",
1947 ceph_cap_string(need), ceph_cap_string(file_wanted));
1948 *err = -EBADF;
1949 ret = 1;
1950 goto out;
1951 }
1952
1953 if (need & CEPH_CAP_FILE_WR) {
1954 if (endoff >= 0 && endoff > (loff_t)ci->i_max_size) {
1955 dout("get_cap_refs %p endoff %llu > maxsize %llu\n",
1956 inode, endoff, ci->i_max_size);
1957 if (endoff > ci->i_wanted_max_size) {
1958 *check_max = 1;
1959 ret = 1;
1960 }
1961 goto out;
1962 }
1963 /*
1964 * If a sync write is in progress, we must wait, so that we
1965 * can get a final snapshot value for size+mtime.
1966 */
1967 if (__ceph_have_pending_cap_snap(ci)) {
1968 dout("get_cap_refs %p cap_snap_pending\n", inode);
1969 goto out;
1970 }
1971 }
1972 have = __ceph_caps_issued(ci, &implemented);
1973
1974 /*
1975 * disallow writes while a truncate is pending
1976 */
1977 if (ci->i_truncate_pending)
1978 have &= ~CEPH_CAP_FILE_WR;
1979
1980 if ((have & need) == need) {
1981 /*
1982 * Look at (implemented & ~have & not) so that we keep waiting
1983 * on transition from wanted -> needed caps. This is needed
1984 * for WRBUFFER|WR -> WR to avoid a new WR sync write from
1985 * going before a prior buffered writeback happens.
1986 */
1987 int not = want & ~(have & need);
1988 int revoking = implemented & ~have;
1989 dout("get_cap_refs %p have %s but not %s (revoking %s)\n",
1990 inode, ceph_cap_string(have), ceph_cap_string(not),
1991 ceph_cap_string(revoking));
1992 if ((revoking & not) == 0) {
1993 *got = need | (have & want);
1994 __take_cap_refs(ci, *got);
1995 ret = 1;
1996 }
1997 } else {
1998 dout("get_cap_refs %p have %s needed %s\n", inode,
1999 ceph_cap_string(have), ceph_cap_string(need));
2000 }
2001out:
2002 spin_unlock(&inode->i_lock);
2003 dout("get_cap_refs %p ret %d got %s\n", inode,
2004 ret, ceph_cap_string(*got));
2005 return ret;
2006}
2007
2008/*
2009 * Check the offset we are writing up to against our current
2010 * max_size. If necessary, tell the MDS we want to write to
2011 * a larger offset.
2012 */
2013static void check_max_size(struct inode *inode, loff_t endoff)
2014{
2015 struct ceph_inode_info *ci = ceph_inode(inode);
2016 int check = 0;
2017
2018 /* do we need to explicitly request a larger max_size? */
2019 spin_lock(&inode->i_lock);
2020 if ((endoff >= ci->i_max_size ||
2021 endoff > (inode->i_size << 1)) &&
2022 endoff > ci->i_wanted_max_size) {
2023 dout("write %p at large endoff %llu, req max_size\n",
2024 inode, endoff);
2025 ci->i_wanted_max_size = endoff;
2026 check = 1;
2027 }
2028 spin_unlock(&inode->i_lock);
2029 if (check)
2030 ceph_check_caps(ci, CHECK_CAPS_AUTHONLY, NULL);
2031}
2032
2033/*
2034 * Wait for caps, and take cap references. If we can't get a WR cap
2035 * due to a small max_size, make sure we check_max_size (and possibly
2036 * ask the mds) so we don't get hung up indefinitely.
2037 */
2038int ceph_get_caps(struct ceph_inode_info *ci, int need, int want, int *got,
2039 loff_t endoff)
2040{
2041 int check_max, ret, err;
2042
2043retry:
2044 if (endoff > 0)
2045 check_max_size(&ci->vfs_inode, endoff);
2046 check_max = 0;
2047 err = 0;
2048 ret = wait_event_interruptible(ci->i_cap_wq,
2049 try_get_cap_refs(ci, need, want,
2050 got, endoff,
2051 &check_max, &err));
2052 if (err)
2053 ret = err;
2054 if (check_max)
2055 goto retry;
2056 return ret;
2057}
2058
2059/*
2060 * Take cap refs. Caller must already know we hold at least one ref
2061 * on the caps in question or we don't know this is safe.
2062 */
2063void ceph_get_cap_refs(struct ceph_inode_info *ci, int caps)
2064{
2065 spin_lock(&ci->vfs_inode.i_lock);
2066 __take_cap_refs(ci, caps);
2067 spin_unlock(&ci->vfs_inode.i_lock);
2068}
2069
2070/*
2071 * Release cap refs.
2072 *
2073 * If we released the last ref on any given cap, call ceph_check_caps
2074 * to release (or schedule a release).
2075 *
2076 * If we are releasing a WR cap (from a sync write), finalize any affected
2077 * cap_snap, and wake up any waiters.
2078 */
2079void ceph_put_cap_refs(struct ceph_inode_info *ci, int had)
2080{
2081 struct inode *inode = &ci->vfs_inode;
2082 int last = 0, put = 0, flushsnaps = 0, wake = 0;
2083 struct ceph_cap_snap *capsnap;
2084
2085 spin_lock(&inode->i_lock);
2086 if (had & CEPH_CAP_PIN)
2087 --ci->i_pin_ref;
2088 if (had & CEPH_CAP_FILE_RD)
2089 if (--ci->i_rd_ref == 0)
2090 last++;
2091 if (had & CEPH_CAP_FILE_CACHE)
2092 if (--ci->i_rdcache_ref == 0)
2093 last++;
2094 if (had & CEPH_CAP_FILE_BUFFER) {
2095 if (--ci->i_wrbuffer_ref == 0) {
2096 last++;
2097 put++;
2098 }
2099 dout("put_cap_refs %p wrbuffer %d -> %d (?)\n",
2100 inode, ci->i_wrbuffer_ref+1, ci->i_wrbuffer_ref);
2101 }
2102 if (had & CEPH_CAP_FILE_WR)
2103 if (--ci->i_wr_ref == 0) {
2104 last++;
2105 if (!list_empty(&ci->i_cap_snaps)) {
2106 capsnap = list_first_entry(&ci->i_cap_snaps,
2107 struct ceph_cap_snap,
2108 ci_item);
2109 if (capsnap->writing) {
2110 capsnap->writing = 0;
2111 flushsnaps =
2112 __ceph_finish_cap_snap(ci,
2113 capsnap);
2114 wake = 1;
2115 }
2116 }
2117 }
2118 spin_unlock(&inode->i_lock);
2119
2120 dout("put_cap_refs %p had %s %s\n", inode, ceph_cap_string(had),
2121 last ? "last" : "");
2122
2123 if (last && !flushsnaps)
2124 ceph_check_caps(ci, 0, NULL);
2125 else if (flushsnaps)
2126 ceph_flush_snaps(ci);
2127 if (wake)
2128 wake_up(&ci->i_cap_wq);
2129 if (put)
2130 iput(inode);
2131}
2132
2133/*
2134 * Release @nr WRBUFFER refs on dirty pages for the given @snapc snap
2135 * context. Adjust per-snap dirty page accounting as appropriate.
2136 * Once all dirty data for a cap_snap is flushed, flush snapped file
2137 * metadata back to the MDS. If we dropped the last ref, call
2138 * ceph_check_caps.
2139 */
2140void ceph_put_wrbuffer_cap_refs(struct ceph_inode_info *ci, int nr,
2141 struct ceph_snap_context *snapc)
2142{
2143 struct inode *inode = &ci->vfs_inode;
2144 int last = 0;
2145 int last_snap = 0;
2146 int found = 0;
2147 struct ceph_cap_snap *capsnap = NULL;
2148
2149 spin_lock(&inode->i_lock);
2150 ci->i_wrbuffer_ref -= nr;
2151 last = !ci->i_wrbuffer_ref;
2152
2153 if (ci->i_head_snapc == snapc) {
2154 ci->i_wrbuffer_ref_head -= nr;
2155 if (!ci->i_wrbuffer_ref_head) {
2156 ceph_put_snap_context(ci->i_head_snapc);
2157 ci->i_head_snapc = NULL;
2158 }
2159 dout("put_wrbuffer_cap_refs on %p head %d/%d -> %d/%d %s\n",
2160 inode,
2161 ci->i_wrbuffer_ref+nr, ci->i_wrbuffer_ref_head+nr,
2162 ci->i_wrbuffer_ref, ci->i_wrbuffer_ref_head,
2163 last ? " LAST" : "");
2164 } else {
2165 list_for_each_entry(capsnap, &ci->i_cap_snaps, ci_item) {
2166 if (capsnap->context == snapc) {
2167 found = 1;
2168 capsnap->dirty_pages -= nr;
2169 last_snap = !capsnap->dirty_pages;
2170 break;
2171 }
2172 }
2173 BUG_ON(!found);
2174 dout("put_wrbuffer_cap_refs on %p cap_snap %p "
2175 " snap %lld %d/%d -> %d/%d %s%s\n",
2176 inode, capsnap, capsnap->context->seq,
2177 ci->i_wrbuffer_ref+nr, capsnap->dirty_pages + nr,
2178 ci->i_wrbuffer_ref, capsnap->dirty_pages,
2179 last ? " (wrbuffer last)" : "",
2180 last_snap ? " (capsnap last)" : "");
2181 }
2182
2183 spin_unlock(&inode->i_lock);
2184
2185 if (last) {
2186 ceph_check_caps(ci, CHECK_CAPS_AUTHONLY, NULL);
2187 iput(inode);
2188 } else if (last_snap) {
2189 ceph_flush_snaps(ci);
2190 wake_up(&ci->i_cap_wq);
2191 }
2192}
2193
2194/*
2195 * Handle a cap GRANT message from the MDS. (Note that a GRANT may
2196 * actually be a revocation if it specifies a smaller cap set.)
2197 *
2198 * caller holds s_mutex.
2199 * return value:
2200 * 0 - ok
2201 * 1 - check_caps on auth cap only (writeback)
2202 * 2 - check_caps (ack revoke)
2203 */
2204static int handle_cap_grant(struct inode *inode, struct ceph_mds_caps *grant,
2205 struct ceph_mds_session *session,
2206 struct ceph_cap *cap,
2207 struct ceph_buffer *xattr_buf)
2208 __releases(inode->i_lock)
2209
2210{
2211 struct ceph_inode_info *ci = ceph_inode(inode);
2212 int mds = session->s_mds;
2213 int seq = le32_to_cpu(grant->seq);
2214 int newcaps = le32_to_cpu(grant->caps);
2215 int issued, implemented, used, wanted, dirty;
2216 u64 size = le64_to_cpu(grant->size);
2217 u64 max_size = le64_to_cpu(grant->max_size);
2218 struct timespec mtime, atime, ctime;
2219 int reply = 0;
2220 int wake = 0;
2221 int writeback = 0;
2222 int revoked_rdcache = 0;
2223 int queue_invalidate = 0;
2224
2225 dout("handle_cap_grant inode %p cap %p mds%d seq %d %s\n",
2226 inode, cap, mds, seq, ceph_cap_string(newcaps));
2227 dout(" size %llu max_size %llu, i_size %llu\n", size, max_size,
2228 inode->i_size);
2229
2230 /*
2231 * If CACHE is being revoked, and we have no dirty buffers,
2232 * try to invalidate (once). (If there are dirty buffers, we
2233 * will invalidate _after_ writeback.)
2234 */
2235 if (((cap->issued & ~newcaps) & CEPH_CAP_FILE_CACHE) &&
2236 !ci->i_wrbuffer_ref) {
2237 if (try_nonblocking_invalidate(inode) == 0) {
2238 revoked_rdcache = 1;
2239 } else {
2240 /* there were locked pages.. invalidate later
2241 in a separate thread. */
2242 if (ci->i_rdcache_revoking != ci->i_rdcache_gen) {
2243 queue_invalidate = 1;
2244 ci->i_rdcache_revoking = ci->i_rdcache_gen;
2245 }
2246 }
2247 }
2248
2249 /* side effects now are allowed */
2250
2251 issued = __ceph_caps_issued(ci, &implemented);
2252 issued |= implemented | __ceph_caps_dirty(ci);
2253
2254 cap->cap_gen = session->s_cap_gen;
2255
2256 __check_cap_issue(ci, cap, newcaps);
2257
2258 if ((issued & CEPH_CAP_AUTH_EXCL) == 0) {
2259 inode->i_mode = le32_to_cpu(grant->mode);
2260 inode->i_uid = le32_to_cpu(grant->uid);
2261 inode->i_gid = le32_to_cpu(grant->gid);
2262 dout("%p mode 0%o uid.gid %d.%d\n", inode, inode->i_mode,
2263 inode->i_uid, inode->i_gid);
2264 }
2265
2266 if ((issued & CEPH_CAP_LINK_EXCL) == 0)
2267 inode->i_nlink = le32_to_cpu(grant->nlink);
2268
2269 if ((issued & CEPH_CAP_XATTR_EXCL) == 0 && grant->xattr_len) {
2270 int len = le32_to_cpu(grant->xattr_len);
2271 u64 version = le64_to_cpu(grant->xattr_version);
2272
2273 if (version > ci->i_xattrs.version) {
2274 dout(" got new xattrs v%llu on %p len %d\n",
2275 version, inode, len);
2276 if (ci->i_xattrs.blob)
2277 ceph_buffer_put(ci->i_xattrs.blob);
2278 ci->i_xattrs.blob = ceph_buffer_get(xattr_buf);
2279 ci->i_xattrs.version = version;
2280 }
2281 }
2282
2283 /* size/ctime/mtime/atime? */
2284 ceph_fill_file_size(inode, issued,
2285 le32_to_cpu(grant->truncate_seq),
2286 le64_to_cpu(grant->truncate_size), size);
2287 ceph_decode_timespec(&mtime, &grant->mtime);
2288 ceph_decode_timespec(&atime, &grant->atime);
2289 ceph_decode_timespec(&ctime, &grant->ctime);
2290 ceph_fill_file_time(inode, issued,
2291 le32_to_cpu(grant->time_warp_seq), &ctime, &mtime,
2292 &atime);
2293
2294 /* max size increase? */
2295 if (max_size != ci->i_max_size) {
2296 dout("max_size %lld -> %llu\n", ci->i_max_size, max_size);
2297 ci->i_max_size = max_size;
2298 if (max_size >= ci->i_wanted_max_size) {
2299 ci->i_wanted_max_size = 0; /* reset */
2300 ci->i_requested_max_size = 0;
2301 }
2302 wake = 1;
2303 }
2304
2305 /* check cap bits */
2306 wanted = __ceph_caps_wanted(ci);
2307 used = __ceph_caps_used(ci);
2308 dirty = __ceph_caps_dirty(ci);
2309 dout(" my wanted = %s, used = %s, dirty %s\n",
2310 ceph_cap_string(wanted),
2311 ceph_cap_string(used),
2312 ceph_cap_string(dirty));
2313 if (wanted != le32_to_cpu(grant->wanted)) {
2314 dout("mds wanted %s -> %s\n",
2315 ceph_cap_string(le32_to_cpu(grant->wanted)),
2316 ceph_cap_string(wanted));
2317 grant->wanted = cpu_to_le32(wanted);
2318 }
2319
2320 cap->seq = seq;
2321
2322 /* file layout may have changed */
2323 ci->i_layout = grant->layout;
2324
2325 /* revocation, grant, or no-op? */
2326 if (cap->issued & ~newcaps) {
2327 dout("revocation: %s -> %s\n", ceph_cap_string(cap->issued),
2328 ceph_cap_string(newcaps));
2329 if ((used & ~newcaps) & CEPH_CAP_FILE_BUFFER)
2330 writeback = 1; /* will delay ack */
2331 else if (dirty & ~newcaps)
2332 reply = 1; /* initiate writeback in check_caps */
2333 else if (((used & ~newcaps) & CEPH_CAP_FILE_CACHE) == 0 ||
2334 revoked_rdcache)
2335 reply = 2; /* send revoke ack in check_caps */
2336 cap->issued = newcaps;
2337 } else if (cap->issued == newcaps) {
2338 dout("caps unchanged: %s -> %s\n",
2339 ceph_cap_string(cap->issued), ceph_cap_string(newcaps));
2340 } else {
2341 dout("grant: %s -> %s\n", ceph_cap_string(cap->issued),
2342 ceph_cap_string(newcaps));
2343 cap->issued = newcaps;
2344 cap->implemented |= newcaps; /* add bits only, to
2345 * avoid stepping on a
2346 * pending revocation */
2347 wake = 1;
2348 }
2349
2350 spin_unlock(&inode->i_lock);
2351 if (writeback)
2352 /*
2353 * queue inode for writeback: we can't actually call
2354 * filemap_write_and_wait, etc. from message handler
2355 * context.
2356 */
2357 ceph_queue_writeback(inode);
2358 if (queue_invalidate)
2359 ceph_queue_invalidate(inode);
2360 if (wake)
2361 wake_up(&ci->i_cap_wq);
2362 return reply;
2363}
2364
2365/*
2366 * Handle FLUSH_ACK from MDS, indicating that metadata we sent to the
2367 * MDS has been safely committed.
2368 */
2369static void handle_cap_flush_ack(struct inode *inode, u64 flush_tid,
2370 struct ceph_mds_caps *m,
2371 struct ceph_mds_session *session,
2372 struct ceph_cap *cap)
2373 __releases(inode->i_lock)
2374{
2375 struct ceph_inode_info *ci = ceph_inode(inode);
2376 struct ceph_mds_client *mdsc = &ceph_client(inode->i_sb)->mdsc;
2377 unsigned seq = le32_to_cpu(m->seq);
2378 int dirty = le32_to_cpu(m->dirty);
2379 int cleaned = 0;
2380 int drop = 0;
2381 int i;
2382
2383 for (i = 0; i < CEPH_CAP_BITS; i++)
2384 if ((dirty & (1 << i)) &&
2385 flush_tid == ci->i_cap_flush_tid[i])
2386 cleaned |= 1 << i;
2387
2388 dout("handle_cap_flush_ack inode %p mds%d seq %d on %s cleaned %s,"
2389 " flushing %s -> %s\n",
2390 inode, session->s_mds, seq, ceph_cap_string(dirty),
2391 ceph_cap_string(cleaned), ceph_cap_string(ci->i_flushing_caps),
2392 ceph_cap_string(ci->i_flushing_caps & ~cleaned));
2393
2394 if (ci->i_flushing_caps == (ci->i_flushing_caps & ~cleaned))
2395 goto out;
2396
2397 ci->i_flushing_caps &= ~cleaned;
2398
2399 spin_lock(&mdsc->cap_dirty_lock);
2400 if (ci->i_flushing_caps == 0) {
2401 list_del_init(&ci->i_flushing_item);
2402 if (!list_empty(&session->s_cap_flushing))
2403 dout(" mds%d still flushing cap on %p\n",
2404 session->s_mds,
2405 &list_entry(session->s_cap_flushing.next,
2406 struct ceph_inode_info,
2407 i_flushing_item)->vfs_inode);
2408 mdsc->num_cap_flushing--;
2409 wake_up(&mdsc->cap_flushing_wq);
2410 dout(" inode %p now !flushing\n", inode);
2411
2412 if (ci->i_dirty_caps == 0) {
2413 dout(" inode %p now clean\n", inode);
2414 BUG_ON(!list_empty(&ci->i_dirty_item));
2415 drop = 1;
2416 } else {
2417 BUG_ON(list_empty(&ci->i_dirty_item));
2418 }
2419 }
2420 spin_unlock(&mdsc->cap_dirty_lock);
2421 wake_up(&ci->i_cap_wq);
2422
2423out:
2424 spin_unlock(&inode->i_lock);
2425 if (drop)
2426 iput(inode);
2427}
2428
2429/*
2430 * Handle FLUSHSNAP_ACK. MDS has flushed snap data to disk and we can
2431 * throw away our cap_snap.
2432 *
2433 * Caller hold s_mutex.
2434 */
2435static void handle_cap_flushsnap_ack(struct inode *inode, u64 flush_tid,
2436 struct ceph_mds_caps *m,
2437 struct ceph_mds_session *session)
2438{
2439 struct ceph_inode_info *ci = ceph_inode(inode);
2440 u64 follows = le64_to_cpu(m->snap_follows);
2441 struct ceph_cap_snap *capsnap;
2442 int drop = 0;
2443
2444 dout("handle_cap_flushsnap_ack inode %p ci %p mds%d follows %lld\n",
2445 inode, ci, session->s_mds, follows);
2446
2447 spin_lock(&inode->i_lock);
2448 list_for_each_entry(capsnap, &ci->i_cap_snaps, ci_item) {
2449 if (capsnap->follows == follows) {
2450 if (capsnap->flush_tid != flush_tid) {
2451 dout(" cap_snap %p follows %lld tid %lld !="
2452 " %lld\n", capsnap, follows,
2453 flush_tid, capsnap->flush_tid);
2454 break;
2455 }
2456 WARN_ON(capsnap->dirty_pages || capsnap->writing);
2457 dout(" removing cap_snap %p follows %lld\n",
2458 capsnap, follows);
2459 ceph_put_snap_context(capsnap->context);
2460 list_del(&capsnap->ci_item);
2461 list_del(&capsnap->flushing_item);
2462 ceph_put_cap_snap(capsnap);
2463 drop = 1;
2464 break;
2465 } else {
2466 dout(" skipping cap_snap %p follows %lld\n",
2467 capsnap, capsnap->follows);
2468 }
2469 }
2470 spin_unlock(&inode->i_lock);
2471 if (drop)
2472 iput(inode);
2473}
2474
2475/*
2476 * Handle TRUNC from MDS, indicating file truncation.
2477 *
2478 * caller hold s_mutex.
2479 */
2480static void handle_cap_trunc(struct inode *inode,
2481 struct ceph_mds_caps *trunc,
2482 struct ceph_mds_session *session)
2483 __releases(inode->i_lock)
2484{
2485 struct ceph_inode_info *ci = ceph_inode(inode);
2486 int mds = session->s_mds;
2487 int seq = le32_to_cpu(trunc->seq);
2488 u32 truncate_seq = le32_to_cpu(trunc->truncate_seq);
2489 u64 truncate_size = le64_to_cpu(trunc->truncate_size);
2490 u64 size = le64_to_cpu(trunc->size);
2491 int implemented = 0;
2492 int dirty = __ceph_caps_dirty(ci);
2493 int issued = __ceph_caps_issued(ceph_inode(inode), &implemented);
2494 int queue_trunc = 0;
2495
2496 issued |= implemented | dirty;
2497
2498 dout("handle_cap_trunc inode %p mds%d seq %d to %lld seq %d\n",
2499 inode, mds, seq, truncate_size, truncate_seq);
2500 queue_trunc = ceph_fill_file_size(inode, issued,
2501 truncate_seq, truncate_size, size);
2502 spin_unlock(&inode->i_lock);
2503
2504 if (queue_trunc)
2505 ceph_queue_vmtruncate(inode);
2506}
2507
2508/*
2509 * Handle EXPORT from MDS. Cap is being migrated _from_ this mds to a
2510 * different one. If we are the most recent migration we've seen (as
2511 * indicated by mseq), make note of the migrating cap bits for the
2512 * duration (until we see the corresponding IMPORT).
2513 *
2514 * caller holds s_mutex
2515 */
2516static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex,
2517 struct ceph_mds_session *session)
2518{
2519 struct ceph_inode_info *ci = ceph_inode(inode);
2520 int mds = session->s_mds;
2521 unsigned mseq = le32_to_cpu(ex->migrate_seq);
2522 struct ceph_cap *cap = NULL, *t;
2523 struct rb_node *p;
2524 int remember = 1;
2525
2526 dout("handle_cap_export inode %p ci %p mds%d mseq %d\n",
2527 inode, ci, mds, mseq);
2528
2529 spin_lock(&inode->i_lock);
2530
2531 /* make sure we haven't seen a higher mseq */
2532 for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
2533 t = rb_entry(p, struct ceph_cap, ci_node);
2534 if (ceph_seq_cmp(t->mseq, mseq) > 0) {
2535 dout(" higher mseq on cap from mds%d\n",
2536 t->session->s_mds);
2537 remember = 0;
2538 }
2539 if (t->session->s_mds == mds)
2540 cap = t;
2541 }
2542
2543 if (cap) {
2544 if (remember) {
2545 /* make note */
2546 ci->i_cap_exporting_mds = mds;
2547 ci->i_cap_exporting_mseq = mseq;
2548 ci->i_cap_exporting_issued = cap->issued;
2549 }
2550 __ceph_remove_cap(cap);
2551 } else {
2552 WARN_ON(!cap);
2553 }
2554
2555 spin_unlock(&inode->i_lock);
2556}
2557
2558/*
2559 * Handle cap IMPORT. If there are temp bits from an older EXPORT,
2560 * clean them up.
2561 *
2562 * caller holds s_mutex.
2563 */
2564static void handle_cap_import(struct ceph_mds_client *mdsc,
2565 struct inode *inode, struct ceph_mds_caps *im,
2566 struct ceph_mds_session *session,
2567 void *snaptrace, int snaptrace_len)
2568{
2569 struct ceph_inode_info *ci = ceph_inode(inode);
2570 int mds = session->s_mds;
2571 unsigned issued = le32_to_cpu(im->caps);
2572 unsigned wanted = le32_to_cpu(im->wanted);
2573 unsigned seq = le32_to_cpu(im->seq);
2574 unsigned mseq = le32_to_cpu(im->migrate_seq);
2575 u64 realmino = le64_to_cpu(im->realm);
2576 u64 cap_id = le64_to_cpu(im->cap_id);
2577
2578 if (ci->i_cap_exporting_mds >= 0 &&
2579 ceph_seq_cmp(ci->i_cap_exporting_mseq, mseq) < 0) {
2580 dout("handle_cap_import inode %p ci %p mds%d mseq %d"
2581 " - cleared exporting from mds%d\n",
2582 inode, ci, mds, mseq,
2583 ci->i_cap_exporting_mds);
2584 ci->i_cap_exporting_issued = 0;
2585 ci->i_cap_exporting_mseq = 0;
2586 ci->i_cap_exporting_mds = -1;
2587 } else {
2588 dout("handle_cap_import inode %p ci %p mds%d mseq %d\n",
2589 inode, ci, mds, mseq);
2590 }
2591
2592 down_write(&mdsc->snap_rwsem);
2593 ceph_update_snap_trace(mdsc, snaptrace, snaptrace+snaptrace_len,
2594 false);
2595 downgrade_write(&mdsc->snap_rwsem);
2596 ceph_add_cap(inode, session, cap_id, -1,
2597 issued, wanted, seq, mseq, realmino, CEPH_CAP_FLAG_AUTH,
2598 NULL /* no caps context */);
2599 try_flush_caps(inode, session, NULL);
2600 up_read(&mdsc->snap_rwsem);
2601}
2602
2603/*
2604 * Handle a caps message from the MDS.
2605 *
2606 * Identify the appropriate session, inode, and call the right handler
2607 * based on the cap op.
2608 */
2609void ceph_handle_caps(struct ceph_mds_session *session,
2610 struct ceph_msg *msg)
2611{
2612 struct ceph_mds_client *mdsc = session->s_mdsc;
2613 struct super_block *sb = mdsc->client->sb;
2614 struct inode *inode;
2615 struct ceph_cap *cap;
2616 struct ceph_mds_caps *h;
2617 int mds = session->s_mds;
2618 int op;
2619 u32 seq;
2620 struct ceph_vino vino;
2621 u64 cap_id;
2622 u64 size, max_size;
2623 u64 tid;
2624 int check_caps = 0;
2625 void *snaptrace;
2626 int r;
2627
2628 dout("handle_caps from mds%d\n", mds);
2629
2630 /* decode */
2631 tid = le64_to_cpu(msg->hdr.tid);
2632 if (msg->front.iov_len < sizeof(*h))
2633 goto bad;
2634 h = msg->front.iov_base;
2635 snaptrace = h + 1;
2636 op = le32_to_cpu(h->op);
2637 vino.ino = le64_to_cpu(h->ino);
2638 vino.snap = CEPH_NOSNAP;
2639 cap_id = le64_to_cpu(h->cap_id);
2640 seq = le32_to_cpu(h->seq);
2641 size = le64_to_cpu(h->size);
2642 max_size = le64_to_cpu(h->max_size);
2643
2644 mutex_lock(&session->s_mutex);
2645 session->s_seq++;
2646 dout(" mds%d seq %lld cap seq %u\n", session->s_mds, session->s_seq,
2647 (unsigned)seq);
2648
2649 /* lookup ino */
2650 inode = ceph_find_inode(sb, vino);
2651 dout(" op %s ino %llx.%llx inode %p\n", ceph_cap_op_name(op), vino.ino,
2652 vino.snap, inode);
2653 if (!inode) {
2654 dout(" i don't have ino %llx\n", vino.ino);
2655 goto done;
2656 }
2657
2658 /* these will work even if we don't have a cap yet */
2659 switch (op) {
2660 case CEPH_CAP_OP_FLUSHSNAP_ACK:
2661 handle_cap_flushsnap_ack(inode, tid, h, session);
2662 goto done;
2663
2664 case CEPH_CAP_OP_EXPORT:
2665 handle_cap_export(inode, h, session);
2666 goto done;
2667
2668 case CEPH_CAP_OP_IMPORT:
2669 handle_cap_import(mdsc, inode, h, session,
2670 snaptrace, le32_to_cpu(h->snap_trace_len));
2671 check_caps = 1; /* we may have sent a RELEASE to the old auth */
2672 goto done;
2673 }
2674
2675 /* the rest require a cap */
2676 spin_lock(&inode->i_lock);
2677 cap = __get_cap_for_mds(ceph_inode(inode), mds);
2678 if (!cap) {
2679 dout("no cap on %p ino %llx.%llx from mds%d, releasing\n",
2680 inode, ceph_ino(inode), ceph_snap(inode), mds);
2681 spin_unlock(&inode->i_lock);
2682 goto done;
2683 }
2684
2685 /* note that each of these drops i_lock for us */
2686 switch (op) {
2687 case CEPH_CAP_OP_REVOKE:
2688 case CEPH_CAP_OP_GRANT:
2689 r = handle_cap_grant(inode, h, session, cap, msg->middle);
2690 if (r == 1)
2691 ceph_check_caps(ceph_inode(inode),
2692 CHECK_CAPS_NODELAY|CHECK_CAPS_AUTHONLY,
2693 session);
2694 else if (r == 2)
2695 ceph_check_caps(ceph_inode(inode),
2696 CHECK_CAPS_NODELAY,
2697 session);
2698 break;
2699
2700 case CEPH_CAP_OP_FLUSH_ACK:
2701 handle_cap_flush_ack(inode, tid, h, session, cap);
2702 break;
2703
2704 case CEPH_CAP_OP_TRUNC:
2705 handle_cap_trunc(inode, h, session);
2706 break;
2707
2708 default:
2709 spin_unlock(&inode->i_lock);
2710 pr_err("ceph_handle_caps: unknown cap op %d %s\n", op,
2711 ceph_cap_op_name(op));
2712 }
2713
2714done:
2715 mutex_unlock(&session->s_mutex);
2716
2717 if (check_caps)
2718 ceph_check_caps(ceph_inode(inode), CHECK_CAPS_NODELAY, NULL);
2719 if (inode)
2720 iput(inode);
2721 return;
2722
2723bad:
2724 pr_err("ceph_handle_caps: corrupt message\n");
2725 ceph_msg_dump(msg);
2726 return;
2727}
2728
2729/*
2730 * Delayed work handler to process end of delayed cap release LRU list.
2731 */
2732void ceph_check_delayed_caps(struct ceph_mds_client *mdsc)
2733{
2734 struct ceph_inode_info *ci;
2735 int flags = CHECK_CAPS_NODELAY;
2736
2737 dout("check_delayed_caps\n");
2738 while (1) {
2739 spin_lock(&mdsc->cap_delay_lock);
2740 if (list_empty(&mdsc->cap_delay_list))
2741 break;
2742 ci = list_first_entry(&mdsc->cap_delay_list,
2743 struct ceph_inode_info,
2744 i_cap_delay_list);
2745 if ((ci->i_ceph_flags & CEPH_I_FLUSH) == 0 &&
2746 time_before(jiffies, ci->i_hold_caps_max))
2747 break;
2748 list_del_init(&ci->i_cap_delay_list);
2749 spin_unlock(&mdsc->cap_delay_lock);
2750 dout("check_delayed_caps on %p\n", &ci->vfs_inode);
2751 ceph_check_caps(ci, flags, NULL);
2752 }
2753 spin_unlock(&mdsc->cap_delay_lock);
2754}
2755
2756/*
2757 * Flush all dirty caps to the mds
2758 */
2759void ceph_flush_dirty_caps(struct ceph_mds_client *mdsc)
2760{
2761 struct ceph_inode_info *ci, *nci = NULL;
2762 struct inode *inode, *ninode = NULL;
2763 struct list_head *p, *n;
2764
2765 dout("flush_dirty_caps\n");
2766 spin_lock(&mdsc->cap_dirty_lock);
2767 list_for_each_safe(p, n, &mdsc->cap_dirty) {
2768 if (nci) {
2769 ci = nci;
2770 inode = ninode;
2771 ci->i_ceph_flags &= ~CEPH_I_NOFLUSH;
2772 dout("flush_dirty_caps inode %p (was next inode)\n",
2773 inode);
2774 } else {
2775 ci = list_entry(p, struct ceph_inode_info,
2776 i_dirty_item);
2777 inode = igrab(&ci->vfs_inode);
2778 BUG_ON(!inode);
2779 dout("flush_dirty_caps inode %p\n", inode);
2780 }
2781 if (n != &mdsc->cap_dirty) {
2782 nci = list_entry(n, struct ceph_inode_info,
2783 i_dirty_item);
2784 ninode = igrab(&nci->vfs_inode);
2785 BUG_ON(!ninode);
2786 nci->i_ceph_flags |= CEPH_I_NOFLUSH;
2787 dout("flush_dirty_caps next inode %p, noflush\n",
2788 ninode);
2789 } else {
2790 nci = NULL;
2791 ninode = NULL;
2792 }
2793 spin_unlock(&mdsc->cap_dirty_lock);
2794 if (inode) {
2795 ceph_check_caps(ci, CHECK_CAPS_NODELAY|CHECK_CAPS_FLUSH,
2796 NULL);
2797 iput(inode);
2798 }
2799 spin_lock(&mdsc->cap_dirty_lock);
2800 }
2801 spin_unlock(&mdsc->cap_dirty_lock);
2802}
2803
2804/*
2805 * Drop open file reference. If we were the last open file,
2806 * we may need to release capabilities to the MDS (or schedule
2807 * their delayed release).
2808 */
2809void ceph_put_fmode(struct ceph_inode_info *ci, int fmode)
2810{
2811 struct inode *inode = &ci->vfs_inode;
2812 int last = 0;
2813
2814 spin_lock(&inode->i_lock);
2815 dout("put_fmode %p fmode %d %d -> %d\n", inode, fmode,
2816 ci->i_nr_by_mode[fmode], ci->i_nr_by_mode[fmode]-1);
2817 BUG_ON(ci->i_nr_by_mode[fmode] == 0);
2818 if (--ci->i_nr_by_mode[fmode] == 0)
2819 last++;
2820 spin_unlock(&inode->i_lock);
2821
2822 if (last && ci->i_vino.snap == CEPH_NOSNAP)
2823 ceph_check_caps(ci, 0, NULL);
2824}
2825
2826/*
2827 * Helpers for embedding cap and dentry lease releases into mds
2828 * requests.
2829 *
2830 * @force is used by dentry_release (below) to force inclusion of a
2831 * record for the directory inode, even when there aren't any caps to
2832 * drop.
2833 */
2834int ceph_encode_inode_release(void **p, struct inode *inode,
2835 int mds, int drop, int unless, int force)
2836{
2837 struct ceph_inode_info *ci = ceph_inode(inode);
2838 struct ceph_cap *cap;
2839 struct ceph_mds_request_release *rel = *p;
2840 int ret = 0;
2841
2842 dout("encode_inode_release %p mds%d drop %s unless %s\n", inode,
2843 mds, ceph_cap_string(drop), ceph_cap_string(unless));
2844
2845 spin_lock(&inode->i_lock);
2846 cap = __get_cap_for_mds(ci, mds);
2847 if (cap && __cap_is_valid(cap)) {
2848 if (force ||
2849 ((cap->issued & drop) &&
2850 (cap->issued & unless) == 0)) {
2851 if ((cap->issued & drop) &&
2852 (cap->issued & unless) == 0) {
2853 dout("encode_inode_release %p cap %p %s -> "
2854 "%s\n", inode, cap,
2855 ceph_cap_string(cap->issued),
2856 ceph_cap_string(cap->issued & ~drop));
2857 cap->issued &= ~drop;
2858 cap->implemented &= ~drop;
2859 if (ci->i_ceph_flags & CEPH_I_NODELAY) {
2860 int wanted = __ceph_caps_wanted(ci);
2861 dout(" wanted %s -> %s (act %s)\n",
2862 ceph_cap_string(cap->mds_wanted),
2863 ceph_cap_string(cap->mds_wanted &
2864 ~wanted),
2865 ceph_cap_string(wanted));
2866 cap->mds_wanted &= wanted;
2867 }
2868 } else {
2869 dout("encode_inode_release %p cap %p %s"
2870 " (force)\n", inode, cap,
2871 ceph_cap_string(cap->issued));
2872 }
2873
2874 rel->ino = cpu_to_le64(ceph_ino(inode));
2875 rel->cap_id = cpu_to_le64(cap->cap_id);
2876 rel->seq = cpu_to_le32(cap->seq);
2877 rel->issue_seq = cpu_to_le32(cap->issue_seq),
2878 rel->mseq = cpu_to_le32(cap->mseq);
2879 rel->caps = cpu_to_le32(cap->issued);
2880 rel->wanted = cpu_to_le32(cap->mds_wanted);
2881 rel->dname_len = 0;
2882 rel->dname_seq = 0;
2883 *p += sizeof(*rel);
2884 ret = 1;
2885 } else {
2886 dout("encode_inode_release %p cap %p %s\n",
2887 inode, cap, ceph_cap_string(cap->issued));
2888 }
2889 }
2890 spin_unlock(&inode->i_lock);
2891 return ret;
2892}
2893
2894int ceph_encode_dentry_release(void **p, struct dentry *dentry,
2895 int mds, int drop, int unless)
2896{
2897 struct inode *dir = dentry->d_parent->d_inode;
2898 struct ceph_mds_request_release *rel = *p;
2899 struct ceph_dentry_info *di = ceph_dentry(dentry);
2900 int force = 0;
2901 int ret;
2902
2903 /*
2904 * force an record for the directory caps if we have a dentry lease.
2905 * this is racy (can't take i_lock and d_lock together), but it
2906 * doesn't have to be perfect; the mds will revoke anything we don't
2907 * release.
2908 */
2909 spin_lock(&dentry->d_lock);
2910 if (di->lease_session && di->lease_session->s_mds == mds)
2911 force = 1;
2912 spin_unlock(&dentry->d_lock);
2913
2914 ret = ceph_encode_inode_release(p, dir, mds, drop, unless, force);
2915
2916 spin_lock(&dentry->d_lock);
2917 if (ret && di->lease_session && di->lease_session->s_mds == mds) {
2918 dout("encode_dentry_release %p mds%d seq %d\n",
2919 dentry, mds, (int)di->lease_seq);
2920 rel->dname_len = cpu_to_le32(dentry->d_name.len);
2921 memcpy(*p, dentry->d_name.name, dentry->d_name.len);
2922 *p += dentry->d_name.len;
2923 rel->dname_seq = cpu_to_le32(di->lease_seq);
2924 }
2925 spin_unlock(&dentry->d_lock);
2926 return ret;
2927}
fs/ceph/ceph_debug.h+37−0
@@ -0,0 +1,37 @@
1#ifndef _FS_CEPH_DEBUG_H
2#define _FS_CEPH_DEBUG_H
3
4#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
5
6#ifdef CONFIG_CEPH_FS_PRETTYDEBUG
7
8/*
9 * wrap pr_debug to include a filename:lineno prefix on each line.
10 * this incurs some overhead (kernel size and execution time) due to
11 * the extra function call at each call site.
12 */
13
14# if defined(DEBUG) || defined(CONFIG_DYNAMIC_DEBUG)
15extern const char *ceph_file_part(const char *s, int len);
16# define dout(fmt, ...) \
17 pr_debug(" %12.12s:%-4d : " fmt, \
18 ceph_file_part(__FILE__, sizeof(__FILE__)), \
19 __LINE__, ##__VA_ARGS__)
20# else
21/* faux printk call just to see any compiler warnings. */
22# define dout(fmt, ...) do { \
23 if (0) \
24 printk(KERN_DEBUG fmt, ##__VA_ARGS__); \
25 } while (0)
26# endif
27
28#else
29
30/*
31 * or, just wrap pr_debug
32 */
33# define dout(fmt, ...) pr_debug(" " fmt, ##__VA_ARGS__)
34
35#endif
36
37#endif
fs/ceph/ceph_frag.c+21−0
@@ -0,0 +1,21 @@
1/*
2 * Ceph 'frag' type
3 */
4#include "types.h"
5
6int ceph_frag_compare(__u32 a, __u32 b)
7{
8 unsigned va = ceph_frag_value(a);
9 unsigned vb = ceph_frag_value(b);
10 if (va < vb)
11 return -1;
12 if (va > vb)
13 return 1;
14 va = ceph_frag_bits(a);
15 vb = ceph_frag_bits(b);
16 if (va < vb)
17 return -1;
18 if (va > vb)
19 return 1;
20 return 0;
21}
fs/ceph/ceph_frag.h+109−0
@@ -0,0 +1,109 @@
1#ifndef _FS_CEPH_FRAG_H
2#define _FS_CEPH_FRAG_H
3
4/*
5 * "Frags" are a way to describe a subset of a 32-bit number space,
6 * using a mask and a value to match against that mask. Any given frag
7 * (subset of the number space) can be partitioned into 2^n sub-frags.
8 *
9 * Frags are encoded into a 32-bit word:
10 * 8 upper bits = "bits"
11 * 24 lower bits = "value"
12 * (We could go to 5+27 bits, but who cares.)
13 *
14 * We use the _most_ significant bits of the 24 bit value. This makes
15 * values logically sort.
16 *
17 * Unfortunately, because the "bits" field is still in the high bits, we
18 * can't sort encoded frags numerically. However, it does allow you
19 * to feed encoded frags as values into frag_contains_value.
20 */
21static inline __u32 ceph_frag_make(__u32 b, __u32 v)
22{
23 return (b << 24) |
24 (v & (0xffffffu << (24-b)) & 0xffffffu);
25}
26static inline __u32 ceph_frag_bits(__u32 f)
27{
28 return f >> 24;
29}
30static inline __u32 ceph_frag_value(__u32 f)
31{
32 return f & 0xffffffu;
33}
34static inline __u32 ceph_frag_mask(__u32 f)
35{
36 return (0xffffffu << (24-ceph_frag_bits(f))) & 0xffffffu;
37}
38static inline __u32 ceph_frag_mask_shift(__u32 f)
39{
40 return 24 - ceph_frag_bits(f);
41}
42
43static inline int ceph_frag_contains_value(__u32 f, __u32 v)
44{
45 return (v & ceph_frag_mask(f)) == ceph_frag_value(f);
46}
47static inline int ceph_frag_contains_frag(__u32 f, __u32 sub)
48{
49 /* is sub as specific as us, and contained by us? */
50 return ceph_frag_bits(sub) >= ceph_frag_bits(f) &&
51 (ceph_frag_value(sub) & ceph_frag_mask(f)) == ceph_frag_value(f);
52}
53
54static inline __u32 ceph_frag_parent(__u32 f)
55{
56 return ceph_frag_make(ceph_frag_bits(f) - 1,
57 ceph_frag_value(f) & (ceph_frag_mask(f) << 1));
58}
59static inline int ceph_frag_is_left_child(__u32 f)
60{
61 return ceph_frag_bits(f) > 0 &&
62 (ceph_frag_value(f) & (0x1000000 >> ceph_frag_bits(f))) == 0;
63}
64static inline int ceph_frag_is_right_child(__u32 f)
65{
66 return ceph_frag_bits(f) > 0 &&
67 (ceph_frag_value(f) & (0x1000000 >> ceph_frag_bits(f))) == 1;
68}
69static inline __u32 ceph_frag_sibling(__u32 f)
70{
71 return ceph_frag_make(ceph_frag_bits(f),
72 ceph_frag_value(f) ^ (0x1000000 >> ceph_frag_bits(f)));
73}
74static inline __u32 ceph_frag_left_child(__u32 f)
75{
76 return ceph_frag_make(ceph_frag_bits(f)+1, ceph_frag_value(f));
77}
78static inline __u32 ceph_frag_right_child(__u32 f)
79{
80 return ceph_frag_make(ceph_frag_bits(f)+1,
81 ceph_frag_value(f) | (0x1000000 >> (1+ceph_frag_bits(f))));
82}
83static inline __u32 ceph_frag_make_child(__u32 f, int by, int i)
84{
85 int newbits = ceph_frag_bits(f) + by;
86 return ceph_frag_make(newbits,
87 ceph_frag_value(f) | (i << (24 - newbits)));
88}
89static inline int ceph_frag_is_leftmost(__u32 f)
90{
91 return ceph_frag_value(f) == 0;
92}
93static inline int ceph_frag_is_rightmost(__u32 f)
94{
95 return ceph_frag_value(f) == ceph_frag_mask(f);
96}
97static inline __u32 ceph_frag_next(__u32 f)
98{
99 return ceph_frag_make(ceph_frag_bits(f),
100 ceph_frag_value(f) + (0x1000000 >> ceph_frag_bits(f)));
101}
102
103/*
104 * comparator to sort frags logically, as when traversing the
105 * number space in ascending order...
106 */
107int ceph_frag_compare(__u32 a, __u32 b);
108
109#endif
fs/ceph/ceph_fs.c+74−0
@@ -0,0 +1,74 @@
1/*
2 * Some non-inline ceph helpers
3 */
4#include "types.h"
5
6/*
7 * return true if @layout appears to be valid
8 */
9int ceph_file_layout_is_valid(const struct ceph_file_layout *layout)
10{
11 __u32 su = le32_to_cpu(layout->fl_stripe_unit);
12 __u32 sc = le32_to_cpu(layout->fl_stripe_count);
13 __u32 os = le32_to_cpu(layout->fl_object_size);
14
15 /* stripe unit, object size must be non-zero, 64k increment */
16 if (!su || (su & (CEPH_MIN_STRIPE_UNIT-1)))
17 return 0;
18 if (!os || (os & (CEPH_MIN_STRIPE_UNIT-1)))
19 return 0;
20 /* object size must be a multiple of stripe unit */
21 if (os < su || os % su)
22 return 0;
23 /* stripe count must be non-zero */
24 if (!sc)
25 return 0;
26 return 1;
27}
28
29
30int ceph_flags_to_mode(int flags)
31{
32#ifdef O_DIRECTORY /* fixme */
33 if ((flags & O_DIRECTORY) == O_DIRECTORY)
34 return CEPH_FILE_MODE_PIN;
35#endif
36#ifdef O_LAZY
37 if (flags & O_LAZY)
38 return CEPH_FILE_MODE_LAZY;
39#endif
40 if ((flags & O_APPEND) == O_APPEND)
41 flags |= O_WRONLY;
42
43 flags &= O_ACCMODE;
44 if ((flags & O_RDWR) == O_RDWR)
45 return CEPH_FILE_MODE_RDWR;
46 if ((flags & O_WRONLY) == O_WRONLY)
47 return CEPH_FILE_MODE_WR;
48 return CEPH_FILE_MODE_RD;
49}
50
51int ceph_caps_for_mode(int mode)
52{
53 switch (mode) {
54 case CEPH_FILE_MODE_PIN:
55 return CEPH_CAP_PIN;
56 case CEPH_FILE_MODE_RD:
57 return CEPH_CAP_PIN | CEPH_CAP_FILE_SHARED |
58 CEPH_CAP_FILE_RD | CEPH_CAP_FILE_CACHE;
59 case CEPH_FILE_MODE_RDWR:
60 return CEPH_CAP_PIN | CEPH_CAP_FILE_SHARED |
61 CEPH_CAP_FILE_EXCL |
62 CEPH_CAP_FILE_RD | CEPH_CAP_FILE_CACHE |
63 CEPH_CAP_FILE_WR | CEPH_CAP_FILE_BUFFER |
64 CEPH_CAP_AUTH_SHARED | CEPH_CAP_AUTH_EXCL |
65 CEPH_CAP_XATTR_SHARED | CEPH_CAP_XATTR_EXCL;
66 case CEPH_FILE_MODE_WR:
67 return CEPH_CAP_PIN | CEPH_CAP_FILE_SHARED |
68 CEPH_CAP_FILE_EXCL |
69 CEPH_CAP_FILE_WR | CEPH_CAP_FILE_BUFFER |
70 CEPH_CAP_AUTH_SHARED | CEPH_CAP_AUTH_EXCL |
71 CEPH_CAP_XATTR_SHARED | CEPH_CAP_XATTR_EXCL;
72 }
73 return 0;
74}
fs/ceph/ceph_fs.h+650−0
@@ -0,0 +1,650 @@
1/*
2 * ceph_fs.h - Ceph constants and data types to share between kernel and
3 * user space.
4 *
5 * Most types in this file are defined as little-endian, and are
6 * primarily intended to describe data structures that pass over the
7 * wire or that are stored on disk.
8 *
9 * LGPL2
10 */
11
12#ifndef _FS_CEPH_CEPH_FS_H
13#define _FS_CEPH_CEPH_FS_H
14
15#include "msgr.h"
16#include "rados.h"
17
18/*
19 * Ceph release version
20 */
21#define CEPH_VERSION_MAJOR 0
22#define CEPH_VERSION_MINOR 19
23#define CEPH_VERSION_PATCH 0
24
25#define _CEPH_STRINGIFY(x) #x
26#define CEPH_STRINGIFY(x) _CEPH_STRINGIFY(x)
27#define CEPH_MAKE_VERSION(x, y, z) CEPH_STRINGIFY(x) "." CEPH_STRINGIFY(y) \
28 "." CEPH_STRINGIFY(z)
29#define CEPH_VERSION CEPH_MAKE_VERSION(CEPH_VERSION_MAJOR, \
30 CEPH_VERSION_MINOR, CEPH_VERSION_PATCH)
31
32/*
33 * subprotocol versions. when specific messages types or high-level
34 * protocols change, bump the affected components. we keep rev
35 * internal cluster protocols separately from the public,
36 * client-facing protocol.
37 */
38#define CEPH_OSD_PROTOCOL 8 /* cluster internal */
39#define CEPH_MDS_PROTOCOL 9 /* cluster internal */
40#define CEPH_MON_PROTOCOL 5 /* cluster internal */
41#define CEPH_OSDC_PROTOCOL 24 /* server/client */
42#define CEPH_MDSC_PROTOCOL 32 /* server/client */
43#define CEPH_MONC_PROTOCOL 15 /* server/client */
44
45
46#define CEPH_INO_ROOT 1
47#define CEPH_INO_CEPH 2 /* hidden .ceph dir */
48
49/* arbitrary limit on max # of monitors (cluster of 3 is typical) */
50#define CEPH_MAX_MON 31
51
52
53/*
54 * feature bits
55 */
56#define CEPH_FEATURE_SUPPORTED 0
57#define CEPH_FEATURE_REQUIRED 0
58
59
60/*
61 * ceph_file_layout - describe data layout for a file/inode
62 */
63struct ceph_file_layout {
64 /* file -> object mapping */
65 __le32 fl_stripe_unit; /* stripe unit, in bytes. must be multiple
66 of page size. */
67 __le32 fl_stripe_count; /* over this many objects */
68 __le32 fl_object_size; /* until objects are this big, then move to
69 new objects */
70 __le32 fl_cas_hash; /* 0 = none; 1 = sha256 */
71
72 /* pg -> disk layout */
73 __le32 fl_object_stripe_unit; /* for per-object parity, if any */
74
75 /* object -> pg layout */
76 __le32 fl_pg_preferred; /* preferred primary for pg (-1 for none) */
77 __le32 fl_pg_pool; /* namespace, crush ruleset, rep level */
78} __attribute__ ((packed));
79
80#define CEPH_MIN_STRIPE_UNIT 65536
81
82int ceph_file_layout_is_valid(const struct ceph_file_layout *layout);
83
84
85/* crypto algorithms */
86#define CEPH_CRYPTO_NONE 0x0
87#define CEPH_CRYPTO_AES 0x1
88
89/* security/authentication protocols */
90#define CEPH_AUTH_UNKNOWN 0x0
91#define CEPH_AUTH_NONE 0x1
92#define CEPH_AUTH_CEPHX 0x2
93
94
95/*********************************************
96 * message layer
97 */
98
99/*
100 * message types
101 */
102
103/* misc */
104#define CEPH_MSG_SHUTDOWN 1
105#define CEPH_MSG_PING 2
106
107/* client <-> monitor */
108#define CEPH_MSG_MON_MAP 4
109#define CEPH_MSG_MON_GET_MAP 5
110#define CEPH_MSG_STATFS 13
111#define CEPH_MSG_STATFS_REPLY 14
112#define CEPH_MSG_MON_SUBSCRIBE 15
113#define CEPH_MSG_MON_SUBSCRIBE_ACK 16
114#define CEPH_MSG_AUTH 17
115#define CEPH_MSG_AUTH_REPLY 18
116
117/* client <-> mds */
118#define CEPH_MSG_MDS_MAP 21
119
120#define CEPH_MSG_CLIENT_SESSION 22
121#define CEPH_MSG_CLIENT_RECONNECT 23
122
123#define CEPH_MSG_CLIENT_REQUEST 24
124#define CEPH_MSG_CLIENT_REQUEST_FORWARD 25
125#define CEPH_MSG_CLIENT_REPLY 26
126#define CEPH_MSG_CLIENT_CAPS 0x310
127#define CEPH_MSG_CLIENT_LEASE 0x311
128#define CEPH_MSG_CLIENT_SNAP 0x312
129#define CEPH_MSG_CLIENT_CAPRELEASE 0x313
130
131/* osd */
132#define CEPH_MSG_OSD_MAP 41
133#define CEPH_MSG_OSD_OP 42
134#define CEPH_MSG_OSD_OPREPLY 43
135
136struct ceph_mon_request_header {
137 __le64 have_version;
138 __le16 session_mon;
139 __le64 session_mon_tid;
140} __attribute__ ((packed));
141
142struct ceph_mon_statfs {
143 struct ceph_mon_request_header monhdr;
144 struct ceph_fsid fsid;
145} __attribute__ ((packed));
146
147struct ceph_statfs {
148 __le64 kb, kb_used, kb_avail;
149 __le64 num_objects;
150} __attribute__ ((packed));
151
152struct ceph_mon_statfs_reply {
153 struct ceph_fsid fsid;
154 __le64 version;
155 struct ceph_statfs st;
156} __attribute__ ((packed));
157
158struct ceph_osd_getmap {
159 struct ceph_mon_request_header monhdr;
160 struct ceph_fsid fsid;
161 __le32 start;
162} __attribute__ ((packed));
163
164struct ceph_mds_getmap {
165 struct ceph_mon_request_header monhdr;
166 struct ceph_fsid fsid;
167} __attribute__ ((packed));
168
169struct ceph_client_mount {
170 struct ceph_mon_request_header monhdr;
171} __attribute__ ((packed));
172
173struct ceph_mon_subscribe_item {
174 __le64 have_version; __le64 have;
175 __u8 onetime;
176} __attribute__ ((packed));
177
178struct ceph_mon_subscribe_ack {
179 __le32 duration; /* seconds */
180 struct ceph_fsid fsid;
181} __attribute__ ((packed));
182
183/*
184 * mds states
185 * > 0 -> in
186 * <= 0 -> out
187 */
188#define CEPH_MDS_STATE_DNE 0 /* down, does not exist. */
189#define CEPH_MDS_STATE_STOPPED -1 /* down, once existed, but no subtrees.
190 empty log. */
191#define CEPH_MDS_STATE_BOOT -4 /* up, boot announcement. */
192#define CEPH_MDS_STATE_STANDBY -5 /* up, idle. waiting for assignment. */
193#define CEPH_MDS_STATE_CREATING -6 /* up, creating MDS instance. */
194#define CEPH_MDS_STATE_STARTING -7 /* up, starting previously stopped mds */
195#define CEPH_MDS_STATE_STANDBY_REPLAY -8 /* up, tailing active node's journal */
196
197#define CEPH_MDS_STATE_REPLAY 8 /* up, replaying journal. */
198#define CEPH_MDS_STATE_RESOLVE 9 /* up, disambiguating distributed
199 operations (import, rename, etc.) */
200#define CEPH_MDS_STATE_RECONNECT 10 /* up, reconnect to clients */
201#define CEPH_MDS_STATE_REJOIN 11 /* up, rejoining distributed cache */
202#define CEPH_MDS_STATE_CLIENTREPLAY 12 /* up, replaying client operations */
203#define CEPH_MDS_STATE_ACTIVE 13 /* up, active */
204#define CEPH_MDS_STATE_STOPPING 14 /* up, but exporting metadata */
205
206extern const char *ceph_mds_state_name(int s);
207
208
209/*
210 * metadata lock types.
211 * - these are bitmasks.. we can compose them
212 * - they also define the lock ordering by the MDS
213 * - a few of these are internal to the mds
214 */
215#define CEPH_LOCK_DN 1
216#define CEPH_LOCK_ISNAP 2
217#define CEPH_LOCK_IVERSION 4 /* mds internal */
218#define CEPH_LOCK_IFILE 8 /* mds internal */
219#define CEPH_LOCK_IAUTH 32
220#define CEPH_LOCK_ILINK 64
221#define CEPH_LOCK_IDFT 128 /* dir frag tree */
222#define CEPH_LOCK_INEST 256 /* mds internal */
223#define CEPH_LOCK_IXATTR 512
224#define CEPH_LOCK_INO 2048 /* immutable inode bits; not a lock */
225
226/* client_session ops */
227enum {
228 CEPH_SESSION_REQUEST_OPEN,
229 CEPH_SESSION_OPEN,
230 CEPH_SESSION_REQUEST_CLOSE,
231 CEPH_SESSION_CLOSE,
232 CEPH_SESSION_REQUEST_RENEWCAPS,
233 CEPH_SESSION_RENEWCAPS,
234 CEPH_SESSION_STALE,
235 CEPH_SESSION_RECALL_STATE,
236};
237
238extern const char *ceph_session_op_name(int op);
239
240struct ceph_mds_session_head {
241 __le32 op;
242 __le64 seq;
243 struct ceph_timespec stamp;
244 __le32 max_caps, max_leases;
245} __attribute__ ((packed));
246
247/* client_request */
248/*
249 * metadata ops.
250 * & 0x001000 -> write op
251 * & 0x010000 -> follow symlink (e.g. stat(), not lstat()).
252 & & 0x100000 -> use weird ino/path trace
253 */
254#define CEPH_MDS_OP_WRITE 0x001000
255enum {
256 CEPH_MDS_OP_LOOKUP = 0x00100,
257 CEPH_MDS_OP_GETATTR = 0x00101,
258 CEPH_MDS_OP_LOOKUPHASH = 0x00102,
259 CEPH_MDS_OP_LOOKUPPARENT = 0x00103,
260
261 CEPH_MDS_OP_SETXATTR = 0x01105,
262 CEPH_MDS_OP_RMXATTR = 0x01106,
263 CEPH_MDS_OP_SETLAYOUT = 0x01107,
264 CEPH_MDS_OP_SETATTR = 0x01108,
265
266 CEPH_MDS_OP_MKNOD = 0x01201,
267 CEPH_MDS_OP_LINK = 0x01202,
268 CEPH_MDS_OP_UNLINK = 0x01203,
269 CEPH_MDS_OP_RENAME = 0x01204,
270 CEPH_MDS_OP_MKDIR = 0x01220,
271 CEPH_MDS_OP_RMDIR = 0x01221,
272 CEPH_MDS_OP_SYMLINK = 0x01222,
273
274 CEPH_MDS_OP_CREATE = 0x01301,
275 CEPH_MDS_OP_OPEN = 0x00302,
276 CEPH_MDS_OP_READDIR = 0x00305,
277
278 CEPH_MDS_OP_LOOKUPSNAP = 0x00400,
279 CEPH_MDS_OP_MKSNAP = 0x01400,
280 CEPH_MDS_OP_RMSNAP = 0x01401,
281 CEPH_MDS_OP_LSSNAP = 0x00402,
282};
283
284extern const char *ceph_mds_op_name(int op);
285
286
287#define CEPH_SETATTR_MODE 1
288#define CEPH_SETATTR_UID 2
289#define CEPH_SETATTR_GID 4
290#define CEPH_SETATTR_MTIME 8
291#define CEPH_SETATTR_ATIME 16
292#define CEPH_SETATTR_SIZE 32
293#define CEPH_SETATTR_CTIME 64
294
295union ceph_mds_request_args {
296 struct {
297 __le32 mask; /* CEPH_CAP_* */
298 } __attribute__ ((packed)) getattr;
299 struct {
300 __le32 mode;
301 __le32 uid;
302 __le32 gid;
303 struct ceph_timespec mtime;
304 struct ceph_timespec atime;
305 __le64 size, old_size; /* old_size needed by truncate */
306 __le32 mask; /* CEPH_SETATTR_* */
307 } __attribute__ ((packed)) setattr;
308 struct {
309 __le32 frag; /* which dir fragment */
310 __le32 max_entries; /* how many dentries to grab */
311 } __attribute__ ((packed)) readdir;
312 struct {
313 __le32 mode;
314 __le32 rdev;
315 } __attribute__ ((packed)) mknod;
316 struct {
317 __le32 mode;
318 } __attribute__ ((packed)) mkdir;
319 struct {
320 __le32 flags;
321 __le32 mode;
322 __le32 stripe_unit; /* layout for newly created file */
323 __le32 stripe_count; /* ... */
324 __le32 object_size;
325 __le32 file_replication;
326 __le32 preferred;
327 } __attribute__ ((packed)) open;
328 struct {
329 __le32 flags;
330 } __attribute__ ((packed)) setxattr;
331 struct {
332 struct ceph_file_layout layout;
333 } __attribute__ ((packed)) setlayout;
334} __attribute__ ((packed));
335
336#define CEPH_MDS_FLAG_REPLAY 1 /* this is a replayed op */
337#define CEPH_MDS_FLAG_WANT_DENTRY 2 /* want dentry in reply */
338
339struct ceph_mds_request_head {
340 __le64 oldest_client_tid;
341 __le32 mdsmap_epoch; /* on client */
342 __le32 flags; /* CEPH_MDS_FLAG_* */
343 __u8 num_retry, num_fwd; /* count retry, fwd attempts */
344 __le16 num_releases; /* # include cap/lease release records */
345 __le32 op; /* mds op code */
346 __le32 caller_uid, caller_gid;
347 __le64 ino; /* use this ino for openc, mkdir, mknod,
348 etc. (if replaying) */
349 union ceph_mds_request_args args;
350} __attribute__ ((packed));
351
352/* cap/lease release record */
353struct ceph_mds_request_release {
354 __le64 ino, cap_id; /* ino and unique cap id */
355 __le32 caps, wanted; /* new issued, wanted */
356 __le32 seq, issue_seq, mseq;
357 __le32 dname_seq; /* if releasing a dentry lease, a */
358 __le32 dname_len; /* string follows. */
359} __attribute__ ((packed));
360
361/* client reply */
362struct ceph_mds_reply_head {
363 __le32 op;
364 __le32 result;
365 __le32 mdsmap_epoch;
366 __u8 safe; /* true if committed to disk */
367 __u8 is_dentry, is_target; /* true if dentry, target inode records
368 are included with reply */
369} __attribute__ ((packed));
370
371/* one for each node split */
372struct ceph_frag_tree_split {
373 __le32 frag; /* this frag splits... */
374 __le32 by; /* ...by this many bits */
375} __attribute__ ((packed));
376
377struct ceph_frag_tree_head {
378 __le32 nsplits; /* num ceph_frag_tree_split records */
379 struct ceph_frag_tree_split splits[];
380} __attribute__ ((packed));
381
382/* capability issue, for bundling with mds reply */
383struct ceph_mds_reply_cap {
384 __le32 caps, wanted; /* caps issued, wanted */
385 __le64 cap_id;
386 __le32 seq, mseq;
387 __le64 realm; /* snap realm */
388 __u8 flags; /* CEPH_CAP_FLAG_* */
389} __attribute__ ((packed));
390
391#define CEPH_CAP_FLAG_AUTH 1 /* cap is issued by auth mds */
392
393/* inode record, for bundling with mds reply */
394struct ceph_mds_reply_inode {
395 __le64 ino;
396 __le64 snapid;
397 __le32 rdev;
398 __le64 version; /* inode version */
399 __le64 xattr_version; /* version for xattr blob */
400 struct ceph_mds_reply_cap cap; /* caps issued for this inode */
401 struct ceph_file_layout layout;
402 struct ceph_timespec ctime, mtime, atime;
403 __le32 time_warp_seq;
404 __le64 size, max_size, truncate_size;
405 __le32 truncate_seq;
406 __le32 mode, uid, gid;
407 __le32 nlink;
408 __le64 files, subdirs, rbytes, rfiles, rsubdirs; /* dir stats */
409 struct ceph_timespec rctime;
410 struct ceph_frag_tree_head fragtree; /* (must be at end of struct) */
411} __attribute__ ((packed));
412/* followed by frag array, then symlink string, then xattr blob */
413
414/* reply_lease follows dname, and reply_inode */
415struct ceph_mds_reply_lease {
416 __le16 mask; /* lease type(s) */
417 __le32 duration_ms; /* lease duration */
418 __le32 seq;
419} __attribute__ ((packed));
420
421struct ceph_mds_reply_dirfrag {
422 __le32 frag; /* fragment */
423 __le32 auth; /* auth mds, if this is a delegation point */
424 __le32 ndist; /* number of mds' this is replicated on */
425 __le32 dist[];
426} __attribute__ ((packed));
427
428/* file access modes */
429#define CEPH_FILE_MODE_PIN 0
430#define CEPH_FILE_MODE_RD 1
431#define CEPH_FILE_MODE_WR 2
432#define CEPH_FILE_MODE_RDWR 3 /* RD | WR */
433#define CEPH_FILE_MODE_LAZY 4 /* lazy io */
434#define CEPH_FILE_MODE_NUM 8 /* bc these are bit fields.. mostly */
435
436int ceph_flags_to_mode(int flags);
437
438
439/* capability bits */
440#define CEPH_CAP_PIN 1 /* no specific capabilities beyond the pin */
441
442/* generic cap bits */
443#define CEPH_CAP_GSHARED 1 /* client can reads */
444#define CEPH_CAP_GEXCL 2 /* client can read and update */
445#define CEPH_CAP_GCACHE 4 /* (file) client can cache reads */
446#define CEPH_CAP_GRD 8 /* (file) client can read */
447#define CEPH_CAP_GWR 16 /* (file) client can write */
448#define CEPH_CAP_GBUFFER 32 /* (file) client can buffer writes */
449#define CEPH_CAP_GWREXTEND 64 /* (file) client can extend EOF */
450#define CEPH_CAP_GLAZYIO 128 /* (file) client can perform lazy io */
451
452/* per-lock shift */
453#define CEPH_CAP_SAUTH 2
454#define CEPH_CAP_SLINK 4
455#define CEPH_CAP_SXATTR 6
456#define CEPH_CAP_SFILE 8 /* goes at the end (uses >2 cap bits) */
457
458#define CEPH_CAP_BITS 16
459
460/* composed values */
461#define CEPH_CAP_AUTH_SHARED (CEPH_CAP_GSHARED << CEPH_CAP_SAUTH)
462#define CEPH_CAP_AUTH_EXCL (CEPH_CAP_GEXCL << CEPH_CAP_SAUTH)
463#define CEPH_CAP_LINK_SHARED (CEPH_CAP_GSHARED << CEPH_CAP_SLINK)
464#define CEPH_CAP_LINK_EXCL (CEPH_CAP_GEXCL << CEPH_CAP_SLINK)
465#define CEPH_CAP_XATTR_SHARED (CEPH_CAP_GSHARED << CEPH_CAP_SXATTR)
466#define CEPH_CAP_XATTR_EXCL (CEPH_CAP_GEXCL << CEPH_CAP_SXATTR)
467#define CEPH_CAP_FILE(x) (x << CEPH_CAP_SFILE)
468#define CEPH_CAP_FILE_SHARED (CEPH_CAP_GSHARED << CEPH_CAP_SFILE)
469#define CEPH_CAP_FILE_EXCL (CEPH_CAP_GEXCL << CEPH_CAP_SFILE)
470#define CEPH_CAP_FILE_CACHE (CEPH_CAP_GCACHE << CEPH_CAP_SFILE)
471#define CEPH_CAP_FILE_RD (CEPH_CAP_GRD << CEPH_CAP_SFILE)
472#define CEPH_CAP_FILE_WR (CEPH_CAP_GWR << CEPH_CAP_SFILE)
473#define CEPH_CAP_FILE_BUFFER (CEPH_CAP_GBUFFER << CEPH_CAP_SFILE)
474#define CEPH_CAP_FILE_WREXTEND (CEPH_CAP_GWREXTEND << CEPH_CAP_SFILE)
475#define CEPH_CAP_FILE_LAZYIO (CEPH_CAP_GLAZYIO << CEPH_CAP_SFILE)
476
477/* cap masks (for getattr) */
478#define CEPH_STAT_CAP_INODE CEPH_CAP_PIN
479#define CEPH_STAT_CAP_TYPE CEPH_CAP_PIN /* mode >> 12 */
480#define CEPH_STAT_CAP_SYMLINK CEPH_CAP_PIN
481#define CEPH_STAT_CAP_UID CEPH_CAP_AUTH_SHARED
482#define CEPH_STAT_CAP_GID CEPH_CAP_AUTH_SHARED
483#define CEPH_STAT_CAP_MODE CEPH_CAP_AUTH_SHARED
484#define CEPH_STAT_CAP_NLINK CEPH_CAP_LINK_SHARED
485#define CEPH_STAT_CAP_LAYOUT CEPH_CAP_FILE_SHARED
486#define CEPH_STAT_CAP_MTIME CEPH_CAP_FILE_SHARED
487#define CEPH_STAT_CAP_SIZE CEPH_CAP_FILE_SHARED
488#define CEPH_STAT_CAP_ATIME CEPH_CAP_FILE_SHARED /* fixme */
489#define CEPH_STAT_CAP_XATTR CEPH_CAP_XATTR_SHARED
490#define CEPH_STAT_CAP_INODE_ALL (CEPH_CAP_PIN | \
491 CEPH_CAP_AUTH_SHARED | \
492 CEPH_CAP_LINK_SHARED | \
493 CEPH_CAP_FILE_SHARED | \
494 CEPH_CAP_XATTR_SHARED)
495
496#define CEPH_CAP_ANY_SHARED (CEPH_CAP_AUTH_SHARED | \
497 CEPH_CAP_LINK_SHARED | \
498 CEPH_CAP_XATTR_SHARED | \
499 CEPH_CAP_FILE_SHARED)
500#define CEPH_CAP_ANY_RD (CEPH_CAP_ANY_SHARED | CEPH_CAP_FILE_RD | \
501 CEPH_CAP_FILE_CACHE)
502
503#define CEPH_CAP_ANY_EXCL (CEPH_CAP_AUTH_EXCL | \
504 CEPH_CAP_LINK_EXCL | \
505 CEPH_CAP_XATTR_EXCL | \
506 CEPH_CAP_FILE_EXCL)
507#define CEPH_CAP_ANY_FILE_WR (CEPH_CAP_FILE_WR | CEPH_CAP_FILE_BUFFER | \
508 CEPH_CAP_FILE_EXCL)
509#define CEPH_CAP_ANY_WR (CEPH_CAP_ANY_EXCL | CEPH_CAP_ANY_FILE_WR)
510#define CEPH_CAP_ANY (CEPH_CAP_ANY_RD | CEPH_CAP_ANY_EXCL | \
511 CEPH_CAP_ANY_FILE_WR | CEPH_CAP_PIN)
512
513#define CEPH_CAP_LOCKS (CEPH_LOCK_IFILE | CEPH_LOCK_IAUTH | CEPH_LOCK_ILINK | \
514 CEPH_LOCK_IXATTR)
515
516int ceph_caps_for_mode(int mode);
517
518enum {
519 CEPH_CAP_OP_GRANT, /* mds->client grant */
520 CEPH_CAP_OP_REVOKE, /* mds->client revoke */
521 CEPH_CAP_OP_TRUNC, /* mds->client trunc notify */
522 CEPH_CAP_OP_EXPORT, /* mds has exported the cap */
523 CEPH_CAP_OP_IMPORT, /* mds has imported the cap */
524 CEPH_CAP_OP_UPDATE, /* client->mds update */
525 CEPH_CAP_OP_DROP, /* client->mds drop cap bits */
526 CEPH_CAP_OP_FLUSH, /* client->mds cap writeback */
527 CEPH_CAP_OP_FLUSH_ACK, /* mds->client flushed */
528 CEPH_CAP_OP_FLUSHSNAP, /* client->mds flush snapped metadata */
529 CEPH_CAP_OP_FLUSHSNAP_ACK, /* mds->client flushed snapped metadata */
530 CEPH_CAP_OP_RELEASE, /* client->mds release (clean) cap */
531 CEPH_CAP_OP_RENEW, /* client->mds renewal request */
532};
533
534extern const char *ceph_cap_op_name(int op);
535
536/*
537 * caps message, used for capability callbacks, acks, requests, etc.
538 */
539struct ceph_mds_caps {
540 __le32 op; /* CEPH_CAP_OP_* */
541 __le64 ino, realm;
542 __le64 cap_id;
543 __le32 seq, issue_seq;
544 __le32 caps, wanted, dirty; /* latest issued/wanted/dirty */
545 __le32 migrate_seq;
546 __le64 snap_follows;
547 __le32 snap_trace_len;
548
549 /* authlock */
550 __le32 uid, gid, mode;
551
552 /* linklock */
553 __le32 nlink;
554
555 /* xattrlock */
556 __le32 xattr_len;
557 __le64 xattr_version;
558
559 /* filelock */
560 __le64 size, max_size, truncate_size;
561 __le32 truncate_seq;
562 struct ceph_timespec mtime, atime, ctime;
563 struct ceph_file_layout layout;
564 __le32 time_warp_seq;
565} __attribute__ ((packed));
566
567/* cap release msg head */
568struct ceph_mds_cap_release {
569 __le32 num; /* number of cap_items that follow */
570} __attribute__ ((packed));
571
572struct ceph_mds_cap_item {
573 __le64 ino;
574 __le64 cap_id;
575 __le32 migrate_seq, seq;
576} __attribute__ ((packed));
577
578#define CEPH_MDS_LEASE_REVOKE 1 /* mds -> client */
579#define CEPH_MDS_LEASE_RELEASE 2 /* client -> mds */
580#define CEPH_MDS_LEASE_RENEW 3 /* client <-> mds */
581#define CEPH_MDS_LEASE_REVOKE_ACK 4 /* client -> mds */
582
583extern const char *ceph_lease_op_name(int o);
584
585/* lease msg header */
586struct ceph_mds_lease {
587 __u8 action; /* CEPH_MDS_LEASE_* */
588 __le16 mask; /* which lease */
589 __le64 ino;
590 __le64 first, last; /* snap range */
591 __le32 seq;
592 __le32 duration_ms; /* duration of renewal */
593} __attribute__ ((packed));
594/* followed by a __le32+string for dname */
595
596/* client reconnect */
597struct ceph_mds_cap_reconnect {
598 __le64 cap_id;
599 __le32 wanted;
600 __le32 issued;
601 __le64 size;
602 struct ceph_timespec mtime, atime;
603 __le64 snaprealm;
604 __le64 pathbase; /* base ino for our path to this ino */
605} __attribute__ ((packed));
606/* followed by encoded string */
607
608struct ceph_mds_snaprealm_reconnect {
609 __le64 ino; /* snap realm base */
610 __le64 seq; /* snap seq for this snap realm */
611 __le64 parent; /* parent realm */
612} __attribute__ ((packed));
613
614/*
615 * snaps
616 */
617enum {
618 CEPH_SNAP_OP_UPDATE, /* CREATE or DESTROY */
619 CEPH_SNAP_OP_CREATE,
620 CEPH_SNAP_OP_DESTROY,
621 CEPH_SNAP_OP_SPLIT,
622};
623
624extern const char *ceph_snap_op_name(int o);
625
626/* snap msg header */
627struct ceph_mds_snap_head {
628 __le32 op; /* CEPH_SNAP_OP_* */
629 __le64 split; /* ino to split off, if any */
630 __le32 num_split_inos; /* # inos belonging to new child realm */
631 __le32 num_split_realms; /* # child realms udner new child realm */
632 __le32 trace_len; /* size of snap trace blob */
633} __attribute__ ((packed));
634/* followed by split ino list, then split realms, then the trace blob */
635
636/*
637 * encode info about a snaprealm, as viewed by a client
638 */
639struct ceph_mds_snap_realm {
640 __le64 ino; /* ino */
641 __le64 created; /* snap: when created */
642 __le64 parent; /* ino: parent realm */
643 __le64 parent_since; /* snap: same parent since */
644 __le64 seq; /* snap: version */
645 __le32 num_snaps;
646 __le32 num_prior_parent_snaps;
647} __attribute__ ((packed));
648/* followed by my snap list, then prior parent snap list */
649
650#endif
fs/ceph/ceph_hash.c+118−0
@@ -0,0 +1,118 @@
1
2#include "types.h"
3
4/*
5 * Robert Jenkin's hash function.
6 * http://burtleburtle.net/bob/hash/evahash.html
7 * This is in the public domain.
8 */
9#define mix(a, b, c) \
10 do { \
11 a = a - b; a = a - c; a = a ^ (c >> 13); \
12 b = b - c; b = b - a; b = b ^ (a << 8); \
13 c = c - a; c = c - b; c = c ^ (b >> 13); \
14 a = a - b; a = a - c; a = a ^ (c >> 12); \
15 b = b - c; b = b - a; b = b ^ (a << 16); \
16 c = c - a; c = c - b; c = c ^ (b >> 5); \
17 a = a - b; a = a - c; a = a ^ (c >> 3); \
18 b = b - c; b = b - a; b = b ^ (a << 10); \
19 c = c - a; c = c - b; c = c ^ (b >> 15); \
20 } while (0)
21
22unsigned ceph_str_hash_rjenkins(const char *str, unsigned length)
23{
24 const unsigned char *k = (const unsigned char *)str;
25 __u32 a, b, c; /* the internal state */
26 __u32 len; /* how many key bytes still need mixing */
27
28 /* Set up the internal state */
29 len = length;
30 a = 0x9e3779b9; /* the golden ratio; an arbitrary value */
31 b = a;
32 c = 0; /* variable initialization of internal state */
33
34 /* handle most of the key */
35 while (len >= 12) {
36 a = a + (k[0] + ((__u32)k[1] << 8) + ((__u32)k[2] << 16) +
37 ((__u32)k[3] << 24));
38 b = b + (k[4] + ((__u32)k[5] << 8) + ((__u32)k[6] << 16) +
39 ((__u32)k[7] << 24));
40 c = c + (k[8] + ((__u32)k[9] << 8) + ((__u32)k[10] << 16) +
41 ((__u32)k[11] << 24));
42 mix(a, b, c);
43 k = k + 12;
44 len = len - 12;
45 }
46
47 /* handle the last 11 bytes */
48 c = c + length;
49 switch (len) { /* all the case statements fall through */
50 case 11:
51 c = c + ((__u32)k[10] << 24);
52 case 10:
53 c = c + ((__u32)k[9] << 16);
54 case 9:
55 c = c + ((__u32)k[8] << 8);
56 /* the first byte of c is reserved for the length */
57 case 8:
58 b = b + ((__u32)k[7] << 24);
59 case 7:
60 b = b + ((__u32)k[6] << 16);
61 case 6:
62 b = b + ((__u32)k[5] << 8);
63 case 5:
64 b = b + k[4];
65 case 4:
66 a = a + ((__u32)k[3] << 24);
67 case 3:
68 a = a + ((__u32)k[2] << 16);
69 case 2:
70 a = a + ((__u32)k[1] << 8);
71 case 1:
72 a = a + k[0];
73 /* case 0: nothing left to add */
74 }
75 mix(a, b, c);
76
77 return c;
78}
79
80/*
81 * linux dcache hash
82 */
83unsigned ceph_str_hash_linux(const char *str, unsigned length)
84{
85 unsigned long hash = 0;
86 unsigned char c;
87
88 while (length--) {
89 c = *str++;
90 hash = (hash + (c << 4) + (c >> 4)) * 11;
91 }
92 return hash;
93}
94
95
96unsigned ceph_str_hash(int type, const char *s, unsigned len)
97{
98 switch (type) {
99 case CEPH_STR_HASH_LINUX:
100 return ceph_str_hash_linux(s, len);
101 case CEPH_STR_HASH_RJENKINS:
102 return ceph_str_hash_rjenkins(s, len);
103 default:
104 return -1;
105 }
106}
107
108const char *ceph_str_hash_name(int type)
109{
110 switch (type) {
111 case CEPH_STR_HASH_LINUX:
112 return "linux";
113 case CEPH_STR_HASH_RJENKINS:
114 return "rjenkins";
115 default:
116 return "unknown";
117 }
118}
fs/ceph/ceph_hash.h+13−0
@@ -0,0 +1,13 @@
1#ifndef _FS_CEPH_HASH_H
2#define _FS_CEPH_HASH_H
3
4#define CEPH_STR_HASH_LINUX 0x1 /* linux dcache hash */
5#define CEPH_STR_HASH_RJENKINS 0x2 /* robert jenkins' */
6
7extern unsigned ceph_str_hash_linux(const char *s, unsigned len);
8extern unsigned ceph_str_hash_rjenkins(const char *s, unsigned len);
9
10extern unsigned ceph_str_hash(int type, const char *s, unsigned len);
11extern const char *ceph_str_hash_name(int type);
12
13#endif
fs/ceph/ceph_strings.c+176−0
@@ -0,0 +1,176 @@
1/*
2 * Ceph string constants
3 */
4#include "types.h"
5
6const char *ceph_entity_type_name(int type)
7{
8 switch (type) {
9 case CEPH_ENTITY_TYPE_MDS: return "mds";
10 case CEPH_ENTITY_TYPE_OSD: return "osd";
11 case CEPH_ENTITY_TYPE_MON: return "mon";
12 case CEPH_ENTITY_TYPE_CLIENT: return "client";
13 case CEPH_ENTITY_TYPE_ADMIN: return "admin";
14 case CEPH_ENTITY_TYPE_AUTH: return "auth";
15 default: return "unknown";
16 }
17}
18
19const char *ceph_osd_op_name(int op)
20{
21 switch (op) {
22 case CEPH_OSD_OP_READ: return "read";
23 case CEPH_OSD_OP_STAT: return "stat";
24
25 case CEPH_OSD_OP_MASKTRUNC: return "masktrunc";
26
27 case CEPH_OSD_OP_WRITE: return "write";
28 case CEPH_OSD_OP_DELETE: return "delete";
29 case CEPH_OSD_OP_TRUNCATE: return "truncate";
30 case CEPH_OSD_OP_ZERO: return "zero";
31 case CEPH_OSD_OP_WRITEFULL: return "writefull";
32
33 case CEPH_OSD_OP_APPEND: return "append";
34 case CEPH_OSD_OP_STARTSYNC: return "startsync";
35 case CEPH_OSD_OP_SETTRUNC: return "settrunc";
36 case CEPH_OSD_OP_TRIMTRUNC: return "trimtrunc";
37
38 case CEPH_OSD_OP_TMAPUP: return "tmapup";
39 case CEPH_OSD_OP_TMAPGET: return "tmapget";
40 case CEPH_OSD_OP_TMAPPUT: return "tmapput";
41
42 case CEPH_OSD_OP_GETXATTR: return "getxattr";
43 case CEPH_OSD_OP_GETXATTRS: return "getxattrs";
44 case CEPH_OSD_OP_SETXATTR: return "setxattr";
45 case CEPH_OSD_OP_SETXATTRS: return "setxattrs";
46 case CEPH_OSD_OP_RESETXATTRS: return "resetxattrs";
47 case CEPH_OSD_OP_RMXATTR: return "rmxattr";
48
49 case CEPH_OSD_OP_PULL: return "pull";
50 case CEPH_OSD_OP_PUSH: return "push";
51 case CEPH_OSD_OP_BALANCEREADS: return "balance-reads";
52 case CEPH_OSD_OP_UNBALANCEREADS: return "unbalance-reads";
53 case CEPH_OSD_OP_SCRUB: return "scrub";
54
55 case CEPH_OSD_OP_WRLOCK: return "wrlock";
56 case CEPH_OSD_OP_WRUNLOCK: return "wrunlock";
57 case CEPH_OSD_OP_RDLOCK: return "rdlock";
58 case CEPH_OSD_OP_RDUNLOCK: return "rdunlock";
59 case CEPH_OSD_OP_UPLOCK: return "uplock";
60 case CEPH_OSD_OP_DNLOCK: return "dnlock";
61
62 case CEPH_OSD_OP_CALL: return "call";
63
64 case CEPH_OSD_OP_PGLS: return "pgls";
65 }
66 return "???";
67}
68
69const char *ceph_mds_state_name(int s)
70{
71 switch (s) {
72 /* down and out */
73 case CEPH_MDS_STATE_DNE: return "down:dne";
74 case CEPH_MDS_STATE_STOPPED: return "down:stopped";
75 /* up and out */
76 case CEPH_MDS_STATE_BOOT: return "up:boot";
77 case CEPH_MDS_STATE_STANDBY: return "up:standby";
78 case CEPH_MDS_STATE_STANDBY_REPLAY: return "up:standby-replay";
79 case CEPH_MDS_STATE_CREATING: return "up:creating";
80 case CEPH_MDS_STATE_STARTING: return "up:starting";
81 /* up and in */
82 case CEPH_MDS_STATE_REPLAY: return "up:replay";
83 case CEPH_MDS_STATE_RESOLVE: return "up:resolve";
84 case CEPH_MDS_STATE_RECONNECT: return "up:reconnect";
85 case CEPH_MDS_STATE_REJOIN: return "up:rejoin";
86 case CEPH_MDS_STATE_CLIENTREPLAY: return "up:clientreplay";
87 case CEPH_MDS_STATE_ACTIVE: return "up:active";
88 case CEPH_MDS_STATE_STOPPING: return "up:stopping";
89 }
90 return "???";
91}
92
93const char *ceph_session_op_name(int op)
94{
95 switch (op) {
96 case CEPH_SESSION_REQUEST_OPEN: return "request_open";
97 case CEPH_SESSION_OPEN: return "open";
98 case CEPH_SESSION_REQUEST_CLOSE: return "request_close";
99 case CEPH_SESSION_CLOSE: return "close";
100 case CEPH_SESSION_REQUEST_RENEWCAPS: return "request_renewcaps";
101 case CEPH_SESSION_RENEWCAPS: return "renewcaps";
102 case CEPH_SESSION_STALE: return "stale";
103 case CEPH_SESSION_RECALL_STATE: return "recall_state";
104 }
105 return "???";
106}
107
108const char *ceph_mds_op_name(int op)
109{
110 switch (op) {
111 case CEPH_MDS_OP_LOOKUP: return "lookup";
112 case CEPH_MDS_OP_LOOKUPHASH: return "lookuphash";
113 case CEPH_MDS_OP_LOOKUPPARENT: return "lookupparent";
114 case CEPH_MDS_OP_GETATTR: return "getattr";
115 case CEPH_MDS_OP_SETXATTR: return "setxattr";
116 case CEPH_MDS_OP_SETATTR: return "setattr";
117 case CEPH_MDS_OP_RMXATTR: return "rmxattr";
118 case CEPH_MDS_OP_READDIR: return "readdir";
119 case CEPH_MDS_OP_MKNOD: return "mknod";
120 case CEPH_MDS_OP_LINK: return "link";
121 case CEPH_MDS_OP_UNLINK: return "unlink";
122 case CEPH_MDS_OP_RENAME: return "rename";
123 case CEPH_MDS_OP_MKDIR: return "mkdir";
124 case CEPH_MDS_OP_RMDIR: return "rmdir";
125 case CEPH_MDS_OP_SYMLINK: return "symlink";
126 case CEPH_MDS_OP_CREATE: return "create";
127 case CEPH_MDS_OP_OPEN: return "open";
128 case CEPH_MDS_OP_LOOKUPSNAP: return "lookupsnap";
129 case CEPH_MDS_OP_LSSNAP: return "lssnap";
130 case CEPH_MDS_OP_MKSNAP: return "mksnap";
131 case CEPH_MDS_OP_RMSNAP: return "rmsnap";
132 }
133 return "???";
134}
135
136const char *ceph_cap_op_name(int op)
137{
138 switch (op) {
139 case CEPH_CAP_OP_GRANT: return "grant";
140 case CEPH_CAP_OP_REVOKE: return "revoke";
141 case CEPH_CAP_OP_TRUNC: return "trunc";
142 case CEPH_CAP_OP_EXPORT: return "export";
143 case CEPH_CAP_OP_IMPORT: return "import";
144 case CEPH_CAP_OP_UPDATE: return "update";
145 case CEPH_CAP_OP_DROP: return "drop";
146 case CEPH_CAP_OP_FLUSH: return "flush";
147 case CEPH_CAP_OP_FLUSH_ACK: return "flush_ack";
148 case CEPH_CAP_OP_FLUSHSNAP: return "flushsnap";
149 case CEPH_CAP_OP_FLUSHSNAP_ACK: return "flushsnap_ack";
150 case CEPH_CAP_OP_RELEASE: return "release";
151 case CEPH_CAP_OP_RENEW: return "renew";
152 }
153 return "???";
154}
155
156const char *ceph_lease_op_name(int o)
157{
158 switch (o) {
159 case CEPH_MDS_LEASE_REVOKE: return "revoke";
160 case CEPH_MDS_LEASE_RELEASE: return "release";
161 case CEPH_MDS_LEASE_RENEW: return "renew";
162 case CEPH_MDS_LEASE_REVOKE_ACK: return "revoke_ack";
163 }
164 return "???";
165}
166
167const char *ceph_snap_op_name(int o)
168{
169 switch (o) {
170 case CEPH_SNAP_OP_UPDATE: return "update";
171 case CEPH_SNAP_OP_CREATE: return "create";
172 case CEPH_SNAP_OP_DESTROY: return "destroy";
173 case CEPH_SNAP_OP_SPLIT: return "split";
174 }
175 return "???";
176}
fs/ceph/crush/crush.c+151−0
@@ -0,0 +1,151 @@
1
2#ifdef __KERNEL__
3# include <linux/slab.h>
4#else
5# include <stdlib.h>
6# include <assert.h>
7# define kfree(x) do { if (x) free(x); } while (0)
8# define BUG_ON(x) assert(!(x))
9#endif
10
11#include "crush.h"
12
13const char *crush_bucket_alg_name(int alg)
14{
15 switch (alg) {
16 case CRUSH_BUCKET_UNIFORM: return "uniform";
17 case CRUSH_BUCKET_LIST: return "list";
18 case CRUSH_BUCKET_TREE: return "tree";
19 case CRUSH_BUCKET_STRAW: return "straw";
20 default: return "unknown";
21 }
22}
23
24/**
25 * crush_get_bucket_item_weight - Get weight of an item in given bucket
26 * @b: bucket pointer
27 * @p: item index in bucket
28 */
29int crush_get_bucket_item_weight(struct crush_bucket *b, int p)
30{
31 if (p >= b->size)
32 return 0;
33
34 switch (b->alg) {
35 case CRUSH_BUCKET_UNIFORM:
36 return ((struct crush_bucket_uniform *)b)->item_weight;
37 case CRUSH_BUCKET_LIST:
38 return ((struct crush_bucket_list *)b)->item_weights[p];
39 case CRUSH_BUCKET_TREE:
40 if (p & 1)
41 return ((struct crush_bucket_tree *)b)->node_weights[p];
42 return 0;
43 case CRUSH_BUCKET_STRAW:
44 return ((struct crush_bucket_straw *)b)->item_weights[p];
45 }
46 return 0;
47}
48
49/**
50 * crush_calc_parents - Calculate parent vectors for the given crush map.
51 * @map: crush_map pointer
52 */
53void crush_calc_parents(struct crush_map *map)
54{
55 int i, b, c;
56
57 for (b = 0; b < map->max_buckets; b++) {
58 if (map->buckets[b] == NULL)
59 continue;
60 for (i = 0; i < map->buckets[b]->size; i++) {
61 c = map->buckets[b]->items[i];
62 BUG_ON(c >= map->max_devices ||
63 c < -map->max_buckets);
64 if (c >= 0)
65 map->device_parents[c] = map->buckets[b]->id;
66 else
67 map->bucket_parents[-1-c] = map->buckets[b]->id;
68 }
69 }
70}
71
72void crush_destroy_bucket_uniform(struct crush_bucket_uniform *b)
73{
74 kfree(b->h.perm);
75 kfree(b->h.items);
76 kfree(b);
77}
78
79void crush_destroy_bucket_list(struct crush_bucket_list *b)
80{
81 kfree(b->item_weights);
82 kfree(b->sum_weights);
83 kfree(b->h.perm);
84 kfree(b->h.items);
85 kfree(b);
86}
87
88void crush_destroy_bucket_tree(struct crush_bucket_tree *b)
89{
90 kfree(b->node_weights);
91 kfree(b);
92}
93
94void crush_destroy_bucket_straw(struct crush_bucket_straw *b)
95{
96 kfree(b->straws);
97 kfree(b->item_weights);
98 kfree(b->h.perm);
99 kfree(b->h.items);
100 kfree(b);
101}
102
103void crush_destroy_bucket(struct crush_bucket *b)
104{
105 switch (b->alg) {
106 case CRUSH_BUCKET_UNIFORM:
107 crush_destroy_bucket_uniform((struct crush_bucket_uniform *)b);
108 break;
109 case CRUSH_BUCKET_LIST:
110 crush_destroy_bucket_list((struct crush_bucket_list *)b);
111 break;
112 case CRUSH_BUCKET_TREE:
113 crush_destroy_bucket_tree((struct crush_bucket_tree *)b);
114 break;
115 case CRUSH_BUCKET_STRAW:
116 crush_destroy_bucket_straw((struct crush_bucket_straw *)b);
117 break;
118 }
119}
120
121/**
122 * crush_destroy - Destroy a crush_map
123 * @map: crush_map pointer
124 */
125void crush_destroy(struct crush_map *map)
126{
127 int b;
128
129 /* buckets */
130 if (map->buckets) {
131 for (b = 0; b < map->max_buckets; b++) {
132 if (map->buckets[b] == NULL)
133 continue;
134 crush_destroy_bucket(map->buckets[b]);
135 }
136 kfree(map->buckets);
137 }
138
139 /* rules */
140 if (map->rules) {
141 for (b = 0; b < map->max_rules; b++)
142 kfree(map->rules[b]);
143 kfree(map->rules);
144 }
145
146 kfree(map->bucket_parents);
147 kfree(map->device_parents);
148 kfree(map);
149}
150
151
fs/ceph/crush/crush.h+180−0
@@ -0,0 +1,180 @@
1#ifndef _CRUSH_CRUSH_H
2#define _CRUSH_CRUSH_H
3
4#include <linux/types.h>
5
6/*
7 * CRUSH is a pseudo-random data distribution algorithm that
8 * efficiently distributes input values (typically, data objects)
9 * across a heterogeneous, structured storage cluster.
10 *
11 * The algorithm was originally described in detail in this paper
12 * (although the algorithm has evolved somewhat since then):
13 *
14 * http://www.ssrc.ucsc.edu/Papers/weil-sc06.pdf
15 *
16 * LGPL2
17 */
18
19
20#define CRUSH_MAGIC 0x00010000ul /* for detecting algorithm revisions */
21
22
23#define CRUSH_MAX_DEPTH 10 /* max crush hierarchy depth */
24#define CRUSH_MAX_SET 10 /* max size of a mapping result */
25
26
27/*
28 * CRUSH uses user-defined "rules" to describe how inputs should be
29 * mapped to devices. A rule consists of sequence of steps to perform
30 * to generate the set of output devices.
31 */
32struct crush_rule_step {
33 __u32 op;
34 __s32 arg1;
35 __s32 arg2;
36};
37
38/* step op codes */
39enum {
40 CRUSH_RULE_NOOP = 0,
41 CRUSH_RULE_TAKE = 1, /* arg1 = value to start with */
42 CRUSH_RULE_CHOOSE_FIRSTN = 2, /* arg1 = num items to pick */
43 /* arg2 = type */
44 CRUSH_RULE_CHOOSE_INDEP = 3, /* same */
45 CRUSH_RULE_EMIT = 4, /* no args */
46 CRUSH_RULE_CHOOSE_LEAF_FIRSTN = 6,
47 CRUSH_RULE_CHOOSE_LEAF_INDEP = 7,
48};
49
50/*
51 * for specifying choose num (arg1) relative to the max parameter
52 * passed to do_rule
53 */
54#define CRUSH_CHOOSE_N 0
55#define CRUSH_CHOOSE_N_MINUS(x) (-(x))
56
57/*
58 * The rule mask is used to describe what the rule is intended for.
59 * Given a ruleset and size of output set, we search through the
60 * rule list for a matching rule_mask.
61 */
62struct crush_rule_mask {
63 __u8 ruleset;
64 __u8 type;
65 __u8 min_size;
66 __u8 max_size;
67};
68
69struct crush_rule {
70 __u32 len;
71 struct crush_rule_mask mask;
72 struct crush_rule_step steps[0];
73};
74
75#define crush_rule_size(len) (sizeof(struct crush_rule) + \
76 (len)*sizeof(struct crush_rule_step))
77
78
79
80/*
81 * A bucket is a named container of other items (either devices or
82 * other buckets). Items within a bucket are chosen using one of a
83 * few different algorithms. The table summarizes how the speed of
84 * each option measures up against mapping stability when items are
85 * added or removed.
86 *
87 * Bucket Alg Speed Additions Removals
88 * ------------------------------------------------
89 * uniform O(1) poor poor
90 * list O(n) optimal poor
91 * tree O(log n) good good
92 * straw O(n) optimal optimal
93 */
94enum {
95 CRUSH_BUCKET_UNIFORM = 1,
96 CRUSH_BUCKET_LIST = 2,
97 CRUSH_BUCKET_TREE = 3,
98 CRUSH_BUCKET_STRAW = 4
99};
100extern const char *crush_bucket_alg_name(int alg);
101
102struct crush_bucket {
103 __s32 id; /* this'll be negative */
104 __u16 type; /* non-zero; type=0 is reserved for devices */
105 __u8 alg; /* one of CRUSH_BUCKET_* */
106 __u8 hash; /* which hash function to use, CRUSH_HASH_* */
107 __u32 weight; /* 16-bit fixed point */
108 __u32 size; /* num items */
109 __s32 *items;
110
111 /*
112 * cached random permutation: used for uniform bucket and for
113 * the linear search fallback for the other bucket types.
114 */
115 __u32 perm_x; /* @x for which *perm is defined */
116 __u32 perm_n; /* num elements of *perm that are permuted/defined */
117 __u32 *perm;
118};
119
120struct crush_bucket_uniform {
121 struct crush_bucket h;
122 __u32 item_weight; /* 16-bit fixed point; all items equally weighted */
123};
124
125struct crush_bucket_list {
126 struct crush_bucket h;
127 __u32 *item_weights; /* 16-bit fixed point */
128 __u32 *sum_weights; /* 16-bit fixed point. element i is sum
129 of weights 0..i, inclusive */
130};
131
132struct crush_bucket_tree {
133 struct crush_bucket h; /* note: h.size is _tree_ size, not number of
134 actual items */
135 __u8 num_nodes;
136 __u32 *node_weights;
137};
138
139struct crush_bucket_straw {
140 struct crush_bucket h;
141 __u32 *item_weights; /* 16-bit fixed point */
142 __u32 *straws; /* 16-bit fixed point */
143};
144
145
146
147/*
148 * CRUSH map includes all buckets, rules, etc.
149 */
150struct crush_map {
151 struct crush_bucket **buckets;
152 struct crush_rule **rules;
153
154 /*
155 * Parent pointers to identify the parent bucket a device or
156 * bucket in the hierarchy. If an item appears more than
157 * once, this is the _last_ time it appeared (where buckets
158 * are processed in bucket id order, from -1 on down to
159 * -max_buckets.
160 */
161 __u32 *bucket_parents;
162 __u32 *device_parents;
163
164 __s32 max_buckets;
165 __u32 max_rules;
166 __s32 max_devices;
167};
168
169
170/* crush.c */
171extern int crush_get_bucket_item_weight(struct crush_bucket *b, int pos);
172extern void crush_calc_parents(struct crush_map *map);
173extern void crush_destroy_bucket_uniform(struct crush_bucket_uniform *b);
174extern void crush_destroy_bucket_list(struct crush_bucket_list *b);
175extern void crush_destroy_bucket_tree(struct crush_bucket_tree *b);
176extern void crush_destroy_bucket_straw(struct crush_bucket_straw *b);
177extern void crush_destroy_bucket(struct crush_bucket *b);
178extern void crush_destroy(struct crush_map *map);
179
180#endif
fs/ceph/crush/hash.c+149−0
@@ -0,0 +1,149 @@
1
2#include <linux/types.h>
3#include "hash.h"
4
5/*
6 * Robert Jenkins' function for mixing 32-bit values
7 * http://burtleburtle.net/bob/hash/evahash.html
8 * a, b = random bits, c = input and output
9 */
10#define crush_hashmix(a, b, c) do { \
11 a = a-b; a = a-c; a = a^(c>>13); \
12 b = b-c; b = b-a; b = b^(a<<8); \
13 c = c-a; c = c-b; c = c^(b>>13); \
14 a = a-b; a = a-c; a = a^(c>>12); \
15 b = b-c; b = b-a; b = b^(a<<16); \
16 c = c-a; c = c-b; c = c^(b>>5); \
17 a = a-b; a = a-c; a = a^(c>>3); \
18 b = b-c; b = b-a; b = b^(a<<10); \
19 c = c-a; c = c-b; c = c^(b>>15); \
20 } while (0)
21
22#define crush_hash_seed 1315423911
23
24static __u32 crush_hash32_rjenkins1(__u32 a)
25{
26 __u32 hash = crush_hash_seed ^ a;
27 __u32 b = a;
28 __u32 x = 231232;
29 __u32 y = 1232;
30 crush_hashmix(b, x, hash);
31 crush_hashmix(y, a, hash);
32 return hash;
33}
34
35static __u32 crush_hash32_rjenkins1_2(__u32 a, __u32 b)
36{
37 __u32 hash = crush_hash_seed ^ a ^ b;
38 __u32 x = 231232;
39 __u32 y = 1232;
40 crush_hashmix(a, b, hash);
41 crush_hashmix(x, a, hash);
42 crush_hashmix(b, y, hash);
43 return hash;
44}
45
46static __u32 crush_hash32_rjenkins1_3(__u32 a, __u32 b, __u32 c)
47{
48 __u32 hash = crush_hash_seed ^ a ^ b ^ c;
49 __u32 x = 231232;
50 __u32 y = 1232;
51 crush_hashmix(a, b, hash);
52 crush_hashmix(c, x, hash);
53 crush_hashmix(y, a, hash);
54 crush_hashmix(b, x, hash);
55 crush_hashmix(y, c, hash);
56 return hash;
57}
58
59static __u32 crush_hash32_rjenkins1_4(__u32 a, __u32 b, __u32 c, __u32 d)
60{
61 __u32 hash = crush_hash_seed ^ a ^ b ^ c ^ d;
62 __u32 x = 231232;
63 __u32 y = 1232;
64 crush_hashmix(a, b, hash);
65 crush_hashmix(c, d, hash);
66 crush_hashmix(a, x, hash);
67 crush_hashmix(y, b, hash);
68 crush_hashmix(c, x, hash);
69 crush_hashmix(y, d, hash);
70 return hash;
71}
72
73static __u32 crush_hash32_rjenkins1_5(__u32 a, __u32 b, __u32 c, __u32 d,
74 __u32 e)
75{
76 __u32 hash = crush_hash_seed ^ a ^ b ^ c ^ d ^ e;
77 __u32 x = 231232;
78 __u32 y = 1232;
79 crush_hashmix(a, b, hash);
80 crush_hashmix(c, d, hash);
81 crush_hashmix(e, x, hash);
82 crush_hashmix(y, a, hash);
83 crush_hashmix(b, x, hash);
84 crush_hashmix(y, c, hash);
85 crush_hashmix(d, x, hash);
86 crush_hashmix(y, e, hash);
87 return hash;
88}
89
90
91__u32 crush_hash32(int type, __u32 a)
92{
93 switch (type) {
94 case CRUSH_HASH_RJENKINS1:
95 return crush_hash32_rjenkins1(a);
96 default:
97 return 0;
98 }
99}
100
101__u32 crush_hash32_2(int type, __u32 a, __u32 b)
102{
103 switch (type) {
104 case CRUSH_HASH_RJENKINS1:
105 return crush_hash32_rjenkins1_2(a, b);
106 default:
107 return 0;
108 }
109}
110
111__u32 crush_hash32_3(int type, __u32 a, __u32 b, __u32 c)
112{
113 switch (type) {
114 case CRUSH_HASH_RJENKINS1:
115 return crush_hash32_rjenkins1_3(a, b, c);
116 default:
117 return 0;
118 }
119}
120
121__u32 crush_hash32_4(int type, __u32 a, __u32 b, __u32 c, __u32 d)
122{
123 switch (type) {
124 case CRUSH_HASH_RJENKINS1:
125 return crush_hash32_rjenkins1_4(a, b, c, d);
126 default:
127 return 0;
128 }
129}
130
131__u32 crush_hash32_5(int type, __u32 a, __u32 b, __u32 c, __u32 d, __u32 e)
132{
133 switch (type) {
134 case CRUSH_HASH_RJENKINS1:
135 return crush_hash32_rjenkins1_5(a, b, c, d, e);
136 default:
137 return 0;
138 }
139}
140
141const char *crush_hash_name(int type)
142{
143 switch (type) {
144 case CRUSH_HASH_RJENKINS1:
145 return "rjenkins1";
146 default:
147 return "unknown";
148 }
149}
fs/ceph/crush/hash.h+17−0
@@ -0,0 +1,17 @@
1#ifndef _CRUSH_HASH_H
2#define _CRUSH_HASH_H
3
4#define CRUSH_HASH_RJENKINS1 0
5
6#define CRUSH_HASH_DEFAULT CRUSH_HASH_RJENKINS1
7
8extern const char *crush_hash_name(int type);
9
10extern __u32 crush_hash32(int type, __u32 a);
11extern __u32 crush_hash32_2(int type, __u32 a, __u32 b);
12extern __u32 crush_hash32_3(int type, __u32 a, __u32 b, __u32 c);
13extern __u32 crush_hash32_4(int type, __u32 a, __u32 b, __u32 c, __u32 d);
14extern __u32 crush_hash32_5(int type, __u32 a, __u32 b, __u32 c, __u32 d,
15 __u32 e);
16
17#endif
fs/ceph/crush/mapper.c+596−0
@@ -0,0 +1,596 @@
1
2#ifdef __KERNEL__
3# include <linux/string.h>
4# include <linux/slab.h>
5# include <linux/bug.h>
6# include <linux/kernel.h>
7# ifndef dprintk
8# define dprintk(args...)
9# endif
10#else
11# include <string.h>
12# include <stdio.h>
13# include <stdlib.h>
14# include <assert.h>
15# define BUG_ON(x) assert(!(x))
16# define dprintk(args...) /* printf(args) */
17# define kmalloc(x, f) malloc(x)
18# define kfree(x) free(x)
19#endif
20
21#include "crush.h"
22#include "hash.h"
23
24/*
25 * Implement the core CRUSH mapping algorithm.
26 */
27
28/**
29 * crush_find_rule - find a crush_rule id for a given ruleset, type, and size.
30 * @map: the crush_map
31 * @ruleset: the storage ruleset id (user defined)
32 * @type: storage ruleset type (user defined)
33 * @size: output set size
34 */
35int crush_find_rule(struct crush_map *map, int ruleset, int type, int size)
36{
37 int i;
38
39 for (i = 0; i < map->max_rules; i++) {
40 if (map->rules[i] &&
41 map->rules[i]->mask.ruleset == ruleset &&
42 map->rules[i]->mask.type == type &&
43 map->rules[i]->mask.min_size <= size &&
44 map->rules[i]->mask.max_size >= size)
45 return i;
46 }
47 return -1;
48}
49
50
51/*
52 * bucket choose methods
53 *
54 * For each bucket algorithm, we have a "choose" method that, given a
55 * crush input @x and replica position (usually, position in output set) @r,
56 * will produce an item in the bucket.
57 */
58
59/*
60 * Choose based on a random permutation of the bucket.
61 *
62 * We used to use some prime number arithmetic to do this, but it
63 * wasn't very random, and had some other bad behaviors. Instead, we
64 * calculate an actual random permutation of the bucket members.
65 * Since this is expensive, we optimize for the r=0 case, which
66 * captures the vast majority of calls.
67 */
68static int bucket_perm_choose(struct crush_bucket *bucket,
69 int x, int r)
70{
71 unsigned pr = r % bucket->size;
72 unsigned i, s;
73
74 /* start a new permutation if @x has changed */
75 if (bucket->perm_x != x || bucket->perm_n == 0) {
76 dprintk("bucket %d new x=%d\n", bucket->id, x);
77 bucket->perm_x = x;
78
79 /* optimize common r=0 case */
80 if (pr == 0) {
81 s = crush_hash32_3(bucket->hash, x, bucket->id, 0) %
82 bucket->size;
83 bucket->perm[0] = s;
84 bucket->perm_n = 0xffff; /* magic value, see below */
85 goto out;
86 }
87
88 for (i = 0; i < bucket->size; i++)
89 bucket->perm[i] = i;
90 bucket->perm_n = 0;
91 } else if (bucket->perm_n == 0xffff) {
92 /* clean up after the r=0 case above */
93 for (i = 1; i < bucket->size; i++)
94 bucket->perm[i] = i;
95 bucket->perm[bucket->perm[0]] = 0;
96 bucket->perm_n = 1;
97 }
98
99 /* calculate permutation up to pr */
100 for (i = 0; i < bucket->perm_n; i++)
101 dprintk(" perm_choose have %d: %d\n", i, bucket->perm[i]);
102 while (bucket->perm_n <= pr) {
103 unsigned p = bucket->perm_n;
104 /* no point in swapping the final entry */
105 if (p < bucket->size - 1) {
106 i = crush_hash32_3(bucket->hash, x, bucket->id, p) %
107 (bucket->size - p);
108 if (i) {
109 unsigned t = bucket->perm[p + i];
110 bucket->perm[p + i] = bucket->perm[p];
111 bucket->perm[p] = t;
112 }
113 dprintk(" perm_choose swap %d with %d\n", p, p+i);
114 }
115 bucket->perm_n++;
116 }
117 for (i = 0; i < bucket->size; i++)
118 dprintk(" perm_choose %d: %d\n", i, bucket->perm[i]);
119
120 s = bucket->perm[pr];
121out:
122 dprintk(" perm_choose %d sz=%d x=%d r=%d (%d) s=%d\n", bucket->id,
123 bucket->size, x, r, pr, s);
124 return bucket->items[s];
125}
126
127/* uniform */
128static int bucket_uniform_choose(struct crush_bucket_uniform *bucket,
129 int x, int r)
130{
131 return bucket_perm_choose(&bucket->h, x, r);
132}
133
134/* list */
135static int bucket_list_choose(struct crush_bucket_list *bucket,
136 int x, int r)
137{
138 int i;
139
140 for (i = bucket->h.size-1; i >= 0; i--) {
141 __u64 w = crush_hash32_4(bucket->h.hash,x, bucket->h.items[i],
142 r, bucket->h.id);
143 w &= 0xffff;
144 dprintk("list_choose i=%d x=%d r=%d item %d weight %x "
145 "sw %x rand %llx",
146 i, x, r, bucket->h.items[i], bucket->item_weights[i],
147 bucket->sum_weights[i], w);
148 w *= bucket->sum_weights[i];
149 w = w >> 16;
150 /*dprintk(" scaled %llx\n", w);*/
151 if (w < bucket->item_weights[i])
152 return bucket->h.items[i];
153 }
154
155 BUG_ON(1);
156 return 0;
157}
158
159
160/* (binary) tree */
161static int height(int n)
162{
163 int h = 0;
164 while ((n & 1) == 0) {
165 h++;
166 n = n >> 1;
167 }
168 return h;
169}
170
171static int left(int x)
172{
173 int h = height(x);
174 return x - (1 << (h-1));
175}
176
177static int right(int x)
178{
179 int h = height(x);
180 return x + (1 << (h-1));
181}
182
183static int terminal(int x)
184{
185 return x & 1;
186}
187
188static int bucket_tree_choose(struct crush_bucket_tree *bucket,
189 int x, int r)
190{
191 int n, l;
192 __u32 w;
193 __u64 t;
194
195 /* start at root */
196 n = bucket->num_nodes >> 1;
197
198 while (!terminal(n)) {
199 /* pick point in [0, w) */
200 w = bucket->node_weights[n];
201 t = (__u64)crush_hash32_4(bucket->h.hash, x, n, r,
202 bucket->h.id) * (__u64)w;
203 t = t >> 32;
204
205 /* descend to the left or right? */
206 l = left(n);
207 if (t < bucket->node_weights[l])
208 n = l;
209 else
210 n = right(n);
211 }
212
213 return bucket->h.items[n >> 1];
214}
215
216
217/* straw */
218
219static int bucket_straw_choose(struct crush_bucket_straw *bucket,
220 int x, int r)
221{
222 int i;
223 int high = 0;
224 __u64 high_draw = 0;
225 __u64 draw;
226
227 for (i = 0; i < bucket->h.size; i++) {
228 draw = crush_hash32_3(bucket->h.hash, x, bucket->h.items[i], r);
229 draw &= 0xffff;
230 draw *= bucket->straws[i];
231 if (i == 0 || draw > high_draw) {
232 high = i;
233 high_draw = draw;
234 }
235 }
236 return bucket->h.items[high];
237}
238
239static int crush_bucket_choose(struct crush_bucket *in, int x, int r)
240{
241 dprintk("choose %d x=%d r=%d\n", in->id, x, r);
242 switch (in->alg) {
243 case CRUSH_BUCKET_UNIFORM:
244 return bucket_uniform_choose((struct crush_bucket_uniform *)in,
245 x, r);
246 case CRUSH_BUCKET_LIST:
247 return bucket_list_choose((struct crush_bucket_list *)in,
248 x, r);
249 case CRUSH_BUCKET_TREE:
250 return bucket_tree_choose((struct crush_bucket_tree *)in,
251 x, r);
252 case CRUSH_BUCKET_STRAW:
253 return bucket_straw_choose((struct crush_bucket_straw *)in,
254 x, r);
255 default:
256 BUG_ON(1);
257 return in->items[0];
258 }
259}
260
261/*
262 * true if device is marked "out" (failed, fully offloaded)
263 * of the cluster
264 */
265static int is_out(struct crush_map *map, __u32 *weight, int item, int x)
266{
267 if (weight[item] >= 0x1000)
268 return 0;
269 if (weight[item] == 0)
270 return 1;
271 if ((crush_hash32_2(CRUSH_HASH_RJENKINS1, x, item) & 0xffff)
272 < weight[item])
273 return 0;
274 return 1;
275}
276
277/**
278 * crush_choose - choose numrep distinct items of given type
279 * @map: the crush_map
280 * @bucket: the bucket we are choose an item from
281 * @x: crush input value
282 * @numrep: the number of items to choose
283 * @type: the type of item to choose
284 * @out: pointer to output vector
285 * @outpos: our position in that vector
286 * @firstn: true if choosing "first n" items, false if choosing "indep"
287 * @recurse_to_leaf: true if we want one device under each item of given type
288 * @out2: second output vector for leaf items (if @recurse_to_leaf)
289 */
290static int crush_choose(struct crush_map *map,
291 struct crush_bucket *bucket,
292 __u32 *weight,
293 int x, int numrep, int type,
294 int *out, int outpos,
295 int firstn, int recurse_to_leaf,
296 int *out2)
297{
298 int rep;
299 int ftotal, flocal;
300 int retry_descent, retry_bucket, skip_rep;
301 struct crush_bucket *in = bucket;
302 int r;
303 int i;
304 int item = 0;
305 int itemtype;
306 int collide, reject;
307 const int orig_tries = 5; /* attempts before we fall back to search */
308 dprintk("choose bucket %d x %d outpos %d\n", bucket->id, x, outpos);
309
310 for (rep = outpos; rep < numrep; rep++) {
311 /* keep trying until we get a non-out, non-colliding item */
312 ftotal = 0;
313 skip_rep = 0;
314 do {
315 retry_descent = 0;
316 in = bucket; /* initial bucket */
317
318 /* choose through intervening buckets */
319 flocal = 0;
320 do {
321 collide = 0;
322 retry_bucket = 0;
323 r = rep;
324 if (in->alg == CRUSH_BUCKET_UNIFORM) {
325 /* be careful */
326 if (firstn || numrep >= in->size)
327 /* r' = r + f_total */
328 r += ftotal;
329 else if (in->size % numrep == 0)
330 /* r'=r+(n+1)*f_local */
331 r += (numrep+1) *
332 (flocal+ftotal);
333 else
334 /* r' = r + n*f_local */
335 r += numrep * (flocal+ftotal);
336 } else {
337 if (firstn)
338 /* r' = r + f_total */
339 r += ftotal;
340 else
341 /* r' = r + n*f_local */
342 r += numrep * (flocal+ftotal);
343 }
344
345 /* bucket choose */
346 if (in->size == 0) {
347 reject = 1;
348 goto reject;
349 }
350 if (flocal >= (in->size>>1) &&
351 flocal > orig_tries)
352 item = bucket_perm_choose(in, x, r);
353 else
354 item = crush_bucket_choose(in, x, r);
355 BUG_ON(item >= map->max_devices);
356
357 /* desired type? */
358 if (item < 0)
359 itemtype = map->buckets[-1-item]->type;
360 else
361 itemtype = 0;
362 dprintk(" item %d type %d\n", item, itemtype);
363
364 /* keep going? */
365 if (itemtype != type) {
366 BUG_ON(item >= 0 ||
367 (-1-item) >= map->max_buckets);
368 in = map->buckets[-1-item];
369 continue;
370 }
371
372 /* collision? */
373 for (i = 0; i < outpos; i++) {
374 if (out[i] == item) {
375 collide = 1;
376 break;
377 }
378 }
379
380 if (recurse_to_leaf &&
381 item < 0 &&
382 crush_choose(map, map->buckets[-1-item],
383 weight,
384 x, outpos+1, 0,
385 out2, outpos,
386 firstn, 0, NULL) <= outpos) {
387 reject = 1;
388 } else {
389 /* out? */
390 if (itemtype == 0)
391 reject = is_out(map, weight,
392 item, x);
393 else
394 reject = 0;
395 }
396
397reject:
398 if (reject || collide) {
399 ftotal++;
400 flocal++;
401
402 if (collide && flocal < 3)
403 /* retry locally a few times */
404 retry_bucket = 1;
405 else if (flocal < in->size + orig_tries)
406 /* exhaustive bucket search */
407 retry_bucket = 1;
408 else if (ftotal < 20)
409 /* then retry descent */
410 retry_descent = 1;
411 else
412 /* else give up */
413 skip_rep = 1;
414 dprintk(" reject %d collide %d "
415 "ftotal %d flocal %d\n",
416 reject, collide, ftotal,
417 flocal);
418 }
419 } while (retry_bucket);
420 } while (retry_descent);
421
422 if (skip_rep) {
423 dprintk("skip rep\n");
424 continue;
425 }
426
427 dprintk("choose got %d\n", item);
428 out[outpos] = item;
429 outpos++;
430 }
431
432 dprintk("choose returns %d\n", outpos);
433 return outpos;
434}
435
436
437/**
438 * crush_do_rule - calculate a mapping with the given input and rule
439 * @map: the crush_map
440 * @ruleno: the rule id
441 * @x: hash input
442 * @result: pointer to result vector
443 * @result_max: maximum result size
444 * @force: force initial replica choice; -1 for none
445 */
446int crush_do_rule(struct crush_map *map,
447 int ruleno, int x, int *result, int result_max,
448 int force, __u32 *weight)
449{
450 int result_len;
451 int force_context[CRUSH_MAX_DEPTH];
452 int force_pos = -1;
453 int a[CRUSH_MAX_SET];
454 int b[CRUSH_MAX_SET];
455 int c[CRUSH_MAX_SET];
456 int recurse_to_leaf;
457 int *w;
458 int wsize = 0;
459 int *o;
460 int osize;
461 int *tmp;
462 struct crush_rule *rule;
463 int step;
464 int i, j;
465 int numrep;
466 int firstn;
467 int rc = -1;
468
469 BUG_ON(ruleno >= map->max_rules);
470
471 rule = map->rules[ruleno];
472 result_len = 0;
473 w = a;
474 o = b;
475
476 /*
477 * determine hierarchical context of force, if any. note
478 * that this may or may not correspond to the specific types
479 * referenced by the crush rule.
480 */
481 if (force >= 0) {
482 if (force >= map->max_devices ||
483 map->device_parents[force] == 0) {
484 /*dprintk("CRUSH: forcefed device dne\n");*/
485 rc = -1; /* force fed device dne */
486 goto out;
487 }
488 if (!is_out(map, weight, force, x)) {
489 while (1) {
490 force_context[++force_pos] = force;
491 if (force >= 0)
492 force = map->device_parents[force];
493 else
494 force = map->bucket_parents[-1-force];
495 if (force == 0)
496 break;
497 }
498 }
499 }
500
501 for (step = 0; step < rule->len; step++) {
502 firstn = 0;
503 switch (rule->steps[step].op) {
504 case CRUSH_RULE_TAKE:
505 w[0] = rule->steps[step].arg1;
506 if (force_pos >= 0) {
507 BUG_ON(force_context[force_pos] != w[0]);
508 force_pos--;
509 }
510 wsize = 1;
511 break;
512
513 case CRUSH_RULE_CHOOSE_LEAF_FIRSTN:
514 case CRUSH_RULE_CHOOSE_FIRSTN:
515 firstn = 1;
516 case CRUSH_RULE_CHOOSE_LEAF_INDEP:
517 case CRUSH_RULE_CHOOSE_INDEP:
518 BUG_ON(wsize == 0);
519
520 recurse_to_leaf =
521 rule->steps[step].op ==
522 CRUSH_RULE_CHOOSE_LEAF_FIRSTN ||
523 rule->steps[step].op ==
524 CRUSH_RULE_CHOOSE_LEAF_INDEP;
525
526 /* reset output */
527 osize = 0;
528
529 for (i = 0; i < wsize; i++) {
530 /*
531 * see CRUSH_N, CRUSH_N_MINUS macros.
532 * basically, numrep <= 0 means relative to
533 * the provided result_max
534 */
535 numrep = rule->steps[step].arg1;
536 if (numrep <= 0) {
537 numrep += result_max;
538 if (numrep <= 0)
539 continue;
540 }
541 j = 0;
542 if (osize == 0 && force_pos >= 0) {
543 /* skip any intermediate types */
544 while (force_pos &&
545 force_context[force_pos] < 0 &&
546 rule->steps[step].arg2 !=
547 map->buckets[-1 -
548 force_context[force_pos]]->type)
549 force_pos--;
550 o[osize] = force_context[force_pos];
551 if (recurse_to_leaf)
552 c[osize] = force_context[0];
553 j++;
554 force_pos--;
555 }
556 osize += crush_choose(map,
557 map->buckets[-1-w[i]],
558 weight,
559 x, numrep,
560 rule->steps[step].arg2,
561 o+osize, j,
562 firstn,
563 recurse_to_leaf, c+osize);
564 }
565
566 if (recurse_to_leaf)
567 /* copy final _leaf_ values to output set */
568 memcpy(o, c, osize*sizeof(*o));
569
570 /* swap t and w arrays */
571 tmp = o;
572 o = w;
573 w = tmp;
574 wsize = osize;
575 break;
576
577
578 case CRUSH_RULE_EMIT:
579 for (i = 0; i < wsize && result_len < result_max; i++) {
580 result[result_len] = w[i];
581 result_len++;
582 }
583 wsize = 0;
584 break;
585
586 default:
587 BUG_ON(1);
588 }
589 }
590 rc = result_len;
591
592out:
593 return rc;
594}
595
596
fs/ceph/crush/mapper.h+20−0
@@ -0,0 +1,20 @@
1#ifndef _CRUSH_MAPPER_H
2#define _CRUSH_MAPPER_H
3
4/*
5 * CRUSH functions for find rules and then mapping an input to an
6 * output set.
7 *
8 * LGPL2
9 */
10
11#include "crush.h"
12
13extern int crush_find_rule(struct crush_map *map, int pool, int type, int size);
14extern int crush_do_rule(struct crush_map *map,
15 int ruleno,
16 int x, int *result, int result_max,
17 int forcefeed, /* -1 for none */
18 __u32 *weights);
19
20#endif
fs/ceph/crypto.c+408−0
@@ -0,0 +1,408 @@
1
2#include "ceph_debug.h"
3
4#include <linux/err.h>
5#include <linux/scatterlist.h>
6#include <crypto/hash.h>
7
8#include "crypto.h"
9#include "decode.h"
10
11int ceph_crypto_key_encode(struct ceph_crypto_key *key, void **p, void *end)
12{
13 if (*p + sizeof(u16) + sizeof(key->created) +
14 sizeof(u16) + key->len > end)
15 return -ERANGE;
16 ceph_encode_16(p, key->type);
17 ceph_encode_copy(p, &key->created, sizeof(key->created));
18 ceph_encode_16(p, key->len);
19 ceph_encode_copy(p, key->key, key->len);
20 return 0;
21}
22
23int ceph_crypto_key_decode(struct ceph_crypto_key *key, void **p, void *end)
24{
25 ceph_decode_need(p, end, 2*sizeof(u16) + sizeof(key->created), bad);
26 key->type = ceph_decode_16(p);
27 ceph_decode_copy(p, &key->created, sizeof(key->created));
28 key->len = ceph_decode_16(p);
29 ceph_decode_need(p, end, key->len, bad);
30 key->key = kmalloc(key->len, GFP_NOFS);
31 if (!key->key)
32 return -ENOMEM;
33 ceph_decode_copy(p, key->key, key->len);
34 return 0;
35
36bad:
37 dout("failed to decode crypto key\n");
38 return -EINVAL;
39}
40
41int ceph_crypto_key_unarmor(struct ceph_crypto_key *key, const char *inkey)
42{
43 int inlen = strlen(inkey);
44 int blen = inlen * 3 / 4;
45 void *buf, *p;
46 int ret;
47
48 dout("crypto_key_unarmor %s\n", inkey);
49 buf = kmalloc(blen, GFP_NOFS);
50 if (!buf)
51 return -ENOMEM;
52 blen = ceph_unarmor(buf, inkey, inkey+inlen);
53 if (blen < 0) {
54 kfree(buf);
55 return blen;
56 }
57
58 p = buf;
59 ret = ceph_crypto_key_decode(key, &p, p + blen);
60 kfree(buf);
61 if (ret)
62 return ret;
63 dout("crypto_key_unarmor key %p type %d len %d\n", key,
64 key->type, key->len);
65 return 0;
66}
67
68
69
70#define AES_KEY_SIZE 16
71
72static struct crypto_blkcipher *ceph_crypto_alloc_cipher(void)
73{
74 return crypto_alloc_blkcipher("cbc(aes)", 0, CRYPTO_ALG_ASYNC);
75}
76
77const u8 *aes_iv = "cephsageyudagreg";
78
79int ceph_aes_encrypt(const void *key, int key_len, void *dst, size_t *dst_len,
80 const void *src, size_t src_len)
81{
82 struct scatterlist sg_in[2], sg_out[1];
83 struct crypto_blkcipher *tfm = ceph_crypto_alloc_cipher();
84 struct blkcipher_desc desc = { .tfm = tfm, .flags = 0 };
85 int ret;
86 void *iv;
87 int ivsize;
88 size_t zero_padding = (0x10 - (src_len & 0x0f));
89 char pad[16];
90
91 if (IS_ERR(tfm))
92 return PTR_ERR(tfm);
93
94 memset(pad, zero_padding, zero_padding);
95
96 *dst_len = src_len + zero_padding;
97
98 crypto_blkcipher_setkey((void *)tfm, key, key_len);
99 sg_init_table(sg_in, 2);
100 sg_set_buf(&sg_in[0], src, src_len);
101 sg_set_buf(&sg_in[1], pad, zero_padding);
102 sg_init_table(sg_out, 1);
103 sg_set_buf(sg_out, dst, *dst_len);
104 iv = crypto_blkcipher_crt(tfm)->iv;
105 ivsize = crypto_blkcipher_ivsize(tfm);
106
107 memcpy(iv, aes_iv, ivsize);
108 /*
109 print_hex_dump(KERN_ERR, "enc key: ", DUMP_PREFIX_NONE, 16, 1,
110 key, key_len, 1);
111 print_hex_dump(KERN_ERR, "enc src: ", DUMP_PREFIX_NONE, 16, 1,
112 src, src_len, 1);
113 print_hex_dump(KERN_ERR, "enc pad: ", DUMP_PREFIX_NONE, 16, 1,
114 pad, zero_padding, 1);
115 */
116 ret = crypto_blkcipher_encrypt(&desc, sg_out, sg_in,
117 src_len + zero_padding);
118 crypto_free_blkcipher(tfm);
119 if (ret < 0)
120 pr_err("ceph_aes_crypt failed %d\n", ret);
121 /*
122 print_hex_dump(KERN_ERR, "enc out: ", DUMP_PREFIX_NONE, 16, 1,
123 dst, *dst_len, 1);
124 */
125 return 0;
126}
127
128int ceph_aes_encrypt2(const void *key, int key_len, void *dst, size_t *dst_len,
129 const void *src1, size_t src1_len,
130 const void *src2, size_t src2_len)
131{
132 struct scatterlist sg_in[3], sg_out[1];
133 struct crypto_blkcipher *tfm = ceph_crypto_alloc_cipher();
134 struct blkcipher_desc desc = { .tfm = tfm, .flags = 0 };
135 int ret;
136 void *iv;
137 int ivsize;
138 size_t zero_padding = (0x10 - ((src1_len + src2_len) & 0x0f));
139 char pad[16];
140
141 if (IS_ERR(tfm))
142 return PTR_ERR(tfm);
143
144 memset(pad, zero_padding, zero_padding);
145
146 *dst_len = src1_len + src2_len + zero_padding;
147
148 crypto_blkcipher_setkey((void *)tfm, key, key_len);
149 sg_init_table(sg_in, 3);
150 sg_set_buf(&sg_in[0], src1, src1_len);
151 sg_set_buf(&sg_in[1], src2, src2_len);
152 sg_set_buf(&sg_in[2], pad, zero_padding);
153 sg_init_table(sg_out, 1);
154 sg_set_buf(sg_out, dst, *dst_len);
155 iv = crypto_blkcipher_crt(tfm)->iv;
156 ivsize = crypto_blkcipher_ivsize(tfm);
157
158 memcpy(iv, aes_iv, ivsize);
159 /*
160 print_hex_dump(KERN_ERR, "enc key: ", DUMP_PREFIX_NONE, 16, 1,
161 key, key_len, 1);
162 print_hex_dump(KERN_ERR, "enc src1: ", DUMP_PREFIX_NONE, 16, 1,
163 src1, src1_len, 1);
164 print_hex_dump(KERN_ERR, "enc src2: ", DUMP_PREFIX_NONE, 16, 1,
165 src2, src2_len, 1);
166 print_hex_dump(KERN_ERR, "enc pad: ", DUMP_PREFIX_NONE, 16, 1,
167 pad, zero_padding, 1);
168 */
169 ret = crypto_blkcipher_encrypt(&desc, sg_out, sg_in,
170 src1_len + src2_len + zero_padding);
171 crypto_free_blkcipher(tfm);
172 if (ret < 0)
173 pr_err("ceph_aes_crypt2 failed %d\n", ret);
174 /*
175 print_hex_dump(KERN_ERR, "enc out: ", DUMP_PREFIX_NONE, 16, 1,
176 dst, *dst_len, 1);
177 */
178 return 0;
179}
180
181int ceph_aes_decrypt(const void *key, int key_len, void *dst, size_t *dst_len,
182 const void *src, size_t src_len)
183{
184 struct scatterlist sg_in[1], sg_out[2];
185 struct crypto_blkcipher *tfm = ceph_crypto_alloc_cipher();
186 struct blkcipher_desc desc = { .tfm = tfm };
187 char pad[16];
188 void *iv;
189 int ivsize;
190 int ret;
191 int last_byte;
192
193 if (IS_ERR(tfm))
194 return PTR_ERR(tfm);
195
196 crypto_blkcipher_setkey((void *)tfm, key, key_len);
197 sg_init_table(sg_in, 1);
198 sg_init_table(sg_out, 2);
199 sg_set_buf(sg_in, src, src_len);
200 sg_set_buf(&sg_out[0], dst, *dst_len);
201 sg_set_buf(&sg_out[1], pad, sizeof(pad));
202
203 iv = crypto_blkcipher_crt(tfm)->iv;
204 ivsize = crypto_blkcipher_ivsize(tfm);
205
206 memcpy(iv, aes_iv, ivsize);
207
208 /*
209 print_hex_dump(KERN_ERR, "dec key: ", DUMP_PREFIX_NONE, 16, 1,
210 key, key_len, 1);
211 print_hex_dump(KERN_ERR, "dec in: ", DUMP_PREFIX_NONE, 16, 1,
212 src, src_len, 1);
213 */
214
215 ret = crypto_blkcipher_decrypt(&desc, sg_out, sg_in, src_len);
216 crypto_free_blkcipher(tfm);
217 if (ret < 0) {
218 pr_err("ceph_aes_decrypt failed %d\n", ret);
219 return ret;
220 }
221
222 if (src_len <= *dst_len)
223 last_byte = ((char *)dst)[src_len - 1];
224 else
225 last_byte = pad[src_len - *dst_len - 1];
226 if (last_byte <= 16 && src_len >= last_byte) {
227 *dst_len = src_len - last_byte;
228 } else {
229 pr_err("ceph_aes_decrypt got bad padding %d on src len %d\n",
230 last_byte, (int)src_len);
231 return -EPERM; /* bad padding */
232 }
233 /*
234 print_hex_dump(KERN_ERR, "dec out: ", DUMP_PREFIX_NONE, 16, 1,
235 dst, *dst_len, 1);
236 */
237 return 0;
238}
239
240int ceph_aes_decrypt2(const void *key, int key_len,
241 void *dst1, size_t *dst1_len,
242 void *dst2, size_t *dst2_len,
243 const void *src, size_t src_len)
244{
245 struct scatterlist sg_in[1], sg_out[3];
246 struct crypto_blkcipher *tfm = ceph_crypto_alloc_cipher();
247 struct blkcipher_desc desc = { .tfm = tfm };
248 char pad[16];
249 void *iv;
250 int ivsize;
251 int ret;
252 int last_byte;
253
254 if (IS_ERR(tfm))
255 return PTR_ERR(tfm);
256
257 sg_init_table(sg_in, 1);
258 sg_set_buf(sg_in, src, src_len);
259 sg_init_table(sg_out, 3);
260 sg_set_buf(&sg_out[0], dst1, *dst1_len);
261 sg_set_buf(&sg_out[1], dst2, *dst2_len);
262 sg_set_buf(&sg_out[2], pad, sizeof(pad));
263
264 crypto_blkcipher_setkey((void *)tfm, key, key_len);
265 iv = crypto_blkcipher_crt(tfm)->iv;
266 ivsize = crypto_blkcipher_ivsize(tfm);
267
268 memcpy(iv, aes_iv, ivsize);
269
270 /*
271 print_hex_dump(KERN_ERR, "dec key: ", DUMP_PREFIX_NONE, 16, 1,
272 key, key_len, 1);
273 print_hex_dump(KERN_ERR, "dec in: ", DUMP_PREFIX_NONE, 16, 1,
274 src, src_len, 1);
275 */
276
277 ret = crypto_blkcipher_decrypt(&desc, sg_out, sg_in, src_len);
278 crypto_free_blkcipher(tfm);
279 if (ret < 0) {
280 pr_err("ceph_aes_decrypt failed %d\n", ret);
281 return ret;
282 }
283
284 if (src_len <= *dst1_len)
285 last_byte = ((char *)dst1)[src_len - 1];
286 else if (src_len <= *dst1_len + *dst2_len)
287 last_byte = ((char *)dst2)[src_len - *dst1_len - 1];
288 else
289 last_byte = pad[src_len - *dst1_len - *dst2_len - 1];
290 if (last_byte <= 16 && src_len >= last_byte) {
291 src_len -= last_byte;
292 } else {
293 pr_err("ceph_aes_decrypt got bad padding %d on src len %d\n",
294 last_byte, (int)src_len);
295 return -EPERM; /* bad padding */
296 }
297
298 if (src_len < *dst1_len) {
299 *dst1_len = src_len;
300 *dst2_len = 0;
301 } else {
302 *dst2_len = src_len - *dst1_len;
303 }
304 /*
305 print_hex_dump(KERN_ERR, "dec out1: ", DUMP_PREFIX_NONE, 16, 1,
306 dst1, *dst1_len, 1);
307 print_hex_dump(KERN_ERR, "dec out2: ", DUMP_PREFIX_NONE, 16, 1,
308 dst2, *dst2_len, 1);
309 */
310
311 return 0;
312}
313
314
315int ceph_decrypt(struct ceph_crypto_key *secret, void *dst, size_t *dst_len,
316 const void *src, size_t src_len)
317{
318 switch (secret->type) {
319 case CEPH_CRYPTO_NONE:
320 if (*dst_len < src_len)
321 return -ERANGE;
322 memcpy(dst, src, src_len);
323 *dst_len = src_len;
324 return 0;
325
326 case CEPH_CRYPTO_AES:
327 return ceph_aes_decrypt(secret->key, secret->len, dst,
328 dst_len, src, src_len);
329
330 default:
331 return -EINVAL;
332 }
333}
334
335int ceph_decrypt2(struct ceph_crypto_key *secret,
336 void *dst1, size_t *dst1_len,
337 void *dst2, size_t *dst2_len,
338 const void *src, size_t src_len)
339{
340 size_t t;
341
342 switch (secret->type) {
343 case CEPH_CRYPTO_NONE:
344 if (*dst1_len + *dst2_len < src_len)
345 return -ERANGE;
346 t = min(*dst1_len, src_len);
347 memcpy(dst1, src, t);
348 *dst1_len = t;
349 src += t;
350 src_len -= t;
351 if (src_len) {
352 t = min(*dst2_len, src_len);
353 memcpy(dst2, src, t);
354 *dst2_len = t;
355 }
356 return 0;
357
358 case CEPH_CRYPTO_AES:
359 return ceph_aes_decrypt2(secret->key, secret->len,
360 dst1, dst1_len, dst2, dst2_len,
361 src, src_len);
362
363 default:
364 return -EINVAL;
365 }
366}
367
368int ceph_encrypt(struct ceph_crypto_key *secret, void *dst, size_t *dst_len,
369 const void *src, size_t src_len)
370{
371 switch (secret->type) {
372 case CEPH_CRYPTO_NONE:
373 if (*dst_len < src_len)
374 return -ERANGE;
375 memcpy(dst, src, src_len);
376 *dst_len = src_len;
377 return 0;
378
379 case CEPH_CRYPTO_AES:
380 return ceph_aes_encrypt(secret->key, secret->len, dst,
381 dst_len, src, src_len);
382
383 default:
384 return -EINVAL;
385 }
386}
387
388int ceph_encrypt2(struct ceph_crypto_key *secret, void *dst, size_t *dst_len,
389 const void *src1, size_t src1_len,
390 const void *src2, size_t src2_len)
391{
392 switch (secret->type) {
393 case CEPH_CRYPTO_NONE:
394 if (*dst_len < src1_len + src2_len)
395 return -ERANGE;
396 memcpy(dst, src1, src1_len);
397 memcpy(dst + src1_len, src2, src2_len);
398 *dst_len = src1_len + src2_len;
399 return 0;
400
401 case CEPH_CRYPTO_AES:
402 return ceph_aes_encrypt2(secret->key, secret->len, dst, dst_len,
403 src1, src1_len, src2, src2_len);
404
405 default:
406 return -EINVAL;
407 }
408}
fs/ceph/crypto.h+48−0
@@ -0,0 +1,48 @@
1#ifndef _FS_CEPH_CRYPTO_H
2#define _FS_CEPH_CRYPTO_H
3
4#include "types.h"
5#include "buffer.h"
6
7/*
8 * cryptographic secret
9 */
10struct ceph_crypto_key {
11 int type;
12 struct ceph_timespec created;
13 int len;
14 void *key;
15};
16
17static inline void ceph_crypto_key_destroy(struct ceph_crypto_key *key)
18{
19 kfree(key->key);
20}
21
22extern int ceph_crypto_key_encode(struct ceph_crypto_key *key,
23 void **p, void *end);
24extern int ceph_crypto_key_decode(struct ceph_crypto_key *key,
25 void **p, void *end);
26extern int ceph_crypto_key_unarmor(struct ceph_crypto_key *key, const char *in);
27
28/* crypto.c */
29extern int ceph_decrypt(struct ceph_crypto_key *secret,
30 void *dst, size_t *dst_len,
31 const void *src, size_t src_len);
32extern int ceph_encrypt(struct ceph_crypto_key *secret,
33 void *dst, size_t *dst_len,
34 const void *src, size_t src_len);
35extern int ceph_decrypt2(struct ceph_crypto_key *secret,
36 void *dst1, size_t *dst1_len,
37 void *dst2, size_t *dst2_len,
38 const void *src, size_t src_len);
39extern int ceph_encrypt2(struct ceph_crypto_key *secret,
40 void *dst, size_t *dst_len,
41 const void *src1, size_t src1_len,
42 const void *src2, size_t src2_len);
43
44/* armor.c */
45extern int ceph_armor(char *dst, const void *src, const void *end);
46extern int ceph_unarmor(void *dst, const char *src, const char *end);
47
48#endif
fs/ceph/debugfs.c+483−0
@@ -0,0 +1,483 @@
1#include "ceph_debug.h"
2
3#include <linux/device.h>
4#include <linux/module.h>
5#include <linux/ctype.h>
6#include <linux/debugfs.h>
7#include <linux/seq_file.h>
8
9#include "super.h"
10#include "mds_client.h"
11#include "mon_client.h"
12#include "auth.h"
13
14#ifdef CONFIG_DEBUG_FS
15
16/*
17 * Implement /sys/kernel/debug/ceph fun
18 *
19 * /sys/kernel/debug/ceph/client* - an instance of the ceph client
20 * .../osdmap - current osdmap
21 * .../mdsmap - current mdsmap
22 * .../monmap - current monmap
23 * .../osdc - active osd requests
24 * .../mdsc - active mds requests
25 * .../monc - mon client state
26 * .../dentry_lru - dump contents of dentry lru
27 * .../caps - expose cap (reservation) stats
28 * .../bdi - symlink to ../../bdi/something
29 */
30
31static struct dentry *ceph_debugfs_dir;
32
33static int monmap_show(struct seq_file *s, void *p)
34{
35 int i;
36 struct ceph_client *client = s->private;
37
38 if (client->monc.monmap == NULL)
39 return 0;
40
41 seq_printf(s, "epoch %d\n", client->monc.monmap->epoch);
42 for (i = 0; i < client->monc.monmap->num_mon; i++) {
43 struct ceph_entity_inst *inst =
44 &client->monc.monmap->mon_inst[i];
45
46 seq_printf(s, "\t%s%lld\t%s\n",
47 ENTITY_NAME(inst->name),
48 pr_addr(&inst->addr.in_addr));
49 }
50 return 0;
51}
52
53static int mdsmap_show(struct seq_file *s, void *p)
54{
55 int i;
56 struct ceph_client *client = s->private;
57
58 if (client->mdsc.mdsmap == NULL)
59 return 0;
60 seq_printf(s, "epoch %d\n", client->mdsc.mdsmap->m_epoch);
61 seq_printf(s, "root %d\n", client->mdsc.mdsmap->m_root);
62 seq_printf(s, "session_timeout %d\n",
63 client->mdsc.mdsmap->m_session_timeout);
64 seq_printf(s, "session_autoclose %d\n",
65 client->mdsc.mdsmap->m_session_autoclose);
66 for (i = 0; i < client->mdsc.mdsmap->m_max_mds; i++) {
67 struct ceph_entity_addr *addr =
68 &client->mdsc.mdsmap->m_info[i].addr;
69 int state = client->mdsc.mdsmap->m_info[i].state;
70
71 seq_printf(s, "\tmds%d\t%s\t(%s)\n", i, pr_addr(&addr->in_addr),
72 ceph_mds_state_name(state));
73 }
74 return 0;
75}
76
77static int osdmap_show(struct seq_file *s, void *p)
78{
79 int i;
80 struct ceph_client *client = s->private;
81 struct rb_node *n;
82
83 if (client->osdc.osdmap == NULL)
84 return 0;
85 seq_printf(s, "epoch %d\n", client->osdc.osdmap->epoch);
86 seq_printf(s, "flags%s%s\n",
87 (client->osdc.osdmap->flags & CEPH_OSDMAP_NEARFULL) ?
88 " NEARFULL" : "",
89 (client->osdc.osdmap->flags & CEPH_OSDMAP_FULL) ?
90 " FULL" : "");
91 for (n = rb_first(&client->osdc.osdmap->pg_pools); n; n = rb_next(n)) {
92 struct ceph_pg_pool_info *pool =
93 rb_entry(n, struct ceph_pg_pool_info, node);
94 seq_printf(s, "pg_pool %d pg_num %d / %d, lpg_num %d / %d\n",
95 pool->id, pool->v.pg_num, pool->pg_num_mask,
96 pool->v.lpg_num, pool->lpg_num_mask);
97 }
98 for (i = 0; i < client->osdc.osdmap->max_osd; i++) {
99 struct ceph_entity_addr *addr =
100 &client->osdc.osdmap->osd_addr[i];
101 int state = client->osdc.osdmap->osd_state[i];
102 char sb[64];
103
104 seq_printf(s, "\tosd%d\t%s\t%3d%%\t(%s)\n",
105 i, pr_addr(&addr->in_addr),
106 ((client->osdc.osdmap->osd_weight[i]*100) >> 16),
107 ceph_osdmap_state_str(sb, sizeof(sb), state));
108 }
109 return 0;
110}
111
112static int monc_show(struct seq_file *s, void *p)
113{
114 struct ceph_client *client = s->private;
115 struct ceph_mon_statfs_request *req;
116 struct ceph_mon_client *monc = &client->monc;
117 struct rb_node *rp;
118
119 mutex_lock(&monc->mutex);
120
121 if (monc->have_mdsmap)
122 seq_printf(s, "have mdsmap %u\n", (unsigned)monc->have_mdsmap);
123 if (monc->have_osdmap)
124 seq_printf(s, "have osdmap %u\n", (unsigned)monc->have_osdmap);
125 if (monc->want_next_osdmap)
126 seq_printf(s, "want next osdmap\n");
127
128 for (rp = rb_first(&monc->statfs_request_tree); rp; rp = rb_next(rp)) {
129 req = rb_entry(rp, struct ceph_mon_statfs_request, node);
130 seq_printf(s, "%lld statfs\n", req->tid);
131 }
132
133 mutex_unlock(&monc->mutex);
134 return 0;
135}
136
137static int mdsc_show(struct seq_file *s, void *p)
138{
139 struct ceph_client *client = s->private;
140 struct ceph_mds_client *mdsc = &client->mdsc;
141 struct ceph_mds_request *req;
142 struct rb_node *rp;
143 int pathlen;
144 u64 pathbase;
145 char *path;
146
147 mutex_lock(&mdsc->mutex);
148 for (rp = rb_first(&mdsc->request_tree); rp; rp = rb_next(rp)) {
149 req = rb_entry(rp, struct ceph_mds_request, r_node);
150
151 if (req->r_request)
152 seq_printf(s, "%lld\tmds%d\t", req->r_tid, req->r_mds);
153 else
154 seq_printf(s, "%lld\t(no request)\t", req->r_tid);
155
156 seq_printf(s, "%s", ceph_mds_op_name(req->r_op));
157
158 if (req->r_got_unsafe)
159 seq_printf(s, "\t(unsafe)");
160 else
161 seq_printf(s, "\t");
162
163 if (req->r_inode) {
164 seq_printf(s, " #%llx", ceph_ino(req->r_inode));
165 } else if (req->r_dentry) {
166 path = ceph_mdsc_build_path(req->r_dentry, &pathlen,
167 &pathbase, 0);
168 spin_lock(&req->r_dentry->d_lock);
169 seq_printf(s, " #%llx/%.*s (%s)",
170 ceph_ino(req->r_dentry->d_parent->d_inode),
171 req->r_dentry->d_name.len,
172 req->r_dentry->d_name.name,
173 path ? path : "");
174 spin_unlock(&req->r_dentry->d_lock);
175 kfree(path);
176 } else if (req->r_path1) {
177 seq_printf(s, " #%llx/%s", req->r_ino1.ino,
178 req->r_path1);
179 }
180
181 if (req->r_old_dentry) {
182 path = ceph_mdsc_build_path(req->r_old_dentry, &pathlen,
183 &pathbase, 0);
184 spin_lock(&req->r_old_dentry->d_lock);
185 seq_printf(s, " #%llx/%.*s (%s)",
186 ceph_ino(req->r_old_dentry->d_parent->d_inode),
187 req->r_old_dentry->d_name.len,
188 req->r_old_dentry->d_name.name,
189 path ? path : "");
190 spin_unlock(&req->r_old_dentry->d_lock);
191 kfree(path);
192 } else if (req->r_path2) {
193 if (req->r_ino2.ino)
194 seq_printf(s, " #%llx/%s", req->r_ino2.ino,
195 req->r_path2);
196 else
197 seq_printf(s, " %s", req->r_path2);
198 }
199
200 seq_printf(s, "\n");
201 }
202 mutex_unlock(&mdsc->mutex);
203
204 return 0;
205}
206
207static int osdc_show(struct seq_file *s, void *pp)
208{
209 struct ceph_client *client = s->private;
210 struct ceph_osd_client *osdc = &client->osdc;
211 struct rb_node *p;
212
213 mutex_lock(&osdc->request_mutex);
214 for (p = rb_first(&osdc->requests); p; p = rb_next(p)) {
215 struct ceph_osd_request *req;
216 struct ceph_osd_request_head *head;
217 struct ceph_osd_op *op;
218 int num_ops;
219 int opcode, olen;
220 int i;
221
222 req = rb_entry(p, struct ceph_osd_request, r_node);
223
224 seq_printf(s, "%lld\tosd%d\t%d.%x\t", req->r_tid,
225 req->r_osd ? req->r_osd->o_osd : -1,
226 le32_to_cpu(req->r_pgid.pool),
227 le16_to_cpu(req->r_pgid.ps));
228
229 head = req->r_request->front.iov_base;
230 op = (void *)(head + 1);
231
232 num_ops = le16_to_cpu(head->num_ops);
233 olen = le32_to_cpu(head->object_len);
234 seq_printf(s, "%.*s", olen,
235 (const char *)(head->ops + num_ops));
236
237 if (req->r_reassert_version.epoch)
238 seq_printf(s, "\t%u'%llu",
239 (unsigned)le32_to_cpu(req->r_reassert_version.epoch),
240 le64_to_cpu(req->r_reassert_version.version));
241 else
242 seq_printf(s, "\t");
243
244 for (i = 0; i < num_ops; i++) {
245 opcode = le16_to_cpu(op->op);
246 seq_printf(s, "\t%s", ceph_osd_op_name(opcode));
247 op++;
248 }
249
250 seq_printf(s, "\n");
251 }
252 mutex_unlock(&osdc->request_mutex);
253 return 0;
254}
255
256static int caps_show(struct seq_file *s, void *p)
257{
258 struct ceph_client *client = p;
259 int total, avail, used, reserved, min;
260
261 ceph_reservation_status(client, &total, &avail, &used, &reserved, &min);
262 seq_printf(s, "total\t\t%d\n"
263 "avail\t\t%d\n"
264 "used\t\t%d\n"
265 "reserved\t%d\n"
266 "min\t%d\n",
267 total, avail, used, reserved, min);
268 return 0;
269}
270
271static int dentry_lru_show(struct seq_file *s, void *ptr)
272{
273 struct ceph_client *client = s->private;
274 struct ceph_mds_client *mdsc = &client->mdsc;
275 struct ceph_dentry_info *di;
276
277 spin_lock(&mdsc->dentry_lru_lock);
278 list_for_each_entry(di, &mdsc->dentry_lru, lru) {
279 struct dentry *dentry = di->dentry;
280 seq_printf(s, "%p %p\t%.*s\n",
281 di, dentry, dentry->d_name.len, dentry->d_name.name);
282 }
283 spin_unlock(&mdsc->dentry_lru_lock);
284
285 return 0;
286}
287
288#define DEFINE_SHOW_FUNC(name) \
289static int name##_open(struct inode *inode, struct file *file) \
290{ \
291 struct seq_file *sf; \
292 int ret; \
293 \
294 ret = single_open(file, name, NULL); \
295 sf = file->private_data; \
296 sf->private = inode->i_private; \
297 return ret; \
298} \
299 \
300static const struct file_operations name##_fops = { \
301 .open = name##_open, \
302 .read = seq_read, \
303 .llseek = seq_lseek, \
304 .release = single_release, \
305};
306
307DEFINE_SHOW_FUNC(monmap_show)
308DEFINE_SHOW_FUNC(mdsmap_show)
309DEFINE_SHOW_FUNC(osdmap_show)
310DEFINE_SHOW_FUNC(monc_show)
311DEFINE_SHOW_FUNC(mdsc_show)
312DEFINE_SHOW_FUNC(osdc_show)
313DEFINE_SHOW_FUNC(dentry_lru_show)
314DEFINE_SHOW_FUNC(caps_show)
315
316static int congestion_kb_set(void *data, u64 val)
317{
318 struct ceph_client *client = (struct ceph_client *)data;
319
320 if (client)
321 client->mount_args->congestion_kb = (int)val;
322
323 return 0;
324}
325
326static int congestion_kb_get(void *data, u64 *val)
327{
328 struct ceph_client *client = (struct ceph_client *)data;
329
330 if (client)
331 *val = (u64)client->mount_args->congestion_kb;
332
333 return 0;
334}
335
336
337DEFINE_SIMPLE_ATTRIBUTE(congestion_kb_fops, congestion_kb_get,
338 congestion_kb_set, "%llu\n");
339
340int __init ceph_debugfs_init(void)
341{
342 ceph_debugfs_dir = debugfs_create_dir("ceph", NULL);
343 if (!ceph_debugfs_dir)
344 return -ENOMEM;
345 return 0;
346}
347
348void ceph_debugfs_cleanup(void)
349{
350 debugfs_remove(ceph_debugfs_dir);
351}
352
353int ceph_debugfs_client_init(struct ceph_client *client)
354{
355 int ret = 0;
356 char name[80];
357
358 snprintf(name, sizeof(name), FSID_FORMAT ".client%lld",
359 PR_FSID(&client->fsid), client->monc.auth->global_id);
360
361 client->debugfs_dir = debugfs_create_dir(name, ceph_debugfs_dir);
362 if (!client->debugfs_dir)
363 goto out;
364
365 client->monc.debugfs_file = debugfs_create_file("monc",
366 0600,
367 client->debugfs_dir,
368 client,
369 &monc_show_fops);
370 if (!client->monc.debugfs_file)
371 goto out;
372
373 client->mdsc.debugfs_file = debugfs_create_file("mdsc",
374 0600,
375 client->debugfs_dir,
376 client,
377 &mdsc_show_fops);
378 if (!client->mdsc.debugfs_file)
379 goto out;
380
381 client->osdc.debugfs_file = debugfs_create_file("osdc",
382 0600,
383 client->debugfs_dir,
384 client,
385 &osdc_show_fops);
386 if (!client->osdc.debugfs_file)
387 goto out;
388
389 client->debugfs_monmap = debugfs_create_file("monmap",
390 0600,
391 client->debugfs_dir,
392 client,
393 &monmap_show_fops);
394 if (!client->debugfs_monmap)
395 goto out;
396
397 client->debugfs_mdsmap = debugfs_create_file("mdsmap",
398 0600,
399 client->debugfs_dir,
400 client,
401 &mdsmap_show_fops);
402 if (!client->debugfs_mdsmap)
403 goto out;
404
405 client->debugfs_osdmap = debugfs_create_file("osdmap",
406 0600,
407 client->debugfs_dir,
408 client,
409 &osdmap_show_fops);
410 if (!client->debugfs_osdmap)
411 goto out;
412
413 client->debugfs_dentry_lru = debugfs_create_file("dentry_lru",
414 0600,
415 client->debugfs_dir,
416 client,
417 &dentry_lru_show_fops);
418 if (!client->debugfs_dentry_lru)
419 goto out;
420
421 client->debugfs_caps = debugfs_create_file("caps",
422 0400,
423 client->debugfs_dir,
424 client,
425 &caps_show_fops);
426 if (!client->debugfs_caps)
427 goto out;
428
429 client->debugfs_congestion_kb = debugfs_create_file("writeback_congestion_kb",
430 0600,
431 client->debugfs_dir,
432 client,
433 &congestion_kb_fops);
434 if (!client->debugfs_congestion_kb)
435 goto out;
436
437 sprintf(name, "../../bdi/%s", dev_name(client->sb->s_bdi->dev));
438 client->debugfs_bdi = debugfs_create_symlink("bdi", client->debugfs_dir,
439 name);
440
441 return 0;
442
443out:
444 ceph_debugfs_client_cleanup(client);
445 return ret;
446}
447
448void ceph_debugfs_client_cleanup(struct ceph_client *client)
449{
450 debugfs_remove(client->debugfs_bdi);
451 debugfs_remove(client->debugfs_caps);
452 debugfs_remove(client->debugfs_dentry_lru);
453 debugfs_remove(client->debugfs_osdmap);
454 debugfs_remove(client->debugfs_mdsmap);
455 debugfs_remove(client->debugfs_monmap);
456 debugfs_remove(client->osdc.debugfs_file);
457 debugfs_remove(client->mdsc.debugfs_file);
458 debugfs_remove(client->monc.debugfs_file);
459 debugfs_remove(client->debugfs_congestion_kb);
460 debugfs_remove(client->debugfs_dir);
461}
462
463#else // CONFIG_DEBUG_FS
464
465int __init ceph_debugfs_init(void)
466{
467 return 0;
468}
469
470void ceph_debugfs_cleanup(void)
471{
472}
473
474int ceph_debugfs_client_init(struct ceph_client *client)
475{
476 return 0;
477}
478
479void ceph_debugfs_client_cleanup(struct ceph_client *client)
480{
481}
482
483#endif // CONFIG_DEBUG_FS
fs/ceph/decode.h+194−0
@@ -0,0 +1,194 @@
1#ifndef __CEPH_DECODE_H
2#define __CEPH_DECODE_H
3
4#include <asm/unaligned.h>
5#include <linux/time.h>
6
7#include "types.h"
8
9/*
10 * in all cases,
11 * void **p pointer to position pointer
12 * void *end pointer to end of buffer (last byte + 1)
13 */
14
15static inline u64 ceph_decode_64(void **p)
16{
17 u64 v = get_unaligned_le64(*p);
18 *p += sizeof(u64);
19 return v;
20}
21static inline u32 ceph_decode_32(void **p)
22{
23 u32 v = get_unaligned_le32(*p);
24 *p += sizeof(u32);
25 return v;
26}
27static inline u16 ceph_decode_16(void **p)
28{
29 u16 v = get_unaligned_le16(*p);
30 *p += sizeof(u16);
31 return v;
32}
33static inline u8 ceph_decode_8(void **p)
34{
35 u8 v = *(u8 *)*p;
36 (*p)++;
37 return v;
38}
39static inline void ceph_decode_copy(void **p, void *pv, size_t n)
40{
41 memcpy(pv, *p, n);
42 *p += n;
43}
44
45/*
46 * bounds check input.
47 */
48#define ceph_decode_need(p, end, n, bad) \
49 do { \
50 if (unlikely(*(p) + (n) > (end))) \
51 goto bad; \
52 } while (0)
53
54#define ceph_decode_64_safe(p, end, v, bad) \
55 do { \
56 ceph_decode_need(p, end, sizeof(u64), bad); \
57 v = ceph_decode_64(p); \
58 } while (0)
59#define ceph_decode_32_safe(p, end, v, bad) \
60 do { \
61 ceph_decode_need(p, end, sizeof(u32), bad); \
62 v = ceph_decode_32(p); \
63 } while (0)
64#define ceph_decode_16_safe(p, end, v, bad) \
65 do { \
66 ceph_decode_need(p, end, sizeof(u16), bad); \
67 v = ceph_decode_16(p); \
68 } while (0)
69#define ceph_decode_8_safe(p, end, v, bad) \
70 do { \
71 ceph_decode_need(p, end, sizeof(u8), bad); \
72 v = ceph_decode_8(p); \
73 } while (0)
74
75#define ceph_decode_copy_safe(p, end, pv, n, bad) \
76 do { \
77 ceph_decode_need(p, end, n, bad); \
78 ceph_decode_copy(p, pv, n); \
79 } while (0)
80
81/*
82 * struct ceph_timespec <-> struct timespec
83 */
84static inline void ceph_decode_timespec(struct timespec *ts,
85 const struct ceph_timespec *tv)
86{
87 ts->tv_sec = le32_to_cpu(tv->tv_sec);
88 ts->tv_nsec = le32_to_cpu(tv->tv_nsec);
89}
90static inline void ceph_encode_timespec(struct ceph_timespec *tv,
91 const struct timespec *ts)
92{
93 tv->tv_sec = cpu_to_le32(ts->tv_sec);
94 tv->tv_nsec = cpu_to_le32(ts->tv_nsec);
95}
96
97/*
98 * sockaddr_storage <-> ceph_sockaddr
99 */
100static inline void ceph_encode_addr(struct ceph_entity_addr *a)
101{
102 a->in_addr.ss_family = htons(a->in_addr.ss_family);
103}
104static inline void ceph_decode_addr(struct ceph_entity_addr *a)
105{
106 a->in_addr.ss_family = ntohs(a->in_addr.ss_family);
107 WARN_ON(a->in_addr.ss_family == 512);
108}
109
110/*
111 * encoders
112 */
113static inline void ceph_encode_64(void **p, u64 v)
114{
115 put_unaligned_le64(v, (__le64 *)*p);
116 *p += sizeof(u64);
117}
118static inline void ceph_encode_32(void **p, u32 v)
119{
120 put_unaligned_le32(v, (__le32 *)*p);
121 *p += sizeof(u32);
122}
123static inline void ceph_encode_16(void **p, u16 v)
124{
125 put_unaligned_le16(v, (__le16 *)*p);
126 *p += sizeof(u16);
127}
128static inline void ceph_encode_8(void **p, u8 v)
129{
130 *(u8 *)*p = v;
131 (*p)++;
132}
133static inline void ceph_encode_copy(void **p, const void *s, int len)
134{
135 memcpy(*p, s, len);
136 *p += len;
137}
138
139/*
140 * filepath, string encoders
141 */
142static inline void ceph_encode_filepath(void **p, void *end,
143 u64 ino, const char *path)
144{
145 u32 len = path ? strlen(path) : 0;
146 BUG_ON(*p + sizeof(ino) + sizeof(len) + len > end);
147 ceph_encode_8(p, 1);
148 ceph_encode_64(p, ino);
149 ceph_encode_32(p, len);
150 if (len)
151 memcpy(*p, path, len);
152 *p += len;
153}
154
155static inline void ceph_encode_string(void **p, void *end,
156 const char *s, u32 len)
157{
158 BUG_ON(*p + sizeof(len) + len > end);
159 ceph_encode_32(p, len);
160 if (len)
161 memcpy(*p, s, len);
162 *p += len;
163}
164
165#define ceph_encode_need(p, end, n, bad) \
166 do { \
167 if (unlikely(*(p) + (n) > (end))) \
168 goto bad; \
169 } while (0)
170
171#define ceph_encode_64_safe(p, end, v, bad) \
172 do { \
173 ceph_encode_need(p, end, sizeof(u64), bad); \
174 ceph_encode_64(p, v); \
175 } while (0)
176#define ceph_encode_32_safe(p, end, v, bad) \
177 do { \
178 ceph_encode_need(p, end, sizeof(u32), bad); \
179 ceph_encode_32(p, v); \
180 } while (0)
181#define ceph_encode_16_safe(p, end, v, bad) \
182 do { \
183 ceph_encode_need(p, end, sizeof(u16), bad); \
184 ceph_encode_16(p, v); \
185 } while (0)
186
187#define ceph_encode_copy_safe(p, end, pv, n, bad) \
188 do { \
189 ceph_encode_need(p, end, n, bad); \
190 ceph_encode_copy(p, pv, n); \
191 } while (0)
192
193
194#endif
fs/ceph/dir.c+1220−0
@@ -0,0 +1,1220 @@
1#include "ceph_debug.h"
2
3#include <linux/spinlock.h>
4#include <linux/fs_struct.h>
5#include <linux/namei.h>
6#include <linux/sched.h>
7
8#include "super.h"
9
10/*
11 * Directory operations: readdir, lookup, create, link, unlink,
12 * rename, etc.
13 */
14
15/*
16 * Ceph MDS operations are specified in terms of a base ino and
17 * relative path. Thus, the client can specify an operation on a
18 * specific inode (e.g., a getattr due to fstat(2)), or as a path
19 * relative to, say, the root directory.
20 *
21 * Normally, we limit ourselves to strict inode ops (no path component)
22 * or dentry operations (a single path component relative to an ino). The
23 * exception to this is open_root_dentry(), which will open the mount
24 * point by name.
25 */
26
27const struct inode_operations ceph_dir_iops;
28const struct file_operations ceph_dir_fops;
29struct dentry_operations ceph_dentry_ops;
30
31/*
32 * Initialize ceph dentry state.
33 */
34int ceph_init_dentry(struct dentry *dentry)
35{
36 struct ceph_dentry_info *di;
37
38 if (dentry->d_fsdata)
39 return 0;
40
41 if (ceph_snap(dentry->d_parent->d_inode) == CEPH_NOSNAP)
42 dentry->d_op = &ceph_dentry_ops;
43 else if (ceph_snap(dentry->d_parent->d_inode) == CEPH_SNAPDIR)
44 dentry->d_op = &ceph_snapdir_dentry_ops;
45 else
46 dentry->d_op = &ceph_snap_dentry_ops;
47
48 di = kmem_cache_alloc(ceph_dentry_cachep, GFP_NOFS);
49 if (!di)
50 return -ENOMEM; /* oh well */
51
52 spin_lock(&dentry->d_lock);
53 if (dentry->d_fsdata) /* lost a race */
54 goto out_unlock;
55 di->dentry = dentry;
56 di->lease_session = NULL;
57 dentry->d_fsdata = di;
58 dentry->d_time = jiffies;
59 ceph_dentry_lru_add(dentry);
60out_unlock:
61 spin_unlock(&dentry->d_lock);
62 return 0;
63}
64
65
66
67/*
68 * for readdir, we encode the directory frag and offset within that
69 * frag into f_pos.
70 */
71static unsigned fpos_frag(loff_t p)
72{
73 return p >> 32;
74}
75static unsigned fpos_off(loff_t p)
76{
77 return p & 0xffffffff;
78}
79
80/*
81 * When possible, we try to satisfy a readdir by peeking at the
82 * dcache. We make this work by carefully ordering dentries on
83 * d_u.d_child when we initially get results back from the MDS, and
84 * falling back to a "normal" sync readdir if any dentries in the dir
85 * are dropped.
86 *
87 * I_COMPLETE tells indicates we have all dentries in the dir. It is
88 * defined IFF we hold CEPH_CAP_FILE_SHARED (which will be revoked by
89 * the MDS if/when the directory is modified).
90 */
91static int __dcache_readdir(struct file *filp,
92 void *dirent, filldir_t filldir)
93{
94 struct inode *inode = filp->f_dentry->d_inode;
95 struct ceph_file_info *fi = filp->private_data;
96 struct dentry *parent = filp->f_dentry;
97 struct inode *dir = parent->d_inode;
98 struct list_head *p;
99 struct dentry *dentry, *last;
100 struct ceph_dentry_info *di;
101 int err = 0;
102
103 /* claim ref on last dentry we returned */
104 last = fi->dentry;
105 fi->dentry = NULL;
106
107 dout("__dcache_readdir %p at %llu (last %p)\n", dir, filp->f_pos,
108 last);
109
110 spin_lock(&dcache_lock);
111
112 /* start at beginning? */
113 if (filp->f_pos == 2 || (last &&
114 filp->f_pos < ceph_dentry(last)->offset)) {
115 if (list_empty(&parent->d_subdirs))
116 goto out_unlock;
117 p = parent->d_subdirs.prev;
118 dout(" initial p %p/%p\n", p->prev, p->next);
119 } else {
120 p = last->d_u.d_child.prev;
121 }
122
123more:
124 dentry = list_entry(p, struct dentry, d_u.d_child);
125 di = ceph_dentry(dentry);
126 while (1) {
127 dout(" p %p/%p d_subdirs %p/%p\n", p->prev, p->next,
128 parent->d_subdirs.prev, parent->d_subdirs.next);
129 if (p == &parent->d_subdirs) {
130 fi->at_end = 1;
131 goto out_unlock;
132 }
133 if (!d_unhashed(dentry) && dentry->d_inode &&
134 ceph_snap(dentry->d_inode) != CEPH_SNAPDIR &&
135 ceph_ino(dentry->d_inode) != CEPH_INO_CEPH &&
136 filp->f_pos <= di->offset)
137 break;
138 dout(" skipping %p %.*s at %llu (%llu)%s%s\n", dentry,
139 dentry->d_name.len, dentry->d_name.name, di->offset,
140 filp->f_pos, d_unhashed(dentry) ? " unhashed" : "",
141 !dentry->d_inode ? " null" : "");
142 p = p->prev;
143 dentry = list_entry(p, struct dentry, d_u.d_child);
144 di = ceph_dentry(dentry);
145 }
146
147 atomic_inc(&dentry->d_count);
148 spin_unlock(&dcache_lock);
149 spin_unlock(&inode->i_lock);
150
151 dout(" %llu (%llu) dentry %p %.*s %p\n", di->offset, filp->f_pos,
152 dentry, dentry->d_name.len, dentry->d_name.name, dentry->d_inode);
153 filp->f_pos = di->offset;
154 err = filldir(dirent, dentry->d_name.name,
155 dentry->d_name.len, di->offset,
156 dentry->d_inode->i_ino,
157 dentry->d_inode->i_mode >> 12);
158
159 if (last) {
160 if (err < 0) {
161 /* remember our position */
162 fi->dentry = last;
163 fi->next_offset = di->offset;
164 } else {
165 dput(last);
166 }
167 last = NULL;
168 }
169
170 spin_lock(&inode->i_lock);
171 spin_lock(&dcache_lock);
172
173 if (err < 0)
174 goto out_unlock;
175
176 last = dentry;
177
178 p = p->prev;
179 filp->f_pos++;
180
181 /* make sure a dentry wasn't dropped while we didn't have dcache_lock */
182 if ((ceph_inode(dir)->i_ceph_flags & CEPH_I_COMPLETE))
183 goto more;
184 dout(" lost I_COMPLETE on %p; falling back to mds\n", dir);
185 err = -EAGAIN;
186
187out_unlock:
188 spin_unlock(&dcache_lock);
189
190 if (last) {
191 spin_unlock(&inode->i_lock);
192 dput(last);
193 spin_lock(&inode->i_lock);
194 }
195
196 return err;
197}
198
199/*
200 * make note of the last dentry we read, so we can
201 * continue at the same lexicographical point,
202 * regardless of what dir changes take place on the
203 * server.
204 */
205static int note_last_dentry(struct ceph_file_info *fi, const char *name,
206 int len)
207{
208 kfree(fi->last_name);
209 fi->last_name = kmalloc(len+1, GFP_NOFS);
210 if (!fi->last_name)
211 return -ENOMEM;
212 memcpy(fi->last_name, name, len);
213 fi->last_name[len] = 0;
214 dout("note_last_dentry '%s'\n", fi->last_name);
215 return 0;
216}
217
218static int ceph_readdir(struct file *filp, void *dirent, filldir_t filldir)
219{
220 struct ceph_file_info *fi = filp->private_data;
221 struct inode *inode = filp->f_dentry->d_inode;
222 struct ceph_inode_info *ci = ceph_inode(inode);
223 struct ceph_client *client = ceph_inode_to_client(inode);
224 struct ceph_mds_client *mdsc = &client->mdsc;
225 unsigned frag = fpos_frag(filp->f_pos);
226 int off = fpos_off(filp->f_pos);
227 int err;
228 u32 ftype;
229 struct ceph_mds_reply_info_parsed *rinfo;
230 const int max_entries = client->mount_args->max_readdir;
231
232 dout("readdir %p filp %p frag %u off %u\n", inode, filp, frag, off);
233 if (fi->at_end)
234 return 0;
235
236 /* always start with . and .. */
237 if (filp->f_pos == 0) {
238 /* note dir version at start of readdir so we can tell
239 * if any dentries get dropped */
240 fi->dir_release_count = ci->i_release_count;
241
242 dout("readdir off 0 -> '.'\n");
243 if (filldir(dirent, ".", 1, ceph_make_fpos(0, 0),
244 inode->i_ino, inode->i_mode >> 12) < 0)
245 return 0;
246 filp->f_pos = 1;
247 off = 1;
248 }
249 if (filp->f_pos == 1) {
250 dout("readdir off 1 -> '..'\n");
251 if (filldir(dirent, "..", 2, ceph_make_fpos(0, 1),
252 filp->f_dentry->d_parent->d_inode->i_ino,
253 inode->i_mode >> 12) < 0)
254 return 0;
255 filp->f_pos = 2;
256 off = 2;
257 }
258
259 /* can we use the dcache? */
260 spin_lock(&inode->i_lock);
261 if ((filp->f_pos == 2 || fi->dentry) &&
262 !ceph_test_opt(client, NOASYNCREADDIR) &&
263 (ci->i_ceph_flags & CEPH_I_COMPLETE) &&
264 __ceph_caps_issued_mask(ci, CEPH_CAP_FILE_SHARED, 1)) {
265 err = __dcache_readdir(filp, dirent, filldir);
266 if (err != -EAGAIN) {
267 spin_unlock(&inode->i_lock);
268 return err;
269 }
270 }
271 spin_unlock(&inode->i_lock);
272 if (fi->dentry) {
273 err = note_last_dentry(fi, fi->dentry->d_name.name,
274 fi->dentry->d_name.len);
275 if (err)
276 return err;
277 dput(fi->dentry);
278 fi->dentry = NULL;
279 }
280
281 /* proceed with a normal readdir */
282
283more:
284 /* do we have the correct frag content buffered? */
285 if (fi->frag != frag || fi->last_readdir == NULL) {
286 struct ceph_mds_request *req;
287 int op = ceph_snap(inode) == CEPH_SNAPDIR ?
288 CEPH_MDS_OP_LSSNAP : CEPH_MDS_OP_READDIR;
289
290 /* discard old result, if any */
291 if (fi->last_readdir)
292 ceph_mdsc_put_request(fi->last_readdir);
293
294 /* requery frag tree, as the frag topology may have changed */
295 frag = ceph_choose_frag(ceph_inode(inode), frag, NULL, NULL);
296
297 dout("readdir fetching %llx.%llx frag %x offset '%s'\n",
298 ceph_vinop(inode), frag, fi->last_name);
299 req = ceph_mdsc_create_request(mdsc, op, USE_AUTH_MDS);
300 if (IS_ERR(req))
301 return PTR_ERR(req);
302 req->r_inode = igrab(inode);
303 req->r_dentry = dget(filp->f_dentry);
304 /* hints to request -> mds selection code */
305 req->r_direct_mode = USE_AUTH_MDS;
306 req->r_direct_hash = ceph_frag_value(frag);
307 req->r_direct_is_hash = true;
308 req->r_path2 = kstrdup(fi->last_name, GFP_NOFS);
309 req->r_readdir_offset = fi->next_offset;
310 req->r_args.readdir.frag = cpu_to_le32(frag);
311 req->r_args.readdir.max_entries = cpu_to_le32(max_entries);
312 req->r_num_caps = max_entries;
313 err = ceph_mdsc_do_request(mdsc, NULL, req);
314 if (err < 0) {
315 ceph_mdsc_put_request(req);
316 return err;
317 }
318 dout("readdir got and parsed readdir result=%d"
319 " on frag %x, end=%d, complete=%d\n", err, frag,
320 (int)req->r_reply_info.dir_end,
321 (int)req->r_reply_info.dir_complete);
322
323 if (!req->r_did_prepopulate) {
324 dout("readdir !did_prepopulate");
325 fi->dir_release_count--; /* preclude I_COMPLETE */
326 }
327
328 /* note next offset and last dentry name */
329 fi->offset = fi->next_offset;
330 fi->last_readdir = req;
331
332 if (req->r_reply_info.dir_end) {
333 kfree(fi->last_name);
334 fi->last_name = NULL;
335 fi->next_offset = 0;
336 } else {
337 rinfo = &req->r_reply_info;
338 err = note_last_dentry(fi,
339 rinfo->dir_dname[rinfo->dir_nr-1],
340 rinfo->dir_dname_len[rinfo->dir_nr-1]);
341 if (err)
342 return err;
343 fi->next_offset += rinfo->dir_nr;
344 }
345 }
346
347 rinfo = &fi->last_readdir->r_reply_info;
348 dout("readdir frag %x num %d off %d chunkoff %d\n", frag,
349 rinfo->dir_nr, off, fi->offset);
350 while (off - fi->offset >= 0 && off - fi->offset < rinfo->dir_nr) {
351 u64 pos = ceph_make_fpos(frag, off);
352 struct ceph_mds_reply_inode *in =
353 rinfo->dir_in[off - fi->offset].in;
354 dout("readdir off %d (%d/%d) -> %lld '%.*s' %p\n",
355 off, off - fi->offset, rinfo->dir_nr, pos,
356 rinfo->dir_dname_len[off - fi->offset],
357 rinfo->dir_dname[off - fi->offset], in);
358 BUG_ON(!in);
359 ftype = le32_to_cpu(in->mode) >> 12;
360 if (filldir(dirent,
361 rinfo->dir_dname[off - fi->offset],
362 rinfo->dir_dname_len[off - fi->offset],
363 pos,
364 le64_to_cpu(in->ino),
365 ftype) < 0) {
366 dout("filldir stopping us...\n");
367 return 0;
368 }
369 off++;
370 filp->f_pos = pos + 1;
371 }
372
373 if (fi->last_name) {
374 ceph_mdsc_put_request(fi->last_readdir);
375 fi->last_readdir = NULL;
376 goto more;
377 }
378
379 /* more frags? */
380 if (!ceph_frag_is_rightmost(frag)) {
381 frag = ceph_frag_next(frag);
382 off = 0;
383 filp->f_pos = ceph_make_fpos(frag, off);
384 dout("readdir next frag is %x\n", frag);
385 goto more;
386 }
387 fi->at_end = 1;
388
389 /*
390 * if dir_release_count still matches the dir, no dentries
391 * were released during the whole readdir, and we should have
392 * the complete dir contents in our cache.
393 */
394 spin_lock(&inode->i_lock);
395 if (ci->i_release_count == fi->dir_release_count) {
396 dout(" marking %p complete\n", inode);
397 ci->i_ceph_flags |= CEPH_I_COMPLETE;
398 ci->i_max_offset = filp->f_pos;
399 }
400 spin_unlock(&inode->i_lock);
401
402 dout("readdir %p filp %p done.\n", inode, filp);
403 return 0;
404}
405
406static void reset_readdir(struct ceph_file_info *fi)
407{
408 if (fi->last_readdir) {
409 ceph_mdsc_put_request(fi->last_readdir);
410 fi->last_readdir = NULL;
411 }
412 kfree(fi->last_name);
413 fi->next_offset = 2; /* compensate for . and .. */
414 if (fi->dentry) {
415 dput(fi->dentry);
416 fi->dentry = NULL;
417 }
418 fi->at_end = 0;
419}
420
421static loff_t ceph_dir_llseek(struct file *file, loff_t offset, int origin)
422{
423 struct ceph_file_info *fi = file->private_data;
424 struct inode *inode = file->f_mapping->host;
425 loff_t old_offset = offset;
426 loff_t retval;
427
428 mutex_lock(&inode->i_mutex);
429 switch (origin) {
430 case SEEK_END:
431 offset += inode->i_size + 2; /* FIXME */
432 break;
433 case SEEK_CUR:
434 offset += file->f_pos;
435 }
436 retval = -EINVAL;
437 if (offset >= 0 && offset <= inode->i_sb->s_maxbytes) {
438 if (offset != file->f_pos) {
439 file->f_pos = offset;
440 file->f_version = 0;
441 fi->at_end = 0;
442 }
443 retval = offset;
444
445 /*
446 * discard buffered readdir content on seekdir(0), or
447 * seek to new frag, or seek prior to current chunk.
448 */
449 if (offset == 0 ||
450 fpos_frag(offset) != fpos_frag(old_offset) ||
451 fpos_off(offset) < fi->offset) {
452 dout("dir_llseek dropping %p content\n", file);
453 reset_readdir(fi);
454 }
455
456 /* bump dir_release_count if we did a forward seek */
457 if (offset > old_offset)
458 fi->dir_release_count--;
459 }
460 mutex_unlock(&inode->i_mutex);
461 return retval;
462}
463
464/*
465 * Process result of a lookup/open request.
466 *
467 * Mainly, make sure we return the final req->r_dentry (if it already
468 * existed) in place of the original VFS-provided dentry when they
469 * differ.
470 *
471 * Gracefully handle the case where the MDS replies with -ENOENT and
472 * no trace (which it may do, at its discretion, e.g., if it doesn't
473 * care to issue a lease on the negative dentry).
474 */
475struct dentry *ceph_finish_lookup(struct ceph_mds_request *req,
476 struct dentry *dentry, int err)
477{
478 struct ceph_client *client = ceph_client(dentry->d_sb);
479 struct inode *parent = dentry->d_parent->d_inode;
480
481 /* .snap dir? */
482 if (err == -ENOENT &&
483 ceph_vino(parent).ino != CEPH_INO_ROOT && /* no .snap in root dir */
484 strcmp(dentry->d_name.name,
485 client->mount_args->snapdir_name) == 0) {
486 struct inode *inode = ceph_get_snapdir(parent);
487 dout("ENOENT on snapdir %p '%.*s', linking to snapdir %p\n",
488 dentry, dentry->d_name.len, dentry->d_name.name, inode);
489 d_add(dentry, inode);
490 err = 0;
491 }
492
493 if (err == -ENOENT) {
494 /* no trace? */
495 err = 0;
496 if (!req->r_reply_info.head->is_dentry) {
497 dout("ENOENT and no trace, dentry %p inode %p\n",
498 dentry, dentry->d_inode);
499 if (dentry->d_inode) {
500 d_drop(dentry);
501 err = -ENOENT;
502 } else {
503 d_add(dentry, NULL);
504 }
505 }
506 }
507 if (err)
508 dentry = ERR_PTR(err);
509 else if (dentry != req->r_dentry)
510 dentry = dget(req->r_dentry); /* we got spliced */
511 else
512 dentry = NULL;
513 return dentry;
514}
515
516static int is_root_ceph_dentry(struct inode *inode, struct dentry *dentry)
517{
518 return ceph_ino(inode) == CEPH_INO_ROOT &&
519 strncmp(dentry->d_name.name, ".ceph", 5) == 0;
520}
521
522/*
523 * Look up a single dir entry. If there is a lookup intent, inform
524 * the MDS so that it gets our 'caps wanted' value in a single op.
525 */
526static struct dentry *ceph_lookup(struct inode *dir, struct dentry *dentry,
527 struct nameidata *nd)
528{
529 struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
530 struct ceph_mds_client *mdsc = &client->mdsc;
531 struct ceph_mds_request *req;
532 int op;
533 int err;
534
535 dout("lookup %p dentry %p '%.*s'\n",
536 dir, dentry, dentry->d_name.len, dentry->d_name.name);
537
538 if (dentry->d_name.len > NAME_MAX)
539 return ERR_PTR(-ENAMETOOLONG);
540
541 err = ceph_init_dentry(dentry);
542 if (err < 0)
543 return ERR_PTR(err);
544
545 /* open (but not create!) intent? */
546 if (nd &&
547 (nd->flags & LOOKUP_OPEN) &&
548 (nd->flags & LOOKUP_CONTINUE) == 0 && /* only open last component */
549 !(nd->intent.open.flags & O_CREAT)) {
550 int mode = nd->intent.open.create_mode & ~current->fs->umask;
551 return ceph_lookup_open(dir, dentry, nd, mode, 1);
552 }
553
554 /* can we conclude ENOENT locally? */
555 if (dentry->d_inode == NULL) {
556 struct ceph_inode_info *ci = ceph_inode(dir);
557 struct ceph_dentry_info *di = ceph_dentry(dentry);
558
559 spin_lock(&dir->i_lock);
560 dout(" dir %p flags are %d\n", dir, ci->i_ceph_flags);
561 if (strncmp(dentry->d_name.name,
562 client->mount_args->snapdir_name,
563 dentry->d_name.len) &&
564 !is_root_ceph_dentry(dir, dentry) &&
565 (ci->i_ceph_flags & CEPH_I_COMPLETE) &&
566 (__ceph_caps_issued_mask(ci, CEPH_CAP_FILE_SHARED, 1))) {
567 di->offset = ci->i_max_offset++;
568 spin_unlock(&dir->i_lock);
569 dout(" dir %p complete, -ENOENT\n", dir);
570 d_add(dentry, NULL);
571 di->lease_shared_gen = ci->i_shared_gen;
572 return NULL;
573 }
574 spin_unlock(&dir->i_lock);
575 }
576
577 op = ceph_snap(dir) == CEPH_SNAPDIR ?
578 CEPH_MDS_OP_LOOKUPSNAP : CEPH_MDS_OP_LOOKUP;
579 req = ceph_mdsc_create_request(mdsc, op, USE_ANY_MDS);
580 if (IS_ERR(req))
581 return ERR_PTR(PTR_ERR(req));
582 req->r_dentry = dget(dentry);
583 req->r_num_caps = 2;
584 /* we only need inode linkage */
585 req->r_args.getattr.mask = cpu_to_le32(CEPH_STAT_CAP_INODE);
586 req->r_locked_dir = dir;
587 err = ceph_mdsc_do_request(mdsc, NULL, req);
588 dentry = ceph_finish_lookup(req, dentry, err);
589 ceph_mdsc_put_request(req); /* will dput(dentry) */
590 dout("lookup result=%p\n", dentry);
591 return dentry;
592}
593
594/*
595 * If we do a create but get no trace back from the MDS, follow up with
596 * a lookup (the VFS expects us to link up the provided dentry).
597 */
598int ceph_handle_notrace_create(struct inode *dir, struct dentry *dentry)
599{
600 struct dentry *result = ceph_lookup(dir, dentry, NULL);
601
602 if (result && !IS_ERR(result)) {
603 /*
604 * We created the item, then did a lookup, and found
605 * it was already linked to another inode we already
606 * had in our cache (and thus got spliced). Link our
607 * dentry to that inode, but don't hash it, just in
608 * case the VFS wants to dereference it.
609 */
610 BUG_ON(!result->d_inode);
611 d_instantiate(dentry, result->d_inode);
612 return 0;
613 }
614 return PTR_ERR(result);
615}
616
617static int ceph_mknod(struct inode *dir, struct dentry *dentry,
618 int mode, dev_t rdev)
619{
620 struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
621 struct ceph_mds_client *mdsc = &client->mdsc;
622 struct ceph_mds_request *req;
623 int err;
624
625 if (ceph_snap(dir) != CEPH_NOSNAP)
626 return -EROFS;
627
628 dout("mknod in dir %p dentry %p mode 0%o rdev %d\n",
629 dir, dentry, mode, rdev);
630 req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_MKNOD, USE_AUTH_MDS);
631 if (IS_ERR(req)) {
632 d_drop(dentry);
633 return PTR_ERR(req);
634 }
635 req->r_dentry = dget(dentry);
636 req->r_num_caps = 2;
637 req->r_locked_dir = dir;
638 req->r_args.mknod.mode = cpu_to_le32(mode);
639 req->r_args.mknod.rdev = cpu_to_le32(rdev);
640 req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
641 req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
642 err = ceph_mdsc_do_request(mdsc, dir, req);
643 if (!err && !req->r_reply_info.head->is_dentry)
644 err = ceph_handle_notrace_create(dir, dentry);
645 ceph_mdsc_put_request(req);
646 if (err)
647 d_drop(dentry);
648 return err;
649}
650
651static int ceph_create(struct inode *dir, struct dentry *dentry, int mode,
652 struct nameidata *nd)
653{
654 dout("create in dir %p dentry %p name '%.*s'\n",
655 dir, dentry, dentry->d_name.len, dentry->d_name.name);
656
657 if (ceph_snap(dir) != CEPH_NOSNAP)
658 return -EROFS;
659
660 if (nd) {
661 BUG_ON((nd->flags & LOOKUP_OPEN) == 0);
662 dentry = ceph_lookup_open(dir, dentry, nd, mode, 0);
663 /* hrm, what should i do here if we get aliased? */
664 if (IS_ERR(dentry))
665 return PTR_ERR(dentry);
666 return 0;
667 }
668
669 /* fall back to mknod */
670 return ceph_mknod(dir, dentry, (mode & ~S_IFMT) | S_IFREG, 0);
671}
672
673static int ceph_symlink(struct inode *dir, struct dentry *dentry,
674 const char *dest)
675{
676 struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
677 struct ceph_mds_client *mdsc = &client->mdsc;
678 struct ceph_mds_request *req;
679 int err;
680
681 if (ceph_snap(dir) != CEPH_NOSNAP)
682 return -EROFS;
683
684 dout("symlink in dir %p dentry %p to '%s'\n", dir, dentry, dest);
685 req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_SYMLINK, USE_AUTH_MDS);
686 if (IS_ERR(req)) {
687 d_drop(dentry);
688 return PTR_ERR(req);
689 }
690 req->r_dentry = dget(dentry);
691 req->r_num_caps = 2;
692 req->r_path2 = kstrdup(dest, GFP_NOFS);
693 req->r_locked_dir = dir;
694 req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
695 req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
696 err = ceph_mdsc_do_request(mdsc, dir, req);
697 if (!err && !req->r_reply_info.head->is_dentry)
698 err = ceph_handle_notrace_create(dir, dentry);
699 ceph_mdsc_put_request(req);
700 if (err)
701 d_drop(dentry);
702 return err;
703}
704
705static int ceph_mkdir(struct inode *dir, struct dentry *dentry, int mode)
706{
707 struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
708 struct ceph_mds_client *mdsc = &client->mdsc;
709 struct ceph_mds_request *req;
710 int err = -EROFS;
711 int op;
712
713 if (ceph_snap(dir) == CEPH_SNAPDIR) {
714 /* mkdir .snap/foo is a MKSNAP */
715 op = CEPH_MDS_OP_MKSNAP;
716 dout("mksnap dir %p snap '%.*s' dn %p\n", dir,
717 dentry->d_name.len, dentry->d_name.name, dentry);
718 } else if (ceph_snap(dir) == CEPH_NOSNAP) {
719 dout("mkdir dir %p dn %p mode 0%o\n", dir, dentry, mode);
720 op = CEPH_MDS_OP_MKDIR;
721 } else {
722 goto out;
723 }
724 req = ceph_mdsc_create_request(mdsc, op, USE_AUTH_MDS);
725 if (IS_ERR(req)) {
726 err = PTR_ERR(req);
727 goto out;
728 }
729
730 req->r_dentry = dget(dentry);
731 req->r_num_caps = 2;
732 req->r_locked_dir = dir;
733 req->r_args.mkdir.mode = cpu_to_le32(mode);
734 req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
735 req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
736 err = ceph_mdsc_do_request(mdsc, dir, req);
737 if (!err && !req->r_reply_info.head->is_dentry)
738 err = ceph_handle_notrace_create(dir, dentry);
739 ceph_mdsc_put_request(req);
740out:
741 if (err < 0)
742 d_drop(dentry);
743 return err;
744}
745
746static int ceph_link(struct dentry *old_dentry, struct inode *dir,
747 struct dentry *dentry)
748{
749 struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
750 struct ceph_mds_client *mdsc = &client->mdsc;
751 struct ceph_mds_request *req;
752 int err;
753
754 if (ceph_snap(dir) != CEPH_NOSNAP)
755 return -EROFS;
756
757 dout("link in dir %p old_dentry %p dentry %p\n", dir,
758 old_dentry, dentry);
759 req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_LINK, USE_AUTH_MDS);
760 if (IS_ERR(req)) {
761 d_drop(dentry);
762 return PTR_ERR(req);
763 }
764 req->r_dentry = dget(dentry);
765 req->r_num_caps = 2;
766 req->r_old_dentry = dget(old_dentry); /* or inode? hrm. */
767 req->r_locked_dir = dir;
768 req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
769 req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
770 err = ceph_mdsc_do_request(mdsc, dir, req);
771 if (err)
772 d_drop(dentry);
773 else if (!req->r_reply_info.head->is_dentry)
774 d_instantiate(dentry, igrab(old_dentry->d_inode));
775 ceph_mdsc_put_request(req);
776 return err;
777}
778
779/*
780 * For a soon-to-be unlinked file, drop the AUTH_RDCACHE caps. If it
781 * looks like the link count will hit 0, drop any other caps (other
782 * than PIN) we don't specifically want (due to the file still being
783 * open).
784 */
785static int drop_caps_for_unlink(struct inode *inode)
786{
787 struct ceph_inode_info *ci = ceph_inode(inode);
788 int drop = CEPH_CAP_LINK_SHARED | CEPH_CAP_LINK_EXCL;
789
790 spin_lock(&inode->i_lock);
791 if (inode->i_nlink == 1) {
792 drop |= ~(__ceph_caps_wanted(ci) | CEPH_CAP_PIN);
793 ci->i_ceph_flags |= CEPH_I_NODELAY;
794 }
795 spin_unlock(&inode->i_lock);
796 return drop;
797}
798
799/*
800 * rmdir and unlink are differ only by the metadata op code
801 */
802static int ceph_unlink(struct inode *dir, struct dentry *dentry)
803{
804 struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
805 struct ceph_mds_client *mdsc = &client->mdsc;
806 struct inode *inode = dentry->d_inode;
807 struct ceph_mds_request *req;
808 int err = -EROFS;
809 int op;
810
811 if (ceph_snap(dir) == CEPH_SNAPDIR) {
812 /* rmdir .snap/foo is RMSNAP */
813 dout("rmsnap dir %p '%.*s' dn %p\n", dir, dentry->d_name.len,
814 dentry->d_name.name, dentry);
815 op = CEPH_MDS_OP_RMSNAP;
816 } else if (ceph_snap(dir) == CEPH_NOSNAP) {
817 dout("unlink/rmdir dir %p dn %p inode %p\n",
818 dir, dentry, inode);
819 op = ((dentry->d_inode->i_mode & S_IFMT) == S_IFDIR) ?
820 CEPH_MDS_OP_RMDIR : CEPH_MDS_OP_UNLINK;
821 } else
822 goto out;
823 req = ceph_mdsc_create_request(mdsc, op, USE_AUTH_MDS);
824 if (IS_ERR(req)) {
825 err = PTR_ERR(req);
826 goto out;
827 }
828 req->r_dentry = dget(dentry);
829 req->r_num_caps = 2;
830 req->r_locked_dir = dir;
831 req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
832 req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
833 req->r_inode_drop = drop_caps_for_unlink(inode);
834 err = ceph_mdsc_do_request(mdsc, dir, req);
835 if (!err && !req->r_reply_info.head->is_dentry)
836 d_delete(dentry);
837 ceph_mdsc_put_request(req);
838out:
839 return err;
840}
841
842static int ceph_rename(struct inode *old_dir, struct dentry *old_dentry,
843 struct inode *new_dir, struct dentry *new_dentry)
844{
845 struct ceph_client *client = ceph_sb_to_client(old_dir->i_sb);
846 struct ceph_mds_client *mdsc = &client->mdsc;
847 struct ceph_mds_request *req;
848 int err;
849
850 if (ceph_snap(old_dir) != ceph_snap(new_dir))
851 return -EXDEV;
852 if (ceph_snap(old_dir) != CEPH_NOSNAP ||
853 ceph_snap(new_dir) != CEPH_NOSNAP)
854 return -EROFS;
855 dout("rename dir %p dentry %p to dir %p dentry %p\n",
856 old_dir, old_dentry, new_dir, new_dentry);
857 req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_RENAME, USE_AUTH_MDS);
858 if (IS_ERR(req))
859 return PTR_ERR(req);
860 req->r_dentry = dget(new_dentry);
861 req->r_num_caps = 2;
862 req->r_old_dentry = dget(old_dentry);
863 req->r_locked_dir = new_dir;
864 req->r_old_dentry_drop = CEPH_CAP_FILE_SHARED;
865 req->r_old_dentry_unless = CEPH_CAP_FILE_EXCL;
866 req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
867 req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
868 /* release LINK_RDCACHE on source inode (mds will lock it) */
869 req->r_old_inode_drop = CEPH_CAP_LINK_SHARED;
870 if (new_dentry->d_inode)
871 req->r_inode_drop = drop_caps_for_unlink(new_dentry->d_inode);
872 err = ceph_mdsc_do_request(mdsc, old_dir, req);
873 if (!err && !req->r_reply_info.head->is_dentry) {
874 /*
875 * Normally d_move() is done by fill_trace (called by
876 * do_request, above). If there is no trace, we need
877 * to do it here.
878 */
879 d_move(old_dentry, new_dentry);
880 }
881 ceph_mdsc_put_request(req);
882 return err;
883}
884
885
886/*
887 * Check if dentry lease is valid. If not, delete the lease. Try to
888 * renew if the least is more than half up.
889 */
890static int dentry_lease_is_valid(struct dentry *dentry)
891{
892 struct ceph_dentry_info *di;
893 struct ceph_mds_session *s;
894 int valid = 0;
895 u32 gen;
896 unsigned long ttl;
897 struct ceph_mds_session *session = NULL;
898 struct inode *dir = NULL;
899 u32 seq = 0;
900
901 spin_lock(&dentry->d_lock);
902 di = ceph_dentry(dentry);
903 if (di && di->lease_session) {
904 s = di->lease_session;
905 spin_lock(&s->s_cap_lock);
906 gen = s->s_cap_gen;
907 ttl = s->s_cap_ttl;
908 spin_unlock(&s->s_cap_lock);
909
910 if (di->lease_gen == gen &&
911 time_before(jiffies, dentry->d_time) &&
912 time_before(jiffies, ttl)) {
913 valid = 1;
914 if (di->lease_renew_after &&
915 time_after(jiffies, di->lease_renew_after)) {
916 /* we should renew */
917 dir = dentry->d_parent->d_inode;
918 session = ceph_get_mds_session(s);
919 seq = di->lease_seq;
920 di->lease_renew_after = 0;
921 di->lease_renew_from = jiffies;
922 }
923 }
924 }
925 spin_unlock(&dentry->d_lock);
926
927 if (session) {
928 ceph_mdsc_lease_send_msg(session, dir, dentry,
929 CEPH_MDS_LEASE_RENEW, seq);
930 ceph_put_mds_session(session);
931 }
932 dout("dentry_lease_is_valid - dentry %p = %d\n", dentry, valid);
933 return valid;
934}
935
936/*
937 * Check if directory-wide content lease/cap is valid.
938 */
939static int dir_lease_is_valid(struct inode *dir, struct dentry *dentry)
940{
941 struct ceph_inode_info *ci = ceph_inode(dir);
942 struct ceph_dentry_info *di = ceph_dentry(dentry);
943 int valid = 0;
944
945 spin_lock(&dir->i_lock);
946 if (ci->i_shared_gen == di->lease_shared_gen)
947 valid = __ceph_caps_issued_mask(ci, CEPH_CAP_FILE_SHARED, 1);
948 spin_unlock(&dir->i_lock);
949 dout("dir_lease_is_valid dir %p v%u dentry %p v%u = %d\n",
950 dir, (unsigned)ci->i_shared_gen, dentry,
951 (unsigned)di->lease_shared_gen, valid);
952 return valid;
953}
954
955/*
956 * Check if cached dentry can be trusted.
957 */
958static int ceph_d_revalidate(struct dentry *dentry, struct nameidata *nd)
959{
960 struct inode *dir = dentry->d_parent->d_inode;
961
962 dout("d_revalidate %p '%.*s' inode %p\n", dentry,
963 dentry->d_name.len, dentry->d_name.name, dentry->d_inode);
964
965 /* always trust cached snapped dentries, snapdir dentry */
966 if (ceph_snap(dir) != CEPH_NOSNAP) {
967 dout("d_revalidate %p '%.*s' inode %p is SNAPPED\n", dentry,
968 dentry->d_name.len, dentry->d_name.name, dentry->d_inode);
969 goto out_touch;
970 }
971 if (dentry->d_inode && ceph_snap(dentry->d_inode) == CEPH_SNAPDIR)
972 goto out_touch;
973
974 if (dentry_lease_is_valid(dentry) ||
975 dir_lease_is_valid(dir, dentry))
976 goto out_touch;
977
978 dout("d_revalidate %p invalid\n", dentry);
979 d_drop(dentry);
980 return 0;
981out_touch:
982 ceph_dentry_lru_touch(dentry);
983 return 1;
984}
985
986/*
987 * When a dentry is released, clear the dir I_COMPLETE if it was part
988 * of the current dir gen.
989 */
990static void ceph_dentry_release(struct dentry *dentry)
991{
992 struct ceph_dentry_info *di = ceph_dentry(dentry);
993 struct inode *parent_inode = dentry->d_parent->d_inode;
994
995 if (parent_inode) {
996 struct ceph_inode_info *ci = ceph_inode(parent_inode);
997
998 spin_lock(&parent_inode->i_lock);
999 if (ci->i_shared_gen == di->lease_shared_gen) {
1000 dout(" clearing %p complete (d_release)\n",
1001 parent_inode);
1002 ci->i_ceph_flags &= ~CEPH_I_COMPLETE;
1003 ci->i_release_count++;
1004 }
1005 spin_unlock(&parent_inode->i_lock);
1006 }
1007 if (di) {
1008 ceph_dentry_lru_del(dentry);
1009 if (di->lease_session)
1010 ceph_put_mds_session(di->lease_session);
1011 kmem_cache_free(ceph_dentry_cachep, di);
1012 dentry->d_fsdata = NULL;
1013 }
1014}
1015
1016static int ceph_snapdir_d_revalidate(struct dentry *dentry,
1017 struct nameidata *nd)
1018{
1019 /*
1020 * Eventually, we'll want to revalidate snapped metadata
1021 * too... probably...
1022 */
1023 return 1;
1024}
1025
1026
1027
1028/*
1029 * read() on a dir. This weird interface hack only works if mounted
1030 * with '-o dirstat'.
1031 */
1032static ssize_t ceph_read_dir(struct file *file, char __user *buf, size_t size,
1033 loff_t *ppos)
1034{
1035 struct ceph_file_info *cf = file->private_data;
1036 struct inode *inode = file->f_dentry->d_inode;
1037 struct ceph_inode_info *ci = ceph_inode(inode);
1038 int left;
1039
1040 if (!ceph_test_opt(ceph_client(inode->i_sb), DIRSTAT))
1041 return -EISDIR;
1042
1043 if (!cf->dir_info) {
1044 cf->dir_info = kmalloc(1024, GFP_NOFS);
1045 if (!cf->dir_info)
1046 return -ENOMEM;
1047 cf->dir_info_len =
1048 sprintf(cf->dir_info,
1049 "entries: %20lld\n"
1050 " files: %20lld\n"
1051 " subdirs: %20lld\n"
1052 "rentries: %20lld\n"
1053 " rfiles: %20lld\n"
1054 " rsubdirs: %20lld\n"
1055 "rbytes: %20lld\n"
1056 "rctime: %10ld.%09ld\n",
1057 ci->i_files + ci->i_subdirs,
1058 ci->i_files,
1059 ci->i_subdirs,
1060 ci->i_rfiles + ci->i_rsubdirs,
1061 ci->i_rfiles,
1062 ci->i_rsubdirs,
1063 ci->i_rbytes,
1064 (long)ci->i_rctime.tv_sec,
1065 (long)ci->i_rctime.tv_nsec);
1066 }
1067
1068 if (*ppos >= cf->dir_info_len)
1069 return 0;
1070 size = min_t(unsigned, size, cf->dir_info_len-*ppos);
1071 left = copy_to_user(buf, cf->dir_info + *ppos, size);
1072 if (left == size)
1073 return -EFAULT;
1074 *ppos += (size - left);
1075 return size - left;
1076}
1077
1078/*
1079 * an fsync() on a dir will wait for any uncommitted directory
1080 * operations to commit.
1081 */
1082static int ceph_dir_fsync(struct file *file, struct dentry *dentry,
1083 int datasync)
1084{
1085 struct inode *inode = dentry->d_inode;
1086 struct ceph_inode_info *ci = ceph_inode(inode);
1087 struct list_head *head = &ci->i_unsafe_dirops;
1088 struct ceph_mds_request *req;
1089 u64 last_tid;
1090 int ret = 0;
1091
1092 dout("dir_fsync %p\n", inode);
1093 spin_lock(&ci->i_unsafe_lock);
1094 if (list_empty(head))
1095 goto out;
1096
1097 req = list_entry(head->prev,
1098 struct ceph_mds_request, r_unsafe_dir_item);
1099 last_tid = req->r_tid;
1100
1101 do {
1102 ceph_mdsc_get_request(req);
1103 spin_unlock(&ci->i_unsafe_lock);
1104 dout("dir_fsync %p wait on tid %llu (until %llu)\n",
1105 inode, req->r_tid, last_tid);
1106 if (req->r_timeout) {
1107 ret = wait_for_completion_timeout(
1108 &req->r_safe_completion, req->r_timeout);
1109 if (ret > 0)
1110 ret = 0;
1111 else if (ret == 0)
1112 ret = -EIO; /* timed out */
1113 } else {
1114 wait_for_completion(&req->r_safe_completion);
1115 }
1116 spin_lock(&ci->i_unsafe_lock);
1117 ceph_mdsc_put_request(req);
1118
1119 if (ret || list_empty(head))
1120 break;
1121 req = list_entry(head->next,
1122 struct ceph_mds_request, r_unsafe_dir_item);
1123 } while (req->r_tid < last_tid);
1124out:
1125 spin_unlock(&ci->i_unsafe_lock);
1126 return ret;
1127}
1128
1129/*
1130 * We maintain a private dentry LRU.
1131 *
1132 * FIXME: this needs to be changed to a per-mds lru to be useful.
1133 */
1134void ceph_dentry_lru_add(struct dentry *dn)
1135{
1136 struct ceph_dentry_info *di = ceph_dentry(dn);
1137 struct ceph_mds_client *mdsc;
1138
1139 dout("dentry_lru_add %p %p '%.*s'\n", di, dn,
1140 dn->d_name.len, dn->d_name.name);
1141 if (di) {
1142 mdsc = &ceph_client(dn->d_sb)->mdsc;
1143 spin_lock(&mdsc->dentry_lru_lock);
1144 list_add_tail(&di->lru, &mdsc->dentry_lru);
1145 mdsc->num_dentry++;
1146 spin_unlock(&mdsc->dentry_lru_lock);
1147 }
1148}
1149
1150void ceph_dentry_lru_touch(struct dentry *dn)
1151{
1152 struct ceph_dentry_info *di = ceph_dentry(dn);
1153 struct ceph_mds_client *mdsc;
1154
1155 dout("dentry_lru_touch %p %p '%.*s'\n", di, dn,
1156 dn->d_name.len, dn->d_name.name);
1157 if (di) {
1158 mdsc = &ceph_client(dn->d_sb)->mdsc;
1159 spin_lock(&mdsc->dentry_lru_lock);
1160 list_move_tail(&di->lru, &mdsc->dentry_lru);
1161 spin_unlock(&mdsc->dentry_lru_lock);
1162 }
1163}
1164
1165void ceph_dentry_lru_del(struct dentry *dn)
1166{
1167 struct ceph_dentry_info *di = ceph_dentry(dn);
1168 struct ceph_mds_client *mdsc;
1169
1170 dout("dentry_lru_del %p %p '%.*s'\n", di, dn,
1171 dn->d_name.len, dn->d_name.name);
1172 if (di) {
1173 mdsc = &ceph_client(dn->d_sb)->mdsc;
1174 spin_lock(&mdsc->dentry_lru_lock);
1175 list_del_init(&di->lru);
1176 mdsc->num_dentry--;
1177 spin_unlock(&mdsc->dentry_lru_lock);
1178 }
1179}
1180
1181const struct file_operations ceph_dir_fops = {
1182 .read = ceph_read_dir,
1183 .readdir = ceph_readdir,
1184 .llseek = ceph_dir_llseek,
1185 .open = ceph_open,
1186 .release = ceph_release,
1187 .unlocked_ioctl = ceph_ioctl,
1188 .fsync = ceph_dir_fsync,
1189};
1190
1191const struct inode_operations ceph_dir_iops = {
1192 .lookup = ceph_lookup,
1193 .permission = ceph_permission,
1194 .getattr = ceph_getattr,
1195 .setattr = ceph_setattr,
1196 .setxattr = ceph_setxattr,
1197 .getxattr = ceph_getxattr,
1198 .listxattr = ceph_listxattr,
1199 .removexattr = ceph_removexattr,
1200 .mknod = ceph_mknod,
1201 .symlink = ceph_symlink,
1202 .mkdir = ceph_mkdir,
1203 .link = ceph_link,
1204 .unlink = ceph_unlink,
1205 .rmdir = ceph_unlink,
1206 .rename = ceph_rename,
1207 .create = ceph_create,
1208};
1209
1210struct dentry_operations ceph_dentry_ops = {
1211 .d_revalidate = ceph_d_revalidate,
1212 .d_release = ceph_dentry_release,
1213};
1214
1215struct dentry_operations ceph_snapdir_dentry_ops = {
1216 .d_revalidate = ceph_snapdir_d_revalidate,
1217};
1218
1219struct dentry_operations ceph_snap_dentry_ops = {
1220};
fs/ceph/export.c+223−0
@@ -0,0 +1,223 @@
1#include "ceph_debug.h"
2
3#include <linux/exportfs.h>
4#include <asm/unaligned.h>
5
6#include "super.h"
7
8/*
9 * NFS export support
10 *
11 * NFS re-export of a ceph mount is, at present, only semireliable.
12 * The basic issue is that the Ceph architectures doesn't lend itself
13 * well to generating filehandles that will remain valid forever.
14 *
15 * So, we do our best. If you're lucky, your inode will be in the
16 * client's cache. If it's not, and you have a connectable fh, then
17 * the MDS server may be able to find it for you. Otherwise, you get
18 * ESTALE.
19 *
20 * There are ways to this more reliable, but in the non-connectable fh
21 * case, we won't every work perfectly, and in the connectable case,
22 * some changes are needed on the MDS side to work better.
23 */
24
25/*
26 * Basic fh
27 */
28struct ceph_nfs_fh {
29 u64 ino;
30} __attribute__ ((packed));
31
32/*
33 * Larger 'connectable' fh that includes parent ino and name hash.
34 * Use this whenever possible, as it works more reliably.
35 */
36struct ceph_nfs_confh {
37 u64 ino, parent_ino;
38 u32 parent_name_hash;
39} __attribute__ ((packed));
40
41static int ceph_encode_fh(struct dentry *dentry, u32 *rawfh, int *max_len,
42 int connectable)
43{
44 struct ceph_nfs_fh *fh = (void *)rawfh;
45 struct ceph_nfs_confh *cfh = (void *)rawfh;
46 struct dentry *parent = dentry->d_parent;
47 struct inode *inode = dentry->d_inode;
48 int type;
49
50 /* don't re-export snaps */
51 if (ceph_snap(inode) != CEPH_NOSNAP)
52 return -EINVAL;
53
54 if (*max_len >= sizeof(*cfh)) {
55 dout("encode_fh %p connectable\n", dentry);
56 cfh->ino = ceph_ino(dentry->d_inode);
57 cfh->parent_ino = ceph_ino(parent->d_inode);
58 cfh->parent_name_hash = parent->d_name.hash;
59 *max_len = sizeof(*cfh);
60 type = 2;
61 } else if (*max_len > sizeof(*fh)) {
62 if (connectable)
63 return -ENOSPC;
64 dout("encode_fh %p\n", dentry);
65 fh->ino = ceph_ino(dentry->d_inode);
66 *max_len = sizeof(*fh);
67 type = 1;
68 } else {
69 return -ENOSPC;
70 }
71 return type;
72}
73
74/*
75 * convert regular fh to dentry
76 *
77 * FIXME: we should try harder by querying the mds for the ino.
78 */
79static struct dentry *__fh_to_dentry(struct super_block *sb,
80 struct ceph_nfs_fh *fh)
81{
82 struct inode *inode;
83 struct dentry *dentry;
84 struct ceph_vino vino;
85 int err;
86
87 dout("__fh_to_dentry %llx\n", fh->ino);
88 vino.ino = fh->ino;
89 vino.snap = CEPH_NOSNAP;
90 inode = ceph_find_inode(sb, vino);
91 if (!inode)
92 return ERR_PTR(-ESTALE);
93
94 dentry = d_obtain_alias(inode);
95 if (!dentry) {
96 pr_err("fh_to_dentry %llx -- inode %p but ENOMEM\n",
97 fh->ino, inode);
98 iput(inode);
99 return ERR_PTR(-ENOMEM);
100 }
101 err = ceph_init_dentry(dentry);
102
103 if (err < 0) {
104 iput(inode);
105 return ERR_PTR(err);
106 }
107 dout("__fh_to_dentry %llx %p dentry %p\n", fh->ino, inode, dentry);
108 return dentry;
109}
110
111/*
112 * convert connectable fh to dentry
113 */
114static struct dentry *__cfh_to_dentry(struct super_block *sb,
115 struct ceph_nfs_confh *cfh)
116{
117 struct ceph_mds_client *mdsc = &ceph_client(sb)->mdsc;
118 struct inode *inode;
119 struct dentry *dentry;
120 struct ceph_vino vino;
121 int err;
122
123 dout("__cfh_to_dentry %llx (%llx/%x)\n",
124 cfh->ino, cfh->parent_ino, cfh->parent_name_hash);
125
126 vino.ino = cfh->ino;
127 vino.snap = CEPH_NOSNAP;
128 inode = ceph_find_inode(sb, vino);
129 if (!inode) {
130 struct ceph_mds_request *req;
131
132 req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_LOOKUPHASH,
133 USE_ANY_MDS);
134 if (IS_ERR(req))
135 return ERR_PTR(PTR_ERR(req));
136
137 req->r_ino1 = vino;
138 req->r_ino2.ino = cfh->parent_ino;
139 req->r_ino2.snap = CEPH_NOSNAP;
140 req->r_path2 = kmalloc(16, GFP_NOFS);
141 snprintf(req->r_path2, 16, "%d", cfh->parent_name_hash);
142 req->r_num_caps = 1;
143 err = ceph_mdsc_do_request(mdsc, NULL, req);
144 ceph_mdsc_put_request(req);
145 inode = ceph_find_inode(sb, vino);
146 if (!inode)
147 return ERR_PTR(err ? err : -ESTALE);
148 }
149
150 dentry = d_obtain_alias(inode);
151 if (!dentry) {
152 pr_err("cfh_to_dentry %llx -- inode %p but ENOMEM\n",
153 cfh->ino, inode);
154 iput(inode);
155 return ERR_PTR(-ENOMEM);
156 }
157 err = ceph_init_dentry(dentry);
158 if (err < 0) {
159 iput(inode);
160 return ERR_PTR(err);
161 }
162 dout("__cfh_to_dentry %llx %p dentry %p\n", cfh->ino, inode, dentry);
163 return dentry;
164}
165
166static struct dentry *ceph_fh_to_dentry(struct super_block *sb, struct fid *fid,
167 int fh_len, int fh_type)
168{
169 if (fh_type == 1)
170 return __fh_to_dentry(sb, (struct ceph_nfs_fh *)fid->raw);
171 else
172 return __cfh_to_dentry(sb, (struct ceph_nfs_confh *)fid->raw);
173}
174
175/*
176 * get parent, if possible.
177 *
178 * FIXME: we could do better by querying the mds to discover the
179 * parent.
180 */
181static struct dentry *ceph_fh_to_parent(struct super_block *sb,
182 struct fid *fid,
183 int fh_len, int fh_type)
184{
185 struct ceph_nfs_confh *cfh = (void *)fid->raw;
186 struct ceph_vino vino;
187 struct inode *inode;
188 struct dentry *dentry;
189 int err;
190
191 if (fh_type == 1)
192 return ERR_PTR(-ESTALE);
193
194 pr_debug("fh_to_parent %llx/%d\n", cfh->parent_ino,
195 cfh->parent_name_hash);
196
197 vino.ino = cfh->ino;
198 vino.snap = CEPH_NOSNAP;
199 inode = ceph_find_inode(sb, vino);
200 if (!inode)
201 return ERR_PTR(-ESTALE);
202
203 dentry = d_obtain_alias(inode);
204 if (!dentry) {
205 pr_err("fh_to_parent %llx -- inode %p but ENOMEM\n",
206 cfh->ino, inode);
207 iput(inode);
208 return ERR_PTR(-ENOMEM);
209 }
210 err = ceph_init_dentry(dentry);
211 if (err < 0) {
212 iput(inode);
213 return ERR_PTR(err);
214 }
215 dout("fh_to_parent %llx %p dentry %p\n", cfh->ino, inode, dentry);
216 return dentry;
217}
218
219const struct export_operations ceph_export_ops = {
220 .encode_fh = ceph_encode_fh,
221 .fh_to_dentry = ceph_fh_to_dentry,
222 .fh_to_parent = ceph_fh_to_parent,
223};
fs/ceph/file.c+937−0
@@ -0,0 +1,937 @@
1#include "ceph_debug.h"
2
3#include <linux/sched.h>
4#include <linux/file.h>
5#include <linux/namei.h>
6#include <linux/writeback.h>
7
8#include "super.h"
9#include "mds_client.h"
10
11/*
12 * Ceph file operations
13 *
14 * Implement basic open/close functionality, and implement
15 * read/write.
16 *
17 * We implement three modes of file I/O:
18 * - buffered uses the generic_file_aio_{read,write} helpers
19 *
20 * - synchronous is used when there is multi-client read/write
21 * sharing, avoids the page cache, and synchronously waits for an
22 * ack from the OSD.
23 *
24 * - direct io takes the variant of the sync path that references
25 * user pages directly.
26 *
27 * fsync() flushes and waits on dirty pages, but just queues metadata
28 * for writeback: since the MDS can recover size and mtime there is no
29 * need to wait for MDS acknowledgement.
30 */
31
32
33/*
34 * Prepare an open request. Preallocate ceph_cap to avoid an
35 * inopportune ENOMEM later.
36 */
37static struct ceph_mds_request *
38prepare_open_request(struct super_block *sb, int flags, int create_mode)
39{
40 struct ceph_client *client = ceph_sb_to_client(sb);
41 struct ceph_mds_client *mdsc = &client->mdsc;
42 struct ceph_mds_request *req;
43 int want_auth = USE_ANY_MDS;
44 int op = (flags & O_CREAT) ? CEPH_MDS_OP_CREATE : CEPH_MDS_OP_OPEN;
45
46 if (flags & (O_WRONLY|O_RDWR|O_CREAT|O_TRUNC))
47 want_auth = USE_AUTH_MDS;
48
49 req = ceph_mdsc_create_request(mdsc, op, want_auth);
50 if (IS_ERR(req))
51 goto out;
52 req->r_fmode = ceph_flags_to_mode(flags);
53 req->r_args.open.flags = cpu_to_le32(flags);
54 req->r_args.open.mode = cpu_to_le32(create_mode);
55 req->r_args.open.preferred = cpu_to_le32(-1);
56out:
57 return req;
58}
59
60/*
61 * initialize private struct file data.
62 * if we fail, clean up by dropping fmode reference on the ceph_inode
63 */
64static int ceph_init_file(struct inode *inode, struct file *file, int fmode)
65{
66 struct ceph_file_info *cf;
67 int ret = 0;
68
69 switch (inode->i_mode & S_IFMT) {
70 case S_IFREG:
71 case S_IFDIR:
72 dout("init_file %p %p 0%o (regular)\n", inode, file,
73 inode->i_mode);
74 cf = kmem_cache_alloc(ceph_file_cachep, GFP_NOFS | __GFP_ZERO);
75 if (cf == NULL) {
76 ceph_put_fmode(ceph_inode(inode), fmode); /* clean up */
77 return -ENOMEM;
78 }
79 cf->fmode = fmode;
80 cf->next_offset = 2;
81 file->private_data = cf;
82 BUG_ON(inode->i_fop->release != ceph_release);
83 break;
84
85 case S_IFLNK:
86 dout("init_file %p %p 0%o (symlink)\n", inode, file,
87 inode->i_mode);
88 ceph_put_fmode(ceph_inode(inode), fmode); /* clean up */
89 break;
90
91 default:
92 dout("init_file %p %p 0%o (special)\n", inode, file,
93 inode->i_mode);
94 /*
95 * we need to drop the open ref now, since we don't
96 * have .release set to ceph_release.
97 */
98 ceph_put_fmode(ceph_inode(inode), fmode); /* clean up */
99 BUG_ON(inode->i_fop->release == ceph_release);
100
101 /* call the proper open fop */
102 ret = inode->i_fop->open(inode, file);
103 }
104 return ret;
105}
106
107/*
108 * If the filp already has private_data, that means the file was
109 * already opened by intent during lookup, and we do nothing.
110 *
111 * If we already have the requisite capabilities, we can satisfy
112 * the open request locally (no need to request new caps from the
113 * MDS). We do, however, need to inform the MDS (asynchronously)
114 * if our wanted caps set expands.
115 */
116int ceph_open(struct inode *inode, struct file *file)
117{
118 struct ceph_inode_info *ci = ceph_inode(inode);
119 struct ceph_client *client = ceph_sb_to_client(inode->i_sb);
120 struct ceph_mds_client *mdsc = &client->mdsc;
121 struct ceph_mds_request *req;
122 struct ceph_file_info *cf = file->private_data;
123 struct inode *parent_inode = file->f_dentry->d_parent->d_inode;
124 int err;
125 int flags, fmode, wanted;
126
127 if (cf) {
128 dout("open file %p is already opened\n", file);
129 return 0;
130 }
131
132 /* filter out O_CREAT|O_EXCL; vfs did that already. yuck. */
133 flags = file->f_flags & ~(O_CREAT|O_EXCL);
134 if (S_ISDIR(inode->i_mode))
135 flags = O_DIRECTORY; /* mds likes to know */
136
137 dout("open inode %p ino %llx.%llx file %p flags %d (%d)\n", inode,
138 ceph_vinop(inode), file, flags, file->f_flags);
139 fmode = ceph_flags_to_mode(flags);
140 wanted = ceph_caps_for_mode(fmode);
141
142 /* snapped files are read-only */
143 if (ceph_snap(inode) != CEPH_NOSNAP && (file->f_mode & FMODE_WRITE))
144 return -EROFS;
145
146 /* trivially open snapdir */
147 if (ceph_snap(inode) == CEPH_SNAPDIR) {
148 spin_lock(&inode->i_lock);
149 __ceph_get_fmode(ci, fmode);
150 spin_unlock(&inode->i_lock);
151 return ceph_init_file(inode, file, fmode);
152 }
153
154 /*
155 * No need to block if we have any caps. Update wanted set
156 * asynchronously.
157 */
158 spin_lock(&inode->i_lock);
159 if (__ceph_is_any_real_caps(ci)) {
160 int mds_wanted = __ceph_caps_mds_wanted(ci);
161 int issued = __ceph_caps_issued(ci, NULL);
162
163 dout("open %p fmode %d want %s issued %s using existing\n",
164 inode, fmode, ceph_cap_string(wanted),
165 ceph_cap_string(issued));
166 __ceph_get_fmode(ci, fmode);
167 spin_unlock(&inode->i_lock);
168
169 /* adjust wanted? */
170 if ((issued & wanted) != wanted &&
171 (mds_wanted & wanted) != wanted &&
172 ceph_snap(inode) != CEPH_SNAPDIR)
173 ceph_check_caps(ci, 0, NULL);
174
175 return ceph_init_file(inode, file, fmode);
176 } else if (ceph_snap(inode) != CEPH_NOSNAP &&
177 (ci->i_snap_caps & wanted) == wanted) {
178 __ceph_get_fmode(ci, fmode);
179 spin_unlock(&inode->i_lock);
180 return ceph_init_file(inode, file, fmode);
181 }
182 spin_unlock(&inode->i_lock);
183
184 dout("open fmode %d wants %s\n", fmode, ceph_cap_string(wanted));
185 req = prepare_open_request(inode->i_sb, flags, 0);
186 if (IS_ERR(req)) {
187 err = PTR_ERR(req);
188 goto out;
189 }
190 req->r_inode = igrab(inode);
191 req->r_num_caps = 1;
192 err = ceph_mdsc_do_request(mdsc, parent_inode, req);
193 if (!err)
194 err = ceph_init_file(inode, file, req->r_fmode);
195 ceph_mdsc_put_request(req);
196 dout("open result=%d on %llx.%llx\n", err, ceph_vinop(inode));
197out:
198 return err;
199}
200
201
202/*
203 * Do a lookup + open with a single request.
204 *
205 * If this succeeds, but some subsequent check in the vfs
206 * may_open() fails, the struct *file gets cleaned up (i.e.
207 * ceph_release gets called). So fear not!
208 */
209/*
210 * flags
211 * path_lookup_open -> LOOKUP_OPEN
212 * path_lookup_create -> LOOKUP_OPEN|LOOKUP_CREATE
213 */
214struct dentry *ceph_lookup_open(struct inode *dir, struct dentry *dentry,
215 struct nameidata *nd, int mode,
216 int locked_dir)
217{
218 struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
219 struct ceph_mds_client *mdsc = &client->mdsc;
220 struct file *file = nd->intent.open.file;
221 struct inode *parent_inode = get_dentry_parent_inode(file->f_dentry);
222 struct ceph_mds_request *req;
223 int err;
224 int flags = nd->intent.open.flags - 1; /* silly vfs! */
225
226 dout("ceph_lookup_open dentry %p '%.*s' flags %d mode 0%o\n",
227 dentry, dentry->d_name.len, dentry->d_name.name, flags, mode);
228
229 /* do the open */
230 req = prepare_open_request(dir->i_sb, flags, mode);
231 if (IS_ERR(req))
232 return ERR_PTR(PTR_ERR(req));
233 req->r_dentry = dget(dentry);
234 req->r_num_caps = 2;
235 if (flags & O_CREAT) {
236 req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
237 req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
238 }
239 req->r_locked_dir = dir; /* caller holds dir->i_mutex */
240 err = ceph_mdsc_do_request(mdsc, parent_inode, req);
241 dentry = ceph_finish_lookup(req, dentry, err);
242 if (!err && (flags & O_CREAT) && !req->r_reply_info.head->is_dentry)
243 err = ceph_handle_notrace_create(dir, dentry);
244 if (!err)
245 err = ceph_init_file(req->r_dentry->d_inode, file,
246 req->r_fmode);
247 ceph_mdsc_put_request(req);
248 dout("ceph_lookup_open result=%p\n", dentry);
249 return dentry;
250}
251
252int ceph_release(struct inode *inode, struct file *file)
253{
254 struct ceph_inode_info *ci = ceph_inode(inode);
255 struct ceph_file_info *cf = file->private_data;
256
257 dout("release inode %p file %p\n", inode, file);
258 ceph_put_fmode(ci, cf->fmode);
259 if (cf->last_readdir)
260 ceph_mdsc_put_request(cf->last_readdir);
261 kfree(cf->last_name);
262 kfree(cf->dir_info);
263 dput(cf->dentry);
264 kmem_cache_free(ceph_file_cachep, cf);
265
266 /* wake up anyone waiting for caps on this inode */
267 wake_up(&ci->i_cap_wq);
268 return 0;
269}
270
271/*
272 * build a vector of user pages
273 */
274static struct page **get_direct_page_vector(const char __user *data,
275 int num_pages,
276 loff_t off, size_t len)
277{
278 struct page **pages;
279 int rc;
280
281 pages = kmalloc(sizeof(*pages) * num_pages, GFP_NOFS);
282 if (!pages)
283 return ERR_PTR(-ENOMEM);
284
285 down_read(&current->mm->mmap_sem);
286 rc = get_user_pages(current, current->mm, (unsigned long)data,
287 num_pages, 0, 0, pages, NULL);
288 up_read(&current->mm->mmap_sem);
289 if (rc < 0)
290 goto fail;
291 return pages;
292
293fail:
294 kfree(pages);
295 return ERR_PTR(rc);
296}
297
298static void put_page_vector(struct page **pages, int num_pages)
299{
300 int i;
301
302 for (i = 0; i < num_pages; i++)
303 put_page(pages[i]);
304 kfree(pages);
305}
306
307void ceph_release_page_vector(struct page **pages, int num_pages)
308{
309 int i;
310
311 for (i = 0; i < num_pages; i++)
312 __free_pages(pages[i], 0);
313 kfree(pages);
314}
315
316/*
317 * allocate a vector new pages
318 */
319static struct page **alloc_page_vector(int num_pages)
320{
321 struct page **pages;
322 int i;
323
324 pages = kmalloc(sizeof(*pages) * num_pages, GFP_NOFS);
325 if (!pages)
326 return ERR_PTR(-ENOMEM);
327 for (i = 0; i < num_pages; i++) {
328 pages[i] = alloc_page(GFP_NOFS);
329 if (pages[i] == NULL) {
330 ceph_release_page_vector(pages, i);
331 return ERR_PTR(-ENOMEM);
332 }
333 }
334 return pages;
335}
336
337/*
338 * copy user data into a page vector
339 */
340static int copy_user_to_page_vector(struct page **pages,
341 const char __user *data,
342 loff_t off, size_t len)
343{
344 int i = 0;
345 int po = off & ~PAGE_CACHE_MASK;
346 int left = len;
347 int l, bad;
348
349 while (left > 0) {
350 l = min_t(int, PAGE_CACHE_SIZE-po, left);
351 bad = copy_from_user(page_address(pages[i]) + po, data, l);
352 if (bad == l)
353 return -EFAULT;
354 data += l - bad;
355 left -= l - bad;
356 po += l - bad;
357 if (po == PAGE_CACHE_SIZE) {
358 po = 0;
359 i++;
360 }
361 }
362 return len;
363}
364
365/*
366 * copy user data from a page vector into a user pointer
367 */
368static int copy_page_vector_to_user(struct page **pages, char __user *data,
369 loff_t off, size_t len)
370{
371 int i = 0;
372 int po = off & ~PAGE_CACHE_MASK;
373 int left = len;
374 int l, bad;
375
376 while (left > 0) {
377 l = min_t(int, left, PAGE_CACHE_SIZE-po);
378 bad = copy_to_user(data, page_address(pages[i]) + po, l);
379 if (bad == l)
380 return -EFAULT;
381 data += l - bad;
382 left -= l - bad;
383 if (po) {
384 po += l - bad;
385 if (po == PAGE_CACHE_SIZE)
386 po = 0;
387 }
388 i++;
389 }
390 return len;
391}
392
393/*
394 * Zero an extent within a page vector. Offset is relative to the
395 * start of the first page.
396 */
397static void zero_page_vector_range(int off, int len, struct page **pages)
398{
399 int i = off >> PAGE_CACHE_SHIFT;
400
401 off &= ~PAGE_CACHE_MASK;
402
403 dout("zero_page_vector_page %u~%u\n", off, len);
404
405 /* leading partial page? */
406 if (off) {
407 int end = min((int)PAGE_CACHE_SIZE, off + len);
408 dout("zeroing %d %p head from %d\n", i, pages[i],
409 (int)off);
410 zero_user_segment(pages[i], off, end);
411 len -= (end - off);
412 i++;
413 }
414 while (len >= PAGE_CACHE_SIZE) {
415 dout("zeroing %d %p len=%d\n", i, pages[i], len);
416 zero_user_segment(pages[i], 0, PAGE_CACHE_SIZE);
417 len -= PAGE_CACHE_SIZE;
418 i++;
419 }
420 /* trailing partial page? */
421 if (len) {
422 dout("zeroing %d %p tail to %d\n", i, pages[i], (int)len);
423 zero_user_segment(pages[i], 0, len);
424 }
425}
426
427
428/*
429 * Read a range of bytes striped over one or more objects. Iterate over
430 * objects we stripe over. (That's not atomic, but good enough for now.)
431 *
432 * If we get a short result from the OSD, check against i_size; we need to
433 * only return a short read to the caller if we hit EOF.
434 */
435static int striped_read(struct inode *inode,
436 u64 off, u64 len,
437 struct page **pages, int num_pages,
438 int *checkeof)
439{
440 struct ceph_client *client = ceph_inode_to_client(inode);
441 struct ceph_inode_info *ci = ceph_inode(inode);
442 u64 pos, this_len;
443 int page_off = off & ~PAGE_CACHE_MASK; /* first byte's offset in page */
444 int left, pages_left;
445 int read;
446 struct page **page_pos;
447 int ret;
448 bool hit_stripe, was_short;
449
450 /*
451 * we may need to do multiple reads. not atomic, unfortunately.
452 */
453 pos = off;
454 left = len;
455 page_pos = pages;
456 pages_left = num_pages;
457 read = 0;
458
459more:
460 this_len = left;
461 ret = ceph_osdc_readpages(&client->osdc, ceph_vino(inode),
462 &ci->i_layout, pos, &this_len,
463 ci->i_truncate_seq,
464 ci->i_truncate_size,
465 page_pos, pages_left);
466 hit_stripe = this_len < left;
467 was_short = ret >= 0 && ret < this_len;
468 if (ret == -ENOENT)
469 ret = 0;
470 dout("striped_read %llu~%u (read %u) got %d%s%s\n", pos, left, read,
471 ret, hit_stripe ? " HITSTRIPE" : "", was_short ? " SHORT" : "");
472
473 if (ret > 0) {
474 int didpages =
475 ((pos & ~PAGE_CACHE_MASK) + ret) >> PAGE_CACHE_SHIFT;
476
477 if (read < pos - off) {
478 dout(" zero gap %llu to %llu\n", off + read, pos);
479 zero_page_vector_range(page_off + read,
480 pos - off - read, pages);
481 }
482 pos += ret;
483 read = pos - off;
484 left -= ret;
485 page_pos += didpages;
486 pages_left -= didpages;
487
488 /* hit stripe? */
489 if (left && hit_stripe)
490 goto more;
491 }
492
493 if (was_short) {
494 /* was original extent fully inside i_size? */
495 if (pos + left <= inode->i_size) {
496 dout("zero tail\n");
497 zero_page_vector_range(page_off + read, len - read,
498 pages);
499 read = len;
500 goto out;
501 }
502
503 /* check i_size */
504 *checkeof = 1;
505 }
506
507out:
508 if (ret >= 0)
509 ret = read;
510 dout("striped_read returns %d\n", ret);
511 return ret;
512}
513
514/*
515 * Completely synchronous read and write methods. Direct from __user
516 * buffer to osd, or directly to user pages (if O_DIRECT).
517 *
518 * If the read spans object boundary, just do multiple reads.
519 */
520static ssize_t ceph_sync_read(struct file *file, char __user *data,
521 unsigned len, loff_t *poff, int *checkeof)
522{
523 struct inode *inode = file->f_dentry->d_inode;
524 struct page **pages;
525 u64 off = *poff;
526 int num_pages = calc_pages_for(off, len);
527 int ret;
528
529 dout("sync_read on file %p %llu~%u %s\n", file, off, len,
530 (file->f_flags & O_DIRECT) ? "O_DIRECT" : "");
531
532 if (file->f_flags & O_DIRECT) {
533 pages = get_direct_page_vector(data, num_pages, off, len);
534
535 /*
536 * flush any page cache pages in this range. this
537 * will make concurrent normal and O_DIRECT io slow,
538 * but it will at least behave sensibly when they are
539 * in sequence.
540 */
541 } else {
542 pages = alloc_page_vector(num_pages);
543 }
544 if (IS_ERR(pages))
545 return PTR_ERR(pages);
546
547 ret = filemap_write_and_wait(inode->i_mapping);
548 if (ret < 0)
549 goto done;
550
551 ret = striped_read(inode, off, len, pages, num_pages, checkeof);
552
553 if (ret >= 0 && (file->f_flags & O_DIRECT) == 0)
554 ret = copy_page_vector_to_user(pages, data, off, ret);
555 if (ret >= 0)
556 *poff = off + ret;
557
558done:
559 if (file->f_flags & O_DIRECT)
560 put_page_vector(pages, num_pages);
561 else
562 ceph_release_page_vector(pages, num_pages);
563 dout("sync_read result %d\n", ret);
564 return ret;
565}
566
567/*
568 * Write commit callback, called if we requested both an ACK and
569 * ONDISK commit reply from the OSD.
570 */
571static void sync_write_commit(struct ceph_osd_request *req,
572 struct ceph_msg *msg)
573{
574 struct ceph_inode_info *ci = ceph_inode(req->r_inode);
575
576 dout("sync_write_commit %p tid %llu\n", req, req->r_tid);
577 spin_lock(&ci->i_unsafe_lock);
578 list_del_init(&req->r_unsafe_item);
579 spin_unlock(&ci->i_unsafe_lock);
580 ceph_put_cap_refs(ci, CEPH_CAP_FILE_WR);
581}
582
583/*
584 * Synchronous write, straight from __user pointer or user pages (if
585 * O_DIRECT).
586 *
587 * If write spans object boundary, just do multiple writes. (For a
588 * correct atomic write, we should e.g. take write locks on all
589 * objects, rollback on failure, etc.)
590 */
591static ssize_t ceph_sync_write(struct file *file, const char __user *data,
592 size_t left, loff_t *offset)
593{
594 struct inode *inode = file->f_dentry->d_inode;
595 struct ceph_inode_info *ci = ceph_inode(inode);
596 struct ceph_client *client = ceph_inode_to_client(inode);
597 struct ceph_osd_request *req;
598 struct page **pages;
599 int num_pages;
600 long long unsigned pos;
601 u64 len;
602 int written = 0;
603 int flags;
604 int do_sync = 0;
605 int check_caps = 0;
606 int ret;
607 struct timespec mtime = CURRENT_TIME;
608
609 if (ceph_snap(file->f_dentry->d_inode) != CEPH_NOSNAP)
610 return -EROFS;
611
612 dout("sync_write on file %p %lld~%u %s\n", file, *offset,
613 (unsigned)left, (file->f_flags & O_DIRECT) ? "O_DIRECT" : "");
614
615 if (file->f_flags & O_APPEND)
616 pos = i_size_read(inode);
617 else
618 pos = *offset;
619
620 ret = filemap_write_and_wait_range(inode->i_mapping, pos, pos + left);
621 if (ret < 0)
622 return ret;
623
624 ret = invalidate_inode_pages2_range(inode->i_mapping,
625 pos >> PAGE_CACHE_SHIFT,
626 (pos + left) >> PAGE_CACHE_SHIFT);
627 if (ret < 0)
628 dout("invalidate_inode_pages2_range returned %d\n", ret);
629
630 flags = CEPH_OSD_FLAG_ORDERSNAP |
631 CEPH_OSD_FLAG_ONDISK |
632 CEPH_OSD_FLAG_WRITE;
633 if ((file->f_flags & (O_SYNC|O_DIRECT)) == 0)
634 flags |= CEPH_OSD_FLAG_ACK;
635 else
636 do_sync = 1;
637
638 /*
639 * we may need to do multiple writes here if we span an object
640 * boundary. this isn't atomic, unfortunately. :(
641 */
642more:
643 len = left;
644 req = ceph_osdc_new_request(&client->osdc, &ci->i_layout,
645 ceph_vino(inode), pos, &len,
646 CEPH_OSD_OP_WRITE, flags,
647 ci->i_snap_realm->cached_context,
648 do_sync,
649 ci->i_truncate_seq, ci->i_truncate_size,
650 &mtime, false, 2);
651 if (IS_ERR(req))
652 return PTR_ERR(req);
653
654 num_pages = calc_pages_for(pos, len);
655
656 if (file->f_flags & O_DIRECT) {
657 pages = get_direct_page_vector(data, num_pages, pos, len);
658 if (IS_ERR(pages)) {
659 ret = PTR_ERR(pages);
660 goto out;
661 }
662
663 /*
664 * throw out any page cache pages in this range. this
665 * may block.
666 */
667 truncate_inode_pages_range(inode->i_mapping, pos, pos+len);
668 } else {
669 pages = alloc_page_vector(num_pages);
670 if (IS_ERR(pages)) {
671 ret = PTR_ERR(pages);
672 goto out;
673 }
674 ret = copy_user_to_page_vector(pages, data, pos, len);
675 if (ret < 0) {
676 ceph_release_page_vector(pages, num_pages);
677 goto out;
678 }
679
680 if ((file->f_flags & O_SYNC) == 0) {
681 /* get a second commit callback */
682 req->r_safe_callback = sync_write_commit;
683 req->r_own_pages = 1;
684 }
685 }
686 req->r_pages = pages;
687 req->r_num_pages = num_pages;
688 req->r_inode = inode;
689
690 ret = ceph_osdc_start_request(&client->osdc, req, false);
691 if (!ret) {
692 if (req->r_safe_callback) {
693 /*
694 * Add to inode unsafe list only after we
695 * start_request so that a tid has been assigned.
696 */
697 spin_lock(&ci->i_unsafe_lock);
698 list_add(&ci->i_unsafe_writes, &req->r_unsafe_item);
699 spin_unlock(&ci->i_unsafe_lock);
700 ceph_get_cap_refs(ci, CEPH_CAP_FILE_WR);
701 }
702 ret = ceph_osdc_wait_request(&client->osdc, req);
703 }
704
705 if (file->f_flags & O_DIRECT)
706 put_page_vector(pages, num_pages);
707 else if (file->f_flags & O_SYNC)
708 ceph_release_page_vector(pages, num_pages);
709
710out:
711 ceph_osdc_put_request(req);
712 if (ret == 0) {
713 pos += len;
714 written += len;
715 left -= len;
716 if (left)
717 goto more;
718
719 ret = written;
720 *offset = pos;
721 if (pos > i_size_read(inode))
722 check_caps = ceph_inode_set_size(inode, pos);
723 if (check_caps)
724 ceph_check_caps(ceph_inode(inode), CHECK_CAPS_AUTHONLY,
725 NULL);
726 }
727 return ret;
728}
729
730/*
731 * Wrap generic_file_aio_read with checks for cap bits on the inode.
732 * Atomically grab references, so that those bits are not released
733 * back to the MDS mid-read.
734 *
735 * Hmm, the sync read case isn't actually async... should it be?
736 */
737static ssize_t ceph_aio_read(struct kiocb *iocb, const struct iovec *iov,
738 unsigned long nr_segs, loff_t pos)
739{
740 struct file *filp = iocb->ki_filp;
741 loff_t *ppos = &iocb->ki_pos;
742 size_t len = iov->iov_len;
743 struct inode *inode = filp->f_dentry->d_inode;
744 struct ceph_inode_info *ci = ceph_inode(inode);
745 void *base = iov->iov_base;
746 ssize_t ret;
747 int got = 0;
748 int checkeof = 0, read = 0;
749
750 dout("aio_read %p %llx.%llx %llu~%u trying to get caps on %p\n",
751 inode, ceph_vinop(inode), pos, (unsigned)len, inode);
752again:
753 __ceph_do_pending_vmtruncate(inode);
754 ret = ceph_get_caps(ci, CEPH_CAP_FILE_RD, CEPH_CAP_FILE_CACHE,
755 &got, -1);
756 if (ret < 0)
757 goto out;
758 dout("aio_read %p %llx.%llx %llu~%u got cap refs on %s\n",
759 inode, ceph_vinop(inode), pos, (unsigned)len,
760 ceph_cap_string(got));
761
762 if ((got & CEPH_CAP_FILE_CACHE) == 0 ||
763 (iocb->ki_filp->f_flags & O_DIRECT) ||
764 (inode->i_sb->s_flags & MS_SYNCHRONOUS))
765 /* hmm, this isn't really async... */
766 ret = ceph_sync_read(filp, base, len, ppos, &checkeof);
767 else
768 ret = generic_file_aio_read(iocb, iov, nr_segs, pos);
769
770out:
771 dout("aio_read %p %llx.%llx dropping cap refs on %s = %d\n",
772 inode, ceph_vinop(inode), ceph_cap_string(got), (int)ret);
773 ceph_put_cap_refs(ci, got);
774
775 if (checkeof && ret >= 0) {
776 int statret = ceph_do_getattr(inode, CEPH_STAT_CAP_SIZE);
777
778 /* hit EOF or hole? */
779 if (statret == 0 && *ppos < inode->i_size) {
780 dout("aio_read sync_read hit hole, reading more\n");
781 read += ret;
782 base += ret;
783 len -= ret;
784 checkeof = 0;
785 goto again;
786 }
787 }
788 if (ret >= 0)
789 ret += read;
790
791 return ret;
792}
793
794/*
795 * Take cap references to avoid releasing caps to MDS mid-write.
796 *
797 * If we are synchronous, and write with an old snap context, the OSD
798 * may return EOLDSNAPC. In that case, retry the write.. _after_
799 * dropping our cap refs and allowing the pending snap to logically
800 * complete _before_ this write occurs.
801 *
802 * If we are near ENOSPC, write synchronously.
803 */
804static ssize_t ceph_aio_write(struct kiocb *iocb, const struct iovec *iov,
805 unsigned long nr_segs, loff_t pos)
806{
807 struct file *file = iocb->ki_filp;
808 struct inode *inode = file->f_dentry->d_inode;
809 struct ceph_inode_info *ci = ceph_inode(inode);
810 struct ceph_osd_client *osdc = &ceph_client(inode->i_sb)->osdc;
811 loff_t endoff = pos + iov->iov_len;
812 int got = 0;
813 int ret, err;
814
815 if (ceph_snap(inode) != CEPH_NOSNAP)
816 return -EROFS;
817
818retry_snap:
819 if (ceph_osdmap_flag(osdc->osdmap, CEPH_OSDMAP_FULL))
820 return -ENOSPC;
821 __ceph_do_pending_vmtruncate(inode);
822 dout("aio_write %p %llx.%llx %llu~%u getting caps. i_size %llu\n",
823 inode, ceph_vinop(inode), pos, (unsigned)iov->iov_len,
824 inode->i_size);
825 ret = ceph_get_caps(ci, CEPH_CAP_FILE_WR, CEPH_CAP_FILE_BUFFER,
826 &got, endoff);
827 if (ret < 0)
828 goto out;
829
830 dout("aio_write %p %llx.%llx %llu~%u got cap refs on %s\n",
831 inode, ceph_vinop(inode), pos, (unsigned)iov->iov_len,
832 ceph_cap_string(got));
833
834 if ((got & CEPH_CAP_FILE_BUFFER) == 0 ||
835 (iocb->ki_filp->f_flags & O_DIRECT) ||
836 (inode->i_sb->s_flags & MS_SYNCHRONOUS)) {
837 ret = ceph_sync_write(file, iov->iov_base, iov->iov_len,
838 &iocb->ki_pos);
839 } else {
840 ret = generic_file_aio_write(iocb, iov, nr_segs, pos);
841
842 if ((ret >= 0 || ret == -EIOCBQUEUED) &&
843 ((file->f_flags & O_SYNC) || IS_SYNC(file->f_mapping->host)
844 || ceph_osdmap_flag(osdc->osdmap, CEPH_OSDMAP_NEARFULL))) {
845 err = vfs_fsync_range(file, file->f_path.dentry,
846 pos, pos + ret - 1, 1);
847 if (err < 0)
848 ret = err;
849 }
850 }
851 if (ret >= 0) {
852 spin_lock(&inode->i_lock);
853 __ceph_mark_dirty_caps(ci, CEPH_CAP_FILE_WR);
854 spin_unlock(&inode->i_lock);
855 }
856
857out:
858 dout("aio_write %p %llx.%llx %llu~%u dropping cap refs on %s\n",
859 inode, ceph_vinop(inode), pos, (unsigned)iov->iov_len,
860 ceph_cap_string(got));
861 ceph_put_cap_refs(ci, got);
862
863 if (ret == -EOLDSNAPC) {
864 dout("aio_write %p %llx.%llx %llu~%u got EOLDSNAPC, retrying\n",
865 inode, ceph_vinop(inode), pos, (unsigned)iov->iov_len);
866 goto retry_snap;
867 }
868
869 return ret;
870}
871
872/*
873 * llseek. be sure to verify file size on SEEK_END.
874 */
875static loff_t ceph_llseek(struct file *file, loff_t offset, int origin)
876{
877 struct inode *inode = file->f_mapping->host;
878 int ret;
879
880 mutex_lock(&inode->i_mutex);
881 __ceph_do_pending_vmtruncate(inode);
882 switch (origin) {
883 case SEEK_END:
884 ret = ceph_do_getattr(inode, CEPH_STAT_CAP_SIZE);
885 if (ret < 0) {
886 offset = ret;
887 goto out;
888 }
889 offset += inode->i_size;
890 break;
891 case SEEK_CUR:
892 /*
893 * Here we special-case the lseek(fd, 0, SEEK_CUR)
894 * position-querying operation. Avoid rewriting the "same"
895 * f_pos value back to the file because a concurrent read(),
896 * write() or lseek() might have altered it
897 */
898 if (offset == 0) {
899 offset = file->f_pos;
900 goto out;
901 }
902 offset += file->f_pos;
903 break;
904 }
905
906 if (offset < 0 || offset > inode->i_sb->s_maxbytes) {
907 offset = -EINVAL;
908 goto out;
909 }
910
911 /* Special lock needed here? */
912 if (offset != file->f_pos) {
913 file->f_pos = offset;
914 file->f_version = 0;
915 }
916
917out:
918 mutex_unlock(&inode->i_mutex);
919 return offset;
920}
921
922const struct file_operations ceph_file_fops = {
923 .open = ceph_open,
924 .release = ceph_release,
925 .llseek = ceph_llseek,
926 .read = do_sync_read,
927 .write = do_sync_write,
928 .aio_read = ceph_aio_read,
929 .aio_write = ceph_aio_write,
930 .mmap = ceph_mmap,
931 .fsync = ceph_fsync,
932 .splice_read = generic_file_splice_read,
933 .splice_write = generic_file_splice_write,
934 .unlocked_ioctl = ceph_ioctl,
935 .compat_ioctl = ceph_ioctl,
936};
937
fs/ceph/inode.c+1750−0
@@ -0,0 +1,1750 @@
1#include "ceph_debug.h"
2
3#include <linux/module.h>
4#include <linux/fs.h>
5#include <linux/smp_lock.h>
6#include <linux/slab.h>
7#include <linux/string.h>
8#include <linux/uaccess.h>
9#include <linux/kernel.h>
10#include <linux/namei.h>
11#include <linux/writeback.h>
12#include <linux/vmalloc.h>
13#include <linux/pagevec.h>
14
15#include "super.h"
16#include "decode.h"
17
18/*
19 * Ceph inode operations
20 *
21 * Implement basic inode helpers (get, alloc) and inode ops (getattr,
22 * setattr, etc.), xattr helpers, and helpers for assimilating
23 * metadata returned by the MDS into our cache.
24 *
25 * Also define helpers for doing asynchronous writeback, invalidation,
26 * and truncation for the benefit of those who can't afford to block
27 * (typically because they are in the message handler path).
28 */
29
30static const struct inode_operations ceph_symlink_iops;
31
32static void ceph_invalidate_work(struct work_struct *work);
33static void ceph_writeback_work(struct work_struct *work);
34static void ceph_vmtruncate_work(struct work_struct *work);
35
36/*
37 * find or create an inode, given the ceph ino number
38 */
39struct inode *ceph_get_inode(struct super_block *sb, struct ceph_vino vino)
40{
41 struct inode *inode;
42 ino_t t = ceph_vino_to_ino(vino);
43
44 inode = iget5_locked(sb, t, ceph_ino_compare, ceph_set_ino_cb, &vino);
45 if (inode == NULL)
46 return ERR_PTR(-ENOMEM);
47 if (inode->i_state & I_NEW) {
48 dout("get_inode created new inode %p %llx.%llx ino %llx\n",
49 inode, ceph_vinop(inode), (u64)inode->i_ino);
50 unlock_new_inode(inode);
51 }
52
53 dout("get_inode on %lu=%llx.%llx got %p\n", inode->i_ino, vino.ino,
54 vino.snap, inode);
55 return inode;
56}
57
58/*
59 * get/constuct snapdir inode for a given directory
60 */
61struct inode *ceph_get_snapdir(struct inode *parent)
62{
63 struct ceph_vino vino = {
64 .ino = ceph_ino(parent),
65 .snap = CEPH_SNAPDIR,
66 };
67 struct inode *inode = ceph_get_inode(parent->i_sb, vino);
68 struct ceph_inode_info *ci = ceph_inode(inode);
69
70 BUG_ON(!S_ISDIR(parent->i_mode));
71 if (IS_ERR(inode))
72 return ERR_PTR(PTR_ERR(inode));
73 inode->i_mode = parent->i_mode;
74 inode->i_uid = parent->i_uid;
75 inode->i_gid = parent->i_gid;
76 inode->i_op = &ceph_dir_iops;
77 inode->i_fop = &ceph_dir_fops;
78 ci->i_snap_caps = CEPH_CAP_PIN; /* so we can open */
79 ci->i_rbytes = 0;
80 return inode;
81}
82
83const struct inode_operations ceph_file_iops = {
84 .permission = ceph_permission,
85 .setattr = ceph_setattr,
86 .getattr = ceph_getattr,
87 .setxattr = ceph_setxattr,
88 .getxattr = ceph_getxattr,
89 .listxattr = ceph_listxattr,
90 .removexattr = ceph_removexattr,
91};
92
93
94/*
95 * We use a 'frag tree' to keep track of the MDS's directory fragments
96 * for a given inode (usually there is just a single fragment). We
97 * need to know when a child frag is delegated to a new MDS, or when
98 * it is flagged as replicated, so we can direct our requests
99 * accordingly.
100 */
101
102/*
103 * find/create a frag in the tree
104 */
105static struct ceph_inode_frag *__get_or_create_frag(struct ceph_inode_info *ci,
106 u32 f)
107{
108 struct rb_node **p;
109 struct rb_node *parent = NULL;
110 struct ceph_inode_frag *frag;
111 int c;
112
113 p = &ci->i_fragtree.rb_node;
114 while (*p) {
115 parent = *p;
116 frag = rb_entry(parent, struct ceph_inode_frag, node);
117 c = ceph_frag_compare(f, frag->frag);
118 if (c < 0)
119 p = &(*p)->rb_left;
120 else if (c > 0)
121 p = &(*p)->rb_right;
122 else
123 return frag;
124 }
125
126 frag = kmalloc(sizeof(*frag), GFP_NOFS);
127 if (!frag) {
128 pr_err("__get_or_create_frag ENOMEM on %p %llx.%llx "
129 "frag %x\n", &ci->vfs_inode,
130 ceph_vinop(&ci->vfs_inode), f);
131 return ERR_PTR(-ENOMEM);
132 }
133 frag->frag = f;
134 frag->split_by = 0;
135 frag->mds = -1;
136 frag->ndist = 0;
137
138 rb_link_node(&frag->node, parent, p);
139 rb_insert_color(&frag->node, &ci->i_fragtree);
140
141 dout("get_or_create_frag added %llx.%llx frag %x\n",
142 ceph_vinop(&ci->vfs_inode), f);
143 return frag;
144}
145
146/*
147 * find a specific frag @f
148 */
149struct ceph_inode_frag *__ceph_find_frag(struct ceph_inode_info *ci, u32 f)
150{
151 struct rb_node *n = ci->i_fragtree.rb_node;
152
153 while (n) {
154 struct ceph_inode_frag *frag =
155 rb_entry(n, struct ceph_inode_frag, node);
156 int c = ceph_frag_compare(f, frag->frag);
157 if (c < 0)
158 n = n->rb_left;
159 else if (c > 0)
160 n = n->rb_right;
161 else
162 return frag;
163 }
164 return NULL;
165}
166
167/*
168 * Choose frag containing the given value @v. If @pfrag is
169 * specified, copy the frag delegation info to the caller if
170 * it is present.
171 */
172u32 ceph_choose_frag(struct ceph_inode_info *ci, u32 v,
173 struct ceph_inode_frag *pfrag,
174 int *found)
175{
176 u32 t = ceph_frag_make(0, 0);
177 struct ceph_inode_frag *frag;
178 unsigned nway, i;
179 u32 n;
180
181 if (found)
182 *found = 0;
183
184 mutex_lock(&ci->i_fragtree_mutex);
185 while (1) {
186 WARN_ON(!ceph_frag_contains_value(t, v));
187 frag = __ceph_find_frag(ci, t);
188 if (!frag)
189 break; /* t is a leaf */
190 if (frag->split_by == 0) {
191 if (pfrag)
192 memcpy(pfrag, frag, sizeof(*pfrag));
193 if (found)
194 *found = 1;
195 break;
196 }
197
198 /* choose child */
199 nway = 1 << frag->split_by;
200 dout("choose_frag(%x) %x splits by %d (%d ways)\n", v, t,
201 frag->split_by, nway);
202 for (i = 0; i < nway; i++) {
203 n = ceph_frag_make_child(t, frag->split_by, i);
204 if (ceph_frag_contains_value(n, v)) {
205 t = n;
206 break;
207 }
208 }
209 BUG_ON(i == nway);
210 }
211 dout("choose_frag(%x) = %x\n", v, t);
212
213 mutex_unlock(&ci->i_fragtree_mutex);
214 return t;
215}
216
217/*
218 * Process dirfrag (delegation) info from the mds. Include leaf
219 * fragment in tree ONLY if ndist > 0. Otherwise, only
220 * branches/splits are included in i_fragtree)
221 */
222static int ceph_fill_dirfrag(struct inode *inode,
223 struct ceph_mds_reply_dirfrag *dirinfo)
224{
225 struct ceph_inode_info *ci = ceph_inode(inode);
226 struct ceph_inode_frag *frag;
227 u32 id = le32_to_cpu(dirinfo->frag);
228 int mds = le32_to_cpu(dirinfo->auth);
229 int ndist = le32_to_cpu(dirinfo->ndist);
230 int i;
231 int err = 0;
232
233 mutex_lock(&ci->i_fragtree_mutex);
234 if (ndist == 0) {
235 /* no delegation info needed. */
236 frag = __ceph_find_frag(ci, id);
237 if (!frag)
238 goto out;
239 if (frag->split_by == 0) {
240 /* tree leaf, remove */
241 dout("fill_dirfrag removed %llx.%llx frag %x"
242 " (no ref)\n", ceph_vinop(inode), id);
243 rb_erase(&frag->node, &ci->i_fragtree);
244 kfree(frag);
245 } else {
246 /* tree branch, keep and clear */
247 dout("fill_dirfrag cleared %llx.%llx frag %x"
248 " referral\n", ceph_vinop(inode), id);
249 frag->mds = -1;
250 frag->ndist = 0;
251 }
252 goto out;
253 }
254
255
256 /* find/add this frag to store mds delegation info */
257 frag = __get_or_create_frag(ci, id);
258 if (IS_ERR(frag)) {
259 /* this is not the end of the world; we can continue
260 with bad/inaccurate delegation info */
261 pr_err("fill_dirfrag ENOMEM on mds ref %llx.%llx fg %x\n",
262 ceph_vinop(inode), le32_to_cpu(dirinfo->frag));
263 err = -ENOMEM;
264 goto out;
265 }
266
267 frag->mds = mds;
268 frag->ndist = min_t(u32, ndist, CEPH_MAX_DIRFRAG_REP);
269 for (i = 0; i < frag->ndist; i++)
270 frag->dist[i] = le32_to_cpu(dirinfo->dist[i]);
271 dout("fill_dirfrag %llx.%llx frag %x ndist=%d\n",
272 ceph_vinop(inode), frag->frag, frag->ndist);
273
274out:
275 mutex_unlock(&ci->i_fragtree_mutex);
276 return err;
277}
278
279
280/*
281 * initialize a newly allocated inode.
282 */
283struct inode *ceph_alloc_inode(struct super_block *sb)
284{
285 struct ceph_inode_info *ci;
286 int i;
287
288 ci = kmem_cache_alloc(ceph_inode_cachep, GFP_NOFS);
289 if (!ci)
290 return NULL;
291
292 dout("alloc_inode %p\n", &ci->vfs_inode);
293
294 ci->i_version = 0;
295 ci->i_time_warp_seq = 0;
296 ci->i_ceph_flags = 0;
297 ci->i_release_count = 0;
298 ci->i_symlink = NULL;
299
300 ci->i_fragtree = RB_ROOT;
301 mutex_init(&ci->i_fragtree_mutex);
302
303 ci->i_xattrs.blob = NULL;
304 ci->i_xattrs.prealloc_blob = NULL;
305 ci->i_xattrs.dirty = false;
306 ci->i_xattrs.index = RB_ROOT;
307 ci->i_xattrs.count = 0;
308 ci->i_xattrs.names_size = 0;
309 ci->i_xattrs.vals_size = 0;
310 ci->i_xattrs.version = 0;
311 ci->i_xattrs.index_version = 0;
312
313 ci->i_caps = RB_ROOT;
314 ci->i_auth_cap = NULL;
315 ci->i_dirty_caps = 0;
316 ci->i_flushing_caps = 0;
317 INIT_LIST_HEAD(&ci->i_dirty_item);
318 INIT_LIST_HEAD(&ci->i_flushing_item);
319 ci->i_cap_flush_seq = 0;
320 ci->i_cap_flush_last_tid = 0;
321 memset(&ci->i_cap_flush_tid, 0, sizeof(ci->i_cap_flush_tid));
322 init_waitqueue_head(&ci->i_cap_wq);
323 ci->i_hold_caps_min = 0;
324 ci->i_hold_caps_max = 0;
325 INIT_LIST_HEAD(&ci->i_cap_delay_list);
326 ci->i_cap_exporting_mds = 0;
327 ci->i_cap_exporting_mseq = 0;
328 ci->i_cap_exporting_issued = 0;
329 INIT_LIST_HEAD(&ci->i_cap_snaps);
330 ci->i_head_snapc = NULL;
331 ci->i_snap_caps = 0;
332
333 for (i = 0; i < CEPH_FILE_MODE_NUM; i++)
334 ci->i_nr_by_mode[i] = 0;
335
336 ci->i_truncate_seq = 0;
337 ci->i_truncate_size = 0;
338 ci->i_truncate_pending = 0;
339
340 ci->i_max_size = 0;
341 ci->i_reported_size = 0;
342 ci->i_wanted_max_size = 0;
343 ci->i_requested_max_size = 0;
344
345 ci->i_pin_ref = 0;
346 ci->i_rd_ref = 0;
347 ci->i_rdcache_ref = 0;
348 ci->i_wr_ref = 0;
349 ci->i_wrbuffer_ref = 0;
350 ci->i_wrbuffer_ref_head = 0;
351 ci->i_shared_gen = 0;
352 ci->i_rdcache_gen = 0;
353 ci->i_rdcache_revoking = 0;
354
355 INIT_LIST_HEAD(&ci->i_unsafe_writes);
356 INIT_LIST_HEAD(&ci->i_unsafe_dirops);
357 spin_lock_init(&ci->i_unsafe_lock);
358
359 ci->i_snap_realm = NULL;
360 INIT_LIST_HEAD(&ci->i_snap_realm_item);
361 INIT_LIST_HEAD(&ci->i_snap_flush_item);
362
363 INIT_WORK(&ci->i_wb_work, ceph_writeback_work);
364 INIT_WORK(&ci->i_pg_inv_work, ceph_invalidate_work);
365
366 INIT_WORK(&ci->i_vmtruncate_work, ceph_vmtruncate_work);
367
368 return &ci->vfs_inode;
369}
370
371void ceph_destroy_inode(struct inode *inode)
372{
373 struct ceph_inode_info *ci = ceph_inode(inode);
374 struct ceph_inode_frag *frag;
375 struct rb_node *n;
376
377 dout("destroy_inode %p ino %llx.%llx\n", inode, ceph_vinop(inode));
378
379 ceph_queue_caps_release(inode);
380
381 kfree(ci->i_symlink);
382 while ((n = rb_first(&ci->i_fragtree)) != NULL) {
383 frag = rb_entry(n, struct ceph_inode_frag, node);
384 rb_erase(n, &ci->i_fragtree);
385 kfree(frag);
386 }
387
388 __ceph_destroy_xattrs(ci);
389 if (ci->i_xattrs.blob)
390 ceph_buffer_put(ci->i_xattrs.blob);
391 if (ci->i_xattrs.prealloc_blob)
392 ceph_buffer_put(ci->i_xattrs.prealloc_blob);
393
394 kmem_cache_free(ceph_inode_cachep, ci);
395}
396
397
398/*
399 * Helpers to fill in size, ctime, mtime, and atime. We have to be
400 * careful because either the client or MDS may have more up to date
401 * info, depending on which capabilities are held, and whether
402 * time_warp_seq or truncate_seq have increased. (Ordinarily, mtime
403 * and size are monotonically increasing, except when utimes() or
404 * truncate() increments the corresponding _seq values.)
405 */
406int ceph_fill_file_size(struct inode *inode, int issued,
407 u32 truncate_seq, u64 truncate_size, u64 size)
408{
409 struct ceph_inode_info *ci = ceph_inode(inode);
410 int queue_trunc = 0;
411
412 if (ceph_seq_cmp(truncate_seq, ci->i_truncate_seq) > 0 ||
413 (truncate_seq == ci->i_truncate_seq && size > inode->i_size)) {
414 dout("size %lld -> %llu\n", inode->i_size, size);
415 inode->i_size = size;
416 inode->i_blocks = (size + (1<<9) - 1) >> 9;
417 ci->i_reported_size = size;
418 if (truncate_seq != ci->i_truncate_seq) {
419 dout("truncate_seq %u -> %u\n",
420 ci->i_truncate_seq, truncate_seq);
421 ci->i_truncate_seq = truncate_seq;
422 /*
423 * If we hold relevant caps, or in the case where we're
424 * not the only client referencing this file and we
425 * don't hold those caps, then we need to check whether
426 * the file is either opened or mmaped
427 */
428 if ((issued & (CEPH_CAP_FILE_CACHE|CEPH_CAP_FILE_RD|
429 CEPH_CAP_FILE_WR|CEPH_CAP_FILE_BUFFER|
430 CEPH_CAP_FILE_EXCL)) ||
431 mapping_mapped(inode->i_mapping) ||
432 __ceph_caps_file_wanted(ci)) {
433 ci->i_truncate_pending++;
434 queue_trunc = 1;
435 }
436 }
437 }
438 if (ceph_seq_cmp(truncate_seq, ci->i_truncate_seq) >= 0 &&
439 ci->i_truncate_size != truncate_size) {
440 dout("truncate_size %lld -> %llu\n", ci->i_truncate_size,
441 truncate_size);
442 ci->i_truncate_size = truncate_size;
443 }
444 return queue_trunc;
445}
446
447void ceph_fill_file_time(struct inode *inode, int issued,
448 u64 time_warp_seq, struct timespec *ctime,
449 struct timespec *mtime, struct timespec *atime)
450{
451 struct ceph_inode_info *ci = ceph_inode(inode);
452 int warn = 0;
453
454 if (issued & (CEPH_CAP_FILE_EXCL|
455 CEPH_CAP_FILE_WR|
456 CEPH_CAP_FILE_BUFFER)) {
457 if (timespec_compare(ctime, &inode->i_ctime) > 0) {
458 dout("ctime %ld.%09ld -> %ld.%09ld inc w/ cap\n",
459 inode->i_ctime.tv_sec, inode->i_ctime.tv_nsec,
460 ctime->tv_sec, ctime->tv_nsec);
461 inode->i_ctime = *ctime;
462 }
463 if (ceph_seq_cmp(time_warp_seq, ci->i_time_warp_seq) > 0) {
464 /* the MDS did a utimes() */
465 dout("mtime %ld.%09ld -> %ld.%09ld "
466 "tw %d -> %d\n",
467 inode->i_mtime.tv_sec, inode->i_mtime.tv_nsec,
468 mtime->tv_sec, mtime->tv_nsec,
469 ci->i_time_warp_seq, (int)time_warp_seq);
470
471 inode->i_mtime = *mtime;
472 inode->i_atime = *atime;
473 ci->i_time_warp_seq = time_warp_seq;
474 } else if (time_warp_seq == ci->i_time_warp_seq) {
475 /* nobody did utimes(); take the max */
476 if (timespec_compare(mtime, &inode->i_mtime) > 0) {
477 dout("mtime %ld.%09ld -> %ld.%09ld inc\n",
478 inode->i_mtime.tv_sec,
479 inode->i_mtime.tv_nsec,
480 mtime->tv_sec, mtime->tv_nsec);
481 inode->i_mtime = *mtime;
482 }
483 if (timespec_compare(atime, &inode->i_atime) > 0) {
484 dout("atime %ld.%09ld -> %ld.%09ld inc\n",
485 inode->i_atime.tv_sec,
486 inode->i_atime.tv_nsec,
487 atime->tv_sec, atime->tv_nsec);
488 inode->i_atime = *atime;
489 }
490 } else if (issued & CEPH_CAP_FILE_EXCL) {
491 /* we did a utimes(); ignore mds values */
492 } else {
493 warn = 1;
494 }
495 } else {
496 /* we have no write caps; whatever the MDS says is true */
497 if (ceph_seq_cmp(time_warp_seq, ci->i_time_warp_seq) >= 0) {
498 inode->i_ctime = *ctime;
499 inode->i_mtime = *mtime;
500 inode->i_atime = *atime;
501 ci->i_time_warp_seq = time_warp_seq;
502 } else {
503 warn = 1;
504 }
505 }
506 if (warn) /* time_warp_seq shouldn't go backwards */
507 dout("%p mds time_warp_seq %llu < %u\n",
508 inode, time_warp_seq, ci->i_time_warp_seq);
509}
510
511/*
512 * Populate an inode based on info from mds. May be called on new or
513 * existing inodes.
514 */
515static int fill_inode(struct inode *inode,
516 struct ceph_mds_reply_info_in *iinfo,
517 struct ceph_mds_reply_dirfrag *dirinfo,
518 struct ceph_mds_session *session,
519 unsigned long ttl_from, int cap_fmode,
520 struct ceph_cap_reservation *caps_reservation)
521{
522 struct ceph_mds_reply_inode *info = iinfo->in;
523 struct ceph_inode_info *ci = ceph_inode(inode);
524 int i;
525 int issued, implemented;
526 struct timespec mtime, atime, ctime;
527 u32 nsplits;
528 struct ceph_buffer *xattr_blob = NULL;
529 int err = 0;
530 int queue_trunc = 0;
531
532 dout("fill_inode %p ino %llx.%llx v %llu had %llu\n",
533 inode, ceph_vinop(inode), le64_to_cpu(info->version),
534 ci->i_version);
535
536 /*
537 * prealloc xattr data, if it looks like we'll need it. only
538 * if len > 4 (meaning there are actually xattrs; the first 4
539 * bytes are the xattr count).
540 */
541 if (iinfo->xattr_len > 4) {
542 xattr_blob = ceph_buffer_new(iinfo->xattr_len, GFP_NOFS);
543 if (!xattr_blob)
544 pr_err("fill_inode ENOMEM xattr blob %d bytes\n",
545 iinfo->xattr_len);
546 }
547
548 spin_lock(&inode->i_lock);
549
550 /*
551 * provided version will be odd if inode value is projected,
552 * even if stable. skip the update if we have a newer info
553 * (e.g., due to inode info racing form multiple MDSs), or if
554 * we are getting projected (unstable) inode info.
555 */
556 if (le64_to_cpu(info->version) > 0 &&
557 (ci->i_version & ~1) > le64_to_cpu(info->version))
558 goto no_change;
559
560 issued = __ceph_caps_issued(ci, &implemented);
561 issued |= implemented | __ceph_caps_dirty(ci);
562
563 /* update inode */
564 ci->i_version = le64_to_cpu(info->version);
565 inode->i_version++;
566 inode->i_rdev = le32_to_cpu(info->rdev);
567
568 if ((issued & CEPH_CAP_AUTH_EXCL) == 0) {
569 inode->i_mode = le32_to_cpu(info->mode);
570 inode->i_uid = le32_to_cpu(info->uid);
571 inode->i_gid = le32_to_cpu(info->gid);
572 dout("%p mode 0%o uid.gid %d.%d\n", inode, inode->i_mode,
573 inode->i_uid, inode->i_gid);
574 }
575
576 if ((issued & CEPH_CAP_LINK_EXCL) == 0)
577 inode->i_nlink = le32_to_cpu(info->nlink);
578
579 /* be careful with mtime, atime, size */
580 ceph_decode_timespec(&atime, &info->atime);
581 ceph_decode_timespec(&mtime, &info->mtime);
582 ceph_decode_timespec(&ctime, &info->ctime);
583 queue_trunc = ceph_fill_file_size(inode, issued,
584 le32_to_cpu(info->truncate_seq),
585 le64_to_cpu(info->truncate_size),
586 le64_to_cpu(info->size));
587 ceph_fill_file_time(inode, issued,
588 le32_to_cpu(info->time_warp_seq),
589 &ctime, &mtime, &atime);
590
591 ci->i_max_size = le64_to_cpu(info->max_size);
592 ci->i_layout = info->layout;
593 inode->i_blkbits = fls(le32_to_cpu(info->layout.fl_stripe_unit)) - 1;
594
595 /* xattrs */
596 /* note that if i_xattrs.len <= 4, i_xattrs.data will still be NULL. */
597 if ((issued & CEPH_CAP_XATTR_EXCL) == 0 &&
598 le64_to_cpu(info->xattr_version) > ci->i_xattrs.version) {
599 if (ci->i_xattrs.blob)
600 ceph_buffer_put(ci->i_xattrs.blob);
601 ci->i_xattrs.blob = xattr_blob;
602 if (xattr_blob)
603 memcpy(ci->i_xattrs.blob->vec.iov_base,
604 iinfo->xattr_data, iinfo->xattr_len);
605 ci->i_xattrs.version = le64_to_cpu(info->xattr_version);
606 }
607
608 inode->i_mapping->a_ops = &ceph_aops;
609 inode->i_mapping->backing_dev_info =
610 &ceph_client(inode->i_sb)->backing_dev_info;
611
612 switch (inode->i_mode & S_IFMT) {
613 case S_IFIFO:
614 case S_IFBLK:
615 case S_IFCHR:
616 case S_IFSOCK:
617 init_special_inode(inode, inode->i_mode, inode->i_rdev);
618 inode->i_op = &ceph_file_iops;
619 break;
620 case S_IFREG:
621 inode->i_op = &ceph_file_iops;
622 inode->i_fop = &ceph_file_fops;
623 break;
624 case S_IFLNK:
625 inode->i_op = &ceph_symlink_iops;
626 if (!ci->i_symlink) {
627 int symlen = iinfo->symlink_len;
628 char *sym;
629
630 BUG_ON(symlen != inode->i_size);
631 spin_unlock(&inode->i_lock);
632
633 err = -ENOMEM;
634 sym = kmalloc(symlen+1, GFP_NOFS);
635 if (!sym)
636 goto out;
637 memcpy(sym, iinfo->symlink, symlen);
638 sym[symlen] = 0;
639
640 spin_lock(&inode->i_lock);
641 if (!ci->i_symlink)
642 ci->i_symlink = sym;
643 else
644 kfree(sym); /* lost a race */
645 }
646 break;
647 case S_IFDIR:
648 inode->i_op = &ceph_dir_iops;
649 inode->i_fop = &ceph_dir_fops;
650
651 ci->i_files = le64_to_cpu(info->files);
652 ci->i_subdirs = le64_to_cpu(info->subdirs);
653 ci->i_rbytes = le64_to_cpu(info->rbytes);
654 ci->i_rfiles = le64_to_cpu(info->rfiles);
655 ci->i_rsubdirs = le64_to_cpu(info->rsubdirs);
656 ceph_decode_timespec(&ci->i_rctime, &info->rctime);
657
658 /* set dir completion flag? */
659 if (ci->i_files == 0 && ci->i_subdirs == 0 &&
660 ceph_snap(inode) == CEPH_NOSNAP &&
661 (le32_to_cpu(info->cap.caps) & CEPH_CAP_FILE_SHARED)) {
662 dout(" marking %p complete (empty)\n", inode);
663 ci->i_ceph_flags |= CEPH_I_COMPLETE;
664 ci->i_max_offset = 2;
665 }
666
667 /* it may be better to set st_size in getattr instead? */
668 if (ceph_test_opt(ceph_client(inode->i_sb), RBYTES))
669 inode->i_size = ci->i_rbytes;
670 break;
671 default:
672 pr_err("fill_inode %llx.%llx BAD mode 0%o\n",
673 ceph_vinop(inode), inode->i_mode);
674 }
675
676no_change:
677 spin_unlock(&inode->i_lock);
678
679 /* queue truncate if we saw i_size decrease */
680 if (queue_trunc)
681 ceph_queue_vmtruncate(inode);
682
683 /* populate frag tree */
684 /* FIXME: move me up, if/when version reflects fragtree changes */
685 nsplits = le32_to_cpu(info->fragtree.nsplits);
686 mutex_lock(&ci->i_fragtree_mutex);
687 for (i = 0; i < nsplits; i++) {
688 u32 id = le32_to_cpu(info->fragtree.splits[i].frag);
689 struct ceph_inode_frag *frag = __get_or_create_frag(ci, id);
690
691 if (IS_ERR(frag))
692 continue;
693 frag->split_by = le32_to_cpu(info->fragtree.splits[i].by);
694 dout(" frag %x split by %d\n", frag->frag, frag->split_by);
695 }
696 mutex_unlock(&ci->i_fragtree_mutex);
697
698 /* were we issued a capability? */
699 if (info->cap.caps) {
700 if (ceph_snap(inode) == CEPH_NOSNAP) {
701 ceph_add_cap(inode, session,
702 le64_to_cpu(info->cap.cap_id),
703 cap_fmode,
704 le32_to_cpu(info->cap.caps),
705 le32_to_cpu(info->cap.wanted),
706 le32_to_cpu(info->cap.seq),
707 le32_to_cpu(info->cap.mseq),
708 le64_to_cpu(info->cap.realm),
709 info->cap.flags,
710 caps_reservation);
711 } else {
712 spin_lock(&inode->i_lock);
713 dout(" %p got snap_caps %s\n", inode,
714 ceph_cap_string(le32_to_cpu(info->cap.caps)));
715 ci->i_snap_caps |= le32_to_cpu(info->cap.caps);
716 if (cap_fmode >= 0)
717 __ceph_get_fmode(ci, cap_fmode);
718 spin_unlock(&inode->i_lock);
719 }
720 }
721
722 /* update delegation info? */
723 if (dirinfo)
724 ceph_fill_dirfrag(inode, dirinfo);
725
726 err = 0;
727
728out:
729 if (xattr_blob)
730 ceph_buffer_put(xattr_blob);
731 return err;
732}
733
734/*
735 * caller should hold session s_mutex.
736 */
737static void update_dentry_lease(struct dentry *dentry,
738 struct ceph_mds_reply_lease *lease,
739 struct ceph_mds_session *session,
740 unsigned long from_time)
741{
742 struct ceph_dentry_info *di = ceph_dentry(dentry);
743 long unsigned duration = le32_to_cpu(lease->duration_ms);
744 long unsigned ttl = from_time + (duration * HZ) / 1000;
745 long unsigned half_ttl = from_time + (duration * HZ / 2) / 1000;
746 struct inode *dir;
747
748 /* only track leases on regular dentries */
749 if (dentry->d_op != &ceph_dentry_ops)
750 return;
751
752 spin_lock(&dentry->d_lock);
753 dout("update_dentry_lease %p mask %d duration %lu ms ttl %lu\n",
754 dentry, le16_to_cpu(lease->mask), duration, ttl);
755
756 /* make lease_rdcache_gen match directory */
757 dir = dentry->d_parent->d_inode;
758 di->lease_shared_gen = ceph_inode(dir)->i_shared_gen;
759
760 if (lease->mask == 0)
761 goto out_unlock;
762
763 if (di->lease_gen == session->s_cap_gen &&
764 time_before(ttl, dentry->d_time))
765 goto out_unlock; /* we already have a newer lease. */
766
767 if (di->lease_session && di->lease_session != session)
768 goto out_unlock;
769
770 ceph_dentry_lru_touch(dentry);
771
772 if (!di->lease_session)
773 di->lease_session = ceph_get_mds_session(session);
774 di->lease_gen = session->s_cap_gen;
775 di->lease_seq = le32_to_cpu(lease->seq);
776 di->lease_renew_after = half_ttl;
777 di->lease_renew_from = 0;
778 dentry->d_time = ttl;
779out_unlock:
780 spin_unlock(&dentry->d_lock);
781 return;
782}
783
784/*
785 * splice a dentry to an inode.
786 * caller must hold directory i_mutex for this to be safe.
787 *
788 * we will only rehash the resulting dentry if @prehash is
789 * true; @prehash will be set to false (for the benefit of
790 * the caller) if we fail.
791 */
792static struct dentry *splice_dentry(struct dentry *dn, struct inode *in,
793 bool *prehash)
794{
795 struct dentry *realdn;
796
797 /* dn must be unhashed */
798 if (!d_unhashed(dn))
799 d_drop(dn);
800 realdn = d_materialise_unique(dn, in);
801 if (IS_ERR(realdn)) {
802 pr_err("splice_dentry error %p inode %p ino %llx.%llx\n",
803 dn, in, ceph_vinop(in));
804 if (prehash)
805 *prehash = false; /* don't rehash on error */
806 dn = realdn; /* note realdn contains the error */
807 goto out;
808 } else if (realdn) {
809 dout("dn %p (%d) spliced with %p (%d) "
810 "inode %p ino %llx.%llx\n",
811 dn, atomic_read(&dn->d_count),
812 realdn, atomic_read(&realdn->d_count),
813 realdn->d_inode, ceph_vinop(realdn->d_inode));
814 dput(dn);
815 dn = realdn;
816 } else {
817 BUG_ON(!ceph_dentry(dn));
818
819 dout("dn %p attached to %p ino %llx.%llx\n",
820 dn, dn->d_inode, ceph_vinop(dn->d_inode));
821 }
822 if ((!prehash || *prehash) && d_unhashed(dn))
823 d_rehash(dn);
824out:
825 return dn;
826}
827
828/*
829 * Set dentry's directory position based on the current dir's max, and
830 * order it in d_subdirs, so that dcache_readdir behaves.
831 */
832static void ceph_set_dentry_offset(struct dentry *dn)
833{
834 struct dentry *dir = dn->d_parent;
835 struct inode *inode = dn->d_parent->d_inode;
836 struct ceph_dentry_info *di;
837
838 BUG_ON(!inode);
839
840 di = ceph_dentry(dn);
841
842 spin_lock(&inode->i_lock);
843 di->offset = ceph_inode(inode)->i_max_offset++;
844 spin_unlock(&inode->i_lock);
845
846 spin_lock(&dcache_lock);
847 spin_lock(&dn->d_lock);
848 list_move_tail(&dir->d_subdirs, &dn->d_u.d_child);
849 dout("set_dentry_offset %p %lld (%p %p)\n", dn, di->offset,
850 dn->d_u.d_child.prev, dn->d_u.d_child.next);
851 spin_unlock(&dn->d_lock);
852 spin_unlock(&dcache_lock);
853}
854
855/*
856 * Incorporate results into the local cache. This is either just
857 * one inode, or a directory, dentry, and possibly linked-to inode (e.g.,
858 * after a lookup).
859 *
860 * A reply may contain
861 * a directory inode along with a dentry.
862 * and/or a target inode
863 *
864 * Called with snap_rwsem (read).
865 */
866int ceph_fill_trace(struct super_block *sb, struct ceph_mds_request *req,
867 struct ceph_mds_session *session)
868{
869 struct ceph_mds_reply_info_parsed *rinfo = &req->r_reply_info;
870 struct inode *in = NULL;
871 struct ceph_mds_reply_inode *ininfo;
872 struct ceph_vino vino;
873 int i = 0;
874 int err = 0;
875
876 dout("fill_trace %p is_dentry %d is_target %d\n", req,
877 rinfo->head->is_dentry, rinfo->head->is_target);
878
879#if 0
880 /*
881 * Debugging hook:
882 *
883 * If we resend completed ops to a recovering mds, we get no
884 * trace. Since that is very rare, pretend this is the case
885 * to ensure the 'no trace' handlers in the callers behave.
886 *
887 * Fill in inodes unconditionally to avoid breaking cap
888 * invariants.
889 */
890 if (rinfo->head->op & CEPH_MDS_OP_WRITE) {
891 pr_info("fill_trace faking empty trace on %lld %s\n",
892 req->r_tid, ceph_mds_op_name(rinfo->head->op));
893 if (rinfo->head->is_dentry) {
894 rinfo->head->is_dentry = 0;
895 err = fill_inode(req->r_locked_dir,
896 &rinfo->diri, rinfo->dirfrag,
897 session, req->r_request_started, -1);
898 }
899 if (rinfo->head->is_target) {
900 rinfo->head->is_target = 0;
901 ininfo = rinfo->targeti.in;
902 vino.ino = le64_to_cpu(ininfo->ino);
903 vino.snap = le64_to_cpu(ininfo->snapid);
904 in = ceph_get_inode(sb, vino);
905 err = fill_inode(in, &rinfo->targeti, NULL,
906 session, req->r_request_started,
907 req->r_fmode);
908 iput(in);
909 }
910 }
911#endif
912
913 if (!rinfo->head->is_target && !rinfo->head->is_dentry) {
914 dout("fill_trace reply is empty!\n");
915 if (rinfo->head->result == 0 && req->r_locked_dir) {
916 struct ceph_inode_info *ci =
917 ceph_inode(req->r_locked_dir);
918 dout(" clearing %p complete (empty trace)\n",
919 req->r_locked_dir);
920 ci->i_ceph_flags &= ~CEPH_I_COMPLETE;
921 ci->i_release_count++;
922 }
923 return 0;
924 }
925
926 if (rinfo->head->is_dentry) {
927 struct inode *dir = req->r_locked_dir;
928
929 err = fill_inode(dir, &rinfo->diri, rinfo->dirfrag,
930 session, req->r_request_started, -1,
931 &req->r_caps_reservation);
932 if (err < 0)
933 return err;
934 }
935
936 if (rinfo->head->is_dentry && !req->r_aborted) {
937 /*
938 * lookup link rename : null -> possibly existing inode
939 * mknod symlink mkdir : null -> new inode
940 * unlink : linked -> null
941 */
942 struct inode *dir = req->r_locked_dir;
943 struct dentry *dn = req->r_dentry;
944 bool have_dir_cap, have_lease;
945
946 BUG_ON(!dn);
947 BUG_ON(!dir);
948 BUG_ON(dn->d_parent->d_inode != dir);
949 BUG_ON(ceph_ino(dir) !=
950 le64_to_cpu(rinfo->diri.in->ino));
951 BUG_ON(ceph_snap(dir) !=
952 le64_to_cpu(rinfo->diri.in->snapid));
953
954 /* do we have a lease on the whole dir? */
955 have_dir_cap =
956 (le32_to_cpu(rinfo->diri.in->cap.caps) &
957 CEPH_CAP_FILE_SHARED);
958
959 /* do we have a dn lease? */
960 have_lease = have_dir_cap ||
961 (le16_to_cpu(rinfo->dlease->mask) &
962 CEPH_LOCK_DN);
963
964 if (!have_lease)
965 dout("fill_trace no dentry lease or dir cap\n");
966
967 /* rename? */
968 if (req->r_old_dentry && req->r_op == CEPH_MDS_OP_RENAME) {
969 dout(" src %p '%.*s' dst %p '%.*s'\n",
970 req->r_old_dentry,
971 req->r_old_dentry->d_name.len,
972 req->r_old_dentry->d_name.name,
973 dn, dn->d_name.len, dn->d_name.name);
974 dout("fill_trace doing d_move %p -> %p\n",
975 req->r_old_dentry, dn);
976 d_move(req->r_old_dentry, dn);
977 dout(" src %p '%.*s' dst %p '%.*s'\n",
978 req->r_old_dentry,
979 req->r_old_dentry->d_name.len,
980 req->r_old_dentry->d_name.name,
981 dn, dn->d_name.len, dn->d_name.name);
982 /* ensure target dentry is invalidated, despite
983 rehashing bug in vfs_rename_dir */
984 dn->d_time = jiffies;
985 ceph_dentry(dn)->lease_shared_gen = 0;
986 /* take overwritten dentry's readdir offset */
987 ceph_dentry(req->r_old_dentry)->offset =
988 ceph_dentry(dn)->offset;
989 dn = req->r_old_dentry; /* use old_dentry */
990 in = dn->d_inode;
991 }
992
993 /* null dentry? */
994 if (!rinfo->head->is_target) {
995 dout("fill_trace null dentry\n");
996 if (dn->d_inode) {
997 dout("d_delete %p\n", dn);
998 d_delete(dn);
999 } else {
1000 dout("d_instantiate %p NULL\n", dn);
1001 d_instantiate(dn, NULL);
1002 if (have_lease && d_unhashed(dn))
1003 d_rehash(dn);
1004 update_dentry_lease(dn, rinfo->dlease,
1005 session,
1006 req->r_request_started);
1007 }
1008 goto done;
1009 }
1010
1011 /* attach proper inode */
1012 ininfo = rinfo->targeti.in;
1013 vino.ino = le64_to_cpu(ininfo->ino);
1014 vino.snap = le64_to_cpu(ininfo->snapid);
1015 if (!dn->d_inode) {
1016 in = ceph_get_inode(sb, vino);
1017 if (IS_ERR(in)) {
1018 pr_err("fill_trace bad get_inode "
1019 "%llx.%llx\n", vino.ino, vino.snap);
1020 err = PTR_ERR(in);
1021 d_delete(dn);
1022 goto done;
1023 }
1024 dn = splice_dentry(dn, in, &have_lease);
1025 if (IS_ERR(dn)) {
1026 err = PTR_ERR(dn);
1027 goto done;
1028 }
1029 req->r_dentry = dn; /* may have spliced */
1030 ceph_set_dentry_offset(dn);
1031 igrab(in);
1032 } else if (ceph_ino(in) == vino.ino &&
1033 ceph_snap(in) == vino.snap) {
1034 igrab(in);
1035 } else {
1036 dout(" %p links to %p %llx.%llx, not %llx.%llx\n",
1037 dn, in, ceph_ino(in), ceph_snap(in),
1038 vino.ino, vino.snap);
1039 have_lease = false;
1040 in = NULL;
1041 }
1042
1043 if (have_lease)
1044 update_dentry_lease(dn, rinfo->dlease, session,
1045 req->r_request_started);
1046 dout(" final dn %p\n", dn);
1047 i++;
1048 } else if (req->r_op == CEPH_MDS_OP_LOOKUPSNAP ||
1049 req->r_op == CEPH_MDS_OP_MKSNAP) {
1050 struct dentry *dn = req->r_dentry;
1051
1052 /* fill out a snapdir LOOKUPSNAP dentry */
1053 BUG_ON(!dn);
1054 BUG_ON(!req->r_locked_dir);
1055 BUG_ON(ceph_snap(req->r_locked_dir) != CEPH_SNAPDIR);
1056 ininfo = rinfo->targeti.in;
1057 vino.ino = le64_to_cpu(ininfo->ino);
1058 vino.snap = le64_to_cpu(ininfo->snapid);
1059 in = ceph_get_inode(sb, vino);
1060 if (IS_ERR(in)) {
1061 pr_err("fill_inode get_inode badness %llx.%llx\n",
1062 vino.ino, vino.snap);
1063 err = PTR_ERR(in);
1064 d_delete(dn);
1065 goto done;
1066 }
1067 dout(" linking snapped dir %p to dn %p\n", in, dn);
1068 dn = splice_dentry(dn, in, NULL);
1069 if (IS_ERR(dn)) {
1070 err = PTR_ERR(dn);
1071 goto done;
1072 }
1073 ceph_set_dentry_offset(dn);
1074 req->r_dentry = dn; /* may have spliced */
1075 igrab(in);
1076 rinfo->head->is_dentry = 1; /* fool notrace handlers */
1077 }
1078
1079 if (rinfo->head->is_target) {
1080 vino.ino = le64_to_cpu(rinfo->targeti.in->ino);
1081 vino.snap = le64_to_cpu(rinfo->targeti.in->snapid);
1082
1083 if (in == NULL || ceph_ino(in) != vino.ino ||
1084 ceph_snap(in) != vino.snap) {
1085 in = ceph_get_inode(sb, vino);
1086 if (IS_ERR(in)) {
1087 err = PTR_ERR(in);
1088 goto done;
1089 }
1090 }
1091 req->r_target_inode = in;
1092
1093 err = fill_inode(in,
1094 &rinfo->targeti, NULL,
1095 session, req->r_request_started,
1096 (le32_to_cpu(rinfo->head->result) == 0) ?
1097 req->r_fmode : -1,
1098 &req->r_caps_reservation);
1099 if (err < 0) {
1100 pr_err("fill_inode badness %p %llx.%llx\n",
1101 in, ceph_vinop(in));
1102 goto done;
1103 }
1104 }
1105
1106done:
1107 dout("fill_trace done err=%d\n", err);
1108 return err;
1109}
1110
1111/*
1112 * Prepopulate our cache with readdir results, leases, etc.
1113 */
1114int ceph_readdir_prepopulate(struct ceph_mds_request *req,
1115 struct ceph_mds_session *session)
1116{
1117 struct dentry *parent = req->r_dentry;
1118 struct ceph_mds_reply_info_parsed *rinfo = &req->r_reply_info;
1119 struct qstr dname;
1120 struct dentry *dn;
1121 struct inode *in;
1122 int err = 0, i;
1123 struct inode *snapdir = NULL;
1124 struct ceph_mds_request_head *rhead = req->r_request->front.iov_base;
1125 u64 frag = le32_to_cpu(rhead->args.readdir.frag);
1126 struct ceph_dentry_info *di;
1127
1128 if (le32_to_cpu(rinfo->head->op) == CEPH_MDS_OP_LSSNAP) {
1129 snapdir = ceph_get_snapdir(parent->d_inode);
1130 parent = d_find_alias(snapdir);
1131 dout("readdir_prepopulate %d items under SNAPDIR dn %p\n",
1132 rinfo->dir_nr, parent);
1133 } else {
1134 dout("readdir_prepopulate %d items under dn %p\n",
1135 rinfo->dir_nr, parent);
1136 if (rinfo->dir_dir)
1137 ceph_fill_dirfrag(parent->d_inode, rinfo->dir_dir);
1138 }
1139
1140 for (i = 0; i < rinfo->dir_nr; i++) {
1141 struct ceph_vino vino;
1142
1143 dname.name = rinfo->dir_dname[i];
1144 dname.len = rinfo->dir_dname_len[i];
1145 dname.hash = full_name_hash(dname.name, dname.len);
1146
1147 vino.ino = le64_to_cpu(rinfo->dir_in[i].in->ino);
1148 vino.snap = le64_to_cpu(rinfo->dir_in[i].in->snapid);
1149
1150retry_lookup:
1151 dn = d_lookup(parent, &dname);
1152 dout("d_lookup on parent=%p name=%.*s got %p\n",
1153 parent, dname.len, dname.name, dn);
1154
1155 if (!dn) {
1156 dn = d_alloc(parent, &dname);
1157 dout("d_alloc %p '%.*s' = %p\n", parent,
1158 dname.len, dname.name, dn);
1159 if (dn == NULL) {
1160 dout("d_alloc badness\n");
1161 err = -ENOMEM;
1162 goto out;
1163 }
1164 err = ceph_init_dentry(dn);
1165 if (err < 0)
1166 goto out;
1167 } else if (dn->d_inode &&
1168 (ceph_ino(dn->d_inode) != vino.ino ||
1169 ceph_snap(dn->d_inode) != vino.snap)) {
1170 dout(" dn %p points to wrong inode %p\n",
1171 dn, dn->d_inode);
1172 d_delete(dn);
1173 dput(dn);
1174 goto retry_lookup;
1175 } else {
1176 /* reorder parent's d_subdirs */
1177 spin_lock(&dcache_lock);
1178 spin_lock(&dn->d_lock);
1179 list_move(&dn->d_u.d_child, &parent->d_subdirs);
1180 spin_unlock(&dn->d_lock);
1181 spin_unlock(&dcache_lock);
1182 }
1183
1184 di = dn->d_fsdata;
1185 di->offset = ceph_make_fpos(frag, i + req->r_readdir_offset);
1186
1187 /* inode */
1188 if (dn->d_inode) {
1189 in = dn->d_inode;
1190 } else {
1191 in = ceph_get_inode(parent->d_sb, vino);
1192 if (in == NULL) {
1193 dout("new_inode badness\n");
1194 d_delete(dn);
1195 dput(dn);
1196 err = -ENOMEM;
1197 goto out;
1198 }
1199 dn = splice_dentry(dn, in, NULL);
1200 }
1201
1202 if (fill_inode(in, &rinfo->dir_in[i], NULL, session,
1203 req->r_request_started, -1,
1204 &req->r_caps_reservation) < 0) {
1205 pr_err("fill_inode badness on %p\n", in);
1206 dput(dn);
1207 continue;
1208 }
1209 update_dentry_lease(dn, rinfo->dir_dlease[i],
1210 req->r_session, req->r_request_started);
1211 dput(dn);
1212 }
1213 req->r_did_prepopulate = true;
1214
1215out:
1216 if (snapdir) {
1217 iput(snapdir);
1218 dput(parent);
1219 }
1220 dout("readdir_prepopulate done\n");
1221 return err;
1222}
1223
1224int ceph_inode_set_size(struct inode *inode, loff_t size)
1225{
1226 struct ceph_inode_info *ci = ceph_inode(inode);
1227 int ret = 0;
1228
1229 spin_lock(&inode->i_lock);
1230 dout("set_size %p %llu -> %llu\n", inode, inode->i_size, size);
1231 inode->i_size = size;
1232 inode->i_blocks = (size + (1 << 9) - 1) >> 9;
1233
1234 /* tell the MDS if we are approaching max_size */
1235 if ((size << 1) >= ci->i_max_size &&
1236 (ci->i_reported_size << 1) < ci->i_max_size)
1237 ret = 1;
1238
1239 spin_unlock(&inode->i_lock);
1240 return ret;
1241}
1242
1243/*
1244 * Write back inode data in a worker thread. (This can't be done
1245 * in the message handler context.)
1246 */
1247void ceph_queue_writeback(struct inode *inode)
1248{
1249 if (queue_work(ceph_inode_to_client(inode)->wb_wq,
1250 &ceph_inode(inode)->i_wb_work)) {
1251 dout("ceph_queue_writeback %p\n", inode);
1252 igrab(inode);
1253 } else {
1254 dout("ceph_queue_writeback %p failed\n", inode);
1255 }
1256}
1257
1258static void ceph_writeback_work(struct work_struct *work)
1259{
1260 struct ceph_inode_info *ci = container_of(work, struct ceph_inode_info,
1261 i_wb_work);
1262 struct inode *inode = &ci->vfs_inode;
1263
1264 dout("writeback %p\n", inode);
1265 filemap_fdatawrite(&inode->i_data);
1266 iput(inode);
1267}
1268
1269/*
1270 * queue an async invalidation
1271 */
1272void ceph_queue_invalidate(struct inode *inode)
1273{
1274 if (queue_work(ceph_inode_to_client(inode)->pg_inv_wq,
1275 &ceph_inode(inode)->i_pg_inv_work)) {
1276 dout("ceph_queue_invalidate %p\n", inode);
1277 igrab(inode);
1278 } else {
1279 dout("ceph_queue_invalidate %p failed\n", inode);
1280 }
1281}
1282
1283/*
1284 * invalidate any pages that are not dirty or under writeback. this
1285 * includes pages that are clean and mapped.
1286 */
1287static void ceph_invalidate_nondirty_pages(struct address_space *mapping)
1288{
1289 struct pagevec pvec;
1290 pgoff_t next = 0;
1291 int i;
1292
1293 pagevec_init(&pvec, 0);
1294 while (pagevec_lookup(&pvec, mapping, next, PAGEVEC_SIZE)) {
1295 for (i = 0; i < pagevec_count(&pvec); i++) {
1296 struct page *page = pvec.pages[i];
1297 pgoff_t index;
1298 int skip_page =
1299 (PageDirty(page) || PageWriteback(page));
1300
1301 if (!skip_page)
1302 skip_page = !trylock_page(page);
1303
1304 /*
1305 * We really shouldn't be looking at the ->index of an
1306 * unlocked page. But we're not allowed to lock these
1307 * pages. So we rely upon nobody altering the ->index
1308 * of this (pinned-by-us) page.
1309 */
1310 index = page->index;
1311 if (index > next)
1312 next = index;
1313 next++;
1314
1315 if (skip_page)
1316 continue;
1317
1318 generic_error_remove_page(mapping, page);
1319 unlock_page(page);
1320 }
1321 pagevec_release(&pvec);
1322 cond_resched();
1323 }
1324}
1325
1326/*
1327 * Invalidate inode pages in a worker thread. (This can't be done
1328 * in the message handler context.)
1329 */
1330static void ceph_invalidate_work(struct work_struct *work)
1331{
1332 struct ceph_inode_info *ci = container_of(work, struct ceph_inode_info,
1333 i_pg_inv_work);
1334 struct inode *inode = &ci->vfs_inode;
1335 u32 orig_gen;
1336 int check = 0;
1337
1338 spin_lock(&inode->i_lock);
1339 dout("invalidate_pages %p gen %d revoking %d\n", inode,
1340 ci->i_rdcache_gen, ci->i_rdcache_revoking);
1341 if (ci->i_rdcache_gen == 0 ||
1342 ci->i_rdcache_revoking != ci->i_rdcache_gen) {
1343 BUG_ON(ci->i_rdcache_revoking > ci->i_rdcache_gen);
1344 /* nevermind! */
1345 ci->i_rdcache_revoking = 0;
1346 spin_unlock(&inode->i_lock);
1347 goto out;
1348 }
1349 orig_gen = ci->i_rdcache_gen;
1350 spin_unlock(&inode->i_lock);
1351
1352 ceph_invalidate_nondirty_pages(inode->i_mapping);
1353
1354 spin_lock(&inode->i_lock);
1355 if (orig_gen == ci->i_rdcache_gen) {
1356 dout("invalidate_pages %p gen %d successful\n", inode,
1357 ci->i_rdcache_gen);
1358 ci->i_rdcache_gen = 0;
1359 ci->i_rdcache_revoking = 0;
1360 check = 1;
1361 } else {
1362 dout("invalidate_pages %p gen %d raced, gen now %d\n",
1363 inode, orig_gen, ci->i_rdcache_gen);
1364 }
1365 spin_unlock(&inode->i_lock);
1366
1367 if (check)
1368 ceph_check_caps(ci, 0, NULL);
1369out:
1370 iput(inode);
1371}
1372
1373
1374/*
1375 * called by trunc_wq; take i_mutex ourselves
1376 *
1377 * We also truncate in a separate thread as well.
1378 */
1379static void ceph_vmtruncate_work(struct work_struct *work)
1380{
1381 struct ceph_inode_info *ci = container_of(work, struct ceph_inode_info,
1382 i_vmtruncate_work);
1383 struct inode *inode = &ci->vfs_inode;
1384
1385 dout("vmtruncate_work %p\n", inode);
1386 mutex_lock(&inode->i_mutex);
1387 __ceph_do_pending_vmtruncate(inode);
1388 mutex_unlock(&inode->i_mutex);
1389 iput(inode);
1390}
1391
1392/*
1393 * Queue an async vmtruncate. If we fail to queue work, we will handle
1394 * the truncation the next time we call __ceph_do_pending_vmtruncate.
1395 */
1396void ceph_queue_vmtruncate(struct inode *inode)
1397{
1398 struct ceph_inode_info *ci = ceph_inode(inode);
1399
1400 if (queue_work(ceph_client(inode->i_sb)->trunc_wq,
1401 &ci->i_vmtruncate_work)) {
1402 dout("ceph_queue_vmtruncate %p\n", inode);
1403 igrab(inode);
1404 } else {
1405 dout("ceph_queue_vmtruncate %p failed, pending=%d\n",
1406 inode, ci->i_truncate_pending);
1407 }
1408}
1409
1410/*
1411 * called with i_mutex held.
1412 *
1413 * Make sure any pending truncation is applied before doing anything
1414 * that may depend on it.
1415 */
1416void __ceph_do_pending_vmtruncate(struct inode *inode)
1417{
1418 struct ceph_inode_info *ci = ceph_inode(inode);
1419 u64 to;
1420 int wrbuffer_refs, wake = 0;
1421
1422retry:
1423 spin_lock(&inode->i_lock);
1424 if (ci->i_truncate_pending == 0) {
1425 dout("__do_pending_vmtruncate %p none pending\n", inode);
1426 spin_unlock(&inode->i_lock);
1427 return;
1428 }
1429
1430 /*
1431 * make sure any dirty snapped pages are flushed before we
1432 * possibly truncate them.. so write AND block!
1433 */
1434 if (ci->i_wrbuffer_ref_head < ci->i_wrbuffer_ref) {
1435 dout("__do_pending_vmtruncate %p flushing snaps first\n",
1436 inode);
1437 spin_unlock(&inode->i_lock);
1438 filemap_write_and_wait_range(&inode->i_data, 0,
1439 inode->i_sb->s_maxbytes);
1440 goto retry;
1441 }
1442
1443 to = ci->i_truncate_size;
1444 wrbuffer_refs = ci->i_wrbuffer_ref;
1445 dout("__do_pending_vmtruncate %p (%d) to %lld\n", inode,
1446 ci->i_truncate_pending, to);
1447 spin_unlock(&inode->i_lock);
1448
1449 truncate_inode_pages(inode->i_mapping, to);
1450
1451 spin_lock(&inode->i_lock);
1452 ci->i_truncate_pending--;
1453 if (ci->i_truncate_pending == 0)
1454 wake = 1;
1455 spin_unlock(&inode->i_lock);
1456
1457 if (wrbuffer_refs == 0)
1458 ceph_check_caps(ci, CHECK_CAPS_AUTHONLY, NULL);
1459 if (wake)
1460 wake_up(&ci->i_cap_wq);
1461}
1462
1463
1464/*
1465 * symlinks
1466 */
1467static void *ceph_sym_follow_link(struct dentry *dentry, struct nameidata *nd)
1468{
1469 struct ceph_inode_info *ci = ceph_inode(dentry->d_inode);
1470 nd_set_link(nd, ci->i_symlink);
1471 return NULL;
1472}
1473
1474static const struct inode_operations ceph_symlink_iops = {
1475 .readlink = generic_readlink,
1476 .follow_link = ceph_sym_follow_link,
1477};
1478
1479/*
1480 * setattr
1481 */
1482int ceph_setattr(struct dentry *dentry, struct iattr *attr)
1483{
1484 struct inode *inode = dentry->d_inode;
1485 struct ceph_inode_info *ci = ceph_inode(inode);
1486 struct inode *parent_inode = dentry->d_parent->d_inode;
1487 const unsigned int ia_valid = attr->ia_valid;
1488 struct ceph_mds_request *req;
1489 struct ceph_mds_client *mdsc = &ceph_client(dentry->d_sb)->mdsc;
1490 int issued;
1491 int release = 0, dirtied = 0;
1492 int mask = 0;
1493 int err = 0;
1494
1495 if (ceph_snap(inode) != CEPH_NOSNAP)
1496 return -EROFS;
1497
1498 __ceph_do_pending_vmtruncate(inode);
1499
1500 err = inode_change_ok(inode, attr);
1501 if (err != 0)
1502 return err;
1503
1504 req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_SETATTR,
1505 USE_AUTH_MDS);
1506 if (IS_ERR(req))
1507 return PTR_ERR(req);
1508
1509 spin_lock(&inode->i_lock);
1510 issued = __ceph_caps_issued(ci, NULL);
1511 dout("setattr %p issued %s\n", inode, ceph_cap_string(issued));
1512
1513 if (ia_valid & ATTR_UID) {
1514 dout("setattr %p uid %d -> %d\n", inode,
1515 inode->i_uid, attr->ia_uid);
1516 if (issued & CEPH_CAP_AUTH_EXCL) {
1517 inode->i_uid = attr->ia_uid;
1518 dirtied |= CEPH_CAP_AUTH_EXCL;
1519 } else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 ||
1520 attr->ia_uid != inode->i_uid) {
1521 req->r_args.setattr.uid = cpu_to_le32(attr->ia_uid);
1522 mask |= CEPH_SETATTR_UID;
1523 release |= CEPH_CAP_AUTH_SHARED;
1524 }
1525 }
1526 if (ia_valid & ATTR_GID) {
1527 dout("setattr %p gid %d -> %d\n", inode,
1528 inode->i_gid, attr->ia_gid);
1529 if (issued & CEPH_CAP_AUTH_EXCL) {
1530 inode->i_gid = attr->ia_gid;
1531 dirtied |= CEPH_CAP_AUTH_EXCL;
1532 } else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 ||
1533 attr->ia_gid != inode->i_gid) {
1534 req->r_args.setattr.gid = cpu_to_le32(attr->ia_gid);
1535 mask |= CEPH_SETATTR_GID;
1536 release |= CEPH_CAP_AUTH_SHARED;
1537 }
1538 }
1539 if (ia_valid & ATTR_MODE) {
1540 dout("setattr %p mode 0%o -> 0%o\n", inode, inode->i_mode,
1541 attr->ia_mode);
1542 if (issued & CEPH_CAP_AUTH_EXCL) {
1543 inode->i_mode = attr->ia_mode;
1544 dirtied |= CEPH_CAP_AUTH_EXCL;
1545 } else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 ||
1546 attr->ia_mode != inode->i_mode) {
1547 req->r_args.setattr.mode = cpu_to_le32(attr->ia_mode);
1548 mask |= CEPH_SETATTR_MODE;
1549 release |= CEPH_CAP_AUTH_SHARED;
1550 }
1551 }
1552
1553 if (ia_valid & ATTR_ATIME) {
1554 dout("setattr %p atime %ld.%ld -> %ld.%ld\n", inode,
1555 inode->i_atime.tv_sec, inode->i_atime.tv_nsec,
1556 attr->ia_atime.tv_sec, attr->ia_atime.tv_nsec);
1557 if (issued & CEPH_CAP_FILE_EXCL) {
1558 ci->i_time_warp_seq++;
1559 inode->i_atime = attr->ia_atime;
1560 dirtied |= CEPH_CAP_FILE_EXCL;
1561 } else if ((issued & CEPH_CAP_FILE_WR) &&
1562 timespec_compare(&inode->i_atime,
1563 &attr->ia_atime) < 0) {
1564 inode->i_atime = attr->ia_atime;
1565 dirtied |= CEPH_CAP_FILE_WR;
1566 } else if ((issued & CEPH_CAP_FILE_SHARED) == 0 ||
1567 !timespec_equal(&inode->i_atime, &attr->ia_atime)) {
1568 ceph_encode_timespec(&req->r_args.setattr.atime,
1569 &attr->ia_atime);
1570 mask |= CEPH_SETATTR_ATIME;
1571 release |= CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_RD |
1572 CEPH_CAP_FILE_WR;
1573 }
1574 }
1575 if (ia_valid & ATTR_MTIME) {
1576 dout("setattr %p mtime %ld.%ld -> %ld.%ld\n", inode,
1577 inode->i_mtime.tv_sec, inode->i_mtime.tv_nsec,
1578 attr->ia_mtime.tv_sec, attr->ia_mtime.tv_nsec);
1579 if (issued & CEPH_CAP_FILE_EXCL) {
1580 ci->i_time_warp_seq++;
1581 inode->i_mtime = attr->ia_mtime;
1582 dirtied |= CEPH_CAP_FILE_EXCL;
1583 } else if ((issued & CEPH_CAP_FILE_WR) &&
1584 timespec_compare(&inode->i_mtime,
1585 &attr->ia_mtime) < 0) {
1586 inode->i_mtime = attr->ia_mtime;
1587 dirtied |= CEPH_CAP_FILE_WR;
1588 } else if ((issued & CEPH_CAP_FILE_SHARED) == 0 ||
1589 !timespec_equal(&inode->i_mtime, &attr->ia_mtime)) {
1590 ceph_encode_timespec(&req->r_args.setattr.mtime,
1591 &attr->ia_mtime);
1592 mask |= CEPH_SETATTR_MTIME;
1593 release |= CEPH_CAP_FILE_SHARED | CEPH_CAP_FILE_RD |
1594 CEPH_CAP_FILE_WR;
1595 }
1596 }
1597 if (ia_valid & ATTR_SIZE) {
1598 dout("setattr %p size %lld -> %lld\n", inode,
1599 inode->i_size, attr->ia_size);
1600 if (attr->ia_size > inode->i_sb->s_maxbytes) {
1601 err = -EINVAL;
1602 goto out;
1603 }
1604 if ((issued & CEPH_CAP_FILE_EXCL) &&
1605 attr->ia_size > inode->i_size) {
1606 inode->i_size = attr->ia_size;
1607 inode->i_blocks =
1608 (attr->ia_size + (1 << 9) - 1) >> 9;
1609 inode->i_ctime = attr->ia_ctime;
1610 ci->i_reported_size = attr->ia_size;
1611 dirtied |= CEPH_CAP_FILE_EXCL;
1612 } else if ((issued & CEPH_CAP_FILE_SHARED) == 0 ||
1613 attr->ia_size != inode->i_size) {
1614 req->r_args.setattr.size = cpu_to_le64(attr->ia_size);
1615 req->r_args.setattr.old_size =
1616 cpu_to_le64(inode->i_size);
1617 mask |= CEPH_SETATTR_SIZE;
1618 release |= CEPH_CAP_FILE_SHARED | CEPH_CAP_FILE_RD |
1619 CEPH_CAP_FILE_WR;
1620 }
1621 }
1622
1623 /* these do nothing */
1624 if (ia_valid & ATTR_CTIME) {
1625 bool only = (ia_valid & (ATTR_SIZE|ATTR_MTIME|ATTR_ATIME|
1626 ATTR_MODE|ATTR_UID|ATTR_GID)) == 0;
1627 dout("setattr %p ctime %ld.%ld -> %ld.%ld (%s)\n", inode,
1628 inode->i_ctime.tv_sec, inode->i_ctime.tv_nsec,
1629 attr->ia_ctime.tv_sec, attr->ia_ctime.tv_nsec,
1630 only ? "ctime only" : "ignored");
1631 inode->i_ctime = attr->ia_ctime;
1632 if (only) {
1633 /*
1634 * if kernel wants to dirty ctime but nothing else,
1635 * we need to choose a cap to dirty under, or do
1636 * a almost-no-op setattr
1637 */
1638 if (issued & CEPH_CAP_AUTH_EXCL)
1639 dirtied |= CEPH_CAP_AUTH_EXCL;
1640 else if (issued & CEPH_CAP_FILE_EXCL)
1641 dirtied |= CEPH_CAP_FILE_EXCL;
1642 else if (issued & CEPH_CAP_XATTR_EXCL)
1643 dirtied |= CEPH_CAP_XATTR_EXCL;
1644 else
1645 mask |= CEPH_SETATTR_CTIME;
1646 }
1647 }
1648 if (ia_valid & ATTR_FILE)
1649 dout("setattr %p ATTR_FILE ... hrm!\n", inode);
1650
1651 if (dirtied) {
1652 __ceph_mark_dirty_caps(ci, dirtied);
1653 inode->i_ctime = CURRENT_TIME;
1654 }
1655
1656 release &= issued;
1657 spin_unlock(&inode->i_lock);
1658
1659 if (mask) {
1660 req->r_inode = igrab(inode);
1661 req->r_inode_drop = release;
1662 req->r_args.setattr.mask = cpu_to_le32(mask);
1663 req->r_num_caps = 1;
1664 err = ceph_mdsc_do_request(mdsc, parent_inode, req);
1665 }
1666 dout("setattr %p result=%d (%s locally, %d remote)\n", inode, err,
1667 ceph_cap_string(dirtied), mask);
1668
1669 ceph_mdsc_put_request(req);
1670 __ceph_do_pending_vmtruncate(inode);
1671 return err;
1672out:
1673 spin_unlock(&inode->i_lock);
1674 ceph_mdsc_put_request(req);
1675 return err;
1676}
1677
1678/*
1679 * Verify that we have a lease on the given mask. If not,
1680 * do a getattr against an mds.
1681 */
1682int ceph_do_getattr(struct inode *inode, int mask)
1683{
1684 struct ceph_client *client = ceph_sb_to_client(inode->i_sb);
1685 struct ceph_mds_client *mdsc = &client->mdsc;
1686 struct ceph_mds_request *req;
1687 int err;
1688
1689 if (ceph_snap(inode) == CEPH_SNAPDIR) {
1690 dout("do_getattr inode %p SNAPDIR\n", inode);
1691 return 0;
1692 }
1693
1694 dout("do_getattr inode %p mask %s\n", inode, ceph_cap_string(mask));
1695 if (ceph_caps_issued_mask(ceph_inode(inode), mask, 1))
1696 return 0;
1697
1698 req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_GETATTR, USE_ANY_MDS);
1699 if (IS_ERR(req))
1700 return PTR_ERR(req);
1701 req->r_inode = igrab(inode);
1702 req->r_num_caps = 1;
1703 req->r_args.getattr.mask = cpu_to_le32(mask);
1704 err = ceph_mdsc_do_request(mdsc, NULL, req);
1705 ceph_mdsc_put_request(req);
1706 dout("do_getattr result=%d\n", err);
1707 return err;
1708}
1709
1710
1711/*
1712 * Check inode permissions. We verify we have a valid value for
1713 * the AUTH cap, then call the generic handler.
1714 */
1715int ceph_permission(struct inode *inode, int mask)
1716{
1717 int err = ceph_do_getattr(inode, CEPH_CAP_AUTH_SHARED);
1718
1719 if (!err)
1720 err = generic_permission(inode, mask, NULL);
1721 return err;
1722}
1723
1724/*
1725 * Get all attributes. Hopefully somedata we'll have a statlite()
1726 * and can limit the fields we require to be accurate.
1727 */
1728int ceph_getattr(struct vfsmount *mnt, struct dentry *dentry,
1729 struct kstat *stat)
1730{
1731 struct inode *inode = dentry->d_inode;
1732 struct ceph_inode_info *ci = ceph_inode(inode);
1733 int err;
1734
1735 err = ceph_do_getattr(inode, CEPH_STAT_CAP_INODE_ALL);
1736 if (!err) {
1737 generic_fillattr(inode, stat);
1738 stat->ino = inode->i_ino;
1739 if (ceph_snap(inode) != CEPH_NOSNAP)
1740 stat->dev = ceph_snap(inode);
1741 else
1742 stat->dev = 0;
1743 if (S_ISDIR(inode->i_mode)) {
1744 stat->size = ci->i_rbytes;
1745 stat->blocks = 0;
1746 stat->blksize = 65536;
1747 }
1748 }
1749 return err;
1750}
fs/ceph/ioctl.c+160−0
@@ -0,0 +1,160 @@
1#include <linux/in.h>
2
3#include "ioctl.h"
4#include "super.h"
5#include "ceph_debug.h"
6
7
8/*
9 * ioctls
10 */
11
12/*
13 * get and set the file layout
14 */
15static long ceph_ioctl_get_layout(struct file *file, void __user *arg)
16{
17 struct ceph_inode_info *ci = ceph_inode(file->f_dentry->d_inode);
18 struct ceph_ioctl_layout l;
19 int err;
20
21 err = ceph_do_getattr(file->f_dentry->d_inode, CEPH_STAT_CAP_LAYOUT);
22 if (!err) {
23 l.stripe_unit = ceph_file_layout_su(ci->i_layout);
24 l.stripe_count = ceph_file_layout_stripe_count(ci->i_layout);
25 l.object_size = ceph_file_layout_object_size(ci->i_layout);
26 l.data_pool = le32_to_cpu(ci->i_layout.fl_pg_pool);
27 l.preferred_osd =
28 (s32)le32_to_cpu(ci->i_layout.fl_pg_preferred);
29 if (copy_to_user(arg, &l, sizeof(l)))
30 return -EFAULT;
31 }
32
33 return err;
34}
35
36static long ceph_ioctl_set_layout(struct file *file, void __user *arg)
37{
38 struct inode *inode = file->f_dentry->d_inode;
39 struct inode *parent_inode = file->f_dentry->d_parent->d_inode;
40 struct ceph_mds_client *mdsc = &ceph_sb_to_client(inode->i_sb)->mdsc;
41 struct ceph_mds_request *req;
42 struct ceph_ioctl_layout l;
43 int err, i;
44
45 /* copy and validate */
46 if (copy_from_user(&l, arg, sizeof(l)))
47 return -EFAULT;
48
49 if ((l.object_size & ~PAGE_MASK) ||
50 (l.stripe_unit & ~PAGE_MASK) ||
51 !l.stripe_unit ||
52 (l.object_size &&
53 (unsigned)l.object_size % (unsigned)l.stripe_unit))
54 return -EINVAL;
55
56 /* make sure it's a valid data pool */
57 if (l.data_pool > 0) {
58 mutex_lock(&mdsc->mutex);
59 err = -EINVAL;
60 for (i = 0; i < mdsc->mdsmap->m_num_data_pg_pools; i++)
61 if (mdsc->mdsmap->m_data_pg_pools[i] == l.data_pool) {
62 err = 0;
63 break;
64 }
65 mutex_unlock(&mdsc->mutex);
66 if (err)
67 return err;
68 }
69
70 req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_SETLAYOUT,
71 USE_AUTH_MDS);
72 if (IS_ERR(req))
73 return PTR_ERR(req);
74 req->r_inode = igrab(inode);
75 req->r_inode_drop = CEPH_CAP_FILE_SHARED | CEPH_CAP_FILE_EXCL;
76
77 req->r_args.setlayout.layout.fl_stripe_unit =
78 cpu_to_le32(l.stripe_unit);
79 req->r_args.setlayout.layout.fl_stripe_count =
80 cpu_to_le32(l.stripe_count);
81 req->r_args.setlayout.layout.fl_object_size =
82 cpu_to_le32(l.object_size);
83 req->r_args.setlayout.layout.fl_pg_pool = cpu_to_le32(l.data_pool);
84 req->r_args.setlayout.layout.fl_pg_preferred =
85 cpu_to_le32(l.preferred_osd);
86
87 err = ceph_mdsc_do_request(mdsc, parent_inode, req);
88 ceph_mdsc_put_request(req);
89 return err;
90}
91
92/*
93 * Return object name, size/offset information, and location (OSD
94 * number, network address) for a given file offset.
95 */
96static long ceph_ioctl_get_dataloc(struct file *file, void __user *arg)
97{
98 struct ceph_ioctl_dataloc dl;
99 struct inode *inode = file->f_dentry->d_inode;
100 struct ceph_inode_info *ci = ceph_inode(inode);
101 struct ceph_osd_client *osdc = &ceph_client(inode->i_sb)->osdc;
102 u64 len = 1, olen;
103 u64 tmp;
104 struct ceph_object_layout ol;
105 struct ceph_pg pgid;
106
107 /* copy and validate */
108 if (copy_from_user(&dl, arg, sizeof(dl)))
109 return -EFAULT;
110
111 down_read(&osdc->map_sem);
112 ceph_calc_file_object_mapping(&ci->i_layout, dl.file_offset, &len,
113 &dl.object_no, &dl.object_offset, &olen);
114 dl.file_offset -= dl.object_offset;
115 dl.object_size = ceph_file_layout_object_size(ci->i_layout);
116 dl.block_size = ceph_file_layout_su(ci->i_layout);
117
118 /* block_offset = object_offset % block_size */
119 tmp = dl.object_offset;
120 dl.block_offset = do_div(tmp, dl.block_size);
121
122 snprintf(dl.object_name, sizeof(dl.object_name), "%llx.%08llx",
123 ceph_ino(inode), dl.object_no);
124 ceph_calc_object_layout(&ol, dl.object_name, &ci->i_layout,
125 osdc->osdmap);
126
127 pgid = ol.ol_pgid;
128 dl.osd = ceph_calc_pg_primary(osdc->osdmap, pgid);
129 if (dl.osd >= 0) {
130 struct ceph_entity_addr *a =
131 ceph_osd_addr(osdc->osdmap, dl.osd);
132 if (a)
133 memcpy(&dl.osd_addr, &a->in_addr, sizeof(dl.osd_addr));
134 } else {
135 memset(&dl.osd_addr, 0, sizeof(dl.osd_addr));
136 }
137 up_read(&osdc->map_sem);
138
139 /* send result back to user */
140 if (copy_to_user(arg, &dl, sizeof(dl)))
141 return -EFAULT;
142
143 return 0;
144}
145
146long ceph_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
147{
148 dout("ioctl file %p cmd %u arg %lu\n", file, cmd, arg);
149 switch (cmd) {
150 case CEPH_IOC_GET_LAYOUT:
151 return ceph_ioctl_get_layout(file, (void __user *)arg);
152
153 case CEPH_IOC_SET_LAYOUT:
154 return ceph_ioctl_set_layout(file, (void __user *)arg);
155
156 case CEPH_IOC_GET_DATALOC:
157 return ceph_ioctl_get_dataloc(file, (void __user *)arg);
158 }
159 return -ENOTTY;
160}
fs/ceph/ioctl.h+40−0
@@ -0,0 +1,40 @@
1#ifndef FS_CEPH_IOCTL_H
2#define FS_CEPH_IOCTL_H
3
4#include <linux/ioctl.h>
5#include <linux/types.h>
6
7#define CEPH_IOCTL_MAGIC 0x97
8
9/* just use u64 to align sanely on all archs */
10struct ceph_ioctl_layout {
11 __u64 stripe_unit, stripe_count, object_size;
12 __u64 data_pool;
13 __s64 preferred_osd;
14};
15
16#define CEPH_IOC_GET_LAYOUT _IOR(CEPH_IOCTL_MAGIC, 1, \
17 struct ceph_ioctl_layout)
18#define CEPH_IOC_SET_LAYOUT _IOW(CEPH_IOCTL_MAGIC, 2, \
19 struct ceph_ioctl_layout)
20
21/*
22 * Extract identity, address of the OSD and object storing a given
23 * file offset.
24 */
25struct ceph_ioctl_dataloc {
26 __u64 file_offset; /* in+out: file offset */
27 __u64 object_offset; /* out: offset in object */
28 __u64 object_no; /* out: object # */
29 __u64 object_size; /* out: object size */
30 char object_name[64]; /* out: object name */
31 __u64 block_offset; /* out: offset in block */
32 __u64 block_size; /* out: block length */
33 __s64 osd; /* out: osd # */
34 struct sockaddr_storage osd_addr; /* out: osd address */
35};
36
37#define CEPH_IOC_GET_DATALOC _IOWR(CEPH_IOCTL_MAGIC, 3, \
38 struct ceph_ioctl_dataloc)
39
40#endif
fs/ceph/mds_client.c+3021−0
@@ -0,0 +1,3021 @@
1#include "ceph_debug.h"
2
3#include <linux/wait.h>
4#include <linux/sched.h>
5
6#include "mds_client.h"
7#include "mon_client.h"
8#include "super.h"
9#include "messenger.h"
10#include "decode.h"
11#include "auth.h"
12#include "pagelist.h"
13
14/*
15 * A cluster of MDS (metadata server) daemons is responsible for
16 * managing the file system namespace (the directory hierarchy and
17 * inodes) and for coordinating shared access to storage. Metadata is
18 * partitioning hierarchically across a number of servers, and that
19 * partition varies over time as the cluster adjusts the distribution
20 * in order to balance load.
21 *
22 * The MDS client is primarily responsible to managing synchronous
23 * metadata requests for operations like open, unlink, and so forth.
24 * If there is a MDS failure, we find out about it when we (possibly
25 * request and) receive a new MDS map, and can resubmit affected
26 * requests.
27 *
28 * For the most part, though, we take advantage of a lossless
29 * communications channel to the MDS, and do not need to worry about
30 * timing out or resubmitting requests.
31 *
32 * We maintain a stateful "session" with each MDS we interact with.
33 * Within each session, we sent periodic heartbeat messages to ensure
34 * any capabilities or leases we have been issues remain valid. If
35 * the session times out and goes stale, our leases and capabilities
36 * are no longer valid.
37 */
38
39static void __wake_requests(struct ceph_mds_client *mdsc,
40 struct list_head *head);
41
42const static struct ceph_connection_operations mds_con_ops;
43
44
45/*
46 * mds reply parsing
47 */
48
49/*
50 * parse individual inode info
51 */
52static int parse_reply_info_in(void **p, void *end,
53 struct ceph_mds_reply_info_in *info)
54{
55 int err = -EIO;
56
57 info->in = *p;
58 *p += sizeof(struct ceph_mds_reply_inode) +
59 sizeof(*info->in->fragtree.splits) *
60 le32_to_cpu(info->in->fragtree.nsplits);
61
62 ceph_decode_32_safe(p, end, info->symlink_len, bad);
63 ceph_decode_need(p, end, info->symlink_len, bad);
64 info->symlink = *p;
65 *p += info->symlink_len;
66
67 ceph_decode_32_safe(p, end, info->xattr_len, bad);
68 ceph_decode_need(p, end, info->xattr_len, bad);
69 info->xattr_data = *p;
70 *p += info->xattr_len;
71 return 0;
72bad:
73 return err;
74}
75
76/*
77 * parse a normal reply, which may contain a (dir+)dentry and/or a
78 * target inode.
79 */
80static int parse_reply_info_trace(void **p, void *end,
81 struct ceph_mds_reply_info_parsed *info)
82{
83 int err;
84
85 if (info->head->is_dentry) {
86 err = parse_reply_info_in(p, end, &info->diri);
87 if (err < 0)
88 goto out_bad;
89
90 if (unlikely(*p + sizeof(*info->dirfrag) > end))
91 goto bad;
92 info->dirfrag = *p;
93 *p += sizeof(*info->dirfrag) +
94 sizeof(u32)*le32_to_cpu(info->dirfrag->ndist);
95 if (unlikely(*p > end))
96 goto bad;
97
98 ceph_decode_32_safe(p, end, info->dname_len, bad);
99 ceph_decode_need(p, end, info->dname_len, bad);
100 info->dname = *p;
101 *p += info->dname_len;
102 info->dlease = *p;
103 *p += sizeof(*info->dlease);
104 }
105
106 if (info->head->is_target) {
107 err = parse_reply_info_in(p, end, &info->targeti);
108 if (err < 0)
109 goto out_bad;
110 }
111
112 if (unlikely(*p != end))
113 goto bad;
114 return 0;
115
116bad:
117 err = -EIO;
118out_bad:
119 pr_err("problem parsing mds trace %d\n", err);
120 return err;
121}
122
123/*
124 * parse readdir results
125 */
126static int parse_reply_info_dir(void **p, void *end,
127 struct ceph_mds_reply_info_parsed *info)
128{
129 u32 num, i = 0;
130 int err;
131
132 info->dir_dir = *p;
133 if (*p + sizeof(*info->dir_dir) > end)
134 goto bad;
135 *p += sizeof(*info->dir_dir) +
136 sizeof(u32)*le32_to_cpu(info->dir_dir->ndist);
137 if (*p > end)
138 goto bad;
139
140 ceph_decode_need(p, end, sizeof(num) + 2, bad);
141 num = ceph_decode_32(p);
142 info->dir_end = ceph_decode_8(p);
143 info->dir_complete = ceph_decode_8(p);
144 if (num == 0)
145 goto done;
146
147 /* alloc large array */
148 info->dir_nr = num;
149 info->dir_in = kcalloc(num, sizeof(*info->dir_in) +
150 sizeof(*info->dir_dname) +
151 sizeof(*info->dir_dname_len) +
152 sizeof(*info->dir_dlease),
153 GFP_NOFS);
154 if (info->dir_in == NULL) {
155 err = -ENOMEM;
156 goto out_bad;
157 }
158 info->dir_dname = (void *)(info->dir_in + num);
159 info->dir_dname_len = (void *)(info->dir_dname + num);
160 info->dir_dlease = (void *)(info->dir_dname_len + num);
161
162 while (num) {
163 /* dentry */
164 ceph_decode_need(p, end, sizeof(u32)*2, bad);
165 info->dir_dname_len[i] = ceph_decode_32(p);
166 ceph_decode_need(p, end, info->dir_dname_len[i], bad);
167 info->dir_dname[i] = *p;
168 *p += info->dir_dname_len[i];
169 dout("parsed dir dname '%.*s'\n", info->dir_dname_len[i],
170 info->dir_dname[i]);
171 info->dir_dlease[i] = *p;
172 *p += sizeof(struct ceph_mds_reply_lease);
173
174 /* inode */
175 err = parse_reply_info_in(p, end, &info->dir_in[i]);
176 if (err < 0)
177 goto out_bad;
178 i++;
179 num--;
180 }
181
182done:
183 if (*p != end)
184 goto bad;
185 return 0;
186
187bad:
188 err = -EIO;
189out_bad:
190 pr_err("problem parsing dir contents %d\n", err);
191 return err;
192}
193
194/*
195 * parse entire mds reply
196 */
197static int parse_reply_info(struct ceph_msg *msg,
198 struct ceph_mds_reply_info_parsed *info)
199{
200 void *p, *end;
201 u32 len;
202 int err;
203
204 info->head = msg->front.iov_base;
205 p = msg->front.iov_base + sizeof(struct ceph_mds_reply_head);
206 end = p + msg->front.iov_len - sizeof(struct ceph_mds_reply_head);
207
208 /* trace */
209 ceph_decode_32_safe(&p, end, len, bad);
210 if (len > 0) {
211 err = parse_reply_info_trace(&p, p+len, info);
212 if (err < 0)
213 goto out_bad;
214 }
215
216 /* dir content */
217 ceph_decode_32_safe(&p, end, len, bad);
218 if (len > 0) {
219 err = parse_reply_info_dir(&p, p+len, info);
220 if (err < 0)
221 goto out_bad;
222 }
223
224 /* snap blob */
225 ceph_decode_32_safe(&p, end, len, bad);
226 info->snapblob_len = len;
227 info->snapblob = p;
228 p += len;
229
230 if (p != end)
231 goto bad;
232 return 0;
233
234bad:
235 err = -EIO;
236out_bad:
237 pr_err("mds parse_reply err %d\n", err);
238 return err;
239}
240
241static void destroy_reply_info(struct ceph_mds_reply_info_parsed *info)
242{
243 kfree(info->dir_in);
244}
245
246
247/*
248 * sessions
249 */
250static const char *session_state_name(int s)
251{
252 switch (s) {
253 case CEPH_MDS_SESSION_NEW: return "new";
254 case CEPH_MDS_SESSION_OPENING: return "opening";
255 case CEPH_MDS_SESSION_OPEN: return "open";
256 case CEPH_MDS_SESSION_HUNG: return "hung";
257 case CEPH_MDS_SESSION_CLOSING: return "closing";
258 case CEPH_MDS_SESSION_RESTARTING: return "restarting";
259 case CEPH_MDS_SESSION_RECONNECTING: return "reconnecting";
260 default: return "???";
261 }
262}
263
264static struct ceph_mds_session *get_session(struct ceph_mds_session *s)
265{
266 if (atomic_inc_not_zero(&s->s_ref)) {
267 dout("mdsc get_session %p %d -> %d\n", s,
268 atomic_read(&s->s_ref)-1, atomic_read(&s->s_ref));
269 return s;
270 } else {
271 dout("mdsc get_session %p 0 -- FAIL", s);
272 return NULL;
273 }
274}
275
276void ceph_put_mds_session(struct ceph_mds_session *s)
277{
278 dout("mdsc put_session %p %d -> %d\n", s,
279 atomic_read(&s->s_ref), atomic_read(&s->s_ref)-1);
280 if (atomic_dec_and_test(&s->s_ref)) {
281 if (s->s_authorizer)
282 s->s_mdsc->client->monc.auth->ops->destroy_authorizer(
283 s->s_mdsc->client->monc.auth, s->s_authorizer);
284 kfree(s);
285 }
286}
287
288/*
289 * called under mdsc->mutex
290 */
291struct ceph_mds_session *__ceph_lookup_mds_session(struct ceph_mds_client *mdsc,
292 int mds)
293{
294 struct ceph_mds_session *session;
295
296 if (mds >= mdsc->max_sessions || mdsc->sessions[mds] == NULL)
297 return NULL;
298 session = mdsc->sessions[mds];
299 dout("lookup_mds_session %p %d\n", session,
300 atomic_read(&session->s_ref));
301 get_session(session);
302 return session;
303}
304
305static bool __have_session(struct ceph_mds_client *mdsc, int mds)
306{
307 if (mds >= mdsc->max_sessions)
308 return false;
309 return mdsc->sessions[mds];
310}
311
312static int __verify_registered_session(struct ceph_mds_client *mdsc,
313 struct ceph_mds_session *s)
314{
315 if (s->s_mds >= mdsc->max_sessions ||
316 mdsc->sessions[s->s_mds] != s)
317 return -ENOENT;
318 return 0;
319}
320
321/*
322 * create+register a new session for given mds.
323 * called under mdsc->mutex.
324 */
325static struct ceph_mds_session *register_session(struct ceph_mds_client *mdsc,
326 int mds)
327{
328 struct ceph_mds_session *s;
329
330 s = kzalloc(sizeof(*s), GFP_NOFS);
331 s->s_mdsc = mdsc;
332 s->s_mds = mds;
333 s->s_state = CEPH_MDS_SESSION_NEW;
334 s->s_ttl = 0;
335 s->s_seq = 0;
336 mutex_init(&s->s_mutex);
337
338 ceph_con_init(mdsc->client->msgr, &s->s_con);
339 s->s_con.private = s;
340 s->s_con.ops = &mds_con_ops;
341 s->s_con.peer_name.type = CEPH_ENTITY_TYPE_MDS;
342 s->s_con.peer_name.num = cpu_to_le64(mds);
343
344 spin_lock_init(&s->s_cap_lock);
345 s->s_cap_gen = 0;
346 s->s_cap_ttl = 0;
347 s->s_renew_requested = 0;
348 s->s_renew_seq = 0;
349 INIT_LIST_HEAD(&s->s_caps);
350 s->s_nr_caps = 0;
351 s->s_trim_caps = 0;
352 atomic_set(&s->s_ref, 1);
353 INIT_LIST_HEAD(&s->s_waiting);
354 INIT_LIST_HEAD(&s->s_unsafe);
355 s->s_num_cap_releases = 0;
356 s->s_cap_iterator = NULL;
357 INIT_LIST_HEAD(&s->s_cap_releases);
358 INIT_LIST_HEAD(&s->s_cap_releases_done);
359 INIT_LIST_HEAD(&s->s_cap_flushing);
360 INIT_LIST_HEAD(&s->s_cap_snaps_flushing);
361
362 dout("register_session mds%d\n", mds);
363 if (mds >= mdsc->max_sessions) {
364 int newmax = 1 << get_count_order(mds+1);
365 struct ceph_mds_session **sa;
366
367 dout("register_session realloc to %d\n", newmax);
368 sa = kcalloc(newmax, sizeof(void *), GFP_NOFS);
369 if (sa == NULL)
370 goto fail_realloc;
371 if (mdsc->sessions) {
372 memcpy(sa, mdsc->sessions,
373 mdsc->max_sessions * sizeof(void *));
374 kfree(mdsc->sessions);
375 }
376 mdsc->sessions = sa;
377 mdsc->max_sessions = newmax;
378 }
379 mdsc->sessions[mds] = s;
380 atomic_inc(&s->s_ref); /* one ref to sessions[], one to caller */
381
382 ceph_con_open(&s->s_con, ceph_mdsmap_get_addr(mdsc->mdsmap, mds));
383
384 return s;
385
386fail_realloc:
387 kfree(s);
388 return ERR_PTR(-ENOMEM);
389}
390
391/*
392 * called under mdsc->mutex
393 */
394static void __unregister_session(struct ceph_mds_client *mdsc,
395 struct ceph_mds_session *s)
396{
397 dout("__unregister_session mds%d %p\n", s->s_mds, s);
398 BUG_ON(mdsc->sessions[s->s_mds] != s);
399 mdsc->sessions[s->s_mds] = NULL;
400 ceph_con_close(&s->s_con);
401 ceph_put_mds_session(s);
402}
403
404/*
405 * drop session refs in request.
406 *
407 * should be last request ref, or hold mdsc->mutex
408 */
409static void put_request_session(struct ceph_mds_request *req)
410{
411 if (req->r_session) {
412 ceph_put_mds_session(req->r_session);
413 req->r_session = NULL;
414 }
415}
416
417void ceph_mdsc_release_request(struct kref *kref)
418{
419 struct ceph_mds_request *req = container_of(kref,
420 struct ceph_mds_request,
421 r_kref);
422 if (req->r_request)
423 ceph_msg_put(req->r_request);
424 if (req->r_reply) {
425 ceph_msg_put(req->r_reply);
426 destroy_reply_info(&req->r_reply_info);
427 }
428 if (req->r_inode) {
429 ceph_put_cap_refs(ceph_inode(req->r_inode),
430 CEPH_CAP_PIN);
431 iput(req->r_inode);
432 }
433 if (req->r_locked_dir)
434 ceph_put_cap_refs(ceph_inode(req->r_locked_dir),
435 CEPH_CAP_PIN);
436 if (req->r_target_inode)
437 iput(req->r_target_inode);
438 if (req->r_dentry)
439 dput(req->r_dentry);
440 if (req->r_old_dentry) {
441 ceph_put_cap_refs(
442 ceph_inode(req->r_old_dentry->d_parent->d_inode),
443 CEPH_CAP_PIN);
444 dput(req->r_old_dentry);
445 }
446 kfree(req->r_path1);
447 kfree(req->r_path2);
448 put_request_session(req);
449 ceph_unreserve_caps(&req->r_caps_reservation);
450 kfree(req);
451}
452
453/*
454 * lookup session, bump ref if found.
455 *
456 * called under mdsc->mutex.
457 */
458static struct ceph_mds_request *__lookup_request(struct ceph_mds_client *mdsc,
459 u64 tid)
460{
461 struct ceph_mds_request *req;
462 struct rb_node *n = mdsc->request_tree.rb_node;
463
464 while (n) {
465 req = rb_entry(n, struct ceph_mds_request, r_node);
466 if (tid < req->r_tid)
467 n = n->rb_left;
468 else if (tid > req->r_tid)
469 n = n->rb_right;
470 else {
471 ceph_mdsc_get_request(req);
472 return req;
473 }
474 }
475 return NULL;
476}
477
478static void __insert_request(struct ceph_mds_client *mdsc,
479 struct ceph_mds_request *new)
480{
481 struct rb_node **p = &mdsc->request_tree.rb_node;
482 struct rb_node *parent = NULL;
483 struct ceph_mds_request *req = NULL;
484
485 while (*p) {
486 parent = *p;
487 req = rb_entry(parent, struct ceph_mds_request, r_node);
488 if (new->r_tid < req->r_tid)
489 p = &(*p)->rb_left;
490 else if (new->r_tid > req->r_tid)
491 p = &(*p)->rb_right;
492 else
493 BUG();
494 }
495
496 rb_link_node(&new->r_node, parent, p);
497 rb_insert_color(&new->r_node, &mdsc->request_tree);
498}
499
500/*
501 * Register an in-flight request, and assign a tid. Link to directory
502 * are modifying (if any).
503 *
504 * Called under mdsc->mutex.
505 */
506static void __register_request(struct ceph_mds_client *mdsc,
507 struct ceph_mds_request *req,
508 struct inode *dir)
509{
510 req->r_tid = ++mdsc->last_tid;
511 if (req->r_num_caps)
512 ceph_reserve_caps(&req->r_caps_reservation, req->r_num_caps);
513 dout("__register_request %p tid %lld\n", req, req->r_tid);
514 ceph_mdsc_get_request(req);
515 __insert_request(mdsc, req);
516
517 if (dir) {
518 struct ceph_inode_info *ci = ceph_inode(dir);
519
520 spin_lock(&ci->i_unsafe_lock);
521 req->r_unsafe_dir = dir;
522 list_add_tail(&req->r_unsafe_dir_item, &ci->i_unsafe_dirops);
523 spin_unlock(&ci->i_unsafe_lock);
524 }
525}
526
527static void __unregister_request(struct ceph_mds_client *mdsc,
528 struct ceph_mds_request *req)
529{
530 dout("__unregister_request %p tid %lld\n", req, req->r_tid);
531 rb_erase(&req->r_node, &mdsc->request_tree);
532 ceph_mdsc_put_request(req);
533
534 if (req->r_unsafe_dir) {
535 struct ceph_inode_info *ci = ceph_inode(req->r_unsafe_dir);
536
537 spin_lock(&ci->i_unsafe_lock);
538 list_del_init(&req->r_unsafe_dir_item);
539 spin_unlock(&ci->i_unsafe_lock);
540 }
541}
542
543/*
544 * Choose mds to send request to next. If there is a hint set in the
545 * request (e.g., due to a prior forward hint from the mds), use that.
546 * Otherwise, consult frag tree and/or caps to identify the
547 * appropriate mds. If all else fails, choose randomly.
548 *
549 * Called under mdsc->mutex.
550 */
551static int __choose_mds(struct ceph_mds_client *mdsc,
552 struct ceph_mds_request *req)
553{
554 struct inode *inode;
555 struct ceph_inode_info *ci;
556 struct ceph_cap *cap;
557 int mode = req->r_direct_mode;
558 int mds = -1;
559 u32 hash = req->r_direct_hash;
560 bool is_hash = req->r_direct_is_hash;
561
562 /*
563 * is there a specific mds we should try? ignore hint if we have
564 * no session and the mds is not up (active or recovering).
565 */
566 if (req->r_resend_mds >= 0 &&
567 (__have_session(mdsc, req->r_resend_mds) ||
568 ceph_mdsmap_get_state(mdsc->mdsmap, req->r_resend_mds) > 0)) {
569 dout("choose_mds using resend_mds mds%d\n",
570 req->r_resend_mds);
571 return req->r_resend_mds;
572 }
573
574 if (mode == USE_RANDOM_MDS)
575 goto random;
576
577 inode = NULL;
578 if (req->r_inode) {
579 inode = req->r_inode;
580 } else if (req->r_dentry) {
581 if (req->r_dentry->d_inode) {
582 inode = req->r_dentry->d_inode;
583 } else {
584 inode = req->r_dentry->d_parent->d_inode;
585 hash = req->r_dentry->d_name.hash;
586 is_hash = true;
587 }
588 }
589 dout("__choose_mds %p is_hash=%d (%d) mode %d\n", inode, (int)is_hash,
590 (int)hash, mode);
591 if (!inode)
592 goto random;
593 ci = ceph_inode(inode);
594
595 if (is_hash && S_ISDIR(inode->i_mode)) {
596 struct ceph_inode_frag frag;
597 int found;
598
599 ceph_choose_frag(ci, hash, &frag, &found);
600 if (found) {
601 if (mode == USE_ANY_MDS && frag.ndist > 0) {
602 u8 r;
603
604 /* choose a random replica */
605 get_random_bytes(&r, 1);
606 r %= frag.ndist;
607 mds = frag.dist[r];
608 dout("choose_mds %p %llx.%llx "
609 "frag %u mds%d (%d/%d)\n",
610 inode, ceph_vinop(inode),
611 frag.frag, frag.mds,
612 (int)r, frag.ndist);
613 return mds;
614 }
615
616 /* since this file/dir wasn't known to be
617 * replicated, then we want to look for the
618 * authoritative mds. */
619 mode = USE_AUTH_MDS;
620 if (frag.mds >= 0) {
621 /* choose auth mds */
622 mds = frag.mds;
623 dout("choose_mds %p %llx.%llx "
624 "frag %u mds%d (auth)\n",
625 inode, ceph_vinop(inode), frag.frag, mds);
626 return mds;
627 }
628 }
629 }
630
631 spin_lock(&inode->i_lock);
632 cap = NULL;
633 if (mode == USE_AUTH_MDS)
634 cap = ci->i_auth_cap;
635 if (!cap && !RB_EMPTY_ROOT(&ci->i_caps))
636 cap = rb_entry(rb_first(&ci->i_caps), struct ceph_cap, ci_node);
637 if (!cap) {
638 spin_unlock(&inode->i_lock);
639 goto random;
640 }
641 mds = cap->session->s_mds;
642 dout("choose_mds %p %llx.%llx mds%d (%scap %p)\n",
643 inode, ceph_vinop(inode), mds,
644 cap == ci->i_auth_cap ? "auth " : "", cap);
645 spin_unlock(&inode->i_lock);
646 return mds;
647
648random:
649 mds = ceph_mdsmap_get_random_mds(mdsc->mdsmap);
650 dout("choose_mds chose random mds%d\n", mds);
651 return mds;
652}
653
654
655/*
656 * session messages
657 */
658static struct ceph_msg *create_session_msg(u32 op, u64 seq)
659{
660 struct ceph_msg *msg;
661 struct ceph_mds_session_head *h;
662
663 msg = ceph_msg_new(CEPH_MSG_CLIENT_SESSION, sizeof(*h), 0, 0, NULL);
664 if (IS_ERR(msg)) {
665 pr_err("create_session_msg ENOMEM creating msg\n");
666 return ERR_PTR(PTR_ERR(msg));
667 }
668 h = msg->front.iov_base;
669 h->op = cpu_to_le32(op);
670 h->seq = cpu_to_le64(seq);
671 return msg;
672}
673
674/*
675 * send session open request.
676 *
677 * called under mdsc->mutex
678 */
679static int __open_session(struct ceph_mds_client *mdsc,
680 struct ceph_mds_session *session)
681{
682 struct ceph_msg *msg;
683 int mstate;
684 int mds = session->s_mds;
685 int err = 0;
686
687 /* wait for mds to go active? */
688 mstate = ceph_mdsmap_get_state(mdsc->mdsmap, mds);
689 dout("open_session to mds%d (%s)\n", mds,
690 ceph_mds_state_name(mstate));
691 session->s_state = CEPH_MDS_SESSION_OPENING;
692 session->s_renew_requested = jiffies;
693
694 /* send connect message */
695 msg = create_session_msg(CEPH_SESSION_REQUEST_OPEN, session->s_seq);
696 if (IS_ERR(msg)) {
697 err = PTR_ERR(msg);
698 goto out;
699 }
700 ceph_con_send(&session->s_con, msg);
701
702out:
703 return 0;
704}
705
706/*
707 * session caps
708 */
709
710/*
711 * Free preallocated cap messages assigned to this session
712 */
713static void cleanup_cap_releases(struct ceph_mds_session *session)
714{
715 struct ceph_msg *msg;
716
717 spin_lock(&session->s_cap_lock);
718 while (!list_empty(&session->s_cap_releases)) {
719 msg = list_first_entry(&session->s_cap_releases,
720 struct ceph_msg, list_head);
721 list_del_init(&msg->list_head);
722 ceph_msg_put(msg);
723 }
724 while (!list_empty(&session->s_cap_releases_done)) {
725 msg = list_first_entry(&session->s_cap_releases_done,
726 struct ceph_msg, list_head);
727 list_del_init(&msg->list_head);
728 ceph_msg_put(msg);
729 }
730 spin_unlock(&session->s_cap_lock);
731}
732
733/*
734 * Helper to safely iterate over all caps associated with a session.
735 *
736 * caller must hold session s_mutex
737 */
738static int iterate_session_caps(struct ceph_mds_session *session,
739 int (*cb)(struct inode *, struct ceph_cap *,
740 void *), void *arg)
741{
742 struct list_head *p;
743 struct ceph_cap *cap;
744 struct inode *inode, *last_inode = NULL;
745 struct ceph_cap *old_cap = NULL;
746 int ret;
747
748 dout("iterate_session_caps %p mds%d\n", session, session->s_mds);
749 spin_lock(&session->s_cap_lock);
750 p = session->s_caps.next;
751 while (p != &session->s_caps) {
752 cap = list_entry(p, struct ceph_cap, session_caps);
753 inode = igrab(&cap->ci->vfs_inode);
754 if (!inode) {
755 p = p->next;
756 continue;
757 }
758 session->s_cap_iterator = cap;
759 spin_unlock(&session->s_cap_lock);
760
761 if (last_inode) {
762 iput(last_inode);
763 last_inode = NULL;
764 }
765 if (old_cap) {
766 ceph_put_cap(old_cap);
767 old_cap = NULL;
768 }
769
770 ret = cb(inode, cap, arg);
771 last_inode = inode;
772
773 spin_lock(&session->s_cap_lock);
774 p = p->next;
775 if (cap->ci == NULL) {
776 dout("iterate_session_caps finishing cap %p removal\n",
777 cap);
778 BUG_ON(cap->session != session);
779 list_del_init(&cap->session_caps);
780 session->s_nr_caps--;
781 cap->session = NULL;
782 old_cap = cap; /* put_cap it w/o locks held */
783 }
784 if (ret < 0)
785 goto out;
786 }
787 ret = 0;
788out:
789 session->s_cap_iterator = NULL;
790 spin_unlock(&session->s_cap_lock);
791
792 if (last_inode)
793 iput(last_inode);
794 if (old_cap)
795 ceph_put_cap(old_cap);
796
797 return ret;
798}
799
800static int remove_session_caps_cb(struct inode *inode, struct ceph_cap *cap,
801 void *arg)
802{
803 struct ceph_inode_info *ci = ceph_inode(inode);
804 dout("removing cap %p, ci is %p, inode is %p\n",
805 cap, ci, &ci->vfs_inode);
806 ceph_remove_cap(cap);
807 return 0;
808}
809
810/*
811 * caller must hold session s_mutex
812 */
813static void remove_session_caps(struct ceph_mds_session *session)
814{
815 dout("remove_session_caps on %p\n", session);
816 iterate_session_caps(session, remove_session_caps_cb, NULL);
817 BUG_ON(session->s_nr_caps > 0);
818 cleanup_cap_releases(session);
819}
820
821/*
822 * wake up any threads waiting on this session's caps. if the cap is
823 * old (didn't get renewed on the client reconnect), remove it now.
824 *
825 * caller must hold s_mutex.
826 */
827static int wake_up_session_cb(struct inode *inode, struct ceph_cap *cap,
828 void *arg)
829{
830 struct ceph_inode_info *ci = ceph_inode(inode);
831
832 wake_up(&ci->i_cap_wq);
833 if (arg) {
834 spin_lock(&inode->i_lock);
835 ci->i_wanted_max_size = 0;
836 ci->i_requested_max_size = 0;
837 spin_unlock(&inode->i_lock);
838 }
839 return 0;
840}
841
842static void wake_up_session_caps(struct ceph_mds_session *session,
843 int reconnect)
844{
845 dout("wake_up_session_caps %p mds%d\n", session, session->s_mds);
846 iterate_session_caps(session, wake_up_session_cb,
847 (void *)(unsigned long)reconnect);
848}
849
850/*
851 * Send periodic message to MDS renewing all currently held caps. The
852 * ack will reset the expiration for all caps from this session.
853 *
854 * caller holds s_mutex
855 */
856static int send_renew_caps(struct ceph_mds_client *mdsc,
857 struct ceph_mds_session *session)
858{
859 struct ceph_msg *msg;
860 int state;
861
862 if (time_after_eq(jiffies, session->s_cap_ttl) &&
863 time_after_eq(session->s_cap_ttl, session->s_renew_requested))
864 pr_info("mds%d caps stale\n", session->s_mds);
865
866 /* do not try to renew caps until a recovering mds has reconnected
867 * with its clients. */
868 state = ceph_mdsmap_get_state(mdsc->mdsmap, session->s_mds);
869 if (state < CEPH_MDS_STATE_RECONNECT) {
870 dout("send_renew_caps ignoring mds%d (%s)\n",
871 session->s_mds, ceph_mds_state_name(state));
872 return 0;
873 }
874
875 dout("send_renew_caps to mds%d (%s)\n", session->s_mds,
876 ceph_mds_state_name(state));
877 session->s_renew_requested = jiffies;
878 msg = create_session_msg(CEPH_SESSION_REQUEST_RENEWCAPS,
879 ++session->s_renew_seq);
880 if (IS_ERR(msg))
881 return PTR_ERR(msg);
882 ceph_con_send(&session->s_con, msg);
883 return 0;
884}
885
886/*
887 * Note new cap ttl, and any transition from stale -> not stale (fresh?).
888 *
889 * Called under session->s_mutex
890 */
891static void renewed_caps(struct ceph_mds_client *mdsc,
892 struct ceph_mds_session *session, int is_renew)
893{
894 int was_stale;
895 int wake = 0;
896
897 spin_lock(&session->s_cap_lock);
898 was_stale = is_renew && (session->s_cap_ttl == 0 ||
899 time_after_eq(jiffies, session->s_cap_ttl));
900
901 session->s_cap_ttl = session->s_renew_requested +
902 mdsc->mdsmap->m_session_timeout*HZ;
903
904 if (was_stale) {
905 if (time_before(jiffies, session->s_cap_ttl)) {
906 pr_info("mds%d caps renewed\n", session->s_mds);
907 wake = 1;
908 } else {
909 pr_info("mds%d caps still stale\n", session->s_mds);
910 }
911 }
912 dout("renewed_caps mds%d ttl now %lu, was %s, now %s\n",
913 session->s_mds, session->s_cap_ttl, was_stale ? "stale" : "fresh",
914 time_before(jiffies, session->s_cap_ttl) ? "stale" : "fresh");
915 spin_unlock(&session->s_cap_lock);
916
917 if (wake)
918 wake_up_session_caps(session, 0);
919}
920
921/*
922 * send a session close request
923 */
924static int request_close_session(struct ceph_mds_client *mdsc,
925 struct ceph_mds_session *session)
926{
927 struct ceph_msg *msg;
928 int err = 0;
929
930 dout("request_close_session mds%d state %s seq %lld\n",
931 session->s_mds, session_state_name(session->s_state),
932 session->s_seq);
933 msg = create_session_msg(CEPH_SESSION_REQUEST_CLOSE, session->s_seq);
934 if (IS_ERR(msg))
935 err = PTR_ERR(msg);
936 else
937 ceph_con_send(&session->s_con, msg);
938 return err;
939}
940
941/*
942 * Called with s_mutex held.
943 */
944static int __close_session(struct ceph_mds_client *mdsc,
945 struct ceph_mds_session *session)
946{
947 if (session->s_state >= CEPH_MDS_SESSION_CLOSING)
948 return 0;
949 session->s_state = CEPH_MDS_SESSION_CLOSING;
950 return request_close_session(mdsc, session);
951}
952
953/*
954 * Trim old(er) caps.
955 *
956 * Because we can't cache an inode without one or more caps, we do
957 * this indirectly: if a cap is unused, we prune its aliases, at which
958 * point the inode will hopefully get dropped to.
959 *
960 * Yes, this is a bit sloppy. Our only real goal here is to respond to
961 * memory pressure from the MDS, though, so it needn't be perfect.
962 */
963static int trim_caps_cb(struct inode *inode, struct ceph_cap *cap, void *arg)
964{
965 struct ceph_mds_session *session = arg;
966 struct ceph_inode_info *ci = ceph_inode(inode);
967 int used, oissued, mine;
968
969 if (session->s_trim_caps <= 0)
970 return -1;
971
972 spin_lock(&inode->i_lock);
973 mine = cap->issued | cap->implemented;
974 used = __ceph_caps_used(ci);
975 oissued = __ceph_caps_issued_other(ci, cap);
976
977 dout("trim_caps_cb %p cap %p mine %s oissued %s used %s\n",
978 inode, cap, ceph_cap_string(mine), ceph_cap_string(oissued),
979 ceph_cap_string(used));
980 if (ci->i_dirty_caps)
981 goto out; /* dirty caps */
982 if ((used & ~oissued) & mine)
983 goto out; /* we need these caps */
984
985 session->s_trim_caps--;
986 if (oissued) {
987 /* we aren't the only cap.. just remove us */
988 __ceph_remove_cap(cap);
989 } else {
990 /* try to drop referring dentries */
991 spin_unlock(&inode->i_lock);
992 d_prune_aliases(inode);
993 dout("trim_caps_cb %p cap %p pruned, count now %d\n",
994 inode, cap, atomic_read(&inode->i_count));
995 return 0;
996 }
997
998out:
999 spin_unlock(&inode->i_lock);
1000 return 0;
1001}
1002
1003/*
1004 * Trim session cap count down to some max number.
1005 */
1006static int trim_caps(struct ceph_mds_client *mdsc,
1007 struct ceph_mds_session *session,
1008 int max_caps)
1009{
1010 int trim_caps = session->s_nr_caps - max_caps;
1011
1012 dout("trim_caps mds%d start: %d / %d, trim %d\n",
1013 session->s_mds, session->s_nr_caps, max_caps, trim_caps);
1014 if (trim_caps > 0) {
1015 session->s_trim_caps = trim_caps;
1016 iterate_session_caps(session, trim_caps_cb, session);
1017 dout("trim_caps mds%d done: %d / %d, trimmed %d\n",
1018 session->s_mds, session->s_nr_caps, max_caps,
1019 trim_caps - session->s_trim_caps);
1020 session->s_trim_caps = 0;
1021 }
1022 return 0;
1023}
1024
1025/*
1026 * Allocate cap_release messages. If there is a partially full message
1027 * in the queue, try to allocate enough to cover it's remainder, so that
1028 * we can send it immediately.
1029 *
1030 * Called under s_mutex.
1031 */
1032static int add_cap_releases(struct ceph_mds_client *mdsc,
1033 struct ceph_mds_session *session,
1034 int extra)
1035{
1036 struct ceph_msg *msg;
1037 struct ceph_mds_cap_release *head;
1038 int err = -ENOMEM;
1039
1040 if (extra < 0)
1041 extra = mdsc->client->mount_args->cap_release_safety;
1042
1043 spin_lock(&session->s_cap_lock);
1044
1045 if (!list_empty(&session->s_cap_releases)) {
1046 msg = list_first_entry(&session->s_cap_releases,
1047 struct ceph_msg,
1048 list_head);
1049 head = msg->front.iov_base;
1050 extra += CEPH_CAPS_PER_RELEASE - le32_to_cpu(head->num);
1051 }
1052
1053 while (session->s_num_cap_releases < session->s_nr_caps + extra) {
1054 spin_unlock(&session->s_cap_lock);
1055 msg = ceph_msg_new(CEPH_MSG_CLIENT_CAPRELEASE, PAGE_CACHE_SIZE,
1056 0, 0, NULL);
1057 if (!msg)
1058 goto out_unlocked;
1059 dout("add_cap_releases %p msg %p now %d\n", session, msg,
1060 (int)msg->front.iov_len);
1061 head = msg->front.iov_base;
1062 head->num = cpu_to_le32(0);
1063 msg->front.iov_len = sizeof(*head);
1064 spin_lock(&session->s_cap_lock);
1065 list_add(&msg->list_head, &session->s_cap_releases);
1066 session->s_num_cap_releases += CEPH_CAPS_PER_RELEASE;
1067 }
1068
1069 if (!list_empty(&session->s_cap_releases)) {
1070 msg = list_first_entry(&session->s_cap_releases,
1071 struct ceph_msg,
1072 list_head);
1073 head = msg->front.iov_base;
1074 if (head->num) {
1075 dout(" queueing non-full %p (%d)\n", msg,
1076 le32_to_cpu(head->num));
1077 list_move_tail(&msg->list_head,
1078 &session->s_cap_releases_done);
1079 session->s_num_cap_releases -=
1080 CEPH_CAPS_PER_RELEASE - le32_to_cpu(head->num);
1081 }
1082 }
1083 err = 0;
1084 spin_unlock(&session->s_cap_lock);
1085out_unlocked:
1086 return err;
1087}
1088
1089/*
1090 * flush all dirty inode data to disk.
1091 *
1092 * returns true if we've flushed through want_flush_seq
1093 */
1094static int check_cap_flush(struct ceph_mds_client *mdsc, u64 want_flush_seq)
1095{
1096 int mds, ret = 1;
1097
1098 dout("check_cap_flush want %lld\n", want_flush_seq);
1099 mutex_lock(&mdsc->mutex);
1100 for (mds = 0; ret && mds < mdsc->max_sessions; mds++) {
1101 struct ceph_mds_session *session = mdsc->sessions[mds];
1102
1103 if (!session)
1104 continue;
1105 get_session(session);
1106 mutex_unlock(&mdsc->mutex);
1107
1108 mutex_lock(&session->s_mutex);
1109 if (!list_empty(&session->s_cap_flushing)) {
1110 struct ceph_inode_info *ci =
1111 list_entry(session->s_cap_flushing.next,
1112 struct ceph_inode_info,
1113 i_flushing_item);
1114 struct inode *inode = &ci->vfs_inode;
1115
1116 spin_lock(&inode->i_lock);
1117 if (ci->i_cap_flush_seq <= want_flush_seq) {
1118 dout("check_cap_flush still flushing %p "
1119 "seq %lld <= %lld to mds%d\n", inode,
1120 ci->i_cap_flush_seq, want_flush_seq,
1121 session->s_mds);
1122 ret = 0;
1123 }
1124 spin_unlock(&inode->i_lock);
1125 }
1126 mutex_unlock(&session->s_mutex);
1127 ceph_put_mds_session(session);
1128
1129 if (!ret)
1130 return ret;
1131 mutex_lock(&mdsc->mutex);
1132 }
1133
1134 mutex_unlock(&mdsc->mutex);
1135 dout("check_cap_flush ok, flushed thru %lld\n", want_flush_seq);
1136 return ret;
1137}
1138
1139/*
1140 * called under s_mutex
1141 */
1142static void send_cap_releases(struct ceph_mds_client *mdsc,
1143 struct ceph_mds_session *session)
1144{
1145 struct ceph_msg *msg;
1146
1147 dout("send_cap_releases mds%d\n", session->s_mds);
1148 while (1) {
1149 spin_lock(&session->s_cap_lock);
1150 if (list_empty(&session->s_cap_releases_done))
1151 break;
1152 msg = list_first_entry(&session->s_cap_releases_done,
1153 struct ceph_msg, list_head);
1154 list_del_init(&msg->list_head);
1155 spin_unlock(&session->s_cap_lock);
1156 msg->hdr.front_len = cpu_to_le32(msg->front.iov_len);
1157 dout("send_cap_releases mds%d %p\n", session->s_mds, msg);
1158 ceph_con_send(&session->s_con, msg);
1159 }
1160 spin_unlock(&session->s_cap_lock);
1161}
1162
1163/*
1164 * requests
1165 */
1166
1167/*
1168 * Create an mds request.
1169 */
1170struct ceph_mds_request *
1171ceph_mdsc_create_request(struct ceph_mds_client *mdsc, int op, int mode)
1172{
1173 struct ceph_mds_request *req = kzalloc(sizeof(*req), GFP_NOFS);
1174
1175 if (!req)
1176 return ERR_PTR(-ENOMEM);
1177
1178 req->r_started = jiffies;
1179 req->r_resend_mds = -1;
1180 INIT_LIST_HEAD(&req->r_unsafe_dir_item);
1181 req->r_fmode = -1;
1182 kref_init(&req->r_kref);
1183 INIT_LIST_HEAD(&req->r_wait);
1184 init_completion(&req->r_completion);
1185 init_completion(&req->r_safe_completion);
1186 INIT_LIST_HEAD(&req->r_unsafe_item);
1187
1188 req->r_op = op;
1189 req->r_direct_mode = mode;
1190 return req;
1191}
1192
1193/*
1194 * return oldest (lowest) request, tid in request tree, 0 if none.
1195 *
1196 * called under mdsc->mutex.
1197 */
1198static struct ceph_mds_request *__get_oldest_req(struct ceph_mds_client *mdsc)
1199{
1200 if (RB_EMPTY_ROOT(&mdsc->request_tree))
1201 return NULL;
1202 return rb_entry(rb_first(&mdsc->request_tree),
1203 struct ceph_mds_request, r_node);
1204}
1205
1206static u64 __get_oldest_tid(struct ceph_mds_client *mdsc)
1207{
1208 struct ceph_mds_request *req = __get_oldest_req(mdsc);
1209
1210 if (req)
1211 return req->r_tid;
1212 return 0;
1213}
1214
1215/*
1216 * Build a dentry's path. Allocate on heap; caller must kfree. Based
1217 * on build_path_from_dentry in fs/cifs/dir.c.
1218 *
1219 * If @stop_on_nosnap, generate path relative to the first non-snapped
1220 * inode.
1221 *
1222 * Encode hidden .snap dirs as a double /, i.e.
1223 * foo/.snap/bar -> foo//bar
1224 */
1225char *ceph_mdsc_build_path(struct dentry *dentry, int *plen, u64 *base,
1226 int stop_on_nosnap)
1227{
1228 struct dentry *temp;
1229 char *path;
1230 int len, pos;
1231
1232 if (dentry == NULL)
1233 return ERR_PTR(-EINVAL);
1234
1235retry:
1236 len = 0;
1237 for (temp = dentry; !IS_ROOT(temp);) {
1238 struct inode *inode = temp->d_inode;
1239 if (inode && ceph_snap(inode) == CEPH_SNAPDIR)
1240 len++; /* slash only */
1241 else if (stop_on_nosnap && inode &&
1242 ceph_snap(inode) == CEPH_NOSNAP)
1243 break;
1244 else
1245 len += 1 + temp->d_name.len;
1246 temp = temp->d_parent;
1247 if (temp == NULL) {
1248 pr_err("build_path_dentry corrupt dentry %p\n", dentry);
1249 return ERR_PTR(-EINVAL);
1250 }
1251 }
1252 if (len)
1253 len--; /* no leading '/' */
1254
1255 path = kmalloc(len+1, GFP_NOFS);
1256 if (path == NULL)
1257 return ERR_PTR(-ENOMEM);
1258 pos = len;
1259 path[pos] = 0; /* trailing null */
1260 for (temp = dentry; !IS_ROOT(temp) && pos != 0; ) {
1261 struct inode *inode = temp->d_inode;
1262
1263 if (inode && ceph_snap(inode) == CEPH_SNAPDIR) {
1264 dout("build_path_dentry path+%d: %p SNAPDIR\n",
1265 pos, temp);
1266 } else if (stop_on_nosnap && inode &&
1267 ceph_snap(inode) == CEPH_NOSNAP) {
1268 break;
1269 } else {
1270 pos -= temp->d_name.len;
1271 if (pos < 0)
1272 break;
1273 strncpy(path + pos, temp->d_name.name,
1274 temp->d_name.len);
1275 dout("build_path_dentry path+%d: %p '%.*s'\n",
1276 pos, temp, temp->d_name.len, path + pos);
1277 }
1278 if (pos)
1279 path[--pos] = '/';
1280 temp = temp->d_parent;
1281 if (temp == NULL) {
1282 pr_err("build_path_dentry corrupt dentry\n");
1283 kfree(path);
1284 return ERR_PTR(-EINVAL);
1285 }
1286 }
1287 if (pos != 0) {
1288 pr_err("build_path_dentry did not end path lookup where "
1289 "expected, namelen is %d, pos is %d\n", len, pos);
1290 /* presumably this is only possible if racing with a
1291 rename of one of the parent directories (we can not
1292 lock the dentries above us to prevent this, but
1293 retrying should be harmless) */
1294 kfree(path);
1295 goto retry;
1296 }
1297
1298 *base = ceph_ino(temp->d_inode);
1299 *plen = len;
1300 dout("build_path_dentry on %p %d built %llx '%.*s'\n",
1301 dentry, atomic_read(&dentry->d_count), *base, len, path);
1302 return path;
1303}
1304
1305static int build_dentry_path(struct dentry *dentry,
1306 const char **ppath, int *ppathlen, u64 *pino,
1307 int *pfreepath)
1308{
1309 char *path;
1310
1311 if (ceph_snap(dentry->d_parent->d_inode) == CEPH_NOSNAP) {
1312 *pino = ceph_ino(dentry->d_parent->d_inode);
1313 *ppath = dentry->d_name.name;
1314 *ppathlen = dentry->d_name.len;
1315 return 0;
1316 }
1317 path = ceph_mdsc_build_path(dentry, ppathlen, pino, 1);
1318 if (IS_ERR(path))
1319 return PTR_ERR(path);
1320 *ppath = path;
1321 *pfreepath = 1;
1322 return 0;
1323}
1324
1325static int build_inode_path(struct inode *inode,
1326 const char **ppath, int *ppathlen, u64 *pino,
1327 int *pfreepath)
1328{
1329 struct dentry *dentry;
1330 char *path;
1331
1332 if (ceph_snap(inode) == CEPH_NOSNAP) {
1333 *pino = ceph_ino(inode);
1334 *ppathlen = 0;
1335 return 0;
1336 }
1337 dentry = d_find_alias(inode);
1338 path = ceph_mdsc_build_path(dentry, ppathlen, pino, 1);
1339 dput(dentry);
1340 if (IS_ERR(path))
1341 return PTR_ERR(path);
1342 *ppath = path;
1343 *pfreepath = 1;
1344 return 0;
1345}
1346
1347/*
1348 * request arguments may be specified via an inode *, a dentry *, or
1349 * an explicit ino+path.
1350 */
1351static int set_request_path_attr(struct inode *rinode, struct dentry *rdentry,
1352 const char *rpath, u64 rino,
1353 const char **ppath, int *pathlen,
1354 u64 *ino, int *freepath)
1355{
1356 int r = 0;
1357
1358 if (rinode) {
1359 r = build_inode_path(rinode, ppath, pathlen, ino, freepath);
1360 dout(" inode %p %llx.%llx\n", rinode, ceph_ino(rinode),
1361 ceph_snap(rinode));
1362 } else if (rdentry) {
1363 r = build_dentry_path(rdentry, ppath, pathlen, ino, freepath);
1364 dout(" dentry %p %llx/%.*s\n", rdentry, *ino, *pathlen,
1365 *ppath);
1366 } else if (rpath) {
1367 *ino = rino;
1368 *ppath = rpath;
1369 *pathlen = strlen(rpath);
1370 dout(" path %.*s\n", *pathlen, rpath);
1371 }
1372
1373 return r;
1374}
1375
1376/*
1377 * called under mdsc->mutex
1378 */
1379static struct ceph_msg *create_request_message(struct ceph_mds_client *mdsc,
1380 struct ceph_mds_request *req,
1381 int mds)
1382{
1383 struct ceph_msg *msg;
1384 struct ceph_mds_request_head *head;
1385 const char *path1 = NULL;
1386 const char *path2 = NULL;
1387 u64 ino1 = 0, ino2 = 0;
1388 int pathlen1 = 0, pathlen2 = 0;
1389 int freepath1 = 0, freepath2 = 0;
1390 int len;
1391 u16 releases;
1392 void *p, *end;
1393 int ret;
1394
1395 ret = set_request_path_attr(req->r_inode, req->r_dentry,
1396 req->r_path1, req->r_ino1.ino,
1397 &path1, &pathlen1, &ino1, &freepath1);
1398 if (ret < 0) {
1399 msg = ERR_PTR(ret);
1400 goto out;
1401 }
1402
1403 ret = set_request_path_attr(NULL, req->r_old_dentry,
1404 req->r_path2, req->r_ino2.ino,
1405 &path2, &pathlen2, &ino2, &freepath2);
1406 if (ret < 0) {
1407 msg = ERR_PTR(ret);
1408 goto out_free1;
1409 }
1410
1411 len = sizeof(*head) +
1412 pathlen1 + pathlen2 + 2*(1 + sizeof(u32) + sizeof(u64));
1413
1414 /* calculate (max) length for cap releases */
1415 len += sizeof(struct ceph_mds_request_release) *
1416 (!!req->r_inode_drop + !!req->r_dentry_drop +
1417 !!req->r_old_inode_drop + !!req->r_old_dentry_drop);
1418 if (req->r_dentry_drop)
1419 len += req->r_dentry->d_name.len;
1420 if (req->r_old_dentry_drop)
1421 len += req->r_old_dentry->d_name.len;
1422
1423 msg = ceph_msg_new(CEPH_MSG_CLIENT_REQUEST, len, 0, 0, NULL);
1424 if (IS_ERR(msg))
1425 goto out_free2;
1426
1427 msg->hdr.tid = cpu_to_le64(req->r_tid);
1428
1429 head = msg->front.iov_base;
1430 p = msg->front.iov_base + sizeof(*head);
1431 end = msg->front.iov_base + msg->front.iov_len;
1432
1433 head->mdsmap_epoch = cpu_to_le32(mdsc->mdsmap->m_epoch);
1434 head->op = cpu_to_le32(req->r_op);
1435 head->caller_uid = cpu_to_le32(current_fsuid());
1436 head->caller_gid = cpu_to_le32(current_fsgid());
1437 head->args = req->r_args;
1438
1439 ceph_encode_filepath(&p, end, ino1, path1);
1440 ceph_encode_filepath(&p, end, ino2, path2);
1441
1442 /* cap releases */
1443 releases = 0;
1444 if (req->r_inode_drop)
1445 releases += ceph_encode_inode_release(&p,
1446 req->r_inode ? req->r_inode : req->r_dentry->d_inode,
1447 mds, req->r_inode_drop, req->r_inode_unless, 0);
1448 if (req->r_dentry_drop)
1449 releases += ceph_encode_dentry_release(&p, req->r_dentry,
1450 mds, req->r_dentry_drop, req->r_dentry_unless);
1451 if (req->r_old_dentry_drop)
1452 releases += ceph_encode_dentry_release(&p, req->r_old_dentry,
1453 mds, req->r_old_dentry_drop, req->r_old_dentry_unless);
1454 if (req->r_old_inode_drop)
1455 releases += ceph_encode_inode_release(&p,
1456 req->r_old_dentry->d_inode,
1457 mds, req->r_old_inode_drop, req->r_old_inode_unless, 0);
1458 head->num_releases = cpu_to_le16(releases);
1459
1460 BUG_ON(p > end);
1461 msg->front.iov_len = p - msg->front.iov_base;
1462 msg->hdr.front_len = cpu_to_le32(msg->front.iov_len);
1463
1464 msg->pages = req->r_pages;
1465 msg->nr_pages = req->r_num_pages;
1466 msg->hdr.data_len = cpu_to_le32(req->r_data_len);
1467 msg->hdr.data_off = cpu_to_le16(0);
1468
1469out_free2:
1470 if (freepath2)
1471 kfree((char *)path2);
1472out_free1:
1473 if (freepath1)
1474 kfree((char *)path1);
1475out:
1476 return msg;
1477}
1478
1479/*
1480 * called under mdsc->mutex if error, under no mutex if
1481 * success.
1482 */
1483static void complete_request(struct ceph_mds_client *mdsc,
1484 struct ceph_mds_request *req)
1485{
1486 if (req->r_callback)
1487 req->r_callback(mdsc, req);
1488 else
1489 complete(&req->r_completion);
1490}
1491
1492/*
1493 * called under mdsc->mutex
1494 */
1495static int __prepare_send_request(struct ceph_mds_client *mdsc,
1496 struct ceph_mds_request *req,
1497 int mds)
1498{
1499 struct ceph_mds_request_head *rhead;
1500 struct ceph_msg *msg;
1501 int flags = 0;
1502
1503 req->r_mds = mds;
1504 req->r_attempts++;
1505 dout("prepare_send_request %p tid %lld %s (attempt %d)\n", req,
1506 req->r_tid, ceph_mds_op_name(req->r_op), req->r_attempts);
1507
1508 if (req->r_request) {
1509 ceph_msg_put(req->r_request);
1510 req->r_request = NULL;
1511 }
1512 msg = create_request_message(mdsc, req, mds);
1513 if (IS_ERR(msg)) {
1514 req->r_reply = ERR_PTR(PTR_ERR(msg));
1515 complete_request(mdsc, req);
1516 return -PTR_ERR(msg);
1517 }
1518 req->r_request = msg;
1519
1520 rhead = msg->front.iov_base;
1521 rhead->oldest_client_tid = cpu_to_le64(__get_oldest_tid(mdsc));
1522 if (req->r_got_unsafe)
1523 flags |= CEPH_MDS_FLAG_REPLAY;
1524 if (req->r_locked_dir)
1525 flags |= CEPH_MDS_FLAG_WANT_DENTRY;
1526 rhead->flags = cpu_to_le32(flags);
1527 rhead->num_fwd = req->r_num_fwd;
1528 rhead->num_retry = req->r_attempts - 1;
1529
1530 dout(" r_locked_dir = %p\n", req->r_locked_dir);
1531
1532 if (req->r_target_inode && req->r_got_unsafe)
1533 rhead->ino = cpu_to_le64(ceph_ino(req->r_target_inode));
1534 else
1535 rhead->ino = 0;
1536 return 0;
1537}
1538
1539/*
1540 * send request, or put it on the appropriate wait list.
1541 */
1542static int __do_request(struct ceph_mds_client *mdsc,
1543 struct ceph_mds_request *req)
1544{
1545 struct ceph_mds_session *session = NULL;
1546 int mds = -1;
1547 int err = -EAGAIN;
1548
1549 if (req->r_reply)
1550 goto out;
1551
1552 if (req->r_timeout &&
1553 time_after_eq(jiffies, req->r_started + req->r_timeout)) {
1554 dout("do_request timed out\n");
1555 err = -EIO;
1556 goto finish;
1557 }
1558
1559 mds = __choose_mds(mdsc, req);
1560 if (mds < 0 ||
1561 ceph_mdsmap_get_state(mdsc->mdsmap, mds) < CEPH_MDS_STATE_ACTIVE) {
1562 dout("do_request no mds or not active, waiting for map\n");
1563 list_add(&req->r_wait, &mdsc->waiting_for_map);
1564 goto out;
1565 }
1566
1567 /* get, open session */
1568 session = __ceph_lookup_mds_session(mdsc, mds);
1569 if (!session)
1570 session = register_session(mdsc, mds);
1571 dout("do_request mds%d session %p state %s\n", mds, session,
1572 session_state_name(session->s_state));
1573 if (session->s_state != CEPH_MDS_SESSION_OPEN &&
1574 session->s_state != CEPH_MDS_SESSION_HUNG) {
1575 if (session->s_state == CEPH_MDS_SESSION_NEW ||
1576 session->s_state == CEPH_MDS_SESSION_CLOSING)
1577 __open_session(mdsc, session);
1578 list_add(&req->r_wait, &session->s_waiting);
1579 goto out_session;
1580 }
1581
1582 /* send request */
1583 req->r_session = get_session(session);
1584 req->r_resend_mds = -1; /* forget any previous mds hint */
1585
1586 if (req->r_request_started == 0) /* note request start time */
1587 req->r_request_started = jiffies;
1588
1589 err = __prepare_send_request(mdsc, req, mds);
1590 if (!err) {
1591 ceph_msg_get(req->r_request);
1592 ceph_con_send(&session->s_con, req->r_request);
1593 }
1594
1595out_session:
1596 ceph_put_mds_session(session);
1597out:
1598 return err;
1599
1600finish:
1601 req->r_reply = ERR_PTR(err);
1602 complete_request(mdsc, req);
1603 goto out;
1604}
1605
1606/*
1607 * called under mdsc->mutex
1608 */
1609static void __wake_requests(struct ceph_mds_client *mdsc,
1610 struct list_head *head)
1611{
1612 struct ceph_mds_request *req, *nreq;
1613
1614 list_for_each_entry_safe(req, nreq, head, r_wait) {
1615 list_del_init(&req->r_wait);
1616 __do_request(mdsc, req);
1617 }
1618}
1619
1620/*
1621 * Wake up threads with requests pending for @mds, so that they can
1622 * resubmit their requests to a possibly different mds. If @all is set,
1623 * wake up if their requests has been forwarded to @mds, too.
1624 */
1625static void kick_requests(struct ceph_mds_client *mdsc, int mds, int all)
1626{
1627 struct ceph_mds_request *req;
1628 struct rb_node *p;
1629
1630 dout("kick_requests mds%d\n", mds);
1631 for (p = rb_first(&mdsc->request_tree); p; p = rb_next(p)) {
1632 req = rb_entry(p, struct ceph_mds_request, r_node);
1633 if (req->r_got_unsafe)
1634 continue;
1635 if (req->r_session &&
1636 req->r_session->s_mds == mds) {
1637 dout(" kicking tid %llu\n", req->r_tid);
1638 put_request_session(req);
1639 __do_request(mdsc, req);
1640 }
1641 }
1642}
1643
1644void ceph_mdsc_submit_request(struct ceph_mds_client *mdsc,
1645 struct ceph_mds_request *req)
1646{
1647 dout("submit_request on %p\n", req);
1648 mutex_lock(&mdsc->mutex);
1649 __register_request(mdsc, req, NULL);
1650 __do_request(mdsc, req);
1651 mutex_unlock(&mdsc->mutex);
1652}
1653
1654/*
1655 * Synchrously perform an mds request. Take care of all of the
1656 * session setup, forwarding, retry details.
1657 */
1658int ceph_mdsc_do_request(struct ceph_mds_client *mdsc,
1659 struct inode *dir,
1660 struct ceph_mds_request *req)
1661{
1662 int err;
1663
1664 dout("do_request on %p\n", req);
1665
1666 /* take CAP_PIN refs for r_inode, r_locked_dir, r_old_dentry */
1667 if (req->r_inode)
1668 ceph_get_cap_refs(ceph_inode(req->r_inode), CEPH_CAP_PIN);
1669 if (req->r_locked_dir)
1670 ceph_get_cap_refs(ceph_inode(req->r_locked_dir), CEPH_CAP_PIN);
1671 if (req->r_old_dentry)
1672 ceph_get_cap_refs(
1673 ceph_inode(req->r_old_dentry->d_parent->d_inode),
1674 CEPH_CAP_PIN);
1675
1676 /* issue */
1677 mutex_lock(&mdsc->mutex);
1678 __register_request(mdsc, req, dir);
1679 __do_request(mdsc, req);
1680
1681 /* wait */
1682 if (!req->r_reply) {
1683 mutex_unlock(&mdsc->mutex);
1684 if (req->r_timeout) {
1685 err = (long)wait_for_completion_interruptible_timeout(
1686 &req->r_completion, req->r_timeout);
1687 if (err == 0)
1688 req->r_reply = ERR_PTR(-EIO);
1689 else if (err < 0)
1690 req->r_reply = ERR_PTR(err);
1691 } else {
1692 err = wait_for_completion_interruptible(
1693 &req->r_completion);
1694 if (err)
1695 req->r_reply = ERR_PTR(err);
1696 }
1697 mutex_lock(&mdsc->mutex);
1698 }
1699
1700 if (IS_ERR(req->r_reply)) {
1701 err = PTR_ERR(req->r_reply);
1702 req->r_reply = NULL;
1703
1704 if (err == -ERESTARTSYS) {
1705 /* aborted */
1706 req->r_aborted = true;
1707
1708 if (req->r_locked_dir &&
1709 (req->r_op & CEPH_MDS_OP_WRITE)) {
1710 struct ceph_inode_info *ci =
1711 ceph_inode(req->r_locked_dir);
1712
1713 dout("aborted, clearing I_COMPLETE on %p\n",
1714 req->r_locked_dir);
1715 spin_lock(&req->r_locked_dir->i_lock);
1716 ci->i_ceph_flags &= ~CEPH_I_COMPLETE;
1717 ci->i_release_count++;
1718 spin_unlock(&req->r_locked_dir->i_lock);
1719 }
1720 } else {
1721 /* clean up this request */
1722 __unregister_request(mdsc, req);
1723 if (!list_empty(&req->r_unsafe_item))
1724 list_del_init(&req->r_unsafe_item);
1725 complete(&req->r_safe_completion);
1726 }
1727 } else if (req->r_err) {
1728 err = req->r_err;
1729 } else {
1730 err = le32_to_cpu(req->r_reply_info.head->result);
1731 }
1732 mutex_unlock(&mdsc->mutex);
1733
1734 dout("do_request %p done, result %d\n", req, err);
1735 return err;
1736}
1737
1738/*
1739 * Handle mds reply.
1740 *
1741 * We take the session mutex and parse and process the reply immediately.
1742 * This preserves the logical ordering of replies, capabilities, etc., sent
1743 * by the MDS as they are applied to our local cache.
1744 */
1745static void handle_reply(struct ceph_mds_session *session, struct ceph_msg *msg)
1746{
1747 struct ceph_mds_client *mdsc = session->s_mdsc;
1748 struct ceph_mds_request *req;
1749 struct ceph_mds_reply_head *head = msg->front.iov_base;
1750 struct ceph_mds_reply_info_parsed *rinfo; /* parsed reply info */
1751 u64 tid;
1752 int err, result;
1753 int mds = session->s_mds;
1754
1755 if (msg->front.iov_len < sizeof(*head)) {
1756 pr_err("mdsc_handle_reply got corrupt (short) reply\n");
1757 ceph_msg_dump(msg);
1758 return;
1759 }
1760
1761 /* get request, session */
1762 tid = le64_to_cpu(msg->hdr.tid);
1763 mutex_lock(&mdsc->mutex);
1764 req = __lookup_request(mdsc, tid);
1765 if (!req) {
1766 dout("handle_reply on unknown tid %llu\n", tid);
1767 mutex_unlock(&mdsc->mutex);
1768 return;
1769 }
1770 dout("handle_reply %p\n", req);
1771
1772 /* correct session? */
1773 if (!req->r_session && req->r_session != session) {
1774 pr_err("mdsc_handle_reply got %llu on session mds%d"
1775 " not mds%d\n", tid, session->s_mds,
1776 req->r_session ? req->r_session->s_mds : -1);
1777 mutex_unlock(&mdsc->mutex);
1778 goto out;
1779 }
1780
1781 /* dup? */
1782 if ((req->r_got_unsafe && !head->safe) ||
1783 (req->r_got_safe && head->safe)) {
1784 pr_warning("got a dup %s reply on %llu from mds%d\n",
1785 head->safe ? "safe" : "unsafe", tid, mds);
1786 mutex_unlock(&mdsc->mutex);
1787 goto out;
1788 }
1789
1790 result = le32_to_cpu(head->result);
1791
1792 /*
1793 * Tolerate 2 consecutive ESTALEs from the same mds.
1794 * FIXME: we should be looking at the cap migrate_seq.
1795 */
1796 if (result == -ESTALE) {
1797 req->r_direct_mode = USE_AUTH_MDS;
1798 req->r_num_stale++;
1799 if (req->r_num_stale <= 2) {
1800 __do_request(mdsc, req);
1801 mutex_unlock(&mdsc->mutex);
1802 goto out;
1803 }
1804 } else {
1805 req->r_num_stale = 0;
1806 }
1807
1808 if (head->safe) {
1809 req->r_got_safe = true;
1810 __unregister_request(mdsc, req);
1811 complete(&req->r_safe_completion);
1812
1813 if (req->r_got_unsafe) {
1814 /*
1815 * We already handled the unsafe response, now do the
1816 * cleanup. No need to examine the response; the MDS
1817 * doesn't include any result info in the safe
1818 * response. And even if it did, there is nothing
1819 * useful we could do with a revised return value.
1820 */
1821 dout("got safe reply %llu, mds%d\n", tid, mds);
1822 list_del_init(&req->r_unsafe_item);
1823
1824 /* last unsafe request during umount? */
1825 if (mdsc->stopping && !__get_oldest_req(mdsc))
1826 complete(&mdsc->safe_umount_waiters);
1827 mutex_unlock(&mdsc->mutex);
1828 goto out;
1829 }
1830 }
1831
1832 BUG_ON(req->r_reply);
1833
1834 if (!head->safe) {
1835 req->r_got_unsafe = true;
1836 list_add_tail(&req->r_unsafe_item, &req->r_session->s_unsafe);
1837 }
1838
1839 dout("handle_reply tid %lld result %d\n", tid, result);
1840 rinfo = &req->r_reply_info;
1841 err = parse_reply_info(msg, rinfo);
1842 mutex_unlock(&mdsc->mutex);
1843
1844 mutex_lock(&session->s_mutex);
1845 if (err < 0) {
1846 pr_err("mdsc_handle_reply got corrupt reply mds%d\n", mds);
1847 ceph_msg_dump(msg);
1848 goto out_err;
1849 }
1850
1851 /* snap trace */
1852 if (rinfo->snapblob_len) {
1853 down_write(&mdsc->snap_rwsem);
1854 ceph_update_snap_trace(mdsc, rinfo->snapblob,
1855 rinfo->snapblob + rinfo->snapblob_len,
1856 le32_to_cpu(head->op) == CEPH_MDS_OP_RMSNAP);
1857 downgrade_write(&mdsc->snap_rwsem);
1858 } else {
1859 down_read(&mdsc->snap_rwsem);
1860 }
1861
1862 /* insert trace into our cache */
1863 err = ceph_fill_trace(mdsc->client->sb, req, req->r_session);
1864 if (err == 0) {
1865 if (result == 0 && rinfo->dir_nr)
1866 ceph_readdir_prepopulate(req, req->r_session);
1867 ceph_unreserve_caps(&req->r_caps_reservation);
1868 }
1869
1870 up_read(&mdsc->snap_rwsem);
1871out_err:
1872 if (err) {
1873 req->r_err = err;
1874 } else {
1875 req->r_reply = msg;
1876 ceph_msg_get(msg);
1877 }
1878
1879 add_cap_releases(mdsc, req->r_session, -1);
1880 mutex_unlock(&session->s_mutex);
1881
1882 /* kick calling process */
1883 complete_request(mdsc, req);
1884out:
1885 ceph_mdsc_put_request(req);
1886 return;
1887}
1888
1889
1890
1891/*
1892 * handle mds notification that our request has been forwarded.
1893 */
1894static void handle_forward(struct ceph_mds_client *mdsc,
1895 struct ceph_mds_session *session,
1896 struct ceph_msg *msg)
1897{
1898 struct ceph_mds_request *req;
1899 u64 tid = le64_to_cpu(msg->hdr.tid);
1900 u32 next_mds;
1901 u32 fwd_seq;
1902 int err = -EINVAL;
1903 void *p = msg->front.iov_base;
1904 void *end = p + msg->front.iov_len;
1905
1906 ceph_decode_need(&p, end, 2*sizeof(u32), bad);
1907 next_mds = ceph_decode_32(&p);
1908 fwd_seq = ceph_decode_32(&p);
1909
1910 mutex_lock(&mdsc->mutex);
1911 req = __lookup_request(mdsc, tid);
1912 if (!req) {
1913 dout("forward %llu to mds%d - req dne\n", tid, next_mds);
1914 goto out; /* dup reply? */
1915 }
1916
1917 if (fwd_seq <= req->r_num_fwd) {
1918 dout("forward %llu to mds%d - old seq %d <= %d\n",
1919 tid, next_mds, req->r_num_fwd, fwd_seq);
1920 } else {
1921 /* resend. forward race not possible; mds would drop */
1922 dout("forward %llu to mds%d (we resend)\n", tid, next_mds);
1923 req->r_num_fwd = fwd_seq;
1924 req->r_resend_mds = next_mds;
1925 put_request_session(req);
1926 __do_request(mdsc, req);
1927 }
1928 ceph_mdsc_put_request(req);
1929out:
1930 mutex_unlock(&mdsc->mutex);
1931 return;
1932
1933bad:
1934 pr_err("mdsc_handle_forward decode error err=%d\n", err);
1935}
1936
1937/*
1938 * handle a mds session control message
1939 */
1940static void handle_session(struct ceph_mds_session *session,
1941 struct ceph_msg *msg)
1942{
1943 struct ceph_mds_client *mdsc = session->s_mdsc;
1944 u32 op;
1945 u64 seq;
1946 int mds = session->s_mds;
1947 struct ceph_mds_session_head *h = msg->front.iov_base;
1948 int wake = 0;
1949
1950 /* decode */
1951 if (msg->front.iov_len != sizeof(*h))
1952 goto bad;
1953 op = le32_to_cpu(h->op);
1954 seq = le64_to_cpu(h->seq);
1955
1956 mutex_lock(&mdsc->mutex);
1957 if (op == CEPH_SESSION_CLOSE)
1958 __unregister_session(mdsc, session);
1959 /* FIXME: this ttl calculation is generous */
1960 session->s_ttl = jiffies + HZ*mdsc->mdsmap->m_session_autoclose;
1961 mutex_unlock(&mdsc->mutex);
1962
1963 mutex_lock(&session->s_mutex);
1964
1965 dout("handle_session mds%d %s %p state %s seq %llu\n",
1966 mds, ceph_session_op_name(op), session,
1967 session_state_name(session->s_state), seq);
1968
1969 if (session->s_state == CEPH_MDS_SESSION_HUNG) {
1970 session->s_state = CEPH_MDS_SESSION_OPEN;
1971 pr_info("mds%d came back\n", session->s_mds);
1972 }
1973
1974 switch (op) {
1975 case CEPH_SESSION_OPEN:
1976 session->s_state = CEPH_MDS_SESSION_OPEN;
1977 renewed_caps(mdsc, session, 0);
1978 wake = 1;
1979 if (mdsc->stopping)
1980 __close_session(mdsc, session);
1981 break;
1982
1983 case CEPH_SESSION_RENEWCAPS:
1984 if (session->s_renew_seq == seq)
1985 renewed_caps(mdsc, session, 1);
1986 break;
1987
1988 case CEPH_SESSION_CLOSE:
1989 remove_session_caps(session);
1990 wake = 1; /* for good measure */
1991 complete(&mdsc->session_close_waiters);
1992 kick_requests(mdsc, mds, 0); /* cur only */
1993 break;
1994
1995 case CEPH_SESSION_STALE:
1996 pr_info("mds%d caps went stale, renewing\n",
1997 session->s_mds);
1998 spin_lock(&session->s_cap_lock);
1999 session->s_cap_gen++;
2000 session->s_cap_ttl = 0;
2001 spin_unlock(&session->s_cap_lock);
2002 send_renew_caps(mdsc, session);
2003 break;
2004
2005 case CEPH_SESSION_RECALL_STATE:
2006 trim_caps(mdsc, session, le32_to_cpu(h->max_caps));
2007 break;
2008
2009 default:
2010 pr_err("mdsc_handle_session bad op %d mds%d\n", op, mds);
2011 WARN_ON(1);
2012 }
2013
2014 mutex_unlock(&session->s_mutex);
2015 if (wake) {
2016 mutex_lock(&mdsc->mutex);
2017 __wake_requests(mdsc, &session->s_waiting);
2018 mutex_unlock(&mdsc->mutex);
2019 }
2020 return;
2021
2022bad:
2023 pr_err("mdsc_handle_session corrupt message mds%d len %d\n", mds,
2024 (int)msg->front.iov_len);
2025 ceph_msg_dump(msg);
2026 return;
2027}
2028
2029
2030/*
2031 * called under session->mutex.
2032 */
2033static void replay_unsafe_requests(struct ceph_mds_client *mdsc,
2034 struct ceph_mds_session *session)
2035{
2036 struct ceph_mds_request *req, *nreq;
2037 int err;
2038
2039 dout("replay_unsafe_requests mds%d\n", session->s_mds);
2040
2041 mutex_lock(&mdsc->mutex);
2042 list_for_each_entry_safe(req, nreq, &session->s_unsafe, r_unsafe_item) {
2043 err = __prepare_send_request(mdsc, req, session->s_mds);
2044 if (!err) {
2045 ceph_msg_get(req->r_request);
2046 ceph_con_send(&session->s_con, req->r_request);
2047 }
2048 }
2049 mutex_unlock(&mdsc->mutex);
2050}
2051
2052/*
2053 * Encode information about a cap for a reconnect with the MDS.
2054 */
2055static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap,
2056 void *arg)
2057{
2058 struct ceph_mds_cap_reconnect rec;
2059 struct ceph_inode_info *ci;
2060 struct ceph_pagelist *pagelist = arg;
2061 char *path;
2062 int pathlen, err;
2063 u64 pathbase;
2064 struct dentry *dentry;
2065
2066 ci = cap->ci;
2067
2068 dout(" adding %p ino %llx.%llx cap %p %lld %s\n",
2069 inode, ceph_vinop(inode), cap, cap->cap_id,
2070 ceph_cap_string(cap->issued));
2071 err = ceph_pagelist_encode_64(pagelist, ceph_ino(inode));
2072 if (err)
2073 return err;
2074
2075 dentry = d_find_alias(inode);
2076 if (dentry) {
2077 path = ceph_mdsc_build_path(dentry, &pathlen, &pathbase, 0);
2078 if (IS_ERR(path)) {
2079 err = PTR_ERR(path);
2080 BUG_ON(err);
2081 }
2082 } else {
2083 path = NULL;
2084 pathlen = 0;
2085 }
2086 err = ceph_pagelist_encode_string(pagelist, path, pathlen);
2087 if (err)
2088 goto out;
2089
2090 spin_lock(&inode->i_lock);
2091 cap->seq = 0; /* reset cap seq */
2092 cap->issue_seq = 0; /* and issue_seq */
2093 rec.cap_id = cpu_to_le64(cap->cap_id);
2094 rec.pathbase = cpu_to_le64(pathbase);
2095 rec.wanted = cpu_to_le32(__ceph_caps_wanted(ci));
2096 rec.issued = cpu_to_le32(cap->issued);
2097 rec.size = cpu_to_le64(inode->i_size);
2098 ceph_encode_timespec(&rec.mtime, &inode->i_mtime);
2099 ceph_encode_timespec(&rec.atime, &inode->i_atime);
2100 rec.snaprealm = cpu_to_le64(ci->i_snap_realm->ino);
2101 spin_unlock(&inode->i_lock);
2102
2103 err = ceph_pagelist_append(pagelist, &rec, sizeof(rec));
2104
2105out:
2106 kfree(path);
2107 dput(dentry);
2108 return err;
2109}
2110
2111
2112/*
2113 * If an MDS fails and recovers, clients need to reconnect in order to
2114 * reestablish shared state. This includes all caps issued through
2115 * this session _and_ the snap_realm hierarchy. Because it's not
2116 * clear which snap realms the mds cares about, we send everything we
2117 * know about.. that ensures we'll then get any new info the
2118 * recovering MDS might have.
2119 *
2120 * This is a relatively heavyweight operation, but it's rare.
2121 *
2122 * called with mdsc->mutex held.
2123 */
2124static void send_mds_reconnect(struct ceph_mds_client *mdsc, int mds)
2125{
2126 struct ceph_mds_session *session = NULL;
2127 struct ceph_msg *reply;
2128 struct rb_node *p;
2129 int err;
2130 struct ceph_pagelist *pagelist;
2131
2132 pr_info("reconnect to recovering mds%d\n", mds);
2133
2134 pagelist = kmalloc(sizeof(*pagelist), GFP_NOFS);
2135 if (!pagelist)
2136 goto fail_nopagelist;
2137 ceph_pagelist_init(pagelist);
2138
2139 reply = ceph_msg_new(CEPH_MSG_CLIENT_RECONNECT, 0, 0, 0, NULL);
2140 if (IS_ERR(reply)) {
2141 err = PTR_ERR(reply);
2142 goto fail_nomsg;
2143 }
2144
2145 /* find session */
2146 session = __ceph_lookup_mds_session(mdsc, mds);
2147 mutex_unlock(&mdsc->mutex); /* drop lock for duration */
2148
2149 if (session) {
2150 mutex_lock(&session->s_mutex);
2151
2152 session->s_state = CEPH_MDS_SESSION_RECONNECTING;
2153 session->s_seq = 0;
2154
2155 ceph_con_open(&session->s_con,
2156 ceph_mdsmap_get_addr(mdsc->mdsmap, mds));
2157
2158 /* replay unsafe requests */
2159 replay_unsafe_requests(mdsc, session);
2160 } else {
2161 dout("no session for mds%d, will send short reconnect\n",
2162 mds);
2163 }
2164
2165 down_read(&mdsc->snap_rwsem);
2166
2167 if (!session)
2168 goto send;
2169 dout("session %p state %s\n", session,
2170 session_state_name(session->s_state));
2171
2172 /* traverse this session's caps */
2173 err = ceph_pagelist_encode_32(pagelist, session->s_nr_caps);
2174 if (err)
2175 goto fail;
2176 err = iterate_session_caps(session, encode_caps_cb, pagelist);
2177 if (err < 0)
2178 goto out;
2179
2180 /*
2181 * snaprealms. we provide mds with the ino, seq (version), and
2182 * parent for all of our realms. If the mds has any newer info,
2183 * it will tell us.
2184 */
2185 for (p = rb_first(&mdsc->snap_realms); p; p = rb_next(p)) {
2186 struct ceph_snap_realm *realm =
2187 rb_entry(p, struct ceph_snap_realm, node);
2188 struct ceph_mds_snaprealm_reconnect sr_rec;
2189
2190 dout(" adding snap realm %llx seq %lld parent %llx\n",
2191 realm->ino, realm->seq, realm->parent_ino);
2192 sr_rec.ino = cpu_to_le64(realm->ino);
2193 sr_rec.seq = cpu_to_le64(realm->seq);
2194 sr_rec.parent = cpu_to_le64(realm->parent_ino);
2195 err = ceph_pagelist_append(pagelist, &sr_rec, sizeof(sr_rec));
2196 if (err)
2197 goto fail;
2198 }
2199
2200send:
2201 reply->pagelist = pagelist;
2202 reply->hdr.data_len = cpu_to_le32(pagelist->length);
2203 reply->nr_pages = calc_pages_for(0, pagelist->length);
2204 ceph_con_send(&session->s_con, reply);
2205
2206 if (session) {
2207 session->s_state = CEPH_MDS_SESSION_OPEN;
2208 __wake_requests(mdsc, &session->s_waiting);
2209 }
2210
2211out:
2212 up_read(&mdsc->snap_rwsem);
2213 if (session) {
2214 mutex_unlock(&session->s_mutex);
2215 ceph_put_mds_session(session);
2216 }
2217 mutex_lock(&mdsc->mutex);
2218 return;
2219
2220fail:
2221 ceph_msg_put(reply);
2222fail_nomsg:
2223 ceph_pagelist_release(pagelist);
2224 kfree(pagelist);
2225fail_nopagelist:
2226 pr_err("ENOMEM preparing reconnect for mds%d\n", mds);
2227 goto out;
2228}
2229
2230
2231/*
2232 * compare old and new mdsmaps, kicking requests
2233 * and closing out old connections as necessary
2234 *
2235 * called under mdsc->mutex.
2236 */
2237static void check_new_map(struct ceph_mds_client *mdsc,
2238 struct ceph_mdsmap *newmap,
2239 struct ceph_mdsmap *oldmap)
2240{
2241 int i;
2242 int oldstate, newstate;
2243 struct ceph_mds_session *s;
2244
2245 dout("check_new_map new %u old %u\n",
2246 newmap->m_epoch, oldmap->m_epoch);
2247
2248 for (i = 0; i < oldmap->m_max_mds && i < mdsc->max_sessions; i++) {
2249 if (mdsc->sessions[i] == NULL)
2250 continue;
2251 s = mdsc->sessions[i];
2252 oldstate = ceph_mdsmap_get_state(oldmap, i);
2253 newstate = ceph_mdsmap_get_state(newmap, i);
2254
2255 dout("check_new_map mds%d state %s -> %s (session %s)\n",
2256 i, ceph_mds_state_name(oldstate),
2257 ceph_mds_state_name(newstate),
2258 session_state_name(s->s_state));
2259
2260 if (memcmp(ceph_mdsmap_get_addr(oldmap, i),
2261 ceph_mdsmap_get_addr(newmap, i),
2262 sizeof(struct ceph_entity_addr))) {
2263 if (s->s_state == CEPH_MDS_SESSION_OPENING) {
2264 /* the session never opened, just close it
2265 * out now */
2266 __wake_requests(mdsc, &s->s_waiting);
2267 __unregister_session(mdsc, s);
2268 } else {
2269 /* just close it */
2270 mutex_unlock(&mdsc->mutex);
2271 mutex_lock(&s->s_mutex);
2272 mutex_lock(&mdsc->mutex);
2273 ceph_con_close(&s->s_con);
2274 mutex_unlock(&s->s_mutex);
2275 s->s_state = CEPH_MDS_SESSION_RESTARTING;
2276 }
2277
2278 /* kick any requests waiting on the recovering mds */
2279 kick_requests(mdsc, i, 1);
2280 } else if (oldstate == newstate) {
2281 continue; /* nothing new with this mds */
2282 }
2283
2284 /*
2285 * send reconnect?
2286 */
2287 if (s->s_state == CEPH_MDS_SESSION_RESTARTING &&
2288 newstate >= CEPH_MDS_STATE_RECONNECT)
2289 send_mds_reconnect(mdsc, i);
2290
2291 /*
2292 * kick requests on any mds that has gone active.
2293 *
2294 * kick requests on cur or forwarder: we may have sent
2295 * the request to mds1, mds1 told us it forwarded it
2296 * to mds2, but then we learn mds1 failed and can't be
2297 * sure it successfully forwarded our request before
2298 * it died.
2299 */
2300 if (oldstate < CEPH_MDS_STATE_ACTIVE &&
2301 newstate >= CEPH_MDS_STATE_ACTIVE) {
2302 pr_info("mds%d reconnect completed\n", s->s_mds);
2303 kick_requests(mdsc, i, 1);
2304 ceph_kick_flushing_caps(mdsc, s);
2305 wake_up_session_caps(s, 1);
2306 }
2307 }
2308}
2309
2310
2311
2312/*
2313 * leases
2314 */
2315
2316/*
2317 * caller must hold session s_mutex, dentry->d_lock
2318 */
2319void __ceph_mdsc_drop_dentry_lease(struct dentry *dentry)
2320{
2321 struct ceph_dentry_info *di = ceph_dentry(dentry);
2322
2323 ceph_put_mds_session(di->lease_session);
2324 di->lease_session = NULL;
2325}
2326
2327static void handle_lease(struct ceph_mds_client *mdsc,
2328 struct ceph_mds_session *session,
2329 struct ceph_msg *msg)
2330{
2331 struct super_block *sb = mdsc->client->sb;
2332 struct inode *inode;
2333 struct ceph_inode_info *ci;
2334 struct dentry *parent, *dentry;
2335 struct ceph_dentry_info *di;
2336 int mds = session->s_mds;
2337 struct ceph_mds_lease *h = msg->front.iov_base;
2338 struct ceph_vino vino;
2339 int mask;
2340 struct qstr dname;
2341 int release = 0;
2342
2343 dout("handle_lease from mds%d\n", mds);
2344
2345 /* decode */
2346 if (msg->front.iov_len < sizeof(*h) + sizeof(u32))
2347 goto bad;
2348 vino.ino = le64_to_cpu(h->ino);
2349 vino.snap = CEPH_NOSNAP;
2350 mask = le16_to_cpu(h->mask);
2351 dname.name = (void *)h + sizeof(*h) + sizeof(u32);
2352 dname.len = msg->front.iov_len - sizeof(*h) - sizeof(u32);
2353 if (dname.len != get_unaligned_le32(h+1))
2354 goto bad;
2355
2356 mutex_lock(&session->s_mutex);
2357 session->s_seq++;
2358
2359 /* lookup inode */
2360 inode = ceph_find_inode(sb, vino);
2361 dout("handle_lease '%s', mask %d, ino %llx %p\n",
2362 ceph_lease_op_name(h->action), mask, vino.ino, inode);
2363 if (inode == NULL) {
2364 dout("handle_lease no inode %llx\n", vino.ino);
2365 goto release;
2366 }
2367 ci = ceph_inode(inode);
2368
2369 /* dentry */
2370 parent = d_find_alias(inode);
2371 if (!parent) {
2372 dout("no parent dentry on inode %p\n", inode);
2373 WARN_ON(1);
2374 goto release; /* hrm... */
2375 }
2376 dname.hash = full_name_hash(dname.name, dname.len);
2377 dentry = d_lookup(parent, &dname);
2378 dput(parent);
2379 if (!dentry)
2380 goto release;
2381
2382 spin_lock(&dentry->d_lock);
2383 di = ceph_dentry(dentry);
2384 switch (h->action) {
2385 case CEPH_MDS_LEASE_REVOKE:
2386 if (di && di->lease_session == session) {
2387 h->seq = cpu_to_le32(di->lease_seq);
2388 __ceph_mdsc_drop_dentry_lease(dentry);
2389 }
2390 release = 1;
2391 break;
2392
2393 case CEPH_MDS_LEASE_RENEW:
2394 if (di && di->lease_session == session &&
2395 di->lease_gen == session->s_cap_gen &&
2396 di->lease_renew_from &&
2397 di->lease_renew_after == 0) {
2398 unsigned long duration =
2399 le32_to_cpu(h->duration_ms) * HZ / 1000;
2400
2401 di->lease_seq = le32_to_cpu(h->seq);
2402 dentry->d_time = di->lease_renew_from + duration;
2403 di->lease_renew_after = di->lease_renew_from +
2404 (duration >> 1);
2405 di->lease_renew_from = 0;
2406 }
2407 break;
2408 }
2409 spin_unlock(&dentry->d_lock);
2410 dput(dentry);
2411
2412 if (!release)
2413 goto out;
2414
2415release:
2416 /* let's just reuse the same message */
2417 h->action = CEPH_MDS_LEASE_REVOKE_ACK;
2418 ceph_msg_get(msg);
2419 ceph_con_send(&session->s_con, msg);
2420
2421out:
2422 iput(inode);
2423 mutex_unlock(&session->s_mutex);
2424 return;
2425
2426bad:
2427 pr_err("corrupt lease message\n");
2428 ceph_msg_dump(msg);
2429}
2430
2431void ceph_mdsc_lease_send_msg(struct ceph_mds_session *session,
2432 struct inode *inode,
2433 struct dentry *dentry, char action,
2434 u32 seq)
2435{
2436 struct ceph_msg *msg;
2437 struct ceph_mds_lease *lease;
2438 int len = sizeof(*lease) + sizeof(u32);
2439 int dnamelen = 0;
2440
2441 dout("lease_send_msg inode %p dentry %p %s to mds%d\n",
2442 inode, dentry, ceph_lease_op_name(action), session->s_mds);
2443 dnamelen = dentry->d_name.len;
2444 len += dnamelen;
2445
2446 msg = ceph_msg_new(CEPH_MSG_CLIENT_LEASE, len, 0, 0, NULL);
2447 if (IS_ERR(msg))
2448 return;
2449 lease = msg->front.iov_base;
2450 lease->action = action;
2451 lease->mask = cpu_to_le16(CEPH_LOCK_DN);
2452 lease->ino = cpu_to_le64(ceph_vino(inode).ino);
2453 lease->first = lease->last = cpu_to_le64(ceph_vino(inode).snap);
2454 lease->seq = cpu_to_le32(seq);
2455 put_unaligned_le32(dnamelen, lease + 1);
2456 memcpy((void *)(lease + 1) + 4, dentry->d_name.name, dnamelen);
2457
2458 /*
2459 * if this is a preemptive lease RELEASE, no need to
2460 * flush request stream, since the actual request will
2461 * soon follow.
2462 */
2463 msg->more_to_follow = (action == CEPH_MDS_LEASE_RELEASE);
2464
2465 ceph_con_send(&session->s_con, msg);
2466}
2467
2468/*
2469 * Preemptively release a lease we expect to invalidate anyway.
2470 * Pass @inode always, @dentry is optional.
2471 */
2472void ceph_mdsc_lease_release(struct ceph_mds_client *mdsc, struct inode *inode,
2473 struct dentry *dentry, int mask)
2474{
2475 struct ceph_dentry_info *di;
2476 struct ceph_mds_session *session;
2477 u32 seq;
2478
2479 BUG_ON(inode == NULL);
2480 BUG_ON(dentry == NULL);
2481 BUG_ON(mask != CEPH_LOCK_DN);
2482
2483 /* is dentry lease valid? */
2484 spin_lock(&dentry->d_lock);
2485 di = ceph_dentry(dentry);
2486 if (!di || !di->lease_session ||
2487 di->lease_session->s_mds < 0 ||
2488 di->lease_gen != di->lease_session->s_cap_gen ||
2489 !time_before(jiffies, dentry->d_time)) {
2490 dout("lease_release inode %p dentry %p -- "
2491 "no lease on %d\n",
2492 inode, dentry, mask);
2493 spin_unlock(&dentry->d_lock);
2494 return;
2495 }
2496
2497 /* we do have a lease on this dentry; note mds and seq */
2498 session = ceph_get_mds_session(di->lease_session);
2499 seq = di->lease_seq;
2500 __ceph_mdsc_drop_dentry_lease(dentry);
2501 spin_unlock(&dentry->d_lock);
2502
2503 dout("lease_release inode %p dentry %p mask %d to mds%d\n",
2504 inode, dentry, mask, session->s_mds);
2505 ceph_mdsc_lease_send_msg(session, inode, dentry,
2506 CEPH_MDS_LEASE_RELEASE, seq);
2507 ceph_put_mds_session(session);
2508}
2509
2510/*
2511 * drop all leases (and dentry refs) in preparation for umount
2512 */
2513static void drop_leases(struct ceph_mds_client *mdsc)
2514{
2515 int i;
2516
2517 dout("drop_leases\n");
2518 mutex_lock(&mdsc->mutex);
2519 for (i = 0; i < mdsc->max_sessions; i++) {
2520 struct ceph_mds_session *s = __ceph_lookup_mds_session(mdsc, i);
2521 if (!s)
2522 continue;
2523 mutex_unlock(&mdsc->mutex);
2524 mutex_lock(&s->s_mutex);
2525 mutex_unlock(&s->s_mutex);
2526 ceph_put_mds_session(s);
2527 mutex_lock(&mdsc->mutex);
2528 }
2529 mutex_unlock(&mdsc->mutex);
2530}
2531
2532
2533
2534/*
2535 * delayed work -- periodically trim expired leases, renew caps with mds
2536 */
2537static void schedule_delayed(struct ceph_mds_client *mdsc)
2538{
2539 int delay = 5;
2540 unsigned hz = round_jiffies_relative(HZ * delay);
2541 schedule_delayed_work(&mdsc->delayed_work, hz);
2542}
2543
2544static void delayed_work(struct work_struct *work)
2545{
2546 int i;
2547 struct ceph_mds_client *mdsc =
2548 container_of(work, struct ceph_mds_client, delayed_work.work);
2549 int renew_interval;
2550 int renew_caps;
2551
2552 dout("mdsc delayed_work\n");
2553 ceph_check_delayed_caps(mdsc);
2554
2555 mutex_lock(&mdsc->mutex);
2556 renew_interval = mdsc->mdsmap->m_session_timeout >> 2;
2557 renew_caps = time_after_eq(jiffies, HZ*renew_interval +
2558 mdsc->last_renew_caps);
2559 if (renew_caps)
2560 mdsc->last_renew_caps = jiffies;
2561
2562 for (i = 0; i < mdsc->max_sessions; i++) {
2563 struct ceph_mds_session *s = __ceph_lookup_mds_session(mdsc, i);
2564 if (s == NULL)
2565 continue;
2566 if (s->s_state == CEPH_MDS_SESSION_CLOSING) {
2567 dout("resending session close request for mds%d\n",
2568 s->s_mds);
2569 request_close_session(mdsc, s);
2570 ceph_put_mds_session(s);
2571 continue;
2572 }
2573 if (s->s_ttl && time_after(jiffies, s->s_ttl)) {
2574 if (s->s_state == CEPH_MDS_SESSION_OPEN) {
2575 s->s_state = CEPH_MDS_SESSION_HUNG;
2576 pr_info("mds%d hung\n", s->s_mds);
2577 }
2578 }
2579 if (s->s_state < CEPH_MDS_SESSION_OPEN) {
2580 /* this mds is failed or recovering, just wait */
2581 ceph_put_mds_session(s);
2582 continue;
2583 }
2584 mutex_unlock(&mdsc->mutex);
2585
2586 mutex_lock(&s->s_mutex);
2587 if (renew_caps)
2588 send_renew_caps(mdsc, s);
2589 else
2590 ceph_con_keepalive(&s->s_con);
2591 add_cap_releases(mdsc, s, -1);
2592 send_cap_releases(mdsc, s);
2593 mutex_unlock(&s->s_mutex);
2594 ceph_put_mds_session(s);
2595
2596 mutex_lock(&mdsc->mutex);
2597 }
2598 mutex_unlock(&mdsc->mutex);
2599
2600 schedule_delayed(mdsc);
2601}
2602
2603
2604int ceph_mdsc_init(struct ceph_mds_client *mdsc, struct ceph_client *client)
2605{
2606 mdsc->client = client;
2607 mutex_init(&mdsc->mutex);
2608 mdsc->mdsmap = kzalloc(sizeof(*mdsc->mdsmap), GFP_NOFS);
2609 init_completion(&mdsc->safe_umount_waiters);
2610 init_completion(&mdsc->session_close_waiters);
2611 INIT_LIST_HEAD(&mdsc->waiting_for_map);
2612 mdsc->sessions = NULL;
2613 mdsc->max_sessions = 0;
2614 mdsc->stopping = 0;
2615 init_rwsem(&mdsc->snap_rwsem);
2616 mdsc->snap_realms = RB_ROOT;
2617 INIT_LIST_HEAD(&mdsc->snap_empty);
2618 spin_lock_init(&mdsc->snap_empty_lock);
2619 mdsc->last_tid = 0;
2620 mdsc->request_tree = RB_ROOT;
2621 INIT_DELAYED_WORK(&mdsc->delayed_work, delayed_work);
2622 mdsc->last_renew_caps = jiffies;
2623 INIT_LIST_HEAD(&mdsc->cap_delay_list);
2624 spin_lock_init(&mdsc->cap_delay_lock);
2625 INIT_LIST_HEAD(&mdsc->snap_flush_list);
2626 spin_lock_init(&mdsc->snap_flush_lock);
2627 mdsc->cap_flush_seq = 0;
2628 INIT_LIST_HEAD(&mdsc->cap_dirty);
2629 mdsc->num_cap_flushing = 0;
2630 spin_lock_init(&mdsc->cap_dirty_lock);
2631 init_waitqueue_head(&mdsc->cap_flushing_wq);
2632 spin_lock_init(&mdsc->dentry_lru_lock);
2633 INIT_LIST_HEAD(&mdsc->dentry_lru);
2634 return 0;
2635}
2636
2637/*
2638 * Wait for safe replies on open mds requests. If we time out, drop
2639 * all requests from the tree to avoid dangling dentry refs.
2640 */
2641static void wait_requests(struct ceph_mds_client *mdsc)
2642{
2643 struct ceph_mds_request *req;
2644 struct ceph_client *client = mdsc->client;
2645
2646 mutex_lock(&mdsc->mutex);
2647 if (__get_oldest_req(mdsc)) {
2648 mutex_unlock(&mdsc->mutex);
2649
2650 dout("wait_requests waiting for requests\n");
2651 wait_for_completion_timeout(&mdsc->safe_umount_waiters,
2652 client->mount_args->mount_timeout * HZ);
2653
2654 /* tear down remaining requests */
2655 mutex_lock(&mdsc->mutex);
2656 while ((req = __get_oldest_req(mdsc))) {
2657 dout("wait_requests timed out on tid %llu\n",
2658 req->r_tid);
2659 __unregister_request(mdsc, req);
2660 }
2661 }
2662 mutex_unlock(&mdsc->mutex);
2663 dout("wait_requests done\n");
2664}
2665
2666/*
2667 * called before mount is ro, and before dentries are torn down.
2668 * (hmm, does this still race with new lookups?)
2669 */
2670void ceph_mdsc_pre_umount(struct ceph_mds_client *mdsc)
2671{
2672 dout("pre_umount\n");
2673 mdsc->stopping = 1;
2674
2675 drop_leases(mdsc);
2676 ceph_flush_dirty_caps(mdsc);
2677 wait_requests(mdsc);
2678}
2679
2680/*
2681 * wait for all write mds requests to flush.
2682 */
2683static void wait_unsafe_requests(struct ceph_mds_client *mdsc, u64 want_tid)
2684{
2685 struct ceph_mds_request *req = NULL;
2686 struct rb_node *n;
2687
2688 mutex_lock(&mdsc->mutex);
2689 dout("wait_unsafe_requests want %lld\n", want_tid);
2690 req = __get_oldest_req(mdsc);
2691 while (req && req->r_tid <= want_tid) {
2692 if ((req->r_op & CEPH_MDS_OP_WRITE)) {
2693 /* write op */
2694 ceph_mdsc_get_request(req);
2695 mutex_unlock(&mdsc->mutex);
2696 dout("wait_unsafe_requests wait on %llu (want %llu)\n",
2697 req->r_tid, want_tid);
2698 wait_for_completion(&req->r_safe_completion);
2699 mutex_lock(&mdsc->mutex);
2700 n = rb_next(&req->r_node);
2701 ceph_mdsc_put_request(req);
2702 } else {
2703 n = rb_next(&req->r_node);
2704 }
2705 if (!n)
2706 break;
2707 req = rb_entry(n, struct ceph_mds_request, r_node);
2708 }
2709 mutex_unlock(&mdsc->mutex);
2710 dout("wait_unsafe_requests done\n");
2711}
2712
2713void ceph_mdsc_sync(struct ceph_mds_client *mdsc)
2714{
2715 u64 want_tid, want_flush;
2716
2717 dout("sync\n");
2718 mutex_lock(&mdsc->mutex);
2719 want_tid = mdsc->last_tid;
2720 want_flush = mdsc->cap_flush_seq;
2721 mutex_unlock(&mdsc->mutex);
2722 dout("sync want tid %lld flush_seq %lld\n", want_tid, want_flush);
2723
2724 ceph_flush_dirty_caps(mdsc);
2725
2726 wait_unsafe_requests(mdsc, want_tid);
2727 wait_event(mdsc->cap_flushing_wq, check_cap_flush(mdsc, want_flush));
2728}
2729
2730
2731/*
2732 * called after sb is ro.
2733 */
2734void ceph_mdsc_close_sessions(struct ceph_mds_client *mdsc)
2735{
2736 struct ceph_mds_session *session;
2737 int i;
2738 int n;
2739 struct ceph_client *client = mdsc->client;
2740 unsigned long started, timeout = client->mount_args->mount_timeout * HZ;
2741
2742 dout("close_sessions\n");
2743
2744 mutex_lock(&mdsc->mutex);
2745
2746 /* close sessions */
2747 started = jiffies;
2748 while (time_before(jiffies, started + timeout)) {
2749 dout("closing sessions\n");
2750 n = 0;
2751 for (i = 0; i < mdsc->max_sessions; i++) {
2752 session = __ceph_lookup_mds_session(mdsc, i);
2753 if (!session)
2754 continue;
2755 mutex_unlock(&mdsc->mutex);
2756 mutex_lock(&session->s_mutex);
2757 __close_session(mdsc, session);
2758 mutex_unlock(&session->s_mutex);
2759 ceph_put_mds_session(session);
2760 mutex_lock(&mdsc->mutex);
2761 n++;
2762 }
2763 if (n == 0)
2764 break;
2765
2766 if (client->mount_state == CEPH_MOUNT_SHUTDOWN)
2767 break;
2768
2769 dout("waiting for sessions to close\n");
2770 mutex_unlock(&mdsc->mutex);
2771 wait_for_completion_timeout(&mdsc->session_close_waiters,
2772 timeout);
2773 mutex_lock(&mdsc->mutex);
2774 }
2775
2776 /* tear down remaining sessions */
2777 for (i = 0; i < mdsc->max_sessions; i++) {
2778 if (mdsc->sessions[i]) {
2779 session = get_session(mdsc->sessions[i]);
2780 __unregister_session(mdsc, session);
2781 mutex_unlock(&mdsc->mutex);
2782 mutex_lock(&session->s_mutex);
2783 remove_session_caps(session);
2784 mutex_unlock(&session->s_mutex);
2785 ceph_put_mds_session(session);
2786 mutex_lock(&mdsc->mutex);
2787 }
2788 }
2789
2790 WARN_ON(!list_empty(&mdsc->cap_delay_list));
2791
2792 mutex_unlock(&mdsc->mutex);
2793
2794 ceph_cleanup_empty_realms(mdsc);
2795
2796 cancel_delayed_work_sync(&mdsc->delayed_work); /* cancel timer */
2797
2798 dout("stopped\n");
2799}
2800
2801void ceph_mdsc_stop(struct ceph_mds_client *mdsc)
2802{
2803 dout("stop\n");
2804 cancel_delayed_work_sync(&mdsc->delayed_work); /* cancel timer */
2805 if (mdsc->mdsmap)
2806 ceph_mdsmap_destroy(mdsc->mdsmap);
2807 kfree(mdsc->sessions);
2808}
2809
2810
2811/*
2812 * handle mds map update.
2813 */
2814void ceph_mdsc_handle_map(struct ceph_mds_client *mdsc, struct ceph_msg *msg)
2815{
2816 u32 epoch;
2817 u32 maplen;
2818 void *p = msg->front.iov_base;
2819 void *end = p + msg->front.iov_len;
2820 struct ceph_mdsmap *newmap, *oldmap;
2821 struct ceph_fsid fsid;
2822 int err = -EINVAL;
2823
2824 ceph_decode_need(&p, end, sizeof(fsid)+2*sizeof(u32), bad);
2825 ceph_decode_copy(&p, &fsid, sizeof(fsid));
2826 if (ceph_check_fsid(mdsc->client, &fsid) < 0)
2827 return;
2828 epoch = ceph_decode_32(&p);
2829 maplen = ceph_decode_32(&p);
2830 dout("handle_map epoch %u len %d\n", epoch, (int)maplen);
2831
2832 /* do we need it? */
2833 ceph_monc_got_mdsmap(&mdsc->client->monc, epoch);
2834 mutex_lock(&mdsc->mutex);
2835 if (mdsc->mdsmap && epoch <= mdsc->mdsmap->m_epoch) {
2836 dout("handle_map epoch %u <= our %u\n",
2837 epoch, mdsc->mdsmap->m_epoch);
2838 mutex_unlock(&mdsc->mutex);
2839 return;
2840 }
2841
2842 newmap = ceph_mdsmap_decode(&p, end);
2843 if (IS_ERR(newmap)) {
2844 err = PTR_ERR(newmap);
2845 goto bad_unlock;
2846 }
2847
2848 /* swap into place */
2849 if (mdsc->mdsmap) {
2850 oldmap = mdsc->mdsmap;
2851 mdsc->mdsmap = newmap;
2852 check_new_map(mdsc, newmap, oldmap);
2853 ceph_mdsmap_destroy(oldmap);
2854 } else {
2855 mdsc->mdsmap = newmap; /* first mds map */
2856 }
2857 mdsc->client->sb->s_maxbytes = mdsc->mdsmap->m_max_file_size;
2858
2859 __wake_requests(mdsc, &mdsc->waiting_for_map);
2860
2861 mutex_unlock(&mdsc->mutex);
2862 schedule_delayed(mdsc);
2863 return;
2864
2865bad_unlock:
2866 mutex_unlock(&mdsc->mutex);
2867bad:
2868 pr_err("error decoding mdsmap %d\n", err);
2869 return;
2870}
2871
2872static struct ceph_connection *con_get(struct ceph_connection *con)
2873{
2874 struct ceph_mds_session *s = con->private;
2875
2876 if (get_session(s)) {
2877 dout("mdsc con_get %p ok (%d)\n", s, atomic_read(&s->s_ref));
2878 return con;
2879 }
2880 dout("mdsc con_get %p FAIL\n", s);
2881 return NULL;
2882}
2883
2884static void con_put(struct ceph_connection *con)
2885{
2886 struct ceph_mds_session *s = con->private;
2887
2888 ceph_put_mds_session(s);
2889 dout("mdsc con_put %p (%d)\n", s, atomic_read(&s->s_ref));
2890}
2891
2892/*
2893 * if the client is unresponsive for long enough, the mds will kill
2894 * the session entirely.
2895 */
2896static void peer_reset(struct ceph_connection *con)
2897{
2898 struct ceph_mds_session *s = con->private;
2899
2900 pr_err("mds%d gave us the boot. IMPLEMENT RECONNECT.\n",
2901 s->s_mds);
2902}
2903
2904static void dispatch(struct ceph_connection *con, struct ceph_msg *msg)
2905{
2906 struct ceph_mds_session *s = con->private;
2907 struct ceph_mds_client *mdsc = s->s_mdsc;
2908 int type = le16_to_cpu(msg->hdr.type);
2909
2910 mutex_lock(&mdsc->mutex);
2911 if (__verify_registered_session(mdsc, s) < 0) {
2912 mutex_unlock(&mdsc->mutex);
2913 goto out;
2914 }
2915 mutex_unlock(&mdsc->mutex);
2916
2917 switch (type) {
2918 case CEPH_MSG_MDS_MAP:
2919 ceph_mdsc_handle_map(mdsc, msg);
2920 break;
2921 case CEPH_MSG_CLIENT_SESSION:
2922 handle_session(s, msg);
2923 break;
2924 case CEPH_MSG_CLIENT_REPLY:
2925 handle_reply(s, msg);
2926 break;
2927 case CEPH_MSG_CLIENT_REQUEST_FORWARD:
2928 handle_forward(mdsc, s, msg);
2929 break;
2930 case CEPH_MSG_CLIENT_CAPS:
2931 ceph_handle_caps(s, msg);
2932 break;
2933 case CEPH_MSG_CLIENT_SNAP:
2934 ceph_handle_snap(mdsc, s, msg);
2935 break;
2936 case CEPH_MSG_CLIENT_LEASE:
2937 handle_lease(mdsc, s, msg);
2938 break;
2939
2940 default:
2941 pr_err("received unknown message type %d %s\n", type,
2942 ceph_msg_type_name(type));
2943 }
2944out:
2945 ceph_msg_put(msg);
2946}
2947
2948/*
2949 * authentication
2950 */
2951static int get_authorizer(struct ceph_connection *con,
2952 void **buf, int *len, int *proto,
2953 void **reply_buf, int *reply_len, int force_new)
2954{
2955 struct ceph_mds_session *s = con->private;
2956 struct ceph_mds_client *mdsc = s->s_mdsc;
2957 struct ceph_auth_client *ac = mdsc->client->monc.auth;
2958 int ret = 0;
2959
2960 if (force_new && s->s_authorizer) {
2961 ac->ops->destroy_authorizer(ac, s->s_authorizer);
2962 s->s_authorizer = NULL;
2963 }
2964 if (s->s_authorizer == NULL) {
2965 if (ac->ops->create_authorizer) {
2966 ret = ac->ops->create_authorizer(
2967 ac, CEPH_ENTITY_TYPE_MDS,
2968 &s->s_authorizer,
2969 &s->s_authorizer_buf,
2970 &s->s_authorizer_buf_len,
2971 &s->s_authorizer_reply_buf,
2972 &s->s_authorizer_reply_buf_len);
2973 if (ret)
2974 return ret;
2975 }
2976 }
2977
2978 *proto = ac->protocol;
2979 *buf = s->s_authorizer_buf;
2980 *len = s->s_authorizer_buf_len;
2981 *reply_buf = s->s_authorizer_reply_buf;
2982 *reply_len = s->s_authorizer_reply_buf_len;
2983 return 0;
2984}
2985
2986
2987static int verify_authorizer_reply(struct ceph_connection *con, int len)
2988{
2989 struct ceph_mds_session *s = con->private;
2990 struct ceph_mds_client *mdsc = s->s_mdsc;
2991 struct ceph_auth_client *ac = mdsc->client->monc.auth;
2992
2993 return ac->ops->verify_authorizer_reply(ac, s->s_authorizer, len);
2994}
2995
2996static int invalidate_authorizer(struct ceph_connection *con)
2997{
2998 struct ceph_mds_session *s = con->private;
2999 struct ceph_mds_client *mdsc = s->s_mdsc;
3000 struct ceph_auth_client *ac = mdsc->client->monc.auth;
3001
3002 if (ac->ops->invalidate_authorizer)
3003 ac->ops->invalidate_authorizer(ac, CEPH_ENTITY_TYPE_MDS);
3004
3005 return ceph_monc_validate_auth(&mdsc->client->monc);
3006}
3007
3008const static struct ceph_connection_operations mds_con_ops = {
3009 .get = con_get,
3010 .put = con_put,
3011 .dispatch = dispatch,
3012 .get_authorizer = get_authorizer,
3013 .verify_authorizer_reply = verify_authorizer_reply,
3014 .invalidate_authorizer = invalidate_authorizer,
3015 .peer_reset = peer_reset,
3016};
3017
3018
3019
3020
3021/* eof */
fs/ceph/mds_client.h+335−0
@@ -0,0 +1,335 @@
1#ifndef _FS_CEPH_MDS_CLIENT_H
2#define _FS_CEPH_MDS_CLIENT_H
3
4#include <linux/completion.h>
5#include <linux/kref.h>
6#include <linux/list.h>
7#include <linux/mutex.h>
8#include <linux/rbtree.h>
9#include <linux/spinlock.h>
10
11#include "types.h"
12#include "messenger.h"
13#include "mdsmap.h"
14
15/*
16 * Some lock dependencies:
17 *
18 * session->s_mutex
19 * mdsc->mutex
20 *
21 * mdsc->snap_rwsem
22 *
23 * inode->i_lock
24 * mdsc->snap_flush_lock
25 * mdsc->cap_delay_lock
26 *
27 */
28
29struct ceph_client;
30struct ceph_cap;
31
32/*
33 * parsed info about a single inode. pointers are into the encoded
34 * on-wire structures within the mds reply message payload.
35 */
36struct ceph_mds_reply_info_in {
37 struct ceph_mds_reply_inode *in;
38 u32 symlink_len;
39 char *symlink;
40 u32 xattr_len;
41 char *xattr_data;
42};
43
44/*
45 * parsed info about an mds reply, including information about the
46 * target inode and/or its parent directory and dentry, and directory
47 * contents (for readdir results).
48 */
49struct ceph_mds_reply_info_parsed {
50 struct ceph_mds_reply_head *head;
51
52 struct ceph_mds_reply_info_in diri, targeti;
53 struct ceph_mds_reply_dirfrag *dirfrag;
54 char *dname;
55 u32 dname_len;
56 struct ceph_mds_reply_lease *dlease;
57
58 struct ceph_mds_reply_dirfrag *dir_dir;
59 int dir_nr;
60 char **dir_dname;
61 u32 *dir_dname_len;
62 struct ceph_mds_reply_lease **dir_dlease;
63 struct ceph_mds_reply_info_in *dir_in;
64 u8 dir_complete, dir_end;
65
66 /* encoded blob describing snapshot contexts for certain
67 operations (e.g., open) */
68 void *snapblob;
69 int snapblob_len;
70};
71
72
73/*
74 * cap releases are batched and sent to the MDS en masse.
75 */
76#define CEPH_CAPS_PER_RELEASE ((PAGE_CACHE_SIZE - \
77 sizeof(struct ceph_mds_cap_release)) / \
78 sizeof(struct ceph_mds_cap_item))
79
80
81/*
82 * state associated with each MDS<->client session
83 */
84enum {
85 CEPH_MDS_SESSION_NEW = 1,
86 CEPH_MDS_SESSION_OPENING = 2,
87 CEPH_MDS_SESSION_OPEN = 3,
88 CEPH_MDS_SESSION_HUNG = 4,
89 CEPH_MDS_SESSION_CLOSING = 5,
90 CEPH_MDS_SESSION_RESTARTING = 6,
91 CEPH_MDS_SESSION_RECONNECTING = 7,
92};
93
94struct ceph_mds_session {
95 struct ceph_mds_client *s_mdsc;
96 int s_mds;
97 int s_state;
98 unsigned long s_ttl; /* time until mds kills us */
99 u64 s_seq; /* incoming msg seq # */
100 struct mutex s_mutex; /* serialize session messages */
101
102 struct ceph_connection s_con;
103
104 struct ceph_authorizer *s_authorizer;
105 void *s_authorizer_buf, *s_authorizer_reply_buf;
106 size_t s_authorizer_buf_len, s_authorizer_reply_buf_len;
107
108 /* protected by s_cap_lock */
109 spinlock_t s_cap_lock;
110 u32 s_cap_gen; /* inc each time we get mds stale msg */
111 unsigned long s_cap_ttl; /* when session caps expire */
112 struct list_head s_caps; /* all caps issued by this session */
113 int s_nr_caps, s_trim_caps;
114 int s_num_cap_releases;
115 struct list_head s_cap_releases; /* waiting cap_release messages */
116 struct list_head s_cap_releases_done; /* ready to send */
117 struct ceph_cap *s_cap_iterator;
118
119 /* protected by mutex */
120 struct list_head s_cap_flushing; /* inodes w/ flushing caps */
121 struct list_head s_cap_snaps_flushing;
122 unsigned long s_renew_requested; /* last time we sent a renew req */
123 u64 s_renew_seq;
124
125 atomic_t s_ref;
126 struct list_head s_waiting; /* waiting requests */
127 struct list_head s_unsafe; /* unsafe requests */
128};
129
130/*
131 * modes of choosing which MDS to send a request to
132 */
133enum {
134 USE_ANY_MDS,
135 USE_RANDOM_MDS,
136 USE_AUTH_MDS, /* prefer authoritative mds for this metadata item */
137};
138
139struct ceph_mds_request;
140struct ceph_mds_client;
141
142/*
143 * request completion callback
144 */
145typedef void (*ceph_mds_request_callback_t) (struct ceph_mds_client *mdsc,
146 struct ceph_mds_request *req);
147
148/*
149 * an in-flight mds request
150 */
151struct ceph_mds_request {
152 u64 r_tid; /* transaction id */
153 struct rb_node r_node;
154
155 int r_op; /* mds op code */
156 int r_mds;
157
158 /* operation on what? */
159 struct inode *r_inode; /* arg1 */
160 struct dentry *r_dentry; /* arg1 */
161 struct dentry *r_old_dentry; /* arg2: rename from or link from */
162 char *r_path1, *r_path2;
163 struct ceph_vino r_ino1, r_ino2;
164
165 struct inode *r_locked_dir; /* dir (if any) i_mutex locked by vfs */
166 struct inode *r_target_inode; /* resulting inode */
167
168 union ceph_mds_request_args r_args;
169 int r_fmode; /* file mode, if expecting cap */
170
171 /* for choosing which mds to send this request to */
172 int r_direct_mode;
173 u32 r_direct_hash; /* choose dir frag based on this dentry hash */
174 bool r_direct_is_hash; /* true if r_direct_hash is valid */
175
176 /* data payload is used for xattr ops */
177 struct page **r_pages;
178 int r_num_pages;
179 int r_data_len;
180
181 /* what caps shall we drop? */
182 int r_inode_drop, r_inode_unless;
183 int r_dentry_drop, r_dentry_unless;
184 int r_old_dentry_drop, r_old_dentry_unless;
185 struct inode *r_old_inode;
186 int r_old_inode_drop, r_old_inode_unless;
187
188 struct ceph_msg *r_request; /* original request */
189 struct ceph_msg *r_reply;
190 struct ceph_mds_reply_info_parsed r_reply_info;
191 int r_err;
192 bool r_aborted;
193
194 unsigned long r_timeout; /* optional. jiffies */
195 unsigned long r_started; /* start time to measure timeout against */
196 unsigned long r_request_started; /* start time for mds request only,
197 used to measure lease durations */
198
199 /* link unsafe requests to parent directory, for fsync */
200 struct inode *r_unsafe_dir;
201 struct list_head r_unsafe_dir_item;
202
203 struct ceph_mds_session *r_session;
204
205 int r_attempts; /* resend attempts */
206 int r_num_fwd; /* number of forward attempts */
207 int r_num_stale;
208 int r_resend_mds; /* mds to resend to next, if any*/
209
210 struct kref r_kref;
211 struct list_head r_wait;
212 struct completion r_completion;
213 struct completion r_safe_completion;
214 ceph_mds_request_callback_t r_callback;
215 struct list_head r_unsafe_item; /* per-session unsafe list item */
216 bool r_got_unsafe, r_got_safe;
217
218 bool r_did_prepopulate;
219 u32 r_readdir_offset;
220
221 struct ceph_cap_reservation r_caps_reservation;
222 int r_num_caps;
223};
224
225/*
226 * mds client state
227 */
228struct ceph_mds_client {
229 struct ceph_client *client;
230 struct mutex mutex; /* all nested structures */
231
232 struct ceph_mdsmap *mdsmap;
233 struct completion safe_umount_waiters, session_close_waiters;
234 struct list_head waiting_for_map;
235
236 struct ceph_mds_session **sessions; /* NULL for mds if no session */
237 int max_sessions; /* len of s_mds_sessions */
238 int stopping; /* true if shutting down */
239
240 /*
241 * snap_rwsem will cover cap linkage into snaprealms, and
242 * realm snap contexts. (later, we can do per-realm snap
243 * contexts locks..) the empty list contains realms with no
244 * references (implying they contain no inodes with caps) that
245 * should be destroyed.
246 */
247 struct rw_semaphore snap_rwsem;
248 struct rb_root snap_realms;
249 struct list_head snap_empty;
250 spinlock_t snap_empty_lock; /* protect snap_empty */
251
252 u64 last_tid; /* most recent mds request */
253 struct rb_root request_tree; /* pending mds requests */
254 struct delayed_work delayed_work; /* delayed work */
255 unsigned long last_renew_caps; /* last time we renewed our caps */
256 struct list_head cap_delay_list; /* caps with delayed release */
257 spinlock_t cap_delay_lock; /* protects cap_delay_list */
258 struct list_head snap_flush_list; /* cap_snaps ready to flush */
259 spinlock_t snap_flush_lock;
260
261 u64 cap_flush_seq;
262 struct list_head cap_dirty; /* inodes with dirty caps */
263 int num_cap_flushing; /* # caps we are flushing */
264 spinlock_t cap_dirty_lock; /* protects above items */
265 wait_queue_head_t cap_flushing_wq;
266
267#ifdef CONFIG_DEBUG_FS
268 struct dentry *debugfs_file;
269#endif
270
271 spinlock_t dentry_lru_lock;
272 struct list_head dentry_lru;
273 int num_dentry;
274};
275
276extern const char *ceph_mds_op_name(int op);
277
278extern struct ceph_mds_session *
279__ceph_lookup_mds_session(struct ceph_mds_client *, int mds);
280
281static inline struct ceph_mds_session *
282ceph_get_mds_session(struct ceph_mds_session *s)
283{
284 atomic_inc(&s->s_ref);
285 return s;
286}
287
288extern void ceph_put_mds_session(struct ceph_mds_session *s);
289
290extern int ceph_send_msg_mds(struct ceph_mds_client *mdsc,
291 struct ceph_msg *msg, int mds);
292
293extern int ceph_mdsc_init(struct ceph_mds_client *mdsc,
294 struct ceph_client *client);
295extern void ceph_mdsc_close_sessions(struct ceph_mds_client *mdsc);
296extern void ceph_mdsc_stop(struct ceph_mds_client *mdsc);
297
298extern void ceph_mdsc_sync(struct ceph_mds_client *mdsc);
299
300extern void ceph_mdsc_lease_release(struct ceph_mds_client *mdsc,
301 struct inode *inode,
302 struct dentry *dn, int mask);
303
304extern struct ceph_mds_request *
305ceph_mdsc_create_request(struct ceph_mds_client *mdsc, int op, int mode);
306extern void ceph_mdsc_submit_request(struct ceph_mds_client *mdsc,
307 struct ceph_mds_request *req);
308extern int ceph_mdsc_do_request(struct ceph_mds_client *mdsc,
309 struct inode *dir,
310 struct ceph_mds_request *req);
311static inline void ceph_mdsc_get_request(struct ceph_mds_request *req)
312{
313 kref_get(&req->r_kref);
314}
315extern void ceph_mdsc_release_request(struct kref *kref);
316static inline void ceph_mdsc_put_request(struct ceph_mds_request *req)
317{
318 kref_put(&req->r_kref, ceph_mdsc_release_request);
319}
320
321extern void ceph_mdsc_pre_umount(struct ceph_mds_client *mdsc);
322
323extern char *ceph_mdsc_build_path(struct dentry *dentry, int *plen, u64 *base,
324 int stop_on_nosnap);
325
326extern void __ceph_mdsc_drop_dentry_lease(struct dentry *dentry);
327extern void ceph_mdsc_lease_send_msg(struct ceph_mds_session *session,
328 struct inode *inode,
329 struct dentry *dentry, char action,
330 u32 seq);
331
332extern void ceph_mdsc_handle_map(struct ceph_mds_client *mdsc,
333 struct ceph_msg *msg);
334
335#endif
fs/ceph/mdsmap.c+174−0
@@ -0,0 +1,174 @@
1#include "ceph_debug.h"
2
3#include <linux/bug.h>
4#include <linux/err.h>
5#include <linux/random.h>
6#include <linux/slab.h>
7#include <linux/types.h>
8
9#include "mdsmap.h"
10#include "messenger.h"
11#include "decode.h"
12
13#include "super.h"
14
15
16/*
17 * choose a random mds that is "up" (i.e. has a state > 0), or -1.
18 */
19int ceph_mdsmap_get_random_mds(struct ceph_mdsmap *m)
20{
21 int n = 0;
22 int i;
23 char r;
24
25 /* count */
26 for (i = 0; i < m->m_max_mds; i++)
27 if (m->m_info[i].state > 0)
28 n++;
29 if (n == 0)
30 return -1;
31
32 /* pick */
33 get_random_bytes(&r, 1);
34 n = r % n;
35 i = 0;
36 for (i = 0; n > 0; i++, n--)
37 while (m->m_info[i].state <= 0)
38 i++;
39
40 return i;
41}
42
43/*
44 * Decode an MDS map
45 *
46 * Ignore any fields we don't care about (there are quite a few of
47 * them).
48 */
49struct ceph_mdsmap *ceph_mdsmap_decode(void **p, void *end)
50{
51 struct ceph_mdsmap *m;
52 const void *start = *p;
53 int i, j, n;
54 int err = -EINVAL;
55 u16 version;
56
57 m = kzalloc(sizeof(*m), GFP_NOFS);
58 if (m == NULL)
59 return ERR_PTR(-ENOMEM);
60
61 ceph_decode_16_safe(p, end, version, bad);
62
63 ceph_decode_need(p, end, 8*sizeof(u32) + sizeof(u64), bad);
64 m->m_epoch = ceph_decode_32(p);
65 m->m_client_epoch = ceph_decode_32(p);
66 m->m_last_failure = ceph_decode_32(p);
67 m->m_root = ceph_decode_32(p);
68 m->m_session_timeout = ceph_decode_32(p);
69 m->m_session_autoclose = ceph_decode_32(p);
70 m->m_max_file_size = ceph_decode_64(p);
71 m->m_max_mds = ceph_decode_32(p);
72
73 m->m_info = kcalloc(m->m_max_mds, sizeof(*m->m_info), GFP_NOFS);
74 if (m->m_info == NULL)
75 goto badmem;
76
77 /* pick out active nodes from mds_info (state > 0) */
78 n = ceph_decode_32(p);
79 for (i = 0; i < n; i++) {
80 u64 global_id;
81 u32 namelen;
82 s32 mds, inc, state;
83 u64 state_seq;
84 u8 infoversion;
85 struct ceph_entity_addr addr;
86 u32 num_export_targets;
87 void *pexport_targets = NULL;
88
89 ceph_decode_need(p, end, sizeof(u64)*2 + 1 + sizeof(u32), bad);
90 global_id = ceph_decode_64(p);
91 infoversion = ceph_decode_8(p);
92 *p += sizeof(u64);
93 namelen = ceph_decode_32(p); /* skip mds name */
94 *p += namelen;
95
96 ceph_decode_need(p, end,
97 4*sizeof(u32) + sizeof(u64) +
98 sizeof(addr) + sizeof(struct ceph_timespec),
99 bad);
100 mds = ceph_decode_32(p);
101 inc = ceph_decode_32(p);
102 state = ceph_decode_32(p);
103 state_seq = ceph_decode_64(p);
104 ceph_decode_copy(p, &addr, sizeof(addr));
105 ceph_decode_addr(&addr);
106 *p += sizeof(struct ceph_timespec);
107 *p += sizeof(u32);
108 ceph_decode_32_safe(p, end, namelen, bad);
109 *p += namelen;
110 if (infoversion >= 2) {
111 ceph_decode_32_safe(p, end, num_export_targets, bad);
112 pexport_targets = *p;
113 *p += num_export_targets * sizeof(u32);
114 } else {
115 num_export_targets = 0;
116 }
117
118 dout("mdsmap_decode %d/%d %lld mds%d.%d %s %s\n",
119 i+1, n, global_id, mds, inc, pr_addr(&addr.in_addr),
120 ceph_mds_state_name(state));
121 if (mds >= 0 && mds < m->m_max_mds && state > 0) {
122 m->m_info[mds].global_id = global_id;
123 m->m_info[mds].state = state;
124 m->m_info[mds].addr = addr;
125 m->m_info[mds].num_export_targets = num_export_targets;
126 if (num_export_targets) {
127 m->m_info[mds].export_targets =
128 kcalloc(num_export_targets, sizeof(u32),
129 GFP_NOFS);
130 for (j = 0; j < num_export_targets; j++)
131 m->m_info[mds].export_targets[j] =
132 ceph_decode_32(&pexport_targets);
133 } else {
134 m->m_info[mds].export_targets = NULL;
135 }
136 }
137 }
138
139 /* pg_pools */
140 ceph_decode_32_safe(p, end, n, bad);
141 m->m_num_data_pg_pools = n;
142 m->m_data_pg_pools = kcalloc(n, sizeof(u32), GFP_NOFS);
143 if (!m->m_data_pg_pools)
144 goto badmem;
145 ceph_decode_need(p, end, sizeof(u32)*(n+1), bad);
146 for (i = 0; i < n; i++)
147 m->m_data_pg_pools[i] = ceph_decode_32(p);
148 m->m_cas_pg_pool = ceph_decode_32(p);
149
150 /* ok, we don't care about the rest. */
151 dout("mdsmap_decode success epoch %u\n", m->m_epoch);
152 return m;
153
154badmem:
155 err = -ENOMEM;
156bad:
157 pr_err("corrupt mdsmap\n");
158 print_hex_dump(KERN_DEBUG, "mdsmap: ",
159 DUMP_PREFIX_OFFSET, 16, 1,
160 start, end - start, true);
161 ceph_mdsmap_destroy(m);
162 return ERR_PTR(-EINVAL);
163}
164
165void ceph_mdsmap_destroy(struct ceph_mdsmap *m)
166{
167 int i;
168
169 for (i = 0; i < m->m_max_mds; i++)
170 kfree(m->m_info[i].export_targets);
171 kfree(m->m_info);
172 kfree(m->m_data_pg_pools);
173 kfree(m);
174}
fs/ceph/mdsmap.h+54−0
@@ -0,0 +1,54 @@
1#ifndef _FS_CEPH_MDSMAP_H
2#define _FS_CEPH_MDSMAP_H
3
4#include "types.h"
5
6/*
7 * mds map - describe servers in the mds cluster.
8 *
9 * we limit fields to those the client actually xcares about
10 */
11struct ceph_mds_info {
12 u64 global_id;
13 struct ceph_entity_addr addr;
14 s32 state;
15 int num_export_targets;
16 u32 *export_targets;
17};
18
19struct ceph_mdsmap {
20 u32 m_epoch, m_client_epoch, m_last_failure;
21 u32 m_root;
22 u32 m_session_timeout; /* seconds */
23 u32 m_session_autoclose; /* seconds */
24 u64 m_max_file_size;
25 u32 m_max_mds; /* size of m_addr, m_state arrays */
26 struct ceph_mds_info *m_info;
27
28 /* which object pools file data can be stored in */
29 int m_num_data_pg_pools;
30 u32 *m_data_pg_pools;
31 u32 m_cas_pg_pool;
32};
33
34static inline struct ceph_entity_addr *
35ceph_mdsmap_get_addr(struct ceph_mdsmap *m, int w)
36{
37 if (w >= m->m_max_mds)
38 return NULL;
39 return &m->m_info[w].addr;
40}
41
42static inline int ceph_mdsmap_get_state(struct ceph_mdsmap *m, int w)
43{
44 BUG_ON(w < 0);
45 if (w >= m->m_max_mds)
46 return CEPH_MDS_STATE_DNE;
47 return m->m_info[w].state;
48}
49
50extern int ceph_mdsmap_get_random_mds(struct ceph_mdsmap *m);
51extern struct ceph_mdsmap *ceph_mdsmap_decode(void **p, void *end);
52extern void ceph_mdsmap_destroy(struct ceph_mdsmap *m);
53
54#endif
fs/ceph/messenger.c+2240−0
@@ -0,0 +1,2240 @@
1#include "ceph_debug.h"
2
3#include <linux/crc32c.h>
4#include <linux/ctype.h>
5#include <linux/highmem.h>
6#include <linux/inet.h>
7#include <linux/kthread.h>
8#include <linux/net.h>
9#include <linux/socket.h>
10#include <linux/string.h>
11#include <net/tcp.h>
12
13#include "super.h"
14#include "messenger.h"
15#include "decode.h"
16#include "pagelist.h"
17
18/*
19 * Ceph uses the messenger to exchange ceph_msg messages with other
20 * hosts in the system. The messenger provides ordered and reliable
21 * delivery. We tolerate TCP disconnects by reconnecting (with
22 * exponential backoff) in the case of a fault (disconnection, bad
23 * crc, protocol error). Acks allow sent messages to be discarded by
24 * the sender.
25 */
26
27/* static tag bytes (protocol control messages) */
28static char tag_msg = CEPH_MSGR_TAG_MSG;
29static char tag_ack = CEPH_MSGR_TAG_ACK;
30static char tag_keepalive = CEPH_MSGR_TAG_KEEPALIVE;
31
32
33static void queue_con(struct ceph_connection *con);
34static void con_work(struct work_struct *);
35static void ceph_fault(struct ceph_connection *con);
36
37const char *ceph_name_type_str(int t)
38{
39 switch (t) {
40 case CEPH_ENTITY_TYPE_MON: return "mon";
41 case CEPH_ENTITY_TYPE_MDS: return "mds";
42 case CEPH_ENTITY_TYPE_OSD: return "osd";
43 case CEPH_ENTITY_TYPE_CLIENT: return "client";
44 case CEPH_ENTITY_TYPE_ADMIN: return "admin";
45 default: return "???";
46 }
47}
48
49/*
50 * nicely render a sockaddr as a string.
51 */
52#define MAX_ADDR_STR 20
53static char addr_str[MAX_ADDR_STR][40];
54static DEFINE_SPINLOCK(addr_str_lock);
55static int last_addr_str;
56
57const char *pr_addr(const struct sockaddr_storage *ss)
58{
59 int i;
60 char *s;
61 struct sockaddr_in *in4 = (void *)ss;
62 unsigned char *quad = (void *)&in4->sin_addr.s_addr;
63 struct sockaddr_in6 *in6 = (void *)ss;
64
65 spin_lock(&addr_str_lock);
66 i = last_addr_str++;
67 if (last_addr_str == MAX_ADDR_STR)
68 last_addr_str = 0;
69 spin_unlock(&addr_str_lock);
70 s = addr_str[i];
71
72 switch (ss->ss_family) {
73 case AF_INET:
74 sprintf(s, "%u.%u.%u.%u:%u",
75 (unsigned int)quad[0],
76 (unsigned int)quad[1],
77 (unsigned int)quad[2],
78 (unsigned int)quad[3],
79 (unsigned int)ntohs(in4->sin_port));
80 break;
81
82 case AF_INET6:
83 sprintf(s, "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x:%u",
84 in6->sin6_addr.s6_addr16[0],
85 in6->sin6_addr.s6_addr16[1],
86 in6->sin6_addr.s6_addr16[2],
87 in6->sin6_addr.s6_addr16[3],
88 in6->sin6_addr.s6_addr16[4],
89 in6->sin6_addr.s6_addr16[5],
90 in6->sin6_addr.s6_addr16[6],
91 in6->sin6_addr.s6_addr16[7],
92 (unsigned int)ntohs(in6->sin6_port));
93 break;
94
95 default:
96 sprintf(s, "(unknown sockaddr family %d)", (int)ss->ss_family);
97 }
98
99 return s;
100}
101
102static void encode_my_addr(struct ceph_messenger *msgr)
103{
104 memcpy(&msgr->my_enc_addr, &msgr->inst.addr, sizeof(msgr->my_enc_addr));
105 ceph_encode_addr(&msgr->my_enc_addr);
106}
107
108/*
109 * work queue for all reading and writing to/from the socket.
110 */
111struct workqueue_struct *ceph_msgr_wq;
112
113int __init ceph_msgr_init(void)
114{
115 ceph_msgr_wq = create_workqueue("ceph-msgr");
116 if (IS_ERR(ceph_msgr_wq)) {
117 int ret = PTR_ERR(ceph_msgr_wq);
118 pr_err("msgr_init failed to create workqueue: %d\n", ret);
119 ceph_msgr_wq = NULL;
120 return ret;
121 }
122 return 0;
123}
124
125void ceph_msgr_exit(void)
126{
127 destroy_workqueue(ceph_msgr_wq);
128}
129
130/*
131 * socket callback functions
132 */
133
134/* data available on socket, or listen socket received a connect */
135static void ceph_data_ready(struct sock *sk, int count_unused)
136{
137 struct ceph_connection *con =
138 (struct ceph_connection *)sk->sk_user_data;
139 if (sk->sk_state != TCP_CLOSE_WAIT) {
140 dout("ceph_data_ready on %p state = %lu, queueing work\n",
141 con, con->state);
142 queue_con(con);
143 }
144}
145
146/* socket has buffer space for writing */
147static void ceph_write_space(struct sock *sk)
148{
149 struct ceph_connection *con =
150 (struct ceph_connection *)sk->sk_user_data;
151
152 /* only queue to workqueue if there is data we want to write. */
153 if (test_bit(WRITE_PENDING, &con->state)) {
154 dout("ceph_write_space %p queueing write work\n", con);
155 queue_con(con);
156 } else {
157 dout("ceph_write_space %p nothing to write\n", con);
158 }
159
160 /* since we have our own write_space, clear the SOCK_NOSPACE flag */
161 clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags);
162}
163
164/* socket's state has changed */
165static void ceph_state_change(struct sock *sk)
166{
167 struct ceph_connection *con =
168 (struct ceph_connection *)sk->sk_user_data;
169
170 dout("ceph_state_change %p state = %lu sk_state = %u\n",
171 con, con->state, sk->sk_state);
172
173 if (test_bit(CLOSED, &con->state))
174 return;
175
176 switch (sk->sk_state) {
177 case TCP_CLOSE:
178 dout("ceph_state_change TCP_CLOSE\n");
179 case TCP_CLOSE_WAIT:
180 dout("ceph_state_change TCP_CLOSE_WAIT\n");
181 if (test_and_set_bit(SOCK_CLOSED, &con->state) == 0) {
182 if (test_bit(CONNECTING, &con->state))
183 con->error_msg = "connection failed";
184 else
185 con->error_msg = "socket closed";
186 queue_con(con);
187 }
188 break;
189 case TCP_ESTABLISHED:
190 dout("ceph_state_change TCP_ESTABLISHED\n");
191 queue_con(con);
192 break;
193 }
194}
195
196/*
197 * set up socket callbacks
198 */
199static void set_sock_callbacks(struct socket *sock,
200 struct ceph_connection *con)
201{
202 struct sock *sk = sock->sk;
203 sk->sk_user_data = (void *)con;
204 sk->sk_data_ready = ceph_data_ready;
205 sk->sk_write_space = ceph_write_space;
206 sk->sk_state_change = ceph_state_change;
207}
208
209
210/*
211 * socket helpers
212 */
213
214/*
215 * initiate connection to a remote socket.
216 */
217static struct socket *ceph_tcp_connect(struct ceph_connection *con)
218{
219 struct sockaddr *paddr = (struct sockaddr *)&con->peer_addr.in_addr;
220 struct socket *sock;
221 int ret;
222
223 BUG_ON(con->sock);
224 ret = sock_create_kern(AF_INET, SOCK_STREAM, IPPROTO_TCP, &sock);
225 if (ret)
226 return ERR_PTR(ret);
227 con->sock = sock;
228 sock->sk->sk_allocation = GFP_NOFS;
229
230 set_sock_callbacks(sock, con);
231
232 dout("connect %s\n", pr_addr(&con->peer_addr.in_addr));
233
234 ret = sock->ops->connect(sock, paddr, sizeof(*paddr), O_NONBLOCK);
235 if (ret == -EINPROGRESS) {
236 dout("connect %s EINPROGRESS sk_state = %u\n",
237 pr_addr(&con->peer_addr.in_addr),
238 sock->sk->sk_state);
239 ret = 0;
240 }
241 if (ret < 0) {
242 pr_err("connect %s error %d\n",
243 pr_addr(&con->peer_addr.in_addr), ret);
244 sock_release(sock);
245 con->sock = NULL;
246 con->error_msg = "connect error";
247 }
248
249 if (ret < 0)
250 return ERR_PTR(ret);
251 return sock;
252}
253
254static int ceph_tcp_recvmsg(struct socket *sock, void *buf, size_t len)
255{
256 struct kvec iov = {buf, len};
257 struct msghdr msg = { .msg_flags = MSG_DONTWAIT | MSG_NOSIGNAL };
258
259 return kernel_recvmsg(sock, &msg, &iov, 1, len, msg.msg_flags);
260}
261
262/*
263 * write something. @more is true if caller will be sending more data
264 * shortly.
265 */
266static int ceph_tcp_sendmsg(struct socket *sock, struct kvec *iov,
267 size_t kvlen, size_t len, int more)
268{
269 struct msghdr msg = { .msg_flags = MSG_DONTWAIT | MSG_NOSIGNAL };
270
271 if (more)
272 msg.msg_flags |= MSG_MORE;
273 else
274 msg.msg_flags |= MSG_EOR; /* superfluous, but what the hell */
275
276 return kernel_sendmsg(sock, &msg, iov, kvlen, len);
277}
278
279
280/*
281 * Shutdown/close the socket for the given connection.
282 */
283static int con_close_socket(struct ceph_connection *con)
284{
285 int rc;
286
287 dout("con_close_socket on %p sock %p\n", con, con->sock);
288 if (!con->sock)
289 return 0;
290 set_bit(SOCK_CLOSED, &con->state);
291 rc = con->sock->ops->shutdown(con->sock, SHUT_RDWR);
292 sock_release(con->sock);
293 con->sock = NULL;
294 clear_bit(SOCK_CLOSED, &con->state);
295 return rc;
296}
297
298/*
299 * Reset a connection. Discard all incoming and outgoing messages
300 * and clear *_seq state.
301 */
302static void ceph_msg_remove(struct ceph_msg *msg)
303{
304 list_del_init(&msg->list_head);
305 ceph_msg_put(msg);
306}
307static void ceph_msg_remove_list(struct list_head *head)
308{
309 while (!list_empty(head)) {
310 struct ceph_msg *msg = list_first_entry(head, struct ceph_msg,
311 list_head);
312 ceph_msg_remove(msg);
313 }
314}
315
316static void reset_connection(struct ceph_connection *con)
317{
318 /* reset connection, out_queue, msg_ and connect_seq */
319 /* discard existing out_queue and msg_seq */
320 ceph_msg_remove_list(&con->out_queue);
321 ceph_msg_remove_list(&con->out_sent);
322
323 if (con->in_msg) {
324 ceph_msg_put(con->in_msg);
325 con->in_msg = NULL;
326 }
327
328 con->connect_seq = 0;
329 con->out_seq = 0;
330 if (con->out_msg) {
331 ceph_msg_put(con->out_msg);
332 con->out_msg = NULL;
333 }
334 con->in_seq = 0;
335}
336
337/*
338 * mark a peer down. drop any open connections.
339 */
340void ceph_con_close(struct ceph_connection *con)
341{
342 dout("con_close %p peer %s\n", con, pr_addr(&con->peer_addr.in_addr));
343 set_bit(CLOSED, &con->state); /* in case there's queued work */
344 clear_bit(STANDBY, &con->state); /* avoid connect_seq bump */
345 clear_bit(LOSSYTX, &con->state); /* so we retry next connect */
346 clear_bit(KEEPALIVE_PENDING, &con->state);
347 clear_bit(WRITE_PENDING, &con->state);
348 mutex_lock(&con->mutex);
349 reset_connection(con);
350 cancel_delayed_work(&con->work);
351 mutex_unlock(&con->mutex);
352 queue_con(con);
353}
354
355/*
356 * Reopen a closed connection, with a new peer address.
357 */
358void ceph_con_open(struct ceph_connection *con, struct ceph_entity_addr *addr)
359{
360 dout("con_open %p %s\n", con, pr_addr(&addr->in_addr));
361 set_bit(OPENING, &con->state);
362 clear_bit(CLOSED, &con->state);
363 memcpy(&con->peer_addr, addr, sizeof(*addr));
364 con->delay = 0; /* reset backoff memory */
365 queue_con(con);
366}
367
368/*
369 * generic get/put
370 */
371struct ceph_connection *ceph_con_get(struct ceph_connection *con)
372{
373 dout("con_get %p nref = %d -> %d\n", con,
374 atomic_read(&con->nref), atomic_read(&con->nref) + 1);
375 if (atomic_inc_not_zero(&con->nref))
376 return con;
377 return NULL;
378}
379
380void ceph_con_put(struct ceph_connection *con)
381{
382 dout("con_put %p nref = %d -> %d\n", con,
383 atomic_read(&con->nref), atomic_read(&con->nref) - 1);
384 BUG_ON(atomic_read(&con->nref) == 0);
385 if (atomic_dec_and_test(&con->nref)) {
386 BUG_ON(con->sock);
387 kfree(con);
388 }
389}
390
391/*
392 * initialize a new connection.
393 */
394void ceph_con_init(struct ceph_messenger *msgr, struct ceph_connection *con)
395{
396 dout("con_init %p\n", con);
397 memset(con, 0, sizeof(*con));
398 atomic_set(&con->nref, 1);
399 con->msgr = msgr;
400 mutex_init(&con->mutex);
401 INIT_LIST_HEAD(&con->out_queue);
402 INIT_LIST_HEAD(&con->out_sent);
403 INIT_DELAYED_WORK(&con->work, con_work);
404}
405
406
407/*
408 * We maintain a global counter to order connection attempts. Get
409 * a unique seq greater than @gt.
410 */
411static u32 get_global_seq(struct ceph_messenger *msgr, u32 gt)
412{
413 u32 ret;
414
415 spin_lock(&msgr->global_seq_lock);
416 if (msgr->global_seq < gt)
417 msgr->global_seq = gt;
418 ret = ++msgr->global_seq;
419 spin_unlock(&msgr->global_seq_lock);
420 return ret;
421}
422
423
424/*
425 * Prepare footer for currently outgoing message, and finish things
426 * off. Assumes out_kvec* are already valid.. we just add on to the end.
427 */
428static void prepare_write_message_footer(struct ceph_connection *con, int v)
429{
430 struct ceph_msg *m = con->out_msg;
431
432 dout("prepare_write_message_footer %p\n", con);
433 con->out_kvec_is_msg = true;
434 con->out_kvec[v].iov_base = &m->footer;
435 con->out_kvec[v].iov_len = sizeof(m->footer);
436 con->out_kvec_bytes += sizeof(m->footer);
437 con->out_kvec_left++;
438 con->out_more = m->more_to_follow;
439 con->out_msg_done = true;
440}
441
442/*
443 * Prepare headers for the next outgoing message.
444 */
445static void prepare_write_message(struct ceph_connection *con)
446{
447 struct ceph_msg *m;
448 int v = 0;
449
450 con->out_kvec_bytes = 0;
451 con->out_kvec_is_msg = true;
452 con->out_msg_done = false;
453
454 /* Sneak an ack in there first? If we can get it into the same
455 * TCP packet that's a good thing. */
456 if (con->in_seq > con->in_seq_acked) {
457 con->in_seq_acked = con->in_seq;
458 con->out_kvec[v].iov_base = &tag_ack;
459 con->out_kvec[v++].iov_len = 1;
460 con->out_temp_ack = cpu_to_le64(con->in_seq_acked);
461 con->out_kvec[v].iov_base = &con->out_temp_ack;
462 con->out_kvec[v++].iov_len = sizeof(con->out_temp_ack);
463 con->out_kvec_bytes = 1 + sizeof(con->out_temp_ack);
464 }
465
466 m = list_first_entry(&con->out_queue,
467 struct ceph_msg, list_head);
468 con->out_msg = m;
469 if (test_bit(LOSSYTX, &con->state)) {
470 list_del_init(&m->list_head);
471 } else {
472 /* put message on sent list */
473 ceph_msg_get(m);
474 list_move_tail(&m->list_head, &con->out_sent);
475 }
476
477 m->hdr.seq = cpu_to_le64(++con->out_seq);
478
479 dout("prepare_write_message %p seq %lld type %d len %d+%d+%d %d pgs\n",
480 m, con->out_seq, le16_to_cpu(m->hdr.type),
481 le32_to_cpu(m->hdr.front_len), le32_to_cpu(m->hdr.middle_len),
482 le32_to_cpu(m->hdr.data_len),
483 m->nr_pages);
484 BUG_ON(le32_to_cpu(m->hdr.front_len) != m->front.iov_len);
485
486 /* tag + hdr + front + middle */
487 con->out_kvec[v].iov_base = &tag_msg;
488 con->out_kvec[v++].iov_len = 1;
489 con->out_kvec[v].iov_base = &m->hdr;
490 con->out_kvec[v++].iov_len = sizeof(m->hdr);
491 con->out_kvec[v++] = m->front;
492 if (m->middle)
493 con->out_kvec[v++] = m->middle->vec;
494 con->out_kvec_left = v;
495 con->out_kvec_bytes += 1 + sizeof(m->hdr) + m->front.iov_len +
496 (m->middle ? m->middle->vec.iov_len : 0);
497 con->out_kvec_cur = con->out_kvec;
498
499 /* fill in crc (except data pages), footer */
500 con->out_msg->hdr.crc =
501 cpu_to_le32(crc32c(0, (void *)&m->hdr,
502 sizeof(m->hdr) - sizeof(m->hdr.crc)));
503 con->out_msg->footer.flags = CEPH_MSG_FOOTER_COMPLETE;
504 con->out_msg->footer.front_crc =
505 cpu_to_le32(crc32c(0, m->front.iov_base, m->front.iov_len));
506 if (m->middle)
507 con->out_msg->footer.middle_crc =
508 cpu_to_le32(crc32c(0, m->middle->vec.iov_base,
509 m->middle->vec.iov_len));
510 else
511 con->out_msg->footer.middle_crc = 0;
512 con->out_msg->footer.data_crc = 0;
513 dout("prepare_write_message front_crc %u data_crc %u\n",
514 le32_to_cpu(con->out_msg->footer.front_crc),
515 le32_to_cpu(con->out_msg->footer.middle_crc));
516
517 /* is there a data payload? */
518 if (le32_to_cpu(m->hdr.data_len) > 0) {
519 /* initialize page iterator */
520 con->out_msg_pos.page = 0;
521 con->out_msg_pos.page_pos =
522 le16_to_cpu(m->hdr.data_off) & ~PAGE_MASK;
523 con->out_msg_pos.data_pos = 0;
524 con->out_msg_pos.did_page_crc = 0;
525 con->out_more = 1; /* data + footer will follow */
526 } else {
527 /* no, queue up footer too and be done */
528 prepare_write_message_footer(con, v);
529 }
530
531 set_bit(WRITE_PENDING, &con->state);
532}
533
534/*
535 * Prepare an ack.
536 */
537static void prepare_write_ack(struct ceph_connection *con)
538{
539 dout("prepare_write_ack %p %llu -> %llu\n", con,
540 con->in_seq_acked, con->in_seq);
541 con->in_seq_acked = con->in_seq;
542
543 con->out_kvec[0].iov_base = &tag_ack;
544 con->out_kvec[0].iov_len = 1;
545 con->out_temp_ack = cpu_to_le64(con->in_seq_acked);
546 con->out_kvec[1].iov_base = &con->out_temp_ack;
547 con->out_kvec[1].iov_len = sizeof(con->out_temp_ack);
548 con->out_kvec_left = 2;
549 con->out_kvec_bytes = 1 + sizeof(con->out_temp_ack);
550 con->out_kvec_cur = con->out_kvec;
551 con->out_more = 1; /* more will follow.. eventually.. */
552 set_bit(WRITE_PENDING, &con->state);
553}
554
555/*
556 * Prepare to write keepalive byte.
557 */
558static void prepare_write_keepalive(struct ceph_connection *con)
559{
560 dout("prepare_write_keepalive %p\n", con);
561 con->out_kvec[0].iov_base = &tag_keepalive;
562 con->out_kvec[0].iov_len = 1;
563 con->out_kvec_left = 1;
564 con->out_kvec_bytes = 1;
565 con->out_kvec_cur = con->out_kvec;
566 set_bit(WRITE_PENDING, &con->state);
567}
568
569/*
570 * Connection negotiation.
571 */
572
573static void prepare_connect_authorizer(struct ceph_connection *con)
574{
575 void *auth_buf;
576 int auth_len = 0;
577 int auth_protocol = 0;
578
579 mutex_unlock(&con->mutex);
580 if (con->ops->get_authorizer)
581 con->ops->get_authorizer(con, &auth_buf, &auth_len,
582 &auth_protocol, &con->auth_reply_buf,
583 &con->auth_reply_buf_len,
584 con->auth_retry);
585 mutex_lock(&con->mutex);
586
587 con->out_connect.authorizer_protocol = cpu_to_le32(auth_protocol);
588 con->out_connect.authorizer_len = cpu_to_le32(auth_len);
589
590 con->out_kvec[con->out_kvec_left].iov_base = auth_buf;
591 con->out_kvec[con->out_kvec_left].iov_len = auth_len;
592 con->out_kvec_left++;
593 con->out_kvec_bytes += auth_len;
594}
595
596/*
597 * We connected to a peer and are saying hello.
598 */
599static void prepare_write_banner(struct ceph_messenger *msgr,
600 struct ceph_connection *con)
601{
602 int len = strlen(CEPH_BANNER);
603
604 con->out_kvec[0].iov_base = CEPH_BANNER;
605 con->out_kvec[0].iov_len = len;
606 con->out_kvec[1].iov_base = &msgr->my_enc_addr;
607 con->out_kvec[1].iov_len = sizeof(msgr->my_enc_addr);
608 con->out_kvec_left = 2;
609 con->out_kvec_bytes = len + sizeof(msgr->my_enc_addr);
610 con->out_kvec_cur = con->out_kvec;
611 con->out_more = 0;
612 set_bit(WRITE_PENDING, &con->state);
613}
614
615static void prepare_write_connect(struct ceph_messenger *msgr,
616 struct ceph_connection *con,
617 int after_banner)
618{
619 unsigned global_seq = get_global_seq(con->msgr, 0);
620 int proto;
621
622 switch (con->peer_name.type) {
623 case CEPH_ENTITY_TYPE_MON:
624 proto = CEPH_MONC_PROTOCOL;
625 break;
626 case CEPH_ENTITY_TYPE_OSD:
627 proto = CEPH_OSDC_PROTOCOL;
628 break;
629 case CEPH_ENTITY_TYPE_MDS:
630 proto = CEPH_MDSC_PROTOCOL;
631 break;
632 default:
633 BUG();
634 }
635
636 dout("prepare_write_connect %p cseq=%d gseq=%d proto=%d\n", con,
637 con->connect_seq, global_seq, proto);
638
639 con->out_connect.features = CEPH_FEATURE_SUPPORTED;
640 con->out_connect.host_type = cpu_to_le32(CEPH_ENTITY_TYPE_CLIENT);
641 con->out_connect.connect_seq = cpu_to_le32(con->connect_seq);
642 con->out_connect.global_seq = cpu_to_le32(global_seq);
643 con->out_connect.protocol_version = cpu_to_le32(proto);
644 con->out_connect.flags = 0;
645
646 if (!after_banner) {
647 con->out_kvec_left = 0;
648 con->out_kvec_bytes = 0;
649 }
650 con->out_kvec[con->out_kvec_left].iov_base = &con->out_connect;
651 con->out_kvec[con->out_kvec_left].iov_len = sizeof(con->out_connect);
652 con->out_kvec_left++;
653 con->out_kvec_bytes += sizeof(con->out_connect);
654 con->out_kvec_cur = con->out_kvec;
655 con->out_more = 0;
656 set_bit(WRITE_PENDING, &con->state);
657
658 prepare_connect_authorizer(con);
659}
660
661
662/*
663 * write as much of pending kvecs to the socket as we can.
664 * 1 -> done
665 * 0 -> socket full, but more to do
666 * <0 -> error
667 */
668static int write_partial_kvec(struct ceph_connection *con)
669{
670 int ret;
671
672 dout("write_partial_kvec %p %d left\n", con, con->out_kvec_bytes);
673 while (con->out_kvec_bytes > 0) {
674 ret = ceph_tcp_sendmsg(con->sock, con->out_kvec_cur,
675 con->out_kvec_left, con->out_kvec_bytes,
676 con->out_more);
677 if (ret <= 0)
678 goto out;
679 con->out_kvec_bytes -= ret;
680 if (con->out_kvec_bytes == 0)
681 break; /* done */
682 while (ret > 0) {
683 if (ret >= con->out_kvec_cur->iov_len) {
684 ret -= con->out_kvec_cur->iov_len;
685 con->out_kvec_cur++;
686 con->out_kvec_left--;
687 } else {
688 con->out_kvec_cur->iov_len -= ret;
689 con->out_kvec_cur->iov_base += ret;
690 ret = 0;
691 break;
692 }
693 }
694 }
695 con->out_kvec_left = 0;
696 con->out_kvec_is_msg = false;
697 ret = 1;
698out:
699 dout("write_partial_kvec %p %d left in %d kvecs ret = %d\n", con,
700 con->out_kvec_bytes, con->out_kvec_left, ret);
701 return ret; /* done! */
702}
703
704/*
705 * Write as much message data payload as we can. If we finish, queue
706 * up the footer.
707 * 1 -> done, footer is now queued in out_kvec[].
708 * 0 -> socket full, but more to do
709 * <0 -> error
710 */
711static int write_partial_msg_pages(struct ceph_connection *con)
712{
713 struct ceph_msg *msg = con->out_msg;
714 unsigned data_len = le32_to_cpu(msg->hdr.data_len);
715 size_t len;
716 int crc = con->msgr->nocrc;
717 int ret;
718
719 dout("write_partial_msg_pages %p msg %p page %d/%d offset %d\n",
720 con, con->out_msg, con->out_msg_pos.page, con->out_msg->nr_pages,
721 con->out_msg_pos.page_pos);
722
723 while (con->out_msg_pos.page < con->out_msg->nr_pages) {
724 struct page *page = NULL;
725 void *kaddr = NULL;
726
727 /*
728 * if we are calculating the data crc (the default), we need
729 * to map the page. if our pages[] has been revoked, use the
730 * zero page.
731 */
732 if (msg->pages) {
733 page = msg->pages[con->out_msg_pos.page];
734 if (crc)
735 kaddr = kmap(page);
736 } else if (msg->pagelist) {
737 page = list_first_entry(&msg->pagelist->head,
738 struct page, lru);
739 if (crc)
740 kaddr = kmap(page);
741 } else {
742 page = con->msgr->zero_page;
743 if (crc)
744 kaddr = page_address(con->msgr->zero_page);
745 }
746 len = min((int)(PAGE_SIZE - con->out_msg_pos.page_pos),
747 (int)(data_len - con->out_msg_pos.data_pos));
748 if (crc && !con->out_msg_pos.did_page_crc) {
749 void *base = kaddr + con->out_msg_pos.page_pos;
750 u32 tmpcrc = le32_to_cpu(con->out_msg->footer.data_crc);
751
752 BUG_ON(kaddr == NULL);
753 con->out_msg->footer.data_crc =
754 cpu_to_le32(crc32c(tmpcrc, base, len));
755 con->out_msg_pos.did_page_crc = 1;
756 }
757
758 ret = kernel_sendpage(con->sock, page,
759 con->out_msg_pos.page_pos, len,
760 MSG_DONTWAIT | MSG_NOSIGNAL |
761 MSG_MORE);
762
763 if (crc && (msg->pages || msg->pagelist))
764 kunmap(page);
765
766 if (ret <= 0)
767 goto out;
768
769 con->out_msg_pos.data_pos += ret;
770 con->out_msg_pos.page_pos += ret;
771 if (ret == len) {
772 con->out_msg_pos.page_pos = 0;
773 con->out_msg_pos.page++;
774 con->out_msg_pos.did_page_crc = 0;
775 if (msg->pagelist)
776 list_move_tail(&page->lru,
777 &msg->pagelist->head);
778 }
779 }
780
781 dout("write_partial_msg_pages %p msg %p done\n", con, msg);
782
783 /* prepare and queue up footer, too */
784 if (!crc)
785 con->out_msg->footer.flags |= CEPH_MSG_FOOTER_NOCRC;
786 con->out_kvec_bytes = 0;
787 con->out_kvec_left = 0;
788 con->out_kvec_cur = con->out_kvec;
789 prepare_write_message_footer(con, 0);
790 ret = 1;
791out:
792 return ret;
793}
794
795/*
796 * write some zeros
797 */
798static int write_partial_skip(struct ceph_connection *con)
799{
800 int ret;
801
802 while (con->out_skip > 0) {
803 struct kvec iov = {
804 .iov_base = page_address(con->msgr->zero_page),
805 .iov_len = min(con->out_skip, (int)PAGE_CACHE_SIZE)
806 };
807
808 ret = ceph_tcp_sendmsg(con->sock, &iov, 1, iov.iov_len, 1);
809 if (ret <= 0)
810 goto out;
811 con->out_skip -= ret;
812 }
813 ret = 1;
814out:
815 return ret;
816}
817
818/*
819 * Prepare to read connection handshake, or an ack.
820 */
821static void prepare_read_banner(struct ceph_connection *con)
822{
823 dout("prepare_read_banner %p\n", con);
824 con->in_base_pos = 0;
825}
826
827static void prepare_read_connect(struct ceph_connection *con)
828{
829 dout("prepare_read_connect %p\n", con);
830 con->in_base_pos = 0;
831}
832
833static void prepare_read_connect_retry(struct ceph_connection *con)
834{
835 dout("prepare_read_connect_retry %p\n", con);
836 con->in_base_pos = strlen(CEPH_BANNER) + sizeof(con->actual_peer_addr)
837 + sizeof(con->peer_addr_for_me);
838}
839
840static void prepare_read_ack(struct ceph_connection *con)
841{
842 dout("prepare_read_ack %p\n", con);
843 con->in_base_pos = 0;
844}
845
846static void prepare_read_tag(struct ceph_connection *con)
847{
848 dout("prepare_read_tag %p\n", con);
849 con->in_base_pos = 0;
850 con->in_tag = CEPH_MSGR_TAG_READY;
851}
852
853/*
854 * Prepare to read a message.
855 */
856static int prepare_read_message(struct ceph_connection *con)
857{
858 dout("prepare_read_message %p\n", con);
859 BUG_ON(con->in_msg != NULL);
860 con->in_base_pos = 0;
861 con->in_front_crc = con->in_middle_crc = con->in_data_crc = 0;
862 return 0;
863}
864
865
866static int read_partial(struct ceph_connection *con,
867 int *to, int size, void *object)
868{
869 *to += size;
870 while (con->in_base_pos < *to) {
871 int left = *to - con->in_base_pos;
872 int have = size - left;
873 int ret = ceph_tcp_recvmsg(con->sock, object + have, left);
874 if (ret <= 0)
875 return ret;
876 con->in_base_pos += ret;
877 }
878 return 1;
879}
880
881
882/*
883 * Read all or part of the connect-side handshake on a new connection
884 */
885static int read_partial_banner(struct ceph_connection *con)
886{
887 int ret, to = 0;
888
889 dout("read_partial_banner %p at %d\n", con, con->in_base_pos);
890
891 /* peer's banner */
892 ret = read_partial(con, &to, strlen(CEPH_BANNER), con->in_banner);
893 if (ret <= 0)
894 goto out;
895 ret = read_partial(con, &to, sizeof(con->actual_peer_addr),
896 &con->actual_peer_addr);
897 if (ret <= 0)
898 goto out;
899 ret = read_partial(con, &to, sizeof(con->peer_addr_for_me),
900 &con->peer_addr_for_me);
901 if (ret <= 0)
902 goto out;
903out:
904 return ret;
905}
906
907static int read_partial_connect(struct ceph_connection *con)
908{
909 int ret, to = 0;
910
911 dout("read_partial_connect %p at %d\n", con, con->in_base_pos);
912
913 ret = read_partial(con, &to, sizeof(con->in_reply), &con->in_reply);
914 if (ret <= 0)
915 goto out;
916 ret = read_partial(con, &to, le32_to_cpu(con->in_reply.authorizer_len),
917 con->auth_reply_buf);
918 if (ret <= 0)
919 goto out;
920
921 dout("read_partial_connect %p tag %d, con_seq = %u, g_seq = %u\n",
922 con, (int)con->in_reply.tag,
923 le32_to_cpu(con->in_reply.connect_seq),
924 le32_to_cpu(con->in_reply.global_seq));
925out:
926 return ret;
927
928}
929
930/*
931 * Verify the hello banner looks okay.
932 */
933static int verify_hello(struct ceph_connection *con)
934{
935 if (memcmp(con->in_banner, CEPH_BANNER, strlen(CEPH_BANNER))) {
936 pr_err("connect to %s got bad banner\n",
937 pr_addr(&con->peer_addr.in_addr));
938 con->error_msg = "protocol error, bad banner";
939 return -1;
940 }
941 return 0;
942}
943
944static bool addr_is_blank(struct sockaddr_storage *ss)
945{
946 switch (ss->ss_family) {
947 case AF_INET:
948 return ((struct sockaddr_in *)ss)->sin_addr.s_addr == 0;
949 case AF_INET6:
950 return
951 ((struct sockaddr_in6 *)ss)->sin6_addr.s6_addr32[0] == 0 &&
952 ((struct sockaddr_in6 *)ss)->sin6_addr.s6_addr32[1] == 0 &&
953 ((struct sockaddr_in6 *)ss)->sin6_addr.s6_addr32[2] == 0 &&
954 ((struct sockaddr_in6 *)ss)->sin6_addr.s6_addr32[3] == 0;
955 }
956 return false;
957}
958
959static int addr_port(struct sockaddr_storage *ss)
960{
961 switch (ss->ss_family) {
962 case AF_INET:
963 return ntohs(((struct sockaddr_in *)ss)->sin_port);
964 case AF_INET6:
965 return ntohs(((struct sockaddr_in6 *)ss)->sin6_port);
966 }
967 return 0;
968}
969
970static void addr_set_port(struct sockaddr_storage *ss, int p)
971{
972 switch (ss->ss_family) {
973 case AF_INET:
974 ((struct sockaddr_in *)ss)->sin_port = htons(p);
975 case AF_INET6:
976 ((struct sockaddr_in6 *)ss)->sin6_port = htons(p);
977 }
978}
979
980/*
981 * Parse an ip[:port] list into an addr array. Use the default
982 * monitor port if a port isn't specified.
983 */
984int ceph_parse_ips(const char *c, const char *end,
985 struct ceph_entity_addr *addr,
986 int max_count, int *count)
987{
988 int i;
989 const char *p = c;
990
991 dout("parse_ips on '%.*s'\n", (int)(end-c), c);
992 for (i = 0; i < max_count; i++) {
993 const char *ipend;
994 struct sockaddr_storage *ss = &addr[i].in_addr;
995 struct sockaddr_in *in4 = (void *)ss;
996 struct sockaddr_in6 *in6 = (void *)ss;
997 int port;
998
999 memset(ss, 0, sizeof(*ss));
1000 if (in4_pton(p, end - p, (u8 *)&in4->sin_addr.s_addr,
1001 ',', &ipend)) {
1002 ss->ss_family = AF_INET;
1003 } else if (in6_pton(p, end - p, (u8 *)&in6->sin6_addr.s6_addr,
1004 ',', &ipend)) {
1005 ss->ss_family = AF_INET6;
1006 } else {
1007 goto bad;
1008 }
1009 p = ipend;
1010
1011 /* port? */
1012 if (p < end && *p == ':') {
1013 port = 0;
1014 p++;
1015 while (p < end && *p >= '0' && *p <= '9') {
1016 port = (port * 10) + (*p - '0');
1017 p++;
1018 }
1019 if (port > 65535 || port == 0)
1020 goto bad;
1021 } else {
1022 port = CEPH_MON_PORT;
1023 }
1024
1025 addr_set_port(ss, port);
1026
1027 dout("parse_ips got %s\n", pr_addr(ss));
1028
1029 if (p == end)
1030 break;
1031 if (*p != ',')
1032 goto bad;
1033 p++;
1034 }
1035
1036 if (p != end)
1037 goto bad;
1038
1039 if (count)
1040 *count = i + 1;
1041 return 0;
1042
1043bad:
1044 pr_err("parse_ips bad ip '%s'\n", c);
1045 return -EINVAL;
1046}
1047
1048static int process_banner(struct ceph_connection *con)
1049{
1050 dout("process_banner on %p\n", con);
1051
1052 if (verify_hello(con) < 0)
1053 return -1;
1054
1055 ceph_decode_addr(&con->actual_peer_addr);
1056 ceph_decode_addr(&con->peer_addr_for_me);
1057
1058 /*
1059 * Make sure the other end is who we wanted. note that the other
1060 * end may not yet know their ip address, so if it's 0.0.0.0, give
1061 * them the benefit of the doubt.
1062 */
1063 if (memcmp(&con->peer_addr, &con->actual_peer_addr,
1064 sizeof(con->peer_addr)) != 0 &&
1065 !(addr_is_blank(&con->actual_peer_addr.in_addr) &&
1066 con->actual_peer_addr.nonce == con->peer_addr.nonce)) {
1067 pr_warning("wrong peer, want %s/%lld, got %s/%lld\n",
1068 pr_addr(&con->peer_addr.in_addr),
1069 le64_to_cpu(con->peer_addr.nonce),
1070 pr_addr(&con->actual_peer_addr.in_addr),
1071 le64_to_cpu(con->actual_peer_addr.nonce));
1072 con->error_msg = "wrong peer at address";
1073 return -1;
1074 }
1075
1076 /*
1077 * did we learn our address?
1078 */
1079 if (addr_is_blank(&con->msgr->inst.addr.in_addr)) {
1080 int port = addr_port(&con->msgr->inst.addr.in_addr);
1081
1082 memcpy(&con->msgr->inst.addr.in_addr,
1083 &con->peer_addr_for_me.in_addr,
1084 sizeof(con->peer_addr_for_me.in_addr));
1085 addr_set_port(&con->msgr->inst.addr.in_addr, port);
1086 encode_my_addr(con->msgr);
1087 dout("process_banner learned my addr is %s\n",
1088 pr_addr(&con->msgr->inst.addr.in_addr));
1089 }
1090
1091 set_bit(NEGOTIATING, &con->state);
1092 prepare_read_connect(con);
1093 return 0;
1094}
1095
1096static void fail_protocol(struct ceph_connection *con)
1097{
1098 reset_connection(con);
1099 set_bit(CLOSED, &con->state); /* in case there's queued work */
1100
1101 mutex_unlock(&con->mutex);
1102 if (con->ops->bad_proto)
1103 con->ops->bad_proto(con);
1104 mutex_lock(&con->mutex);
1105}
1106
1107static int process_connect(struct ceph_connection *con)
1108{
1109 u64 sup_feat = CEPH_FEATURE_SUPPORTED;
1110 u64 req_feat = CEPH_FEATURE_REQUIRED;
1111 u64 server_feat = le64_to_cpu(con->in_reply.features);
1112
1113 dout("process_connect on %p tag %d\n", con, (int)con->in_tag);
1114
1115 switch (con->in_reply.tag) {
1116 case CEPH_MSGR_TAG_FEATURES:
1117 pr_err("%s%lld %s feature set mismatch,"
1118 " my %llx < server's %llx, missing %llx\n",
1119 ENTITY_NAME(con->peer_name),
1120 pr_addr(&con->peer_addr.in_addr),
1121 sup_feat, server_feat, server_feat & ~sup_feat);
1122 con->error_msg = "missing required protocol features";
1123 fail_protocol(con);
1124 return -1;
1125
1126 case CEPH_MSGR_TAG_BADPROTOVER:
1127 pr_err("%s%lld %s protocol version mismatch,"
1128 " my %d != server's %d\n",
1129 ENTITY_NAME(con->peer_name),
1130 pr_addr(&con->peer_addr.in_addr),
1131 le32_to_cpu(con->out_connect.protocol_version),
1132 le32_to_cpu(con->in_reply.protocol_version));
1133 con->error_msg = "protocol version mismatch";
1134 fail_protocol(con);
1135 return -1;
1136
1137 case CEPH_MSGR_TAG_BADAUTHORIZER:
1138 con->auth_retry++;
1139 dout("process_connect %p got BADAUTHORIZER attempt %d\n", con,
1140 con->auth_retry);
1141 if (con->auth_retry == 2) {
1142 con->error_msg = "connect authorization failure";
1143 reset_connection(con);
1144 set_bit(CLOSED, &con->state);
1145 return -1;
1146 }
1147 con->auth_retry = 1;
1148 prepare_write_connect(con->msgr, con, 0);
1149 prepare_read_connect_retry(con);
1150 break;
1151
1152 case CEPH_MSGR_TAG_RESETSESSION:
1153 /*
1154 * If we connected with a large connect_seq but the peer
1155 * has no record of a session with us (no connection, or
1156 * connect_seq == 0), they will send RESETSESION to indicate
1157 * that they must have reset their session, and may have
1158 * dropped messages.
1159 */
1160 dout("process_connect got RESET peer seq %u\n",
1161 le32_to_cpu(con->in_connect.connect_seq));
1162 pr_err("%s%lld %s connection reset\n",
1163 ENTITY_NAME(con->peer_name),
1164 pr_addr(&con->peer_addr.in_addr));
1165 reset_connection(con);
1166 prepare_write_connect(con->msgr, con, 0);
1167 prepare_read_connect(con);
1168
1169 /* Tell ceph about it. */
1170 mutex_unlock(&con->mutex);
1171 pr_info("reset on %s%lld\n", ENTITY_NAME(con->peer_name));
1172 if (con->ops->peer_reset)
1173 con->ops->peer_reset(con);
1174 mutex_lock(&con->mutex);
1175 break;
1176
1177 case CEPH_MSGR_TAG_RETRY_SESSION:
1178 /*
1179 * If we sent a smaller connect_seq than the peer has, try
1180 * again with a larger value.
1181 */
1182 dout("process_connect got RETRY my seq = %u, peer_seq = %u\n",
1183 le32_to_cpu(con->out_connect.connect_seq),
1184 le32_to_cpu(con->in_connect.connect_seq));
1185 con->connect_seq = le32_to_cpu(con->in_connect.connect_seq);
1186 prepare_write_connect(con->msgr, con, 0);
1187 prepare_read_connect(con);
1188 break;
1189
1190 case CEPH_MSGR_TAG_RETRY_GLOBAL:
1191 /*
1192 * If we sent a smaller global_seq than the peer has, try
1193 * again with a larger value.
1194 */
1195 dout("process_connect got RETRY_GLOBAL my %u peer_gseq %u\n",
1196 con->peer_global_seq,
1197 le32_to_cpu(con->in_connect.global_seq));
1198 get_global_seq(con->msgr,
1199 le32_to_cpu(con->in_connect.global_seq));
1200 prepare_write_connect(con->msgr, con, 0);
1201 prepare_read_connect(con);
1202 break;
1203
1204 case CEPH_MSGR_TAG_READY:
1205 if (req_feat & ~server_feat) {
1206 pr_err("%s%lld %s protocol feature mismatch,"
1207 " my required %llx > server's %llx, need %llx\n",
1208 ENTITY_NAME(con->peer_name),
1209 pr_addr(&con->peer_addr.in_addr),
1210 req_feat, server_feat, req_feat & ~server_feat);
1211 con->error_msg = "missing required protocol features";
1212 fail_protocol(con);
1213 return -1;
1214 }
1215 clear_bit(CONNECTING, &con->state);
1216 con->peer_global_seq = le32_to_cpu(con->in_reply.global_seq);
1217 con->connect_seq++;
1218 dout("process_connect got READY gseq %d cseq %d (%d)\n",
1219 con->peer_global_seq,
1220 le32_to_cpu(con->in_reply.connect_seq),
1221 con->connect_seq);
1222 WARN_ON(con->connect_seq !=
1223 le32_to_cpu(con->in_reply.connect_seq));
1224
1225 if (con->in_reply.flags & CEPH_MSG_CONNECT_LOSSY)
1226 set_bit(LOSSYTX, &con->state);
1227
1228 prepare_read_tag(con);
1229 break;
1230
1231 case CEPH_MSGR_TAG_WAIT:
1232 /*
1233 * If there is a connection race (we are opening
1234 * connections to each other), one of us may just have
1235 * to WAIT. This shouldn't happen if we are the
1236 * client.
1237 */
1238 pr_err("process_connect peer connecting WAIT\n");
1239
1240 default:
1241 pr_err("connect protocol error, will retry\n");
1242 con->error_msg = "protocol error, garbage tag during connect";
1243 return -1;
1244 }
1245 return 0;
1246}
1247
1248
1249/*
1250 * read (part of) an ack
1251 */
1252static int read_partial_ack(struct ceph_connection *con)
1253{
1254 int to = 0;
1255
1256 return read_partial(con, &to, sizeof(con->in_temp_ack),
1257 &con->in_temp_ack);
1258}
1259
1260
1261/*
1262 * We can finally discard anything that's been acked.
1263 */
1264static void process_ack(struct ceph_connection *con)
1265{
1266 struct ceph_msg *m;
1267 u64 ack = le64_to_cpu(con->in_temp_ack);
1268 u64 seq;
1269
1270 while (!list_empty(&con->out_sent)) {
1271 m = list_first_entry(&con->out_sent, struct ceph_msg,
1272 list_head);
1273 seq = le64_to_cpu(m->hdr.seq);
1274 if (seq > ack)
1275 break;
1276 dout("got ack for seq %llu type %d at %p\n", seq,
1277 le16_to_cpu(m->hdr.type), m);
1278 ceph_msg_remove(m);
1279 }
1280 prepare_read_tag(con);
1281}
1282
1283
1284
1285
1286static int read_partial_message_section(struct ceph_connection *con,
1287 struct kvec *section, unsigned int sec_len,
1288 u32 *crc)
1289{
1290 int left;
1291 int ret;
1292
1293 BUG_ON(!section);
1294
1295 while (section->iov_len < sec_len) {
1296 BUG_ON(section->iov_base == NULL);
1297 left = sec_len - section->iov_len;
1298 ret = ceph_tcp_recvmsg(con->sock, (char *)section->iov_base +
1299 section->iov_len, left);
1300 if (ret <= 0)
1301 return ret;
1302 section->iov_len += ret;
1303 if (section->iov_len == sec_len)
1304 *crc = crc32c(0, section->iov_base,
1305 section->iov_len);
1306 }
1307
1308 return 1;
1309}
1310
1311static struct ceph_msg *ceph_alloc_msg(struct ceph_connection *con,
1312 struct ceph_msg_header *hdr,
1313 int *skip);
1314/*
1315 * read (part of) a message.
1316 */
1317static int read_partial_message(struct ceph_connection *con)
1318{
1319 struct ceph_msg *m = con->in_msg;
1320 void *p;
1321 int ret;
1322 int to, left;
1323 unsigned front_len, middle_len, data_len, data_off;
1324 int datacrc = con->msgr->nocrc;
1325 int skip;
1326
1327 dout("read_partial_message con %p msg %p\n", con, m);
1328
1329 /* header */
1330 while (con->in_base_pos < sizeof(con->in_hdr)) {
1331 left = sizeof(con->in_hdr) - con->in_base_pos;
1332 ret = ceph_tcp_recvmsg(con->sock,
1333 (char *)&con->in_hdr + con->in_base_pos,
1334 left);
1335 if (ret <= 0)
1336 return ret;
1337 con->in_base_pos += ret;
1338 if (con->in_base_pos == sizeof(con->in_hdr)) {
1339 u32 crc = crc32c(0, (void *)&con->in_hdr,
1340 sizeof(con->in_hdr) - sizeof(con->in_hdr.crc));
1341 if (crc != le32_to_cpu(con->in_hdr.crc)) {
1342 pr_err("read_partial_message bad hdr "
1343 " crc %u != expected %u\n",
1344 crc, con->in_hdr.crc);
1345 return -EBADMSG;
1346 }
1347 }
1348 }
1349 front_len = le32_to_cpu(con->in_hdr.front_len);
1350 if (front_len > CEPH_MSG_MAX_FRONT_LEN)
1351 return -EIO;
1352 middle_len = le32_to_cpu(con->in_hdr.middle_len);
1353 if (middle_len > CEPH_MSG_MAX_DATA_LEN)
1354 return -EIO;
1355 data_len = le32_to_cpu(con->in_hdr.data_len);
1356 if (data_len > CEPH_MSG_MAX_DATA_LEN)
1357 return -EIO;
1358 data_off = le16_to_cpu(con->in_hdr.data_off);
1359
1360 /* allocate message? */
1361 if (!con->in_msg) {
1362 dout("got hdr type %d front %d data %d\n", con->in_hdr.type,
1363 con->in_hdr.front_len, con->in_hdr.data_len);
1364 con->in_msg = ceph_alloc_msg(con, &con->in_hdr, &skip);
1365 if (skip) {
1366 /* skip this message */
1367 dout("alloc_msg returned NULL, skipping message\n");
1368 con->in_base_pos = -front_len - middle_len - data_len -
1369 sizeof(m->footer);
1370 con->in_tag = CEPH_MSGR_TAG_READY;
1371 return 0;
1372 }
1373 if (IS_ERR(con->in_msg)) {
1374 ret = PTR_ERR(con->in_msg);
1375 con->in_msg = NULL;
1376 con->error_msg =
1377 "error allocating memory for incoming message";
1378 return ret;
1379 }
1380 m = con->in_msg;
1381 m->front.iov_len = 0; /* haven't read it yet */
1382 if (m->middle)
1383 m->middle->vec.iov_len = 0;
1384
1385 con->in_msg_pos.page = 0;
1386 con->in_msg_pos.page_pos = data_off & ~PAGE_MASK;
1387 con->in_msg_pos.data_pos = 0;
1388 }
1389
1390 /* front */
1391 ret = read_partial_message_section(con, &m->front, front_len,
1392 &con->in_front_crc);
1393 if (ret <= 0)
1394 return ret;
1395
1396 /* middle */
1397 if (m->middle) {
1398 ret = read_partial_message_section(con, &m->middle->vec, middle_len,
1399 &con->in_middle_crc);
1400 if (ret <= 0)
1401 return ret;
1402 }
1403
1404 /* (page) data */
1405 while (con->in_msg_pos.data_pos < data_len) {
1406 left = min((int)(data_len - con->in_msg_pos.data_pos),
1407 (int)(PAGE_SIZE - con->in_msg_pos.page_pos));
1408 BUG_ON(m->pages == NULL);
1409 p = kmap(m->pages[con->in_msg_pos.page]);
1410 ret = ceph_tcp_recvmsg(con->sock, p + con->in_msg_pos.page_pos,
1411 left);
1412 if (ret > 0 && datacrc)
1413 con->in_data_crc =
1414 crc32c(con->in_data_crc,
1415 p + con->in_msg_pos.page_pos, ret);
1416 kunmap(m->pages[con->in_msg_pos.page]);
1417 if (ret <= 0)
1418 return ret;
1419 con->in_msg_pos.data_pos += ret;
1420 con->in_msg_pos.page_pos += ret;
1421 if (con->in_msg_pos.page_pos == PAGE_SIZE) {
1422 con->in_msg_pos.page_pos = 0;
1423 con->in_msg_pos.page++;
1424 }
1425 }
1426
1427 /* footer */
1428 to = sizeof(m->hdr) + sizeof(m->footer);
1429 while (con->in_base_pos < to) {
1430 left = to - con->in_base_pos;
1431 ret = ceph_tcp_recvmsg(con->sock, (char *)&m->footer +
1432 (con->in_base_pos - sizeof(m->hdr)),
1433 left);
1434 if (ret <= 0)
1435 return ret;
1436 con->in_base_pos += ret;
1437 }
1438 dout("read_partial_message got msg %p %d (%u) + %d (%u) + %d (%u)\n",
1439 m, front_len, m->footer.front_crc, middle_len,
1440 m->footer.middle_crc, data_len, m->footer.data_crc);
1441
1442 /* crc ok? */
1443 if (con->in_front_crc != le32_to_cpu(m->footer.front_crc)) {
1444 pr_err("read_partial_message %p front crc %u != exp. %u\n",
1445 m, con->in_front_crc, m->footer.front_crc);
1446 return -EBADMSG;
1447 }
1448 if (con->in_middle_crc != le32_to_cpu(m->footer.middle_crc)) {
1449 pr_err("read_partial_message %p middle crc %u != exp %u\n",
1450 m, con->in_middle_crc, m->footer.middle_crc);
1451 return -EBADMSG;
1452 }
1453 if (datacrc &&
1454 (m->footer.flags & CEPH_MSG_FOOTER_NOCRC) == 0 &&
1455 con->in_data_crc != le32_to_cpu(m->footer.data_crc)) {
1456 pr_err("read_partial_message %p data crc %u != exp. %u\n", m,
1457 con->in_data_crc, le32_to_cpu(m->footer.data_crc));
1458 return -EBADMSG;
1459 }
1460
1461 return 1; /* done! */
1462}
1463
1464/*
1465 * Process message. This happens in the worker thread. The callback should
1466 * be careful not to do anything that waits on other incoming messages or it
1467 * may deadlock.
1468 */
1469static void process_message(struct ceph_connection *con)
1470{
1471 struct ceph_msg *msg;
1472
1473 msg = con->in_msg;
1474 con->in_msg = NULL;
1475
1476 /* if first message, set peer_name */
1477 if (con->peer_name.type == 0)
1478 con->peer_name = msg->hdr.src.name;
1479
1480 con->in_seq++;
1481 mutex_unlock(&con->mutex);
1482
1483 dout("===== %p %llu from %s%lld %d=%s len %d+%d (%u %u %u) =====\n",
1484 msg, le64_to_cpu(msg->hdr.seq),
1485 ENTITY_NAME(msg->hdr.src.name),
1486 le16_to_cpu(msg->hdr.type),
1487 ceph_msg_type_name(le16_to_cpu(msg->hdr.type)),
1488 le32_to_cpu(msg->hdr.front_len),
1489 le32_to_cpu(msg->hdr.data_len),
1490 con->in_front_crc, con->in_middle_crc, con->in_data_crc);
1491 con->ops->dispatch(con, msg);
1492
1493 mutex_lock(&con->mutex);
1494 prepare_read_tag(con);
1495}
1496
1497
1498/*
1499 * Write something to the socket. Called in a worker thread when the
1500 * socket appears to be writeable and we have something ready to send.
1501 */
1502static int try_write(struct ceph_connection *con)
1503{
1504 struct ceph_messenger *msgr = con->msgr;
1505 int ret = 1;
1506
1507 dout("try_write start %p state %lu nref %d\n", con, con->state,
1508 atomic_read(&con->nref));
1509
1510 mutex_lock(&con->mutex);
1511more:
1512 dout("try_write out_kvec_bytes %d\n", con->out_kvec_bytes);
1513
1514 /* open the socket first? */
1515 if (con->sock == NULL) {
1516 /*
1517 * if we were STANDBY and are reconnecting _this_
1518 * connection, bump connect_seq now. Always bump
1519 * global_seq.
1520 */
1521 if (test_and_clear_bit(STANDBY, &con->state))
1522 con->connect_seq++;
1523
1524 prepare_write_banner(msgr, con);
1525 prepare_write_connect(msgr, con, 1);
1526 prepare_read_banner(con);
1527 set_bit(CONNECTING, &con->state);
1528 clear_bit(NEGOTIATING, &con->state);
1529
1530 BUG_ON(con->in_msg);
1531 con->in_tag = CEPH_MSGR_TAG_READY;
1532 dout("try_write initiating connect on %p new state %lu\n",
1533 con, con->state);
1534 con->sock = ceph_tcp_connect(con);
1535 if (IS_ERR(con->sock)) {
1536 con->sock = NULL;
1537 con->error_msg = "connect error";
1538 ret = -1;
1539 goto out;
1540 }
1541 }
1542
1543more_kvec:
1544 /* kvec data queued? */
1545 if (con->out_skip) {
1546 ret = write_partial_skip(con);
1547 if (ret <= 0)
1548 goto done;
1549 if (ret < 0) {
1550 dout("try_write write_partial_skip err %d\n", ret);
1551 goto done;
1552 }
1553 }
1554 if (con->out_kvec_left) {
1555 ret = write_partial_kvec(con);
1556 if (ret <= 0)
1557 goto done;
1558 }
1559
1560 /* msg pages? */
1561 if (con->out_msg) {
1562 if (con->out_msg_done) {
1563 ceph_msg_put(con->out_msg);
1564 con->out_msg = NULL; /* we're done with this one */
1565 goto do_next;
1566 }
1567
1568 ret = write_partial_msg_pages(con);
1569 if (ret == 1)
1570 goto more_kvec; /* we need to send the footer, too! */
1571 if (ret == 0)
1572 goto done;
1573 if (ret < 0) {
1574 dout("try_write write_partial_msg_pages err %d\n",
1575 ret);
1576 goto done;
1577 }
1578 }
1579
1580do_next:
1581 if (!test_bit(CONNECTING, &con->state)) {
1582 /* is anything else pending? */
1583 if (!list_empty(&con->out_queue)) {
1584 prepare_write_message(con);
1585 goto more;
1586 }
1587 if (con->in_seq > con->in_seq_acked) {
1588 prepare_write_ack(con);
1589 goto more;
1590 }
1591 if (test_and_clear_bit(KEEPALIVE_PENDING, &con->state)) {
1592 prepare_write_keepalive(con);
1593 goto more;
1594 }
1595 }
1596
1597 /* Nothing to do! */
1598 clear_bit(WRITE_PENDING, &con->state);
1599 dout("try_write nothing else to write.\n");
1600done:
1601 ret = 0;
1602out:
1603 mutex_unlock(&con->mutex);
1604 dout("try_write done on %p\n", con);
1605 return ret;
1606}
1607
1608
1609
1610/*
1611 * Read what we can from the socket.
1612 */
1613static int try_read(struct ceph_connection *con)
1614{
1615 struct ceph_messenger *msgr;
1616 int ret = -1;
1617
1618 if (!con->sock)
1619 return 0;
1620
1621 if (test_bit(STANDBY, &con->state))
1622 return 0;
1623
1624 dout("try_read start on %p\n", con);
1625 msgr = con->msgr;
1626
1627 mutex_lock(&con->mutex);
1628
1629more:
1630 dout("try_read tag %d in_base_pos %d\n", (int)con->in_tag,
1631 con->in_base_pos);
1632 if (test_bit(CONNECTING, &con->state)) {
1633 if (!test_bit(NEGOTIATING, &con->state)) {
1634 dout("try_read connecting\n");
1635 ret = read_partial_banner(con);
1636 if (ret <= 0)
1637 goto done;
1638 if (process_banner(con) < 0) {
1639 ret = -1;
1640 goto out;
1641 }
1642 }
1643 ret = read_partial_connect(con);
1644 if (ret <= 0)
1645 goto done;
1646 if (process_connect(con) < 0) {
1647 ret = -1;
1648 goto out;
1649 }
1650 goto more;
1651 }
1652
1653 if (con->in_base_pos < 0) {
1654 /*
1655 * skipping + discarding content.
1656 *
1657 * FIXME: there must be a better way to do this!
1658 */
1659 static char buf[1024];
1660 int skip = min(1024, -con->in_base_pos);
1661 dout("skipping %d / %d bytes\n", skip, -con->in_base_pos);
1662 ret = ceph_tcp_recvmsg(con->sock, buf, skip);
1663 if (ret <= 0)
1664 goto done;
1665 con->in_base_pos += ret;
1666 if (con->in_base_pos)
1667 goto more;
1668 }
1669 if (con->in_tag == CEPH_MSGR_TAG_READY) {
1670 /*
1671 * what's next?
1672 */
1673 ret = ceph_tcp_recvmsg(con->sock, &con->in_tag, 1);
1674 if (ret <= 0)
1675 goto done;
1676 dout("try_read got tag %d\n", (int)con->in_tag);
1677 switch (con->in_tag) {
1678 case CEPH_MSGR_TAG_MSG:
1679 prepare_read_message(con);
1680 break;
1681 case CEPH_MSGR_TAG_ACK:
1682 prepare_read_ack(con);
1683 break;
1684 case CEPH_MSGR_TAG_CLOSE:
1685 set_bit(CLOSED, &con->state); /* fixme */
1686 goto done;
1687 default:
1688 goto bad_tag;
1689 }
1690 }
1691 if (con->in_tag == CEPH_MSGR_TAG_MSG) {
1692 ret = read_partial_message(con);
1693 if (ret <= 0) {
1694 switch (ret) {
1695 case -EBADMSG:
1696 con->error_msg = "bad crc";
1697 ret = -EIO;
1698 goto out;
1699 case -EIO:
1700 con->error_msg = "io error";
1701 goto out;
1702 default:
1703 goto done;
1704 }
1705 }
1706 if (con->in_tag == CEPH_MSGR_TAG_READY)
1707 goto more;
1708 process_message(con);
1709 goto more;
1710 }
1711 if (con->in_tag == CEPH_MSGR_TAG_ACK) {
1712 ret = read_partial_ack(con);
1713 if (ret <= 0)
1714 goto done;
1715 process_ack(con);
1716 goto more;
1717 }
1718
1719done:
1720 ret = 0;
1721out:
1722 mutex_unlock(&con->mutex);
1723 dout("try_read done on %p\n", con);
1724 return ret;
1725
1726bad_tag:
1727 pr_err("try_read bad con->in_tag = %d\n", (int)con->in_tag);
1728 con->error_msg = "protocol error, garbage tag";
1729 ret = -1;
1730 goto out;
1731}
1732
1733
1734/*
1735 * Atomically queue work on a connection. Bump @con reference to
1736 * avoid races with connection teardown.
1737 *
1738 * There is some trickery going on with QUEUED and BUSY because we
1739 * only want a _single_ thread operating on each connection at any
1740 * point in time, but we want to use all available CPUs.
1741 *
1742 * The worker thread only proceeds if it can atomically set BUSY. It
1743 * clears QUEUED and does it's thing. When it thinks it's done, it
1744 * clears BUSY, then rechecks QUEUED.. if it's set again, it loops
1745 * (tries again to set BUSY).
1746 *
1747 * To queue work, we first set QUEUED, _then_ if BUSY isn't set, we
1748 * try to queue work. If that fails (work is already queued, or BUSY)
1749 * we give up (work also already being done or is queued) but leave QUEUED
1750 * set so that the worker thread will loop if necessary.
1751 */
1752static void queue_con(struct ceph_connection *con)
1753{
1754 if (test_bit(DEAD, &con->state)) {
1755 dout("queue_con %p ignoring: DEAD\n",
1756 con);
1757 return;
1758 }
1759
1760 if (!con->ops->get(con)) {
1761 dout("queue_con %p ref count 0\n", con);
1762 return;
1763 }
1764
1765 set_bit(QUEUED, &con->state);
1766 if (test_bit(BUSY, &con->state)) {
1767 dout("queue_con %p - already BUSY\n", con);
1768 con->ops->put(con);
1769 } else if (!queue_work(ceph_msgr_wq, &con->work.work)) {
1770 dout("queue_con %p - already queued\n", con);
1771 con->ops->put(con);
1772 } else {
1773 dout("queue_con %p\n", con);
1774 }
1775}
1776
1777/*
1778 * Do some work on a connection. Drop a connection ref when we're done.
1779 */
1780static void con_work(struct work_struct *work)
1781{
1782 struct ceph_connection *con = container_of(work, struct ceph_connection,
1783 work.work);
1784 int backoff = 0;
1785
1786more:
1787 if (test_and_set_bit(BUSY, &con->state) != 0) {
1788 dout("con_work %p BUSY already set\n", con);
1789 goto out;
1790 }
1791 dout("con_work %p start, clearing QUEUED\n", con);
1792 clear_bit(QUEUED, &con->state);
1793
1794 if (test_bit(CLOSED, &con->state)) { /* e.g. if we are replaced */
1795 dout("con_work CLOSED\n");
1796 con_close_socket(con);
1797 goto done;
1798 }
1799 if (test_and_clear_bit(OPENING, &con->state)) {
1800 /* reopen w/ new peer */
1801 dout("con_work OPENING\n");
1802 con_close_socket(con);
1803 }
1804
1805 if (test_and_clear_bit(SOCK_CLOSED, &con->state) ||
1806 try_read(con) < 0 ||
1807 try_write(con) < 0) {
1808 backoff = 1;
1809 ceph_fault(con); /* error/fault path */
1810 }
1811
1812done:
1813 clear_bit(BUSY, &con->state);
1814 dout("con->state=%lu\n", con->state);
1815 if (test_bit(QUEUED, &con->state)) {
1816 if (!backoff || test_bit(OPENING, &con->state)) {
1817 dout("con_work %p QUEUED reset, looping\n", con);
1818 goto more;
1819 }
1820 dout("con_work %p QUEUED reset, but just faulted\n", con);
1821 clear_bit(QUEUED, &con->state);
1822 }
1823 dout("con_work %p done\n", con);
1824
1825out:
1826 con->ops->put(con);
1827}
1828
1829
1830/*
1831 * Generic error/fault handler. A retry mechanism is used with
1832 * exponential backoff
1833 */
1834static void ceph_fault(struct ceph_connection *con)
1835{
1836 pr_err("%s%lld %s %s\n", ENTITY_NAME(con->peer_name),
1837 pr_addr(&con->peer_addr.in_addr), con->error_msg);
1838 dout("fault %p state %lu to peer %s\n",
1839 con, con->state, pr_addr(&con->peer_addr.in_addr));
1840
1841 if (test_bit(LOSSYTX, &con->state)) {
1842 dout("fault on LOSSYTX channel\n");
1843 goto out;
1844 }
1845
1846 clear_bit(BUSY, &con->state); /* to avoid an improbable race */
1847
1848 mutex_lock(&con->mutex);
1849 if (test_bit(CLOSED, &con->state))
1850 goto out_unlock;
1851
1852 con_close_socket(con);
1853
1854 if (con->in_msg) {
1855 ceph_msg_put(con->in_msg);
1856 con->in_msg = NULL;
1857 }
1858
1859 /* Requeue anything that hasn't been acked */
1860 list_splice_init(&con->out_sent, &con->out_queue);
1861
1862 /* If there are no messages in the queue, place the connection
1863 * in a STANDBY state (i.e., don't try to reconnect just yet). */
1864 if (list_empty(&con->out_queue) && !con->out_keepalive_pending) {
1865 dout("fault setting STANDBY\n");
1866 set_bit(STANDBY, &con->state);
1867 } else {
1868 /* retry after a delay. */
1869 if (con->delay == 0)
1870 con->delay = BASE_DELAY_INTERVAL;
1871 else if (con->delay < MAX_DELAY_INTERVAL)
1872 con->delay *= 2;
1873 dout("fault queueing %p delay %lu\n", con, con->delay);
1874 con->ops->get(con);
1875 if (queue_delayed_work(ceph_msgr_wq, &con->work,
1876 round_jiffies_relative(con->delay)) == 0)
1877 con->ops->put(con);
1878 }
1879
1880out_unlock:
1881 mutex_unlock(&con->mutex);
1882out:
1883 /*
1884 * in case we faulted due to authentication, invalidate our
1885 * current tickets so that we can get new ones.
1886 */
1887 if (con->auth_retry && con->ops->invalidate_authorizer) {
1888 dout("calling invalidate_authorizer()\n");
1889 con->ops->invalidate_authorizer(con);
1890 }
1891
1892 if (con->ops->fault)
1893 con->ops->fault(con);
1894}
1895
1896
1897
1898/*
1899 * create a new messenger instance
1900 */
1901struct ceph_messenger *ceph_messenger_create(struct ceph_entity_addr *myaddr)
1902{
1903 struct ceph_messenger *msgr;
1904
1905 msgr = kzalloc(sizeof(*msgr), GFP_KERNEL);
1906 if (msgr == NULL)
1907 return ERR_PTR(-ENOMEM);
1908
1909 spin_lock_init(&msgr->global_seq_lock);
1910
1911 /* the zero page is needed if a request is "canceled" while the message
1912 * is being written over the socket */
1913 msgr->zero_page = alloc_page(GFP_KERNEL | __GFP_ZERO);
1914 if (!msgr->zero_page) {
1915 kfree(msgr);
1916 return ERR_PTR(-ENOMEM);
1917 }
1918 kmap(msgr->zero_page);
1919
1920 if (myaddr)
1921 msgr->inst.addr = *myaddr;
1922
1923 /* select a random nonce */
1924 msgr->inst.addr.type = 0;
1925 get_random_bytes(&msgr->inst.addr.nonce, sizeof(msgr->inst.addr.nonce));
1926 encode_my_addr(msgr);
1927
1928 dout("messenger_create %p\n", msgr);
1929 return msgr;
1930}
1931
1932void ceph_messenger_destroy(struct ceph_messenger *msgr)
1933{
1934 dout("destroy %p\n", msgr);
1935 kunmap(msgr->zero_page);
1936 __free_page(msgr->zero_page);
1937 kfree(msgr);
1938 dout("destroyed messenger %p\n", msgr);
1939}
1940
1941/*
1942 * Queue up an outgoing message on the given connection.
1943 */
1944void ceph_con_send(struct ceph_connection *con, struct ceph_msg *msg)
1945{
1946 if (test_bit(CLOSED, &con->state)) {
1947 dout("con_send %p closed, dropping %p\n", con, msg);
1948 ceph_msg_put(msg);
1949 return;
1950 }
1951
1952 /* set src+dst */
1953 msg->hdr.src.name = con->msgr->inst.name;
1954 msg->hdr.src.addr = con->msgr->my_enc_addr;
1955 msg->hdr.orig_src = msg->hdr.src;
1956
1957 BUG_ON(msg->front.iov_len != le32_to_cpu(msg->hdr.front_len));
1958
1959 /* queue */
1960 mutex_lock(&con->mutex);
1961 BUG_ON(!list_empty(&msg->list_head));
1962 list_add_tail(&msg->list_head, &con->out_queue);
1963 dout("----- %p to %s%lld %d=%s len %d+%d+%d -----\n", msg,
1964 ENTITY_NAME(con->peer_name), le16_to_cpu(msg->hdr.type),
1965 ceph_msg_type_name(le16_to_cpu(msg->hdr.type)),
1966 le32_to_cpu(msg->hdr.front_len),
1967 le32_to_cpu(msg->hdr.middle_len),
1968 le32_to_cpu(msg->hdr.data_len));
1969 mutex_unlock(&con->mutex);
1970
1971 /* if there wasn't anything waiting to send before, queue
1972 * new work */
1973 if (test_and_set_bit(WRITE_PENDING, &con->state) == 0)
1974 queue_con(con);
1975}
1976
1977/*
1978 * Revoke a message that was previously queued for send
1979 */
1980void ceph_con_revoke(struct ceph_connection *con, struct ceph_msg *msg)
1981{
1982 mutex_lock(&con->mutex);
1983 if (!list_empty(&msg->list_head)) {
1984 dout("con_revoke %p msg %p\n", con, msg);
1985 list_del_init(&msg->list_head);
1986 ceph_msg_put(msg);
1987 msg->hdr.seq = 0;
1988 if (con->out_msg == msg) {
1989 ceph_msg_put(con->out_msg);
1990 con->out_msg = NULL;
1991 }
1992 if (con->out_kvec_is_msg) {
1993 con->out_skip = con->out_kvec_bytes;
1994 con->out_kvec_is_msg = false;
1995 }
1996 } else {
1997 dout("con_revoke %p msg %p - not queued (sent?)\n", con, msg);
1998 }
1999 mutex_unlock(&con->mutex);
2000}
2001
2002/*
2003 * Revoke a message that we may be reading data into
2004 */
2005void ceph_con_revoke_message(struct ceph_connection *con, struct ceph_msg *msg)
2006{
2007 mutex_lock(&con->mutex);
2008 if (con->in_msg && con->in_msg == msg) {
2009 unsigned front_len = le32_to_cpu(con->in_hdr.front_len);
2010 unsigned middle_len = le32_to_cpu(con->in_hdr.middle_len);
2011 unsigned data_len = le32_to_cpu(con->in_hdr.data_len);
2012
2013 /* skip rest of message */
2014 dout("con_revoke_pages %p msg %p revoked\n", con, msg);
2015 con->in_base_pos = con->in_base_pos -
2016 sizeof(struct ceph_msg_header) -
2017 front_len -
2018 middle_len -
2019 data_len -
2020 sizeof(struct ceph_msg_footer);
2021 ceph_msg_put(con->in_msg);
2022 con->in_msg = NULL;
2023 con->in_tag = CEPH_MSGR_TAG_READY;
2024 } else {
2025 dout("con_revoke_pages %p msg %p pages %p no-op\n",
2026 con, con->in_msg, msg);
2027 }
2028 mutex_unlock(&con->mutex);
2029}
2030
2031/*
2032 * Queue a keepalive byte to ensure the tcp connection is alive.
2033 */
2034void ceph_con_keepalive(struct ceph_connection *con)
2035{
2036 if (test_and_set_bit(KEEPALIVE_PENDING, &con->state) == 0 &&
2037 test_and_set_bit(WRITE_PENDING, &con->state) == 0)
2038 queue_con(con);
2039}
2040
2041
2042/*
2043 * construct a new message with given type, size
2044 * the new msg has a ref count of 1.
2045 */
2046struct ceph_msg *ceph_msg_new(int type, int front_len,
2047 int page_len, int page_off, struct page **pages)
2048{
2049 struct ceph_msg *m;
2050
2051 m = kmalloc(sizeof(*m), GFP_NOFS);
2052 if (m == NULL)
2053 goto out;
2054 kref_init(&m->kref);
2055 INIT_LIST_HEAD(&m->list_head);
2056
2057 m->hdr.type = cpu_to_le16(type);
2058 m->hdr.front_len = cpu_to_le32(front_len);
2059 m->hdr.middle_len = 0;
2060 m->hdr.data_len = cpu_to_le32(page_len);
2061 m->hdr.data_off = cpu_to_le16(page_off);
2062 m->hdr.priority = cpu_to_le16(CEPH_MSG_PRIO_DEFAULT);
2063 m->footer.front_crc = 0;
2064 m->footer.middle_crc = 0;
2065 m->footer.data_crc = 0;
2066 m->front_max = front_len;
2067 m->front_is_vmalloc = false;
2068 m->more_to_follow = false;
2069 m->pool = NULL;
2070
2071 /* front */
2072 if (front_len) {
2073 if (front_len > PAGE_CACHE_SIZE) {
2074 m->front.iov_base = __vmalloc(front_len, GFP_NOFS,
2075 PAGE_KERNEL);
2076 m->front_is_vmalloc = true;
2077 } else {
2078 m->front.iov_base = kmalloc(front_len, GFP_NOFS);
2079 }
2080 if (m->front.iov_base == NULL) {
2081 pr_err("msg_new can't allocate %d bytes\n",
2082 front_len);
2083 goto out2;
2084 }
2085 } else {
2086 m->front.iov_base = NULL;
2087 }
2088 m->front.iov_len = front_len;
2089
2090 /* middle */
2091 m->middle = NULL;
2092
2093 /* data */
2094 m->nr_pages = calc_pages_for(page_off, page_len);
2095 m->pages = pages;
2096 m->pagelist = NULL;
2097
2098 dout("ceph_msg_new %p page %d~%d -> %d\n", m, page_off, page_len,
2099 m->nr_pages);
2100 return m;
2101
2102out2:
2103 ceph_msg_put(m);
2104out:
2105 pr_err("msg_new can't create type %d len %d\n", type, front_len);
2106 return ERR_PTR(-ENOMEM);
2107}
2108
2109/*
2110 * Allocate "middle" portion of a message, if it is needed and wasn't
2111 * allocated by alloc_msg. This allows us to read a small fixed-size
2112 * per-type header in the front and then gracefully fail (i.e.,
2113 * propagate the error to the caller based on info in the front) when
2114 * the middle is too large.
2115 */
2116static int ceph_alloc_middle(struct ceph_connection *con, struct ceph_msg *msg)
2117{
2118 int type = le16_to_cpu(msg->hdr.type);
2119 int middle_len = le32_to_cpu(msg->hdr.middle_len);
2120
2121 dout("alloc_middle %p type %d %s middle_len %d\n", msg, type,
2122 ceph_msg_type_name(type), middle_len);
2123 BUG_ON(!middle_len);
2124 BUG_ON(msg->middle);
2125
2126 msg->middle = ceph_buffer_new(middle_len, GFP_NOFS);
2127 if (!msg->middle)
2128 return -ENOMEM;
2129 return 0;
2130}
2131
2132/*
2133 * Generic message allocator, for incoming messages.
2134 */
2135static struct ceph_msg *ceph_alloc_msg(struct ceph_connection *con,
2136 struct ceph_msg_header *hdr,
2137 int *skip)
2138{
2139 int type = le16_to_cpu(hdr->type);
2140 int front_len = le32_to_cpu(hdr->front_len);
2141 int middle_len = le32_to_cpu(hdr->middle_len);
2142 struct ceph_msg *msg = NULL;
2143 int ret;
2144
2145 if (con->ops->alloc_msg) {
2146 mutex_unlock(&con->mutex);
2147 msg = con->ops->alloc_msg(con, hdr, skip);
2148 mutex_lock(&con->mutex);
2149 if (IS_ERR(msg))
2150 return msg;
2151
2152 if (*skip)
2153 return NULL;
2154 }
2155 if (!msg) {
2156 *skip = 0;
2157 msg = ceph_msg_new(type, front_len, 0, 0, NULL);
2158 if (!msg) {
2159 pr_err("unable to allocate msg type %d len %d\n",
2160 type, front_len);
2161 return ERR_PTR(-ENOMEM);
2162 }
2163 }
2164 memcpy(&msg->hdr, &con->in_hdr, sizeof(con->in_hdr));
2165
2166 if (middle_len) {
2167 ret = ceph_alloc_middle(con, msg);
2168
2169 if (ret < 0) {
2170 ceph_msg_put(msg);
2171 return msg;
2172 }
2173 }
2174
2175 return msg;
2176}
2177
2178
2179/*
2180 * Free a generically kmalloc'd message.
2181 */
2182void ceph_msg_kfree(struct ceph_msg *m)
2183{
2184 dout("msg_kfree %p\n", m);
2185 if (m->front_is_vmalloc)
2186 vfree(m->front.iov_base);
2187 else
2188 kfree(m->front.iov_base);
2189 kfree(m);
2190}
2191
2192/*
2193 * Drop a msg ref. Destroy as needed.
2194 */
2195void ceph_msg_last_put(struct kref *kref)
2196{
2197 struct ceph_msg *m = container_of(kref, struct ceph_msg, kref);
2198
2199 dout("ceph_msg_put last one on %p\n", m);
2200 WARN_ON(!list_empty(&m->list_head));
2201
2202 /* drop middle, data, if any */
2203 if (m->middle) {
2204 ceph_buffer_put(m->middle);
2205 m->middle = NULL;
2206 }
2207 m->nr_pages = 0;
2208 m->pages = NULL;
2209
2210 if (m->pagelist) {
2211 ceph_pagelist_release(m->pagelist);
2212 kfree(m->pagelist);
2213 m->pagelist = NULL;
2214 }
2215
2216 if (m->pool)
2217 ceph_msgpool_put(m->pool, m);
2218 else
2219 ceph_msg_kfree(m);
2220}
2221
2222void ceph_msg_dump(struct ceph_msg *msg)
2223{
2224 pr_debug("msg_dump %p (front_max %d nr_pages %d)\n", msg,
2225 msg->front_max, msg->nr_pages);
2226 print_hex_dump(KERN_DEBUG, "header: ",
2227 DUMP_PREFIX_OFFSET, 16, 1,
2228 &msg->hdr, sizeof(msg->hdr), true);
2229 print_hex_dump(KERN_DEBUG, " front: ",
2230 DUMP_PREFIX_OFFSET, 16, 1,
2231 msg->front.iov_base, msg->front.iov_len, true);
2232 if (msg->middle)
2233 print_hex_dump(KERN_DEBUG, "middle: ",
2234 DUMP_PREFIX_OFFSET, 16, 1,
2235 msg->middle->vec.iov_base,
2236 msg->middle->vec.iov_len, true);
2237 print_hex_dump(KERN_DEBUG, "footer: ",
2238 DUMP_PREFIX_OFFSET, 16, 1,
2239 &msg->footer, sizeof(msg->footer), true);
2240}
fs/ceph/messenger.h+254−0
@@ -0,0 +1,254 @@
1#ifndef __FS_CEPH_MESSENGER_H
2#define __FS_CEPH_MESSENGER_H
3
4#include <linux/kref.h>
5#include <linux/mutex.h>
6#include <linux/net.h>
7#include <linux/radix-tree.h>
8#include <linux/uio.h>
9#include <linux/version.h>
10#include <linux/workqueue.h>
11
12#include "types.h"
13#include "buffer.h"
14
15struct ceph_msg;
16struct ceph_connection;
17
18extern struct workqueue_struct *ceph_msgr_wq; /* receive work queue */
19
20/*
21 * Ceph defines these callbacks for handling connection events.
22 */
23struct ceph_connection_operations {
24 struct ceph_connection *(*get)(struct ceph_connection *);
25 void (*put)(struct ceph_connection *);
26
27 /* handle an incoming message. */
28 void (*dispatch) (struct ceph_connection *con, struct ceph_msg *m);
29
30 /* authorize an outgoing connection */
31 int (*get_authorizer) (struct ceph_connection *con,
32 void **buf, int *len, int *proto,
33 void **reply_buf, int *reply_len, int force_new);
34 int (*verify_authorizer_reply) (struct ceph_connection *con, int len);
35 int (*invalidate_authorizer)(struct ceph_connection *con);
36
37 /* protocol version mismatch */
38 void (*bad_proto) (struct ceph_connection *con);
39
40 /* there was some error on the socket (disconnect, whatever) */
41 void (*fault) (struct ceph_connection *con);
42
43 /* a remote host as terminated a message exchange session, and messages
44 * we sent (or they tried to send us) may be lost. */
45 void (*peer_reset) (struct ceph_connection *con);
46
47 struct ceph_msg * (*alloc_msg) (struct ceph_connection *con,
48 struct ceph_msg_header *hdr,
49 int *skip);
50};
51
52extern const char *ceph_name_type_str(int t);
53
54/* use format string %s%d */
55#define ENTITY_NAME(n) ceph_name_type_str((n).type), le64_to_cpu((n).num)
56
57struct ceph_messenger {
58 struct ceph_entity_inst inst; /* my name+address */
59 struct ceph_entity_addr my_enc_addr;
60 struct page *zero_page; /* used in certain error cases */
61
62 bool nocrc;
63
64 /*
65 * the global_seq counts connections i (attempt to) initiate
66 * in order to disambiguate certain connect race conditions.
67 */
68 u32 global_seq;
69 spinlock_t global_seq_lock;
70};
71
72/*
73 * a single message. it contains a header (src, dest, message type, etc.),
74 * footer (crc values, mainly), a "front" message body, and possibly a
75 * data payload (stored in some number of pages).
76 */
77struct ceph_msg {
78 struct ceph_msg_header hdr; /* header */
79 struct ceph_msg_footer footer; /* footer */
80 struct kvec front; /* unaligned blobs of message */
81 struct ceph_buffer *middle;
82 struct page **pages; /* data payload. NOT OWNER. */
83 unsigned nr_pages; /* size of page array */
84 struct ceph_pagelist *pagelist; /* instead of pages */
85 struct list_head list_head;
86 struct kref kref;
87 bool front_is_vmalloc;
88 bool more_to_follow;
89 int front_max;
90
91 struct ceph_msgpool *pool;
92};
93
94struct ceph_msg_pos {
95 int page, page_pos; /* which page; offset in page */
96 int data_pos; /* offset in data payload */
97 int did_page_crc; /* true if we've calculated crc for current page */
98};
99
100/* ceph connection fault delay defaults, for exponential backoff */
101#define BASE_DELAY_INTERVAL (HZ/2)
102#define MAX_DELAY_INTERVAL (5 * 60 * HZ)
103
104/*
105 * ceph_connection state bit flags
106 *
107 * QUEUED and BUSY are used together to ensure that only a single
108 * thread is currently opening, reading or writing data to the socket.
109 */
110#define LOSSYTX 0 /* we can close channel or drop messages on errors */
111#define CONNECTING 1
112#define NEGOTIATING 2
113#define KEEPALIVE_PENDING 3
114#define WRITE_PENDING 4 /* we have data ready to send */
115#define QUEUED 5 /* there is work queued on this connection */
116#define BUSY 6 /* work is being done */
117#define STANDBY 8 /* no outgoing messages, socket closed. we keep
118 * the ceph_connection around to maintain shared
119 * state with the peer. */
120#define CLOSED 10 /* we've closed the connection */
121#define SOCK_CLOSED 11 /* socket state changed to closed */
122#define OPENING 13 /* open connection w/ (possibly new) peer */
123#define DEAD 14 /* dead, about to kfree */
124
125/*
126 * A single connection with another host.
127 *
128 * We maintain a queue of outgoing messages, and some session state to
129 * ensure that we can preserve the lossless, ordered delivery of
130 * messages in the case of a TCP disconnect.
131 */
132struct ceph_connection {
133 void *private;
134 atomic_t nref;
135
136 const struct ceph_connection_operations *ops;
137
138 struct ceph_messenger *msgr;
139 struct socket *sock;
140 unsigned long state; /* connection state (see flags above) */
141 const char *error_msg; /* error message, if any */
142
143 struct ceph_entity_addr peer_addr; /* peer address */
144 struct ceph_entity_name peer_name; /* peer name */
145 struct ceph_entity_addr peer_addr_for_me;
146 u32 connect_seq; /* identify the most recent connection
147 attempt for this connection, client */
148 u32 peer_global_seq; /* peer's global seq for this connection */
149
150 int auth_retry; /* true if we need a newer authorizer */
151 void *auth_reply_buf; /* where to put the authorizer reply */
152 int auth_reply_buf_len;
153
154 struct mutex mutex;
155
156 /* out queue */
157 struct list_head out_queue;
158 struct list_head out_sent; /* sending or sent but unacked */
159 u64 out_seq; /* last message queued for send */
160 u64 out_seq_sent; /* last message sent */
161 bool out_keepalive_pending;
162
163 u64 in_seq, in_seq_acked; /* last message received, acked */
164
165 /* connection negotiation temps */
166 char in_banner[CEPH_BANNER_MAX_LEN];
167 union {
168 struct { /* outgoing connection */
169 struct ceph_msg_connect out_connect;
170 struct ceph_msg_connect_reply in_reply;
171 };
172 struct { /* incoming */
173 struct ceph_msg_connect in_connect;
174 struct ceph_msg_connect_reply out_reply;
175 };
176 };
177 struct ceph_entity_addr actual_peer_addr;
178
179 /* message out temps */
180 struct ceph_msg *out_msg; /* sending message (== tail of
181 out_sent) */
182 bool out_msg_done;
183 struct ceph_msg_pos out_msg_pos;
184
185 struct kvec out_kvec[8], /* sending header/footer data */
186 *out_kvec_cur;
187 int out_kvec_left; /* kvec's left in out_kvec */
188 int out_skip; /* skip this many bytes */
189 int out_kvec_bytes; /* total bytes left */
190 bool out_kvec_is_msg; /* kvec refers to out_msg */
191 int out_more; /* there is more data after the kvecs */
192 __le64 out_temp_ack; /* for writing an ack */
193
194 /* message in temps */
195 struct ceph_msg_header in_hdr;
196 struct ceph_msg *in_msg;
197 struct ceph_msg_pos in_msg_pos;
198 u32 in_front_crc, in_middle_crc, in_data_crc; /* calculated crc */
199
200 char in_tag; /* protocol control byte */
201 int in_base_pos; /* bytes read */
202 __le64 in_temp_ack; /* for reading an ack */
203
204 struct delayed_work work; /* send|recv work */
205 unsigned long delay; /* current delay interval */
206};
207
208
209extern const char *pr_addr(const struct sockaddr_storage *ss);
210extern int ceph_parse_ips(const char *c, const char *end,
211 struct ceph_entity_addr *addr,
212 int max_count, int *count);
213
214
215extern int ceph_msgr_init(void);
216extern void ceph_msgr_exit(void);
217
218extern struct ceph_messenger *ceph_messenger_create(
219 struct ceph_entity_addr *myaddr);
220extern void ceph_messenger_destroy(struct ceph_messenger *);
221
222extern void ceph_con_init(struct ceph_messenger *msgr,
223 struct ceph_connection *con);
224extern void ceph_con_open(struct ceph_connection *con,
225 struct ceph_entity_addr *addr);
226extern void ceph_con_close(struct ceph_connection *con);
227extern void ceph_con_send(struct ceph_connection *con, struct ceph_msg *msg);
228extern void ceph_con_revoke(struct ceph_connection *con, struct ceph_msg *msg);
229extern void ceph_con_revoke_message(struct ceph_connection *con,
230 struct ceph_msg *msg);
231extern void ceph_con_keepalive(struct ceph_connection *con);
232extern struct ceph_connection *ceph_con_get(struct ceph_connection *con);
233extern void ceph_con_put(struct ceph_connection *con);
234
235extern struct ceph_msg *ceph_msg_new(int type, int front_len,
236 int page_len, int page_off,
237 struct page **pages);
238extern void ceph_msg_kfree(struct ceph_msg *m);
239
240
241static inline struct ceph_msg *ceph_msg_get(struct ceph_msg *msg)
242{
243 kref_get(&msg->kref);
244 return msg;
245}
246extern void ceph_msg_last_put(struct kref *kref);
247static inline void ceph_msg_put(struct ceph_msg *msg)
248{
249 kref_put(&msg->kref, ceph_msg_last_put);
250}
251
252extern void ceph_msg_dump(struct ceph_msg *msg);
253
254#endif
fs/ceph/mon_client.c+834−0
@@ -0,0 +1,834 @@
1#include "ceph_debug.h"
2
3#include <linux/types.h>
4#include <linux/random.h>
5#include <linux/sched.h>
6
7#include "mon_client.h"
8#include "super.h"
9#include "auth.h"
10#include "decode.h"
11
12/*
13 * Interact with Ceph monitor cluster. Handle requests for new map
14 * versions, and periodically resend as needed. Also implement
15 * statfs() and umount().
16 *
17 * A small cluster of Ceph "monitors" are responsible for managing critical
18 * cluster configuration and state information. An odd number (e.g., 3, 5)
19 * of cmon daemons use a modified version of the Paxos part-time parliament
20 * algorithm to manage the MDS map (mds cluster membership), OSD map, and
21 * list of clients who have mounted the file system.
22 *
23 * We maintain an open, active session with a monitor at all times in order to
24 * receive timely MDSMap updates. We periodically send a keepalive byte on the
25 * TCP socket to ensure we detect a failure. If the connection does break, we
26 * randomly hunt for a new monitor. Once the connection is reestablished, we
27 * resend any outstanding requests.
28 */
29
30const static struct ceph_connection_operations mon_con_ops;
31
32static int __validate_auth(struct ceph_mon_client *monc);
33
34/*
35 * Decode a monmap blob (e.g., during mount).
36 */
37struct ceph_monmap *ceph_monmap_decode(void *p, void *end)
38{
39 struct ceph_monmap *m = NULL;
40 int i, err = -EINVAL;
41 struct ceph_fsid fsid;
42 u32 epoch, num_mon;
43 u16 version;
44 u32 len;
45
46 ceph_decode_32_safe(&p, end, len, bad);
47 ceph_decode_need(&p, end, len, bad);
48
49 dout("monmap_decode %p %p len %d\n", p, end, (int)(end-p));
50
51 ceph_decode_16_safe(&p, end, version, bad);
52
53 ceph_decode_need(&p, end, sizeof(fsid) + 2*sizeof(u32), bad);
54 ceph_decode_copy(&p, &fsid, sizeof(fsid));
55 epoch = ceph_decode_32(&p);
56
57 num_mon = ceph_decode_32(&p);
58 ceph_decode_need(&p, end, num_mon*sizeof(m->mon_inst[0]), bad);
59
60 if (num_mon >= CEPH_MAX_MON)
61 goto bad;
62 m = kmalloc(sizeof(*m) + sizeof(m->mon_inst[0])*num_mon, GFP_NOFS);
63 if (m == NULL)
64 return ERR_PTR(-ENOMEM);
65 m->fsid = fsid;
66 m->epoch = epoch;
67 m->num_mon = num_mon;
68 ceph_decode_copy(&p, m->mon_inst, num_mon*sizeof(m->mon_inst[0]));
69 for (i = 0; i < num_mon; i++)
70 ceph_decode_addr(&m->mon_inst[i].addr);
71
72 dout("monmap_decode epoch %d, num_mon %d\n", m->epoch,
73 m->num_mon);
74 for (i = 0; i < m->num_mon; i++)
75 dout("monmap_decode mon%d is %s\n", i,
76 pr_addr(&m->mon_inst[i].addr.in_addr));
77 return m;
78
79bad:
80 dout("monmap_decode failed with %d\n", err);
81 kfree(m);
82 return ERR_PTR(err);
83}
84
85/*
86 * return true if *addr is included in the monmap.
87 */
88int ceph_monmap_contains(struct ceph_monmap *m, struct ceph_entity_addr *addr)
89{
90 int i;
91
92 for (i = 0; i < m->num_mon; i++)
93 if (memcmp(addr, &m->mon_inst[i].addr, sizeof(*addr)) == 0)
94 return 1;
95 return 0;
96}
97
98/*
99 * Send an auth request.
100 */
101static void __send_prepared_auth_request(struct ceph_mon_client *monc, int len)
102{
103 monc->pending_auth = 1;
104 monc->m_auth->front.iov_len = len;
105 monc->m_auth->hdr.front_len = cpu_to_le32(len);
106 ceph_msg_get(monc->m_auth); /* keep our ref */
107 ceph_con_send(monc->con, monc->m_auth);
108}
109
110/*
111 * Close monitor session, if any.
112 */
113static void __close_session(struct ceph_mon_client *monc)
114{
115 if (monc->con) {
116 dout("__close_session closing mon%d\n", monc->cur_mon);
117 ceph_con_revoke(monc->con, monc->m_auth);
118 ceph_con_close(monc->con);
119 monc->cur_mon = -1;
120 monc->pending_auth = 0;
121 ceph_auth_reset(monc->auth);
122 }
123}
124
125/*
126 * Open a session with a (new) monitor.
127 */
128static int __open_session(struct ceph_mon_client *monc)
129{
130 char r;
131 int ret;
132
133 if (monc->cur_mon < 0) {
134 get_random_bytes(&r, 1);
135 monc->cur_mon = r % monc->monmap->num_mon;
136 dout("open_session num=%d r=%d -> mon%d\n",
137 monc->monmap->num_mon, r, monc->cur_mon);
138 monc->sub_sent = 0;
139 monc->sub_renew_after = jiffies; /* i.e., expired */
140 monc->want_next_osdmap = !!monc->want_next_osdmap;
141
142 dout("open_session mon%d opening\n", monc->cur_mon);
143 monc->con->peer_name.type = CEPH_ENTITY_TYPE_MON;
144 monc->con->peer_name.num = cpu_to_le64(monc->cur_mon);
145 ceph_con_open(monc->con,
146 &monc->monmap->mon_inst[monc->cur_mon].addr);
147
148 /* initiatiate authentication handshake */
149 ret = ceph_auth_build_hello(monc->auth,
150 monc->m_auth->front.iov_base,
151 monc->m_auth->front_max);
152 __send_prepared_auth_request(monc, ret);
153 } else {
154 dout("open_session mon%d already open\n", monc->cur_mon);
155 }
156 return 0;
157}
158
159static bool __sub_expired(struct ceph_mon_client *monc)
160{
161 return time_after_eq(jiffies, monc->sub_renew_after);
162}
163
164/*
165 * Reschedule delayed work timer.
166 */
167static void __schedule_delayed(struct ceph_mon_client *monc)
168{
169 unsigned delay;
170
171 if (monc->cur_mon < 0 || __sub_expired(monc))
172 delay = 10 * HZ;
173 else
174 delay = 20 * HZ;
175 dout("__schedule_delayed after %u\n", delay);
176 schedule_delayed_work(&monc->delayed_work, delay);
177}
178
179/*
180 * Send subscribe request for mdsmap and/or osdmap.
181 */
182static void __send_subscribe(struct ceph_mon_client *monc)
183{
184 dout("__send_subscribe sub_sent=%u exp=%u want_osd=%d\n",
185 (unsigned)monc->sub_sent, __sub_expired(monc),
186 monc->want_next_osdmap);
187 if ((__sub_expired(monc) && !monc->sub_sent) ||
188 monc->want_next_osdmap == 1) {
189 struct ceph_msg *msg;
190 struct ceph_mon_subscribe_item *i;
191 void *p, *end;
192
193 msg = ceph_msg_new(CEPH_MSG_MON_SUBSCRIBE, 96, 0, 0, NULL);
194 if (!msg)
195 return;
196
197 p = msg->front.iov_base;
198 end = p + msg->front.iov_len;
199
200 dout("__send_subscribe to 'mdsmap' %u+\n",
201 (unsigned)monc->have_mdsmap);
202 if (monc->want_next_osdmap) {
203 dout("__send_subscribe to 'osdmap' %u\n",
204 (unsigned)monc->have_osdmap);
205 ceph_encode_32(&p, 3);
206 ceph_encode_string(&p, end, "osdmap", 6);
207 i = p;
208 i->have = cpu_to_le64(monc->have_osdmap);
209 i->onetime = 1;
210 p += sizeof(*i);
211 monc->want_next_osdmap = 2; /* requested */
212 } else {
213 ceph_encode_32(&p, 2);
214 }
215 ceph_encode_string(&p, end, "mdsmap", 6);
216 i = p;
217 i->have = cpu_to_le64(monc->have_mdsmap);
218 i->onetime = 0;
219 p += sizeof(*i);
220 ceph_encode_string(&p, end, "monmap", 6);
221 i = p;
222 i->have = 0;
223 i->onetime = 0;

611 more lines aren’t shown here.

fs/ceph/mon_client.h+119−0

This diff is large: its 120 lines aren’t shown here.

fs/ceph/msgpool.c+186−0

This diff is large: its 187 lines aren’t shown here.

fs/ceph/msgpool.h+27−0

This diff is large: its 28 lines aren’t shown here.

fs/ceph/msgr.h+158−0

This diff is large: its 159 lines aren’t shown here.

fs/ceph/osd_client.c+1537−0

This diff is large: its 1538 lines aren’t shown here.

fs/ceph/osd_client.h+166−0

This diff is large: its 167 lines aren’t shown here.

fs/ceph/osdmap.c+1019−0

This diff is large: its 1020 lines aren’t shown here.

fs/ceph/osdmap.h+125−0

This diff is large: its 126 lines aren’t shown here.

fs/ceph/pagelist.c+54−0

This diff is large: its 55 lines aren’t shown here.

fs/ceph/pagelist.h+54−0

This diff is large: its 55 lines aren’t shown here.

fs/ceph/rados.h+374−0

This diff is large: its 375 lines aren’t shown here.

fs/ceph/snap.c+904−0

This diff is large: its 905 lines aren’t shown here.

fs/ceph/super.c+1030−0

This diff is large: its 1031 lines aren’t shown here.

fs/ceph/super.h+901−0

This diff is large: its 902 lines aren’t shown here.

fs/ceph/types.h+29−0

This diff is large: its 30 lines aren’t shown here.

fs/ceph/xattr.c+844−0

This diff is large: its 845 lines aren’t shown here.

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t