History

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client

* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client: (205 commits) ceph: update for write_inode API change ceph: reset osd after relevant messages timed out ceph: fix flush_dirty_caps race with caps migration ceph: include migrating caps in issued set ceph: fix osdmap decoding when pools include (removed) snaps ceph: return EBADF if waiting for caps on closed file ceph: set osd request message front length correctly ceph: reset front len on return to msgpool; BUG on mismatched front iov ceph: fix snaptrace decoding on cap migration between mds ceph: use single osd op reply msg ceph: reset bits on connection close ceph: remove bogus mds forward warning ceph: remove fragile __map_osds optimization ceph: fix connection fault STANDBY check ceph: invalidate_authorizer without con->mutex held ceph: don't clobber write return value when using O_SYNC ceph: fix client_request_forward decoding ceph: drop messages on unregistered mds sessions; cleanup ceph: fix comments, locking in destroy_inode ceph: move dereference after NULL test ... Fix trivial conflicts in Documentation/ioctl/ioctl-number.txt

Linus Torvalds committed 16 years ago

Commit fc7f99c2 parents 0a492fd f1a3d5767 files · +28066 −0Browse files

Unified · Split

67 files
  1. Documentation/filesystems/ceph.txt +139 −0
  2. Documentation/ioctl/ioctl-number.txt +1 −0
  3. MAINTAINERS +9 −0
  4. fs/Kconfig +1 −0
  5. fs/Makefile +1 −0
  6. fs/ceph/Kconfig +27 −0
  7. fs/ceph/Makefile +39 −0
  8. fs/ceph/README +20 −0
  9. fs/ceph/addr.c +1188 −0
  10. fs/ceph/armor.c +99 −0
  11. fs/ceph/auth.c +257 −0
  12. fs/ceph/auth.h +84 −0
  13. fs/ceph/auth_none.c +121 −0
  14. fs/ceph/auth_none.h +28 −0
  15. fs/ceph/auth_x.c +656 −0
  16. fs/ceph/auth_x.h +49 −0
  17. fs/ceph/auth_x_protocol.h +90 −0
  18. fs/ceph/buffer.c +78 −0
  19. fs/ceph/buffer.h +39 −0
  20. fs/ceph/caps.c +2927 −0
  21. fs/ceph/ceph_debug.h +37 −0
  22. fs/ceph/ceph_frag.c +21 −0
  23. fs/ceph/ceph_frag.h +109 −0
  24. fs/ceph/ceph_fs.c +74 −0
  25. fs/ceph/ceph_fs.h +650 −0
  26. fs/ceph/ceph_hash.c +118 −0
  27. fs/ceph/ceph_hash.h +13 −0
  28. fs/ceph/ceph_strings.c +176 −0
  29. fs/ceph/crush/crush.c +151 −0
  30. fs/ceph/crush/crush.h +180 −0
  31. fs/ceph/crush/hash.c +149 −0
  32. fs/ceph/crush/hash.h +17 −0
  33. fs/ceph/crush/mapper.c +596 −0
  34. fs/ceph/crush/mapper.h +20 −0
  35. fs/ceph/crypto.c +408 −0
  36. fs/ceph/crypto.h +48 −0
  37. fs/ceph/debugfs.c +483 −0
  38. fs/ceph/decode.h +194 −0
  39. fs/ceph/dir.c +1220 −0
  40. fs/ceph/export.c +223 −0
  41. fs/ceph/file.c +937 −0
  42. fs/ceph/inode.c +1750 −0
  43. fs/ceph/ioctl.c +160 −0
  44. fs/ceph/ioctl.h +40 −0
  45. fs/ceph/mds_client.c +3021 −0
  46. fs/ceph/mds_client.h +335 −0
  47. fs/ceph/mdsmap.c +174 −0
  48. fs/ceph/mdsmap.h +54 −0
  49. fs/ceph/messenger.c +2240 −0
  50. fs/ceph/messenger.h +254 −0
  51. fs/ceph/mon_client.c +834 −0
  52. fs/ceph/mon_client.h +119 −0
  53. fs/ceph/msgpool.c +186 −0
  54. fs/ceph/msgpool.h +27 −0
  55. fs/ceph/msgr.h +158 −0
  56. fs/ceph/osd_client.c +1537 −0
  57. fs/ceph/osd_client.h +166 −0
  58. fs/ceph/osdmap.c +1019 −0
  59. fs/ceph/osdmap.h +125 −0
  60. fs/ceph/pagelist.c +54 −0
  61. fs/ceph/pagelist.h +54 −0
  62. fs/ceph/rados.h +374 −0
  63. fs/ceph/snap.c +904 −0
  64. fs/ceph/super.c +1030 −0
  65. fs/ceph/super.h +901 −0
  66. fs/ceph/types.h +29 −0
  67. fs/ceph/xattr.c +844 −0
Documentation/filesystems/ceph.txt+139−0
@@ -0,0 +1,139 @@
1+Ceph Distributed File System
2+============================
3+
4+Ceph is a distributed network file system designed to provide good
5+performance, reliability, and scalability.
6+
7+Basic features include:
8+
9+ * POSIX semantics
10+ * Seamless scaling from 1 to many thousands of nodes
11+ * High availability and reliability. No single points of failure.
12+ * N-way replication of data across storage nodes
13+ * Fast recovery from node failures
14+ * Automatic rebalancing of data on node addition/removal
15+ * Easy deployment: most FS components are userspace daemons
16+
17+Also,
18+ * Flexible snapshots (on any directory)
19+ * Recursive accounting (nested files, directories, bytes)
20+
21+In contrast to cluster filesystems like GFS, OCFS2, and GPFS that rely
22+on symmetric access by all clients to shared block devices, Ceph
23+separates data and metadata management into independent server
24+clusters, similar to Lustre. Unlike Lustre, however, metadata and
25+storage nodes run entirely as user space daemons. Storage nodes
26+utilize btrfs to store data objects, leveraging its advanced features
27+(checksumming, metadata replication, etc.). File data is striped
28+across storage nodes in large chunks to distribute workload and
29+facilitate high throughputs. When storage nodes fail, data is
30+re-replicated in a distributed fashion by the storage nodes themselves
31+(with some minimal coordination from a cluster monitor), making the
32+system extremely efficient and scalable.
33+
34+Metadata servers effectively form a large, consistent, distributed
35+in-memory cache above the file namespace that is extremely scalable,
36+dynamically redistributes metadata in response to workload changes,
37+and can tolerate arbitrary (well, non-Byzantine) node failures. The
38+metadata server takes a somewhat unconventional approach to metadata
39+storage to significantly improve performance for common workloads. In
40+particular, inodes with only a single link are embedded in
41+directories, allowing entire directories of dentries and inodes to be
42+loaded into its cache with a single I/O operation. The contents of
43+extremely large directories can be fragmented and managed by
44+independent metadata servers, allowing scalable concurrent access.
45+
46+The system offers automatic data rebalancing/migration when scaling
47+from a small cluster of just a few nodes to many hundreds, without
48+requiring an administrator carve the data set into static volumes or
49+go through the tedious process of migrating data between servers.
50+When the file system approaches full, new nodes can be easily added
51+and things will "just work."
52+
53+Ceph includes flexible snapshot mechanism that allows a user to create
54+a snapshot on any subdirectory (and its nested contents) in the
55+system. Snapshot creation and deletion are as simple as 'mkdir
56+.snap/foo' and 'rmdir .snap/foo'.
57+
58+Ceph also provides some recursive accounting on directories for nested
59+files and bytes. That is, a 'getfattr -d foo' on any directory in the
60+system will reveal the total number of nested regular files and
61+subdirectories, and a summation of all nested file sizes. This makes
62+the identification of large disk space consumers relatively quick, as
63+no 'du' or similar recursive scan of the file system is required.
64+
65+
66+Mount Syntax
67+============
68+
69+The basic mount syntax is:
70+
71+ # mount -t ceph monip[:port][,monip2[:port]...]:/[subdir] mnt
72+
73+You only need to specify a single monitor, as the client will get the
74+full list when it connects. (However, if the monitor you specify
75+happens to be down, the mount won't succeed.) The port can be left
76+off if the monitor is using the default. So if the monitor is at
77+1.2.3.4,
78+
79+ # mount -t ceph 1.2.3.4:/ /mnt/ceph
80+
81+is sufficient. If /sbin/mount.ceph is installed, a hostname can be
82+used instead of an IP address.
83+
84+
85+
86+Mount Options
87+=============
88+
89+ ip=A.B.C.D[:N]
90+ Specify the IP and/or port the client should bind to locally.
91+ There is normally not much reason to do this. If the IP is not
92+ specified, the client's IP address is determined by looking at the
93+ address it's connection to the monitor originates from.
94+
95+ wsize=X
96+ Specify the maximum write size in bytes. By default there is no
97+ maximu. Ceph will normally size writes based on the file stripe
98+ size.
99+
100+ rsize=X
101+ Specify the maximum readahead.
102+
103+ mount_timeout=X
104+ Specify the timeout value for mount (in seconds), in the case
105+ of a non-responsive Ceph file system. The default is 30
106+ seconds.
107+
108+ rbytes
109+ When stat() is called on a directory, set st_size to 'rbytes',
110+ the summation of file sizes over all files nested beneath that
111+ directory. This is the default.
112+
113+ norbytes
114+ When stat() is called on a directory, set st_size to the
115+ number of entries in that directory.
116+
117+ nocrc
118+ Disable CRC32C calculation for data writes. If set, the OSD
119+ must rely on TCP's error correction to detect data corruption
120+ in the data payload.
121+
122+ noasyncreaddir
123+ Disable client's use its local cache to satisfy readdir
124+ requests. (This does not change correctness; the client uses
125+ cached metadata only when a lease or capability ensures it is
126+ valid.)
127+
128+
129+More Information
130+================
131+
132+For more information on Ceph, see the home page at
133+ http://ceph.newdream.net/
134+
135+The Linux kernel client source tree is available at
136+ git://ceph.newdream.net/linux-ceph-client.git
137+
138+and the source for the full system is at
139+ git://ceph.newdream.net/ceph.git
Documentation/ioctl/ioctl-number.txt+1−0
@@ -291,6 +291,7 @@ Code Seq#(hex) Include File Comments · 290 unmodified lines
291291 0x92 00-0F drivers/usb/mon/mon_bin.c
292292 0x93 60-7F linux/auto_fs.h
293293 0x94 all fs/btrfs/ioctl.h
294+0x97 00-7F fs/ceph/ioctl.h Ceph file system
294295 0x99 00-0F 537-Addinboard driver
295296 <mailto:buk@buks.ipn.de>
296297 0xA0 all linux/sdp/sdp.h Industrial Device Project
MAINTAINERS+9−0
@@ -1441,6 +1441,15 @@ F: arch/powerpc/include/asm/spu*.h · 1440 unmodified lines
14411441 F: arch/powerpc/oprofile/*cell*
14421442 F: arch/powerpc/platforms/cell/
14431443
1444+CEPH DISTRIBUTED FILE SYSTEM CLIENT
1445+M: Sage Weil <sage@newdream.net>
1446+L: ceph-devel@lists.sourceforge.net
1447+W: http://ceph.newdream.net/
1448+T: git git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client.git
1449+S: Supported
1450+F: Documentation/filesystems/ceph.txt
1451+F: fs/ceph
1452+
14441453 CERTIFIED WIRELESS USB (WUSB) SUBSYSTEM:
14451454 M: David Vrabel <david.vrabel@csr.com>
14461455 L: linux-usb@vger.kernel.org
fs/Kconfig+1−0
@@ -235,6 +235,7 @@ config NFS_COMMON · 234 unmodified lines
235235
236236 source "net/sunrpc/Kconfig"
237237 source "fs/smbfs/Kconfig"
238+source "fs/ceph/Kconfig"
238239 source "fs/cifs/Kconfig"
239240 source "fs/ncpfs/Kconfig"
240241 source "fs/coda/Kconfig"
fs/Makefile+1−0
@@ -125,3 +125,4 @@ obj-$(CONFIG_OCFS2_FS) += ocfs2/ · 124 unmodified lines
125125 obj-$(CONFIG_BTRFS_FS) += btrfs/
126126 obj-$(CONFIG_GFS2_FS) += gfs2/
127127 obj-$(CONFIG_EXOFS_FS) += exofs/
128+obj-$(CONFIG_CEPH_FS) += ceph/
fs/ceph/Kconfig+27−0
@@ -0,0 +1,27 @@
1+config CEPH_FS
2+ tristate "Ceph distributed file system (EXPERIMENTAL)"
3+ depends on INET && EXPERIMENTAL
4+ select LIBCRC32C
5+ select CONFIG_CRYPTO_AES
6+ help
7+ Choose Y or M here to include support for mounting the
8+ experimental Ceph distributed file system. Ceph is an extremely
9+ scalable file system designed to provide high performance,
10+ reliable access to petabytes of storage.
11+
12+ More information at http://ceph.newdream.net/.
13+
14+ If unsure, say N.
15+
16+config CEPH_FS_PRETTYDEBUG
17+ bool "Include file:line in ceph debug output"
18+ depends on CEPH_FS
19+ default n
20+ help
21+ If you say Y here, debug output will include a filename and
22+ line to aid debugging. This icnreases kernel size and slows
23+ execution slightly when debug call sites are enabled (e.g.,
24+ via CONFIG_DYNAMIC_DEBUG).
25+
26+ If unsure, say N.
27+
fs/ceph/Makefile+39−0
@@ -0,0 +1,39 @@
1+#
2+# Makefile for CEPH filesystem.
3+#
4+
5+ifneq ($(KERNELRELEASE),)
6+
7+obj-$(CONFIG_CEPH_FS) += ceph.o
8+
9+ceph-objs := super.o inode.o dir.o file.o addr.o ioctl.o \
10+ export.o caps.o snap.o xattr.o \
11+ messenger.o msgpool.o buffer.o pagelist.o \
12+ mds_client.o mdsmap.o \
13+ mon_client.o \
14+ osd_client.o osdmap.o crush/crush.o crush/mapper.o crush/hash.o \
15+ debugfs.o \
16+ auth.o auth_none.o \
17+ crypto.o armor.o \
18+ auth_x.o \
19+ ceph_fs.o ceph_strings.o ceph_hash.o ceph_frag.o
20+
21+else
22+#Otherwise we were called directly from the command
23+# line; invoke the kernel build system.
24+
25+KERNELDIR ?= /lib/modules/$(shell uname -r)/build
26+PWD := $(shell pwd)
27+
28+default: all
29+
30+all:
31+ $(MAKE) -C $(KERNELDIR) M=$(PWD) CONFIG_CEPH_FS=m modules
32+
33+modules_install:
34+ $(MAKE) -C $(KERNELDIR) M=$(PWD) CONFIG_CEPH_FS=m modules_install
35+
36+clean:
37+ $(MAKE) -C $(KERNELDIR) M=$(PWD) clean
38+
39+endif
fs/ceph/README+20−0
@@ -0,0 +1,20 @@
1+#
2+# The following files are shared by (and manually synchronized
3+# between) the Ceph userland and kernel client.
4+#
5+# userland kernel
6+src/include/ceph_fs.h fs/ceph/ceph_fs.h
7+src/include/ceph_fs.cc fs/ceph/ceph_fs.c
8+src/include/msgr.h fs/ceph/msgr.h
9+src/include/rados.h fs/ceph/rados.h
10+src/include/ceph_strings.cc fs/ceph/ceph_strings.c
11+src/include/ceph_frag.h fs/ceph/ceph_frag.h
12+src/include/ceph_frag.cc fs/ceph/ceph_frag.c
13+src/include/ceph_hash.h fs/ceph/ceph_hash.h
14+src/include/ceph_hash.cc fs/ceph/ceph_hash.c
15+src/crush/crush.c fs/ceph/crush/crush.c
16+src/crush/crush.h fs/ceph/crush/crush.h
17+src/crush/mapper.c fs/ceph/crush/mapper.c
18+src/crush/mapper.h fs/ceph/crush/mapper.h
19+src/crush/hash.h fs/ceph/crush/hash.h
20+src/crush/hash.c fs/ceph/crush/hash.c
fs/ceph/addr.c+1188−0
@@ -0,0 +1,1188 @@
1+#include "ceph_debug.h"
2+
3+#include <linux/backing-dev.h>
4+#include <linux/fs.h>
5+#include <linux/mm.h>
6+#include <linux/pagemap.h>
7+#include <linux/writeback.h> /* generic_writepages */
8+#include <linux/pagevec.h>
9+#include <linux/task_io_accounting_ops.h>
10+
11+#include "super.h"
12+#include "osd_client.h"
13+
14+/*
15+ * Ceph address space ops.
16+ *
17+ * There are a few funny things going on here.
18+ *
19+ * The page->private field is used to reference a struct
20+ * ceph_snap_context for _every_ dirty page. This indicates which
21+ * snapshot the page was logically dirtied in, and thus which snap
22+ * context needs to be associated with the osd write during writeback.
23+ *
24+ * Similarly, struct ceph_inode_info maintains a set of counters to
25+ * count dirty pages on the inode. In the absense of snapshots,
26+ * i_wrbuffer_ref == i_wrbuffer_ref_head == the dirty page count.
27+ *
28+ * When a snapshot is taken (that is, when the client receives
29+ * notification that a snapshot was taken), each inode with caps and
30+ * with dirty pages (dirty pages implies there is a cap) gets a new
31+ * ceph_cap_snap in the i_cap_snaps list (which is sorted in ascending
32+ * order, new snaps go to the tail). The i_wrbuffer_ref_head count is
33+ * moved to capsnap->dirty. (Unless a sync write is currently in
34+ * progress. In that case, the capsnap is said to be "pending", new
35+ * writes cannot start, and the capsnap isn't "finalized" until the
36+ * write completes (or fails) and a final size/mtime for the inode for
37+ * that snap can be settled upon.) i_wrbuffer_ref_head is reset to 0.
38+ *
39+ * On writeback, we must submit writes to the osd IN SNAP ORDER. So,
40+ * we look for the first capsnap in i_cap_snaps and write out pages in
41+ * that snap context _only_. Then we move on to the next capsnap,
42+ * eventually reaching the "live" or "head" context (i.e., pages that
43+ * are not yet snapped) and are writing the most recently dirtied
44+ * pages.
45+ *
46+ * Invalidate and so forth must take care to ensure the dirty page
47+ * accounting is preserved.
48+ */
49+
50+#define CONGESTION_ON_THRESH(congestion_kb) (congestion_kb >> (PAGE_SHIFT-10))
51+#define CONGESTION_OFF_THRESH(congestion_kb) \
52+ (CONGESTION_ON_THRESH(congestion_kb) - \
53+ (CONGESTION_ON_THRESH(congestion_kb) >> 2))
54+
55+
56+
57+/*
58+ * Dirty a page. Optimistically adjust accounting, on the assumption
59+ * that we won't race with invalidate. If we do, readjust.
60+ */
61+static int ceph_set_page_dirty(struct page *page)
62+{
63+ struct address_space *mapping = page->mapping;
64+ struct inode *inode;
65+ struct ceph_inode_info *ci;
66+ int undo = 0;
67+ struct ceph_snap_context *snapc;
68+
69+ if (unlikely(!mapping))
70+ return !TestSetPageDirty(page);
71+
72+ if (TestSetPageDirty(page)) {
73+ dout("%p set_page_dirty %p idx %lu -- already dirty\n",
74+ mapping->host, page, page->index);
75+ return 0;
76+ }
77+
78+ inode = mapping->host;
79+ ci = ceph_inode(inode);
80+
81+ /*
82+ * Note that we're grabbing a snapc ref here without holding
83+ * any locks!
84+ */
85+ snapc = ceph_get_snap_context(ci->i_snap_realm->cached_context);
86+
87+ /* dirty the head */
88+ spin_lock(&inode->i_lock);
89+ if (ci->i_wrbuffer_ref_head == 0)
90+ ci->i_head_snapc = ceph_get_snap_context(snapc);
91+ ++ci->i_wrbuffer_ref_head;
92+ if (ci->i_wrbuffer_ref == 0)
93+ igrab(inode);
94+ ++ci->i_wrbuffer_ref;
95+ dout("%p set_page_dirty %p idx %lu head %d/%d -> %d/%d "
96+ "snapc %p seq %lld (%d snaps)\n",
97+ mapping->host, page, page->index,
98+ ci->i_wrbuffer_ref-1, ci->i_wrbuffer_ref_head-1,
99+ ci->i_wrbuffer_ref, ci->i_wrbuffer_ref_head,
100+ snapc, snapc->seq, snapc->num_snaps);
101+ spin_unlock(&inode->i_lock);
102+
103+ /* now adjust page */
104+ spin_lock_irq(&mapping->tree_lock);
105+ if (page->mapping) { /* Race with truncate? */
106+ WARN_ON_ONCE(!PageUptodate(page));
107+
108+ if (mapping_cap_account_dirty(mapping)) {
109+ __inc_zone_page_state(page, NR_FILE_DIRTY);
110+ __inc_bdi_stat(mapping->backing_dev_info,
111+ BDI_RECLAIMABLE);
112+ task_io_account_write(PAGE_CACHE_SIZE);
113+ }
114+ radix_tree_tag_set(&mapping->page_tree,
115+ page_index(page), PAGECACHE_TAG_DIRTY);
116+
117+ /*
118+ * Reference snap context in page->private. Also set
119+ * PagePrivate so that we get invalidatepage callback.
120+ */
121+ page->private = (unsigned long)snapc;
122+ SetPagePrivate(page);
123+ } else {
124+ dout("ANON set_page_dirty %p (raced truncate?)\n", page);
125+ undo = 1;
126+ }
127+
128+ spin_unlock_irq(&mapping->tree_lock);
129+
130+ if (undo)
131+ /* whoops, we failed to dirty the page */
132+ ceph_put_wrbuffer_cap_refs(ci, 1, snapc);
133+
134+ __mark_inode_dirty(mapping->host, I_DIRTY_PAGES);
135+
136+ BUG_ON(!PageDirty(page));
137+ return 1;
138+}
139+
140+/*
141+ * If we are truncating the full page (i.e. offset == 0), adjust the
142+ * dirty page counters appropriately. Only called if there is private
143+ * data on the page.
144+ */
145+static void ceph_invalidatepage(struct page *page, unsigned long offset)
146+{
147+ struct inode *inode;
148+ struct ceph_inode_info *ci;
149+ struct ceph_snap_context *snapc = (void *)page->private;
150+
151+ BUG_ON(!PageLocked(page));
152+ BUG_ON(!page->private);
153+ BUG_ON(!PagePrivate(page));
154+ BUG_ON(!page->mapping);
155+
156+ inode = page->mapping->host;
157+
158+ /*
159+ * We can get non-dirty pages here due to races between
160+ * set_page_dirty and truncate_complete_page; just spit out a
161+ * warning, in case we end up with accounting problems later.
162+ */
163+ if (!PageDirty(page))
164+ pr_err("%p invalidatepage %p page not dirty\n", inode, page);
165+
166+ if (offset == 0)
167+ ClearPageChecked(page);
168+
169+ ci = ceph_inode(inode);
170+ if (offset == 0) {
171+ dout("%p invalidatepage %p idx %lu full dirty page %lu\n",
172+ inode, page, page->index, offset);
173+ ceph_put_wrbuffer_cap_refs(ci, 1, snapc);
174+ ceph_put_snap_context(snapc);
175+ page->private = 0;
176+ ClearPagePrivate(page);
177+ } else {
178+ dout("%p invalidatepage %p idx %lu partial dirty page\n",
179+ inode, page, page->index);
180+ }
181+}
182+
183+/* just a sanity check */
184+static int ceph_releasepage(struct page *page, gfp_t g)
185+{
186+ struct inode *inode = page->mapping ? page->mapping->host : NULL;
187+ dout("%p releasepage %p idx %lu\n", inode, page, page->index);
188+ WARN_ON(PageDirty(page));
189+ WARN_ON(page->private);
190+ WARN_ON(PagePrivate(page));
191+ return 0;
192+}
193+
194+/*
195+ * read a single page, without unlocking it.
196+ */
197+static int readpage_nounlock(struct file *filp, struct page *page)
198+{
199+ struct inode *inode = filp->f_dentry->d_inode;
200+ struct ceph_inode_info *ci = ceph_inode(inode);
201+ struct ceph_osd_client *osdc = &ceph_inode_to_client(inode)->osdc;
202+ int err = 0;
203+ u64 len = PAGE_CACHE_SIZE;
204+
205+ dout("readpage inode %p file %p page %p index %lu\n",
206+ inode, filp, page, page->index);
207+ err = ceph_osdc_readpages(osdc, ceph_vino(inode), &ci->i_layout,
208+ page->index << PAGE_CACHE_SHIFT, &len,
209+ ci->i_truncate_seq, ci->i_truncate_size,
210+ &page, 1);
211+ if (err == -ENOENT)
212+ err = 0;
213+ if (err < 0) {
214+ SetPageError(page);
215+ goto out;
216+ } else if (err < PAGE_CACHE_SIZE) {
217+ /* zero fill remainder of page */
218+ zero_user_segment(page, err, PAGE_CACHE_SIZE);
219+ }
220+ SetPageUptodate(page);
221+
222+out:
223+ return err < 0 ? err : 0;
224+}
225+
226+static int ceph_readpage(struct file *filp, struct page *page)
227+{
228+ int r = readpage_nounlock(filp, page);
229+ unlock_page(page);
230+ return r;
231+}
232+
233+/*
234+ * Build a vector of contiguous pages from the provided page list.
235+ */
236+static struct page **page_vector_from_list(struct list_head *page_list,
237+ unsigned *nr_pages)
238+{
239+ struct page **pages;
240+ struct page *page;
241+ int next_index, contig_pages = 0;
242+
243+ /* build page vector */
244+ pages = kmalloc(sizeof(*pages) * *nr_pages, GFP_NOFS);
245+ if (!pages)
246+ return ERR_PTR(-ENOMEM);
247+
248+ BUG_ON(list_empty(page_list));
249+ next_index = list_entry(page_list->prev, struct page, lru)->index;
250+ list_for_each_entry_reverse(page, page_list, lru) {
251+ if (page->index == next_index) {
252+ dout("readpages page %d %p\n", contig_pages, page);
253+ pages[contig_pages] = page;
254+ contig_pages++;
255+ next_index++;
256+ } else {
257+ break;
258+ }
259+ }
260+ *nr_pages = contig_pages;
261+ return pages;
262+}
263+
264+/*
265+ * Read multiple pages. Leave pages we don't read + unlock in page_list;
266+ * the caller (VM) cleans them up.
267+ */
268+static int ceph_readpages(struct file *file, struct address_space *mapping,
269+ struct list_head *page_list, unsigned nr_pages)
270+{
271+ struct inode *inode = file->f_dentry->d_inode;
272+ struct ceph_inode_info *ci = ceph_inode(inode);
273+ struct ceph_osd_client *osdc = &ceph_inode_to_client(inode)->osdc;
274+ int rc = 0;
275+ struct page **pages;
276+ struct pagevec pvec;
277+ loff_t offset;
278+ u64 len;
279+
280+ dout("readpages %p file %p nr_pages %d\n",
281+ inode, file, nr_pages);
282+
283+ pages = page_vector_from_list(page_list, &nr_pages);
284+ if (IS_ERR(pages))
285+ return PTR_ERR(pages);
286+
287+ /* guess read extent */
288+ offset = pages[0]->index << PAGE_CACHE_SHIFT;
289+ len = nr_pages << PAGE_CACHE_SHIFT;
290+ rc = ceph_osdc_readpages(osdc, ceph_vino(inode), &ci->i_layout,
291+ offset, &len,
292+ ci->i_truncate_seq, ci->i_truncate_size,
293+ pages, nr_pages);
294+ if (rc == -ENOENT)
295+ rc = 0;
296+ if (rc < 0)
297+ goto out;
298+
299+ /* set uptodate and add to lru in pagevec-sized chunks */
300+ pagevec_init(&pvec, 0);
301+ for (; !list_empty(page_list) && len > 0;
302+ rc -= PAGE_CACHE_SIZE, len -= PAGE_CACHE_SIZE) {
303+ struct page *page =
304+ list_entry(page_list->prev, struct page, lru);
305+
306+ list_del(&page->lru);
307+
308+ if (rc < (int)PAGE_CACHE_SIZE) {
309+ /* zero (remainder of) page */
310+ int s = rc < 0 ? 0 : rc;
311+ zero_user_segment(page, s, PAGE_CACHE_SIZE);
312+ }
313+
314+ if (add_to_page_cache(page, mapping, page->index, GFP_NOFS)) {
315+ page_cache_release(page);
316+ dout("readpages %p add_to_page_cache failed %p\n",
317+ inode, page);
318+ continue;
319+ }
320+ dout("readpages %p adding %p idx %lu\n", inode, page,
321+ page->index);
322+ flush_dcache_page(page);
323+ SetPageUptodate(page);
324+ unlock_page(page);
325+ if (pagevec_add(&pvec, page) == 0)
326+ pagevec_lru_add_file(&pvec); /* add to lru */
327+ }
328+ pagevec_lru_add_file(&pvec);
329+ rc = 0;
330+
331+out:
332+ kfree(pages);
333+ return rc;
334+}
335+
336+/*
337+ * Get ref for the oldest snapc for an inode with dirty data... that is, the
338+ * only snap context we are allowed to write back.
339+ *
340+ * Caller holds i_lock.
341+ */
342+static struct ceph_snap_context *__get_oldest_context(struct inode *inode,
343+ u64 *snap_size)
344+{
345+ struct ceph_inode_info *ci = ceph_inode(inode);
346+ struct ceph_snap_context *snapc = NULL;
347+ struct ceph_cap_snap *capsnap = NULL;
348+
349+ list_for_each_entry(capsnap, &ci->i_cap_snaps, ci_item) {
350+ dout(" cap_snap %p snapc %p has %d dirty pages\n", capsnap,
351+ capsnap->context, capsnap->dirty_pages);
352+ if (capsnap->dirty_pages) {
353+ snapc = ceph_get_snap_context(capsnap->context);
354+ if (snap_size)
355+ *snap_size = capsnap->size;
356+ break;
357+ }
358+ }
359+ if (!snapc && ci->i_snap_realm) {
360+ snapc = ceph_get_snap_context(ci->i_snap_realm->cached_context);
361+ dout(" head snapc %p has %d dirty pages\n",
362+ snapc, ci->i_wrbuffer_ref_head);
363+ }
364+ return snapc;
365+}
366+
367+static struct ceph_snap_context *get_oldest_context(struct inode *inode,
368+ u64 *snap_size)
369+{
370+ struct ceph_snap_context *snapc = NULL;
371+
372+ spin_lock(&inode->i_lock);
373+ snapc = __get_oldest_context(inode, snap_size);
374+ spin_unlock(&inode->i_lock);
375+ return snapc;
376+}
377+
378+/*
379+ * Write a single page, but leave the page locked.
380+ *
381+ * If we get a write error, set the page error bit, but still adjust the
382+ * dirty page accounting (i.e., page is no longer dirty).
383+ */
384+static int writepage_nounlock(struct page *page, struct writeback_control *wbc)
385+{
386+ struct inode *inode;
387+ struct ceph_inode_info *ci;
388+ struct ceph_client *client;
389+ struct ceph_osd_client *osdc;
390+ loff_t page_off = page->index << PAGE_CACHE_SHIFT;
391+ int len = PAGE_CACHE_SIZE;
392+ loff_t i_size;
393+ int err = 0;
394+ struct ceph_snap_context *snapc;
395+ u64 snap_size = 0;
396+ long writeback_stat;
397+
398+ dout("writepage %p idx %lu\n", page, page->index);
399+
400+ if (!page->mapping || !page->mapping->host) {
401+ dout("writepage %p - no mapping\n", page);
402+ return -EFAULT;
403+ }
404+ inode = page->mapping->host;
405+ ci = ceph_inode(inode);
406+ client = ceph_inode_to_client(inode);
407+ osdc = &client->osdc;
408+
409+ /* verify this is a writeable snap context */
410+ snapc = (void *)page->private;
411+ if (snapc == NULL) {
412+ dout("writepage %p page %p not dirty?\n", inode, page);
413+ goto out;
414+ }
415+ if (snapc != get_oldest_context(inode, &snap_size)) {
416+ dout("writepage %p page %p snapc %p not writeable - noop\n",
417+ inode, page, (void *)page->private);
418+ /* we should only noop if called by kswapd */
419+ WARN_ON((current->flags & PF_MEMALLOC) == 0);
420+ goto out;
421+ }
422+
423+ /* is this a partial page at end of file? */
424+ if (snap_size)
425+ i_size = snap_size;
426+ else
427+ i_size = i_size_read(inode);
428+ if (i_size < page_off + len)
429+ len = i_size - page_off;
430+
431+ dout("writepage %p page %p index %lu on %llu~%u\n",
432+ inode, page, page->index, page_off, len);
433+
434+ writeback_stat = atomic_long_inc_return(&client->writeback_count);
435+ if (writeback_stat >
436+ CONGESTION_ON_THRESH(client->mount_args->congestion_kb))
437+ set_bdi_congested(&client->backing_dev_info, BLK_RW_ASYNC);
438+
439+ set_page_writeback(page);
440+ err = ceph_osdc_writepages(osdc, ceph_vino(inode),
441+ &ci->i_layout, snapc,
442+ page_off, len,
443+ ci->i_truncate_seq, ci->i_truncate_size,
444+ &inode->i_mtime,
445+ &page, 1, 0, 0, true);
446+ if (err < 0) {
447+ dout("writepage setting page/mapping error %d %p\n", err, page);
448+ SetPageError(page);
449+ mapping_set_error(&inode->i_data, err);
450+ if (wbc)
451+ wbc->pages_skipped++;
452+ } else {
453+ dout("writepage cleaned page %p\n", page);
454+ err = 0; /* vfs expects us to return 0 */
455+ }
456+ page->private = 0;
457+ ClearPagePrivate(page);
458+ end_page_writeback(page);
459+ ceph_put_wrbuffer_cap_refs(ci, 1, snapc);
460+ ceph_put_snap_context(snapc);
461+out:
462+ return err;
463+}
464+
465+static int ceph_writepage(struct page *page, struct writeback_control *wbc)
466+{
467+ int err;
468+ struct inode *inode = page->mapping->host;
469+ BUG_ON(!inode);
470+ igrab(inode);
471+ err = writepage_nounlock(page, wbc);
472+ unlock_page(page);
473+ iput(inode);
474+ return err;
475+}
476+
477+
478+/*
479+ * lame release_pages helper. release_pages() isn't exported to
480+ * modules.
481+ */
482+static void ceph_release_pages(struct page **pages, int num)
483+{
484+ struct pagevec pvec;
485+ int i;
486+
487+ pagevec_init(&pvec, 0);
488+ for (i = 0; i < num; i++) {
489+ if (pagevec_add(&pvec, pages[i]) == 0)
490+ pagevec_release(&pvec);
491+ }
492+ pagevec_release(&pvec);
493+}
494+
495+
496+/*
497+ * async writeback completion handler.
498+ *
499+ * If we get an error, set the mapping error bit, but not the individual
500+ * page error bits.
501+ */
502+static void writepages_finish(struct ceph_osd_request *req,
503+ struct ceph_msg *msg)
504+{
505+ struct inode *inode = req->r_inode;
506+ struct ceph_osd_reply_head *replyhead;
507+ struct ceph_osd_op *op;
508+ struct ceph_inode_info *ci = ceph_inode(inode);
509+ unsigned wrote;
510+ struct page *page;
511+ int i;
512+ struct ceph_snap_context *snapc = req->r_snapc;
513+ struct address_space *mapping = inode->i_mapping;
514+ struct writeback_control *wbc = req->r_wbc;
515+ __s32 rc = -EIO;
516+ u64 bytes = 0;
517+ struct ceph_client *client = ceph_inode_to_client(inode);
518+ long writeback_stat;
519+ unsigned issued = __ceph_caps_issued(ci, NULL);
520+
521+ /* parse reply */
522+ replyhead = msg->front.iov_base;
523+ WARN_ON(le32_to_cpu(replyhead->num_ops) == 0);
524+ op = (void *)(replyhead + 1);
525+ rc = le32_to_cpu(replyhead->result);
526+ bytes = le64_to_cpu(op->extent.length);
527+
528+ if (rc >= 0) {
529+ /*
530+ * Assume we wrote the pages we originally sent. The
531+ * osd might reply with fewer pages if our writeback
532+ * raced with a truncation and was adjusted at the osd,
533+ * so don't believe the reply.
534+ */
535+ wrote = req->r_num_pages;
536+ } else {
537+ wrote = 0;
538+ mapping_set_error(mapping, rc);
539+ }
540+ dout("writepages_finish %p rc %d bytes %llu wrote %d (pages)\n",
541+ inode, rc, bytes, wrote);
542+
543+ /* clean all pages */
544+ for (i = 0; i < req->r_num_pages; i++) {
545+ page = req->r_pages[i];
546+ BUG_ON(!page);
547+ WARN_ON(!PageUptodate(page));
548+
549+ writeback_stat =
550+ atomic_long_dec_return(&client->writeback_count);
551+ if (writeback_stat <
552+ CONGESTION_OFF_THRESH(client->mount_args->congestion_kb))
553+ clear_bdi_congested(&client->backing_dev_info,
554+ BLK_RW_ASYNC);
555+
556+ if (i >= wrote) {
557+ dout("inode %p skipping page %p\n", inode, page);
558+ wbc->pages_skipped++;
559+ }
560+ page->private = 0;
561+ ClearPagePrivate(page);
562+ ceph_put_snap_context(snapc);
563+ dout("unlocking %d %p\n", i, page);
564+ end_page_writeback(page);
565+
566+ /*
567+ * We lost the cache cap, need to truncate the page before
568+ * it is unlocked, otherwise we'd truncate it later in the
569+ * page truncation thread, possibly losing some data that
570+ * raced its way in
571+ */
572+ if ((issued & CEPH_CAP_FILE_CACHE) == 0)
573+ generic_error_remove_page(inode->i_mapping, page);
574+
575+ unlock_page(page);
576+ }
577+ dout("%p wrote+cleaned %d pages\n", inode, wrote);
578+ ceph_put_wrbuffer_cap_refs(ci, req->r_num_pages, snapc);
579+
580+ ceph_release_pages(req->r_pages, req->r_num_pages);
581+ if (req->r_pages_from_pool)
582+ mempool_free(req->r_pages,
583+ ceph_client(inode->i_sb)->wb_pagevec_pool);
584+ else
585+ kfree(req->r_pages);
586+ ceph_osdc_put_request(req);
587+}
588+
589+/*
590+ * allocate a page vec, either directly, or if necessary, via a the
591+ * mempool. we avoid the mempool if we can because req->r_num_pages
592+ * may be less than the maximum write size.
593+ */
594+static void alloc_page_vec(struct ceph_client *client,
595+ struct ceph_osd_request *req)
596+{
597+ req->r_pages = kmalloc(sizeof(struct page *) * req->r_num_pages,
598+ GFP_NOFS);
599+ if (!req->r_pages) {
600+ req->r_pages = mempool_alloc(client->wb_pagevec_pool, GFP_NOFS);
601+ req->r_pages_from_pool = 1;
602+ WARN_ON(!req->r_pages);
603+ }
604+}
605+
606+/*
607+ * initiate async writeback
608+ */
609+static int ceph_writepages_start(struct address_space *mapping,
610+ struct writeback_control *wbc)
611+{
612+ struct inode *inode = mapping->host;
613+ struct backing_dev_info *bdi = mapping->backing_dev_info;
614+ struct ceph_inode_info *ci = ceph_inode(inode);
615+ struct ceph_client *client;
616+ pgoff_t index, start, end;
617+ int range_whole = 0;
618+ int should_loop = 1;
619+ pgoff_t max_pages = 0, max_pages_ever = 0;
620+ struct ceph_snap_context *snapc = NULL, *last_snapc = NULL;
621+ struct pagevec pvec;
622+ int done = 0;
623+ int rc = 0;
624+ unsigned wsize = 1 << inode->i_blkbits;
625+ struct ceph_osd_request *req = NULL;
626+ int do_sync;
627+ u64 snap_size = 0;
628+
629+ /*
630+ * Include a 'sync' in the OSD request if this is a data
631+ * integrity write (e.g., O_SYNC write or fsync()), or if our
632+ * cap is being revoked.
633+ */
634+ do_sync = wbc->sync_mode == WB_SYNC_ALL;
635+ if (ceph_caps_revoking(ci, CEPH_CAP_FILE_BUFFER))
636+ do_sync = 1;
637+ dout("writepages_start %p dosync=%d (mode=%s)\n",
638+ inode, do_sync,
639+ wbc->sync_mode == WB_SYNC_NONE ? "NONE" :
640+ (wbc->sync_mode == WB_SYNC_ALL ? "ALL" : "HOLD"));
641+
642+ client = ceph_inode_to_client(inode);
643+ if (client->mount_state == CEPH_MOUNT_SHUTDOWN) {
644+ pr_warning("writepage_start %p on forced umount\n", inode);
645+ return -EIO; /* we're in a forced umount, don't write! */
646+ }
647+ if (client->mount_args->wsize && client->mount_args->wsize < wsize)
648+ wsize = client->mount_args->wsize;
649+ if (wsize < PAGE_CACHE_SIZE)
650+ wsize = PAGE_CACHE_SIZE;
651+ max_pages_ever = wsize >> PAGE_CACHE_SHIFT;
652+
653+ pagevec_init(&pvec, 0);
654+
655+ /* ?? */
656+ if (wbc->nonblocking && bdi_write_congested(bdi)) {
657+ dout(" writepages congested\n");
658+ wbc->encountered_congestion = 1;
659+ goto out_final;
660+ }
661+
662+ /* where to start/end? */
663+ if (wbc->range_cyclic) {
664+ start = mapping->writeback_index; /* Start from prev offset */
665+ end = -1;
666+ dout(" cyclic, start at %lu\n", start);
667+ } else {
668+ start = wbc->range_start >> PAGE_CACHE_SHIFT;
669+ end = wbc->range_end >> PAGE_CACHE_SHIFT;
670+ if (wbc->range_start == 0 && wbc->range_end == LLONG_MAX)
671+ range_whole = 1;
672+ should_loop = 0;
673+ dout(" not cyclic, %lu to %lu\n", start, end);
674+ }
675+ index = start;
676+
677+retry:
678+ /* find oldest snap context with dirty data */
679+ ceph_put_snap_context(snapc);
680+ snapc = get_oldest_context(inode, &snap_size);
681+ if (!snapc) {
682+ /* hmm, why does writepages get called when there
683+ is no dirty data? */
684+ dout(" no snap context with dirty data?\n");
685+ goto out;
686+ }
687+ dout(" oldest snapc is %p seq %lld (%d snaps)\n",
688+ snapc, snapc->seq, snapc->num_snaps);
689+ if (last_snapc && snapc != last_snapc) {
690+ /* if we switched to a newer snapc, restart our scan at the
691+ * start of the original file range. */
692+ dout(" snapc differs from last pass, restarting at %lu\n",
693+ index);
694+ index = start;
695+ }
696+ last_snapc = snapc;
697+
698+ while (!done && index <= end) {
699+ unsigned i;
700+ int first;
701+ pgoff_t next;
702+ int pvec_pages, locked_pages;
703+ struct page *page;
704+ int want;
705+ u64 offset, len;
706+ struct ceph_osd_request_head *reqhead;
707+ struct ceph_osd_op *op;
708+ long writeback_stat;
709+
710+ next = 0;
711+ locked_pages = 0;
712+ max_pages = max_pages_ever;
713+
714+get_more_pages:
715+ first = -1;
716+ want = min(end - index,
717+ min((pgoff_t)PAGEVEC_SIZE,
718+ max_pages - (pgoff_t)locked_pages) - 1)
719+ + 1;
720+ pvec_pages = pagevec_lookup_tag(&pvec, mapping, &index,
721+ PAGECACHE_TAG_DIRTY,
722+ want);
723+ dout("pagevec_lookup_tag got %d\n", pvec_pages);
724+ if (!pvec_pages && !locked_pages)
725+ break;
726+ for (i = 0; i < pvec_pages && locked_pages < max_pages; i++) {
727+ page = pvec.pages[i];
728+ dout("? %p idx %lu\n", page, page->index);
729+ if (locked_pages == 0)
730+ lock_page(page); /* first page */
731+ else if (!trylock_page(page))
732+ break;
733+
734+ /* only dirty pages, or our accounting breaks */
735+ if (unlikely(!PageDirty(page)) ||
736+ unlikely(page->mapping != mapping)) {
737+ dout("!dirty or !mapping %p\n", page);
738+ unlock_page(page);
739+ break;
740+ }
741+ if (!wbc->range_cyclic && page->index > end) {
742+ dout("end of range %p\n", page);
743+ done = 1;
744+ unlock_page(page);
745+ break;
746+ }
747+ if (next && (page->index != next)) {
748+ dout("not consecutive %p\n", page);
749+ unlock_page(page);
750+ break;
751+ }
752+ if (wbc->sync_mode != WB_SYNC_NONE) {
753+ dout("waiting on writeback %p\n", page);
754+ wait_on_page_writeback(page);
755+ }
756+ if ((snap_size && page_offset(page) > snap_size) ||
757+ (!snap_size &&
758+ page_offset(page) > i_size_read(inode))) {
759+ dout("%p page eof %llu\n", page, snap_size ?
760+ snap_size : i_size_read(inode));
761+ done = 1;
762+ unlock_page(page);
763+ break;
764+ }
765+ if (PageWriteback(page)) {
766+ dout("%p under writeback\n", page);
767+ unlock_page(page);
768+ break;
769+ }
770+
771+ /* only if matching snap context */
772+ if (snapc != (void *)page->private) {
773+ dout("page snapc %p != oldest %p\n",
774+ (void *)page->private, snapc);
775+ unlock_page(page);
776+ if (!locked_pages)
777+ continue; /* keep looking for snap */
778+ break;
779+ }
780+
781+ if (!clear_page_dirty_for_io(page)) {
782+ dout("%p !clear_page_dirty_for_io\n", page);
783+ unlock_page(page);
784+ break;
785+ }
786+
787+ /* ok */
788+ if (locked_pages == 0) {
789+ /* prepare async write request */
790+ offset = page->index << PAGE_CACHE_SHIFT;
791+ len = wsize;
792+ req = ceph_osdc_new_request(&client->osdc,
793+ &ci->i_layout,
794+ ceph_vino(inode),
795+ offset, &len,
796+ CEPH_OSD_OP_WRITE,
797+ CEPH_OSD_FLAG_WRITE |
798+ CEPH_OSD_FLAG_ONDISK,
799+ snapc, do_sync,
800+ ci->i_truncate_seq,
801+ ci->i_truncate_size,
802+ &inode->i_mtime, true, 1);
803+ max_pages = req->r_num_pages;
804+
805+ alloc_page_vec(client, req);
806+ req->r_callback = writepages_finish;
807+ req->r_inode = inode;
808+ req->r_wbc = wbc;
809+ }
810+
811+ /* note position of first page in pvec */
812+ if (first < 0)
813+ first = i;
814+ dout("%p will write page %p idx %lu\n",
815+ inode, page, page->index);
816+
817+ writeback_stat = atomic_long_inc_return(&client->writeback_count);
818+ if (writeback_stat > CONGESTION_ON_THRESH(client->mount_args->congestion_kb)) {
819+ set_bdi_congested(&client->backing_dev_info, BLK_RW_ASYNC);
820+ }
821+
822+ set_page_writeback(page);
823+ req->r_pages[locked_pages] = page;
824+ locked_pages++;
825+ next = page->index + 1;
826+ }
827+
828+ /* did we get anything? */
829+ if (!locked_pages)
830+ goto release_pvec_pages;
831+ if (i) {
832+ int j;
833+ BUG_ON(!locked_pages || first < 0);
834+
835+ if (pvec_pages && i == pvec_pages &&
836+ locked_pages < max_pages) {
837+ dout("reached end pvec, trying for more\n");
838+ pagevec_reinit(&pvec);
839+ goto get_more_pages;
840+ }
841+
842+ /* shift unused pages over in the pvec... we
843+ * will need to release them below. */
844+ for (j = i; j < pvec_pages; j++) {
845+ dout(" pvec leftover page %p\n",
846+ pvec.pages[j]);
847+ pvec.pages[j-i+first] = pvec.pages[j];
848+ }
849+ pvec.nr -= i-first;
850+ }
851+
852+ /* submit the write */
853+ offset = req->r_pages[0]->index << PAGE_CACHE_SHIFT;
854+ len = min((snap_size ? snap_size : i_size_read(inode)) - offset,
855+ (u64)locked_pages << PAGE_CACHE_SHIFT);
856+ dout("writepages got %d pages at %llu~%llu\n",
857+ locked_pages, offset, len);
858+
859+ /* revise final length, page count */
860+ req->r_num_pages = locked_pages;
861+ reqhead = req->r_request->front.iov_base;
862+ op = (void *)(reqhead + 1);
863+ op->extent.length = cpu_to_le64(len);
864+ op->payload_len = cpu_to_le32(len);
865+ req->r_request->hdr.data_len = cpu_to_le32(len);
866+
867+ ceph_osdc_start_request(&client->osdc, req, true);
868+ req = NULL;
869+
870+ /* continue? */
871+ index = next;
872+ wbc->nr_to_write -= locked_pages;
873+ if (wbc->nr_to_write <= 0)
874+ done = 1;
875+
876+release_pvec_pages:
877+ dout("pagevec_release on %d pages (%p)\n", (int)pvec.nr,
878+ pvec.nr ? pvec.pages[0] : NULL);
879+ pagevec_release(&pvec);
880+
881+ if (locked_pages && !done)
882+ goto retry;
883+ }
884+
885+ if (should_loop && !done) {
886+ /* more to do; loop back to beginning of file */
887+ dout("writepages looping back to beginning of file\n");
888+ should_loop = 0;
889+ index = 0;
890+ goto retry;
891+ }
892+
893+ if (wbc->range_cyclic || (range_whole && wbc->nr_to_write > 0))
894+ mapping->writeback_index = index;
895+
896+out:
897+ if (req)
898+ ceph_osdc_put_request(req);
899+ if (rc > 0)
900+ rc = 0; /* vfs expects us to return 0 */
901+ ceph_put_snap_context(snapc);
902+ dout("writepages done, rc = %d\n", rc);
903+out_final:
904+ return rc;
905+}
906+
907+
908+
909+/*
910+ * See if a given @snapc is either writeable, or already written.
911+ */
912+static int context_is_writeable_or_written(struct inode *inode,
913+ struct ceph_snap_context *snapc)
914+{
915+ struct ceph_snap_context *oldest = get_oldest_context(inode, NULL);
916+ return !oldest || snapc->seq <= oldest->seq;
917+}
918+
919+/*
920+ * We are only allowed to write into/dirty the page if the page is
921+ * clean, or already dirty within the same snap context.
922+ */
923+static int ceph_update_writeable_page(struct file *file,
924+ loff_t pos, unsigned len,
925+ struct page *page)
926+{
927+ struct inode *inode = file->f_dentry->d_inode;
928+ struct ceph_inode_info *ci = ceph_inode(inode);
929+ struct ceph_mds_client *mdsc = &ceph_inode_to_client(inode)->mdsc;
930+ loff_t page_off = pos & PAGE_CACHE_MASK;
931+ int pos_in_page = pos & ~PAGE_CACHE_MASK;
932+ int end_in_page = pos_in_page + len;
933+ loff_t i_size;
934+ struct ceph_snap_context *snapc;
935+ int r;
936+
937+retry_locked:
938+ /* writepages currently holds page lock, but if we change that later, */
939+ wait_on_page_writeback(page);
940+
941+ /* check snap context */
942+ BUG_ON(!ci->i_snap_realm);
943+ down_read(&mdsc->snap_rwsem);
944+ BUG_ON(!ci->i_snap_realm->cached_context);
945+ if (page->private &&
946+ (void *)page->private != ci->i_snap_realm->cached_context) {
947+ /*
948+ * this page is already dirty in another (older) snap
949+ * context! is it writeable now?
950+ */
951+ snapc = get_oldest_context(inode, NULL);
952+ up_read(&mdsc->snap_rwsem);
953+
954+ if (snapc != (void *)page->private) {
955+ dout(" page %p snapc %p not current or oldest\n",
956+ page, (void *)page->private);
957+ /*
958+ * queue for writeback, and wait for snapc to
959+ * be writeable or written
960+ */
961+ snapc = ceph_get_snap_context((void *)page->private);
962+ unlock_page(page);
963+ ceph_queue_writeback(inode);
964+ wait_event_interruptible(ci->i_cap_wq,
965+ context_is_writeable_or_written(inode, snapc));
966+ ceph_put_snap_context(snapc);
967+ return -EAGAIN;
968+ }
969+
970+ /* yay, writeable, do it now (without dropping page lock) */
971+ dout(" page %p snapc %p not current, but oldest\n",
972+ page, snapc);
973+ if (!clear_page_dirty_for_io(page))
974+ goto retry_locked;
975+ r = writepage_nounlock(page, NULL);
976+ if (r < 0)
977+ goto fail_nosnap;
978+ goto retry_locked;
979+ }
980+
981+ if (PageUptodate(page)) {
982+ dout(" page %p already uptodate\n", page);
983+ return 0;
984+ }
985+
986+ /* full page? */
987+ if (pos_in_page == 0 && len == PAGE_CACHE_SIZE)
988+ return 0;
989+
990+ /* past end of file? */
991+ i_size = inode->i_size; /* caller holds i_mutex */
992+
993+ if (i_size + len > inode->i_sb->s_maxbytes) {
994+ /* file is too big */
995+ r = -EINVAL;
996+ goto fail;
997+ }
998+
999+ if (page_off >= i_size ||
1000+ (pos_in_page == 0 && (pos+len) >= i_size &&
1001+ end_in_page - pos_in_page != PAGE_CACHE_SIZE)) {
1002+ dout(" zeroing %p 0 - %d and %d - %d\n",
1003+ page, pos_in_page, end_in_page, (int)PAGE_CACHE_SIZE);
1004+ zero_user_segments(page,
1005+ 0, pos_in_page,
1006+ end_in_page, PAGE_CACHE_SIZE);
1007+ return 0;
1008+ }
1009+
1010+ /* we need to read it. */
1011+ up_read(&mdsc->snap_rwsem);
1012+ r = readpage_nounlock(file, page);
1013+ if (r < 0)
1014+ goto fail_nosnap;
1015+ goto retry_locked;
1016+
1017+fail:
1018+ up_read(&mdsc->snap_rwsem);
1019+fail_nosnap:
1020+ unlock_page(page);
1021+ return r;
1022+}
1023+
1024+/*
1025+ * We are only allowed to write into/dirty the page if the page is
1026+ * clean, or already dirty within the same snap context.
1027+ */
1028+static int ceph_write_begin(struct file *file, struct address_space *mapping,
1029+ loff_t pos, unsigned len, unsigned flags,
1030+ struct page **pagep, void **fsdata)
1031+{
1032+ struct inode *inode = file->f_dentry->d_inode;
1033+ struct page *page;
1034+ pgoff_t index = pos >> PAGE_CACHE_SHIFT;
1035+ int r;
1036+
1037+ do {
1038+ /* get a page*/
1039+ page = grab_cache_page_write_begin(mapping, index, 0);
1040+ if (!page)
1041+ return -ENOMEM;
1042+ *pagep = page;
1043+
1044+ dout("write_begin file %p inode %p page %p %d~%d\n", file,
1045+ inode, page, (int)pos, (int)len);
1046+
1047+ r = ceph_update_writeable_page(file, pos, len, page);
1048+ } while (r == -EAGAIN);
1049+
1050+ return r;
1051+}
1052+
1053+/*
1054+ * we don't do anything in here that simple_write_end doesn't do
1055+ * except adjust dirty page accounting and drop read lock on
1056+ * mdsc->snap_rwsem.
1057+ */
1058+static int ceph_write_end(struct file *file, struct address_space *mapping,
1059+ loff_t pos, unsigned len, unsigned copied,
1060+ struct page *page, void *fsdata)
1061+{
1062+ struct inode *inode = file->f_dentry->d_inode;
1063+ struct ceph_client *client = ceph_inode_to_client(inode);
1064+ struct ceph_mds_client *mdsc = &client->mdsc;
1065+ unsigned from = pos & (PAGE_CACHE_SIZE - 1);
1066+ int check_cap = 0;
1067+
1068+ dout("write_end file %p inode %p page %p %d~%d (%d)\n", file,
1069+ inode, page, (int)pos, (int)copied, (int)len);
1070+
1071+ /* zero the stale part of the page if we did a short copy */
1072+ if (copied < len)
1073+ zero_user_segment(page, from+copied, len);
1074+
1075+ /* did file size increase? */
1076+ /* (no need for i_size_read(); we caller holds i_mutex */
1077+ if (pos+copied > inode->i_size)
1078+ check_cap = ceph_inode_set_size(inode, pos+copied);
1079+
1080+ if (!PageUptodate(page))
1081+ SetPageUptodate(page);
1082+
1083+ set_page_dirty(page);
1084+
1085+ unlock_page(page);
1086+ up_read(&mdsc->snap_rwsem);
1087+ page_cache_release(page);
1088+
1089+ if (check_cap)
1090+ ceph_check_caps(ceph_inode(inode), CHECK_CAPS_AUTHONLY, NULL);
1091+
1092+ return copied;
1093+}
1094+
1095+/*
1096+ * we set .direct_IO to indicate direct io is supported, but since we
1097+ * intercept O_DIRECT reads and writes early, this function should
1098+ * never get called.
1099+ */
1100+static ssize_t ceph_direct_io(int rw, struct kiocb *iocb,
1101+ const struct iovec *iov,
1102+ loff_t pos, unsigned long nr_segs)
1103+{
1104+ WARN_ON(1);
1105+ return -EINVAL;
1106+}
1107+
1108+const struct address_space_operations ceph_aops = {
1109+ .readpage = ceph_readpage,
1110+ .readpages = ceph_readpages,
1111+ .writepage = ceph_writepage,
1112+ .writepages = ceph_writepages_start,
1113+ .write_begin = ceph_write_begin,
1114+ .write_end = ceph_write_end,
1115+ .set_page_dirty = ceph_set_page_dirty,
1116+ .invalidatepage = ceph_invalidatepage,
1117+ .releasepage = ceph_releasepage,
1118+ .direct_IO = ceph_direct_io,
1119+};
1120+
1121+
1122+/*
1123+ * vm ops
1124+ */
1125+
1126+/*
1127+ * Reuse write_begin here for simplicity.
1128+ */
1129+static int ceph_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf)
1130+{
1131+ struct inode *inode = vma->vm_file->f_dentry->d_inode;
1132+ struct page *page = vmf->page;
1133+ struct ceph_mds_client *mdsc = &ceph_inode_to_client(inode)->mdsc;
1134+ loff_t off = page->index << PAGE_CACHE_SHIFT;
1135+ loff_t size, len;
1136+ int ret;
1137+
1138+ size = i_size_read(inode);
1139+ if (off + PAGE_CACHE_SIZE <= size)
1140+ len = PAGE_CACHE_SIZE;
1141+ else
1142+ len = size & ~PAGE_CACHE_MASK;
1143+
1144+ dout("page_mkwrite %p %llu~%llu page %p idx %lu\n", inode,
1145+ off, len, page, page->index);
1146+
1147+ lock_page(page);
1148+
1149+ ret = VM_FAULT_NOPAGE;
1150+ if ((off > size) ||
1151+ (page->mapping != inode->i_mapping))
1152+ goto out;
1153+
1154+ ret = ceph_update_writeable_page(vma->vm_file, off, len, page);
1155+ if (ret == 0) {
1156+ /* success. we'll keep the page locked. */
1157+ set_page_dirty(page);
1158+ up_read(&mdsc->snap_rwsem);
1159+ ret = VM_FAULT_LOCKED;
1160+ } else {
1161+ if (ret == -ENOMEM)
1162+ ret = VM_FAULT_OOM;
1163+ else
1164+ ret = VM_FAULT_SIGBUS;
1165+ }
1166+out:
1167+ dout("page_mkwrite %p %llu~%llu = %d\n", inode, off, len, ret);
1168+ if (ret != VM_FAULT_LOCKED)
1169+ unlock_page(page);
1170+ return ret;
1171+}
1172+
1173+static struct vm_operations_struct ceph_vmops = {
1174+ .fault = filemap_fault,
1175+ .page_mkwrite = ceph_page_mkwrite,
1176+};
1177+
1178+int ceph_mmap(struct file *file, struct vm_area_struct *vma)
1179+{
1180+ struct address_space *mapping = file->f_mapping;
1181+
1182+ if (!mapping->a_ops->readpage)
1183+ return -ENOEXEC;
1184+ file_accessed(file);
1185+ vma->vm_ops = &ceph_vmops;
1186+ vma->vm_flags |= VM_CAN_NONLINEAR;
1187+ return 0;
1188+}
fs/ceph/armor.c+99−0
@@ -0,0 +1,99 @@
1+
2+#include <linux/errno.h>
3+
4+/*
5+ * base64 encode/decode.
6+ */
7+
8+const char *pem_key = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
9+
10+static int encode_bits(int c)
11+{
12+ return pem_key[c];
13+}
14+
15+static int decode_bits(char c)
16+{
17+ if (c >= 'A' && c <= 'Z')
18+ return c - 'A';
19+ if (c >= 'a' && c <= 'z')
20+ return c - 'a' + 26;
21+ if (c >= '0' && c <= '9')
22+ return c - '0' + 52;
23+ if (c == '+')
24+ return 62;
25+ if (c == '/')
26+ return 63;
27+ if (c == '=')
28+ return 0; /* just non-negative, please */
29+ return -EINVAL;
30+}
31+
32+int ceph_armor(char *dst, const char *src, const char *end)
33+{
34+ int olen = 0;
35+ int line = 0;
36+
37+ while (src < end) {
38+ unsigned char a, b, c;
39+
40+ a = *src++;
41+ *dst++ = encode_bits(a >> 2);
42+ if (src < end) {
43+ b = *src++;
44+ *dst++ = encode_bits(((a & 3) << 4) | (b >> 4));
45+ if (src < end) {
46+ c = *src++;
47+ *dst++ = encode_bits(((b & 15) << 2) |
48+ (c >> 6));
49+ *dst++ = encode_bits(c & 63);
50+ } else {
51+ *dst++ = encode_bits((b & 15) << 2);
52+ *dst++ = '=';
53+ }
54+ } else {
55+ *dst++ = encode_bits(((a & 3) << 4));
56+ *dst++ = '=';
57+ *dst++ = '=';
58+ }
59+ olen += 4;
60+ line += 4;
61+ if (line == 64) {
62+ line = 0;
63+ *(dst++) = '\n';
64+ olen++;
65+ }
66+ }
67+ return olen;
68+}
69+
70+int ceph_unarmor(char *dst, const char *src, const char *end)
71+{
72+ int olen = 0;
73+
74+ while (src < end) {
75+ int a, b, c, d;
76+
77+ if (src < end && src[0] == '\n')
78+ src++;
79+ if (src + 4 > end)
80+ return -EINVAL;
81+ a = decode_bits(src[0]);
82+ b = decode_bits(src[1]);
83+ c = decode_bits(src[2]);
84+ d = decode_bits(src[3]);
85+ if (a < 0 || b < 0 || c < 0 || d < 0)
86+ return -EINVAL;
87+
88+ *dst++ = (a << 2) | (b >> 4);
89+ if (src[2] == '=')
90+ return olen + 1;
91+ *dst++ = ((b & 15) << 4) | (c >> 2);
92+ if (src[3] == '=')
93+ return olen + 2;
94+ *dst++ = ((c & 3) << 6) | d;
95+ olen += 3;
96+ src += 4;
97+ }
98+ return olen;
99+}
fs/ceph/auth.c+257−0
@@ -0,0 +1,257 @@
1+#include "ceph_debug.h"
2+
3+#include <linux/module.h>
4+#include <linux/err.h>
5+
6+#include "types.h"
7+#include "auth_none.h"
8+#include "auth_x.h"
9+#include "decode.h"
10+#include "super.h"
11+
12+#include "messenger.h"
13+
14+/*
15+ * get protocol handler
16+ */
17+static u32 supported_protocols[] = {
18+ CEPH_AUTH_NONE,
19+ CEPH_AUTH_CEPHX
20+};
21+
22+int ceph_auth_init_protocol(struct ceph_auth_client *ac, int protocol)
23+{
24+ switch (protocol) {
25+ case CEPH_AUTH_NONE:
26+ return ceph_auth_none_init(ac);
27+ case CEPH_AUTH_CEPHX:
28+ return ceph_x_init(ac);
29+ default:
30+ return -ENOENT;
31+ }
32+}
33+
34+/*
35+ * setup, teardown.
36+ */
37+struct ceph_auth_client *ceph_auth_init(const char *name, const char *secret)
38+{
39+ struct ceph_auth_client *ac;
40+ int ret;
41+
42+ dout("auth_init name '%s' secret '%s'\n", name, secret);
43+
44+ ret = -ENOMEM;
45+ ac = kzalloc(sizeof(*ac), GFP_NOFS);
46+ if (!ac)
47+ goto out;
48+
49+ ac->negotiating = true;
50+ if (name)
51+ ac->name = name;
52+ else
53+ ac->name = CEPH_AUTH_NAME_DEFAULT;
54+ dout("auth_init name %s secret %s\n", ac->name, secret);
55+ ac->secret = secret;
56+ return ac;
57+
58+out:
59+ return ERR_PTR(ret);
60+}
61+
62+void ceph_auth_destroy(struct ceph_auth_client *ac)
63+{
64+ dout("auth_destroy %p\n", ac);
65+ if (ac->ops)
66+ ac->ops->destroy(ac);
67+ kfree(ac);
68+}
69+
70+/*
71+ * Reset occurs when reconnecting to the monitor.
72+ */
73+void ceph_auth_reset(struct ceph_auth_client *ac)
74+{
75+ dout("auth_reset %p\n", ac);
76+ if (ac->ops && !ac->negotiating)
77+ ac->ops->reset(ac);
78+ ac->negotiating = true;
79+}
80+
81+int ceph_entity_name_encode(const char *name, void **p, void *end)
82+{
83+ int len = strlen(name);
84+
85+ if (*p + 2*sizeof(u32) + len > end)
86+ return -ERANGE;
87+ ceph_encode_32(p, CEPH_ENTITY_TYPE_CLIENT);
88+ ceph_encode_32(p, len);
89+ ceph_encode_copy(p, name, len);
90+ return 0;
91+}
92+
93+/*
94+ * Initiate protocol negotiation with monitor. Include entity name
95+ * and list supported protocols.
96+ */
97+int ceph_auth_build_hello(struct ceph_auth_client *ac, void *buf, size_t len)
98+{
99+ struct ceph_mon_request_header *monhdr = buf;
100+ void *p = monhdr + 1, *end = buf + len, *lenp;
101+ int i, num;
102+ int ret;
103+
104+ dout("auth_build_hello\n");
105+ monhdr->have_version = 0;
106+ monhdr->session_mon = cpu_to_le16(-1);
107+ monhdr->session_mon_tid = 0;
108+
109+ ceph_encode_32(&p, 0); /* no protocol, yet */
110+
111+ lenp = p;
112+ p += sizeof(u32);
113+
114+ ceph_decode_need(&p, end, 1 + sizeof(u32), bad);
115+ ceph_encode_8(&p, 1);
116+ num = ARRAY_SIZE(supported_protocols);
117+ ceph_encode_32(&p, num);
118+ ceph_decode_need(&p, end, num * sizeof(u32), bad);
119+ for (i = 0; i < num; i++)
120+ ceph_encode_32(&p, supported_protocols[i]);
121+
122+ ret = ceph_entity_name_encode(ac->name, &p, end);
123+ if (ret < 0)
124+ return ret;
125+ ceph_decode_need(&p, end, sizeof(u64), bad);
126+ ceph_encode_64(&p, ac->global_id);
127+
128+ ceph_encode_32(&lenp, p - lenp - sizeof(u32));
129+ return p - buf;
130+
131+bad:
132+ return -ERANGE;
133+}
134+
135+int ceph_build_auth_request(struct ceph_auth_client *ac,
136+ void *msg_buf, size_t msg_len)
137+{
138+ struct ceph_mon_request_header *monhdr = msg_buf;
139+ void *p = monhdr + 1;
140+ void *end = msg_buf + msg_len;
141+ int ret;
142+
143+ monhdr->have_version = 0;
144+ monhdr->session_mon = cpu_to_le16(-1);
145+ monhdr->session_mon_tid = 0;
146+
147+ ceph_encode_32(&p, ac->protocol);
148+
149+ ret = ac->ops->build_request(ac, p + sizeof(u32), end);
150+ if (ret < 0) {
151+ pr_err("error %d building request\n", ret);
152+ return ret;
153+ }
154+ dout(" built request %d bytes\n", ret);
155+ ceph_encode_32(&p, ret);
156+ return p + ret - msg_buf;
157+}
158+
159+/*
160+ * Handle auth message from monitor.
161+ */
162+int ceph_handle_auth_reply(struct ceph_auth_client *ac,
163+ void *buf, size_t len,
164+ void *reply_buf, size_t reply_len)
165+{
166+ void *p = buf;
167+ void *end = buf + len;
168+ int protocol;
169+ s32 result;
170+ u64 global_id;
171+ void *payload, *payload_end;
172+ int payload_len;
173+ char *result_msg;
174+ int result_msg_len;
175+ int ret = -EINVAL;
176+
177+ dout("handle_auth_reply %p %p\n", p, end);
178+ ceph_decode_need(&p, end, sizeof(u32) * 3 + sizeof(u64), bad);
179+ protocol = ceph_decode_32(&p);
180+ result = ceph_decode_32(&p);
181+ global_id = ceph_decode_64(&p);
182+ payload_len = ceph_decode_32(&p);
183+ payload = p;
184+ p += payload_len;
185+ ceph_decode_need(&p, end, sizeof(u32), bad);
186+ result_msg_len = ceph_decode_32(&p);
187+ result_msg = p;
188+ p += result_msg_len;
189+ if (p != end)
190+ goto bad;
191+
192+ dout(" result %d '%.*s' gid %llu len %d\n", result, result_msg_len,
193+ result_msg, global_id, payload_len);
194+
195+ payload_end = payload + payload_len;
196+
197+ if (global_id && ac->global_id != global_id) {
198+ dout(" set global_id %lld -> %lld\n", ac->global_id, global_id);
199+ ac->global_id = global_id;
200+ }
201+
202+ if (ac->negotiating) {
203+ /* server does not support our protocols? */
204+ if (!protocol && result < 0) {
205+ ret = result;
206+ goto out;
207+ }
208+ /* set up (new) protocol handler? */
209+ if (ac->protocol && ac->protocol != protocol) {
210+ ac->ops->destroy(ac);
211+ ac->protocol = 0;
212+ ac->ops = NULL;
213+ }
214+ if (ac->protocol != protocol) {
215+ ret = ceph_auth_init_protocol(ac, protocol);
216+ if (ret) {
217+ pr_err("error %d on auth protocol %d init\n",
218+ ret, protocol);
219+ goto out;
220+ }
221+ }
222+
223+ ac->negotiating = false;
224+ }
225+
226+ ret = ac->ops->handle_reply(ac, result, payload, payload_end);
227+ if (ret == -EAGAIN) {
228+ return ceph_build_auth_request(ac, reply_buf, reply_len);
229+ } else if (ret) {
230+ pr_err("authentication error %d\n", ret);
231+ return ret;
232+ }
233+ return 0;
234+
235+bad:
236+ pr_err("failed to decode auth msg\n");
237+out:
238+ return ret;
239+}
240+
241+int ceph_build_auth(struct ceph_auth_client *ac,
242+ void *msg_buf, size_t msg_len)
243+{
244+ if (!ac->protocol)
245+ return ceph_auth_build_hello(ac, msg_buf, msg_len);
246+ BUG_ON(!ac->ops);
247+ if (!ac->ops->is_authenticated(ac))
248+ return ceph_build_auth_request(ac, msg_buf, msg_len);
249+ return 0;
250+}
251+
252+int ceph_auth_is_authenticated(struct ceph_auth_client *ac)
253+{
254+ if (!ac->ops)
255+ return 0;
256+ return ac->ops->is_authenticated(ac);
257+}
fs/ceph/auth.h+84−0
@@ -0,0 +1,84 @@
1+#ifndef _FS_CEPH_AUTH_H
2+#define _FS_CEPH_AUTH_H
3+
4+#include "types.h"
5+#include "buffer.h"
6+
7+/*
8+ * Abstract interface for communicating with the authenticate module.
9+ * There is some handshake that takes place between us and the monitor
10+ * to acquire the necessary keys. These are used to generate an
11+ * 'authorizer' that we use when connecting to a service (mds, osd).
12+ */
13+
14+struct ceph_auth_client;
15+struct ceph_authorizer;
16+
17+struct ceph_auth_client_ops {
18+ /*
19+ * true if we are authenticated and can connect to
20+ * services.
21+ */
22+ int (*is_authenticated)(struct ceph_auth_client *ac);
23+
24+ /*
25+ * build requests and process replies during monitor
26+ * handshake. if handle_reply returns -EAGAIN, we build
27+ * another request.
28+ */
29+ int (*build_request)(struct ceph_auth_client *ac, void *buf, void *end);
30+ int (*handle_reply)(struct ceph_auth_client *ac, int result,
31+ void *buf, void *end);
32+
33+ /*
34+ * Create authorizer for connecting to a service, and verify
35+ * the response to authenticate the service.
36+ */
37+ int (*create_authorizer)(struct ceph_auth_client *ac, int peer_type,
38+ struct ceph_authorizer **a,
39+ void **buf, size_t *len,
40+ void **reply_buf, size_t *reply_len);
41+ int (*verify_authorizer_reply)(struct ceph_auth_client *ac,
42+ struct ceph_authorizer *a, size_t len);
43+ void (*destroy_authorizer)(struct ceph_auth_client *ac,
44+ struct ceph_authorizer *a);
45+ void (*invalidate_authorizer)(struct ceph_auth_client *ac,
46+ int peer_type);
47+
48+ /* reset when we (re)connect to a monitor */
49+ void (*reset)(struct ceph_auth_client *ac);
50+
51+ void (*destroy)(struct ceph_auth_client *ac);
52+};
53+
54+struct ceph_auth_client {
55+ u32 protocol; /* CEPH_AUTH_* */
56+ void *private; /* for use by protocol implementation */
57+ const struct ceph_auth_client_ops *ops; /* null iff protocol==0 */
58+
59+ bool negotiating; /* true if negotiating protocol */
60+ const char *name; /* entity name */
61+ u64 global_id; /* our unique id in system */
62+ const char *secret; /* our secret key */
63+ unsigned want_keys; /* which services we want */
64+};
65+
66+extern struct ceph_auth_client *ceph_auth_init(const char *name,
67+ const char *secret);
68+extern void ceph_auth_destroy(struct ceph_auth_client *ac);
69+
70+extern void ceph_auth_reset(struct ceph_auth_client *ac);
71+
72+extern int ceph_auth_build_hello(struct ceph_auth_client *ac,
73+ void *buf, size_t len);
74+extern int ceph_handle_auth_reply(struct ceph_auth_client *ac,
75+ void *buf, size_t len,
76+ void *reply_buf, size_t reply_len);
77+extern int ceph_entity_name_encode(const char *name, void **p, void *end);
78+
79+extern int ceph_build_auth(struct ceph_auth_client *ac,
80+ void *msg_buf, size_t msg_len);
81+
82+extern int ceph_auth_is_authenticated(struct ceph_auth_client *ac);
83+
84+#endif
fs/ceph/auth_none.c+121−0
@@ -0,0 +1,121 @@
1+
2+#include "ceph_debug.h"
3+
4+#include <linux/err.h>
5+#include <linux/module.h>
6+#include <linux/random.h>
7+
8+#include "auth_none.h"
9+#include "auth.h"
10+#include "decode.h"
11+
12+static void reset(struct ceph_auth_client *ac)
13+{
14+ struct ceph_auth_none_info *xi = ac->private;
15+
16+ xi->starting = true;
17+ xi->built_authorizer = false;
18+}
19+
20+static void destroy(struct ceph_auth_client *ac)
21+{
22+ kfree(ac->private);
23+ ac->private = NULL;
24+}
25+
26+static int is_authenticated(struct ceph_auth_client *ac)
27+{
28+ struct ceph_auth_none_info *xi = ac->private;
29+
30+ return !xi->starting;
31+}
32+
33+/*
34+ * the generic auth code decode the global_id, and we carry no actual
35+ * authenticate state, so nothing happens here.
36+ */
37+static int handle_reply(struct ceph_auth_client *ac, int result,
38+ void *buf, void *end)
39+{
40+ struct ceph_auth_none_info *xi = ac->private;
41+
42+ xi->starting = false;
43+ return result;
44+}
45+
46+/*
47+ * build an 'authorizer' with our entity_name and global_id. we can
48+ * reuse a single static copy since it is identical for all services
49+ * we connect to.
50+ */
51+static int ceph_auth_none_create_authorizer(
52+ struct ceph_auth_client *ac, int peer_type,
53+ struct ceph_authorizer **a,
54+ void **buf, size_t *len,
55+ void **reply_buf, size_t *reply_len)
56+{
57+ struct ceph_auth_none_info *ai = ac->private;
58+ struct ceph_none_authorizer *au = &ai->au;
59+ void *p, *end;
60+ int ret;
61+
62+ if (!ai->built_authorizer) {
63+ p = au->buf;
64+ end = p + sizeof(au->buf);
65+ ceph_encode_8(&p, 1);
66+ ret = ceph_entity_name_encode(ac->name, &p, end - 8);
67+ if (ret < 0)
68+ goto bad;
69+ ceph_decode_need(&p, end, sizeof(u64), bad2);
70+ ceph_encode_64(&p, ac->global_id);
71+ au->buf_len = p - (void *)au->buf;
72+ ai->built_authorizer = true;
73+ dout("built authorizer len %d\n", au->buf_len);
74+ }
75+
76+ *a = (struct ceph_authorizer *)au;
77+ *buf = au->buf;
78+ *len = au->buf_len;
79+ *reply_buf = au->reply_buf;
80+ *reply_len = sizeof(au->reply_buf);
81+ return 0;
82+
83+bad2:
84+ ret = -ERANGE;
85+bad:
86+ return ret;
87+}
88+
89+static void ceph_auth_none_destroy_authorizer(struct ceph_auth_client *ac,
90+ struct ceph_authorizer *a)
91+{
92+ /* nothing to do */
93+}
94+
95+static const struct ceph_auth_client_ops ceph_auth_none_ops = {
96+ .reset = reset,
97+ .destroy = destroy,
98+ .is_authenticated = is_authenticated,
99+ .handle_reply = handle_reply,
100+ .create_authorizer = ceph_auth_none_create_authorizer,
101+ .destroy_authorizer = ceph_auth_none_destroy_authorizer,
102+};
103+
104+int ceph_auth_none_init(struct ceph_auth_client *ac)
105+{
106+ struct ceph_auth_none_info *xi;
107+
108+ dout("ceph_auth_none_init %p\n", ac);
109+ xi = kzalloc(sizeof(*xi), GFP_NOFS);
110+ if (!xi)
111+ return -ENOMEM;
112+
113+ xi->starting = true;
114+ xi->built_authorizer = false;
115+
116+ ac->protocol = CEPH_AUTH_NONE;
117+ ac->private = xi;
118+ ac->ops = &ceph_auth_none_ops;
119+ return 0;
120+}
121+
fs/ceph/auth_none.h+28−0
@@ -0,0 +1,28 @@
1+#ifndef _FS_CEPH_AUTH_NONE_H
2+#define _FS_CEPH_AUTH_NONE_H
3+
4+#include "auth.h"
5+
6+/*
7+ * null security mode.
8+ *
9+ * we use a single static authorizer that simply encodes our entity name
10+ * and global id.
11+ */
12+
13+struct ceph_none_authorizer {
14+ char buf[128];
15+ int buf_len;
16+ char reply_buf[0];
17+};
18+
19+struct ceph_auth_none_info {
20+ bool starting;
21+ bool built_authorizer;
22+ struct ceph_none_authorizer au; /* we only need one; it's static */
23+};
24+
25+extern int ceph_auth_none_init(struct ceph_auth_client *ac);
26+
27+#endif
28+
fs/ceph/auth_x.c+656−0
@@ -0,0 +1,656 @@
1+
2+#include "ceph_debug.h"
3+
4+#include <linux/err.h>
5+#include <linux/module.h>
6+#include <linux/random.h>
7+
8+#include "auth_x.h"
9+#include "auth_x_protocol.h"
10+#include "crypto.h"
11+#include "auth.h"
12+#include "decode.h"
13+
14+struct kmem_cache *ceph_x_ticketbuf_cachep;
15+
16+#define TEMP_TICKET_BUF_LEN 256
17+
18+static void ceph_x_validate_tickets(struct ceph_auth_client *ac, int *pneed);
19+
20+static int ceph_x_is_authenticated(struct ceph_auth_client *ac)
21+{
22+ struct ceph_x_info *xi = ac->private;
23+ int need;
24+
25+ ceph_x_validate_tickets(ac, &need);
26+ dout("ceph_x_is_authenticated want=%d need=%d have=%d\n",
27+ ac->want_keys, need, xi->have_keys);
28+ return (ac->want_keys & xi->have_keys) == ac->want_keys;
29+}
30+
31+static int ceph_x_encrypt(struct ceph_crypto_key *secret,
32+ void *ibuf, int ilen, void *obuf, size_t olen)
33+{
34+ struct ceph_x_encrypt_header head = {
35+ .struct_v = 1,
36+ .magic = cpu_to_le64(CEPHX_ENC_MAGIC)
37+ };
38+ size_t len = olen - sizeof(u32);
39+ int ret;
40+
41+ ret = ceph_encrypt2(secret, obuf + sizeof(u32), &len,
42+ &head, sizeof(head), ibuf, ilen);
43+ if (ret)
44+ return ret;
45+ ceph_encode_32(&obuf, len);
46+ return len + sizeof(u32);
47+}
48+
49+static int ceph_x_decrypt(struct ceph_crypto_key *secret,
50+ void **p, void *end, void *obuf, size_t olen)
51+{
52+ struct ceph_x_encrypt_header head;
53+ size_t head_len = sizeof(head);
54+ int len, ret;
55+
56+ len = ceph_decode_32(p);
57+ if (*p + len > end)
58+ return -EINVAL;
59+
60+ dout("ceph_x_decrypt len %d\n", len);
61+ ret = ceph_decrypt2(secret, &head, &head_len, obuf, &olen,
62+ *p, len);
63+ if (ret)
64+ return ret;
65+ if (head.struct_v != 1 || le64_to_cpu(head.magic) != CEPHX_ENC_MAGIC)
66+ return -EPERM;
67+ *p += len;
68+ return olen;
69+}
70+
71+/*
72+ * get existing (or insert new) ticket handler
73+ */
74+struct ceph_x_ticket_handler *get_ticket_handler(struct ceph_auth_client *ac,
75+ int service)
76+{
77+ struct ceph_x_ticket_handler *th;
78+ struct ceph_x_info *xi = ac->private;
79+ struct rb_node *parent = NULL, **p = &xi->ticket_handlers.rb_node;
80+
81+ while (*p) {
82+ parent = *p;
83+ th = rb_entry(parent, struct ceph_x_ticket_handler, node);
84+ if (service < th->service)
85+ p = &(*p)->rb_left;
86+ else if (service > th->service)
87+ p = &(*p)->rb_right;
88+ else
89+ return th;
90+ }
91+
92+ /* add it */
93+ th = kzalloc(sizeof(*th), GFP_NOFS);
94+ if (!th)
95+ return ERR_PTR(-ENOMEM);
96+ th->service = service;
97+ rb_link_node(&th->node, parent, p);
98+ rb_insert_color(&th->node, &xi->ticket_handlers);
99+ return th;
100+}
101+
102+static void remove_ticket_handler(struct ceph_auth_client *ac,
103+ struct ceph_x_ticket_handler *th)
104+{
105+ struct ceph_x_info *xi = ac->private;
106+
107+ dout("remove_ticket_handler %p %d\n", th, th->service);
108+ rb_erase(&th->node, &xi->ticket_handlers);
109+ ceph_crypto_key_destroy(&th->session_key);
110+ if (th->ticket_blob)
111+ ceph_buffer_put(th->ticket_blob);
112+ kfree(th);
113+}
114+
115+static int ceph_x_proc_ticket_reply(struct ceph_auth_client *ac,
116+ struct ceph_crypto_key *secret,
117+ void *buf, void *end)
118+{
119+ struct ceph_x_info *xi = ac->private;
120+ int num;
121+ void *p = buf;
122+ int ret;
123+ char *dbuf;
124+ char *ticket_buf;
125+ u8 struct_v;
126+
127+ dbuf = kmem_cache_alloc(ceph_x_ticketbuf_cachep, GFP_NOFS | GFP_ATOMIC);
128+ if (!dbuf)
129+ return -ENOMEM;
130+
131+ ret = -ENOMEM;
132+ ticket_buf = kmem_cache_alloc(ceph_x_ticketbuf_cachep,
133+ GFP_NOFS | GFP_ATOMIC);
134+ if (!ticket_buf)
135+ goto out_dbuf;
136+
137+ ceph_decode_need(&p, end, 1 + sizeof(u32), bad);
138+ struct_v = ceph_decode_8(&p);
139+ if (struct_v != 1)
140+ goto bad;
141+ num = ceph_decode_32(&p);
142+ dout("%d tickets\n", num);
143+ while (num--) {
144+ int type;
145+ u8 struct_v;
146+ struct ceph_x_ticket_handler *th;
147+ void *dp, *dend;
148+ int dlen;
149+ char is_enc;
150+ struct timespec validity;
151+ struct ceph_crypto_key old_key;
152+ void *tp, *tpend;
153+
154+ ceph_decode_need(&p, end, sizeof(u32) + 1, bad);
155+
156+ type = ceph_decode_32(&p);
157+ dout(" ticket type %d %s\n", type, ceph_entity_type_name(type));
158+
159+ struct_v = ceph_decode_8(&p);
160+ if (struct_v != 1)
161+ goto bad;
162+
163+ th = get_ticket_handler(ac, type);
164+ if (IS_ERR(th)) {
165+ ret = PTR_ERR(th);
166+ goto out;
167+ }
168+
169+ /* blob for me */
170+ dlen = ceph_x_decrypt(secret, &p, end, dbuf,
171+ TEMP_TICKET_BUF_LEN);
172+ if (dlen <= 0) {
173+ ret = dlen;
174+ goto out;
175+ }
176+ dout(" decrypted %d bytes\n", dlen);
177+ dend = dbuf + dlen;
178+ dp = dbuf;
179+
180+ struct_v = ceph_decode_8(&dp);
181+ if (struct_v != 1)
182+ goto bad;
183+
184+ memcpy(&old_key, &th->session_key, sizeof(old_key));
185+ ret = ceph_crypto_key_decode(&th->session_key, &dp, dend);
186+ if (ret)
187+ goto out;
188+
189+ ceph_decode_copy(&dp, &th->validity, sizeof(th->validity));
190+ ceph_decode_timespec(&validity, &th->validity);
191+ th->expires = get_seconds() + validity.tv_sec;
192+ th->renew_after = th->expires - (validity.tv_sec / 4);
193+ dout(" expires=%lu renew_after=%lu\n", th->expires,
194+ th->renew_after);
195+
196+ /* ticket blob for service */
197+ ceph_decode_8_safe(&p, end, is_enc, bad);
198+ tp = ticket_buf;
199+ if (is_enc) {
200+ /* encrypted */
201+ dout(" encrypted ticket\n");
202+ dlen = ceph_x_decrypt(&old_key, &p, end, ticket_buf,
203+ TEMP_TICKET_BUF_LEN);
204+ if (dlen < 0) {
205+ ret = dlen;
206+ goto out;
207+ }
208+ dlen = ceph_decode_32(&tp);
209+ } else {
210+ /* unencrypted */
211+ ceph_decode_32_safe(&p, end, dlen, bad);
212+ ceph_decode_need(&p, end, dlen, bad);
213+ ceph_decode_copy(&p, ticket_buf, dlen);
214+ }
215+ tpend = tp + dlen;
216+ dout(" ticket blob is %d bytes\n", dlen);
217+ ceph_decode_need(&tp, tpend, 1 + sizeof(u64), bad);
218+ struct_v = ceph_decode_8(&tp);
219+ th->secret_id = ceph_decode_64(&tp);
220+ ret = ceph_decode_buffer(&th->ticket_blob, &tp, tpend);
221+ if (ret)
222+ goto out;
223+ dout(" got ticket service %d (%s) secret_id %lld len %d\n",
224+ type, ceph_entity_type_name(type), th->secret_id,
225+ (int)th->ticket_blob->vec.iov_len);
226+ xi->have_keys |= th->service;
227+ }
228+
229+ ret = 0;
230+out:
231+ kmem_cache_free(ceph_x_ticketbuf_cachep, ticket_buf);
232+out_dbuf:
233+ kmem_cache_free(ceph_x_ticketbuf_cachep, dbuf);
234+ return ret;
235+
236+bad:
237+ ret = -EINVAL;
238+ goto out;
239+}
240+
241+static int ceph_x_build_authorizer(struct ceph_auth_client *ac,
242+ struct ceph_x_ticket_handler *th,
243+ struct ceph_x_authorizer *au)
244+{
245+ int len;
246+ struct ceph_x_authorize_a *msg_a;
247+ struct ceph_x_authorize_b msg_b;
248+ void *p, *end;
249+ int ret;
250+ int ticket_blob_len =
251+ (th->ticket_blob ? th->ticket_blob->vec.iov_len : 0);
252+
253+ dout("build_authorizer for %s %p\n",
254+ ceph_entity_type_name(th->service), au);
255+
256+ len = sizeof(*msg_a) + sizeof(msg_b) + sizeof(u32) +
257+ ticket_blob_len + 16;
258+ dout(" need len %d\n", len);
259+ if (au->buf && au->buf->alloc_len < len) {
260+ ceph_buffer_put(au->buf);
261+ au->buf = NULL;
262+ }
263+ if (!au->buf) {
264+ au->buf = ceph_buffer_new(len, GFP_NOFS);
265+ if (!au->buf)
266+ return -ENOMEM;
267+ }
268+ au->service = th->service;
269+
270+ msg_a = au->buf->vec.iov_base;
271+ msg_a->struct_v = 1;
272+ msg_a->global_id = cpu_to_le64(ac->global_id);
273+ msg_a->service_id = cpu_to_le32(th->service);
274+ msg_a->ticket_blob.struct_v = 1;
275+ msg_a->ticket_blob.secret_id = cpu_to_le64(th->secret_id);
276+ msg_a->ticket_blob.blob_len = cpu_to_le32(ticket_blob_len);
277+ if (ticket_blob_len) {
278+ memcpy(msg_a->ticket_blob.blob, th->ticket_blob->vec.iov_base,
279+ th->ticket_blob->vec.iov_len);
280+ }
281+ dout(" th %p secret_id %lld %lld\n", th, th->secret_id,
282+ le64_to_cpu(msg_a->ticket_blob.secret_id));
283+
284+ p = msg_a + 1;
285+ p += ticket_blob_len;
286+ end = au->buf->vec.iov_base + au->buf->vec.iov_len;
287+
288+ get_random_bytes(&au->nonce, sizeof(au->nonce));
289+ msg_b.struct_v = 1;
290+ msg_b.nonce = cpu_to_le64(au->nonce);
291+ ret = ceph_x_encrypt(&th->session_key, &msg_b, sizeof(msg_b),
292+ p, end - p);
293+ if (ret < 0)
294+ goto out_buf;
295+ p += ret;
296+ au->buf->vec.iov_len = p - au->buf->vec.iov_base;
297+ dout(" built authorizer nonce %llx len %d\n", au->nonce,
298+ (int)au->buf->vec.iov_len);
299+ return 0;
300+
301+out_buf:
302+ ceph_buffer_put(au->buf);
303+ au->buf = NULL;
304+ return ret;
305+}
306+
307+static int ceph_x_encode_ticket(struct ceph_x_ticket_handler *th,
308+ void **p, void *end)
309+{
310+ ceph_decode_need(p, end, 1 + sizeof(u64), bad);
311+ ceph_encode_8(p, 1);
312+ ceph_encode_64(p, th->secret_id);
313+ if (th->ticket_blob) {
314+ const char *buf = th->ticket_blob->vec.iov_base;
315+ u32 len = th->ticket_blob->vec.iov_len;
316+
317+ ceph_encode_32_safe(p, end, len, bad);
318+ ceph_encode_copy_safe(p, end, buf, len, bad);
319+ } else {
320+ ceph_encode_32_safe(p, end, 0, bad);
321+ }
322+
323+ return 0;
324+bad:
325+ return -ERANGE;
326+}
327+
328+static void ceph_x_validate_tickets(struct ceph_auth_client *ac, int *pneed)
329+{
330+ int want = ac->want_keys;
331+ struct ceph_x_info *xi = ac->private;
332+ int service;
333+
334+ *pneed = ac->want_keys & ~(xi->have_keys);
335+
336+ for (service = 1; service <= want; service <<= 1) {
337+ struct ceph_x_ticket_handler *th;
338+
339+ if (!(ac->want_keys & service))
340+ continue;
341+
342+ if (*pneed & service)
343+ continue;
344+
345+ th = get_ticket_handler(ac, service);
346+
347+ if (!th) {
348+ *pneed |= service;
349+ continue;
350+ }
351+
352+ if (get_seconds() >= th->renew_after)
353+ *pneed |= service;
354+ if (get_seconds() >= th->expires)
355+ xi->have_keys &= ~service;
356+ }
357+}
358+
359+
360+static int ceph_x_build_request(struct ceph_auth_client *ac,
361+ void *buf, void *end)
362+{
363+ struct ceph_x_info *xi = ac->private;
364+ int need;
365+ struct ceph_x_request_header *head = buf;
366+ int ret;
367+ struct ceph_x_ticket_handler *th =
368+ get_ticket_handler(ac, CEPH_ENTITY_TYPE_AUTH);
369+
370+ ceph_x_validate_tickets(ac, &need);
371+
372+ dout("build_request want %x have %x need %x\n",
373+ ac->want_keys, xi->have_keys, need);
374+
375+ if (need & CEPH_ENTITY_TYPE_AUTH) {
376+ struct ceph_x_authenticate *auth = (void *)(head + 1);
377+ void *p = auth + 1;
378+ struct ceph_x_challenge_blob tmp;
379+ char tmp_enc[40];
380+ u64 *u;
381+
382+ if (p > end)
383+ return -ERANGE;
384+
385+ dout(" get_auth_session_key\n");
386+ head->op = cpu_to_le16(CEPHX_GET_AUTH_SESSION_KEY);
387+
388+ /* encrypt and hash */
389+ get_random_bytes(&auth->client_challenge, sizeof(u64));
390+ tmp.client_challenge = auth->client_challenge;
391+ tmp.server_challenge = cpu_to_le64(xi->server_challenge);
392+ ret = ceph_x_encrypt(&xi->secret, &tmp, sizeof(tmp),
393+ tmp_enc, sizeof(tmp_enc));
394+ if (ret < 0)
395+ return ret;
396+
397+ auth->struct_v = 1;
398+ auth->key = 0;
399+ for (u = (u64 *)tmp_enc; u + 1 <= (u64 *)(tmp_enc + ret); u++)
400+ auth->key ^= *u;
401+ dout(" server_challenge %llx client_challenge %llx key %llx\n",
402+ xi->server_challenge, le64_to_cpu(auth->client_challenge),
403+ le64_to_cpu(auth->key));
404+
405+ /* now encode the old ticket if exists */
406+ ret = ceph_x_encode_ticket(th, &p, end);
407+ if (ret < 0)
408+ return ret;
409+
410+ return p - buf;
411+ }
412+
413+ if (need) {
414+ void *p = head + 1;
415+ struct ceph_x_service_ticket_request *req;
416+
417+ if (p > end)
418+ return -ERANGE;
419+ head->op = cpu_to_le16(CEPHX_GET_PRINCIPAL_SESSION_KEY);
420+
421+ BUG_ON(!th);
422+ ret = ceph_x_build_authorizer(ac, th, &xi->auth_authorizer);
423+ if (ret)
424+ return ret;
425+ ceph_encode_copy(&p, xi->auth_authorizer.buf->vec.iov_base,
426+ xi->auth_authorizer.buf->vec.iov_len);
427+
428+ req = p;
429+ req->keys = cpu_to_le32(need);
430+ p += sizeof(*req);
431+ return p - buf;
432+ }
433+
434+ return 0;
435+}
436+
437+static int ceph_x_handle_reply(struct ceph_auth_client *ac, int result,
438+ void *buf, void *end)
439+{
440+ struct ceph_x_info *xi = ac->private;
441+ struct ceph_x_reply_header *head = buf;
442+ struct ceph_x_ticket_handler *th;
443+ int len = end - buf;
444+ int op;
445+ int ret;
446+
447+ if (result)
448+ return result; /* XXX hmm? */
449+
450+ if (xi->starting) {
451+ /* it's a hello */
452+ struct ceph_x_server_challenge *sc = buf;
453+
454+ if (len != sizeof(*sc))
455+ return -EINVAL;
456+ xi->server_challenge = le64_to_cpu(sc->server_challenge);
457+ dout("handle_reply got server challenge %llx\n",
458+ xi->server_challenge);
459+ xi->starting = false;
460+ xi->have_keys &= ~CEPH_ENTITY_TYPE_AUTH;
461+ return -EAGAIN;
462+ }
463+
464+ op = le32_to_cpu(head->op);
465+ result = le32_to_cpu(head->result);
466+ dout("handle_reply op %d result %d\n", op, result);
467+ switch (op) {
468+ case CEPHX_GET_AUTH_SESSION_KEY:
469+ /* verify auth key */
470+ ret = ceph_x_proc_ticket_reply(ac, &xi->secret,
471+ buf + sizeof(*head), end);
472+ break;
473+
474+ case CEPHX_GET_PRINCIPAL_SESSION_KEY:
475+ th = get_ticket_handler(ac, CEPH_ENTITY_TYPE_AUTH);
476+ BUG_ON(!th);
477+ ret = ceph_x_proc_ticket_reply(ac, &th->session_key,
478+ buf + sizeof(*head), end);
479+ break;
480+
481+ default:
482+ return -EINVAL;
483+ }
484+ if (ret)
485+ return ret;
486+ if (ac->want_keys == xi->have_keys)
487+ return 0;
488+ return -EAGAIN;
489+}
490+
491+static int ceph_x_create_authorizer(
492+ struct ceph_auth_client *ac, int peer_type,
493+ struct ceph_authorizer **a,
494+ void **buf, size_t *len,
495+ void **reply_buf, size_t *reply_len)
496+{
497+ struct ceph_x_authorizer *au;
498+ struct ceph_x_ticket_handler *th;
499+ int ret;
500+
501+ th = get_ticket_handler(ac, peer_type);
502+ if (IS_ERR(th))
503+ return PTR_ERR(th);
504+
505+ au = kzalloc(sizeof(*au), GFP_NOFS);
506+ if (!au)
507+ return -ENOMEM;
508+
509+ ret = ceph_x_build_authorizer(ac, th, au);
510+ if (ret) {
511+ kfree(au);
512+ return ret;
513+ }
514+
515+ *a = (struct ceph_authorizer *)au;
516+ *buf = au->buf->vec.iov_base;
517+ *len = au->buf->vec.iov_len;
518+ *reply_buf = au->reply_buf;
519+ *reply_len = sizeof(au->reply_buf);
520+ return 0;
521+}
522+
523+static int ceph_x_verify_authorizer_reply(struct ceph_auth_client *ac,
524+ struct ceph_authorizer *a, size_t len)
525+{
526+ struct ceph_x_authorizer *au = (void *)a;
527+ struct ceph_x_ticket_handler *th;
528+ int ret = 0;
529+ struct ceph_x_authorize_reply reply;
530+ void *p = au->reply_buf;
531+ void *end = p + sizeof(au->reply_buf);
532+
533+ th = get_ticket_handler(ac, au->service);
534+ if (!th)
535+ return -EIO; /* hrm! */
536+ ret = ceph_x_decrypt(&th->session_key, &p, end, &reply, sizeof(reply));
537+ if (ret < 0)
538+ return ret;
539+ if (ret != sizeof(reply))
540+ return -EPERM;
541+
542+ if (au->nonce + 1 != le64_to_cpu(reply.nonce_plus_one))
543+ ret = -EPERM;
544+ else
545+ ret = 0;
546+ dout("verify_authorizer_reply nonce %llx got %llx ret %d\n",
547+ au->nonce, le64_to_cpu(reply.nonce_plus_one), ret);
548+ return ret;
549+}
550+
551+static void ceph_x_destroy_authorizer(struct ceph_auth_client *ac,
552+ struct ceph_authorizer *a)
553+{
554+ struct ceph_x_authorizer *au = (void *)a;
555+
556+ ceph_buffer_put(au->buf);
557+ kfree(au);
558+}
559+
560+
561+static void ceph_x_reset(struct ceph_auth_client *ac)
562+{
563+ struct ceph_x_info *xi = ac->private;
564+
565+ dout("reset\n");
566+ xi->starting = true;
567+ xi->server_challenge = 0;
568+}
569+
570+static void ceph_x_destroy(struct ceph_auth_client *ac)
571+{
572+ struct ceph_x_info *xi = ac->private;
573+ struct rb_node *p;
574+
575+ dout("ceph_x_destroy %p\n", ac);
576+ ceph_crypto_key_destroy(&xi->secret);
577+
578+ while ((p = rb_first(&xi->ticket_handlers)) != NULL) {
579+ struct ceph_x_ticket_handler *th =
580+ rb_entry(p, struct ceph_x_ticket_handler, node);
581+ remove_ticket_handler(ac, th);
582+ }
583+
584+ kmem_cache_destroy(ceph_x_ticketbuf_cachep);
585+
586+ kfree(ac->private);
587+ ac->private = NULL;
588+}
589+
590+static void ceph_x_invalidate_authorizer(struct ceph_auth_client *ac,
591+ int peer_type)
592+{
593+ struct ceph_x_ticket_handler *th;
594+
595+ th = get_ticket_handler(ac, peer_type);
596+ if (th && !IS_ERR(th))
597+ remove_ticket_handler(ac, th);
598+}
599+
600+
601+static const struct ceph_auth_client_ops ceph_x_ops = {
602+ .is_authenticated = ceph_x_is_authenticated,
603+ .build_request = ceph_x_build_request,
604+ .handle_reply = ceph_x_handle_reply,
605+ .create_authorizer = ceph_x_create_authorizer,
606+ .verify_authorizer_reply = ceph_x_verify_authorizer_reply,
607+ .destroy_authorizer = ceph_x_destroy_authorizer,
608+ .invalidate_authorizer = ceph_x_invalidate_authorizer,
609+ .reset = ceph_x_reset,
610+ .destroy = ceph_x_destroy,
611+};
612+
613+
614+int ceph_x_init(struct ceph_auth_client *ac)
615+{
616+ struct ceph_x_info *xi;
617+ int ret;
618+
619+ dout("ceph_x_init %p\n", ac);
620+ xi = kzalloc(sizeof(*xi), GFP_NOFS);
621+ if (!xi)
622+ return -ENOMEM;
623+
624+ ret = -ENOMEM;
625+ ceph_x_ticketbuf_cachep = kmem_cache_create("ceph_x_ticketbuf",
626+ TEMP_TICKET_BUF_LEN, 8,
627+ (SLAB_RECLAIM_ACCOUNT|SLAB_MEM_SPREAD),
628+ NULL);
629+ if (!ceph_x_ticketbuf_cachep)
630+ goto done_nomem;
631+ ret = -EINVAL;
632+ if (!ac->secret) {
633+ pr_err("no secret set (for auth_x protocol)\n");
634+ goto done_nomem;
635+ }
636+
637+ ret = ceph_crypto_key_unarmor(&xi->secret, ac->secret);
638+ if (ret)
639+ goto done_nomem;
640+
641+ xi->starting = true;
642+ xi->ticket_handlers = RB_ROOT;
643+
644+ ac->protocol = CEPH_AUTH_CEPHX;
645+ ac->private = xi;
646+ ac->ops = &ceph_x_ops;
647+ return 0;
648+
649+done_nomem:
650+ kfree(xi);
651+ if (ceph_x_ticketbuf_cachep)
652+ kmem_cache_destroy(ceph_x_ticketbuf_cachep);
653+ return ret;
654+}
655+
656+
fs/ceph/auth_x.h+49−0
@@ -0,0 +1,49 @@
1+#ifndef _FS_CEPH_AUTH_X_H
2+#define _FS_CEPH_AUTH_X_H
3+
4+#include <linux/rbtree.h>
5+
6+#include "crypto.h"
7+#include "auth.h"
8+#include "auth_x_protocol.h"
9+
10+/*
11+ * Handle ticket for a single service.
12+ */
13+struct ceph_x_ticket_handler {
14+ struct rb_node node;
15+ unsigned service;
16+
17+ struct ceph_crypto_key session_key;
18+ struct ceph_timespec validity;
19+
20+ u64 secret_id;
21+ struct ceph_buffer *ticket_blob;
22+
23+ unsigned long renew_after, expires;
24+};
25+
26+
27+struct ceph_x_authorizer {
28+ struct ceph_buffer *buf;
29+ unsigned service;
30+ u64 nonce;
31+ char reply_buf[128]; /* big enough for encrypted blob */
32+};
33+
34+struct ceph_x_info {
35+ struct ceph_crypto_key secret;
36+
37+ bool starting;
38+ u64 server_challenge;
39+
40+ unsigned have_keys;
41+ struct rb_root ticket_handlers;
42+
43+ struct ceph_x_authorizer auth_authorizer;
44+};
45+
46+extern int ceph_x_init(struct ceph_auth_client *ac);
47+
48+#endif
49+
fs/ceph/auth_x_protocol.h+90−0
@@ -0,0 +1,90 @@
1+#ifndef __FS_CEPH_AUTH_X_PROTOCOL
2+#define __FS_CEPH_AUTH_X_PROTOCOL
3+
4+#define CEPHX_GET_AUTH_SESSION_KEY 0x0100
5+#define CEPHX_GET_PRINCIPAL_SESSION_KEY 0x0200
6+#define CEPHX_GET_ROTATING_KEY 0x0400
7+
8+/* common bits */
9+struct ceph_x_ticket_blob {
10+ __u8 struct_v;
11+ __le64 secret_id;
12+ __le32 blob_len;
13+ char blob[];
14+} __attribute__ ((packed));
15+
16+
17+/* common request/reply headers */
18+struct ceph_x_request_header {
19+ __le16 op;
20+} __attribute__ ((packed));
21+
22+struct ceph_x_reply_header {
23+ __le16 op;
24+ __le32 result;
25+} __attribute__ ((packed));
26+
27+
28+/* authenticate handshake */
29+
30+/* initial hello (no reply header) */
31+struct ceph_x_server_challenge {
32+ __u8 struct_v;
33+ __le64 server_challenge;
34+} __attribute__ ((packed));
35+
36+struct ceph_x_authenticate {
37+ __u8 struct_v;
38+ __le64 client_challenge;
39+ __le64 key;
40+ /* ticket blob */
41+} __attribute__ ((packed));
42+
43+struct ceph_x_service_ticket_request {
44+ __u8 struct_v;
45+ __le32 keys;
46+} __attribute__ ((packed));
47+
48+struct ceph_x_challenge_blob {
49+ __le64 server_challenge;
50+ __le64 client_challenge;
51+} __attribute__ ((packed));
52+
53+
54+
55+/* authorize handshake */
56+
57+/*
58+ * The authorizer consists of two pieces:
59+ * a - service id, ticket blob
60+ * b - encrypted with session key
61+ */
62+struct ceph_x_authorize_a {
63+ __u8 struct_v;
64+ __le64 global_id;
65+ __le32 service_id;
66+ struct ceph_x_ticket_blob ticket_blob;
67+} __attribute__ ((packed));
68+
69+struct ceph_x_authorize_b {
70+ __u8 struct_v;
71+ __le64 nonce;
72+} __attribute__ ((packed));
73+
74+struct ceph_x_authorize_reply {
75+ __u8 struct_v;
76+ __le64 nonce_plus_one;
77+} __attribute__ ((packed));
78+
79+
80+/*
81+ * encyption bundle
82+ */
83+#define CEPHX_ENC_MAGIC 0xff009cad8826aa55ull
84+
85+struct ceph_x_encrypt_header {
86+ __u8 struct_v;
87+ __le64 magic;
88+} __attribute__ ((packed));
89+
90+#endif
fs/ceph/buffer.c+78−0
@@ -0,0 +1,78 @@
1+
2+#include "ceph_debug.h"
3+#include "buffer.h"
4+#include "decode.h"
5+
6+struct ceph_buffer *ceph_buffer_new(size_t len, gfp_t gfp)
7+{
8+ struct ceph_buffer *b;
9+
10+ b = kmalloc(sizeof(*b), gfp);
11+ if (!b)
12+ return NULL;
13+
14+ b->vec.iov_base = kmalloc(len, gfp | __GFP_NOWARN);
15+ if (b->vec.iov_base) {
16+ b->is_vmalloc = false;
17+ } else {
18+ b->vec.iov_base = __vmalloc(len, gfp, PAGE_KERNEL);
19+ if (!b->vec.iov_base) {
20+ kfree(b);
21+ return NULL;
22+ }
23+ b->is_vmalloc = true;
24+ }
25+
26+ kref_init(&b->kref);
27+ b->alloc_len = len;
28+ b->vec.iov_len = len;
29+ dout("buffer_new %p\n", b);
30+ return b;
31+}
32+
33+void ceph_buffer_release(struct kref *kref)
34+{
35+ struct ceph_buffer *b = container_of(kref, struct ceph_buffer, kref);
36+
37+ dout("buffer_release %p\n", b);
38+ if (b->vec.iov_base) {
39+ if (b->is_vmalloc)
40+ vfree(b->vec.iov_base);
41+ else
42+ kfree(b->vec.iov_base);
43+ }
44+ kfree(b);
45+}
46+
47+int ceph_buffer_alloc(struct ceph_buffer *b, int len, gfp_t gfp)
48+{
49+ b->vec.iov_base = kmalloc(len, gfp | __GFP_NOWARN);
50+ if (b->vec.iov_base) {
51+ b->is_vmalloc = false;
52+ } else {
53+ b->vec.iov_base = __vmalloc(len, gfp, PAGE_KERNEL);
54+ b->is_vmalloc = true;
55+ }
56+ if (!b->vec.iov_base)
57+ return -ENOMEM;
58+ b->alloc_len = len;
59+ b->vec.iov_len = len;
60+ return 0;
61+}
62+
63+int ceph_decode_buffer(struct ceph_buffer **b, void **p, void *end)
64+{
65+ size_t len;
66+
67+ ceph_decode_need(p, end, sizeof(u32), bad);
68+ len = ceph_decode_32(p);
69+ dout("decode_buffer len %d\n", (int)len);
70+ ceph_decode_need(p, end, len, bad);
71+ *b = ceph_buffer_new(len, GFP_NOFS);
72+ if (!*b)
73+ return -ENOMEM;
74+ ceph_decode_copy(p, (*b)->vec.iov_base, len);
75+ return 0;
76+bad:
77+ return -EINVAL;
78+}
fs/ceph/buffer.h+39−0
@@ -0,0 +1,39 @@
1+#ifndef __FS_CEPH_BUFFER_H
2+#define __FS_CEPH_BUFFER_H
3+
4+#include <linux/kref.h>
5+#include <linux/mm.h>
6+#include <linux/vmalloc.h>
7+#include <linux/types.h>
8+#include <linux/uio.h>
9+
10+/*
11+ * a simple reference counted buffer.
12+ *
13+ * use kmalloc for small sizes (<= one page), vmalloc for larger
14+ * sizes.
15+ */
16+struct ceph_buffer {
17+ struct kref kref;
18+ struct kvec vec;
19+ size_t alloc_len;
20+ bool is_vmalloc;
21+};
22+
23+extern struct ceph_buffer *ceph_buffer_new(size_t len, gfp_t gfp);
24+extern void ceph_buffer_release(struct kref *kref);
25+
26+static inline struct ceph_buffer *ceph_buffer_get(struct ceph_buffer *b)
27+{
28+ kref_get(&b->kref);
29+ return b;
30+}
31+
32+static inline void ceph_buffer_put(struct ceph_buffer *b)
33+{
34+ kref_put(&b->kref, ceph_buffer_release);
35+}
36+
37+extern int ceph_decode_buffer(struct ceph_buffer **b, void **p, void *end);
38+
39+#endif
fs/ceph/caps.c+2927−0
@@ -0,0 +1,2927 @@
1+#include "ceph_debug.h"
2+
3+#include <linux/fs.h>
4+#include <linux/kernel.h>
5+#include <linux/sched.h>
6+#include <linux/vmalloc.h>
7+#include <linux/wait.h>
8+#include <linux/writeback.h>
9+
10+#include "super.h"
11+#include "decode.h"
12+#include "messenger.h"
13+
14+/*
15+ * Capability management
16+ *
17+ * The Ceph metadata servers control client access to inode metadata
18+ * and file data by issuing capabilities, granting clients permission
19+ * to read and/or write both inode field and file data to OSDs
20+ * (storage nodes). Each capability consists of a set of bits
21+ * indicating which operations are allowed.
22+ *
23+ * If the client holds a *_SHARED cap, the client has a coherent value
24+ * that can be safely read from the cached inode.
25+ *
26+ * In the case of a *_EXCL (exclusive) or FILE_WR capabilities, the
27+ * client is allowed to change inode attributes (e.g., file size,
28+ * mtime), note its dirty state in the ceph_cap, and asynchronously
29+ * flush that metadata change to the MDS.
30+ *
31+ * In the event of a conflicting operation (perhaps by another
32+ * client), the MDS will revoke the conflicting client capabilities.
33+ *
34+ * In order for a client to cache an inode, it must hold a capability
35+ * with at least one MDS server. When inodes are released, release
36+ * notifications are batched and periodically sent en masse to the MDS
37+ * cluster to release server state.
38+ */
39+
40+
41+/*
42+ * Generate readable cap strings for debugging output.
43+ */
44+#define MAX_CAP_STR 20
45+static char cap_str[MAX_CAP_STR][40];
46+static DEFINE_SPINLOCK(cap_str_lock);
47+static int last_cap_str;
48+
49+static char *gcap_string(char *s, int c)
50+{
51+ if (c & CEPH_CAP_GSHARED)
52+ *s++ = 's';
53+ if (c & CEPH_CAP_GEXCL)
54+ *s++ = 'x';
55+ if (c & CEPH_CAP_GCACHE)
56+ *s++ = 'c';
57+ if (c & CEPH_CAP_GRD)
58+ *s++ = 'r';
59+ if (c & CEPH_CAP_GWR)
60+ *s++ = 'w';
61+ if (c & CEPH_CAP_GBUFFER)
62+ *s++ = 'b';
63+ if (c & CEPH_CAP_GLAZYIO)
64+ *s++ = 'l';
65+ return s;
66+}
67+
68+const char *ceph_cap_string(int caps)
69+{
70+ int i;
71+ char *s;
72+ int c;
73+
74+ spin_lock(&cap_str_lock);
75+ i = last_cap_str++;
76+ if (last_cap_str == MAX_CAP_STR)
77+ last_cap_str = 0;
78+ spin_unlock(&cap_str_lock);
79+
80+ s = cap_str[i];
81+
82+ if (caps & CEPH_CAP_PIN)
83+ *s++ = 'p';
84+
85+ c = (caps >> CEPH_CAP_SAUTH) & 3;
86+ if (c) {
87+ *s++ = 'A';
88+ s = gcap_string(s, c);
89+ }
90+
91+ c = (caps >> CEPH_CAP_SLINK) & 3;
92+ if (c) {
93+ *s++ = 'L';
94+ s = gcap_string(s, c);
95+ }
96+
97+ c = (caps >> CEPH_CAP_SXATTR) & 3;
98+ if (c) {
99+ *s++ = 'X';
100+ s = gcap_string(s, c);
101+ }
102+
103+ c = caps >> CEPH_CAP_SFILE;
104+ if (c) {
105+ *s++ = 'F';
106+ s = gcap_string(s, c);
107+ }
108+
109+ if (s == cap_str[i])
110+ *s++ = '-';
111+ *s = 0;
112+ return cap_str[i];
113+}
114+
115+/*
116+ * Cap reservations
117+ *
118+ * Maintain a global pool of preallocated struct ceph_caps, referenced
119+ * by struct ceph_caps_reservations. This ensures that we preallocate
120+ * memory needed to successfully process an MDS response. (If an MDS
121+ * sends us cap information and we fail to process it, we will have
122+ * problems due to the client and MDS being out of sync.)
123+ *
124+ * Reservations are 'owned' by a ceph_cap_reservation context.
125+ */
126+static spinlock_t caps_list_lock;
127+static struct list_head caps_list; /* unused (reserved or unreserved) */
128+static int caps_total_count; /* total caps allocated */
129+static int caps_use_count; /* in use */
130+static int caps_reserve_count; /* unused, reserved */
131+static int caps_avail_count; /* unused, unreserved */
132+static int caps_min_count; /* keep at least this many (unreserved) */
133+
134+void __init ceph_caps_init(void)
135+{
136+ INIT_LIST_HEAD(&caps_list);
137+ spin_lock_init(&caps_list_lock);
138+}
139+
140+void ceph_caps_finalize(void)
141+{
142+ struct ceph_cap *cap;
143+
144+ spin_lock(&caps_list_lock);
145+ while (!list_empty(&caps_list)) {
146+ cap = list_first_entry(&caps_list, struct ceph_cap, caps_item);
147+ list_del(&cap->caps_item);
148+ kmem_cache_free(ceph_cap_cachep, cap);
149+ }
150+ caps_total_count = 0;
151+ caps_avail_count = 0;
152+ caps_use_count = 0;
153+ caps_reserve_count = 0;
154+ caps_min_count = 0;
155+ spin_unlock(&caps_list_lock);
156+}
157+
158+void ceph_adjust_min_caps(int delta)
159+{
160+ spin_lock(&caps_list_lock);
161+ caps_min_count += delta;
162+ BUG_ON(caps_min_count < 0);
163+ spin_unlock(&caps_list_lock);
164+}
165+
166+int ceph_reserve_caps(struct ceph_cap_reservation *ctx, int need)
167+{
168+ int i;
169+ struct ceph_cap *cap;
170+ int have;
171+ int alloc = 0;
172+ LIST_HEAD(newcaps);
173+ int ret = 0;
174+
175+ dout("reserve caps ctx=%p need=%d\n", ctx, need);
176+
177+ /* first reserve any caps that are already allocated */
178+ spin_lock(&caps_list_lock);
179+ if (caps_avail_count >= need)
180+ have = need;
181+ else
182+ have = caps_avail_count;
183+ caps_avail_count -= have;
184+ caps_reserve_count += have;
185+ BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
186+ caps_avail_count);
187+ spin_unlock(&caps_list_lock);
188+
189+ for (i = have; i < need; i++) {
190+ cap = kmem_cache_alloc(ceph_cap_cachep, GFP_NOFS);
191+ if (!cap) {
192+ ret = -ENOMEM;
193+ goto out_alloc_count;
194+ }
195+ list_add(&cap->caps_item, &newcaps);
196+ alloc++;
197+ }
198+ BUG_ON(have + alloc != need);
199+
200+ spin_lock(&caps_list_lock);
201+ caps_total_count += alloc;
202+ caps_reserve_count += alloc;
203+ list_splice(&newcaps, &caps_list);
204+
205+ BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
206+ caps_avail_count);
207+ spin_unlock(&caps_list_lock);
208+
209+ ctx->count = need;
210+ dout("reserve caps ctx=%p %d = %d used + %d resv + %d avail\n",
211+ ctx, caps_total_count, caps_use_count, caps_reserve_count,
212+ caps_avail_count);
213+ return 0;
214+
215+out_alloc_count:
216+ /* we didn't manage to reserve as much as we needed */
217+ pr_warning("reserve caps ctx=%p ENOMEM need=%d got=%d\n",
218+ ctx, need, have);
219+ return ret;
220+}
221+
222+int ceph_unreserve_caps(struct ceph_cap_reservation *ctx)
223+{
224+ dout("unreserve caps ctx=%p count=%d\n", ctx, ctx->count);
225+ if (ctx->count) {
226+ spin_lock(&caps_list_lock);
227+ BUG_ON(caps_reserve_count < ctx->count);
228+ caps_reserve_count -= ctx->count;
229+ caps_avail_count += ctx->count;
230+ ctx->count = 0;
231+ dout("unreserve caps %d = %d used + %d resv + %d avail\n",
232+ caps_total_count, caps_use_count, caps_reserve_count,
233+ caps_avail_count);
234+ BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
235+ caps_avail_count);
236+ spin_unlock(&caps_list_lock);
237+ }
238+ return 0;
239+}
240+
241+static struct ceph_cap *get_cap(struct ceph_cap_reservation *ctx)
242+{
243+ struct ceph_cap *cap = NULL;
244+
245+ /* temporary, until we do something about cap import/export */
246+ if (!ctx)
247+ return kmem_cache_alloc(ceph_cap_cachep, GFP_NOFS);
248+
249+ spin_lock(&caps_list_lock);
250+ dout("get_cap ctx=%p (%d) %d = %d used + %d resv + %d avail\n",
251+ ctx, ctx->count, caps_total_count, caps_use_count,
252+ caps_reserve_count, caps_avail_count);
253+ BUG_ON(!ctx->count);
254+ BUG_ON(ctx->count > caps_reserve_count);
255+ BUG_ON(list_empty(&caps_list));
256+
257+ ctx->count--;
258+ caps_reserve_count--;
259+ caps_use_count++;
260+
261+ cap = list_first_entry(&caps_list, struct ceph_cap, caps_item);
262+ list_del(&cap->caps_item);
263+
264+ BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
265+ caps_avail_count);
266+ spin_unlock(&caps_list_lock);
267+ return cap;
268+}
269+
270+void ceph_put_cap(struct ceph_cap *cap)
271+{
272+ spin_lock(&caps_list_lock);
273+ dout("put_cap %p %d = %d used + %d resv + %d avail\n",
274+ cap, caps_total_count, caps_use_count,
275+ caps_reserve_count, caps_avail_count);
276+ caps_use_count--;
277+ /*
278+ * Keep some preallocated caps around (ceph_min_count), to
279+ * avoid lots of free/alloc churn.
280+ */
281+ if (caps_avail_count >= caps_reserve_count + caps_min_count) {
282+ caps_total_count--;
283+ kmem_cache_free(ceph_cap_cachep, cap);
284+ } else {
285+ caps_avail_count++;
286+ list_add(&cap->caps_item, &caps_list);
287+ }
288+
289+ BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
290+ caps_avail_count);
291+ spin_unlock(&caps_list_lock);
292+}
293+
294+void ceph_reservation_status(struct ceph_client *client,
295+ int *total, int *avail, int *used, int *reserved,
296+ int *min)
297+{
298+ if (total)
299+ *total = caps_total_count;
300+ if (avail)
301+ *avail = caps_avail_count;
302+ if (used)
303+ *used = caps_use_count;
304+ if (reserved)
305+ *reserved = caps_reserve_count;
306+ if (min)
307+ *min = caps_min_count;
308+}
309+
310+/*
311+ * Find ceph_cap for given mds, if any.
312+ *
313+ * Called with i_lock held.
314+ */
315+static struct ceph_cap *__get_cap_for_mds(struct ceph_inode_info *ci, int mds)
316+{
317+ struct ceph_cap *cap;
318+ struct rb_node *n = ci->i_caps.rb_node;
319+
320+ while (n) {
321+ cap = rb_entry(n, struct ceph_cap, ci_node);
322+ if (mds < cap->mds)
323+ n = n->rb_left;
324+ else if (mds > cap->mds)
325+ n = n->rb_right;
326+ else
327+ return cap;
328+ }
329+ return NULL;
330+}
331+
332+/*
333+ * Return id of any MDS with a cap, preferably FILE_WR|WRBUFFER|EXCL, else
334+ * -1.
335+ */
336+static int __ceph_get_cap_mds(struct ceph_inode_info *ci, u32 *mseq)
337+{
338+ struct ceph_cap *cap;
339+ int mds = -1;
340+ struct rb_node *p;
341+
342+ /* prefer mds with WR|WRBUFFER|EXCL caps */
343+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
344+ cap = rb_entry(p, struct ceph_cap, ci_node);
345+ mds = cap->mds;
346+ if (mseq)
347+ *mseq = cap->mseq;
348+ if (cap->issued & (CEPH_CAP_FILE_WR |
349+ CEPH_CAP_FILE_BUFFER |
350+ CEPH_CAP_FILE_EXCL))
351+ break;
352+ }
353+ return mds;
354+}
355+
356+int ceph_get_cap_mds(struct inode *inode)
357+{
358+ int mds;
359+ spin_lock(&inode->i_lock);
360+ mds = __ceph_get_cap_mds(ceph_inode(inode), NULL);
361+ spin_unlock(&inode->i_lock);
362+ return mds;
363+}
364+
365+/*
366+ * Called under i_lock.
367+ */
368+static void __insert_cap_node(struct ceph_inode_info *ci,
369+ struct ceph_cap *new)
370+{
371+ struct rb_node **p = &ci->i_caps.rb_node;
372+ struct rb_node *parent = NULL;
373+ struct ceph_cap *cap = NULL;
374+
375+ while (*p) {
376+ parent = *p;
377+ cap = rb_entry(parent, struct ceph_cap, ci_node);
378+ if (new->mds < cap->mds)
379+ p = &(*p)->rb_left;
380+ else if (new->mds > cap->mds)
381+ p = &(*p)->rb_right;
382+ else
383+ BUG();
384+ }
385+
386+ rb_link_node(&new->ci_node, parent, p);
387+ rb_insert_color(&new->ci_node, &ci->i_caps);
388+}
389+
390+/*
391+ * (re)set cap hold timeouts, which control the delayed release
392+ * of unused caps back to the MDS. Should be called on cap use.
393+ */
394+static void __cap_set_timeouts(struct ceph_mds_client *mdsc,
395+ struct ceph_inode_info *ci)
396+{
397+ struct ceph_mount_args *ma = mdsc->client->mount_args;
398+
399+ ci->i_hold_caps_min = round_jiffies(jiffies +
400+ ma->caps_wanted_delay_min * HZ);
401+ ci->i_hold_caps_max = round_jiffies(jiffies +
402+ ma->caps_wanted_delay_max * HZ);
403+ dout("__cap_set_timeouts %p min %lu max %lu\n", &ci->vfs_inode,
404+ ci->i_hold_caps_min - jiffies, ci->i_hold_caps_max - jiffies);
405+}
406+
407+/*
408+ * (Re)queue cap at the end of the delayed cap release list.
409+ *
410+ * If I_FLUSH is set, leave the inode at the front of the list.
411+ *
412+ * Caller holds i_lock
413+ * -> we take mdsc->cap_delay_lock
414+ */
415+static void __cap_delay_requeue(struct ceph_mds_client *mdsc,
416+ struct ceph_inode_info *ci)
417+{
418+ __cap_set_timeouts(mdsc, ci);
419+ dout("__cap_delay_requeue %p flags %d at %lu\n", &ci->vfs_inode,
420+ ci->i_ceph_flags, ci->i_hold_caps_max);
421+ if (!mdsc->stopping) {
422+ spin_lock(&mdsc->cap_delay_lock);
423+ if (!list_empty(&ci->i_cap_delay_list)) {
424+ if (ci->i_ceph_flags & CEPH_I_FLUSH)
425+ goto no_change;
426+ list_del_init(&ci->i_cap_delay_list);
427+ }
428+ list_add_tail(&ci->i_cap_delay_list, &mdsc->cap_delay_list);
429+no_change:
430+ spin_unlock(&mdsc->cap_delay_lock);
431+ }
432+}
433+
434+/*
435+ * Queue an inode for immediate writeback. Mark inode with I_FLUSH,
436+ * indicating we should send a cap message to flush dirty metadata
437+ * asap, and move to the front of the delayed cap list.
438+ */
439+static void __cap_delay_requeue_front(struct ceph_mds_client *mdsc,
440+ struct ceph_inode_info *ci)
441+{
442+ dout("__cap_delay_requeue_front %p\n", &ci->vfs_inode);
443+ spin_lock(&mdsc->cap_delay_lock);
444+ ci->i_ceph_flags |= CEPH_I_FLUSH;
445+ if (!list_empty(&ci->i_cap_delay_list))
446+ list_del_init(&ci->i_cap_delay_list);
447+ list_add(&ci->i_cap_delay_list, &mdsc->cap_delay_list);
448+ spin_unlock(&mdsc->cap_delay_lock);
449+}
450+
451+/*
452+ * Cancel delayed work on cap.
453+ *
454+ * Caller must hold i_lock.
455+ */
456+static void __cap_delay_cancel(struct ceph_mds_client *mdsc,
457+ struct ceph_inode_info *ci)
458+{
459+ dout("__cap_delay_cancel %p\n", &ci->vfs_inode);
460+ if (list_empty(&ci->i_cap_delay_list))
461+ return;
462+ spin_lock(&mdsc->cap_delay_lock);
463+ list_del_init(&ci->i_cap_delay_list);
464+ spin_unlock(&mdsc->cap_delay_lock);
465+}
466+
467+/*
468+ * Common issue checks for add_cap, handle_cap_grant.
469+ */
470+static void __check_cap_issue(struct ceph_inode_info *ci, struct ceph_cap *cap,
471+ unsigned issued)
472+{
473+ unsigned had = __ceph_caps_issued(ci, NULL);
474+
475+ /*
476+ * Each time we receive FILE_CACHE anew, we increment
477+ * i_rdcache_gen.
478+ */
479+ if ((issued & CEPH_CAP_FILE_CACHE) &&
480+ (had & CEPH_CAP_FILE_CACHE) == 0)
481+ ci->i_rdcache_gen++;
482+
483+ /*
484+ * if we are newly issued FILE_SHARED, clear I_COMPLETE; we
485+ * don't know what happened to this directory while we didn't
486+ * have the cap.
487+ */
488+ if ((issued & CEPH_CAP_FILE_SHARED) &&
489+ (had & CEPH_CAP_FILE_SHARED) == 0) {
490+ ci->i_shared_gen++;
491+ if (S_ISDIR(ci->vfs_inode.i_mode)) {
492+ dout(" marking %p NOT complete\n", &ci->vfs_inode);
493+ ci->i_ceph_flags &= ~CEPH_I_COMPLETE;
494+ }
495+ }
496+}
497+
498+/*
499+ * Add a capability under the given MDS session.
500+ *
501+ * Caller should hold session snap_rwsem (read) and s_mutex.
502+ *
503+ * @fmode is the open file mode, if we are opening a file, otherwise
504+ * it is < 0. (This is so we can atomically add the cap and add an
505+ * open file reference to it.)
506+ */
507+int ceph_add_cap(struct inode *inode,
508+ struct ceph_mds_session *session, u64 cap_id,
509+ int fmode, unsigned issued, unsigned wanted,
510+ unsigned seq, unsigned mseq, u64 realmino, int flags,
511+ struct ceph_cap_reservation *caps_reservation)
512+{
513+ struct ceph_mds_client *mdsc = &ceph_inode_to_client(inode)->mdsc;
514+ struct ceph_inode_info *ci = ceph_inode(inode);
515+ struct ceph_cap *new_cap = NULL;
516+ struct ceph_cap *cap;
517+ int mds = session->s_mds;
518+ int actual_wanted;
519+
520+ dout("add_cap %p mds%d cap %llx %s seq %d\n", inode,
521+ session->s_mds, cap_id, ceph_cap_string(issued), seq);
522+
523+ /*
524+ * If we are opening the file, include file mode wanted bits
525+ * in wanted.
526+ */
527+ if (fmode >= 0)
528+ wanted |= ceph_caps_for_mode(fmode);
529+
530+retry:
531+ spin_lock(&inode->i_lock);
532+ cap = __get_cap_for_mds(ci, mds);
533+ if (!cap) {
534+ if (new_cap) {
535+ cap = new_cap;
536+ new_cap = NULL;
537+ } else {
538+ spin_unlock(&inode->i_lock);
539+ new_cap = get_cap(caps_reservation);
540+ if (new_cap == NULL)
541+ return -ENOMEM;
542+ goto retry;
543+ }
544+
545+ cap->issued = 0;
546+ cap->implemented = 0;
547+ cap->mds = mds;
548+ cap->mds_wanted = 0;
549+
550+ cap->ci = ci;
551+ __insert_cap_node(ci, cap);
552+
553+ /* clear out old exporting info? (i.e. on cap import) */
554+ if (ci->i_cap_exporting_mds == mds) {
555+ ci->i_cap_exporting_issued = 0;
556+ ci->i_cap_exporting_mseq = 0;
557+ ci->i_cap_exporting_mds = -1;
558+ }
559+
560+ /* add to session cap list */
561+ cap->session = session;
562+ spin_lock(&session->s_cap_lock);
563+ list_add_tail(&cap->session_caps, &session->s_caps);
564+ session->s_nr_caps++;
565+ spin_unlock(&session->s_cap_lock);
566+ }
567+
568+ if (!ci->i_snap_realm) {
569+ /*
570+ * add this inode to the appropriate snap realm
571+ */
572+ struct ceph_snap_realm *realm = ceph_lookup_snap_realm(mdsc,
573+ realmino);
574+ if (realm) {
575+ ceph_get_snap_realm(mdsc, realm);
576+ spin_lock(&realm->inodes_with_caps_lock);
577+ ci->i_snap_realm = realm;
578+ list_add(&ci->i_snap_realm_item,
579+ &realm->inodes_with_caps);
580+ spin_unlock(&realm->inodes_with_caps_lock);
581+ } else {
582+ pr_err("ceph_add_cap: couldn't find snap realm %llx\n",
583+ realmino);
584+ }
585+ }
586+
587+ __check_cap_issue(ci, cap, issued);
588+
589+ /*
590+ * If we are issued caps we don't want, or the mds' wanted
591+ * value appears to be off, queue a check so we'll release
592+ * later and/or update the mds wanted value.
593+ */
594+ actual_wanted = __ceph_caps_wanted(ci);
595+ if ((wanted & ~actual_wanted) ||
596+ (issued & ~actual_wanted & CEPH_CAP_ANY_WR)) {
597+ dout(" issued %s, mds wanted %s, actual %s, queueing\n",
598+ ceph_cap_string(issued), ceph_cap_string(wanted),
599+ ceph_cap_string(actual_wanted));
600+ __cap_delay_requeue(mdsc, ci);
601+ }
602+
603+ if (flags & CEPH_CAP_FLAG_AUTH)
604+ ci->i_auth_cap = cap;
605+ else if (ci->i_auth_cap == cap)
606+ ci->i_auth_cap = NULL;
607+
608+ dout("add_cap inode %p (%llx.%llx) cap %p %s now %s seq %d mds%d\n",
609+ inode, ceph_vinop(inode), cap, ceph_cap_string(issued),
610+ ceph_cap_string(issued|cap->issued), seq, mds);
611+ cap->cap_id = cap_id;
612+ cap->issued = issued;
613+ cap->implemented |= issued;
614+ cap->mds_wanted |= wanted;
615+ cap->seq = seq;
616+ cap->issue_seq = seq;
617+ cap->mseq = mseq;
618+ cap->cap_gen = session->s_cap_gen;
619+
620+ if (fmode >= 0)
621+ __ceph_get_fmode(ci, fmode);
622+ spin_unlock(&inode->i_lock);
623+ wake_up(&ci->i_cap_wq);
624+ return 0;
625+}
626+
627+/*
628+ * Return true if cap has not timed out and belongs to the current
629+ * generation of the MDS session (i.e. has not gone 'stale' due to
630+ * us losing touch with the mds).
631+ */
632+static int __cap_is_valid(struct ceph_cap *cap)
633+{
634+ unsigned long ttl;
635+ u32 gen;
636+
637+ spin_lock(&cap->session->s_cap_lock);
638+ gen = cap->session->s_cap_gen;
639+ ttl = cap->session->s_cap_ttl;
640+ spin_unlock(&cap->session->s_cap_lock);
641+
642+ if (cap->cap_gen < gen || time_after_eq(jiffies, ttl)) {
643+ dout("__cap_is_valid %p cap %p issued %s "
644+ "but STALE (gen %u vs %u)\n", &cap->ci->vfs_inode,
645+ cap, ceph_cap_string(cap->issued), cap->cap_gen, gen);
646+ return 0;
647+ }
648+
649+ return 1;
650+}
651+
652+/*
653+ * Return set of valid cap bits issued to us. Note that caps time
654+ * out, and may be invalidated in bulk if the client session times out
655+ * and session->s_cap_gen is bumped.
656+ */
657+int __ceph_caps_issued(struct ceph_inode_info *ci, int *implemented)
658+{
659+ int have = ci->i_snap_caps | ci->i_cap_exporting_issued;
660+ struct ceph_cap *cap;
661+ struct rb_node *p;
662+
663+ if (implemented)
664+ *implemented = 0;
665+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
666+ cap = rb_entry(p, struct ceph_cap, ci_node);
667+ if (!__cap_is_valid(cap))
668+ continue;
669+ dout("__ceph_caps_issued %p cap %p issued %s\n",
670+ &ci->vfs_inode, cap, ceph_cap_string(cap->issued));
671+ have |= cap->issued;
672+ if (implemented)
673+ *implemented |= cap->implemented;
674+ }
675+ return have;
676+}
677+
678+/*
679+ * Get cap bits issued by caps other than @ocap
680+ */
681+int __ceph_caps_issued_other(struct ceph_inode_info *ci, struct ceph_cap *ocap)
682+{
683+ int have = ci->i_snap_caps;
684+ struct ceph_cap *cap;
685+ struct rb_node *p;
686+
687+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
688+ cap = rb_entry(p, struct ceph_cap, ci_node);
689+ if (cap == ocap)
690+ continue;
691+ if (!__cap_is_valid(cap))
692+ continue;
693+ have |= cap->issued;
694+ }
695+ return have;
696+}
697+
698+/*
699+ * Move a cap to the end of the LRU (oldest caps at list head, newest
700+ * at list tail).
701+ */
702+static void __touch_cap(struct ceph_cap *cap)
703+{
704+ struct ceph_mds_session *s = cap->session;
705+
706+ spin_lock(&s->s_cap_lock);
707+ if (s->s_cap_iterator == NULL) {
708+ dout("__touch_cap %p cap %p mds%d\n", &cap->ci->vfs_inode, cap,
709+ s->s_mds);
710+ list_move_tail(&cap->session_caps, &s->s_caps);
711+ } else {
712+ dout("__touch_cap %p cap %p mds%d NOP, iterating over caps\n",
713+ &cap->ci->vfs_inode, cap, s->s_mds);
714+ }
715+ spin_unlock(&s->s_cap_lock);
716+}
717+
718+/*
719+ * Check if we hold the given mask. If so, move the cap(s) to the
720+ * front of their respective LRUs. (This is the preferred way for
721+ * callers to check for caps they want.)
722+ */
723+int __ceph_caps_issued_mask(struct ceph_inode_info *ci, int mask, int touch)
724+{
725+ struct ceph_cap *cap;
726+ struct rb_node *p;
727+ int have = ci->i_snap_caps;
728+
729+ if ((have & mask) == mask) {
730+ dout("__ceph_caps_issued_mask %p snap issued %s"
731+ " (mask %s)\n", &ci->vfs_inode,
732+ ceph_cap_string(have),
733+ ceph_cap_string(mask));
734+ return 1;
735+ }
736+
737+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
738+ cap = rb_entry(p, struct ceph_cap, ci_node);
739+ if (!__cap_is_valid(cap))
740+ continue;
741+ if ((cap->issued & mask) == mask) {
742+ dout("__ceph_caps_issued_mask %p cap %p issued %s"
743+ " (mask %s)\n", &ci->vfs_inode, cap,
744+ ceph_cap_string(cap->issued),
745+ ceph_cap_string(mask));
746+ if (touch)
747+ __touch_cap(cap);
748+ return 1;
749+ }
750+
751+ /* does a combination of caps satisfy mask? */
752+ have |= cap->issued;
753+ if ((have & mask) == mask) {
754+ dout("__ceph_caps_issued_mask %p combo issued %s"
755+ " (mask %s)\n", &ci->vfs_inode,
756+ ceph_cap_string(cap->issued),
757+ ceph_cap_string(mask));
758+ if (touch) {
759+ struct rb_node *q;
760+
761+ /* touch this + preceeding caps */
762+ __touch_cap(cap);
763+ for (q = rb_first(&ci->i_caps); q != p;
764+ q = rb_next(q)) {
765+ cap = rb_entry(q, struct ceph_cap,
766+ ci_node);
767+ if (!__cap_is_valid(cap))
768+ continue;
769+ __touch_cap(cap);
770+ }
771+ }
772+ return 1;
773+ }
774+ }
775+
776+ return 0;
777+}
778+
779+/*
780+ * Return true if mask caps are currently being revoked by an MDS.
781+ */
782+int ceph_caps_revoking(struct ceph_inode_info *ci, int mask)
783+{
784+ struct inode *inode = &ci->vfs_inode;
785+ struct ceph_cap *cap;
786+ struct rb_node *p;
787+ int ret = 0;
788+
789+ spin_lock(&inode->i_lock);
790+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
791+ cap = rb_entry(p, struct ceph_cap, ci_node);
792+ if (__cap_is_valid(cap) &&
793+ (cap->implemented & ~cap->issued & mask)) {
794+ ret = 1;
795+ break;
796+ }
797+ }
798+ spin_unlock(&inode->i_lock);
799+ dout("ceph_caps_revoking %p %s = %d\n", inode,
800+ ceph_cap_string(mask), ret);
801+ return ret;
802+}
803+
804+int __ceph_caps_used(struct ceph_inode_info *ci)
805+{
806+ int used = 0;
807+ if (ci->i_pin_ref)
808+ used |= CEPH_CAP_PIN;
809+ if (ci->i_rd_ref)
810+ used |= CEPH_CAP_FILE_RD;
811+ if (ci->i_rdcache_ref || ci->i_rdcache_gen)
812+ used |= CEPH_CAP_FILE_CACHE;
813+ if (ci->i_wr_ref)
814+ used |= CEPH_CAP_FILE_WR;
815+ if (ci->i_wrbuffer_ref)
816+ used |= CEPH_CAP_FILE_BUFFER;
817+ return used;
818+}
819+
820+/*
821+ * wanted, by virtue of open file modes
822+ */
823+int __ceph_caps_file_wanted(struct ceph_inode_info *ci)
824+{
825+ int want = 0;
826+ int mode;
827+ for (mode = 0; mode < 4; mode++)
828+ if (ci->i_nr_by_mode[mode])
829+ want |= ceph_caps_for_mode(mode);
830+ return want;
831+}
832+
833+/*
834+ * Return caps we have registered with the MDS(s) as 'wanted'.
835+ */
836+int __ceph_caps_mds_wanted(struct ceph_inode_info *ci)
837+{
838+ struct ceph_cap *cap;
839+ struct rb_node *p;
840+ int mds_wanted = 0;
841+
842+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
843+ cap = rb_entry(p, struct ceph_cap, ci_node);
844+ if (!__cap_is_valid(cap))
845+ continue;
846+ mds_wanted |= cap->mds_wanted;
847+ }
848+ return mds_wanted;
849+}
850+
851+/*
852+ * called under i_lock
853+ */
854+static int __ceph_is_any_caps(struct ceph_inode_info *ci)
855+{
856+ return !RB_EMPTY_ROOT(&ci->i_caps) || ci->i_cap_exporting_mds >= 0;
857+}
858+
859+/*
860+ * caller should hold i_lock.
861+ * caller will not hold session s_mutex if called from destroy_inode.
862+ */
863+void __ceph_remove_cap(struct ceph_cap *cap)
864+{
865+ struct ceph_mds_session *session = cap->session;
866+ struct ceph_inode_info *ci = cap->ci;
867+ struct ceph_mds_client *mdsc = &ceph_client(ci->vfs_inode.i_sb)->mdsc;
868+
869+ dout("__ceph_remove_cap %p from %p\n", cap, &ci->vfs_inode);
870+
871+ /* remove from inode list */
872+ rb_erase(&cap->ci_node, &ci->i_caps);
873+ cap->ci = NULL;
874+ if (ci->i_auth_cap == cap)
875+ ci->i_auth_cap = NULL;
876+
877+ /* remove from session list */
878+ spin_lock(&session->s_cap_lock);
879+ if (session->s_cap_iterator == cap) {
880+ /* not yet, we are iterating over this very cap */
881+ dout("__ceph_remove_cap delaying %p removal from session %p\n",
882+ cap, cap->session);
883+ } else {
884+ list_del_init(&cap->session_caps);
885+ session->s_nr_caps--;
886+ cap->session = NULL;
887+ }
888+ spin_unlock(&session->s_cap_lock);
889+
890+ if (cap->session == NULL)
891+ ceph_put_cap(cap);
892+
893+ if (!__ceph_is_any_caps(ci) && ci->i_snap_realm) {
894+ struct ceph_snap_realm *realm = ci->i_snap_realm;
895+ spin_lock(&realm->inodes_with_caps_lock);
896+ list_del_init(&ci->i_snap_realm_item);
897+ ci->i_snap_realm_counter++;
898+ ci->i_snap_realm = NULL;
899+ spin_unlock(&realm->inodes_with_caps_lock);
900+ ceph_put_snap_realm(mdsc, realm);
901+ }
902+ if (!__ceph_is_any_real_caps(ci))
903+ __cap_delay_cancel(mdsc, ci);
904+}
905+
906+/*
907+ * Build and send a cap message to the given MDS.
908+ *
909+ * Caller should be holding s_mutex.
910+ */
911+static int send_cap_msg(struct ceph_mds_session *session,
912+ u64 ino, u64 cid, int op,
913+ int caps, int wanted, int dirty,
914+ u32 seq, u64 flush_tid, u32 issue_seq, u32 mseq,
915+ u64 size, u64 max_size,
916+ struct timespec *mtime, struct timespec *atime,
917+ u64 time_warp_seq,
918+ uid_t uid, gid_t gid, mode_t mode,
919+ u64 xattr_version,
920+ struct ceph_buffer *xattrs_buf,
921+ u64 follows)
922+{
923+ struct ceph_mds_caps *fc;
924+ struct ceph_msg *msg;
925+
926+ dout("send_cap_msg %s %llx %llx caps %s wanted %s dirty %s"
927+ " seq %u/%u mseq %u follows %lld size %llu/%llu"
928+ " xattr_ver %llu xattr_len %d\n", ceph_cap_op_name(op),
929+ cid, ino, ceph_cap_string(caps), ceph_cap_string(wanted),
930+ ceph_cap_string(dirty),
931+ seq, issue_seq, mseq, follows, size, max_size,
932+ xattr_version, xattrs_buf ? (int)xattrs_buf->vec.iov_len : 0);
933+
934+ msg = ceph_msg_new(CEPH_MSG_CLIENT_CAPS, sizeof(*fc), 0, 0, NULL);
935+ if (IS_ERR(msg))
936+ return PTR_ERR(msg);
937+
938+ msg->hdr.tid = cpu_to_le64(flush_tid);
939+
940+ fc = msg->front.iov_base;
941+ memset(fc, 0, sizeof(*fc));
942+
943+ fc->cap_id = cpu_to_le64(cid);
944+ fc->op = cpu_to_le32(op);
945+ fc->seq = cpu_to_le32(seq);
946+ fc->issue_seq = cpu_to_le32(issue_seq);
947+ fc->migrate_seq = cpu_to_le32(mseq);
948+ fc->caps = cpu_to_le32(caps);
949+ fc->wanted = cpu_to_le32(wanted);
950+ fc->dirty = cpu_to_le32(dirty);
951+ fc->ino = cpu_to_le64(ino);
952+ fc->snap_follows = cpu_to_le64(follows);
953+
954+ fc->size = cpu_to_le64(size);
955+ fc->max_size = cpu_to_le64(max_size);
956+ if (mtime)
957+ ceph_encode_timespec(&fc->mtime, mtime);
958+ if (atime)
959+ ceph_encode_timespec(&fc->atime, atime);
960+ fc->time_warp_seq = cpu_to_le32(time_warp_seq);
961+
962+ fc->uid = cpu_to_le32(uid);
963+ fc->gid = cpu_to_le32(gid);
964+ fc->mode = cpu_to_le32(mode);
965+
966+ fc->xattr_version = cpu_to_le64(xattr_version);
967+ if (xattrs_buf) {
968+ msg->middle = ceph_buffer_get(xattrs_buf);
969+ fc->xattr_len = cpu_to_le32(xattrs_buf->vec.iov_len);
970+ msg->hdr.middle_len = cpu_to_le32(xattrs_buf->vec.iov_len);
971+ }
972+
973+ ceph_con_send(&session->s_con, msg);
974+ return 0;
975+}
976+
977+/*
978+ * Queue cap releases when an inode is dropped from our cache. Since
979+ * inode is about to be destroyed, there is no need for i_lock.
980+ */
981+void ceph_queue_caps_release(struct inode *inode)
982+{
983+ struct ceph_inode_info *ci = ceph_inode(inode);
984+ struct rb_node *p;
985+
986+ p = rb_first(&ci->i_caps);
987+ while (p) {
988+ struct ceph_cap *cap = rb_entry(p, struct ceph_cap, ci_node);
989+ struct ceph_mds_session *session = cap->session;
990+ struct ceph_msg *msg;
991+ struct ceph_mds_cap_release *head;
992+ struct ceph_mds_cap_item *item;
993+
994+ spin_lock(&session->s_cap_lock);
995+ BUG_ON(!session->s_num_cap_releases);
996+ msg = list_first_entry(&session->s_cap_releases,
997+ struct ceph_msg, list_head);
998+
999+ dout(" adding %p release to mds%d msg %p (%d left)\n",
1000+ inode, session->s_mds, msg, session->s_num_cap_releases);
1001+
1002+ BUG_ON(msg->front.iov_len + sizeof(*item) > PAGE_CACHE_SIZE);
1003+ head = msg->front.iov_base;
1004+ head->num = cpu_to_le32(le32_to_cpu(head->num) + 1);
1005+ item = msg->front.iov_base + msg->front.iov_len;
1006+ item->ino = cpu_to_le64(ceph_ino(inode));
1007+ item->cap_id = cpu_to_le64(cap->cap_id);
1008+ item->migrate_seq = cpu_to_le32(cap->mseq);
1009+ item->seq = cpu_to_le32(cap->issue_seq);
1010+
1011+ session->s_num_cap_releases--;
1012+
1013+ msg->front.iov_len += sizeof(*item);
1014+ if (le32_to_cpu(head->num) == CEPH_CAPS_PER_RELEASE) {
1015+ dout(" release msg %p full\n", msg);
1016+ list_move_tail(&msg->list_head,
1017+ &session->s_cap_releases_done);
1018+ } else {
1019+ dout(" release msg %p at %d/%d (%d)\n", msg,
1020+ (int)le32_to_cpu(head->num),
1021+ (int)CEPH_CAPS_PER_RELEASE,
1022+ (int)msg->front.iov_len);
1023+ }
1024+ spin_unlock(&session->s_cap_lock);
1025+ p = rb_next(p);
1026+ __ceph_remove_cap(cap);
1027+ }
1028+}
1029+
1030+/*
1031+ * Send a cap msg on the given inode. Update our caps state, then
1032+ * drop i_lock and send the message.
1033+ *
1034+ * Make note of max_size reported/requested from mds, revoked caps
1035+ * that have now been implemented.
1036+ *
1037+ * Make half-hearted attempt ot to invalidate page cache if we are
1038+ * dropping RDCACHE. Note that this will leave behind locked pages
1039+ * that we'll then need to deal with elsewhere.
1040+ *
1041+ * Return non-zero if delayed release, or we experienced an error
1042+ * such that the caller should requeue + retry later.
1043+ *
1044+ * called with i_lock, then drops it.
1045+ * caller should hold snap_rwsem (read), s_mutex.
1046+ */
1047+static int __send_cap(struct ceph_mds_client *mdsc, struct ceph_cap *cap,
1048+ int op, int used, int want, int retain, int flushing,
1049+ unsigned *pflush_tid)
1050+ __releases(cap->ci->vfs_inode->i_lock)
1051+{
1052+ struct ceph_inode_info *ci = cap->ci;
1053+ struct inode *inode = &ci->vfs_inode;
1054+ u64 cap_id = cap->cap_id;
1055+ int held, revoking, dropping, keep;
1056+ u64 seq, issue_seq, mseq, time_warp_seq, follows;
1057+ u64 size, max_size;
1058+ struct timespec mtime, atime;
1059+ int wake = 0;
1060+ mode_t mode;
1061+ uid_t uid;
1062+ gid_t gid;
1063+ struct ceph_mds_session *session;
1064+ u64 xattr_version = 0;
1065+ int delayed = 0;
1066+ u64 flush_tid = 0;
1067+ int i;
1068+ int ret;
1069+
1070+ held = cap->issued | cap->implemented;
1071+ revoking = cap->implemented & ~cap->issued;
1072+ retain &= ~revoking;
1073+ dropping = cap->issued & ~retain;
1074+
1075+ dout("__send_cap %p cap %p session %p %s -> %s (revoking %s)\n",
1076+ inode, cap, cap->session,
1077+ ceph_cap_string(held), ceph_cap_string(held & retain),
1078+ ceph_cap_string(revoking));
1079+ BUG_ON((retain & CEPH_CAP_PIN) == 0);
1080+
1081+ session = cap->session;
1082+
1083+ /* don't release wanted unless we've waited a bit. */
1084+ if ((ci->i_ceph_flags & CEPH_I_NODELAY) == 0 &&
1085+ time_before(jiffies, ci->i_hold_caps_min)) {
1086+ dout(" delaying issued %s -> %s, wanted %s -> %s on send\n",
1087+ ceph_cap_string(cap->issued),
1088+ ceph_cap_string(cap->issued & retain),
1089+ ceph_cap_string(cap->mds_wanted),
1090+ ceph_cap_string(want));
1091+ want |= cap->mds_wanted;
1092+ retain |= cap->issued;
1093+ delayed = 1;
1094+ }
1095+ ci->i_ceph_flags &= ~(CEPH_I_NODELAY | CEPH_I_FLUSH);
1096+
1097+ cap->issued &= retain; /* drop bits we don't want */
1098+ if (cap->implemented & ~cap->issued) {
1099+ /*
1100+ * Wake up any waiters on wanted -> needed transition.
1101+ * This is due to the weird transition from buffered
1102+ * to sync IO... we need to flush dirty pages _before_
1103+ * allowing sync writes to avoid reordering.
1104+ */
1105+ wake = 1;
1106+ }
1107+ cap->implemented &= cap->issued | used;
1108+ cap->mds_wanted = want;
1109+
1110+ if (flushing) {
1111+ /*
1112+ * assign a tid for flush operations so we can avoid
1113+ * flush1 -> dirty1 -> flush2 -> flushack1 -> mark
1114+ * clean type races. track latest tid for every bit
1115+ * so we can handle flush AxFw, flush Fw, and have the
1116+ * first ack clean Ax.
1117+ */
1118+ flush_tid = ++ci->i_cap_flush_last_tid;
1119+ if (pflush_tid)
1120+ *pflush_tid = flush_tid;
1121+ dout(" cap_flush_tid %d\n", (int)flush_tid);
1122+ for (i = 0; i < CEPH_CAP_BITS; i++)
1123+ if (flushing & (1 << i))
1124+ ci->i_cap_flush_tid[i] = flush_tid;
1125+ }
1126+
1127+ keep = cap->implemented;
1128+ seq = cap->seq;
1129+ issue_seq = cap->issue_seq;
1130+ mseq = cap->mseq;
1131+ size = inode->i_size;
1132+ ci->i_reported_size = size;
1133+ max_size = ci->i_wanted_max_size;
1134+ ci->i_requested_max_size = max_size;
1135+ mtime = inode->i_mtime;
1136+ atime = inode->i_atime;
1137+ time_warp_seq = ci->i_time_warp_seq;
1138+ follows = ci->i_snap_realm->cached_context->seq;
1139+ uid = inode->i_uid;
1140+ gid = inode->i_gid;
1141+ mode = inode->i_mode;
1142+
1143+ if (dropping & CEPH_CAP_XATTR_EXCL) {
1144+ __ceph_build_xattrs_blob(ci);
1145+ xattr_version = ci->i_xattrs.version + 1;
1146+ }
1147+
1148+ spin_unlock(&inode->i_lock);
1149+
1150+ ret = send_cap_msg(session, ceph_vino(inode).ino, cap_id,
1151+ op, keep, want, flushing, seq, flush_tid, issue_seq, mseq,
1152+ size, max_size, &mtime, &atime, time_warp_seq,
1153+ uid, gid, mode,
1154+ xattr_version,
1155+ (flushing & CEPH_CAP_XATTR_EXCL) ? ci->i_xattrs.blob : NULL,
1156+ follows);
1157+ if (ret < 0) {
1158+ dout("error sending cap msg, must requeue %p\n", inode);
1159+ delayed = 1;
1160+ }
1161+
1162+ if (wake)
1163+ wake_up(&ci->i_cap_wq);
1164+
1165+ return delayed;
1166+}
1167+
1168+/*
1169+ * When a snapshot is taken, clients accumulate dirty metadata on
1170+ * inodes with capabilities in ceph_cap_snaps to describe the file
1171+ * state at the time the snapshot was taken. This must be flushed
1172+ * asynchronously back to the MDS once sync writes complete and dirty
1173+ * data is written out.
1174+ *
1175+ * Called under i_lock. Takes s_mutex as needed.
1176+ */
1177+void __ceph_flush_snaps(struct ceph_inode_info *ci,
1178+ struct ceph_mds_session **psession)
1179+{
1180+ struct inode *inode = &ci->vfs_inode;
1181+ int mds;
1182+ struct ceph_cap_snap *capsnap;
1183+ u32 mseq;
1184+ struct ceph_mds_client *mdsc = &ceph_inode_to_client(inode)->mdsc;
1185+ struct ceph_mds_session *session = NULL; /* if session != NULL, we hold
1186+ session->s_mutex */
1187+ u64 next_follows = 0; /* keep track of how far we've gotten through the
1188+ i_cap_snaps list, and skip these entries next time
1189+ around to avoid an infinite loop */
1190+
1191+ if (psession)
1192+ session = *psession;
1193+
1194+ dout("__flush_snaps %p\n", inode);
1195+retry:
1196+ list_for_each_entry(capsnap, &ci->i_cap_snaps, ci_item) {
1197+ /* avoid an infiniute loop after retry */
1198+ if (capsnap->follows < next_follows)
1199+ continue;
1200+ /*
1201+ * we need to wait for sync writes to complete and for dirty
1202+ * pages to be written out.
1203+ */
1204+ if (capsnap->dirty_pages || capsnap->writing)
1205+ continue;
1206+
1207+ /* pick mds, take s_mutex */
1208+ mds = __ceph_get_cap_mds(ci, &mseq);
1209+ if (session && session->s_mds != mds) {
1210+ dout("oops, wrong session %p mutex\n", session);
1211+ mutex_unlock(&session->s_mutex);
1212+ ceph_put_mds_session(session);
1213+ session = NULL;
1214+ }
1215+ if (!session) {
1216+ spin_unlock(&inode->i_lock);
1217+ mutex_lock(&mdsc->mutex);
1218+ session = __ceph_lookup_mds_session(mdsc, mds);
1219+ mutex_unlock(&mdsc->mutex);
1220+ if (session) {
1221+ dout("inverting session/ino locks on %p\n",
1222+ session);
1223+ mutex_lock(&session->s_mutex);
1224+ }
1225+ /*
1226+ * if session == NULL, we raced against a cap
1227+ * deletion. retry, and we'll get a better
1228+ * @mds value next time.
1229+ */
1230+ spin_lock(&inode->i_lock);
1231+ goto retry;
1232+ }
1233+
1234+ capsnap->flush_tid = ++ci->i_cap_flush_last_tid;
1235+ atomic_inc(&capsnap->nref);
1236+ if (!list_empty(&capsnap->flushing_item))
1237+ list_del_init(&capsnap->flushing_item);
1238+ list_add_tail(&capsnap->flushing_item,
1239+ &session->s_cap_snaps_flushing);
1240+ spin_unlock(&inode->i_lock);
1241+
1242+ dout("flush_snaps %p cap_snap %p follows %lld size %llu\n",
1243+ inode, capsnap, next_follows, capsnap->size);
1244+ send_cap_msg(session, ceph_vino(inode).ino, 0,
1245+ CEPH_CAP_OP_FLUSHSNAP, capsnap->issued, 0,
1246+ capsnap->dirty, 0, capsnap->flush_tid, 0, mseq,
1247+ capsnap->size, 0,
1248+ &capsnap->mtime, &capsnap->atime,
1249+ capsnap->time_warp_seq,
1250+ capsnap->uid, capsnap->gid, capsnap->mode,
1251+ 0, NULL,
1252+ capsnap->follows);
1253+
1254+ next_follows = capsnap->follows + 1;
1255+ ceph_put_cap_snap(capsnap);
1256+
1257+ spin_lock(&inode->i_lock);
1258+ goto retry;
1259+ }
1260+
1261+ /* we flushed them all; remove this inode from the queue */
1262+ spin_lock(&mdsc->snap_flush_lock);
1263+ list_del_init(&ci->i_snap_flush_item);
1264+ spin_unlock(&mdsc->snap_flush_lock);
1265+
1266+ if (psession)
1267+ *psession = session;
1268+ else if (session) {
1269+ mutex_unlock(&session->s_mutex);
1270+ ceph_put_mds_session(session);
1271+ }
1272+}
1273+
1274+static void ceph_flush_snaps(struct ceph_inode_info *ci)
1275+{
1276+ struct inode *inode = &ci->vfs_inode;
1277+
1278+ spin_lock(&inode->i_lock);
1279+ __ceph_flush_snaps(ci, NULL);
1280+ spin_unlock(&inode->i_lock);
1281+}
1282+
1283+/*
1284+ * Mark caps dirty. If inode is newly dirty, add to the global dirty
1285+ * list.
1286+ */
1287+void __ceph_mark_dirty_caps(struct ceph_inode_info *ci, int mask)
1288+{
1289+ struct ceph_mds_client *mdsc = &ceph_client(ci->vfs_inode.i_sb)->mdsc;
1290+ struct inode *inode = &ci->vfs_inode;
1291+ int was = ci->i_dirty_caps;
1292+ int dirty = 0;
1293+
1294+ dout("__mark_dirty_caps %p %s dirty %s -> %s\n", &ci->vfs_inode,
1295+ ceph_cap_string(mask), ceph_cap_string(was),
1296+ ceph_cap_string(was | mask));
1297+ ci->i_dirty_caps |= mask;
1298+ if (was == 0) {
1299+ dout(" inode %p now dirty\n", &ci->vfs_inode);
1300+ BUG_ON(!list_empty(&ci->i_dirty_item));
1301+ spin_lock(&mdsc->cap_dirty_lock);
1302+ list_add(&ci->i_dirty_item, &mdsc->cap_dirty);
1303+ spin_unlock(&mdsc->cap_dirty_lock);
1304+ if (ci->i_flushing_caps == 0) {
1305+ igrab(inode);
1306+ dirty |= I_DIRTY_SYNC;
1307+ }
1308+ }
1309+ BUG_ON(list_empty(&ci->i_dirty_item));
1310+ if (((was | ci->i_flushing_caps) & CEPH_CAP_FILE_BUFFER) &&
1311+ (mask & CEPH_CAP_FILE_BUFFER))
1312+ dirty |= I_DIRTY_DATASYNC;
1313+ if (dirty)
1314+ __mark_inode_dirty(inode, dirty);
1315+ __cap_delay_requeue(mdsc, ci);
1316+}
1317+
1318+/*
1319+ * Add dirty inode to the flushing list. Assigned a seq number so we
1320+ * can wait for caps to flush without starving.
1321+ *
1322+ * Called under i_lock.
1323+ */
1324+static int __mark_caps_flushing(struct inode *inode,
1325+ struct ceph_mds_session *session)
1326+{
1327+ struct ceph_mds_client *mdsc = &ceph_client(inode->i_sb)->mdsc;
1328+ struct ceph_inode_info *ci = ceph_inode(inode);
1329+ int flushing;
1330+
1331+ BUG_ON(ci->i_dirty_caps == 0);
1332+ BUG_ON(list_empty(&ci->i_dirty_item));
1333+
1334+ flushing = ci->i_dirty_caps;
1335+ dout("__mark_caps_flushing flushing %s, flushing_caps %s -> %s\n",
1336+ ceph_cap_string(flushing),
1337+ ceph_cap_string(ci->i_flushing_caps),
1338+ ceph_cap_string(ci->i_flushing_caps | flushing));
1339+ ci->i_flushing_caps |= flushing;
1340+ ci->i_dirty_caps = 0;
1341+ dout(" inode %p now !dirty\n", inode);
1342+
1343+ spin_lock(&mdsc->cap_dirty_lock);
1344+ list_del_init(&ci->i_dirty_item);
1345+
1346+ ci->i_cap_flush_seq = ++mdsc->cap_flush_seq;
1347+ if (list_empty(&ci->i_flushing_item)) {
1348+ list_add_tail(&ci->i_flushing_item, &session->s_cap_flushing);
1349+ mdsc->num_cap_flushing++;
1350+ dout(" inode %p now flushing seq %lld\n", inode,
1351+ ci->i_cap_flush_seq);
1352+ } else {
1353+ list_move_tail(&ci->i_flushing_item, &session->s_cap_flushing);
1354+ dout(" inode %p now flushing (more) seq %lld\n", inode,
1355+ ci->i_cap_flush_seq);
1356+ }
1357+ spin_unlock(&mdsc->cap_dirty_lock);
1358+
1359+ return flushing;
1360+}
1361+
1362+/*
1363+ * try to invalidate mapping pages without blocking.
1364+ */
1365+static int mapping_is_empty(struct address_space *mapping)
1366+{
1367+ struct page *page = find_get_page(mapping, 0);
1368+
1369+ if (!page)
1370+ return 1;
1371+
1372+ put_page(page);
1373+ return 0;
1374+}
1375+
1376+static int try_nonblocking_invalidate(struct inode *inode)
1377+{
1378+ struct ceph_inode_info *ci = ceph_inode(inode);
1379+ u32 invalidating_gen = ci->i_rdcache_gen;
1380+
1381+ spin_unlock(&inode->i_lock);
1382+ invalidate_mapping_pages(&inode->i_data, 0, -1);
1383+ spin_lock(&inode->i_lock);
1384+
1385+ if (mapping_is_empty(&inode->i_data) &&
1386+ invalidating_gen == ci->i_rdcache_gen) {
1387+ /* success. */
1388+ dout("try_nonblocking_invalidate %p success\n", inode);
1389+ ci->i_rdcache_gen = 0;
1390+ ci->i_rdcache_revoking = 0;
1391+ return 0;
1392+ }
1393+ dout("try_nonblocking_invalidate %p failed\n", inode);
1394+ return -1;
1395+}
1396+
1397+/*
1398+ * Swiss army knife function to examine currently used and wanted
1399+ * versus held caps. Release, flush, ack revoked caps to mds as
1400+ * appropriate.
1401+ *
1402+ * CHECK_CAPS_NODELAY - caller is delayed work and we should not delay
1403+ * cap release further.
1404+ * CHECK_CAPS_AUTHONLY - we should only check the auth cap
1405+ * CHECK_CAPS_FLUSH - we should flush any dirty caps immediately, without
1406+ * further delay.
1407+ */
1408+void ceph_check_caps(struct ceph_inode_info *ci, int flags,
1409+ struct ceph_mds_session *session)
1410+{
1411+ struct ceph_client *client = ceph_inode_to_client(&ci->vfs_inode);
1412+ struct ceph_mds_client *mdsc = &client->mdsc;
1413+ struct inode *inode = &ci->vfs_inode;
1414+ struct ceph_cap *cap;
1415+ int file_wanted, used;
1416+ int took_snap_rwsem = 0; /* true if mdsc->snap_rwsem held */
1417+ int drop_session_lock = session ? 0 : 1;
1418+ int issued, implemented, want, retain, revoking, flushing = 0;
1419+ int mds = -1; /* keep track of how far we've gone through i_caps list
1420+ to avoid an infinite loop on retry */
1421+ struct rb_node *p;
1422+ int tried_invalidate = 0;
1423+ int delayed = 0, sent = 0, force_requeue = 0, num;
1424+ int queue_invalidate = 0;
1425+ int is_delayed = flags & CHECK_CAPS_NODELAY;
1426+
1427+ /* if we are unmounting, flush any unused caps immediately. */
1428+ if (mdsc->stopping)
1429+ is_delayed = 1;
1430+
1431+ spin_lock(&inode->i_lock);
1432+
1433+ if (ci->i_ceph_flags & CEPH_I_FLUSH)
1434+ flags |= CHECK_CAPS_FLUSH;
1435+
1436+ /* flush snaps first time around only */
1437+ if (!list_empty(&ci->i_cap_snaps))
1438+ __ceph_flush_snaps(ci, &session);
1439+ goto retry_locked;
1440+retry:
1441+ spin_lock(&inode->i_lock);
1442+retry_locked:
1443+ file_wanted = __ceph_caps_file_wanted(ci);
1444+ used = __ceph_caps_used(ci);
1445+ want = file_wanted | used;
1446+ issued = __ceph_caps_issued(ci, &implemented);
1447+ revoking = implemented & ~issued;
1448+
1449+ retain = want | CEPH_CAP_PIN;
1450+ if (!mdsc->stopping && inode->i_nlink > 0) {
1451+ if (want) {
1452+ retain |= CEPH_CAP_ANY; /* be greedy */
1453+ } else {
1454+ retain |= CEPH_CAP_ANY_SHARED;
1455+ /*
1456+ * keep RD only if we didn't have the file open RW,
1457+ * because then the mds would revoke it anyway to
1458+ * journal max_size=0.
1459+ */
1460+ if (ci->i_max_size == 0)
1461+ retain |= CEPH_CAP_ANY_RD;
1462+ }
1463+ }
1464+
1465+ dout("check_caps %p file_want %s used %s dirty %s flushing %s"
1466+ " issued %s revoking %s retain %s %s%s%s\n", inode,
1467+ ceph_cap_string(file_wanted),
1468+ ceph_cap_string(used), ceph_cap_string(ci->i_dirty_caps),
1469+ ceph_cap_string(ci->i_flushing_caps),
1470+ ceph_cap_string(issued), ceph_cap_string(revoking),
1471+ ceph_cap_string(retain),
1472+ (flags & CHECK_CAPS_AUTHONLY) ? " AUTHONLY" : "",
1473+ (flags & CHECK_CAPS_NODELAY) ? " NODELAY" : "",
1474+ (flags & CHECK_CAPS_FLUSH) ? " FLUSH" : "");
1475+
1476+ /*
1477+ * If we no longer need to hold onto old our caps, and we may
1478+ * have cached pages, but don't want them, then try to invalidate.
1479+ * If we fail, it's because pages are locked.... try again later.
1480+ */
1481+ if ((!is_delayed || mdsc->stopping) &&
1482+ ci->i_wrbuffer_ref == 0 && /* no dirty pages... */
1483+ ci->i_rdcache_gen && /* may have cached pages */
1484+ (file_wanted == 0 || /* no open files */
1485+ (revoking & CEPH_CAP_FILE_CACHE)) && /* or revoking cache */
1486+ !tried_invalidate) {
1487+ dout("check_caps trying to invalidate on %p\n", inode);
1488+ if (try_nonblocking_invalidate(inode) < 0) {
1489+ if (revoking & CEPH_CAP_FILE_CACHE) {
1490+ dout("check_caps queuing invalidate\n");
1491+ queue_invalidate = 1;
1492+ ci->i_rdcache_revoking = ci->i_rdcache_gen;
1493+ } else {
1494+ dout("check_caps failed to invalidate pages\n");
1495+ /* we failed to invalidate pages. check these
1496+ caps again later. */
1497+ force_requeue = 1;
1498+ __cap_set_timeouts(mdsc, ci);
1499+ }
1500+ }
1501+ tried_invalidate = 1;
1502+ goto retry_locked;
1503+ }
1504+
1505+ num = 0;
1506+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
1507+ cap = rb_entry(p, struct ceph_cap, ci_node);
1508+ num++;
1509+
1510+ /* avoid looping forever */
1511+ if (mds >= cap->mds ||
1512+ ((flags & CHECK_CAPS_AUTHONLY) && cap != ci->i_auth_cap))
1513+ continue;
1514+
1515+ /* NOTE: no side-effects allowed, until we take s_mutex */
1516+
1517+ revoking = cap->implemented & ~cap->issued;
1518+ if (revoking)
1519+ dout(" mds%d revoking %s\n", cap->mds,
1520+ ceph_cap_string(revoking));
1521+
1522+ if (cap == ci->i_auth_cap &&
1523+ (cap->issued & CEPH_CAP_FILE_WR)) {
1524+ /* request larger max_size from MDS? */
1525+ if (ci->i_wanted_max_size > ci->i_max_size &&
1526+ ci->i_wanted_max_size > ci->i_requested_max_size) {
1527+ dout("requesting new max_size\n");
1528+ goto ack;
1529+ }
1530+
1531+ /* approaching file_max? */
1532+ if ((inode->i_size << 1) >= ci->i_max_size &&
1533+ (ci->i_reported_size << 1) < ci->i_max_size) {
1534+ dout("i_size approaching max_size\n");
1535+ goto ack;
1536+ }
1537+ }
1538+ /* flush anything dirty? */
1539+ if (cap == ci->i_auth_cap && (flags & CHECK_CAPS_FLUSH) &&
1540+ ci->i_dirty_caps) {
1541+ dout("flushing dirty caps\n");
1542+ goto ack;
1543+ }
1544+
1545+ /* completed revocation? going down and there are no caps? */
1546+ if (revoking && (revoking & used) == 0) {
1547+ dout("completed revocation of %s\n",
1548+ ceph_cap_string(cap->implemented & ~cap->issued));
1549+ goto ack;
1550+ }
1551+
1552+ /* want more caps from mds? */
1553+ if (want & ~(cap->mds_wanted | cap->issued))
1554+ goto ack;
1555+
1556+ /* things we might delay */
1557+ if ((cap->issued & ~retain) == 0 &&
1558+ cap->mds_wanted == want)
1559+ continue; /* nope, all good */
1560+
1561+ if (is_delayed)
1562+ goto ack;
1563+
1564+ /* delay? */
1565+ if ((ci->i_ceph_flags & CEPH_I_NODELAY) == 0 &&
1566+ time_before(jiffies, ci->i_hold_caps_max)) {
1567+ dout(" delaying issued %s -> %s, wanted %s -> %s\n",
1568+ ceph_cap_string(cap->issued),
1569+ ceph_cap_string(cap->issued & retain),
1570+ ceph_cap_string(cap->mds_wanted),
1571+ ceph_cap_string(want));
1572+ delayed++;
1573+ continue;
1574+ }
1575+
1576+ack:
1577+ if (ci->i_ceph_flags & CEPH_I_NOFLUSH) {
1578+ dout(" skipping %p I_NOFLUSH set\n", inode);
1579+ continue;
1580+ }
1581+
1582+ if (session && session != cap->session) {
1583+ dout("oops, wrong session %p mutex\n", session);
1584+ mutex_unlock(&session->s_mutex);
1585+ session = NULL;
1586+ }
1587+ if (!session) {
1588+ session = cap->session;
1589+ if (mutex_trylock(&session->s_mutex) == 0) {
1590+ dout("inverting session/ino locks on %p\n",
1591+ session);
1592+ spin_unlock(&inode->i_lock);
1593+ if (took_snap_rwsem) {
1594+ up_read(&mdsc->snap_rwsem);
1595+ took_snap_rwsem = 0;
1596+ }
1597+ mutex_lock(&session->s_mutex);
1598+ goto retry;
1599+ }
1600+ }
1601+ /* take snap_rwsem after session mutex */
1602+ if (!took_snap_rwsem) {
1603+ if (down_read_trylock(&mdsc->snap_rwsem) == 0) {
1604+ dout("inverting snap/in locks on %p\n",
1605+ inode);
1606+ spin_unlock(&inode->i_lock);
1607+ down_read(&mdsc->snap_rwsem);
1608+ took_snap_rwsem = 1;
1609+ goto retry;
1610+ }
1611+ took_snap_rwsem = 1;
1612+ }
1613+
1614+ if (cap == ci->i_auth_cap && ci->i_dirty_caps)
1615+ flushing = __mark_caps_flushing(inode, session);
1616+
1617+ mds = cap->mds; /* remember mds, so we don't repeat */
1618+ sent++;
1619+
1620+ /* __send_cap drops i_lock */
1621+ delayed += __send_cap(mdsc, cap, CEPH_CAP_OP_UPDATE, used, want,
1622+ retain, flushing, NULL);
1623+ goto retry; /* retake i_lock and restart our cap scan. */
1624+ }
1625+
1626+ /*
1627+ * Reschedule delayed caps release if we delayed anything,
1628+ * otherwise cancel.
1629+ */
1630+ if (delayed && is_delayed)
1631+ force_requeue = 1; /* __send_cap delayed release; requeue */
1632+ if (!delayed && !is_delayed)
1633+ __cap_delay_cancel(mdsc, ci);
1634+ else if (!is_delayed || force_requeue)
1635+ __cap_delay_requeue(mdsc, ci);
1636+
1637+ spin_unlock(&inode->i_lock);
1638+
1639+ if (queue_invalidate)
1640+ ceph_queue_invalidate(inode);
1641+
1642+ if (session && drop_session_lock)
1643+ mutex_unlock(&session->s_mutex);
1644+ if (took_snap_rwsem)
1645+ up_read(&mdsc->snap_rwsem);
1646+}
1647+
1648+/*
1649+ * Try to flush dirty caps back to the auth mds.
1650+ */
1651+static int try_flush_caps(struct inode *inode, struct ceph_mds_session *session,
1652+ unsigned *flush_tid)
1653+{
1654+ struct ceph_mds_client *mdsc = &ceph_client(inode->i_sb)->mdsc;
1655+ struct ceph_inode_info *ci = ceph_inode(inode);
1656+ int unlock_session = session ? 0 : 1;
1657+ int flushing = 0;
1658+
1659+retry:
1660+ spin_lock(&inode->i_lock);
1661+ if (ci->i_ceph_flags & CEPH_I_NOFLUSH) {
1662+ dout("try_flush_caps skipping %p I_NOFLUSH set\n", inode);
1663+ goto out;
1664+ }
1665+ if (ci->i_dirty_caps && ci->i_auth_cap) {
1666+ struct ceph_cap *cap = ci->i_auth_cap;
1667+ int used = __ceph_caps_used(ci);
1668+ int want = __ceph_caps_wanted(ci);
1669+ int delayed;
1670+
1671+ if (!session) {
1672+ spin_unlock(&inode->i_lock);
1673+ session = cap->session;
1674+ mutex_lock(&session->s_mutex);
1675+ goto retry;
1676+ }
1677+ BUG_ON(session != cap->session);
1678+ if (cap->session->s_state < CEPH_MDS_SESSION_OPEN)
1679+ goto out;
1680+
1681+ flushing = __mark_caps_flushing(inode, session);
1682+
1683+ /* __send_cap drops i_lock */
1684+ delayed = __send_cap(mdsc, cap, CEPH_CAP_OP_FLUSH, used, want,
1685+ cap->issued | cap->implemented, flushing,
1686+ flush_tid);
1687+ if (!delayed)
1688+ goto out_unlocked;
1689+
1690+ spin_lock(&inode->i_lock);
1691+ __cap_delay_requeue(mdsc, ci);
1692+ }
1693+out:
1694+ spin_unlock(&inode->i_lock);
1695+out_unlocked:
1696+ if (session && unlock_session)
1697+ mutex_unlock(&session->s_mutex);
1698+ return flushing;
1699+}
1700+
1701+/*
1702+ * Return true if we've flushed caps through the given flush_tid.
1703+ */
1704+static int caps_are_flushed(struct inode *inode, unsigned tid)
1705+{
1706+ struct ceph_inode_info *ci = ceph_inode(inode);
1707+ int dirty, i, ret = 1;
1708+
1709+ spin_lock(&inode->i_lock);
1710+ dirty = __ceph_caps_dirty(ci);
1711+ for (i = 0; i < CEPH_CAP_BITS; i++)
1712+ if ((ci->i_flushing_caps & (1 << i)) &&
1713+ ci->i_cap_flush_tid[i] <= tid) {
1714+ /* still flushing this bit */
1715+ ret = 0;
1716+ break;
1717+ }
1718+ spin_unlock(&inode->i_lock);
1719+ return ret;
1720+}
1721+
1722+/*
1723+ * Wait on any unsafe replies for the given inode. First wait on the
1724+ * newest request, and make that the upper bound. Then, if there are
1725+ * more requests, keep waiting on the oldest as long as it is still older
1726+ * than the original request.
1727+ */
1728+static void sync_write_wait(struct inode *inode)
1729+{
1730+ struct ceph_inode_info *ci = ceph_inode(inode);
1731+ struct list_head *head = &ci->i_unsafe_writes;
1732+ struct ceph_osd_request *req;
1733+ u64 last_tid;
1734+
1735+ spin_lock(&ci->i_unsafe_lock);
1736+ if (list_empty(head))
1737+ goto out;
1738+
1739+ /* set upper bound as _last_ entry in chain */
1740+ req = list_entry(head->prev, struct ceph_osd_request,
1741+ r_unsafe_item);
1742+ last_tid = req->r_tid;
1743+
1744+ do {
1745+ ceph_osdc_get_request(req);
1746+ spin_unlock(&ci->i_unsafe_lock);
1747+ dout("sync_write_wait on tid %llu (until %llu)\n",
1748+ req->r_tid, last_tid);
1749+ wait_for_completion(&req->r_safe_completion);
1750+ spin_lock(&ci->i_unsafe_lock);
1751+ ceph_osdc_put_request(req);
1752+
1753+ /*
1754+ * from here on look at first entry in chain, since we
1755+ * only want to wait for anything older than last_tid
1756+ */
1757+ if (list_empty(head))
1758+ break;
1759+ req = list_entry(head->next, struct ceph_osd_request,
1760+ r_unsafe_item);
1761+ } while (req->r_tid < last_tid);
1762+out:
1763+ spin_unlock(&ci->i_unsafe_lock);
1764+}
1765+
1766+int ceph_fsync(struct file *file, struct dentry *dentry, int datasync)
1767+{
1768+ struct inode *inode = dentry->d_inode;
1769+ struct ceph_inode_info *ci = ceph_inode(inode);
1770+ unsigned flush_tid;
1771+ int ret;
1772+ int dirty;
1773+
1774+ dout("fsync %p%s\n", inode, datasync ? " datasync" : "");
1775+ sync_write_wait(inode);
1776+
1777+ ret = filemap_write_and_wait(inode->i_mapping);
1778+ if (ret < 0)
1779+ return ret;
1780+
1781+ dirty = try_flush_caps(inode, NULL, &flush_tid);
1782+ dout("fsync dirty caps are %s\n", ceph_cap_string(dirty));
1783+
1784+ /*
1785+ * only wait on non-file metadata writeback (the mds
1786+ * can recover size and mtime, so we don't need to
1787+ * wait for that)
1788+ */
1789+ if (!datasync && (dirty & ~CEPH_CAP_ANY_FILE_WR)) {
1790+ dout("fsync waiting for flush_tid %u\n", flush_tid);
1791+ ret = wait_event_interruptible(ci->i_cap_wq,
1792+ caps_are_flushed(inode, flush_tid));
1793+ }
1794+
1795+ dout("fsync %p%s done\n", inode, datasync ? " datasync" : "");
1796+ return ret;
1797+}
1798+
1799+/*
1800+ * Flush any dirty caps back to the mds. If we aren't asked to wait,
1801+ * queue inode for flush but don't do so immediately, because we can
1802+ * get by with fewer MDS messages if we wait for data writeback to
1803+ * complete first.
1804+ */
1805+int ceph_write_inode(struct inode *inode, struct writeback_control *wbc)
1806+{
1807+ struct ceph_inode_info *ci = ceph_inode(inode);
1808+ unsigned flush_tid;
1809+ int err = 0;
1810+ int dirty;
1811+ int wait = wbc->sync_mode == WB_SYNC_ALL;
1812+
1813+ dout("write_inode %p wait=%d\n", inode, wait);
1814+ if (wait) {
1815+ dirty = try_flush_caps(inode, NULL, &flush_tid);
1816+ if (dirty)
1817+ err = wait_event_interruptible(ci->i_cap_wq,
1818+ caps_are_flushed(inode, flush_tid));
1819+ } else {
1820+ struct ceph_mds_client *mdsc = &ceph_client(inode->i_sb)->mdsc;
1821+
1822+ spin_lock(&inode->i_lock);
1823+ if (__ceph_caps_dirty(ci))
1824+ __cap_delay_requeue_front(mdsc, ci);
1825+ spin_unlock(&inode->i_lock);
1826+ }
1827+ return err;
1828+}
1829+
1830+/*
1831+ * After a recovering MDS goes active, we need to resend any caps
1832+ * we were flushing.
1833+ *
1834+ * Caller holds session->s_mutex.
1835+ */
1836+static void kick_flushing_capsnaps(struct ceph_mds_client *mdsc,
1837+ struct ceph_mds_session *session)
1838+{
1839+ struct ceph_cap_snap *capsnap;
1840+
1841+ dout("kick_flushing_capsnaps mds%d\n", session->s_mds);
1842+ list_for_each_entry(capsnap, &session->s_cap_snaps_flushing,
1843+ flushing_item) {
1844+ struct ceph_inode_info *ci = capsnap->ci;
1845+ struct inode *inode = &ci->vfs_inode;
1846+ struct ceph_cap *cap;
1847+
1848+ spin_lock(&inode->i_lock);
1849+ cap = ci->i_auth_cap;
1850+ if (cap && cap->session == session) {
1851+ dout("kick_flushing_caps %p cap %p capsnap %p\n", inode,
1852+ cap, capsnap);
1853+ __ceph_flush_snaps(ci, &session);
1854+ } else {
1855+ pr_err("%p auth cap %p not mds%d ???\n", inode,
1856+ cap, session->s_mds);
1857+ spin_unlock(&inode->i_lock);
1858+ }
1859+ }
1860+}
1861+
1862+void ceph_kick_flushing_caps(struct ceph_mds_client *mdsc,
1863+ struct ceph_mds_session *session)
1864+{
1865+ struct ceph_inode_info *ci;
1866+
1867+ kick_flushing_capsnaps(mdsc, session);
1868+
1869+ dout("kick_flushing_caps mds%d\n", session->s_mds);
1870+ list_for_each_entry(ci, &session->s_cap_flushing, i_flushing_item) {
1871+ struct inode *inode = &ci->vfs_inode;
1872+ struct ceph_cap *cap;
1873+ int delayed = 0;
1874+
1875+ spin_lock(&inode->i_lock);
1876+ cap = ci->i_auth_cap;
1877+ if (cap && cap->session == session) {
1878+ dout("kick_flushing_caps %p cap %p %s\n", inode,
1879+ cap, ceph_cap_string(ci->i_flushing_caps));
1880+ delayed = __send_cap(mdsc, cap, CEPH_CAP_OP_FLUSH,
1881+ __ceph_caps_used(ci),
1882+ __ceph_caps_wanted(ci),
1883+ cap->issued | cap->implemented,
1884+ ci->i_flushing_caps, NULL);
1885+ if (delayed) {
1886+ spin_lock(&inode->i_lock);
1887+ __cap_delay_requeue(mdsc, ci);
1888+ spin_unlock(&inode->i_lock);
1889+ }
1890+ } else {
1891+ pr_err("%p auth cap %p not mds%d ???\n", inode,
1892+ cap, session->s_mds);
1893+ spin_unlock(&inode->i_lock);
1894+ }
1895+ }
1896+}
1897+
1898+
1899+/*
1900+ * Take references to capabilities we hold, so that we don't release
1901+ * them to the MDS prematurely.
1902+ *
1903+ * Protected by i_lock.
1904+ */
1905+static void __take_cap_refs(struct ceph_inode_info *ci, int got)
1906+{
1907+ if (got & CEPH_CAP_PIN)
1908+ ci->i_pin_ref++;
1909+ if (got & CEPH_CAP_FILE_RD)
1910+ ci->i_rd_ref++;
1911+ if (got & CEPH_CAP_FILE_CACHE)
1912+ ci->i_rdcache_ref++;
1913+ if (got & CEPH_CAP_FILE_WR)
1914+ ci->i_wr_ref++;
1915+ if (got & CEPH_CAP_FILE_BUFFER) {
1916+ if (ci->i_wrbuffer_ref == 0)
1917+ igrab(&ci->vfs_inode);
1918+ ci->i_wrbuffer_ref++;
1919+ dout("__take_cap_refs %p wrbuffer %d -> %d (?)\n",
1920+ &ci->vfs_inode, ci->i_wrbuffer_ref-1, ci->i_wrbuffer_ref);
1921+ }
1922+}
1923+
1924+/*
1925+ * Try to grab cap references. Specify those refs we @want, and the
1926+ * minimal set we @need. Also include the larger offset we are writing
1927+ * to (when applicable), and check against max_size here as well.
1928+ * Note that caller is responsible for ensuring max_size increases are
1929+ * requested from the MDS.
1930+ */
1931+static int try_get_cap_refs(struct ceph_inode_info *ci, int need, int want,
1932+ int *got, loff_t endoff, int *check_max, int *err)
1933+{
1934+ struct inode *inode = &ci->vfs_inode;
1935+ int ret = 0;
1936+ int have, implemented;
1937+ int file_wanted;
1938+
1939+ dout("get_cap_refs %p need %s want %s\n", inode,
1940+ ceph_cap_string(need), ceph_cap_string(want));
1941+ spin_lock(&inode->i_lock);
1942+
1943+ /* make sure file is actually open */
1944+ file_wanted = __ceph_caps_file_wanted(ci);
1945+ if ((file_wanted & need) == 0) {
1946+ dout("try_get_cap_refs need %s file_wanted %s, EBADF\n",
1947+ ceph_cap_string(need), ceph_cap_string(file_wanted));
1948+ *err = -EBADF;
1949+ ret = 1;
1950+ goto out;
1951+ }
1952+
1953+ if (need & CEPH_CAP_FILE_WR) {
1954+ if (endoff >= 0 && endoff > (loff_t)ci->i_max_size) {
1955+ dout("get_cap_refs %p endoff %llu > maxsize %llu\n",
1956+ inode, endoff, ci->i_max_size);
1957+ if (endoff > ci->i_wanted_max_size) {
1958+ *check_max = 1;
1959+ ret = 1;
1960+ }
1961+ goto out;
1962+ }
1963+ /*
1964+ * If a sync write is in progress, we must wait, so that we
1965+ * can get a final snapshot value for size+mtime.
1966+ */
1967+ if (__ceph_have_pending_cap_snap(ci)) {
1968+ dout("get_cap_refs %p cap_snap_pending\n", inode);
1969+ goto out;
1970+ }
1971+ }
1972+ have = __ceph_caps_issued(ci, &implemented);
1973+
1974+ /*
1975+ * disallow writes while a truncate is pending
1976+ */
1977+ if (ci->i_truncate_pending)
1978+ have &= ~CEPH_CAP_FILE_WR;
1979+
1980+ if ((have & need) == need) {
1981+ /*
1982+ * Look at (implemented & ~have & not) so that we keep waiting
1983+ * on transition from wanted -> needed caps. This is needed
1984+ * for WRBUFFER|WR -> WR to avoid a new WR sync write from
1985+ * going before a prior buffered writeback happens.
1986+ */
1987+ int not = want & ~(have & need);
1988+ int revoking = implemented & ~have;
1989+ dout("get_cap_refs %p have %s but not %s (revoking %s)\n",
1990+ inode, ceph_cap_string(have), ceph_cap_string(not),
1991+ ceph_cap_string(revoking));
1992+ if ((revoking & not) == 0) {
1993+ *got = need | (have & want);
1994+ __take_cap_refs(ci, *got);
1995+ ret = 1;
1996+ }
1997+ } else {
1998+ dout("get_cap_refs %p have %s needed %s\n", inode,
1999+ ceph_cap_string(have), ceph_cap_string(need));
2000+ }
2001+out:
2002+ spin_unlock(&inode->i_lock);
2003+ dout("get_cap_refs %p ret %d got %s\n", inode,
2004+ ret, ceph_cap_string(*got));
2005+ return ret;
2006+}
2007+
2008+/*
2009+ * Check the offset we are writing up to against our current
2010+ * max_size. If necessary, tell the MDS we want to write to
2011+ * a larger offset.
2012+ */
2013+static void check_max_size(struct inode *inode, loff_t endoff)
2014+{
2015+ struct ceph_inode_info *ci = ceph_inode(inode);
2016+ int check = 0;
2017+
2018+ /* do we need to explicitly request a larger max_size? */
2019+ spin_lock(&inode->i_lock);
2020+ if ((endoff >= ci->i_max_size ||
2021+ endoff > (inode->i_size << 1)) &&
2022+ endoff > ci->i_wanted_max_size) {
2023+ dout("write %p at large endoff %llu, req max_size\n",
2024+ inode, endoff);
2025+ ci->i_wanted_max_size = endoff;
2026+ check = 1;
2027+ }
2028+ spin_unlock(&inode->i_lock);
2029+ if (check)
2030+ ceph_check_caps(ci, CHECK_CAPS_AUTHONLY, NULL);
2031+}
2032+
2033+/*
2034+ * Wait for caps, and take cap references. If we can't get a WR cap
2035+ * due to a small max_size, make sure we check_max_size (and possibly
2036+ * ask the mds) so we don't get hung up indefinitely.
2037+ */
2038+int ceph_get_caps(struct ceph_inode_info *ci, int need, int want, int *got,
2039+ loff_t endoff)
2040+{
2041+ int check_max, ret, err;
2042+
2043+retry:
2044+ if (endoff > 0)
2045+ check_max_size(&ci->vfs_inode, endoff);
2046+ check_max = 0;
2047+ err = 0;
2048+ ret = wait_event_interruptible(ci->i_cap_wq,
2049+ try_get_cap_refs(ci, need, want,
2050+ got, endoff,
2051+ &check_max, &err));
2052+ if (err)
2053+ ret = err;
2054+ if (check_max)
2055+ goto retry;
2056+ return ret;
2057+}
2058+
2059+/*
2060+ * Take cap refs. Caller must already know we hold at least one ref
2061+ * on the caps in question or we don't know this is safe.
2062+ */
2063+void ceph_get_cap_refs(struct ceph_inode_info *ci, int caps)
2064+{
2065+ spin_lock(&ci->vfs_inode.i_lock);
2066+ __take_cap_refs(ci, caps);
2067+ spin_unlock(&ci->vfs_inode.i_lock);
2068+}
2069+
2070+/*
2071+ * Release cap refs.
2072+ *
2073+ * If we released the last ref on any given cap, call ceph_check_caps
2074+ * to release (or schedule a release).
2075+ *
2076+ * If we are releasing a WR cap (from a sync write), finalize any affected
2077+ * cap_snap, and wake up any waiters.
2078+ */
2079+void ceph_put_cap_refs(struct ceph_inode_info *ci, int had)
2080+{
2081+ struct inode *inode = &ci->vfs_inode;
2082+ int last = 0, put = 0, flushsnaps = 0, wake = 0;
2083+ struct ceph_cap_snap *capsnap;
2084+
2085+ spin_lock(&inode->i_lock);
2086+ if (had & CEPH_CAP_PIN)
2087+ --ci->i_pin_ref;
2088+ if (had & CEPH_CAP_FILE_RD)
2089+ if (--ci->i_rd_ref == 0)
2090+ last++;
2091+ if (had & CEPH_CAP_FILE_CACHE)
2092+ if (--ci->i_rdcache_ref == 0)
2093+ last++;
2094+ if (had & CEPH_CAP_FILE_BUFFER) {
2095+ if (--ci->i_wrbuffer_ref == 0) {
2096+ last++;
2097+ put++;
2098+ }
2099+ dout("put_cap_refs %p wrbuffer %d -> %d (?)\n",
2100+ inode, ci->i_wrbuffer_ref+1, ci->i_wrbuffer_ref);
2101+ }
2102+ if (had & CEPH_CAP_FILE_WR)
2103+ if (--ci->i_wr_ref == 0) {
2104+ last++;
2105+ if (!list_empty(&ci->i_cap_snaps)) {
2106+ capsnap = list_first_entry(&ci->i_cap_snaps,
2107+ struct ceph_cap_snap,
2108+ ci_item);
2109+ if (capsnap->writing) {
2110+ capsnap->writing = 0;
2111+ flushsnaps =
2112+ __ceph_finish_cap_snap(ci,
2113+ capsnap);
2114+ wake = 1;
2115+ }
2116+ }
2117+ }
2118+ spin_unlock(&inode->i_lock);
2119+
2120+ dout("put_cap_refs %p had %s %s\n", inode, ceph_cap_string(had),
2121+ last ? "last" : "");
2122+
2123+ if (last && !flushsnaps)
2124+ ceph_check_caps(ci, 0, NULL);
2125+ else if (flushsnaps)
2126+ ceph_flush_snaps(ci);
2127+ if (wake)
2128+ wake_up(&ci->i_cap_wq);
2129+ if (put)
2130+ iput(inode);
2131+}
2132+
2133+/*
2134+ * Release @nr WRBUFFER refs on dirty pages for the given @snapc snap
2135+ * context. Adjust per-snap dirty page accounting as appropriate.
2136+ * Once all dirty data for a cap_snap is flushed, flush snapped file
2137+ * metadata back to the MDS. If we dropped the last ref, call
2138+ * ceph_check_caps.
2139+ */
2140+void ceph_put_wrbuffer_cap_refs(struct ceph_inode_info *ci, int nr,
2141+ struct ceph_snap_context *snapc)
2142+{
2143+ struct inode *inode = &ci->vfs_inode;
2144+ int last = 0;
2145+ int last_snap = 0;
2146+ int found = 0;
2147+ struct ceph_cap_snap *capsnap = NULL;
2148+
2149+ spin_lock(&inode->i_lock);
2150+ ci->i_wrbuffer_ref -= nr;
2151+ last = !ci->i_wrbuffer_ref;
2152+
2153+ if (ci->i_head_snapc == snapc) {
2154+ ci->i_wrbuffer_ref_head -= nr;
2155+ if (!ci->i_wrbuffer_ref_head) {
2156+ ceph_put_snap_context(ci->i_head_snapc);
2157+ ci->i_head_snapc = NULL;
2158+ }
2159+ dout("put_wrbuffer_cap_refs on %p head %d/%d -> %d/%d %s\n",
2160+ inode,
2161+ ci->i_wrbuffer_ref+nr, ci->i_wrbuffer_ref_head+nr,
2162+ ci->i_wrbuffer_ref, ci->i_wrbuffer_ref_head,
2163+ last ? " LAST" : "");
2164+ } else {
2165+ list_for_each_entry(capsnap, &ci->i_cap_snaps, ci_item) {
2166+ if (capsnap->context == snapc) {
2167+ found = 1;
2168+ capsnap->dirty_pages -= nr;
2169+ last_snap = !capsnap->dirty_pages;
2170+ break;
2171+ }
2172+ }
2173+ BUG_ON(!found);
2174+ dout("put_wrbuffer_cap_refs on %p cap_snap %p "
2175+ " snap %lld %d/%d -> %d/%d %s%s\n",
2176+ inode, capsnap, capsnap->context->seq,
2177+ ci->i_wrbuffer_ref+nr, capsnap->dirty_pages + nr,
2178+ ci->i_wrbuffer_ref, capsnap->dirty_pages,
2179+ last ? " (wrbuffer last)" : "",
2180+ last_snap ? " (capsnap last)" : "");
2181+ }
2182+
2183+ spin_unlock(&inode->i_lock);
2184+
2185+ if (last) {
2186+ ceph_check_caps(ci, CHECK_CAPS_AUTHONLY, NULL);
2187+ iput(inode);
2188+ } else if (last_snap) {
2189+ ceph_flush_snaps(ci);
2190+ wake_up(&ci->i_cap_wq);
2191+ }
2192+}
2193+
2194+/*
2195+ * Handle a cap GRANT message from the MDS. (Note that a GRANT may
2196+ * actually be a revocation if it specifies a smaller cap set.)
2197+ *
2198+ * caller holds s_mutex.
2199+ * return value:
2200+ * 0 - ok
2201+ * 1 - check_caps on auth cap only (writeback)
2202+ * 2 - check_caps (ack revoke)
2203+ */
2204+static int handle_cap_grant(struct inode *inode, struct ceph_mds_caps *grant,
2205+ struct ceph_mds_session *session,
2206+ struct ceph_cap *cap,
2207+ struct ceph_buffer *xattr_buf)
2208+ __releases(inode->i_lock)
2209+
2210+{
2211+ struct ceph_inode_info *ci = ceph_inode(inode);
2212+ int mds = session->s_mds;
2213+ int seq = le32_to_cpu(grant->seq);
2214+ int newcaps = le32_to_cpu(grant->caps);
2215+ int issued, implemented, used, wanted, dirty;
2216+ u64 size = le64_to_cpu(grant->size);
2217+ u64 max_size = le64_to_cpu(grant->max_size);
2218+ struct timespec mtime, atime, ctime;
2219+ int reply = 0;
2220+ int wake = 0;
2221+ int writeback = 0;
2222+ int revoked_rdcache = 0;
2223+ int queue_invalidate = 0;
2224+
2225+ dout("handle_cap_grant inode %p cap %p mds%d seq %d %s\n",
2226+ inode, cap, mds, seq, ceph_cap_string(newcaps));
2227+ dout(" size %llu max_size %llu, i_size %llu\n", size, max_size,
2228+ inode->i_size);
2229+
2230+ /*
2231+ * If CACHE is being revoked, and we have no dirty buffers,
2232+ * try to invalidate (once). (If there are dirty buffers, we
2233+ * will invalidate _after_ writeback.)
2234+ */
2235+ if (((cap->issued & ~newcaps) & CEPH_CAP_FILE_CACHE) &&
2236+ !ci->i_wrbuffer_ref) {
2237+ if (try_nonblocking_invalidate(inode) == 0) {
2238+ revoked_rdcache = 1;
2239+ } else {
2240+ /* there were locked pages.. invalidate later
2241+ in a separate thread. */
2242+ if (ci->i_rdcache_revoking != ci->i_rdcache_gen) {
2243+ queue_invalidate = 1;
2244+ ci->i_rdcache_revoking = ci->i_rdcache_gen;
2245+ }
2246+ }
2247+ }
2248+
2249+ /* side effects now are allowed */
2250+
2251+ issued = __ceph_caps_issued(ci, &implemented);
2252+ issued |= implemented | __ceph_caps_dirty(ci);
2253+
2254+ cap->cap_gen = session->s_cap_gen;
2255+
2256+ __check_cap_issue(ci, cap, newcaps);
2257+
2258+ if ((issued & CEPH_CAP_AUTH_EXCL) == 0) {
2259+ inode->i_mode = le32_to_cpu(grant->mode);
2260+ inode->i_uid = le32_to_cpu(grant->uid);
2261+ inode->i_gid = le32_to_cpu(grant->gid);
2262+ dout("%p mode 0%o uid.gid %d.%d\n", inode, inode->i_mode,
2263+ inode->i_uid, inode->i_gid);
2264+ }
2265+
2266+ if ((issued & CEPH_CAP_LINK_EXCL) == 0)
2267+ inode->i_nlink = le32_to_cpu(grant->nlink);
2268+
2269+ if ((issued & CEPH_CAP_XATTR_EXCL) == 0 && grant->xattr_len) {
2270+ int len = le32_to_cpu(grant->xattr_len);
2271+ u64 version = le64_to_cpu(grant->xattr_version);
2272+
2273+ if (version > ci->i_xattrs.version) {
2274+ dout(" got new xattrs v%llu on %p len %d\n",
2275+ version, inode, len);
2276+ if (ci->i_xattrs.blob)
2277+ ceph_buffer_put(ci->i_xattrs.blob);
2278+ ci->i_xattrs.blob = ceph_buffer_get(xattr_buf);
2279+ ci->i_xattrs.version = version;
2280+ }
2281+ }
2282+
2283+ /* size/ctime/mtime/atime? */
2284+ ceph_fill_file_size(inode, issued,
2285+ le32_to_cpu(grant->truncate_seq),
2286+ le64_to_cpu(grant->truncate_size), size);
2287+ ceph_decode_timespec(&mtime, &grant->mtime);
2288+ ceph_decode_timespec(&atime, &grant->atime);
2289+ ceph_decode_timespec(&ctime, &grant->ctime);
2290+ ceph_fill_file_time(inode, issued,
2291+ le32_to_cpu(grant->time_warp_seq), &ctime, &mtime,
2292+ &atime);
2293+
2294+ /* max size increase? */
2295+ if (max_size != ci->i_max_size) {
2296+ dout("max_size %lld -> %llu\n", ci->i_max_size, max_size);
2297+ ci->i_max_size = max_size;
2298+ if (max_size >= ci->i_wanted_max_size) {
2299+ ci->i_wanted_max_size = 0; /* reset */
2300+ ci->i_requested_max_size = 0;
2301+ }
2302+ wake = 1;
2303+ }
2304+
2305+ /* check cap bits */
2306+ wanted = __ceph_caps_wanted(ci);
2307+ used = __ceph_caps_used(ci);
2308+ dirty = __ceph_caps_dirty(ci);
2309+ dout(" my wanted = %s, used = %s, dirty %s\n",
2310+ ceph_cap_string(wanted),
2311+ ceph_cap_string(used),
2312+ ceph_cap_string(dirty));
2313+ if (wanted != le32_to_cpu(grant->wanted)) {
2314+ dout("mds wanted %s -> %s\n",
2315+ ceph_cap_string(le32_to_cpu(grant->wanted)),
2316+ ceph_cap_string(wanted));
2317+ grant->wanted = cpu_to_le32(wanted);
2318+ }
2319+
2320+ cap->seq = seq;
2321+
2322+ /* file layout may have changed */
2323+ ci->i_layout = grant->layout;
2324+
2325+ /* revocation, grant, or no-op? */
2326+ if (cap->issued & ~newcaps) {
2327+ dout("revocation: %s -> %s\n", ceph_cap_string(cap->issued),
2328+ ceph_cap_string(newcaps));
2329+ if ((used & ~newcaps) & CEPH_CAP_FILE_BUFFER)
2330+ writeback = 1; /* will delay ack */
2331+ else if (dirty & ~newcaps)
2332+ reply = 1; /* initiate writeback in check_caps */
2333+ else if (((used & ~newcaps) & CEPH_CAP_FILE_CACHE) == 0 ||
2334+ revoked_rdcache)
2335+ reply = 2; /* send revoke ack in check_caps */
2336+ cap->issued = newcaps;
2337+ } else if (cap->issued == newcaps) {
2338+ dout("caps unchanged: %s -> %s\n",
2339+ ceph_cap_string(cap->issued), ceph_cap_string(newcaps));
2340+ } else {
2341+ dout("grant: %s -> %s\n", ceph_cap_string(cap->issued),
2342+ ceph_cap_string(newcaps));
2343+ cap->issued = newcaps;
2344+ cap->implemented |= newcaps; /* add bits only, to
2345+ * avoid stepping on a
2346+ * pending revocation */
2347+ wake = 1;
2348+ }
2349+
2350+ spin_unlock(&inode->i_lock);
2351+ if (writeback)
2352+ /*
2353+ * queue inode for writeback: we can't actually call
2354+ * filemap_write_and_wait, etc. from message handler
2355+ * context.
2356+ */
2357+ ceph_queue_writeback(inode);
2358+ if (queue_invalidate)
2359+ ceph_queue_invalidate(inode);
2360+ if (wake)
2361+ wake_up(&ci->i_cap_wq);
2362+ return reply;
2363+}
2364+
2365+/*
2366+ * Handle FLUSH_ACK from MDS, indicating that metadata we sent to the
2367+ * MDS has been safely committed.
2368+ */
2369+static void handle_cap_flush_ack(struct inode *inode, u64 flush_tid,
2370+ struct ceph_mds_caps *m,
2371+ struct ceph_mds_session *session,
2372+ struct ceph_cap *cap)
2373+ __releases(inode->i_lock)
2374+{
2375+ struct ceph_inode_info *ci = ceph_inode(inode);
2376+ struct ceph_mds_client *mdsc = &ceph_client(inode->i_sb)->mdsc;
2377+ unsigned seq = le32_to_cpu(m->seq);
2378+ int dirty = le32_to_cpu(m->dirty);
2379+ int cleaned = 0;
2380+ int drop = 0;
2381+ int i;
2382+
2383+ for (i = 0; i < CEPH_CAP_BITS; i++)
2384+ if ((dirty & (1 << i)) &&
2385+ flush_tid == ci->i_cap_flush_tid[i])
2386+ cleaned |= 1 << i;
2387+
2388+ dout("handle_cap_flush_ack inode %p mds%d seq %d on %s cleaned %s,"
2389+ " flushing %s -> %s\n",
2390+ inode, session->s_mds, seq, ceph_cap_string(dirty),
2391+ ceph_cap_string(cleaned), ceph_cap_string(ci->i_flushing_caps),
2392+ ceph_cap_string(ci->i_flushing_caps & ~cleaned));
2393+
2394+ if (ci->i_flushing_caps == (ci->i_flushing_caps & ~cleaned))
2395+ goto out;
2396+
2397+ ci->i_flushing_caps &= ~cleaned;
2398+
2399+ spin_lock(&mdsc->cap_dirty_lock);
2400+ if (ci->i_flushing_caps == 0) {
2401+ list_del_init(&ci->i_flushing_item);
2402+ if (!list_empty(&session->s_cap_flushing))
2403+ dout(" mds%d still flushing cap on %p\n",
2404+ session->s_mds,
2405+ &list_entry(session->s_cap_flushing.next,
2406+ struct ceph_inode_info,
2407+ i_flushing_item)->vfs_inode);
2408+ mdsc->num_cap_flushing--;
2409+ wake_up(&mdsc->cap_flushing_wq);
2410+ dout(" inode %p now !flushing\n", inode);
2411+
2412+ if (ci->i_dirty_caps == 0) {
2413+ dout(" inode %p now clean\n", inode);
2414+ BUG_ON(!list_empty(&ci->i_dirty_item));
2415+ drop = 1;
2416+ } else {
2417+ BUG_ON(list_empty(&ci->i_dirty_item));
2418+ }
2419+ }
2420+ spin_unlock(&mdsc->cap_dirty_lock);
2421+ wake_up(&ci->i_cap_wq);
2422+
2423+out:
2424+ spin_unlock(&inode->i_lock);
2425+ if (drop)
2426+ iput(inode);
2427+}
2428+
2429+/*
2430+ * Handle FLUSHSNAP_ACK. MDS has flushed snap data to disk and we can
2431+ * throw away our cap_snap.
2432+ *
2433+ * Caller hold s_mutex.
2434+ */
2435+static void handle_cap_flushsnap_ack(struct inode *inode, u64 flush_tid,
2436+ struct ceph_mds_caps *m,
2437+ struct ceph_mds_session *session)
2438+{
2439+ struct ceph_inode_info *ci = ceph_inode(inode);
2440+ u64 follows = le64_to_cpu(m->snap_follows);
2441+ struct ceph_cap_snap *capsnap;
2442+ int drop = 0;
2443+
2444+ dout("handle_cap_flushsnap_ack inode %p ci %p mds%d follows %lld\n",
2445+ inode, ci, session->s_mds, follows);
2446+
2447+ spin_lock(&inode->i_lock);
2448+ list_for_each_entry(capsnap, &ci->i_cap_snaps, ci_item) {
2449+ if (capsnap->follows == follows) {
2450+ if (capsnap->flush_tid != flush_tid) {
2451+ dout(" cap_snap %p follows %lld tid %lld !="
2452+ " %lld\n", capsnap, follows,
2453+ flush_tid, capsnap->flush_tid);
2454+ break;
2455+ }
2456+ WARN_ON(capsnap->dirty_pages || capsnap->writing);
2457+ dout(" removing cap_snap %p follows %lld\n",
2458+ capsnap, follows);
2459+ ceph_put_snap_context(capsnap->context);
2460+ list_del(&capsnap->ci_item);
2461+ list_del(&capsnap->flushing_item);
2462+ ceph_put_cap_snap(capsnap);
2463+ drop = 1;
2464+ break;
2465+ } else {
2466+ dout(" skipping cap_snap %p follows %lld\n",
2467+ capsnap, capsnap->follows);
2468+ }
2469+ }
2470+ spin_unlock(&inode->i_lock);
2471+ if (drop)
2472+ iput(inode);
2473+}
2474+
2475+/*
2476+ * Handle TRUNC from MDS, indicating file truncation.
2477+ *
2478+ * caller hold s_mutex.
2479+ */
2480+static void handle_cap_trunc(struct inode *inode,
2481+ struct ceph_mds_caps *trunc,
2482+ struct ceph_mds_session *session)
2483+ __releases(inode->i_lock)
2484+{
2485+ struct ceph_inode_info *ci = ceph_inode(inode);
2486+ int mds = session->s_mds;
2487+ int seq = le32_to_cpu(trunc->seq);
2488+ u32 truncate_seq = le32_to_cpu(trunc->truncate_seq);
2489+ u64 truncate_size = le64_to_cpu(trunc->truncate_size);
2490+ u64 size = le64_to_cpu(trunc->size);
2491+ int implemented = 0;
2492+ int dirty = __ceph_caps_dirty(ci);
2493+ int issued = __ceph_caps_issued(ceph_inode(inode), &implemented);
2494+ int queue_trunc = 0;
2495+
2496+ issued |= implemented | dirty;
2497+
2498+ dout("handle_cap_trunc inode %p mds%d seq %d to %lld seq %d\n",
2499+ inode, mds, seq, truncate_size, truncate_seq);
2500+ queue_trunc = ceph_fill_file_size(inode, issued,
2501+ truncate_seq, truncate_size, size);
2502+ spin_unlock(&inode->i_lock);
2503+
2504+ if (queue_trunc)
2505+ ceph_queue_vmtruncate(inode);
2506+}
2507+
2508+/*
2509+ * Handle EXPORT from MDS. Cap is being migrated _from_ this mds to a
2510+ * different one. If we are the most recent migration we've seen (as
2511+ * indicated by mseq), make note of the migrating cap bits for the
2512+ * duration (until we see the corresponding IMPORT).
2513+ *
2514+ * caller holds s_mutex
2515+ */
2516+static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex,
2517+ struct ceph_mds_session *session)
2518+{
2519+ struct ceph_inode_info *ci = ceph_inode(inode);
2520+ int mds = session->s_mds;
2521+ unsigned mseq = le32_to_cpu(ex->migrate_seq);
2522+ struct ceph_cap *cap = NULL, *t;
2523+ struct rb_node *p;
2524+ int remember = 1;
2525+
2526+ dout("handle_cap_export inode %p ci %p mds%d mseq %d\n",
2527+ inode, ci, mds, mseq);
2528+
2529+ spin_lock(&inode->i_lock);
2530+
2531+ /* make sure we haven't seen a higher mseq */
2532+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
2533+ t = rb_entry(p, struct ceph_cap, ci_node);
2534+ if (ceph_seq_cmp(t->mseq, mseq) > 0) {
2535+ dout(" higher mseq on cap from mds%d\n",
2536+ t->session->s_mds);
2537+ remember = 0;
2538+ }
2539+ if (t->session->s_mds == mds)
2540+ cap = t;
2541+ }
2542+
2543+ if (cap) {
2544+ if (remember) {
2545+ /* make note */
2546+ ci->i_cap_exporting_mds = mds;
2547+ ci->i_cap_exporting_mseq = mseq;
2548+ ci->i_cap_exporting_issued = cap->issued;
2549+ }
2550+ __ceph_remove_cap(cap);
2551+ } else {
2552+ WARN_ON(!cap);
2553+ }
2554+
2555+ spin_unlock(&inode->i_lock);
2556+}
2557+
2558+/*
2559+ * Handle cap IMPORT. If there are temp bits from an older EXPORT,
2560+ * clean them up.
2561+ *
2562+ * caller holds s_mutex.
2563+ */
2564+static void handle_cap_import(struct ceph_mds_client *mdsc,
2565+ struct inode *inode, struct ceph_mds_caps *im,
2566+ struct ceph_mds_session *session,
2567+ void *snaptrace, int snaptrace_len)
2568+{
2569+ struct ceph_inode_info *ci = ceph_inode(inode);
2570+ int mds = session->s_mds;
2571+ unsigned issued = le32_to_cpu(im->caps);
2572+ unsigned wanted = le32_to_cpu(im->wanted);
2573+ unsigned seq = le32_to_cpu(im->seq);
2574+ unsigned mseq = le32_to_cpu(im->migrate_seq);
2575+ u64 realmino = le64_to_cpu(im->realm);
2576+ u64 cap_id = le64_to_cpu(im->cap_id);
2577+
2578+ if (ci->i_cap_exporting_mds >= 0 &&
2579+ ceph_seq_cmp(ci->i_cap_exporting_mseq, mseq) < 0) {
2580+ dout("handle_cap_import inode %p ci %p mds%d mseq %d"
2581+ " - cleared exporting from mds%d\n",
2582+ inode, ci, mds, mseq,
2583+ ci->i_cap_exporting_mds);
2584+ ci->i_cap_exporting_issued = 0;
2585+ ci->i_cap_exporting_mseq = 0;
2586+ ci->i_cap_exporting_mds = -1;
2587+ } else {
2588+ dout("handle_cap_import inode %p ci %p mds%d mseq %d\n",
2589+ inode, ci, mds, mseq);
2590+ }
2591+
2592+ down_write(&mdsc->snap_rwsem);
2593+ ceph_update_snap_trace(mdsc, snaptrace, snaptrace+snaptrace_len,
2594+ false);
2595+ downgrade_write(&mdsc->snap_rwsem);
2596+ ceph_add_cap(inode, session, cap_id, -1,
2597+ issued, wanted, seq, mseq, realmino, CEPH_CAP_FLAG_AUTH,
2598+ NULL /* no caps context */);
2599+ try_flush_caps(inode, session, NULL);
2600+ up_read(&mdsc->snap_rwsem);
2601+}
2602+
2603+/*
2604+ * Handle a caps message from the MDS.
2605+ *
2606+ * Identify the appropriate session, inode, and call the right handler
2607+ * based on the cap op.
2608+ */
2609+void ceph_handle_caps(struct ceph_mds_session *session,
2610+ struct ceph_msg *msg)
2611+{
2612+ struct ceph_mds_client *mdsc = session->s_mdsc;
2613+ struct super_block *sb = mdsc->client->sb;
2614+ struct inode *inode;
2615+ struct ceph_cap *cap;
2616+ struct ceph_mds_caps *h;
2617+ int mds = session->s_mds;
2618+ int op;
2619+ u32 seq;
2620+ struct ceph_vino vino;
2621+ u64 cap_id;
2622+ u64 size, max_size;
2623+ u64 tid;
2624+ int check_caps = 0;
2625+ void *snaptrace;
2626+ int r;
2627+
2628+ dout("handle_caps from mds%d\n", mds);
2629+
2630+ /* decode */
2631+ tid = le64_to_cpu(msg->hdr.tid);
2632+ if (msg->front.iov_len < sizeof(*h))
2633+ goto bad;
2634+ h = msg->front.iov_base;
2635+ snaptrace = h + 1;
2636+ op = le32_to_cpu(h->op);
2637+ vino.ino = le64_to_cpu(h->ino);
2638+ vino.snap = CEPH_NOSNAP;
2639+ cap_id = le64_to_cpu(h->cap_id);
2640+ seq = le32_to_cpu(h->seq);
2641+ size = le64_to_cpu(h->size);
2642+ max_size = le64_to_cpu(h->max_size);
2643+
2644+ mutex_lock(&session->s_mutex);
2645+ session->s_seq++;
2646+ dout(" mds%d seq %lld cap seq %u\n", session->s_mds, session->s_seq,
2647+ (unsigned)seq);
2648+
2649+ /* lookup ino */
2650+ inode = ceph_find_inode(sb, vino);
2651+ dout(" op %s ino %llx.%llx inode %p\n", ceph_cap_op_name(op), vino.ino,
2652+ vino.snap, inode);
2653+ if (!inode) {
2654+ dout(" i don't have ino %llx\n", vino.ino);
2655+ goto done;
2656+ }
2657+
2658+ /* these will work even if we don't have a cap yet */
2659+ switch (op) {
2660+ case CEPH_CAP_OP_FLUSHSNAP_ACK:
2661+ handle_cap_flushsnap_ack(inode, tid, h, session);
2662+ goto done;
2663+
2664+ case CEPH_CAP_OP_EXPORT:
2665+ handle_cap_export(inode, h, session);
2666+ goto done;
2667+
2668+ case CEPH_CAP_OP_IMPORT:
2669+ handle_cap_import(mdsc, inode, h, session,
2670+ snaptrace, le32_to_cpu(h->snap_trace_len));
2671+ check_caps = 1; /* we may have sent a RELEASE to the old auth */
2672+ goto done;
2673+ }
2674+
2675+ /* the rest require a cap */
2676+ spin_lock(&inode->i_lock);
2677+ cap = __get_cap_for_mds(ceph_inode(inode), mds);
2678+ if (!cap) {
2679+ dout("no cap on %p ino %llx.%llx from mds%d, releasing\n",
2680+ inode, ceph_ino(inode), ceph_snap(inode), mds);
2681+ spin_unlock(&inode->i_lock);
2682+ goto done;
2683+ }
2684+
2685+ /* note that each of these drops i_lock for us */
2686+ switch (op) {
2687+ case CEPH_CAP_OP_REVOKE:
2688+ case CEPH_CAP_OP_GRANT:
2689+ r = handle_cap_grant(inode, h, session, cap, msg->middle);
2690+ if (r == 1)
2691+ ceph_check_caps(ceph_inode(inode),
2692+ CHECK_CAPS_NODELAY|CHECK_CAPS_AUTHONLY,
2693+ session);
2694+ else if (r == 2)
2695+ ceph_check_caps(ceph_inode(inode),
2696+ CHECK_CAPS_NODELAY,
2697+ session);
2698+ break;
2699+
2700+ case CEPH_CAP_OP_FLUSH_ACK:
2701+ handle_cap_flush_ack(inode, tid, h, session, cap);
2702+ break;
2703+
2704+ case CEPH_CAP_OP_TRUNC:
2705+ handle_cap_trunc(inode, h, session);
2706+ break;
2707+
2708+ default:
2709+ spin_unlock(&inode->i_lock);
2710+ pr_err("ceph_handle_caps: unknown cap op %d %s\n", op,
2711+ ceph_cap_op_name(op));
2712+ }
2713+
2714+done:
2715+ mutex_unlock(&session->s_mutex);
2716+
2717+ if (check_caps)
2718+ ceph_check_caps(ceph_inode(inode), CHECK_CAPS_NODELAY, NULL);
2719+ if (inode)
2720+ iput(inode);
2721+ return;
2722+
2723+bad:
2724+ pr_err("ceph_handle_caps: corrupt message\n");
2725+ ceph_msg_dump(msg);
2726+ return;
2727+}
2728+
2729+/*
2730+ * Delayed work handler to process end of delayed cap release LRU list.
2731+ */
2732+void ceph_check_delayed_caps(struct ceph_mds_client *mdsc)
2733+{
2734+ struct ceph_inode_info *ci;
2735+ int flags = CHECK_CAPS_NODELAY;
2736+
2737+ dout("check_delayed_caps\n");
2738+ while (1) {
2739+ spin_lock(&mdsc->cap_delay_lock);
2740+ if (list_empty(&mdsc->cap_delay_list))
2741+ break;
2742+ ci = list_first_entry(&mdsc->cap_delay_list,
2743+ struct ceph_inode_info,
2744+ i_cap_delay_list);
2745+ if ((ci->i_ceph_flags & CEPH_I_FLUSH) == 0 &&
2746+ time_before(jiffies, ci->i_hold_caps_max))
2747+ break;
2748+ list_del_init(&ci->i_cap_delay_list);
2749+ spin_unlock(&mdsc->cap_delay_lock);
2750+ dout("check_delayed_caps on %p\n", &ci->vfs_inode);
2751+ ceph_check_caps(ci, flags, NULL);
2752+ }
2753+ spin_unlock(&mdsc->cap_delay_lock);
2754+}
2755+
2756+/*
2757+ * Flush all dirty caps to the mds
2758+ */
2759+void ceph_flush_dirty_caps(struct ceph_mds_client *mdsc)
2760+{
2761+ struct ceph_inode_info *ci, *nci = NULL;
2762+ struct inode *inode, *ninode = NULL;
2763+ struct list_head *p, *n;
2764+
2765+ dout("flush_dirty_caps\n");
2766+ spin_lock(&mdsc->cap_dirty_lock);
2767+ list_for_each_safe(p, n, &mdsc->cap_dirty) {
2768+ if (nci) {
2769+ ci = nci;
2770+ inode = ninode;
2771+ ci->i_ceph_flags &= ~CEPH_I_NOFLUSH;
2772+ dout("flush_dirty_caps inode %p (was next inode)\n",
2773+ inode);
2774+ } else {
2775+ ci = list_entry(p, struct ceph_inode_info,
2776+ i_dirty_item);
2777+ inode = igrab(&ci->vfs_inode);
2778+ BUG_ON(!inode);
2779+ dout("flush_dirty_caps inode %p\n", inode);
2780+ }
2781+ if (n != &mdsc->cap_dirty) {
2782+ nci = list_entry(n, struct ceph_inode_info,
2783+ i_dirty_item);
2784+ ninode = igrab(&nci->vfs_inode);
2785+ BUG_ON(!ninode);
2786+ nci->i_ceph_flags |= CEPH_I_NOFLUSH;
2787+ dout("flush_dirty_caps next inode %p, noflush\n",
2788+ ninode);
2789+ } else {
2790+ nci = NULL;
2791+ ninode = NULL;
2792+ }
2793+ spin_unlock(&mdsc->cap_dirty_lock);
2794+ if (inode) {
2795+ ceph_check_caps(ci, CHECK_CAPS_NODELAY|CHECK_CAPS_FLUSH,
2796+ NULL);
2797+ iput(inode);
2798+ }
2799+ spin_lock(&mdsc->cap_dirty_lock);
2800+ }
2801+ spin_unlock(&mdsc->cap_dirty_lock);
2802+}
2803+
2804+/*
2805+ * Drop open file reference. If we were the last open file,
2806+ * we may need to release capabilities to the MDS (or schedule
2807+ * their delayed release).
2808+ */
2809+void ceph_put_fmode(struct ceph_inode_info *ci, int fmode)
2810+{
2811+ struct inode *inode = &ci->vfs_inode;
2812+ int last = 0;
2813+
2814+ spin_lock(&inode->i_lock);
2815+ dout("put_fmode %p fmode %d %d -> %d\n", inode, fmode,
2816+ ci->i_nr_by_mode[fmode], ci->i_nr_by_mode[fmode]-1);
2817+ BUG_ON(ci->i_nr_by_mode[fmode] == 0);
2818+ if (--ci->i_nr_by_mode[fmode] == 0)
2819+ last++;
2820+ spin_unlock(&inode->i_lock);
2821+
2822+ if (last && ci->i_vino.snap == CEPH_NOSNAP)
2823+ ceph_check_caps(ci, 0, NULL);
2824+}
2825+
2826+/*
2827+ * Helpers for embedding cap and dentry lease releases into mds
2828+ * requests.
2829+ *
2830+ * @force is used by dentry_release (below) to force inclusion of a
2831+ * record for the directory inode, even when there aren't any caps to
2832+ * drop.
2833+ */
2834+int ceph_encode_inode_release(void **p, struct inode *inode,
2835+ int mds, int drop, int unless, int force)
2836+{
2837+ struct ceph_inode_info *ci = ceph_inode(inode);
2838+ struct ceph_cap *cap;
2839+ struct ceph_mds_request_release *rel = *p;
2840+ int ret = 0;
2841+
2842+ dout("encode_inode_release %p mds%d drop %s unless %s\n", inode,
2843+ mds, ceph_cap_string(drop), ceph_cap_string(unless));
2844+
2845+ spin_lock(&inode->i_lock);
2846+ cap = __get_cap_for_mds(ci, mds);
2847+ if (cap && __cap_is_valid(cap)) {
2848+ if (force ||
2849+ ((cap->issued & drop) &&
2850+ (cap->issued & unless) == 0)) {
2851+ if ((cap->issued & drop) &&
2852+ (cap->issued & unless) == 0) {
2853+ dout("encode_inode_release %p cap %p %s -> "
2854+ "%s\n", inode, cap,
2855+ ceph_cap_string(cap->issued),
2856+ ceph_cap_string(cap->issued & ~drop));
2857+ cap->issued &= ~drop;
2858+ cap->implemented &= ~drop;
2859+ if (ci->i_ceph_flags & CEPH_I_NODELAY) {
2860+ int wanted = __ceph_caps_wanted(ci);
2861+ dout(" wanted %s -> %s (act %s)\n",
2862+ ceph_cap_string(cap->mds_wanted),
2863+ ceph_cap_string(cap->mds_wanted &
2864+ ~wanted),
2865+ ceph_cap_string(wanted));
2866+ cap->mds_wanted &= wanted;
2867+ }
2868+ } else {
2869+ dout("encode_inode_release %p cap %p %s"
2870+ " (force)\n", inode, cap,
2871+ ceph_cap_string(cap->issued));
2872+ }
2873+
2874+ rel->ino = cpu_to_le64(ceph_ino(inode));
2875+ rel->cap_id = cpu_to_le64(cap->cap_id);
2876+ rel->seq = cpu_to_le32(cap->seq);
2877+ rel->issue_seq = cpu_to_le32(cap->issue_seq),
2878+ rel->mseq = cpu_to_le32(cap->mseq);
2879+ rel->caps = cpu_to_le32(cap->issued);
2880+ rel->wanted = cpu_to_le32(cap->mds_wanted);
2881+ rel->dname_len = 0;
2882+ rel->dname_seq = 0;
2883+ *p += sizeof(*rel);
2884+ ret = 1;
2885+ } else {
2886+ dout("encode_inode_release %p cap %p %s\n",
2887+ inode, cap, ceph_cap_string(cap->issued));
2888+ }
2889+ }
2890+ spin_unlock(&inode->i_lock);
2891+ return ret;
2892+}
2893+
2894+int ceph_encode_dentry_release(void **p, struct dentry *dentry,
2895+ int mds, int drop, int unless)
2896+{
2897+ struct inode *dir = dentry->d_parent->d_inode;
2898+ struct ceph_mds_request_release *rel = *p;
2899+ struct ceph_dentry_info *di = ceph_dentry(dentry);
2900+ int force = 0;
2901+ int ret;
2902+
2903+ /*
2904+ * force an record for the directory caps if we have a dentry lease.
2905+ * this is racy (can't take i_lock and d_lock together), but it
2906+ * doesn't have to be perfect; the mds will revoke anything we don't
2907+ * release.
2908+ */
2909+ spin_lock(&dentry->d_lock);
2910+ if (di->lease_session && di->lease_session->s_mds == mds)
2911+ force = 1;
2912+ spin_unlock(&dentry->d_lock);
2913+
2914+ ret = ceph_encode_inode_release(p, dir, mds, drop, unless, force);
2915+
2916+ spin_lock(&dentry->d_lock);
2917+ if (ret && di->lease_session && di->lease_session->s_mds == mds) {
2918+ dout("encode_dentry_release %p mds%d seq %d\n",
2919+ dentry, mds, (int)di->lease_seq);
2920+ rel->dname_len = cpu_to_le32(dentry->d_name.len);
2921+ memcpy(*p, dentry->d_name.name, dentry->d_name.len);
2922+ *p += dentry->d_name.len;
2923+ rel->dname_seq = cpu_to_le32(di->lease_seq);
2924+ }
2925+ spin_unlock(&dentry->d_lock);
2926+ return ret;
2927+}
fs/ceph/ceph_debug.h+37−0
@@ -0,0 +1,37 @@
1+#ifndef _FS_CEPH_DEBUG_H
2+#define _FS_CEPH_DEBUG_H
3+
4+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
5+
6+#ifdef CONFIG_CEPH_FS_PRETTYDEBUG
7+
8+/*
9+ * wrap pr_debug to include a filename:lineno prefix on each line.
10+ * this incurs some overhead (kernel size and execution time) due to
11+ * the extra function call at each call site.
12+ */
13+
14+# if defined(DEBUG) || defined(CONFIG_DYNAMIC_DEBUG)
15+extern const char *ceph_file_part(const char *s, int len);
16+# define dout(fmt, ...) \
17+ pr_debug(" %12.12s:%-4d : " fmt, \
18+ ceph_file_part(__FILE__, sizeof(__FILE__)), \
19+ __LINE__, ##__VA_ARGS__)
20+# else
21+/* faux printk call just to see any compiler warnings. */
22+# define dout(fmt, ...) do { \
23+ if (0) \
24+ printk(KERN_DEBUG fmt, ##__VA_ARGS__); \
25+ } while (0)
26+# endif
27+
28+#else
29+
30+/*
31+ * or, just wrap pr_debug
32+ */
33+# define dout(fmt, ...) pr_debug(" " fmt, ##__VA_ARGS__)
34+
35+#endif
36+
37+#endif
fs/ceph/ceph_frag.c+21−0
@@ -0,0 +1,21 @@
1+/*
2+ * Ceph 'frag' type
3+ */
4+#include "types.h"
5+
6+int ceph_frag_compare(__u32 a, __u32 b)
7+{
8+ unsigned va = ceph_frag_value(a);
9+ unsigned vb = ceph_frag_value(b);
10+ if (va < vb)
11+ return -1;
12+ if (va > vb)
13+ return 1;
14+ va = ceph_frag_bits(a);
15+ vb = ceph_frag_bits(b);
16+ if (va < vb)
17+ return -1;
18+ if (va > vb)
19+ return 1;
20+ return 0;
21+}
fs/ceph/ceph_frag.h+109−0
@@ -0,0 +1,109 @@
1+#ifndef _FS_CEPH_FRAG_H
2+#define _FS_CEPH_FRAG_H
3+
4+/*
5+ * "Frags" are a way to describe a subset of a 32-bit number space,
6+ * using a mask and a value to match against that mask. Any given frag
7+ * (subset of the number space) can be partitioned into 2^n sub-frags.
8+ *
9+ * Frags are encoded into a 32-bit word:
10+ * 8 upper bits = "bits"
11+ * 24 lower bits = "value"
12+ * (We could go to 5+27 bits, but who cares.)
13+ *
14+ * We use the _most_ significant bits of the 24 bit value. This makes
15+ * values logically sort.
16+ *
17+ * Unfortunately, because the "bits" field is still in the high bits, we
18+ * can't sort encoded frags numerically. However, it does allow you
19+ * to feed encoded frags as values into frag_contains_value.
20+ */
21+static inline __u32 ceph_frag_make(__u32 b, __u32 v)
22+{
23+ return (b << 24) |
24+ (v & (0xffffffu << (24-b)) & 0xffffffu);
25+}
26+static inline __u32 ceph_frag_bits(__u32 f)
27+{
28+ return f >> 24;
29+}
30+static inline __u32 ceph_frag_value(__u32 f)
31+{
32+ return f & 0xffffffu;
33+}
34+static inline __u32 ceph_frag_mask(__u32 f)
35+{
36+ return (0xffffffu << (24-ceph_frag_bits(f))) & 0xffffffu;
37+}
38+static inline __u32 ceph_frag_mask_shift(__u32 f)
39+{
40+ return 24 - ceph_frag_bits(f);
41+}
42+
43+static inline int ceph_frag_contains_value(__u32 f, __u32 v)
44+{
45+ return (v & ceph_frag_mask(f)) == ceph_frag_value(f);
46+}
47+static inline int ceph_frag_contains_frag(__u32 f, __u32 sub)
48+{
49+ /* is sub as specific as us, and contained by us? */
50+ return ceph_frag_bits(sub) >= ceph_frag_bits(f) &&
51+ (ceph_frag_value(sub) & ceph_frag_mask(f)) == ceph_frag_value(f);
52+}
53+
54+static inline __u32 ceph_frag_parent(__u32 f)
55+{
56+ return ceph_frag_make(ceph_frag_bits(f) - 1,
57+ ceph_frag_value(f) & (ceph_frag_mask(f) << 1));
58+}
59+static inline int ceph_frag_is_left_child(__u32 f)
60+{
61+ return ceph_frag_bits(f) > 0 &&
62+ (ceph_frag_value(f) & (0x1000000 >> ceph_frag_bits(f))) == 0;
63+}
64+static inline int ceph_frag_is_right_child(__u32 f)
65+{
66+ return ceph_frag_bits(f) > 0 &&
67+ (ceph_frag_value(f) & (0x1000000 >> ceph_frag_bits(f))) == 1;
68+}
69+static inline __u32 ceph_frag_sibling(__u32 f)
70+{
71+ return ceph_frag_make(ceph_frag_bits(f),
72+ ceph_frag_value(f) ^ (0x1000000 >> ceph_frag_bits(f)));
73+}
74+static inline __u32 ceph_frag_left_child(__u32 f)
75+{
76+ return ceph_frag_make(ceph_frag_bits(f)+1, ceph_frag_value(f));
77+}
78+static inline __u32 ceph_frag_right_child(__u32 f)
79+{
80+ return ceph_frag_make(ceph_frag_bits(f)+1,
81+ ceph_frag_value(f) | (0x1000000 >> (1+ceph_frag_bits(f))));
82+}
83+static inline __u32 ceph_frag_make_child(__u32 f, int by, int i)
84+{
85+ int newbits = ceph_frag_bits(f) + by;
86+ return ceph_frag_make(newbits,
87+ ceph_frag_value(f) | (i << (24 - newbits)));
88+}
89+static inline int ceph_frag_is_leftmost(__u32 f)
90+{
91+ return ceph_frag_value(f) == 0;
92+}
93+static inline int ceph_frag_is_rightmost(__u32 f)
94+{
95+ return ceph_frag_value(f) == ceph_frag_mask(f);
96+}
97+static inline __u32 ceph_frag_next(__u32 f)
98+{
99+ return ceph_frag_make(ceph_frag_bits(f),
100+ ceph_frag_value(f) + (0x1000000 >> ceph_frag_bits(f)));
101+}
102+
103+/*
104+ * comparator to sort frags logically, as when traversing the
105+ * number space in ascending order...
106+ */
107+int ceph_frag_compare(__u32 a, __u32 b);
108+
109+#endif
fs/ceph/ceph_fs.c+74−0
@@ -0,0 +1,74 @@
1+/*
2+ * Some non-inline ceph helpers
3+ */
4+#include "types.h"
5+
6+/*
7+ * return true if @layout appears to be valid
8+ */
9+int ceph_file_layout_is_valid(const struct ceph_file_layout *layout)
10+{
11+ __u32 su = le32_to_cpu(layout->fl_stripe_unit);
12+ __u32 sc = le32_to_cpu(layout->fl_stripe_count);
13+ __u32 os = le32_to_cpu(layout->fl_object_size);
14+
15+ /* stripe unit, object size must be non-zero, 64k increment */
16+ if (!su || (su & (CEPH_MIN_STRIPE_UNIT-1)))
17+ return 0;
18+ if (!os || (os & (CEPH_MIN_STRIPE_UNIT-1)))
19+ return 0;
20+ /* object size must be a multiple of stripe unit */
21+ if (os < su || os % su)
22+ return 0;
23+ /* stripe count must be non-zero */
24+ if (!sc)
25+ return 0;
26+ return 1;
27+}
28+
29+
30+int ceph_flags_to_mode(int flags)
31+{
32+#ifdef O_DIRECTORY /* fixme */
33+ if ((flags & O_DIRECTORY) == O_DIRECTORY)
34+ return CEPH_FILE_MODE_PIN;
35+#endif
36+#ifdef O_LAZY
37+ if (flags & O_LAZY)
38+ return CEPH_FILE_MODE_LAZY;
39+#endif
40+ if ((flags & O_APPEND) == O_APPEND)
41+ flags |= O_WRONLY;
42+
43+ flags &= O_ACCMODE;
44+ if ((flags & O_RDWR) == O_RDWR)
45+ return CEPH_FILE_MODE_RDWR;
46+ if ((flags & O_WRONLY) == O_WRONLY)
47+ return CEPH_FILE_MODE_WR;
48+ return CEPH_FILE_MODE_RD;
49+}
50+
51+int ceph_caps_for_mode(int mode)
52+{
53+ switch (mode) {
54+ case CEPH_FILE_MODE_PIN:
55+ return CEPH_CAP_PIN;
56+ case CEPH_FILE_MODE_RD:
57+ return CEPH_CAP_PIN | CEPH_CAP_FILE_SHARED |
58+ CEPH_CAP_FILE_RD | CEPH_CAP_FILE_CACHE;
59+ case CEPH_FILE_MODE_RDWR:
60+ return CEPH_CAP_PIN | CEPH_CAP_FILE_SHARED |
61+ CEPH_CAP_FILE_EXCL |
62+ CEPH_CAP_FILE_RD | CEPH_CAP_FILE_CACHE |
63+ CEPH_CAP_FILE_WR | CEPH_CAP_FILE_BUFFER |
64+ CEPH_CAP_AUTH_SHARED | CEPH_CAP_AUTH_EXCL |
65+ CEPH_CAP_XATTR_SHARED | CEPH_CAP_XATTR_EXCL;
66+ case CEPH_FILE_MODE_WR:
67+ return CEPH_CAP_PIN | CEPH_CAP_FILE_SHARED |
68+ CEPH_CAP_FILE_EXCL |
69+ CEPH_CAP_FILE_WR | CEPH_CAP_FILE_BUFFER |
70+ CEPH_CAP_AUTH_SHARED | CEPH_CAP_AUTH_EXCL |
71+ CEPH_CAP_XATTR_SHARED | CEPH_CAP_XATTR_EXCL;
72+ }
73+ return 0;
74+}
fs/ceph/ceph_fs.h+650−0
@@ -0,0 +1,650 @@
1+/*
2+ * ceph_fs.h - Ceph constants and data types to share between kernel and
3+ * user space.
4+ *
5+ * Most types in this file are defined as little-endian, and are
6+ * primarily intended to describe data structures that pass over the
7+ * wire or that are stored on disk.
8+ *
9+ * LGPL2
10+ */
11+
12+#ifndef _FS_CEPH_CEPH_FS_H
13+#define _FS_CEPH_CEPH_FS_H
14+
15+#include "msgr.h"
16+#include "rados.h"
17+
18+/*
19+ * Ceph release version
20+ */
21+#define CEPH_VERSION_MAJOR 0
22+#define CEPH_VERSION_MINOR 19
23+#define CEPH_VERSION_PATCH 0
24+
25+#define _CEPH_STRINGIFY(x) #x
26+#define CEPH_STRINGIFY(x) _CEPH_STRINGIFY(x)
27+#define CEPH_MAKE_VERSION(x, y, z) CEPH_STRINGIFY(x) "." CEPH_STRINGIFY(y) \
28+ "." CEPH_STRINGIFY(z)
29+#define CEPH_VERSION CEPH_MAKE_VERSION(CEPH_VERSION_MAJOR, \
30+ CEPH_VERSION_MINOR, CEPH_VERSION_PATCH)
31+
32+/*
33+ * subprotocol versions. when specific messages types or high-level
34+ * protocols change, bump the affected components. we keep rev
35+ * internal cluster protocols separately from the public,
36+ * client-facing protocol.
37+ */
38+#define CEPH_OSD_PROTOCOL 8 /* cluster internal */
39+#define CEPH_MDS_PROTOCOL 9 /* cluster internal */
40+#define CEPH_MON_PROTOCOL 5 /* cluster internal */
41+#define CEPH_OSDC_PROTOCOL 24 /* server/client */
42+#define CEPH_MDSC_PROTOCOL 32 /* server/client */
43+#define CEPH_MONC_PROTOCOL 15 /* server/client */
44+
45+
46+#define CEPH_INO_ROOT 1
47+#define CEPH_INO_CEPH 2 /* hidden .ceph dir */
48+
49+/* arbitrary limit on max # of monitors (cluster of 3 is typical) */
50+#define CEPH_MAX_MON 31
51+
52+
53+/*
54+ * feature bits
55+ */
56+#define CEPH_FEATURE_SUPPORTED 0
57+#define CEPH_FEATURE_REQUIRED 0
58+
59+
60+/*
61+ * ceph_file_layout - describe data layout for a file/inode
62+ */
63+struct ceph_file_layout {
64+ /* file -> object mapping */
65+ __le32 fl_stripe_unit; /* stripe unit, in bytes. must be multiple
66+ of page size. */
67+ __le32 fl_stripe_count; /* over this many objects */
68+ __le32 fl_object_size; /* until objects are this big, then move to
69+ new objects */
70+ __le32 fl_cas_hash; /* 0 = none; 1 = sha256 */
71+
72+ /* pg -> disk layout */
73+ __le32 fl_object_stripe_unit; /* for per-object parity, if any */
74+
75+ /* object -> pg layout */
76+ __le32 fl_pg_preferred; /* preferred primary for pg (-1 for none) */
77+ __le32 fl_pg_pool; /* namespace, crush ruleset, rep level */
78+} __attribute__ ((packed));
79+
80+#define CEPH_MIN_STRIPE_UNIT 65536
81+
82+int ceph_file_layout_is_valid(const struct ceph_file_layout *layout);
83+
84+
85+/* crypto algorithms */
86+#define CEPH_CRYPTO_NONE 0x0
87+#define CEPH_CRYPTO_AES 0x1
88+
89+/* security/authentication protocols */
90+#define CEPH_AUTH_UNKNOWN 0x0
91+#define CEPH_AUTH_NONE 0x1
92+#define CEPH_AUTH_CEPHX 0x2
93+
94+
95+/*********************************************
96+ * message layer
97+ */
98+
99+/*
100+ * message types
101+ */
102+
103+/* misc */
104+#define CEPH_MSG_SHUTDOWN 1
105+#define CEPH_MSG_PING 2
106+
107+/* client <-> monitor */
108+#define CEPH_MSG_MON_MAP 4
109+#define CEPH_MSG_MON_GET_MAP 5
110+#define CEPH_MSG_STATFS 13
111+#define CEPH_MSG_STATFS_REPLY 14
112+#define CEPH_MSG_MON_SUBSCRIBE 15
113+#define CEPH_MSG_MON_SUBSCRIBE_ACK 16
114+#define CEPH_MSG_AUTH 17
115+#define CEPH_MSG_AUTH_REPLY 18
116+
117+/* client <-> mds */
118+#define CEPH_MSG_MDS_MAP 21
119+
120+#define CEPH_MSG_CLIENT_SESSION 22
121+#define CEPH_MSG_CLIENT_RECONNECT 23
122+
123+#define CEPH_MSG_CLIENT_REQUEST 24
124+#define CEPH_MSG_CLIENT_REQUEST_FORWARD 25
125+#define CEPH_MSG_CLIENT_REPLY 26
126+#define CEPH_MSG_CLIENT_CAPS 0x310
127+#define CEPH_MSG_CLIENT_LEASE 0x311
128+#define CEPH_MSG_CLIENT_SNAP 0x312
129+#define CEPH_MSG_CLIENT_CAPRELEASE 0x313
130+
131+/* osd */
132+#define CEPH_MSG_OSD_MAP 41
133+#define CEPH_MSG_OSD_OP 42
134+#define CEPH_MSG_OSD_OPREPLY 43
135+
136+struct ceph_mon_request_header {
137+ __le64 have_version;
138+ __le16 session_mon;
139+ __le64 session_mon_tid;
140+} __attribute__ ((packed));
141+
142+struct ceph_mon_statfs {
143+ struct ceph_mon_request_header monhdr;
144+ struct ceph_fsid fsid;
145+} __attribute__ ((packed));
146+
147+struct ceph_statfs {
148+ __le64 kb, kb_used, kb_avail;
149+ __le64 num_objects;
150+} __attribute__ ((packed));
151+
152+struct ceph_mon_statfs_reply {
153+ struct ceph_fsid fsid;
154+ __le64 version;
155+ struct ceph_statfs st;
156+} __attribute__ ((packed));
157+
158+struct ceph_osd_getmap {
159+ struct ceph_mon_request_header monhdr;
160+ struct ceph_fsid fsid;
161+ __le32 start;
162+} __attribute__ ((packed));
163+
164+struct ceph_mds_getmap {
165+ struct ceph_mon_request_header monhdr;
166+ struct ceph_fsid fsid;
167+} __attribute__ ((packed));
168+
169+struct ceph_client_mount {
170+ struct ceph_mon_request_header monhdr;
171+} __attribute__ ((packed));
172+
173+struct ceph_mon_subscribe_item {
174+ __le64 have_version; __le64 have;
175+ __u8 onetime;
176+} __attribute__ ((packed));
177+
178+struct ceph_mon_subscribe_ack {
179+ __le32 duration; /* seconds */
180+ struct ceph_fsid fsid;
181+} __attribute__ ((packed));
182+
183+/*
184+ * mds states
185+ * > 0 -> in
186+ * <= 0 -> out
187+ */
188+#define CEPH_MDS_STATE_DNE 0 /* down, does not exist. */
189+#define CEPH_MDS_STATE_STOPPED -1 /* down, once existed, but no subtrees.
190+ empty log. */
191+#define CEPH_MDS_STATE_BOOT -4 /* up, boot announcement. */
192+#define CEPH_MDS_STATE_STANDBY -5 /* up, idle. waiting for assignment. */
193+#define CEPH_MDS_STATE_CREATING -6 /* up, creating MDS instance. */
194+#define CEPH_MDS_STATE_STARTING -7 /* up, starting previously stopped mds */
195+#define CEPH_MDS_STATE_STANDBY_REPLAY -8 /* up, tailing active node's journal */
196+
197+#define CEPH_MDS_STATE_REPLAY 8 /* up, replaying journal. */
198+#define CEPH_MDS_STATE_RESOLVE 9 /* up, disambiguating distributed
199+ operations (import, rename, etc.) */
200+#define CEPH_MDS_STATE_RECONNECT 10 /* up, reconnect to clients */
201+#define CEPH_MDS_STATE_REJOIN 11 /* up, rejoining distributed cache */
202+#define CEPH_MDS_STATE_CLIENTREPLAY 12 /* up, replaying client operations */
203+#define CEPH_MDS_STATE_ACTIVE 13 /* up, active */
204+#define CEPH_MDS_STATE_STOPPING 14 /* up, but exporting metadata */
205+
206+extern const char *ceph_mds_state_name(int s);
207+
208+
209+/*
210+ * metadata lock types.
211+ * - these are bitmasks.. we can compose them
212+ * - they also define the lock ordering by the MDS
213+ * - a few of these are internal to the mds
214+ */
215+#define CEPH_LOCK_DN 1
216+#define CEPH_LOCK_ISNAP 2
217+#define CEPH_LOCK_IVERSION 4 /* mds internal */
218+#define CEPH_LOCK_IFILE 8 /* mds internal */
219+#define CEPH_LOCK_IAUTH 32
220+#define CEPH_LOCK_ILINK 64
221+#define CEPH_LOCK_IDFT 128 /* dir frag tree */
222+#define CEPH_LOCK_INEST 256 /* mds internal */
223+#define CEPH_LOCK_IXATTR 512
224+#define CEPH_LOCK_INO 2048 /* immutable inode bits; not a lock */
225+
226+/* client_session ops */
227+enum {
228+ CEPH_SESSION_REQUEST_OPEN,
229+ CEPH_SESSION_OPEN,
230+ CEPH_SESSION_REQUEST_CLOSE,
231+ CEPH_SESSION_CLOSE,
232+ CEPH_SESSION_REQUEST_RENEWCAPS,
233+ CEPH_SESSION_RENEWCAPS,
234+ CEPH_SESSION_STALE,
235+ CEPH_SESSION_RECALL_STATE,
236+};
237+
238+extern const char *ceph_session_op_name(int op);
239+
240+struct ceph_mds_session_head {
241+ __le32 op;
242+ __le64 seq;
243+ struct ceph_timespec stamp;
244+ __le32 max_caps, max_leases;
245+} __attribute__ ((packed));
246+
247+/* client_request */
248+/*
249+ * metadata ops.
250+ * & 0x001000 -> write op
251+ * & 0x010000 -> follow symlink (e.g. stat(), not lstat()).
252+ & & 0x100000 -> use weird ino/path trace
253+ */
254+#define CEPH_MDS_OP_WRITE 0x001000
255+enum {
256+ CEPH_MDS_OP_LOOKUP = 0x00100,
257+ CEPH_MDS_OP_GETATTR = 0x00101,
258+ CEPH_MDS_OP_LOOKUPHASH = 0x00102,
259+ CEPH_MDS_OP_LOOKUPPARENT = 0x00103,
260+
261+ CEPH_MDS_OP_SETXATTR = 0x01105,
262+ CEPH_MDS_OP_RMXATTR = 0x01106,
263+ CEPH_MDS_OP_SETLAYOUT = 0x01107,
264+ CEPH_MDS_OP_SETATTR = 0x01108,
265+
266+ CEPH_MDS_OP_MKNOD = 0x01201,
267+ CEPH_MDS_OP_LINK = 0x01202,
268+ CEPH_MDS_OP_UNLINK = 0x01203,
269+ CEPH_MDS_OP_RENAME = 0x01204,
270+ CEPH_MDS_OP_MKDIR = 0x01220,
271+ CEPH_MDS_OP_RMDIR = 0x01221,
272+ CEPH_MDS_OP_SYMLINK = 0x01222,
273+
274+ CEPH_MDS_OP_CREATE = 0x01301,
275+ CEPH_MDS_OP_OPEN = 0x00302,
276+ CEPH_MDS_OP_READDIR = 0x00305,
277+
278+ CEPH_MDS_OP_LOOKUPSNAP = 0x00400,
279+ CEPH_MDS_OP_MKSNAP = 0x01400,
280+ CEPH_MDS_OP_RMSNAP = 0x01401,
281+ CEPH_MDS_OP_LSSNAP = 0x00402,
282+};
283+
284+extern const char *ceph_mds_op_name(int op);
285+
286+
287+#define CEPH_SETATTR_MODE 1
288+#define CEPH_SETATTR_UID 2
289+#define CEPH_SETATTR_GID 4
290+#define CEPH_SETATTR_MTIME 8
291+#define CEPH_SETATTR_ATIME 16
292+#define CEPH_SETATTR_SIZE 32
293+#define CEPH_SETATTR_CTIME 64
294+
295+union ceph_mds_request_args {
296+ struct {
297+ __le32 mask; /* CEPH_CAP_* */
298+ } __attribute__ ((packed)) getattr;
299+ struct {
300+ __le32 mode;
301+ __le32 uid;
302+ __le32 gid;
303+ struct ceph_timespec mtime;
304+ struct ceph_timespec atime;
305+ __le64 size, old_size; /* old_size needed by truncate */
306+ __le32 mask; /* CEPH_SETATTR_* */
307+ } __attribute__ ((packed)) setattr;
308+ struct {
309+ __le32 frag; /* which dir fragment */
310+ __le32 max_entries; /* how many dentries to grab */
311+ } __attribute__ ((packed)) readdir;
312+ struct {
313+ __le32 mode;
314+ __le32 rdev;
315+ } __attribute__ ((packed)) mknod;
316+ struct {
317+ __le32 mode;
318+ } __attribute__ ((packed)) mkdir;
319+ struct {
320+ __le32 flags;
321+ __le32 mode;
322+ __le32 stripe_unit; /* layout for newly created file */
323+ __le32 stripe_count; /* ... */
324+ __le32 object_size;
325+ __le32 file_replication;
326+ __le32 preferred;
327+ } __attribute__ ((packed)) open;
328+ struct {
329+ __le32 flags;
330+ } __attribute__ ((packed)) setxattr;
331+ struct {
332+ struct ceph_file_layout layout;
333+ } __attribute__ ((packed)) setlayout;
334+} __attribute__ ((packed));
335+
336+#define CEPH_MDS_FLAG_REPLAY 1 /* this is a replayed op */
337+#define CEPH_MDS_FLAG_WANT_DENTRY 2 /* want dentry in reply */
338+
339+struct ceph_mds_request_head {
340+ __le64 oldest_client_tid;
341+ __le32 mdsmap_epoch; /* on client */
342+ __le32 flags; /* CEPH_MDS_FLAG_* */
343+ __u8 num_retry, num_fwd; /* count retry, fwd attempts */
344+ __le16 num_releases; /* # include cap/lease release records */
345+ __le32 op; /* mds op code */
346+ __le32 caller_uid, caller_gid;
347+ __le64 ino; /* use this ino for openc, mkdir, mknod,
348+ etc. (if replaying) */
349+ union ceph_mds_request_args args;
350+} __attribute__ ((packed));
351+
352+/* cap/lease release record */
353+struct ceph_mds_request_release {
354+ __le64 ino, cap_id; /* ino and unique cap id */
355+ __le32 caps, wanted; /* new issued, wanted */
356+ __le32 seq, issue_seq, mseq;
357+ __le32 dname_seq; /* if releasing a dentry lease, a */
358+ __le32 dname_len; /* string follows. */
359+} __attribute__ ((packed));
360+
361+/* client reply */
362+struct ceph_mds_reply_head {
363+ __le32 op;
364+ __le32 result;
365+ __le32 mdsmap_epoch;
366+ __u8 safe; /* true if committed to disk */
367+ __u8 is_dentry, is_target; /* true if dentry, target inode records
368+ are included with reply */
369+} __attribute__ ((packed));
370+
371+/* one for each node split */
372+struct ceph_frag_tree_split {
373+ __le32 frag; /* this frag splits... */
374+ __le32 by; /* ...by this many bits */
375+} __attribute__ ((packed));
376+
377+struct ceph_frag_tree_head {
378+ __le32 nsplits; /* num ceph_frag_tree_split records */
379+ struct ceph_frag_tree_split splits[];
380+} __attribute__ ((packed));
381+
382+/* capability issue, for bundling with mds reply */
383+struct ceph_mds_reply_cap {
384+ __le32 caps, wanted; /* caps issued, wanted */
385+ __le64 cap_id;
386+ __le32 seq, mseq;
387+ __le64 realm; /* snap realm */
388+ __u8 flags; /* CEPH_CAP_FLAG_* */
389+} __attribute__ ((packed));
390+
391+#define CEPH_CAP_FLAG_AUTH 1 /* cap is issued by auth mds */
392+
393+/* inode record, for bundling with mds reply */
394+struct ceph_mds_reply_inode {
395+ __le64 ino;
396+ __le64 snapid;
397+ __le32 rdev;
398+ __le64 version; /* inode version */
399+ __le64 xattr_version; /* version for xattr blob */
400+ struct ceph_mds_reply_cap cap; /* caps issued for this inode */
401+ struct ceph_file_layout layout;
402+ struct ceph_timespec ctime, mtime, atime;
403+ __le32 time_warp_seq;
404+ __le64 size, max_size, truncate_size;
405+ __le32 truncate_seq;
406+ __le32 mode, uid, gid;
407+ __le32 nlink;
408+ __le64 files, subdirs, rbytes, rfiles, rsubdirs; /* dir stats */
409+ struct ceph_timespec rctime;
410+ struct ceph_frag_tree_head fragtree; /* (must be at end of struct) */
411+} __attribute__ ((packed));
412+/* followed by frag array, then symlink string, then xattr blob */
413+
414+/* reply_lease follows dname, and reply_inode */
415+struct ceph_mds_reply_lease {
416+ __le16 mask; /* lease type(s) */
417+ __le32 duration_ms; /* lease duration */
418+ __le32 seq;
419+} __attribute__ ((packed));
420+
421+struct ceph_mds_reply_dirfrag {
422+ __le32 frag; /* fragment */
423+ __le32 auth; /* auth mds, if this is a delegation point */
424+ __le32 ndist; /* number of mds' this is replicated on */
425+ __le32 dist[];
426+} __attribute__ ((packed));
427+
428+/* file access modes */
429+#define CEPH_FILE_MODE_PIN 0
430+#define CEPH_FILE_MODE_RD 1
431+#define CEPH_FILE_MODE_WR 2
432+#define CEPH_FILE_MODE_RDWR 3 /* RD | WR */
433+#define CEPH_FILE_MODE_LAZY 4 /* lazy io */
434+#define CEPH_FILE_MODE_NUM 8 /* bc these are bit fields.. mostly */
435+
436+int ceph_flags_to_mode(int flags);
437+
438+
439+/* capability bits */
440+#define CEPH_CAP_PIN 1 /* no specific capabilities beyond the pin */
441+
442+/* generic cap bits */
443+#define CEPH_CAP_GSHARED 1 /* client can reads */
444+#define CEPH_CAP_GEXCL 2 /* client can read and update */
445+#define CEPH_CAP_GCACHE 4 /* (file) client can cache reads */
446+#define CEPH_CAP_GRD 8 /* (file) client can read */
447+#define CEPH_CAP_GWR 16 /* (file) client can write */
448+#define CEPH_CAP_GBUFFER 32 /* (file) client can buffer writes */
449+#define CEPH_CAP_GWREXTEND 64 /* (file) client can extend EOF */
450+#define CEPH_CAP_GLAZYIO 128 /* (file) client can perform lazy io */
451+
452+/* per-lock shift */
453+#define CEPH_CAP_SAUTH 2
454+#define CEPH_CAP_SLINK 4
455+#define CEPH_CAP_SXATTR 6
456+#define CEPH_CAP_SFILE 8 /* goes at the end (uses >2 cap bits) */
457+
458+#define CEPH_CAP_BITS 16
459+
460+/* composed values */
461+#define CEPH_CAP_AUTH_SHARED (CEPH_CAP_GSHARED << CEPH_CAP_SAUTH)
462+#define CEPH_CAP_AUTH_EXCL (CEPH_CAP_GEXCL << CEPH_CAP_SAUTH)
463+#define CEPH_CAP_LINK_SHARED (CEPH_CAP_GSHARED << CEPH_CAP_SLINK)
464+#define CEPH_CAP_LINK_EXCL (CEPH_CAP_GEXCL << CEPH_CAP_SLINK)
465+#define CEPH_CAP_XATTR_SHARED (CEPH_CAP_GSHARED << CEPH_CAP_SXATTR)
466+#define CEPH_CAP_XATTR_EXCL (CEPH_CAP_GEXCL << CEPH_CAP_SXATTR)
467+#define CEPH_CAP_FILE(x) (x << CEPH_CAP_SFILE)
468+#define CEPH_CAP_FILE_SHARED (CEPH_CAP_GSHARED << CEPH_CAP_SFILE)
469+#define CEPH_CAP_FILE_EXCL (CEPH_CAP_GEXCL << CEPH_CAP_SFILE)
470+#define CEPH_CAP_FILE_CACHE (CEPH_CAP_GCACHE << CEPH_CAP_SFILE)
471+#define CEPH_CAP_FILE_RD (CEPH_CAP_GRD << CEPH_CAP_SFILE)
472+#define CEPH_CAP_FILE_WR (CEPH_CAP_GWR << CEPH_CAP_SFILE)
473+#define CEPH_CAP_FILE_BUFFER (CEPH_CAP_GBUFFER << CEPH_CAP_SFILE)
474+#define CEPH_CAP_FILE_WREXTEND (CEPH_CAP_GWREXTEND << CEPH_CAP_SFILE)
475+#define CEPH_CAP_FILE_LAZYIO (CEPH_CAP_GLAZYIO << CEPH_CAP_SFILE)
476+
477+/* cap masks (for getattr) */
478+#define CEPH_STAT_CAP_INODE CEPH_CAP_PIN
479+#define CEPH_STAT_CAP_TYPE CEPH_CAP_PIN /* mode >> 12 */
480+#define CEPH_STAT_CAP_SYMLINK CEPH_CAP_PIN
481+#define CEPH_STAT_CAP_UID CEPH_CAP_AUTH_SHARED
482+#define CEPH_STAT_CAP_GID CEPH_CAP_AUTH_SHARED
483+#define CEPH_STAT_CAP_MODE CEPH_CAP_AUTH_SHARED
484+#define CEPH_STAT_CAP_NLINK CEPH_CAP_LINK_SHARED
485+#define CEPH_STAT_CAP_LAYOUT CEPH_CAP_FILE_SHARED
486+#define CEPH_STAT_CAP_MTIME CEPH_CAP_FILE_SHARED
487+#define CEPH_STAT_CAP_SIZE CEPH_CAP_FILE_SHARED
488+#define CEPH_STAT_CAP_ATIME CEPH_CAP_FILE_SHARED /* fixme */
489+#define CEPH_STAT_CAP_XATTR CEPH_CAP_XATTR_SHARED
490+#define CEPH_STAT_CAP_INODE_ALL (CEPH_CAP_PIN | \
491+ CEPH_CAP_AUTH_SHARED | \
492+ CEPH_CAP_LINK_SHARED | \
493+ CEPH_CAP_FILE_SHARED | \
494+ CEPH_CAP_XATTR_SHARED)
495+
496+#define CEPH_CAP_ANY_SHARED (CEPH_CAP_AUTH_SHARED | \
497+ CEPH_CAP_LINK_SHARED | \
498+ CEPH_CAP_XATTR_SHARED | \
499+ CEPH_CAP_FILE_SHARED)
500+#define CEPH_CAP_ANY_RD (CEPH_CAP_ANY_SHARED | CEPH_CAP_FILE_RD | \
501+ CEPH_CAP_FILE_CACHE)
502+
503+#define CEPH_CAP_ANY_EXCL (CEPH_CAP_AUTH_EXCL | \
504+ CEPH_CAP_LINK_EXCL | \
505+ CEPH_CAP_XATTR_EXCL | \
506+ CEPH_CAP_FILE_EXCL)
507+#define CEPH_CAP_ANY_FILE_WR (CEPH_CAP_FILE_WR | CEPH_CAP_FILE_BUFFER | \
508+ CEPH_CAP_FILE_EXCL)
509+#define CEPH_CAP_ANY_WR (CEPH_CAP_ANY_EXCL | CEPH_CAP_ANY_FILE_WR)
510+#define CEPH_CAP_ANY (CEPH_CAP_ANY_RD | CEPH_CAP_ANY_EXCL | \
511+ CEPH_CAP_ANY_FILE_WR | CEPH_CAP_PIN)
512+
513+#define CEPH_CAP_LOCKS (CEPH_LOCK_IFILE | CEPH_LOCK_IAUTH | CEPH_LOCK_ILINK | \
514+ CEPH_LOCK_IXATTR)
515+
516+int ceph_caps_for_mode(int mode);
517+
518+enum {
519+ CEPH_CAP_OP_GRANT, /* mds->client grant */
520+ CEPH_CAP_OP_REVOKE, /* mds->client revoke */
521+ CEPH_CAP_OP_TRUNC, /* mds->client trunc notify */
522+ CEPH_CAP_OP_EXPORT, /* mds has exported the cap */
523+ CEPH_CAP_OP_IMPORT, /* mds has imported the cap */
524+ CEPH_CAP_OP_UPDATE, /* client->mds update */
525+ CEPH_CAP_OP_DROP, /* client->mds drop cap bits */
526+ CEPH_CAP_OP_FLUSH, /* client->mds cap writeback */
527+ CEPH_CAP_OP_FLUSH_ACK, /* mds->client flushed */
528+ CEPH_CAP_OP_FLUSHSNAP, /* client->mds flush snapped metadata */
529+ CEPH_CAP_OP_FLUSHSNAP_ACK, /* mds->client flushed snapped metadata */
530+ CEPH_CAP_OP_RELEASE, /* client->mds release (clean) cap */
531+ CEPH_CAP_OP_RENEW, /* client->mds renewal request */
532+};
533+
534+extern const char *ceph_cap_op_name(int op);
535+
536+/*
537+ * caps message, used for capability callbacks, acks, requests, etc.
538+ */
539+struct ceph_mds_caps {
540+ __le32 op; /* CEPH_CAP_OP_* */
541+ __le64 ino, realm;
542+ __le64 cap_id;
543+ __le32 seq, issue_seq;
544+ __le32 caps, wanted, dirty; /* latest issued/wanted/dirty */
545+ __le32 migrate_seq;
546+ __le64 snap_follows;
547+ __le32 snap_trace_len;
548+
549+ /* authlock */
550+ __le32 uid, gid, mode;
551+
552+ /* linklock */
553+ __le32 nlink;
554+
555+ /* xattrlock */
556+ __le32 xattr_len;
557+ __le64 xattr_version;
558+
559+ /* filelock */
560+ __le64 size, max_size, truncate_size;
561+ __le32 truncate_seq;
562+ struct ceph_timespec mtime, atime, ctime;
563+ struct ceph_file_layout layout;
564+ __le32 time_warp_seq;
565+} __attribute__ ((packed));
566+
567+/* cap release msg head */
568+struct ceph_mds_cap_release {
569+ __le32 num; /* number of cap_items that follow */
570+} __attribute__ ((packed));
571+
572+struct ceph_mds_cap_item {
573+ __le64 ino;
574+ __le64 cap_id;
575+ __le32 migrate_seq, seq;
576+} __attribute__ ((packed));
577+
578+#define CEPH_MDS_LEASE_REVOKE 1 /* mds -> client */
579+#define CEPH_MDS_LEASE_RELEASE 2 /* client -> mds */
580+#define CEPH_MDS_LEASE_RENEW 3 /* client <-> mds */
581+#define CEPH_MDS_LEASE_REVOKE_ACK 4 /* client -> mds */
582+
583+extern const char *ceph_lease_op_name(int o);
584+
585+/* lease msg header */
586+struct ceph_mds_lease {
587+ __u8 action; /* CEPH_MDS_LEASE_* */
588+ __le16 mask; /* which lease */
589+ __le64 ino;
590+ __le64 first, last; /* snap range */
591+ __le32 seq;
592+ __le32 duration_ms; /* duration of renewal */
593+} __attribute__ ((packed));
594+/* followed by a __le32+string for dname */
595+
596+/* client reconnect */
597+struct ceph_mds_cap_reconnect {
598+ __le64 cap_id;
599+ __le32 wanted;
600+ __le32 issued;
601+ __le64 size;
602+ struct ceph_timespec mtime, atime;
603+ __le64 snaprealm;
604+ __le64 pathbase; /* base ino for our path to this ino */
605+} __attribute__ ((packed));
606+/* followed by encoded string */
607+
608+struct ceph_mds_snaprealm_reconnect {
609+ __le64 ino; /* snap realm base */
610+ __le64 seq; /* snap seq for this snap realm */
611+ __le64 parent; /* parent realm */
612+} __attribute__ ((packed));
613+
614+/*
615+ * snaps
616+ */
617+enum {
618+ CEPH_SNAP_OP_UPDATE, /* CREATE or DESTROY */
619+ CEPH_SNAP_OP_CREATE,
620+ CEPH_SNAP_OP_DESTROY,
621+ CEPH_SNAP_OP_SPLIT,
622+};
623+
624+extern const char *ceph_snap_op_name(int o);
625+
626+/* snap msg header */
627+struct ceph_mds_snap_head {
628+ __le32 op; /* CEPH_SNAP_OP_* */
629+ __le64 split; /* ino to split off, if any */
630+ __le32 num_split_inos; /* # inos belonging to new child realm */
631+ __le32 num_split_realms; /* # child realms udner new child realm */
632+ __le32 trace_len; /* size of snap trace blob */
633+} __attribute__ ((packed));
634+/* followed by split ino list, then split realms, then the trace blob */
635+
636+/*
637+ * encode info about a snaprealm, as viewed by a client
638+ */
639+struct ceph_mds_snap_realm {
640+ __le64 ino; /* ino */
641+ __le64 created; /* snap: when created */
642+ __le64 parent; /* ino: parent realm */
643+ __le64 parent_since; /* snap: same parent since */
644+ __le64 seq; /* snap: version */
645+ __le32 num_snaps;
646+ __le32 num_prior_parent_snaps;
647+} __attribute__ ((packed));
648+/* followed by my snap list, then prior parent snap list */
649+
650+#endif
fs/ceph/ceph_hash.c+118−0
@@ -0,0 +1,118 @@
1+
2+#include "types.h"
3+
4+/*
5+ * Robert Jenkin's hash function.
6+ * http://burtleburtle.net/bob/hash/evahash.html
7+ * This is in the public domain.
8+ */
9+#define mix(a, b, c) \
10+ do { \
11+ a = a - b; a = a - c; a = a ^ (c >> 13); \
12+ b = b - c; b = b - a; b = b ^ (a << 8); \
13+ c = c - a; c = c - b; c = c ^ (b >> 13); \
14+ a = a - b; a = a - c; a = a ^ (c >> 12); \
15+ b = b - c; b = b - a; b = b ^ (a << 16); \
16+ c = c - a; c = c - b; c = c ^ (b >> 5); \
17+ a = a - b; a = a - c; a = a ^ (c >> 3); \
18+ b = b - c; b = b - a; b = b ^ (a << 10); \
19+ c = c - a; c = c - b; c = c ^ (b >> 15); \
20+ } while (0)
21+
22+unsigned ceph_str_hash_rjenkins(const char *str, unsigned length)
23+{
24+ const unsigned char *k = (const unsigned char *)str;
25+ __u32 a, b, c; /* the internal state */
26+ __u32 len; /* how many key bytes still need mixing */
27+
28+ /* Set up the internal state */
29+ len = length;
30+ a = 0x9e3779b9; /* the golden ratio; an arbitrary value */
31+ b = a;
32+ c = 0; /* variable initialization of internal state */
33+
34+ /* handle most of the key */
35+ while (len >= 12) {
36+ a = a + (k[0] + ((__u32)k[1] << 8) + ((__u32)k[2] << 16) +
37+ ((__u32)k[3] << 24));
38+ b = b + (k[4] + ((__u32)k[5] << 8) + ((__u32)k[6] << 16) +
39+ ((__u32)k[7] << 24));
40+ c = c + (k[8] + ((__u32)k[9] << 8) + ((__u32)k[10] << 16) +
41+ ((__u32)k[11] << 24));
42+ mix(a, b, c);
43+ k = k + 12;
44+ len = len - 12;
45+ }
46+
47+ /* handle the last 11 bytes */
48+ c = c + length;
49+ switch (len) { /* all the case statements fall through */
50+ case 11:
51+ c = c + ((__u32)k[10] << 24);
52+ case 10:
53+ c = c + ((__u32)k[9] << 16);
54+ case 9:
55+ c = c + ((__u32)k[8] << 8);
56+ /* the first byte of c is reserved for the length */
57+ case 8:
58+ b = b + ((__u32)k[7] << 24);
59+ case 7:
60+ b = b + ((__u32)k[6] << 16);
61+ case 6:
62+ b = b + ((__u32)k[5] << 8);
63+ case 5:
64+ b = b + k[4];
65+ case 4:
66+ a = a + ((__u32)k[3] << 24);
67+ case 3:
68+ a = a + ((__u32)k[2] << 16);
69+ case 2:
70+ a = a + ((__u32)k[1] << 8);
71+ case 1:
72+ a = a + k[0];
73+ /* case 0: nothing left to add */
74+ }
75+ mix(a, b, c);
76+
77+ return c;
78+}
79+
80+/*
81+ * linux dcache hash
82+ */
83+unsigned ceph_str_hash_linux(const char *str, unsigned length)
84+{
85+ unsigned long hash = 0;
86+ unsigned char c;
87+
88+ while (length--) {
89+ c = *str++;
90+ hash = (hash + (c << 4) + (c >> 4)) * 11;
91+ }
92+ return hash;
93+}
94+
95+
96+unsigned ceph_str_hash(int type, const char *s, unsigned len)
97+{
98+ switch (type) {
99+ case CEPH_STR_HASH_LINUX:
100+ return ceph_str_hash_linux(s, len);
101+ case CEPH_STR_HASH_RJENKINS:
102+ return ceph_str_hash_rjenkins(s, len);
103+ default:
104+ return -1;
105+ }
106+}
107+
108+const char *ceph_str_hash_name(int type)
109+{
110+ switch (type) {
111+ case CEPH_STR_HASH_LINUX:
112+ return "linux";
113+ case CEPH_STR_HASH_RJENKINS:
114+ return "rjenkins";
115+ default:
116+ return "unknown";
117+ }
118+}
fs/ceph/ceph_hash.h+13−0
@@ -0,0 +1,13 @@
1+#ifndef _FS_CEPH_HASH_H
2+#define _FS_CEPH_HASH_H
3+
4+#define CEPH_STR_HASH_LINUX 0x1 /* linux dcache hash */
5+#define CEPH_STR_HASH_RJENKINS 0x2 /* robert jenkins' */
6+
7+extern unsigned ceph_str_hash_linux(const char *s, unsigned len);
8+extern unsigned ceph_str_hash_rjenkins(const char *s, unsigned len);
9+
10+extern unsigned ceph_str_hash(int type, const char *s, unsigned len);
11+extern const char *ceph_str_hash_name(int type);
12+
13+#endif
fs/ceph/ceph_strings.c+176−0
@@ -0,0 +1,176 @@
1+/*
2+ * Ceph string constants
3+ */
4+#include "types.h"
5+
6+const char *ceph_entity_type_name(int type)
7+{
8+ switch (type) {
9+ case CEPH_ENTITY_TYPE_MDS: return "mds";
10+ case CEPH_ENTITY_TYPE_OSD: return "osd";
11+ case CEPH_ENTITY_TYPE_MON: return "mon";
12+ case CEPH_ENTITY_TYPE_CLIENT: return "client";
13+ case CEPH_ENTITY_TYPE_ADMIN: return "admin";
14+ case CEPH_ENTITY_TYPE_AUTH: return "auth";
15+ default: return "unknown";
16+ }
17+}
18+
19+const char *ceph_osd_op_name(int op)
20+{
21+ switch (op) {
22+ case CEPH_OSD_OP_READ: return "read";
23+ case CEPH_OSD_OP_STAT: return "stat";
24+
25+ case CEPH_OSD_OP_MASKTRUNC: return "masktrunc";
26+
27+ case CEPH_OSD_OP_WRITE: return "write";
28+ case CEPH_OSD_OP_DELETE: return "delete";
29+ case CEPH_OSD_OP_TRUNCATE: return "truncate";
30+ case CEPH_OSD_OP_ZERO: return "zero";
31+ case CEPH_OSD_OP_WRITEFULL: return "writefull";
32+
33+ case CEPH_OSD_OP_APPEND: return "append";
34+ case CEPH_OSD_OP_STARTSYNC: return "startsync";
35+ case CEPH_OSD_OP_SETTRUNC: return "settrunc";
36+ case CEPH_OSD_OP_TRIMTRUNC: return "trimtrunc";
37+
38+ case CEPH_OSD_OP_TMAPUP: return "tmapup";
39+ case CEPH_OSD_OP_TMAPGET: return "tmapget";
40+ case CEPH_OSD_OP_TMAPPUT: return "tmapput";
41+
42+ case CEPH_OSD_OP_GETXATTR: return "getxattr";
43+ case CEPH_OSD_OP_GETXATTRS: return "getxattrs";
44+ case CEPH_OSD_OP_SETXATTR: return "setxattr";
45+ case CEPH_OSD_OP_SETXATTRS: return "setxattrs";
46+ case CEPH_OSD_OP_RESETXATTRS: return "resetxattrs";
47+ case CEPH_OSD_OP_RMXATTR: return "rmxattr";
48+
49+ case CEPH_OSD_OP_PULL: return "pull";
50+ case CEPH_OSD_OP_PUSH: return "push";
51+ case CEPH_OSD_OP_BALANCEREADS: return "balance-reads";
52+ case CEPH_OSD_OP_UNBALANCEREADS: return "unbalance-reads";
53+ case CEPH_OSD_OP_SCRUB: return "scrub";
54+
55+ case CEPH_OSD_OP_WRLOCK: return "wrlock";
56+ case CEPH_OSD_OP_WRUNLOCK: return "wrunlock";
57+ case CEPH_OSD_OP_RDLOCK: return "rdlock";
58+ case CEPH_OSD_OP_RDUNLOCK: return "rdunlock";
59+ case CEPH_OSD_OP_UPLOCK: return "uplock";
60+ case CEPH_OSD_OP_DNLOCK: return "dnlock";
61+
62+ case CEPH_OSD_OP_CALL: return "call";
63+
64+ case CEPH_OSD_OP_PGLS: return "pgls";
65+ }
66+ return "???";
67+}
68+
69+const char *ceph_mds_state_name(int s)
70+{
71+ switch (s) {
72+ /* down and out */
73+ case CEPH_MDS_STATE_DNE: return "down:dne";
74+ case CEPH_MDS_STATE_STOPPED: return "down:stopped";
75+ /* up and out */
76+ case CEPH_MDS_STATE_BOOT: return "up:boot";
77+ case CEPH_MDS_STATE_STANDBY: return "up:standby";
78+ case CEPH_MDS_STATE_STANDBY_REPLAY: return "up:standby-replay";
79+ case CEPH_MDS_STATE_CREATING: return "up:creating";
80+ case CEPH_MDS_STATE_STARTING: return "up:starting";
81+ /* up and in */
82+ case CEPH_MDS_STATE_REPLAY: return "up:replay";
83+ case CEPH_MDS_STATE_RESOLVE: return "up:resolve";
84+ case CEPH_MDS_STATE_RECONNECT: return "up:reconnect";
85+ case CEPH_MDS_STATE_REJOIN: return "up:rejoin";
86+ case CEPH_MDS_STATE_CLIENTREPLAY: return "up:clientreplay";
87+ case CEPH_MDS_STATE_ACTIVE: return "up:active";
88+ case CEPH_MDS_STATE_STOPPING: return "up:stopping";
89+ }
90+ return "???";
91+}
92+
93+const char *ceph_session_op_name(int op)
94+{
95+ switch (op) {
96+ case CEPH_SESSION_REQUEST_OPEN: return "request_open";
97+ case CEPH_SESSION_OPEN: return "open";
98+ case CEPH_SESSION_REQUEST_CLOSE: return "request_close";
99+ case CEPH_SESSION_CLOSE: return "close";
100+ case CEPH_SESSION_REQUEST_RENEWCAPS: return "request_renewcaps";
101+ case CEPH_SESSION_RENEWCAPS: return "renewcaps";
102+ case CEPH_SESSION_STALE: return "stale";
103+ case CEPH_SESSION_RECALL_STATE: return "recall_state";
104+ }
105+ return "???";
106+}
107+
108+const char *ceph_mds_op_name(int op)
109+{
110+ switch (op) {
111+ case CEPH_MDS_OP_LOOKUP: return "lookup";
112+ case CEPH_MDS_OP_LOOKUPHASH: return "lookuphash";
113+ case CEPH_MDS_OP_LOOKUPPARENT: return "lookupparent";
114+ case CEPH_MDS_OP_GETATTR: return "getattr";
115+ case CEPH_MDS_OP_SETXATTR: return "setxattr";
116+ case CEPH_MDS_OP_SETATTR: return "setattr";
117+ case CEPH_MDS_OP_RMXATTR: return "rmxattr";
118+ case CEPH_MDS_OP_READDIR: return "readdir";
119+ case CEPH_MDS_OP_MKNOD: return "mknod";
120+ case CEPH_MDS_OP_LINK: return "link";
121+ case CEPH_MDS_OP_UNLINK: return "unlink";
122+ case CEPH_MDS_OP_RENAME: return "rename";
123+ case CEPH_MDS_OP_MKDIR: return "mkdir";
124+ case CEPH_MDS_OP_RMDIR: return "rmdir";
125+ case CEPH_MDS_OP_SYMLINK: return "symlink";
126+ case CEPH_MDS_OP_CREATE: return "create";
127+ case CEPH_MDS_OP_OPEN: return "open";
128+ case CEPH_MDS_OP_LOOKUPSNAP: return "lookupsnap";
129+ case CEPH_MDS_OP_LSSNAP: return "lssnap";
130+ case CEPH_MDS_OP_MKSNAP: return "mksnap";
131+ case CEPH_MDS_OP_RMSNAP: return "rmsnap";
132+ }
133+ return "???";
134+}
135+
136+const char *ceph_cap_op_name(int op)
137+{
138+ switch (op) {
139+ case CEPH_CAP_OP_GRANT: return "grant";
140+ case CEPH_CAP_OP_REVOKE: return "revoke";
141+ case CEPH_CAP_OP_TRUNC: return "trunc";
142+ case CEPH_CAP_OP_EXPORT: return "export";
143+ case CEPH_CAP_OP_IMPORT: return "import";
144+ case CEPH_CAP_OP_UPDATE: return "update";
145+ case CEPH_CAP_OP_DROP: return "drop";
146+ case CEPH_CAP_OP_FLUSH: return "flush";
147+ case CEPH_CAP_OP_FLUSH_ACK: return "flush_ack";
148+ case CEPH_CAP_OP_FLUSHSNAP: return "flushsnap";
149+ case CEPH_CAP_OP_FLUSHSNAP_ACK: return "flushsnap_ack";
150+ case CEPH_CAP_OP_RELEASE: return "release";
151+ case CEPH_CAP_OP_RENEW: return "renew";
152+ }
153+ return "???";
154+}
155+
156+const char *ceph_lease_op_name(int o)
157+{
158+ switch (o) {
159+ case CEPH_MDS_LEASE_REVOKE: return "revoke";
160+ case CEPH_MDS_LEASE_RELEASE: return "release";
161+ case CEPH_MDS_LEASE_RENEW: return "renew";
162+ case CEPH_MDS_LEASE_REVOKE_ACK: return "revoke_ack";
163+ }
164+ return "???";
165+}
166+
167+const char *ceph_snap_op_name(int o)
168+{
169+ switch (o) {
170+ case CEPH_SNAP_OP_UPDATE: return "update";
171+ case CEPH_SNAP_OP_CREATE: return "create";
172+ case CEPH_SNAP_OP_DESTROY: return "destroy";
173+ case CEPH_SNAP_OP_SPLIT: return "split";
174+ }
175+ return "???";
176+}
fs/ceph/crush/crush.c+151−0
@@ -0,0 +1,151 @@
1+
2+#ifdef __KERNEL__
3+# include <linux/slab.h>
4+#else
5+# include <stdlib.h>
6+# include <assert.h>
7+# define kfree(x) do { if (x) free(x); } while (0)
8+# define BUG_ON(x) assert(!(x))
9+#endif
10+
11+#include "crush.h"
12+
13+const char *crush_bucket_alg_name(int alg)
14+{
15+ switch (alg) {
16+ case CRUSH_BUCKET_UNIFORM: return "uniform";
17+ case CRUSH_BUCKET_LIST: return "list";
18+ case CRUSH_BUCKET_TREE: return "tree";
19+ case CRUSH_BUCKET_STRAW: return "straw";
20+ default: return "unknown";
21+ }
22+}
23+
24+/**
25+ * crush_get_bucket_item_weight - Get weight of an item in given bucket
26+ * @b: bucket pointer
27+ * @p: item index in bucket
28+ */
29+int crush_get_bucket_item_weight(struct crush_bucket *b, int p)
30+{
31+ if (p >= b->size)
32+ return 0;
33+
34+ switch (b->alg) {
35+ case CRUSH_BUCKET_UNIFORM:
36+ return ((struct crush_bucket_uniform *)b)->item_weight;
37+ case CRUSH_BUCKET_LIST:
38+ return ((struct crush_bucket_list *)b)->item_weights[p];
39+ case CRUSH_BUCKET_TREE:
40+ if (p & 1)
41+ return ((struct crush_bucket_tree *)b)->node_weights[p];
42+ return 0;
43+ case CRUSH_BUCKET_STRAW:
44+ return ((struct crush_bucket_straw *)b)->item_weights[p];
45+ }
46+ return 0;
47+}
48+
49+/**
50+ * crush_calc_parents - Calculate parent vectors for the given crush map.
51+ * @map: crush_map pointer
52+ */
53+void crush_calc_parents(struct crush_map *map)
54+{
55+ int i, b, c;
56+
57+ for (b = 0; b < map->max_buckets; b++) {
58+ if (map->buckets[b] == NULL)
59+ continue;
60+ for (i = 0; i < map->buckets[b]->size; i++) {
61+ c = map->buckets[b]->items[i];
62+ BUG_ON(c >= map->max_devices ||
63+ c < -map->max_buckets);
64+ if (c >= 0)
65+ map->device_parents[c] = map->buckets[b]->id;
66+ else
67+ map->bucket_parents[-1-c] = map->buckets[b]->id;
68+ }
69+ }
70+}
71+
72+void crush_destroy_bucket_uniform(struct crush_bucket_uniform *b)
73+{
74+ kfree(b->h.perm);
75+ kfree(b->h.items);
76+ kfree(b);
77+}
78+
79+void crush_destroy_bucket_list(struct crush_bucket_list *b)
80+{
81+ kfree(b->item_weights);
82+ kfree(b->sum_weights);
83+ kfree(b->h.perm);
84+ kfree(b->h.items);
85+ kfree(b);
86+}
87+
88+void crush_destroy_bucket_tree(struct crush_bucket_tree *b)
89+{
90+ kfree(b->node_weights);
91+ kfree(b);
92+}
93+
94+void crush_destroy_bucket_straw(struct crush_bucket_straw *b)
95+{
96+ kfree(b->straws);
97+ kfree(b->item_weights);
98+ kfree(b->h.perm);
99+ kfree(b->h.items);
100+ kfree(b);
101+}
102+
103+void crush_destroy_bucket(struct crush_bucket *b)
104+{
105+ switch (b->alg) {
106+ case CRUSH_BUCKET_UNIFORM:
107+ crush_destroy_bucket_uniform((struct crush_bucket_uniform *)b);
108+ break;
109+ case CRUSH_BUCKET_LIST:
110+ crush_destroy_bucket_list((struct crush_bucket_list *)b);
111+ break;
112+ case CRUSH_BUCKET_TREE:
113+ crush_destroy_bucket_tree((struct crush_bucket_tree *)b);
114+ break;
115+ case CRUSH_BUCKET_STRAW:
116+ crush_destroy_bucket_straw((struct crush_bucket_straw *)b);
117+ break;
118+ }
119+}
120+
121+/**
122+ * crush_destroy - Destroy a crush_map
123+ * @map: crush_map pointer
124+ */
125+void crush_destroy(struct crush_map *map)
126+{
127+ int b;
128+
129+ /* buckets */
130+ if (map->buckets) {
131+ for (b = 0; b < map->max_buckets; b++) {
132+ if (map->buckets[b] == NULL)
133+ continue;
134+ crush_destroy_bucket(map->buckets[b]);
135+ }
136+ kfree(map->buckets);
137+ }
138+
139+ /* rules */
140+ if (map->rules) {
141+ for (b = 0; b < map->max_rules; b++)
142+ kfree(map->rules[b]);
143+ kfree(map->rules);
144+ }
145+
146+ kfree(map->bucket_parents);
147+ kfree(map->device_parents);
148+ kfree(map);
149+}
150+
151+
fs/ceph/crush/crush.h+180−0
@@ -0,0 +1,180 @@
1+#ifndef _CRUSH_CRUSH_H
2+#define _CRUSH_CRUSH_H
3+
4+#include <linux/types.h>
5+
6+/*
7+ * CRUSH is a pseudo-random data distribution algorithm that
8+ * efficiently distributes input values (typically, data objects)
9+ * across a heterogeneous, structured storage cluster.
10+ *
11+ * The algorithm was originally described in detail in this paper
12+ * (although the algorithm has evolved somewhat since then):
13+ *
14+ * http://www.ssrc.ucsc.edu/Papers/weil-sc06.pdf
15+ *
16+ * LGPL2
17+ */
18+
19+
20+#define CRUSH_MAGIC 0x00010000ul /* for detecting algorithm revisions */
21+
22+
23+#define CRUSH_MAX_DEPTH 10 /* max crush hierarchy depth */
24+#define CRUSH_MAX_SET 10 /* max size of a mapping result */
25+
26+
27+/*
28+ * CRUSH uses user-defined "rules" to describe how inputs should be
29+ * mapped to devices. A rule consists of sequence of steps to perform
30+ * to generate the set of output devices.
31+ */
32+struct crush_rule_step {
33+ __u32 op;
34+ __s32 arg1;
35+ __s32 arg2;
36+};
37+
38+/* step op codes */
39+enum {
40+ CRUSH_RULE_NOOP = 0,
41+ CRUSH_RULE_TAKE = 1, /* arg1 = value to start with */
42+ CRUSH_RULE_CHOOSE_FIRSTN = 2, /* arg1 = num items to pick */
43+ /* arg2 = type */
44+ CRUSH_RULE_CHOOSE_INDEP = 3, /* same */
45+ CRUSH_RULE_EMIT = 4, /* no args */
46+ CRUSH_RULE_CHOOSE_LEAF_FIRSTN = 6,
47+ CRUSH_RULE_CHOOSE_LEAF_INDEP = 7,
48+};
49+
50+/*
51+ * for specifying choose num (arg1) relative to the max parameter
52+ * passed to do_rule
53+ */
54+#define CRUSH_CHOOSE_N 0
55+#define CRUSH_CHOOSE_N_MINUS(x) (-(x))
56+
57+/*
58+ * The rule mask is used to describe what the rule is intended for.
59+ * Given a ruleset and size of output set, we search through the
60+ * rule list for a matching rule_mask.
61+ */
62+struct crush_rule_mask {
63+ __u8 ruleset;
64+ __u8 type;
65+ __u8 min_size;
66+ __u8 max_size;
67+};
68+
69+struct crush_rule {
70+ __u32 len;
71+ struct crush_rule_mask mask;
72+ struct crush_rule_step steps[0];
73+};
74+
75+#define crush_rule_size(len) (sizeof(struct crush_rule) + \
76+ (len)*sizeof(struct crush_rule_step))
77+
78+
79+
80+/*
81+ * A bucket is a named container of other items (either devices or
82+ * other buckets). Items within a bucket are chosen using one of a
83+ * few different algorithms. The table summarizes how the speed of
84+ * each option measures up against mapping stability when items are
85+ * added or removed.
86+ *
87+ * Bucket Alg Speed Additions Removals
88+ * ------------------------------------------------
89+ * uniform O(1) poor poor
90+ * list O(n) optimal poor
91+ * tree O(log n) good good
92+ * straw O(n) optimal optimal
93+ */
94+enum {
95+ CRUSH_BUCKET_UNIFORM = 1,
96+ CRUSH_BUCKET_LIST = 2,
97+ CRUSH_BUCKET_TREE = 3,
98+ CRUSH_BUCKET_STRAW = 4
99+};
100+extern const char *crush_bucket_alg_name(int alg);
101+
102+struct crush_bucket {
103+ __s32 id; /* this'll be negative */
104+ __u16 type; /* non-zero; type=0 is reserved for devices */
105+ __u8 alg; /* one of CRUSH_BUCKET_* */
106+ __u8 hash; /* which hash function to use, CRUSH_HASH_* */
107+ __u32 weight; /* 16-bit fixed point */
108+ __u32 size; /* num items */
109+ __s32 *items;
110+
111+ /*
112+ * cached random permutation: used for uniform bucket and for
113+ * the linear search fallback for the other bucket types.
114+ */
115+ __u32 perm_x; /* @x for which *perm is defined */
116+ __u32 perm_n; /* num elements of *perm that are permuted/defined */
117+ __u32 *perm;
118+};
119+
120+struct crush_bucket_uniform {
121+ struct crush_bucket h;
122+ __u32 item_weight; /* 16-bit fixed point; all items equally weighted */
123+};
124+
125+struct crush_bucket_list {
126+ struct crush_bucket h;
127+ __u32 *item_weights; /* 16-bit fixed point */
128+ __u32 *sum_weights; /* 16-bit fixed point. element i is sum
129+ of weights 0..i, inclusive */
130+};
131+
132+struct crush_bucket_tree {
133+ struct crush_bucket h; /* note: h.size is _tree_ size, not number of
134+ actual items */
135+ __u8 num_nodes;
136+ __u32 *node_weights;
137+};
138+
139+struct crush_bucket_straw {
140+ struct crush_bucket h;
141+ __u32 *item_weights; /* 16-bit fixed point */
142+ __u32 *straws; /* 16-bit fixed point */
143+};
144+
145+
146+
147+/*
148+ * CRUSH map includes all buckets, rules, etc.
149+ */
150+struct crush_map {
151+ struct crush_bucket **buckets;
152+ struct crush_rule **rules;
153+
154+ /*
155+ * Parent pointers to identify the parent bucket a device or
156+ * bucket in the hierarchy. If an item appears more than
157+ * once, this is the _last_ time it appeared (where buckets
158+ * are processed in bucket id order, from -1 on down to
159+ * -max_buckets.
160+ */
161+ __u32 *bucket_parents;
162+ __u32 *device_parents;
163+
164+ __s32 max_buckets;
165+ __u32 max_rules;
166+ __s32 max_devices;
167+};
168+
169+
170+/* crush.c */
171+extern int crush_get_bucket_item_weight(struct crush_bucket *b, int pos);
172+extern void crush_calc_parents(struct crush_map *map);
173+extern void crush_destroy_bucket_uniform(struct crush_bucket_uniform *b);
174+extern void crush_destroy_bucket_list(struct crush_bucket_list *b);
175+extern void crush_destroy_bucket_tree(struct crush_bucket_tree *b);
176+extern void crush_destroy_bucket_straw(struct crush_bucket_straw *b);
177+extern void crush_destroy_bucket(struct crush_bucket *b);
178+extern void crush_destroy(struct crush_map *map);
179+
180+#endif
fs/ceph/crush/hash.c+149−0
@@ -0,0 +1,149 @@
1+
2+#include <linux/types.h>
3+#include "hash.h"
4+
5+/*
6+ * Robert Jenkins' function for mixing 32-bit values
7+ * http://burtleburtle.net/bob/hash/evahash.html
8+ * a, b = random bits, c = input and output
9+ */
10+#define crush_hashmix(a, b, c) do { \
11+ a = a-b; a = a-c; a = a^(c>>13); \
12+ b = b-c; b = b-a; b = b^(a<<8); \
13+ c = c-a; c = c-b; c = c^(b>>13); \
14+ a = a-b; a = a-c; a = a^(c>>12); \
15+ b = b-c; b = b-a; b = b^(a<<16); \
16+ c = c-a; c = c-b; c = c^(b>>5); \
17+ a = a-b; a = a-c; a = a^(c>>3); \
18+ b = b-c; b = b-a; b = b^(a<<10); \
19+ c = c-a; c = c-b; c = c^(b>>15); \
20+ } while (0)
21+
22+#define crush_hash_seed 1315423911
23+
24+static __u32 crush_hash32_rjenkins1(__u32 a)
25+{
26+ __u32 hash = crush_hash_seed ^ a;
27+ __u32 b = a;
28+ __u32 x = 231232;
29+ __u32 y = 1232;
30+ crush_hashmix(b, x, hash);
31+ crush_hashmix(y, a, hash);
32+ return hash;
33+}
34+
35+static __u32 crush_hash32_rjenkins1_2(__u32 a, __u32 b)
36+{
37+ __u32 hash = crush_hash_seed ^ a ^ b;
38+ __u32 x = 231232;
39+ __u32 y = 1232;
40+ crush_hashmix(a, b, hash);
41+ crush_hashmix(x, a, hash);
42+ crush_hashmix(b, y, hash);
43+ return hash;
44+}
45+
46+static __u32 crush_hash32_rjenkins1_3(__u32 a, __u32 b, __u32 c)
47+{
48+ __u32 hash = crush_hash_seed ^ a ^ b ^ c;
49+ __u32 x = 231232;
50+ __u32 y = 1232;
51+ crush_hashmix(a, b, hash);
52+ crush_hashmix(c, x, hash);
53+ crush_hashmix(y, a, hash);
54+ crush_hashmix(b, x, hash);
55+ crush_hashmix(y, c, hash);
56+ return hash;
57+}
58+
59+static __u32 crush_hash32_rjenkins1_4(__u32 a, __u32 b, __u32 c, __u32 d)
60+{
61+ __u32 hash = crush_hash_seed ^ a ^ b ^ c ^ d;
62+ __u32 x = 231232;
63+ __u32 y = 1232;
64+ crush_hashmix(a, b, hash);
65+ crush_hashmix(c, d, hash);
66+ crush_hashmix(a, x, hash);
67+ crush_hashmix(y, b, hash);
68+ crush_hashmix(c, x, hash);
69+ crush_hashmix(y, d, hash);
70+ return hash;
71+}
72+
73+static __u32 crush_hash32_rjenkins1_5(__u32 a, __u32 b, __u32 c, __u32 d,
74+ __u32 e)
75+{
76+ __u32 hash = crush_hash_seed ^ a ^ b ^ c ^ d ^ e;
77+ __u32 x = 231232;
78+ __u32 y = 1232;
79+ crush_hashmix(a, b, hash);
80+ crush_hashmix(c, d, hash);
81+ crush_hashmix(e, x, hash);
82+ crush_hashmix(y, a, hash);
83+ crush_hashmix(b, x, hash);
84+ crush_hashmix(y, c, hash);
85+ crush_hashmix(d, x, hash);
86+ crush_hashmix(y, e, hash);
87+ return hash;
88+}
89+
90+
91+__u32 crush_hash32(int type, __u32 a)
92+{
93+ switch (type) {
94+ case CRUSH_HASH_RJENKINS1:
95+ return crush_hash32_rjenkins1(a);
96+ default:
97+ return 0;
98+ }
99+}
100+
101+__u32 crush_hash32_2(int type, __u32 a, __u32 b)
102+{
103+ switch (type) {
104+ case CRUSH_HASH_RJENKINS1:
105+ return crush_hash32_rjenkins1_2(a, b);
106+ default:
107+ return 0;
108+ }
109+}
110+
111+__u32 crush_hash32_3(int type, __u32 a, __u32 b, __u32 c)
112+{
113+ switch (type) {
114+ case CRUSH_HASH_RJENKINS1:
115+ return crush_hash32_rjenkins1_3(a, b, c);
116+ default:
117+ return 0;
118+ }
119+}
120+
121+__u32 crush_hash32_4(int type, __u32 a, __u32 b, __u32 c, __u32 d)
122+{
123+ switch (type) {
124+ case CRUSH_HASH_RJENKINS1:
125+ return crush_hash32_rjenkins1_4(a, b, c, d);
126+ default:
127+ return 0;
128+ }
129+}
130+
131+__u32 crush_hash32_5(int type, __u32 a, __u32 b, __u32 c, __u32 d, __u32 e)
132+{
133+ switch (type) {
134+ case CRUSH_HASH_RJENKINS1:
135+ return crush_hash32_rjenkins1_5(a, b, c, d, e);
136+ default:
137+ return 0;
138+ }
139+}
140+
141+const char *crush_hash_name(int type)
142+{
143+ switch (type) {
144+ case CRUSH_HASH_RJENKINS1:
145+ return "rjenkins1";
146+ default:
147+ return "unknown";
148+ }
149+}
fs/ceph/crush/hash.h+17−0
@@ -0,0 +1,17 @@
1+#ifndef _CRUSH_HASH_H
2+#define _CRUSH_HASH_H
3+
4+#define CRUSH_HASH_RJENKINS1 0
5+
6+#define CRUSH_HASH_DEFAULT CRUSH_HASH_RJENKINS1
7+
8+extern const char *crush_hash_name(int type);
9+
10+extern __u32 crush_hash32(int type, __u32 a);
11+extern __u32 crush_hash32_2(int type, __u32 a, __u32 b);
12+extern __u32 crush_hash32_3(int type, __u32 a, __u32 b, __u32 c);
13+extern __u32 crush_hash32_4(int type, __u32 a, __u32 b, __u32 c, __u32 d);
14+extern __u32 crush_hash32_5(int type, __u32 a, __u32 b, __u32 c, __u32 d,
15+ __u32 e);
16+
17+#endif
fs/ceph/crush/mapper.c+596−0
@@ -0,0 +1,596 @@
1+
2+#ifdef __KERNEL__
3+# include <linux/string.h>
4+# include <linux/slab.h>
5+# include <linux/bug.h>
6+# include <linux/kernel.h>
7+# ifndef dprintk
8+# define dprintk(args...)
9+# endif
10+#else
11+# include <string.h>
12+# include <stdio.h>
13+# include <stdlib.h>
14+# include <assert.h>
15+# define BUG_ON(x) assert(!(x))
16+# define dprintk(args...) /* printf(args) */
17+# define kmalloc(x, f) malloc(x)
18+# define kfree(x) free(x)
19+#endif
20+
21+#include "crush.h"
22+#include "hash.h"
23+
24+/*
25+ * Implement the core CRUSH mapping algorithm.
26+ */
27+
28+/**
29+ * crush_find_rule - find a crush_rule id for a given ruleset, type, and size.
30+ * @map: the crush_map
31+ * @ruleset: the storage ruleset id (user defined)
32+ * @type: storage ruleset type (user defined)
33+ * @size: output set size
34+ */
35+int crush_find_rule(struct crush_map *map, int ruleset, int type, int size)
36+{
37+ int i;
38+
39+ for (i = 0; i < map->max_rules; i++) {
40+ if (map->rules[i] &&
41+ map->rules[i]->mask.ruleset == ruleset &&
42+ map->rules[i]->mask.type == type &&
43+ map->rules[i]->mask.min_size <= size &&
44+ map->rules[i]->mask.max_size >= size)
45+ return i;
46+ }
47+ return -1;
48+}
49+
50+
51+/*
52+ * bucket choose methods
53+ *
54+ * For each bucket algorithm, we have a "choose" method that, given a
55+ * crush input @x and replica position (usually, position in output set) @r,
56+ * will produce an item in the bucket.
57+ */
58+
59+/*
60+ * Choose based on a random permutation of the bucket.
61+ *
62+ * We used to use some prime number arithmetic to do this, but it
63+ * wasn't very random, and had some other bad behaviors. Instead, we
64+ * calculate an actual random permutation of the bucket members.
65+ * Since this is expensive, we optimize for the r=0 case, which
66+ * captures the vast majority of calls.
67+ */
68+static int bucket_perm_choose(struct crush_bucket *bucket,
69+ int x, int r)
70+{
71+ unsigned pr = r % bucket->size;
72+ unsigned i, s;
73+
74+ /* start a new permutation if @x has changed */
75+ if (bucket->perm_x != x || bucket->perm_n == 0) {
76+ dprintk("bucket %d new x=%d\n", bucket->id, x);
77+ bucket->perm_x = x;
78+
79+ /* optimize common r=0 case */
80+ if (pr == 0) {
81+ s = crush_hash32_3(bucket->hash, x, bucket->id, 0) %
82+ bucket->size;
83+ bucket->perm[0] = s;
84+ bucket->perm_n = 0xffff; /* magic value, see below */
85+ goto out;
86+ }
87+
88+ for (i = 0; i < bucket->size; i++)
89+ bucket->perm[i] = i;
90+ bucket->perm_n = 0;
91+ } else if (bucket->perm_n == 0xffff) {
92+ /* clean up after the r=0 case above */
93+ for (i = 1; i < bucket->size; i++)
94+ bucket->perm[i] = i;
95+ bucket->perm[bucket->perm[0]] = 0;
96+ bucket->perm_n = 1;
97+ }
98+
99+ /* calculate permutation up to pr */
100+ for (i = 0; i < bucket->perm_n; i++)
101+ dprintk(" perm_choose have %d: %d\n", i, bucket->perm[i]);
102+ while (bucket->perm_n <= pr) {
103+ unsigned p = bucket->perm_n;
104+ /* no point in swapping the final entry */
105+ if (p < bucket->size - 1) {
106+ i = crush_hash32_3(bucket->hash, x, bucket->id, p) %
107+ (bucket->size - p);
108+ if (i) {
109+ unsigned t = bucket->perm[p + i];
110+ bucket->perm[p + i] = bucket->perm[p];
111+ bucket->perm[p] = t;
112+ }
113+ dprintk(" perm_choose swap %d with %d\n", p, p+i);
114+ }
115+ bucket->perm_n++;
116+ }
117+ for (i = 0; i < bucket->size; i++)
118+ dprintk(" perm_choose %d: %d\n", i, bucket->perm[i]);
119+
120+ s = bucket->perm[pr];
121+out:
122+ dprintk(" perm_choose %d sz=%d x=%d r=%d (%d) s=%d\n", bucket->id,
123+ bucket->size, x, r, pr, s);
124+ return bucket->items[s];
125+}
126+
127+/* uniform */
128+static int bucket_uniform_choose(struct crush_bucket_uniform *bucket,
129+ int x, int r)
130+{
131+ return bucket_perm_choose(&bucket->h, x, r);
132+}
133+
134+/* list */
135+static int bucket_list_choose(struct crush_bucket_list *bucket,
136+ int x, int r)
137+{
138+ int i;
139+
140+ for (i = bucket->h.size-1; i >= 0; i--) {
141+ __u64 w = crush_hash32_4(bucket->h.hash,x, bucket->h.items[i],
142+ r, bucket->h.id);
143+ w &= 0xffff;
144+ dprintk("list_choose i=%d x=%d r=%d item %d weight %x "
145+ "sw %x rand %llx",
146+ i, x, r, bucket->h.items[i], bucket->item_weights[i],
147+ bucket->sum_weights[i], w);
148+ w *= bucket->sum_weights[i];
149+ w = w >> 16;
150+ /*dprintk(" scaled %llx\n", w);*/
151+ if (w < bucket->item_weights[i])
152+ return bucket->h.items[i];
153+ }
154+
155+ BUG_ON(1);
156+ return 0;
157+}
158+
159+
160+/* (binary) tree */
161+static int height(int n)
162+{
163+ int h = 0;
164+ while ((n & 1) == 0) {
165+ h++;
166+ n = n >> 1;
167+ }
168+ return h;
169+}
170+
171+static int left(int x)
172+{
173+ int h = height(x);
174+ return x - (1 << (h-1));
175+}
176+
177+static int right(int x)
178+{
179+ int h = height(x);
180+ return x + (1 << (h-1));
181+}
182+
183+static int terminal(int x)
184+{
185+ return x & 1;
186+}
187+
188+static int bucket_tree_choose(struct crush_bucket_tree *bucket,
189+ int x, int r)
190+{
191+ int n, l;
192+ __u32 w;
193+ __u64 t;
194+
195+ /* start at root */
196+ n = bucket->num_nodes >> 1;
197+
198+ while (!terminal(n)) {
199+ /* pick point in [0, w) */
200+ w = bucket->node_weights[n];
201+ t = (__u64)crush_hash32_4(bucket->h.hash, x, n, r,
202+ bucket->h.id) * (__u64)w;
203+ t = t >> 32;
204+
205+ /* descend to the left or right? */
206+ l = left(n);
207+ if (t < bucket->node_weights[l])
208+ n = l;
209+ else
210+ n = right(n);
211+ }
212+
213+ return bucket->h.items[n >> 1];
214+}
215+
216+
217+/* straw */
218+
219+static int bucket_straw_choose(struct crush_bucket_straw *bucket,
220+ int x, int r)
221+{
222+ int i;
223+ int high = 0;
224+ __u64 high_draw = 0;
225+ __u64 draw;
226+
227+ for (i = 0; i < bucket->h.size; i++) {
228+ draw = crush_hash32_3(bucket->h.hash, x, bucket->h.items[i], r);
229+ draw &= 0xffff;
230+ draw *= bucket->straws[i];
231+ if (i == 0 || draw > high_draw) {
232+ high = i;
233+ high_draw = draw;
234+ }
235+ }
236+ return bucket->h.items[high];
237+}
238+
239+static int crush_bucket_choose(struct crush_bucket *in, int x, int r)
240+{
241+ dprintk("choose %d x=%d r=%d\n", in->id, x, r);
242+ switch (in->alg) {
243+ case CRUSH_BUCKET_UNIFORM:
244+ return bucket_uniform_choose((struct crush_bucket_uniform *)in,
245+ x, r);
246+ case CRUSH_BUCKET_LIST:
247+ return bucket_list_choose((struct crush_bucket_list *)in,
248+ x, r);
249+ case CRUSH_BUCKET_TREE:
250+ return bucket_tree_choose((struct crush_bucket_tree *)in,
251+ x, r);
252+ case CRUSH_BUCKET_STRAW:
253+ return bucket_straw_choose((struct crush_bucket_straw *)in,
254+ x, r);
255+ default:
256+ BUG_ON(1);
257+ return in->items[0];
258+ }
259+}
260+
261+/*
262+ * true if device is marked "out" (failed, fully offloaded)
263+ * of the cluster
264+ */
265+static int is_out(struct crush_map *map, __u32 *weight, int item, int x)
266+{
267+ if (weight[item] >= 0x1000)
268+ return 0;
269+ if (weight[item] == 0)
270+ return 1;
271+ if ((crush_hash32_2(CRUSH_HASH_RJENKINS1, x, item) & 0xffff)
272+ < weight[item])
273+ return 0;
274+ return 1;
275+}
276+
277+/**
278+ * crush_choose - choose numrep distinct items of given type
279+ * @map: the crush_map
280+ * @bucket: the bucket we are choose an item from
281+ * @x: crush input value
282+ * @numrep: the number of items to choose
283+ * @type: the type of item to choose
284+ * @out: pointer to output vector
285+ * @outpos: our position in that vector
286+ * @firstn: true if choosing "first n" items, false if choosing "indep"
287+ * @recurse_to_leaf: true if we want one device under each item of given type
288+ * @out2: second output vector for leaf items (if @recurse_to_leaf)
289+ */
290+static int crush_choose(struct crush_map *map,
291+ struct crush_bucket *bucket,
292+ __u32 *weight,
293+ int x, int numrep, int type,
294+ int *out, int outpos,
295+ int firstn, int recurse_to_leaf,
296+ int *out2)
297+{
298+ int rep;
299+ int ftotal, flocal;
300+ int retry_descent, retry_bucket, skip_rep;
301+ struct crush_bucket *in = bucket;
302+ int r;
303+ int i;
304+ int item = 0;
305+ int itemtype;
306+ int collide, reject;
307+ const int orig_tries = 5; /* attempts before we fall back to search */
308+ dprintk("choose bucket %d x %d outpos %d\n", bucket->id, x, outpos);
309+
310+ for (rep = outpos; rep < numrep; rep++) {
311+ /* keep trying until we get a non-out, non-colliding item */
312+ ftotal = 0;
313+ skip_rep = 0;
314+ do {
315+ retry_descent = 0;
316+ in = bucket; /* initial bucket */
317+
318+ /* choose through intervening buckets */
319+ flocal = 0;
320+ do {
321+ collide = 0;
322+ retry_bucket = 0;
323+ r = rep;
324+ if (in->alg == CRUSH_BUCKET_UNIFORM) {
325+ /* be careful */
326+ if (firstn || numrep >= in->size)
327+ /* r' = r + f_total */
328+ r += ftotal;
329+ else if (in->size % numrep == 0)
330+ /* r'=r+(n+1)*f_local */
331+ r += (numrep+1) *
332+ (flocal+ftotal);
333+ else
334+ /* r' = r + n*f_local */
335+ r += numrep * (flocal+ftotal);
336+ } else {
337+ if (firstn)
338+ /* r' = r + f_total */
339+ r += ftotal;
340+ else
341+ /* r' = r + n*f_local */
342+ r += numrep * (flocal+ftotal);
343+ }
344+
345+ /* bucket choose */
346+ if (in->size == 0) {
347+ reject = 1;
348+ goto reject;
349+ }
350+ if (flocal >= (in->size>>1) &&
351+ flocal > orig_tries)
352+ item = bucket_perm_choose(in, x, r);
353+ else
354+ item = crush_bucket_choose(in, x, r);
355+ BUG_ON(item >= map->max_devices);
356+
357+ /* desired type? */
358+ if (item < 0)
359+ itemtype = map->buckets[-1-item]->type;
360+ else
361+ itemtype = 0;
362+ dprintk(" item %d type %d\n", item, itemtype);
363+
364+ /* keep going? */
365+ if (itemtype != type) {
366+ BUG_ON(item >= 0 ||
367+ (-1-item) >= map->max_buckets);
368+ in = map->buckets[-1-item];
369+ continue;
370+ }
371+
372+ /* collision? */
373+ for (i = 0; i < outpos; i++) {
374+ if (out[i] == item) {
375+ collide = 1;
376+ break;
377+ }
378+ }
379+
380+ if (recurse_to_leaf &&
381+ item < 0 &&
382+ crush_choose(map, map->buckets[-1-item],
383+ weight,
384+ x, outpos+1, 0,
385+ out2, outpos,
386+ firstn, 0, NULL) <= outpos) {
387+ reject = 1;
388+ } else {
389+ /* out? */
390+ if (itemtype == 0)
391+ reject = is_out(map, weight,
392+ item, x);
393+ else
394+ reject = 0;
395+ }
396+
397+reject:
398+ if (reject || collide) {
399+ ftotal++;
400+ flocal++;
401+
402+ if (collide && flocal < 3)
403+ /* retry locally a few times */
404+ retry_bucket = 1;
405+ else if (flocal < in->size + orig_tries)
406+ /* exhaustive bucket search */
407+ retry_bucket = 1;
408+ else if (ftotal < 20)
409+ /* then retry descent */
410+ retry_descent = 1;
411+ else
412+ /* else give up */
413+ skip_rep = 1;
414+ dprintk(" reject %d collide %d "
415+ "ftotal %d flocal %d\n",
416+ reject, collide, ftotal,
417+ flocal);
418+ }
419+ } while (retry_bucket);
420+ } while (retry_descent);
421+
422+ if (skip_rep) {
423+ dprintk("skip rep\n");
424+ continue;
425+ }
426+
427+ dprintk("choose got %d\n", item);
428+ out[outpos] = item;
429+ outpos++;
430+ }
431+
432+ dprintk("choose returns %d\n", outpos);
433+ return outpos;
434+}
435+
436+
437+/**
438+ * crush_do_rule - calculate a mapping with the given input and rule
439+ * @map: the crush_map
440+ * @ruleno: the rule id
441+ * @x: hash input
442+ * @result: pointer to result vector
443+ * @result_max: maximum result size
444+ * @force: force initial replica choice; -1 for none
445+ */
446+int crush_do_rule(struct crush_map *map,
447+ int ruleno, int x, int *result, int result_max,
448+ int force, __u32 *weight)
449+{
450+ int result_len;
451+ int force_context[CRUSH_MAX_DEPTH];
452+ int force_pos = -1;
453+ int a[CRUSH_MAX_SET];
454+ int b[CRUSH_MAX_SET];
455+ int c[CRUSH_MAX_SET];
456+ int recurse_to_leaf;
457+ int *w;
458+ int wsize = 0;
459+ int *o;
460+ int osize;
461+ int *tmp;
462+ struct crush_rule *rule;
463+ int step;
464+ int i, j;
465+ int numrep;
466+ int firstn;
467+ int rc = -1;
468+
469+ BUG_ON(ruleno >= map->max_rules);
470+
471+ rule = map->rules[ruleno];
472+ result_len = 0;
473+ w = a;
474+ o = b;
475+
476+ /*
477+ * determine hierarchical context of force, if any. note
478+ * that this may or may not correspond to the specific types
479+ * referenced by the crush rule.
480+ */
481+ if (force >= 0) {
482+ if (force >= map->max_devices ||
483+ map->device_parents[force] == 0) {
484+ /*dprintk("CRUSH: forcefed device dne\n");*/
485+ rc = -1; /* force fed device dne */
486+ goto out;
487+ }
488+ if (!is_out(map, weight, force, x)) {
489+ while (1) {
490+ force_context[++force_pos] = force;
491+ if (force >= 0)
492+ force = map->device_parents[force];
493+ else
494+ force = map->bucket_parents[-1-force];
495+ if (force == 0)
496+ break;
497+ }
498+ }
499+ }
500+
501+ for (step = 0; step < rule->len; step++) {
502+ firstn = 0;
503+ switch (rule->steps[step].op) {
504+ case CRUSH_RULE_TAKE:
505+ w[0] = rule->steps[step].arg1;
506+ if (force_pos >= 0) {
507+ BUG_ON(force_context[force_pos] != w[0]);
508+ force_pos--;
509+ }
510+ wsize = 1;
511+ break;
512+
513+ case CRUSH_RULE_CHOOSE_LEAF_FIRSTN:
514+ case CRUSH_RULE_CHOOSE_FIRSTN:
515+ firstn = 1;
516+ case CRUSH_RULE_CHOOSE_LEAF_INDEP:
517+ case CRUSH_RULE_CHOOSE_INDEP:
518+ BUG_ON(wsize == 0);
519+
520+ recurse_to_leaf =
521+ rule->steps[step].op ==
522+ CRUSH_RULE_CHOOSE_LEAF_FIRSTN ||
523+ rule->steps[step].op ==
524+ CRUSH_RULE_CHOOSE_LEAF_INDEP;
525+
526+ /* reset output */
527+ osize = 0;
528+
529+ for (i = 0; i < wsize; i++) {
530+ /*
531+ * see CRUSH_N, CRUSH_N_MINUS macros.
532+ * basically, numrep <= 0 means relative to
533+ * the provided result_max
534+ */
535+ numrep = rule->steps[step].arg1;
536+ if (numrep <= 0) {
537+ numrep += result_max;
538+ if (numrep <= 0)
539+ continue;
540+ }
541+ j = 0;
542+ if (osize == 0 && force_pos >= 0) {
543+ /* skip any intermediate types */
544+ while (force_pos &&
545+ force_context[force_pos] < 0 &&
546+ rule->steps[step].arg2 !=
547+ map->buckets[-1 -
548+ force_context[force_pos]]->type)
549+ force_pos--;
550+ o[osize] = force_context[force_pos];
551+ if (recurse_to_leaf)
552+ c[osize] = force_context[0];
553+ j++;
554+ force_pos--;
555+ }
556+ osize += crush_choose(map,
557+ map->buckets[-1-w[i]],
558+ weight,
559+ x, numrep,
560+ rule->steps[step].arg2,
561+ o+osize, j,
562+ firstn,
563+ recurse_to_leaf, c+osize);
564+ }
565+
566+ if (recurse_to_leaf)
567+ /* copy final _leaf_ values to output set */
568+ memcpy(o, c, osize*sizeof(*o));
569+
570+ /* swap t and w arrays */
571+ tmp = o;
572+ o = w;
573+ w = tmp;
574+ wsize = osize;
575+ break;
576+
577+
578+ case CRUSH_RULE_EMIT:
579+ for (i = 0; i < wsize && result_len < result_max; i++) {
580+ result[result_len] = w[i];
581+ result_len++;
582+ }
583+ wsize = 0;
584+ break;
585+
586+ default:
587+ BUG_ON(1);
588+ }
589+ }
590+ rc = result_len;
591+
592+out:
593+ return rc;
594+}
595+
596+
fs/ceph/crush/mapper.h+20−0
@@ -0,0 +1,20 @@
1+#ifndef _CRUSH_MAPPER_H
2+#define _CRUSH_MAPPER_H
3+
4+/*
5+ * CRUSH functions for find rules and then mapping an input to an
6+ * output set.
7+ *
8+ * LGPL2
9+ */
10+
11+#include "crush.h"
12+
13+extern int crush_find_rule(struct crush_map *map, int pool, int type, int size);
14+extern int crush_do_rule(struct crush_map *map,
15+ int ruleno,
16+ int x, int *result, int result_max,
17+ int forcefeed, /* -1 for none */
18+ __u32 *weights);
19+
20+#endif
fs/ceph/crypto.c+408−0
@@ -0,0 +1,408 @@
1+
2+#include "ceph_debug.h"
3+
4+#include <linux/err.h>
5+#include <linux/scatterlist.h>
6+#include <crypto/hash.h>
7+
8+#include "crypto.h"
9+#include "decode.h"
10+
11+int ceph_crypto_key_encode(struct ceph_crypto_key *key, void **p, void *end)
12+{
13+ if (*p + sizeof(u16) + sizeof(key->created) +
14+ sizeof(u16) + key->len > end)
15+ return -ERANGE;
16+ ceph_encode_16(p, key->type);
17+ ceph_encode_copy(p, &key->created, sizeof(key->created));
18+ ceph_encode_16(p, key->len);
19+ ceph_encode_copy(p, key->key, key->len);
20+ return 0;
21+}
22+
23+int ceph_crypto_key_decode(struct ceph_crypto_key *key, void **p, void *end)
24+{
25+ ceph_decode_need(p, end, 2*sizeof(u16) + sizeof(key->created), bad);
26+ key->type = ceph_decode_16(p);
27+ ceph_decode_copy(p, &key->created, sizeof(key->created));
28+ key->len = ceph_decode_16(p);
29+ ceph_decode_need(p, end, key->len, bad);
30+ key->key = kmalloc(key->len, GFP_NOFS);
31+ if (!key->key)
32+ return -ENOMEM;
33+ ceph_decode_copy(p, key->key, key->len);
34+ return 0;
35+
36+bad:
37+ dout("failed to decode crypto key\n");
38+ return -EINVAL;
39+}
40+
41+int ceph_crypto_key_unarmor(struct ceph_crypto_key *key, const char *inkey)
42+{
43+ int inlen = strlen(inkey);
44+ int blen = inlen * 3 / 4;
45+ void *buf, *p;
46+ int ret;
47+
48+ dout("crypto_key_unarmor %s\n", inkey);
49+ buf = kmalloc(blen, GFP_NOFS);
50+ if (!buf)
51+ return -ENOMEM;
52+ blen = ceph_unarmor(buf, inkey, inkey+inlen);
53+ if (blen < 0) {
54+ kfree(buf);
55+ return blen;
56+ }
57+
58+ p = buf;
59+ ret = ceph_crypto_key_decode(key, &p, p + blen);
60+ kfree(buf);
61+ if (ret)
62+ return ret;
63+ dout("crypto_key_unarmor key %p type %d len %d\n", key,
64+ key->type, key->len);
65+ return 0;
66+}
67+
68+
69+
70+#define AES_KEY_SIZE 16
71+
72+static struct crypto_blkcipher *ceph_crypto_alloc_cipher(void)
73+{
74+ return crypto_alloc_blkcipher("cbc(aes)", 0, CRYPTO_ALG_ASYNC);
75+}
76+
77+const u8 *aes_iv = "cephsageyudagreg";
78+
79+int ceph_aes_encrypt(const void *key, int key_len, void *dst, size_t *dst_len,
80+ const void *src, size_t src_len)
81+{
82+ struct scatterlist sg_in[2], sg_out[1];
83+ struct crypto_blkcipher *tfm = ceph_crypto_alloc_cipher();
84+ struct blkcipher_desc desc = { .tfm = tfm, .flags = 0 };
85+ int ret;
86+ void *iv;
87+ int ivsize;
88+ size_t zero_padding = (0x10 - (src_len & 0x0f));
89+ char pad[16];
90+
91+ if (IS_ERR(tfm))
92+ return PTR_ERR(tfm);
93+
94+ memset(pad, zero_padding, zero_padding);
95+
96+ *dst_len = src_len + zero_padding;
97+
98+ crypto_blkcipher_setkey((void *)tfm, key, key_len);
99+ sg_init_table(sg_in, 2);
100+ sg_set_buf(&sg_in[0], src, src_len);
101+ sg_set_buf(&sg_in[1], pad, zero_padding);
102+ sg_init_table(sg_out, 1);
103+ sg_set_buf(sg_out, dst, *dst_len);
104+ iv = crypto_blkcipher_crt(tfm)->iv;
105+ ivsize = crypto_blkcipher_ivsize(tfm);
106+
107+ memcpy(iv, aes_iv, ivsize);
108+ /*
109+ print_hex_dump(KERN_ERR, "enc key: ", DUMP_PREFIX_NONE, 16, 1,
110+ key, key_len, 1);
111+ print_hex_dump(KERN_ERR, "enc src: ", DUMP_PREFIX_NONE, 16, 1,
112+ src, src_len, 1);
113+ print_hex_dump(KERN_ERR, "enc pad: ", DUMP_PREFIX_NONE, 16, 1,
114+ pad, zero_padding, 1);
115+ */
116+ ret = crypto_blkcipher_encrypt(&desc, sg_out, sg_in,
117+ src_len + zero_padding);
118+ crypto_free_blkcipher(tfm);
119+ if (ret < 0)
120+ pr_err("ceph_aes_crypt failed %d\n", ret);
121+ /*
122+ print_hex_dump(KERN_ERR, "enc out: ", DUMP_PREFIX_NONE, 16, 1,
123+ dst, *dst_len, 1);
124+ */
125+ return 0;
126+}
127+
128+int ceph_aes_encrypt2(const void *key, int key_len, void *dst, size_t *dst_len,
129+ const void *src1, size_t src1_len,
130+ const void *src2, size_t src2_len)
131+{
132+ struct scatterlist sg_in[3], sg_out[1];
133+ struct crypto_blkcipher *tfm = ceph_crypto_alloc_cipher();
134+ struct blkcipher_desc desc = { .tfm = tfm, .flags = 0 };
135+ int ret;
136+ void *iv;
137+ int ivsize;
138+ size_t zero_padding = (0x10 - ((src1_len + src2_len) & 0x0f));
139+ char pad[16];
140+
141+ if (IS_ERR(tfm))
142+ return PTR_ERR(tfm);
143+
144+ memset(pad, zero_padding, zero_padding);
145+
146+ *dst_len = src1_len + src2_len + zero_padding;
147+
148+ crypto_blkcipher_setkey((void *)tfm, key, key_len);
149+ sg_init_table(sg_in, 3);
150+ sg_set_buf(&sg_in[0], src1, src1_len);
151+ sg_set_buf(&sg_in[1], src2, src2_len);
152+ sg_set_buf(&sg_in[2], pad, zero_padding);
153+ sg_init_table(sg_out, 1);
154+ sg_set_buf(sg_out, dst, *dst_len);
155+ iv = crypto_blkcipher_crt(tfm)->iv;
156+ ivsize = crypto_blkcipher_ivsize(tfm);
157+
158+ memcpy(iv, aes_iv, ivsize);
159+ /*
160+ print_hex_dump(KERN_ERR, "enc key: ", DUMP_PREFIX_NONE, 16, 1,
161+ key, key_len, 1);
162+ print_hex_dump(KERN_ERR, "enc src1: ", DUMP_PREFIX_NONE, 16, 1,
163+ src1, src1_len, 1);
164+ print_hex_dump(KERN_ERR, "enc src2: ", DUMP_PREFIX_NONE, 16, 1,
165+ src2, src2_len, 1);
166+ print_hex_dump(KERN_ERR, "enc pad: ", DUMP_PREFIX_NONE, 16, 1,
167+ pad, zero_padding, 1);
168+ */
169+ ret = crypto_blkcipher_encrypt(&desc, sg_out, sg_in,
170+ src1_len + src2_len + zero_padding);
171+ crypto_free_blkcipher(tfm);
172+ if (ret < 0)
173+ pr_err("ceph_aes_crypt2 failed %d\n", ret);
174+ /*
175+ print_hex_dump(KERN_ERR, "enc out: ", DUMP_PREFIX_NONE, 16, 1,
176+ dst, *dst_len, 1);
177+ */
178+ return 0;
179+}
180+
181+int ceph_aes_decrypt(const void *key, int key_len, void *dst, size_t *dst_len,
182+ const void *src, size_t src_len)
183+{
184+ struct scatterlist sg_in[1], sg_out[2];
185+ struct crypto_blkcipher *tfm = ceph_crypto_alloc_cipher();
186+ struct blkcipher_desc desc = { .tfm = tfm };
187+ char pad[16];
188+ void *iv;
189+ int ivsize;
190+ int ret;
191+ int last_byte;
192+
193+ if (IS_ERR(tfm))
194+ return PTR_ERR(tfm);
195+
196+ crypto_blkcipher_setkey((void *)tfm, key, key_len);
197+ sg_init_table(sg_in, 1);
198+ sg_init_table(sg_out, 2);
199+ sg_set_buf(sg_in, src, src_len);
200+ sg_set_buf(&sg_out[0], dst, *dst_len);
201+ sg_set_buf(&sg_out[1], pad, sizeof(pad));
202+
203+ iv = crypto_blkcipher_crt(tfm)->iv;
204+ ivsize = crypto_blkcipher_ivsize(tfm);
205+
206+ memcpy(iv, aes_iv, ivsize);
207+
208+ /*
209+ print_hex_dump(KERN_ERR, "dec key: ", DUMP_PREFIX_NONE, 16, 1,
210+ key, key_len, 1);
211+ print_hex_dump(KERN_ERR, "dec in: ", DUMP_PREFIX_NONE, 16, 1,
212+ src, src_len, 1);
213+ */
214+
215+ ret = crypto_blkcipher_decrypt(&desc, sg_out, sg_in, src_len);
216+ crypto_free_blkcipher(tfm);
217+ if (ret < 0) {
218+ pr_err("ceph_aes_decrypt failed %d\n", ret);
219+ return ret;
220+ }
221+
222+ if (src_len <= *dst_len)
223+ last_byte = ((char *)dst)[src_len - 1];
224+ else
225+ last_byte = pad[src_len - *dst_len - 1];
226+ if (last_byte <= 16 && src_len >= last_byte) {
227+ *dst_len = src_len - last_byte;
228+ } else {
229+ pr_err("ceph_aes_decrypt got bad padding %d on src len %d\n",
230+ last_byte, (int)src_len);
231+ return -EPERM; /* bad padding */
232+ }
233+ /*
234+ print_hex_dump(KERN_ERR, "dec out: ", DUMP_PREFIX_NONE, 16, 1,
235+ dst, *dst_len, 1);
236+ */
237+ return 0;
238+}
239+
240+int ceph_aes_decrypt2(const void *key, int key_len,
241+ void *dst1, size_t *dst1_len,
242+ void *dst2, size_t *dst2_len,
243+ const void *src, size_t src_len)
244+{
245+ struct scatterlist sg_in[1], sg_out[3];
246+ struct crypto_blkcipher *tfm = ceph_crypto_alloc_cipher();
247+ struct blkcipher_desc desc = { .tfm = tfm };
248+ char pad[16];
249+ void *iv;
250+ int ivsize;
251+ int ret;
252+ int last_byte;
253+
254+ if (IS_ERR(tfm))
255+ return PTR_ERR(tfm);
256+
257+ sg_init_table(sg_in, 1);
258+ sg_set_buf(sg_in, src, src_len);
259+ sg_init_table(sg_out, 3);
260+ sg_set_buf(&sg_out[0], dst1, *dst1_len);
261+ sg_set_buf(&sg_out[1], dst2, *dst2_len);
262+ sg_set_buf(&sg_out[2], pad, sizeof(pad));
263+
264+ crypto_blkcipher_setkey((void *)tfm, key, key_len);
265+ iv = crypto_blkcipher_crt(tfm)->iv;
266+ ivsize = crypto_blkcipher_ivsize(tfm);
267+
268+ memcpy(iv, aes_iv, ivsize);
269+
270+ /*
271+ print_hex_dump(KERN_ERR, "dec key: ", DUMP_PREFIX_NONE, 16, 1,
272+ key, key_len, 1);
273+ print_hex_dump(KERN_ERR, "dec in: ", DUMP_PREFIX_NONE, 16, 1,
274+ src, src_len, 1);
275+ */
276+
277+ ret = crypto_blkcipher_decrypt(&desc, sg_out, sg_in, src_len);
278+ crypto_free_blkcipher(tfm);
279+ if (ret < 0) {
280+ pr_err("ceph_aes_decrypt failed %d\n", ret);
281+ return ret;
282+ }
283+
284+ if (src_len <= *dst1_len)
285+ last_byte = ((char *)dst1)[src_len - 1];
286+ else if (src_len <= *dst1_len + *dst2_len)
287+ last_byte = ((char *)dst2)[src_len - *dst1_len - 1];
288+ else
289+ last_byte = pad[src_len - *dst1_len - *dst2_len - 1];
290+ if (last_byte <= 16 && src_len >= last_byte) {
291+ src_len -= last_byte;
292+ } else {
293+ pr_err("ceph_aes_decrypt got bad padding %d on src len %d\n",
294+ last_byte, (int)src_len);
295+ return -EPERM; /* bad padding */
296+ }
297+
298+ if (src_len < *dst1_len) {
299+ *dst1_len = src_len;
300+ *dst2_len = 0;
301+ } else {
302+ *dst2_len = src_len - *dst1_len;
303+ }
304+ /*
305+ print_hex_dump(KERN_ERR, "dec out1: ", DUMP_PREFIX_NONE, 16, 1,
306+ dst1, *dst1_len, 1);
307+ print_hex_dump(KERN_ERR, "dec out2: ", DUMP_PREFIX_NONE, 16, 1,
308+ dst2, *dst2_len, 1);
309+ */
310+
311+ return 0;
312+}
313+
314+
315+int ceph_decrypt(struct ceph_crypto_key *secret, void *dst, size_t *dst_len,
316+ const void *src, size_t src_len)
317+{
318+ switch (secret->type) {
319+ case CEPH_CRYPTO_NONE:
320+ if (*dst_len < src_len)
321+ return -ERANGE;
322+ memcpy(dst, src, src_len);
323+ *dst_len = src_len;
324+ return 0;
325+
326+ case CEPH_CRYPTO_AES:
327+ return ceph_aes_decrypt(secret->key, secret->len, dst,
328+ dst_len, src, src_len);
329+
330+ default:
331+ return -EINVAL;
332+ }
333+}
334+
335+int ceph_decrypt2(struct ceph_crypto_key *secret,
336+ void *dst1, size_t *dst1_len,
337+ void *dst2, size_t *dst2_len,
338+ const void *src, size_t src_len)
339+{
340+ size_t t;
341+
342+ switch (secret->type) {
343+ case CEPH_CRYPTO_NONE:
344+ if (*dst1_len + *dst2_len < src_len)
345+ return -ERANGE;
346+ t = min(*dst1_len, src_len);
347+ memcpy(dst1, src, t);
348+ *dst1_len = t;
349+ src += t;
350+ src_len -= t;
351+ if (src_len) {
352+ t = min(*dst2_len, src_len);
353+ memcpy(dst2, src, t);
354+ *dst2_len = t;
355+ }
356+ return 0;
357+
358+ case CEPH_CRYPTO_AES:
359+ return ceph_aes_decrypt2(secret->key, secret->len,
360+ dst1, dst1_len, dst2, dst2_len,
361+ src, src_len);
362+
363+ default:
364+ return -EINVAL;
365+ }
366+}
367+
368+int ceph_encrypt(struct ceph_crypto_key *secret, void *dst, size_t *dst_len,
369+ const void *src, size_t src_len)
370+{
371+ switch (secret->type) {
372+ case CEPH_CRYPTO_NONE:
373+ if (*dst_len < src_len)
374+ return -ERANGE;
375+ memcpy(dst, src, src_len);
376+ *dst_len = src_len;
377+ return 0;
378+
379+ case CEPH_CRYPTO_AES:
380+ return ceph_aes_encrypt(secret->key, secret->len, dst,
381+ dst_len, src, src_len);
382+
383+ default:
384+ return -EINVAL;
385+ }
386+}
387+
388+int ceph_encrypt2(struct ceph_crypto_key *secret, void *dst, size_t *dst_len,
389+ const void *src1, size_t src1_len,
390+ const void *src2, size_t src2_len)
391+{
392+ switch (secret->type) {
393+ case CEPH_CRYPTO_NONE:
394+ if (*dst_len < src1_len + src2_len)
395+ return -ERANGE;
396+ memcpy(dst, src1, src1_len);
397+ memcpy(dst + src1_len, src2, src2_len);
398+ *dst_len = src1_len + src2_len;
399+ return 0;
400+
401+ case CEPH_CRYPTO_AES:
402+ return ceph_aes_encrypt2(secret->key, secret->len, dst, dst_len,
403+ src1, src1_len, src2, src2_len);
404+
405+ default:
406+ return -EINVAL;
407+ }
408+}
fs/ceph/crypto.h+48−0
@@ -0,0 +1,48 @@
1+#ifndef _FS_CEPH_CRYPTO_H
2+#define _FS_CEPH_CRYPTO_H
3+
4+#include "types.h"
5+#include "buffer.h"
6+
7+/*
8+ * cryptographic secret
9+ */
10+struct ceph_crypto_key {
11+ int type;
12+ struct ceph_timespec created;
13+ int len;
14+ void *key;
15+};
16+
17+static inline void ceph_crypto_key_destroy(struct ceph_crypto_key *key)
18+{
19+ kfree(key->key);
20+}
21+
22+extern int ceph_crypto_key_encode(struct ceph_crypto_key *key,
23+ void **p, void *end);
24+extern int ceph_crypto_key_decode(struct ceph_crypto_key *key,
25+ void **p, void *end);
26+extern int ceph_crypto_key_unarmor(struct ceph_crypto_key *key, const char *in);
27+
28+/* crypto.c */
29+extern int ceph_decrypt(struct ceph_crypto_key *secret,
30+ void *dst, size_t *dst_len,
31+ const void *src, size_t src_len);
32+extern int ceph_encrypt(struct ceph_crypto_key *secret,
33+ void *dst, size_t *dst_len,
34+ const void *src, size_t src_len);
35+extern int ceph_decrypt2(struct ceph_crypto_key *secret,
36+ void *dst1, size_t *dst1_len,
37+ void *dst2, size_t *dst2_len,
38+ const void *src, size_t src_len);
39+extern int ceph_encrypt2(struct ceph_crypto_key *secret,
40+ void *dst, size_t *dst_len,
41+ const void *src1, size_t src1_len,
42+ const void *src2, size_t src2_len);
43+
44+/* armor.c */
45+extern int ceph_armor(char *dst, const void *src, const void *end);
46+extern int ceph_unarmor(void *dst, const char *src, const char *end);
47+
48+#endif
fs/ceph/debugfs.c+483−0
@@ -0,0 +1,483 @@
1+#include "ceph_debug.h"
2+
3+#include <linux/device.h>
4+#include <linux/module.h>
5+#include <linux/ctype.h>
6+#include <linux/debugfs.h>
7+#include <linux/seq_file.h>
8+
9+#include "super.h"
10+#include "mds_client.h"
11+#include "mon_client.h"
12+#include "auth.h"
13+
14+#ifdef CONFIG_DEBUG_FS
15+
16+/*
17+ * Implement /sys/kernel/debug/ceph fun
18+ *
19+ * /sys/kernel/debug/ceph/client* - an instance of the ceph client
20+ * .../osdmap - current osdmap
21+ * .../mdsmap - current mdsmap
22+ * .../monmap - current monmap
23+ * .../osdc - active osd requests
24+ * .../mdsc - active mds requests
25+ * .../monc - mon client state
26+ * .../dentry_lru - dump contents of dentry lru
27+ * .../caps - expose cap (reservation) stats
28+ * .../bdi - symlink to ../../bdi/something
29+ */
30+
31+static struct dentry *ceph_debugfs_dir;
32+
33+static int monmap_show(struct seq_file *s, void *p)
34+{
35+ int i;
36+ struct ceph_client *client = s->private;
37+
38+ if (client->monc.monmap == NULL)
39+ return 0;
40+
41+ seq_printf(s, "epoch %d\n", client->monc.monmap->epoch);
42+ for (i = 0; i < client->monc.monmap->num_mon; i++) {
43+ struct ceph_entity_inst *inst =
44+ &client->monc.monmap->mon_inst[i];
45+
46+ seq_printf(s, "\t%s%lld\t%s\n",
47+ ENTITY_NAME(inst->name),
48+ pr_addr(&inst->addr.in_addr));
49+ }
50+ return 0;
51+}
52+
53+static int mdsmap_show(struct seq_file *s, void *p)
54+{
55+ int i;
56+ struct ceph_client *client = s->private;
57+
58+ if (client->mdsc.mdsmap == NULL)
59+ return 0;
60+ seq_printf(s, "epoch %d\n", client->mdsc.mdsmap->m_epoch);
61+ seq_printf(s, "root %d\n", client->mdsc.mdsmap->m_root);
62+ seq_printf(s, "session_timeout %d\n",
63+ client->mdsc.mdsmap->m_session_timeout);
64+ seq_printf(s, "session_autoclose %d\n",
65+ client->mdsc.mdsmap->m_session_autoclose);
66+ for (i = 0; i < client->mdsc.mdsmap->m_max_mds; i++) {
67+ struct ceph_entity_addr *addr =
68+ &client->mdsc.mdsmap->m_info[i].addr;
69+ int state = client->mdsc.mdsmap->m_info[i].state;
70+
71+ seq_printf(s, "\tmds%d\t%s\t(%s)\n", i, pr_addr(&addr->in_addr),
72+ ceph_mds_state_name(state));
73+ }
74+ return 0;
75+}
76+
77+static int osdmap_show(struct seq_file *s, void *p)
78+{
79+ int i;
80+ struct ceph_client *client = s->private;
81+ struct rb_node *n;
82+
83+ if (client->osdc.osdmap == NULL)
84+ return 0;
85+ seq_printf(s, "epoch %d\n", client->osdc.osdmap->epoch);
86+ seq_printf(s, "flags%s%s\n",
87+ (client->osdc.osdmap->flags & CEPH_OSDMAP_NEARFULL) ?
88+ " NEARFULL" : "",
89+ (client->osdc.osdmap->flags & CEPH_OSDMAP_FULL) ?
90+ " FULL" : "");
91+ for (n = rb_first(&client->osdc.osdmap->pg_pools); n; n = rb_next(n)) {
92+ struct ceph_pg_pool_info *pool =
93+ rb_entry(n, struct ceph_pg_pool_info, node);
94+ seq_printf(s, "pg_pool %d pg_num %d / %d, lpg_num %d / %d\n",
95+ pool->id, pool->v.pg_num, pool->pg_num_mask,
96+ pool->v.lpg_num, pool->lpg_num_mask);
97+ }
98+ for (i = 0; i < client->osdc.osdmap->max_osd; i++) {
99+ struct ceph_entity_addr *addr =
100+ &client->osdc.osdmap->osd_addr[i];
101+ int state = client->osdc.osdmap->osd_state[i];
102+ char sb[64];
103+
104+ seq_printf(s, "\tosd%d\t%s\t%3d%%\t(%s)\n",
105+ i, pr_addr(&addr->in_addr),
106+ ((client->osdc.osdmap->osd_weight[i]*100) >> 16),
107+ ceph_osdmap_state_str(sb, sizeof(sb), state));
108+ }
109+ return 0;
110+}
111+
112+static int monc_show(struct seq_file *s, void *p)
113+{
114+ struct ceph_client *client = s->private;
115+ struct ceph_mon_statfs_request *req;
116+ struct ceph_mon_client *monc = &client->monc;
117+ struct rb_node *rp;
118+
119+ mutex_lock(&monc->mutex);
120+
121+ if (monc->have_mdsmap)
122+ seq_printf(s, "have mdsmap %u\n", (unsigned)monc->have_mdsmap);
123+ if (monc->have_osdmap)
124+ seq_printf(s, "have osdmap %u\n", (unsigned)monc->have_osdmap);
125+ if (monc->want_next_osdmap)
126+ seq_printf(s, "want next osdmap\n");
127+
128+ for (rp = rb_first(&monc->statfs_request_tree); rp; rp = rb_next(rp)) {
129+ req = rb_entry(rp, struct ceph_mon_statfs_request, node);
130+ seq_printf(s, "%lld statfs\n", req->tid);
131+ }
132+
133+ mutex_unlock(&monc->mutex);
134+ return 0;
135+}
136+
137+static int mdsc_show(struct seq_file *s, void *p)
138+{
139+ struct ceph_client *client = s->private;
140+ struct ceph_mds_client *mdsc = &client->mdsc;
141+ struct ceph_mds_request *req;
142+ struct rb_node *rp;
143+ int pathlen;
144+ u64 pathbase;
145+ char *path;
146+
147+ mutex_lock(&mdsc->mutex);
148+ for (rp = rb_first(&mdsc->request_tree); rp; rp = rb_next(rp)) {
149+ req = rb_entry(rp, struct ceph_mds_request, r_node);
150+
151+ if (req->r_request)
152+ seq_printf(s, "%lld\tmds%d\t", req->r_tid, req->r_mds);
153+ else
154+ seq_printf(s, "%lld\t(no request)\t", req->r_tid);
155+
156+ seq_printf(s, "%s", ceph_mds_op_name(req->r_op));
157+
158+ if (req->r_got_unsafe)
159+ seq_printf(s, "\t(unsafe)");
160+ else
161+ seq_printf(s, "\t");
162+
163+ if (req->r_inode) {
164+ seq_printf(s, " #%llx", ceph_ino(req->r_inode));
165+ } else if (req->r_dentry) {
166+ path = ceph_mdsc_build_path(req->r_dentry, &pathlen,
167+ &pathbase, 0);
168+ spin_lock(&req->r_dentry->d_lock);
169+ seq_printf(s, " #%llx/%.*s (%s)",
170+ ceph_ino(req->r_dentry->d_parent->d_inode),
171+ req->r_dentry->d_name.len,
172+ req->r_dentry->d_name.name,
173+ path ? path : "");
174+ spin_unlock(&req->r_dentry->d_lock);
175+ kfree(path);
176+ } else if (req->r_path1) {
177+ seq_printf(s, " #%llx/%s", req->r_ino1.ino,
178+ req->r_path1);
179+ }
180+
181+ if (req->r_old_dentry) {
182+ path = ceph_mdsc_build_path(req->r_old_dentry, &pathlen,
183+ &pathbase, 0);
184+ spin_lock(&req->r_old_dentry->d_lock);
185+ seq_printf(s, " #%llx/%.*s (%s)",
186+ ceph_ino(req->r_old_dentry->d_parent->d_inode),
187+ req->r_old_dentry->d_name.len,
188+ req->r_old_dentry->d_name.name,
189+ path ? path : "");
190+ spin_unlock(&req->r_old_dentry->d_lock);
191+ kfree(path);
192+ } else if (req->r_path2) {
193+ if (req->r_ino2.ino)
194+ seq_printf(s, " #%llx/%s", req->r_ino2.ino,
195+ req->r_path2);
196+ else
197+ seq_printf(s, " %s", req->r_path2);
198+ }
199+
200+ seq_printf(s, "\n");
201+ }
202+ mutex_unlock(&mdsc->mutex);
203+
204+ return 0;
205+}
206+
207+static int osdc_show(struct seq_file *s, void *pp)
208+{
209+ struct ceph_client *client = s->private;
210+ struct ceph_osd_client *osdc = &client->osdc;
211+ struct rb_node *p;
212+
213+ mutex_lock(&osdc->request_mutex);
214+ for (p = rb_first(&osdc->requests); p; p = rb_next(p)) {
215+ struct ceph_osd_request *req;
216+ struct ceph_osd_request_head *head;
217+ struct ceph_osd_op *op;
218+ int num_ops;
219+ int opcode, olen;
220+ int i;
221+
222+ req = rb_entry(p, struct ceph_osd_request, r_node);
223+
224+ seq_printf(s, "%lld\tosd%d\t%d.%x\t", req->r_tid,
225+ req->r_osd ? req->r_osd->o_osd : -1,
226+ le32_to_cpu(req->r_pgid.pool),
227+ le16_to_cpu(req->r_pgid.ps));
228+
229+ head = req->r_request->front.iov_base;
230+ op = (void *)(head + 1);
231+
232+ num_ops = le16_to_cpu(head->num_ops);
233+ olen = le32_to_cpu(head->object_len);
234+ seq_printf(s, "%.*s", olen,
235+ (const char *)(head->ops + num_ops));
236+
237+ if (req->r_reassert_version.epoch)
238+ seq_printf(s, "\t%u'%llu",
239+ (unsigned)le32_to_cpu(req->r_reassert_version.epoch),
240+ le64_to_cpu(req->r_reassert_version.version));
241+ else
242+ seq_printf(s, "\t");
243+
244+ for (i = 0; i < num_ops; i++) {
245+ opcode = le16_to_cpu(op->op);
246+ seq_printf(s, "\t%s", ceph_osd_op_name(opcode));
247+ op++;
248+ }
249+
250+ seq_printf(s, "\n");
251+ }
252+ mutex_unlock(&osdc->request_mutex);
253+ return 0;
254+}
255+
256+static int caps_show(struct seq_file *s, void *p)
257+{
258+ struct ceph_client *client = p;
259+ int total, avail, used, reserved, min;
260+
261+ ceph_reservation_status(client, &total, &avail, &used, &reserved, &min);
262+ seq_printf(s, "total\t\t%d\n"
263+ "avail\t\t%d\n"
264+ "used\t\t%d\n"
265+ "reserved\t%d\n"
266+ "min\t%d\n",
267+ total, avail, used, reserved, min);
268+ return 0;
269+}
270+
271+static int dentry_lru_show(struct seq_file *s, void *ptr)
272+{
273+ struct ceph_client *client = s->private;
274+ struct ceph_mds_client *mdsc = &client->mdsc;
275+ struct ceph_dentry_info *di;
276+
277+ spin_lock(&mdsc->dentry_lru_lock);
278+ list_for_each_entry(di, &mdsc->dentry_lru, lru) {
279+ struct dentry *dentry = di->dentry;
280+ seq_printf(s, "%p %p\t%.*s\n",
281+ di, dentry, dentry->d_name.len, dentry->d_name.name);
282+ }
283+ spin_unlock(&mdsc->dentry_lru_lock);
284+
285+ return 0;
286+}
287+
288+#define DEFINE_SHOW_FUNC(name) \
289+static int name##_open(struct inode *inode, struct file *file) \
290+{ \
291+ struct seq_file *sf; \
292+ int ret; \
293+ \
294+ ret = single_open(file, name, NULL); \
295+ sf = file->private_data; \
296+ sf->private = inode->i_private; \
297+ return ret; \
298+} \
299+ \
300+static const struct file_operations name##_fops = { \
301+ .open = name##_open, \
302+ .read = seq_read, \
303+ .llseek = seq_lseek, \
304+ .release = single_release, \
305+};
306+
307+DEFINE_SHOW_FUNC(monmap_show)
308+DEFINE_SHOW_FUNC(mdsmap_show)
309+DEFINE_SHOW_FUNC(osdmap_show)
310+DEFINE_SHOW_FUNC(monc_show)
311+DEFINE_SHOW_FUNC(mdsc_show)
312+DEFINE_SHOW_FUNC(osdc_show)
313+DEFINE_SHOW_FUNC(dentry_lru_show)
314+DEFINE_SHOW_FUNC(caps_show)
315+
316+static int congestion_kb_set(void *data, u64 val)
317+{
318+ struct ceph_client *client = (struct ceph_client *)data;
319+
320+ if (client)
321+ client->mount_args->congestion_kb = (int)val;
322+
323+ return 0;
324+}
325+
326+static int congestion_kb_get(void *data, u64 *val)
327+{
328+ struct ceph_client *client = (struct ceph_client *)data;
329+
330+ if (client)
331+ *val = (u64)client->mount_args->congestion_kb;
332+
333+ return 0;
334+}
335+
336+
337+DEFINE_SIMPLE_ATTRIBUTE(congestion_kb_fops, congestion_kb_get,
338+ congestion_kb_set, "%llu\n");
339+
340+int __init ceph_debugfs_init(void)
341+{
342+ ceph_debugfs_dir = debugfs_create_dir("ceph", NULL);
343+ if (!ceph_debugfs_dir)
344+ return -ENOMEM;
345+ return 0;
346+}
347+
348+void ceph_debugfs_cleanup(void)
349+{
350+ debugfs_remove(ceph_debugfs_dir);
351+}
352+
353+int ceph_debugfs_client_init(struct ceph_client *client)
354+{
355+ int ret = 0;
356+ char name[80];
357+
358+ snprintf(name, sizeof(name), FSID_FORMAT ".client%lld",
359+ PR_FSID(&client->fsid), client->monc.auth->global_id);
360+
361+ client->debugfs_dir = debugfs_create_dir(name, ceph_debugfs_dir);
362+ if (!client->debugfs_dir)
363+ goto out;
364+
365+ client->monc.debugfs_file = debugfs_create_file("monc",
366+ 0600,
367+ client->debugfs_dir,
368+ client,
369+ &monc_show_fops);
370+ if (!client->monc.debugfs_file)
371+ goto out;
372+
373+ client->mdsc.debugfs_file = debugfs_create_file("mdsc",
374+ 0600,
375+ client->debugfs_dir,
376+ client,
377+ &mdsc_show_fops);
378+ if (!client->mdsc.debugfs_file)
379+ goto out;
380+
381+ client->osdc.debugfs_file = debugfs_create_file("osdc",
382+ 0600,
383+ client->debugfs_dir,
384+ client,
385+ &osdc_show_fops);
386+ if (!client->osdc.debugfs_file)
387+ goto out;
388+
389+ client->debugfs_monmap = debugfs_create_file("monmap",
390+ 0600,
391+ client->debugfs_dir,
392+ client,
393+ &monmap_show_fops);
394+ if (!client->debugfs_monmap)
395+ goto out;
396+
397+ client->debugfs_mdsmap = debugfs_create_file("mdsmap",
398+ 0600,
399+ client->debugfs_dir,
400+ client,
401+ &mdsmap_show_fops);
402+ if (!client->debugfs_mdsmap)
403+ goto out;
404+
405+ client->debugfs_osdmap = debugfs_create_file("osdmap",
406+ 0600,
407+ client->debugfs_dir,
408+ client,
409+ &osdmap_show_fops);
410+ if (!client->debugfs_osdmap)
411+ goto out;
412+
413+ client->debugfs_dentry_lru = debugfs_create_file("dentry_lru",
414+ 0600,
415+ client->debugfs_dir,
416+ client,
417+ &dentry_lru_show_fops);
418+ if (!client->debugfs_dentry_lru)
419+ goto out;
420+
421+ client->debugfs_caps = debugfs_create_file("caps",
422+ 0400,
423+ client->debugfs_dir,
424+ client,
425+ &caps_show_fops);
426+ if (!client->debugfs_caps)
427+ goto out;
428+
429+ client->debugfs_congestion_kb = debugfs_create_file("writeback_congestion_kb",
430+ 0600,
431+ client->debugfs_dir,
432+ client,
433+ &congestion_kb_fops);
434+ if (!client->debugfs_congestion_kb)
435+ goto out;
436+
437+ sprintf(name, "../../bdi/%s", dev_name(client->sb->s_bdi->dev));
438+ client->debugfs_bdi = debugfs_create_symlink("bdi", client->debugfs_dir,
439+ name);
440+
441+ return 0;
442+
443+out:
444+ ceph_debugfs_client_cleanup(client);
445+ return ret;
446+}
447+
448+void ceph_debugfs_client_cleanup(struct ceph_client *client)
449+{
450+ debugfs_remove(client->debugfs_bdi);
451+ debugfs_remove(client->debugfs_caps);
452+ debugfs_remove(client->debugfs_dentry_lru);
453+ debugfs_remove(client->debugfs_osdmap);
454+ debugfs_remove(client->debugfs_mdsmap);
455+ debugfs_remove(client->debugfs_monmap);
456+ debugfs_remove(client->osdc.debugfs_file);
457+ debugfs_remove(client->mdsc.debugfs_file);
458+ debugfs_remove(client->monc.debugfs_file);
459+ debugfs_remove(client->debugfs_congestion_kb);
460+ debugfs_remove(client->debugfs_dir);
461+}
462+
463+#else // CONFIG_DEBUG_FS
464+
465+int __init ceph_debugfs_init(void)
466+{
467+ return 0;
468+}
469+
470+void ceph_debugfs_cleanup(void)
471+{
472+}
473+
474+int ceph_debugfs_client_init(struct ceph_client *client)
475+{
476+ return 0;
477+}
478+
479+void ceph_debugfs_client_cleanup(struct ceph_client *client)
480+{
481+}
482+
483+#endif // CONFIG_DEBUG_FS
fs/ceph/decode.h+194−0
@@ -0,0 +1,194 @@
1+#ifndef __CEPH_DECODE_H
2+#define __CEPH_DECODE_H
3+
4+#include <asm/unaligned.h>
5+#include <linux/time.h>
6+
7+#include "types.h"
8+
9+/*
10+ * in all cases,
11+ * void **p pointer to position pointer
12+ * void *end pointer to end of buffer (last byte + 1)
13+ */
14+
15+static inline u64 ceph_decode_64(void **p)
16+{
17+ u64 v = get_unaligned_le64(*p);
18+ *p += sizeof(u64);
19+ return v;
20+}
21+static inline u32 ceph_decode_32(void **p)
22+{
23+ u32 v = get_unaligned_le32(*p);
24+ *p += sizeof(u32);
25+ return v;
26+}
27+static inline u16 ceph_decode_16(void **p)
28+{
29+ u16 v = get_unaligned_le16(*p);
30+ *p += sizeof(u16);
31+ return v;
32+}
33+static inline u8 ceph_decode_8(void **p)
34+{
35+ u8 v = *(u8 *)*p;
36+ (*p)++;
37+ return v;
38+}
39+static inline void ceph_decode_copy(void **p, void *pv, size_t n)
40+{
41+ memcpy(pv, *p, n);
42+ *p += n;
43+}
44+
45+/*
46+ * bounds check input.
47+ */
48+#define ceph_decode_need(p, end, n, bad) \
49+ do { \
50+ if (unlikely(*(p) + (n) > (end))) \
51+ goto bad; \
52+ } while (0)
53+
54+#define ceph_decode_64_safe(p, end, v, bad) \
55+ do { \
56+ ceph_decode_need(p, end, sizeof(u64), bad); \
57+ v = ceph_decode_64(p); \
58+ } while (0)
59+#define ceph_decode_32_safe(p, end, v, bad) \
60+ do { \
61+ ceph_decode_need(p, end, sizeof(u32), bad); \
62+ v = ceph_decode_32(p); \
63+ } while (0)
64+#define ceph_decode_16_safe(p, end, v, bad) \
65+ do { \
66+ ceph_decode_need(p, end, sizeof(u16), bad); \
67+ v = ceph_decode_16(p); \
68+ } while (0)
69+#define ceph_decode_8_safe(p, end, v, bad) \
70+ do { \
71+ ceph_decode_need(p, end, sizeof(u8), bad); \
72+ v = ceph_decode_8(p); \
73+ } while (0)
74+
75+#define ceph_decode_copy_safe(p, end, pv, n, bad) \
76+ do { \
77+ ceph_decode_need(p, end, n, bad); \
78+ ceph_decode_copy(p, pv, n); \
79+ } while (0)
80+
81+/*
82+ * struct ceph_timespec <-> struct timespec
83+ */
84+static inline void ceph_decode_timespec(struct timespec *ts,
85+ const struct ceph_timespec *tv)
86+{
87+ ts->tv_sec = le32_to_cpu(tv->tv_sec);
88+ ts->tv_nsec = le32_to_cpu(tv->tv_nsec);
89+}
90+static inline void ceph_encode_timespec(struct ceph_timespec *tv,
91+ const struct timespec *ts)
92+{
93+ tv->tv_sec = cpu_to_le32(ts->tv_sec);
94+ tv->tv_nsec = cpu_to_le32(ts->tv_nsec);
95+}
96+
97+/*
98+ * sockaddr_storage <-> ceph_sockaddr
99+ */
100+static inline void ceph_encode_addr(struct ceph_entity_addr *a)
101+{
102+ a->in_addr.ss_family = htons(a->in_addr.ss_family);
103+}
104+static inline void ceph_decode_addr(struct ceph_entity_addr *a)
105+{
106+ a->in_addr.ss_family = ntohs(a->in_addr.ss_family);
107+ WARN_ON(a->in_addr.ss_family == 512);
108+}
109+
110+/*
111+ * encoders
112+ */
113+static inline void ceph_encode_64(void **p, u64 v)
114+{
115+ put_unaligned_le64(v, (__le64 *)*p);
116+ *p += sizeof(u64);
117+}
118+static inline void ceph_encode_32(void **p, u32 v)
119+{
120+ put_unaligned_le32(v, (__le32 *)*p);
121+ *p += sizeof(u32);
122+}
123+static inline void ceph_encode_16(void **p, u16 v)
124+{
125+ put_unaligned_le16(v, (__le16 *)*p);
126+ *p += sizeof(u16);
127+}
128+static inline void ceph_encode_8(void **p, u8 v)
129+{
130+ *(u8 *)*p = v;
131+ (*p)++;
132+}
133+static inline void ceph_encode_copy(void **p, const void *s, int len)
134+{
135+ memcpy(*p, s, len);
136+ *p += len;
137+}
138+
139+/*
140+ * filepath, string encoders
141+ */
142+static inline void ceph_encode_filepath(void **p, void *end,
143+ u64 ino, const char *path)
144+{
145+ u32 len = path ? strlen(path) : 0;
146+ BUG_ON(*p + sizeof(ino) + sizeof(len) + len > end);
147+ ceph_encode_8(p, 1);
148+ ceph_encode_64(p, ino);
149+ ceph_encode_32(p, len);
150+ if (len)
151+ memcpy(*p, path, len);
152+ *p += len;
153+}
154+
155+static inline void ceph_encode_string(void **p, void *end,
156+ const char *s, u32 len)
157+{
158+ BUG_ON(*p + sizeof(len) + len > end);
159+ ceph_encode_32(p, len);
160+ if (len)
161+ memcpy(*p, s, len);
162+ *p += len;
163+}
164+
165+#define ceph_encode_need(p, end, n, bad) \
166+ do { \
167+ if (unlikely(*(p) + (n) > (end))) \
168+ goto bad; \
169+ } while (0)
170+
171+#define ceph_encode_64_safe(p, end, v, bad) \
172+ do { \
173+ ceph_encode_need(p, end, sizeof(u64), bad); \
174+ ceph_encode_64(p, v); \
175+ } while (0)
176+#define ceph_encode_32_safe(p, end, v, bad) \
177+ do { \
178+ ceph_encode_need(p, end, sizeof(u32), bad); \
179+ ceph_encode_32(p, v); \
180+ } while (0)
181+#define ceph_encode_16_safe(p, end, v, bad) \
182+ do { \
183+ ceph_encode_need(p, end, sizeof(u16), bad); \
184+ ceph_encode_16(p, v); \
185+ } while (0)
186+
187+#define ceph_encode_copy_safe(p, end, pv, n, bad) \
188+ do { \
189+ ceph_encode_need(p, end, n, bad); \
190+ ceph_encode_copy(p, pv, n); \
191+ } while (0)
192+
193+
194+#endif
fs/ceph/dir.c+1220−0
@@ -0,0 +1,1220 @@
1+#include "ceph_debug.h"
2+
3+#include <linux/spinlock.h>
4+#include <linux/fs_struct.h>
5+#include <linux/namei.h>
6+#include <linux/sched.h>
7+
8+#include "super.h"
9+
10+/*
11+ * Directory operations: readdir, lookup, create, link, unlink,
12+ * rename, etc.
13+ */
14+
15+/*
16+ * Ceph MDS operations are specified in terms of a base ino and
17+ * relative path. Thus, the client can specify an operation on a
18+ * specific inode (e.g., a getattr due to fstat(2)), or as a path
19+ * relative to, say, the root directory.
20+ *
21+ * Normally, we limit ourselves to strict inode ops (no path component)
22+ * or dentry operations (a single path component relative to an ino). The
23+ * exception to this is open_root_dentry(), which will open the mount
24+ * point by name.
25+ */
26+
27+const struct inode_operations ceph_dir_iops;
28+const struct file_operations ceph_dir_fops;
29+struct dentry_operations ceph_dentry_ops;
30+
31+/*
32+ * Initialize ceph dentry state.
33+ */
34+int ceph_init_dentry(struct dentry *dentry)
35+{
36+ struct ceph_dentry_info *di;
37+
38+ if (dentry->d_fsdata)
39+ return 0;
40+
41+ if (ceph_snap(dentry->d_parent->d_inode) == CEPH_NOSNAP)
42+ dentry->d_op = &ceph_dentry_ops;
43+ else if (ceph_snap(dentry->d_parent->d_inode) == CEPH_SNAPDIR)
44+ dentry->d_op = &ceph_snapdir_dentry_ops;
45+ else
46+ dentry->d_op = &ceph_snap_dentry_ops;
47+
48+ di = kmem_cache_alloc(ceph_dentry_cachep, GFP_NOFS);
49+ if (!di)
50+ return -ENOMEM; /* oh well */
51+
52+ spin_lock(&dentry->d_lock);
53+ if (dentry->d_fsdata) /* lost a race */
54+ goto out_unlock;
55+ di->dentry = dentry;
56+ di->lease_session = NULL;
57+ dentry->d_fsdata = di;
58+ dentry->d_time = jiffies;
59+ ceph_dentry_lru_add(dentry);
60+out_unlock:
61+ spin_unlock(&dentry->d_lock);
62+ return 0;
63+}
64+
65+
66+
67+/*
68+ * for readdir, we encode the directory frag and offset within that
69+ * frag into f_pos.
70+ */
71+static unsigned fpos_frag(loff_t p)
72+{
73+ return p >> 32;
74+}
75+static unsigned fpos_off(loff_t p)
76+{
77+ return p & 0xffffffff;
78+}
79+
80+/*
81+ * When possible, we try to satisfy a readdir by peeking at the
82+ * dcache. We make this work by carefully ordering dentries on
83+ * d_u.d_child when we initially get results back from the MDS, and
84+ * falling back to a "normal" sync readdir if any dentries in the dir
85+ * are dropped.
86+ *
87+ * I_COMPLETE tells indicates we have all dentries in the dir. It is
88+ * defined IFF we hold CEPH_CAP_FILE_SHARED (which will be revoked by
89+ * the MDS if/when the directory is modified).
90+ */
91+static int __dcache_readdir(struct file *filp,
92+ void *dirent, filldir_t filldir)
93+{
94+ struct inode *inode = filp->f_dentry->d_inode;
95+ struct ceph_file_info *fi = filp->private_data;
96+ struct dentry *parent = filp->f_dentry;
97+ struct inode *dir = parent->d_inode;
98+ struct list_head *p;
99+ struct dentry *dentry, *last;
100+ struct ceph_dentry_info *di;
101+ int err = 0;
102+
103+ /* claim ref on last dentry we returned */
104+ last = fi->dentry;
105+ fi->dentry = NULL;
106+
107+ dout("__dcache_readdir %p at %llu (last %p)\n", dir, filp->f_pos,
108+ last);
109+
110+ spin_lock(&dcache_lock);
111+
112+ /* start at beginning? */
113+ if (filp->f_pos == 2 || (last &&
114+ filp->f_pos < ceph_dentry(last)->offset)) {
115+ if (list_empty(&parent->d_subdirs))
116+ goto out_unlock;
117+ p = parent->d_subdirs.prev;
118+ dout(" initial p %p/%p\n", p->prev, p->next);
119+ } else {
120+ p = last->d_u.d_child.prev;
121+ }
122+
123+more:
124+ dentry = list_entry(p, struct dentry, d_u.d_child);
125+ di = ceph_dentry(dentry);
126+ while (1) {
127+ dout(" p %p/%p d_subdirs %p/%p\n", p->prev, p->next,
128+ parent->d_subdirs.prev, parent->d_subdirs.next);
129+ if (p == &parent->d_subdirs) {
130+ fi->at_end = 1;
131+ goto out_unlock;
132+ }
133+ if (!d_unhashed(dentry) && dentry->d_inode &&
134+ ceph_snap(dentry->d_inode) != CEPH_SNAPDIR &&
135+ ceph_ino(dentry->d_inode) != CEPH_INO_CEPH &&
136+ filp->f_pos <= di->offset)
137+ break;
138+ dout(" skipping %p %.*s at %llu (%llu)%s%s\n", dentry,
139+ dentry->d_name.len, dentry->d_name.name, di->offset,
140+ filp->f_pos, d_unhashed(dentry) ? " unhashed" : "",
141+ !dentry->d_inode ? " null" : "");
142+ p = p->prev;
143+ dentry = list_entry(p, struct dentry, d_u.d_child);
144+ di = ceph_dentry(dentry);
145+ }
146+
147+ atomic_inc(&dentry->d_count);
148+ spin_unlock(&dcache_lock);
149+ spin_unlock(&inode->i_lock);
150+
151+ dout(" %llu (%llu) dentry %p %.*s %p\n", di->offset, filp->f_pos,
152+ dentry, dentry->d_name.len, dentry->d_name.name, dentry->d_inode);
153+ filp->f_pos = di->offset;
154+ err = filldir(dirent, dentry->d_name.name,
155+ dentry->d_name.len, di->offset,
156+ dentry->d_inode->i_ino,
157+ dentry->d_inode->i_mode >> 12);
158+
159+ if (last) {
160+ if (err < 0) {
161+ /* remember our position */
162+ fi->dentry = last;
163+ fi->next_offset = di->offset;
164+ } else {
165+ dput(last);
166+ }
167+ last = NULL;
168+ }
169+
170+ spin_lock(&inode->i_lock);
171+ spin_lock(&dcache_lock);
172+
173+ if (err < 0)
174+ goto out_unlock;
175+
176+ last = dentry;
177+
178+ p = p->prev;
179+ filp->f_pos++;
180+
181+ /* make sure a dentry wasn't dropped while we didn't have dcache_lock */
182+ if ((ceph_inode(dir)->i_ceph_flags & CEPH_I_COMPLETE))
183+ goto more;
184+ dout(" lost I_COMPLETE on %p; falling back to mds\n", dir);
185+ err = -EAGAIN;
186+
187+out_unlock:
188+ spin_unlock(&dcache_lock);
189+
190+ if (last) {
191+ spin_unlock(&inode->i_lock);
192+ dput(last);
193+ spin_lock(&inode->i_lock);
194+ }
195+
196+ return err;
197+}
198+
199+/*
200+ * make note of the last dentry we read, so we can
201+ * continue at the same lexicographical point,
202+ * regardless of what dir changes take place on the
203+ * server.
204+ */
205+static int note_last_dentry(struct ceph_file_info *fi, const char *name,
206+ int len)
207+{
208+ kfree(fi->last_name);
209+ fi->last_name = kmalloc(len+1, GFP_NOFS);
210+ if (!fi->last_name)
211+ return -ENOMEM;
212+ memcpy(fi->last_name, name, len);
213+ fi->last_name[len] = 0;
214+ dout("note_last_dentry '%s'\n", fi->last_name);
215+ return 0;
216+}
217+
218+static int ceph_readdir(struct file *filp, void *dirent, filldir_t filldir)
219+{
220+ struct ceph_file_info *fi = filp->private_data;
221+ struct inode *inode = filp->f_dentry->d_inode;
222+ struct ceph_inode_info *ci = ceph_inode(inode);
223+ struct ceph_client *client = ceph_inode_to_client(inode);
224+ struct ceph_mds_client *mdsc = &client->mdsc;
225+ unsigned frag = fpos_frag(filp->f_pos);
226+ int off = fpos_off(filp->f_pos);
227+ int err;
228+ u32 ftype;
229+ struct ceph_mds_reply_info_parsed *rinfo;
230+ const int max_entries = client->mount_args->max_readdir;
231+
232+ dout("readdir %p filp %p frag %u off %u\n", inode, filp, frag, off);
233+ if (fi->at_end)
234+ return 0;
235+
236+ /* always start with . and .. */
237+ if (filp->f_pos == 0) {
238+ /* note dir version at start of readdir so we can tell
239+ * if any dentries get dropped */
240+ fi->dir_release_count = ci->i_release_count;
241+
242+ dout("readdir off 0 -> '.'\n");
243+ if (filldir(dirent, ".", 1, ceph_make_fpos(0, 0),
244+ inode->i_ino, inode->i_mode >> 12) < 0)
245+ return 0;
246+ filp->f_pos = 1;
247+ off = 1;
248+ }
249+ if (filp->f_pos == 1) {
250+ dout("readdir off 1 -> '..'\n");
251+ if (filldir(dirent, "..", 2, ceph_make_fpos(0, 1),
252+ filp->f_dentry->d_parent->d_inode->i_ino,
253+ inode->i_mode >> 12) < 0)
254+ return 0;
255+ filp->f_pos = 2;
256+ off = 2;
257+ }
258+
259+ /* can we use the dcache? */
260+ spin_lock(&inode->i_lock);
261+ if ((filp->f_pos == 2 || fi->dentry) &&
262+ !ceph_test_opt(client, NOASYNCREADDIR) &&
263+ (ci->i_ceph_flags & CEPH_I_COMPLETE) &&
264+ __ceph_caps_issued_mask(ci, CEPH_CAP_FILE_SHARED, 1)) {
265+ err = __dcache_readdir(filp, dirent, filldir);
266+ if (err != -EAGAIN) {
267+ spin_unlock(&inode->i_lock);
268+ return err;
269+ }
270+ }
271+ spin_unlock(&inode->i_lock);
272+ if (fi->dentry) {
273+ err = note_last_dentry(fi, fi->dentry->d_name.name,
274+ fi->dentry->d_name.len);
275+ if (err)
276+ return err;
277+ dput(fi->dentry);
278+ fi->dentry = NULL;
279+ }
280+
281+ /* proceed with a normal readdir */
282+
283+more:
284+ /* do we have the correct frag content buffered? */
285+ if (fi->frag != frag || fi->last_readdir == NULL) {
286+ struct ceph_mds_request *req;
287+ int op = ceph_snap(inode) == CEPH_SNAPDIR ?
288+ CEPH_MDS_OP_LSSNAP : CEPH_MDS_OP_READDIR;
289+
290+ /* discard old result, if any */
291+ if (fi->last_readdir)
292+ ceph_mdsc_put_request(fi->last_readdir);
293+
294+ /* requery frag tree, as the frag topology may have changed */
295+ frag = ceph_choose_frag(ceph_inode(inode), frag, NULL, NULL);
296+
297+ dout("readdir fetching %llx.%llx frag %x offset '%s'\n",
298+ ceph_vinop(inode), frag, fi->last_name);
299+ req = ceph_mdsc_create_request(mdsc, op, USE_AUTH_MDS);
300+ if (IS_ERR(req))
301+ return PTR_ERR(req);
302+ req->r_inode = igrab(inode);
303+ req->r_dentry = dget(filp->f_dentry);
304+ /* hints to request -> mds selection code */
305+ req->r_direct_mode = USE_AUTH_MDS;
306+ req->r_direct_hash = ceph_frag_value(frag);
307+ req->r_direct_is_hash = true;
308+ req->r_path2 = kstrdup(fi->last_name, GFP_NOFS);
309+ req->r_readdir_offset = fi->next_offset;
310+ req->r_args.readdir.frag = cpu_to_le32(frag);
311+ req->r_args.readdir.max_entries = cpu_to_le32(max_entries);
312+ req->r_num_caps = max_entries;
313+ err = ceph_mdsc_do_request(mdsc, NULL, req);
314+ if (err < 0) {
315+ ceph_mdsc_put_request(req);
316+ return err;
317+ }
318+ dout("readdir got and parsed readdir result=%d"
319+ " on frag %x, end=%d, complete=%d\n", err, frag,
320+ (int)req->r_reply_info.dir_end,
321+ (int)req->r_reply_info.dir_complete);
322+
323+ if (!req->r_did_prepopulate) {
324+ dout("readdir !did_prepopulate");
325+ fi->dir_release_count--; /* preclude I_COMPLETE */
326+ }
327+
328+ /* note next offset and last dentry name */
329+ fi->offset = fi->next_offset;
330+ fi->last_readdir = req;
331+
332+ if (req->r_reply_info.dir_end) {
333+ kfree(fi->last_name);
334+ fi->last_name = NULL;
335+ fi->next_offset = 0;
336+ } else {
337+ rinfo = &req->r_reply_info;
338+ err = note_last_dentry(fi,
339+ rinfo->dir_dname[rinfo->dir_nr-1],
340+ rinfo->dir_dname_len[rinfo->dir_nr-1]);
341+ if (err)
342+ return err;
343+ fi->next_offset += rinfo->dir_nr;
344+ }
345+ }
346+
347+ rinfo = &fi->last_readdir->r_reply_info;
348+ dout("readdir frag %x num %d off %d chunkoff %d\n", frag,
349+ rinfo->dir_nr, off, fi->offset);
350+ while (off - fi->offset >= 0 && off - fi->offset < rinfo->dir_nr) {
351+ u64 pos = ceph_make_fpos(frag, off);
352+ struct ceph_mds_reply_inode *in =
353+ rinfo->dir_in[off - fi->offset].in;
354+ dout("readdir off %d (%d/%d) -> %lld '%.*s' %p\n",
355+ off, off - fi->offset, rinfo->dir_nr, pos,
356+ rinfo->dir_dname_len[off - fi->offset],
357+ rinfo->dir_dname[off - fi->offset], in);
358+ BUG_ON(!in);
359+ ftype = le32_to_cpu(in->mode) >> 12;
360+ if (filldir(dirent,
361+ rinfo->dir_dname[off - fi->offset],
362+ rinfo->dir_dname_len[off - fi->offset],
363+ pos,
364+ le64_to_cpu(in->ino),
365+ ftype) < 0) {
366+ dout("filldir stopping us...\n");
367+ return 0;
368+ }
369+ off++;
370+ filp->f_pos = pos + 1;
371+ }
372+
373+ if (fi->last_name) {
374+ ceph_mdsc_put_request(fi->last_readdir);
375+ fi->last_readdir = NULL;
376+ goto more;
377+ }
378+
379+ /* more frags? */
380+ if (!ceph_frag_is_rightmost(frag)) {
381+ frag = ceph_frag_next(frag);
382+ off = 0;
383+ filp->f_pos = ceph_make_fpos(frag, off);
384+ dout("readdir next frag is %x\n", frag);
385+ goto more;
386+ }
387+ fi->at_end = 1;
388+
389+ /*
390+ * if dir_release_count still matches the dir, no dentries
391+ * were released during the whole readdir, and we should have
392+ * the complete dir contents in our cache.
393+ */
394+ spin_lock(&inode->i_lock);
395+ if (ci->i_release_count == fi->dir_release_count) {
396+ dout(" marking %p complete\n", inode);
397+ ci->i_ceph_flags |= CEPH_I_COMPLETE;
398+ ci->i_max_offset = filp->f_pos;
399+ }
400+ spin_unlock(&inode->i_lock);
401+
402+ dout("readdir %p filp %p done.\n", inode, filp);
403+ return 0;
404+}
405+
406+static void reset_readdir(struct ceph_file_info *fi)
407+{
408+ if (fi->last_readdir) {
409+ ceph_mdsc_put_request(fi->last_readdir);
410+ fi->last_readdir = NULL;
411+ }
412+ kfree(fi->last_name);
413+ fi->next_offset = 2; /* compensate for . and .. */
414+ if (fi->dentry) {
415+ dput(fi->dentry);
416+ fi->dentry = NULL;
417+ }
418+ fi->at_end = 0;
419+}
420+
421+static loff_t ceph_dir_llseek(struct file *file, loff_t offset, int origin)
422+{
423+ struct ceph_file_info *fi = file->private_data;
424+ struct inode *inode = file->f_mapping->host;
425+ loff_t old_offset = offset;
426+ loff_t retval;
427+
428+ mutex_lock(&inode->i_mutex);
429+ switch (origin) {
430+ case SEEK_END:
431+ offset += inode->i_size + 2; /* FIXME */
432+ break;
433+ case SEEK_CUR:
434+ offset += file->f_pos;
435+ }
436+ retval = -EINVAL;
437+ if (offset >= 0 && offset <= inode->i_sb->s_maxbytes) {
438+ if (offset != file->f_pos) {
439+ file->f_pos = offset;
440+ file->f_version = 0;
441+ fi->at_end = 0;
442+ }
443+ retval = offset;
444+
445+ /*
446+ * discard buffered readdir content on seekdir(0), or
447+ * seek to new frag, or seek prior to current chunk.
448+ */
449+ if (offset == 0 ||
450+ fpos_frag(offset) != fpos_frag(old_offset) ||
451+ fpos_off(offset) < fi->offset) {
452+ dout("dir_llseek dropping %p content\n", file);
453+ reset_readdir(fi);
454+ }
455+
456+ /* bump dir_release_count if we did a forward seek */
457+ if (offset > old_offset)
458+ fi->dir_release_count--;
459+ }
460+ mutex_unlock(&inode->i_mutex);
461+ return retval;
462+}
463+
464+/*
465+ * Process result of a lookup/open request.
466+ *
467+ * Mainly, make sure we return the final req->r_dentry (if it already
468+ * existed) in place of the original VFS-provided dentry when they
469+ * differ.
470+ *
471+ * Gracefully handle the case where the MDS replies with -ENOENT and
472+ * no trace (which it may do, at its discretion, e.g., if it doesn't
473+ * care to issue a lease on the negative dentry).
474+ */
475+struct dentry *ceph_finish_lookup(struct ceph_mds_request *req,
476+ struct dentry *dentry, int err)
477+{
478+ struct ceph_client *client = ceph_client(dentry->d_sb);
479+ struct inode *parent = dentry->d_parent->d_inode;
480+
481+ /* .snap dir? */
482+ if (err == -ENOENT &&
483+ ceph_vino(parent).ino != CEPH_INO_ROOT && /* no .snap in root dir */
484+ strcmp(dentry->d_name.name,
485+ client->mount_args->snapdir_name) == 0) {
486+ struct inode *inode = ceph_get_snapdir(parent);
487+ dout("ENOENT on snapdir %p '%.*s', linking to snapdir %p\n",
488+ dentry, dentry->d_name.len, dentry->d_name.name, inode);
489+ d_add(dentry, inode);
490+ err = 0;
491+ }
492+
493+ if (err == -ENOENT) {
494+ /* no trace? */
495+ err = 0;
496+ if (!req->r_reply_info.head->is_dentry) {
497+ dout("ENOENT and no trace, dentry %p inode %p\n",
498+ dentry, dentry->d_inode);
499+ if (dentry->d_inode) {
500+ d_drop(dentry);
501+ err = -ENOENT;
502+ } else {
503+ d_add(dentry, NULL);
504+ }
505+ }
506+ }
507+ if (err)
508+ dentry = ERR_PTR(err);
509+ else if (dentry != req->r_dentry)
510+ dentry = dget(req->r_dentry); /* we got spliced */
511+ else
512+ dentry = NULL;
513+ return dentry;
514+}
515+
516+static int is_root_ceph_dentry(struct inode *inode, struct dentry *dentry)
517+{
518+ return ceph_ino(inode) == CEPH_INO_ROOT &&
519+ strncmp(dentry->d_name.name, ".ceph", 5) == 0;
520+}
521+
522+/*
523+ * Look up a single dir entry. If there is a lookup intent, inform
524+ * the MDS so that it gets our 'caps wanted' value in a single op.
525+ */
526+static struct dentry *ceph_lookup(struct inode *dir, struct dentry *dentry,
527+ struct nameidata *nd)
528+{
529+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
530+ struct ceph_mds_client *mdsc = &client->mdsc;
531+ struct ceph_mds_request *req;
532+ int op;
533+ int err;
534+
535+ dout("lookup %p dentry %p '%.*s'\n",
536+ dir, dentry, dentry->d_name.len, dentry->d_name.name);
537+
538+ if (dentry->d_name.len > NAME_MAX)
539+ return ERR_PTR(-ENAMETOOLONG);
540+
541+ err = ceph_init_dentry(dentry);
542+ if (err < 0)
543+ return ERR_PTR(err);
544+
545+ /* open (but not create!) intent? */
546+ if (nd &&
547+ (nd->flags & LOOKUP_OPEN) &&
548+ (nd->flags & LOOKUP_CONTINUE) == 0 && /* only open last component */
549+ !(nd->intent.open.flags & O_CREAT)) {
550+ int mode = nd->intent.open.create_mode & ~current->fs->umask;
551+ return ceph_lookup_open(dir, dentry, nd, mode, 1);
552+ }
553+
554+ /* can we conclude ENOENT locally? */
555+ if (dentry->d_inode == NULL) {
556+ struct ceph_inode_info *ci = ceph_inode(dir);
557+ struct ceph_dentry_info *di = ceph_dentry(dentry);
558+
559+ spin_lock(&dir->i_lock);
560+ dout(" dir %p flags are %d\n", dir, ci->i_ceph_flags);
561+ if (strncmp(dentry->d_name.name,
562+ client->mount_args->snapdir_name,
563+ dentry->d_name.len) &&
564+ !is_root_ceph_dentry(dir, dentry) &&
565+ (ci->i_ceph_flags & CEPH_I_COMPLETE) &&
566+ (__ceph_caps_issued_mask(ci, CEPH_CAP_FILE_SHARED, 1))) {
567+ di->offset = ci->i_max_offset++;
568+ spin_unlock(&dir->i_lock);
569+ dout(" dir %p complete, -ENOENT\n", dir);
570+ d_add(dentry, NULL);
571+ di->lease_shared_gen = ci->i_shared_gen;
572+ return NULL;
573+ }
574+ spin_unlock(&dir->i_lock);
575+ }
576+
577+ op = ceph_snap(dir) == CEPH_SNAPDIR ?
578+ CEPH_MDS_OP_LOOKUPSNAP : CEPH_MDS_OP_LOOKUP;
579+ req = ceph_mdsc_create_request(mdsc, op, USE_ANY_MDS);
580+ if (IS_ERR(req))
581+ return ERR_PTR(PTR_ERR(req));
582+ req->r_dentry = dget(dentry);
583+ req->r_num_caps = 2;
584+ /* we only need inode linkage */
585+ req->r_args.getattr.mask = cpu_to_le32(CEPH_STAT_CAP_INODE);
586+ req->r_locked_dir = dir;
587+ err = ceph_mdsc_do_request(mdsc, NULL, req);
588+ dentry = ceph_finish_lookup(req, dentry, err);
589+ ceph_mdsc_put_request(req); /* will dput(dentry) */
590+ dout("lookup result=%p\n", dentry);
591+ return dentry;
592+}
593+
594+/*
595+ * If we do a create but get no trace back from the MDS, follow up with
596+ * a lookup (the VFS expects us to link up the provided dentry).
597+ */
598+int ceph_handle_notrace_create(struct inode *dir, struct dentry *dentry)
599+{
600+ struct dentry *result = ceph_lookup(dir, dentry, NULL);
601+
602+ if (result && !IS_ERR(result)) {
603+ /*
604+ * We created the item, then did a lookup, and found
605+ * it was already linked to another inode we already
606+ * had in our cache (and thus got spliced). Link our
607+ * dentry to that inode, but don't hash it, just in
608+ * case the VFS wants to dereference it.
609+ */
610+ BUG_ON(!result->d_inode);
611+ d_instantiate(dentry, result->d_inode);
612+ return 0;
613+ }
614+ return PTR_ERR(result);
615+}
616+
617+static int ceph_mknod(struct inode *dir, struct dentry *dentry,
618+ int mode, dev_t rdev)
619+{
620+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
621+ struct ceph_mds_client *mdsc = &client->mdsc;
622+ struct ceph_mds_request *req;
623+ int err;
624+
625+ if (ceph_snap(dir) != CEPH_NOSNAP)
626+ return -EROFS;
627+
628+ dout("mknod in dir %p dentry %p mode 0%o rdev %d\n",
629+ dir, dentry, mode, rdev);
630+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_MKNOD, USE_AUTH_MDS);
631+ if (IS_ERR(req)) {
632+ d_drop(dentry);
633+ return PTR_ERR(req);
634+ }
635+ req->r_dentry = dget(dentry);
636+ req->r_num_caps = 2;
637+ req->r_locked_dir = dir;
638+ req->r_args.mknod.mode = cpu_to_le32(mode);
639+ req->r_args.mknod.rdev = cpu_to_le32(rdev);
640+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
641+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
642+ err = ceph_mdsc_do_request(mdsc, dir, req);
643+ if (!err && !req->r_reply_info.head->is_dentry)
644+ err = ceph_handle_notrace_create(dir, dentry);
645+ ceph_mdsc_put_request(req);
646+ if (err)
647+ d_drop(dentry);
648+ return err;
649+}
650+
651+static int ceph_create(struct inode *dir, struct dentry *dentry, int mode,
652+ struct nameidata *nd)
653+{
654+ dout("create in dir %p dentry %p name '%.*s'\n",
655+ dir, dentry, dentry->d_name.len, dentry->d_name.name);
656+
657+ if (ceph_snap(dir) != CEPH_NOSNAP)
658+ return -EROFS;
659+
660+ if (nd) {
661+ BUG_ON((nd->flags & LOOKUP_OPEN) == 0);
662+ dentry = ceph_lookup_open(dir, dentry, nd, mode, 0);
663+ /* hrm, what should i do here if we get aliased? */
664+ if (IS_ERR(dentry))
665+ return PTR_ERR(dentry);
666+ return 0;
667+ }
668+
669+ /* fall back to mknod */
670+ return ceph_mknod(dir, dentry, (mode & ~S_IFMT) | S_IFREG, 0);
671+}
672+
673+static int ceph_symlink(struct inode *dir, struct dentry *dentry,
674+ const char *dest)
675+{
676+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
677+ struct ceph_mds_client *mdsc = &client->mdsc;
678+ struct ceph_mds_request *req;
679+ int err;
680+
681+ if (ceph_snap(dir) != CEPH_NOSNAP)
682+ return -EROFS;
683+
684+ dout("symlink in dir %p dentry %p to '%s'\n", dir, dentry, dest);
685+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_SYMLINK, USE_AUTH_MDS);
686+ if (IS_ERR(req)) {
687+ d_drop(dentry);
688+ return PTR_ERR(req);
689+ }
690+ req->r_dentry = dget(dentry);
691+ req->r_num_caps = 2;
692+ req->r_path2 = kstrdup(dest, GFP_NOFS);
693+ req->r_locked_dir = dir;
694+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
695+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
696+ err = ceph_mdsc_do_request(mdsc, dir, req);
697+ if (!err && !req->r_reply_info.head->is_dentry)
698+ err = ceph_handle_notrace_create(dir, dentry);
699+ ceph_mdsc_put_request(req);
700+ if (err)
701+ d_drop(dentry);
702+ return err;
703+}
704+
705+static int ceph_mkdir(struct inode *dir, struct dentry *dentry, int mode)
706+{
707+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
708+ struct ceph_mds_client *mdsc = &client->mdsc;
709+ struct ceph_mds_request *req;
710+ int err = -EROFS;
711+ int op;
712+
713+ if (ceph_snap(dir) == CEPH_SNAPDIR) {
714+ /* mkdir .snap/foo is a MKSNAP */
715+ op = CEPH_MDS_OP_MKSNAP;
716+ dout("mksnap dir %p snap '%.*s' dn %p\n", dir,
717+ dentry->d_name.len, dentry->d_name.name, dentry);
718+ } else if (ceph_snap(dir) == CEPH_NOSNAP) {
719+ dout("mkdir dir %p dn %p mode 0%o\n", dir, dentry, mode);
720+ op = CEPH_MDS_OP_MKDIR;
721+ } else {
722+ goto out;
723+ }
724+ req = ceph_mdsc_create_request(mdsc, op, USE_AUTH_MDS);
725+ if (IS_ERR(req)) {
726+ err = PTR_ERR(req);
727+ goto out;
728+ }
729+
730+ req->r_dentry = dget(dentry);
731+ req->r_num_caps = 2;
732+ req->r_locked_dir = dir;
733+ req->r_args.mkdir.mode = cpu_to_le32(mode);
734+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
735+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
736+ err = ceph_mdsc_do_request(mdsc, dir, req);
737+ if (!err && !req->r_reply_info.head->is_dentry)
738+ err = ceph_handle_notrace_create(dir, dentry);
739+ ceph_mdsc_put_request(req);
740+out:
741+ if (err < 0)
742+ d_drop(dentry);
743+ return err;
744+}
745+
746+static int ceph_link(struct dentry *old_dentry, struct inode *dir,
747+ struct dentry *dentry)
748+{
749+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
750+ struct ceph_mds_client *mdsc = &client->mdsc;
751+ struct ceph_mds_request *req;
752+ int err;
753+
754+ if (ceph_snap(dir) != CEPH_NOSNAP)
755+ return -EROFS;
756+
757+ dout("link in dir %p old_dentry %p dentry %p\n", dir,
758+ old_dentry, dentry);
759+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_LINK, USE_AUTH_MDS);
760+ if (IS_ERR(req)) {
761+ d_drop(dentry);
762+ return PTR_ERR(req);
763+ }
764+ req->r_dentry = dget(dentry);
765+ req->r_num_caps = 2;
766+ req->r_old_dentry = dget(old_dentry); /* or inode? hrm. */
767+ req->r_locked_dir = dir;
768+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
769+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
770+ err = ceph_mdsc_do_request(mdsc, dir, req);
771+ if (err)
772+ d_drop(dentry);
773+ else if (!req->r_reply_info.head->is_dentry)
774+ d_instantiate(dentry, igrab(old_dentry->d_inode));
775+ ceph_mdsc_put_request(req);
776+ return err;
777+}
778+
779+/*
780+ * For a soon-to-be unlinked file, drop the AUTH_RDCACHE caps. If it
781+ * looks like the link count will hit 0, drop any other caps (other
782+ * than PIN) we don't specifically want (due to the file still being
783+ * open).
784+ */
785+static int drop_caps_for_unlink(struct inode *inode)
786+{
787+ struct ceph_inode_info *ci = ceph_inode(inode);
788+ int drop = CEPH_CAP_LINK_SHARED | CEPH_CAP_LINK_EXCL;
789+
790+ spin_lock(&inode->i_lock);
791+ if (inode->i_nlink == 1) {
792+ drop |= ~(__ceph_caps_wanted(ci) | CEPH_CAP_PIN);
793+ ci->i_ceph_flags |= CEPH_I_NODELAY;
794+ }
795+ spin_unlock(&inode->i_lock);
796+ return drop;
797+}
798+
799+/*
800+ * rmdir and unlink are differ only by the metadata op code
801+ */
802+static int ceph_unlink(struct inode *dir, struct dentry *dentry)
803+{
804+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
805+ struct ceph_mds_client *mdsc = &client->mdsc;
806+ struct inode *inode = dentry->d_inode;
807+ struct ceph_mds_request *req;
808+ int err = -EROFS;
809+ int op;
810+
811+ if (ceph_snap(dir) == CEPH_SNAPDIR) {
812+ /* rmdir .snap/foo is RMSNAP */
813+ dout("rmsnap dir %p '%.*s' dn %p\n", dir, dentry->d_name.len,
814+ dentry->d_name.name, dentry);
815+ op = CEPH_MDS_OP_RMSNAP;
816+ } else if (ceph_snap(dir) == CEPH_NOSNAP) {
817+ dout("unlink/rmdir dir %p dn %p inode %p\n",
818+ dir, dentry, inode);
819+ op = ((dentry->d_inode->i_mode & S_IFMT) == S_IFDIR) ?
820+ CEPH_MDS_OP_RMDIR : CEPH_MDS_OP_UNLINK;
821+ } else
822+ goto out;
823+ req = ceph_mdsc_create_request(mdsc, op, USE_AUTH_MDS);
824+ if (IS_ERR(req)) {
825+ err = PTR_ERR(req);
826+ goto out;
827+ }
828+ req->r_dentry = dget(dentry);
829+ req->r_num_caps = 2;
830+ req->r_locked_dir = dir;
831+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
832+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
833+ req->r_inode_drop = drop_caps_for_unlink(inode);
834+ err = ceph_mdsc_do_request(mdsc, dir, req);
835+ if (!err && !req->r_reply_info.head->is_dentry)
836+ d_delete(dentry);
837+ ceph_mdsc_put_request(req);
838+out:
839+ return err;
840+}
841+
842+static int ceph_rename(struct inode *old_dir, struct dentry *old_dentry,
843+ struct inode *new_dir, struct dentry *new_dentry)
844+{
845+ struct ceph_client *client = ceph_sb_to_client(old_dir->i_sb);
846+ struct ceph_mds_client *mdsc = &client->mdsc;
847+ struct ceph_mds_request *req;
848+ int err;
849+
850+ if (ceph_snap(old_dir) != ceph_snap(new_dir))
851+ return -EXDEV;
852+ if (ceph_snap(old_dir) != CEPH_NOSNAP ||
853+ ceph_snap(new_dir) != CEPH_NOSNAP)
854+ return -EROFS;
855+ dout("rename dir %p dentry %p to dir %p dentry %p\n",
856+ old_dir, old_dentry, new_dir, new_dentry);
857+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_RENAME, USE_AUTH_MDS);
858+ if (IS_ERR(req))
859+ return PTR_ERR(req);
860+ req->r_dentry = dget(new_dentry);
861+ req->r_num_caps = 2;
862+ req->r_old_dentry = dget(old_dentry);
863+ req->r_locked_dir = new_dir;
864+ req->r_old_dentry_drop = CEPH_CAP_FILE_SHARED;
865+ req->r_old_dentry_unless = CEPH_CAP_FILE_EXCL;
866+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
867+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
868+ /* release LINK_RDCACHE on source inode (mds will lock it) */
869+ req->r_old_inode_drop = CEPH_CAP_LINK_SHARED;
870+ if (new_dentry->d_inode)
871+ req->r_inode_drop = drop_caps_for_unlink(new_dentry->d_inode);
872+ err = ceph_mdsc_do_request(mdsc, old_dir, req);
873+ if (!err && !req->r_reply_info.head->is_dentry) {
874+ /*
875+ * Normally d_move() is done by fill_trace (called by
876+ * do_request, above). If there is no trace, we need
877+ * to do it here.
878+ */
879+ d_move(old_dentry, new_dentry);
880+ }
881+ ceph_mdsc_put_request(req);
882+ return err;
883+}
884+
885+
886+/*
887+ * Check if dentry lease is valid. If not, delete the lease. Try to
888+ * renew if the least is more than half up.
889+ */
890+static int dentry_lease_is_valid(struct dentry *dentry)
891+{
892+ struct ceph_dentry_info *di;
893+ struct ceph_mds_session *s;
894+ int valid = 0;
895+ u32 gen;
896+ unsigned long ttl;
897+ struct ceph_mds_session *session = NULL;
898+ struct inode *dir = NULL;
899+ u32 seq = 0;
900+
901+ spin_lock(&dentry->d_lock);
902+ di = ceph_dentry(dentry);
903+ if (di && di->lease_session) {
904+ s = di->lease_session;
905+ spin_lock(&s->s_cap_lock);
906+ gen = s->s_cap_gen;
907+ ttl = s->s_cap_ttl;
908+ spin_unlock(&s->s_cap_lock);
909+
910+ if (di->lease_gen == gen &&
911+ time_before(jiffies, dentry->d_time) &&
912+ time_before(jiffies, ttl)) {
913+ valid = 1;
914+ if (di->lease_renew_after &&
915+ time_after(jiffies, di->lease_renew_after)) {
916+ /* we should renew */
917+ dir = dentry->d_parent->d_inode;
918+ session = ceph_get_mds_session(s);
919+ seq = di->lease_seq;
920+ di->lease_renew_after = 0;
921+ di->lease_renew_from = jiffies;
922+ }
923+ }
924+ }
925+ spin_unlock(&dentry->d_lock);
926+
927+ if (session) {
928+ ceph_mdsc_lease_send_msg(session, dir, dentry,
929+ CEPH_MDS_LEASE_RENEW, seq);
930+ ceph_put_mds_session(session);
931+ }
932+ dout("dentry_lease_is_valid - dentry %p = %d\n", dentry, valid);
933+ return valid;
934+}
935+
936+/*
937+ * Check if directory-wide content lease/cap is valid.
938+ */
939+static int dir_lease_is_valid(struct inode *dir, struct dentry *dentry)
940+{
941+ struct ceph_inode_info *ci = ceph_inode(dir);
942+ struct ceph_dentry_info *di = ceph_dentry(dentry);
943+ int valid = 0;
944+
945+ spin_lock(&dir->i_lock);
946+ if (ci->i_shared_gen == di->lease_shared_gen)
947+ valid = __ceph_caps_issued_mask(ci, CEPH_CAP_FILE_SHARED, 1);
948+ spin_unlock(&dir->i_lock);
949+ dout("dir_lease_is_valid dir %p v%u dentry %p v%u = %d\n",
950+ dir, (unsigned)ci->i_shared_gen, dentry,
951+ (unsigned)di->lease_shared_gen, valid);
952+ return valid;
953+}
954+
955+/*
956+ * Check if cached dentry can be trusted.
957+ */
958+static int ceph_d_revalidate(struct dentry *dentry, struct nameidata *nd)
959+{
960+ struct inode *dir = dentry->d_parent->d_inode;
961+
962+ dout("d_revalidate %p '%.*s' inode %p\n", dentry,
963+ dentry->d_name.len, dentry->d_name.name, dentry->d_inode);
964+
965+ /* always trust cached snapped dentries, snapdir dentry */
966+ if (ceph_snap(dir) != CEPH_NOSNAP) {
967+ dout("d_revalidate %p '%.*s' inode %p is SNAPPED\n", dentry,
968+ dentry->d_name.len, dentry->d_name.name, dentry->d_inode);
969+ goto out_touch;
970+ }
971+ if (dentry->d_inode && ceph_snap(dentry->d_inode) == CEPH_SNAPDIR)
972+ goto out_touch;
973+
974+ if (dentry_lease_is_valid(dentry) ||
975+ dir_lease_is_valid(dir, dentry))
976+ goto out_touch;
977+
978+ dout("d_revalidate %p invalid\n", dentry);
979+ d_drop(dentry);
980+ return 0;
981+out_touch:
982+ ceph_dentry_lru_touch(dentry);
983+ return 1;
984+}
985+
986+/*
987+ * When a dentry is released, clear the dir I_COMPLETE if it was part
988+ * of the current dir gen.
989+ */
990+static void ceph_dentry_release(struct dentry *dentry)
991+{
992+ struct ceph_dentry_info *di = ceph_dentry(dentry);
993+ struct inode *parent_inode = dentry->d_parent->d_inode;
994+
995+ if (parent_inode) {
996+ struct ceph_inode_info *ci = ceph_inode(parent_inode);
997+
998+ spin_lock(&parent_inode->i_lock);
999+ if (ci->i_shared_gen == di->lease_shared_gen) {
1000+ dout(" clearing %p complete (d_release)\n",
1001+ parent_inode);
1002+ ci->i_ceph_flags &= ~CEPH_I_COMPLETE;
1003+ ci->i_release_count++;
1004+ }
1005+ spin_unlock(&parent_inode->i_lock);
1006+ }
1007+ if (di) {
1008+ ceph_dentry_lru_del(dentry);
1009+ if (di->lease_session)
1010+ ceph_put_mds_session(di->lease_session);
1011+ kmem_cache_free(ceph_dentry_cachep, di);
1012+ dentry->d_fsdata = NULL;
1013+ }
1014+}
1015+
1016+static int ceph_snapdir_d_revalidate(struct dentry *dentry,
1017+ struct nameidata *nd)
1018+{
1019+ /*
1020+ * Eventually, we'll want to revalidate snapped metadata
1021+ * too... probably...
1022+ */
1023+ return 1;
1024+}
1025+
1026+
1027+
1028+/*
1029+ * read() on a dir. This weird interface hack only works if mounted
1030+ * with '-o dirstat'.
1031+ */
1032+static ssize_t ceph_read_dir(struct file *file, char __user *buf, size_t size,
1033+ loff_t *ppos)
1034+{
1035+ struct ceph_file_info *cf = file->private_data;
1036+ struct inode *inode = file->f_dentry->d_inode;
1037+ struct ceph_inode_info *ci = ceph_inode(inode);
1038+ int left;
1039+
1040+ if (!ceph_test_opt(ceph_client(inode->i_sb), DIRSTAT))
1041+ return -EISDIR;
1042+
1043+ if (!cf->dir_info) {
1044+ cf->dir_info = kmalloc(1024, GFP_NOFS);
1045+ if (!cf->dir_info)
1046+ return -ENOMEM;
1047+ cf->dir_info_len =
1048+ sprintf(cf->dir_info,
1049+ "entries: %20lld\n"
1050+ " files: %20lld\n"
1051+ " subdirs: %20lld\n"
1052+ "rentries: %20lld\n"
1053+ " rfiles: %20lld\n"
1054+ " rsubdirs: %20lld\n"
1055+ "rbytes: %20lld\n"
1056+ "rctime: %10ld.%09ld\n",
1057+ ci->i_files + ci->i_subdirs,
1058+ ci->i_files,
1059+ ci->i_subdirs,
1060+ ci->i_rfiles + ci->i_rsubdirs,
1061+ ci->i_rfiles,
1062+ ci->i_rsubdirs,
1063+ ci->i_rbytes,
1064+ (long)ci->i_rctime.tv_sec,
1065+ (long)ci->i_rctime.tv_nsec);
1066+ }
1067+
1068+ if (*ppos >= cf->dir_info_len)
1069+ return 0;
1070+ size = min_t(unsigned, size, cf->dir_info_len-*ppos);
1071+ left = copy_to_user(buf, cf->dir_info + *ppos, size);
1072+ if (left == size)
1073+ return -EFAULT;
1074+ *ppos += (size - left);
1075+ return size - left;
1076+}
1077+
1078+/*
1079+ * an fsync() on a dir will wait for any uncommitted directory
1080+ * operations to commit.
1081+ */
1082+static int ceph_dir_fsync(struct file *file, struct dentry *dentry,
1083+ int datasync)
1084+{
1085+ struct inode *inode = dentry->d_inode;
1086+ struct ceph_inode_info *ci = ceph_inode(inode);
1087+ struct list_head *head = &ci->i_unsafe_dirops;
1088+ struct ceph_mds_request *req;
1089+ u64 last_tid;
1090+ int ret = 0;
1091+
1092+ dout("dir_fsync %p\n", inode);
1093+ spin_lock(&ci->i_unsafe_lock);
1094+ if (list_empty(head))
1095+ goto out;
1096+
1097+ req = list_entry(head->prev,
1098+ struct ceph_mds_request, r_unsafe_dir_item);
1099+ last_tid = req->r_tid;
1100+
1101+ do {
1102+ ceph_mdsc_get_request(req);
1103+ spin_unlock(&ci->i_unsafe_lock);
1104+ dout("dir_fsync %p wait on tid %llu (until %llu)\n",
1105+ inode, req->r_tid, last_tid);
1106+ if (req->r_timeout) {
1107+ ret = wait_for_completion_timeout(
1108+ &req->r_safe_completion, req->r_timeout);
1109+ if (ret > 0)
1110+ ret = 0;
1111+ else if (ret == 0)
1112+ ret = -EIO; /* timed out */
1113+ } else {
1114+ wait_for_completion(&req->r_safe_completion);
1115+ }
1116+ spin_lock(&ci->i_unsafe_lock);
1117+ ceph_mdsc_put_request(req);
1118+
1119+ if (ret || list_empty(head))
1120+ break;
1121+ req = list_entry(head->next,
1122+ struct ceph_mds_request, r_unsafe_dir_item);
1123+ } while (req->r_tid < last_tid);
1124+out:
1125+ spin_unlock(&ci->i_unsafe_lock);
1126+ return ret;
1127+}
1128+
1129+/*
1130+ * We maintain a private dentry LRU.
1131+ *
1132+ * FIXME: this needs to be changed to a per-mds lru to be useful.
1133+ */
1134+void ceph_dentry_lru_add(struct dentry *dn)
1135+{
1136+ struct ceph_dentry_info *di = ceph_dentry(dn);
1137+ struct ceph_mds_client *mdsc;
1138+
1139+ dout("dentry_lru_add %p %p '%.*s'\n", di, dn,
1140+ dn->d_name.len, dn->d_name.name);
1141+ if (di) {
1142+ mdsc = &ceph_client(dn->d_sb)->mdsc;
1143+ spin_lock(&mdsc->dentry_lru_lock);
1144+ list_add_tail(&di->lru, &mdsc->dentry_lru);
1145+ mdsc->num_dentry++;
1146+ spin_unlock(&mdsc->dentry_lru_lock);
1147+ }
1148+}
1149+
1150+void ceph_dentry_lru_touch(struct dentry *dn)
1151+{
1152+ struct ceph_dentry_info *di = ceph_dentry(dn);
1153+ struct ceph_mds_client *mdsc;
1154+
1155+ dout("dentry_lru_touch %p %p '%.*s'\n", di, dn,
1156+ dn->d_name.len, dn->d_name.name);
1157+ if (di) {
1158+ mdsc = &ceph_client(dn->d_sb)->mdsc;
1159+ spin_lock(&mdsc->dentry_lru_lock);
1160+ list_move_tail(&di->lru, &mdsc->dentry_lru);
1161+ spin_unlock(&mdsc->dentry_lru_lock);
1162+ }
1163+}
1164+
1165+void ceph_dentry_lru_del(struct dentry *dn)
1166+{
1167+ struct ceph_dentry_info *di = ceph_dentry(dn);
1168+ struct ceph_mds_client *mdsc;
1169+
1170+ dout("dentry_lru_del %p %p '%.*s'\n", di, dn,
1171+ dn->d_name.len, dn->d_name.name);
1172+ if (di) {
1173+ mdsc = &ceph_client(dn->d_sb)->mdsc;
1174+ spin_lock(&mdsc->dentry_lru_lock);
1175+ list_del_init(&di->lru);
1176+ mdsc->num_dentry--;
1177+ spin_unlock(&mdsc->dentry_lru_lock);
1178+ }
1179+}
1180+
1181+const struct file_operations ceph_dir_fops = {
1182+ .read = ceph_read_dir,
1183+ .readdir = ceph_readdir,
1184+ .llseek = ceph_dir_llseek,
1185+ .open = ceph_open,
1186+ .release = ceph_release,
1187+ .unlocked_ioctl = ceph_ioctl,
1188+ .fsync = ceph_dir_fsync,
1189+};
1190+
1191+const struct inode_operations ceph_dir_iops = {
1192+ .lookup = ceph_lookup,
1193+ .permission = ceph_permission,
1194+ .getattr = ceph_getattr,
1195+ .setattr = ceph_setattr,
1196+ .setxattr = ceph_setxattr,
1197+ .getxattr = ceph_getxattr,
1198+ .listxattr = ceph_listxattr,
1199+ .removexattr = ceph_removexattr,
1200+ .mknod = ceph_mknod,
1201+ .symlink = ceph_symlink,
1202+ .mkdir = ceph_mkdir,
1203+ .link = ceph_link,
1204+ .unlink = ceph_unlink,
1205+ .rmdir = ceph_unlink,
1206+ .rename = ceph_rename,
1207+ .create = ceph_create,
1208+};
1209+
1210+struct dentry_operations ceph_dentry_ops = {
1211+ .d_revalidate = ceph_d_revalidate,
1212+ .d_release = ceph_dentry_release,
1213+};
1214+
1215+struct dentry_operations ceph_snapdir_dentry_ops = {
1216+ .d_revalidate = ceph_snapdir_d_revalidate,
1217+};
1218+
1219+struct dentry_operations ceph_snap_dentry_ops = {
1220+};
fs/ceph/export.c+223−0
@@ -0,0 +1,223 @@
1+#include "ceph_debug.h"
2+
3+#include <linux/exportfs.h>
4+#include <asm/unaligned.h>
5+
6+#include "super.h"
7+
8+/*
9+ * NFS export support
10+ *
11+ * NFS re-export of a ceph mount is, at present, only semireliable.
12+ * The basic issue is that the Ceph architectures doesn't lend itself
13+ * well to generating filehandles that will remain valid forever.
14+ *
15+ * So, we do our best. If you're lucky, your inode will be in the
16+ * client's cache. If it's not, and you have a connectable fh, then
17+ * the MDS server may be able to find it for you. Otherwise, you get
18+ * ESTALE.
19+ *
20+ * There are ways to this more reliable, but in the non-connectable fh
21+ * case, we won't every work perfectly, and in the connectable case,
22+ * some changes are needed on the MDS side to work better.
23+ */
24+
25+/*
26+ * Basic fh
27+ */
28+struct ceph_nfs_fh {
29+ u64 ino;
30+} __attribute__ ((packed));
31+
32+/*
33+ * Larger 'connectable' fh that includes parent ino and name hash.
34+ * Use this whenever possible, as it works more reliably.
35+ */
36+struct ceph_nfs_confh {
37+ u64 ino, parent_ino;
38+ u32 parent_name_hash;
39+} __attribute__ ((packed));
40+
41+static int ceph_encode_fh(struct dentry *dentry, u32 *rawfh, int *max_len,
42+ int connectable)
43+{
44+ struct ceph_nfs_fh *fh = (void *)rawfh;
45+ struct ceph_nfs_confh *cfh = (void *)rawfh;
46+ struct dentry *parent = dentry->d_parent;
47+ struct inode *inode = dentry->d_inode;
48+ int type;
49+
50+ /* don't re-export snaps */
51+ if (ceph_snap(inode) != CEPH_NOSNAP)
52+ return -EINVAL;
53+
54+ if (*max_len >= sizeof(*cfh)) {
55+ dout("encode_fh %p connectable\n", dentry);
56+ cfh->ino = ceph_ino(dentry->d_inode);
57+ cfh->parent_ino = ceph_ino(parent->d_inode);
58+ cfh->parent_name_hash = parent->d_name.hash;
59+ *max_len = sizeof(*cfh);
60+ type = 2;
61+ } else if (*max_len > sizeof(*fh)) {
62+ if (connectable)
63+ return -ENOSPC;
64+ dout("encode_fh %p\n", dentry);
65+ fh->ino = ceph_ino(dentry->d_inode);
66+ *max_len = sizeof(*fh);
67+ type = 1;
68+ } else {
69+ return -ENOSPC;
70+ }
71+ return type;
72+}
73+
74+/*
75+ * convert regular fh to dentry
76+ *
77+ * FIXME: we should try harder by querying the mds for the ino.
78+ */
79+static struct dentry *__fh_to_dentry(struct super_block *sb,
80+ struct ceph_nfs_fh *fh)
81+{
82+ struct inode *inode;
83+ struct dentry *dentry;
84+ struct ceph_vino vino;
85+ int err;
86+
87+ dout("__fh_to_dentry %llx\n", fh->ino);
88+ vino.ino = fh->ino;
89+ vino.snap = CEPH_NOSNAP;
90+ inode = ceph_find_inode(sb, vino);
91+ if (!inode)
92+ return ERR_PTR(-ESTALE);
93+
94+ dentry = d_obtain_alias(inode);
95+ if (!dentry) {
96+ pr_err("fh_to_dentry %llx -- inode %p but ENOMEM\n",
97+ fh->ino, inode);
98+ iput(inode);
99+ return ERR_PTR(-ENOMEM);
100+ }
101+ err = ceph_init_dentry(dentry);
102+
103+ if (err < 0) {
104+ iput(inode);
105+ return ERR_PTR(err);
106+ }
107+ dout("__fh_to_dentry %llx %p dentry %p\n", fh->ino, inode, dentry);
108+ return dentry;
109+}
110+
111+/*
112+ * convert connectable fh to dentry
113+ */
114+static struct dentry *__cfh_to_dentry(struct super_block *sb,
115+ struct ceph_nfs_confh *cfh)
116+{
117+ struct ceph_mds_client *mdsc = &ceph_client(sb)->mdsc;
118+ struct inode *inode;
119+ struct dentry *dentry;
120+ struct ceph_vino vino;
121+ int err;
122+
123+ dout("__cfh_to_dentry %llx (%llx/%x)\n",
124+ cfh->ino, cfh->parent_ino, cfh->parent_name_hash);
125+
126+ vino.ino = cfh->ino;
127+ vino.snap = CEPH_NOSNAP;
128+ inode = ceph_find_inode(sb, vino);
129+ if (!inode) {
130+ struct ceph_mds_request *req;
131+
132+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_LOOKUPHASH,
133+ USE_ANY_MDS);
134+ if (IS_ERR(req))
135+ return ERR_PTR(PTR_ERR(req));
136+
137+ req->r_ino1 = vino;
138+ req->r_ino2.ino = cfh->parent_ino;
139+ req->r_ino2.snap = CEPH_NOSNAP;
140+ req->r_path2 = kmalloc(16, GFP_NOFS);
141+ snprintf(req->r_path2, 16, "%d", cfh->parent_name_hash);
142+ req->r_num_caps = 1;
143+ err = ceph_mdsc_do_request(mdsc, NULL, req);
144+ ceph_mdsc_put_request(req);
145+ inode = ceph_find_inode(sb, vino);
146+ if (!inode)
147+ return ERR_PTR(err ? err : -ESTALE);
148+ }
149+
150+ dentry = d_obtain_alias(inode);
151+ if (!dentry) {
152+ pr_err("cfh_to_dentry %llx -- inode %p but ENOMEM\n",
153+ cfh->ino, inode);
154+ iput(inode);
155+ return ERR_PTR(-ENOMEM);
156+ }
157+ err = ceph_init_dentry(dentry);
158+ if (err < 0) {
159+ iput(inode);
160+ return ERR_PTR(err);
161+ }
162+ dout("__cfh_to_dentry %llx %p dentry %p\n", cfh->ino, inode, dentry);
163+ return dentry;
164+}
165+
166+static struct dentry *ceph_fh_to_dentry(struct super_block *sb, struct fid *fid,
167+ int fh_len, int fh_type)
168+{
169+ if (fh_type == 1)
170+ return __fh_to_dentry(sb, (struct ceph_nfs_fh *)fid->raw);
171+ else
172+ return __cfh_to_dentry(sb, (struct ceph_nfs_confh *)fid->raw);
173+}
174+
175+/*
176+ * get parent, if possible.
177+ *
178+ * FIXME: we could do better by querying the mds to discover the
179+ * parent.
180+ */
181+static struct dentry *ceph_fh_to_parent(struct super_block *sb,
182+ struct fid *fid,
183+ int fh_len, int fh_type)
184+{
185+ struct ceph_nfs_confh *cfh = (void *)fid->raw;
186+ struct ceph_vino vino;
187+ struct inode *inode;
188+ struct dentry *dentry;
189+ int err;
190+
191+ if (fh_type == 1)
192+ return ERR_PTR(-ESTALE);
193+
194+ pr_debug("fh_to_parent %llx/%d\n", cfh->parent_ino,
195+ cfh->parent_name_hash);
196+
197+ vino.ino = cfh->ino;
198+ vino.snap = CEPH_NOSNAP;
199+ inode = ceph_find_inode(sb, vino);
200+ if (!inode)
201+ return ERR_PTR(-ESTALE);
202+
203+ dentry = d_obtain_alias(inode);
204+ if (!dentry) {
205+ pr_err("fh_to_parent %llx -- inode %p but ENOMEM\n",
206+ cfh->ino, inode);
207+ iput(inode);
208+ return ERR_PTR(-ENOMEM);
209+ }
210+ err = ceph_init_dentry(dentry);
211+ if (err < 0) {
212+ iput(inode);
213+ return ERR_PTR(err);
214+ }
215+ dout("fh_to_parent %llx %p dentry %p\n", cfh->ino, inode, dentry);
216+ return dentry;
217+}
218+
219+const struct export_operations ceph_export_ops = {
220+ .encode_fh = ceph_encode_fh,
221+ .fh_to_dentry = ceph_fh_to_dentry,
222+ .fh_to_parent = ceph_fh_to_parent,
223+};
fs/ceph/file.c+937−0
@@ -0,0 +1,937 @@
1+#include "ceph_debug.h"
2+
3+#include <linux/sched.h>
4+#include <linux/file.h>
5+#include <linux/namei.h>
6+#include <linux/writeback.h>
7+
8+#include "super.h"
9+#include "mds_client.h"
10+
11+/*
12+ * Ceph file operations
13+ *
14+ * Implement basic open/close functionality, and implement
15+ * read/write.
16+ *
17+ * We implement three modes of file I/O:
18+ * - buffered uses the generic_file_aio_{read,write} helpers
19+ *
20+ * - synchronous is used when there is multi-client read/write
21+ * sharing, avoids the page cache, and synchronously waits for an
22+ * ack from the OSD.
23+ *
24+ * - direct io takes the variant of the sync path that references
25+ * user pages directly.
26+ *
27+ * fsync() flushes and waits on dirty pages, but just queues metadata
28+ * for writeback: since the MDS can recover size and mtime there is no
29+ * need to wait for MDS acknowledgement.
30+ */
31+
32+
33+/*
34+ * Prepare an open request. Preallocate ceph_cap to avoid an
35+ * inopportune ENOMEM later.
36+ */
37+static struct ceph_mds_request *
38+prepare_open_request(struct super_block *sb, int flags, int create_mode)
39+{
40+ struct ceph_client *client = ceph_sb_to_client(sb);
41+ struct ceph_mds_client *mdsc = &client->mdsc;
42+ struct ceph_mds_request *req;
43+ int want_auth = USE_ANY_MDS;
44+ int op = (flags & O_CREAT) ? CEPH_MDS_OP_CREATE : CEPH_MDS_OP_OPEN;
45+
46+ if (flags & (O_WRONLY|O_RDWR|O_CREAT|O_TRUNC))
47+ want_auth = USE_AUTH_MDS;
48+
49+ req = ceph_mdsc_create_request(mdsc, op, want_auth);
50+ if (IS_ERR(req))
51+ goto out;
52+ req->r_fmode = ceph_flags_to_mode(flags);
53+ req->r_args.open.flags = cpu_to_le32(flags);
54+ req->r_args.open.mode = cpu_to_le32(create_mode);
55+ req->r_args.open.preferred = cpu_to_le32(-1);
56+out:
57+ return req;
58+}
59+
60+/*
61+ * initialize private struct file data.
62+ * if we fail, clean up by dropping fmode reference on the ceph_inode
63+ */
64+static int ceph_init_file(struct inode *inode, struct file *file, int fmode)
65+{
66+ struct ceph_file_info *cf;
67+ int ret = 0;
68+
69+ switch (inode->i_mode & S_IFMT) {
70+ case S_IFREG:
71+ case S_IFDIR:
72+ dout("init_file %p %p 0%o (regular)\n", inode, file,
73+ inode->i_mode);
74+ cf = kmem_cache_alloc(ceph_file_cachep, GFP_NOFS | __GFP_ZERO);
75+ if (cf == NULL) {
76+ ceph_put_fmode(ceph_inode(inode), fmode); /* clean up */
77+ return -ENOMEM;
78+ }
79+ cf->fmode = fmode;
80+ cf->next_offset = 2;
81+ file->private_data = cf;
82+ BUG_ON(inode->i_fop->release != ceph_release);
83+ break;
84+
85+ case S_IFLNK:
86+ dout("init_file %p %p 0%o (symlink)\n", inode, file,
87+ inode->i_mode);
88+ ceph_put_fmode(ceph_inode(inode), fmode); /* clean up */
89+ break;
90+
91+ default:
92+ dout("init_file %p %p 0%o (special)\n", inode, file,
93+ inode->i_mode);
94+ /*
95+ * we need to drop the open ref now, since we don't
96+ * have .release set to ceph_release.
97+ */
98+ ceph_put_fmode(ceph_inode(inode), fmode); /* clean up */
99+ BUG_ON(inode->i_fop->release == ceph_release);
100+
101+ /* call the proper open fop */
102+ ret = inode->i_fop->open(inode, file);
103+ }
104+ return ret;
105+}
106+
107+/*
108+ * If the filp already has private_data, that means the file was
109+ * already opened by intent during lookup, and we do nothing.
110+ *
111+ * If we already have the requisite capabilities, we can satisfy
112+ * the open request locally (no need to request new caps from the
113+ * MDS). We do, however, need to inform the MDS (asynchronously)
114+ * if our wanted caps set expands.
115+ */
116+int ceph_open(struct inode *inode, struct file *file)
117+{
118+ struct ceph_inode_info *ci = ceph_inode(inode);
119+ struct ceph_client *client = ceph_sb_to_client(inode->i_sb);
120+ struct ceph_mds_client *mdsc = &client->mdsc;
121+ struct ceph_mds_request *req;
122+ struct ceph_file_info *cf = file->private_data;
123+ struct inode *parent_inode = file->f_dentry->d_parent->d_inode;
124+ int err;
125+ int flags, fmode, wanted;
126+
127+ if (cf) {
128+ dout("open file %p is already opened\n", file);
129+ return 0;
130+ }
131+
132+ /* filter out O_CREAT|O_EXCL; vfs did that already. yuck. */
133+ flags = file->f_flags & ~(O_CREAT|O_EXCL);
134+ if (S_ISDIR(inode->i_mode))
135+ flags = O_DIRECTORY; /* mds likes to know */
136+
137+ dout("open inode %p ino %llx.%llx file %p flags %d (%d)\n", inode,
138+ ceph_vinop(inode), file, flags, file->f_flags);
139+ fmode = ceph_flags_to_mode(flags);
140+ wanted = ceph_caps_for_mode(fmode);
141+
142+ /* snapped files are read-only */
143+ if (ceph_snap(inode) != CEPH_NOSNAP && (file->f_mode & FMODE_WRITE))
144+ return -EROFS;
145+
146+ /* trivially open snapdir */
147+ if (ceph_snap(inode) == CEPH_SNAPDIR) {
148+ spin_lock(&inode->i_lock);
149+ __ceph_get_fmode(ci, fmode);
150+ spin_unlock(&inode->i_lock);
151+ return ceph_init_file(inode, file, fmode);
152+ }
153+
154+ /*
155+ * No need to block if we have any caps. Update wanted set
156+ * asynchronously.
157+ */
158+ spin_lock(&inode->i_lock);
159+ if (__ceph_is_any_real_caps(ci)) {
160+ int mds_wanted = __ceph_caps_mds_wanted(ci);
161+ int issued = __ceph_caps_issued(ci, NULL);
162+
163+ dout("open %p fmode %d want %s issued %s using existing\n",
164+ inode, fmode, ceph_cap_string(wanted),
165+ ceph_cap_string(issued));
166+ __ceph_get_fmode(ci, fmode);
167+ spin_unlock(&inode->i_lock);
168+
169+ /* adjust wanted? */
170+ if ((issued & wanted) != wanted &&
171+ (mds_wanted & wanted) != wanted &&
172+ ceph_snap(inode) != CEPH_SNAPDIR)
173+ ceph_check_caps(ci, 0, NULL);
174+
175+ return ceph_init_file(inode, file, fmode);
176+ } else if (ceph_snap(inode) != CEPH_NOSNAP &&
177+ (ci->i_snap_caps & wanted) == wanted) {
178+ __ceph_get_fmode(ci, fmode);
179+ spin_unlock(&inode->i_lock);
180+ return ceph_init_file(inode, file, fmode);
181+ }
182+ spin_unlock(&inode->i_lock);
183+
184+ dout("open fmode %d wants %s\n", fmode, ceph_cap_string(wanted));
185+ req = prepare_open_request(inode->i_sb, flags, 0);
186+ if (IS_ERR(req)) {
187+ err = PTR_ERR(req);
188+ goto out;
189+ }
190+ req->r_inode = igrab(inode);
191+ req->r_num_caps = 1;
192+ err = ceph_mdsc_do_request(mdsc, parent_inode, req);
193+ if (!err)
194+ err = ceph_init_file(inode, file, req->r_fmode);
195+ ceph_mdsc_put_request(req);
196+ dout("open result=%d on %llx.%llx\n", err, ceph_vinop(inode));
197+out:
198+ return err;
199+}
200+
201+
202+/*
203+ * Do a lookup + open with a single request.
204+ *
205+ * If this succeeds, but some subsequent check in the vfs
206+ * may_open() fails, the struct *file gets cleaned up (i.e.
207+ * ceph_release gets called). So fear not!
208+ */
209+/*
210+ * flags
211+ * path_lookup_open -> LOOKUP_OPEN
212+ * path_lookup_create -> LOOKUP_OPEN|LOOKUP_CREATE
213+ */
214+struct dentry *ceph_lookup_open(struct inode *dir, struct dentry *dentry,
215+ struct nameidata *nd, int mode,
216+ int locked_dir)
217+{
218+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
219+ struct ceph_mds_client *mdsc = &client->mdsc;
220+ struct file *file = nd->intent.open.file;
221+ struct inode *parent_inode = get_dentry_parent_inode(file->f_dentry);
222+ struct ceph_mds_request *req;
223+ int err;
224+ int flags = nd->intent.open.flags - 1; /* silly vfs! */
225+
226+ dout("ceph_lookup_open dentry %p '%.*s' flags %d mode 0%o\n",
227+ dentry, dentry->d_name.len, dentry->d_name.name, flags, mode);
228+
229+ /* do the open */
230+ req = prepare_open_request(dir->i_sb, flags, mode);
231+ if (IS_ERR(req))
232+ return ERR_PTR(PTR_ERR(req));
233+ req->r_dentry = dget(dentry);
234+ req->r_num_caps = 2;
235+ if (flags & O_CREAT) {
236+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
237+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
238+ }
239+ req->r_locked_dir = dir; /* caller holds dir->i_mutex */
240+ err = ceph_mdsc_do_request(mdsc, parent_inode, req);
241+ dentry = ceph_finish_lookup(req, dentry, err);
242+ if (!err && (flags & O_CREAT) && !req->r_reply_info.head->is_dentry)
243+ err = ceph_handle_notrace_create(dir, dentry);
244+ if (!err)
245+ err = ceph_init_file(req->r_dentry->d_inode, file,
246+ req->r_fmode);
247+ ceph_mdsc_put_request(req);
248+ dout("ceph_lookup_open result=%p\n", dentry);
249+ return dentry;
250+}
251+
252+int ceph_release(struct inode *inode, struct file *file)
253+{
254+ struct ceph_inode_info *ci = ceph_inode(inode);
255+ struct ceph_file_info *cf = file->private_data;
256+
257+ dout("release inode %p file %p\n", inode, file);
258+ ceph_put_fmode(ci, cf->fmode);
259+ if (cf->last_readdir)
260+ ceph_mdsc_put_request(cf->last_readdir);
261+ kfree(cf->last_name);
262+ kfree(cf->dir_info);
263+ dput(cf->dentry);
264+ kmem_cache_free(ceph_file_cachep, cf);
265+
266+ /* wake up anyone waiting for caps on this inode */
267+ wake_up(&ci->i_cap_wq);
268+ return 0;
269+}
270+
271+/*
272+ * build a vector of user pages
273+ */
274+static struct page **get_direct_page_vector(const char __user *data,
275+ int num_pages,
276+ loff_t off, size_t len)
277+{
278+ struct page **pages;
279+ int rc;
280+
281+ pages = kmalloc(sizeof(*pages) * num_pages, GFP_NOFS);
282+ if (!pages)
283+ return ERR_PTR(-ENOMEM);
284+
285+ down_read(&current->mm->mmap_sem);
286+ rc = get_user_pages(current, current->mm, (unsigned long)data,
287+ num_pages, 0, 0, pages, NULL);
288+ up_read(&current->mm->mmap_sem);
289+ if (rc < 0)
290+ goto fail;
291+ return pages;
292+
293+fail:
294+ kfree(pages);
295+ return ERR_PTR(rc);
296+}
297+
298+static void put_page_vector(struct page **pages, int num_pages)
299+{
300+ int i;
301+
302+ for (i = 0; i < num_pages; i++)
303+ put_page(pages[i]);
304+ kfree(pages);
305+}
306+
307+void ceph_release_page_vector(struct page **pages, int num_pages)
308+{
309+ int i;
310+
311+ for (i = 0; i < num_pages; i++)
312+ __free_pages(pages[i], 0);
313+ kfree(pages);
314+}
315+
316+/*
317+ * allocate a vector new pages
318+ */
319+static struct page **alloc_page_vector(int num_pages)
320+{
321+ struct page **pages;
322+ int i;
323+
324+ pages = kmalloc(sizeof(*pages) * num_pages, GFP_NOFS);
325+ if (!pages)
326+ return ERR_PTR(-ENOMEM);
327+ for (i = 0; i < num_pages; i++) {
328+ pages[i] = alloc_page(GFP_NOFS);
329+ if (pages[i] == NULL) {
330+ ceph_release_page_vector(pages, i);
331+ return ERR_PTR(-ENOMEM);
332+ }
333+ }
334+ return pages;
335+}
336+
337+/*
338+ * copy user data into a page vector
339+ */
340+static int copy_user_to_page_vector(struct page **pages,
341+ const char __user *data,
342+ loff_t off, size_t len)
343+{
344+ int i = 0;
345+ int po = off & ~PAGE_CACHE_MASK;
346+ int left = len;
347+ int l, bad;
348+
349+ while (left > 0) {
350+ l = min_t(int, PAGE_CACHE_SIZE-po, left);
351+ bad = copy_from_user(page_address(pages[i]) + po, data, l);
352+ if (bad == l)
353+ return -EFAULT;
354+ data += l - bad;
355+ left -= l - bad;
356+ po += l - bad;
357+ if (po == PAGE_CACHE_SIZE) {
358+ po = 0;
359+ i++;
360+ }
361+ }
362+ return len;
363+}
364+
365+/*
366+ * copy user data from a page vector into a user pointer
367+ */
368+static int copy_page_vector_to_user(struct page **pages, char __user *data,
369+ loff_t off, size_t len)
370+{
371+ int i = 0;
372+ int po = off & ~PAGE_CACHE_MASK;
373+ int left = len;
374+ int l, bad;
375+
376+ while (left > 0) {
377+ l = min_t(int, left, PAGE_CACHE_SIZE-po);
378+ bad = copy_to_user(data, page_address(pages[i]) + po, l);
379+ if (bad == l)
380+ return -EFAULT;
381+ data += l - bad;
382+ left -= l - bad;
383+ if (po) {
384+ po += l - bad;
385+ if (po == PAGE_CACHE_SIZE)
386+ po = 0;
387+ }
388+ i++;
389+ }
390+ return len;
391+}
392+
393+/*
394+ * Zero an extent within a page vector. Offset is relative to the
395+ * start of the first page.
396+ */
397+static void zero_page_vector_range(int off, int len, struct page **pages)
398+{
399+ int i = off >> PAGE_CACHE_SHIFT;
400+
401+ off &= ~PAGE_CACHE_MASK;
402+
403+ dout("zero_page_vector_page %u~%u\n", off, len);
404+
405+ /* leading partial page? */
406+ if (off) {
407+ int end = min((int)PAGE_CACHE_SIZE, off + len);
408+ dout("zeroing %d %p head from %d\n", i, pages[i],
409+ (int)off);
410+ zero_user_segment(pages[i], off, end);
411+ len -= (end - off);
412+ i++;
413+ }
414+ while (len >= PAGE_CACHE_SIZE) {
415+ dout("zeroing %d %p len=%d\n", i, pages[i], len);
416+ zero_user_segment(pages[i], 0, PAGE_CACHE_SIZE);
417+ len -= PAGE_CACHE_SIZE;
418+ i++;
419+ }
420+ /* trailing partial page? */
421+ if (len) {
422+ dout("zeroing %d %p tail to %d\n", i, pages[i], (int)len);
423+ zero_user_segment(pages[i], 0, len);
424+ }
425+}
426+
427+
428+/*
429+ * Read a range of bytes striped over one or more objects. Iterate over
430+ * objects we stripe over. (That's not atomic, but good enough for now.)
431+ *
432+ * If we get a short result from the OSD, check against i_size; we need to
433+ * only return a short read to the caller if we hit EOF.
434+ */
435+static int striped_read(struct inode *inode,
436+ u64 off, u64 len,
437+ struct page **pages, int num_pages,
438+ int *checkeof)
439+{
440+ struct ceph_client *client = ceph_inode_to_client(inode);
441+ struct ceph_inode_info *ci = ceph_inode(inode);
442+ u64 pos, this_len;
443+ int page_off = off & ~PAGE_CACHE_MASK; /* first byte's offset in page */
444+ int left, pages_left;
445+ int read;
446+ struct page **page_pos;
447+ int ret;
448+ bool hit_stripe, was_short;
449+
450+ /*
451+ * we may need to do multiple reads. not atomic, unfortunately.
452+ */
453+ pos = off;
454+ left = len;
455+ page_pos = pages;
456+ pages_left = num_pages;
457+ read = 0;
458+
459+more:
460+ this_len = left;
461+ ret = ceph_osdc_readpages(&client->osdc, ceph_vino(inode),
462+ &ci->i_layout, pos, &this_len,
463+ ci->i_truncate_seq,
464+ ci->i_truncate_size,
465+ page_pos, pages_left);
466+ hit_stripe = this_len < left;
467+ was_short = ret >= 0 && ret < this_len;
468+ if (ret == -ENOENT)
469+ ret = 0;
470+ dout("striped_read %llu~%u (read %u) got %d%s%s\n", pos, left, read,
471+ ret, hit_stripe ? " HITSTRIPE" : "", was_short ? " SHORT" : "");
472+
473+ if (ret > 0) {
474+ int didpages =
475+ ((pos & ~PAGE_CACHE_MASK) + ret) >> PAGE_CACHE_SHIFT;
476+
477+ if (read < pos - off) {
478+ dout(" zero gap %llu to %llu\n", off + read, pos);
479+ zero_page_vector_range(page_off + read,
480+ pos - off - read, pages);
481+ }
482+ pos += ret;
483+ read = pos - off;
484+ left -= ret;
485+ page_pos += didpages;
486+ pages_left -= didpages;
487+
488+ /* hit stripe? */
489+ if (left && hit_stripe)
490+ goto more;
491+ }
492+
493+ if (was_short) {
494+ /* was original extent fully inside i_size? */
495+ if (pos + left <= inode->i_size) {
496+ dout("zero tail\n");
497+ zero_page_vector_range(page_off + read, len - read,
498+ pages);
499+ read = len;
500+ goto out;
501+ }
502+
503+ /* check i_size */
504+ *checkeof = 1;
505+ }
506+
507+out:
508+ if (ret >= 0)
509+ ret = read;
510+ dout("striped_read returns %d\n", ret);
511+ return ret;
512+}
513+
514+/*
515+ * Completely synchronous read and write methods. Direct from __user
516+ * buffer to osd, or directly to user pages (if O_DIRECT).
517+ *
518+ * If the read spans object boundary, just do multiple reads.
519+ */
520+static ssize_t ceph_sync_read(struct file *file, char __user *data,
521+ unsigned len, loff_t *poff, int *checkeof)
522+{
523+ struct inode *inode = file->f_dentry->d_inode;
524+ struct page **pages;
525+ u64 off = *poff;
526+ int num_pages = calc_pages_for(off, len);
527+ int ret;
528+
529+ dout("sync_read on file %p %llu~%u %s\n", file, off, len,
530+ (file->f_flags & O_DIRECT) ? "O_DIRECT" : "");
531+
532+ if (file->f_flags & O_DIRECT) {
533+ pages = get_direct_page_vector(data, num_pages, off, len);
534+
535+ /*
536+ * flush any page cache pages in this range. this
537+ * will make concurrent normal and O_DIRECT io slow,
538+ * but it will at least behave sensibly when they are
539+ * in sequence.
540+ */
541+ } else {
542+ pages = alloc_page_vector(num_pages);
543+ }
544+ if (IS_ERR(pages))
545+ return PTR_ERR(pages);
546+
547+ ret = filemap_write_and_wait(inode->i_mapping);
548+ if (ret < 0)
549+ goto done;
550+
551+ ret = striped_read(inode, off, len, pages, num_pages, checkeof);
552+
553+ if (ret >= 0 && (file->f_flags & O_DIRECT) == 0)
554+ ret = copy_page_vector_to_user(pages, data, off, ret);
555+ if (ret >= 0)
556+ *poff = off + ret;
557+
558+done:
559+ if (file->f_flags & O_DIRECT)
560+ put_page_vector(pages, num_pages);
561+ else
562+ ceph_release_page_vector(pages, num_pages);
563+ dout("sync_read result %d\n", ret);
564+ return ret;
565+}
566+
567+/*
568+ * Write commit callback, called if we requested both an ACK and
569+ * ONDISK commit reply from the OSD.
570+ */
571+static void sync_write_commit(struct ceph_osd_request *req,
572+ struct ceph_msg *msg)
573+{
574+ struct ceph_inode_info *ci = ceph_inode(req->r_inode);
575+
576+ dout("sync_write_commit %p tid %llu\n", req, req->r_tid);
577+ spin_lock(&ci->i_unsafe_lock);
578+ list_del_init(&req->r_unsafe_item);
579+ spin_unlock(&ci->i_unsafe_lock);
580+ ceph_put_cap_refs(ci, CEPH_CAP_FILE_WR);
581+}
582+
583+/*
584+ * Synchronous write, straight from __user pointer or user pages (if
585+ * O_DIRECT).
586+ *
587+ * If write spans object boundary, just do multiple writes. (For a
588+ * correct atomic write, we should e.g. take write locks on all
589+ * objects, rollback on failure, etc.)
590+ */
591+static ssize_t ceph_sync_write(struct file *file, const char __user *data,
592+ size_t left, loff_t *offset)
593+{
594+ struct inode *inode = file->f_dentry->d_inode;
595+ struct ceph_inode_info *ci = ceph_inode(inode);
596+ struct ceph_client *client = ceph_inode_to_client(inode);
597+ struct ceph_osd_request *req;
598+ struct page **pages;
599+ int num_pages;
600+ long long unsigned pos;
601+ u64 len;
602+ int written = 0;
603+ int flags;
604+ int do_sync = 0;
605+ int check_caps = 0;
606+ int ret;
607+ struct timespec mtime = CURRENT_TIME;
608+
609+ if (ceph_snap(file->f_dentry->d_inode) != CEPH_NOSNAP)
610+ return -EROFS;
611+
612+ dout("sync_write on file %p %lld~%u %s\n", file, *offset,
613+ (unsigned)left, (file->f_flags & O_DIRECT) ? "O_DIRECT" : "");
614+
615+ if (file->f_flags & O_APPEND)
616+ pos = i_size_read(inode);
617+ else
618+ pos = *offset;
619+
620+ ret = filemap_write_and_wait_range(inode->i_mapping, pos, pos + left);
621+ if (ret < 0)
622+ return ret;
623+
624+ ret = invalidate_inode_pages2_range(inode->i_mapping,
625+ pos >> PAGE_CACHE_SHIFT,
626+ (pos + left) >> PAGE_CACHE_SHIFT);
627+ if (ret < 0)
628+ dout("invalidate_inode_pages2_range returned %d\n", ret);
629+
630+ flags = CEPH_OSD_FLAG_ORDERSNAP |
631+ CEPH_OSD_FLAG_ONDISK |
632+ CEPH_OSD_FLAG_WRITE;
633+ if ((file->f_flags & (O_SYNC|O_DIRECT)) == 0)
634+ flags |= CEPH_OSD_FLAG_ACK;
635+ else
636+ do_sync = 1;
637+
638+ /*
639+ * we may need to do multiple writes here if we span an object
640+ * boundary. this isn't atomic, unfortunately. :(
641+ */
642+more:
643+ len = left;
644+ req = ceph_osdc_new_request(&client->osdc, &ci->i_layout,
645+ ceph_vino(inode), pos, &len,
646+ CEPH_OSD_OP_WRITE, flags,
647+ ci->i_snap_realm->cached_context,
648+ do_sync,
649+ ci->i_truncate_seq, ci->i_truncate_size,
650+ &mtime, false, 2);
651+ if (IS_ERR(req))
652+ return PTR_ERR(req);
653+
654+ num_pages = calc_pages_for(pos, len);
655+
656+ if (file->f_flags & O_DIRECT) {
657+ pages = get_direct_page_vector(data, num_pages, pos, len);
658+ if (IS_ERR(pages)) {
659+ ret = PTR_ERR(pages);
660+ goto out;
661+ }
662+
663+ /*
664+ * throw out any page cache pages in this range. this
665+ * may block.
666+ */
667+ truncate_inode_pages_range(inode->i_mapping, pos, pos+len);
668+ } else {
669+ pages = alloc_page_vector(num_pages);
670+ if (IS_ERR(pages)) {
671+ ret = PTR_ERR(pages);
672+ goto out;
673+ }
674+ ret = copy_user_to_page_vector(pages, data, pos, len);
675+ if (ret < 0) {
676+ ceph_release_page_vector(pages, num_pages);
677+ goto out;
678+ }
679+
680+ if ((file->f_flags & O_SYNC) == 0) {
681+ /* get a second commit callback */
682+ req->r_safe_callback = sync_write_commit;
683+ req->r_own_pages = 1;
684+ }
685+ }
686+ req->r_pages = pages;
687+ req->r_num_pages = num_pages;
688+ req->r_inode = inode;
689+
690+ ret = ceph_osdc_start_request(&client->osdc, req, false);
691+ if (!ret) {
692+ if (req->r_safe_callback) {
693+ /*
694+ * Add to inode unsafe list only after we
695+ * start_request so that a tid has been assigned.
696+ */
697+ spin_lock(&ci->i_unsafe_lock);
698+ list_add(&ci->i_unsafe_writes, &req->r_unsafe_item);
699+ spin_unlock(&ci->i_unsafe_lock);
700+ ceph_get_cap_refs(ci, CEPH_CAP_FILE_WR);
701+ }
702+ ret = ceph_osdc_wait_request(&client->osdc, req);
703+ }
704+
705+ if (file->f_flags & O_DIRECT)
706+ put_page_vector(pages, num_pages);
707+ else if (file->f_flags & O_SYNC)
708+ ceph_release_page_vector(pages, num_pages);
709+
710+out:
711+ ceph_osdc_put_request(req);
712+ if (ret == 0) {
713+ pos += len;
714+ written += len;
715+ left -= len;
716+ if (left)
717+ goto more;
718+
719+ ret = written;
720+ *offset = pos;
721+ if (pos > i_size_read(inode))
722+ check_caps = ceph_inode_set_size(inode, pos);
723+ if (check_caps)
724+ ceph_check_caps(ceph_inode(inode), CHECK_CAPS_AUTHONLY,
725+ NULL);
726+ }
727+ return ret;
728+}
729+
730+/*
731+ * Wrap generic_file_aio_read with checks for cap bits on the inode.
732+ * Atomically grab references, so that those bits are not released
733+ * back to the MDS mid-read.
734+ *
735+ * Hmm, the sync read case isn't actually async... should it be?
736+ */
737+static ssize_t ceph_aio_read(struct kiocb *iocb, const struct iovec *iov,
738+ unsigned long nr_segs, loff_t pos)
739+{
740+ struct file *filp = iocb->ki_filp;
741+ loff_t *ppos = &iocb->ki_pos;
742+ size_t len = iov->iov_len;
743+ struct inode *inode = filp->f_dentry->d_inode;
744+ struct ceph_inode_info *ci = ceph_inode(inode);
745+ void *base = iov->iov_base;
746+ ssize_t ret;
747+ int got = 0;
748+ int checkeof = 0, read = 0;
749+
750+ dout("aio_read %p %llx.%llx %llu~%u trying to get caps on %p\n",
751+ inode, ceph_vinop(inode), pos, (unsigned)len, inode);
752+again:
753+ __ceph_do_pending_vmtruncate(inode);
754+ ret = ceph_get_caps(ci, CEPH_CAP_FILE_RD, CEPH_CAP_FILE_CACHE,
755+ &got, -1);
756+ if (ret < 0)
757+ goto out;
758+ dout("aio_read %p %llx.%llx %llu~%u got cap refs on %s\n",
759+ inode, ceph_vinop(inode), pos, (unsigned)len,
760+ ceph_cap_string(got));
761+
762+ if ((got & CEPH_CAP_FILE_CACHE) == 0 ||
763+ (iocb->ki_filp->f_flags & O_DIRECT) ||
764+ (inode->i_sb->s_flags & MS_SYNCHRONOUS))
765+ /* hmm, this isn't really async... */
766+ ret = ceph_sync_read(filp, base, len, ppos, &checkeof);
767+ else
768+ ret = generic_file_aio_read(iocb, iov, nr_segs, pos);
769+
770+out:
771+ dout("aio_read %p %llx.%llx dropping cap refs on %s = %d\n",
772+ inode, ceph_vinop(inode), ceph_cap_string(got), (int)ret);
773+ ceph_put_cap_refs(ci, got);
774+
775+ if (checkeof && ret >= 0) {
776+ int statret = ceph_do_getattr(inode, CEPH_STAT_CAP_SIZE);
777+
778+ /* hit EOF or hole? */
779+ if (statret == 0 && *ppos < inode->i_size) {
780+ dout("aio_read sync_read hit hole, reading more\n");
781+ read += ret;
782+ base += ret;
783+ len -= ret;
784+ checkeof = 0;
785+ goto again;
786+ }
787+ }
788+ if (ret >= 0)
789+ ret += read;
790+
791+ return ret;
792+}
793+
794+/*
795+ * Take cap references to avoid releasing caps to MDS mid-write.
796+ *
797+ * If we are synchronous, and write with an old snap context, the OSD
798+ * may return EOLDSNAPC. In that case, retry the write.. _after_
799+ * dropping our cap refs and allowing the pending snap to logically
800+ * complete _before_ this write occurs.
801+ *
802+ * If we are near ENOSPC, write synchronously.
803+ */
804+static ssize_t ceph_aio_write(struct kiocb *iocb, const struct iovec *iov,
805+ unsigned long nr_segs, loff_t pos)
806+{
807+ struct file *file = iocb->ki_filp;
808+ struct inode *inode = file->f_dentry->d_inode;
809+ struct ceph_inode_info *ci = ceph_inode(inode);
810+ struct ceph_osd_client *osdc = &ceph_client(inode->i_sb)->osdc;
811+ loff_t endoff = pos + iov->iov_len;
812+ int got = 0;
813+ int ret, err;
814+
815+ if (ceph_snap(inode) != CEPH_NOSNAP)
816+ return -EROFS;
817+
818+retry_snap:
819+ if (ceph_osdmap_flag(osdc->osdmap, CEPH_OSDMAP_FULL))
820+ return -ENOSPC;
821+ __ceph_do_pending_vmtruncate(inode);
822+ dout("aio_write %p %llx.%llx %llu~%u getting caps. i_size %llu\n",
823+ inode, ceph_vinop(inode), pos, (unsigned)iov->iov_len,
824+ inode->i_size);
825+ ret = ceph_get_caps(ci, CEPH_CAP_FILE_WR, CEPH_CAP_FILE_BUFFER,
826+ &got, endoff);
827+ if (ret < 0)
828+ goto out;
829+
830+ dout("aio_write %p %llx.%llx %llu~%u got cap refs on %s\n",
831+ inode, ceph_vinop(inode), pos, (unsigned)iov->iov_len,
832+ ceph_cap_string(got));
833+
834+ if ((got & CEPH_CAP_FILE_BUFFER) == 0 ||
835+ (iocb->ki_filp->f_flags & O_DIRECT) ||
836+ (inode->i_sb->s_flags & MS_SYNCHRONOUS)) {
837+ ret = ceph_sync_write(file, iov->iov_base, iov->iov_len,
838+ &iocb->ki_pos);
839+ } else {
840+ ret = generic_file_aio_write(iocb, iov, nr_segs, pos);
841+
842+ if ((ret >= 0 || ret == -EIOCBQUEUED) &&
843+ ((file->f_flags & O_SYNC) || IS_SYNC(file->f_mapping->host)
844+ || ceph_osdmap_flag(osdc->osdmap, CEPH_OSDMAP_NEARFULL))) {
845+ err = vfs_fsync_range(file, file->f_path.dentry,
846+ pos, pos + ret - 1, 1);
847+ if (err < 0)
848+ ret = err;
849+ }
850+ }
851+ if (ret >= 0) {
852+ spin_lock(&inode->i_lock);
853+ __ceph_mark_dirty_caps(ci, CEPH_CAP_FILE_WR);
854+ spin_unlock(&inode->i_lock);
855+ }
856+
857+out:
858+ dout("aio_write %p %llx.%llx %llu~%u dropping cap refs on %s\n",
859+ inode, ceph_vinop(inode), pos, (unsigned)iov->iov_len,
860+ ceph_cap_string(got));
861+ ceph_put_cap_refs(ci, got);
862+
863+ if (ret == -EOLDSNAPC) {
864+ dout("aio_write %p %llx.%llx %llu~%u got EOLDSNAPC, retrying\n",
865+ inode, ceph_vinop(inode), pos, (unsigned)iov->iov_len);
866+ goto retry_snap;
867+ }
868+
869+ return ret;
870+}
871+
872+/*
873+ * llseek. be sure to verify file size on SEEK_END.
874+ */
875+static loff_t ceph_llseek(struct file *file, loff_t offset, int origin)
876+{
877+ struct inode *inode = file->f_mapping->host;
878+ int ret;
879+
880+ mutex_lock(&inode->i_mutex);
881+ __ceph_do_pending_vmtruncate(inode);
882+ switch (origin) {
883+ case SEEK_END:
884+ ret = ceph_do_getattr(inode, CEPH_STAT_CAP_SIZE);
885+ if (ret < 0) {
886+ offset = ret;
887+ goto out;
888+ }
889+ offset += inode->i_size;
890+ break;
891+ case SEEK_CUR:
892+ /*
893+ * Here we special-case the lseek(fd, 0, SEEK_CUR)
894+ * position-querying operation. Avoid rewriting the "same"
895+ * f_pos value back to the file because a concurrent read(),
896+ * write() or lseek() might have altered it
897+ */
898+ if (offset == 0) {
899+ offset = file->f_pos;
900+ goto out;
901+ }
902+ offset += file->f_pos;
903+ break;
904+ }
905+
906+ if (offset < 0 || offset > inode->i_sb->s_maxbytes) {
907+ offset = -EINVAL;
908+ goto out;
909+ }
910+
911+ /* Special lock needed here? */
912+ if (offset != file->f_pos) {
913+ file->f_pos = offset;
914+ file->f_version = 0;
915+ }
916+
917+out:
918+ mutex_unlock(&inode->i_mutex);
919+ return offset;
920+}
921+
922+const struct file_operations ceph_file_fops = {
923+ .open = ceph_open,
924+ .release = ceph_release,
925+ .llseek = ceph_llseek,
926+ .read = do_sync_read,
927+ .write = do_sync_write,
928+ .aio_read = ceph_aio_read,
929+ .aio_write = ceph_aio_write,
930+ .mmap = ceph_mmap,
931+ .fsync = ceph_fsync,
932+ .splice_read = generic_file_splice_read,
933+ .splice_write = generic_file_splice_write,
934+ .unlocked_ioctl = ceph_ioctl,
935+ .compat_ioctl = ceph_ioctl,
936+};
937+
fs/ceph/inode.c+1750−0
@@ -0,0 +1,1750 @@
1+#include "ceph_debug.h"
2+
3+#include <linux/module.h>
4+#include <linux/fs.h>
5+#include <linux/smp_lock.h>
6+#include <linux/slab.h>
7+#include <linux/string.h>
8+#include <linux/uaccess.h>
9+#include <linux/kernel.h>
10+#include <linux/namei.h>
11+#include <linux/writeback.h>
12+#include <linux/vmalloc.h>
13+#include <linux/pagevec.h>
14+
15+#include "super.h"
16+#include "decode.h"
17+
18+/*
19+ * Ceph inode operations
20+ *
21+ * Implement basic inode helpers (get, alloc) and inode ops (getattr,
22+ * setattr, etc.), xattr helpers, and helpers for assimilating
23+ * metadata returned by the MDS into our cache.
24+ *
25+ * Also define helpers for doing asynchronous writeback, invalidation,
26+ * and truncation for the benefit of those who can't afford to block
27+ * (typically because they are in the message handler path).
28+ */
29+
30+static const struct inode_operations ceph_symlink_iops;
31+
32+static void ceph_invalidate_work(struct work_struct *work);
33+static void ceph_writeback_work(struct work_struct *work);
34+static void ceph_vmtruncate_work(struct work_struct *work);
35+
36+/*
37+ * find or create an inode, given the ceph ino number
38+ */
39+struct inode *ceph_get_inode(struct super_block *sb, struct ceph_vino vino)
40+{
41+ struct inode *inode;
42+ ino_t t = ceph_vino_to_ino(vino);
43+
44+ inode = iget5_locked(sb, t, ceph_ino_compare, ceph_set_ino_cb, &vino);
45+ if (inode == NULL)
46+ return ERR_PTR(-ENOMEM);
47+ if (inode->i_state & I_NEW) {
48+ dout("get_inode created new inode %p %llx.%llx ino %llx\n",
49+ inode, ceph_vinop(inode), (u64)inode->i_ino);
50+ unlock_new_inode(inode);
51+ }
52+
53+ dout("get_inode on %lu=%llx.%llx got %p\n", inode->i_ino, vino.ino,
54+ vino.snap, inode);
55+ return inode;
56+}
57+
58+/*
59+ * get/constuct snapdir inode for a given directory
60+ */
61+struct inode *ceph_get_snapdir(struct inode *parent)
62+{
63+ struct ceph_vino vino = {
64+ .ino = ceph_ino(parent),
65+ .snap = CEPH_SNAPDIR,
66+ };
67+ struct inode *inode = ceph_get_inode(parent->i_sb, vino);
68+ struct ceph_inode_info *ci = ceph_inode(inode);
69+
70+ BUG_ON(!S_ISDIR(parent->i_mode));
71+ if (IS_ERR(inode))
72+ return ERR_PTR(PTR_ERR(inode));
73+ inode->i_mode = parent->i_mode;
74+ inode->i_uid = parent->i_uid;
75+ inode->i_gid = parent->i_gid;
76+ inode->i_op = &ceph_dir_iops;
77+ inode->i_fop = &ceph_dir_fops;
78+ ci->i_snap_caps = CEPH_CAP_PIN; /* so we can open */
79+ ci->i_rbytes = 0;
80+ return inode;
81+}
82+
83+const struct inode_operations ceph_file_iops = {
84+ .permission = ceph_permission,
85+ .setattr = ceph_setattr,
86+ .getattr = ceph_getattr,
87+ .setxattr = ceph_setxattr,
88+ .getxattr = ceph_getxattr,
89+ .listxattr = ceph_listxattr,
90+ .removexattr = ceph_removexattr,
91+};
92+
93+
94+/*
95+ * We use a 'frag tree' to keep track of the MDS's directory fragments
96+ * for a given inode (usually there is just a single fragment). We
97+ * need to know when a child frag is delegated to a new MDS, or when
98+ * it is flagged as replicated, so we can direct our requests
99+ * accordingly.
100+ */
101+
102+/*
103+ * find/create a frag in the tree
104+ */
105+static struct ceph_inode_frag *__get_or_create_frag(struct ceph_inode_info *ci,
106+ u32 f)
107+{
108+ struct rb_node **p;
109+ struct rb_node *parent = NULL;
110+ struct ceph_inode_frag *frag;
111+ int c;
112+
113+ p = &ci->i_fragtree.rb_node;
114+ while (*p) {
115+ parent = *p;
116+ frag = rb_entry(parent, struct ceph_inode_frag, node);
117+ c = ceph_frag_compare(f, frag->frag);
118+ if (c < 0)
119+ p = &(*p)->rb_left;
120+ else if (c > 0)
121+ p = &(*p)->rb_right;
122+ else
123+ return frag;
124+ }
125+
126+ frag = kmalloc(sizeof(*frag), GFP_NOFS);
127+ if (!frag) {
128+ pr_err("__get_or_create_frag ENOMEM on %p %llx.%llx "
129+ "frag %x\n", &ci->vfs_inode,
130+ ceph_vinop(&ci->vfs_inode), f);
131+ return ERR_PTR(-ENOMEM);
132+ }
133+ frag->frag = f;
134+ frag->split_by = 0;
135+ frag->mds = -1;
136+ frag->ndist = 0;
137+
138+ rb_link_node(&frag->node, parent, p);
139+ rb_insert_color(&frag->node, &ci->i_fragtree);
140+
141+ dout("get_or_create_frag added %llx.%llx frag %x\n",
142+ ceph_vinop(&ci->vfs_inode), f);
143+ return frag;
144+}
145+
146+/*
147+ * find a specific frag @f
148+ */
149+struct ceph_inode_frag *__ceph_find_frag(struct ceph_inode_info *ci, u32 f)
150+{
151+ struct rb_node *n = ci->i_fragtree.rb_node;
152+
153+ while (n) {
154+ struct ceph_inode_frag *frag =
155+ rb_entry(n, struct ceph_inode_frag, node);
156+ int c = ceph_frag_compare(f, frag->frag);
157+ if (c < 0)
158+ n = n->rb_left;
159+ else if (c > 0)
160+ n = n->rb_right;
161+ else
162+ return frag;
163+ }
164+ return NULL;
165+}
166+
167+/*
168+ * Choose frag containing the given value @v. If @pfrag is
169+ * specified, copy the frag delegation info to the caller if
170+ * it is present.
171+ */
172+u32 ceph_choose_frag(struct ceph_inode_info *ci, u32 v,
173+ struct ceph_inode_frag *pfrag,
174+ int *found)
175+{
176+ u32 t = ceph_frag_make(0, 0);
177+ struct ceph_inode_frag *frag;
178+ unsigned nway, i;
179+ u32 n;
180+
181+ if (found)
182+ *found = 0;
183+
184+ mutex_lock(&ci->i_fragtree_mutex);
185+ while (1) {
186+ WARN_ON(!ceph_frag_contains_value(t, v));
187+ frag = __ceph_find_frag(ci, t);
188+ if (!frag)
189+ break; /* t is a leaf */
190+ if (frag->split_by == 0) {
191+ if (pfrag)
192+ memcpy(pfrag, frag, sizeof(*pfrag));
193+ if (found)
194+ *found = 1;
195+ break;
196+ }
197+
198+ /* choose child */
199+ nway = 1 << frag->split_by;
200+ dout("choose_frag(%x) %x splits by %d (%d ways)\n", v, t,
201+ frag->split_by, nway);
202+ for (i = 0; i < nway; i++) {
203+ n = ceph_frag_make_child(t, frag->split_by, i);
204+ if (ceph_frag_contains_value(n, v)) {
205+ t = n;
206+ break;
207+ }
208+ }
209+ BUG_ON(i == nway);
210+ }
211+ dout("choose_frag(%x) = %x\n", v, t);
212+
213+ mutex_unlock(&ci->i_fragtree_mutex);
214+ return t;
215+}
216+
217+/*
218+ * Process dirfrag (delegation) info from the mds. Include leaf
219+ * fragment in tree ONLY if ndist > 0. Otherwise, only
220+ * branches/splits are included in i_fragtree)
221+ */
222+static int ceph_fill_dirfrag(struct inode *inode,
223+ struct ceph_mds_reply_dirfrag *dirinfo)
224+{
225+ struct ceph_inode_info *ci = ceph_inode(inode);
226+ struct ceph_inode_frag *frag;
227+ u32 id = le32_to_cpu(dirinfo->frag);
228+ int mds = le32_to_cpu(dirinfo->auth);
229+ int ndist = le32_to_cpu(dirinfo->ndist);
230+ int i;
231+ int err = 0;
232+
233+ mutex_lock(&ci->i_fragtree_mutex);
234+ if (ndist == 0) {
235+ /* no delegation info needed. */
236+ frag = __ceph_find_frag(ci, id);
237+ if (!frag)
238+ goto out;
239+ if (frag->split_by == 0) {
240+ /* tree leaf, remove */
241+ dout("fill_dirfrag removed %llx.%llx frag %x"
242+ " (no ref)\n", ceph_vinop(inode), id);
243+ rb_erase(&frag->node, &ci->i_fragtree);
244+ kfree(frag);
245+ } else {
246+ /* tree branch, keep and clear */
247+ dout("fill_dirfrag cleared %llx.%llx frag %x"
248+ " referral\n", ceph_vinop(inode), id);
249+ frag->mds = -1;
250+ frag->ndist = 0;
251+ }
252+ goto out;
253+ }
254+
255+
256+ /* find/add this frag to store mds delegation info */
257+ frag = __get_or_create_frag(ci, id);
258+ if (IS_ERR(frag)) {
259+ /* this is not the end of the world; we can continue
260+ with bad/inaccurate delegation info */
261+ pr_err("fill_dirfrag ENOMEM on mds ref %llx.%llx fg %x\n",
262+ ceph_vinop(inode), le32_to_cpu(dirinfo->frag));
263+ err = -ENOMEM;
264+ goto out;
265+ }
266+
267+ frag->mds = mds;
268+ frag->ndist = min_t(u32, ndist, CEPH_MAX_DIRFRAG_REP);
269+ for (i = 0; i < frag->ndist; i++)
270+ frag->dist[i] = le32_to_cpu(dirinfo->dist[i]);
271+ dout("fill_dirfrag %llx.%llx frag %x ndist=%d\n",
272+ ceph_vinop(inode), frag->frag, frag->ndist);
273+
274+out:
275+ mutex_unlock(&ci->i_fragtree_mutex);
276+ return err;
277+}
278+
279+
280+/*
281+ * initialize a newly allocated inode.
282+ */
283+struct inode *ceph_alloc_inode(struct super_block *sb)
284+{
285+ struct ceph_inode_info *ci;
286+ int i;
287+
288+ ci = kmem_cache_alloc(ceph_inode_cachep, GFP_NOFS);
289+ if (!ci)
290+ return NULL;
291+
292+ dout("alloc_inode %p\n", &ci->vfs_inode);
293+
294+ ci->i_version = 0;
295+ ci->i_time_warp_seq = 0;
296+ ci->i_ceph_flags = 0;
297+ ci->i_release_count = 0;
298+ ci->i_symlink = NULL;
299+
300+ ci->i_fragtree = RB_ROOT;
301+ mutex_init(&ci->i_fragtree_mutex);
302+
303+ ci->i_xattrs.blob = NULL;
304+ ci->i_xattrs.prealloc_blob = NULL;
305+ ci->i_xattrs.dirty = false;
306+ ci->i_xattrs.index = RB_ROOT;
307+ ci->i_xattrs.count = 0;
308+ ci->i_xattrs.names_size = 0;
309+ ci->i_xattrs.vals_size = 0;
310+ ci->i_xattrs.version = 0;
311+ ci->i_xattrs.index_version = 0;
312+
313+ ci->i_caps = RB_ROOT;
314+ ci->i_auth_cap = NULL;
315+ ci->i_dirty_caps = 0;
316+ ci->i_flushing_caps = 0;
317+ INIT_LIST_HEAD(&ci->i_dirty_item);
318+ INIT_LIST_HEAD(&ci->i_flushing_item);
319+ ci->i_cap_flush_seq = 0;
320+ ci->i_cap_flush_last_tid = 0;
321+ memset(&ci->i_cap_flush_tid, 0, sizeof(ci->i_cap_flush_tid));
322+ init_waitqueue_head(&ci->i_cap_wq);
323+ ci->i_hold_caps_min = 0;
324+ ci->i_hold_caps_max = 0;
325+ INIT_LIST_HEAD(&ci->i_cap_delay_list);
326+ ci->i_cap_exporting_mds = 0;
327+ ci->i_cap_exporting_mseq = 0;
328+ ci->i_cap_exporting_issued = 0;
329+ INIT_LIST_HEAD(&ci->i_cap_snaps);
330+ ci->i_head_snapc = NULL;
331+ ci->i_snap_caps = 0;
332+
333+ for (i = 0; i < CEPH_FILE_MODE_NUM; i++)
334+ ci->i_nr_by_mode[i] = 0;
335+
336+ ci->i_truncate_seq = 0;
337+ ci->i_truncate_size = 0;
338+ ci->i_truncate_pending = 0;
339+
340+ ci->i_max_size = 0;
341+ ci->i_reported_size = 0;
342+ ci->i_wanted_max_size = 0;
343+ ci->i_requested_max_size = 0;
344+
345+ ci->i_pin_ref = 0;
346+ ci->i_rd_ref = 0;
347+ ci->i_rdcache_ref = 0;
348+ ci->i_wr_ref = 0;
349+ ci->i_wrbuffer_ref = 0;
350+ ci->i_wrbuffer_ref_head = 0;
351+ ci->i_shared_gen = 0;
352+ ci->i_rdcache_gen = 0;
353+ ci->i_rdcache_revoking = 0;
354+
355+ INIT_LIST_HEAD(&ci->i_unsafe_writes);
356+ INIT_LIST_HEAD(&ci->i_unsafe_dirops);
357+ spin_lock_init(&ci->i_unsafe_lock);
358+
359+ ci->i_snap_realm = NULL;
360+ INIT_LIST_HEAD(&ci->i_snap_realm_item);
361+ INIT_LIST_HEAD(&ci->i_snap_flush_item);
362+
363+ INIT_WORK(&ci->i_wb_work, ceph_writeback_work);
364+ INIT_WORK(&ci->i_pg_inv_work, ceph_invalidate_work);
365+
366+ INIT_WORK(&ci->i_vmtruncate_work, ceph_vmtruncate_work);
367+
368+ return &ci->vfs_inode;
369+}
370+
371+void ceph_destroy_inode(struct inode *inode)
372+{
373+ struct ceph_inode_info *ci = ceph_inode(inode);
374+ struct ceph_inode_frag *frag;
375+ struct rb_node *n;
376+
377+ dout("destroy_inode %p ino %llx.%llx\n", inode, ceph_vinop(inode));
378+
379+ ceph_queue_caps_release(inode);
380+
381+ kfree(ci->i_symlink);
382+ while ((n = rb_first(&ci->i_fragtree)) != NULL) {
383+ frag = rb_entry(n, struct ceph_inode_frag, node);
384+ rb_erase(n, &ci->i_fragtree);
385+ kfree(frag);
386+ }
387+
388+ __ceph_destroy_xattrs(ci);
389+ if (ci->i_xattrs.blob)
390+ ceph_buffer_put(ci->i_xattrs.blob);
391+ if (ci->i_xattrs.prealloc_blob)
392+ ceph_buffer_put(ci->i_xattrs.prealloc_blob);
393+
394+ kmem_cache_free(ceph_inode_cachep, ci);
395+}
396+
397+
398+/*
399+ * Helpers to fill in size, ctime, mtime, and atime. We have to be
400+ * careful because either the client or MDS may have more up to date
401+ * info, depending on which capabilities are held, and whether
402+ * time_warp_seq or truncate_seq have increased. (Ordinarily, mtime
403+ * and size are monotonically increasing, except when utimes() or
404+ * truncate() increments the corresponding _seq values.)
405+ */
406+int ceph_fill_file_size(struct inode *inode, int issued,
407+ u32 truncate_seq, u64 truncate_size, u64 size)
408+{
409+ struct ceph_inode_info *ci = ceph_inode(inode);
410+ int queue_trunc = 0;
411+
412+ if (ceph_seq_cmp(truncate_seq, ci->i_truncate_seq) > 0 ||
413+ (truncate_seq == ci->i_truncate_seq && size > inode->i_size)) {
414+ dout("size %lld -> %llu\n", inode->i_size, size);
415+ inode->i_size = size;
416+ inode->i_blocks = (size + (1<<9) - 1) >> 9;
417+ ci->i_reported_size = size;
418+ if (truncate_seq != ci->i_truncate_seq) {
419+ dout("truncate_seq %u -> %u\n",
420+ ci->i_truncate_seq, truncate_seq);
421+ ci->i_truncate_seq = truncate_seq;
422+ /*
423+ * If we hold relevant caps, or in the case where we're
424+ * not the only client referencing this file and we
425+ * don't hold those caps, then we need to check whether
426+ * the file is either opened or mmaped
427+ */
428+ if ((issued & (CEPH_CAP_FILE_CACHE|CEPH_CAP_FILE_RD|
429+ CEPH_CAP_FILE_WR|CEPH_CAP_FILE_BUFFER|
430+ CEPH_CAP_FILE_EXCL)) ||
431+ mapping_mapped(inode->i_mapping) ||
432+ __ceph_caps_file_wanted(ci)) {
433+ ci->i_truncate_pending++;
434+ queue_trunc = 1;
435+ }
436+ }
437+ }
438+ if (ceph_seq_cmp(truncate_seq, ci->i_truncate_seq) >= 0 &&
439+ ci->i_truncate_size != truncate_size) {
440+ dout("truncate_size %lld -> %llu\n", ci->i_truncate_size,
441+ truncate_size);
442+ ci->i_truncate_size = truncate_size;
443+ }
444+ return queue_trunc;
445+}
446+
447+void ceph_fill_file_time(struct inode *inode, int issued,
448+ u64 time_warp_seq, struct timespec *ctime,
449+ struct timespec *mtime, struct timespec *atime)
450+{
451+ struct ceph_inode_info *ci = ceph_inode(inode);
452+ int warn = 0;
453+
454+ if (issued & (CEPH_CAP_FILE_EXCL|
455+ CEPH_CAP_FILE_WR|
456+ CEPH_CAP_FILE_BUFFER)) {
457+ if (timespec_compare(ctime, &inode->i_ctime) > 0) {
458+ dout("ctime %ld.%09ld -> %ld.%09ld inc w/ cap\n",
459+ inode->i_ctime.tv_sec, inode->i_ctime.tv_nsec,
460+ ctime->tv_sec, ctime->tv_nsec);
461+ inode->i_ctime = *ctime;
462+ }
463+ if (ceph_seq_cmp(time_warp_seq, ci->i_time_warp_seq) > 0) {
464+ /* the MDS did a utimes() */
465+ dout("mtime %ld.%09ld -> %ld.%09ld "
466+ "tw %d -> %d\n",
467+ inode->i_mtime.tv_sec, inode->i_mtime.tv_nsec,
468+ mtime->tv_sec, mtime->tv_nsec,
469+ ci->i_time_warp_seq, (int)time_warp_seq);
470+
471+ inode->i_mtime = *mtime;
472+ inode->i_atime = *atime;
473+ ci->i_time_warp_seq = time_warp_seq;
474+ } else if (time_warp_seq == ci->i_time_warp_seq) {
475+ /* nobody did utimes(); take the max */
476+ if (timespec_compare(mtime, &inode->i_mtime) > 0) {
477+ dout("mtime %ld.%09ld -> %ld.%09ld inc\n",
478+ inode->i_mtime.tv_sec,
479+ inode->i_mtime.tv_nsec,
480+ mtime->tv_sec, mtime->tv_nsec);
481+ inode->i_mtime = *mtime;
482+ }
483+ if (timespec_compare(atime, &inode->i_atime) > 0) {
484+ dout("atime %ld.%09ld -> %ld.%09ld inc\n",
485+ inode->i_atime.tv_sec,
486+ inode->i_atime.tv_nsec,
487+ atime->tv_sec, atime->tv_nsec);
488+ inode->i_atime = *atime;
489+ }
490+ } else if (issued & CEPH_CAP_FILE_EXCL) {
491+ /* we did a utimes(); ignore mds values */
492+ } else {
493+ warn = 1;
494+ }
495+ } else {
496+ /* we have no write caps; whatever the MDS says is true */
497+ if (ceph_seq_cmp(time_warp_seq, ci->i_time_warp_seq) >= 0) {
498+ inode->i_ctime = *ctime;
499+ inode->i_mtime = *mtime;
500+ inode->i_atime = *atime;
501+ ci->i_time_warp_seq = time_warp_seq;
502+ } else {
503+ warn = 1;
504+ }
505+ }
506+ if (warn) /* time_warp_seq shouldn't go backwards */
507+ dout("%p mds time_warp_seq %llu < %u\n",
508+ inode, time_warp_seq, ci->i_time_warp_seq);
509+}
510+
511+/*
512+ * Populate an inode based on info from mds. May be called on new or
513+ * existing inodes.
514+ */
515+static int fill_inode(struct inode *inode,
516+ struct ceph_mds_reply_info_in *iinfo,
517+ struct ceph_mds_reply_dirfrag *dirinfo,
518+ struct ceph_mds_session *session,
519+ unsigned long ttl_from, int cap_fmode,
520+ struct ceph_cap_reservation *caps_reservation)
521+{
522+ struct ceph_mds_reply_inode *info = iinfo->in;
523+ struct ceph_inode_info *ci = ceph_inode(inode);
524+ int i;
525+ int issued, implemented;
526+ struct timespec mtime, atime, ctime;
527+ u32 nsplits;
528+ struct ceph_buffer *xattr_blob = NULL;
529+ int err = 0;
530+ int queue_trunc = 0;
531+
532+ dout("fill_inode %p ino %llx.%llx v %llu had %llu\n",
533+ inode, ceph_vinop(inode), le64_to_cpu(info->version),
534+ ci->i_version);
535+
536+ /*
537+ * prealloc xattr data, if it looks like we'll need it. only
538+ * if len > 4 (meaning there are actually xattrs; the first 4
539+ * bytes are the xattr count).
540+ */
541+ if (iinfo->xattr_len > 4) {
542+ xattr_blob = ceph_buffer_new(iinfo->xattr_len, GFP_NOFS);
543+ if (!xattr_blob)
544+ pr_err("fill_inode ENOMEM xattr blob %d bytes\n",
545+ iinfo->xattr_len);
546+ }
547+
548+ spin_lock(&inode->i_lock);
549+
550+ /*
551+ * provided version will be odd if inode value is projected,
552+ * even if stable. skip the update if we have a newer info
553+ * (e.g., due to inode info racing form multiple MDSs), or if
554+ * we are getting projected (unstable) inode info.
555+ */
556+ if (le64_to_cpu(info->version) > 0 &&
557+ (ci->i_version & ~1) > le64_to_cpu(info->version))
558+ goto no_change;
559+
560+ issued = __ceph_caps_issued(ci, &implemented);
561+ issued |= implemented | __ceph_caps_dirty(ci);
562+
563+ /* update inode */
564+ ci->i_version = le64_to_cpu(info->version);
565+ inode->i_version++;
566+ inode->i_rdev = le32_to_cpu(info->rdev);
567+
568+ if ((issued & CEPH_CAP_AUTH_EXCL) == 0) {
569+ inode->i_mode = le32_to_cpu(info->mode);
570+ inode->i_uid = le32_to_cpu(info->uid);
571+ inode->i_gid = le32_to_cpu(info->gid);
572+ dout("%p mode 0%o uid.gid %d.%d\n", inode, inode->i_mode,
573+ inode->i_uid, inode->i_gid);
574+ }
575+
576+ if ((issued & CEPH_CAP_LINK_EXCL) == 0)
577+ inode->i_nlink = le32_to_cpu(info->nlink);
578+
579+ /* be careful with mtime, atime, size */
580+ ceph_decode_timespec(&atime, &info->atime);
581+ ceph_decode_timespec(&mtime, &info->mtime);
582+ ceph_decode_timespec(&ctime, &info->ctime);
583+ queue_trunc = ceph_fill_file_size(inode, issued,
584+ le32_to_cpu(info->truncate_seq),
585+ le64_to_cpu(info->truncate_size),
586+ le64_to_cpu(info->size));
587+ ceph_fill_file_time(inode, issued,
588+ le32_to_cpu(info->time_warp_seq),
589+ &ctime, &mtime, &atime);
590+
591+ ci->i_max_size = le64_to_cpu(info->max_size);
592+ ci->i_layout = info->layout;
593+ inode->i_blkbits = fls(le32_to_cpu(info->layout.fl_stripe_unit)) - 1;
594+
595+ /* xattrs */
596+ /* note that if i_xattrs.len <= 4, i_xattrs.data will still be NULL. */
597+ if ((issued & CEPH_CAP_XATTR_EXCL) == 0 &&
598+ le64_to_cpu(info->xattr_version) > ci->i_xattrs.version) {
599+ if (ci->i_xattrs.blob)
600+ ceph_buffer_put(ci->i_xattrs.blob);
601+ ci->i_xattrs.blob = xattr_blob;
602+ if (xattr_blob)
603+ memcpy(ci->i_xattrs.blob->vec.iov_base,
604+ iinfo->xattr_data, iinfo->xattr_len);
605+ ci->i_xattrs.version = le64_to_cpu(info->xattr_version);
606+ }
607+
608+ inode->i_mapping->a_ops = &ceph_aops;
609+ inode->i_mapping->backing_dev_info =
610+ &ceph_client(inode->i_sb)->backing_dev_info;
611+
612+ switch (inode->i_mode & S_IFMT) {
613+ case S_IFIFO:
614+ case S_IFBLK:
615+ case S_IFCHR:
616+ case S_IFSOCK:
617+ init_special_inode(inode, inode->i_mode, inode->i_rdev);
618+ inode->i_op = &ceph_file_iops;
619+ break;
620+ case S_IFREG:
621+ inode->i_op = &ceph_file_iops;
622+ inode->i_fop = &ceph_file_fops;
623+ break;
624+ case S_IFLNK:
625+ inode->i_op = &ceph_symlink_iops;
626+ if (!ci->i_symlink) {
627+ int symlen = iinfo->symlink_len;
628+ char *sym;
629+
630+ BUG_ON(symlen != inode->i_size);
631+ spin_unlock(&inode->i_lock);
632+
633+ err = -ENOMEM;
634+ sym = kmalloc(symlen+1, GFP_NOFS);
635+ if (!sym)
636+ goto out;
637+ memcpy(sym, iinfo->symlink, symlen);
638+ sym[symlen] = 0;
639+
640+ spin_lock(&inode->i_lock);
641+ if (!ci->i_symlink)
642+ ci->i_symlink = sym;
643+ else
644+ kfree(sym); /* lost a race */
645+ }
646+ break;
647+ case S_IFDIR:
648+ inode->i_op = &ceph_dir_iops;
649+ inode->i_fop = &ceph_dir_fops;
650+
651+ ci->i_files = le64_to_cpu(info->files);
652+ ci->i_subdirs = le64_to_cpu(info->subdirs);
653+ ci->i_rbytes = le64_to_cpu(info->rbytes);
654+ ci->i_rfiles = le64_to_cpu(info->rfiles);
655+ ci->i_rsubdirs = le64_to_cpu(info->rsubdirs);
656+ ceph_decode_timespec(&ci->i_rctime, &info->rctime);
657+
658+ /* set dir completion flag? */
659+ if (ci->i_files == 0 && ci->i_subdirs == 0 &&
660+ ceph_snap(inode) == CEPH_NOSNAP &&
661+ (le32_to_cpu(info->cap.caps) & CEPH_CAP_FILE_SHARED)) {
662+ dout(" marking %p complete (empty)\n", inode);
663+ ci->i_ceph_flags |= CEPH_I_COMPLETE;
664+ ci->i_max_offset = 2;
665+ }
666+
667+ /* it may be better to set st_size in getattr instead? */
668+ if (ceph_test_opt(ceph_client(inode->i_sb), RBYTES))
669+ inode->i_size = ci->i_rbytes;
670+ break;
671+ default:
672+ pr_err("fill_inode %llx.%llx BAD mode 0%o\n",
673+ ceph_vinop(inode), inode->i_mode);
674+ }
675+
676+no_change:
677+ spin_unlock(&inode->i_lock);
678+
679+ /* queue truncate if we saw i_size decrease */
680+ if (queue_trunc)
681+ ceph_queue_vmtruncate(inode);
682+
683+ /* populate frag tree */
684+ /* FIXME: move me up, if/when version reflects fragtree changes */
685+ nsplits = le32_to_cpu(info->fragtree.nsplits);
686+ mutex_lock(&ci->i_fragtree_mutex);
687+ for (i = 0; i < nsplits; i++) {
688+ u32 id = le32_to_cpu(info->fragtree.splits[i].frag);
689+ struct ceph_inode_frag *frag = __get_or_create_frag(ci, id);
690+
691+ if (IS_ERR(frag))
692+ continue;
693+ frag->split_by = le32_to_cpu(info->fragtree.splits[i].by);
694+ dout(" frag %x split by %d\n", frag->frag, frag->split_by);
695+ }
696+ mutex_unlock(&ci->i_fragtree_mutex);
697+
698+ /* were we issued a capability? */
699+ if (info->cap.caps) {
700+ if (ceph_snap(inode) == CEPH_NOSNAP) {
701+ ceph_add_cap(inode, session,
702+ le64_to_cpu(info->cap.cap_id),
703+ cap_fmode,
704+ le32_to_cpu(info->cap.caps),
705+ le32_to_cpu(info->cap.wanted),
706+ le32_to_cpu(info->cap.seq),
707+ le32_to_cpu(info->cap.mseq),
708+ le64_to_cpu(info->cap.realm),
709+ info->cap.flags,
710+ caps_reservation);
711+ } else {
712+ spin_lock(&inode->i_lock);
713+ dout(" %p got snap_caps %s\n", inode,
714+ ceph_cap_string(le32_to_cpu(info->cap.caps)));
715+ ci->i_snap_caps |= le32_to_cpu(info->cap.caps);
716+ if (cap_fmode >= 0)
717+ __ceph_get_fmode(ci, cap_fmode);
718+ spin_unlock(&inode->i_lock);
719+ }
720+ }
721+
722+ /* update delegation info? */
723+ if (dirinfo)
724+ ceph_fill_dirfrag(inode, dirinfo);
725+
726+ err = 0;
727+
728+out:
729+ if (xattr_blob)
730+ ceph_buffer_put(xattr_blob);
731+ return err;
732+}
733+
734+/*
735+ * caller should hold session s_mutex.
736+ */
737+static void update_dentry_lease(struct dentry *dentry,
738+ struct ceph_mds_reply_lease *lease,
739+ struct ceph_mds_session *session,
740+ unsigned long from_time)
741+{
742+ struct ceph_dentry_info *di = ceph_dentry(dentry);
743+ long unsigned duration = le32_to_cpu(lease->duration_ms);
744+ long unsigned ttl = from_time + (duration * HZ) / 1000;
745+ long unsigned half_ttl = from_time + (duration * HZ / 2) / 1000;
746+ struct inode *dir;
747+
748+ /* only track leases on regular dentries */
749+ if (dentry->d_op != &ceph_dentry_ops)
750+ return;
751+
752+ spin_lock(&dentry->d_lock);
753+ dout("update_dentry_lease %p mask %d duration %lu ms ttl %lu\n",
754+ dentry, le16_to_cpu(lease->mask), duration, ttl);
755+
756+ /* make lease_rdcache_gen match directory */
757+ dir = dentry->d_parent->d_inode;
758+ di->lease_shared_gen = ceph_inode(dir)->i_shared_gen;
759+
760+ if (lease->mask == 0)
761+ goto out_unlock;
762+
763+ if (di->lease_gen == session->s_cap_gen &&
764+ time_before(ttl, dentry->d_time))
765+ goto out_unlock; /* we already have a newer lease. */
766+
767+ if (di->lease_session && di->lease_session != session)
768+ goto out_unlock;
769+
770+ ceph_dentry_lru_touch(dentry);
771+
772+ if (!di->lease_session)
773+ di->lease_session = ceph_get_mds_session(session);
774+ di->lease_gen = session->s_cap_gen;
775+ di->lease_seq = le32_to_cpu(lease->seq);
776+ di->lease_renew_after = half_ttl;
777+ di->lease_renew_from = 0;
778+ dentry->d_time = ttl;
779+out_unlock:
780+ spin_unlock(&dentry->d_lock);
781+ return;
782+}
783+
784+/*
785+ * splice a dentry to an inode.
786+ * caller must hold directory i_mutex for this to be safe.
787+ *
788+ * we will only rehash the resulting dentry if @prehash is
789+ * true; @prehash will be set to false (for the benefit of
790+ * the caller) if we fail.
791+ */
792+static struct dentry *splice_dentry(struct dentry *dn, struct inode *in,
793+ bool *prehash)
794+{
795+ struct dentry *realdn;
796+
797+ /* dn must be unhashed */
798+ if (!d_unhashed(dn))
799+ d_drop(dn);
800+ realdn = d_materialise_unique(dn, in);
801+ if (IS_ERR(realdn)) {
802+ pr_err("splice_dentry error %p inode %p ino %llx.%llx\n",
803+ dn, in, ceph_vinop(in));
804+ if (prehash)
805+ *prehash = false; /* don't rehash on error */
806+ dn = realdn; /* note realdn contains the error */
807+ goto out;
808+ } else if (realdn) {
809+ dout("dn %p (%d) spliced with %p (%d) "
810+ "inode %p ino %llx.%llx\n",
811+ dn, atomic_read(&dn->d_count),
812+ realdn, atomic_read(&realdn->d_count),
813+ realdn->d_inode, ceph_vinop(realdn->d_inode));
814+ dput(dn);
815+ dn = realdn;
816+ } else {
817+ BUG_ON(!ceph_dentry(dn));
818+
819+ dout("dn %p attached to %p ino %llx.%llx\n",
820+ dn, dn->d_inode, ceph_vinop(dn->d_inode));
821+ }
822+ if ((!prehash || *prehash) && d_unhashed(dn))
823+ d_rehash(dn);
824+out:
825+ return dn;
826+}
827+
828+/*
829+ * Set dentry's directory position based on the current dir's max, and
830+ * order it in d_subdirs, so that dcache_readdir behaves.
831+ */
832+static void ceph_set_dentry_offset(struct dentry *dn)
833+{
834+ struct dentry *dir = dn->d_parent;
835+ struct inode *inode = dn->d_parent->d_inode;
836+ struct ceph_dentry_info *di;
837+
838+ BUG_ON(!inode);
839+
840+ di = ceph_dentry(dn);
841+
842+ spin_lock(&inode->i_lock);
843+ di->offset = ceph_inode(inode)->i_max_offset++;
844+ spin_unlock(&inode->i_lock);
845+
846+ spin_lock(&dcache_lock);
847+ spin_lock(&dn->d_lock);
848+ list_move_tail(&dir->d_subdirs, &dn->d_u.d_child);
849+ dout("set_dentry_offset %p %lld (%p %p)\n", dn, di->offset,
850+ dn->d_u.d_child.prev, dn->d_u.d_child.next);
851+ spin_unlock(&dn->d_lock);
852+ spin_unlock(&dcache_lock);
853+}
854+
855+/*
856+ * Incorporate results into the local cache. This is either just
857+ * one inode, or a directory, dentry, and possibly linked-to inode (e.g.,
858+ * after a lookup).
859+ *
860+ * A reply may contain
861+ * a directory inode along with a dentry.
862+ * and/or a target inode
863+ *
864+ * Called with snap_rwsem (read).
865+ */
866+int ceph_fill_trace(struct super_block *sb, struct ceph_mds_request *req,
867+ struct ceph_mds_session *session)
868+{
869+ struct ceph_mds_reply_info_parsed *rinfo = &req->r_reply_info;
870+ struct inode *in = NULL;
871+ struct ceph_mds_reply_inode *ininfo;
872+ struct ceph_vino vino;
873+ int i = 0;
874+ int err = 0;
875+
876+ dout("fill_trace %p is_dentry %d is_target %d\n", req,
877+ rinfo->head->is_dentry, rinfo->head->is_target);
878+
879+#if 0
880+ /*
881+ * Debugging hook:
882+ *
883+ * If we resend completed ops to a recovering mds, we get no
884+ * trace. Since that is very rare, pretend this is the case
885+ * to ensure the 'no trace' handlers in the callers behave.
886+ *
887+ * Fill in inodes unconditionally to avoid breaking cap
888+ * invariants.
889+ */
890+ if (rinfo->head->op & CEPH_MDS_OP_WRITE) {
891+ pr_info("fill_trace faking empty trace on %lld %s\n",
892+ req->r_tid, ceph_mds_op_name(rinfo->head->op));
893+ if (rinfo->head->is_dentry) {
894+ rinfo->head->is_dentry = 0;
895+ err = fill_inode(req->r_locked_dir,
896+ &rinfo->diri, rinfo->dirfrag,
897+ session, req->r_request_started, -1);
898+ }
899+ if (rinfo->head->is_target) {
900+ rinfo->head->is_target = 0;
901+ ininfo = rinfo->targeti.in;
902+ vino.ino = le64_to_cpu(ininfo->ino);
903+ vino.snap = le64_to_cpu(ininfo->snapid);
904+ in = ceph_get_inode(sb, vino);
905+ err = fill_inode(in, &rinfo->targeti, NULL,
906+ session, req->r_request_started,
907+ req->r_fmode);
908+ iput(in);
909+ }
910+ }
911+#endif
912+
913+ if (!rinfo->head->is_target && !rinfo->head->is_dentry) {
914+ dout("fill_trace reply is empty!\n");
915+ if (rinfo->head->result == 0 && req->r_locked_dir) {
916+ struct ceph_inode_info *ci =
917+ ceph_inode(req->r_locked_dir);
918+ dout(" clearing %p complete (empty trace)\n",
919+ req->r_locked_dir);
920+ ci->i_ceph_flags &= ~CEPH_I_COMPLETE;
921+ ci->i_release_count++;
922+ }
923+ return 0;
924+ }
925+
926+ if (rinfo->head->is_dentry) {
927+ struct inode *dir = req->r_locked_dir;
928+
929+ err = fill_inode(dir, &rinfo->diri, rinfo->dirfrag,
930+ session, req->r_request_started, -1,
931+ &req->r_caps_reservation);
932+ if (err < 0)
933+ return err;
934+ }
935+
936+ if (rinfo->head->is_dentry && !req->r_aborted) {
937+ /*
938+ * lookup link rename : null -> possibly existing inode
939+ * mknod symlink mkdir : null -> new inode
940+ * unlink : linked -> null
941+ */
942+ struct inode *dir = req->r_locked_dir;
943+ struct dentry *dn = req->r_dentry;
944+ bool have_dir_cap, have_lease;
945+
946+ BUG_ON(!dn);
947+ BUG_ON(!dir);
948+ BUG_ON(dn->d_parent->d_inode != dir);
949+ BUG_ON(ceph_ino(dir) !=
950+ le64_to_cpu(rinfo->diri.in->ino));
951+ BUG_ON(ceph_snap(dir) !=
952+ le64_to_cpu(rinfo->diri.in->snapid));
953+
954+ /* do we have a lease on the whole dir? */
955+ have_dir_cap =
956+ (le32_to_cpu(rinfo->diri.in->cap.caps) &
957+ CEPH_CAP_FILE_SHARED);
958+
959+ /* do we have a dn lease? */
960+ have_lease = have_dir_cap ||
961+ (le16_to_cpu(rinfo->dlease->mask) &
962+ CEPH_LOCK_DN);
963+
964+ if (!have_lease)
965+ dout("fill_trace no dentry lease or dir cap\n");
966+
967+ /* rename? */
968+ if (req->r_old_dentry && req->r_op == CEPH_MDS_OP_RENAME) {
969+ dout(" src %p '%.*s' dst %p '%.*s'\n",
970+ req->r_old_dentry,
971+ req->r_old_dentry->d_name.len,
972+ req->r_old_dentry->d_name.name,
973+ dn, dn->d_name.len, dn->d_name.name);
974+ dout("fill_trace doing d_move %p -> %p\n",
975+ req->r_old_dentry, dn);
976+ d_move(req->r_old_dentry, dn);
977+ dout(" src %p '%.*s' dst %p '%.*s'\n",
978+ req->r_old_dentry,
979+ req->r_old_dentry->d_name.len,
980+ req->r_old_dentry->d_name.name,
981+ dn, dn->d_name.len, dn->d_name.name);
982+ /* ensure target dentry is invalidated, despite
983+ rehashing bug in vfs_rename_dir */
984+ dn->d_time = jiffies;
985+ ceph_dentry(dn)->lease_shared_gen = 0;
986+ /* take overwritten dentry's readdir offset */
987+ ceph_dentry(req->r_old_dentry)->offset =
988+ ceph_dentry(dn)->offset;
989+ dn = req->r_old_dentry; /* use old_dentry */
990+ in = dn->d_inode;
991+ }
992+
993+ /* null dentry? */
994+ if (!rinfo->head->is_target) {
995+ dout("fill_trace null dentry\n");
996+ if (dn->d_inode) {
997+ dout("d_delete %p\n", dn);
998+ d_delete(dn);
999+ } else {
1000+ dout("d_instantiate %p NULL\n", dn);
1001+ d_instantiate(dn, NULL);
1002+ if (have_lease && d_unhashed(dn))
1003+ d_rehash(dn);
1004+ update_dentry_lease(dn, rinfo->dlease,
1005+ session,
1006+ req->r_request_started);
1007+ }
1008+ goto done;
1009+ }
1010+
1011+ /* attach proper inode */
1012+ ininfo = rinfo->targeti.in;
1013+ vino.ino = le64_to_cpu(ininfo->ino);
1014+ vino.snap = le64_to_cpu(ininfo->snapid);
1015+ if (!dn->d_inode) {
1016+ in = ceph_get_inode(sb, vino);
1017+ if (IS_ERR(in)) {
1018+ pr_err("fill_trace bad get_inode "
1019+ "%llx.%llx\n", vino.ino, vino.snap);
1020+ err = PTR_ERR(in);
1021+ d_delete(dn);
1022+ goto done;
1023+ }
1024+ dn = splice_dentry(dn, in, &have_lease);
1025+ if (IS_ERR(dn)) {
1026+ err = PTR_ERR(dn);
1027+ goto done;
1028+ }
1029+ req->r_dentry = dn; /* may have spliced */
1030+ ceph_set_dentry_offset(dn);
1031+ igrab(in);
1032+ } else if (ceph_ino(in) == vino.ino &&
1033+ ceph_snap(in) == vino.snap) {
1034+ igrab(in);
1035+ } else {
1036+ dout(" %p links to %p %llx.%llx, not %llx.%llx\n",
1037+ dn, in, ceph_ino(in), ceph_snap(in),
1038+ vino.ino, vino.snap);
1039+ have_lease = false;
1040+ in = NULL;
1041+ }
1042+
1043+ if (have_lease)
1044+ update_dentry_lease(dn, rinfo->dlease, session,
1045+ req->r_request_started);
1046+ dout(" final dn %p\n", dn);
1047+ i++;
1048+ } else if (req->r_op == CEPH_MDS_OP_LOOKUPSNAP ||
1049+ req->r_op == CEPH_MDS_OP_MKSNAP) {
1050+ struct dentry *dn = req->r_dentry;
1051+
1052+ /* fill out a snapdir LOOKUPSNAP dentry */
1053+ BUG_ON(!dn);
1054+ BUG_ON(!req->r_locked_dir);
1055+ BUG_ON(ceph_snap(req->r_locked_dir) != CEPH_SNAPDIR);
1056+ ininfo = rinfo->targeti.in;
1057+ vino.ino = le64_to_cpu(ininfo->ino);
1058+ vino.snap = le64_to_cpu(ininfo->snapid);
1059+ in = ceph_get_inode(sb, vino);
1060+ if (IS_ERR(in)) {
1061+ pr_err("fill_inode get_inode badness %llx.%llx\n",
1062+ vino.ino, vino.snap);
1063+ err = PTR_ERR(in);
1064+ d_delete(dn);
1065+ goto done;
1066+ }
1067+ dout(" linking snapped dir %p to dn %p\n", in, dn);
1068+ dn = splice_dentry(dn, in, NULL);
1069+ if (IS_ERR(dn)) {
1070+ err = PTR_ERR(dn);
1071+ goto done;
1072+ }
1073+ ceph_set_dentry_offset(dn);
1074+ req->r_dentry = dn; /* may have spliced */
1075+ igrab(in);
1076+ rinfo->head->is_dentry = 1; /* fool notrace handlers */
1077+ }
1078+
1079+ if (rinfo->head->is_target) {
1080+ vino.ino = le64_to_cpu(rinfo->targeti.in->ino);
1081+ vino.snap = le64_to_cpu(rinfo->targeti.in->snapid);
1082+
1083+ if (in == NULL || ceph_ino(in) != vino.ino ||
1084+ ceph_snap(in) != vino.snap) {
1085+ in = ceph_get_inode(sb, vino);
1086+ if (IS_ERR(in)) {
1087+ err = PTR_ERR(in);
1088+ goto done;
1089+ }
1090+ }
1091+ req->r_target_inode = in;
1092+
1093+ err = fill_inode(in,
1094+ &rinfo->targeti, NULL,
1095+ session, req->r_request_started,
1096+ (le32_to_cpu(rinfo->head->result) == 0) ?
1097+ req->r_fmode : -1,
1098+ &req->r_caps_reservation);
1099+ if (err < 0) {
1100+ pr_err("fill_inode badness %p %llx.%llx\n",
1101+ in, ceph_vinop(in));
1102+ goto done;
1103+ }
1104+ }
1105+
1106+done:
1107+ dout("fill_trace done err=%d\n", err);
1108+ return err;
1109+}
1110+
1111+/*
1112+ * Prepopulate our cache with readdir results, leases, etc.
1113+ */
1114+int ceph_readdir_prepopulate(struct ceph_mds_request *req,
1115+ struct ceph_mds_session *session)
1116+{
1117+ struct dentry *parent = req->r_dentry;
1118+ struct ceph_mds_reply_info_parsed *rinfo = &req->r_reply_info;
1119+ struct qstr dname;
1120+ struct dentry *dn;
1121+ struct inode *in;
1122+ int err = 0, i;
1123+ struct inode *snapdir = NULL;
1124+ struct ceph_mds_request_head *rhead = req->r_request->front.iov_base;
1125+ u64 frag = le32_to_cpu(rhead->args.readdir.frag);
1126+ struct ceph_dentry_info *di;
1127+
1128+ if (le32_to_cpu(rinfo->head->op) == CEPH_MDS_OP_LSSNAP) {
1129+ snapdir = ceph_get_snapdir(parent->d_inode);
1130+ parent = d_find_alias(snapdir);
1131+ dout("readdir_prepopulate %d items under SNAPDIR dn %p\n",
1132+ rinfo->dir_nr, parent);
1133+ } else {
1134+ dout("readdir_prepopulate %d items under dn %p\n",
1135+ rinfo->dir_nr, parent);
1136+ if (rinfo->dir_dir)
1137+ ceph_fill_dirfrag(parent->d_inode, rinfo->dir_dir);
1138+ }
1139+
1140+ for (i = 0; i < rinfo->dir_nr; i++) {
1141+ struct ceph_vino vino;
1142+
1143+ dname.name = rinfo->dir_dname[i];
1144+ dname.len = rinfo->dir_dname_len[i];
1145+ dname.hash = full_name_hash(dname.name, dname.len);
1146+
1147+ vino.ino = le64_to_cpu(rinfo->dir_in[i].in->ino);
1148+ vino.snap = le64_to_cpu(rinfo->dir_in[i].in->snapid);
1149+
1150+retry_lookup:
1151+ dn = d_lookup(parent, &dname);
1152+ dout("d_lookup on parent=%p name=%.*s got %p\n",
1153+ parent, dname.len, dname.name, dn);
1154+
1155+ if (!dn) {
1156+ dn = d_alloc(parent, &dname);
1157+ dout("d_alloc %p '%.*s' = %p\n", parent,
1158+ dname.len, dname.name, dn);
1159+ if (dn == NULL) {
1160+ dout("d_alloc badness\n");
1161+ err = -ENOMEM;
1162+ goto out;
1163+ }
1164+ err = ceph_init_dentry(dn);
1165+ if (err < 0)
1166+ goto out;
1167+ } else if (dn->d_inode &&
1168+ (ceph_ino(dn->d_inode) != vino.ino ||
1169+ ceph_snap(dn->d_inode) != vino.snap)) {
1170+ dout(" dn %p points to wrong inode %p\n",
1171+ dn, dn->d_inode);
1172+ d_delete(dn);
1173+ dput(dn);
1174+ goto retry_lookup;
1175+ } else {
1176+ /* reorder parent's d_subdirs */
1177+ spin_lock(&dcache_lock);
1178+ spin_lock(&dn->d_lock);
1179+ list_move(&dn->d_u.d_child, &parent->d_subdirs);
1180+ spin_unlock(&dn->d_lock);
1181+ spin_unlock(&dcache_lock);
1182+ }
1183+
1184+ di = dn->d_fsdata;
1185+ di->offset = ceph_make_fpos(frag, i + req->r_readdir_offset);
1186+
1187+ /* inode */
1188+ if (dn->d_inode) {
1189+ in = dn->d_inode;
1190+ } else {
1191+ in = ceph_get_inode(parent->d_sb, vino);
1192+ if (in == NULL) {
1193+ dout("new_inode badness\n");
1194+ d_delete(dn);
1195+ dput(dn);
1196+ err = -ENOMEM;
1197+ goto out;
1198+ }
1199+ dn = splice_dentry(dn, in, NULL);
1200+ }
1201+
1202+ if (fill_inode(in, &rinfo->dir_in[i], NULL, session,
1203+ req->r_request_started, -1,
1204+ &req->r_caps_reservation) < 0) {
1205+ pr_err("fill_inode badness on %p\n", in);
1206+ dput(dn);
1207+ continue;
1208+ }
1209+ update_dentry_lease(dn, rinfo->dir_dlease[i],
1210+ req->r_session, req->r_request_started);
1211+ dput(dn);
1212+ }
1213+ req->r_did_prepopulate = true;
1214+
1215+out:
1216+ if (snapdir) {
1217+ iput(snapdir);
1218+ dput(parent);
1219+ }
1220+ dout("readdir_prepopulate done\n");
1221+ return err;
1222+}
1223+
1224+int ceph_inode_set_size(struct inode *inode, loff_t size)
1225+{
1226+ struct ceph_inode_info *ci = ceph_inode(inode);
1227+ int ret = 0;
1228+
1229+ spin_lock(&inode->i_lock);
1230+ dout("set_size %p %llu -> %llu\n", inode, inode->i_size, size);
1231+ inode->i_size = size;
1232+ inode->i_blocks = (size + (1 << 9) - 1) >> 9;
1233+
1234+ /* tell the MDS if we are approaching max_size */
1235+ if ((size << 1) >= ci->i_max_size &&
1236+ (ci->i_reported_size << 1) < ci->i_max_size)
1237+ ret = 1;
1238+
1239+ spin_unlock(&inode->i_lock);
1240+ return ret;
1241+}
1242+
1243+/*
1244+ * Write back inode data in a worker thread. (This can't be done
1245+ * in the message handler context.)
1246+ */
1247+void ceph_queue_writeback(struct inode *inode)
1248+{
1249+ if (queue_work(ceph_inode_to_client(inode)->wb_wq,
1250+ &ceph_inode(inode)->i_wb_work)) {
1251+ dout("ceph_queue_writeback %p\n", inode);
1252+ igrab(inode);
1253+ } else {
1254+ dout("ceph_queue_writeback %p failed\n", inode);
1255+ }
1256+}
1257+
1258+static void ceph_writeback_work(struct work_struct *work)
1259+{
1260+ struct ceph_inode_info *ci = container_of(work, struct ceph_inode_info,
1261+ i_wb_work);
1262+ struct inode *inode = &ci->vfs_inode;
1263+
1264+ dout("writeback %p\n", inode);
1265+ filemap_fdatawrite(&inode->i_data);
1266+ iput(inode);
1267+}
1268+
1269+/*
1270+ * queue an async invalidation
1271+ */
1272+void ceph_queue_invalidate(struct inode *inode)
1273+{
1274+ if (queue_work(ceph_inode_to_client(inode)->pg_inv_wq,
1275+ &ceph_inode(inode)->i_pg_inv_work)) {
1276+ dout("ceph_queue_invalidate %p\n", inode);
1277+ igrab(inode);
1278+ } else {
1279+ dout("ceph_queue_invalidate %p failed\n", inode);
1280+ }
1281+}
1282+
1283+/*
1284+ * invalidate any pages that are not dirty or under writeback. this
1285+ * includes pages that are clean and mapped.
1286+ */
1287+static void ceph_invalidate_nondirty_pages(struct address_space *mapping)
1288+{
1289+ struct pagevec pvec;
1290+ pgoff_t next = 0;
1291+ int i;
1292+
1293+ pagevec_init(&pvec, 0);
1294+ while (pagevec_lookup(&pvec, mapping, next, PAGEVEC_SIZE)) {
1295+ for (i = 0; i < pagevec_count(&pvec); i++) {
1296+ struct page *page = pvec.pages[i];
1297+ pgoff_t index;
1298+ int skip_page =
1299+ (PageDirty(page) || PageWriteback(page));
1300+
1301+ if (!skip_page)
1302+ skip_page = !trylock_page(page);
1303+
1304+ /*
1305+ * We really shouldn't be looking at the ->index of an
1306+ * unlocked page. But we're not allowed to lock these
1307+ * pages. So we rely upon nobody altering the ->index
1308+ * of this (pinned-by-us) page.
1309+ */
1310+ index = page->index;
1311+ if (index > next)
1312+ next = index;
1313+ next++;
1314+
1315+ if (skip_page)
1316+ continue;
1317+
1318+ generic_error_remove_page(mapping, page);
1319+ unlock_page(page);
1320+ }
1321+ pagevec_release(&pvec);
1322+ cond_resched();
1323+ }
1324+}
1325+
1326+/*
1327+ * Invalidate inode pages in a worker thread. (This can't be done
1328+ * in the message handler context.)
1329+ */
1330+static void ceph_invalidate_work(struct work_struct *work)
1331+{
1332+ struct ceph_inode_info *ci = container_of(work, struct ceph_inode_info,
1333+ i_pg_inv_work);
1334+ struct inode *inode = &ci->vfs_inode;
1335+ u32 orig_gen;
1336+ int check = 0;
1337+
1338+ spin_lock(&inode->i_lock);
1339+ dout("invalidate_pages %p gen %d revoking %d\n", inode,
1340+ ci->i_rdcache_gen, ci->i_rdcache_revoking);
1341+ if (ci->i_rdcache_gen == 0 ||
1342+ ci->i_rdcache_revoking != ci->i_rdcache_gen) {
1343+ BUG_ON(ci->i_rdcache_revoking > ci->i_rdcache_gen);
1344+ /* nevermind! */
1345+ ci->i_rdcache_revoking = 0;
1346+ spin_unlock(&inode->i_lock);
1347+ goto out;
1348+ }
1349+ orig_gen = ci->i_rdcache_gen;
1350+ spin_unlock(&inode->i_lock);
1351+
1352+ ceph_invalidate_nondirty_pages(inode->i_mapping);
1353+
1354+ spin_lock(&inode->i_lock);
1355+ if (orig_gen == ci->i_rdcache_gen) {
1356+ dout("invalidate_pages %p gen %d successful\n", inode,
1357+ ci->i_rdcache_gen);
1358+ ci->i_rdcache_gen = 0;
1359+ ci->i_rdcache_revoking = 0;
1360+ check = 1;
1361+ } else {
1362+ dout("invalidate_pages %p gen %d raced, gen now %d\n",
1363+ inode, orig_gen, ci->i_rdcache_gen);
1364+ }
1365+ spin_unlock(&inode->i_lock);
1366+
1367+ if (check)
1368+ ceph_check_caps(ci, 0, NULL);
1369+out:
1370+ iput(inode);
1371+}
1372+
1373+
1374+/*
1375+ * called by trunc_wq; take i_mutex ourselves
1376+ *
1377+ * We also truncate in a separate thread as well.
1378+ */
1379+static void ceph_vmtruncate_work(struct work_struct *work)
1380+{
1381+ struct ceph_inode_info *ci = container_of(work, struct ceph_inode_info,
1382+ i_vmtruncate_work);
1383+ struct inode *inode = &ci->vfs_inode;
1384+
1385+ dout("vmtruncate_work %p\n", inode);
1386+ mutex_lock(&inode->i_mutex);
1387+ __ceph_do_pending_vmtruncate(inode);
1388+ mutex_unlock(&inode->i_mutex);
1389+ iput(inode);
1390+}
1391+
1392+/*
1393+ * Queue an async vmtruncate. If we fail to queue work, we will handle
1394+ * the truncation the next time we call __ceph_do_pending_vmtruncate.
1395+ */
1396+void ceph_queue_vmtruncate(struct inode *inode)
1397+{
1398+ struct ceph_inode_info *ci = ceph_inode(inode);
1399+
1400+ if (queue_work(ceph_client(inode->i_sb)->trunc_wq,
1401+ &ci->i_vmtruncate_work)) {
1402+ dout("ceph_queue_vmtruncate %p\n", inode);
1403+ igrab(inode);
1404+ } else {
1405+ dout("ceph_queue_vmtruncate %p failed, pending=%d\n",
1406+ inode, ci->i_truncate_pending);
1407+ }
1408+}
1409+
1410+/*
1411+ * called with i_mutex held.
1412+ *
1413+ * Make sure any pending truncation is applied before doing anything
1414+ * that may depend on it.
1415+ */
1416+void __ceph_do_pending_vmtruncate(struct inode *inode)
1417+{
1418+ struct ceph_inode_info *ci = ceph_inode(inode);
1419+ u64 to;
1420+ int wrbuffer_refs, wake = 0;
1421+
1422+retry:
1423+ spin_lock(&inode->i_lock);
1424+ if (ci->i_truncate_pending == 0) {
1425+ dout("__do_pending_vmtruncate %p none pending\n", inode);
1426+ spin_unlock(&inode->i_lock);
1427+ return;
1428+ }
1429+
1430+ /*
1431+ * make sure any dirty snapped pages are flushed before we
1432+ * possibly truncate them.. so write AND block!
1433+ */
1434+ if (ci->i_wrbuffer_ref_head < ci->i_wrbuffer_ref) {
1435+ dout("__do_pending_vmtruncate %p flushing snaps first\n",
1436+ inode);
1437+ spin_unlock(&inode->i_lock);
1438+ filemap_write_and_wait_range(&inode->i_data, 0,
1439+ inode->i_sb->s_maxbytes);
1440+ goto retry;
1441+ }
1442+
1443+ to = ci->i_truncate_size;
1444+ wrbuffer_refs = ci->i_wrbuffer_ref;
1445+ dout("__do_pending_vmtruncate %p (%d) to %lld\n", inode,
1446+ ci->i_truncate_pending, to);
1447+ spin_unlock(&inode->i_lock);
1448+
1449+ truncate_inode_pages(inode->i_mapping, to);
1450+
1451+ spin_lock(&inode->i_lock);
1452+ ci->i_truncate_pending--;
1453+ if (ci->i_truncate_pending == 0)
1454+ wake = 1;
1455+ spin_unlock(&inode->i_lock);
1456+
1457+ if (wrbuffer_refs == 0)
1458+ ceph_check_caps(ci, CHECK_CAPS_AUTHONLY, NULL);
1459+ if (wake)
1460+ wake_up(&ci->i_cap_wq);
1461+}
1462+
1463+
1464+/*
1465+ * symlinks
1466+ */
1467+static void *ceph_sym_follow_link(struct dentry *dentry, struct nameidata *nd)
1468+{
1469+ struct ceph_inode_info *ci = ceph_inode(dentry->d_inode);
1470+ nd_set_link(nd, ci->i_symlink);
1471+ return NULL;
1472+}
1473+
1474+static const struct inode_operations ceph_symlink_iops = {
1475+ .readlink = generic_readlink,
1476+ .follow_link = ceph_sym_follow_link,
1477+};
1478+
1479+/*
1480+ * setattr
1481+ */
1482+int ceph_setattr(struct dentry *dentry, struct iattr *attr)
1483+{
1484+ struct inode *inode = dentry->d_inode;
1485+ struct ceph_inode_info *ci = ceph_inode(inode);
1486+ struct inode *parent_inode = dentry->d_parent->d_inode;
1487+ const unsigned int ia_valid = attr->ia_valid;
1488+ struct ceph_mds_request *req;
1489+ struct ceph_mds_client *mdsc = &ceph_client(dentry->d_sb)->mdsc;
1490+ int issued;
1491+ int release = 0, dirtied = 0;
1492+ int mask = 0;
1493+ int err = 0;
1494+
1495+ if (ceph_snap(inode) != CEPH_NOSNAP)
1496+ return -EROFS;
1497+
1498+ __ceph_do_pending_vmtruncate(inode);
1499+
1500+ err = inode_change_ok(inode, attr);
1501+ if (err != 0)
1502+ return err;
1503+
1504+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_SETATTR,
1505+ USE_AUTH_MDS);
1506+ if (IS_ERR(req))
1507+ return PTR_ERR(req);
1508+
1509+ spin_lock(&inode->i_lock);
1510+ issued = __ceph_caps_issued(ci, NULL);
1511+ dout("setattr %p issued %s\n", inode, ceph_cap_string(issued));
1512+
1513+ if (ia_valid & ATTR_UID) {
1514+ dout("setattr %p uid %d -> %d\n", inode,
1515+ inode->i_uid, attr->ia_uid);
1516+ if (issued & CEPH_CAP_AUTH_EXCL) {
1517+ inode->i_uid = attr->ia_uid;
1518+ dirtied |= CEPH_CAP_AUTH_EXCL;
1519+ } else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 ||
1520+ attr->ia_uid != inode->i_uid) {
1521+ req->r_args.setattr.uid = cpu_to_le32(attr->ia_uid);
1522+ mask |= CEPH_SETATTR_UID;
1523+ release |= CEPH_CAP_AUTH_SHARED;
1524+ }
1525+ }
1526+ if (ia_valid & ATTR_GID) {
1527+ dout("setattr %p gid %d -> %d\n", inode,
1528+ inode->i_gid, attr->ia_gid);
1529+ if (issued & CEPH_CAP_AUTH_EXCL) {
1530+ inode->i_gid = attr->ia_gid;
1531+ dirtied |= CEPH_CAP_AUTH_EXCL;
1532+ } else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 ||
1533+ attr->ia_gid != inode->i_gid) {
1534+ req->r_args.setattr.gid = cpu_to_le32(attr->ia_gid);
1535+ mask |= CEPH_SETATTR_GID;
1536+ release |= CEPH_CAP_AUTH_SHARED;
1537+ }
1538+ }
1539+ if (ia_valid & ATTR_MODE) {
1540+ dout("setattr %p mode 0%o -> 0%o\n", inode, inode->i_mode,
1541+ attr->ia_mode);
1542+ if (issued & CEPH_CAP_AUTH_EXCL) {
1543+ inode->i_mode = attr->ia_mode;
1544+ dirtied |= CEPH_CAP_AUTH_EXCL;
1545+ } else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 ||
1546+ attr->ia_mode != inode->i_mode) {
1547+ req->r_args.setattr.mode = cpu_to_le32(attr->ia_mode);
1548+ mask |= CEPH_SETATTR_MODE;
1549+ release |= CEPH_CAP_AUTH_SHARED;
1550+ }
1551+ }
1552+
1553+ if (ia_valid & ATTR_ATIME) {
1554+ dout("setattr %p atime %ld.%ld -> %ld.%ld\n", inode,
1555+ inode->i_atime.tv_sec, inode->i_atime.tv_nsec,
1556+ attr->ia_atime.tv_sec, attr->ia_atime.tv_nsec);
1557+ if (issued & CEPH_CAP_FILE_EXCL) {
1558+ ci->i_time_warp_seq++;
1559+ inode->i_atime = attr->ia_atime;
1560+ dirtied |= CEPH_CAP_FILE_EXCL;
1561+ } else if ((issued & CEPH_CAP_FILE_WR) &&
1562+ timespec_compare(&inode->i_atime,
1563+ &attr->ia_atime) < 0) {
1564+ inode->i_atime = attr->ia_atime;
1565+ dirtied |= CEPH_CAP_FILE_WR;
1566+ } else if ((issued & CEPH_CAP_FILE_SHARED) == 0 ||
1567+ !timespec_equal(&inode->i_atime, &attr->ia_atime)) {
1568+ ceph_encode_timespec(&req->r_args.setattr.atime,
1569+ &attr->ia_atime);
1570+ mask |= CEPH_SETATTR_ATIME;
1571+ release |= CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_RD |
1572+ CEPH_CAP_FILE_WR;
1573+ }
1574+ }
1575+ if (ia_valid & ATTR_MTIME) {
1576+ dout("setattr %p mtime %ld.%ld -> %ld.%ld\n", inode,
1577+ inode->i_mtime.tv_sec, inode->i_mtime.tv_nsec,
1578+ attr->ia_mtime.tv_sec, attr->ia_mtime.tv_nsec);
1579+ if (issued & CEPH_CAP_FILE_EXCL) {
1580+ ci->i_time_warp_seq++;
1581+ inode->i_mtime = attr->ia_mtime;
1582+ dirtied |= CEPH_CAP_FILE_EXCL;
1583+ } else if ((issued & CEPH_CAP_FILE_WR) &&
1584+ timespec_compare(&inode->i_mtime,
1585+ &attr->ia_mtime) < 0) {
1586+ inode->i_mtime = attr->ia_mtime;
1587+ dirtied |= CEPH_CAP_FILE_WR;
1588+ } else if ((issued & CEPH_CAP_FILE_SHARED) == 0 ||
1589+ !timespec_equal(&inode->i_mtime, &attr->ia_mtime)) {
1590+ ceph_encode_timespec(&req->r_args.setattr.mtime,
1591+ &attr->ia_mtime);
1592+ mask |= CEPH_SETATTR_MTIME;
1593+ release |= CEPH_CAP_FILE_SHARED | CEPH_CAP_FILE_RD |
1594+ CEPH_CAP_FILE_WR;
1595+ }
1596+ }
1597+ if (ia_valid & ATTR_SIZE) {
1598+ dout("setattr %p size %lld -> %lld\n", inode,
1599+ inode->i_size, attr->ia_size);
1600+ if (attr->ia_size > inode->i_sb->s_maxbytes) {
1601+ err = -EINVAL;
1602+ goto out;
1603+ }
1604+ if ((issued & CEPH_CAP_FILE_EXCL) &&
1605+ attr->ia_size > inode->i_size) {
1606+ inode->i_size = attr->ia_size;
1607+ inode->i_blocks =
1608+ (attr->ia_size + (1 << 9) - 1) >> 9;
1609+ inode->i_ctime = attr->ia_ctime;
1610+ ci->i_reported_size = attr->ia_size;
1611+ dirtied |= CEPH_CAP_FILE_EXCL;
1612+ } else if ((issued & CEPH_CAP_FILE_SHARED) == 0 ||
1613+ attr->ia_size != inode->i_size) {
1614+ req->r_args.setattr.size = cpu_to_le64(attr->ia_size);
1615+ req->r_args.setattr.old_size =
1616+ cpu_to_le64(inode->i_size);
1617+ mask |= CEPH_SETATTR_SIZE;
1618+ release |= CEPH_CAP_FILE_SHARED | CEPH_CAP_FILE_RD |
1619+ CEPH_CAP_FILE_WR;
1620+ }
1621+ }
1622+
1623+ /* these do nothing */
1624+ if (ia_valid & ATTR_CTIME) {
1625+ bool only = (ia_valid & (ATTR_SIZE|ATTR_MTIME|ATTR_ATIME|
1626+ ATTR_MODE|ATTR_UID|ATTR_GID)) == 0;
1627+ dout("setattr %p ctime %ld.%ld -> %ld.%ld (%s)\n", inode,
1628+ inode->i_ctime.tv_sec, inode->i_ctime.tv_nsec,
1629+ attr->ia_ctime.tv_sec, attr->ia_ctime.tv_nsec,
1630+ only ? "ctime only" : "ignored");
1631+ inode->i_ctime = attr->ia_ctime;
1632+ if (only) {
1633+ /*
1634+ * if kernel wants to dirty ctime but nothing else,
1635+ * we need to choose a cap to dirty under, or do
1636+ * a almost-no-op setattr
1637+ */
1638+ if (issued & CEPH_CAP_AUTH_EXCL)
1639+ dirtied |= CEPH_CAP_AUTH_EXCL;
1640+ else if (issued & CEPH_CAP_FILE_EXCL)
1641+ dirtied |= CEPH_CAP_FILE_EXCL;
1642+ else if (issued & CEPH_CAP_XATTR_EXCL)
1643+ dirtied |= CEPH_CAP_XATTR_EXCL;
1644+ else
1645+ mask |= CEPH_SETATTR_CTIME;
1646+ }
1647+ }
1648+ if (ia_valid & ATTR_FILE)
1649+ dout("setattr %p ATTR_FILE ... hrm!\n", inode);
1650+
1651+ if (dirtied) {
1652+ __ceph_mark_dirty_caps(ci, dirtied);
1653+ inode->i_ctime = CURRENT_TIME;
1654+ }
1655+
1656+ release &= issued;
1657+ spin_unlock(&inode->i_lock);
1658+
1659+ if (mask) {
1660+ req->r_inode = igrab(inode);
1661+ req->r_inode_drop = release;
1662+ req->r_args.setattr.mask = cpu_to_le32(mask);
1663+ req->r_num_caps = 1;
1664+ err = ceph_mdsc_do_request(mdsc, parent_inode, req);
1665+ }
1666+ dout("setattr %p result=%d (%s locally, %d remote)\n", inode, err,
1667+ ceph_cap_string(dirtied), mask);
1668+
1669+ ceph_mdsc_put_request(req);
1670+ __ceph_do_pending_vmtruncate(inode);
1671+ return err;
1672+out:
1673+ spin_unlock(&inode->i_lock);
1674+ ceph_mdsc_put_request(req);
1675+ return err;
1676+}
1677+
1678+/*
1679+ * Verify that we have a lease on the given mask. If not,
1680+ * do a getattr against an mds.
1681+ */
1682+int ceph_do_getattr(struct inode *inode, int mask)
1683+{
1684+ struct ceph_client *client = ceph_sb_to_client(inode->i_sb);
1685+ struct ceph_mds_client *mdsc = &client->mdsc;
1686+ struct ceph_mds_request *req;
1687+ int err;
1688+
1689+ if (ceph_snap(inode) == CEPH_SNAPDIR) {
1690+ dout("do_getattr inode %p SNAPDIR\n", inode);
1691+ return 0;
1692+ }
1693+
1694+ dout("do_getattr inode %p mask %s\n", inode, ceph_cap_string(mask));
1695+ if (ceph_caps_issued_mask(ceph_inode(inode), mask, 1))
1696+ return 0;
1697+
1698+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_GETATTR, USE_ANY_MDS);
1699+ if (IS_ERR(req))
1700+ return PTR_ERR(req);
1701+ req->r_inode = igrab(inode);
1702+ req->r_num_caps = 1;
1703+ req->r_args.getattr.mask = cpu_to_le32(mask);
1704+ err = ceph_mdsc_do_request(mdsc, NULL, req);
1705+ ceph_mdsc_put_request(req);
1706+ dout("do_getattr result=%d\n", err);
1707+ return err;
1708+}
1709+
1710+
1711+/*
1712+ * Check inode permissions. We verify we have a valid value for
1713+ * the AUTH cap, then call the generic handler.
1714+ */
1715+int ceph_permission(struct inode *inode, int mask)
1716+{
1717+ int err = ceph_do_getattr(inode, CEPH_CAP_AUTH_SHARED);
1718+
1719+ if (!err)
1720+ err = generic_permission(inode, mask, NULL);
1721+ return err;
1722+}
1723+
1724+/*
1725+ * Get all attributes. Hopefully somedata we'll have a statlite()
1726+ * and can limit the fields we require to be accurate.
1727+ */
1728+int ceph_getattr(struct vfsmount *mnt, struct dentry *dentry,
1729+ struct kstat *stat)
1730+{
1731+ struct inode *inode = dentry->d_inode;
1732+ struct ceph_inode_info *ci = ceph_inode(inode);
1733+ int err;
1734+
1735+ err = ceph_do_getattr(inode, CEPH_STAT_CAP_INODE_ALL);
1736+ if (!err) {
1737+ generic_fillattr(inode, stat);
1738+ stat->ino = inode->i_ino;
1739+ if (ceph_snap(inode) != CEPH_NOSNAP)
1740+ stat->dev = ceph_snap(inode);
1741+ else
1742+ stat->dev = 0;
1743+ if (S_ISDIR(inode->i_mode)) {
1744+ stat->size = ci->i_rbytes;
1745+ stat->blocks = 0;
1746+ stat->blksize = 65536;
1747+ }
1748+ }
1749+ return err;
1750+}
fs/ceph/ioctl.c+160−0
@@ -0,0 +1,160 @@
1+#include <linux/in.h>
2+
3+#include "ioctl.h"
4+#include "super.h"
5+#include "ceph_debug.h"
6+
7+
8+/*
9+ * ioctls
10+ */
11+
12+/*
13+ * get and set the file layout
14+ */
15+static long ceph_ioctl_get_layout(struct file *file, void __user *arg)
16+{
17+ struct ceph_inode_info *ci = ceph_inode(file->f_dentry->d_inode);
18+ struct ceph_ioctl_layout l;
19+ int err;
20+
21+ err = ceph_do_getattr(file->f_dentry->d_inode, CEPH_STAT_CAP_LAYOUT);
22+ if (!err) {
23+ l.stripe_unit = ceph_file_layout_su(ci->i_layout);
24+ l.stripe_count = ceph_file_layout_stripe_count(ci->i_layout);
25+ l.object_size = ceph_file_layout_object_size(ci->i_layout);
26+ l.data_pool = le32_to_cpu(ci->i_layout.fl_pg_pool);
27+ l.preferred_osd =
28+ (s32)le32_to_cpu(ci->i_layout.fl_pg_preferred);
29+ if (copy_to_user(arg, &l, sizeof(l)))
30+ return -EFAULT;
31+ }
32+
33+ return err;
34+}
35+
36+static long ceph_ioctl_set_layout(struct file *file, void __user *arg)
37+{
38+ struct inode *inode = file->f_dentry->d_inode;
39+ struct inode *parent_inode = file->f_dentry->d_parent->d_inode;
40+ struct ceph_mds_client *mdsc = &ceph_sb_to_client(inode->i_sb)->mdsc;
41+ struct ceph_mds_request *req;
42+ struct ceph_ioctl_layout l;
43+ int err, i;
44+
45+ /* copy and validate */
46+ if (copy_from_user(&l, arg, sizeof(l)))
47+ return -EFAULT;
48+
49+ if ((l.object_size & ~PAGE_MASK) ||
50+ (l.stripe_unit & ~PAGE_MASK) ||
51+ !l.stripe_unit ||
52+ (l.object_size &&
53+ (unsigned)l.object_size % (unsigned)l.stripe_unit))
54+ return -EINVAL;
55+
56+ /* make sure it's a valid data pool */
57+ if (l.data_pool > 0) {
58+ mutex_lock(&mdsc->mutex);
59+ err = -EINVAL;
60+ for (i = 0; i < mdsc->mdsmap->m_num_data_pg_pools; i++)
61+ if (mdsc->mdsmap->m_data_pg_pools[i] == l.data_pool) {
62+ err = 0;
63+ break;
64+ }
65+ mutex_unlock(&mdsc->mutex);
66+ if (err)
67+ return err;
68+ }
69+
70+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_SETLAYOUT,
71+ USE_AUTH_MDS);
72+ if (IS_ERR(req))
73+ return PTR_ERR(req);
74+ req->r_inode = igrab(inode);
75+ req->r_inode_drop = CEPH_CAP_FILE_SHARED | CEPH_CAP_FILE_EXCL;
76+
77+ req->r_args.setlayout.layout.fl_stripe_unit =
78+ cpu_to_le32(l.stripe_unit);
79+ req->r_args.setlayout.layout.fl_stripe_count =
80+ cpu_to_le32(l.stripe_count);
81+ req->r_args.setlayout.layout.fl_object_size =
82+ cpu_to_le32(l.object_size);
83+ req->r_args.setlayout.layout.fl_pg_pool = cpu_to_le32(l.data_pool);
84+ req->r_args.setlayout.layout.fl_pg_preferred =
85+ cpu_to_le32(l.preferred_osd);
86+
87+ err = ceph_mdsc_do_request(mdsc, parent_inode, req);
88+ ceph_mdsc_put_request(req);
89+ return err;
90+}
91+
92+/*
93+ * Return object name, size/offset information, and location (OSD
94+ * number, network address) for a given file offset.
95+ */
96+static long ceph_ioctl_get_dataloc(struct file *file, void __user *arg)
97+{
98+ struct ceph_ioctl_dataloc dl;
99+ struct inode *inode = file->f_dentry->d_inode;
100+ struct ceph_inode_info *ci = ceph_inode(inode);
101+ struct ceph_osd_client *osdc = &ceph_client(inode->i_sb)->osdc;
102+ u64 len = 1, olen;
103+ u64 tmp;
104+ struct ceph_object_layout ol;
105+ struct ceph_pg pgid;
106+
107+ /* copy and validate */
108+ if (copy_from_user(&dl, arg, sizeof(dl)))
109+ return -EFAULT;
110+
111+ down_read(&osdc->map_sem);
112+ ceph_calc_file_object_mapping(&ci->i_layout, dl.file_offset, &len,
113+ &dl.object_no, &dl.object_offset, &olen);
114+ dl.file_offset -= dl.object_offset;
115+ dl.object_size = ceph_file_layout_object_size(ci->i_layout);
116+ dl.block_size = ceph_file_layout_su(ci->i_layout);
117+
118+ /* block_offset = object_offset % block_size */
119+ tmp = dl.object_offset;
120+ dl.block_offset = do_div(tmp, dl.block_size);
121+
122+ snprintf(dl.object_name, sizeof(dl.object_name), "%llx.%08llx",
123+ ceph_ino(inode), dl.object_no);
124+ ceph_calc_object_layout(&ol, dl.object_name, &ci->i_layout,
125+ osdc->osdmap);
126+
127+ pgid = ol.ol_pgid;
128+ dl.osd = ceph_calc_pg_primary(osdc->osdmap, pgid);
129+ if (dl.osd >= 0) {
130+ struct ceph_entity_addr *a =
131+ ceph_osd_addr(osdc->osdmap, dl.osd);
132+ if (a)
133+ memcpy(&dl.osd_addr, &a->in_addr, sizeof(dl.osd_addr));
134+ } else {
135+ memset(&dl.osd_addr, 0, sizeof(dl.osd_addr));
136+ }
137+ up_read(&osdc->map_sem);
138+
139+ /* send result back to user */
140+ if (copy_to_user(arg, &dl, sizeof(dl)))
141+ return -EFAULT;
142+
143+ return 0;
144+}
145+
146+long ceph_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
147+{
148+ dout("ioctl file %p cmd %u arg %lu\n", file, cmd, arg);
149+ switch (cmd) {
150+ case CEPH_IOC_GET_LAYOUT:
151+ return ceph_ioctl_get_layout(file, (void __user *)arg);
152+
153+ case CEPH_IOC_SET_LAYOUT:
154+ return ceph_ioctl_set_layout(file, (void __user *)arg);
155+
156+ case CEPH_IOC_GET_DATALOC:
157+ return ceph_ioctl_get_dataloc(file, (void __user *)arg);
158+ }
159+ return -ENOTTY;
160+}
fs/ceph/ioctl.h+40−0
@@ -0,0 +1,40 @@
1+#ifndef FS_CEPH_IOCTL_H
2+#define FS_CEPH_IOCTL_H
3+
4+#include <linux/ioctl.h>
5+#include <linux/types.h>
6+
7+#define CEPH_IOCTL_MAGIC 0x97
8+
9+/* just use u64 to align sanely on all archs */
10+struct ceph_ioctl_layout {
11+ __u64 stripe_unit, stripe_count, object_size;
12+ __u64 data_pool;
13+ __s64 preferred_osd;
14+};
15+
16+#define CEPH_IOC_GET_LAYOUT _IOR(CEPH_IOCTL_MAGIC, 1, \
17+ struct ceph_ioctl_layout)
18+#define CEPH_IOC_SET_LAYOUT _IOW(CEPH_IOCTL_MAGIC, 2, \
19+ struct ceph_ioctl_layout)
20+
21+/*
22+ * Extract identity, address of the OSD and object storing a given
23+ * file offset.
24+ */
25+struct ceph_ioctl_dataloc {
26+ __u64 file_offset; /* in+out: file offset */
27+ __u64 object_offset; /* out: offset in object */
28+ __u64 object_no; /* out: object # */
29+ __u64 object_size; /* out: object size */
30+ char object_name[64]; /* out: object name */
31+ __u64 block_offset; /* out: offset in block */
32+ __u64 block_size; /* out: block length */
33+ __s64 osd; /* out: osd # */
34+ struct sockaddr_storage osd_addr; /* out: osd address */
35+};
36+
37+#define CEPH_IOC_GET_DATALOC _IOWR(CEPH_IOCTL_MAGIC, 3, \
38+ struct ceph_ioctl_dataloc)
39+
40+#endif
fs/ceph/mds_client.c+3021−0
@@ -0,0 +1,3021 @@
1+#include "ceph_debug.h"
2+
3+#include <linux/wait.h>
4+#include <linux/sched.h>
5+
6+#include "mds_client.h"
7+#include "mon_client.h"
8+#include "super.h"
9+#include "messenger.h"
10+#include "decode.h"
11+#include "auth.h"
12+#include "pagelist.h"
13+
14+/*
15+ * A cluster of MDS (metadata server) daemons is responsible for
16+ * managing the file system namespace (the directory hierarchy and
17+ * inodes) and for coordinating shared access to storage. Metadata is
18+ * partitioning hierarchically across a number of servers, and that
19+ * partition varies over time as the cluster adjusts the distribution
20+ * in order to balance load.
21+ *
22+ * The MDS client is primarily responsible to managing synchronous
23+ * metadata requests for operations like open, unlink, and so forth.
24+ * If there is a MDS failure, we find out about it when we (possibly
25+ * request and) receive a new MDS map, and can resubmit affected
26+ * requests.
27+ *
28+ * For the most part, though, we take advantage of a lossless
29+ * communications channel to the MDS, and do not need to worry about
30+ * timing out or resubmitting requests.
31+ *
32+ * We maintain a stateful "session" with each MDS we interact with.
33+ * Within each session, we sent periodic heartbeat messages to ensure
34+ * any capabilities or leases we have been issues remain valid. If
35+ * the session times out and goes stale, our leases and capabilities
36+ * are no longer valid.
37+ */
38+
39+static void __wake_requests(struct ceph_mds_client *mdsc,
40+ struct list_head *head);
41+
42+const static struct ceph_connection_operations mds_con_ops;
43+
44+
45+/*
46+ * mds reply parsing
47+ */
48+
49+/*
50+ * parse individual inode info
51+ */
52+static int parse_reply_info_in(void **p, void *end,
53+ struct ceph_mds_reply_info_in *info)
54+{
55+ int err = -EIO;
56+
57+ info->in = *p;
58+ *p += sizeof(struct ceph_mds_reply_inode) +
59+ sizeof(*info->in->fragtree.splits) *
60+ le32_to_cpu(info->in->fragtree.nsplits);
61+
62+ ceph_decode_32_safe(p, end, info->symlink_len, bad);
63+ ceph_decode_need(p, end, info->symlink_len, bad);
64+ info->symlink = *p;
65+ *p += info->symlink_len;
66+
67+ ceph_decode_32_safe(p, end, info->xattr_len, bad);
68+ ceph_decode_need(p, end, info->xattr_len, bad);
69+ info->xattr_data = *p;
70+ *p += info->xattr_len;
71+ return 0;
72+bad:
73+ return err;
74+}
75+
76+/*
77+ * parse a normal reply, which may contain a (dir+)dentry and/or a
78+ * target inode.
79+ */
80+static int parse_reply_info_trace(void **p, void *end,
81+ struct ceph_mds_reply_info_parsed *info)
82+{
83+ int err;
84+
85+ if (info->head->is_dentry) {
86+ err = parse_reply_info_in(p, end, &info->diri);
87+ if (err < 0)
88+ goto out_bad;
89+
90+ if (unlikely(*p + sizeof(*info->dirfrag) > end))
91+ goto bad;
92+ info->dirfrag = *p;
93+ *p += sizeof(*info->dirfrag) +
94+ sizeof(u32)*le32_to_cpu(info->dirfrag->ndist);
95+ if (unlikely(*p > end))
96+ goto bad;
97+
98+ ceph_decode_32_safe(p, end, info->dname_len, bad);
99+ ceph_decode_need(p, end, info->dname_len, bad);
100+ info->dname = *p;
101+ *p += info->dname_len;
102+ info->dlease = *p;
103+ *p += sizeof(*info->dlease);
104+ }
105+
106+ if (info->head->is_target) {
107+ err = parse_reply_info_in(p, end, &info->targeti);
108+ if (err < 0)
109+ goto out_bad;
110+ }
111+
112+ if (unlikely(*p != end))
113+ goto bad;
114+ return 0;
115+
116+bad:
117+ err = -EIO;
118+out_bad:
119+ pr_err("problem parsing mds trace %d\n", err);
120+ return err;
121+}
122+
123+/*
124+ * parse readdir results
125+ */
126+static int parse_reply_info_dir(void **p, void *end,
127+ struct ceph_mds_reply_info_parsed *info)
128+{
129+ u32 num, i = 0;
130+ int err;
131+
132+ info->dir_dir = *p;
133+ if (*p + sizeof(*info->dir_dir) > end)
134+ goto bad;
135+ *p += sizeof(*info->dir_dir) +
136+ sizeof(u32)*le32_to_cpu(info->dir_dir->ndist);
137+ if (*p > end)
138+ goto bad;
139+
140+ ceph_decode_need(p, end, sizeof(num) + 2, bad);
141+ num = ceph_decode_32(p);
142+ info->dir_end = ceph_decode_8(p);
143+ info->dir_complete = ceph_decode_8(p);
144+ if (num == 0)
145+ goto done;
146+
147+ /* alloc large array */
148+ info->dir_nr = num;
149+ info->dir_in = kcalloc(num, sizeof(*info->dir_in) +
150+ sizeof(*info->dir_dname) +
151+ sizeof(*info->dir_dname_len) +
152+ sizeof(*info->dir_dlease),
153+ GFP_NOFS);
154+ if (info->dir_in == NULL) {
155+ err = -ENOMEM;
156+ goto out_bad;
157+ }
158+ info->dir_dname = (void *)(info->dir_in + num);
159+ info->dir_dname_len = (void *)(info->dir_dname + num);
160+ info->dir_dlease = (void *)(info->dir_dname_len + num);
161+
162+ while (num) {
163+ /* dentry */
164+ ceph_decode_need(p, end, sizeof(u32)*2, bad);
165+ info->dir_dname_len[i] = ceph_decode_32(p);
166+ ceph_decode_need(p, end, info->dir_dname_len[i], bad);
167+ info->dir_dname[i] = *p;
168+ *p += info->dir_dname_len[i];
169+ dout("parsed dir dname '%.*s'\n", info->dir_dname_len[i],
170+ info->dir_dname[i]);
171+ info->dir_dlease[i] = *p;
172+ *p += sizeof(struct ceph_mds_reply_lease);
173+
174+ /* inode */
175+ err = parse_reply_info_in(p, end, &info->dir_in[i]);
176+ if (err < 0)
177+ goto out_bad;
178+ i++;
179+ num--;
180+ }
181+
182+done:
183+ if (*p != end)
184+ goto bad;
185+ return 0;
186+
187+bad:
188+ err = -EIO;
189+out_bad:
190+ pr_err("problem parsing dir contents %d\n", err);
191+ return err;
192+}
193+
194+/*
195+ * parse entire mds reply
196+ */
197+static int parse_reply_info(struct ceph_msg *msg,
198+ struct ceph_mds_reply_info_parsed *info)
199+{
200+ void *p, *end;
201+ u32 len;
202+ int err;
203+
204+ info->head = msg->front.iov_base;
205+ p = msg->front.iov_base + sizeof(struct ceph_mds_reply_head);
206+ end = p + msg->front.iov_len - sizeof(struct ceph_mds_reply_head);
207+
208+ /* trace */
209+ ceph_decode_32_safe(&p, end, len, bad);
210+ if (len > 0) {
211+ err = parse_reply_info_trace(&p, p+len, info);
212+ if (err < 0)
213+ goto out_bad;
214+ }
215+
216+ /* dir content */
217+ ceph_decode_32_safe(&p, end, len, bad);
218+ if (len > 0) {
219+ err = parse_reply_info_dir(&p, p+len, info);
220+ if (err < 0)
221+ goto out_bad;
222+ }
223+
224+ /* snap blob */
225+ ceph_decode_32_safe(&p, end, len, bad);
226+ info->snapblob_len = len;
227+ info->snapblob = p;
228+ p += len;
229+
230+ if (p != end)
231+ goto bad;
232+ return 0;
233+
234+bad:
235+ err = -EIO;
236+out_bad:
237+ pr_err("mds parse_reply err %d\n", err);
238+ return err;
239+}
240+
241+static void destroy_reply_info(struct ceph_mds_reply_info_parsed *info)
242+{
243+ kfree(info->dir_in);
244+}
245+
246+
247+/*
248+ * sessions
249+ */
250+static const char *session_state_name(int s)
251+{
252+ switch (s) {
253+ case CEPH_MDS_SESSION_NEW: return "new";
254+ case CEPH_MDS_SESSION_OPENING: return "opening";
255+ case CEPH_MDS_SESSION_OPEN: return "open";
256+ case CEPH_MDS_SESSION_HUNG: return "hung";
257+ case CEPH_MDS_SESSION_CLOSING: return "closing";
258+ case CEPH_MDS_SESSION_RESTARTING: return "restarting";
259+ case CEPH_MDS_SESSION_RECONNECTING: return "reconnecting";
260+ default: return "???";
261+ }
262+}
263+
264+static struct ceph_mds_session *get_session(struct ceph_mds_session *s)
265+{
266+ if (atomic_inc_not_zero(&s->s_ref)) {
267+ dout("mdsc get_session %p %d -> %d\n", s,
268+ atomic_read(&s->s_ref)-1, atomic_read(&s->s_ref));
269+ return s;
270+ } else {
271+ dout("mdsc get_session %p 0 -- FAIL", s);
272+ return NULL;
273+ }
274+}
275+
276+void ceph_put_mds_session(struct ceph_mds_session *s)
277+{
278+ dout("mdsc put_session %p %d -> %d\n", s,
279+ atomic_read(&s->s_ref), atomic_read(&s->s_ref)-1);
280+ if (atomic_dec_and_test(&s->s_ref)) {
281+ if (s->s_authorizer)
282+ s->s_mdsc->client->monc.auth->ops->destroy_authorizer(
283+ s->s_mdsc->client->monc.auth, s->s_authorizer);
284+ kfree(s);
285+ }
286+}
287+
288+/*
289+ * called under mdsc->mutex
290+ */
291+struct ceph_mds_session *__ceph_lookup_mds_session(struct ceph_mds_client *mdsc,
292+ int mds)
293+{
294+ struct ceph_mds_session *session;
295+
296+ if (mds >= mdsc->max_sessions || mdsc->sessions[mds] == NULL)
297+ return NULL;
298+ session = mdsc->sessions[mds];
299+ dout("lookup_mds_session %p %d\n", session,
300+ atomic_read(&session->s_ref));
301+ get_session(session);
302+ return session;
303+}
304+
305+static bool __have_session(struct ceph_mds_client *mdsc, int mds)
306+{
307+ if (mds >= mdsc->max_sessions)
308+ return false;
309+ return mdsc->sessions[mds];
310+}
311+
312+static int __verify_registered_session(struct ceph_mds_client *mdsc,
313+ struct ceph_mds_session *s)
314+{
315+ if (s->s_mds >= mdsc->max_sessions ||
316+ mdsc->sessions[s->s_mds] != s)
317+ return -ENOENT;
318+ return 0;
319+}
320+
321+/*
322+ * create+register a new session for given mds.
323+ * called under mdsc->mutex.
324+ */
325+static struct ceph_mds_session *register_session(struct ceph_mds_client *mdsc,
326+ int mds)
327+{
328+ struct ceph_mds_session *s;
329+
330+ s = kzalloc(sizeof(*s), GFP_NOFS);
331+ s->s_mdsc = mdsc;
332+ s->s_mds = mds;
333+ s->s_state = CEPH_MDS_SESSION_NEW;
334+ s->s_ttl = 0;
335+ s->s_seq = 0;
336+ mutex_init(&s->s_mutex);
337+
338+ ceph_con_init(mdsc->client->msgr, &s->s_con);
339+ s->s_con.private = s;
340+ s->s_con.ops = &mds_con_ops;
341+ s->s_con.peer_name.type = CEPH_ENTITY_TYPE_MDS;
342+ s->s_con.peer_name.num = cpu_to_le64(mds);
343+
344+ spin_lock_init(&s->s_cap_lock);
345+ s->s_cap_gen = 0;
346+ s->s_cap_ttl = 0;
347+ s->s_renew_requested = 0;
348+ s->s_renew_seq = 0;
349+ INIT_LIST_HEAD(&s->s_caps);
350+ s->s_nr_caps = 0;
351+ s->s_trim_caps = 0;
352+ atomic_set(&s->s_ref, 1);
353+ INIT_LIST_HEAD(&s->s_waiting);
354+ INIT_LIST_HEAD(&s->s_unsafe);
355+ s->s_num_cap_releases = 0;
356+ s->s_cap_iterator = NULL;
357+ INIT_LIST_HEAD(&s->s_cap_releases);
358+ INIT_LIST_HEAD(&s->s_cap_releases_done);
359+ INIT_LIST_HEAD(&s->s_cap_flushing);
360+ INIT_LIST_HEAD(&s->s_cap_snaps_flushing);
361+
362+ dout("register_session mds%d\n", mds);
363+ if (mds >= mdsc->max_sessions) {
364+ int newmax = 1 << get_count_order(mds+1);
365+ struct ceph_mds_session **sa;
366+
367+ dout("register_session realloc to %d\n", newmax);
368+ sa = kcalloc(newmax, sizeof(void *), GFP_NOFS);
369+ if (sa == NULL)
370+ goto fail_realloc;
371+ if (mdsc->sessions) {
372+ memcpy(sa, mdsc->sessions,
373+ mdsc->max_sessions * sizeof(void *));
374+ kfree(mdsc->sessions);
375+ }
376+ mdsc->sessions = sa;
377+ mdsc->max_sessions = newmax;
378+ }
379+ mdsc->sessions[mds] = s;
380+ atomic_inc(&s->s_ref); /* one ref to sessions[], one to caller */
381+
382+ ceph_con_open(&s->s_con, ceph_mdsmap_get_addr(mdsc->mdsmap, mds));
383+
384+ return s;
385+
386+fail_realloc:
387+ kfree(s);
388+ return ERR_PTR(-ENOMEM);
389+}
390+
391+/*
392+ * called under mdsc->mutex
393+ */
394+static void __unregister_session(struct ceph_mds_client *mdsc,
395+ struct ceph_mds_session *s)
396+{
397+ dout("__unregister_session mds%d %p\n", s->s_mds, s);
398+ BUG_ON(mdsc->sessions[s->s_mds] != s);
399+ mdsc->sessions[s->s_mds] = NULL;
400+ ceph_con_close(&s->s_con);
401+ ceph_put_mds_session(s);
402+}
403+
404+/*
405+ * drop session refs in request.
406+ *
407+ * should be last request ref, or hold mdsc->mutex
408+ */
409+static void put_request_session(struct ceph_mds_request *req)
410+{
411+ if (req->r_session) {
412+ ceph_put_mds_session(req->r_session);
413+ req->r_session = NULL;
414+ }
415+}
416+
417+void ceph_mdsc_release_request(struct kref *kref)
418+{
419+ struct ceph_mds_request *req = container_of(kref,
420+ struct ceph_mds_request,
421+ r_kref);
422+ if (req->r_request)
423+ ceph_msg_put(req->r_request);
424+ if (req->r_reply) {
425+ ceph_msg_put(req->r_reply);
426+ destroy_reply_info(&req->r_reply_info);
427+ }
428+ if (req->r_inode) {
429+ ceph_put_cap_refs(ceph_inode(req->r_inode),
430+ CEPH_CAP_PIN);
431+ iput(req->r_inode);
432+ }
433+ if (req->r_locked_dir)
434+ ceph_put_cap_refs(ceph_inode(req->r_locked_dir),
435+ CEPH_CAP_PIN);
436+ if (req->r_target_inode)
437+ iput(req->r_target_inode);
438+ if (req->r_dentry)
439+ dput(req->r_dentry);
440+ if (req->r_old_dentry) {
441+ ceph_put_cap_refs(
442+ ceph_inode(req->r_old_dentry->d_parent->d_inode),
443+ CEPH_CAP_PIN);
444+ dput(req->r_old_dentry);
445+ }
446+ kfree(req->r_path1);
447+ kfree(req->r_path2);
448+ put_request_session(req);
449+ ceph_unreserve_caps(&req->r_caps_reservation);
450+ kfree(req);
451+}
452+
453+/*
454+ * lookup session, bump ref if found.
455+ *
456+ * called under mdsc->mutex.
457+ */
458+static struct ceph_mds_request *__lookup_request(struct ceph_mds_client *mdsc,
459+ u64 tid)
460+{
461+ struct ceph_mds_request *req;
462+ struct rb_node *n = mdsc->request_tree.rb_node;
463+
464+ while (n) {
465+ req = rb_entry(n, struct ceph_mds_request, r_node);
466+ if (tid < req->r_tid)
467+ n = n->rb_left;
468+ else if (tid > req->r_tid)
469+ n = n->rb_right;
470+ else {
471+ ceph_mdsc_get_request(req);
472+ return req;
473+ }
474+ }
475+ return NULL;
476+}
477+
478+static void __insert_request(struct ceph_mds_client *mdsc,
479+ struct ceph_mds_request *new)
480+{
481+ struct rb_node **p = &mdsc->request_tree.rb_node;
482+ struct rb_node *parent = NULL;
483+ struct ceph_mds_request *req = NULL;
484+
485+ while (*p) {
486+ parent = *p;
487+ req = rb_entry(parent, struct ceph_mds_request, r_node);
488+ if (new->r_tid < req->r_tid)
489+ p = &(*p)->rb_left;
490+ else if (new->r_tid > req->r_tid)
491+ p = &(*p)->rb_right;
492+ else
493+ BUG();
494+ }
495+
496+ rb_link_node(&new->r_node, parent, p);
497+ rb_insert_color(&new->r_node, &mdsc->request_tree);
498+}
499+
500+/*
501+ * Register an in-flight request, and assign a tid. Link to directory
502+ * are modifying (if any).
503+ *
504+ * Called under mdsc->mutex.
505+ */
506+static void __register_request(struct ceph_mds_client *mdsc,
507+ struct ceph_mds_request *req,
508+ struct inode *dir)
509+{
510+ req->r_tid = ++mdsc->last_tid;
511+ if (req->r_num_caps)
512+ ceph_reserve_caps(&req->r_caps_reservation, req->r_num_caps);
513+ dout("__register_request %p tid %lld\n", req, req->r_tid);
514+ ceph_mdsc_get_request(req);
515+ __insert_request(mdsc, req);
516+
517+ if (dir) {
518+ struct ceph_inode_info *ci = ceph_inode(dir);
519+
520+ spin_lock(&ci->i_unsafe_lock);
521+ req->r_unsafe_dir = dir;
522+ list_add_tail(&req->r_unsafe_dir_item, &ci->i_unsafe_dirops);
523+ spin_unlock(&ci->i_unsafe_lock);
524+ }
525+}
526+
527+static void __unregister_request(struct ceph_mds_client *mdsc,
528+ struct ceph_mds_request *req)
529+{
530+ dout("__unregister_request %p tid %lld\n", req, req->r_tid);
531+ rb_erase(&req->r_node, &mdsc->request_tree);
532+ ceph_mdsc_put_request(req);
533+
534+ if (req->r_unsafe_dir) {
535+ struct ceph_inode_info *ci = ceph_inode(req->r_unsafe_dir);
536+
537+ spin_lock(&ci->i_unsafe_lock);
538+ list_del_init(&req->r_unsafe_dir_item);
539+ spin_unlock(&ci->i_unsafe_lock);
540+ }
541+}
542+
543+/*
544+ * Choose mds to send request to next. If there is a hint set in the
545+ * request (e.g., due to a prior forward hint from the mds), use that.
546+ * Otherwise, consult frag tree and/or caps to identify the
547+ * appropriate mds. If all else fails, choose randomly.
548+ *
549+ * Called under mdsc->mutex.
550+ */
551+static int __choose_mds(struct ceph_mds_client *mdsc,
552+ struct ceph_mds_request *req)
553+{
554+ struct inode *inode;
555+ struct ceph_inode_info *ci;
556+ struct ceph_cap *cap;
557+ int mode = req->r_direct_mode;
558+ int mds = -1;
559+ u32 hash = req->r_direct_hash;
560+ bool is_hash = req->r_direct_is_hash;
561+
562+ /*
563+ * is there a specific mds we should try? ignore hint if we have
564+ * no session and the mds is not up (active or recovering).
565+ */
566+ if (req->r_resend_mds >= 0 &&
567+ (__have_session(mdsc, req->r_resend_mds) ||
568+ ceph_mdsmap_get_state(mdsc->mdsmap, req->r_resend_mds) > 0)) {
569+ dout("choose_mds using resend_mds mds%d\n",
570+ req->r_resend_mds);
571+ return req->r_resend_mds;
572+ }
573+
574+ if (mode == USE_RANDOM_MDS)
575+ goto random;
576+
577+ inode = NULL;
578+ if (req->r_inode) {
579+ inode = req->r_inode;
580+ } else if (req->r_dentry) {
581+ if (req->r_dentry->d_inode) {
582+ inode = req->r_dentry->d_inode;
583+ } else {
584+ inode = req->r_dentry->d_parent->d_inode;
585+ hash = req->r_dentry->d_name.hash;
586+ is_hash = true;
587+ }
588+ }
589+ dout("__choose_mds %p is_hash=%d (%d) mode %d\n", inode, (int)is_hash,
590+ (int)hash, mode);
591+ if (!inode)
592+ goto random;
593+ ci = ceph_inode(inode);
594+
595+ if (is_hash && S_ISDIR(inode->i_mode)) {
596+ struct ceph_inode_frag frag;
597+ int found;
598+
599+ ceph_choose_frag(ci, hash, &frag, &found);
600+ if (found) {
601+ if (mode == USE_ANY_MDS && frag.ndist > 0) {
602+ u8 r;
603+
604+ /* choose a random replica */
605+ get_random_bytes(&r, 1);
606+ r %= frag.ndist;
607+ mds = frag.dist[r];
608+ dout("choose_mds %p %llx.%llx "
609+ "frag %u mds%d (%d/%d)\n",
610+ inode, ceph_vinop(inode),
611+ frag.frag, frag.mds,
612+ (int)r, frag.ndist);
613+ return mds;
614+ }
615+
616+ /* since this file/dir wasn't known to be
617+ * replicated, then we want to look for the
618+ * authoritative mds. */
619+ mode = USE_AUTH_MDS;
620+ if (frag.mds >= 0) {
621+ /* choose auth mds */
622+ mds = frag.mds;
623+ dout("choose_mds %p %llx.%llx "
624+ "frag %u mds%d (auth)\n",
625+ inode, ceph_vinop(inode), frag.frag, mds);
626+ return mds;
627+ }
628+ }
629+ }
630+
631+ spin_lock(&inode->i_lock);
632+ cap = NULL;
633+ if (mode == USE_AUTH_MDS)
634+ cap = ci->i_auth_cap;
635+ if (!cap && !RB_EMPTY_ROOT(&ci->i_caps))
636+ cap = rb_entry(rb_first(&ci->i_caps), struct ceph_cap, ci_node);
637+ if (!cap) {
638+ spin_unlock(&inode->i_lock);
639+ goto random;
640+ }
641+ mds = cap->session->s_mds;
642+ dout("choose_mds %p %llx.%llx mds%d (%scap %p)\n",
643+ inode, ceph_vinop(inode), mds,
644+ cap == ci->i_auth_cap ? "auth " : "", cap);
645+ spin_unlock(&inode->i_lock);
646+ return mds;
647+
648+random:
649+ mds = ceph_mdsmap_get_random_mds(mdsc->mdsmap);
650+ dout("choose_mds chose random mds%d\n", mds);
651+ return mds;
652+}
653+
654+
655+/*
656+ * session messages
657+ */
658+static struct ceph_msg *create_session_msg(u32 op, u64 seq)
659+{
660+ struct ceph_msg *msg;
661+ struct ceph_mds_session_head *h;
662+
663+ msg = ceph_msg_new(CEPH_MSG_CLIENT_SESSION, sizeof(*h), 0, 0, NULL);
664+ if (IS_ERR(msg)) {
665+ pr_err("create_session_msg ENOMEM creating msg\n");
666+ return ERR_PTR(PTR_ERR(msg));
667+ }
668+ h = msg->front.iov_base;
669+ h->op = cpu_to_le32(op);
670+ h->seq = cpu_to_le64(seq);
671+ return msg;
672+}
673+
674+/*
675+ * send session open request.
676+ *
677+ * called under mdsc->mutex
678+ */
679+static int __open_session(struct ceph_mds_client *mdsc,
680+ struct ceph_mds_session *session)
681+{
682+ struct ceph_msg *msg;
683+ int mstate;
684+ int mds = session->s_mds;
685+ int err = 0;
686+
687+ /* wait for mds to go active? */
688+ mstate = ceph_mdsmap_get_state(mdsc->mdsmap, mds);
689+ dout("open_session to mds%d (%s)\n", mds,
690+ ceph_mds_state_name(mstate));
691+ session->s_state = CEPH_MDS_SESSION_OPENING;
692+ session->s_renew_requested = jiffies;
693+
694+ /* send connect message */
695+ msg = create_session_msg(CEPH_SESSION_REQUEST_OPEN, session->s_seq);
696+ if (IS_ERR(msg)) {
697+ err = PTR_ERR(msg);
698+ goto out;
699+ }
700+ ceph_con_send(&session->s_con, msg);
701+
702+out:
703+ return 0;
704+}
705+
706+/*
707+ * session caps
708+ */
709+
710+/*
711+ * Free preallocated cap messages assigned to this session
712+ */
713+static void cleanup_cap_releases(struct ceph_mds_session *session)
714+{
715+ struct ceph_msg *msg;
716+
717+ spin_lock(&session->s_cap_lock);
718+ while (!list_empty(&session->s_cap_releases)) {
719+ msg = list_first_entry(&session->s_cap_releases,
720+ struct ceph_msg, list_head);
721+ list_del_init(&msg->list_head);
722+ ceph_msg_put(msg);
723+ }
724+ while (!list_empty(&session->s_cap_releases_done)) {
725+ msg = list_first_entry(&session->s_cap_releases_done,
726+ struct ceph_msg, list_head);
727+ list_del_init(&msg->list_head);
728+ ceph_msg_put(msg);
729+ }
730+ spin_unlock(&session->s_cap_lock);
731+}
732+
733+/*
734+ * Helper to safely iterate over all caps associated with a session.
735+ *
736+ * caller must hold session s_mutex
737+ */
738+static int iterate_session_caps(struct ceph_mds_session *session,
739+ int (*cb)(struct inode *, struct ceph_cap *,
740+ void *), void *arg)
741+{
742+ struct list_head *p;
743+ struct ceph_cap *cap;
744+ struct inode *inode, *last_inode = NULL;
745+ struct ceph_cap *old_cap = NULL;
746+ int ret;
747+
748+ dout("iterate_session_caps %p mds%d\n", session, session->s_mds);
749+ spin_lock(&session->s_cap_lock);
750+ p = session->s_caps.next;
751+ while (p != &session->s_caps) {
752+ cap = list_entry(p, struct ceph_cap, session_caps);
753+ inode = igrab(&cap->ci->vfs_inode);
754+ if (!inode) {
755+ p = p->next;
756+ continue;
757+ }
758+ session->s_cap_iterator = cap;
759+ spin_unlock(&session->s_cap_lock);
760+
761+ if (last_inode) {
762+ iput(last_inode);
763+ last_inode = NULL;
764+ }
765+ if (old_cap) {
766+ ceph_put_cap(old_cap);
767+ old_cap = NULL;
768+ }
769+
770+ ret = cb(inode, cap, arg);
771+ last_inode = inode;
772+
773+ spin_lock(&session->s_cap_lock);
774+ p = p->next;
775+ if (cap->ci == NULL) {
776+ dout("iterate_session_caps finishing cap %p removal\n",
777+ cap);
778+ BUG_ON(cap->session != session);
779+ list_del_init(&cap->session_caps);
780+ session->s_nr_caps--;
781+ cap->session = NULL;
782+ old_cap = cap; /* put_cap it w/o locks held */
783+ }
784+ if (ret < 0)
785+ goto out;
786+ }
787+ ret = 0;
788+out:
789+ session->s_cap_iterator = NULL;
790+ spin_unlock(&session->s_cap_lock);
791+
792+ if (last_inode)
793+ iput(last_inode);
794+ if (old_cap)
795+ ceph_put_cap(old_cap);
796+
797+ return ret;
798+}
799+
800+static int remove_session_caps_cb(struct inode *inode, struct ceph_cap *cap,
801+ void *arg)
802+{
803+ struct ceph_inode_info *ci = ceph_inode(inode);
804+ dout("removing cap %p, ci is %p, inode is %p\n",
805+ cap, ci, &ci->vfs_inode);
806+ ceph_remove_cap(cap);
807+ return 0;
808+}
809+
810+/*
811+ * caller must hold session s_mutex
812+ */
813+static void remove_session_caps(struct ceph_mds_session *session)
814+{
815+ dout("remove_session_caps on %p\n", session);
816+ iterate_session_caps(session, remove_session_caps_cb, NULL);
817+ BUG_ON(session->s_nr_caps > 0);
818+ cleanup_cap_releases(session);
819+}
820+
821+/*
822+ * wake up any threads waiting on this session's caps. if the cap is
823+ * old (didn't get renewed on the client reconnect), remove it now.
824+ *
825+ * caller must hold s_mutex.
826+ */
827+static int wake_up_session_cb(struct inode *inode, struct ceph_cap *cap,
828+ void *arg)
829+{
830+ struct ceph_inode_info *ci = ceph_inode(inode);
831+
832+ wake_up(&ci->i_cap_wq);
833+ if (arg) {
834+ spin_lock(&inode->i_lock);
835+ ci->i_wanted_max_size = 0;
836+ ci->i_requested_max_size = 0;
837+ spin_unlock(&inode->i_lock);
838+ }
839+ return 0;
840+}
841+
842+static void wake_up_session_caps(struct ceph_mds_session *session,
843+ int reconnect)
844+{
845+ dout("wake_up_session_caps %p mds%d\n", session, session->s_mds);
846+ iterate_session_caps(session, wake_up_session_cb,
847+ (void *)(unsigned long)reconnect);
848+}
849+
850+/*
851+ * Send periodic message to MDS renewing all currently held caps. The
852+ * ack will reset the expiration for all caps from this session.
853+ *
854+ * caller holds s_mutex
855+ */
856+static int send_renew_caps(struct ceph_mds_client *mdsc,
857+ struct ceph_mds_session *session)
858+{
859+ struct ceph_msg *msg;
860+ int state;
861+
862+ if (time_after_eq(jiffies, session->s_cap_ttl) &&
863+ time_after_eq(session->s_cap_ttl, session->s_renew_requested))
864+ pr_info("mds%d caps stale\n", session->s_mds);
865+
866+ /* do not try to renew caps until a recovering mds has reconnected
867+ * with its clients. */
868+ state = ceph_mdsmap_get_state(mdsc->mdsmap, session->s_mds);
869+ if (state < CEPH_MDS_STATE_RECONNECT) {
870+ dout("send_renew_caps ignoring mds%d (%s)\n",
871+ session->s_mds, ceph_mds_state_name(state));
872+ return 0;
873+ }
874+
875+ dout("send_renew_caps to mds%d (%s)\n", session->s_mds,
876+ ceph_mds_state_name(state));
877+ session->s_renew_requested = jiffies;
878+ msg = create_session_msg(CEPH_SESSION_REQUEST_RENEWCAPS,
879+ ++session->s_renew_seq);
880+ if (IS_ERR(msg))
881+ return PTR_ERR(msg);
882+ ceph_con_send(&session->s_con, msg);
883+ return 0;
884+}
885+
886+/*
887+ * Note new cap ttl, and any transition from stale -> not stale (fresh?).
888+ *
889+ * Called under session->s_mutex
890+ */
891+static void renewed_caps(struct ceph_mds_client *mdsc,
892+ struct ceph_mds_session *session, int is_renew)
893+{
894+ int was_stale;
895+ int wake = 0;
896+
897+ spin_lock(&session->s_cap_lock);
898+ was_stale = is_renew && (session->s_cap_ttl == 0 ||
899+ time_after_eq(jiffies, session->s_cap_ttl));
900+
901+ session->s_cap_ttl = session->s_renew_requested +
902+ mdsc->mdsmap->m_session_timeout*HZ;
903+
904+ if (was_stale) {
905+ if (time_before(jiffies, session->s_cap_ttl)) {
906+ pr_info("mds%d caps renewed\n", session->s_mds);
907+ wake = 1;
908+ } else {
909+ pr_info("mds%d caps still stale\n", session->s_mds);
910+ }
911+ }
912+ dout("renewed_caps mds%d ttl now %lu, was %s, now %s\n",
913+ session->s_mds, session->s_cap_ttl, was_stale ? "stale" : "fresh",
914+ time_before(jiffies, session->s_cap_ttl) ? "stale" : "fresh");
915+ spin_unlock(&session->s_cap_lock);
916+
917+ if (wake)
918+ wake_up_session_caps(session, 0);
919+}
920+
921+/*
922+ * send a session close request
923+ */
924+static int request_close_session(struct ceph_mds_client *mdsc,
925+ struct ceph_mds_session *session)
926+{
927+ struct ceph_msg *msg;
928+ int err = 0;
929+
930+ dout("request_close_session mds%d state %s seq %lld\n",
931+ session->s_mds, session_state_name(session->s_state),
932+ session->s_seq);
933+ msg = create_session_msg(CEPH_SESSION_REQUEST_CLOSE, session->s_seq);
934+ if (IS_ERR(msg))
935+ err = PTR_ERR(msg);
936+ else
937+ ceph_con_send(&session->s_con, msg);
938+ return err;
939+}
940+
941+/*
942+ * Called with s_mutex held.
943+ */
944+static int __close_session(struct ceph_mds_client *mdsc,
945+ struct ceph_mds_session *session)
946+{
947+ if (session->s_state >= CEPH_MDS_SESSION_CLOSING)
948+ return 0;
949+ session->s_state = CEPH_MDS_SESSION_CLOSING;
950+ return request_close_session(mdsc, session);
951+}
952+
953+/*
954+ * Trim old(er) caps.
955+ *
956+ * Because we can't cache an inode without one or more caps, we do
957+ * this indirectly: if a cap is unused, we prune its aliases, at which
958+ * point the inode will hopefully get dropped to.
959+ *
960+ * Yes, this is a bit sloppy. Our only real goal here is to respond to
961+ * memory pressure from the MDS, though, so it needn't be perfect.
962+ */
963+static int trim_caps_cb(struct inode *inode, struct ceph_cap *cap, void *arg)
964+{
965+ struct ceph_mds_session *session = arg;
966+ struct ceph_inode_info *ci = ceph_inode(inode);
967+ int used, oissued, mine;
968+
969+ if (session->s_trim_caps <= 0)
970+ return -1;
971+
972+ spin_lock(&inode->i_lock);
973+ mine = cap->issued | cap->implemented;
974+ used = __ceph_caps_used(ci);
975+ oissued = __ceph_caps_issued_other(ci, cap);
976+
977+ dout("trim_caps_cb %p cap %p mine %s oissued %s used %s\n",
978+ inode, cap, ceph_cap_string(mine), ceph_cap_string(oissued),
979+ ceph_cap_string(used));
980+ if (ci->i_dirty_caps)
981+ goto out; /* dirty caps */
982+ if ((used & ~oissued) & mine)
983+ goto out; /* we need these caps */
984+
985+ session->s_trim_caps--;
986+ if (oissued) {
987+ /* we aren't the only cap.. just remove us */
988+ __ceph_remove_cap(cap);
989+ } else {
990+ /* try to drop referring dentries */
991+ spin_unlock(&inode->i_lock);
992+ d_prune_aliases(inode);
993+ dout("trim_caps_cb %p cap %p pruned, count now %d\n",
994+ inode, cap, atomic_read(&inode->i_count));
995+ return 0;
996+ }
997+
998+out:
999+ spin_unlock(&inode->i_lock);
1000+ return 0;
1001+}
1002+
1003+/*
1004+ * Trim session cap count down to some max number.
1005+ */
1006+static int trim_caps(struct ceph_mds_client *mdsc,
1007+ struct ceph_mds_session *session,
1008+ int max_caps)
1009+{
1010+ int trim_caps = session->s_nr_caps - max_caps;
1011+
1012+ dout("trim_caps mds%d start: %d / %d, trim %d\n",
1013+ session->s_mds, session->s_nr_caps, max_caps, trim_caps);
1014+ if (trim_caps > 0) {
1015+ session->s_trim_caps = trim_caps;
1016+ iterate_session_caps(session, trim_caps_cb, session);
1017+ dout("trim_caps mds%d done: %d / %d, trimmed %d\n",
1018+ session->s_mds, session->s_nr_caps, max_caps,
1019+ trim_caps - session->s_trim_caps);
1020+ session->s_trim_caps = 0;
1021+ }
1022+ return 0;
1023+}
1024+
1025+/*
1026+ * Allocate cap_release messages. If there is a partially full message
1027+ * in the queue, try to allocate enough to cover it's remainder, so that
1028+ * we can send it immediately.
1029+ *
1030+ * Called under s_mutex.
1031+ */
1032+static int add_cap_releases(struct ceph_mds_client *mdsc,
1033+ struct ceph_mds_session *session,
1034+ int extra)
1035+{
1036+ struct ceph_msg *msg;
1037+ struct ceph_mds_cap_release *head;
1038+ int err = -ENOMEM;
1039+
1040+ if (extra < 0)
1041+ extra = mdsc->client->mount_args->cap_release_safety;
1042+
1043+ spin_lock(&session->s_cap_lock);
1044+
1045+ if (!list_empty(&session->s_cap_releases)) {
1046+ msg = list_first_entry(&session->s_cap_releases,
1047+ struct ceph_msg,
1048+ list_head);
1049+ head = msg->front.iov_base;
1050+ extra += CEPH_CAPS_PER_RELEASE - le32_to_cpu(head->num);
1051+ }
1052+
1053+ while (session->s_num_cap_releases < session->s_nr_caps + extra) {
1054+ spin_unlock(&session->s_cap_lock);
1055+ msg = ceph_msg_new(CEPH_MSG_CLIENT_CAPRELEASE, PAGE_CACHE_SIZE,
1056+ 0, 0, NULL);
1057+ if (!msg)
1058+ goto out_unlocked;
1059+ dout("add_cap_releases %p msg %p now %d\n", session, msg,
1060+ (int)msg->front.iov_len);
1061+ head = msg->front.iov_base;
1062+ head->num = cpu_to_le32(0);
1063+ msg->front.iov_len = sizeof(*head);
1064+ spin_lock(&session->s_cap_lock);
1065+ list_add(&msg->list_head, &session->s_cap_releases);
1066+ session->s_num_cap_releases += CEPH_CAPS_PER_RELEASE;
1067+ }
1068+
1069+ if (!list_empty(&session->s_cap_releases)) {
1070+ msg = list_first_entry(&session->s_cap_releases,
1071+ struct ceph_msg,
1072+ list_head);
1073+ head = msg->front.iov_base;
1074+ if (head->num) {
1075+ dout(" queueing non-full %p (%d)\n", msg,
1076+ le32_to_cpu(head->num));
1077+ list_move_tail(&msg->list_head,
1078+ &session->s_cap_releases_done);
1079+ session->s_num_cap_releases -=
1080+ CEPH_CAPS_PER_RELEASE - le32_to_cpu(head->num);
1081+ }
1082+ }
1083+ err = 0;
1084+ spin_unlock(&session->s_cap_lock);
1085+out_unlocked:
1086+ return err;
1087+}
1088+
1089+/*
1090+ * flush all dirty inode data to disk.
1091+ *
1092+ * returns true if we've flushed through want_flush_seq
1093+ */
1094+static int check_cap_flush(struct ceph_mds_client *mdsc, u64 want_flush_seq)
1095+{
1096+ int mds, ret = 1;
1097+
1098+ dout("check_cap_flush want %lld\n", want_flush_seq);
1099+ mutex_lock(&mdsc->mutex);
1100+ for (mds = 0; ret && mds < mdsc->max_sessions; mds++) {
1101+ struct ceph_mds_session *session = mdsc->sessions[mds];
1102+
1103+ if (!session)
1104+ continue;
1105+ get_session(session);
1106+ mutex_unlock(&mdsc->mutex);
1107+
1108+ mutex_lock(&session->s_mutex);
1109+ if (!list_empty(&session->s_cap_flushing)) {
1110+ struct ceph_inode_info *ci =
1111+ list_entry(session->s_cap_flushing.next,
1112+ struct ceph_inode_info,
1113+ i_flushing_item);
1114+ struct inode *inode = &ci->vfs_inode;
1115+
1116+ spin_lock(&inode->i_lock);
1117+ if (ci->i_cap_flush_seq <= want_flush_seq) {
1118+ dout("check_cap_flush still flushing %p "
1119+ "seq %lld <= %lld to mds%d\n", inode,
1120+ ci->i_cap_flush_seq, want_flush_seq,
1121+ session->s_mds);
1122+ ret = 0;
1123+ }
1124+ spin_unlock(&inode->i_lock);
1125+ }
1126+ mutex_unlock(&session->s_mutex);
1127+ ceph_put_mds_session(session);
1128+
1129+ if (!ret)
1130+ return ret;
1131+ mutex_lock(&mdsc->mutex);
1132+ }
1133+
1134+ mutex_unlock(&mdsc->mutex);
1135+ dout("check_cap_flush ok, flushed thru %lld\n", want_flush_seq);
1136+ return ret;
1137+}
1138+
1139+/*
1140+ * called under s_mutex
1141+ */
1142+static void send_cap_releases(struct ceph_mds_client *mdsc,
1143+ struct ceph_mds_session *session)
1144+{
1145+ struct ceph_msg *msg;
1146+
1147+ dout("send_cap_releases mds%d\n", session->s_mds);
1148+ while (1) {
1149+ spin_lock(&session->s_cap_lock);
1150+ if (list_empty(&session->s_cap_releases_done))
1151+ break;
1152+ msg = list_first_entry(&session->s_cap_releases_done,
1153+ struct ceph_msg, list_head);
1154+ list_del_init(&msg->list_head);
1155+ spin_unlock(&session->s_cap_lock);
1156+ msg->hdr.front_len = cpu_to_le32(msg->front.iov_len);
1157+ dout("send_cap_releases mds%d %p\n", session->s_mds, msg);
1158+ ceph_con_send(&session->s_con, msg);
1159+ }
1160+ spin_unlock(&session->s_cap_lock);
1161+}
1162+
1163+/*
1164+ * requests
1165+ */
1166+
1167+/*
1168+ * Create an mds request.
1169+ */
1170+struct ceph_mds_request *
1171+ceph_mdsc_create_request(struct ceph_mds_client *mdsc, int op, int mode)
1172+{
1173+ struct ceph_mds_request *req = kzalloc(sizeof(*req), GFP_NOFS);
1174+
1175+ if (!req)
1176+ return ERR_PTR(-ENOMEM);
1177+
1178+ req->r_started = jiffies;
1179+ req->r_resend_mds = -1;
1180+ INIT_LIST_HEAD(&req->r_unsafe_dir_item);
1181+ req->r_fmode = -1;
1182+ kref_init(&req->r_kref);
1183+ INIT_LIST_HEAD(&req->r_wait);
1184+ init_completion(&req->r_completion);
1185+ init_completion(&req->r_safe_completion);
1186+ INIT_LIST_HEAD(&req->r_unsafe_item);
1187+
1188+ req->r_op = op;
1189+ req->r_direct_mode = mode;
1190+ return req;
1191+}
1192+
1193+/*
1194+ * return oldest (lowest) request, tid in request tree, 0 if none.
1195+ *
1196+ * called under mdsc->mutex.
1197+ */
1198+static struct ceph_mds_request *__get_oldest_req(struct ceph_mds_client *mdsc)
1199+{
1200+ if (RB_EMPTY_ROOT(&mdsc->request_tree))
1201+ return NULL;
1202+ return rb_entry(rb_first(&mdsc->request_tree),
1203+ struct ceph_mds_request, r_node);
1204+}
1205+
1206+static u64 __get_oldest_tid(struct ceph_mds_client *mdsc)
1207+{
1208+ struct ceph_mds_request *req = __get_oldest_req(mdsc);
1209+
1210+ if (req)
1211+ return req->r_tid;
1212+ return 0;
1213+}
1214+
1215+/*
1216+ * Build a dentry's path. Allocate on heap; caller must kfree. Based
1217+ * on build_path_from_dentry in fs/cifs/dir.c.
1218+ *
1219+ * If @stop_on_nosnap, generate path relative to the first non-snapped
1220+ * inode.
1221+ *
1222+ * Encode hidden .snap dirs as a double /, i.e.
1223+ * foo/.snap/bar -> foo//bar
1224+ */
1225+char *ceph_mdsc_build_path(struct dentry *dentry, int *plen, u64 *base,
1226+ int stop_on_nosnap)
1227+{
1228+ struct dentry *temp;
1229+ char *path;
1230+ int len, pos;
1231+
1232+ if (dentry == NULL)
1233+ return ERR_PTR(-EINVAL);
1234+
1235+retry:
1236+ len = 0;
1237+ for (temp = dentry; !IS_ROOT(temp);) {
1238+ struct inode *inode = temp->d_inode;
1239+ if (inode && ceph_snap(inode) == CEPH_SNAPDIR)
1240+ len++; /* slash only */
1241+ else if (stop_on_nosnap && inode &&
1242+ ceph_snap(inode) == CEPH_NOSNAP)
1243+ break;
1244+ else
1245+ len += 1 + temp->d_name.len;
1246+ temp = temp->d_parent;
1247+ if (temp == NULL) {
1248+ pr_err("build_path_dentry corrupt dentry %p\n", dentry);
1249+ return ERR_PTR(-EINVAL);
1250+ }
1251+ }
1252+ if (len)
1253+ len--; /* no leading '/' */
1254+
1255+ path = kmalloc(len+1, GFP_NOFS);
1256+ if (path == NULL)
1257+ return ERR_PTR(-ENOMEM);
1258+ pos = len;
1259+ path[pos] = 0; /* trailing null */
1260+ for (temp = dentry; !IS_ROOT(temp) && pos != 0; ) {
1261+ struct inode *inode = temp->d_inode;
1262+
1263+ if (inode && ceph_snap(inode) == CEPH_SNAPDIR) {
1264+ dout("build_path_dentry path+%d: %p SNAPDIR\n",
1265+ pos, temp);
1266+ } else if (stop_on_nosnap && inode &&
1267+ ceph_snap(inode) == CEPH_NOSNAP) {
1268+ break;
1269+ } else {
1270+ pos -= temp->d_name.len;
1271+ if (pos < 0)
1272+ break;
1273+ strncpy(path + pos, temp->d_name.name,
1274+ temp->d_name.len);
1275+ dout("build_path_dentry path+%d: %p '%.*s'\n",
1276+ pos, temp, temp->d_name.len, path + pos);
1277+ }
1278+ if (pos)
1279+ path[--pos] = '/';
1280+ temp = temp->d_parent;
1281+ if (temp == NULL) {
1282+ pr_err("build_path_dentry corrupt dentry\n");
1283+ kfree(path);
1284+ return ERR_PTR(-EINVAL);
1285+ }
1286+ }
1287+ if (pos != 0) {
1288+ pr_err("build_path_dentry did not end path lookup where "
1289+ "expected, namelen is %d, pos is %d\n", len, pos);
1290+ /* presumably this is only possible if racing with a
1291+ rename of one of the parent directories (we can not
1292+ lock the dentries above us to prevent this, but
1293+ retrying should be harmless) */
1294+ kfree(path);
1295+ goto retry;
1296+ }
1297+
1298+ *base = ceph_ino(temp->d_inode);
1299+ *plen = len;
1300+ dout("build_path_dentry on %p %d built %llx '%.*s'\n",
1301+ dentry, atomic_read(&dentry->d_count), *base, len, path);
1302+ return path;
1303+}
1304+
1305+static int build_dentry_path(struct dentry *dentry,
1306+ const char **ppath, int *ppathlen, u64 *pino,
1307+ int *pfreepath)
1308+{
1309+ char *path;
1310+
1311+ if (ceph_snap(dentry->d_parent->d_inode) == CEPH_NOSNAP) {
1312+ *pino = ceph_ino(dentry->d_parent->d_inode);
1313+ *ppath = dentry->d_name.name;
1314+ *ppathlen = dentry->d_name.len;
1315+ return 0;
1316+ }
1317+ path = ceph_mdsc_build_path(dentry, ppathlen, pino, 1);
1318+ if (IS_ERR(path))
1319+ return PTR_ERR(path);
1320+ *ppath = path;
1321+ *pfreepath = 1;
1322+ return 0;
1323+}
1324+
1325+static int build_inode_path(struct inode *inode,
1326+ const char **ppath, int *ppathlen, u64 *pino,
1327+ int *pfreepath)
1328+{
1329+ struct dentry *dentry;
1330+ char *path;
1331+
1332+ if (ceph_snap(inode) == CEPH_NOSNAP) {
1333+ *pino = ceph_ino(inode);
1334+ *ppathlen = 0;
1335+ return 0;
1336+ }
1337+ dentry = d_find_alias(inode);
1338+ path = ceph_mdsc_build_path(dentry, ppathlen, pino, 1);
1339+ dput(dentry);
1340+ if (IS_ERR(path))
1341+ return PTR_ERR(path);
1342+ *ppath = path;
1343+ *pfreepath = 1;
1344+ return 0;
1345+}
1346+
1347+/*
1348+ * request arguments may be specified via an inode *, a dentry *, or
1349+ * an explicit ino+path.
1350+ */
1351+static int set_request_path_attr(struct inode *rinode, struct dentry *rdentry,
1352+ const char *rpath, u64 rino,
1353+ const char **ppath, int *pathlen,
1354+ u64 *ino, int *freepath)
1355+{
1356+ int r = 0;
1357+
1358+ if (rinode) {
1359+ r = build_inode_path(rinode, ppath, pathlen, ino, freepath);
1360+ dout(" inode %p %llx.%llx\n", rinode, ceph_ino(rinode),
1361+ ceph_snap(rinode));
1362+ } else if (rdentry) {
1363+ r = build_dentry_path(rdentry, ppath, pathlen, ino, freepath);
1364+ dout(" dentry %p %llx/%.*s\n", rdentry, *ino, *pathlen,
1365+ *ppath);
1366+ } else if (rpath) {
1367+ *ino = rino;
1368+ *ppath = rpath;
1369+ *pathlen = strlen(rpath);
1370+ dout(" path %.*s\n", *pathlen, rpath);
1371+ }
1372+
1373+ return r;
1374+}
1375+
1376+/*
1377+ * called under mdsc->mutex
1378+ */
1379+static struct ceph_msg *create_request_message(struct ceph_mds_client *mdsc,
1380+ struct ceph_mds_request *req,
1381+ int mds)
1382+{
1383+ struct ceph_msg *msg;
1384+ struct ceph_mds_request_head *head;
1385+ const char *path1 = NULL;
1386+ const char *path2 = NULL;
1387+ u64 ino1 = 0, ino2 = 0;
1388+ int pathlen1 = 0, pathlen2 = 0;
1389+ int freepath1 = 0, freepath2 = 0;
1390+ int len;
1391+ u16 releases;
1392+ void *p, *end;
1393+ int ret;
1394+
1395+ ret = set_request_path_attr(req->r_inode, req->r_dentry,
1396+ req->r_path1, req->r_ino1.ino,
1397+ &path1, &pathlen1, &ino1, &freepath1);
1398+ if (ret < 0) {
1399+ msg = ERR_PTR(ret);
1400+ goto out;
1401+ }
1402+
1403+ ret = set_request_path_attr(NULL, req->r_old_dentry,
1404+ req->r_path2, req->r_ino2.ino,
1405+ &path2, &pathlen2, &ino2, &freepath2);
1406+ if (ret < 0) {
1407+ msg = ERR_PTR(ret);
1408+ goto out_free1;
1409+ }
1410+
1411+ len = sizeof(*head) +
1412+ pathlen1 + pathlen2 + 2*(1 + sizeof(u32) + sizeof(u64));
1413+
1414+ /* calculate (max) length for cap releases */
1415+ len += sizeof(struct ceph_mds_request_release) *
1416+ (!!req->r_inode_drop + !!req->r_dentry_drop +
1417+ !!req->r_old_inode_drop + !!req->r_old_dentry_drop);
1418+ if (req->r_dentry_drop)
1419+ len += req->r_dentry->d_name.len;
1420+ if (req->r_old_dentry_drop)
1421+ len += req->r_old_dentry->d_name.len;
1422+
1423+ msg = ceph_msg_new(CEPH_MSG_CLIENT_REQUEST, len, 0, 0, NULL);
1424+ if (IS_ERR(msg))
1425+ goto out_free2;
1426+
1427+ msg->hdr.tid = cpu_to_le64(req->r_tid);
1428+
1429+ head = msg->front.iov_base;
1430+ p = msg->front.iov_base + sizeof(*head);
1431+ end = msg->front.iov_base + msg->front.iov_len;
1432+
1433+ head->mdsmap_epoch = cpu_to_le32(mdsc->mdsmap->m_epoch);
1434+ head->op = cpu_to_le32(req->r_op);
1435+ head->caller_uid = cpu_to_le32(current_fsuid());
1436+ head->caller_gid = cpu_to_le32(current_fsgid());
1437+ head->args = req->r_args;
1438+
1439+ ceph_encode_filepath(&p, end, ino1, path1);
1440+ ceph_encode_filepath(&p, end, ino2, path2);
1441+
1442+ /* cap releases */
1443+ releases = 0;
1444+ if (req->r_inode_drop)
1445+ releases += ceph_encode_inode_release(&p,
1446+ req->r_inode ? req->r_inode : req->r_dentry->d_inode,
1447+ mds, req->r_inode_drop, req->r_inode_unless, 0);
1448+ if (req->r_dentry_drop)
1449+ releases += ceph_encode_dentry_release(&p, req->r_dentry,
1450+ mds, req->r_dentry_drop, req->r_dentry_unless);
1451+ if (req->r_old_dentry_drop)
1452+ releases += ceph_encode_dentry_release(&p, req->r_old_dentry,
1453+ mds, req->r_old_dentry_drop, req->r_old_dentry_unless);
1454+ if (req->r_old_inode_drop)
1455+ releases += ceph_encode_inode_release(&p,
1456+ req->r_old_dentry->d_inode,
1457+ mds, req->r_old_inode_drop, req->r_old_inode_unless, 0);
1458+ head->num_releases = cpu_to_le16(releases);
1459+
1460+ BUG_ON(p > end);
1461+ msg->front.iov_len = p - msg->front.iov_base;
1462+ msg->hdr.front_len = cpu_to_le32(msg->front.iov_len);
1463+
1464+ msg->pages = req->r_pages;
1465+ msg->nr_pages = req->r_num_pages;
1466+ msg->hdr.data_len = cpu_to_le32(req->r_data_len);
1467+ msg->hdr.data_off = cpu_to_le16(0);
1468+
1469+out_free2:
1470+ if (freepath2)
1471+ kfree((char *)path2);
1472+out_free1:
1473+ if (freepath1)
1474+ kfree((char *)path1);
1475+out:
1476+ return msg;
1477+}
1478+
1479+/*
1480+ * called under mdsc->mutex if error, under no mutex if
1481+ * success.
1482+ */
1483+static void complete_request(struct ceph_mds_client *mdsc,
1484+ struct ceph_mds_request *req)
1485+{
1486+ if (req->r_callback)
1487+ req->r_callback(mdsc, req);
1488+ else
1489+ complete(&req->r_completion);
1490+}
1491+
1492+/*
1493+ * called under mdsc->mutex
1494+ */
1495+static int __prepare_send_request(struct ceph_mds_client *mdsc,
1496+ struct ceph_mds_request *req,
1497+ int mds)
1498+{
1499+ struct ceph_mds_request_head *rhead;
1500+ struct ceph_msg *msg;
1501+ int flags = 0;
1502+
1503+ req->r_mds = mds;
1504+ req->r_attempts++;
1505+ dout("prepare_send_request %p tid %lld %s (attempt %d)\n", req,
1506+ req->r_tid, ceph_mds_op_name(req->r_op), req->r_attempts);
1507+
1508+ if (req->r_request) {
1509+ ceph_msg_put(req->r_request);
1510+ req->r_request = NULL;
1511+ }
1512+ msg = create_request_message(mdsc, req, mds);
1513+ if (IS_ERR(msg)) {
1514+ req->r_reply = ERR_PTR(PTR_ERR(msg));
1515+ complete_request(mdsc, req);
1516+ return -PTR_ERR(msg);
1517+ }
1518+ req->r_request = msg;
1519+
1520+ rhead = msg->front.iov_base;
1521+ rhead->oldest_client_tid = cpu_to_le64(__get_oldest_tid(mdsc));
1522+ if (req->r_got_unsafe)
1523+ flags |= CEPH_MDS_FLAG_REPLAY;
1524+ if (req->r_locked_dir)
1525+ flags |= CEPH_MDS_FLAG_WANT_DENTRY;
1526+ rhead->flags = cpu_to_le32(flags);
1527+ rhead->num_fwd = req->r_num_fwd;
1528+ rhead->num_retry = req->r_attempts - 1;
1529+
1530+ dout(" r_locked_dir = %p\n", req->r_locked_dir);
1531+
1532+ if (req->r_target_inode && req->r_got_unsafe)
1533+ rhead->ino = cpu_to_le64(ceph_ino(req->r_target_inode));
1534+ else
1535+ rhead->ino = 0;
1536+ return 0;
1537+}
1538+
1539+/*
1540+ * send request, or put it on the appropriate wait list.
1541+ */
1542+static int __do_request(struct ceph_mds_client *mdsc,
1543+ struct ceph_mds_request *req)
1544+{
1545+ struct ceph_mds_session *session = NULL;
1546+ int mds = -1;
1547+ int err = -EAGAIN;
1548+
1549+ if (req->r_reply)
1550+ goto out;
1551+
1552+ if (req->r_timeout &&
1553+ time_after_eq(jiffies, req->r_started + req->r_timeout)) {
1554+ dout("do_request timed out\n");
1555+ err = -EIO;
1556+ goto finish;
1557+ }
1558+
1559+ mds = __choose_mds(mdsc, req);
1560+ if (mds < 0 ||
1561+ ceph_mdsmap_get_state(mdsc->mdsmap, mds) < CEPH_MDS_STATE_ACTIVE) {
1562+ dout("do_request no mds or not active, waiting for map\n");
1563+ list_add(&req->r_wait, &mdsc->waiting_for_map);
1564+ goto out;
1565+ }
1566+
1567+ /* get, open session */
1568+ session = __ceph_lookup_mds_session(mdsc, mds);
1569+ if (!session)
1570+ session = register_session(mdsc, mds);
1571+ dout("do_request mds%d session %p state %s\n", mds, session,
1572+ session_state_name(session->s_state));
1573+ if (session->s_state != CEPH_MDS_SESSION_OPEN &&
1574+ session->s_state != CEPH_MDS_SESSION_HUNG) {
1575+ if (session->s_state == CEPH_MDS_SESSION_NEW ||
1576+ session->s_state == CEPH_MDS_SESSION_CLOSING)
1577+ __open_session(mdsc, session);
1578+ list_add(&req->r_wait, &session->s_waiting);
1579+ goto out_session;
1580+ }
1581+
1582+ /* send request */
1583+ req->r_session = get_session(session);
1584+ req->r_resend_mds = -1; /* forget any previous mds hint */
1585+
1586+ if (req->r_request_started == 0) /* note request start time */
1587+ req->r_request_started = jiffies;
1588+
1589+ err = __prepare_send_request(mdsc, req, mds);
1590+ if (!err) {
1591+ ceph_msg_get(req->r_request);
1592+ ceph_con_send(&session->s_con, req->r_request);
1593+ }
1594+
1595+out_session:
1596+ ceph_put_mds_session(session);
1597+out:
1598+ return err;
1599+
1600+finish:
1601+ req->r_reply = ERR_PTR(err);
1602+ complete_request(mdsc, req);
1603+ goto out;
1604+}
1605+
1606+/*
1607+ * called under mdsc->mutex
1608+ */
1609+static void __wake_requests(struct ceph_mds_client *mdsc,
1610+ struct list_head *head)
1611+{
1612+ struct ceph_mds_request *req, *nreq;
1613+
1614+ list_for_each_entry_safe(req, nreq, head, r_wait) {
1615+ list_del_init(&req->r_wait);
1616+ __do_request(mdsc, req);
1617+ }
1618+}
1619+
1620+/*
1621+ * Wake up threads with requests pending for @mds, so that they can
1622+ * resubmit their requests to a possibly different mds. If @all is set,
1623+ * wake up if their requests has been forwarded to @mds, too.
1624+ */
1625+static void kick_requests(struct ceph_mds_client *mdsc, int mds, int all)
1626+{
1627+ struct ceph_mds_request *req;
1628+ struct rb_node *p;
1629+
1630+ dout("kick_requests mds%d\n", mds);
1631+ for (p = rb_first(&mdsc->request_tree); p; p = rb_next(p)) {
1632+ req = rb_entry(p, struct ceph_mds_request, r_node);
1633+ if (req->r_got_unsafe)
1634+ continue;
1635+ if (req->r_session &&
1636+ req->r_session->s_mds == mds) {
1637+ dout(" kicking tid %llu\n", req->r_tid);
1638+ put_request_session(req);
1639+ __do_request(mdsc, req);
1640+ }
1641+ }
1642+}
1643+
1644+void ceph_mdsc_submit_request(struct ceph_mds_client *mdsc,
1645+ struct ceph_mds_request *req)
1646+{
1647+ dout("submit_request on %p\n", req);
1648+ mutex_lock(&mdsc->mutex);
1649+ __register_request(mdsc, req, NULL);
1650+ __do_request(mdsc, req);
1651+ mutex_unlock(&mdsc->mutex);
1652+}
1653+
1654+/*
1655+ * Synchrously perform an mds request. Take care of all of the
1656+ * session setup, forwarding, retry details.
1657+ */
1658+int ceph_mdsc_do_request(struct ceph_mds_client *mdsc,
1659+ struct inode *dir,
1660+ struct ceph_mds_request *req)
1661+{
1662+ int err;
1663+
1664+ dout("do_request on %p\n", req);
1665+
1666+ /* take CAP_PIN refs for r_inode, r_locked_dir, r_old_dentry */
1667+ if (req->r_inode)
1668+ ceph_get_cap_refs(ceph_inode(req->r_inode), CEPH_CAP_PIN);
1669+ if (req->r_locked_dir)
1670+ ceph_get_cap_refs(ceph_inode(req->r_locked_dir), CEPH_CAP_PIN);
1671+ if (req->r_old_dentry)
1672+ ceph_get_cap_refs(
1673+ ceph_inode(req->r_old_dentry->d_parent->d_inode),
1674+ CEPH_CAP_PIN);
1675+
1676+ /* issue */
1677+ mutex_lock(&mdsc->mutex);
1678+ __register_request(mdsc, req, dir);
1679+ __do_request(mdsc, req);
1680+
1681+ /* wait */
1682+ if (!req->r_reply) {
1683+ mutex_unlock(&mdsc->mutex);
1684+ if (req->r_timeout) {
1685+ err = (long)wait_for_completion_interruptible_timeout(
1686+ &req->r_completion, req->r_timeout);
1687+ if (err == 0)
1688+ req->r_reply = ERR_PTR(-EIO);
1689+ else if (err < 0)
1690+ req->r_reply = ERR_PTR(err);
1691+ } else {
1692+ err = wait_for_completion_interruptible(
1693+ &req->r_completion);
1694+ if (err)
1695+ req->r_reply = ERR_PTR(err);
1696+ }
1697+ mutex_lock(&mdsc->mutex);
1698+ }
1699+
1700+ if (IS_ERR(req->r_reply)) {
1701+ err = PTR_ERR(req->r_reply);
1702+ req->r_reply = NULL;
1703+
1704+ if (err == -ERESTARTSYS) {
1705+ /* aborted */
1706+ req->r_aborted = true;
1707+
1708+ if (req->r_locked_dir &&
1709+ (req->r_op & CEPH_MDS_OP_WRITE)) {
1710+ struct ceph_inode_info *ci =
1711+ ceph_inode(req->r_locked_dir);
1712+
1713+ dout("aborted, clearing I_COMPLETE on %p\n",
1714+ req->r_locked_dir);
1715+ spin_lock(&req->r_locked_dir->i_lock);
1716+ ci->i_ceph_flags &= ~CEPH_I_COMPLETE;
1717+ ci->i_release_count++;
1718+ spin_unlock(&req->r_locked_dir->i_lock);
1719+ }
1720+ } else {
1721+ /* clean up this request */
1722+ __unregister_request(mdsc, req);
1723+ if (!list_empty(&req->r_unsafe_item))
1724+ list_del_init(&req->r_unsafe_item);
1725+ complete(&req->r_safe_completion);
1726+ }
1727+ } else if (req->r_err) {
1728+ err = req->r_err;
1729+ } else {
1730+ err = le32_to_cpu(req->r_reply_info.head->result);
1731+ }
1732+ mutex_unlock(&mdsc->mutex);
1733+
1734+ dout("do_request %p done, result %d\n", req, err);
1735+ return err;
1736+}
1737+
1738+/*
1739+ * Handle mds reply.
1740+ *
1741+ * We take the session mutex and parse and process the reply immediately.
1742+ * This preserves the logical ordering of replies, capabilities, etc., sent
1743+ * by the MDS as they are applied to our local cache.
1744+ */
1745+static void handle_reply(struct ceph_mds_session *session, struct ceph_msg *msg)
1746+{
1747+ struct ceph_mds_client *mdsc = session->s_mdsc;
1748+ struct ceph_mds_request *req;
1749+ struct ceph_mds_reply_head *head = msg->front.iov_base;
1750+ struct ceph_mds_reply_info_parsed *rinfo; /* parsed reply info */
1751+ u64 tid;
1752+ int err, result;
1753+ int mds = session->s_mds;
1754+
1755+ if (msg->front.iov_len < sizeof(*head)) {
1756+ pr_err("mdsc_handle_reply got corrupt (short) reply\n");
1757+ ceph_msg_dump(msg);
1758+ return;
1759+ }
1760+
1761+ /* get request, session */
1762+ tid = le64_to_cpu(msg->hdr.tid);
1763+ mutex_lock(&mdsc->mutex);
1764+ req = __lookup_request(mdsc, tid);
1765+ if (!req) {
1766+ dout("handle_reply on unknown tid %llu\n", tid);
1767+ mutex_unlock(&mdsc->mutex);
1768+ return;
1769+ }
1770+ dout("handle_reply %p\n", req);
1771+
1772+ /* correct session? */
1773+ if (!req->r_session && req->r_session != session) {
1774+ pr_err("mdsc_handle_reply got %llu on session mds%d"
1775+ " not mds%d\n", tid, session->s_mds,
1776+ req->r_session ? req->r_session->s_mds : -1);
1777+ mutex_unlock(&mdsc->mutex);
1778+ goto out;
1779+ }
1780+
1781+ /* dup? */
1782+ if ((req->r_got_unsafe && !head->safe) ||
1783+ (req->r_got_safe && head->safe)) {
1784+ pr_warning("got a dup %s reply on %llu from mds%d\n",
1785+ head->safe ? "safe" : "unsafe", tid, mds);
1786+ mutex_unlock(&mdsc->mutex);
1787+ goto out;
1788+ }
1789+
1790+ result = le32_to_cpu(head->result);
1791+
1792+ /*
1793+ * Tolerate 2 consecutive ESTALEs from the same mds.
1794+ * FIXME: we should be looking at the cap migrate_seq.
1795+ */
1796+ if (result == -ESTALE) {
1797+ req->r_direct_mode = USE_AUTH_MDS;
1798+ req->r_num_stale++;
1799+ if (req->r_num_stale <= 2) {
1800+ __do_request(mdsc, req);
1801+ mutex_unlock(&mdsc->mutex);
1802+ goto out;
1803+ }
1804+ } else {
1805+ req->r_num_stale = 0;
1806+ }
1807+
1808+ if (head->safe) {
1809+ req->r_got_safe = true;
1810+ __unregister_request(mdsc, req);
1811+ complete(&req->r_safe_completion);
1812+
1813+ if (req->r_got_unsafe) {
1814+ /*
1815+ * We already handled the unsafe response, now do the
1816+ * cleanup. No need to examine the response; the MDS
1817+ * doesn't include any result info in the safe
1818+ * response. And even if it did, there is nothing
1819+ * useful we could do with a revised return value.
1820+ */
1821+ dout("got safe reply %llu, mds%d\n", tid, mds);
1822+ list_del_init(&req->r_unsafe_item);
1823+
1824+ /* last unsafe request during umount? */
1825+ if (mdsc->stopping && !__get_oldest_req(mdsc))
1826+ complete(&mdsc->safe_umount_waiters);
1827+ mutex_unlock(&mdsc->mutex);
1828+ goto out;
1829+ }
1830+ }
1831+
1832+ BUG_ON(req->r_reply);
1833+
1834+ if (!head->safe) {
1835+ req->r_got_unsafe = true;
1836+ list_add_tail(&req->r_unsafe_item, &req->r_session->s_unsafe);
1837+ }
1838+
1839+ dout("handle_reply tid %lld result %d\n", tid, result);
1840+ rinfo = &req->r_reply_info;
1841+ err = parse_reply_info(msg, rinfo);
1842+ mutex_unlock(&mdsc->mutex);
1843+
1844+ mutex_lock(&session->s_mutex);
1845+ if (err < 0) {
1846+ pr_err("mdsc_handle_reply got corrupt reply mds%d\n", mds);
1847+ ceph_msg_dump(msg);
1848+ goto out_err;
1849+ }
1850+
1851+ /* snap trace */
1852+ if (rinfo->snapblob_len) {
1853+ down_write(&mdsc->snap_rwsem);
1854+ ceph_update_snap_trace(mdsc, rinfo->snapblob,
1855+ rinfo->snapblob + rinfo->snapblob_len,
1856+ le32_to_cpu(head->op) == CEPH_MDS_OP_RMSNAP);
1857+ downgrade_write(&mdsc->snap_rwsem);
1858+ } else {
1859+ down_read(&mdsc->snap_rwsem);
1860+ }
1861+
1862+ /* insert trace into our cache */
1863+ err = ceph_fill_trace(mdsc->client->sb, req, req->r_session);
1864+ if (err == 0) {
1865+ if (result == 0 && rinfo->dir_nr)
1866+ ceph_readdir_prepopulate(req, req->r_session);
1867+ ceph_unreserve_caps(&req->r_caps_reservation);
1868+ }
1869+
1870+ up_read(&mdsc->snap_rwsem);
1871+out_err:
1872+ if (err) {
1873+ req->r_err = err;
1874+ } else {
1875+ req->r_reply = msg;
1876+ ceph_msg_get(msg);
1877+ }
1878+
1879+ add_cap_releases(mdsc, req->r_session, -1);
1880+ mutex_unlock(&session->s_mutex);
1881+
1882+ /* kick calling process */
1883+ complete_request(mdsc, req);
1884+out:
1885+ ceph_mdsc_put_request(req);
1886+ return;
1887+}
1888+
1889+
1890+
1891+/*
1892+ * handle mds notification that our request has been forwarded.
1893+ */
1894+static void handle_forward(struct ceph_mds_client *mdsc,
1895+ struct ceph_mds_session *session,
1896+ struct ceph_msg *msg)
1897+{
1898+ struct ceph_mds_request *req;
1899+ u64 tid = le64_to_cpu(msg->hdr.tid);
1900+ u32 next_mds;
1901+ u32 fwd_seq;
1902+ int err = -EINVAL;
1903+ void *p = msg->front.iov_base;
1904+ void *end = p + msg->front.iov_len;
1905+
1906+ ceph_decode_need(&p, end, 2*sizeof(u32), bad);
1907+ next_mds = ceph_decode_32(&p);
1908+ fwd_seq = ceph_decode_32(&p);
1909+
1910+ mutex_lock(&mdsc->mutex);
1911+ req = __lookup_request(mdsc, tid);
1912+ if (!req) {
1913+ dout("forward %llu to mds%d - req dne\n", tid, next_mds);
1914+ goto out; /* dup reply? */
1915+ }
1916+
1917+ if (fwd_seq <= req->r_num_fwd) {
1918+ dout("forward %llu to mds%d - old seq %d <= %d\n",
1919+ tid, next_mds, req->r_num_fwd, fwd_seq);
1920+ } else {
1921+ /* resend. forward race not possible; mds would drop */
1922+ dout("forward %llu to mds%d (we resend)\n", tid, next_mds);
1923+ req->r_num_fwd = fwd_seq;
1924+ req->r_resend_mds = next_mds;
1925+ put_request_session(req);
1926+ __do_request(mdsc, req);
1927+ }
1928+ ceph_mdsc_put_request(req);
1929+out:
1930+ mutex_unlock(&mdsc->mutex);
1931+ return;
1932+
1933+bad:
1934+ pr_err("mdsc_handle_forward decode error err=%d\n", err);
1935+}
1936+
1937+/*
1938+ * handle a mds session control message
1939+ */
1940+static void handle_session(struct ceph_mds_session *session,
1941+ struct ceph_msg *msg)
1942+{
1943+ struct ceph_mds_client *mdsc = session->s_mdsc;
1944+ u32 op;
1945+ u64 seq;
1946+ int mds = session->s_mds;
1947+ struct ceph_mds_session_head *h = msg->front.iov_base;
1948+ int wake = 0;
1949+
1950+ /* decode */
1951+ if (msg->front.iov_len != sizeof(*h))
1952+ goto bad;
1953+ op = le32_to_cpu(h->op);
1954+ seq = le64_to_cpu(h->seq);
1955+
1956+ mutex_lock(&mdsc->mutex);
1957+ if (op == CEPH_SESSION_CLOSE)
1958+ __unregister_session(mdsc, session);
1959+ /* FIXME: this ttl calculation is generous */
1960+ session->s_ttl = jiffies + HZ*mdsc->mdsmap->m_session_autoclose;
1961+ mutex_unlock(&mdsc->mutex);
1962+
1963+ mutex_lock(&session->s_mutex);
1964+
1965+ dout("handle_session mds%d %s %p state %s seq %llu\n",
1966+ mds, ceph_session_op_name(op), session,
1967+ session_state_name(session->s_state), seq);
1968+
1969+ if (session->s_state == CEPH_MDS_SESSION_HUNG) {
1970+ session->s_state = CEPH_MDS_SESSION_OPEN;
1971+ pr_info("mds%d came back\n", session->s_mds);
1972+ }
1973+
1974+ switch (op) {
1975+ case CEPH_SESSION_OPEN:
1976+ session->s_state = CEPH_MDS_SESSION_OPEN;
1977+ renewed_caps(mdsc, session, 0);
1978+ wake = 1;
1979+ if (mdsc->stopping)
1980+ __close_session(mdsc, session);
1981+ break;
1982+
1983+ case CEPH_SESSION_RENEWCAPS:
1984+ if (session->s_renew_seq == seq)
1985+ renewed_caps(mdsc, session, 1);
1986+ break;
1987+
1988+ case CEPH_SESSION_CLOSE:
1989+ remove_session_caps(session);
1990+ wake = 1; /* for good measure */
1991+ complete(&mdsc->session_close_waiters);
1992+ kick_requests(mdsc, mds, 0); /* cur only */
1993+ break;
1994+
1995+ case CEPH_SESSION_STALE:
1996+ pr_info("mds%d caps went stale, renewing\n",
1997+ session->s_mds);
1998+ spin_lock(&session->s_cap_lock);
1999+ session->s_cap_gen++;
2000+ session->s_cap_ttl = 0;
2001+ spin_unlock(&session->s_cap_lock);
2002+ send_renew_caps(mdsc, session);
2003+ break;
2004+
2005+ case CEPH_SESSION_RECALL_STATE:
2006+ trim_caps(mdsc, session, le32_to_cpu(h->max_caps));
2007+ break;
2008+
2009+ default:
2010+ pr_err("mdsc_handle_session bad op %d mds%d\n", op, mds);
2011+ WARN_ON(1);
2012+ }
2013+
2014+ mutex_unlock(&session->s_mutex);
2015+ if (wake) {
2016+ mutex_lock(&mdsc->mutex);
2017+ __wake_requests(mdsc, &session->s_waiting);
2018+ mutex_unlock(&mdsc->mutex);
2019+ }
2020+ return;
2021+
2022+bad:
2023+ pr_err("mdsc_handle_session corrupt message mds%d len %d\n", mds,
2024+ (int)msg->front.iov_len);
2025+ ceph_msg_dump(msg);
2026+ return;
2027+}
2028+
2029+
2030+/*
2031+ * called under session->mutex.
2032+ */
2033+static void replay_unsafe_requests(struct ceph_mds_client *mdsc,
2034+ struct ceph_mds_session *session)
2035+{
2036+ struct ceph_mds_request *req, *nreq;
2037+ int err;
2038+
2039+ dout("replay_unsafe_requests mds%d\n", session->s_mds);
2040+
2041+ mutex_lock(&mdsc->mutex);
2042+ list_for_each_entry_safe(req, nreq, &session->s_unsafe, r_unsafe_item) {
2043+ err = __prepare_send_request(mdsc, req, session->s_mds);
2044+ if (!err) {
2045+ ceph_msg_get(req->r_request);
2046+ ceph_con_send(&session->s_con, req->r_request);
2047+ }
2048+ }
2049+ mutex_unlock(&mdsc->mutex);
2050+}
2051+
2052+/*
2053+ * Encode information about a cap for a reconnect with the MDS.
2054+ */
2055+static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap,
2056+ void *arg)
2057+{
2058+ struct ceph_mds_cap_reconnect rec;
2059+ struct ceph_inode_info *ci;
2060+ struct ceph_pagelist *pagelist = arg;
2061+ char *path;
2062+ int pathlen, err;
2063+ u64 pathbase;
2064+ struct dentry *dentry;
2065+
2066+ ci = cap->ci;
2067+
2068+ dout(" adding %p ino %llx.%llx cap %p %lld %s\n",
2069+ inode, ceph_vinop(inode), cap, cap->cap_id,
2070+ ceph_cap_string(cap->issued));
2071+ err = ceph_pagelist_encode_64(pagelist, ceph_ino(inode));
2072+ if (err)
2073+ return err;
2074+
2075+ dentry = d_find_alias(inode);
2076+ if (dentry) {
2077+ path = ceph_mdsc_build_path(dentry, &pathlen, &pathbase, 0);
2078+ if (IS_ERR(path)) {
2079+ err = PTR_ERR(path);
2080+ BUG_ON(err);
2081+ }
2082+ } else {
2083+ path = NULL;
2084+ pathlen = 0;
2085+ }
2086+ err = ceph_pagelist_encode_string(pagelist, path, pathlen);
2087+ if (err)
2088+ goto out;
2089+
2090+ spin_lock(&inode->i_lock);
2091+ cap->seq = 0; /* reset cap seq */
2092+ cap->issue_seq = 0; /* and issue_seq */
2093+ rec.cap_id = cpu_to_le64(cap->cap_id);
2094+ rec.pathbase = cpu_to_le64(pathbase);
2095+ rec.wanted = cpu_to_le32(__ceph_caps_wanted(ci));
2096+ rec.issued = cpu_to_le32(cap->issued);
2097+ rec.size = cpu_to_le64(inode->i_size);
2098+ ceph_encode_timespec(&rec.mtime, &inode->i_mtime);
2099+ ceph_encode_timespec(&rec.atime, &inode->i_atime);
2100+ rec.snaprealm = cpu_to_le64(ci->i_snap_realm->ino);
2101+ spin_unlock(&inode->i_lock);
2102+
2103+ err = ceph_pagelist_append(pagelist, &rec, sizeof(rec));
2104+
2105+out:
2106+ kfree(path);
2107+ dput(dentry);
2108+ return err;
2109+}
2110+
2111+
2112+/*
2113+ * If an MDS fails and recovers, clients need to reconnect in order to
2114+ * reestablish shared state. This includes all caps issued through
2115+ * this session _and_ the snap_realm hierarchy. Because it's not
2116+ * clear which snap realms the mds cares about, we send everything we
2117+ * know about.. that ensures we'll then get any new info the
2118+ * recovering MDS might have.
2119+ *
2120+ * This is a relatively heavyweight operation, but it's rare.
2121+ *
2122+ * called with mdsc->mutex held.
2123+ */
2124+static void send_mds_reconnect(struct ceph_mds_client *mdsc, int mds)
2125+{
2126+ struct ceph_mds_session *session = NULL;
2127+ struct ceph_msg *reply;
2128+ struct rb_node *p;
2129+ int err;
2130+ struct ceph_pagelist *pagelist;
2131+
2132+ pr_info("reconnect to recovering mds%d\n", mds);
2133+
2134+ pagelist = kmalloc(sizeof(*pagelist), GFP_NOFS);
2135+ if (!pagelist)
2136+ goto fail_nopagelist;
2137+ ceph_pagelist_init(pagelist);
2138+
2139+ reply = ceph_msg_new(CEPH_MSG_CLIENT_RECONNECT, 0, 0, 0, NULL);
2140+ if (IS_ERR(reply)) {
2141+ err = PTR_ERR(reply);
2142+ goto fail_nomsg;
2143+ }
2144+
2145+ /* find session */
2146+ session = __ceph_lookup_mds_session(mdsc, mds);
2147+ mutex_unlock(&mdsc->mutex); /* drop lock for duration */
2148+
2149+ if (session) {
2150+ mutex_lock(&session->s_mutex);
2151+
2152+ session->s_state = CEPH_MDS_SESSION_RECONNECTING;
2153+ session->s_seq = 0;
2154+
2155+ ceph_con_open(&session->s_con,
2156+ ceph_mdsmap_get_addr(mdsc->mdsmap, mds));
2157+
2158+ /* replay unsafe requests */
2159+ replay_unsafe_requests(mdsc, session);
2160+ } else {
2161+ dout("no session for mds%d, will send short reconnect\n",
2162+ mds);
2163+ }
2164+
2165+ down_read(&mdsc->snap_rwsem);
2166+
2167+ if (!session)
2168+ goto send;
2169+ dout("session %p state %s\n", session,
2170+ session_state_name(session->s_state));
2171+
2172+ /* traverse this session's caps */
2173+ err = ceph_pagelist_encode_32(pagelist, session->s_nr_caps);
2174+ if (err)
2175+ goto fail;
2176+ err = iterate_session_caps(session, encode_caps_cb, pagelist);
2177+ if (err < 0)
2178+ goto out;
2179+
2180+ /*
2181+ * snaprealms. we provide mds with the ino, seq (version), and
2182+ * parent for all of our realms. If the mds has any newer info,
2183+ * it will tell us.
2184+ */
2185+ for (p = rb_first(&mdsc->snap_realms); p; p = rb_next(p)) {
2186+ struct ceph_snap_realm *realm =
2187+ rb_entry(p, struct ceph_snap_realm, node);
2188+ struct ceph_mds_snaprealm_reconnect sr_rec;
2189+
2190+ dout(" adding snap realm %llx seq %lld parent %llx\n",
2191+ realm->ino, realm->seq, realm->parent_ino);
2192+ sr_rec.ino = cpu_to_le64(realm->ino);
2193+ sr_rec.seq = cpu_to_le64(realm->seq);
2194+ sr_rec.parent = cpu_to_le64(realm->parent_ino);
2195+ err = ceph_pagelist_append(pagelist, &sr_rec, sizeof(sr_rec));
2196+ if (err)
2197+ goto fail;
2198+ }
2199+
2200+send:
2201+ reply->pagelist = pagelist;
2202+ reply->hdr.data_len = cpu_to_le32(pagelist->length);
2203+ reply->nr_pages = calc_pages_for(0, pagelist->length);
2204+ ceph_con_send(&session->s_con, reply);
2205+
2206+ if (session) {
2207+ session->s_state = CEPH_MDS_SESSION_OPEN;
2208+ __wake_requests(mdsc, &session->s_waiting);
2209+ }
2210+
2211+out:
2212+ up_read(&mdsc->snap_rwsem);
2213+ if (session) {
2214+ mutex_unlock(&session->s_mutex);
2215+ ceph_put_mds_session(session);
2216+ }
2217+ mutex_lock(&mdsc->mutex);
2218+ return;
2219+
2220+fail:
2221+ ceph_msg_put(reply);
2222+fail_nomsg:
2223+ ceph_pagelist_release(pagelist);
2224+ kfree(pagelist);
2225+fail_nopagelist:
2226+ pr_err("ENOMEM preparing reconnect for mds%d\n", mds);
2227+ goto out;
2228+}
2229+
2230+
2231+/*
2232+ * compare old and new mdsmaps, kicking requests
2233+ * and closing out old connections as necessary
2234+ *
2235+ * called under mdsc->mutex.
2236+ */
2237+static void check_new_map(struct ceph_mds_client *mdsc,
2238+ struct ceph_mdsmap *newmap,
2239+ struct ceph_mdsmap *oldmap)
2240+{
2241+ int i;
2242+ int oldstate, newstate;
2243+ struct ceph_mds_session *s;
2244+
2245+ dout("check_new_map new %u old %u\n",
2246+ newmap->m_epoch, oldmap->m_epoch);
2247+
2248+ for (i = 0; i < oldmap->m_max_mds && i < mdsc->max_sessions; i++) {
2249+ if (mdsc->sessions[i] == NULL)
2250+ continue;
2251+ s = mdsc->sessions[i];
2252+ oldstate = ceph_mdsmap_get_state(oldmap, i);
2253+ newstate = ceph_mdsmap_get_state(newmap, i);
2254+
2255+ dout("check_new_map mds%d state %s -> %s (session %s)\n",
2256+ i, ceph_mds_state_name(oldstate),
2257+ ceph_mds_state_name(newstate),
2258+ session_state_name(s->s_state));
2259+
2260+ if (memcmp(ceph_mdsmap_get_addr(oldmap, i),
2261+ ceph_mdsmap_get_addr(newmap, i),
2262+ sizeof(struct ceph_entity_addr))) {
2263+ if (s->s_state == CEPH_MDS_SESSION_OPENING) {
2264+ /* the session never opened, just close it
2265+ * out now */
2266+ __wake_requests(mdsc, &s->s_waiting);
2267+ __unregister_session(mdsc, s);
2268+ } else {
2269+ /* just close it */
2270+ mutex_unlock(&mdsc->mutex);
2271+ mutex_lock(&s->s_mutex);
2272+ mutex_lock(&mdsc->mutex);
2273+ ceph_con_close(&s->s_con);
2274+ mutex_unlock(&s->s_mutex);
2275+ s->s_state = CEPH_MDS_SESSION_RESTARTING;
2276+ }
2277+
2278+ /* kick any requests waiting on the recovering mds */
2279+ kick_requests(mdsc, i, 1);
2280+ } else if (oldstate == newstate) {
2281+ continue; /* nothing new with this mds */
2282+ }
2283+
2284+ /*
2285+ * send reconnect?
2286+ */
2287+ if (s->s_state == CEPH_MDS_SESSION_RESTARTING &&
2288+ newstate >= CEPH_MDS_STATE_RECONNECT)
2289+ send_mds_reconnect(mdsc, i);
2290+
2291+ /*
2292+ * kick requests on any mds that has gone active.
2293+ *
2294+ * kick requests on cur or forwarder: we may have sent
2295+ * the request to mds1, mds1 told us it forwarded it
2296+ * to mds2, but then we learn mds1 failed and can't be
2297+ * sure it successfully forwarded our request before
2298+ * it died.
2299+ */
2300+ if (oldstate < CEPH_MDS_STATE_ACTIVE &&
2301+ newstate >= CEPH_MDS_STATE_ACTIVE) {
2302+ pr_info("mds%d reconnect completed\n", s->s_mds);
2303+ kick_requests(mdsc, i, 1);
2304+ ceph_kick_flushing_caps(mdsc, s);
2305+ wake_up_session_caps(s, 1);
2306+ }
2307+ }
2308+}
2309+
2310+
2311+
2312+/*
2313+ * leases
2314+ */
2315+
2316+/*
2317+ * caller must hold session s_mutex, dentry->d_lock
2318+ */
2319+void __ceph_mdsc_drop_dentry_lease(struct dentry *dentry)
2320+{
2321+ struct ceph_dentry_info *di = ceph_dentry(dentry);
2322+
2323+ ceph_put_mds_session(di->lease_session);
2324+ di->lease_session = NULL;
2325+}
2326+
2327+static void handle_lease(struct ceph_mds_client *mdsc,
2328+ struct ceph_mds_session *session,
2329+ struct ceph_msg *msg)
2330+{
2331+ struct super_block *sb = mdsc->client->sb;
2332+ struct inode *inode;
2333+ struct ceph_inode_info *ci;
2334+ struct dentry *parent, *dentry;
2335+ struct ceph_dentry_info *di;
2336+ int mds = session->s_mds;
2337+ struct ceph_mds_lease *h = msg->front.iov_base;
2338+ struct ceph_vino vino;
2339+ int mask;
2340+ struct qstr dname;
2341+ int release = 0;
2342+
2343+ dout("handle_lease from mds%d\n", mds);
2344+
2345+ /* decode */
2346+ if (msg->front.iov_len < sizeof(*h) + sizeof(u32))
2347+ goto bad;
2348+ vino.ino = le64_to_cpu(h->ino);
2349+ vino.snap = CEPH_NOSNAP;
2350+ mask = le16_to_cpu(h->mask);
2351+ dname.name = (void *)h + sizeof(*h) + sizeof(u32);
2352+ dname.len = msg->front.iov_len - sizeof(*h) - sizeof(u32);
2353+ if (dname.len != get_unaligned_le32(h+1))
2354+ goto bad;
2355+
2356+ mutex_lock(&session->s_mutex);
2357+ session->s_seq++;
2358+
2359+ /* lookup inode */
2360+ inode = ceph_find_inode(sb, vino);
2361+ dout("handle_lease '%s', mask %d, ino %llx %p\n",
2362+ ceph_lease_op_name(h->action), mask, vino.ino, inode);
2363+ if (inode == NULL) {
2364+ dout("handle_lease no inode %llx\n", vino.ino);
2365+ goto release;
2366+ }
2367+ ci = ceph_inode(inode);
2368+
2369+ /* dentry */
2370+ parent = d_find_alias(inode);
2371+ if (!parent) {
2372+ dout("no parent dentry on inode %p\n", inode);
2373+ WARN_ON(1);
2374+ goto release; /* hrm... */
2375+ }
2376+ dname.hash = full_name_hash(dname.name, dname.len);
2377+ dentry = d_lookup(parent, &dname);
2378+ dput(parent);
2379+ if (!dentry)
2380+ goto release;
2381+
2382+ spin_lock(&dentry->d_lock);
2383+ di = ceph_dentry(dentry);
2384+ switch (h->action) {
2385+ case CEPH_MDS_LEASE_REVOKE:
2386+ if (di && di->lease_session == session) {
2387+ h->seq = cpu_to_le32(di->lease_seq);
2388+ __ceph_mdsc_drop_dentry_lease(dentry);
2389+ }
2390+ release = 1;
2391+ break;
2392+
2393+ case CEPH_MDS_LEASE_RENEW:
2394+ if (di && di->lease_session == session &&
2395+ di->lease_gen == session->s_cap_gen &&
2396+ di->lease_renew_from &&
2397+ di->lease_renew_after == 0) {
2398+ unsigned long duration =
2399+ le32_to_cpu(h->duration_ms) * HZ / 1000;
2400+
2401+ di->lease_seq = le32_to_cpu(h->seq);
2402+ dentry->d_time = di->lease_renew_from + duration;
2403+ di->lease_renew_after = di->lease_renew_from +
2404+ (duration >> 1);
2405+ di->lease_renew_from = 0;
2406+ }
2407+ break;
2408+ }
2409+ spin_unlock(&dentry->d_lock);
2410+ dput(dentry);
2411+
2412+ if (!release)
2413+ goto out;
2414+
2415+release:
2416+ /* let's just reuse the same message */
2417+ h->action = CEPH_MDS_LEASE_REVOKE_ACK;
2418+ ceph_msg_get(msg);
2419+ ceph_con_send(&session->s_con, msg);
2420+
2421+out:
2422+ iput(inode);
2423+ mutex_unlock(&session->s_mutex);
2424+ return;
2425+
2426+bad:
2427+ pr_err("corrupt lease message\n");
2428+ ceph_msg_dump(msg);
2429+}
2430+
2431+void ceph_mdsc_lease_send_msg(struct ceph_mds_session *session,
2432+ struct inode *inode,
2433+ struct dentry *dentry, char action,
2434+ u32 seq)
2435+{
2436+ struct ceph_msg *msg;
2437+ struct ceph_mds_lease *lease;
2438+ int len = sizeof(*lease) + sizeof(u32);
2439+ int dnamelen = 0;
2440+
2441+ dout("lease_send_msg inode %p dentry %p %s to mds%d\n",
2442+ inode, dentry, ceph_lease_op_name(action), session->s_mds);
2443+ dnamelen = dentry->d_name.len;
2444+ len += dnamelen;
2445+
2446+ msg = ceph_msg_new(CEPH_MSG_CLIENT_LEASE, len, 0, 0, NULL);
2447+ if (IS_ERR(msg))
2448+ return;
2449+ lease = msg->front.iov_base;
2450+ lease->action = action;
2451+ lease->mask = cpu_to_le16(CEPH_LOCK_DN);
2452+ lease->ino = cpu_to_le64(ceph_vino(inode).ino);
2453+ lease->first = lease->last = cpu_to_le64(ceph_vino(inode).snap);
2454+ lease->seq = cpu_to_le32(seq);
2455+ put_unaligned_le32(dnamelen, lease + 1);
2456+ memcpy((void *)(lease + 1) + 4, dentry->d_name.name, dnamelen);
2457+
2458+ /*
2459+ * if this is a preemptive lease RELEASE, no need to
2460+ * flush request stream, since the actual request will
2461+ * soon follow.
2462+ */
2463+ msg->more_to_follow = (action == CEPH_MDS_LEASE_RELEASE);
2464+
2465+ ceph_con_send(&session->s_con, msg);
2466+}
2467+
2468+/*
2469+ * Preemptively release a lease we expect to invalidate anyway.
2470+ * Pass @inode always, @dentry is optional.
2471+ */
2472+void ceph_mdsc_lease_release(struct ceph_mds_client *mdsc, struct inode *inode,
2473+ struct dentry *dentry, int mask)
2474+{
2475+ struct ceph_dentry_info *di;
2476+ struct ceph_mds_session *session;
2477+ u32 seq;
2478+
2479+ BUG_ON(inode == NULL);
2480+ BUG_ON(dentry == NULL);
2481+ BUG_ON(mask != CEPH_LOCK_DN);
2482+
2483+ /* is dentry lease valid? */
2484+ spin_lock(&dentry->d_lock);
2485+ di = ceph_dentry(dentry);
2486+ if (!di || !di->lease_session ||
2487+ di->lease_session->s_mds < 0 ||
2488+ di->lease_gen != di->lease_session->s_cap_gen ||
2489+ !time_before(jiffies, dentry->d_time)) {
2490+ dout("lease_release inode %p dentry %p -- "
2491+ "no lease on %d\n",
2492+ inode, dentry, mask);
2493+ spin_unlock(&dentry->d_lock);
2494+ return;
2495+ }
2496+
2497+ /* we do have a lease on this dentry; note mds and seq */
2498+ session = ceph_get_mds_session(di->lease_session);
2499+ seq = di->lease_seq;
2500+ __ceph_mdsc_drop_dentry_lease(dentry);
2501+ spin_unlock(&dentry->d_lock);
2502+
2503+ dout("lease_release inode %p dentry %p mask %d to mds%d\n",
2504+ inode, dentry, mask, session->s_mds);
2505+ ceph_mdsc_lease_send_msg(session, inode, dentry,
2506+ CEPH_MDS_LEASE_RELEASE, seq);
2507+ ceph_put_mds_session(session);
2508+}
2509+
2510+/*
2511+ * drop all leases (and dentry refs) in preparation for umount
2512+ */
2513+static void drop_leases(struct ceph_mds_client *mdsc)
2514+{
2515+ int i;
2516+
2517+ dout("drop_leases\n");
2518+ mutex_lock(&mdsc->mutex);
2519+ for (i = 0; i < mdsc->max_sessions; i++) {
2520+ struct ceph_mds_session *s = __ceph_lookup_mds_session(mdsc, i);
2521+ if (!s)
2522+ continue;
2523+ mutex_unlock(&mdsc->mutex);
2524+ mutex_lock(&s->s_mutex);
2525+ mutex_unlock(&s->s_mutex);
2526+ ceph_put_mds_session(s);
2527+ mutex_lock(&mdsc->mutex);
2528+ }
2529+ mutex_unlock(&mdsc->mutex);
2530+}
2531+
2532+
2533+
2534+/*
2535+ * delayed work -- periodically trim expired leases, renew caps with mds
2536+ */
2537+static void schedule_delayed(struct ceph_mds_client *mdsc)
2538+{
2539+ int delay = 5;
2540+ unsigned hz = round_jiffies_relative(HZ * delay);
2541+ schedule_delayed_work(&mdsc->delayed_work, hz);
2542+}
2543+
2544+static void delayed_work(struct work_struct *work)
2545+{
2546+ int i;
2547+ struct ceph_mds_client *mdsc =
2548+ container_of(work, struct ceph_mds_client, delayed_work.work);
2549+ int renew_interval;
2550+ int renew_caps;
2551+
2552+ dout("mdsc delayed_work\n");
2553+ ceph_check_delayed_caps(mdsc);
2554+
2555+ mutex_lock(&mdsc->mutex);
2556+ renew_interval = mdsc->mdsmap->m_session_timeout >> 2;
2557+ renew_caps = time_after_eq(jiffies, HZ*renew_interval +
2558+ mdsc->last_renew_caps);
2559+ if (renew_caps)
2560+ mdsc->last_renew_caps = jiffies;
2561+
2562+ for (i = 0; i < mdsc->max_sessions; i++) {
2563+ struct ceph_mds_session *s = __ceph_lookup_mds_session(mdsc, i);
2564+ if (s == NULL)
2565+ continue;
2566+ if (s->s_state == CEPH_MDS_SESSION_CLOSING) {
2567+ dout("resending session close request for mds%d\n",
2568+ s->s_mds);
2569+ request_close_session(mdsc, s);
2570+ ceph_put_mds_session(s);
2571+ continue;
2572+ }
2573+ if (s->s_ttl && time_after(jiffies, s->s_ttl)) {
2574+ if (s->s_state == CEPH_MDS_SESSION_OPEN) {
2575+ s->s_state = CEPH_MDS_SESSION_HUNG;
2576+ pr_info("mds%d hung\n", s->s_mds);
2577+ }
2578+ }
2579+ if (s->s_state < CEPH_MDS_SESSION_OPEN) {
2580+ /* this mds is failed or recovering, just wait */
2581+ ceph_put_mds_session(s);
2582+ continue;
2583+ }
2584+ mutex_unlock(&mdsc->mutex);
2585+
2586+ mutex_lock(&s->s_mutex);
2587+ if (renew_caps)
2588+ send_renew_caps(mdsc, s);
2589+ else
2590+ ceph_con_keepalive(&s->s_con);
2591+ add_cap_releases(mdsc, s, -1);
2592+ send_cap_releases(mdsc, s);
2593+ mutex_unlock(&s->s_mutex);
2594+ ceph_put_mds_session(s);
2595+
2596+ mutex_lock(&mdsc->mutex);
2597+ }
2598+ mutex_unlock(&mdsc->mutex);
2599+
2600+ schedule_delayed(mdsc);
2601+}
2602+
2603+
2604+int ceph_mdsc_init(struct ceph_mds_client *mdsc, struct ceph_client *client)
2605+{
2606+ mdsc->client = client;
2607+ mutex_init(&mdsc->mutex);
2608+ mdsc->mdsmap = kzalloc(sizeof(*mdsc->mdsmap), GFP_NOFS);
2609+ init_completion(&mdsc->safe_umount_waiters);
2610+ init_completion(&mdsc->session_close_waiters);
2611+ INIT_LIST_HEAD(&mdsc->waiting_for_map);
2612+ mdsc->sessions = NULL;
2613+ mdsc->max_sessions = 0;
2614+ mdsc->stopping = 0;
2615+ init_rwsem(&mdsc->snap_rwsem);
2616+ mdsc->snap_realms = RB_ROOT;
2617+ INIT_LIST_HEAD(&mdsc->snap_empty);
2618+ spin_lock_init(&mdsc->snap_empty_lock);
2619+ mdsc->last_tid = 0;
2620+ mdsc->request_tree = RB_ROOT;
2621+ INIT_DELAYED_WORK(&mdsc->delayed_work, delayed_work);
2622+ mdsc->last_renew_caps = jiffies;
2623+ INIT_LIST_HEAD(&mdsc->cap_delay_list);
2624+ spin_lock_init(&mdsc->cap_delay_lock);
2625+ INIT_LIST_HEAD(&mdsc->snap_flush_list);
2626+ spin_lock_init(&mdsc->snap_flush_lock);
2627+ mdsc->cap_flush_seq = 0;
2628+ INIT_LIST_HEAD(&mdsc->cap_dirty);
2629+ mdsc->num_cap_flushing = 0;
2630+ spin_lock_init(&mdsc->cap_dirty_lock);
2631+ init_waitqueue_head(&mdsc->cap_flushing_wq);
2632+ spin_lock_init(&mdsc->dentry_lru_lock);
2633+ INIT_LIST_HEAD(&mdsc->dentry_lru);
2634+ return 0;
2635+}
2636+
2637+/*
2638+ * Wait for safe replies on open mds requests. If we time out, drop
2639+ * all requests from the tree to avoid dangling dentry refs.
2640+ */
2641+static void wait_requests(struct ceph_mds_client *mdsc)
2642+{
2643+ struct ceph_mds_request *req;
2644+ struct ceph_client *client = mdsc->client;
2645+
2646+ mutex_lock(&mdsc->mutex);
2647+ if (__get_oldest_req(mdsc)) {
2648+ mutex_unlock(&mdsc->mutex);
2649+
2650+ dout("wait_requests waiting for requests\n");
2651+ wait_for_completion_timeout(&mdsc->safe_umount_waiters,
2652+ client->mount_args->mount_timeout * HZ);
2653+
2654+ /* tear down remaining requests */
2655+ mutex_lock(&mdsc->mutex);
2656+ while ((req = __get_oldest_req(mdsc))) {
2657+ dout("wait_requests timed out on tid %llu\n",
2658+ req->r_tid);
2659+ __unregister_request(mdsc, req);
2660+ }
2661+ }
2662+ mutex_unlock(&mdsc->mutex);
2663+ dout("wait_requests done\n");
2664+}
2665+
2666+/*
2667+ * called before mount is ro, and before dentries are torn down.
2668+ * (hmm, does this still race with new lookups?)
2669+ */
2670+void ceph_mdsc_pre_umount(struct ceph_mds_client *mdsc)
2671+{
2672+ dout("pre_umount\n");
2673+ mdsc->stopping = 1;
2674+
2675+ drop_leases(mdsc);
2676+ ceph_flush_dirty_caps(mdsc);
2677+ wait_requests(mdsc);
2678+}
2679+
2680+/*
2681+ * wait for all write mds requests to flush.
2682+ */
2683+static void wait_unsafe_requests(struct ceph_mds_client *mdsc, u64 want_tid)
2684+{
2685+ struct ceph_mds_request *req = NULL;
2686+ struct rb_node *n;
2687+
2688+ mutex_lock(&mdsc->mutex);
2689+ dout("wait_unsafe_requests want %lld\n", want_tid);
2690+ req = __get_oldest_req(mdsc);
2691+ while (req && req->r_tid <= want_tid) {
2692+ if ((req->r_op & CEPH_MDS_OP_WRITE)) {
2693+ /* write op */
2694+ ceph_mdsc_get_request(req);
2695+ mutex_unlock(&mdsc->mutex);
2696+ dout("wait_unsafe_requests wait on %llu (want %llu)\n",
2697+ req->r_tid, want_tid);
2698+ wait_for_completion(&req->r_safe_completion);
2699+ mutex_lock(&mdsc->mutex);
2700+ n = rb_next(&req->r_node);
2701+ ceph_mdsc_put_request(req);
2702+ } else {
2703+ n = rb_next(&req->r_node);
2704+ }
2705+ if (!n)
2706+ break;
2707+ req = rb_entry(n, struct ceph_mds_request, r_node);
2708+ }
2709+ mutex_unlock(&mdsc->mutex);
2710+ dout("wait_unsafe_requests done\n");
2711+}
2712+
2713+void ceph_mdsc_sync(struct ceph_mds_client *mdsc)
2714+{
2715+ u64 want_tid, want_flush;
2716+
2717+ dout("sync\n");
2718+ mutex_lock(&mdsc->mutex);
2719+ want_tid = mdsc->last_tid;
2720+ want_flush = mdsc->cap_flush_seq;
2721+ mutex_unlock(&mdsc->mutex);
2722+ dout("sync want tid %lld flush_seq %lld\n", want_tid, want_flush);
2723+
2724+ ceph_flush_dirty_caps(mdsc);
2725+
2726+ wait_unsafe_requests(mdsc, want_tid);
2727+ wait_event(mdsc->cap_flushing_wq, check_cap_flush(mdsc, want_flush));
2728+}
2729+
2730+
2731+/*
2732+ * called after sb is ro.
2733+ */
2734+void ceph_mdsc_close_sessions(struct ceph_mds_client *mdsc)
2735+{
2736+ struct ceph_mds_session *session;
2737+ int i;
2738+ int n;
2739+ struct ceph_client *client = mdsc->client;
2740+ unsigned long started, timeout = client->mount_args->mount_timeout * HZ;
2741+
2742+ dout("close_sessions\n");
2743+
2744+ mutex_lock(&mdsc->mutex);
2745+
2746+ /* close sessions */
2747+ started = jiffies;
2748+ while (time_before(jiffies, started + timeout)) {
2749+ dout("closing sessions\n");
2750+ n = 0;
2751+ for (i = 0; i < mdsc->max_sessions; i++) {
2752+ session = __ceph_lookup_mds_session(mdsc, i);
2753+ if (!session)
2754+ continue;
2755+ mutex_unlock(&mdsc->mutex);
2756+ mutex_lock(&session->s_mutex);
2757+ __close_session(mdsc, session);
2758+ mutex_unlock(&session->s_mutex);
2759+ ceph_put_mds_session(session);
2760+ mutex_lock(&mdsc->mutex);
2761+ n++;
2762+ }
2763+ if (n == 0)
2764+ break;
2765+
2766+ if (client->mount_state == CEPH_MOUNT_SHUTDOWN)
2767+ break;
2768+
2769+ dout("waiting for sessions to close\n");
2770+ mutex_unlock(&mdsc->mutex);
2771+ wait_for_completion_timeout(&mdsc->session_close_waiters,
2772+ timeout);
2773+ mutex_lock(&mdsc->mutex);
2774+ }
2775+
2776+ /* tear down remaining sessions */
2777+ for (i = 0; i < mdsc->max_sessions; i++) {
2778+ if (mdsc->sessions[i]) {
2779+ session = get_session(mdsc->sessions[i]);
2780+ __unregister_session(mdsc, session);
2781+ mutex_unlock(&mdsc->mutex);
2782+ mutex_lock(&session->s_mutex);
2783+ remove_session_caps(session);
2784+ mutex_unlock(&session->s_mutex);
2785+ ceph_put_mds_session(session);
2786+ mutex_lock(&mdsc->mutex);
2787+ }
2788+ }
2789+
2790+ WARN_ON(!list_empty(&mdsc->cap_delay_list));
2791+
2792+ mutex_unlock(&mdsc->mutex);
2793+
2794+ ceph_cleanup_empty_realms(mdsc);
2795+
2796+ cancel_delayed_work_sync(&mdsc->delayed_work); /* cancel timer */
2797+
2798+ dout("stopped\n");
2799+}
2800+
2801+void ceph_mdsc_stop(struct ceph_mds_client *mdsc)
2802+{
2803+ dout("stop\n");
2804+ cancel_delayed_work_sync(&mdsc->delayed_work); /* cancel timer */
2805+ if (mdsc->mdsmap)
2806+ ceph_mdsmap_destroy(mdsc->mdsmap);
2807+ kfree(mdsc->sessions);
2808+}
2809+
2810+
2811+/*
2812+ * handle mds map update.
2813+ */
2814+void ceph_mdsc_handle_map(struct ceph_mds_client *mdsc, struct ceph_msg *msg)
2815+{
2816+ u32 epoch;
2817+ u32 maplen;
2818+ void *p = msg->front.iov_base;
2819+ void *end = p + msg->front.iov_len;
2820+ struct ceph_mdsmap *newmap, *oldmap;
2821+ struct ceph_fsid fsid;
2822+ int err = -EINVAL;
2823+
2824+ ceph_decode_need(&p, end, sizeof(fsid)+2*sizeof(u32), bad);
2825+ ceph_decode_copy(&p, &fsid, sizeof(fsid));
2826+ if (ceph_check_fsid(mdsc->client, &fsid) < 0)
2827+ return;
2828+ epoch = ceph_decode_32(&p);
2829+ maplen = ceph_decode_32(&p);
2830+ dout("handle_map epoch %u len %d\n", epoch, (int)maplen);
2831+
2832+ /* do we need it? */
2833+ ceph_monc_got_mdsmap(&mdsc->client->monc, epoch);
2834+ mutex_lock(&mdsc->mutex);
2835+ if (mdsc->mdsmap && epoch <= mdsc->mdsmap->m_epoch) {
2836+ dout("handle_map epoch %u <= our %u\n",
2837+ epoch, mdsc->mdsmap->m_epoch);
2838+ mutex_unlock(&mdsc->mutex);
2839+ return;
2840+ }
2841+
2842+ newmap = ceph_mdsmap_decode(&p, end);
2843+ if (IS_ERR(newmap)) {
2844+ err = PTR_ERR(newmap);
2845+ goto bad_unlock;
2846+ }
2847+
2848+ /* swap into place */
2849+ if (mdsc->mdsmap) {
2850+ oldmap = mdsc->mdsmap;
2851+ mdsc->mdsmap = newmap;
2852+ check_new_map(mdsc, newmap, oldmap);
2853+ ceph_mdsmap_destroy(oldmap);
2854+ } else {
2855+ mdsc->mdsmap = newmap; /* first mds map */
2856+ }
2857+ mdsc->client->sb->s_maxbytes = mdsc->mdsmap->m_max_file_size;
2858+
2859+ __wake_requests(mdsc, &mdsc->waiting_for_map);
2860+
2861+ mutex_unlock(&mdsc->mutex);
2862+ schedule_delayed(mdsc);
2863+ return;
2864+
2865+bad_unlock:
2866+ mutex_unlock(&mdsc->mutex);
2867+bad:
2868+ pr_err("error decoding mdsmap %d\n", err);
2869+ return;
2870+}
2871+
2872+static struct ceph_connection *con_get(struct ceph_connection *con)
2873+{
2874+ struct ceph_mds_session *s = con->private;
2875+
2876+ if (get_session(s)) {
2877+ dout("mdsc con_get %p ok (%d)\n", s, atomic_read(&s->s_ref));
2878+ return con;
2879+ }
2880+ dout("mdsc con_get %p FAIL\n", s);
2881+ return NULL;
2882+}
2883+
2884+static void con_put(struct ceph_connection *con)
2885+{
2886+ struct ceph_mds_session *s = con->private;
2887+
2888+ ceph_put_mds_session(s);
2889+ dout("mdsc con_put %p (%d)\n", s, atomic_read(&s->s_ref));
2890+}
2891+
2892+/*
2893+ * if the client is unresponsive for long enough, the mds will kill
2894+ * the session entirely.
2895+ */
2896+static void peer_reset(struct ceph_connection *con)
2897+{
2898+ struct ceph_mds_session *s = con->private;
2899+
2900+ pr_err("mds%d gave us the boot. IMPLEMENT RECONNECT.\n",
2901+ s->s_mds);
2902+}
2903+
2904+static void dispatch(struct ceph_connection *con, struct ceph_msg *msg)
2905+{
2906+ struct ceph_mds_session *s = con->private;
2907+ struct ceph_mds_client *mdsc = s->s_mdsc;
2908+ int type = le16_to_cpu(msg->hdr.type);
2909+
2910+ mutex_lock(&mdsc->mutex);
2911+ if (__verify_registered_session(mdsc, s) < 0) {
2912+ mutex_unlock(&mdsc->mutex);
2913+ goto out;
2914+ }
2915+ mutex_unlock(&mdsc->mutex);
2916+
2917+ switch (type) {
2918+ case CEPH_MSG_MDS_MAP:
2919+ ceph_mdsc_handle_map(mdsc, msg);
2920+ break;
2921+ case CEPH_MSG_CLIENT_SESSION:
2922+ handle_session(s, msg);
2923+ break;
2924+ case CEPH_MSG_CLIENT_REPLY:
2925+ handle_reply(s, msg);
2926+ break;
2927+ case CEPH_MSG_CLIENT_REQUEST_FORWARD:
2928+ handle_forward(mdsc, s, msg);
2929+ break;
2930+ case CEPH_MSG_CLIENT_CAPS:
2931+ ceph_handle_caps(s, msg);
2932+ break;
2933+ case CEPH_MSG_CLIENT_SNAP:
2934+ ceph_handle_snap(mdsc, s, msg);
2935+ break;
2936+ case CEPH_MSG_CLIENT_LEASE:
2937+ handle_lease(mdsc, s, msg);
2938+ break;
2939+
2940+ default:
2941+ pr_err("received unknown message type %d %s\n", type,
2942+ ceph_msg_type_name(type));
2943+ }
2944+out:
2945+ ceph_msg_put(msg);
2946+}
2947+
2948+/*
2949+ * authentication
2950+ */
2951+static int get_authorizer(struct ceph_connection *con,
2952+ void **buf, int *len, int *proto,
2953+ void **reply_buf, int *reply_len, int force_new)
2954+{
2955+ struct ceph_mds_session *s = con->private;
2956+ struct ceph_mds_client *mdsc = s->s_mdsc;
2957+ struct ceph_auth_client *ac = mdsc->client->monc.auth;
2958+ int ret = 0;
2959+
2960+ if (force_new && s->s_authorizer) {
2961+ ac->ops->destroy_authorizer(ac, s->s_authorizer);
2962+ s->s_authorizer = NULL;
2963+ }
2964+ if (s->s_authorizer == NULL) {
2965+ if (ac->ops->create_authorizer) {
2966+ ret = ac->ops->create_authorizer(
2967+ ac, CEPH_ENTITY_TYPE_MDS,
2968+ &s->s_authorizer,
2969+ &s->s_authorizer_buf,
2970+ &s->s_authorizer_buf_len,
2971+ &s->s_authorizer_reply_buf,
2972+ &s->s_authorizer_reply_buf_len);
2973+ if (ret)
2974+ return ret;
2975+ }
2976+ }
2977+
2978+ *proto = ac->protocol;
2979+ *buf = s->s_authorizer_buf;
2980+ *len = s->s_authorizer_buf_len;
2981+ *reply_buf = s->s_authorizer_reply_buf;
2982+ *reply_len = s->s_authorizer_reply_buf_len;
2983+ return 0;
2984+}
2985+
2986+
2987+static int verify_authorizer_reply(struct ceph_connection *con, int len)
2988+{
2989+ struct ceph_mds_session *s = con->private;
2990+ struct ceph_mds_client *mdsc = s->s_mdsc;
2991+ struct ceph_auth_client *ac = mdsc->client->monc.auth;
2992+
2993+ return ac->ops->verify_authorizer_reply(ac, s->s_authorizer, len);
2994+}
2995+
2996+static int invalidate_authorizer(struct ceph_connection *con)
2997+{
2998+ struct ceph_mds_session *s = con->private;
2999+ struct ceph_mds_client *mdsc = s->s_mdsc;
3000+ struct ceph_auth_client *ac = mdsc->client->monc.auth;
3001+
3002+ if (ac->ops->invalidate_authorizer)
3003+ ac->ops->invalidate_authorizer(ac, CEPH_ENTITY_TYPE_MDS);
3004+
3005+ return ceph_monc_validate_auth(&mdsc->client->monc);
3006+}
3007+
3008+const static struct ceph_connection_operations mds_con_ops = {
3009+ .get = con_get,
3010+ .put = con_put,
3011+ .dispatch = dispatch,
3012+ .get_authorizer = get_authorizer,
3013+ .verify_authorizer_reply = verify_authorizer_reply,
3014+ .invalidate_authorizer = invalidate_authorizer,
3015+ .peer_reset = peer_reset,
3016+};
3017+
3018+
3019+
3020+
3021+/* eof */
fs/ceph/mds_client.h+335−0
@@ -0,0 +1,335 @@
1+#ifndef _FS_CEPH_MDS_CLIENT_H
2+#define _FS_CEPH_MDS_CLIENT_H
3+
4+#include <linux/completion.h>
5+#include <linux/kref.h>
6+#include <linux/list.h>
7+#include <linux/mutex.h>
8+#include <linux/rbtree.h>
9+#include <linux/spinlock.h>
10+
11+#include "types.h"
12+#include "messenger.h"
13+#include "mdsmap.h"
14+
15+/*
16+ * Some lock dependencies:
17+ *
18+ * session->s_mutex
19+ * mdsc->mutex
20+ *
21+ * mdsc->snap_rwsem
22+ *
23+ * inode->i_lock
24+ * mdsc->snap_flush_lock
25+ * mdsc->cap_delay_lock
26+ *
27+ */
28+
29+struct ceph_client;
30+struct ceph_cap;
31+
32+/*
33+ * parsed info about a single inode. pointers are into the encoded
34+ * on-wire structures within the mds reply message payload.
35+ */
36+struct ceph_mds_reply_info_in {
37+ struct ceph_mds_reply_inode *in;
38+ u32 symlink_len;
39+ char *symlink;
40+ u32 xattr_len;
41+ char *xattr_data;
42+};
43+
44+/*
45+ * parsed info about an mds reply, including information about the
46+ * target inode and/or its parent directory and dentry, and directory
47+ * contents (for readdir results).
48+ */
49+struct ceph_mds_reply_info_parsed {
50+ struct ceph_mds_reply_head *head;
51+
52+ struct ceph_mds_reply_info_in diri, targeti;
53+ struct ceph_mds_reply_dirfrag *dirfrag;
54+ char *dname;
55+ u32 dname_len;
56+ struct ceph_mds_reply_lease *dlease;
57+
58+ struct ceph_mds_reply_dirfrag *dir_dir;
59+ int dir_nr;
60+ char **dir_dname;
61+ u32 *dir_dname_len;
62+ struct ceph_mds_reply_lease **dir_dlease;
63+ struct ceph_mds_reply_info_in *dir_in;
64+ u8 dir_complete, dir_end;
65+
66+ /* encoded blob describing snapshot contexts for certain
67+ operations (e.g., open) */
68+ void *snapblob;
69+ int snapblob_len;
70+};
71+
72+
73+/*
74+ * cap releases are batched and sent to the MDS en masse.
75+ */
76+#define CEPH_CAPS_PER_RELEASE ((PAGE_CACHE_SIZE - \
77+ sizeof(struct ceph_mds_cap_release)) / \
78+ sizeof(struct ceph_mds_cap_item))
79+
80+
81+/*
82+ * state associated with each MDS<->client session
83+ */
84+enum {
85+ CEPH_MDS_SESSION_NEW = 1,
86+ CEPH_MDS_SESSION_OPENING = 2,
87+ CEPH_MDS_SESSION_OPEN = 3,
88+ CEPH_MDS_SESSION_HUNG = 4,
89+ CEPH_MDS_SESSION_CLOSING = 5,
90+ CEPH_MDS_SESSION_RESTARTING = 6,
91+ CEPH_MDS_SESSION_RECONNECTING = 7,
92+};
93+
94+struct ceph_mds_session {
95+ struct ceph_mds_client *s_mdsc;
96+ int s_mds;
97+ int s_state;
98+ unsigned long s_ttl; /* time until mds kills us */
99+ u64 s_seq; /* incoming msg seq # */
100+ struct mutex s_mutex; /* serialize session messages */
101+
102+ struct ceph_connection s_con;
103+
104+ struct ceph_authorizer *s_authorizer;
105+ void *s_authorizer_buf, *s_authorizer_reply_buf;
106+ size_t s_authorizer_buf_len, s_authorizer_reply_buf_len;
107+
108+ /* protected by s_cap_lock */
109+ spinlock_t s_cap_lock;
110+ u32 s_cap_gen; /* inc each time we get mds stale msg */
111+ unsigned long s_cap_ttl; /* when session caps expire */
112+ struct list_head s_caps; /* all caps issued by this session */
113+ int s_nr_caps, s_trim_caps;
114+ int s_num_cap_releases;
115+ struct list_head s_cap_releases; /* waiting cap_release messages */
116+ struct list_head s_cap_releases_done; /* ready to send */
117+ struct ceph_cap *s_cap_iterator;
118+
119+ /* protected by mutex */
120+ struct list_head s_cap_flushing; /* inodes w/ flushing caps */
121+ struct list_head s_cap_snaps_flushing;
122+ unsigned long s_renew_requested; /* last time we sent a renew req */
123+ u64 s_renew_seq;
124+
125+ atomic_t s_ref;
126+ struct list_head s_waiting; /* waiting requests */
127+ struct list_head s_unsafe; /* unsafe requests */
128+};
129+
130+/*
131+ * modes of choosing which MDS to send a request to
132+ */
133+enum {
134+ USE_ANY_MDS,
135+ USE_RANDOM_MDS,
136+ USE_AUTH_MDS, /* prefer authoritative mds for this metadata item */
137+};
138+
139+struct ceph_mds_request;
140+struct ceph_mds_client;
141+
142+/*
143+ * request completion callback
144+ */
145+typedef void (*ceph_mds_request_callback_t) (struct ceph_mds_client *mdsc,
146+ struct ceph_mds_request *req);
147+
148+/*
149+ * an in-flight mds request
150+ */
151+struct ceph_mds_request {
152+ u64 r_tid; /* transaction id */
153+ struct rb_node r_node;
154+
155+ int r_op; /* mds op code */
156+ int r_mds;
157+
158+ /* operation on what? */
159+ struct inode *r_inode; /* arg1 */
160+ struct dentry *r_dentry; /* arg1 */
161+ struct dentry *r_old_dentry; /* arg2: rename from or link from */
162+ char *r_path1, *r_path2;
163+ struct ceph_vino r_ino1, r_ino2;
164+
165+ struct inode *r_locked_dir; /* dir (if any) i_mutex locked by vfs */
166+ struct inode *r_target_inode; /* resulting inode */
167+
168+ union ceph_mds_request_args r_args;
169+ int r_fmode; /* file mode, if expecting cap */
170+
171+ /* for choosing which mds to send this request to */
172+ int r_direct_mode;
173+ u32 r_direct_hash; /* choose dir frag based on this dentry hash */
174+ bool r_direct_is_hash; /* true if r_direct_hash is valid */
175+
176+ /* data payload is used for xattr ops */
177+ struct page **r_pages;
178+ int r_num_pages;
179+ int r_data_len;
180+
181+ /* what caps shall we drop? */
182+ int r_inode_drop, r_inode_unless;
183+ int r_dentry_drop, r_dentry_unless;
184+ int r_old_dentry_drop, r_old_dentry_unless;
185+ struct inode *r_old_inode;
186+ int r_old_inode_drop, r_old_inode_unless;
187+
188+ struct ceph_msg *r_request; /* original request */
189+ struct ceph_msg *r_reply;
190+ struct ceph_mds_reply_info_parsed r_reply_info;
191+ int r_err;
192+ bool r_aborted;
193+
194+ unsigned long r_timeout; /* optional. jiffies */
195+ unsigned long r_started; /* start time to measure timeout against */
196+ unsigned long r_request_started; /* start time for mds request only,
197+ used to measure lease durations */
198+
199+ /* link unsafe requests to parent directory, for fsync */
200+ struct inode *r_unsafe_dir;
201+ struct list_head r_unsafe_dir_item;
202+
203+ struct ceph_mds_session *r_session;
204+
205+ int r_attempts; /* resend attempts */
206+ int r_num_fwd; /* number of forward attempts */
207+ int r_num_stale;
208+ int r_resend_mds; /* mds to resend to next, if any*/
209+
210+ struct kref r_kref;
211+ struct list_head r_wait;
212+ struct completion r_completion;
213+ struct completion r_safe_completion;
214+ ceph_mds_request_callback_t r_callback;
215+ struct list_head r_unsafe_item; /* per-session unsafe list item */
216+ bool r_got_unsafe, r_got_safe;
217+
218+ bool r_did_prepopulate;
219+ u32 r_readdir_offset;
220+
221+ struct ceph_cap_reservation r_caps_reservation;
222+ int r_num_caps;
223+};
224+
225+/*
226+ * mds client state
227+ */
228+struct ceph_mds_client {
229+ struct ceph_client *client;
230+ struct mutex mutex; /* all nested structures */
231+
232+ struct ceph_mdsmap *mdsmap;
233+ struct completion safe_umount_waiters, session_close_waiters;
234+ struct list_head waiting_for_map;
235+
236+ struct ceph_mds_session **sessions; /* NULL for mds if no session */
237+ int max_sessions; /* len of s_mds_sessions */
238+ int stopping; /* true if shutting down */
239+
240+ /*
241+ * snap_rwsem will cover cap linkage into snaprealms, and
242+ * realm snap contexts. (later, we can do per-realm snap
243+ * contexts locks..) the empty list contains realms with no
244+ * references (implying they contain no inodes with caps) that
245+ * should be destroyed.
246+ */
247+ struct rw_semaphore snap_rwsem;
248+ struct rb_root snap_realms;
249+ struct list_head snap_empty;
250+ spinlock_t snap_empty_lock; /* protect snap_empty */
251+
252+ u64 last_tid; /* most recent mds request */
253+ struct rb_root request_tree; /* pending mds requests */
254+ struct delayed_work delayed_work; /* delayed work */
255+ unsigned long last_renew_caps; /* last time we renewed our caps */
256+ struct list_head cap_delay_list; /* caps with delayed release */
257+ spinlock_t cap_delay_lock; /* protects cap_delay_list */
258+ struct list_head snap_flush_list; /* cap_snaps ready to flush */
259+ spinlock_t snap_flush_lock;
260+
261+ u64 cap_flush_seq;
262+ struct list_head cap_dirty; /* inodes with dirty caps */
263+ int num_cap_flushing; /* # caps we are flushing */
264+ spinlock_t cap_dirty_lock; /* protects above items */
265+ wait_queue_head_t cap_flushing_wq;
266+
267+#ifdef CONFIG_DEBUG_FS
268+ struct dentry *debugfs_file;
269+#endif
270+
271+ spinlock_t dentry_lru_lock;
272+ struct list_head dentry_lru;
273+ int num_dentry;
274+};
275+
276+extern const char *ceph_mds_op_name(int op);
277+
278+extern struct ceph_mds_session *
279+__ceph_lookup_mds_session(struct ceph_mds_client *, int mds);
280+
281+static inline struct ceph_mds_session *
282+ceph_get_mds_session(struct ceph_mds_session *s)
283+{
284+ atomic_inc(&s->s_ref);
285+ return s;
286+}
287+
288+extern void ceph_put_mds_session(struct ceph_mds_session *s);
289+
290+extern int ceph_send_msg_mds(struct ceph_mds_client *mdsc,
291+ struct ceph_msg *msg, int mds);
292+
293+extern int ceph_mdsc_init(struct ceph_mds_client *mdsc,
294+ struct ceph_client *client);
295+extern void ceph_mdsc_close_sessions(struct ceph_mds_client *mdsc);
296+extern void ceph_mdsc_stop(struct ceph_mds_client *mdsc);
297+
298+extern void ceph_mdsc_sync(struct ceph_mds_client *mdsc);
299+
300+extern void ceph_mdsc_lease_release(struct ceph_mds_client *mdsc,
301+ struct inode *inode,
302+ struct dentry *dn, int mask);
303+
304+extern struct ceph_mds_request *
305+ceph_mdsc_create_request(struct ceph_mds_client *mdsc, int op, int mode);
306+extern void ceph_mdsc_submit_request(struct ceph_mds_client *mdsc,
307+ struct ceph_mds_request *req);
308+extern int ceph_mdsc_do_request(struct ceph_mds_client *mdsc,
309+ struct inode *dir,
310+ struct ceph_mds_request *req);
311+static inline void ceph_mdsc_get_request(struct ceph_mds_request *req)
312+{
313+ kref_get(&req->r_kref);
314+}
315+extern void ceph_mdsc_release_request(struct kref *kref);
316+static inline void ceph_mdsc_put_request(struct ceph_mds_request *req)
317+{
318+ kref_put(&req->r_kref, ceph_mdsc_release_request);
319+}
320+
321+extern void ceph_mdsc_pre_umount(struct ceph_mds_client *mdsc);
322+
323+extern char *ceph_mdsc_build_path(struct dentry *dentry, int *plen, u64 *base,
324+ int stop_on_nosnap);
325+
326+extern void __ceph_mdsc_drop_dentry_lease(struct dentry *dentry);
327+extern void ceph_mdsc_lease_send_msg(struct ceph_mds_session *session,
328+ struct inode *inode,
329+ struct dentry *dentry, char action,
330+ u32 seq);
331+
332+extern void ceph_mdsc_handle_map(struct ceph_mds_client *mdsc,
333+ struct ceph_msg *msg);
334+
335+#endif
fs/ceph/mdsmap.c+174−0
@@ -0,0 +1,174 @@
1+#include "ceph_debug.h"
2+
3+#include <linux/bug.h>
4+#include <linux/err.h>
5+#include <linux/random.h>
6+#include <linux/slab.h>
7+#include <linux/types.h>
8+
9+#include "mdsmap.h"
10+#include "messenger.h"
11+#include "decode.h"
12+
13+#include "super.h"
14+
15+
16+/*
17+ * choose a random mds that is "up" (i.e. has a state > 0), or -1.
18+ */
19+int ceph_mdsmap_get_random_mds(struct ceph_mdsmap *m)
20+{
21+ int n = 0;
22+ int i;
23+ char r;
24+
25+ /* count */
26+ for (i = 0; i < m->m_max_mds; i++)
27+ if (m->m_info[i].state > 0)
28+ n++;
29+ if (n == 0)
30+ return -1;
31+
32+ /* pick */
33+ get_random_bytes(&r, 1);
34+ n = r % n;
35+ i = 0;
36+ for (i = 0; n > 0; i++, n--)
37+ while (m->m_info[i].state <= 0)
38+ i++;
39+
40+ return i;
41+}
42+
43+/*
44+ * Decode an MDS map
45+ *
46+ * Ignore any fields we don't care about (there are quite a few of
47+ * them).
48+ */
49+struct ceph_mdsmap *ceph_mdsmap_decode(void **p, void *end)
50+{
51+ struct ceph_mdsmap *m;
52+ const void *start = *p;
53+ int i, j, n;
54+ int err = -EINVAL;
55+ u16 version;
56+
57+ m = kzalloc(sizeof(*m), GFP_NOFS);
58+ if (m == NULL)
59+ return ERR_PTR(-ENOMEM);
60+
61+ ceph_decode_16_safe(p, end, version, bad);
62+
63+ ceph_decode_need(p, end, 8*sizeof(u32) + sizeof(u64), bad);
64+ m->m_epoch = ceph_decode_32(p);
65+ m->m_client_epoch = ceph_decode_32(p);
66+ m->m_last_failure = ceph_decode_32(p);
67+ m->m_root = ceph_decode_32(p);
68+ m->m_session_timeout = ceph_decode_32(p);
69+ m->m_session_autoclose = ceph_decode_32(p);
70+ m->m_max_file_size = ceph_decode_64(p);
71+ m->m_max_mds = ceph_decode_32(p);
72+
73+ m->m_info = kcalloc(m->m_max_mds, sizeof(*m->m_info), GFP_NOFS);
74+ if (m->m_info == NULL)
75+ goto badmem;
76+
77+ /* pick out active nodes from mds_info (state > 0) */
78+ n = ceph_decode_32(p);
79+ for (i = 0; i < n; i++) {
80+ u64 global_id;
81+ u32 namelen;
82+ s32 mds, inc, state;
83+ u64 state_seq;
84+ u8 infoversion;
85+ struct ceph_entity_addr addr;
86+ u32 num_export_targets;
87+ void *pexport_targets = NULL;
88+
89+ ceph_decode_need(p, end, sizeof(u64)*2 + 1 + sizeof(u32), bad);
90+ global_id = ceph_decode_64(p);
91+ infoversion = ceph_decode_8(p);
92+ *p += sizeof(u64);
93+ namelen = ceph_decode_32(p); /* skip mds name */
94+ *p += namelen;
95+
96+ ceph_decode_need(p, end,
97+ 4*sizeof(u32) + sizeof(u64) +
98+ sizeof(addr) + sizeof(struct ceph_timespec),
99+ bad);
100+ mds = ceph_decode_32(p);
101+ inc = ceph_decode_32(p);
102+ state = ceph_decode_32(p);
103+ state_seq = ceph_decode_64(p);
104+ ceph_decode_copy(p, &addr, sizeof(addr));
105+ ceph_decode_addr(&addr);
106+ *p += sizeof(struct ceph_timespec);
107+ *p += sizeof(u32);
108+ ceph_decode_32_safe(p, end, namelen, bad);
109+ *p += namelen;
110+ if (infoversion >= 2) {
111+ ceph_decode_32_safe(p, end, num_export_targets, bad);
112+ pexport_targets = *p;
113+ *p += num_export_targets * sizeof(u32);
114+ } else {
115+ num_export_targets = 0;
116+ }
117+
118+ dout("mdsmap_decode %d/%d %lld mds%d.%d %s %s\n",
119+ i+1, n, global_id, mds, inc, pr_addr(&addr.in_addr),
120+ ceph_mds_state_name(state));
121+ if (mds >= 0 && mds < m->m_max_mds && state > 0) {
122+ m->m_info[mds].global_id = global_id;
123+ m->m_info[mds].state = state;
124+ m->m_info[mds].addr = addr;
125+ m->m_info[mds].num_export_targets = num_export_targets;
126+ if (num_export_targets) {
127+ m->m_info[mds].export_targets =
128+ kcalloc(num_export_targets, sizeof(u32),
129+ GFP_NOFS);
130+ for (j = 0; j < num_export_targets; j++)
131+ m->m_info[mds].export_targets[j] =
132+ ceph_decode_32(&pexport_targets);
133+ } else {
134+ m->m_info[mds].export_targets = NULL;
135+ }
136+ }
137+ }
138+
139+ /* pg_pools */
140+ ceph_decode_32_safe(p, end, n, bad);
141+ m->m_num_data_pg_pools = n;
142+ m->m_data_pg_pools = kcalloc(n, sizeof(u32), GFP_NOFS);
143+ if (!m->m_data_pg_pools)
144+ goto badmem;
145+ ceph_decode_need(p, end, sizeof(u32)*(n+1), bad);
146+ for (i = 0; i < n; i++)
147+ m->m_data_pg_pools[i] = ceph_decode_32(p);
148+ m->m_cas_pg_pool = ceph_decode_32(p);
149+
150+ /* ok, we don't care about the rest. */
151+ dout("mdsmap_decode success epoch %u\n", m->m_epoch);
152+ return m;
153+
154+badmem:
155+ err = -ENOMEM;
156+bad:
157+ pr_err("corrupt mdsmap\n");
158+ print_hex_dump(KERN_DEBUG, "mdsmap: ",
159+ DUMP_PREFIX_OFFSET, 16, 1,
160+ start, end - start, true);
161+ ceph_mdsmap_destroy(m);
162+ return ERR_PTR(-EINVAL);
163+}
164+
165+void ceph_mdsmap_destroy(struct ceph_mdsmap *m)
166+{
167+ int i;
168+
169+ for (i = 0; i < m->m_max_mds; i++)
170+ kfree(m->m_info[i].export_targets);
171+ kfree(m->m_info);
172+ kfree(m->m_data_pg_pools);
173+ kfree(m);
174+}
fs/ceph/mdsmap.h+54−0
@@ -0,0 +1,54 @@
1+#ifndef _FS_CEPH_MDSMAP_H
2+#define _FS_CEPH_MDSMAP_H
3+
4+#include "types.h"
5+
6+/*
7+ * mds map - describe servers in the mds cluster.
8+ *
9+ * we limit fields to those the client actually xcares about
10+ */
11+struct ceph_mds_info {
12+ u64 global_id;
13+ struct ceph_entity_addr addr;
14+ s32 state;
15+ int num_export_targets;
16+ u32 *export_targets;
17+};
18+
19+struct ceph_mdsmap {
20+ u32 m_epoch, m_client_epoch, m_last_failure;
21+ u32 m_root;
22+ u32 m_session_timeout; /* seconds */
23+ u32 m_session_autoclose; /* seconds */
24+ u64 m_max_file_size;
25+ u32 m_max_mds; /* size of m_addr, m_state arrays */
26+ struct ceph_mds_info *m_info;
27+
28+ /* which object pools file data can be stored in */
29+ int m_num_data_pg_pools;
30+ u32 *m_data_pg_pools;
31+ u32 m_cas_pg_pool;
32+};
33+
34+static inline struct ceph_entity_addr *
35+ceph_mdsmap_get_addr(struct ceph_mdsmap *m, int w)
36+{
37+ if (w >= m->m_max_mds)
38+ return NULL;
39+ return &m->m_info[w].addr;
40+}
41+
42+static inline int ceph_mdsmap_get_state(struct ceph_mdsmap *m, int w)
43+{
44+ BUG_ON(w < 0);
45+ if (w >= m->m_max_mds)
46+ return CEPH_MDS_STATE_DNE;
47+ return m->m_info[w].state;
48+}
49+
50+extern int ceph_mdsmap_get_random_mds(struct ceph_mdsmap *m);
51+extern struct ceph_mdsmap *ceph_mdsmap_decode(void **p, void *end);
52+extern void ceph_mdsmap_destroy(struct ceph_mdsmap *m);
53+
54+#endif
fs/ceph/messenger.c+2240−0
@@ -0,0 +1,2240 @@
1+#include "ceph_debug.h"
2+
3+#include <linux/crc32c.h>
4+#include <linux/ctype.h>
5+#include <linux/highmem.h>
6+#include <linux/inet.h>
7+#include <linux/kthread.h>
8+#include <linux/net.h>
9+#include <linux/socket.h>
10+#include <linux/string.h>
11+#include <net/tcp.h>
12+
13+#include "super.h"
14+#include "messenger.h"
15+#include "decode.h"
16+#include "pagelist.h"
17+
18+/*
19+ * Ceph uses the messenger to exchange ceph_msg messages with other
20+ * hosts in the system. The messenger provides ordered and reliable
21+ * delivery. We tolerate TCP disconnects by reconnecting (with
22+ * exponential backoff) in the case of a fault (disconnection, bad
23+ * crc, protocol error). Acks allow sent messages to be discarded by
24+ * the sender.
25+ */
26+
27+/* static tag bytes (protocol control messages) */
28+static char tag_msg = CEPH_MSGR_TAG_MSG;
29+static char tag_ack = CEPH_MSGR_TAG_ACK;
30+static char tag_keepalive = CEPH_MSGR_TAG_KEEPALIVE;
31+
32+
33+static void queue_con(struct ceph_connection *con);
34+static void con_work(struct work_struct *);
35+static void ceph_fault(struct ceph_connection *con);
36+
37+const char *ceph_name_type_str(int t)
38+{
39+ switch (t) {
40+ case CEPH_ENTITY_TYPE_MON: return "mon";
41+ case CEPH_ENTITY_TYPE_MDS: return "mds";
42+ case CEPH_ENTITY_TYPE_OSD: return "osd";
43+ case CEPH_ENTITY_TYPE_CLIENT: return "client";
44+ case CEPH_ENTITY_TYPE_ADMIN: return "admin";
45+ default: return "???";
46+ }
47+}
48+
49+/*
50+ * nicely render a sockaddr as a string.
51+ */
52+#define MAX_ADDR_STR 20
53+static char addr_str[MAX_ADDR_STR][40];
54+static DEFINE_SPINLOCK(addr_str_lock);
55+static int last_addr_str;
56+
57+const char *pr_addr(const struct sockaddr_storage *ss)
58+{
59+ int i;
60+ char *s;
61+ struct sockaddr_in *in4 = (void *)ss;
62+ unsigned char *quad = (void *)&in4->sin_addr.s_addr;
63+ struct sockaddr_in6 *in6 = (void *)ss;
64+
65+ spin_lock(&addr_str_lock);
66+ i = last_addr_str++;
67+ if (last_addr_str == MAX_ADDR_STR)
68+ last_addr_str = 0;
69+ spin_unlock(&addr_str_lock);
70+ s = addr_str[i];
71+
72+ switch (ss->ss_family) {
73+ case AF_INET:
74+ sprintf(s, "%u.%u.%u.%u:%u",
75+ (unsigned int)quad[0],
76+ (unsigned int)quad[1],
77+ (unsigned int)quad[2],
78+ (unsigned int)quad[3],
79+ (unsigned int)ntohs(in4->sin_port));
80+ break;
81+
82+ case AF_INET6:
83+ sprintf(s, "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x:%u",
84+ in6->sin6_addr.s6_addr16[0],
85+ in6->sin6_addr.s6_addr16[1],
86+ in6->sin6_addr.s6_addr16[2],
87+ in6->sin6_addr.s6_addr16[3],
88+ in6->sin6_addr.s6_addr16[4],
89+ in6->sin6_addr.s6_addr16[5],
90+ in6->sin6_addr.s6_addr16[6],
91+ in6->sin6_addr.s6_addr16[7],
92+ (unsigned int)ntohs(in6->sin6_port));
93+ break;
94+
95+ default:
96+ sprintf(s, "(unknown sockaddr family %d)", (int)ss->ss_family);
97+ }
98+
99+ return s;
100+}
101+
102+static void encode_my_addr(struct ceph_messenger *msgr)
103+{
104+ memcpy(&msgr->my_enc_addr, &msgr->inst.addr, sizeof(msgr->my_enc_addr));
105+ ceph_encode_addr(&msgr->my_enc_addr);
106+}
107+
108+/*
109+ * work queue for all reading and writing to/from the socket.
110+ */
111+struct workqueue_struct *ceph_msgr_wq;
112+
113+int __init ceph_msgr_init(void)
114+{
115+ ceph_msgr_wq = create_workqueue("ceph-msgr");
116+ if (IS_ERR(ceph_msgr_wq)) {
117+ int ret = PTR_ERR(ceph_msgr_wq);
118+ pr_err("msgr_init failed to create workqueue: %d\n", ret);
119+ ceph_msgr_wq = NULL;
120+ return ret;
121+ }
122+ return 0;
123+}
124+
125+void ceph_msgr_exit(void)
126+{
127+ destroy_workqueue(ceph_msgr_wq);
128+}
129+
130+/*
131+ * socket callback functions
132+ */
133+
134+/* data available on socket, or listen socket received a connect */
135+static void ceph_data_ready(struct sock *sk, int count_unused)
136+{
137+ struct ceph_connection *con =
138+ (struct ceph_connection *)sk->sk_user_data;
139+ if (sk->sk_state != TCP_CLOSE_WAIT) {
140+ dout("ceph_data_ready on %p state = %lu, queueing work\n",
141+ con, con->state);
142+ queue_con(con);
143+ }
144+}
145+
146+/* socket has buffer space for writing */
147+static void ceph_write_space(struct sock *sk)
148+{
149+ struct ceph_connection *con =
150+ (struct ceph_connection *)sk->sk_user_data;
151+
152+ /* only queue to workqueue if there is data we want to write. */
153+ if (test_bit(WRITE_PENDING, &con->state)) {
154+ dout("ceph_write_space %p queueing write work\n", con);
155+ queue_con(con);
156+ } else {
157+ dout("ceph_write_space %p nothing to write\n", con);
158+ }
159+
160+ /* since we have our own write_space, clear the SOCK_NOSPACE flag */
161+ clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags);
162+}
163+
164+/* socket's state has changed */
165+static void ceph_state_change(struct sock *sk)
166+{
167+ struct ceph_connection *con =
168+ (struct ceph_connection *)sk->sk_user_data;
169+
170+ dout("ceph_state_change %p state = %lu sk_state = %u\n",
171+ con, con->state, sk->sk_state);
172+
173+ if (test_bit(CLOSED, &con->state))
174+ return;
175+
176+ switch (sk->sk_state) {
177+ case TCP_CLOSE:
178+ dout("ceph_state_change TCP_CLOSE\n");
179+ case TCP_CLOSE_WAIT:
180+ dout("ceph_state_change TCP_CLOSE_WAIT\n");
181+ if (test_and_set_bit(SOCK_CLOSED, &con->state) == 0) {
182+ if (test_bit(CONNECTING, &con->state))
183+ con->error_msg = "connection failed";
184+ else
185+ con->error_msg = "socket closed";
186+ queue_con(con);
187+ }
188+ break;
189+ case TCP_ESTABLISHED:
190+ dout("ceph_state_change TCP_ESTABLISHED\n");
191+ queue_con(con);
192+ break;
193+ }
194+}
195+
196+/*
197+ * set up socket callbacks
198+ */
199+static void set_sock_callbacks(struct socket *sock,
200+ struct ceph_connection *con)
201+{
202+ struct sock *sk = sock->sk;
203+ sk->sk_user_data = (void *)con;
204+ sk->sk_data_ready = ceph_data_ready;
205+ sk->sk_write_space = ceph_write_space;
206+ sk->sk_state_change = ceph_state_change;
207+}
208+
209+
210+/*
211+ * socket helpers
212+ */
213+
214+/*
215+ * initiate connection to a remote socket.
216+ */
217+static struct socket *ceph_tcp_connect(struct ceph_connection *con)
218+{
219+ struct sockaddr *paddr = (struct sockaddr *)&con->peer_addr.in_addr;
220+ struct socket *sock;
221+ int ret;
222+
223+ BUG_ON(con->sock);
224+ ret = sock_create_kern(AF_INET, SOCK_STREAM, IPPROTO_TCP, &sock);
225+ if (ret)
226+ return ERR_PTR(ret);
227+ con->sock = sock;
228+ sock->sk->sk_allocation = GFP_NOFS;
229+
230+ set_sock_callbacks(sock, con);
231+
232+ dout("connect %s\n", pr_addr(&con->peer_addr.in_addr));
233+
234+ ret = sock->ops->connect(sock, paddr, sizeof(*paddr), O_NONBLOCK);
235+ if (ret == -EINPROGRESS) {
236+ dout("connect %s EINPROGRESS sk_state = %u\n",
237+ pr_addr(&con->peer_addr.in_addr),
238+ sock->sk->sk_state);
239+ ret = 0;
240+ }
241+ if (ret < 0) {
242+ pr_err("connect %s error %d\n",
243+ pr_addr(&con->peer_addr.in_addr), ret);
244+ sock_release(sock);
245+ con->sock = NULL;
246+ con->error_msg = "connect error";
247+ }
248+
249+ if (ret < 0)
250+ return ERR_PTR(ret);
251+ return sock;
252+}
253+
254+static int ceph_tcp_recvmsg(struct socket *sock, void *buf, size_t len)
255+{
256+ struct kvec iov = {buf, len};
257+ struct msghdr msg = { .msg_flags = MSG_DONTWAIT | MSG_NOSIGNAL };
258+
259+ return kernel_recvmsg(sock, &msg, &iov, 1, len, msg.msg_flags);
260+}
261+
262+/*
263+ * write something. @more is true if caller will be sending more data
264+ * shortly.
265+ */
266+static int ceph_tcp_sendmsg(struct socket *sock, struct kvec *iov,
267+ size_t kvlen, size_t len, int more)
268+{
269+ struct msghdr msg = { .msg_flags = MSG_DONTWAIT | MSG_NOSIGNAL };
270+
271+ if (more)
272+ msg.msg_flags |= MSG_MORE;
273+ else
274+ msg.msg_flags |= MSG_EOR; /* superfluous, but what the hell */
275+
276+ return kernel_sendmsg(sock, &msg, iov, kvlen, len);
277+}
278+
279+
280+/*
281+ * Shutdown/close the socket for the given connection.
282+ */
283+static int con_close_socket(struct ceph_connection *con)
284+{
285+ int rc;
286+
287+ dout("con_close_socket on %p sock %p\n", con, con->sock);
288+ if (!con->sock)
289+ return 0;
290+ set_bit(SOCK_CLOSED, &con->state);
291+ rc = con->sock->ops->shutdown(con->sock, SHUT_RDWR);
292+ sock_release(con->sock);
293+ con->sock = NULL;
294+ clear_bit(SOCK_CLOSED, &con->state);
295+ return rc;
296+}
297+
298+/*
299+ * Reset a connection. Discard all incoming and outgoing messages
300+ * and clear *_seq state.
301+ */
302+static void ceph_msg_remove(struct ceph_msg *msg)
303+{
304+ list_del_init(&msg->list_head);
305+ ceph_msg_put(msg);
306+}
307+static void ceph_msg_remove_list(struct list_head *head)
308+{
309+ while (!list_empty(head)) {
310+ struct ceph_msg *msg = list_first_entry(head, struct ceph_msg,
311+ list_head);
312+ ceph_msg_remove(msg);
313+ }
314+}
315+
316+static void reset_connection(struct ceph_connection *con)
317+{
318+ /* reset connection, out_queue, msg_ and connect_seq */
319+ /* discard existing out_queue and msg_seq */
320+ ceph_msg_remove_list(&con->out_queue);
321+ ceph_msg_remove_list(&con->out_sent);
322+
323+ if (con->in_msg) {
324+ ceph_msg_put(con->in_msg);
325+ con->in_msg = NULL;
326+ }
327+
328+ con->connect_seq = 0;
329+ con->out_seq = 0;
330+ if (con->out_msg) {
331+ ceph_msg_put(con->out_msg);
332+ con->out_msg = NULL;
333+ }
334+ con->in_seq = 0;
335+}
336+
337+/*
338+ * mark a peer down. drop any open connections.
339+ */
340+void ceph_con_close(struct ceph_connection *con)
341+{
342+ dout("con_close %p peer %s\n", con, pr_addr(&con->peer_addr.in_addr));
343+ set_bit(CLOSED, &con->state); /* in case there's queued work */
344+ clear_bit(STANDBY, &con->state); /* avoid connect_seq bump */
345+ clear_bit(LOSSYTX, &con->state); /* so we retry next connect */
346+ clear_bit(KEEPALIVE_PENDING, &con->state);
347+ clear_bit(WRITE_PENDING, &con->state);
348+ mutex_lock(&con->mutex);
349+ reset_connection(con);
350+ cancel_delayed_work(&con->work);
351+ mutex_unlock(&con->mutex);
352+ queue_con(con);
353+}
354+
355+/*
356+ * Reopen a closed connection, with a new peer address.
357+ */
358+void ceph_con_open(struct ceph_connection *con, struct ceph_entity_addr *addr)
359+{
360+ dout("con_open %p %s\n", con, pr_addr(&addr->in_addr));
361+ set_bit(OPENING, &con->state);
362+ clear_bit(CLOSED, &con->state);
363+ memcpy(&con->peer_addr, addr, sizeof(*addr));
364+ con->delay = 0; /* reset backoff memory */
365+ queue_con(con);
366+}
367+
368+/*
369+ * generic get/put
370+ */
371+struct ceph_connection *ceph_con_get(struct ceph_connection *con)
372+{
373+ dout("con_get %p nref = %d -> %d\n", con,
374+ atomic_read(&con->nref), atomic_read(&con->nref) + 1);
375+ if (atomic_inc_not_zero(&con->nref))
376+ return con;
377+ return NULL;
378+}
379+
380+void ceph_con_put(struct ceph_connection *con)
381+{
382+ dout("con_put %p nref = %d -> %d\n", con,
383+ atomic_read(&con->nref), atomic_read(&con->nref) - 1);
384+ BUG_ON(atomic_read(&con->nref) == 0);
385+ if (atomic_dec_and_test(&con->nref)) {
386+ BUG_ON(con->sock);
387+ kfree(con);
388+ }
389+}
390+
391+/*
392+ * initialize a new connection.
393+ */
394+void ceph_con_init(struct ceph_messenger *msgr, struct ceph_connection *con)
395+{
396+ dout("con_init %p\n", con);
397+ memset(con, 0, sizeof(*con));
398+ atomic_set(&con->nref, 1);
399+ con->msgr = msgr;
400+ mutex_init(&con->mutex);
401+ INIT_LIST_HEAD(&con->out_queue);
402+ INIT_LIST_HEAD(&con->out_sent);
403+ INIT_DELAYED_WORK(&con->work, con_work);
404+}
405+
406+
407+/*
408+ * We maintain a global counter to order connection attempts. Get
409+ * a unique seq greater than @gt.
410+ */
411+static u32 get_global_seq(struct ceph_messenger *msgr, u32 gt)
412+{
413+ u32 ret;
414+
415+ spin_lock(&msgr->global_seq_lock);
416+ if (msgr->global_seq < gt)
417+ msgr->global_seq = gt;
418+ ret = ++msgr->global_seq;
419+ spin_unlock(&msgr->global_seq_lock);
420+ return ret;
421+}
422+
423+
424+/*
425+ * Prepare footer for currently outgoing message, and finish things
426+ * off. Assumes out_kvec* are already valid.. we just add on to the end.
427+ */
428+static void prepare_write_message_footer(struct ceph_connection *con, int v)
429+{
430+ struct ceph_msg *m = con->out_msg;
431+
432+ dout("prepare_write_message_footer %p\n", con);
433+ con->out_kvec_is_msg = true;
434+ con->out_kvec[v].iov_base = &m->footer;
435+ con->out_kvec[v].iov_len = sizeof(m->footer);
436+ con->out_kvec_bytes += sizeof(m->footer);
437+ con->out_kvec_left++;
438+ con->out_more = m->more_to_follow;
439+ con->out_msg_done = true;
440+}
441+
442+/*
443+ * Prepare headers for the next outgoing message.
444+ */
445+static void prepare_write_message(struct ceph_connection *con)
446+{
447+ struct ceph_msg *m;
448+ int v = 0;
449+
450+ con->out_kvec_bytes = 0;
451+ con->out_kvec_is_msg = true;
452+ con->out_msg_done = false;
453+
454+ /* Sneak an ack in there first? If we can get it into the same
455+ * TCP packet that's a good thing. */
456+ if (con->in_seq > con->in_seq_acked) {
457+ con->in_seq_acked = con->in_seq;
458+ con->out_kvec[v].iov_base = &tag_ack;
459+ con->out_kvec[v++].iov_len = 1;
460+ con->out_temp_ack = cpu_to_le64(con->in_seq_acked);
461+ con->out_kvec[v].iov_base = &con->out_temp_ack;
462+ con->out_kvec[v++].iov_len = sizeof(con->out_temp_ack);
463+ con->out_kvec_bytes = 1 + sizeof(con->out_temp_ack);
464+ }
465+
466+ m = list_first_entry(&con->out_queue,
467+ struct ceph_msg, list_head);
468+ con->out_msg = m;
469+ if (test_bit(LOSSYTX, &con->state)) {
470+ list_del_init(&m->list_head);
471+ } else {
472+ /* put message on sent list */
473+ ceph_msg_get(m);
474+ list_move_tail(&m->list_head, &con->out_sent);
475+ }
476+
477+ m->hdr.seq = cpu_to_le64(++con->out_seq);
478+
479+ dout("prepare_write_message %p seq %lld type %d len %d+%d+%d %d pgs\n",
480+ m, con->out_seq, le16_to_cpu(m->hdr.type),
481+ le32_to_cpu(m->hdr.front_len), le32_to_cpu(m->hdr.middle_len),
482+ le32_to_cpu(m->hdr.data_len),
483+ m->nr_pages);
484+ BUG_ON(le32_to_cpu(m->hdr.front_len) != m->front.iov_len);
485+
486+ /* tag + hdr + front + middle */
487+ con->out_kvec[v].iov_base = &tag_msg;
488+ con->out_kvec[v++].iov_len = 1;
489+ con->out_kvec[v].iov_base = &m->hdr;
490+ con->out_kvec[v++].iov_len = sizeof(m->hdr);
491+ con->out_kvec[v++] = m->front;
492+ if (m->middle)
493+ con->out_kvec[v++] = m->middle->vec;
494+ con->out_kvec_left = v;
495+ con->out_kvec_bytes += 1 + sizeof(m->hdr) + m->front.iov_len +
496+ (m->middle ? m->middle->vec.iov_len : 0);
497+ con->out_kvec_cur = con->out_kvec;
498+
499+ /* fill in crc (except data pages), footer */
500+ con->out_msg->hdr.crc =
501+ cpu_to_le32(crc32c(0, (void *)&m->hdr,
502+ sizeof(m->hdr) - sizeof(m->hdr.crc)));
503+ con->out_msg->footer.flags = CEPH_MSG_FOOTER_COMPLETE;
504+ con->out_msg->footer.front_crc =
505+ cpu_to_le32(crc32c(0, m->front.iov_base, m->front.iov_len));
506+ if (m->middle)
507+ con->out_msg->footer.middle_crc =
508+ cpu_to_le32(crc32c(0, m->middle->vec.iov_base,
509+ m->middle->vec.iov_len));
510+ else
511+ con->out_msg->footer.middle_crc = 0;
512+ con->out_msg->footer.data_crc = 0;
513+ dout("prepare_write_message front_crc %u data_crc %u\n",
514+ le32_to_cpu(con->out_msg->footer.front_crc),
515+ le32_to_cpu(con->out_msg->footer.middle_crc));
516+
517+ /* is there a data payload? */
518+ if (le32_to_cpu(m->hdr.data_len) > 0) {
519+ /* initialize page iterator */
520+ con->out_msg_pos.page = 0;
521+ con->out_msg_pos.page_pos =
522+ le16_to_cpu(m->hdr.data_off) & ~PAGE_MASK;
523+ con->out_msg_pos.data_pos = 0;
524+ con->out_msg_pos.did_page_crc = 0;
525+ con->out_more = 1; /* data + footer will follow */
526+ } else {
527+ /* no, queue up footer too and be done */
528+ prepare_write_message_footer(con, v);
529+ }
530+
531+ set_bit(WRITE_PENDING, &con->state);
532+}
533+
534+/*
535+ * Prepare an ack.
536+ */
537+static void prepare_write_ack(struct ceph_connection *con)
538+{
539+ dout("prepare_write_ack %p %llu -> %llu\n", con,
540+ con->in_seq_acked, con->in_seq);
541+ con->in_seq_acked = con->in_seq;
542+
543+ con->out_kvec[0].iov_base = &tag_ack;
544+ con->out_kvec[0].iov_len = 1;
545+ con->out_temp_ack = cpu_to_le64(con->in_seq_acked);
546+ con->out_kvec[1].iov_base = &con->out_temp_ack;
547+ con->out_kvec[1].iov_len = sizeof(con->out_temp_ack);
548+ con->out_kvec_left = 2;
549+ con->out_kvec_bytes = 1 + sizeof(con->out_temp_ack);
550+ con->out_kvec_cur = con->out_kvec;
551+ con->out_more = 1; /* more will follow.. eventually.. */
552+ set_bit(WRITE_PENDING, &con->state);
553+}
554+
555+/*
556+ * Prepare to write keepalive byte.
557+ */
558+static void prepare_write_keepalive(struct ceph_connection *con)
559+{
560+ dout("prepare_write_keepalive %p\n", con);
561+ con->out_kvec[0].iov_base = &tag_keepalive;
562+ con->out_kvec[0].iov_len = 1;
563+ con->out_kvec_left = 1;
564+ con->out_kvec_bytes = 1;
565+ con->out_kvec_cur = con->out_kvec;
566+ set_bit(WRITE_PENDING, &con->state);
567+}
568+
569+/*
570+ * Connection negotiation.
571+ */
572+
573+static void prepare_connect_authorizer(struct ceph_connection *con)
574+{
575+ void *auth_buf;
576+ int auth_len = 0;
577+ int auth_protocol = 0;
578+
579+ mutex_unlock(&con->mutex);
580+ if (con->ops->get_authorizer)
581+ con->ops->get_authorizer(con, &auth_buf, &auth_len,
582+ &auth_protocol, &con->auth_reply_buf,
583+ &con->auth_reply_buf_len,
584+ con->auth_retry);
585+ mutex_lock(&con->mutex);
586+
587+ con->out_connect.authorizer_protocol = cpu_to_le32(auth_protocol);
588+ con->out_connect.authorizer_len = cpu_to_le32(auth_len);
589+
590+ con->out_kvec[con->out_kvec_left].iov_base = auth_buf;
591+ con->out_kvec[con->out_kvec_left].iov_len = auth_len;
592+ con->out_kvec_left++;
593+ con->out_kvec_bytes += auth_len;
594+}
595+
596+/*
597+ * We connected to a peer and are saying hello.
598+ */
599+static void prepare_write_banner(struct ceph_messenger *msgr,
600+ struct ceph_connection *con)
601+{
602+ int len = strlen(CEPH_BANNER);
603+
604+ con->out_kvec[0].iov_base = CEPH_BANNER;
605+ con->out_kvec[0].iov_len = len;
606+ con->out_kvec[1].iov_base = &msgr->my_enc_addr;
607+ con->out_kvec[1].iov_len = sizeof(msgr->my_enc_addr);
608+ con->out_kvec_left = 2;
609+ con->out_kvec_bytes = len + sizeof(msgr->my_enc_addr);
610+ con->out_kvec_cur = con->out_kvec;
611+ con->out_more = 0;
612+ set_bit(WRITE_PENDING, &con->state);
613+}
614+
615+static void prepare_write_connect(struct ceph_messenger *msgr,
616+ struct ceph_connection *con,
617+ int after_banner)
618+{
619+ unsigned global_seq = get_global_seq(con->msgr, 0);
620+ int proto;
621+
622+ switch (con->peer_name.type) {
623+ case CEPH_ENTITY_TYPE_MON:
624+ proto = CEPH_MONC_PROTOCOL;
625+ break;
626+ case CEPH_ENTITY_TYPE_OSD:
627+ proto = CEPH_OSDC_PROTOCOL;
628+ break;
629+ case CEPH_ENTITY_TYPE_MDS:
630+ proto = CEPH_MDSC_PROTOCOL;
631+ break;
632+ default:
633+ BUG();
634+ }
635+
636+ dout("prepare_write_connect %p cseq=%d gseq=%d proto=%d\n", con,
637+ con->connect_seq, global_seq, proto);
638+
639+ con->out_connect.features = CEPH_FEATURE_SUPPORTED;
640+ con->out_connect.host_type = cpu_to_le32(CEPH_ENTITY_TYPE_CLIENT);
641+ con->out_connect.connect_seq = cpu_to_le32(con->connect_seq);
642+ con->out_connect.global_seq = cpu_to_le32(global_seq);
643+ con->out_connect.protocol_version = cpu_to_le32(proto);
644+ con->out_connect.flags = 0;
645+
646+ if (!after_banner) {
647+ con->out_kvec_left = 0;
648+ con->out_kvec_bytes = 0;
649+ }
650+ con->out_kvec[con->out_kvec_left].iov_base = &con->out_connect;
651+ con->out_kvec[con->out_kvec_left].iov_len = sizeof(con->out_connect);
652+ con->out_kvec_left++;
653+ con->out_kvec_bytes += sizeof(con->out_connect);
654+ con->out_kvec_cur = con->out_kvec;
655+ con->out_more = 0;
656+ set_bit(WRITE_PENDING, &con->state);
657+
658+ prepare_connect_authorizer(con);
659+}
660+
661+
662+/*
663+ * write as much of pending kvecs to the socket as we can.
664+ * 1 -> done
665+ * 0 -> socket full, but more to do
666+ * <0 -> error
667+ */
668+static int write_partial_kvec(struct ceph_connection *con)
669+{
670+ int ret;
671+
672+ dout("write_partial_kvec %p %d left\n", con, con->out_kvec_bytes);
673+ while (con->out_kvec_bytes > 0) {
674+ ret = ceph_tcp_sendmsg(con->sock, con->out_kvec_cur,
675+ con->out_kvec_left, con->out_kvec_bytes,
676+ con->out_more);
677+ if (ret <= 0)
678+ goto out;
679+ con->out_kvec_bytes -= ret;
680+ if (con->out_kvec_bytes == 0)
681+ break; /* done */
682+ while (ret > 0) {
683+ if (ret >= con->out_kvec_cur->iov_len) {
684+ ret -= con->out_kvec_cur->iov_len;
685+ con->out_kvec_cur++;
686+ con->out_kvec_left--;
687+ } else {
688+ con->out_kvec_cur->iov_len -= ret;
689+ con->out_kvec_cur->iov_base += ret;
690+ ret = 0;
691+ break;
692+ }
693+ }
694+ }
695+ con->out_kvec_left = 0;
696+ con->out_kvec_is_msg = false;
697+ ret = 1;
698+out:
699+ dout("write_partial_kvec %p %d left in %d kvecs ret = %d\n", con,
700+ con->out_kvec_bytes, con->out_kvec_left, ret);
701+ return ret; /* done! */
702+}
703+
704+/*
705+ * Write as much message data payload as we can. If we finish, queue
706+ * up the footer.
707+ * 1 -> done, footer is now queued in out_kvec[].
708+ * 0 -> socket full, but more to do
709+ * <0 -> error
710+ */
711+static int write_partial_msg_pages(struct ceph_connection *con)
712+{
713+ struct ceph_msg *msg = con->out_msg;
714+ unsigned data_len = le32_to_cpu(msg->hdr.data_len);
715+ size_t len;
716+ int crc = con->msgr->nocrc;
717+ int ret;
718+
719+ dout("write_partial_msg_pages %p msg %p page %d/%d offset %d\n",
720+ con, con->out_msg, con->out_msg_pos.page, con->out_msg->nr_pages,
721+ con->out_msg_pos.page_pos);
722+
723+ while (con->out_msg_pos.page < con->out_msg->nr_pages) {
724+ struct page *page = NULL;
725+ void *kaddr = NULL;
726+
727+ /*
728+ * if we are calculating the data crc (the default), we need
729+ * to map the page. if our pages[] has been revoked, use the
730+ * zero page.
731+ */
732+ if (msg->pages) {
733+ page = msg->pages[con->out_msg_pos.page];
734+ if (crc)
735+ kaddr = kmap(page);
736+ } else if (msg->pagelist) {
737+ page = list_first_entry(&msg->pagelist->head,
738+ struct page, lru);
739+ if (crc)
740+ kaddr = kmap(page);
741+ } else {
742+ page = con->msgr->zero_page;
743+ if (crc)
744+ kaddr = page_address(con->msgr->zero_page);
745+ }
746+ len = min((int)(PAGE_SIZE - con->out_msg_pos.page_pos),
747+ (int)(data_len - con->out_msg_pos.data_pos));
748+ if (crc && !con->out_msg_pos.did_page_crc) {
749+ void *base = kaddr + con->out_msg_pos.page_pos;
750+ u32 tmpcrc = le32_to_cpu(con->out_msg->footer.data_crc);
751+
752+ BUG_ON(kaddr == NULL);
753+ con->out_msg->footer.data_crc =
754+ cpu_to_le32(crc32c(tmpcrc, base, len));
755+ con->out_msg_pos.did_page_crc = 1;
756+ }
757+
758+ ret = kernel_sendpage(con->sock, page,
759+ con->out_msg_pos.page_pos, len,
760+ MSG_DONTWAIT | MSG_NOSIGNAL |
761+ MSG_MORE);
762+
763+ if (crc && (msg->pages || msg->pagelist))
764+ kunmap(page);
765+
766+ if (ret <= 0)
767+ goto out;
768+
769+ con->out_msg_pos.data_pos += ret;
770+ con->out_msg_pos.page_pos += ret;
771+ if (ret == len) {
772+ con->out_msg_pos.page_pos = 0;
773+ con->out_msg_pos.page++;
774+ con->out_msg_pos.did_page_crc = 0;
775+ if (msg->pagelist)
776+ list_move_tail(&page->lru,
777+ &msg->pagelist->head);
778+ }
779+ }
780+
781+ dout("write_partial_msg_pages %p msg %p done\n", con, msg);
782+
783+ /* prepare and queue up footer, too */
784+ if (!crc)
785+ con->out_msg->footer.flags |= CEPH_MSG_FOOTER_NOCRC;
786+ con->out_kvec_bytes = 0;
787+ con->out_kvec_left = 0;
788+ con->out_kvec_cur = con->out_kvec;
789+ prepare_write_message_footer(con, 0);
790+ ret = 1;
791+out:
792+ return ret;
793+}
794+
795+/*
796+ * write some zeros
797+ */
798+static int write_partial_skip(struct ceph_connection *con)
799+{
800+ int ret;
801+
802+ while (con->out_skip > 0) {
803+ struct kvec iov = {
804+ .iov_base = page_address(con->msgr->zero_page),
805+ .iov_len = min(con->out_skip, (int)PAGE_CACHE_SIZE)
806+ };
807+
808+ ret = ceph_tcp_sendmsg(con->sock, &iov, 1, iov.iov_len, 1);
809+ if (ret <= 0)
810+ goto out;
811+ con->out_skip -= ret;
812+ }
813+ ret = 1;
814+out:
815+ return ret;
816+}
817+
818+/*
819+ * Prepare to read connection handshake, or an ack.
820+ */
821+static void prepare_read_banner(struct ceph_connection *con)
822+{
823+ dout("prepare_read_banner %p\n", con);
824+ con->in_base_pos = 0;
825+}
826+
827+static void prepare_read_connect(struct ceph_connection *con)
828+{
829+ dout("prepare_read_connect %p\n", con);
830+ con->in_base_pos = 0;
831+}
832+
833+static void prepare_read_connect_retry(struct ceph_connection *con)
834+{
835+ dout("prepare_read_connect_retry %p\n", con);
836+ con->in_base_pos = strlen(CEPH_BANNER) + sizeof(con->actual_peer_addr)
837+ + sizeof(con->peer_addr_for_me);
838+}
839+
840+static void prepare_read_ack(struct ceph_connection *con)
841+{
842+ dout("prepare_read_ack %p\n", con);
843+ con->in_base_pos = 0;
844+}
845+
846+static void prepare_read_tag(struct ceph_connection *con)
847+{
848+ dout("prepare_read_tag %p\n", con);
849+ con->in_base_pos = 0;
850+ con->in_tag = CEPH_MSGR_TAG_READY;
851+}
852+
853+/*
854+ * Prepare to read a message.
855+ */
856+static int prepare_read_message(struct ceph_connection *con)
857+{
858+ dout("prepare_read_message %p\n", con);
859+ BUG_ON(con->in_msg != NULL);
860+ con->in_base_pos = 0;
861+ con->in_front_crc = con->in_middle_crc = con->in_data_crc = 0;
862+ return 0;
863+}
864+
865+
866+static int read_partial(struct ceph_connection *con,
867+ int *to, int size, void *object)
868+{
869+ *to += size;
870+ while (con->in_base_pos < *to) {
871+ int left = *to - con->in_base_pos;
872+ int have = size - left;
873+ int ret = ceph_tcp_recvmsg(con->sock, object + have, left);
874+ if (ret <= 0)
875+ return ret;
876+ con->in_base_pos += ret;
877+ }
878+ return 1;
879+}
880+
881+
882+/*
883+ * Read all or part of the connect-side handshake on a new connection
884+ */
885+static int read_partial_banner(struct ceph_connection *con)
886+{
887+ int ret, to = 0;
888+
889+ dout("read_partial_banner %p at %d\n", con, con->in_base_pos);
890+
891+ /* peer's banner */
892+ ret = read_partial(con, &to, strlen(CEPH_BANNER), con->in_banner);
893+ if (ret <= 0)
894+ goto out;
895+ ret = read_partial(con, &to, sizeof(con->actual_peer_addr),
896+ &con->actual_peer_addr);
897+ if (ret <= 0)
898+ goto out;
899+ ret = read_partial(con, &to, sizeof(con->peer_addr_for_me),
900+ &con->peer_addr_for_me);
901+ if (ret <= 0)
902+ goto out;
903+out:
904+ return ret;
905+}
906+
907+static int read_partial_connect(struct ceph_connection *con)
908+{
909+ int ret, to = 0;
910+
911+ dout("read_partial_connect %p at %d\n", con, con->in_base_pos);
912+
913+ ret = read_partial(con, &to, sizeof(con->in_reply), &con->in_reply);
914+ if (ret <= 0)
915+ goto out;
916+ ret = read_partial(con, &to, le32_to_cpu(con->in_reply.authorizer_len),
917+ con->auth_reply_buf);
918+ if (ret <= 0)
919+ goto out;
920+
921+ dout("read_partial_connect %p tag %d, con_seq = %u, g_seq = %u\n",
922+ con, (int)con->in_reply.tag,
923+ le32_to_cpu(con->in_reply.connect_seq),
924+ le32_to_cpu(con->in_reply.global_seq));
925+out:
926+ return ret;
927+
928+}
929+
930+/*
931+ * Verify the hello banner looks okay.
932+ */
933+static int verify_hello(struct ceph_connection *con)
934+{
935+ if (memcmp(con->in_banner, CEPH_BANNER, strlen(CEPH_BANNER))) {
936+ pr_err("connect to %s got bad banner\n",
937+ pr_addr(&con->peer_addr.in_addr));
938+ con->error_msg = "protocol error, bad banner";
939+ return -1;
940+ }
941+ return 0;
942+}
943+
944+static bool addr_is_blank(struct sockaddr_storage *ss)
945+{
946+ switch (ss->ss_family) {
947+ case AF_INET:
948+ return ((struct sockaddr_in *)ss)->sin_addr.s_addr == 0;
949+ case AF_INET6:
950+ return
951+ ((struct sockaddr_in6 *)ss)->sin6_addr.s6_addr32[0] == 0 &&
952+ ((struct sockaddr_in6 *)ss)->sin6_addr.s6_addr32[1] == 0 &&
953+ ((struct sockaddr_in6 *)ss)->sin6_addr.s6_addr32[2] == 0 &&
954+ ((struct sockaddr_in6 *)ss)->sin6_addr.s6_addr32[3] == 0;
955+ }
956+ return false;
957+}
958+
959+static int addr_port(struct sockaddr_storage *ss)
960+{
961+ switch (ss->ss_family) {
962+ case AF_INET:
963+ return ntohs(((struct sockaddr_in *)ss)->sin_port);
964+ case AF_INET6:
965+ return ntohs(((struct sockaddr_in6 *)ss)->sin6_port);
966+ }
967+ return 0;
968+}
969+
970+static void addr_set_port(struct sockaddr_storage *ss, int p)
971+{
972+ switch (ss->ss_family) {
973+ case AF_INET:
974+ ((struct sockaddr_in *)ss)->sin_port = htons(p);
975+ case AF_INET6:
976+ ((struct sockaddr_in6 *)ss)->sin6_port = htons(p);
977+ }
978+}
979+
980+/*
981+ * Parse an ip[:port] list into an addr array. Use the default
982+ * monitor port if a port isn't specified.
983+ */
984+int ceph_parse_ips(const char *c, const char *end,
985+ struct ceph_entity_addr *addr,
986+ int max_count, int *count)
987+{
988+ int i;
989+ const char *p = c;
990+
991+ dout("parse_ips on '%.*s'\n", (int)(end-c), c);
992+ for (i = 0; i < max_count; i++) {
993+ const char *ipend;
994+ struct sockaddr_storage *ss = &addr[i].in_addr;
995+ struct sockaddr_in *in4 = (void *)ss;
996+ struct sockaddr_in6 *in6 = (void *)ss;
997+ int port;
998+
999+ memset(ss, 0, sizeof(*ss));
1000+ if (in4_pton(p, end - p, (u8 *)&in4->sin_addr.s_addr,
1001+ ',', &ipend)) {
1002+ ss->ss_family = AF_INET;
1003+ } else if (in6_pton(p, end - p, (u8 *)&in6->sin6_addr.s6_addr,
1004+ ',', &ipend)) {
1005+ ss->ss_family = AF_INET6;
1006+ } else {
1007+ goto bad;
1008+ }
1009+ p = ipend;
1010+
1011+ /* port? */
1012+ if (p < end && *p == ':') {
1013+ port = 0;
1014+ p++;
1015+ while (p < end && *p >= '0' && *p <= '9') {
1016+ port = (port * 10) + (*p - '0');
1017+ p++;
1018+ }
1019+ if (port > 65535 || port == 0)
1020+ goto bad;
1021+ } else {
1022+ port = CEPH_MON_PORT;
1023+ }
1024+
1025+ addr_set_port(ss, port);
1026+
1027+ dout("parse_ips got %s\n", pr_addr(ss));
1028+
1029+ if (p == end)
1030+ break;
1031+ if (*p != ',')
1032+ goto bad;
1033+ p++;
1034+ }
1035+
1036+ if (p != end)
1037+ goto bad;
1038+
1039+ if (count)
1040+ *count = i + 1;
1041+ return 0;
1042+
1043+bad:
1044+ pr_err("parse_ips bad ip '%s'\n", c);
1045+ return -EINVAL;
1046+}
1047+
1048+static int process_banner(struct ceph_connection *con)
1049+{
1050+ dout("process_banner on %p\n", con);
1051+
1052+ if (verify_hello(con) < 0)
1053+ return -1;
1054+
1055+ ceph_decode_addr(&con->actual_peer_addr);
1056+ ceph_decode_addr(&con->peer_addr_for_me);
1057+
1058+ /*
1059+ * Make sure the other end is who we wanted. note that the other
1060+ * end may not yet know their ip address, so if it's 0.0.0.0, give
1061+ * them the benefit of the doubt.
1062+ */
1063+ if (memcmp(&con->peer_addr, &con->actual_peer_addr,
1064+ sizeof(con->peer_addr)) != 0 &&
1065+ !(addr_is_blank(&con->actual_peer_addr.in_addr) &&
1066+ con->actual_peer_addr.nonce == con->peer_addr.nonce)) {
1067+ pr_warning("wrong peer, want %s/%lld, got %s/%lld\n",
1068+ pr_addr(&con->peer_addr.in_addr),
1069+ le64_to_cpu(con->peer_addr.nonce),
1070+ pr_addr(&con->actual_peer_addr.in_addr),
1071+ le64_to_cpu(con->actual_peer_addr.nonce));
1072+ con->error_msg = "wrong peer at address";
1073+ return -1;
1074+ }
1075+
1076+ /*
1077+ * did we learn our address?
1078+ */
1079+ if (addr_is_blank(&con->msgr->inst.addr.in_addr)) {
1080+ int port = addr_port(&con->msgr->inst.addr.in_addr);
1081+
1082+ memcpy(&con->msgr->inst.addr.in_addr,
1083+ &con->peer_addr_for_me.in_addr,
1084+ sizeof(con->peer_addr_for_me.in_addr));
1085+ addr_set_port(&con->msgr->inst.addr.in_addr, port);
1086+ encode_my_addr(con->msgr);
1087+ dout("process_banner learned my addr is %s\n",
1088+ pr_addr(&con->msgr->inst.addr.in_addr));
1089+ }
1090+
1091+ set_bit(NEGOTIATING, &con->state);
1092+ prepare_read_connect(con);
1093+ return 0;
1094+}
1095+
1096+static void fail_protocol(struct ceph_connection *con)
1097+{
1098+ reset_connection(con);
1099+ set_bit(CLOSED, &con->state); /* in case there's queued work */
1100+
1101+ mutex_unlock(&con->mutex);
1102+ if (con->ops->bad_proto)
1103+ con->ops->bad_proto(con);
1104+ mutex_lock(&con->mutex);
1105+}
1106+
1107+static int process_connect(struct ceph_connection *con)
1108+{
1109+ u64 sup_feat = CEPH_FEATURE_SUPPORTED;
1110+ u64 req_feat = CEPH_FEATURE_REQUIRED;
1111+ u64 server_feat = le64_to_cpu(con->in_reply.features);
1112+
1113+ dout("process_connect on %p tag %d\n", con, (int)con->in_tag);
1114+
1115+ switch (con->in_reply.tag) {
1116+ case CEPH_MSGR_TAG_FEATURES:
1117+ pr_err("%s%lld %s feature set mismatch,"
1118+ " my %llx < server's %llx, missing %llx\n",
1119+ ENTITY_NAME(con->peer_name),
1120+ pr_addr(&con->peer_addr.in_addr),
1121+ sup_feat, server_feat, server_feat & ~sup_feat);
1122+ con->error_msg = "missing required protocol features";
1123+ fail_protocol(con);
1124+ return -1;
1125+
1126+ case CEPH_MSGR_TAG_BADPROTOVER:
1127+ pr_err("%s%lld %s protocol version mismatch,"
1128+ " my %d != server's %d\n",
1129+ ENTITY_NAME(con->peer_name),
1130+ pr_addr(&con->peer_addr.in_addr),
1131+ le32_to_cpu(con->out_connect.protocol_version),
1132+ le32_to_cpu(con->in_reply.protocol_version));
1133+ con->error_msg = "protocol version mismatch";
1134+ fail_protocol(con);
1135+ return -1;
1136+
1137+ case CEPH_MSGR_TAG_BADAUTHORIZER:
1138+ con->auth_retry++;
1139+ dout("process_connect %p got BADAUTHORIZER attempt %d\n", con,
1140+ con->auth_retry);
1141+ if (con->auth_retry == 2) {
1142+ con->error_msg = "connect authorization failure";
1143+ reset_connection(con);
1144+ set_bit(CLOSED, &con->state);
1145+ return -1;
1146+ }
1147+ con->auth_retry = 1;
1148+ prepare_write_connect(con->msgr, con, 0);
1149+ prepare_read_connect_retry(con);
1150+ break;
1151+
1152+ case CEPH_MSGR_TAG_RESETSESSION:
1153+ /*
1154+ * If we connected with a large connect_seq but the peer
1155+ * has no record of a session with us (no connection, or
1156+ * connect_seq == 0), they will send RESETSESION to indicate
1157+ * that they must have reset their session, and may have
1158+ * dropped messages.
1159+ */
1160+ dout("process_connect got RESET peer seq %u\n",
1161+ le32_to_cpu(con->in_connect.connect_seq));
1162+ pr_err("%s%lld %s connection reset\n",
1163+ ENTITY_NAME(con->peer_name),
1164+ pr_addr(&con->peer_addr.in_addr));
1165+ reset_connection(con);
1166+ prepare_write_connect(con->msgr, con, 0);
1167+ prepare_read_connect(con);
1168+
1169+ /* Tell ceph about it. */
1170+ mutex_unlock(&con->mutex);
1171+ pr_info("reset on %s%lld\n", ENTITY_NAME(con->peer_name));
1172+ if (con->ops->peer_reset)
1173+ con->ops->peer_reset(con);
1174+ mutex_lock(&con->mutex);
1175+ break;
1176+
1177+ case CEPH_MSGR_TAG_RETRY_SESSION:
1178+ /*
1179+ * If we sent a smaller connect_seq than the peer has, try
1180+ * again with a larger value.
1181+ */
1182+ dout("process_connect got RETRY my seq = %u, peer_seq = %u\n",
1183+ le32_to_cpu(con->out_connect.connect_seq),
1184+ le32_to_cpu(con->in_connect.connect_seq));
1185+ con->connect_seq = le32_to_cpu(con->in_connect.connect_seq);
1186+ prepare_write_connect(con->msgr, con, 0);
1187+ prepare_read_connect(con);
1188+ break;
1189+
1190+ case CEPH_MSGR_TAG_RETRY_GLOBAL:
1191+ /*
1192+ * If we sent a smaller global_seq than the peer has, try
1193+ * again with a larger value.
1194+ */
1195+ dout("process_connect got RETRY_GLOBAL my %u peer_gseq %u\n",
1196+ con->peer_global_seq,
1197+ le32_to_cpu(con->in_connect.global_seq));
1198+ get_global_seq(con->msgr,
1199+ le32_to_cpu(con->in_connect.global_seq));
1200+ prepare_write_connect(con->msgr, con, 0);
1201+ prepare_read_connect(con);
1202+ break;
1203+
1204+ case CEPH_MSGR_TAG_READY:
1205+ if (req_feat & ~server_feat) {
1206+ pr_err("%s%lld %s protocol feature mismatch,"
1207+ " my required %llx > server's %llx, need %llx\n",
1208+ ENTITY_NAME(con->peer_name),
1209+ pr_addr(&con->peer_addr.in_addr),
1210+ req_feat, server_feat, req_feat & ~server_feat);
1211+ con->error_msg = "missing required protocol features";
1212+ fail_protocol(con);
1213+ return -1;
1214+ }
1215+ clear_bit(CONNECTING, &con->state);
1216+ con->peer_global_seq = le32_to_cpu(con->in_reply.global_seq);
1217+ con->connect_seq++;
1218+ dout("process_connect got READY gseq %d cseq %d (%d)\n",
1219+ con->peer_global_seq,
1220+ le32_to_cpu(con->in_reply.connect_seq),
1221+ con->connect_seq);
1222+ WARN_ON(con->connect_seq !=
1223+ le32_to_cpu(con->in_reply.connect_seq));
1224+
1225+ if (con->in_reply.flags & CEPH_MSG_CONNECT_LOSSY)
1226+ set_bit(LOSSYTX, &con->state);
1227+
1228+ prepare_read_tag(con);
1229+ break;
1230+
1231+ case CEPH_MSGR_TAG_WAIT:
1232+ /*
1233+ * If there is a connection race (we are opening
1234+ * connections to each other), one of us may just have
1235+ * to WAIT. This shouldn't happen if we are the
1236+ * client.
1237+ */
1238+ pr_err("process_connect peer connecting WAIT\n");
1239+
1240+ default:
1241+ pr_err("connect protocol error, will retry\n");
1242+ con->error_msg = "protocol error, garbage tag during connect";
1243+ return -1;
1244+ }
1245+ return 0;
1246+}
1247+
1248+
1249+/*
1250+ * read (part of) an ack
1251+ */
1252+static int read_partial_ack(struct ceph_connection *con)
1253+{
1254+ int to = 0;
1255+
1256+ return read_partial(con, &to, sizeof(con->in_temp_ack),
1257+ &con->in_temp_ack);
1258+}
1259+
1260+
1261+/*
1262+ * We can finally discard anything that's been acked.
1263+ */
1264+static void process_ack(struct ceph_connection *con)
1265+{
1266+ struct ceph_msg *m;
1267+ u64 ack = le64_to_cpu(con->in_temp_ack);
1268+ u64 seq;
1269+
1270+ while (!list_empty(&con->out_sent)) {
1271+ m = list_first_entry(&con->out_sent, struct ceph_msg,
1272+ list_head);
1273+ seq = le64_to_cpu(m->hdr.seq);
1274+ if (seq > ack)
1275+ break;
1276+ dout("got ack for seq %llu type %d at %p\n", seq,
1277+ le16_to_cpu(m->hdr.type), m);
1278+ ceph_msg_remove(m);
1279+ }
1280+ prepare_read_tag(con);
1281+}
1282+
1283+
1284+
1285+
1286+static int read_partial_message_section(struct ceph_connection *con,
1287+ struct kvec *section, unsigned int sec_len,
1288+ u32 *crc)
1289+{
1290+ int left;
1291+ int ret;
1292+
1293+ BUG_ON(!section);
1294+
1295+ while (section->iov_len < sec_len) {
1296+ BUG_ON(section->iov_base == NULL);
1297+ left = sec_len - section->iov_len;
1298+ ret = ceph_tcp_recvmsg(con->sock, (char *)section->iov_base +
1299+ section->iov_len, left);
1300+ if (ret <= 0)
1301+ return ret;
1302+ section->iov_len += ret;
1303+ if (section->iov_len == sec_len)
1304+ *crc = crc32c(0, section->iov_base,
1305+ section->iov_len);
1306+ }
1307+
1308+ return 1;
1309+}
1310+
1311+static struct ceph_msg *ceph_alloc_msg(struct ceph_connection *con,
1312+ struct ceph_msg_header *hdr,
1313+ int *skip);
1314+/*
1315+ * read (part of) a message.
1316+ */
1317+static int read_partial_message(struct ceph_connection *con)
1318+{
1319+ struct ceph_msg *m = con->in_msg;
1320+ void *p;
1321+ int ret;
1322+ int to, left;
1323+ unsigned front_len, middle_len, data_len, data_off;
1324+ int datacrc = con->msgr->nocrc;
1325+ int skip;
1326+
1327+ dout("read_partial_message con %p msg %p\n", con, m);
1328+
1329+ /* header */
1330+ while (con->in_base_pos < sizeof(con->in_hdr)) {
1331+ left = sizeof(con->in_hdr) - con->in_base_pos;
1332+ ret = ceph_tcp_recvmsg(con->sock,
1333+ (char *)&con->in_hdr + con->in_base_pos,
1334+ left);
1335+ if (ret <= 0)
1336+ return ret;
1337+ con->in_base_pos += ret;
1338+ if (con->in_base_pos == sizeof(con->in_hdr)) {
1339+ u32 crc = crc32c(0, (void *)&con->in_hdr,
1340+ sizeof(con->in_hdr) - sizeof(con->in_hdr.crc));
1341+ if (crc != le32_to_cpu(con->in_hdr.crc)) {
1342+ pr_err("read_partial_message bad hdr "
1343+ " crc %u != expected %u\n",
1344+ crc, con->in_hdr.crc);
1345+ return -EBADMSG;
1346+ }
1347+ }
1348+ }
1349+ front_len = le32_to_cpu(con->in_hdr.front_len);
1350+ if (front_len > CEPH_MSG_MAX_FRONT_LEN)
1351+ return -EIO;
1352+ middle_len = le32_to_cpu(con->in_hdr.middle_len);
1353+ if (middle_len > CEPH_MSG_MAX_DATA_LEN)
1354+ return -EIO;
1355+ data_len = le32_to_cpu(con->in_hdr.data_len);
1356+ if (data_len > CEPH_MSG_MAX_DATA_LEN)
1357+ return -EIO;
1358+ data_off = le16_to_cpu(con->in_hdr.data_off);
1359+
1360+ /* allocate message? */
1361+ if (!con->in_msg) {
1362+ dout("got hdr type %d front %d data %d\n", con->in_hdr.type,
1363+ con->in_hdr.front_len, con->in_hdr.data_len);
1364+ con->in_msg = ceph_alloc_msg(con, &con->in_hdr, &skip);
1365+ if (skip) {
1366+ /* skip this message */
1367+ dout("alloc_msg returned NULL, skipping message\n");
1368+ con->in_base_pos = -front_len - middle_len - data_len -
1369+ sizeof(m->footer);
1370+ con->in_tag = CEPH_MSGR_TAG_READY;
1371+ return 0;
1372+ }
1373+ if (IS_ERR(con->in_msg)) {
1374+ ret = PTR_ERR(con->in_msg);
1375+ con->in_msg = NULL;
1376+ con->error_msg =
1377+ "error allocating memory for incoming message";
1378+ return ret;
1379+ }
1380+ m = con->in_msg;
1381+ m->front.iov_len = 0; /* haven't read it yet */
1382+ if (m->middle)
1383+ m->middle->vec.iov_len = 0;
1384+
1385+ con->in_msg_pos.page = 0;
1386+ con->in_msg_pos.page_pos = data_off & ~PAGE_MASK;
1387+ con->in_msg_pos.data_pos = 0;
1388+ }
1389+
1390+ /* front */
1391+ ret = read_partial_message_section(con, &m->front, front_len,
1392+ &con->in_front_crc);
1393+ if (ret <= 0)
1394+ return ret;
1395+
1396+ /* middle */
1397+ if (m->middle) {
1398+ ret = read_partial_message_section(con, &m->middle->vec, middle_len,
1399+ &con->in_middle_crc);
1400+ if (ret <= 0)
1401+ return ret;
1402+ }
1403+
1404+ /* (page) data */
1405+ while (con->in_msg_pos.data_pos < data_len) {
1406+ left = min((int)(data_len - con->in_msg_pos.data_pos),
1407+ (int)(PAGE_SIZE - con->in_msg_pos.page_pos));
1408+ BUG_ON(m->pages == NULL);
1409+ p = kmap(m->pages[con->in_msg_pos.page]);
1410+ ret = ceph_tcp_recvmsg(con->sock, p + con->in_msg_pos.page_pos,
1411+ left);
1412+ if (ret > 0 && datacrc)
1413+ con->in_data_crc =
1414+ crc32c(con->in_data_crc,
1415+ p + con->in_msg_pos.page_pos, ret);
1416+ kunmap(m->pages[con->in_msg_pos.page]);
1417+ if (ret <= 0)
1418+ return ret;
1419+ con->in_msg_pos.data_pos += ret;
1420+ con->in_msg_pos.page_pos += ret;
1421+ if (con->in_msg_pos.page_pos == PAGE_SIZE) {
1422+ con->in_msg_pos.page_pos = 0;
1423+ con->in_msg_pos.page++;
1424+ }
1425+ }
1426+
1427+ /* footer */
1428+ to = sizeof(m->hdr) + sizeof(m->footer);
1429+ while (con->in_base_pos < to) {
1430+ left = to - con->in_base_pos;
1431+ ret = ceph_tcp_recvmsg(con->sock, (char *)&m->footer +
1432+ (con->in_base_pos - sizeof(m->hdr)),
1433+ left);
1434+ if (ret <= 0)
1435+ return ret;
1436+ con->in_base_pos += ret;
1437+ }
1438+ dout("read_partial_message got msg %p %d (%u) + %d (%u) + %d (%u)\n",
1439+ m, front_len, m->footer.front_crc, middle_len,
1440+ m->footer.middle_crc, data_len, m->footer.data_crc);
1441+
1442+ /* crc ok? */
1443+ if (con->in_front_crc != le32_to_cpu(m->footer.front_crc)) {
1444+ pr_err("read_partial_message %p front crc %u != exp. %u\n",
1445+ m, con->in_front_crc, m->footer.front_crc);
1446+ return -EBADMSG;
1447+ }
1448+ if (con->in_middle_crc != le32_to_cpu(m->footer.middle_crc)) {
1449+ pr_err("read_partial_message %p middle crc %u != exp %u\n",
1450+ m, con->in_middle_crc, m->footer.middle_crc);
1451+ return -EBADMSG;
1452+ }
1453+ if (datacrc &&
1454+ (m->footer.flags & CEPH_MSG_FOOTER_NOCRC) == 0 &&
1455+ con->in_data_crc != le32_to_cpu(m->footer.data_crc)) {
1456+ pr_err("read_partial_message %p data crc %u != exp. %u\n", m,
1457+ con->in_data_crc, le32_to_cpu(m->footer.data_crc));
1458+ return -EBADMSG;
1459+ }
1460+
1461+ return 1; /* done! */
1462+}
1463+
1464+/*
1465+ * Process message. This happens in the worker thread. The callback should
1466+ * be careful not to do anything that waits on other incoming messages or it
1467+ * may deadlock.
1468+ */
1469+static void process_message(struct ceph_connection *con)
1470+{
1471+ struct ceph_msg *msg;
1472+
1473+ msg = con->in_msg;
1474+ con->in_msg = NULL;
1475+
1476+ /* if first message, set peer_name */
1477+ if (con->peer_name.type == 0)
1478+ con->peer_name = msg->hdr.src.name;
1479+
1480+ con->in_seq++;
1481+ mutex_unlock(&con->mutex);
1482+
1483+ dout("===== %p %llu from %s%lld %d=%s len %d+%d (%u %u %u) =====\n",
1484+ msg, le64_to_cpu(msg->hdr.seq),
1485+ ENTITY_NAME(msg->hdr.src.name),
1486+ le16_to_cpu(msg->hdr.type),
1487+ ceph_msg_type_name(le16_to_cpu(msg->hdr.type)),
1488+ le32_to_cpu(msg->hdr.front_len),
1489+ le32_to_cpu(msg->hdr.data_len),
1490+ con->in_front_crc, con->in_middle_crc, con->in_data_crc);
1491+ con->ops->dispatch(con, msg);
1492+
1493+ mutex_lock(&con->mutex);
1494+ prepare_read_tag(con);
1495+}
1496+
1497+
1498+/*
1499+ * Write something to the socket. Called in a worker thread when the
1500+ * socket appears to be writeable and we have something ready to send.
1501+ */
1502+static int try_write(struct ceph_connection *con)
1503+{
1504+ struct ceph_messenger *msgr = con->msgr;
1505+ int ret = 1;
1506+
1507+ dout("try_write start %p state %lu nref %d\n", con, con->state,
1508+ atomic_read(&con->nref));
1509+
1510+ mutex_lock(&con->mutex);
1511+more:
1512+ dout("try_write out_kvec_bytes %d\n", con->out_kvec_bytes);
1513+
1514+ /* open the socket first? */
1515+ if (con->sock == NULL) {
1516+ /*
1517+ * if we were STANDBY and are reconnecting _this_
1518+ * connection, bump connect_seq now. Always bump
1519+ * global_seq.
1520+ */
1521+ if (test_and_clear_bit(STANDBY, &con->state))
1522+ con->connect_seq++;
1523+
1524+ prepare_write_banner(msgr, con);
1525+ prepare_write_connect(msgr, con, 1);
1526+ prepare_read_banner(con);
1527+ set_bit(CONNECTING, &con->state);
1528+ clear_bit(NEGOTIATING, &con->state);
1529+
1530+ BUG_ON(con->in_msg);
1531+ con->in_tag = CEPH_MSGR_TAG_READY;
1532+ dout("try_write initiating connect on %p new state %lu\n",
1533+ con, con->state);
1534+ con->sock = ceph_tcp_connect(con);
1535+ if (IS_ERR(con->sock)) {
1536+ con->sock = NULL;
1537+ con->error_msg = "connect error";
1538+ ret = -1;
1539+ goto out;
1540+ }
1541+ }
1542+
1543+more_kvec:
1544+ /* kvec data queued? */
1545+ if (con->out_skip) {
1546+ ret = write_partial_skip(con);
1547+ if (ret <= 0)
1548+ goto done;
1549+ if (ret < 0) {
1550+ dout("try_write write_partial_skip err %d\n", ret);
1551+ goto done;
1552+ }
1553+ }
1554+ if (con->out_kvec_left) {
1555+ ret = write_partial_kvec(con);
1556+ if (ret <= 0)
1557+ goto done;
1558+ }
1559+
1560+ /* msg pages? */
1561+ if (con->out_msg) {
1562+ if (con->out_msg_done) {
1563+ ceph_msg_put(con->out_msg);
1564+ con->out_msg = NULL; /* we're done with this one */
1565+ goto do_next;
1566+ }
1567+
1568+ ret = write_partial_msg_pages(con);
1569+ if (ret == 1)
1570+ goto more_kvec; /* we need to send the footer, too! */
1571+ if (ret == 0)
1572+ goto done;
1573+ if (ret < 0) {
1574+ dout("try_write write_partial_msg_pages err %d\n",
1575+ ret);
1576+ goto done;
1577+ }
1578+ }
1579+
1580+do_next:
1581+ if (!test_bit(CONNECTING, &con->state)) {
1582+ /* is anything else pending? */
1583+ if (!list_empty(&con->out_queue)) {
1584+ prepare_write_message(con);
1585+ goto more;
1586+ }
1587+ if (con->in_seq > con->in_seq_acked) {
1588+ prepare_write_ack(con);
1589+ goto more;
1590+ }
1591+ if (test_and_clear_bit(KEEPALIVE_PENDING, &con->state)) {
1592+ prepare_write_keepalive(con);
1593+ goto more;
1594+ }
1595+ }
1596+
1597+ /* Nothing to do! */
1598+ clear_bit(WRITE_PENDING, &con->state);
1599+ dout("try_write nothing else to write.\n");
1600+done:
1601+ ret = 0;
1602+out:
1603+ mutex_unlock(&con->mutex);
1604+ dout("try_write done on %p\n", con);
1605+ return ret;
1606+}
1607+
1608+
1609+
1610+/*
1611+ * Read what we can from the socket.
1612+ */
1613+static int try_read(struct ceph_connection *con)
1614+{
1615+ struct ceph_messenger *msgr;
1616+ int ret = -1;
1617+
1618+ if (!con->sock)
1619+ return 0;
1620+
1621+ if (test_bit(STANDBY, &con->state))
1622+ return 0;
1623+
1624+ dout("try_read start on %p\n", con);
1625+ msgr = con->msgr;
1626+
1627+ mutex_lock(&con->mutex);
1628+
1629+more:
1630+ dout("try_read tag %d in_base_pos %d\n", (int)con->in_tag,
1631+ con->in_base_pos);
1632+ if (test_bit(CONNECTING, &con->state)) {
1633+ if (!test_bit(NEGOTIATING, &con->state)) {
1634+ dout("try_read connecting\n");
1635+ ret = read_partial_banner(con);
1636+ if (ret <= 0)
1637+ goto done;
1638+ if (process_banner(con) < 0) {
1639+ ret = -1;
1640+ goto out;
1641+ }
1642+ }
1643+ ret = read_partial_connect(con);
1644+ if (ret <= 0)
1645+ goto done;
1646+ if (process_connect(con) < 0) {
1647+ ret = -1;
1648+ goto out;
1649+ }
1650+ goto more;
1651+ }
1652+
1653+ if (con->in_base_pos < 0) {
1654+ /*
1655+ * skipping + discarding content.
1656+ *
1657+ * FIXME: there must be a better way to do this!
1658+ */
1659+ static char buf[1024];
1660+ int skip = min(1024, -con->in_base_pos);
1661+ dout("skipping %d / %d bytes\n", skip, -con->in_base_pos);
1662+ ret = ceph_tcp_recvmsg(con->sock, buf, skip);
1663+ if (ret <= 0)
1664+ goto done;
1665+ con->in_base_pos += ret;
1666+ if (con->in_base_pos)
1667+ goto more;
1668+ }
1669+ if (con->in_tag == CEPH_MSGR_TAG_READY) {
1670+ /*
1671+ * what's next?
1672+ */
1673+ ret = ceph_tcp_recvmsg(con->sock, &con->in_tag, 1);
1674+ if (ret <= 0)
1675+ goto done;
1676+ dout("try_read got tag %d\n", (int)con->in_tag);
1677+ switch (con->in_tag) {
1678+ case CEPH_MSGR_TAG_MSG:
1679+ prepare_read_message(con);
1680+ break;
1681+ case CEPH_MSGR_TAG_ACK:
1682+ prepare_read_ack(con);
1683+ break;
1684+ case CEPH_MSGR_TAG_CLOSE:
1685+ set_bit(CLOSED, &con->state); /* fixme */
1686+ goto done;
1687+ default:
1688+ goto bad_tag;
1689+ }
1690+ }
1691+ if (con->in_tag == CEPH_MSGR_TAG_MSG) {
1692+ ret = read_partial_message(con);
1693+ if (ret <= 0) {
1694+ switch (ret) {
1695+ case -EBADMSG:
1696+ con->error_msg = "bad crc";
1697+ ret = -EIO;
1698+ goto out;
1699+ case -EIO:
1700+ con->error_msg = "io error";
1701+ goto out;
1702+ default:
1703+ goto done;
1704+ }
1705+ }
1706+ if (con->in_tag == CEPH_MSGR_TAG_READY)
1707+ goto more;
1708+ process_message(con);
1709+ goto more;
1710+ }
1711+ if (con->in_tag == CEPH_MSGR_TAG_ACK) {
1712+ ret = read_partial_ack(con);
1713+ if (ret <= 0)
1714+ goto done;
1715+ process_ack(con);
1716+ goto more;
1717+ }
1718+
1719+done:
1720+ ret = 0;
1721+out:
1722+ mutex_unlock(&con->mutex);
1723+ dout("try_read done on %p\n", con);
1724+ return ret;
1725+
1726+bad_tag:
1727+ pr_err("try_read bad con->in_tag = %d\n", (int)con->in_tag);
1728+ con->error_msg = "protocol error, garbage tag";
1729+ ret = -1;
1730+ goto out;
1731+}
1732+
1733+
1734+/*
1735+ * Atomically queue work on a connection. Bump @con reference to
1736+ * avoid races with connection teardown.
1737+ *
1738+ * There is some trickery going on with QUEUED and BUSY because we
1739+ * only want a _single_ thread operating on each connection at any
1740+ * point in time, but we want to use all available CPUs.
1741+ *
1742+ * The worker thread only proceeds if it can atomically set BUSY. It
1743+ * clears QUEUED and does it's thing. When it thinks it's done, it
1744+ * clears BUSY, then rechecks QUEUED.. if it's set again, it loops
1745+ * (tries again to set BUSY).
1746+ *
1747+ * To queue work, we first set QUEUED, _then_ if BUSY isn't set, we
1748+ * try to queue work. If that fails (work is already queued, or BUSY)
1749+ * we give up (work also already being done or is queued) but leave QUEUED
1750+ * set so that the worker thread will loop if necessary.
1751+ */
1752+static void queue_con(struct ceph_connection *con)
1753+{
1754+ if (test_bit(DEAD, &con->state)) {
1755+ dout("queue_con %p ignoring: DEAD\n",
1756+ con);
1757+ return;
1758+ }
1759+
1760+ if (!con->ops->get(con)) {
1761+ dout("queue_con %p ref count 0\n", con);
1762+ return;
1763+ }
1764+
1765+ set_bit(QUEUED, &con->state);
1766+ if (test_bit(BUSY, &con->state)) {
1767+ dout("queue_con %p - already BUSY\n", con);
1768+ con->ops->put(con);
1769+ } else if (!queue_work(ceph_msgr_wq, &con->work.work)) {
1770+ dout("queue_con %p - already queued\n", con);
1771+ con->ops->put(con);
1772+ } else {
1773+ dout("queue_con %p\n", con);
1774+ }
1775+}
1776+
1777+/*
1778+ * Do some work on a connection. Drop a connection ref when we're done.
1779+ */
1780+static void con_work(struct work_struct *work)
1781+{
1782+ struct ceph_connection *con = container_of(work, struct ceph_connection,
1783+ work.work);
1784+ int backoff = 0;
1785+
1786+more:
1787+ if (test_and_set_bit(BUSY, &con->state) != 0) {
1788+ dout("con_work %p BUSY already set\n", con);
1789+ goto out;
1790+ }
1791+ dout("con_work %p start, clearing QUEUED\n", con);
1792+ clear_bit(QUEUED, &con->state);
1793+
1794+ if (test_bit(CLOSED, &con->state)) { /* e.g. if we are replaced */
1795+ dout("con_work CLOSED\n");
1796+ con_close_socket(con);
1797+ goto done;
1798+ }
1799+ if (test_and_clear_bit(OPENING, &con->state)) {
1800+ /* reopen w/ new peer */
1801+ dout("con_work OPENING\n");
1802+ con_close_socket(con);
1803+ }
1804+
1805+ if (test_and_clear_bit(SOCK_CLOSED, &con->state) ||
1806+ try_read(con) < 0 ||
1807+ try_write(con) < 0) {
1808+ backoff = 1;
1809+ ceph_fault(con); /* error/fault path */
1810+ }
1811+
1812+done:
1813+ clear_bit(BUSY, &con->state);
1814+ dout("con->state=%lu\n", con->state);
1815+ if (test_bit(QUEUED, &con->state)) {
1816+ if (!backoff || test_bit(OPENING, &con->state)) {
1817+ dout("con_work %p QUEUED reset, looping\n", con);
1818+ goto more;
1819+ }
1820+ dout("con_work %p QUEUED reset, but just faulted\n", con);
1821+ clear_bit(QUEUED, &con->state);
1822+ }
1823+ dout("con_work %p done\n", con);
1824+
1825+out:
1826+ con->ops->put(con);
1827+}
1828+
1829+
1830+/*
1831+ * Generic error/fault handler. A retry mechanism is used with
1832+ * exponential backoff
1833+ */
1834+static void ceph_fault(struct ceph_connection *con)
1835+{
1836+ pr_err("%s%lld %s %s\n", ENTITY_NAME(con->peer_name),
1837+ pr_addr(&con->peer_addr.in_addr), con->error_msg);
1838+ dout("fault %p state %lu to peer %s\n",
1839+ con, con->state, pr_addr(&con->peer_addr.in_addr));
1840+
1841+ if (test_bit(LOSSYTX, &con->state)) {
1842+ dout("fault on LOSSYTX channel\n");
1843+ goto out;
1844+ }
1845+
1846+ clear_bit(BUSY, &con->state); /* to avoid an improbable race */
1847+
1848+ mutex_lock(&con->mutex);
1849+ if (test_bit(CLOSED, &con->state))
1850+ goto out_unlock;
1851+
1852+ con_close_socket(con);
1853+
1854+ if (con->in_msg) {
1855+ ceph_msg_put(con->in_msg);
1856+ con->in_msg = NULL;
1857+ }
1858+
1859+ /* Requeue anything that hasn't been acked */
1860+ list_splice_init(&con->out_sent, &con->out_queue);
1861+
1862+ /* If there are no messages in the queue, place the connection
1863+ * in a STANDBY state (i.e., don't try to reconnect just yet). */
1864+ if (list_empty(&con->out_queue) && !con->out_keepalive_pending) {
1865+ dout("fault setting STANDBY\n");
1866+ set_bit(STANDBY, &con->state);
1867+ } else {
1868+ /* retry after a delay. */
1869+ if (con->delay == 0)
1870+ con->delay = BASE_DELAY_INTERVAL;
1871+ else if (con->delay < MAX_DELAY_INTERVAL)
1872+ con->delay *= 2;
1873+ dout("fault queueing %p delay %lu\n", con, con->delay);
1874+ con->ops->get(con);
1875+ if (queue_delayed_work(ceph_msgr_wq, &con->work,
1876+ round_jiffies_relative(con->delay)) == 0)
1877+ con->ops->put(con);
1878+ }
1879+
1880+out_unlock:
1881+ mutex_unlock(&con->mutex);
1882+out:
1883+ /*
1884+ * in case we faulted due to authentication, invalidate our
1885+ * current tickets so that we can get new ones.
1886+ */
1887+ if (con->auth_retry && con->ops->invalidate_authorizer) {
1888+ dout("calling invalidate_authorizer()\n");
1889+ con->ops->invalidate_authorizer(con);
1890+ }
1891+
1892+ if (con->ops->fault)
1893+ con->ops->fault(con);
1894+}
1895+
1896+
1897+
1898+/*
1899+ * create a new messenger instance
1900+ */
1901+struct ceph_messenger *ceph_messenger_create(struct ceph_entity_addr *myaddr)
1902+{
1903+ struct ceph_messenger *msgr;
1904+
1905+ msgr = kzalloc(sizeof(*msgr), GFP_KERNEL);
1906+ if (msgr == NULL)
1907+ return ERR_PTR(-ENOMEM);
1908+
1909+ spin_lock_init(&msgr->global_seq_lock);
1910+
1911+ /* the zero page is needed if a request is "canceled" while the message
1912+ * is being written over the socket */
1913+ msgr->zero_page = alloc_page(GFP_KERNEL | __GFP_ZERO);
1914+ if (!msgr->zero_page) {
1915+ kfree(msgr);
1916+ return ERR_PTR(-ENOMEM);
1917+ }
1918+ kmap(msgr->zero_page);
1919+
1920+ if (myaddr)
1921+ msgr->inst.addr = *myaddr;
1922+
1923+ /* select a random nonce */
1924+ msgr->inst.addr.type = 0;
1925+ get_random_bytes(&msgr->inst.addr.nonce, sizeof(msgr->inst.addr.nonce));
1926+ encode_my_addr(msgr);
1927+
1928+ dout("messenger_create %p\n", msgr);
1929+ return msgr;
1930+}
1931+
1932+void ceph_messenger_destroy(struct ceph_messenger *msgr)
1933+{
1934+ dout("destroy %p\n", msgr);
1935+ kunmap(msgr->zero_page);
1936+ __free_page(msgr->zero_page);
1937+ kfree(msgr);
1938+ dout("destroyed messenger %p\n", msgr);
1939+}
1940+
1941+/*
1942+ * Queue up an outgoing message on the given connection.
1943+ */
1944+void ceph_con_send(struct ceph_connection *con, struct ceph_msg *msg)
1945+{
1946+ if (test_bit(CLOSED, &con->state)) {
1947+ dout("con_send %p closed, dropping %p\n", con, msg);
1948+ ceph_msg_put(msg);
1949+ return;
1950+ }
1951+
1952+ /* set src+dst */
1953+ msg->hdr.src.name = con->msgr->inst.name;
1954+ msg->hdr.src.addr = con->msgr->my_enc_addr;
1955+ msg->hdr.orig_src = msg->hdr.src;
1956+
1957+ BUG_ON(msg->front.iov_len != le32_to_cpu(msg->hdr.front_len));
1958+
1959+ /* queue */
1960+ mutex_lock(&con->mutex);
1961+ BUG_ON(!list_empty(&msg->list_head));
1962+ list_add_tail(&msg->list_head, &con->out_queue);
1963+ dout("----- %p to %s%lld %d=%s len %d+%d+%d -----\n", msg,
1964+ ENTITY_NAME(con->peer_name), le16_to_cpu(msg->hdr.type),
1965+ ceph_msg_type_name(le16_to_cpu(msg->hdr.type)),
1966+ le32_to_cpu(msg->hdr.front_len),
1967+ le32_to_cpu(msg->hdr.middle_len),
1968+ le32_to_cpu(msg->hdr.data_len));
1969+ mutex_unlock(&con->mutex);
1970+
1971+ /* if there wasn't anything waiting to send before, queue
1972+ * new work */
1973+ if (test_and_set_bit(WRITE_PENDING, &con->state) == 0)
1974+ queue_con(con);
1975+}
1976+
1977+/*
1978+ * Revoke a message that was previously queued for send
1979+ */
1980+void ceph_con_revoke(struct ceph_connection *con, struct ceph_msg *msg)
1981+{
1982+ mutex_lock(&con->mutex);
1983+ if (!list_empty(&msg->list_head)) {
1984+ dout("con_revoke %p msg %p\n", con, msg);
1985+ list_del_init(&msg->list_head);
1986+ ceph_msg_put(msg);
1987+ msg->hdr.seq = 0;
1988+ if (con->out_msg == msg) {
1989+ ceph_msg_put(con->out_msg);
1990+ con->out_msg = NULL;
1991+ }
1992+ if (con->out_kvec_is_msg) {
1993+ con->out_skip = con->out_kvec_bytes;
1994+ con->out_kvec_is_msg = false;
1995+ }
1996+ } else {
1997+ dout("con_revoke %p msg %p - not queued (sent?)\n", con, msg);
1998+ }
1999+ mutex_unlock(&con->mutex);
2000+}
2001+
2002+/*
2003+ * Revoke a message that we may be reading data into
2004+ */
2005+void ceph_con_revoke_message(struct ceph_connection *con, struct ceph_msg *msg)
2006+{
2007+ mutex_lock(&con->mutex);
2008+ if (con->in_msg && con->in_msg == msg) {
2009+ unsigned front_len = le32_to_cpu(con->in_hdr.front_len);
2010+ unsigned middle_len = le32_to_cpu(con->in_hdr.middle_len);
2011+ unsigned data_len = le32_to_cpu(con->in_hdr.data_len);
2012+
2013+ /* skip rest of message */
2014+ dout("con_revoke_pages %p msg %p revoked\n", con, msg);
2015+ con->in_base_pos = con->in_base_pos -
2016+ sizeof(struct ceph_msg_header) -
2017+ front_len -
2018+ middle_len -
2019+ data_len -
2020+ sizeof(struct ceph_msg_footer);
2021+ ceph_msg_put(con->in_msg);
2022+ con->in_msg = NULL;
2023+ con->in_tag = CEPH_MSGR_TAG_READY;
2024+ } else {
2025+ dout("con_revoke_pages %p msg %p pages %p no-op\n",
2026+ con, con->in_msg, msg);
2027+ }
2028+ mutex_unlock(&con->mutex);
2029+}
2030+
2031+/*
2032+ * Queue a keepalive byte to ensure the tcp connection is alive.
2033+ */
2034+void ceph_con_keepalive(struct ceph_connection *con)
2035+{
2036+ if (test_and_set_bit(KEEPALIVE_PENDING, &con->state) == 0 &&
2037+ test_and_set_bit(WRITE_PENDING, &con->state) == 0)
2038+ queue_con(con);
2039+}
2040+
2041+
2042+/*
2043+ * construct a new message with given type, size
2044+ * the new msg has a ref count of 1.
2045+ */
2046+struct ceph_msg *ceph_msg_new(int type, int front_len,
2047+ int page_len, int page_off, struct page **pages)
2048+{
2049+ struct ceph_msg *m;
2050+
2051+ m = kmalloc(sizeof(*m), GFP_NOFS);
2052+ if (m == NULL)
2053+ goto out;
2054+ kref_init(&m->kref);
2055+ INIT_LIST_HEAD(&m->list_head);
2056+
2057+ m->hdr.type = cpu_to_le16(type);
2058+ m->hdr.front_len = cpu_to_le32(front_len);
2059+ m->hdr.middle_len = 0;
2060+ m->hdr.data_len = cpu_to_le32(page_len);
2061+ m->hdr.data_off = cpu_to_le16(page_off);
2062+ m->hdr.priority = cpu_to_le16(CEPH_MSG_PRIO_DEFAULT);
2063+ m->footer.front_crc = 0;
2064+ m->footer.middle_crc = 0;
2065+ m->footer.data_crc = 0;
2066+ m->front_max = front_len;
2067+ m->front_is_vmalloc = false;
2068+ m->more_to_follow = false;
2069+ m->pool = NULL;
2070+
2071+ /* front */
2072+ if (front_len) {
2073+ if (front_len > PAGE_CACHE_SIZE) {
2074+ m->front.iov_base = __vmalloc(front_len, GFP_NOFS,
2075+ PAGE_KERNEL);
2076+ m->front_is_vmalloc = true;
2077+ } else {
2078+ m->front.iov_base = kmalloc(front_len, GFP_NOFS);
2079+ }
2080+ if (m->front.iov_base == NULL) {
2081+ pr_err("msg_new can't allocate %d bytes\n",
2082+ front_len);
2083+ goto out2;
2084+ }
2085+ } else {
2086+ m->front.iov_base = NULL;
2087+ }
2088+ m->front.iov_len = front_len;
2089+
2090+ /* middle */
2091+ m->middle = NULL;
2092+
2093+ /* data */
2094+ m->nr_pages = calc_pages_for(page_off, page_len);
2095+ m->pages = pages;
2096+ m->pagelist = NULL;
2097+
2098+ dout("ceph_msg_new %p page %d~%d -> %d\n", m, page_off, page_len,
2099+ m->nr_pages);
2100+ return m;
2101+
2102+out2:
2103+ ceph_msg_put(m);
2104+out:
2105+ pr_err("msg_new can't create type %d len %d\n", type, front_len);
2106+ return ERR_PTR(-ENOMEM);
2107+}
2108+
2109+/*
2110+ * Allocate "middle" portion of a message, if it is needed and wasn't
2111+ * allocated by alloc_msg. This allows us to read a small fixed-size
2112+ * per-type header in the front and then gracefully fail (i.e.,
2113+ * propagate the error to the caller based on info in the front) when
2114+ * the middle is too large.
2115+ */
2116+static int ceph_alloc_middle(struct ceph_connection *con, struct ceph_msg *msg)
2117+{
2118+ int type = le16_to_cpu(msg->hdr.type);
2119+ int middle_len = le32_to_cpu(msg->hdr.middle_len);
2120+
2121+ dout("alloc_middle %p type %d %s middle_len %d\n", msg, type,
2122+ ceph_msg_type_name(type), middle_len);
2123+ BUG_ON(!middle_len);
2124+ BUG_ON(msg->middle);
2125+
2126+ msg->middle = ceph_buffer_new(middle_len, GFP_NOFS);
2127+ if (!msg->middle)
2128+ return -ENOMEM;
2129+ return 0;
2130+}
2131+
2132+/*
2133+ * Generic message allocator, for incoming messages.
2134+ */
2135+static struct ceph_msg *ceph_alloc_msg(struct ceph_connection *con,
2136+ struct ceph_msg_header *hdr,
2137+ int *skip)
2138+{
2139+ int type = le16_to_cpu(hdr->type);
2140+ int front_len = le32_to_cpu(hdr->front_len);
2141+ int middle_len = le32_to_cpu(hdr->middle_len);
2142+ struct ceph_msg *msg = NULL;
2143+ int ret;
2144+
2145+ if (con->ops->alloc_msg) {
2146+ mutex_unlock(&con->mutex);
2147+ msg = con->ops->alloc_msg(con, hdr, skip);
2148+ mutex_lock(&con->mutex);
2149+ if (IS_ERR(msg))
2150+ return msg;
2151+
2152+ if (*skip)
2153+ return NULL;
2154+ }
2155+ if (!msg) {
2156+ *skip = 0;
2157+ msg = ceph_msg_new(type, front_len, 0, 0, NULL);
2158+ if (!msg) {
2159+ pr_err("unable to allocate msg type %d len %d\n",
2160+ type, front_len);
2161+ return ERR_PTR(-ENOMEM);
2162+ }
2163+ }
2164+ memcpy(&msg->hdr, &con->in_hdr, sizeof(con->in_hdr));
2165+
2166+ if (middle_len) {
2167+ ret = ceph_alloc_middle(con, msg);
2168+
2169+ if (ret < 0) {
2170+ ceph_msg_put(msg);
2171+ return msg;
2172+ }
2173+ }
2174+
2175+ return msg;
2176+}
2177+
2178+
2179+/*
2180+ * Free a generically kmalloc'd message.
2181+ */
2182+void ceph_msg_kfree(struct ceph_msg *m)
2183+{
2184+ dout("msg_kfree %p\n", m);
2185+ if (m->front_is_vmalloc)
2186+ vfree(m->front.iov_base);
2187+ else
2188+ kfree(m->front.iov_base);
2189+ kfree(m);
2190+}
2191+
2192+/*
2193+ * Drop a msg ref. Destroy as needed.
2194+ */
2195+void ceph_msg_last_put(struct kref *kref)
2196+{
2197+ struct ceph_msg *m = container_of(kref, struct ceph_msg, kref);
2198+
2199+ dout("ceph_msg_put last one on %p\n", m);
2200+ WARN_ON(!list_empty(&m->list_head));
2201+
2202+ /* drop middle, data, if any */
2203+ if (m->middle) {
2204+ ceph_buffer_put(m->middle);
2205+ m->middle = NULL;
2206+ }
2207+ m->nr_pages = 0;
2208+ m->pages = NULL;
2209+
2210+ if (m->pagelist) {
2211+ ceph_pagelist_release(m->pagelist);
2212+ kfree(m->pagelist);
2213+ m->pagelist = NULL;
2214+ }
2215+
2216+ if (m->pool)
2217+ ceph_msgpool_put(m->pool, m);
2218+ else
2219+ ceph_msg_kfree(m);
2220+}
2221+
2222+void ceph_msg_dump(struct ceph_msg *msg)
2223+{
2224+ pr_debug("msg_dump %p (front_max %d nr_pages %d)\n", msg,
2225+ msg->front_max, msg->nr_pages);
2226+ print_hex_dump(KERN_DEBUG, "header: ",
2227+ DUMP_PREFIX_OFFSET, 16, 1,
2228+ &msg->hdr, sizeof(msg->hdr), true);
2229+ print_hex_dump(KERN_DEBUG, " front: ",
2230+ DUMP_PREFIX_OFFSET, 16, 1,
2231+ msg->front.iov_base, msg->front.iov_len, true);
2232+ if (msg->middle)
2233+ print_hex_dump(KERN_DEBUG, "middle: ",
2234+ DUMP_PREFIX_OFFSET, 16, 1,
2235+ msg->middle->vec.iov_base,
2236+ msg->middle->vec.iov_len, true);
2237+ print_hex_dump(KERN_DEBUG, "footer: ",
2238+ DUMP_PREFIX_OFFSET, 16, 1,
2239+ &msg->footer, sizeof(msg->footer), true);
2240+}
fs/ceph/messenger.h+254−0
@@ -0,0 +1,254 @@
1+#ifndef __FS_CEPH_MESSENGER_H
2+#define __FS_CEPH_MESSENGER_H
3+
4+#include <linux/kref.h>
5+#include <linux/mutex.h>
6+#include <linux/net.h>
7+#include <linux/radix-tree.h>
8+#include <linux/uio.h>
9+#include <linux/version.h>
10+#include <linux/workqueue.h>
11+
12+#include "types.h"
13+#include "buffer.h"
14+
15+struct ceph_msg;
16+struct ceph_connection;
17+
18+extern struct workqueue_struct *ceph_msgr_wq; /* receive work queue */
19+
20+/*
21+ * Ceph defines these callbacks for handling connection events.
22+ */
23+struct ceph_connection_operations {
24+ struct ceph_connection *(*get)(struct ceph_connection *);
25+ void (*put)(struct ceph_connection *);
26+
27+ /* handle an incoming message. */
28+ void (*dispatch) (struct ceph_connection *con, struct ceph_msg *m);
29+
30+ /* authorize an outgoing connection */
31+ int (*get_authorizer) (struct ceph_connection *con,
32+ void **buf, int *len, int *proto,
33+ void **reply_buf, int *reply_len, int force_new);
34+ int (*verify_authorizer_reply) (struct ceph_connection *con, int len);
35+ int (*invalidate_authorizer)(struct ceph_connection *con);
36+
37+ /* protocol version mismatch */
38+ void (*bad_proto) (struct ceph_connection *con);
39+
40+ /* there was some error on the socket (disconnect, whatever) */
41+ void (*fault) (struct ceph_connection *con);
42+
43+ /* a remote host as terminated a message exchange session, and messages
44+ * we sent (or they tried to send us) may be lost. */
45+ void (*peer_reset) (struct ceph_connection *con);
46+
47+ struct ceph_msg * (*alloc_msg) (struct ceph_connection *con,
48+ struct ceph_msg_header *hdr,
49+ int *skip);
50+};
51+
52+extern const char *ceph_name_type_str(int t);
53+
54+/* use format string %s%d */
55+#define ENTITY_NAME(n) ceph_name_type_str((n).type), le64_to_cpu((n).num)
56+
57+struct ceph_messenger {
58+ struct ceph_entity_inst inst; /* my name+address */
59+ struct ceph_entity_addr my_enc_addr;
60+ struct page *zero_page; /* used in certain error cases */
61+
62+ bool nocrc;
63+
64+ /*
65+ * the global_seq counts connections i (attempt to) initiate
66+ * in order to disambiguate certain connect race conditions.
67+ */
68+ u32 global_seq;
69+ spinlock_t global_seq_lock;
70+};
71+
72+/*
73+ * a single message. it contains a header (src, dest, message type, etc.),
74+ * footer (crc values, mainly), a "front" message body, and possibly a
75+ * data payload (stored in some number of pages).
76+ */
77+struct ceph_msg {
78+ struct ceph_msg_header hdr; /* header */
79+ struct ceph_msg_footer footer; /* footer */
80+ struct kvec front; /* unaligned blobs of message */
81+ struct ceph_buffer *middle;
82+ struct page **pages; /* data payload. NOT OWNER. */
83+ unsigned nr_pages; /* size of page array */
84+ struct ceph_pagelist *pagelist; /* instead of pages */
85+ struct list_head list_head;
86+ struct kref kref;
87+ bool front_is_vmalloc;
88+ bool more_to_follow;
89+ int front_max;
90+
91+ struct ceph_msgpool *pool;
92+};
93+
94+struct ceph_msg_pos {
95+ int page, page_pos; /* which page; offset in page */
96+ int data_pos; /* offset in data payload */
97+ int did_page_crc; /* true if we've calculated crc for current page */
98+};
99+
100+/* ceph connection fault delay defaults, for exponential backoff */
101+#define BASE_DELAY_INTERVAL (HZ/2)
102+#define MAX_DELAY_INTERVAL (5 * 60 * HZ)
103+
104+/*
105+ * ceph_connection state bit flags
106+ *
107+ * QUEUED and BUSY are used together to ensure that only a single
108+ * thread is currently opening, reading or writing data to the socket.
109+ */
110+#define LOSSYTX 0 /* we can close channel or drop messages on errors */
111+#define CONNECTING 1
112+#define NEGOTIATING 2
113+#define KEEPALIVE_PENDING 3
114+#define WRITE_PENDING 4 /* we have data ready to send */
115+#define QUEUED 5 /* there is work queued on this connection */
116+#define BUSY 6 /* work is being done */
117+#define STANDBY 8 /* no outgoing messages, socket closed. we keep
118+ * the ceph_connection around to maintain shared
119+ * state with the peer. */
120+#define CLOSED 10 /* we've closed the connection */
121+#define SOCK_CLOSED 11 /* socket state changed to closed */
122+#define OPENING 13 /* open connection w/ (possibly new) peer */
123+#define DEAD 14 /* dead, about to kfree */
124+
125+/*
126+ * A single connection with another host.
127+ *
128+ * We maintain a queue of outgoing messages, and some session state to
129+ * ensure that we can preserve the lossless, ordered delivery of
130+ * messages in the case of a TCP disconnect.
131+ */
132+struct ceph_connection {
133+ void *private;
134+ atomic_t nref;
135+
136+ const struct ceph_connection_operations *ops;
137+
138+ struct ceph_messenger *msgr;
139+ struct socket *sock;
140+ unsigned long state; /* connection state (see flags above) */
141+ const char *error_msg; /* error message, if any */
142+
143+ struct ceph_entity_addr peer_addr; /* peer address */
144+ struct ceph_entity_name peer_name; /* peer name */
145+ struct ceph_entity_addr peer_addr_for_me;
146+ u32 connect_seq; /* identify the most recent connection
147+ attempt for this connection, client */
148+ u32 peer_global_seq; /* peer's global seq for this connection */
149+
150+ int auth_retry; /* true if we need a newer authorizer */
151+ void *auth_reply_buf; /* where to put the authorizer reply */
152+ int auth_reply_buf_len;
153+
154+ struct mutex mutex;
155+
156+ /* out queue */
157+ struct list_head out_queue;
158+ struct list_head out_sent; /* sending or sent but unacked */
159+ u64 out_seq; /* last message queued for send */
160+ u64 out_seq_sent; /* last message sent */
161+ bool out_keepalive_pending;
162+
163+ u64 in_seq, in_seq_acked; /* last message received, acked */
164+
165+ /* connection negotiation temps */
166+ char in_banner[CEPH_BANNER_MAX_LEN];
167+ union {
168+ struct { /* outgoing connection */
169+ struct ceph_msg_connect out_connect;
170+ struct ceph_msg_connect_reply in_reply;
171+ };
172+ struct { /* incoming */
173+ struct ceph_msg_connect in_connect;
174+ struct ceph_msg_connect_reply out_reply;
175+ };
176+ };
177+ struct ceph_entity_addr actual_peer_addr;
178+
179+ /* message out temps */
180+ struct ceph_msg *out_msg; /* sending message (== tail of
181+ out_sent) */
182+ bool out_msg_done;
183+ struct ceph_msg_pos out_msg_pos;
184+
185+ struct kvec out_kvec[8], /* sending header/footer data */
186+ *out_kvec_cur;
187+ int out_kvec_left; /* kvec's left in out_kvec */
188+ int out_skip; /* skip this many bytes */
189+ int out_kvec_bytes; /* total bytes left */
190+ bool out_kvec_is_msg; /* kvec refers to out_msg */
191+ int out_more; /* there is more data after the kvecs */
192+ __le64 out_temp_ack; /* for writing an ack */
193+
194+ /* message in temps */
195+ struct ceph_msg_header in_hdr;
196+ struct ceph_msg *in_msg;
197+ struct ceph_msg_pos in_msg_pos;
198+ u32 in_front_crc, in_middle_crc, in_data_crc; /* calculated crc */
199+
200+ char in_tag; /* protocol control byte */
201+ int in_base_pos; /* bytes read */
202+ __le64 in_temp_ack; /* for reading an ack */
203+
204+ struct delayed_work work; /* send|recv work */
205+ unsigned long delay; /* current delay interval */
206+};
207+
208+
209+extern const char *pr_addr(const struct sockaddr_storage *ss);
210+extern int ceph_parse_ips(const char *c, const char *end,
211+ struct ceph_entity_addr *addr,
212+ int max_count, int *count);
213+
214+
215+extern int ceph_msgr_init(void);
216+extern void ceph_msgr_exit(void);
217+
218+extern struct ceph_messenger *ceph_messenger_create(
219+ struct ceph_entity_addr *myaddr);
220+extern void ceph_messenger_destroy(struct ceph_messenger *);
221+
222+extern void ceph_con_init(struct ceph_messenger *msgr,
223+ struct ceph_connection *con);
224+extern void ceph_con_open(struct ceph_connection *con,
225+ struct ceph_entity_addr *addr);
226+extern void ceph_con_close(struct ceph_connection *con);
227+extern void ceph_con_send(struct ceph_connection *con, struct ceph_msg *msg);
228+extern void ceph_con_revoke(struct ceph_connection *con, struct ceph_msg *msg);
229+extern void ceph_con_revoke_message(struct ceph_connection *con,
230+ struct ceph_msg *msg);
231+extern void ceph_con_keepalive(struct ceph_connection *con);
232+extern struct ceph_connection *ceph_con_get(struct ceph_connection *con);
233+extern void ceph_con_put(struct ceph_connection *con);
234+
235+extern struct ceph_msg *ceph_msg_new(int type, int front_len,
236+ int page_len, int page_off,
237+ struct page **pages);
238+extern void ceph_msg_kfree(struct ceph_msg *m);
239+
240+
241+static inline struct ceph_msg *ceph_msg_get(struct ceph_msg *msg)
242+{
243+ kref_get(&msg->kref);
244+ return msg;
245+}
246+extern void ceph_msg_last_put(struct kref *kref);
247+static inline void ceph_msg_put(struct ceph_msg *msg)
248+{
249+ kref_put(&msg->kref, ceph_msg_last_put);
250+}
251+
252+extern void ceph_msg_dump(struct ceph_msg *msg);
253+
254+#endif
fs/ceph/mon_client.c+834−0
@@ -0,0 +1,834 @@
1+#include "ceph_debug.h"
2+
3+#include <linux/types.h>
4+#include <linux/random.h>
5+#include <linux/sched.h>
6+
7+#include "mon_client.h"
8+#include "super.h"
9+#include "auth.h"
10+#include "decode.h"
11+
12+/*
13+ * Interact with Ceph monitor cluster. Handle requests for new map
14+ * versions, and periodically resend as needed. Also implement
15+ * statfs() and umount().
16+ *
17+ * A small cluster of Ceph "monitors" are responsible for managing critical
18+ * cluster configuration and state information. An odd number (e.g., 3, 5)
19+ * of cmon daemons use a modified version of the Paxos part-time parliament
20+ * algorithm to manage the MDS map (mds cluster membership), OSD map, and
21+ * list of clients who have mounted the file system.
22+ *
23+ * We maintain an open, active session with a monitor at all times in order to
24+ * receive timely MDSMap updates. We periodically send a keepalive byte on the
25+ * TCP socket to ensure we detect a failure. If the connection does break, we
26+ * randomly hunt for a new monitor. Once the connection is reestablished, we
27+ * resend any outstanding requests.
28+ */
29+
30+const static struct ceph_connection_operations mon_con_ops;
31+
32+static int __validate_auth(struct ceph_mon_client *monc);
33+
34+/*
35+ * Decode a monmap blob (e.g., during mount).
36+ */
37+struct ceph_monmap *ceph_monmap_decode(void *p, void *end)
38+{
39+ struct ceph_monmap *m = NULL;
40+ int i, err = -EINVAL;
41+ struct ceph_fsid fsid;
42+ u32 epoch, num_mon;
43+ u16 version;
44+ u32 len;
45+
46+ ceph_decode_32_safe(&p, end, len, bad);
47+ ceph_decode_need(&p, end, len, bad);
48+
49+ dout("monmap_decode %p %p len %d\n", p, end, (int)(end-p));
50+
51+ ceph_decode_16_safe(&p, end, version, bad);
52+
53+ ceph_decode_need(&p, end, sizeof(fsid) + 2*sizeof(u32), bad);
54+ ceph_decode_copy(&p, &fsid, sizeof(fsid));
55+ epoch = ceph_decode_32(&p);
56+
57+ num_mon = ceph_decode_32(&p);
58+ ceph_decode_need(&p, end, num_mon*sizeof(m->mon_inst[0]), bad);
59+
60+ if (num_mon >= CEPH_MAX_MON)
61+ goto bad;
62+ m = kmalloc(sizeof(*m) + sizeof(m->mon_inst[0])*num_mon, GFP_NOFS);
63+ if (m == NULL)
64+ return ERR_PTR(-ENOMEM);
65+ m->fsid = fsid;
66+ m->epoch = epoch;
67+ m->num_mon = num_mon;
68+ ceph_decode_copy(&p, m->mon_inst, num_mon*sizeof(m->mon_inst[0]));
69+ for (i = 0; i < num_mon; i++)
70+ ceph_decode_addr(&m->mon_inst[i].addr);
71+
72+ dout("monmap_decode epoch %d, num_mon %d\n", m->epoch,
73+ m->num_mon);
74+ for (i = 0; i < m->num_mon; i++)
75+ dout("monmap_decode mon%d is %s\n", i,
76+ pr_addr(&m->mon_inst[i].addr.in_addr));
77+ return m;
78+
79+bad:
80+ dout("monmap_decode failed with %d\n", err);
81+ kfree(m);
82+ return ERR_PTR(err);
83+}
84+
85+/*
86+ * return true if *addr is included in the monmap.
87+ */
88+int ceph_monmap_contains(struct ceph_monmap *m, struct ceph_entity_addr *addr)
89+{
90+ int i;
91+
92+ for (i = 0; i < m->num_mon; i++)
93+ if (memcmp(addr, &m->mon_inst[i].addr, sizeof(*addr)) == 0)
94+ return 1;
95+ return 0;
96+}
97+
98+/*
99+ * Send an auth request.
100+ */
101+static void __send_prepared_auth_request(struct ceph_mon_client *monc, int len)
102+{
103+ monc->pending_auth = 1;
104+ monc->m_auth->front.iov_len = len;
105+ monc->m_auth->hdr.front_len = cpu_to_le32(len);
106+ ceph_msg_get(monc->m_auth); /* keep our ref */
107+ ceph_con_send(monc->con, monc->m_auth);
108+}
109+
110+/*
111+ * Close monitor session, if any.
112+ */
113+static void __close_session(struct ceph_mon_client *monc)
114+{
115+ if (monc->con) {
116+ dout("__close_session closing mon%d\n", monc->cur_mon);
117+ ceph_con_revoke(monc->con, monc->m_auth);
118+ ceph_con_close(monc->con);
119+ monc->cur_mon = -1;
120+ monc->pending_auth = 0;
121+ ceph_auth_reset(monc->auth);
122+ }
123+}
124+
125+/*
126+ * Open a session with a (new) monitor.
127+ */
128+static int __open_session(struct ceph_mon_client *monc)
129+{
130+ char r;
131+ int ret;
132+
133+ if (monc->cur_mon < 0) {
134+ get_random_bytes(&r, 1);
135+ monc->cur_mon = r % monc->monmap->num_mon;
136+ dout("open_session num=%d r=%d -> mon%d\n",
137+ monc->monmap->num_mon, r, monc->cur_mon);
138+ monc->sub_sent = 0;
139+ monc->sub_renew_after = jiffies; /* i.e., expired */
140+ monc->want_next_osdmap = !!monc->want_next_osdmap;
141+
142+ dout("open_session mon%d opening\n", monc->cur_mon);
143+ monc->con->peer_name.type = CEPH_ENTITY_TYPE_MON;
144+ monc->con->peer_name.num = cpu_to_le64(monc->cur_mon);
145+ ceph_con_open(monc->con,
146+ &monc->monmap->mon_inst[monc->cur_mon].addr);
147+
148+ /* initiatiate authentication handshake */
149+ ret = ceph_auth_build_hello(monc->auth,
150+ monc->m_auth->front.iov_base,
151+ monc->m_auth->front_max);
152+ __send_prepared_auth_request(monc, ret);
153+ } else {
154+ dout("open_session mon%d already open\n", monc->cur_mon);
155+ }
156+ return 0;
157+}
158+
159+static bool __sub_expired(struct ceph_mon_client *monc)
160+{
161+ return time_after_eq(jiffies, monc->sub_renew_after);
162+}
163+
164+/*
165+ * Reschedule delayed work timer.
166+ */
167+static void __schedule_delayed(struct ceph_mon_client *monc)
168+{
169+ unsigned delay;
170+
171+ if (monc->cur_mon < 0 || __sub_expired(monc))
172+ delay = 10 * HZ;
173+ else
174+ delay = 20 * HZ;
175+ dout("__schedule_delayed after %u\n", delay);
176+ schedule_delayed_work(&monc->delayed_work, delay);
177+}
178+
179+/*
180+ * Send subscribe request for mdsmap and/or osdmap.
181+ */
182+static void __send_subscribe(struct ceph_mon_client *monc)
183+{
184+ dout("__send_subscribe sub_sent=%u exp=%u want_osd=%d\n",
185+ (unsigned)monc->sub_sent, __sub_expired(monc),
186+ monc->want_next_osdmap);
187+ if ((__sub_expired(monc) && !monc->sub_sent) ||
188+ monc->want_next_osdmap == 1) {
189+ struct ceph_msg *msg;
190+ struct ceph_mon_subscribe_item *i;
191+ void *p, *end;
192+
193+ msg = ceph_msg_new(CEPH_MSG_MON_SUBSCRIBE, 96, 0, 0, NULL);
194+ if (!msg)
195+ return;
196+
197+ p = msg->front.iov_base;
198+ end = p + msg->front.iov_len;
199+
200+ dout("__send_subscribe to 'mdsmap' %u+\n",
201+ (unsigned)monc->have_mdsmap);
202+ if (monc->want_next_osdmap) {
203+ dout("__send_subscribe to 'osdmap' %u\n",
204+ (unsigned)monc->have_osdmap);
205+ ceph_encode_32(&p, 3);
206+ ceph_encode_string(&p, end, "osdmap", 6);
207+ i = p;
208+ i->have = cpu_to_le64(monc->have_osdmap);
209+ i->onetime = 1;
210+ p += sizeof(*i);
211+ monc->want_next_osdmap = 2; /* requested */
212+ } else {
213+ ceph_encode_32(&p, 2);
214+ }
215+ ceph_encode_string(&p, end, "mdsmap", 6);
216+ i = p;
217+ i->have = cpu_to_le64(monc->have_mdsmap);
218+ i->onetime = 0;
219+ p += sizeof(*i);
220+ ceph_encode_string(&p, end, "monmap", 6);
221+ i = p;
222+ i->have = 0;
223+ i->onetime = 0;

611 more lines aren’t shown here.

fs/ceph/mon_client.h+119−0

This diff is large: its 120 lines aren’t shown here.

fs/ceph/msgpool.c+186−0

This diff is large: its 187 lines aren’t shown here.

fs/ceph/msgpool.h+27−0

This diff is large: its 28 lines aren’t shown here.

fs/ceph/msgr.h+158−0

This diff is large: its 159 lines aren’t shown here.

fs/ceph/osd_client.c+1537−0

This diff is large: its 1538 lines aren’t shown here.

fs/ceph/osd_client.h+166−0

This diff is large: its 167 lines aren’t shown here.

fs/ceph/osdmap.c+1019−0

This diff is large: its 1020 lines aren’t shown here.

fs/ceph/osdmap.h+125−0

This diff is large: its 126 lines aren’t shown here.

fs/ceph/pagelist.c+54−0

This diff is large: its 55 lines aren’t shown here.

fs/ceph/pagelist.h+54−0

This diff is large: its 55 lines aren’t shown here.

fs/ceph/rados.h+374−0

This diff is large: its 375 lines aren’t shown here.

fs/ceph/snap.c+904−0

This diff is large: its 905 lines aren’t shown here.

fs/ceph/super.c+1030−0

This diff is large: its 1031 lines aren’t shown here.

fs/ceph/super.h+901−0

This diff is large: its 902 lines aren’t shown here.

fs/ceph/types.h+29−0

This diff is large: its 30 lines aren’t shown here.

fs/ceph/xattr.c+844−0

This diff is large: its 845 lines aren’t shown here.

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t