Commit Graph

166563 Commits

Author SHA1 Message Date
Sage Weil
fb690390e3 ceph: make CRUSH hash function a bucket property
Make the integer hash function a property of the bucket it is used on.  This
allows us to gracefully add support for new hash functions without starting
from scatch.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-07 20:18:22 -08:00
Sage Weil
1654dd0cf5 ceph: make object hash a pg_pool property
The object will be hashed to a placement seed (ps) based on the pg_pool's
hash function.  This allows new hashes to be introduced into an existing
object store, or selection of a hash appropriate to the objects that
will be stored in a particular pool.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-06 21:55:25 -08:00
Sage Weil
cfbbcd24a6 ceph: use strong hash function for mapping objects to pgs
We were using the (weak) dcache hash function, but it was leaving lower
bits consecutive for consecutive (inode) objects.  We really want to make
the object to pg mapping random and uniform, so use a proper hash function
here.

This is Robert Jenkin's public domain hash function (with some minor
cleanup):
	http://burtleburtle.net/bob/hash/evahash.html

This is a protocol revision.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-06 16:44:05 -08:00
Sage Weil
c6cf726316 ceph: make CRUSH hash functions non-inline
These are way to big to be inline.  I missed crush/* when doing the inline
audit for akpm's review.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-06 16:39:26 -08:00
Sage Weil
1bdb70e590 ceph: clean up 'osd%d down' console msg
No ceph prefix.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-06 13:57:49 -08:00
Sage Weil
f28bcfbe66 ceph: convert port endianness
The port is informational only, but we should make it correct.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-04 16:36:12 -08:00
Sage Weil
6a18be16f7 ceph: fix sparse endian warning
Use the __le macro, even though for -1 it doesn't matter.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-04 16:36:12 -08:00
Sage Weil
51042122d4 ceph: fix endian conversions for ceph_pg
The endian conversions don't quite work with the old union ceph_pg.  Just
make it a regular struct, and make each field __le.  This is simpler and it
has the added bonus of actually working.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-04 16:36:07 -08:00
Sage Weil
63f2d21195 ceph: use fixed endian encoding for ceph_entity_addr
We exchange struct ceph_entity_addr over the wire and store it on disk.
The sockaddr_storage.ss_family field, however, is host endianness.  So,
fix ss_family endianness to big endian when sending/receiving over the
wire.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-03 15:17:56 -08:00
Sage Weil
859e7b1493 ceph: init/destroy bdi in client create/destroy helpers
This keeps bdi setup/teardown in line with client life cycle.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-02 09:32:47 -08:00
Sage Weil
33aa96e743 crush: always return a value from crush_bucket_choose
Even when we encounter a corrupt bucket.  We still BUG().  This fixes
the warning

fs/ceph/crush/mapper.c: In function 'crush_choose':
fs/ceph/crush/mapper.c:352: warning: control may reach end of non-void function
'crush_bucket_choose' being inlined

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-01 17:53:24 -08:00
Sage Weil
63ff78b25c ceph: fix uninitialized err variable
Fixes warning
fs/ceph/xattr.c: In function '__build_xattrs':
fs/ceph/xattr.c:353: warning: 'err' may be used uninitialized in this function

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-01 17:51:15 -08:00
Noah Watkins
ff1d1f7179 ceph: fix intra strip unit length calculation
Commit 645a102581 fixes calculation of object
offset for layouts with multiple stripes per object. This updates the
calculation of the length written to take into account multiple stripes per
object.

Signed-off-by: Noah Watkins <noah@noahdesu.com>
Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-30 13:56:14 -07:00
Sage Weil
645a102581 ceph: fix object striping calculation for non-default striping schemes
We were incorrectly calculationing of object offset.  If we have multiple
stripe units per object, we need to shift to the start of the current
su in addition to the offset within the su.

Also rename bno to ono (object number) to avoid some variable naming
confusion.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-28 17:45:41 -07:00
Sage Weil
5600f5ebd3 ceph: correct comment to match striping calculation
The object extent offset is the file offset _modulo_ the stripe unit.
The code was correct, the comment was wrong.

Reported-by: Noah Watkins <jayhawk@soe.ucsc.edu>
Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-28 17:45:37 -07:00
Noah Watkins
35e054a66e ceph: remove redundant use of le32_to_cpu
Using stripe unit size calculated and saved on the stack to avoid
a redundant call to le32_to_cpu.

Signed-off-by: Noah Watkins <noah@noahdesu.com>
Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-28 17:44:37 -07:00
Noah Watkins
fbbccec9c6 ceph: replace list_entry with container_of
Usage of non-list.h list_entry function for container_of
functionality replaced with direct use of container_of.

Signed-off-by: Noah Watkins <noah@noahdesu.com>
Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-28 17:44:22 -07:00
Sage Weil
6b8051855d ceph: allocate and parse mount args before client instance
This simplifies much of the error handling during mount.  It also means
that we have the mount args before client creation, and we can initialize
based on those options.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-27 11:57:03 -07:00
Sage Weil
e53c2fe075 ceph: fix, clean up string mount arg parsing
Clearly demark int and string argument options, and do not try to convert
string arguments to ints.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-27 11:17:25 -07:00
Sage Weil
6ca874e92d ceph: silence uninitialized variable warning
Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-26 22:07:59 -07:00
Sage Weil
7b813c4602 ceph: reduce parse_mount_args stack usage
Since we've increased the max mon count, we shouldn't put the addr array
on the parse_mount_args stack.  Put it on the heap instead.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-26 22:07:53 -07:00
Sage Weil
ecb19c4649 ceph: remove small mon addr limit; use CEPH_MAX_MON where appropriate
Get rid of separate max mon limit; use the system limit instead.  This
allows mounts when there are lots of mon addrs provided by mount.ceph (as
with a host with lots of A/AAAA records).

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-22 10:53:17 -07:00
Sage Weil
232d4b0131 ceph: move directory size logic to ceph_getattr
We can't fill i_size with rbytes at the fill_file_size stage without
adding additional checks for directories.  Notably, we want st_blocks
to remain 0 on directories so that 'du' still works.

Fill in i_blocks, i_size specially in ceph_getattr instead.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-21 11:24:36 -07:00
Sage Weil
bb097ffaf8 ceph: v0.17 of client
Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-19 16:17:31 -07:00
Sage Weil
ee7fdfaff7 ceph: include preferred osd in placement seed
Mix the preferred osd (if any) into the placement seed that is fed into
the CRUSH object placement calculation.  This prevents all the placement
pgs from peering with the same osds.

Rev the osd client protocol with this change.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-19 11:42:41 -07:00
Sage Weil
8fa9765576 ceph: enable readahead
Initialized bdi->ra_pages to enable readahead.  Use 512KB default.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-16 14:44:43 -07:00
Sage Weil
76e3b390d4 ceph: move dirty caps code around
Cleanup only.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-15 18:14:44 -07:00
Sage Weil
8f3bc053c6 ceph: warn on allocation from msgpool with larger front_len
Pass the front_len we need when pulling a message off a msgpool,
and WARN if it is greater than the pool's size.  Then try to
allocate a new message (to continue without failing).

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-15 18:14:43 -07:00
Sage Weil
07bd10fb98 ceph: correct subscribe_ack msgpool payload size
Defined a struct for the SUBSCRIBE_ACK, and use that to size
the msgpool.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-15 18:14:42 -07:00
Sage Weil
afcdaea3f2 ceph: flush dirty caps via the cap_dirty list
Previously we were flushing dirty caps by passing an extra flag
when traversing the delayed caps list.  Besides being a bit ugly,
that can also miss caps that are dirty but didn't result in a
cap requeue: notably, mark_caps_dirty().

Separate the flushing into a separate helper, and traverse the
cap_dirty list.

This also brings i_dirty_item in line with i_dirty_caps: we are
on the list IFF caps != 0.  We carry an inode ref IFF
dirty_caps|flushing_caps != 0.

Lose the unused return value from __ceph_mark_caps_dirty().

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-15 18:14:35 -07:00
Sage Weil
cdc35f9627 ceph: move generic flushing code into helper
Both callers of __mark_caps_flushing() do the same work; move it
into the helper.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-14 14:43:56 -07:00
Sage Weil
f2cf418cec ceph: initialize sb->s_bdi, bdi_unregister after kill_anon_super
Writeback doesn't work without the bdi set, and writeback on
umount doesn't work if we unregister the bdi too early.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-14 14:09:07 -07:00
Sage Weil
c89136ea42 ceph: convert encode/decode macros to inlines
This avoids the fugly pass by reference and makes the code a bit easier
to read.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-14 09:59:09 -07:00
Sage Weil
535bbb5307 ceph: add version field to message header
This makes it easier for individual message types to indicate
their particular encoding, and make future changes backward
compatible.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-13 12:55:26 -07:00
Sage Weil
572033069d ceph: remove unused CEPH_MSG_{OSD,MDS}_GETMAP
Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-12 10:29:44 -07:00
Sage Weil
8fc57da4d3 ceph: ignore trailing data in monamp
This lets us extend the format more easily.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-12 10:29:24 -07:00
Sage Weil
752727a1b2 ceph: add file layout validation
This tracks updates to code shared with userspace.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-09 16:39:30 -07:00
Sage Weil
13e38c8ae7 ceph: update to mon client protocol v15
The mon request headers now include session_mon information that must
be properly initialized.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-09 16:39:27 -07:00
Sage Weil
266673db42 ceph: cancel osd requests before resending them
This ensures we don't submit the same request twice if we are kicking a
specific osd (as with an osd_reset), or when we hit a transient error and
resend.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-09 11:58:20 -07:00
Sage Weil
81b024e70f ceph: reset osd session on fault, not peer_reset
The peer_reset just takes longer (until we reconnect and discover the osd
dropped the session... which it will).

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-09 11:58:15 -07:00
Sage Weil
991abb6ecf ceph: fail gracefully on corrupt osdmap (bad pg_temp mapping)
Return an error and report a corrupt map instead of crying BUG().

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-09 11:58:11 -07:00
Sage Weil
0ba6478df7 ceph: revoke osd request message on request completion
If an osd has failed or returned and a request has been sent twice, it's
possible to get a reply and unregister the request while the request
message is queued for delivery.  Since the message references the caller's
page vector, we need to revoke it before completing.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-09 11:58:07 -07:00
Sage Weil
c1ea8823be ceph: fix osd request submission race
The osd request submission path registers the request, drops and retakes
the request_mutex, then sends it to the OSD.  A racing kick_requests could
sent it during that interval, causing the same msg to be sent twice and
BUGing in the msgr.

Fix by only sending the message if it hasn't been touched by other
threads.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-09 11:58:03 -07:00
Sage Weil
0656d11ba6 ceph: renew mon subscription before it expires
Be conservative: renew subscription once half the interval has expired.

Do not reuse sub expiration to control hunting.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-08 10:39:08 -07:00
Sage Weil
e251e28808 ceph: fix mdsmap decoding when multiple mds's are present
A misplaced sizeof() around namelen was throwing things off.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-07 16:38:55 -07:00
Sage Weil
b28813a61d ceph: gracefully avoid empty crush buckets
This avoids a divide by zero when the input and/or map are
malformed.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-07 10:59:34 -07:00
Sage Weil
b195befd9a ceph: include preferred_osd in file layout virtual xattr
Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-07 10:59:30 -07:00
Sage Weil
fa0b72e9e2 ceph: show meaningful version on module load
Kill the old git revision; print the ceph version and protocol
versions instead.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-07 10:59:10 -07:00
Sage Weil
e324b8f991 ceph: document shared files in README
Document files shared between kernel and user code trees.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-06 12:21:17 -07:00
Sage Weil
9030aaf9bf ceph: Kconfig, Makefile
Kconfig options and Makefile.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-06 11:31:15 -07:00