Easy workaround for a lockdep splat - and since bch2_prt_backtrace() is
only used in debug code this is fine.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
This improves io_opts() and makes it a non-inline function - it's big
enough that it probably shouldn't be.
Also, bch_io_opts no longer needs fields for whether options are
defined, so we can slim it down a bit.
We'd like to stop passing around the full bch_io_opts, but that'll be
tricky because of bch2_rebalance_add_key().
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Provide inline versions of some allocation functions
- bch2_alloc_sectors_done_inlined()
- bch2_alloc_sectors_append_ptrs_inlined()
and use them in the core IO path.
Also, inline bch2_extent_update_i_size_sectors() and
bch2_bkey_append_ptr().
In the core write path, function call overhead matters - every function
call is a jump to a new location and a potential cache miss.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
This separates out the slowpath into a separate function, and inlines
bch2_alloc_v4_mut into bch2_trans_start_alloc_update(), the main place
it's called.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
When we need to add more replicas to an extent, it might be the case
that we already have a replica on every device, but some of them are
cached.
This patch fixes a bug where we'd spin on that extent because the write
path fails to find a device we can allocate from: we allow allocating
from devices that already have cached replicas on them, and change
bch2_data_update_index_update() to drop the cached replica if needed.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
When errors=panic, we need to dump transaction updates before calling
bch2_inconsistent_error().
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
We now have bch2_trans_inconsistent() which generically does the same
thing - dumps pending btree transaction updates.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Old inode formats don't have all the fields of the current inode format:
when unpacking inodes in the current format we can thus skip zeroing out
the destination buffer, but that doesn't work on for the old formats.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
In debug mode, we now track where btree iterators and paths are
initialized/allocated - helpful in tracking down btree path overflows.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
This is something we need to do more widely: instead of bothering with
GFP_NOIO/GFP_NOFS, if we need to allocate memory while holding locks:
- first attempt the allocation with GFP_NOWAIT
- if that fails, drop btree locks with bch2_trans_unlock(), then
retry with GFP_KERNEL.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
We need to take a ref on a path while we're traversing it: this fixes a
bug with paths getting reused while being traversed, in the key cache
fill code.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
The btree node read path has the ability to kick off an asynchronous
btree node rewrite if we saw and corrected an error. Previously this was
only used for errors that caused one of the replicas to be unusable -
this patch plumbs it through to all error paths, so that normal fsck
errors can be corrected.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
This patch
- Adds a mechanism for queuing up journal entries prior to the journal
being started, which will be used for early journal log messages
- Adds bch2_fs_log_msg() and improves bch2_trans_log_msg(), which now
take format strings. bch2_fs_log_msg() can be used before or after
the journal has been started, and will use the appropriate mechanism.
- Deletes the now obsolete bch2_journal_log_msg()
- And adds more log messages to the recovery path - messages for
journal/filesystem started, journal entries being blacklisted, and
journal replay starting/finishing.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
With CONFIG_FORTIFY_SOURCE, the compiler attempts to warn about mempcys
that extend past struct field boundaries. This results in some spurious
warnings where we use embedded variable length structs, this patch
switches to unsafe_mecpy() to fix the warnings.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
It's now legal for the pin fifo to be empty, which means this code needs
to be updated in order to not hit an assert.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
While a btree transaction is running, we hold a SRCU read lock on the
btree key cache that prevents btree key cache keys from being freed -
this is so that relock() operations won't access freed memory.
The downside of this is that long running btree transactions prevent
memory from being freed from the key cache. This adds a check in
bch2_trans_begin() - if the transaction has been running longer than 1
second, drop and retake the SRCU read lock and zero out pointers to
unlock key cache paths.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Add a few easy unlikely() optimizations. These are mainly worthwhile
because the compiler will (usually) put the branch-not-taken path at the
end of the function, meaning better icache utilization.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
This introduces some new conveniences, to help cut down on boilerplate:
- bch2_trans_kmalloc_nomemzero() - performance optimiation
- bch2_bkey_make_mut()
- bch2_bkey_get_mut()
- bch2_bkey_get_mut_typed()
- bch2_bkey_alloc()
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
If it so happens that we crash while dirty, meaning we don't have the
superblock clean section, and we erroneously mark a journal entry we
wrote as blacklisted, we won't be able to recover.
This patch fixes this by adding a fallback: if we've got no superblock
clean section, and no non-ignored journal entries, we try the most
recent ignored journal entry.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
We weren't resetting filesystem & device usage when restarting gc, which
was spotted when free bucket counters overflowed - whoops.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Previously, we were journalling extra_journal_entries (which is used for
new btree roots, and irreversably mutates system state) before calling
bch2_trans_fs_usage_apply(), which can fail - whoops.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
When we started stashing the key being overwritten in
btree_insert_entry, this introduced a typical iterator invalidation
problem, triggered by btree node splits or resorts.
Previously, dealt with this by unconditionally re-validating those
stashed pointers in the transaction commit path. This patch gets rid of
that by doing it only when needed, in bch2_trans_node_add() or
bch2_trans_node_reinit_iter().
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
We shouldn't be overloading standard error codes now that we have
provisions for bcachefs-specific errorcodes: this patch converts super.c
and super-io.c to per error site errcodes, with a bit of cleanup.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
This isn't actually an error condition, this just indicates a normal
shutdown - no reason for these to be in the log.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bch2_btree_iter_peek_upto() in snapshots mode may need to keep a
btree_path for the insert position, not just the position of the key
we're returning. The code was incorrectly assuming this would be in the
same btree node - we were missing a bch2_btree_path_traverse() call.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
bch2_journal_keys_peek_upto() was comparing against btree_id & level
incorrectly - fix this by using __journal_key_cmp().
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
This fixes a (harmless) broken invariant in __bch2_btree_path_set_pos():
iterators to interior nodes should point to the first non whiteout.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
This just cleans up and simplifies the code that decides where to resume
writing in the journal - when the code was originally written we weren't
saving the precise location of every journal write found.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
