More testing of NOR flash against power cuts showed that sometimes
eraseblocks may be unwritable, and we cannot really invalidate
them before erasure. But in this case the eraseblock probably
contains garbage anyway, and we do not have to invalidate the
headers. This assumption might be not true, but this is at least
what I have observed. So if we cannot invalidate the headers,
we make sure that the PEB does not contain valid VID header.
If this is true, everything is fine, otherwise we panic.
Useful for debugging problems, compiled in only if UBI debugging
is enabled. This patch also makes the UBI writing function dump
the flash if it fails to write.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
In case of NOR flash, UBI zeroes EC and VID headers' magic,
in order to detect interrupted erasures. It first zeroes out
the EC magic, then VID magic. However, if a power cut happens
in between, we'll end up with a corrupted EC header and a valid
VID header, in which case UBI accepts the PEB, but prints a
warning. This patch makes sure we first zero out the VID
magic, then the EC magic, not vice versa. This is just a
small amendment to prevent warning messages.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
The recent "UBI: fix NOR flash recovery" introduced compilation
warnings which were immediately spotted by our linux-next keeper.
This patch fixes them.
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
This commit fixes NOR flash recovery issues observed with Spansion
S29GL512N NOR.
When NOR erases, it first fills PEBs with zeroes, then sets all bytes
to 0xFF. Filling with zeroes starts from the end of the PEB. And when
power is cut, this results in PEBs containing correct EC and VID headers
but corrupted with zeros at the end. This confuses UBI and it mistakinly
accepts these PEBs and associate them with LEBs.
Fis this issue by zeroing EC and VID magics before erasing PEBs, to
make UBI later refuse zem.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Move the image seq. number handling from I/O level to the scanning
lever, where it really belongs to. Move the @image_seq_set variable
to the @struct ubi_scan_info structure, which exists only during
scanning.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
An image sequence number is added to the UBI erase-counter header
to be able determine if the root file system contains a mixture
of old and new images (because the flashing failed to complete).
A change to nolo is also needed for this to take effect.
Signed-off-by: Adrian Hunter <adrian.hunter@nokia.com>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
The 'paranoid_check_empty()' is bogus because, which is easilly
seen on NOR flash, which has long erase cycles, and which may
easilly end-up with half-erased eraseblocks. In this case the
paranoid check fails. I is just wrong to assume that PEBs which
do not have EC headers always contain all 0xFF. Such assumption
should not be made on the I/O level, which is quite low.
Thus, just kill the check.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
This patch adds code which makes sure eraseblocks contain all 0xFF
bytes before starting using them. The verification is done only when
debugging checks are enabled.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
When paranoid checs are enabled, the 'io_paral' test from the
'mtd-utils' package fails. The symptoms are:
UBI error: paranoid_check_all_ff: flash region at PEB 3973:512, length 15872 does not contain all 0xFF bytes
UBI error: paranoid_check_all_ff: paranoid check failed for PEB 3973
UBI: hex dump of the 512-16384 region
It turned out to be a bug in the checking function. Suppose there
are 2 tasks - A and B. Task A is the wear-levelling working
('wear_leveling_worker()'). It is reading the VID header to find
which LEB this PEB belongs to. Say, task A is reading header
of PEB X. Suppose PEB X is unmapped, and has no VID header.
Task B is trying to write to PEB X.
Task A: in 'ubi_io_read_vid_hdr()': reads the VID header from PEB X.
The read data contain all 0xFF bytes.
Task B: writes VID header and some data to PEB X
Task A: assumes PEB X is empty, calls 'paranoid_check_all_ff()', which
fails.
The solution for this problem is to make 'paranoid_check_all_ff()'
re-read the VID header, re-check it, and only if it is not there,
check the rest. This now implemented by the 'paranoid_check_empty()'
function.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
The 'ubi_io_read_vid_hdr()' and 'ubi_io_read_ec_hdr()' function
have the 'verbose' argument which controls whether they should
print a warning if the VID/EC header was not found or was corrupted.
Some callers require the headers to be OK, and pass 1. Some allow
a corrupted/not present header, and pass 0.
if (UBI_IO_DEBUG)
verbose = 1;
And UBI_IO_DEBUG is 1 if CONFIG_MTD_UBI_DEBUG_MSG_BLD is true. So in
this case the warning is printed all the time. This confuses people.
Thus, do not print the messages as warnings if UBI_IO_DEBUG is true,
but print them as debugging messages instead.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
No functional changes, just tweak comments to make kernel-doc
work fine and stop complaining.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Just out or curiousity ran checkpatch.pl for whole UBI,
and discovered there are quite a few of stylistic issues.
Fix them.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
If bit-flips happen often, UBI prints to many messages. Lessen
the amount by only printing the messages when the PEB has been
scrubbed. Also, print torturing messages.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Hch asked not to use "unit" for sub-systems, let it be so.
Also some other commentaries modifications.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Make I/O function to be always verbose when about CRC errors
and magic number errors when I/O debugging is enabled.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
When NAND detects an ECC error, it returns -EBADMSG. It does not
stop reading requested data if one page has an ECC error, it keeps
going and reads all the requested data. If it fails to read all
the data, it does not return -EBADMSG, but returns the error code
which reflects the reason of the failure.
But some drivers may have bugs (e.g., OneNAND had) and stop reading
after the first ECC error, so it returns -EBADMSG. In turn, UBI
propagates this up to the caller. The caller will treat this as
"all the requested data was read, but there was an ECC error".
So we change the error code to -EIO if it is -EBADMSG and the read
length is less then the requested length. We also add an assertion,
so if UBI debugging is enabled, UBI will bug.
Pointed-to-by: Adrian Hunter <ext-adrian.hunter@nokia.com>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Similar reason as in case of the previous patch: it causes
deadlocks if a filesystem with writeback support works on top
of UBI. So pre-allocate needed buffers when attaching MTD device.
We also need mutexes to protect the buffers, but they do not
cause much contantion because they are used in recovery, torture,
and WL copy routines, which are called seldom.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Use GFP_NOFS flag when allocating memory on I/O path, because otherwise
we may deadlock the filesystem which works on top of us. We observed
the deadlocks with UBIFS. Example:
VFS->FS lock a lock->UBI->kmalloc()->VFS writeback->FS locks the same
lock again.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
I hit those situations and found out lack of print messages. Add more prints
when erase problems occur.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
There is signed multiplication assigned to unsigned ei.addr in io.c.
This causes wrong addresses for big multiplication.This patch solves the
problem.
Signed-off-by: Brijesh Singh <brijesh.s.singh@gmail.com>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Kill UBI's homegrown endianess handling and replace it with
the standard kernel endianess handling.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
UBI allocates temporary buffers of PEB size, which may be 256KiB.
Use vmalloc instead of kmalloc for such big temporary buffers.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
UBI (Latin: "where?") manages multiple logical volumes on a single
flash device, specifically supporting NAND flash devices. UBI provides
a flexible partitioning concept which still allows for wear-levelling
across the whole flash device.
In a sense, UBI may be compared to the Logical Volume Manager
(LVM). Whereas LVM maps logical sector numbers to physical HDD sector
numbers, UBI maps logical eraseblocks to physical eraseblocks.
More information may be found at
http://www.linux-mtd.infradead.org/doc/ubi.html
Partitioning/Re-partitioning
An UBI volume occupies a certain number of erase blocks. This is
limited by a configured maximum volume size, which could also be
viewed as the partition size. Each individual UBI volume's size can
be changed independently of the other UBI volumes, provided that the
sum of all volume sizes doesn't exceed a certain limit.
UBI supports dynamic volumes and static volumes. Static volumes are
read-only and their contents are protected by CRC check sums.
Bad eraseblocks handling
UBI transparently handles bad eraseblocks. When a physical
eraseblock becomes bad, it is substituted by a good physical
eraseblock, and the user does not even notice this.
Scrubbing
On a NAND flash bit flips can occur on any write operation,
sometimes also on read. If bit flips persist on the device, at first
they can still be corrected by ECC, but once they accumulate,
correction will become impossible. Thus it is best to actively scrub
the affected eraseblock, by first copying it to a free eraseblock
and then erasing the original. The UBI layer performs this type of
scrubbing under the covers, transparently to the UBI volume users.
Erase Counts
UBI maintains an erase count header per eraseblock. This frees
higher-level layers (like file systems) from doing this and allows
for centralized erase count management instead. The erase counts are
used by the wear-levelling algorithm in the UBI layer. The algorithm
itself is exchangeable.
Booting from NAND
For booting directly from NAND flash the hardware must at least be
capable of fetching and executing a small portion of the NAND
flash. Some NAND flash controllers have this kind of support. They
usually limit the window to a few kilobytes in erase block 0. This
"initial program loader" (IPL) must then contain sufficient logic to
load and execute the next boot phase.
Due to bad eraseblocks, which may be randomly scattered over the
flash device, it is problematic to store the "secondary program
loader" (SPL) statically. Also, due to bit-flips it may become
corrupted over time. UBI allows to solve this problem gracefully by
storing the SPL in a small static UBI volume.
UBI volumes vs. static partitions
UBI volumes are still very similar to static MTD partitions:
* both consist of eraseblocks (logical eraseblocks in case of UBI
volumes, and physical eraseblocks in case of static partitions;
* both support three basic operations - read, write, erase.
But UBI volumes have the following advantages over traditional
static MTD partitions:
* there are no eraseblock wear-leveling constraints in case of UBI
volumes, so the user should not care about this;
* there are no bit-flips and bad eraseblocks in case of UBI volumes.
So, UBI volumes may be considered as flash devices with relaxed
restrictions.
Where can it be found?
Documentation, kernel code and applications can be found in the MTD
gits.
What are the applications for?
The applications help to create binary flash images for two purposes: pfi
files (partial flash images) for in-system update of UBI volumes, and plain
binary images, with or without OOB data in case of NAND, for a manufacturing
step. Furthermore some tools are/and will be created that allow flash content
analysis after a system has crashed..
Who did UBI?
The original ideas, where UBI is based on, were developed by Andreas
Arnez, Frank Haverkamp and Thomas Gleixner. Josh W. Boyer and some others
were involved too. The implementation of the kernel layer was done by Artem
B. Bityutskiy. The user-space applications and tools were written by Oliver
Lohmann with contributions from Frank Haverkamp, Andreas Arnez, and Artem.
Joern Engel contributed a patch which modifies JFFS2 so that it can be run on
a UBI volume. Thomas Gleixner did modifications to the NAND layer. Alexander
Schmidt made some testing work as well as core functionality improvements.
Signed-off-by: Artem B. Bityutskiy <dedekind@linutronix.de>
Signed-off-by: Frank Haverkamp <haver@vnet.ibm.com>
