123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191 |
- NXFFS README
- ^^^^^^^^^^^^
- This README file contains information about the implementation of the NuttX
- wear-leveling FLASH file system, NXFFS.
- Contents:
- General NXFFS organization
- General operation
- Headers
- NXFFS Limitations
- Multiple Writers
- ioctls
- Things to Do
- General NXFFS organization
- ==========================
- The following example assumes 4 logical blocks per FLASH erase block. The
- actual relationship is determined by the FLASH geometry reported by the MTD
- driver.
- ERASE LOGICAL Inodes begin with a inode header. inode may
- BLOCK BLOCK CONTENTS be marked as "deleted," pending re-packing.
- n 4*n --+--------------+
- |BBBBBBBBBBBBBB| Logic block header
- |IIIIIIIIIIIIII| Inodes begin with a inode header
- |DDDDDDDDDDDDDD| Data block containing inode data block
- | (Inode Data) |
- 4*n+1 --+--------------+
- |BBBBBBBBBBBBBB| Logic block header
- |DDDDDDDDDDDDDD| Inodes may consist of multiple data blocks
- | (Inode Data) |
- |IIIIIIIIIIIIII| Next inode header
- | | Possibly a few unused bytes at the end of a block
- 4*n+2 --+--------------+
- |BBBBBBBBBBBBBB| Logic block header
- |DDDDDDDDDDDDDD|
- | (Inode Data) |
- 4*n+3 --+--------------+
- |BBBBBBBBBBBBBB| Logic block header
- |IIIIIIIIIIIIII| Next inode header
- |DDDDDDDDDDDDDD|
- | (Inode Data) |
- n+1 4*(n+1) --+--------------+
- |BBBBBBBBBBBBBB| Logic block header
- | | All FLASH is unused after the end of the final
- | | inode.
- --+--------------+
- General operation
- =================
- Inodes are written starting at the beginning of FLASH. As inodes are
- deleted, they are marked as deleted but not removed. As new inodes are
- written, allocations proceed to toward the end of the FLASH -- thus,
- supporting wear leveling by using all FLASH blocks equally.
- When the FLASH becomes full (no more space at the end of the FLASH), a
- re-packing operation must be performed: All inodes marked deleted are
- finally removed and the remaining inodes are packed at the beginning of
- the FLASH. Allocations then continue at the freed FLASH memory at the
- end of the FLASH.
- Headers
- =======
- BLOCK HEADER:
- The block header is used to determine if the block has every been
- formatted and also indicates bad blocks which should never be used.
- INODE HEADER:
- Each inode begins with an inode header that contains, among other things,
- the name of the inode, the offset to the first data block, and the
- length of the inode data.
- At present, the only kind of inode support is a file. So for now, the
- term file and inode are interchangeable.
- INODE DATA HEADER:
- Inode data is enclosed in a data header. For a given inode, there
- is at most one inode data block per logical block. If the inode data
- spans more than one logical block, then the inode data may be enclosed
- in multiple data blocks, one per logical block.
- NXFFS Limitations
- =================
- This implementation is very simple as, as a result, has several limitations
- that you should be aware before opting to use NXFFS:
- 1. Since the files are contiguous in FLASH and since allocations always
- proceed toward the end of the FLASH, there can only be one file opened
- for writing at a time. Multiple files may be opened for reading.
- 2. Files may not be increased in size after they have been closed. The
- O_APPEND open flag is not supported.
- 3. Files are always written sequential. Seeking within a file opened for
- writing will not work.
- 4. There are no directories, however, '/' may be used within a file name
- string providing some illusion of directories.
- 5. Files may be opened for reading or for writing, but not both: The O_RDWR
- open flag is not supported.
- 6. The re-packing process occurs only during a write when the free FLASH
- memory at the end of the FLASH is exhausted. Thus, occasionally, file
- writing may take a long time.
- 7. Another limitation is that there can be only a single NXFFS volume
- mounted at any time. This has to do with the fact that we bind to
- an MTD driver (instead of a block driver) and bypass all of the normal
- mount operations.
- Multiple Writers
- ================
- As mentioned in the limitations above, there can be only one file opened
- for writing at a time. If one thread has a file opened for writing and
- another thread attempts to open a file for writing, then that second
- thread will be blocked and will have to wait for the first thread to
- close the file.
- Such behavior may or may not be a problem for your application, depending
- (1) how long the first thread keeps the file open for writing and (2) how
- critical the behavior of the second thread is. Note that writing to FLASH
- can always trigger a major FLASH reorganization and, hence, there is no
- way to guarantee the first condition: The first thread may have the file
- open for a long time even if it only intends to write a small amount.
- Also note that a deadlock condition would occur if the SAME thread
- attempted to open two files for writing. The thread would would be
- blocked waiting for itself to close the first file.
- ioctls
- ======
- The file system supports to ioctls:
- FIOC_REFORMAT: Will force the flash to be erased and a fresh, empty
- NXFFS file system to be written on it.
- FIOC_OPTIMIZE: Will force immediate repacking of the file system. This
- will avoid the delays to repack the file system in the emergency case
- when all of the FLASH memory has been used. Instead, you can defer
- the garbage collection to time when the system is not busy. Calling
- this function on a thrashing file system will increase the amount of
- wear on the FLASH if you use this frequently!
- Things to Do
- ============
- - The statfs() implementation is minimal. It should have some calculation
- of the f_bfree, f_bavail, f_files, f_ffree return values.
- - There are too many allocs and frees. More structures may need to be
- pre-allocated.
- - The file name is always extracted and held in allocated, variable-length
- memory. The file name is not used during reading and eliminating the
- file name in the entry structure would improve performance.
- - There is a big inefficiency in reading. On each read, the logic searches
- for the read position from the beginning of the file each time. This
- may be necessary whenever an lseek() is done, but not in general. Read
- performance could be improved by keeping FLASH offset and read positional
- information in the read open file structure.
- - Fault tolerance must be improved. We need to be absolutely certain that
- any FLASH errors do not cause the file system to behavior incorrectly.
- - Wear leveling might be improved (?). Files are re-packed at the front
- of FLASH as part of the clean-up operation. However, that means the files
- that are not modified often become fixed in place at the beginning of
- FLASH. This reduces the size of the pool moving files at the end of the
- FLASH. As the file system becomes more filled with fixed files at the
- front of the device, the level of wear on the blocks at the end of the
- FLASH increases.
- - When the time comes to reorganization the FLASH, the system may be
- unavailable for a long time. That is a bad behavior. What is needed,
- I think, is a garbage collection task that runs periodically so that
- when the big reorganization event occurs, most of the work is already
- done. That garbage collection should search for valid blocks that no
- longer contain valid data. It should pre-erase them, put them in
- a good but empty state... all ready for file system re-organization.
- NOTE: There is the FIOC_OPTIMIZE IOCTL command that can be used by an
- application for force garbage collection when the system is not busy.
- If used judiciously by the application, this can eliminate the problem.
- - And worse, when NXFSS reorganization the FLASH a power cycle can
- damage the file system content if it happens at the wrong time.
- - The current design does not permit re-opening of files for write access
- unless the file is truncated to zero length. This effectively prohibits
- implementation of a proper truncate() method which should alter the
- size of a previously written file. There is some fragmentary logic in
- place but even this is conditioned out with __NO_TRUNCATE_SUPPORT__.
|