On 2/25/21 6:56 PM, John Reiser wrote:

...

Tools such as ls or stat report the size of a directory. Of course it is not the content size. stat -c %s  /home/sergio/.config 6550

What does 6550 mean in btrfs context?

Regardless of filesystem type, the size of a directory is the sum of the sizes of the struct linux_dirent (or linux_dirent64) for the filenames of the contained files. See the manual page "man 2 getdents".

That's not correct; dirents are in-memory structures, unrelated to what the [fl]stat(2) interface used by ls returns for a directory.

The size returned by stat(2) (aka ls) of a directory inode is filesystem implementation dependent, and AFAIK has no well-defined meaning. stat(2) refers to st_size only for files and symlinks, not for directories. Same w/ POSIX:

off_t st_size For regular files, the file size in bytes. For symbolic links, the length in bytes of the pathname contained in the symbolic link.

So, as one example on ext4 - directories never shrink.

# mkdir dir # ls -ld dir drwxr-xr-x. 2 root root 4096 Feb 26 00:48 dir

# touch dir/123456789 # ls -ld dir drwxr-xr-x. 2 root root 4096 Feb 26 01:00 dir

# for I in `seq 1 2000`; do touch dir/longfilename$I; done # ls -ld dir drwxr-xr-x. 2 root root 69632 Feb 26 00:49 dir

# rm -f dir/* # ls -ld dir drwxr-xr-x. 2 root root 69632 Feb 26 00:49 dir

69632 byte directory with no files in it, wheeee.

xfs is different:

# mkdir dir # ls -ld dir drwxr-xr-x. 2 root root 6 Feb 26 00:58 dir

# touch dir/123456789 # ls -ld dir drwxr-xr-x. 2 root root 23 Feb 26 00:59 dir

# for I in `seq 1 2000`; do touch dir/longfilename$I; done # ls -ld dir drwxr-xr-x. 2 root root 65536 Feb 26 00:59 dir

# rm -f dir/* # ls -ld dir drwxr-xr-x. 2 root root 6 Feb 26 00:59 dir

btrfs is still different:

# mkdir dir # ls -ld dir drwxr-xr-x. 1 root root 0 Feb 26 01:05 dir

# touch dir/123456789 # ls -ld dir drwxr-xr-x. 1 root root 18 Feb 26 01:06 dir

# for I in `seq 1 2000`; do touch dir/longfilename$I; done # ls -ld dir drwxr-xr-x. 1 root root 61804 Feb 26 01:06 dir

# rm -f dir/* # ls -ld dir drwxr-xr-x. 1 root root 0 Feb 26 01:06 dir

In short, "size" of a dir doesn't tell you much.

-Eric

On 2/26/21 7:07 AM, Eric Sandeen wrote:

In short, "size" of a dir doesn't tell you much.

In ext4 it tells you (proportionally to other dirs) how many files you have got at maximum in that directory in its entire history. :-)

Regards.

On 3/16/21, David Howells wrote:

John Reiser jreiser@bitwagon.com wrote:

...

See the manual page "man 2 getdents".

Um, which bit? I don't see anything obvious to that end.

On that manual page: ===== The system call getdents() reads several linux_dirent structures from the directory referred to by the open file descriptor fd into the buffer pointed to by dirp. [snip]] On success, the number of bytes read is returned. =====

So the return value is related to the size of the directory; the sum of the values returned before End-Of-File should be quite close to the .st_size of the directory. If a program is walking through the directory, reading all the entries via getdents64(), then .st_size of the directory is the only thing known in advance about the total size. (Of course anything involving a directory can depend on concurrent create/delete/rename of files within the directory.)

On AFS directories are handled as files that the filesystem downloads and parses locally. The size returned in st_size is the size of the directory content blob.

* Steven Whitehouse:

If you are looking for a hint on how large a buffer to allocate, then st_blksize is generally used as a hint for directory reads, or otherwise, a fixed size buffer or a page or two. The st_size field is meaningless for directories and you'll get all kinds of odd results depending on the filesystem that is in use, so best avoided,

One caveat is that st_blksize can be unreasonably large, particularly with NFS and large configured read buffers. So maybe cap that value at 32 KiB, but then why not use 32 KiB unconditionally?

Another issue is that too small a buffer might make some directories unreadable, given that Linux does not actually enforce NAME_MAX. Some file systems support NAME_MAX Unicode codepoints in a name, which is encoded to roughly 0.75 KiB in UTF-8.

Thanks, Florian

John Reiser jreiser@bitwagon.com wrote:

On 3/16/21, David Howells wrote:

...

John Reiser jreiser@bitwagon.com wrote:

...

See the manual page "man 2 getdents".

Um, which bit? I don't see anything obvious to that end.

On that manual page:

The system call getdents() reads several linux_dirent structures from the directory referred to by the open file descriptor fd into the buffer pointed to by dirp. [snip]] On success, the number of bytes read is returned. =====

It doesn't say anything about the size of the directory there. "Number of bytes read is returned" should be taken as how much of the user buffer was filled - information you need to know to be able to parse it.

Further, there's getdents() and there's getdents64() and their structs are of different sizes. By your logic st_size would have to be the number of linux_dirent structs for use with the former and the number of linux_dirent64 structs for use with the latter... And then there's readdir() as well with it's old_linux_dirent struct.

So, no, it cannot work like that.

David