cluster-commits March 2016

cluster-commits@lists.stg.fedorahosted.org

2 participants
13 discussions

gfs2-utils: master - glocktop: Fix a tight loop under nohup
by Andrew Price 22 Mar '16

22 Mar '16

Gitweb: http://git.fedorahosted.org/git/?p=gfs2-utils.git;a=commitdiff;h=72553fb7cd… Commit: 72553fb7cdfa7d69e0d7473cfd388078d7dedae5 Parent: 88207327890914416881ee096d4a9a96806fb355 Author: Andrew Price <anprice(a)redhat.com> AuthorDate: Tue Mar 1 22:16:41 2016 +0000 Committer: Andrew Price <anprice(a)redhat.com> CommitterDate: Tue Mar 22 14:48:30 2016 +0000 glocktop: Fix a tight loop under nohup When glocktop is run under nohup, /dev/null is redirected to stdin, which flags up EOF and causes the file descriptor to always be in a ready state when select()ed. This causes select() to return immediately instead of timing out after the refresh delay. Remove stdin from the fd_set when we get EOF to avoid getting into a tight loop when run under nohup. Also fix up some messages relating to the refresh interval. Signed-off-by: Andrew Price <anprice(a)redhat.com> --- gfs2/glocktop/glocktop.c | 20 ++++++++++++++------ 1 files changed, 14 insertions(+), 6 deletions(-) diff --git a/gfs2/glocktop/glocktop.c b/gfs2/glocktop/glocktop.c index 43c3c3f..b418d2a 100644 --- a/gfs2/glocktop/glocktop.c +++ b/gfs2/glocktop/glocktop.c @@ -147,7 +147,6 @@ enum summary_types { char debugfs[PATH_MAX]; int termcols = 80, termlines = 30, done = 0; unsigned glocks = 0; -int refresh_time = REFRESH_TIME; const char *termtype; WINDOW *wind; int bufsize = 4 * 1024 * 1024; @@ -1625,7 +1624,7 @@ static void usage(void) "[-c] [-D] [-H] [-r] [-t]\n"); printf("\n"); printf("-i : Runs glocktop in interactive mode.\n"); - printf("-d : delay between refreshes, in seconds (default: 3).\n"); + printf("-d : delay between refreshes, in seconds (default: %d).\n", REFRESH_TIME); printf("-n : stop after <iter> refreshes.\n"); printf("-H : don't show Held glocks, even if not waited on, excluding " "iopen\n"); @@ -1645,7 +1644,7 @@ int main(int argc, char **argv) char fn[PATH_MAX]; struct dirent *dent; int retval; - struct timeval tv; + int refresh_time = REFRESH_TIME; fd_set readfds; char string[96]; int ch, i, dlmwaiters = 0, dlmgrants = 0; @@ -1654,6 +1653,7 @@ int main(int argc, char **argv) int show_held = 1, help = 0; int interactive = 0; int summary = 10; + int nfds = STDIN_FILENO + 1; prog_name = argv[0]; memset(glock, 0, sizeof(glock)); @@ -1669,7 +1669,7 @@ int main(int argc, char **argv) refresh_time = atoi(optarg); if (refresh_time < 1) { fprintf(stderr, "Error: delay %d too small; " - "must be > 1\n", refresh_time); + "must be at least 1\n", refresh_time); exit(-1); } break; @@ -1759,6 +1759,8 @@ int main(int argc, char **argv) } while (!done) { + struct timeval tv; + sprintf(fn, "%s/gfs2/", debugfs); dir = opendir(fn); @@ -1829,8 +1831,9 @@ int main(int argc, char **argv) tv.tv_sec = refresh_time; tv.tv_usec = 0; FD_ZERO(&readfds); - FD_SET(0, &readfds); - retval = select(1, &readfds, NULL, NULL, &tv); + if (nfds != 0) + FD_SET(STDIN_FILENO, &readfds); + retval = select(nfds, &readfds, NULL, NULL, &tv); if (retval) { if (interactive) ch = getch(); @@ -1856,6 +1859,11 @@ int main(int argc, char **argv) if (refresh_time < 1) refresh_time = 1; break; + /* When we get EOF on stdin, remove it from the fd_set + to avoid shorting out the select() */ + case EOF: + nfds = 0; + break; } } iters_done++;

1 0

gfs2-utils: master - mkfs.gfs2(8) man page improvements
by Andrew Price 22 Mar '16

22 Mar '16

Gitweb: http://git.fedorahosted.org/git/?p=gfs2-utils.git;a=commitdiff;h=8820732789… Commit: 88207327890914416881ee096d4a9a96806fb355 Parent: 0103dc07a3bc9a5bccca499f2362ed91e0eb2acd Author: Andrew Price <anprice(a)redhat.com> AuthorDate: Mon Feb 22 16:56:02 2016 +0000 Committer: Andrew Price <anprice(a)redhat.com> CommitterDate: Tue Mar 22 14:46:15 2016 +0000 mkfs.gfs2(8) man page improvements This is a fairly substantial overhaul of the mkfs.gfs2 man page. Among the changes are: - Remove references to gfs2_mkfs (bz#1283866) - Advise users to stick to the defaults - Clarify the implications of some options (e.g. -j) - Use gfs2 instead of Global File System - Remove unnecessary CamelCase and Title Case - Various language and formatting improvements Signed-off-by: Andrew Price <anprice(a)redhat.com> --- gfs2/man/mkfs.gfs2.8 | 157 ++++++++++++++++++++++++++++---------------------- 1 files changed, 88 insertions(+), 69 deletions(-) diff --git a/gfs2/man/mkfs.gfs2.8 b/gfs2/man/mkfs.gfs2.8 index f480082..342a636 100644 --- a/gfs2/man/mkfs.gfs2.8 +++ b/gfs2/man/mkfs.gfs2.8 @@ -1,55 +1,61 @@ .TH mkfs.gfs2 8 .SH NAME -mkfs.gfs2 - Make a GFS2 filesystem +mkfs.gfs2 - create a gfs2 filesystem .SH SYNOPSIS .B mkfs.gfs2 -[\fIOPTION\fR]... \fIDEVICE\fR \fI[ block-count ]\fR +[\fIoptions\fR] \fIdevice\fR \fI[block-count]\fR .SH DESCRIPTION -mkfs.gfs2 is used to create a Global File System. +mkfs.gfs2 is used to create a gfs2 file system. .SH OPTIONS +The default values of the following options have been chosen for best results. +In most cases, there should be no need to choose different values. The +exceptions to this are the number of journals (\fB-j\fP) and the lock table +(\fB-t\fP), as these options will be specific to your cluster. .TP -\fB-b\fP \fIBlockSize\fR -Set the filesystem block size to \fIBlockSize\fR (must be a power of -two). The minimum block size is 512. The FS block size cannot exceed -the machine's memory page size. On the most architectures (i386, -x86_64, s390, s390x), the memory page size is 4096 bytes. On other -architectures it may be bigger. The default block size is 4096 bytes. -In general, GFS2 filesystems should not deviate from the default value. +\fB-b\fP \fIbytes\fR +Set the filesystem block size to \fIbytes\fR which must be a power of two. The +minimum block size is 512 and the block size cannot exceed the machine's memory +page size, which on most architectures is 4096 bytes. The default block size +is 4096 bytes. .TP -\fB-c\fP \fIMegaBytes\fR -Initial size of each journal's quota change file +\fB-c\fP \fImegabytes\fR +Initial size of each journal's quota change file. The default is 1MB. .TP \fB-D\fP Enable debugging output. .TP \fB-h\fP -Print out a help message describing available -options, then exit. +Print out a help message describing the available options, then exit. .TP -\fB-J\fP \fIMegaBytes\fR -The size of the journals in Megabytes. The default journal size is -128 megabytes. The minimum size is 8 megabytes. +\fB-J\fP \fImegabytes\fR +The size of each journal. The default journal size is 128 megabytes and the +minimum size is 8 megabytes. .TP -\fB-j\fP \fINumber\fR -The number of journals for gfs2_mkfs to create. You need at least one -journal per machine that will mount the filesystem. If this option is -not specified, one journal will be created. +\fB-j\fP \fIjournals\fR +The number of journals for mkfs.gfs2 to create. At least one journal is +required for each machine that will mount the filesystem concurrently. If this +option is not specified, only one journal will be created. This number may be +used as an indicator of the number of nodes in the cluster in order to optimize +the layout of the filesystem. As such, it is best to set this option with the +maximum number of mounters in mind than to add more journals later. .TP \fB-K\fP -Keep, do not attempt to discard blocks at mkfs time (discarding blocks -initially is useful on solid state devices and sparse / thin-provisioned -storage). +Do not attempt to discard the block device contents. Issuing discards to the +device allows some solid state devices and sparse or thin-provisioned storage +devices to optimise free space. Other devices may emulate this behaviour by +zeroing the device contents, which can be a slow process. .TP \fB-O\fP -This option prevents gfs2_mkfs from asking for confirmation before writing -the filesystem. +Override. This option prevents mkfs.gfs2 from asking for confirmation before +writing the filesystem. .TP \fB-o\fP -Specify extended options. Multiple options can be separated by commas. Valid extended options are: +Specify extended options. Multiple options can be separated by commas. Valid +extended options are: .RS 1.0i .TP .BI help @@ -65,7 +71,9 @@ option. .TP .BI swidth= bytes This is used to specify the stripe width for a RAID device or striped logical -volume. This option ensures that resource groups will be stripe aligned and overrides the stripe width value obtained by probing the device. This value must be a multiple of the +volume. This option ensures that resource groups will be stripe aligned and +overrides the stripe width value obtained by probing the device. This value +must be a multiple of the .I sunit option and must also be specified with it. .TP @@ -79,51 +87,62 @@ and extended options. .RE .TP -\fB-p\fP \fILockProtoName\fR -LockProtoName is the name of the locking protocol to use. Acceptable -locking protocols are \fIlock_dlm\fR (for shared storage) or if you are -using GFS2 as a local filesystem (\fB1 node only\fP), you can specify the -\fIlock_nolock\fR protocol. If this option is not specified, -\fIlock_dlm\fR protocol will be assumed. +\fB-p\fP \fIprotocol\fR +Specify the locking protocol to use when no locking protocol is specified at +mount time. Valid locking protocols are: +.RS 1.0i +.TP +.BI lock_dlm +This is the default. It enables DLM-based locking for use in shared storage +configurations. +.TP +.BI lock_nolock +This enables single-node locking +.RE .TP \fB-q\fP -Be quiet. Don't print anything. -.TP -\fB-r\fP \fIMegaBytes\fR -gfs2_mkfs will try to make Resource Groups about this big. -Minimum RG size is 32 MB. Maximum RG size is 2048 MB. -A large RG size may increase performance on very large file systems. -If not specified, mkfs.gfs2 will choose the RG size based on the size -of the file system: average size file systems will have 256 MB RGs, and -bigger file systems will have bigger RGs for better performance. -.TP -\fB-t\fP \fILockTableName\fR -The lock table field appropriate to the lock module you're using. -It is \fIclustername:fsname\fR. -Clustername must match that in cluster.conf; only members of this -cluster are permitted to use this file system (1 to 32 characters). -Fsname is a unique file system name used to distinguish this GFS2 file -system from others created (1 to 30 characters). Lock_nolock doesn't -use this field. Valid \fIclustername\fRs and \fIfsname\fRs may only contain -alphanumeric characters, hyphens (-) and underscores (_). +Quiet mode. Do not print anything. +.TP +\fB-r\fP \fImegabytes\fR +mkfs.gfs2 will try to make resource groups approximately this large. The +minimum resource group size is 32 MB and the maximum is 2048 MB. A large +resource group size may increase performance on very large file systems. If +not specified, mkfs.gfs2 will choose the resource group size based on the +size and alignment characteristics of the target device. +.TP +\fB-t\fP \fIclustername:lockspace\fR +The "lock table" pair used to uniquely identify this filesystem in a cluster. +The cluster name segment (maximum 32 characters) must match the name given to +your cluster in its configuration; only members of this cluster are permitted +to use this file system. The lockspace segment (maximum 30 characters) is a +unique file system name used to distinguish this gfs2 file system. Valid +\fIclustername\fRs and \fIlockspace\fRs may only contain alphanumeric +characters, hyphens (-) and underscores (_). .TP \fB-V\fP Print program version information, then exit. - .TP -[ \fIblock-count\fR ] -Make the file system this many blocks in size. If not specified, the -entire length of the specified device is used. - +\fIblock-count\fR +Use \fIblock-count\fR as the size of the filesystem instead of using the whole +device. \fIblock-count\fR is specified as a number of filesystem blocks. .SH EXAMPLE -.TP -gfs2_mkfs -t mycluster:mygfs2 -p lock_dlm -j 2 /dev/vg0/mygfs2 -This will make a Global File System on the block device -"/dev/vg0/mygfs2". It will belong to "mycluster" and register itself -as wanting locking for "mygfs2". It will use DLM for locking and make -two journals. -.TP -gfs2_mkfs -t mycluster:mygfs2 -p lock_nolock -j 3 /dev/vg0/mygfs2 -This will make a Global File System on the block device -"/dev/vg0/mygfs2". It will belong to "mycluster" and but have no -cluster locking. It will have three journals. +.nf +.RS +# mkfs.gfs2 -t mycluster:mygfs2 -p lock_dlm -j 2 /dev/vg0/lv_gfs2 +.PP +This will create a gfs2 filesystem on the block device /dev/vg0/lv_gfs2. It +will belong to a cluster named "mycluster" and use the "mygfs2" lock space. It +will use DLM for locking and create journals for a two-node cluster. +.PP +# mkfs.gfs2 -t mycluster:mygfs2 -p lock_nolock -j 3 /dev/vg0/lv_gfs2 +.PP +This will create a filesystem on the block device /dev/vg0/lv_gfs2. It +will belong to a cluster named "mycluster" and use the "mygfs2" lockspace, but +it will have no cluster locking by default as lock_nolock is used. It will +have journals for a three-node cluster. +.RE +.fi +.SH SEE ALSO +.BR gfs2 (5), +.BR gfs2_jadd (8), +.BR gfs2_grow (8)

1 0

gfs2-utils: master - fsck.gfs2: Detect multiple rgrp grow segments
by Bob Peterson 22 Mar '16

22 Mar '16

Gitweb: http://git.fedorahosted.org/git/?p=gfs2-utils.git;a=commitdiff;h=0103dc07a3… Commit: 0103dc07a3bc9a5bccca499f2362ed91e0eb2acd Parent: f460cf2d34d212a12857e89ee36d3a96d01933cb Author: Bob Peterson <rpeterso(a)redhat.com> AuthorDate: Fri Feb 12 15:08:54 2016 -0500 Committer: Bob Peterson <rpeterso(a)redhat.com> CommitterDate: Thu Feb 25 11:48:12 2016 -0500 fsck.gfs2: Detect multiple rgrp grow segments This patch gives fsck.gfs2's rgrepair code the capability to detect multiple gfs2_grow segments and repair them accordingly. Signed-off-by: Bob Peterson <rpeterso(a)redhat.com> --- gfs2/fsck/initialize.c | 2 +- gfs2/fsck/rgrepair.c | 226 ++++++++++++++++++++++++++++++++++-------------- 2 files changed, 160 insertions(+), 68 deletions(-) diff --git a/gfs2/fsck/initialize.c b/gfs2/fsck/initialize.c index 09ad660..84d39cd 100644 --- a/gfs2/fsck/initialize.c +++ b/gfs2/fsck/initialize.c @@ -640,7 +640,7 @@ static int fetch_rgrps(struct gfs2_sbd *sdp) log_notice(_("(level %d passed)\n"), trust_lvl + 1); break; } else { - if (ret < 0) + if (ret == -1) log_err( _("(level %d failed: %s)\n"), trust_lvl + 1, fail_desc[trust_lvl]); else diff --git a/gfs2/fsck/rgrepair.c b/gfs2/fsck/rgrepair.c index e80a7af..55057ce 100644 --- a/gfs2/fsck/rgrepair.c +++ b/gfs2/fsck/rgrepair.c @@ -21,6 +21,7 @@ struct special_blocks false_rgrps; #define BAD_RG_PERCENT_TOLERANCE 11 #define AWAY_FROM_BITMAPS 0x1000 +#define MAX_RGSEGMENTS 20 #define ri_equal(ondisk, expected, field) (ondisk.field == expected.field) @@ -95,72 +96,146 @@ static int is_false_rg(uint64_t block) * look like twice the distance. If we can find 6 of them, that * should be enough to figure out the correct layout. * This also figures out first_rg_dist since that's always different. + * + * This function was revised to return the number of segments, usually 2. + * The shortest distance is now returned in the highest entry in rg_dist */ -static uint64_t find_shortest_rgdist(struct gfs2_sbd *sdp, - uint64_t *initial_first_rg_dist, - uint64_t *first_rg_dist) +static int find_shortest_rgdist(struct gfs2_sbd *sdp, uint64_t *dist_array, + int *dist_cnt) { - uint64_t blk, block_of_last_rg, shortest_dist_btwn_rgs; + uint64_t blk, block_last_rg, shortest_dist_btwn_rgs; struct gfs2_buffer_head *bh; - int number_of_rgs = 0; + int rgs_sampled = 0; struct gfs2_rindex buf, tmpndx; + uint64_t initial_first_rg_dist; + int gsegment = 0; + int is_rgrp; /* Figure out if there are any RG-looking blocks in the journal we need to ignore. */ find_journaled_rgs(sdp); - *initial_first_rg_dist = *first_rg_dist = sdp->sb_addr + 1; - block_of_last_rg = sdp->sb_addr + 1; + initial_first_rg_dist = dist_array[0] = block_last_rg = + sdp->sb_addr + 1; shortest_dist_btwn_rgs = sdp->device.length; - for (blk = sdp->sb_addr + 1; - blk < sdp->device.length && number_of_rgs < 6; blk++) { - bh = bread(sdp, blk); - if (((blk == sdp->sb_addr + 1) || - (!gfs2_check_meta(bh, GFS2_METATYPE_RG))) && - !is_false_rg(blk)) { - log_debug( _("rgrp found at block 0x%llx\n"), - (unsigned long long)blk); - if (blk > sdp->sb_addr + 1) { - uint64_t rgdist; - - rgdist = blk - block_of_last_rg; - log_debug("dist 0x%llx = 0x%llx - 0x%llx", - (unsigned long long)rgdist, - (unsigned long long)blk, - (unsigned long long)block_of_last_rg); - /* ----------------------------------------- */ - /* We found an RG. Check to see if we need */ - /* to set the first_rg_dist based on whether */ - /* it's still at its initial value (i.e. the */ - /* fs.) The first rg distance is different */ - /* from the rest because of the superblock */ - /* and 64K dead space. */ - /* ----------------------------------------- */ - if (*first_rg_dist == *initial_first_rg_dist) - *first_rg_dist = rgdist; - if (rgdist < shortest_dist_btwn_rgs) { - shortest_dist_btwn_rgs = rgdist; - log_debug( _("(shortest so far)\n")); + for (blk = sdp->sb_addr + 1; blk < sdp->device.length; blk++) { + uint64_t dist; + + if (blk == sdp->sb_addr + 1) + is_rgrp = 1; + else if (is_false_rg(blk)) + is_rgrp = 0; + else { + bh = bread(sdp, blk); + is_rgrp = (gfs2_check_meta(bh, GFS2_METATYPE_RG) == 0); + brelse(bh); + } + if (!is_rgrp) { + if (rgs_sampled >= 6) { + uint64_t nblk; + + log_info(_("rgrp not found at block 0x%llx. " + "Last found rgrp was 0x%llx. " + "Checking the next one.\n"), + (unsigned long long)blk, + (unsigned long long)block_last_rg); + /* check for just a damaged rgrp */ + nblk = blk + dist_array[gsegment]; + if (is_false_rg(nblk)) { + is_rgrp = 0; + } else { + bh = bread(sdp, nblk); + is_rgrp = (((gfs2_check_meta(bh, + GFS2_METATYPE_RG) == 0))); + brelse(bh); + } + if (is_rgrp) { + log_info(_("Next rgrp is intact, so " + "this one is damaged.\n")); + blk = nblk - 1; + dist_cnt[gsegment]++; + continue; } - else - log_debug("\n"); + log_info(_("Looking for new segment.\n")); + blk -= 16; + rgs_sampled = 0; + shortest_dist_btwn_rgs = sdp->device.length; + /* That last one didn't pan out, so: */ + dist_cnt[gsegment]--; + gsegment++; } - block_of_last_rg = blk; - number_of_rgs++; - blk += 250; /* skip ahead for performance */ + if ((blk - block_last_rg) > (524288 * 2)) { + log_info(_("No rgrps were found within 4GB " + "of the last rgrp. Must be the " + "end of the file system.\n")); + + break; + } + continue; } - brelse(bh); + + dist_cnt[gsegment]++; + if (rgs_sampled >= 6) { + block_last_rg = blk; + blk += dist_array[gsegment] - 1; /* prev value in + array minus 1. */ + continue; + } + log_info(_("segment %d: rgrp found at block 0x%llx\n"), + gsegment + 1, (unsigned long long)blk); + dist = blk - block_last_rg; + if (blk > sdp->sb_addr + 1) { /* not the very first rgrp */ + + log_info("dist 0x%llx = 0x%llx - 0x%llx ", + (unsigned long long)dist, + (unsigned long long)blk, + (unsigned long long)block_last_rg); + /** + * We found an RG. Check to see if we need to set the + * first_rg_dist based on whether it is still at its + * initial value (i.e. the fs.) The first rg distance + * is different from the rest because of the + * superblock and 64K dead space. + **/ + if (dist_array[0] == initial_first_rg_dist) { + dist_array[0] = dist; + dist_cnt[0] = 1; + rgs_sampled = 0; + } + if (dist < shortest_dist_btwn_rgs) { + shortest_dist_btwn_rgs = dist; + log_info( _("(shortest so far)")); + } + log_info("\n"); + if (++rgs_sampled == 6) { + dist_array[gsegment] = shortest_dist_btwn_rgs; + log_info(_("Settled on distance 0x%llx for " + "segment %d\n"), + (unsigned long long) + dist_array[gsegment], gsegment + 1); + } + } else { + gsegment++; + } + block_last_rg = blk; + if (rgs_sampled < 6) + blk += 250; /* skip ahead for performance */ + else + blk += shortest_dist_btwn_rgs - 1; + } + if (gsegment > MAX_RGSEGMENTS) { + log_err(_("Maximum number of rgrp grow segments reached.\n")); + log_err(_("This file system cannot be repaired with fsck.\n")); + gsegment = 0; + goto out; } /* -------------------------------------------------------------- */ /* Sanity-check our first_rg_dist. If RG #2 got nuked, the */ /* first_rg_dist would measure from #1 to #3, which would be bad. */ /* We need to take remedial measures to fix it (from the index). */ /* -------------------------------------------------------------- */ - log_debug( _("First rgrp distance: 0x%llx\n"), (unsigned long long)*first_rg_dist); - log_debug( _("Distance between rgrps: 0x%llx\n"), - (unsigned long long)shortest_dist_btwn_rgs); - if (*first_rg_dist >= shortest_dist_btwn_rgs + + if (*dist_array >= shortest_dist_btwn_rgs + (shortest_dist_btwn_rgs / 4)) { /* read in the second RG index entry for this subd. */ gfs2_readi(sdp->md.riinode, (char *)&buf, @@ -169,21 +244,21 @@ static uint64_t find_shortest_rgdist(struct gfs2_sbd *sdp, gfs2_rindex_in(&tmpndx, (char *)&buf); if (tmpndx.ri_addr > sdp->sb_addr + 1) { /* sanity check */ log_warn( _("rgrp 2 is damaged: getting dist from index: ")); - *first_rg_dist = tmpndx.ri_addr - (sdp->sb_addr + 1); - log_warn("0x%llx\n", (unsigned long long)*first_rg_dist); + *dist_array = tmpndx.ri_addr - (sdp->sb_addr + 1); + log_warn("0x%llx\n", (unsigned long long)*dist_array); } else { log_warn( _("rgrp index 2 is damaged: extrapolating dist: ")); - *first_rg_dist = sdp->device.length - - (sdp->rgrps - 1) * + *dist_array = sdp->device.length - (sdp->rgrps - 1) * (sdp->device.length / sdp->rgrps); - log_warn("0x%llx\n", (unsigned long long)*first_rg_dist); + log_warn("0x%llx\n", (unsigned long long)*dist_array); } log_debug( _("Adjusted first rgrp distance: 0x%llx\n"), - (unsigned long long)*first_rg_dist); + (unsigned long long)*dist_array); } /* if first RG distance is within tolerance */ +out: gfs2_special_free(&false_rgrps); - return shortest_dist_btwn_rgs; + return gsegment; } /* @@ -445,14 +520,15 @@ static int gfs2_rindex_rebuild(struct gfs2_sbd *sdp, int *num_rgs, { struct osi_node *n, *next = NULL; struct gfs2_buffer_head *bh; - uint64_t shortest_dist_btwn_rgs; + uint64_t rg_dist[MAX_RGSEGMENTS] = {0, }; + int rg_dcnt[MAX_RGSEGMENTS] = {0, }; uint64_t blk; uint64_t fwd_block, block_bump; - uint64_t first_rg_dist, initial_first_rg_dist; struct rgrp_tree *calc_rgd, *prev_rgd; - int number_of_rgs, rgi; + int number_of_rgs, rgi, segment_rgs; int rg_was_fnd = FALSE, corrupt_rgs = 0; - int error = -1, j; + int error = -1, j, i; + int grow_segments, segment = 0; /* * In order to continue, we need to initialize the jindex. We need @@ -469,17 +545,17 @@ static int gfs2_rindex_rebuild(struct gfs2_sbd *sdp, int *num_rgs, } sdp->rgcalc.osi_node = NULL; - initial_first_rg_dist = first_rg_dist = sdp->sb_addr + 1; - shortest_dist_btwn_rgs = find_shortest_rgdist(sdp, - &initial_first_rg_dist, - &first_rg_dist); - number_of_rgs = 0; + grow_segments = find_shortest_rgdist(sdp, &rg_dist[0], &rg_dcnt[0]); + for (i = 0; i < grow_segments; i++) + log_info(_("Segment %d: rgrp distance: 0x%llx, count: %d\n"), + i + 1, (unsigned long long)rg_dist[i], rg_dcnt[i]); + number_of_rgs = segment_rgs = 0; /* -------------------------------------------------------------- */ /* Now go through the RGs and verify their integrity, fixing as */ /* needed when corruption is encountered. */ /* -------------------------------------------------------------- */ prev_rgd = NULL; - block_bump = first_rg_dist; + block_bump = rg_dist[0]; blk = sdp->sb_addr + 1; while (blk <= sdp->device.length) { log_debug( _("Block 0x%llx\n"), (unsigned long long)blk); @@ -542,6 +618,7 @@ static int gfs2_rindex_rebuild(struct gfs2_sbd *sdp, int *num_rgs, (unsigned long)prev_rgd->ri.ri_data); } number_of_rgs++; + segment_rgs++; if (rg_was_fnd) log_info( _(" rgrp %d at block 0x%llx intact\n"), number_of_rgs, (unsigned long long)blk); @@ -552,10 +629,16 @@ static int gfs2_rindex_rebuild(struct gfs2_sbd *sdp, int *num_rgs, /* * Figure out where our next rgrp should be. */ - if (blk == sdp->sb_addr + 1) - block_bump = first_rg_dist; - else if (!gfs_grow) { - block_bump = shortest_dist_btwn_rgs; + if ((blk == sdp->sb_addr + 1) || (!gfs_grow)) { + block_bump = rg_dist[segment]; + if (segment_rgs >= rg_dcnt[segment]) { + log_debug(_("End of segment %d\n"), ++segment); + segment_rgs = 0; + if (segment >= grow_segments) { + log_debug(_("Last segment.\n")); + break; + } + } /* if we have uniformly-spaced rgrps, there may be some wasted space at the end of the device. Since we don't want to create a short rgrp and @@ -1142,5 +1225,14 @@ int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count, int *sane) *rg_count = rg; gfs2_rgrp_free(&sdp->rgcalc); gfs2_rgrp_free(&sdp->rgtree); + /* We shouldn't need to worry about getting the user's permission to + make changes here. If b_modified is true, they already gave their + permission. */ + if (sdp->md.riinode->i_bh->b_modified) { + log_debug("Syncing rindex inode changes to disk.\n"); + gfs2_dinode_out(&sdp->md.riinode->i_di, + sdp->md.riinode->i_bh); + bwrite(sdp->md.riinode->i_bh); + } return 0; }

1 0

gfs2-utils: master - fsck.gfs2: Do not try to overrun the max rgrps that fit inside rindex
by Bob Peterson 22 Mar '16

22 Mar '16

Gitweb: http://git.fedorahosted.org/git/?p=gfs2-utils.git;a=commitdiff;h=f460cf2d34… Commit: f460cf2d34d212a12857e89ee36d3a96d01933cb Parent: 11d171e73e0f960c65e50e2a5fc3fac136d1076c Author: Bob Peterson <rpeterso(a)redhat.com> AuthorDate: Fri Feb 12 15:01:07 2016 -0500 Committer: Bob Peterson <rpeterso(a)redhat.com> CommitterDate: Thu Feb 25 11:48:05 2016 -0500 fsck.gfs2: Do not try to overrun the max rgrps that fit inside rindex When rindex is being repaired, it may calculate a new set of rgrps, then replace the old rgrps with the new ones. The trouble is, if we've calculated more rgrps than will fit inside rindex, it will try to write past the end of rindex. That's unacceptable because we cannot allow block allocations until after pass1 has finished, and rg repair happens well before pass1 starts. The failing situation is for cases where, for example, a file system was extended via lvextend, thus allowing for many more rgrps to be calculated than originally fit, but the rindex is still in a pre-gfs2_grow state where its size can only fit a smaller number of rgrps. In that case, we want fsck.gfs2 to repair what rgrps it can, but stop then it reaches the limits of the rindex file size. Signed-off-by: Bob Peterson <rpeterso(a)redhat.com> --- gfs2/fsck/rgrepair.c | 37 ++++++++++++++++++++++++++++++------- 1 files changed, 30 insertions(+), 7 deletions(-) diff --git a/gfs2/fsck/rgrepair.c b/gfs2/fsck/rgrepair.c index 1fefc34..e80a7af 100644 --- a/gfs2/fsck/rgrepair.c +++ b/gfs2/fsck/rgrepair.c @@ -950,13 +950,34 @@ int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count, int *sane) log_warn( _("L%d: number of rgs expected = %lld.\n"), trust_lvl + 1, (unsigned long long)sdp->rgrps); if (calc_rg_count != sdp->rgrps) { + int most_that_fit; + log_warn( _("L%d: They don't match; either (1) the fs was " "extended, (2) an odd\n"), trust_lvl + 1); log_warn( _("L%d: rgrp size was used, or (3) we have a corrupt " "rg index.\n"), trust_lvl + 1); - gfs2_rgrp_free(&sdp->rgcalc); - gfs2_rgrp_free(&sdp->rgtree); - return -1; + /* If the trust level is open_minded, we would have calculated + the rindex based on the device size. If it's not the same + number, don't trust it. Complain about the discrepancy, + then try again with a little more distrust. */ + if ((trust_lvl < distrust) || + !query( _("Attempt to use what rgrps we can? (y/n)"))) { + gfs2_rgrp_free(&sdp->rgcalc); + gfs2_rgrp_free(&sdp->rgtree); + log_err(_("The rindex was not repaired.\n")); + return -1; + } + /* We cannot grow rindex at this point. Since pass1 has not + yet run, we can't allocate blocks. Therefore we must use + whatever will fix in the space given. */ + most_that_fit = sdp->md.riinode->i_di.di_size / + sizeof(struct gfs2_rindex); + log_debug(_("The most we can fit is %d rgrps\n"), + most_that_fit); + if (most_that_fit < calc_rg_count) + calc_rg_count = most_that_fit; + log_err(_("Attempting to fix rindex with %d rgrps.\n"), + calc_rg_count); } /* ------------------------------------------------------------- */ /* Now compare the rindex to what we think it should be. */ @@ -967,7 +988,7 @@ int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count, int *sane) /* ------------------------------------------------------------- */ discrepancies = 0; for (rg = 0, n = osi_first(&sdp->rgtree), e = osi_first(&sdp->rgcalc); - n && e && !fsck_abort; rg++) { + n && e && !fsck_abort && rg < calc_rg_count; rg++) { struct rgrp_tree *expected, *actual; next = osi_next(n); @@ -1016,13 +1037,15 @@ int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count, int *sane) return -1; } } + log_debug("Calculated %d rgrps: Total: %d Match: %d Mismatch: %d\n", + calc_rg_count, rg, rg - discrepancies, discrepancies); /* ------------------------------------------------------------- */ /* Now compare the rindex to what we think it should be. */ /* Our rindex should be pretty predictable unless we've grown */ /* so look for index problems first before looking at the rgs. */ /* ------------------------------------------------------------- */ for (rg = 0, n = osi_first(&sdp->rgtree), e = osi_first(&sdp->rgcalc); - e && !fsck_abort; rg++) { + e && !fsck_abort && rg < calc_rg_count; rg++) { struct rgrp_tree *expected, *actual; if (n) @@ -1091,8 +1114,8 @@ int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count, int *sane) /* Now we can somewhat trust the rindex and the RG addresses, */ /* so let's read them in, check them and optionally fix them. */ /* ------------------------------------------------------------- */ - for (rg = 0, n = osi_first(&sdp->rgtree); n && !fsck_abort; - n = next, rg++) { + for (rg = 0, n = osi_first(&sdp->rgtree); n && !fsck_abort && + rg < calc_rg_count; n = next, rg++) { struct rgrp_tree *rgd; uint64_t prev_err = 0, errblock; int i;

1 0

gfs2-utils: master - fsck.gfs2: Add ability to fix rindex file size
by Bob Peterson 22 Mar '16

22 Mar '16

Gitweb: http://git.fedorahosted.org/git/?p=gfs2-utils.git;a=commitdiff;h=11d171e73e… Commit: 11d171e73e0f960c65e50e2a5fc3fac136d1076c Parent: 648a5d7c208bc9e342db31b1c79f019f448b5295 Author: Bob Peterson <rpeterso(a)redhat.com> AuthorDate: Fri Feb 12 14:58:29 2016 -0500 Committer: Bob Peterson <rpeterso(a)redhat.com> CommitterDate: Thu Feb 25 11:47:40 2016 -0500 fsck.gfs2: Add ability to fix rindex file size Before this patch, fsck.gfs2 checked the size of rindex, and if it wasn't evenly divisible by the size of an rindex entry, it threw up its hands and gave up. This patch allows fsck.gfs2 to repair the rindex size, basically rounding down to the largest number of rgrps that will fit. Signed-off-by: Bob Peterson <rpeterso(a)redhat.com> --- gfs2/fsck/rgrepair.c | 16 ++++++++++++---- 1 files changed, 12 insertions(+), 4 deletions(-) diff --git a/gfs2/fsck/rgrepair.c b/gfs2/fsck/rgrepair.c index 6e56f91..1fefc34 100644 --- a/gfs2/fsck/rgrepair.c +++ b/gfs2/fsck/rgrepair.c @@ -934,10 +934,18 @@ int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count, int *sane) sdp->rgtree.osi_node = NULL; /* Just to be safe */ rindex_read(sdp, 0, &sdp->rgrps, sane); if (sdp->md.riinode->i_di.di_size % sizeof(struct gfs2_rindex)) { - log_warn( _("WARNING: rindex file is corrupt.\n")); - gfs2_rgrp_free(&sdp->rgcalc); - gfs2_rgrp_free(&sdp->rgtree); - return -1; + log_warn( _("WARNING: rindex file has an invalid size.\n")); + if (!query( _("Truncate the rindex size? (y/n)"))) { + log_err(_("The rindex was not repaired.\n")); + gfs2_rgrp_free(&sdp->rgcalc); + gfs2_rgrp_free(&sdp->rgtree); + return -1; + } + sdp->md.riinode->i_di.di_size /= sizeof(struct gfs2_rindex); + sdp->md.riinode->i_di.di_size *= sizeof(struct gfs2_rindex); + bmodified(sdp->md.riinode->i_bh); + log_err(_("Changing rindex size to %lld.\n"), + (unsigned long long)sdp->md.riinode->i_di.di_size); } log_warn( _("L%d: number of rgs expected = %lld.\n"), trust_lvl + 1, (unsigned long long)sdp->rgrps);

1 0

gfs2-utils: master - fsck.gfs2: Ditch variable rgcount_from_index
by Bob Peterson 22 Mar '16

22 Mar '16

Gitweb: http://git.fedorahosted.org/git/?p=gfs2-utils.git;a=commitdiff;h=648a5d7c20… Commit: 648a5d7c208bc9e342db31b1c79f019f448b5295 Parent: e095bebc69475ff9b9369ef9350d7cc7eb50d47d Author: Bob Peterson <rpeterso(a)redhat.com> AuthorDate: Fri Feb 12 14:53:10 2016 -0500 Committer: Bob Peterson <rpeterso(a)redhat.com> CommitterDate: Thu Feb 25 11:00:19 2016 -0500 fsck.gfs2: Ditch variable rgcount_from_index This patch eliminates variable rgcount_from_index from function rg_repair. It wasn't really being used anyway. The important count of resource groups from the rindex is sdp->rgrps, so use that instead. Also, eliminate setting that from the calculated number of rgrps. We've already got calc_rg_count for that. Signed-off-by: Bob Peterson <rpeterso(a)redhat.com> --- gfs2/fsck/rgrepair.c | 6 ++---- 1 files changed, 2 insertions(+), 4 deletions(-) diff --git a/gfs2/fsck/rgrepair.c b/gfs2/fsck/rgrepair.c index 5bf82db..6e56f91 100644 --- a/gfs2/fsck/rgrepair.c +++ b/gfs2/fsck/rgrepair.c @@ -880,7 +880,7 @@ int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count, int *sane) { struct osi_node *n, *next = NULL, *e, *enext; int error, discrepancies, percent; - int calc_rg_count = 0, rgcount_from_index, rg; + int calc_rg_count = 0, rg; struct gfs2_rindex buf; if (trust_lvl == blind_faith) @@ -919,7 +919,6 @@ int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count, int *sane) gfs2_rgrp_free(&sdp->rgcalc); return -1; } - sdp->rgrps = calc_rg_count; } else if (trust_lvl == indignation) { /* If we can't trust anything */ /* Free previous incarnations in memory, if any. */ gfs2_rgrp_free(&sdp->rgtree); @@ -930,11 +929,10 @@ int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count, int *sane) gfs2_rgrp_free(&sdp->rgcalc); return -1; } - sdp->rgrps = calc_rg_count; } /* Read in the rindex */ sdp->rgtree.osi_node = NULL; /* Just to be safe */ - rindex_read(sdp, 0, &rgcount_from_index, sane); + rindex_read(sdp, 0, &sdp->rgrps, sane); if (sdp->md.riinode->i_di.di_size % sizeof(struct gfs2_rindex)) { log_warn( _("WARNING: rindex file is corrupt.\n")); gfs2_rgrp_free(&sdp->rgcalc);

1 0

gfs2-utils: master - fsck.gfs2: Minor reformatting
by Bob Peterson 22 Mar '16

22 Mar '16

Gitweb: http://git.fedorahosted.org/git/?p=gfs2-utils.git;a=commitdiff;h=e095bebc69… Commit: e095bebc69475ff9b9369ef9350d7cc7eb50d47d Parent: f904f38134faffeaad7393eeae33a4c1c1285a45 Author: Bob Peterson <rpeterso(a)redhat.com> AuthorDate: Fri Feb 12 14:25:33 2016 -0500 Committer: Bob Peterson <rpeterso(a)redhat.com> CommitterDate: Thu Feb 25 10:59:47 2016 -0500 fsck.gfs2: Minor reformatting This patch just changes some formatting inside the code and eliminates a message that doesn't need to be there. Signed-off-by: Bob Peterson <rpeterso(a)redhat.com> --- gfs2/fsck/rgrepair.c | 20 +++++++++----------- 1 files changed, 9 insertions(+), 11 deletions(-) diff --git a/gfs2/fsck/rgrepair.c b/gfs2/fsck/rgrepair.c index f0759c1..5bf82db 100644 --- a/gfs2/fsck/rgrepair.c +++ b/gfs2/fsck/rgrepair.c @@ -171,8 +171,7 @@ static uint64_t find_shortest_rgdist(struct gfs2_sbd *sdp, log_warn( _("rgrp 2 is damaged: getting dist from index: ")); *first_rg_dist = tmpndx.ri_addr - (sdp->sb_addr + 1); log_warn("0x%llx\n", (unsigned long long)*first_rg_dist); - } - else { + } else { log_warn( _("rgrp index 2 is damaged: extrapolating dist: ")); *first_rg_dist = sdp->device.length - (sdp->rgrps - 1) * @@ -500,8 +499,8 @@ static int gfs2_rindex_rebuild(struct gfs2_sbd *sdp, int *num_rgs, /* ------------------------------------------------- */ corrupt_rgs++; if (corrupt_rgs < 5) - log_debug( _("Missing or damaged rgrp at block " - "%llu (0x%llx)\n"), + log_debug(_("Missing or damaged rgrp at block " + "%llu (0x%llx)\n"), (unsigned long long)blk, (unsigned long long)blk); else { @@ -664,7 +663,8 @@ static uint64_t how_many_rgrps(struct gfs2_sbd *sdp, struct device *dev, int rgs sdp->rgsize += GFS2_DEFAULT_RGSIZE; /* bigger rgs */ } - log_debug(" rg sz = %"PRIu32"\n nrgrp = %"PRIu64"\n", sdp->rgsize, nrgrp); + log_debug(" rg sz = %"PRIu32"\n nrgrp = %"PRIu64"\n", sdp->rgsize, + nrgrp); return nrgrp; } @@ -728,10 +728,10 @@ static void compute_rgrp_layout(struct gfs2_sbd *sdp, struct osi_root *rgtree, i (nrgrp - 1) * (dev->length / nrgrp); } rl->start = rgaddr; - printf("%d: start: %" PRIu64 " (0x%" + /* printf("%d: start: %" PRIu64 " (0x%" PRIx64 "), length = %"PRIu64" (0x%" PRIx64 ")\n", rgrp + 1, rl->start, rl->start, - rl->length, rl->length); + rl->length, rl->length);*/ rlast = rl; } @@ -909,8 +909,7 @@ int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count, int *sane) gfs2_rgrp_free(&sdp->rgcalc); return -1; } - } - else if (trust_lvl == distrust) { /* If we can't trust RG index */ + } else if (trust_lvl == distrust) { /* If we can't trust RG index */ /* Free previous incarnations in memory, if any. */ gfs2_rgrp_free(&sdp->rgtree); @@ -921,8 +920,7 @@ int rg_repair(struct gfs2_sbd *sdp, int trust_lvl, int *rg_count, int *sane) return -1; } sdp->rgrps = calc_rg_count; - } - else if (trust_lvl == indignation) { /* If we can't trust anything */ + } else if (trust_lvl == indignation) { /* If we can't trust anything */ /* Free previous incarnations in memory, if any. */ gfs2_rgrp_free(&sdp->rgtree);

1 0

gfs2-utils: master - fsck.gfs2: better reporting of false positive rgrp identification
by Bob Peterson 22 Mar '16

22 Mar '16

Gitweb: http://git.fedorahosted.org/git/?p=gfs2-utils.git;a=commitdiff;h=f904f38134… Commit: f904f38134faffeaad7393eeae33a4c1c1285a45 Parent: 318f97f9e8e7602dea585c1626d4e4b0ac5fc0b4 Author: Bob Peterson <rpeterso(a)redhat.com> AuthorDate: Fri Feb 12 14:18:20 2016 -0500 Committer: Bob Peterson <rpeterso(a)redhat.com> CommitterDate: Thu Feb 25 10:59:25 2016 -0500 fsck.gfs2: better reporting of false positive rgrp identification This patch adds some more information to the debug messages related to the identification of "false positive" rgrps. That is, rgrps that aren't really rgrps because they reside inside a journal. Signed-off-by: Bob Peterson <rpeterso(a)redhat.com> --- gfs2/fsck/rgrepair.c | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-) diff --git a/gfs2/fsck/rgrepair.c b/gfs2/fsck/rgrepair.c index 9ef97b8..f0759c1 100644 --- a/gfs2/fsck/rgrepair.c +++ b/gfs2/fsck/rgrepair.c @@ -55,12 +55,16 @@ static void find_journaled_rgs(struct gfs2_sbd *sdp) uint32_t extlen; struct gfs2_inode *ip; struct gfs2_buffer_head *bh; + int false_count; osi_list_init(&false_rgrps.list); for (j = 0; j < sdp->md.journals; j++) { - log_debug( _("Checking for rgrps in journal%d.\n"), j); ip = sdp->md.journal[j]; + log_debug(_("Checking for rgrps in journal%d which starts " + "at block 0x%llx.\n"), j, + (unsigned long long)ip->i_di.di_num.no_addr); jblocks = ip->i_di.di_size / sdp->sd_sb.sb_bsize; + false_count = 0; for (b = 0; b < jblocks; b++) { block_map(ip, b, &new, &dblock, &extlen, 0); if (!dblock) @@ -68,10 +72,12 @@ static void find_journaled_rgs(struct gfs2_sbd *sdp) bh = bread(sdp, dblock); if (!gfs2_check_meta(bh, GFS2_METATYPE_RG)) { /* False rgrp found at block dblock */ + false_count++; gfs2_special_set(&false_rgrps, dblock); } brelse(bh); } + log_debug("\n%d false positives identified.\n", false_count); } }

1 0

gfs2-utils: master - fsck.gfs2: Read jindex before making rindex repairs
by Bob Peterson 22 Mar '16

22 Mar '16

Gitweb: http://git.fedorahosted.org/git/?p=gfs2-utils.git;a=commitdiff;h=318f97f9e8… Commit: 318f97f9e8e7602dea585c1626d4e4b0ac5fc0b4 Parent: 4724a1d7137ae76738b14f8299aa0d24fe41abb1 Author: Bob Peterson <rpeterso(a)redhat.com> AuthorDate: Thu Aug 6 12:06:35 2015 -0500 Committer: Bob Peterson <rpeterso(a)redhat.com> CommitterDate: Thu Feb 25 10:58:11 2016 -0500 fsck.gfs2: Read jindex before making rindex repairs In most cases, the rindex needs to be read into memory in case the journals or jindex are corrupt and need repairs. However, in some rare cases, the rindex needs repairs, and in the rindex repair code it needs to read in the jindex and journals in order to filter out rgrp records that appear in the journals. This prevents the rgrp records inside journals from being treated as real rgrps, rather than false-positives. This patch also fixes a segfault in the rgrp code for cases of extremely corrupt rindex files where the rgrp has no buffers. Signed-off-by: Bob Peterson <rpeterso(a)redhat.com> --- gfs2/fsck/fs_recovery.c | 138 ++++++++++++++++++++++++++++++++++++++++++++++ gfs2/fsck/fs_recovery.h | 1 + gfs2/fsck/initialize.c | 140 ++--------------------------------------------- gfs2/fsck/rgrepair.c | 38 ++++++++++-- gfs2/libgfs2/rgrp.c | 20 ++++--- 5 files changed, 186 insertions(+), 151 deletions(-) diff --git a/gfs2/fsck/fs_recovery.c b/gfs2/fsck/fs_recovery.c index ce11fb4..17b2a3a 100644 --- a/gfs2/fsck/fs_recovery.c +++ b/gfs2/fsck/fs_recovery.c @@ -812,3 +812,141 @@ int ji_update(struct gfs2_sbd *sdp) } return 0; } + +static void bad_journalname(const char *filename, int len) +{ + if (len >= 64) + len = 63; + log_debug(_("Journal index entry '%.*s' has an invalid filename.\n"), + len, filename); +} + +/** + * check_jindex_dent - check the jindex directory entries + * + * This function makes sure the directory entries of the jindex are valid. + * If they're not '.' or '..' they better have the form journalXXX. + */ +static int check_jindex_dent(struct gfs2_inode *ip, struct gfs2_dirent *dent, + struct gfs2_dirent *prev_de, + struct gfs2_buffer_head *bh, char *filename, + uint32_t *count, int *lindex, void *priv) +{ + struct gfs2_dirent dentry, *de; + int i; + + memset(&dentry, 0, sizeof(struct gfs2_dirent)); + gfs2_dirent_in(&dentry, (char *)dent); + de = &dentry; + + if (de->de_name_len == 1 && filename[0] == '.') + goto dirent_good; + if (de->de_name_len == 2 && filename[0] == '.' && filename[1] == '.') + goto dirent_good; + + if ((de->de_name_len >= 11) || /* "journal9999" */ + (de->de_name_len <= 7) || + (strncmp(filename, "journal", 7))) { + bad_journalname(filename, de->de_name_len); + return -1; + } + for (i = 7; i < de->de_name_len; i++) { + if (filename[i] < '0' || filename[i] > '9') { + bad_journalname(filename, de->de_name_len); + return -2; + } + } + +dirent_good: + /* Return the number of leaf entries so metawalk doesn't flag this + leaf as having none. */ + *count = be16_to_cpu(((struct gfs2_leaf *)bh->b_data)->lf_entries); + return 0; +} + +struct metawalk_fxns jindex_check_fxns = { + .private = NULL, + .check_dentry = check_jindex_dent, +}; + +/** + * init_jindex - read in the rindex file + */ +int init_jindex(struct gfs2_sbd *sdp, int allow_ji_rebuild) +{ + /******************************************************************* + ****************** Fill in journal information ****************** + *******************************************************************/ + + log_debug(_("Validating the journal index.\n")); + /* rgrepair requires the journals be read in in order to distinguish + "real" rgrps from rgrps that are just copies left in journals. */ + if (sdp->gfs1) + sdp->md.jiinode = lgfs2_inode_read(sdp, sbd1->sb_jindex_di.no_addr); + else + gfs2_lookupi(sdp->master_dir, "jindex", 6, &sdp->md.jiinode); + + if (!sdp->md.jiinode) { + int err; + + if (!allow_ji_rebuild) { + log_crit(_("Error: jindex and rindex files are both " + "corrupt.\n")); + return -1; + } + if (!query( _("The gfs2 system jindex inode is missing. " + "Okay to rebuild it? (y/n) "))) { + log_crit(_("Error: cannot proceed without a valid " + "jindex file.\n")); + return -1; + } + + err = build_jindex(sdp); + if (err) { + log_crit(_("Error %d rebuilding jindex\n"), err); + return err; + } + gfs2_lookupi(sdp->master_dir, "jindex", 6, &sdp->md.jiinode); + } + + /* check for irrelevant entries in jindex. Can't use check_dir because + that creates and destroys the inode, which we don't want. */ + if (!sdp->gfs1) { + int error; + + log_debug(_("Checking the integrity of the journal index.\n")); + if (sdp->md.jiinode->i_di.di_flags & GFS2_DIF_EXHASH) + error = check_leaf_blks(sdp->md.jiinode, + &jindex_check_fxns); + else + error = check_linear_dir(sdp->md.jiinode, + sdp->md.jiinode->i_bh, + &jindex_check_fxns); + if (error) { + log_err(_("The system journal index is damaged.\n")); + if (!query( _("Okay to rebuild it? (y/n) "))) { + log_crit(_("Error: cannot proceed without a " + "valid jindex file.\n")); + return -1; + } + inode_put(&sdp->md.jiinode); + gfs2_dirent_del(sdp->master_dir, "jindex", 6); + log_err(_("Corrupt journal index was removed.\n")); + error = build_jindex(sdp); + if (error) { + log_err(_("Error rebuilding journal " + "index: Cannot continue.\n")); + return error; + } + gfs2_lookupi(sdp->master_dir, "jindex", 6, + &sdp->md.jiinode); + } + } + + /* read in the ji data */ + if (ji_update(sdp)){ + log_err( _("Unable to read jindex inode.\n")); + return -1; + } + return 0; +} diff --git a/gfs2/fsck/fs_recovery.h b/gfs2/fsck/fs_recovery.h index 1a7aae3..d687627 100644 --- a/gfs2/fsck/fs_recovery.h +++ b/gfs2/fsck/fs_recovery.h @@ -8,5 +8,6 @@ extern int replay_journals(struct gfs2_sbd *sdp, int preen, int force_check, extern int preen_is_safe(struct gfs2_sbd *sdp, int preen, int force_check); extern int ji_update(struct gfs2_sbd *sdp); +extern int init_jindex(struct gfs2_sbd *sdp, int allow_ji_rebuild); #endif /* __FS_RECOVERY_H__ */ diff --git a/gfs2/fsck/initialize.c b/gfs2/fsck/initialize.c index 4a31927..09ad660 100644 --- a/gfs2/fsck/initialize.c +++ b/gfs2/fsck/initialize.c @@ -1511,139 +1511,6 @@ static int init_rindex(struct gfs2_sbd *sdp) return 0; } -static void bad_journalname(const char *filename, int len) -{ - if (len >= 64) - len = 63; - log_debug(_("Journal index entry '%.*s' has an invalid filename.\n"), - len, filename); -} - -/** - * check_jindex_dent - check the jindex directory entries - * - * This function makes sure the directory entries of the jindex are valid. - * If they're not '.' or '..' they better have the form journalXXX. - */ -static int check_jindex_dent(struct gfs2_inode *ip, struct gfs2_dirent *dent, - struct gfs2_dirent *prev_de, - struct gfs2_buffer_head *bh, char *filename, - uint32_t *count, int *lindex, void *priv) -{ - struct gfs2_dirent dentry, *de; - int i; - - memset(&dentry, 0, sizeof(struct gfs2_dirent)); - gfs2_dirent_in(&dentry, (char *)dent); - de = &dentry; - - if (de->de_name_len == 1 && filename[0] == '.') - goto dirent_good; - if (de->de_name_len == 2 && filename[0] == '.' && filename[1] == '.') - goto dirent_good; - - if ((de->de_name_len >= 11) || /* "journal9999" */ - (de->de_name_len <= 7) || - (strncmp(filename, "journal", 7))) { - bad_journalname(filename, de->de_name_len); - return -1; - } - for (i = 7; i < de->de_name_len; i++) { - if (filename[i] < '0' || filename[i] > '9') { - bad_journalname(filename, de->de_name_len); - return -2; - } - } - -dirent_good: - /* Return the number of leaf entries so metawalk doesn't flag this - leaf as having none. */ - *count = be16_to_cpu(((struct gfs2_leaf *)bh->b_data)->lf_entries); - return 0; -} - -struct metawalk_fxns jindex_check_fxns = { - .private = NULL, - .check_dentry = check_jindex_dent, -}; - -/** - * init_jindex - read in the rindex file - */ -static int init_jindex(struct gfs2_sbd *sdp) -{ - /******************************************************************* - ****************** Fill in journal information ****************** - *******************************************************************/ - - log_debug(_("Validating the journal index.\n")); - /* rgrepair requires the journals be read in in order to distinguish - "real" rgrps from rgrps that are just copies left in journals. */ - if (sdp->gfs1) - sdp->md.jiinode = lgfs2_inode_read(sdp, sbd1->sb_jindex_di.no_addr); - else - gfs2_lookupi(sdp->master_dir, "jindex", 6, &sdp->md.jiinode); - - if (!sdp->md.jiinode) { - int err; - - if (!query( _("The gfs2 system jindex inode is missing. " - "Okay to rebuild it? (y/n) "))) { - log_crit(_("Error: cannot proceed without a valid " - "jindex file.\n")); - return -1; - } - - err = build_jindex(sdp); - if (err) { - log_crit(_("Error %d rebuilding jindex\n"), err); - return err; - } - gfs2_lookupi(sdp->master_dir, "jindex", 6, &sdp->md.jiinode); - } - - /* check for irrelevant entries in jindex. Can't use check_dir because - that creates and destroys the inode, which we don't want. */ - if (!sdp->gfs1) { - int error; - - log_debug(_("Checking the integrity of the journal index.\n")); - if (sdp->md.jiinode->i_di.di_flags & GFS2_DIF_EXHASH) - error = check_leaf_blks(sdp->md.jiinode, - &jindex_check_fxns); - else - error = check_linear_dir(sdp->md.jiinode, - sdp->md.jiinode->i_bh, - &jindex_check_fxns); - if (error) { - log_err(_("The system journal index is damaged.\n")); - if (!query( _("Okay to rebuild it? (y/n) "))) { - log_crit(_("Error: cannot proceed without a " - "valid jindex file.\n")); - return -1; - } - inode_put(&sdp->md.jiinode); - gfs2_dirent_del(sdp->master_dir, "jindex", 6); - log_err(_("Corrupt journal index was removed.\n")); - error = build_jindex(sdp); - if (error) { - log_err(_("Error rebuilding journal " - "index: Cannot continue.\n")); - return error; - } - gfs2_lookupi(sdp->master_dir, "jindex", 6, - &sdp->md.jiinode); - } - } - - /* read in the ji data */ - if (ji_update(sdp)){ - log_err( _("Unable to read jindex inode.\n")); - return -1; - } - return 0; -} - /** * initialize - initialize superblock pointer * @@ -1735,7 +1602,10 @@ int initialize(struct gfs2_sbd *sdp, int force_check, int preen, lookup_per_node(sdp, 0); /* We need rindex first in case jindex is missing and needs to read - in the rgrps before rebuilding it. */ + in the rgrps before rebuilding it. However, note that if the rindex + is damaged, we need the journals to repair it. That's because the + journals likely contain rgrps and bitmaps, which we need to ignore + when we're trying to find the rgrps. */ if (init_rindex(sdp)) return FSCK_ERROR; @@ -1745,7 +1615,7 @@ int initialize(struct gfs2_sbd *sdp, int force_check, int preen, /* We need to read in jindex in order to replay the journals. If there's an error, we may proceed and let init_system_inodes try to rebuild it. */ - if (init_jindex(sdp) == 0) { + if (init_jindex(sdp, 1) == 0) { /* If GFS, rebuild the journals. If GFS2, replay them. We don't have the smarts to replay GFS1 journals (neither did gfs_fsck). */ diff --git a/gfs2/fsck/rgrepair.c b/gfs2/fsck/rgrepair.c index 12e474b..9ef97b8 100644 --- a/gfs2/fsck/rgrepair.c +++ b/gfs2/fsck/rgrepair.c @@ -14,6 +14,7 @@ #include "libgfs2.h" #include "osi_list.h" #include "fsck.h" +#include "fs_recovery.h" int rindex_modified = FALSE; struct special_blocks false_rgrps; @@ -41,6 +42,10 @@ struct special_blocks false_rgrps; * case, we don't want to mistake these blocks that look just a real RG * for a real RG block. These are "fake" RGs that need to be ignored for * the purposes of finding where things are. + * + * NOTE: This function assumes that the jindex and journals have been read in, + * which isn't often the case. Normally the rindex needs to be read in + * first. If the rindex is damaged, that's not an option. */ static void find_journaled_rgs(struct gfs2_sbd *sdp) { @@ -62,8 +67,7 @@ static void find_journaled_rgs(struct gfs2_sbd *sdp) break; bh = bread(sdp, dblock); if (!gfs2_check_meta(bh, GFS2_METATYPE_RG)) { - log_debug( _("False rgrp found at block 0x%llx\n"), - (unsigned long long)dblock); + /* False rgrp found at block dblock */ gfs2_special_set(&false_rgrps, dblock); } brelse(bh); @@ -443,6 +447,21 @@ static int gfs2_rindex_rebuild(struct gfs2_sbd *sdp, int *num_rgs, struct rgrp_tree *calc_rgd, *prev_rgd; int number_of_rgs, rgi; int rg_was_fnd = FALSE, corrupt_rgs = 0; + int error = -1, j; + + /* + * In order to continue, we need to initialize the jindex. We need + * the journals in order to correctly eliminate false positives during + * rgrp repair. IOW, we need to properly ignore rgrps that appear in + * the journals, and we can only do that if we have the journals. + * To make matters worse, journals may span several (small) rgrps, + * so we can't go by the rgrps. + */ + if (init_jindex(sdp, 0) != 0) { + log_crit(_("Error: Can't read jindex required for rindex " + "repairs.\n")); + return -1; + } sdp->rgcalc.osi_node = NULL; initial_first_rg_dist = first_rg_dist = sdp->sb_addr + 1; @@ -466,7 +485,7 @@ static int gfs2_rindex_rebuild(struct gfs2_sbd *sdp, int *num_rgs, calc_rgd = rgrp_insert(&sdp->rgcalc, blk); if (!calc_rgd) { log_crit( _("Can't allocate memory for rgrp repair.\n")); - return -1; + goto out; } calc_rgd->ri.ri_length = 1; if (!rg_was_fnd) { /* if not an RG */ @@ -483,7 +502,7 @@ static int gfs2_rindex_rebuild(struct gfs2_sbd *sdp, int *num_rgs, log_crit( _("Error: too many missing or " "damaged rgrps using this method. " "Time to try another method.\n")); - return -1; + goto out; } } /* ------------------------------------------------ */ @@ -519,10 +538,10 @@ static int gfs2_rindex_rebuild(struct gfs2_sbd *sdp, int *num_rgs, } number_of_rgs++; if (rg_was_fnd) - log_info( _(" rgrp %d at block 0x%llx intact"), + log_info( _(" rgrp %d at block 0x%llx intact\n"), number_of_rgs, (unsigned long long)blk); else - log_warn( _("* rgrp %d at block 0x%llx *** DAMAGED ***"), + log_warn( _("* rgrp %d at block 0x%llx *** DAMAGED ***\n"), number_of_rgs, (unsigned long long)blk); prev_rgd = calc_rgd; /* @@ -587,7 +606,12 @@ static int gfs2_rindex_rebuild(struct gfs2_sbd *sdp, int *num_rgs, calc_rgd->ri.ri_bitbytes); } *num_rgs = number_of_rgs; - return 0; + error = 0; +out: + for (j = 0; j < sdp->md.journals; j++) + inode_put(&sdp->md.journal[j]); + free(sdp->md.journal); + return error; } #define DIV_RU(x, y) (((x) + (y) - 1) / (y)) diff --git a/gfs2/libgfs2/rgrp.c b/gfs2/libgfs2/rgrp.c index c5671f0..766bdfc 100644 --- a/gfs2/libgfs2/rgrp.c +++ b/gfs2/libgfs2/rgrp.c @@ -250,17 +250,19 @@ void gfs2_rgrp_free(struct osi_root *rgrp_tree) while ((n = osi_first(rgrp_tree))) { rgd = (struct rgrp_tree *)n; - if (rgd->bits && rgd->bits[0].bi_bh) { /* if a buffer exists */ - rgs_since_sync++; - if (rgs_since_sync >= RG_SYNC_TOLERANCE) { - if (!sdp) - sdp = rgd->bits[0].bi_bh->sdp; - fsync(sdp->device_fd); - rgs_since_sync = 0; + if (rgd->bits) { + if (rgd->bits[0].bi_bh) { /* if a buffer exists */ + rgs_since_sync++; + if (rgs_since_sync >= RG_SYNC_TOLERANCE) { + if (!sdp) + sdp = rgd->bits[0].bi_bh->sdp; + fsync(sdp->device_fd); + rgs_since_sync = 0; + } + gfs2_rgrp_relse(rgd); /* free them all. */ } - gfs2_rgrp_relse(rgd); /* free them all. */ + free(rgd->bits); } - free(rgd->bits); osi_erase(&rgd->node, rgrp_tree); free(rgd); }

1 0

gfs2-utils: master - gfs2_edit: Don't reference an empty rgrp tree
by Bob Peterson 22 Mar '16

22 Mar '16

Gitweb: http://git.fedorahosted.org/git/?p=gfs2-utils.git;a=commitdiff;h=4724a1d713… Commit: 4724a1d7137ae76738b14f8299aa0d24fe41abb1 Parent: f20b97397da9491e4fe51189fb2a0f6d47a289ea Author: Bob Peterson <rpeterso(a)redhat.com> AuthorDate: Tue Feb 9 09:53:01 2016 -0500 Committer: Bob Peterson <rpeterso(a)redhat.com> CommitterDate: Thu Feb 25 10:54:50 2016 -0500 gfs2_edit: Don't reference an empty rgrp tree This patch adds a check to function read_rindex in gfs2_edit which makes sure the rgrp rb_tree is not empty before dereferencing it. Signed-off-by: Bob Peterson <rpeterso(a)redhat.com> --- gfs2/edit/hexedit.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/gfs2/edit/hexedit.c b/gfs2/edit/hexedit.c index 606bb5a..1fea017 100644 --- a/gfs2/edit/hexedit.c +++ b/gfs2/edit/hexedit.c @@ -1025,8 +1025,10 @@ static int read_rindex(void) if (sbd.md.riinode) /* If we found the rindex */ rindex_read(&sbd, 0, &count, &sane); - ri = &((struct rgrp_tree *)osi_last(&sbd.rgtree))->ri; - sbd.fssize = ri->ri_data0 + ri->ri_data; + if (!OSI_EMPTY_ROOT(&sbd.rgtree)) { + ri = &((struct rgrp_tree *)osi_last(&sbd.rgtree))->ri; + sbd.fssize = ri->ri_data0 + ri->ri_data; + } return 0; }

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

cluster-commits March 2016