I'm not involved in the STIG making "process" but I deal with the "product" on a daily basis. There are at least seven problems with the product that make me think the process does not include "thoughtful review".
Malformed XML -- When I opened a ticket with DISA on the STIGs having thinkg like "<" or ">" characters written as "<" or ">", the response seemed to be "Well, the tools all work with it." Well, that means people have to code around your code.
Bad characters in the text -- Better response from them on the ticket where I noted non-ASCII type characters in the text. "We'll get to it."
Incorrect commands -- V-38660 in RHEL 6 V1R12. The command "grep 'v1|v2c|com2sec' /etc/snmp/snmpd.conf" might work as an "egrep".
Unclear language -- For parameters in files like "system-auth", don't put the entire line when you only mean one parameter. Otherwise an unskilled auditor or admin will just use that one line, which might not be what's needed. Often even more stringent commands can be used but the admin could get ding'd because the line does not match exactly.
Different VIDs for the same thing in different STIGs -- The precedent is established for using the same Vulnerability ID (VID) in different OS STIGs when the issue and fix are the same. However, this good idea seems to come and go, there are many VIDs for the same thing in different OS STIGs.
Different VIDs for the same thing in the same STIG -- In the early RHEL 6 STIG there were 6 different VIDs for the exact same problem. I haven't checked lately but I think there are still a few duplicates.
Partially automated tools -- There are ~264 VIDs in RHEL 6 V1R12, of which ~97 get marked as "Not Reviewed" because the benchmark file doesn't automate the checks. I'm not a great coder and I've spent a little time automating about 75-85% of those checks. Why can't the benchmarks do a better job than I?
The Open Source process prioritizes "better product" over "good feelings". I'm glad DISA is looking at using the process to improve the product. When it comes right down to it, the STIGs are written to help protect the war fighter and our national security. It might get a little heated but our nation and our service members deserve our absolute best effort.
Leam
p.s. Watch me make a mistake in checking this. I have an excuse though, flu.
scap-security-guide@lists.fedorahosted.org