In PR 4447 [0], looks like test cases for a single rule are being ran multiple times per profile:
On 6/24/19 11:49 AM, Gabriel Gaspar Becker wrote:
|INFO - Script both-correct.pass.sh using profile xccdf_org.ssgproject.content_profile_cui OK INFO - Script both-correct.pass.sh using profile xccdf_org.ssgproject.content_profile_ospp OK|
How come the test harness tests the same rule multiple times?
Why not test each rule once? Seems like it would reduce the testing time significantly.
[0] https://github.com/ComplianceAsCode/content/pull/4447.patch
This can happen when you want to ensure that a test run for every platform, for example, if cui profile exists in RHE7 but ospp don't and RHEL8 otherwise, this would ensure that the test scenario will be available to run for both RHEL7 and RHEL8.
This for sure can increase the testing time because sometimes it can run twice, but future features of the test suite will use the so called virtual profile "(all)" which selects all available rules and override the header "profiles" from the test scenario implying that the test scenario will only run once.
There are two pull requests [1][2] at the moment (one merged) to improve usability and performance of test suite:
[1] https://github.com/ComplianceAsCode/content/pull/4419 [2] https://github.com/ComplianceAsCode/content/pull/4441
On Mon, Jun 24, 2019 at 10:50 PM Shawn Wells shawn@redhat.com wrote:
In PR 4447 [0], looks like test cases for a single rule are being ran multiple times per profile: On 6/24/19 11:49 AM, Gabriel Gaspar Becker wrote:
INFO - Script both-correct.pass.sh using profile xccdf_org.ssgproject.content_profile_cui OK INFO - Script both-correct.pass.sh using profile xccdf_org.ssgproject.content_profile_ospp OK
How come the test harness tests the same rule multiple times?
Why not test each rule once? Seems like it would reduce the testing time significantly.
[0] https://github.com/ComplianceAsCode/content/pull/4447.patch _______________________________________________ scap-security-guide mailing list -- scap-security-guide@lists.fedorahosted.org To unsubscribe send an email to scap-security-guide-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/scap-security-guide@lists.fedor...
scap-security-guide@lists.fedorahosted.org