Skip to content

[RFE] Implement Bitmask Exit Codes for Granular CI/CD Control #372

@tregubovav-dev

Description

@tregubovav-dev

Description:
Currently, the DUnitX Console Runner aggregates all negative outcomes into a single generic exit code (usually 1), or 100 for command-line errors.

While the console output provides a summary (Ignored/Passed/Leaked/Failed/Errored), the process exit code is binary (Success/Fail). This forces CI/CD pipelines to parse the text output or the XML log to distinguish between critical failures (e.g., Exceptions/Crashes) and manageable issues (e.g., Memory Leaks or Failed Assertions).

Proposal:
Introduce a mechanism (potentially opt-in via a switch like --exitmode:Bitmask) to return the exit code as a bitmask. This allows build scripts to make intelligent decisions (e.g., "Fail build on Errors, but only Warn on Leaks").

Proposed Bit Mapping:
The exit code would be calculated by OR-ing the following flags:

Bit Hex Decimal Condition
0 0x01 1 Internal/CMD Error (Bad arguments, runner crashed)
1 0x02 2 Tests Errored (Unhandled Exceptions)
2 0x04 4 Tests Failed (Assertion failures)
3 0x08 8 Tests Leaked (Memory leaks detected)
4 0x10 16 Tests Ignored (See note below)

Logic:

ExitCode := 0;
if HasCmdLineError then ExitCode := ExitCode or $01;
if ErrorCount > 0  then ExitCode := ExitCode or $02;
if FailCount > 0   then ExitCode := ExitCode or $04;
if LeakCount > 0   then ExitCode := ExitCode or $08;
// Optional: Only if --fail-on-ignored is set
if IgnoreCount > 0 then ExitCode := ExitCode or $10;

Use Case Example:
A CI pipeline could execute the tests and handle the result like this (pseudocode):

run_tests.exe
CODE=$?

# If Bit 1 (Error) or Bit 2 (Fail) is set -> Critical Failure
if (( CODE & 6 )); then
  echo "Tests Failed or Errored!"
  exit 1
fi

# If Bit 3 (Leak) is set -> Warning only
if (( CODE & 8 )); then
  echo "WARNING: Memory Leaks detected"
  # Do not exit 1, allow build to proceed
fi

Benefits:

  1. Eliminates the need for fragile Log/XML parsing in build scripts.
  2. Allows distinct handling of Memory Leaks vs. Logic Failures.
  3. Allows differentiating between "Tests Failed" and "Runner Failed" (Bad Params).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions