Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show evaluated test arguments for functions with 1 & 2 arguments #57825

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

omus
Copy link
Member

@omus omus commented Mar 18, 2025

When using @test any function call which has 1 or 2 arguments only will always display the evaluated arguments. Having this makes it much easier at a glance to debug the problem:

julia> using Test

julia> x = [1,2];

julia> y = [1,3];

julia> @test issubset(x, y)
Test Failed at REPL[5]:1
  Expression: issubset(x, y)
   Evaluated: issubset([1, 2], [1, 3])

ERROR: There was an error during testing

Alternatively, users could use @testset let however it can be quite tedious to add those blocks to test suites.

Alternative to:

@nsajko nsajko added the testsystem The unit testing framework and Test stdlib label Mar 19, 2025
omus added 2 commits March 19, 2025 09:26
Printing resulted from calling `record` on a `Test.Error` and
`Test.Fail` was not actually tested at this point and was just creating
noise in our test results. I've confirmed that later tests to at least
execute the `print_result` code path.
Use of `@test_deprecated` requires that the deprecation warning uses the
term "deprecated" in the warning when using `--depwarn=yes`.
@LilithHafner
Copy link
Member

My concern in #57820 (review) is a fair bit stronger here: sometimes the arguments might be large when printed.

@omus
Copy link
Member Author

omus commented Mar 19, 2025

My concern in #57820 (review) is a fair bit stronger here: sometimes the arguments might be large when printed.

That is also a concern of mine. I'll look into a possible solution with displaysize yet.

Update: we're already using print with the limit context enabled so this doesn't get too crazy

@omus
Copy link
Member Author

omus commented Mar 19, 2025

Digging into this mystery:

julia> using Test

julia> @test 1 .== 1
Error During Test at REPL[2]:1
  Test threw exception
  Expression: 1 .== 1
  UndefVarError: `.==` not defined in `Main`
  Suggestion: check for spelling errors or missing imports.
  Stacktrace:
   [1] top-level scope
     @ REPL[2]:1
   [2] macro expansion
     @ ~/Development/Julia/aarch64/latest/usr/share/julia/stdlib/v1.13/Test/src/Test.jl:705 [inlined]
ERROR: There was an error during testin

@omus
Copy link
Member Author

omus commented Mar 19, 2025

The change required to support broadcasting is big enough that I'll make a separate PR for it. Previously, broadcasted functions never showed evaluated arguments so I'll just follow that behavior.

@omus
Copy link
Member Author

omus commented Mar 20, 2025

CI failures look unrelated to me

@omus omus requested a review from dnadlinger March 20, 2025 18:16
@omus omus requested review from LilithHafner and removed request for dnadlinger March 20, 2025 18:38
Copy link
Member

@vtjnash vtjnash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM, though I don't see why there is a restriction on being a "simple call", when the printing logic doesn't care about that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testsystem The unit testing framework and Test stdlib
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants