Repository navigation
fix highlighter bugs #4022
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+108
−15
Merged
fix highlighter bugs #4022
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
d2eb9a3
Fix patterns within regions failing on specific line lengths and columns
8debef1
Remove bad/unnecessary recursive highlight call
d6e0e38
Fix nested regions causing skipping highlighting parent region
7d1fc82
Don't needlessly match region patterns outside the region
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -55,6 +55,7 @@ testgen: | |
|
|
||
| test: | ||
| go test ./internal/... | ||
| go test ./pkg/... | ||
| go test ./cmd/... | ||
|
|
||
| bench: | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| package highlight | ||
|
redsti-github marked this conversation as resolved.
Resolved
|
||
|
|
||
| import ( | ||
| "strings" | ||
| "testing" | ||
| ) | ||
|
|
||
| func testHighlighter(t *testing.T, syntax string) *Highlighter { | ||
| t.Helper() | ||
| f, err := ParseFile([]byte(syntax)) | ||
| if err != nil { | ||
| t.Fatalf("ParseFile: %v", err) | ||
| } | ||
| def, err := ParseDef(f, &Header{}) | ||
| if err != nil { | ||
| t.Fatalf("ParseDef: %v", err) | ||
| } | ||
| return NewHighlighter(def) | ||
| } | ||
|
|
||
| // The rules inside a region must be applied no matter where on the line the | ||
| // region begins. The end match is located in the region's own slice of the | ||
| // line, so its index must not be compared against the region's absolute start | ||
| // offset: when the two happened to be equal, every rule inside the region was | ||
| // skipped and the contents were left with the region's own group. | ||
| const regionSyntax = `filetype: test | ||
|
|
||
| detect: | ||
| filename: "\\.test$" | ||
|
|
||
| rules: | ||
| - comment: | ||
| start: "//" | ||
| end: "$" | ||
| rules: | ||
| - todo: "TODO" | ||
| ` | ||
|
|
||
| func TestRegionRulesAppliedAtAnyStartOffset(t *testing.T) { | ||
| for prefix := 0; prefix < 24; prefix++ { | ||
| line := strings.Repeat("x", prefix) + "// TODO" | ||
|
|
||
| h := testHighlighter(t, regionSyntax) | ||
|
|
||
| todo, ok := Groups["todo"] | ||
| if !ok { | ||
| t.Fatal("todo group not defined") | ||
| } | ||
|
|
||
| matches := h.HighlightString(line) | ||
|
|
||
| g, ok2 := matches[0][prefix+3] | ||
| if !ok2 || g != todo { | ||
| t.Errorf("prefix %d: %q: todo inside the comment was not highlighted", prefix, line) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // The rules inside a region must be applied even if it contains a nested region. | ||
| const regionSyntax2 = `filetype: test | ||
|
|
||
| detect: | ||
| filename: "\\.test$" | ||
|
|
||
| rules: | ||
| - comment: | ||
| start: "//" | ||
| end: "$" | ||
| rules: | ||
| - todo: "TODO" | ||
| - comment: | ||
| start: "<" | ||
| end: ">" | ||
| rules: | ||
| - error: "FIXME" | ||
| ` | ||
|
|
||
| func TestRegionRulesApplyWithNestedRegion(t *testing.T) { | ||
| line := "xxx // TODO < FIXME > " | ||
|
|
||
| h := testHighlighter(t, regionSyntax2) | ||
|
|
||
| todo, ok := Groups["todo"] | ||
| if !ok { | ||
| t.Fatal("todo group not defined") | ||
| } | ||
|
|
||
| matches := h.HighlightString(line) | ||
|
|
||
| g, ok2 := matches[0][7] | ||
| if !ok2 || g != todo { | ||
| t.Errorf("%q: todo inside the comment was not highlighted", line) | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see you removed this
highlightEmptyRegion()call. Is it because you considered it redundant and pointless (even independently of this fix)?It seems indeed redundant and pointless: it highlights the same exact range as the
highlightRegion()call right below it? So, we first highlight it as an "empty" region i.e. pretending thatfirstRegionis nil, and then we overwrite this bogus highlighting with the correct highlighting in the belowhighlightRegion()? What's going on?Actually it is a question to @JoeKar who added this
highlightEmptyRegion()in 7e8ff8c#diff-16d0164b7fcc4350a9a7a5827b930e8de29487a35a3afa8d9e165b8e2af0f13aR161. What was the reason for it? And why was it correct to removeh.highlightRegion(highlights, start, false, lineNum, sliceEnd(line, firstLoc[0]), curRegion, statesOnly), replacing it with thishighlightEmptyRegion()?...Hmm, isn't this whole fix basically a revert of 7e8ff8c? @JoeKar what do you say?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I remember correctly, I introduced this back then to highlight potential current patterns as long as no further nested regions are available.
But to be honest: As I mentioned in another issue, I gave up trying to understand this highlighting logic and discarded it in favor of #3127 (which I more or less exchanged locally too).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AFAICS 7e8ff8c moved highlighting current region after recursing into nested region, while this fix moves it back before recursing into nested region. So, does this fix break whatever 7e8ff8c was fixing, or not? Can we check that? (Do we remember how to check that?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, 7e8ff8c just moved the "highlight the part that is past the end of the region and return" part to after "highlight the current region".
But it still has "highlight nested region and return" (lines 177-184) before "highlight the current region" (lines 186-205), which this change swaps.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I see.
Still, what about my earlier question (I'm just curious): did you remove this
highlightEmptyRegion()call because you considered it unneeded regardless of this change, or is this removal specifically related to this change?BTW if the former, it would be probably better to do this removal in a separate commit before this one (for easier review, to not mix two different changes in the same commit)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed it because it looked either unnecesarry, or plain wrong (why are we highlighting the same region, twice, with different rules?).
Testing it now from the first commit in this PR (3098b61), i got it misbehaving with these rules:
which given the following text:
highlights everything as you would expect, except for the last
xxx, which getstodoinstead ofidentifier, which it clearly shouldn't. (Because the "comment" region callshighlightEmptyRegionwhich highlightsxxxas "todo".)I'll split it out into it's own commit later today.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, thanks.