Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ testgen:

test:
go test ./internal/...
go test ./pkg/...
go test ./cmd/...

bench:
Expand Down
28 changes: 13 additions & 15 deletions pkg/highlight/highlighter.go
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ func (h *Highlighter) highlightRegion(highlights LineMatch, start int, canMatchE
searchNesting := true
endLoc := findIndex(curRegion.end, curRegion.skip, line)
if endLoc != nil {
if start == endLoc[0] {
if endLoc[0] == 0 {
searchNesting = false
} else {
firstLoc = endLoc
Expand All @@ -142,30 +142,20 @@ func (h *Highlighter) highlightRegion(highlights LineMatch, start int, canMatchE
}
}
}
if firstRegion != nil && firstLoc[0] != lineLen {
if !statesOnly {
highlights[start+firstLoc[0]] = firstRegion.limitGroup
}
h.highlightEmptyRegion(highlights, start+firstLoc[1], canMatchEnd, lineNum, sliceStart(line, firstLoc[1]), statesOnly)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you removed this highlightEmptyRegion() call. Is it because you considered it redundant and pointless (even independently of this fix)?

It seems indeed redundant and pointless: it highlights the same exact range as the highlightRegion() call right below it? So, we first highlight it as an "empty" region i.e. pretending that firstRegion is nil, and then we overwrite this bogus highlighting with the correct highlighting in the below highlightRegion()? What's going on?

Actually it is a question to @JoeKar who added this highlightEmptyRegion() in 7e8ff8c#diff-16d0164b7fcc4350a9a7a5827b930e8de29487a35a3afa8d9e165b8e2af0f13aR161. What was the reason for it? And why was it correct to remove h.highlightRegion(highlights, start, false, lineNum, sliceEnd(line, firstLoc[0]), curRegion, statesOnly), replacing it with this highlightEmptyRegion()?

...Hmm, isn't this whole fix basically a revert of 7e8ff8c? @JoeKar what do you say?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I remember correctly, I introduced this back then to highlight potential current patterns as long as no further nested regions are available.
But to be honest: As I mentioned in another issue, I gave up trying to understand this highlighting logic and discarded it in favor of #3127 (which I more or less exchanged locally too).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICS 7e8ff8c moved highlighting current region after recursing into nested region, while this fix moves it back before recursing into nested region. So, does this fix break whatever 7e8ff8c was fixing, or not? Can we check that? (Do we remember how to check that?)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, 7e8ff8c just moved the "highlight the part that is past the end of the region and return" part to after "highlight the current region".

But it still has "highlight nested region and return" (lines 177-184) before "highlight the current region" (lines 186-205), which this change swaps.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I see.

Still, what about my earlier question (I'm just curious): did you remove this highlightEmptyRegion() call because you considered it unneeded regardless of this change, or is this removal specifically related to this change?

BTW if the former, it would be probably better to do this removal in a separate commit before this one (for easier review, to not mix two different changes in the same commit)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed it because it looked either unnecesarry, or plain wrong (why are we highlighting the same region, twice, with different rules?).

Testing it now from the first commit in this PR (3098b61), i got it misbehaving with these rules:

rules:
    - todo: "xxx"
    - comment:
        start: "//"
        end: "$"
        rules:
            - identifier:
                start: "<"
                end: ">"
                rules:
                    - todo: "todo"

which given the following text:

xxx
// xxx < xxx >

highlights everything as you would expect, except for the last xxx, which gets todo instead of identifier, which it clearly shouldn't. (Because the "comment" region calls highlightEmptyRegion which highlights xxx as "todo".)

I'll split it out into it's own commit later today.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks.

h.highlightRegion(highlights, start+firstLoc[1], canMatchEnd, lineNum, sliceStart(line, firstLoc[1]), firstRegion, statesOnly)
return highlights
}

if !statesOnly {
fullHighlights := make([]Group, lineLen)
fullHighlights := make([]Group, firstLoc[0])
for i := 0; i < len(fullHighlights); i++ {
fullHighlights[i] = curRegion.group
}

if searchNesting {
for _, p := range curRegion.rules.patterns {
if curRegion.group == curRegion.limitGroup || p.group == curRegion.limitGroup {
matches := findAllIndex(p.regex, line)
matches := findAllIndex(p.regex, sliceEnd(line, firstLoc[0]))
for _, m := range matches {
if (endLoc == nil) || (m[0] < endLoc[0]) {
for i := m[0]; i < m[1]; i++ {
fullHighlights[i] = p.group
}
for i := m[0]; i < m[1]; i++ {
fullHighlights[i] = p.group
}
}
}
Expand All @@ -178,6 +168,14 @@ func (h *Highlighter) highlightRegion(highlights LineMatch, start int, canMatchE
}
}

if firstRegion != nil && firstLoc[0] != lineLen {
if !statesOnly {
highlights[start+firstLoc[0]] = firstRegion.limitGroup
}
h.highlightRegion(highlights, start+firstLoc[1], canMatchEnd, lineNum, sliceStart(line, firstLoc[1]), firstRegion, statesOnly)
return highlights
}

loc := endLoc
if loc != nil {
if !statesOnly {
Expand Down
94 changes: 94 additions & 0 deletions pkg/highlight/highlighter_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
package highlight
Comment thread
redsti-github marked this conversation as resolved.
Resolved

import (
"strings"
"testing"
)

func testHighlighter(t *testing.T, syntax string) *Highlighter {
t.Helper()
f, err := ParseFile([]byte(syntax))
if err != nil {
t.Fatalf("ParseFile: %v", err)
}
def, err := ParseDef(f, &Header{})
if err != nil {
t.Fatalf("ParseDef: %v", err)
}
return NewHighlighter(def)
}

// The rules inside a region must be applied no matter where on the line the
// region begins. The end match is located in the region's own slice of the
// line, so its index must not be compared against the region's absolute start
// offset: when the two happened to be equal, every rule inside the region was
// skipped and the contents were left with the region's own group.
const regionSyntax = `filetype: test

detect:
filename: "\\.test$"

rules:
- comment:
start: "//"
end: "$"
rules:
- todo: "TODO"
`

func TestRegionRulesAppliedAtAnyStartOffset(t *testing.T) {
for prefix := 0; prefix < 24; prefix++ {
line := strings.Repeat("x", prefix) + "// TODO"

h := testHighlighter(t, regionSyntax)

todo, ok := Groups["todo"]
if !ok {
t.Fatal("todo group not defined")
}

matches := h.HighlightString(line)

g, ok2 := matches[0][prefix+3]
if !ok2 || g != todo {
t.Errorf("prefix %d: %q: todo inside the comment was not highlighted", prefix, line)
}
}
}

// The rules inside a region must be applied even if it contains a nested region.
const regionSyntax2 = `filetype: test

detect:
filename: "\\.test$"

rules:
- comment:
start: "//"
end: "$"
rules:
- todo: "TODO"
- comment:
start: "<"
end: ">"
rules:
- error: "FIXME"
`

func TestRegionRulesApplyWithNestedRegion(t *testing.T) {
line := "xxx // TODO < FIXME > "

h := testHighlighter(t, regionSyntax2)

todo, ok := Groups["todo"]
if !ok {
t.Fatal("todo group not defined")
}

matches := h.HighlightString(line)

g, ok2 := matches[0][7]
if !ok2 || g != todo {
t.Errorf("%q: todo inside the comment was not highlighted", line)
}
}
Loading