Skip to content

Conversation

emma-sg
Copy link
Member

@emma-sg emma-sg commented Oct 15, 2025

Closes #2729

Changes

Adds a filter to the workflow list that allows filtering by last crawl state.

This filter includes all crawl states as options, as opposed to the archived item filter which only includes "finished" states.

Screenshot

Screenshot 2025-10-15 at 5 35 47 PM

@emma-sg emma-sg requested review from SuaYoo and tw4l October 15, 2025 21:36
@ikreymer ikreymer added this to the 1.19.4 Release milestone Oct 15, 2025
@tw4l tw4l force-pushed the filter-by-last-crawl-state branch from a1e1d2e to 649e8bc Compare October 16, 2025 16:07
Copy link
Member

@SuaYoo SuaYoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since workflows only do one crawl at a time, I think leaving off "Last" is still legible and more concise: "Crawl Status"? I don't feel too strongly about this, though, if the consensus is to keep it.

@emma-sg
Copy link
Member Author

emma-sg commented Oct 20, 2025

Yeah, I guess it depends whether we think of "crawl" here as "the process of crawling" (generalized) vs "an instance of this process of crawling" (specific). My impulse would be towards the latter, especially since we've got a list of crawls that you can see from the archived item page & from individual workflows. I worry that without "last" users might expect this to filter by statuses from all past crawls, or something like that.

@tw4l
Copy link
Member

tw4l commented Oct 20, 2025

Yeah, I guess it depends whether we think of "crawl" here as "the process of crawling" (generalized) vs "an instance of this process of crawling" (specific). My impulse would be towards the latter, especially since we've got a list of crawls that you can see from the archived item page & from individual workflows. I worry that without "last" users might expect this to filter by statuses from all past crawls, or something like that.

Could also be "Latest Crawl" to match the corresponding tab in the workflow?

@emma-sg
Copy link
Member Author

emma-sg commented Oct 20, 2025

Oh true, I forgot we'd used "Latest" — I think that's slightly clearer (though slightly longer too), I'll update it.

@SuaYoo
Copy link
Member

SuaYoo commented Oct 20, 2025

I worry that without "last" users might expect this to filter by statuses from all past crawls, or something like that.

FYI we do mention in the user guide that workflow statuses are either inherited or their "own":

Finished crawl workflows inherit the status of the last archived item they created. Crawl workflows that are in progress maintain their own statuses.

This makes me think just "Status" is the most accurate label, since that's how it's also labeled in the workflow detail view:

Screenshot 2025-10-20 at 1 34 35 PM

@emma-sg
Copy link
Member Author

emma-sg commented Oct 20, 2025

Okay, I see your point, though I do still wonder if it's confusing in the info bar — I almost wonder if "Latest Crawl - Complete" would make more sense there too. That's out of scope of this pr though. For now, I wonder if we could eliminate the "status" part as well, and have the filter just say "Latest Crawl"...? I'm not sure, I feel like I could be convinced either way: the way it's explained in the docs makes just "Status" make more sense, but I do also kiiinda feel like that's confusing too, since it suggests that it's sort of a separate state from the latest crawl's state, which slightly conflicts with how it's displayed in the workflow's crawls list where both the workflow and the currently-running crawl have a state.

@SuaYoo
Copy link
Member

SuaYoo commented Oct 20, 2025

I do also kiiinda feel like that's confusing too, since it suggests that it's sort of a separate state from the latest crawl's state, which slightly conflicts with how it's displayed in the workflow's crawls list where both the workflow and the currently-running crawl have a state.

Yeah that's fair. There's also this issue that hasn't been prioritized yet that proposes changing the workflows list columns to match the details, so we can reassess the filter label when we update the workflow columns.

Screenshot 2025-10-20 at 1 32 51 PM

Copy link
Member

@tw4l tw4l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backend changes look good!

@emma-sg emma-sg merged commit 6b183f8 into main Oct 21, 2025
29 checks passed
@emma-sg emma-sg deleted the filter-by-last-crawl-state branch October 21, 2025 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Task]: Filter/sort workflows by last crawl state

4 participants