Skip to content

[WIP] Cfg loops #17175

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

[WIP] Cfg loops #17175

wants to merge 5 commits into from

Conversation

dylwil3
Copy link
Collaborator

@dylwil3 dylwil3 commented Apr 3, 2025

WIP

@MichaReiser
Copy link
Member

I think there are a few lifetimes that can be simplified in the mermaid testing code

Subject: [PATCH] Align server indexing with CLI behavior
---
Index: crates/ruff_python_semantic/src/cfg/graph.rs
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/crates/ruff_python_semantic/src/cfg/graph.rs b/crates/ruff_python_semantic/src/cfg/graph.rs
--- a/crates/ruff_python_semantic/src/cfg/graph.rs	(revision dca1340348d73b1bf1cd10de593b1ef01d9363ec)
+++ b/crates/ruff_python_semantic/src/cfg/graph.rs	(date 1743696804592)
@@ -49,7 +49,7 @@
 
     /// Returns the [`Edges`] going out of the basic block at the given index
     pub fn outgoing(&self, block: BlockId) -> &Edges {
-        &self.blocks[block].out
+        self.blocks[block].out
     }
 
     /// Returns an iterator over the indices of the direct predecessors of the block at the given index
@@ -132,7 +132,7 @@
     }
 
     /// Returns iterator over [`Condition`]s which must be satisfied to traverse corresponding edge
-    pub fn conditions(&self) -> impl ExactSizeIterator<Item = &Condition> {
+    pub fn conditions(&self) -> impl ExactSizeIterator<Item = &Condition<'stmt>> {
         self.conditions.iter()
     }
 
@@ -140,7 +140,7 @@
         self.targets.is_empty()
     }
 
-    pub fn filter_targets_by_conditions<'a, T: FnMut(&Condition) -> bool + 'a>(
+    pub fn filter_targets_by_conditions<'a: 'stmt, T: FnMut(&Condition) -> bool + 'a>(
         &'a self,
         mut predicate: T,
     ) -> impl Iterator<Item = BlockId> + 'a {
Index: crates/ruff_python_semantic/src/cfg/visualize.rs
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/crates/ruff_python_semantic/src/cfg/visualize.rs b/crates/ruff_python_semantic/src/cfg/visualize.rs
--- a/crates/ruff_python_semantic/src/cfg/visualize.rs	(revision dca1340348d73b1bf1cd10de593b1ef01d9363ec)
+++ b/crates/ruff_python_semantic/src/cfg/visualize.rs	(date 1743696538214)
@@ -10,7 +10,7 @@
     CFGWithSource::new(graph, source).draw_graph()
 }
 
-trait MermaidGraph<'a>: DirectedGraph<'a> {
+trait MermaidGraph: DirectedGraph {
     fn draw_node(&self, node: Self::Node) -> MermaidNode;
     fn draw_edges(&self, node: Self::Node) -> impl Iterator<Item = (Self::Node, MermaidEdge)>;
 
@@ -146,7 +146,7 @@
     }
 }
 
-pub trait DirectedGraph<'a> {
+pub trait DirectedGraph {
     type Node: Idx;
 
     fn num_nodes(&self) -> usize;
@@ -154,18 +154,18 @@
     fn successors(&self, node: Self::Node) -> impl ExactSizeIterator<Item = Self::Node> + '_;
 }
 
-struct CFGWithSource<'stmt> {
+struct CFGWithSource<'source, 'stmt> {
     cfg: ControlFlowGraph<'stmt>,
-    source: &'stmt str,
+    source: &'source str,
 }
 
-impl<'stmt> CFGWithSource<'stmt> {
-    fn new(cfg: ControlFlowGraph<'stmt>, source: &'stmt str) -> Self {
+impl<'source, 'stmt> CFGWithSource<'source, 'stmt> {
+    fn new(cfg: ControlFlowGraph<'stmt>, source: &'source str) -> Self {
         Self { cfg, source }
     }
 }
 
-impl<'stmt> DirectedGraph<'stmt> for CFGWithSource<'stmt> {
+impl DirectedGraph for CFGWithSource<'_, '_> {
     type Node = BlockId;
 
     fn num_nodes(&self) -> usize {
@@ -181,7 +181,7 @@
     }
 }
 
-impl<'stmt> MermaidGraph<'stmt> for CFGWithSource<'stmt> {
+impl MermaidGraph for CFGWithSource<'_, '_> {
     fn draw_node(&self, node: Self::Node) -> MermaidNode {
         let statements: Vec<String> = self
             .cfg

But I don't see an obvious work around for the lifetime issues other than using an IndexVec stored on ControlFlowGraph to store all the conditions and then store the ConditionId in Edges::conditions (or to use a Vec in Edges::conditions)

@dylwil3
Copy link
Collaborator Author

dylwil3 commented Apr 4, 2025

Hmm... not sure what I did wrong but I applied your patch exactly and the code did not compile. This seems to be the problem:

     /// Returns the [`Edges`] going out of the basic block at the given index
     pub fn outgoing(&self, block: BlockId) -> &Edges {
-        &self.blocks[block].out
+        self.blocks[block].out
     }

This doesn't agree with the return signature. If we remove the borrow we get a lifetime issue. It all works fine if we switch from SmallVec to Vec though so I'm gonna do that for now, and we can revisit a more performant data structure a little later.

Copy link
Contributor

github-actions bot commented Apr 4, 2025

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+166 -1 violations, +0 -0 fixes in 19 projects; 36 projects unchanged)

DisnakeDev/disnake (+7 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ disnake/ext/commands/converter.py:1019:9: PLW0101 Unreachable code in `convert`
+ disnake/ext/commands/slash_core.py:56:5: PLW0101 Unreachable code in `_autocomplete`
+ disnake/message.py:1328:9: PLW0101 Unreachable code in `_clear_emoji`
+ disnake/state.py:2531:9: PLW0101 Unreachable code in `_delay_ready`
+ disnake/ui/select/base.py:234:13: PLW0101 Unreachable code in `_transform_default_values`
+ scripts/codemods/overloads_no_missing.py:29:9: PLW0101 Unreachable code in `leave_FunctionDef`
+ scripts/codemods/typed_permissions.py:84:9: PLW0101 Unreachable code in `leave_ClassDef`

apache/airflow (+30 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ airflow-core/src/airflow/example_dags/plugins/workday.py:57:9: PLW0101 Unreachable code in `get_next_workday`
+ airflow-core/src/airflow/models/dag.py:2551:5: PLW0101 Unreachable code in `_run_task`
+ airflow-core/tests/integration/otel/dags/otel_test_dag_with_pause_between_tasks.py:139:5: PLW0101 Unreachable code in `paused_task`
+ airflow-core/tests/integration/otel/dags/otel_test_dag_with_pause_in_task.py:74:5: PLW0101 Unreachable code in `task1`
+ airflow-core/tests/unit/serialization/test_dag_serialization.py:602:9: PLW0101 Unreachable code in `test_deserialization_across_process`
+ providers/amazon/src/airflow/providers/amazon/aws/hooks/quicksight.py:169:9: PLW0101 Unreachable code in `wait_for_state`
+ providers/amazon/src/airflow/providers/amazon/aws/hooks/redshift_data.py:269:9: PLW0101 Unreachable code in `get_table_primary_key`
+ providers/amazon/src/airflow/providers/amazon/aws/hooks/sagemaker.py:760:9: PLW0101 Unreachable code in `check_status`
+ providers/amazon/src/airflow/providers/amazon/aws/hooks/sagemaker.py:843:9: PLW0101 Unreachable code in `check_training_status_with_log`
+ providers/amazon/src/airflow/providers/amazon/aws/transfers/dynamodb_to_s3.py:249:9: PLW0101 Unreachable code in `_scan_dynamodb_and_upload_to_s3`
... 20 additional changes omitted for project

apache/superset (+4 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ superset/commands/importers/v1/utils.py:232:5: PLW0101 Unreachable code in `get_resource_mappings_batched`
+ superset/migrations/shared/native_filters.py:95:13: PLW0101 Unreachable code in `convert_filter_scopes_to_native_filters`
+ superset/tasks/cache.py:311:5: PLW0101 Unreachable code in `cache_warmup`
+ superset/utils/slack.py:88:5: PLW0101 Unreachable code in `get_channels`

aws/aws-sam-cli (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ tests/integration/local/common_utils.py:34:5: PLW0101 Unreachable code in `wait_for_local_process`

bokeh/bokeh (+2 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ tests/support/plugins/file_server.py:93:9: PLW0101 Unreachable code in `__init__`
+ tests/support/plugins/selenium.py:79:9: PLW0101 Unreachable code in `chrome`

freedomofpress/securedrop (+5 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ securedrop/management/run.py:125:9: PLW0101 Unreachable code in `monitor`
+ securedrop/pretty_bad_protocol/_meta.py:666:9: PLW0101 Unreachable code in `_read_response`
+ securedrop/pretty_bad_protocol/_meta.py:689:9: PLW0101 Unreachable code in `_read_data`
+ securedrop/pretty_bad_protocol/_util.py:137:5: PLW0101 Unreachable code in `_copy_data`
+ securedrop/tests/test_secure_tempfile.py:92:5: PLW0101 Unreachable code in `test_buffered_read`

fronzbot/blinkpy (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ blinksync/blinksync.py:102:5: PLW0101 Unreachable code in `main`

ibis-project/ibis (+3 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ ibis/backends/impala/metadata.py:153:9: PLW0101 Unreachable code in `_parse_schema`
+ ibis/backends/impala/metadata.py:174:9: PLW0101 Unreachable code in `_parse_info`
+ ibis/backends/impala/metadata.py:258:9: PLW0101 Unreachable code in `_parse_nested_params`

langchain-ai/langchain (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ libs/core/langchain_core/runnables/base.py:5161:13: PLW0101 Unreachable code in `astream_events`

latchbio/latch (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ src/latch_cli/tinyrequests.py:111:5: PLW0101 Unreachable code in `_req`

milvus-io/pymilvus (+10 -1 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ examples/bulk_import/example_bulkinsert_csv.py:170:5: PLW0101 Unreachable code in `wait_tasks_to_state`
+ examples/bulk_import/example_bulkinsert_json.py:212:5: PLW0101 Unreachable code in `wait_tasks_to_state`
+ examples/bulk_import/example_bulkinsert_numpy.py:214:5: PLW0101 Unreachable code in `wait_tasks_to_state`
+ examples/bulk_import/example_bulkinsert_withfunction.py:170:5: PLW0101 Unreachable code in `wait_tasks_to_state`
+ examples/bulk_import/example_bulkwriter.py:407:5: PLW0101 Unreachable code in `call_bulkinsert`
+ examples/bulk_import/example_bulkwriter_with_nullable.py:319:5: PLW0101 Unreachable code in `call_bulkinsert`
- examples/iterator/iterator.py:64:9: PLW0101 Unreachable code in `external_filter_func`
+ examples/iterator/iterator.py:65:9: PLW0101 Unreachable code in `external_filter_func`
+ examples/orm/iterator.py:124:5: PLW0101 Unreachable code in `query_iterate_collection_no_offset`
+ pymilvus/client/search_iterator.py:150:9: PLW0101 Unreachable code in `next`
... 1 additional changes omitted for project

pandas-dev/pandas (+5 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ pandas/_version.py:111:5: PLW0101 Unreachable code in `run_command`
+ pandas/core/sorting.py:208:5: PLW0101 Unreachable code in `get_group_index`
+ pandas/io/html.py:1007:5: PLW0101 Unreachable code in `_parse`
+ pandas/tests/extension/test_categorical.py:39:5: PLW0101 Unreachable code in `make_data`
+ pandas/tests/io/test_sql.py:1169:5: PLW0101 Unreachable code in `test_read_iris_query_string_with_parameter`

pypa/cibuildwheel (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ cibuildwheel/oci_container.py:461:9: PLW0101 Unreachable code in `call`

rotki/rotki (+51 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ rotkehlchen/accounting/accountant.py:236:9: PLW0101 Unreachable code in `process_history`
+ rotkehlchen/chain/base/modules/echo/decoder.py:156:9: PLW0101 Unreachable code in `_process_funding`
+ rotkehlchen/chain/ethereum/etherscan.py:113:9: PLW0101 Unreachable code in `get_withdrawals`
+ rotkehlchen/chain/ethereum/modules/compound/v2/decoder.py:384:9: PLW0101 Unreachable code in `decode_comp_claim`
+ rotkehlchen/chain/ethereum/modules/curve/crvusd/decoder.py:207:9: PLW0101 Unreachable code in `_decode_peg_keeper_update`
+ rotkehlchen/chain/ethereum/modules/golem/decoder.py:71:9: PLW0101 Unreachable code in `_decode_migration`
+ rotkehlchen/chain/ethereum/modules/lido/decoder.py:83:9: PLW0101 Unreachable code in `_decode_lido_staking_in_steth`
+ rotkehlchen/chain/ethereum/modules/shutter/decoder.py:77:9: PLW0101 Unreachable code in `_decode_shutter_claim`
+ rotkehlchen/chain/ethereum/modules/yearn/decoder.py:239:9: PLW0101 Unreachable code in `_handle_transfer_events`
+ rotkehlchen/chain/ethereum/modules/yearn/utils.py:132:5: PLW0101 Unreachable code in `_query_yearn_vaults`
+ rotkehlchen/chain/evm/decoding/aave/common.py:442:9: PLW0101 Unreachable code in `_decode_incentives_common`
+ rotkehlchen/chain/evm/decoding/balancer/utils.py:103:5: PLW0101 Unreachable code in `query_balancer_pools`
+ rotkehlchen/chain/evm/decoding/compound/v3/decoder.py:206:9: PLW0101 Unreachable code in `_decode_supply_or_repay_event`
+ rotkehlchen/chain/evm/decoding/compound/v3/decoder.py:356:9: PLW0101 Unreachable code in `_decode_collateral_movement`
+ rotkehlchen/chain/evm/decoding/curve/decoder.py:621:13: PLW0101 Unreachable code in `_decode_deposit_and_stake`
... 36 additional changes omitted for project

zulip/zulip (+17 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ zerver/actions/message_flags.py:93:5: PLW0101 Unreachable code in `do_mark_all_as_read`
+ zerver/actions/realm_settings.py:671:9: PLW0101 Unreachable code in `do_delete_all_realm_attachments`
+ zerver/data_import/slack.py:1687:5: PLW0101 Unreachable code in `get_slack_api_data`
+ zerver/lib/cache.py:327:5: PLW0101 Unreachable code in `cache_delete_many`
+ zerver/lib/export.py:2693:5: PLW0101 Unreachable code in `get_id_list_gently_from_database`
+ zerver/lib/export.py:2708:5: PLW0101 Unreachable code in `chunkify`
+ zerver/lib/import_realm.py:1904:5: PLW0101 Unreachable code in `get_incoming_message_ids`
+ zerver/lib/retention.py:179:5: PLW0101 Unreachable code in `run_archiving`
+ zerver/lib/soft_deactivation.py:299:5: PLW0101 Unreachable code in `do_soft_deactivate_users`
+ zerver/lib/templates.py:190:5: PLW0101 Unreachable code in `webpack_entry`
... 7 additional changes omitted for project

... Truncated remaining completed project reports due to GitHub comment length restrictions

Changes by rule (1 rules affected)

code total + violation - violation + fix - fix
PLW0101 167 166 1 0 0

@dylwil3 dylwil3 mentioned this pull request Apr 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants