Skip to content

remove duplicates feature #1312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
romainds-tech opened this issue Feb 13, 2025 · 1 comment
Open

remove duplicates feature #1312

romainds-tech opened this issue Feb 13, 2025 · 1 comment
Labels
feature help wanted Extra attention is needed

Comments

@romainds-tech
Copy link

I have a table with a lot of entries (+440,000).

However, I have a lot of duplicates (a duplicate can be defined when the same column in 2 rows is identical). And I can't see how I can simply delete them from the interface? Here's an example of the SQL query I had to make when connecting to the database (in the docker container). :

WITH duplicates AS (
  SELECT 
    ctid,
    ROW_NUMBER() OVER (
      PARTITION BY “column_name”
      ORDER BY ctid
    ) AS rn
  FROM “schema_name”. “table_name”
)
DELETE FROM “schema_name”. “table_name”
WHERE ctid IN (
  SELECT ctid
  FROM duplicates
  WHERE rn > 1
);

If this feature is added at the interface level when you're on a grid, there could be a sort of 'delete duplicates' button with a column to be defined. I don't know if the scope of this feature stops there, or if it encompasses other possibilities for deleting duplicates.

@tea-artist tea-artist added help wanted Extra attention is needed feature labels Feb 14, 2025
@tea-artist
Copy link
Contributor

Removing duplicate content involves different strategies and is a "write" operation. It is better to implement this requirement through plugins.

We are perfecting the infrastructure of the plug-in system to support such requirements flexibly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants