Skip to content

Date column in PK error #101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kolya7k opened this issue Feb 1, 2025 · 2 comments
Open

Date column in PK error #101

kolya7k opened this issue Feb 1, 2025 · 2 comments

Comments

@kolya7k
Copy link

kolya7k commented Feb 1, 2025

Table

CREATE TABLE IF NOT EXISTS `counters_daily` (
  `type` int UNSIGNED NOT NULL,
  `data` bigint UNSIGNED NOT NULL,
  `date` date NOT NULL,
  `value` bigint NOT NULL,
  PRIMARY KEY (`type`,`date`,`data`),
  KEY `date` (`date`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci ROW_FORMAT=DYNAMIC;
Traceback (most recent call last):
  File "/root/venv/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator.py", line 209, in run
    self.perform_initial_replication()
  File "/root/venv/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator.py", line 253, in perform_initial_replication
    self.perform_initial_replication_table(table)
  File "/root/venv/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator.py", line 338, in perform_initial_replication_table
    max_primary_key = max(max_primary_key, record_primary_key)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '>' not supported between instances of 'datetime.date' and 'str'
Traceback (most recent call last):
  File "/root/venv/bin/mysql_ch_replicator", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/root/venv/lib/python3.12/site-packages/mysql_ch_replicator/main.py", line 152, in main
    run_db_replicator(args, config)
  File "/root/venv/lib/python3.12/site-packages/mysql_ch_replicator/main.py", line 98, in run_db_replicator
    db_replicator.run()
  File "/root/venv/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator.py", line 209, in run
    self.perform_initial_replication()
  File "/root/venv/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator.py", line 253, in perform_initial_replication
    self.perform_initial_replication_table(table)
  File "/root/venv/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator.py", line 338, in perform_initial_replication_table
    max_primary_key = max(max_primary_key, record_primary_key)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '>' not supported between instances of 'datetime.date' and 'str'
@kolya7k
Copy link
Author

kolya7k commented Feb 1, 2025

Perhaps it may be fixed adding

from datetime import datetime, date

    def to_date_if_str(self, value):
        if isinstance(value, str):
            stripped_val = value.strip("\"'")
            try:
                return datetime.strptime(stripped_val, '%Y-%m-%d').date()
            except ValueError:
                return value
        return value

and changing

            for record in records:
                record_primary_key = [record[key_idx] for key_idx in primary_key_ids]
                record_primary_key = [self.to_date_if_str(val) for val in record_primary_key]

                if max_primary_key is None:
                    max_primary_key = record_primary_key
                else:
                    max_primary_key = [self.to_date_if_str(val) for val in max_primary_key]
                    max_primary_key = max(max_primary_key, record_primary_key)

@kolya7k
Copy link
Author

kolya7k commented Feb 2, 2025

Here is the complex PR #102

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant