Skip to content

Conversation

XiaotianZha
Copy link

@XiaotianZha XiaotianZha commented Oct 16, 2025

To fix timestamp format problem
ValueError: unconverted data remains when parsing with format "%Y-%m-%d": " 09:30:00-04:00", at position 109. You might want to try:
- passing format if your strings have a consistent format;
- passing format='ISO8601' if your strings are all ISO8601 but not necessarily in exactly the same format;
- passing format='mixed', and the format will be inferred for each element individually. You might want to use dayfirst alongside this.

Description

Motivation and Context

How Has This Been Tested?

  • Pass the test by running: pytest qlib/tests/test_all_pipeline.py under upper directory of qlib.
  • If you are adding a new feature, test on your own test scripts.

Screenshots of Test Results (if appropriate):

  1. Pipeline test:
  2. Your own tests:

Types of changes

  • Fix bugs
  • Add new feature
  • Update documentation

To fix timestamp format problem
ValueError: unconverted data remains when parsing with format "%Y-%m-%d": " 09:30:00-04:00", at position 109. You might want to try:
    - passing `format` if your strings have a consistent format;
    - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
    - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.
@SunsetWolf
Copy link
Collaborator

Hi, @XiaotianZha

Thanks for your attention and contribution to qlib,

I noticed that when I get the daily frequency data from yahooquery, there are some data with the time format of the last data like this %Y-%m-%d %H:%M:%S+08:00, which means that this data is the intraday data of the current stock, and the time displayed is the time of the last refresh. Stock data with this date format will show a similar error when normalizing:

ValueError: unconverted data remains when parsing with format "%Y-%m-%d": " 09:30:00+08:00"

If this data is forced to be formatted as %Y-%m-%d, it is equivalent to treating the intraday data as the closing data, which is incorrect data for daily data.

For this problem, you can refer to the previous PR 2015.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants