Issue with parsing dates (valid_until)

[prijic87]

Hello,

I stumbled upon an issue with parsing of dates from my MISP instance, where misp2sentinel tries to upload expired indicators using the new Upload Indicators API. The integration is working, and I can see indicators in Sentinel, but it seems that certain indicators are uploaded which should not, as they are expired, or I misunderstood how the script behaves. The script is run in dry run mode, to prevent sending indicators to Sentinel again.

MISP Version: 2.5.6

Config.py:
misp_event_filters = {
"published": 1,
"tags": ["redacted],
#"enforceWarninglist": True,
"includeEventTags": True,
"publish_timestamp": "1d",
}

days_to_expire = 50 # Graph API and Upload Indicators
days_to_expire_start = "current_date" # Upload Indicators API only. Start counting from "valid_from" | "current_date" ;
days_to_expire_mapping = { # Upload indicators API only. Mapping for expiration of specific indicator types
"ipv4-addr": 180,
"ipv6-addr": 180,
"domain-name": 180,
"url": 365,
}

verbose_log = True
write_parsed_indicators = True # Upload Indicators only
write_parsed_eventid = True

misp_flatten_attributes = True # Convert all attributes in objects to "atomic" attributes. This can help when attributes are not "mapped" in misp-stix
dry_run = True

I have integration with 3rd party TI provider, where MISP is being updated in a way that script edits the existing events, and then publishes them, so published timestamp is changed every time the Event has been updated.

For testing purposes, I have modified one event, and added a custom tag, so the filter will return only one event which at the time of writing has attributes: 438 (89 Objects).

I have also included all of the debug options, and made a very small modification to script so I can better understand what is going on. The modification is on line 124 of _get_misp_events_stix to include the Event ID and valid_until date.

The Issue:
2025-02-21 13:48:13,723 - misp2sentinel - DEBUG - Add [file:hashes.MD5 = '21e4cade48fdfb98587173a793883bd1'] to list of indicators to upload in event 9205, valid_until: 2025-04-12T13:48:13Z
2025-02-21 13:48:13,725 - misp2sentinel - DEBUG - Add [file:hashes.SHA1 = '6bd8edb83f47892343aa6e7639c62de24db606e4'] to list of indicators to upload in event 9205, valid_until: 2025-04-12T13:48:13Z
2025-02-21 13:48:13,727 - misp2sentinel - DEBUG - Add [file:hashes.SHA256 = '38a67836c20704a1f2f740403437fda61d02c34c4db653e8ee2a059ed444806c'] to list of indicators to upload in event 9205, valid_until: 2025-04-12T13:48:13Z

...

The file:hashes are part of MISP object, which I initially thought it is linked with Issue regarding the IDS flag MISP/misp-stix#48 but I later found that in this MISP event, there are more objects, which are ignored by the script except this one.

The script added 50 days from the current date (21.02.2025)

Other issues are with domain names, where script tries to add 4 indicators as it thinks they are valid, but in MISP there is no difference from other entries in the same event that are skipped as outdated by the script:

2025-02-21 13:48:14,053 - misp2sentinel - ERROR - Skipping outdated indicator [url:value = 'http://207.148.121.45/stubs/nt/Salary-Benefits-Coach-2024.pdf'] in event 9205, valid_until: 2024-04-16T11:15:26Z
2025-02-21 13:48:14,056 - misp2sentinel - ERROR - Skipping outdated indicator [url:value = 'http://207.148.121.45/stubs/nt/NT_v2.4.2_2_SC.cmd'] in event 9205, valid_until: 2024-04-16T11:15:26Z
2025-02-21 13:48:14,058 - misp2sentinel - DEBUG - Add [url:value = 'http://207.148.121.45/stubs/nt/nt_v2.4.2_1_startup.cmd'] to list of indicators to upload in event 9205, valid_until: 2026-02-21T13:48:14Z
2025-02-21 13:48:14,061 - misp2sentinel - DEBUG - Add [url:value = 'http://207.148.121.45/stubs/nt/nt_v2.4.2_ch3_startup.exe'] to list of indicators to upload in event 9205, valid_until: 2026-02-21T13:48:14Z
2025-02-21 13:48:14,063 - misp2sentinel - ERROR - Skipping outdated indicator [url:value = 'http://207.148.121.45/stubs/nt/nt.ps1'] in event 9205, valid_until: 2024-04-19T19:27:10Z
2025-02-21 13:48:14,066 - misp2sentinel - ERROR - Skipping outdated indicator [domain-name:value = 'uyt1n8ded9fb380.com'] in event 9205, valid_until: 2024-11-13T17:51:36Z
2025-02-21 13:48:14,068 - misp2sentinel - DEBUG - Add [domain-name:value = 'karishmaerik324.com'] to list of indicators to upload in event 9205, valid_until: 2025-08-20T13:48:14Z
2025-02-21 13:48:14,070 - misp2sentinel - DEBUG - Add [domain-name:value = 'lechiwees.fr'] to list of indicators to upload in event 9205, valid_until: 2025-08-20T13:48:14Z
2025-02-21 13:48:14,072 - misp2sentinel - ERROR - Skipping outdated indicator [domain-name:value = 'ninedragonsfortress.com'] in event 9205, valid_until: 2024-11-19T07:58:36Z
2025-02-21 13:48:14,074 - misp2sentinel - ERROR - Skipping outdated indicator [domain-name:value = 'deolahuman.shop'] in event 9205, valid_until: 2024-10-02T20:29:39Z
2025-02-21 13:48:14,076 - misp2sentinel - ERROR - Skipping outdated indicator [domain-name:value = 'mina324234.com'] in event 9205, valid_until: 2024-12-19T23:18:47Z

I cannot understand how these entries are added to the list of indicators to be sent, while others are skipped as outdated.

Many thanks!
Luka