valid URL containing an underscore cannot be added

[gingerbeardman]

From #202

Confirming a valid URL containing an _ (underscore) cannot be added to the system

Screenshot

Example URLs

• http://infoseek_rip.g.ribbon.to/crspalm.hp.infoseek.co.jp/
• http://a_pollett.tripod.com/mayumihf.htm

Bug

So there is a problem with the process of checking a URLs, as both of the above are valid.

[sissbruecker]

Yeah, URL validation is not perfect, and from what I understand this is quite opinionated. For now linkding just uses the default Djanog URL validator, so you could also report it there. As a workaround linkding provides an option to disable URL validation: https://github.com/sissbruecker/linkding/blob/master/docs/Options.md#ld_disable_url_validation

[gingerbeardman]

Django won't fix, since 10 years

https://code.djangoproject.com/ticket/18517

[gingerbeardman]

I'm here one year later trying linkding again (thanks to Browser Companion app for iOS/macOS) and these two URLs failed to import again, not that I expected it to have changed.

I'm trying a workaround encoding the underscore as %5F

I was able to add them as wayback machine versions.

eg

https://web.archive.org/web/20230504175240/http://infoseek_rip.g.ribbon.to/crspalm.hp.infoseek.co.jp/

😄

[gingerbeardman]

Here's another one:

• https://rvm_io.global.ssl.fastly.net/binaries/ubuntu/20.04/x86_64/

---

Yeah, URL validation is not perfect, and from what I understand this is quite opinionated. For now linkding just uses the default Djanog URL validator, so you could also report it there. As a workaround linkding provides an option to disable URL validation:
https://github.com/sissbruecker/linkding/blob/master/docs/Options.md#ld_disable_url_validation

The above link has since been broken (quick google) it's now at:

https://github.com/sissbruecker/linkding/blob/master/docs/src/content/docs/options.md

[webysther]

Switch to use https://github.com/python-validators/validators/ can be the trick, support very well RFC and can be used to expand in future to support another types of url like custom schema and local uri using feature-flag.

[webysther]

I'm here one year later trying linkding again (thanks to Browser Companion app for iOS/macOS) and these two URLs failed to import again, not that I expected it to have changed.

I'm trying a workaround encoding the underscore as %5F

I was able to add them as wayback machine versions.

eg

https://web.archive.org/web/20230504175240/http://infoseek_rip.g.ribbon.to/crspalm.hp.infoseek.co.jp/

😄

Why not disable the validation to keep using it?

[webysther]

One important point: this is not fully the same as #957 because the point about the another one is not validation is similarity support to avoid more wide duplicate than strict equal-only check.

[gingerbeardman]

I simply do not want to turn off validation, though I might change my mind about that in the future.