Throw on {write.folder-storage.path,write.object-storage.path} properties

[Fokko]

Another way of warning non-Java users about the deprecated properties

• write.folder-storage.path
• write.object-storage.path

This will only warn when write.data.path is not set. So hopefully the number of occurrences in the logs will be limited.

[snazy]

I wonder whether the now "illegal" properties should also be caught when setting namespace/table/view properties, so that it becomes impossible to set those properties (and run into hard failures when accessing data).

There seem to be more usages of OBJECT_STORE_PATH and WRITE_FOLDER_STORAGE_LOCATION - e.g. IcebergToGlueConverter and RewriteTablePathUtil

[pvary]

Instead of scattering these exceptions around the code, could we add a new DEPRECATED_PROPERTIES set like the currently existing RESERVED_PROPERTIES?

We could throw an exception if the property from this list is written, or read.
This way we can establish a single place where the properties are deprecated.

[snazy]

+1

[Fokko]

I've refactored the code to use the DEPRECATED_PROPERTIES.

My opinion is that we should not fail too fast, because that's annoying to the user. For example, here when you have write.data.path and any deprecated property, it will not fail because write.data.path takes priority. Also, other properties, such as write.manifest-lists.enabled are harmless to have them, since it is now enabled by default. Fail on reading or writing any deprecated property is a bigger discussion, and should happen on the spec level because this is bigger than just the Java implementation, and would like to decouple that from this PR.

[Fokko]

There seem to be more usages of OBJECT_STORE_PATH and WRITE_FOLDER_STORAGE_LOCATION - e.g. IcebergToGlueConverter and RewriteTablePathUtil

I think we can just remove them with 2.0, where they are used it is harmless (removing them when you copy a table, rewriting them when you write the base location, etc).

[pvary]

maybe checkDeprecation?

[stevenzwu]

checkDeprecation is also not accurate. This method actually does both: get property value and check deprecations.

the implementation/error msg is tied to TableProperties.WRITE_DATA_LOCATION. Maybe call this getLegacyDataLocation or sth like that.

The other approach is to define DEPRECATED_PROPERTIES in TableProperties class as a map (key = deprecated property, value = new property). maybe we can get to it when the need comes. right now, we can keep the change local to LocationProviders class

[Fokko]

Went for getAndCheckLegacyLocation since it also applies to metadata. And yes, everything is private, so we can always move to the TableProperties.

[snazy]

I wonder whether the now "illegal" properties should also be caught when setting namespace/table/view properties, so that it becomes impossible to set those properties (and run into hard failures when accessing data).

I think that would quite useful to prevent users from putting in any of the "illegal" properties. WDYT?

[Fokko]

I think that would quite useful to prevent users from putting in any of the "illegal" properties. WDYT?

I'm not a fan of it, because I think it is very hard to enforce across implementations. For example, we don't support the ones that were deprecated in this PR in PyIceberg/Rust/Go, let other vendors. Since we need to maintain this list, we're also not able to completely remove them at some point.