Add optional libhdfs backend for HDFS sources

[Copilot]

Nebula Importer’s HDFS reader only supported the pure-Go client path, which is not suitable for clusters that require Hadoop’s native client behavior such as KMS-backed encryption zones. This change adds an opt-in libhdfs backend and the config needed to route HDFS reads through the native Hadoop stack when required.

• Backend selection

  • Adds sources[].hdfs.backend with go as the default and libhdfs as an opt-in path.
  • Keeps the existing HDFS behavior unchanged unless backend: libhdfs is explicitly configured.

• Native HDFS path

  • Adds a new libhdfs source implementation for open/read/stat/list/glob operations.
  • Uses Hadoop’s native client entrypoints rather than the pure-Go HDFS library when the native backend is selected.
  • Includes a non-libhdfs stub so default builds continue to work without CGO or native Hadoop libraries.

• Hadoop/Kerberos configuration

  • Adds HDFS config fields for native client setup:
    • hadoopConfigDir
    • hadoopHome
    • javaHome
    • coreSiteFile
    • hdfsSiteFile
  • Wires Kerberos-related settings into the native path, including support for krb5ConfigFile and ccacheFile.
  • Resolves auxiliary HDFS config file paths relative to the importer config file, consistent with other source path handling.

• Build and docs

  • Adds a dedicated build-libhdfs make target for CGO + libhdfs builds.
  • Documents the new backend and configuration surface in the HDFS docs.
  • Adds an example config for a libhdfs-backed HDFS source.

Example:

sources:
  - hdfs:
      backend: libhdfs
      address: "hdfs://nameservice1"
      user: "hdfs"
      hadoopConfigDir: "/etc/hadoop/conf"
      krb5ConfigFile: "/etc/krb5.conf"
      ccacheFile: "/tmp/krb5cc_1000"
      path: "/secure/events/20190918.export.csv"

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}