|
| 1 | +--- |
| 2 | +title: "Apache Hive : Setting up Metastore backed by MariaDB" |
| 3 | +date: 2025-11-05 |
| 4 | +--- |
| 5 | + |
| 6 | +# Apache Hive : Setting up Metastore backed by MariaDB |
| 7 | + |
| 8 | +{{< toc >}} |
| 9 | + |
| 10 | +## Note |
| 11 | + |
| 12 | +**Starting from mysql-connector-java 8.0.12, using the default MySQL driver the Metastore cannot be up to service.** |
| 13 | + |
| 14 | +## Introduction |
| 15 | + |
| 16 | +From mysql-connector-java 8.0.12, MySQL driver issues a getSQLKeywords call for retrieving this database's keywords, triggered by MySQLAdapter(DataNucleus) on Metastore initialization. |
| 17 | +However, the back table `KEYWORDS` in MariaDB diverged from that in MySQL, which makes the Metastore fail to start, an exception thrown like: |
| 18 | + |
| 19 | +``` |
| 20 | +Caused by: java.sql.SQLSyntaxErrorException: Unknown column 'RESERVED' in 'where clause' |
| 21 | + at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120) ~[mysql-connector-java.jar:8.0.22] |
| 22 | + at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97) ~[mysql-connector-java.jar:8.0.22] |
| 23 | + at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) ~[mysql-connector-java.jar:8.0.22] |
| 24 | + at com.mysql.cj.jdbc.StatementImpl.executeQuery(StatementImpl.java:1200) ~[mysql-connector-java.jar:8.0.22] |
| 25 | + at com.mysql.cj.jdbc.DatabaseMetaDataUsingInfoSchema.getSQLKeywords(DatabaseMetaDataUsingInfoSchema.java:1178) ~[mysql-connector-java.jar:8.0.22] |
| 26 | + at org.datanucleus.store.rdbms.adapter.BaseDatastoreAdapter.<init>(BaseDatastoreAdapter.java:288) ~[datanucleus-rdbms-5.2.10.jar:?] |
| 27 | +``` |
| 28 | +We suggest switching the MySQL driver to the MariaDB if you have seen the same issue. |
| 29 | + |
| 30 | +## Configuration |
| 31 | + |
| 32 | +Before start, make sure MariaDB is accessible, and create a dedicated database and a secure user account that the Hive Metastore will use to connect. |
| 33 | +Download the MariaDB JDBC Connector JAR (e.g., mariadb-java-client-x.x.x.jar) and place it in the Hive `$HIVE_HOME/lib` directory. |
| 34 | + |
| 35 | +For the first time to start the Hive Metastore service, the database must be initialized with the Hive schema. You can use the `schematool` utility provided with the Hive installation. |
| 36 | +Run this command from the Hive installation directory (`$HIVE_HOME/bin`): |
| 37 | + |
| 38 | +```Bash |
| 39 | + |
| 40 | +schematool -driver org.mariadb.jdbc.Driver -dbType mysql -initOrUpgradeSchema -url "jdbc:mariadb://<MariaDB_Host>:<Port>/${db}" -userName hive_user -passWord your_strong_password |
| 41 | +``` |
| 42 | + |
| 43 | +The main configuration file for Metastore is `$HIVE_HOME/conf/hive-site.xml`, this is where you tell the Metastore service (HMS) how to connect to your MariaDB database. |
| 44 | +The following XML properties must be added or updated in `hive-site.xml`: |
| 45 | + |
| 46 | +```xml |
| 47 | +<configuration> |
| 48 | + <property> |
| 49 | + <name>javax.jdo.option.ConnectionURL</name> |
| 50 | + <value>jdbc:mariadb://<MariaDB_Host>:<Port>/${db}?createDatabaseIfNotExist=false&useSSL=false</value> |
| 51 | + <description>JDBC connection URL for the Metastore database.</description> |
| 52 | + </property> |
| 53 | + |
| 54 | + <property> |
| 55 | + <name>javax.jdo.option.ConnectionDriverName</name> |
| 56 | + <value>org.mariadb.jdbc.Driver</value> |
| 57 | + <description>The JDBC driver class name for MariaDB.</description> |
| 58 | + </property> |
| 59 | + |
| 60 | + <property> |
| 61 | + <name>javax.jdo.option.ConnectionUserName</name> |
| 62 | + <value>hive_user</value> |
| 63 | + <description>Username for the Metastore database connection.</description> |
| 64 | + </property> |
| 65 | + |
| 66 | + <property> |
| 67 | + <name>javax.jdo.option.ConnectionPassword</name> |
| 68 | + <value>your_strong_password</value> |
| 69 | + <description>Password for the Metastore database connection.</description> |
| 70 | + </property> |
| 71 | +</configuration> |
| 72 | +``` |
| 73 | + |
| 74 | +**Important:** |
| 75 | +- Replace `<MariaDB_Host>:<Port>` with the correct hostname/IP addresses. |
| 76 | +- The JDBC URL, `&` must be escaped as `&` in XML. |
| 77 | + |
| 78 | +After all configurations are complete, you can start the Hive Metastore service. |
| 79 | +```bash |
| 80 | +hive --service metastore |
| 81 | +``` |
| 82 | + |
| 83 | +Want a quick play on the new driver? try our [Docker image](../setting-up-hive-with-docker): |
| 84 | + |
| 85 | + |
| 86 | + |
| 87 | + |
0 commit comments