Skip to content

Commit 7ad1448

Browse files
committed
Support for Sparse vector and documentation changes
1 parent a99d1ae commit 7ad1448

19 files changed

+731
-115
lines changed

doc/src/release_notes.rst

+2
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@ node-oracledb `v6.8.0 <https://github.com/oracle/node-oracledb/compare/v6.7.1...
1414
Common Changes
1515
++++++++++++++
1616

17+
#) Added support for Oracle Database 23ai sparse vectors.
18+
1719
#) Fixed :attr:`~dbObject.length` property for the database object
1820
collection types, which was broken from node-oracledb 6.0.
1921

doc/src/user_guide/initialization.rst

+36-31
Original file line numberDiff line numberDiff line change
@@ -500,51 +500,56 @@ explicitly specified or a default location will be used. Do one of:
500500
501501
.. _environmentvariables:
502502
503-
Oracle Environment Variables for node-oracledb Thick Mode
504-
=========================================================
505-
506-
Some common environment variables that influence node-oracledb in Thick mode
507-
are shown below. The variables that may be needed depend on how Node.js is
508-
installed, how you connect to the database, and what optional settings are
509-
desired. It is recommended to set Oracle variables in the environment before
510-
invoking Node.js, however they may also be set in application code as long as
511-
they are set before node-oracledb is first used. System environment variables
512-
like ``LD_LIBRARY_PATH`` must be set before Node.js starts.
513-
514-
.. note::
515-
516-
The variables listed below are only supported in the node-oracledb Thick
517-
mode, with the exception of the ``TNS_ADMIN`` and ``ORA_SDTZ`` which
518-
are also supported in the node-oracledb Thin mode.
519-
520-
.. list-table-with-summary:: Common Oracle Environment Variables
503+
Oracle Environment Variables for node-oracledb
504+
==============================================
505+
506+
Some common environment variables that influence node-oracledb are shown
507+
below. The variables that may be needed depend on how Node.js is installed,
508+
how you connect to the database, and what optional settings are desired. It is
509+
recommended to set Oracle variables in the environment before invoking
510+
Node.js, however they may also be set in application code as long as they are
511+
set before node-oracledb is first used. System environment variables like
512+
``LD_LIBRARY_PATH`` must be set before Node.js starts.
513+
514+
.. list-table-with-summary:: Common Oracle Environment Variables supported by node-oracledb
521515
:header-rows: 1
522516
:class: wy-table-responsive
523517
:align: center
524-
:widths: 20 30
525-
:summary: The first column displays the common Oracle Environment Variable. The second column, Purpose, describes what the environment variable is used for.
518+
:widths: 20 40 10
519+
:name: _oracle_environment_variables
520+
:summary: The first column displays the common Oracle Environment Variable. The second column, Purpose, describes what the environment variable is used for. The third column displays whether the environment variable can be used in the node-oracledb Thin mode, Thick mode, or both modes.
526521
527522
* - Oracle Environment Variables
528523
- Purpose
524+
- Node-oracledb Mode
529525
* - ``LD_LIBRARY_PATH``
530526
- The library search path for Linux and some UNIX platforms. Set this to the directory containing the Oracle Client libraries, for example ``/opt/oracle/instantclient_23_5`` or ``$ORACLE_HOME/lib``. The variable needs to be set in the environment before Node.js is invoked. The variable is not needed if the libraries are located by an alternative method, such as from running ``ldconfig``. On some UNIX platforms, an OS specific equivalent such as ``LIBPATH`` or ``SHLIB_PATH`` is used instead of ``LD_LIBRARY_PATH``.
531-
* - ``PATH``
532-
- The library search path for Windows should include the location where ``OCI.DLL`` is found. Not needed if you pass :ref:`libDir <odbinitoracleclientattrsopts>` when calling :meth:`oracledb.initOracleClient()`.
533-
* - ``TNS_ADMIN``
534-
- The location of the optional :ref:`Oracle Net configuration files <tnsadmin>` and :ref:`Oracle Client configuration files <oraaccess>`, including ``tnsnames.ora``, ``sqlnet.ora``, and ``oraaccess.xml``, if they are not in a default location. The :ref:`configDir <odbinitoracleclientattrsopts>` value in a call to :meth:`oracledb.initOracleClient()` overrides ``TNS_ADMIN``.
527+
- Thick
528+
* - ``NLS_DATE_FORMAT``, ``NLS_TIMESTAMP_FORMAT``
529+
- See :ref:`Fetching Numbers and Dates as String <fetchasstringhandling>`. The variables are ignored if ``NLS_LANG`` is not set.
530+
- Thick
531+
* - ``NLS_LANG``
532+
- Determines the ‘national language support’ globalization options for node-oracledb. If not set, a default value will be chosen by Oracle.
533+
Note that node-oracledb will always uses the AL32UTF8 character set. See :ref:`Globalization and National Language Support (NLS) <nls>`.
534+
- Thick
535+
* - ``NLS_NUMERIC_CHARACTERS``
536+
- See :ref:`Fetching Numbers and Dates as String <fetchasstringhandling>`. The variables are ignored if ``NLS_LANG`` is not set.
537+
- Thick
535538
* - ``ORA_SDTZ``
536539
- The default session time zone, see :ref:`Fetching Dates and Timestamps <datehandling>`.
540+
- Both
537541
* - ``ORA_TZFILE``
538542
- The name of the Oracle time zone file to use. See :ref:`oratzfile`.
543+
- Thick
539544
* - ``ORACLE_HOME``
540545
- The directory containing the Oracle Database software. This directory must be accessible by the Node.js process. This variable should *not* be set if node-oracledb uses Oracle Instant Client.
541-
* - ``NLS_LANG``
542-
- Determines the ‘national language support’ globalization options for node-oracledb. If not set, a default value will be chosen by Oracle.
543-
Note that node-oracledb will always uses the AL32UTF8 character set. See :ref:`Globalization and National Language Support (NLS) <nls>`.
544-
* - ``NLS_DATE_FORMAT``, ``NLS_TIMESTAMP_FORMAT``
545-
- See :ref:`Fetching Numbers and Dates as String <fetchasstringhandling>`. The variables are ignored if ``NLS_LANG`` is not set.
546-
* - ``NLS_NUMERIC_CHARACTERS``
547-
- See :ref:`Fetching Numbers and Dates as String <fetchasstringhandling>`. The variables are ignored if ``NLS_LANG`` is not set.
546+
- Thick
547+
* - ``PATH``
548+
- The library search path for Windows should include the location where ``OCI.DLL`` is found. Not needed if you pass :ref:`libDir <odbinitoracleclientattrsopts>` when calling :meth:`oracledb.initOracleClient()`.
549+
- Thick
550+
* - ``TNS_ADMIN``
551+
- The location of the optional :ref:`Oracle Net configuration files <tnsadmin>` and :ref:`Oracle Client configuration files <oraaccess>`, including ``tnsnames.ora``, ``sqlnet.ora``, and ``oraaccess.xml``, if they are not in a default location. The :ref:`configDir <odbinitoracleclientattrsopts>` value in a call to :meth:`oracledb.initOracleClient()` overrides ``TNS_ADMIN``.
552+
- Both
548553
549554
Scripts for Setting the Default Environment in a Database Installation
550555
----------------------------------------------------------------------

examples/vectorSparse.js

+186
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
/* Copyright (c) 2025, Oracle and/or its affiliates. */
2+
3+
/******************************************************************************
4+
*
5+
* This software is dual-licensed to you under the Universal Permissive License
6+
* (UPL) 1.0 as shown at https://oss.oracle.com/licenses/upl and Apache License
7+
* 2.0 as shown at http://www.apache.org/licenses/LICENSE-2.0. You may choose
8+
* either license.
9+
*
10+
* If you elect to accept the software under the Apache License, Version 2.0,
11+
* the following applies:
12+
*
13+
* Licensed under the Apache License, Version 2.0 (the "License");
14+
* you may not use this file except in compliance with the License.
15+
* You may obtain a copy of the License at
16+
*
17+
* https://www.apache.org/licenses/LICENSE-2.0
18+
*
19+
* Unless required by applicable law or agreed to in writing, software
20+
* distributed under the License is distributed on an "AS IS" BASIS,
21+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
22+
* See the License for the specific language governing permissions and
23+
* limitations under the License.
24+
*
25+
* NAME
26+
* vectortypesparse.js
27+
*
28+
* DESCRIPTION
29+
* Insert and query SPARSE VECTOR columns.
30+
*
31+
*
32+
*****************************************************************************/
33+
34+
'use strict';
35+
36+
Error.stackTraceLimit = 50;
37+
38+
const oracledb = require('oracledb');
39+
const assert = require('assert');
40+
const dbConfig = require('./dbconfig.js');
41+
const tableName = 'testvectorsparse';
42+
43+
if (process.env.NODE_ORACLEDB_DRIVER_MODE === 'thick') {
44+
let clientOpts = {};
45+
// On Windows and macOS Intel platforms, set the environment
46+
// variable NODE_ORACLEDB_CLIENT_LIB_DIR to the Oracle Client library path
47+
if (process.platform === 'win32' || (process.platform === 'darwin' && process.arch === 'x64')) {
48+
clientOpts = { libDir: process.env.NODE_ORACLEDB_CLIENT_LIB_DIR };
49+
}
50+
oracledb.initOracleClient(clientOpts); // enable node-oracledb Thick mode
51+
}
52+
53+
oracledb.outFormat = oracledb.OUT_FORMAT_OBJECT;
54+
55+
async function run() {
56+
57+
const connection = await oracledb.getConnection(dbConfig);
58+
59+
try {
60+
let result;
61+
const serverVersion = connection.oracleServerVersion;
62+
if (serverVersion < 2306000000) {
63+
console.log(`DB version ${serverVersion} does not support VECTOR type`);
64+
return;
65+
}
66+
67+
console.log('Creating table');
68+
await connection.execute(`DROP TABLE if exists ${tableName}`);
69+
await connection.execute(`CREATE TABLE ${tableName} (id NUMBER GENERATED ALWAYS AS IDENTITY,
70+
sparseF64 VECTOR(4, float64, SPARSE), sparseFlexF64 VECTOR(*, float64, SPARSE),
71+
denseF64 VECTOR(2, float64), denseFlexF64 VECTOR(*, float64))`);
72+
73+
const arr = [39, -65];
74+
const queryVector = new Float64Array([39, -65]);
75+
const float64arr1 = new Float64Array(arr);
76+
const float64arr2 = new Float64Array([-34, 23]);
77+
const float64arr3 = new Float64Array([-34, 23, 32, 12]);
78+
const sparseString = '[4, [1, 3], [39, -65]]'; // totalDims, indexArray, valueArray.
79+
let sparsevec = new oracledb.SparseVector({ values: float64arr1, indices: [1, 3], numDimensions: 4 });
80+
81+
const binds = {
82+
sparse: { type: oracledb.DB_TYPE_VECTOR, val: sparsevec },
83+
dense: { type: oracledb.DB_TYPE_VECTOR, val: float64arr2 }
84+
};
85+
86+
const denseArray = sparsevec.dense();
87+
console.log(' dense vector ', denseArray);
88+
89+
console.log('Inserting SparseVector instance created from POJO');
90+
result = await connection.execute(`insert into ${tableName} values(DEFAULT, :1, :2, :3, :4)`,
91+
[
92+
sparsevec,
93+
sparsevec,
94+
float64arr1,
95+
float64arr1
96+
]);
97+
98+
console.log('Inserting string data of sparse format');
99+
result = await connection.execute(`insert into ${tableName} values(DEFAULT, :sparse, :sparse, :dense, :dense)`,
100+
[sparseString, sparseString, float64arr1, float64arr1]);
101+
102+
console.log('Inserting SparseVector instance created from string');
103+
sparsevec = new oracledb.SparseVector(sparseString);
104+
result = await connection.execute(`insert into ${tableName} values(DEFAULT, :1, :2, :3, :4)`,
105+
[
106+
sparsevec,
107+
sparsevec,
108+
float64arr1,
109+
float64arr1
110+
]);
111+
112+
console.log('Inserting SparseVector instance created from dense Array');
113+
sparsevec = new oracledb.SparseVector(denseArray);
114+
result = await connection.execute(`insert into ${tableName} values(DEFAULT, :1, :2, :3, :4)`,
115+
[
116+
sparsevec,
117+
sparsevec,
118+
float64arr1,
119+
float64arr1
120+
]);
121+
122+
console.log('Inserting Dense vector into Sparse Flex dimensions column');
123+
let sql = `insert into ${tableName} values(DEFAULT, :sparse, :dense, :dense, :dense)`;
124+
result = await connection.execute(sql, binds);
125+
126+
console.log('Inserting Sparse vector into Dense Flex dimensions column');
127+
sql = `insert into ${tableName} values(DEFAULT, :sparse, :sparse, :dense, :sparse)`;
128+
result = await connection.execute(sql, binds);
129+
130+
console.log('Query Results:');
131+
result = await connection.execute(
132+
`select * from ${tableName} ORDER BY id`);
133+
console.log("Query metadata:", result.metaData);
134+
for (const val of result.rows) {
135+
console.log("Query rows:", JSON.stringify(val));
136+
}
137+
138+
// Inserting Dense vector of different dimensions into Sparse Fixed dimensions column
139+
sql = `insert into ${tableName} values(DEFAULT, :dense, :sparse, :dense, :dense)`;
140+
await assert.rejects(
141+
async () => await connection.execute(sql,
142+
{
143+
sparse: { type: oracledb.DB_TYPE_VECTOR, val: sparsevec },
144+
dense: { type: oracledb.DB_TYPE_VECTOR, val: float64arr2 }
145+
}
146+
),
147+
/ORA-51803:/
148+
);
149+
150+
// Inserting Dense vector of same dimensions into Sparse Fixed dimensions column
151+
sql = `insert into ${tableName} values(DEFAULT, :dense, :sparse, :dense, :dense)`;
152+
await assert.rejects(
153+
async () => await connection.execute(sql,
154+
{
155+
sparse: { type: oracledb.DB_TYPE_VECTOR, val: sparsevec },
156+
dense: { type: oracledb.DB_TYPE_VECTOR, val: float64arr3 }
157+
}
158+
),
159+
/ORA-51803:/
160+
);
161+
162+
// Inserting Sparse vector into Dense Fixed dimensions column
163+
sql = `insert into ${tableName} values(DEFAULT, :sparse, :sparse, :sparse, :dense)`;
164+
await assert.rejects(
165+
async () => await connection.execute(sql, binds),
166+
/ORA-51803:/
167+
);
168+
169+
const sparseQueryVec = new oracledb.SparseVector({ values: queryVector, indices: [2, 4], numDimensions: 4 });
170+
console.log('vector distance with Query ', queryVector);
171+
console.log(await connection.execute(`select vector_distance (sparseF64, :1) from ${tableName}`, [sparseQueryVec]));
172+
173+
} catch (err) {
174+
console.error(err);
175+
} finally {
176+
if (connection) {
177+
try {
178+
await connection.close();
179+
} catch (err) {
180+
console.error(err);
181+
}
182+
}
183+
}
184+
}
185+
186+
run();

lib/errors.js

+28
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,13 @@ const ERR_INVALID_BRANCH_SIZE = 154;
164164
const ERR_OPERATION_NOT_SUPPORTED_ON_BFILE = 155;
165165
const ERR_OPERATION_ONLY_SUPPORTED_ON_BFILE = 156;
166166
const ERR_EXECMANY_NOT_ALLOWED_ON_QUERIES = 157;
167+
const ERR_VECTOR_SPARSE_INDICES_IS_NOT_ARRAY = 158;
168+
const ERR_VECTOR_SPARSE_VALUES_IS_NOT_ARRAY = 159;
169+
const ERR_VECTOR_SPARSE_DIMS_IS_NOT_INTEGER = 160;
170+
const ERR_VECTOR_SPARSE_INDICES_VALUES_NOT_EQUAL = 161;
171+
const ERR_VECTOR_SPARSE_INVALID_JSON = 162;
172+
const ERR_VECTOR_SPARSE_INVALID_STRING = 163;
173+
const ERR_VECTOR_SPARSE_INVALID_INPUT = 164;
167174

168175
// Oracle Net layer errors start from 500
169176
const ERR_CONNECTION_CLOSED = 500;
@@ -467,6 +474,20 @@ messages.set(ERR_OPERATION_ONLY_SUPPORTED_ON_BFILE, // NJS-156
467474
'operation is only supported on BFILE LOBs');
468475
messages.set(ERR_EXECMANY_NOT_ALLOWED_ON_QUERIES, // NJS-157
469476
'executeMany() cannot be used with SELECT statement or WITH SQL clause');
477+
messages.set(ERR_VECTOR_SPARSE_INDICES_IS_NOT_ARRAY, // NJS-158
478+
'SPARSE VECTOR indices is not an Array');
479+
messages.set(ERR_VECTOR_SPARSE_VALUES_IS_NOT_ARRAY, // NJS-159
480+
'SPARSE VECTOR values is not an Array');
481+
messages.set(ERR_VECTOR_SPARSE_DIMS_IS_NOT_INTEGER, // NJS-160
482+
'SPARSE VECTOR dimensions is not an Positive Integer');
483+
messages.set(ERR_VECTOR_SPARSE_INDICES_VALUES_NOT_EQUAL, // NJS-161
484+
'SPARSE VECTOR indices and values must be of same length');
485+
messages.set(ERR_VECTOR_SPARSE_INVALID_JSON, // NJS-162
486+
'SPARSE VECTOR string data is not valid JSON');
487+
messages.set(ERR_VECTOR_SPARSE_INVALID_STRING, // NJS-163
488+
'SPARSE VECTOR string data Array should have exactly 3 elements');
489+
messages.set(ERR_VECTOR_SPARSE_INVALID_INPUT, // NJS-164
490+
'SPARSE VECTOR Invalid Input Data');
470491

471492
// Oracle Net layer errors
472493

@@ -909,6 +930,13 @@ module.exports = {
909930
ERR_OPERATION_NOT_SUPPORTED_ON_BFILE,
910931
ERR_OPERATION_ONLY_SUPPORTED_ON_BFILE,
911932
ERR_EXECMANY_NOT_ALLOWED_ON_QUERIES,
933+
ERR_VECTOR_SPARSE_INDICES_IS_NOT_ARRAY,
934+
ERR_VECTOR_SPARSE_VALUES_IS_NOT_ARRAY,
935+
ERR_VECTOR_SPARSE_DIMS_IS_NOT_INTEGER,
936+
ERR_VECTOR_SPARSE_INDICES_VALUES_NOT_EQUAL,
937+
ERR_VECTOR_SPARSE_INVALID_JSON,
938+
ERR_VECTOR_SPARSE_INVALID_STRING,
939+
ERR_VECTOR_SPARSE_INVALID_INPUT,
912940
WRN_COMPILATION_CREATE,
913941
assert,
914942
assertArgCount,

lib/impl/datahandlers/constants.js

+2
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,12 @@ module.exports = {
3434
TNS_VECTOR_MAGIC_BYTE: 0xDB,
3535
TNS_VECTOR_VERSION_BASE: 0,
3636
TNS_VECTOR_VERSION_WITH_BINARY: 1,
37+
TNS_VECTOR_VERSION_WITH_SPARSE: 2,
3738

3839
// vector flags
3940
TNS_VECTOR_FLAG_NORMSRC: 0x0010,
4041
TNS_VECTOR_FLAG_NORM: 0x0002,
42+
TNS_VECTOR_FLAG_SPARSE: 0x0020,
4143

4244
// base JSON constants
4345
TNS_JSON_MAGIC_BYTE_1: 0xff,

0 commit comments

Comments
 (0)