Skip to content

Update transaction and query options specifications for GRPC and HTTP #931

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 28, 2025
Merged

Update transaction and query options specifications for GRPC and HTTP #931

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
92e7c1d
Update HTTP's answerCountLimit behavior for write queries
farost Apr 17, 2025
545642b
Add transaction and query options descriptions
farost Apr 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 21 additions & 11 deletions drivers/modules/ROOT/pages/http/api-reference.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -748,8 +748,18 @@ Open a new transaction and receive a unique transaction id.
[cols="h,3a"]
|===
| Field | Description
| `schemaLockAcquireTimeoutMillis` | Timeout for a schema transaction to acquire the exclusive schema lock of the database. Can be used to wait until a previous schema transaction finishes and releases the exclusivity lock. Specified in milliseconds. **Default:** 10 seconds.
| `transactionTimeoutMillis` | The maximum amount of time a transaction can stay opened. It will be closed automatically without preserving its changes and finishing its active queries after this timeout. Specified in milliseconds. **Default:** 5 minutes.
| `transactionTimeoutMillis` |
include::{page-version}@manual:resources:partial$options-descriptions/transaction.adoc[tag=transaction-timeout]

Specified in milliseconds. **Default:**
include::{page-version}@manual:resources:partial$options-descriptions/transaction.adoc[tag=transaction-timeout-default]


| `schemaLockAcquireTimeoutMillis` |
include::{page-version}@manual:resources:partial$options-descriptions/transaction.adoc[tag=schema-lock-acquire-timeout]

Specified in milliseconds. **Default:**
include::{page-version}@manual:resources:partial$options-descriptions/transaction.adoc[tag=schema-lock-acquire-timeout-default]
|===
// end::transaction-options[]

Expand Down Expand Up @@ -1043,19 +1053,19 @@ This endpoint allows running multiple sequential queries before committing.
[cols="h,3a"]
|===
| Field | Description
| `includeInstanceTypes` | An optimization flag: whether to include the types of the returned instance concepts in concept row responses or not. **Default:** true.
| `answerCountLimit` | The maximum allowed size of concept rows or concept documents answers returned. Used to limit the network load. **Default:** 10 000.
| `includeInstanceTypes` |
include::{page-version}@manual:resources:partial$options-descriptions/query.adoc[tag=include-instance-types]

If used in a read query, at most `answerCountLimit` answers will be returned. If there are more answers cut, a relevant `warning` will be provided in the response.
**Default:**
include::{page-version}@manual:resources:partial$options-descriptions/query.adoc[tag=include-instance-types-default]

If used in a write query and is hit, the operation will fail completely.

[NOTE]
====
This behaviour will be changed in the next stable version of TypeDB:
| `answerCountLimit` |
include::{page-version}@manual:resources:partial$options-descriptions/query.adoc[tag=answer-count-limit]
If there are more answers cut, a relevant `warning` will be provided in the response.

If used in a write query, at most `answerCountLimit` answers will be returned. All changes, including both returned and not returned, will be applied.
====
**Default:**
include::{page-version}@manual:resources:partial$options-descriptions/query.adoc[tag=answer-count-limit-default]

|===
// end::query-options[]
Expand Down
52 changes: 33 additions & 19 deletions drivers/modules/ROOT/pages/http/index.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ The TypeDB HTTP endpoint can be used to perform database management, user manage

== Server configuration

TypeDB server by runs both a gRPC (standard for most of the client applications) and an HTTP endpoint by default. Both endpoints have common authentication and xref:{page-version}@manual::configure/encryption.adoc[encryption settings]. Additionally, it is possible to disable the HTTP endpoint and change its default port.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are many formatting updates. I will highlight what I actually changed

TypeDB server by runs both a gRPC (standard for most of the client applications) and an HTTP endpoint by default.
Both endpoints have common authentication and xref:{page-version}@manual::configure/encryption.adoc[encryption settings].
Additionally, it is possible to disable the HTTP endpoint and change its default port.

[NOTE]
====
Expand All @@ -18,13 +20,16 @@ Please visit xref:{page-version}@manual::configure/server.adoc#_command_line_arg
[#_authentication]
=== Authentication

Mostly all methods require Token-based authorization. These temporary tokens can be generated through a xref:{page-version}@drivers::http/api-reference.adoc#_sign_in[`POST` request to `localhost:8000/v1/signin`]. Provide your TypeDB user credentials in the request's body.
Mostly all methods require Token-based authorization.
These temporary tokens can be generated through a xref:{page-version}@drivers::http/api-reference.adoc#_sign_in[`POST` request to `localhost:8000/v1/signin`].
Provide your TypeDB user credentials in the request's body.

Change the xref:{page-version}@manual::configure/server.adoc#_command_line_arguments[`server.authentication.token_ttl_seconds`] parameter in the server's configuration to modify the time the tokens will remain valid before a new sign in request is required.

=== Encryption

If xref:{page-version}@manual::configure/encryption.adoc[server encryption] is set up, its settings are also applied to the HTTP endpoint, and the endpoint access will be additionally protected. Visit the xref:{page-version}@manual::configure/server.adoc#_encryption[server configuration] page for more details.
If xref:{page-version}@manual::configure/encryption.adoc[server encryption] is set up, its settings are also applied to the HTTP endpoint, and the endpoint access will be additionally protected.
Visit the xref:{page-version}@manual::configure/server.adoc#_encryption[server configuration] page for more details.

=== CORS

Expand All @@ -39,14 +44,18 @@ See xref:{page-version}@drivers::http/api-reference.adoc[] to access available d

=== Understanding query answers

The TypeDB HTTP endpoint supports all types of xref:{page-version}@manual::queries/transactions.adoc[transactions] and xref:{page-version}@manual::queries/index.adoc[queries]. The common xref:{page-version}@manual::queries/answers.adoc[query answer type], a list of concept rows, is represented in the HTTP endpoint like the following document:
The TypeDB HTTP endpoint supports all types of xref:{page-version}@manual::queries/transactions.adoc[transactions] and xref:{page-version}@manual::queries/index.adoc[queries].
The common xref:{page-version}@manual::queries/answers.adoc[query answer type], a list of concept rows, is represented in the HTTP endpoint like the following document:

.Concept rows answer
[%collapsible]
====
Each successful query response contains a query type, an answer type, an optional warning message, and the list of answers containing concepts (if applicable).

The `answers` field contains a list of documents with variables as keys and concepts as values. Each concept contains a `kind` field for unambiguous parsing. Each type has a label, and each instance has an optional description of its type, if the specific query option (`includeInstanceTypes`) is enabled. See all the possible concept variants below.
The `answers` field contains a list of documents with variables as keys and concepts as values.
Each concept contains a `kind` field for unambiguous parsing.
Each type has a label, and each instance has an optional description of its type, if the specific query option (`includeInstanceTypes`) is enabled.
See all the possible concept variants below.

include::{page-version}@drivers:resources:partial$http-api/responses/query-concept-rows-example.json.adoc[]
====
Expand All @@ -57,7 +66,8 @@ The result above can be achieved in two different ways.

This will except you to manually open, close, and commit transactions used for querying.

Open a transaction using a `POST` request `/v1/transactions/open`. Provide an authorization token in the header (see xref:{page-version}@drivers::http/index.adoc#_authentication[authentication] above) and a JSON body, containing information about the target database and required transaction type:
Open a transaction using a `POST` request `/v1/transactions/open`.
Provide an authorization token in the header (see xref:{page-version}@drivers::http/index.adoc#_authentication[authentication] above) and a JSON body, containing information about the target database and required transaction type:

[source,json]
----
Expand All @@ -69,7 +79,9 @@ Open a transaction using a `POST` request `/v1/transactions/open`. Provide an au

[NOTE]
====
Schema transactions have an exclusive lock on the database and prevent other transactions from opening. If you don't close a schema transaction and lose its ID, it will be closed automatically based on the transaction timeout specified in the request (this parameter can be added to the request above):
Schema transactions have an exclusive lock on the database and prevent other transactions from opening.
If you don't close a schema transaction and lose its ID, it will be closed automatically based on the transaction timeout specified in the request (this parameter can be added to the request above):

[source,json]
----
"transactionOptions": {
Expand All @@ -79,6 +91,7 @@ Schema transactions have an exclusive lock on the database and prevent other tra
====

If everything is correct, you will receive a response containing a body like:

[source,json]
----
{
Expand All @@ -87,6 +100,7 @@ If everything is correct, you will receive a response containing a body like:
----

Then, send a `POST` query request to `v1/transactions/e1f8583c-2a03-4aac-a260-ec186369e86f/query` with the same authorization token in the header and the following JSON body included:

[source,json]
----
{
Expand All @@ -101,9 +115,12 @@ Don't forget to close the transaction when the work is done.

=== One-shot query

To avoid manual transaction management, a one-shot query endpoint can be used. It opens and automatically closes or commits a transaction for each query sent.
To avoid manual transaction management, a one-shot query endpoint can be used.
It opens and automatically closes or commits a transaction for each query sent.

Send a single `POST` request to `/v1/query`.
Provide an authorization token in the header (see xref:{page-version}@drivers::http/index.adoc#_authentication[authentication] above) and the following body containing information about the target database, transaction type required, query, and optional options:

Send a single `POST` request to `/v1/query`. Provide an authorization token in the header (see xref:{page-version}@drivers::http/index.adoc#_authentication[authentication] above) and the following body containing information about the target database, transaction type required, query, and optional options:
[source,json]
----
{
Expand All @@ -121,21 +138,18 @@ With this, you don't need to worry about forgotten transactions.

=== Running big queries

The current version of the HTTP endpoint does not support query answer streaming. Unlike in gRPC, the query results will be fully consumed before an initial answer is received on the client side, and the whole list of concept rows or documents will be returned in a single response.
The current version of the HTTP endpoint does not support query answer streaming.
Unlike in gRPC, the query results will be fully consumed before an initial answer is received on the client side, and the whole list of concept rows or documents will be returned in a single response.

While this mechanism will be enhanced in the future, for safety purposes, please use a special query option `answerCountLimit` to limit the amount of produced results and avoid too long query execution. The default value of ten thousand answers can be extended if you are ready for the consequences.
While this mechanism will be enhanced in the future, for safety purposes, please use a special query option `answerCountLimit` to limit the amount of produced results and avoid too long query execution.
The default value of ten thousand answers can be extended if you are ready for the consequences.

If this limit is hit:

- Read queries will return `206 Partial Content` with all the answers
processed;
- Write queries will stop their execution with an error, and the transaction will be closed without preserving intermediate results.

[NOTE]
====
This behaviour will be changed in the next stable version of TypeDB:
- Read queries will return `206 Partial Content` with all the answers processed;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I've changed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

- Write queries will be fully executed, and their changes can be committed.
The results will be returned with `206 Partial Content`.

- Write queries will be fully executed, and their changes can be committed. The results will be returned with the `206 Partial Content` code.
====

For example:
Expand Down
18 changes: 9 additions & 9 deletions manual/modules/ROOT/pages/migration/2_to_3.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
= TypeDB 2.x to 3.0
= TypeDB 2.x to 3.x

TypeDB 3.0 is a major leap forward, offering significantly improved performance and introducing a wide range of changes across almost every aspect of the database.
TypeDB 3.x is a major leap forward, offering significantly improved performance and introducing a wide range of changes across almost every aspect of the database.
This page outlines the key updates you should know, alongside recommendations for server and driver upgrades.

== Migration process
Expand Down Expand Up @@ -36,7 +36,9 @@ Click on a feature below for detailed documentation and examples.

* **Sessions Removed:** Connections to TypeDB now only require opening a `schema`, `write`, or `read` xref:{page-version}@manual::queries/transactions.adoc[transaction].
* **Integer Value Type Renamed:** `long` is now xref:{page-version}@typeql::values/integer.adoc[`integer`].
* **Kinds Introduced:** no more predefined root types and their subtyping through `person sub entity`, `friendship sub relation`, or `name sub attribute`. Each new type should be declared with an explicit kind: `entity person`, `relation friendship`, and `attribute name`. Check out xref:{page-version}@manual::schema/schema_structure.adoc[] for more.
* **Kinds Introduced:** no more predefined root types and their subtyping through `person sub entity`, `friendship sub relation`, or `name sub attribute`.
Each new type should be declared with an explicit kind: `entity person`, `relation friendship`, and `attribute name`.
Check out xref:{page-version}@manual::schema/schema_structure.adoc[] for more.
* **Expanded User Management:** Authorization and user management are now included in all TypeDB editions.
Default credentials, if applicable, can be found in the xref:{page-version}@manual::connect/index.adoc[Connection Manual].
* **Updated Query Syntax:**
Expand Down Expand Up @@ -80,20 +82,18 @@ Take into account that some of your processes can be blocked if you previously r

* Database import and export tools.
* Configuration files (use CLI as a temporary solution).
* Transaction options.
* `update` queries (use `delete` + `insert` as a workaround).
* `contains` and `like` operators for `string` s.
* Instantiation restriction for inherited `owns` and `plays` (used to `as`). A better way is planned to be released in the future (NOTE: you don't need the `as` keyword to <<#_specialize, specialize>> an ownership of an inherited super attribute type with an ownership of its subtype).
* Instantiation restriction for inherited `owns` and `plays` (used to `as`).
A better way is planned to be released in the future (NOTE: you don't need the `as` keyword to <<#_specialize, specialize>> an ownership of an inherited super attribute type with an ownership of its subtype).
* Scaling support for Cloud and Enterprise editions.

=== TypeDB Drivers

* Node.js, C, C++, and C# drivers are not yet available.
Consider using one of xref:{page-version}@drivers::index.adoc[the available languages] or xref:{page-version}@drivers::http/index.adoc[the HTTP Endpoint].

=== TypeDB Studio

* **Graph visualizer** has been disabled due to incoming migration to an xref:{page-version}@manual::tools/code.adoc[IDE plugin]. Please consider using side tools for graph visualization if required.
* **Type browser** is temporarily unavailable and planned to be reimplemented in the xref:{page-version}@manual::tools/code.adoc[IDE plugin].
* **Graph visualizer** and **Type browser** are temporarily unavailable, but will be reimplemented and enhanced in the new TypeDB Studio soon.

[#_having_troubles]
== Having troubles?
Expand Down
Loading