Add AS OF SYSTEM TIME support for CockroachDB temporal queries

[mackinleysmith]

CockroachDB AS OF SYSTEM TIME Support

This pull request implements support for CockroachDB's "AS OF SYSTEM TIME" feature in GORM.

Docs can be found on Cockroach's website at: https://www.cockroachlabs.com/docs/stable/as-of-system-time.

Overview

The "AS OF SYSTEM TIME" feature allows you to query data as it existed at a specific point in time, which is useful for:

• Point-in-time recovery
• Temporal queries
• Auditing historical data
• Debugging data issues

Usage

Basic Usage

// Query data as it existed 1 hour ago
db.AsOfSystemTime("-1h").Find(&users)

// Query data as it existed at a specific timestamp
db.AsOfSystemTime(time.Date(2023, 1, 1, 12, 0, 0, 0, time.UTC)).Find(&users)

// Query data as it existed 30 minutes ago
db.AsOfSystemTime("-30m").Find(&users)

Chaining with Other Methods

// Combine with WHERE clauses
db.Model(&User{}).
   AsOfSystemTime("-1h").
   Where("name = ?", "john").
   Find(&users)

// Combine with SELECT
db.Model(&User{}).
   AsOfSystemTime("-2h").
   Select("id", "name", "email").
   Find(&users)

// Combine with JOINs
db.Model(&User{}).
   AsOfSystemTime("-1h").
   Joins("Company").
   Find(&users)

Supported Time Formats

The method accepts two types of input:

1. Time strings (raw SQL expressions):

   db.AsOfSystemTime("-1h")           // 1 hour ago
   db.AsOfSystemTime("-30m")          // 30 minutes ago
   db.AsOfSystemTime("now() - interval '1 day'")  // 1 day ago

2. Go time.Time values:

   db.AsOfSystemTime(time.Date(2023, 1, 1, 12, 0, 0, 0, time.UTC))
   db.AsOfSystemTime(time.Now().Add(-time.Hour))

Database Support

This feature is only supported by CockroachDB. If you try to use it with other databases, GORM will return an error.

Generated SQL

The feature generates SQL like this:

-- For time strings
SELECT * FROM users AS OF SYSTEM TIME -1h

-- For timestamps
SELECT * FROM users AS OF SYSTEM TIME '2023-01-01 12:00:00+00'

-- With other clauses
SELECT id, name FROM users AS OF SYSTEM TIME -1h WHERE name = 'john'

-- With JOINs (AS OF SYSTEM TIME appears after JOINs)
SELECT * FROM users
INNER JOIN companies ON companies.id = users.company_id
AS OF SYSTEM TIME -1h

Important: The "AS OF SYSTEM TIME" clause is always positioned at the end of the FROM clause, after any JOINs. This is the correct SQL syntax for CockroachDB.

Error Handling

The method will return an error if:

• The database dialect is not CockroachDB
• An unsupported timestamp type is provided

// This will return an error with non-CockroachDB databases
err := db.AsOfSystemTime("-1h").Find(&users).Error
if err != nil {
    log.Printf("Error: %v", err)
}

Implementation Details

The feature is implemented through:

1. A new AsOfSystemTime clause type in the clause package
2. Integration with the existing FROM clause
3. A new AsOfSystemTime() method in the chainable API
4. Proper dialect checking to ensure CockroachDB compatibility

Examples

Complete Example

package main

import (
    "log"
    "time"

    "gorm.io/gorm"
    "gorm.io/driver/postgres" // CockroachDB driver
)

type User struct {
    ID        uint      `gorm:"primarykey"`
    Name      string
    Email     string
    CreatedAt time.Time
    UpdatedAt time.Time
}

func main() {
    // Connect to CockroachDB
    db, err := gorm.Open(postgres.Open("your-connection-string"), &gorm.Config{})
    if err != nil {
        log.Fatal(err)
    }

    var users []User

    // Query users as they existed 1 hour ago
    err = db.AsOfSystemTime("-1h").Find(&users).Error
    if err != nil {
        log.Printf("Error querying historical data: %v", err)
        return
    }

    log.Printf("Found %d users from 1 hour ago", len(users))

    // Query users as they existed at a specific time
    historicalTime := time.Date(2023, 1, 1, 12, 0, 0, 0, time.UTC)
    err = db.AsOfSystemTime(historicalTime).Find(&users).Error
    if err != nil {
        log.Printf("Error querying historical data: %v", err)
        return
    }

    log.Printf("Found %d users from %v", len(users), historicalTime)
}

Notes

• The feature requires CockroachDB 20.1 or later
• Performance may vary depending on the amount of historical data
• The feature works with all GORM query methods (Find, First, Take, etc.)
• It can be combined with all other GORM features (WHERE, JOIN, SELECT, etc.)
• The "AS OF SYSTEM TIME" clause is always positioned after JOINs in the generated SQL

[propel-code-bot]

Add CockroachDB AS OF SYSTEM TIME Temporal Query Support in GORM

This PR introduces explicit support for CockroachDB's AS OF SYSTEM TIME temporal query syntax into GORM. It provides a chainable API method (AsOfSystemTime) that lets users query historical data as it existed at a specific point or interval, and updates the SQL clause builder logic so that the AS OF SYSTEM TIME clause is appended correctly after tables and joins. The implementation handles both raw SQL time expressions and strongly-typed time.Time objects, and is integrated with GORM's FROM clause. There is extensive testing to verify the correct SQL output for various scenarios.

Key Changes

• Added new file clause/as_of_system_time.go defining the AsOfSystemTime clause with logic for building the SQL fragment and merging clauses.
• Modified clause/from.go to embed an optional AsOfSystemTime pointer and render the clause after joins if set.
• Added the DB.AsOfSystemTime(timestamp interface{}) and DB.AsOfSystemTimeNow() chainable methods in chainable_api.go for API-level user access.
• Created and expanded clause/as_of_system_time_test.go to cover clause construction, SQL output, name interface conformance, and clause merging behaviors.
• Sanitized the Raw SQL input for AS OF SYSTEM TIME by removing semicolons to mitigate SQL injection risks.
• Extended documentation and code comments for the new methods and clause fields.

Affected Areas

• chainable_api.go (public API changes, new methods)
• clause/as_of_system_time.go (new clause, SQL generation)
• clause/from.go (FROM clause struct and SQL builder)
• clause/as_of_system_time_test.go (tests for clause behavior)

This summary was automatically generated by @propel-code-bot

[mackinleysmith]

@jinzhu who should i speak with about getting this merged in? Is there anything I can do to be helpful in that process?

[jinzhu]

Hi @mackinleysmith,

Thank you for your PR!

I believe a better approach would be to add an AsOfSystemTime method in the CockroachDB driver. This method could return a clause.Builder, similar to how clause.From is implemented. The driver would then be responsible for incorporating this clause into the final SQL during statement building.

You can refer to this implementation as an example: https://github.com/go-gorm/postgres/blob/c65a34eaf8229cd55f5a0febaf69c30250be3c2a/postgres.go#L85

Let me know what you think!

[mackinleysmith]

@jinzhu as far as I understand, CockroachDB does not have its own driver for Gorm. It seems the recommended approach is to use the postgres driver, since Cockroach implements the postgres dialect. Please let me know if I'm missing something -- I'd be glad to take a stab at it if I am.

[jinzhu]

I think it would be more reasonable to create a dedicated GORM CockroachDB driver based on the PostgreSQL driver.
This way, we can encapsulate the complexity inside the driver and better support CockroachDB-specific features in the future.