Skip to content

OffsetPaginatorConfig should accept offset_body_path and limit_body_path #3255

@kinghuang

Description

@kinghuang

Feature description

The OffsetPaginator has parameters for setting the offset and limit in the request JSON body (offset_body_path and limit_body_path) in addition to setting them in query parameters (offset_param and limit_param). But, it's not possible to declaratively set these parameters in a RESTAPIConfig because OffsetPaginatorConfig doesn't include offset_body_path and limit_body_path.

Are you a dlt user?

Yes, I'm already a dlt user.

Use case

I want to declare and configure an offset paginator that uses offset_body_path and limit_body_path as a dict.

"resource_defaults": {
    "endpoint": {
        "paginator": {
            "type": "offset",
            "limit": 1000,
            "offset": 0,
            "offset_body_path": "skip",
            "limit_body_path": "top",
            "total_path": None,
            "stop_after_empty_page": True,
            "maximum_offset": 50000,
        },
    },
},

Proposed solution

Add offset_body_path and limit_body_path as optional fields on OffsetPaginatorConfig.

Related issues

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions