You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: document the RequestQueueClient v4 migration
Expand the `RequestQueueClient` migration table in the v4 upgrading guide with
the full method mapping, the new fetch/handle/reclaim lifecycle, the
`isEmpty` semantics, and the `RequestQueueV1`-as-alias note. List the removed
head/lock types.
Closes#3075.
Copy file name to clipboardExpand all lines: docs/upgrading/upgrading_v4.md
+18-1Lines changed: 18 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -560,11 +560,28 @@ The sub-client interfaces (`DatasetClient`, `KeyValueStoreClient`, `RequestQueue
560
560
561
561
**`RequestQueueClient`:**
562
562
563
+
The request queue client was reduced from 12 methods to 9. The distributed-locking protocol (`listAndLockHead` → `prolongRequestLock` → `deleteRequestLock`) and the queue-head/consistency bookkeeping that used to live in the `RequestQueue` frontend have been removed from the interface; coordinating multiple clients accessing the same queue (e.g. request locking on the Apify platform) is now an internal concern of the client implementation.
|`listHead(opts?)`|`fetchNextRequest()` (returns a single request, marks it in progress) |
572
+
|`listAndLockHead(opts)`| Removed (locking is internal to the client) |
573
+
|`prolongRequestLock(id, opts)`| Removed |
574
+
|`deleteRequestLock(id, opts?)`| Removed |
565
575
|`deleteRequest(id)`| Removed |
576
+
|_(n/a)_|`isEmpty()` (new — `true` when no pending requests remain) |
577
+
578
+
The lifecycle is now: `fetchNextRequest()` hands out a pending request and marks it in progress; once processed, call `markRequestAsHandled(request)`; on failure call `reclaimRequest(request, { forefront? })` to return it to the queue.
579
+
580
+
`RequestQueueClient.isEmpty()` reports whether any **pending** request remains (i.e. whether the next `fetchNextRequest()` would return `null`); requests that are in progress are not counted.
581
+
582
+
`RequestQueueV1` and `RequestQueueV2` no longer differ in behavior — both are thin frontends over the slim client — and `RequestQueueV1` is now a deprecated alias of `RequestQueue`.
566
583
567
-
**Removed types** from `@crawlee/types`: `DatasetClientUpdateOptions`, `KeyValueStoreClientUpdateOptions`, `KeyValueStoreRecordOptions`, `KeyValueStoreClientListData`, `KeyValueStoreClientGetRecordOptions`. `KeyValueStoreClientListOptions` was renamed to `KeyValueStoreListKeysOptions`.
584
+
**Removed types** from `@crawlee/types`: `DatasetClientUpdateOptions`, `KeyValueStoreClientUpdateOptions`, `KeyValueStoreRecordOptions`, `KeyValueStoreClientListData`, `KeyValueStoreClientGetRecordOptions`, `QueueHead`, `RequestQueueHeadItem`, `ListOptions`, `ListAndLockOptions`, `ListAndLockHeadResult`, `ProlongRequestLockOptions`, `ProlongRequestLockResult`, `DeleteRequestLockOptions`. `KeyValueStoreClientListOptions` was renamed to `KeyValueStoreListKeysOptions`.
568
585
569
586
The high-level storage classes (`Dataset`, `KeyValueStore`, `RequestQueue`) now receive their sub-client directly in the constructor options instead of receiving a `StorageClient` and calling its methods.
0 commit comments