Add Merge-Patch Support Handling by jason-fox · Pull Request #1283 · telefonicaid/iotagent-node-lib

jason-fox · 2022-08-25T16:27:10Z

merge-patch is an advanced operation which would be usefully supported by some robotics IoT Agents - e.g. ROS2, OPC-UA. This PR adds a new merge-patch handler and forwards NGSI-LD merge-patch requests to the handler as necessary.

Update server
Add Unit Tests
Update Docs

Not all IoT Agents will need to support advanced services such as merge-patch, NGSI-LD null or datasetId, flags have been included to allow support for these payloads to be withdrawn.

jason-fox · 2022-08-25T16:40:50Z

When in ld mode, the defaults for the NSGI-LD server of the IoT Agent handling null and datasetId should both be true, as supporting both is standard for NSGI-LD 1.5 - how this works in practice will depend on the handler within the relevant IoT Agent.

For example, a payload like this

{
  "position": [
    {
      "type": "Property",
      "value": [ 1, 2, 3 ],
      "datasetId": "urn:ngsi-ld:this"
    },
    {
      "type": "Property",
      "value": [ 28, -104, 23 ],
      "datasetId": "urn:ngsi-ld:that"
    }
  ]
}

Will send two copies of the position attribute to be processed. Whether that is to be interpreted as "two sequential commands" or "do nothing at all with attributes containing a datasetId" will depend on the handler in the IoT Agent.

Without further changes downstream, at the moment, the IoT Agent for JSON and IoT Agent for Ultralight will accept this as two sequential commands on the default datasetId - additional code will be required to allow for other options.

Update expectation since merge-patch needs to retrieve lazy attributes as well.

jason-fox · 2022-08-25T17:13:07Z

This PR will close the remaining issues from #1259

The main part still missing is an ability to reject null and/or datasetId on purpose - i.e. "This IoT Agent can't understand that payload" - and make that configurable through config.js and environment variables.

mapedraza · 2023-03-07T09:56:13Z

doc/installationguide.md

+| IOTA_LD_SUPPORT_NULL                 | `server.ldSupport.null`         |
+| IOTA_LD_SUPPORT_DATASET_ID           | `server.ldSupport.datasetId`    |


Those parameters need to be documented

Fixed af14fde

mapedraza · 2023-03-07T10:02:22Z

lib/services/devices/deviceService.js

                    deviceObj.type = deviceData.type;
                    deviceObj.staticAttributes = deviceData.staticAttributes;
                    deviceObj.commands = deviceData.commands;
+                    deviceObj.lazy = deviceData.lazy;


Could you explain why you included this line and how it is related with functionalities implemented in this PR (merge patch)

This is necessary to ensure that a lazy attribute may be merged-patched (see use on line lib/services/northBound/contextServer-NGSI-LD.js - 455)

mapedraza · 2023-03-07T10:06:39Z

CHANGES_NEXT_RELEASE entry is missing

jason-fox · 2023-03-10T13:18:10Z

CHANGES_NEXT_RELEASE entry is missing

Fixed ce7746b

fgalan · 2023-03-14T15:38:59Z

@jason-fox please fix the small conflict in CHANGES_NEXT_RELEASE file so this PR becomes mergeable again.

CHANGES_NEXT_RELEASE

mapedraza · 2023-03-14T17:10:53Z

@jason-fox weirdly, test are failing

825 passing (25s)
[1802](https://github.com/telefonicaid/iotagent-node-lib/actions/runs/4418276954/jobs/7745258389#step:5:1803)
  11 pending
[1803](https://github.com/telefonicaid/iotagent-node-lib/actions/runs/4418276954/jobs/7745258389#step:5:1804)
  1 failing
[1804](https://github.com/telefonicaid/iotagent-node-lib/actions/runs/4418276954/jobs/7745258389#step:5:1805)

[1805](https://github.com/telefonicaid/iotagent-node-lib/actions/runs/4418276954/jobs/7745258389#step:5:1806)
  1) NGSI-LD - Merge-Patch functionalities
[1806](https://github.com/telefonicaid/iotagent-node-lib/actions/runs/4418276954/jobs/7745258389#step:5:1807)
       When a merge-patch PATCH  arrives to the IoT Agent as Context Provider
[1807](https://github.com/telefonicaid/iotagent-node-lib/actions/runs/4418276954/jobs/7745258389#step:5:1808)
         should call the client handler once:
[1808](https://github.com/telefonicaid/iotagent-node-lib/actions/runs/4418276954/jobs/7745258389#step:5:1809)
     Error: Timeout of 8000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/runner/work/iotagent-node-lib/iotagent-node-lib/test/unit/ngsi-ld/lazyAndCommands/merge-patch-test.js)
[1809](https://github.com/telefonicaid/iotagent-node-lib/actions/runs/4418276954/jobs/7745258389#step:5:1810)
      at listOnTimeout (internal/timers.js:557:17)
[1810](https://github.com/telefonicaid/iotagent-node-lib/actions/runs/4418276954/jobs/7745258389#step:5:1811)
      at processTimers (internal/timers.js:500:7)
[1811](https://github.com/telefonicaid/iotagent-node-lib/actions/runs/4418276954/jobs/7745258389#step:5:1812)

Co-authored-by: Fermín Galán Márquez <fgalan@users.noreply.github.com>

jason-fox · 2023-03-14T19:37:50Z

test was failing

@mapedraza this is because this PR predates #1302 which has just been merged, and therefore the updated 1.6.1 NGSI-LD registration is required when provisioning - fixed 8510ad2

mapedraza

LGTM

fgalan

LGTM

jason-fox added 6 commits August 22, 2022 08:33

merge-patch in progress

46a21d8

Completing Merge Patch code

397debf

Add lazy

603d604

Correct param

7d58442

Allow datasetId and null to be unsupported

ddc1af2

Simplify names and update documentation

a614463

Merge branch 'master' into feature/merge-patch

d4d8f6c

Update expectation since merge-patch needs to retrieve lazy attributes as well.

jason-fox and others added 3 commits August 30, 2022 20:31

correct attribute name.

e2fb50b

Merge branch 'master' into feature/merge-patch

2148a25

Merge branch 'master' into feature/merge-patch

6ca98a4

mapedraza reviewed Mar 7, 2023

View reviewed changes

Update CNR

ce7746b

jason-fox added 3 commits March 10, 2023 14:35

Add documentation.

af14fde

Fix typo.

477f298

Remove space

e2d57bc

Merge branch 'master' into feature/merge-patch

8d7b391

fgalan reviewed Mar 14, 2023

View reviewed changes

CHANGES_NEXT_RELEASE Outdated Show resolved Hide resolved

jason-fox and others added 3 commits March 14, 2023 18:30

Fixing test expectation

8510ad2

Upgrade Merge to NGSI 1.6.1

2959af4

Update CHANGES_NEXT_RELEASE

1b18903

Co-authored-by: Fermín Galán Márquez <fgalan@users.noreply.github.com>

Correct merge => mergeEntity in registration operations filter

bcc8ba5

mapedraza approved these changes Mar 15, 2023

View reviewed changes

fgalan approved these changes Mar 15, 2023

View reviewed changes

fgalan merged commit f7badab into telefonicaid:master Mar 15, 2023

jason-fox deleted the feature/merge-patch branch April 26, 2023 06:53

		\| IOTA_LD_SUPPORT_NULL \| `server.ldSupport.null` \|
		\| IOTA_LD_SUPPORT_DATASET_ID \| `server.ldSupport.datasetId` \|

Conversation

jason-fox commented Aug 25, 2022

Uh oh!

jason-fox commented Aug 25, 2022

Uh oh!

jason-fox commented Aug 25, 2022

Uh oh!

mapedraza Mar 7, 2023

Choose a reason for hiding this comment

Uh oh!

jason-fox Mar 10, 2023

Choose a reason for hiding this comment

Uh oh!

mapedraza Mar 7, 2023

Choose a reason for hiding this comment

Uh oh!

jason-fox Mar 10, 2023

Choose a reason for hiding this comment

Uh oh!

mapedraza commented Mar 7, 2023

Uh oh!

jason-fox commented Mar 10, 2023

Uh oh!

fgalan commented Mar 14, 2023

Uh oh!

Uh oh!

mapedraza commented Mar 14, 2023

Uh oh!

jason-fox commented Mar 14, 2023

Uh oh!

mapedraza left a comment

Choose a reason for hiding this comment

Uh oh!

fgalan left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants