How to Add Resiliency to Errors in a no_std and no_alloc Context ?

[joboudreault]

Goal

Being able to completely ignore a field in a JSON string in a no_std and no_alloc environment with the serde_json_core crate. The JSON processed may be of two forms :

Successful response :

{
    "status": "success",
    "message": "",
    "data": {"some_data": 42}
}

Unsuccessful response :

{
    "status": "error",
    "message": "error message",
    "data": {"unknown_field": "some_string"}
}

The JSON is then converted into this Rust structure :

pub struct ApiResponse<'a, T> {
    pub status: ApiStatus,
    pub message: &'a str,
    pub data: Option<T>,
}

If the status field is "error", then the data field must be completely ignored even if there are unknown fields of any kind (string, number, object, null, ...).

Problem

Currently : there is no way to perform this deserialization without the serde_json_core::from_slice() function returning Result::Err(_). Thus, the message field is not accessible.

Expected : there exists a way to successfully deserialize the JSON string whether it is an error or not, and access the error message.

Note : All attempts have a Minimal Reproducible Example (MRE) in this GitHub repository (here).

Attempt 1 (normal)

Here, we tried simply using the #[derive(Deserialize)] macro.

Deserialization error :

Error: CustomError;; JSON does not match deserializer’s expected format.

Attempt 2

Here, we tried to deserialize with a custom deserializer thinking that serde will return None if it cannot successfully convert the data field into T.

Deserialization error :

Error: CustomError;; JSON does not match deserializer’s expected format.

Attempt 3

Here, we tried to deserialize with a custom deserializer thinking that the Result returned by the <T as Deserialize>::deserialize() function may be intercepted and mapped to Ok(None)

Deserialization error :

Error: TrailingCharacters;; JSON has non-whitespace trailing characters after the value.

Attempt 4

Here, we tried to deserialize with a custom deserializer and custom Visitor emptying all fields in the data field's object. In this attempt, we realized that all the functions in the Deserializer<'de> trait are taking ownership which disallow us to perform an operation trying to convert T or to None depending on the status field.

Deserialization panic message :

Body is a 'Data' structure: ExpectedObjectCommaOrEnd

Note: All attempts have a MRE on this GitHub repository (here).

Possible Solution

There exists a solution when using serde_json and allocating a HashMap as described in issue #1583. However, we cannot use this method because there must not be any allocator, thus HashMap cannot be used.

Environment

All compilation is run with the following Rust version :

$ cargo -Vv
cargo 1.79.0 (ffa9cf99a 2024-06-03)
release: 1.79.0
commit-hash: ffa9cf99a594e59032757403d4c780b46dc2c43a
commit-date: 2024-06-03
host: x86_64-unknown-linux-gnu
libgit2: 1.7.2 (sys:0.18.3 vendored)
libcurl: 8.6.0-DEV (sys:0.4.72+curl-8.6.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Arch Linux [64-bit]

$ rustc -Vv
rustc 1.79.0 (129f3b996 2024-06-10)
binary: rustc
commit-hash: 129f3b9964af4d4a709d1383930ade12dfe7c081
commit-date: 2024-06-10
host: x86_64-unknown-linux-gnu
release: 1.79.0
LLVM version: 18.1.7

[dtolnay]

serde_json_core lives in a different repo. https://github.com/rust-embedded-community/serde-json-core

[joboudreault]

Sorry for this inconvenience. Here is the Issue in the serde-json-core repository for history : How to Add Resiliency to Errors in a no_std and no_alloc Context ?