Opcua client fails to connect to Prosys Simulation Server configured with multiple endpoints

[DOhlsson]

I am trying to connect to Prosys Simulation Server while developing code with this library.

If I connect with SecurityMode None or Sign, it works without issues. But when connecting to the server with the SignAndEncrypt mode and SecurityPolicy Basic256Sha256 it fails with strange errors. I have tried using the simple-client from the examples folder, both with version 0.12 and the latest commit cf2d1bd on master.

simple-client 0.12 output with RUST_OPCUA_LOG=debug

2024-08-07 14:11:00.426 DEBUG opcua::client::message_queue             Request 2 was processed by the server
2024-08-07 14:11:00.438 DEBUG opcua::client::comms::tcp_transport      receive chunk intermediate 1002:272
2024-08-07 14:11:00.482 DEBUG opcua::client::comms::tcp_transport      receive chunk intermediate 1002:273
2024-08-07 14:11:00.482 ERROR opcua::types::service_types::enums       Invalid value 257 for enum MessageSecurityMode
2024-08-07 14:11:00.482 DEBUG opcua::core::comms::chunker              Cannot decode message CreateSessionResponse_Encoding_DefaultBinary, err = StatusCode(IS_ERROR | BadUnexpectedError)
2024-08-07 14:11:00.482 INFO  opcua::client::comms::tcp_transport      ReadState has dropped
2024-08-07 14:11:00.482 INFO  opcua::client::comms::tcp_transport      WriteState has dropped
2024-08-07 14:11:00.482 DEBUG opcua::client::comms::tcp_transport      Closing connection because the read loop terminated
2024-08-07 14:11:10.426 INFO  opcua::client::session::session_state    Timeout waiting for response from server
2024-08-07 14:11:10.426 INFO  opcua::client::message_queue             Request 2 has timed out and any response will be ignored
2024-08-07 14:11:10.426 ERROR opcua::client::client                    Got an error while creating the default session - BadTimeout
2024-08-07 14:11:10.426 DEBUG opcua::client::session::session_state    Session was closed with status = BadServiceUnsupported
2024-08-07 14:11:10.426 INFO  opcua::client::session::session          Session has dropped
2024-08-07 14:11:10.426 INFO  opcua::client::comms::tcp_transport      TcpTransport has dropped
2024-08-07 14:11:10.426 INFO  opcua::client::session::session_state    SessionState has dropped

The message Invalid value 257 for enum MessageSecurityMode changes values to 257, 513 and 769 depending on which Endpoints I have opened on the server. Interestingly the values correspond to 0x0101, 0x0201 and 0x0301.

The only way I can get the simple-client to connect to this server with SignAndEncrypt is if I configure the server with only a single endpoint for SignAndEncrypt and the Basic256Sha256 policy.

I would like to be able to connect to the server with any possible combination of endpoints. If there is anything else I can provide to help debugging this issue I will gladly provide.

Thank you for your time creating this otherwise wonderful library!

[einarmo]

This is strange, I wonder if it's a binary layout issue? MessageSecurityMode is quite clearly either 1, 2, or 3. This is presumably in the endpointDescriptions array in create session response. I double checked, and I'm quite sure we're decoding that value correctly, treating it as a 32 bit number.

It would be good to run wireshark and see what it thinks the payload is, but it really does look to me like the server is setting an invalid value for that field, for some reason.

I can't find anything in the standard for modifying that specific enum, at least.

[einarmo]

A workaround that I think can be done with the upcoming rewrite of the type generation is to decode invalid values to the INVALID_0 MessageSecurityMode, which might solve the issue, that might be how other clients tolerate this server, though I don't know for sure.

[DOhlsson]

I tried analyzing the network packets with Wireshark, but it apparently does not support decrypting OPC UA.

I have however gotten a bit closer to figuring it out. The data being parsed out when decoding the GetEndpointsResponse contains a null byte at the end of each chunk. So an extra byte of 0x00 is added into the middle of the server certificate, causing MessageSecurityMode::decode() to read the last byte of the certificate and then the first 3 bytes from the MSM.

I have not figured out if this is caused by this library or by the server.

[DOhlsson]

I have found the cause of this issue. It seems that this code does not take into account that there is a paddingSize byte between the message data and the signature. See spec.

This causes each chunk to become corrupt and contain an extra 0 at the end.

[einarmo]

I feel like I would need to spend a lot more time reading to figure out if you're right, but this is really good catch if so. I wonder if this applies to the send-side as well?

[einarmo]

Trying to investigate this by connecting to a server in the reference SDK and I can't make that work either, which may or may not be related.

[einarmo]

Trying to investigate this by connecting to a server in the reference SDK and I can't make that work either, which may or may not be related.

Not related, just a mistake on my end, and a tiny bug I will fix, not related to your problem.

[DOhlsson]

Trying to investigate this by connecting to a server in the reference SDK and I can't make that work either, which may or may not be related.

Not related, just a mistake on my end, and a tiny bug I will fix, not related to your problem.

It occurs for me when there are many endpoints enabled on the server, so that the message gets chunked up.

[einarmo]

Managed to reproduce, I think, got a BadDecodingError once I got the server to split the message. Took a while because I found a bug in the reference SDK. Clearly nobody can be trusted to correctly implement this standard.

Different issue than you got, but that's probably just a coincidence, I imagine you would have gotten a similar error if the decoder managed to continue: Decoding error - Error { kind: UnexpectedEof, message: "failed to fill whole buffer" }.

[einarmo]

Got it.

if self.security_mode == MessageSecurityMode::SignAndEncrypt {
    Self::remove_signature_and_padding(
        decrypted_data,
        decrypted_size,
        signature_size,
        &self.decoding_options,
    )?
} else {
    Self::update_message_size_and_truncate(
        decrypted_data, 
        decrypted_size - signature_size,
        &self.decoding_options
    )?
}

and

fn remove_signature_and_padding(
    mut data: Vec<u8>,
    decrypted_size: usize,
    signature_size: usize,
    decoding_options: &DecodingOptions
) -> Result<Vec<u8>, StatusCode> {
    let padding_start = decrypted_size - signature_size - 1;
    let padding_size = data[padding_start];
    
    for i in ((padding_start - padding_size as usize)..padding_start).rev() {
        if data[i] != padding_size {
            error!("Failed to verify padding, got {} when looking at byte {}, expected {}",
                data[i], i, padding_size);
            return Err(StatusCode::BadDecodingError);
        }
    }
    // Remove one extra byte for the padding size.
    let message_size = decrypted_size - signature_size - padding_size as usize - 1;
    Self::update_message_size(&mut data, message_size, decoding_options)?;
    data.truncate(message_size);
    Ok(data)
}

I did this deep in my own branch since the compile times are so much better, but I'll see about porting my changes, which includes some other tiny fixes, to the current master branch. I also want to see if I can write a test, though since the server uses the same code it's a bit limited in usefulness.

[einarmo]

This also does not do anything about ExtraPaddingSize, I need to make a sample that actually has a key larger than 2048 bytes to test that.

[einarmo]

The standard also talks about Authenticated encryption algorithms, which don't have padding? We also need to look into that.

The secure_channel code is a total mess at this point, it could use a major cleanup and some more structure, but I want to put together a set of tests against a reference server before we do that.

[DOhlsson]

Great work! If there is any code you'd like me to try out I'd be glad to help.

Different issue than you got, but that's probably just a coincidence, I imagine you would have gotten a similar error if the decoder managed to continue: Decoding error - Error { kind: UnexpectedEof, message: "failed to fill whole buffer" }.

I get errors relating to decoding of strings. Because the length becomes crazy when the parsed fields are off. Decoding Error with statuscode BadDecodingError. String buf length 151391536 exceeds decoding limit 65535.

[einarmo]

It would be great if you could give #377 a try @DOhlsson, it should fix your issue.