JWE/JWT support currently broken in master branch

[taniwallach]

The master branch was properly handling problemJWT being sent by the XBlock under development as of commit d2d844e (Merge pull request #62 from drdrew42/feature/strict-production-mode), but now (after commits from Aug 22) it is not.

In the Docker logs for the container I see:

[2021-09-01 06:35:06.27719] [21] [trace] [tPiMnSYHXP7n] POST "/render-api"
[2021-09-01 06:35:06.27744] [21] [trace] [tPiMnSYHXP7n] Routing to controller "RenderApp::Controller::Render" and action "problem"
[2021-09-01 06:35:06.27784] [21] [info] [tPiMnSYHXP7n] Received JWT: using problemJWT
problemSeed, sourceFilePath, numIncorrect, showComments, showHints, psvn, outputFormat, problemSourceURL, format, answersSubmitted, language, displayMode, showSolutions, permissionLevel, numCorrect, problemJWT, showSummary, includeTags, processAnswers, problemSource, aud at /usr/app/lib/RenderApp/Controller/Render.pm line 64.
[2021-09-01 06:35:06.27866] [21] [info] [tPiMnSYHXP7n] CREATED: Problem created from webwork-open-problem-library/OpenProblemLibrary/Dartmouth/setMTWCh2S4/problem_5.pg with random seed #1795344405
[2021-09-01 06:35:06.35775] [21] [debug] [tPiMnSYHXP7n] 500 Render failed: Can't locate object method "decode" via package "RenderApp::Model::JWT" (perhaps you forgot to load "RenderApp::Model::JWT"?) at /usr/app/lib/RenderApp/Controller/RenderProblem.pm line 480.
[2021-09-01 06:35:06.35794] [21] [warn] [tPiMnSYHXP7n] Render failed: Can't locate object method "decode" via package "RenderApp::Model::JWT" (perhaps you forgot to load "RenderApp::Model::JWT"?) at /usr/app/lib/RenderApp/Controller/RenderProblem.pm line 480.
[2021-09-01 06:35:06.35826] [21] [debug] [tPiMnSYHXP7n] 500 Internal Server Error (0.081055s, 12.337/s)
[2021-09-01 06:35:06.35861] [21] [error] [tPiMnSYHXP7n] TRASH: [80.1ms] render from webwork-open-problem-library/OpenProblemLibrary/Dartmouth/setMTWCh2S4/problem_5.pg failed with error: Internal Server Error

The problem may be related to the fact that 6b0e059#diff-d7ce13f075eeb062a67c5ad1b95701a86ad1d0e80679b0f12d14e39e7c31ec97 has

    my $session = RenderApp::Model::JWT->decode($inputs_ref->{sessionJWT});

but it seems that #71 which is still a pending PR contains lib/RenderApp/Model/JWT.pm which is a new file.

Bottom line the current status of the master branch is broken.

[drdrew42]

Thanks @taniwallach -- this was supposed to be part of the JWT-JWE-evolution branch

From the commit message it seems that I was intending to update the problemUUID (where I had been seeing 'wide character' warnings), and accidentally got a lot more than I bargained for.

I've rebased so that the commit is now where it belongs, and our main branch should be back to full-strength

[drdrew42]

I think I've now got all the branches un-tangled and rebased -- recommend everyone do a git reset --soft since histories have been re-written...

[taniwallach]

@drdrew42 - Thanks!

The revised main branch is working again for me.

[taniwallach]

This is a bit off topic, but relates to the message above regarding the "wide character in print" the problematic commit referred to (maybe relating to problemUUID but maybe somewhere else).

Reminder: encode_utf8 is probably usually not the best choice for encoding into UTF-8. It does not output "strict" UTF-8, and the Perl documentation on encode_utf8 recommends not using encode_utf8 for data exchange. Usually, it is better to use Encode::encode("UTF-8",$source) instead of Encode::encode_utf8($source).

Regarding problemUUID in lib/RenderApp/Controller/RenderProblem.pm where you have $inputs_ref->{problemUUID} = md5_hex(Encode::encode_utf8($source));. That conversion is only for internal use only, so it is probably not really critical to make the change to the "better" method of encoding. Using encode_utf8 may even have a certain benefit, as the Perl docs say that encode_utf8 does not fail due to the "lax" UTF-8 it allows.

Note: PG is stuck using encode_utf8 and some other less preferable methods of handling conversion to/from UTF-8, as Encode::encode("UTF-8", ) does not work in WWSafe. For example. that is why pg/lib/WeBWorK/PG/ImageGenerator.pm is stuck using encode_utf8. See the discussions at openwebwork/webwork2#982 and
openwebwork/webwork2#1154 . That sort of restriction will also apply to things which the renderer runs inside a WWSafe setting.

[drdrew42]

Thanks for the clarification -- you're right that problemUUID is only parsing UTF-8 for the purposes of generating a unique hash, but in general I will heed this advice.

Maybe we can also find a way to push Encode into the safe so that the operations mask doesn't block Encode::encode()...