|
| 1 | +# A short and accurate title |
| 2 | + |
| 3 | +## Preamble |
| 4 | + |
| 5 | + Author: Philipp Böschen <[email protected]> |
| 6 | + Sponsor: |
| 7 | + ID: 0034 |
| 8 | + Status: Draft |
| 9 | + |
| 10 | +## Abstract |
| 11 | + |
| 12 | +This PPC suggests the implementation of a new perlrun option `-j` that works in conjunction with `-p` or `-n` to automatically feed the stdin that is parsed into `JSON::PP::decode_json($_);` |
| 13 | +A working example can be found at: https://github.com/Perl/perl5/pull/22718 |
| 14 | + |
| 15 | +## Motivation |
| 16 | + |
| 17 | +Ultimately this change would make Perl another step further to being extremely useful in adhoc sysadmin tasks, a role that is currently covered by, usually, the `jq` tool. |
| 18 | +I personally have found that `jq`, while useful for quick extracts, becomes very cumbersome if you have to do conditional logic, which is where `perl -jnle` would shine since usually a simple `grep` or `map` can work wonders on getting good data quickly. |
| 19 | +JSON has largely become ubiquitous with modern API handling, so I feel like it's not overly specific to let Perl "natively" handle it. |
| 20 | + |
| 21 | +As the [PRs](https://github.com/Perl/perl5/pull/22718) discussion shows, there are of course alternatives, but none of them seem to be as accessible as a `-j` flag, especially if we are thinking of where `jq` is often applied: |
| 22 | +- On cloud-init scripts where Perl is present but not really considered |
| 23 | +- In a bare bones debugging environment it gets often pulled in, Perl would already be there |
| 24 | + |
| 25 | +## Rationale |
| 26 | + |
| 27 | +I have tested the aformentioned [PR](https://github.com/Perl/perl5/pull/22718) for a while at work and privately and found it quite useful in place of `jq`. It has one less dependency on my existing debugging kit and performs reasonably well. |
| 28 | + |
| 29 | +Any other proposed solutions always come with the catch that now I have to bring said solution along with me to the debugging target, be it a source filter or just a whole different tool written in Perl itself. |
| 30 | + |
| 31 | +There are some thoughts on the "-p" interaction for things that ingest JSON and don't want to produce a new JSON from it, I personally am unsure of what the ideal here would be right now. |
| 32 | + |
| 33 | +## Specification |
| 34 | + |
| 35 | +This would be an extension of https://perldoc.perl.org/perlrun. |
| 36 | +```pod |
| 37 | +=item -j |
| 38 | +
|
| 39 | +causes Perl to consume the lines that "-n" provides to be fed into C<JSON::PP::decode_json($_);> to give access to line delimited JSON feeds as Perl hashes directly. This will fail with the usual errors from C<JSON::PP::decode_json> if the line that is read is not valid JSON. |
| 40 | +When used with both "-n" and -p", "-j" calls C<print JSON::PP::encode_json($_);> to properly print out JSON again. |
| 41 | +``` |
| 42 | +## Backwards Compatibility |
| 43 | +
|
| 44 | +This change can be backported into all versions that have a working `JSON::PP` module. |
| 45 | +
|
| 46 | +## Security Implications |
| 47 | +
|
| 48 | +This would open up a similar security exposure as any perl scripts have that use `JSON::PP`. Of course if this feature gets widely adopted into server bootstraps the criticality of `JSON::PP` becomes more vital over time. |
| 49 | +
|
| 50 | +## Examples |
| 51 | +
|
| 52 | +Structured logging is the obvious use case for this feature so you can for example force `journald` into line by line JSON mode and then parse it like so to list all root processes logs: |
| 53 | +``` |
| 54 | +sudo journalctl -f -o json | ./perl -njle 'if ( $_->{"_GID"} == 0 ) {print $_->{"MESSAGE"}}' |
| 55 | +``` |
| 56 | +
|
| 57 | +This would work similarly for most structured logging applications that have somewhat all agreed on JSON being the format of choice. |
| 58 | +
|
| 59 | +A quick example from parsing audit logs for Hashicorp Vault: |
| 60 | +
|
| 61 | +``` |
| 62 | +tail -f /var/log/vault/audit.log | perl -jnle 'if ( $_->{request}{client_id} eq "some_client" ) { print "$_->{request}{operation},$_->{request}{path}\n" }' |
| 63 | +update,auth/token/create |
| 64 | +... |
| 65 | +``` |
| 66 | +
|
| 67 | +## Prototype Implementation |
| 68 | +
|
| 69 | +https://github.com/Perl/perl5/pull/22718 implements the full PPC while also passing all tests. |
| 70 | +
|
| 71 | +## Future Scope |
| 72 | +
|
| 73 | +There should be no additions would be that `JSON::PP` starts supporting different things. |
| 74 | +Some work could be done on formatting the output in various ways, like supporting pretty printing. |
| 75 | +
|
| 76 | +## Rejected Ideas |
| 77 | +
|
| 78 | +As mentioned above there has been some discussion around writing a custom Perl application/module that does similar things or using a source filter. All of these solutions fall a bit short on the portability of being in the main Perl runtime which is the allure of this flag since it would remove one more dependency from running systems. |
| 79 | +
|
| 80 | +## Open Issues |
| 81 | +
|
| 82 | +
|
| 83 | +## Copyright |
| 84 | +
|
| 85 | +Copyright (C) 2024, Philipp Böschen |
| 86 | +
|
| 87 | +This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. |
| 88 | +
|
0 commit comments