Skip to content

[ISSUE 17] implement drop_pii for arrays and maps#46

Open
oxg558 wants to merge 17 commits into
masterfrom
issue-17-drop-pii-is-broken
Open

[ISSUE 17] implement drop_pii for arrays and maps#46
oxg558 wants to merge 17 commits into
masterfrom
issue-17-drop-pii-is-broken

Conversation

@oxg558
Copy link
Copy Markdown
Contributor

@oxg558 oxg558 commented May 20, 2026

Fixes #17 by implementing drop_pii for arrays and maps that have record values.

Also adds a topological sort to the generate_code function. This makes it easier to generate more complex schemas at runtime in tests like those written here for drop_pii.

@oxg558 oxg558 force-pushed the issue-17-drop-pii-is-broken branch from ecb1955 to 3813299 Compare May 21, 2026 11:19
@oxg558 oxg558 force-pushed the issue-17-drop-pii-is-broken branch from 3813299 to 7edb635 Compare May 21, 2026 11:38
@oxg558 oxg558 force-pushed the issue-17-drop-pii-is-broken branch from 4987730 to 5d2454f Compare May 21, 2026 12:06
@oxg558 oxg558 changed the title [ISSUE 17] fix drop_pii [ISSUE 17] implement drop_pii for arrays May 21, 2026
@oxg558 oxg558 changed the title [ISSUE 17] implement drop_pii for arrays [ISSUE 17] implement drop_pii for arrays and maps May 21, 2026
@oxg558 oxg558 force-pushed the issue-17-drop-pii-is-broken branch from eaab6fd to 2015180 Compare May 21, 2026 16:10
@oxg558 oxg558 force-pushed the issue-17-drop-pii-is-broken branch from 434a08f to abc9549 Compare May 21, 2026 16:16
@oxg558 oxg558 force-pushed the issue-17-drop-pii-is-broken branch from abc9549 to 129bb9f Compare June 1, 2026 16:19
for people like me who struggle a bit reading the macro code
@oxg558 oxg558 force-pushed the issue-17-drop-pii-is-broken branch from 129bb9f to ae62d8a Compare June 1, 2026 16:20
@oxg558 oxg558 marked this pull request as ready for review June 1, 2026 16:22
@oxg558 oxg558 requested a review from a team as a code owner June 1, 2026 16:22
@oxg558 oxg558 requested a review from a team June 1, 2026 16:23
Copy link
Copy Markdown

@hpatoio hpatoio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a comment.

Few more generic questions:

  • Can we tag a major version ? Unless there is a good reason not to do it. Asked a similar question here
  • Is there anything ppl must do when updating to this new version ?

Comment thread README.md
@hpatoio hpatoio self-requested a review June 5, 2026 07:13
hpatoio
hpatoio previously approved these changes Jun 5, 2026
cpiemontese
cpiemontese previously approved these changes Jun 5, 2026
@oxg558 oxg558 dismissed stale reviews from cpiemontese and hpatoio via d0a4ce7 June 5, 2026 08:23
@oxg558 oxg558 requested review from cpiemontese and removed request for cpiemontese June 5, 2026 08:24
Comment thread lib/avrogen/avro/types/map.ex Outdated

unquote_splicing(value_clauses)

def unquote(values_fn)(value), do: value
Copy link
Copy Markdown
Contributor Author

@oxg558 oxg558 Jun 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an issue here for maps of maps: the value_clauses can also contain a helper function for an inner type which would mean the final unquote would leave the function clauses out of order, e.g.:

  def drop_pii_raw_payloads(value) when is_map(value) do
    Enum.reduce(value, %{}, fn {k, v}, acc ->
      Elixir.Map.put(acc, k, drop_pii_raw_payloads_values(v))
    end)
  end

  def drop_pii_raw_payloads_values(value) when is_map(value) do
    Enum.reduce(value, %{}, fn {k, v}, acc ->
      Elixir.Map.put(acc, k, drop_pii_raw_payloads_values_values(v))
    end)
  end

  def drop_pii_raw_payloads_values_values(%RawPayload{} = value) do
    RawPayload.drop_pii(value)
  end

  def drop_pii_raw_payloads_values_values(value) do
    value
  end

  def drop_pii_raw_payloads_values(value) do       # <----- out of order
    value
  end

This generates a compile warning

@oxg558 oxg558 force-pushed the issue-17-drop-pii-is-broken branch from 57f0be6 to 8ed9908 Compare June 5, 2026 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

drop_pii seems broken

3 participants