Design input needed - where should tag-based access control live?

[bharos]

I am working on a design for tag-based access control in Gravitino. Before I finalize it,
there is one structural decision I would like consensus on, so it would be great to get some inputs here than me bringing a design that has already assumed an answer.

Grants today are object-identified, and authorization never reads tags. So there is no way
to express something like:

role analyst may read tables tagged certified in schema lakehouse.finance

I would like to add an access rule of the shape
(principal, action, effect, tag condition, scope), evaluated inside the existing
authorization-expression path. With policy-on-tag merged, there are two plausible homes for
it.

Option A — an access_control policy type, reusing Policy and the policy-tag
relation. Principal, action and effect live in the policy's content; the authorizer calls
ObjectPolicyResolver and keeps policies of that type. Almost no new plumbing.

POST /api/metalakes/prod/policies
{ "name": "analyst_can_read", "policyType": "access_control", "enabled": true,
  "content": { "role": "analyst", "action": "SELECT_TABLE", "effect": "ALLOW" } }

PUT /api/metalakes/prod/tags/certified/policies/analyst_can_read
{ "selector": { "type": "ALL_VALUES" } }

Option B — a separate entity with principal, action, effect and scope as columns,
reading tags directly rather than through Policy or ObjectPolicyResolver. It would
reuse PolicySelector so both features express tag conditions the same way. Costs a new
table across three dialects, a REST resource and client support.

POST /api/metalakes/prod/tag-access-policies
{ "principal": { "type": "ROLE", "name": "analyst" },
  "objectType": "TABLE", "action": "SELECT_TABLE", "scope": "lakehouse.finance",
  "effect": "ALLOW",
  "requireTags": [ { "tag": "certified", "selector": { "type": "ALL_VALUES" } } ] }

Which authority should create an access rule?

Under A, creating that rule needs APPLY_POLICY on the policy and APPLY_TAG on the tag.
It needs no permission on the objects the rule affects — the call names no object — and
does not involve MANAGE_GRANTS, which governs granting privileges today via
hasMetadataPrivilegePermission. The rule still grants SELECT_TABLE on every object
carrying the tag.

Scope differs too. APPLY_TAG binds only at METALAKE / TAG and APPLY_POLICY only at
METALAKE / POLICY (ApplyTag.canBindTo), so access rules could only be administered
metalake-wide. MANAGE_GRANTS binds at every object type, and its binding at a parent covers all children.
Under B the create is checked against the rule's
scope, so it can use whichever authority the project considers correct and be delegated
per catalog or schema.

Is APPLY_POLICY + APPLY_TAG the intended authority for creating access rules, or
should they require grant authority on the data they affect?

A related point: principal and effect

Neither has a home in policy-on-tag today — the relation has no principal column (the
policy-on-tag doc(#12177) says the selector "does not support ... principals, scopes, or general
expressions"), and enabled / disabled is not allow / deny.

• Under A both would live inside the policy's opaque content (like a JSON text), so rules cannot be
  looked up by role, and deleting a role leaves rules behind that still name it - a later
  role created with the same name silently inherits them.
• Under B both are columns, so they can be indexed and can cascade on role deletion.

Guidance on either point would help a lot before I take the design further.

[roryqi]

Thanks for your discussion. I prefer solution A.
But we can create a new selector for the access-control policy.
We have a selector called metadata object type selector or other similar names.

{
   type: "OBJECT_TYPE"
    value: "ROLE"
}

So only the role can use the policy. Other objects won't use the policy

[bharos]

Option A sounds good in terms of alignment with current policy on tag support.

Three things I'd like to settle before writing the design doc.

1. DENY can be suppressed by re-tagging

Tag values are nearest-wins and ancestor values are discarded rather than merged. So:

• lakehouse.finance is tagged sensitivity=restricted
• a DENY policy is bound to sensitivity with TAG_VALUE("restricted")
• the owner of lakehouse.finance.orders tags their own table sensitivity=public
• the effective value at the table is now public, the selector stops matching, and the deny silently no longer applies

That's a normal operation for a table owner, so it breaks by accident as easily as on purpose. It isn't specific to access control either — a masking policy bound to classification=pii on a schema can be escaped the same way.

Three directions I can see:

• match DENY against the accumulated assignments over the object and all ancestors, with no suppression, while ALLOW keeps nearest-wins — the schema's restricted stays visible at the table, so the deny still matches. This needs EffectiveTagResolver to return every assignment with its source object rather than only the winner per tag name;
• let a tag be marked non-overridable, so once the schema sets sensitivity, a table underneath cannot re-assign it at all;
• rely on APPLY_TAG being per-tag — if only the governance team holds APPLY_TAG on sensitivity, the table owner cannot set it in the first place. Works today, but it's an operational convention rather than something the system enforces.

I'm leaning towards the first option, as it changes no write path, keeps nearest-wins for ALLOW, and still works when a table legitimately tightens a value rather than loosening it . It also looks cheapest now, since EffectiveTagResolver isn't implemented yet.

The trade-off is that a deny could no longer be lifted by re-tagging, so exemptions would have to be explicit. Does that match how you'd want it to work?

2. Role deletion

The principal lives in content, so there's no cascade -> deleting a role leaves rules that still name it, and a role later created with the same name would inherit them. I am thinking maybe we can add a role-deletion hook that removes or disables those policies?

[roryqi]

1. I prefer not allowing deny privilege in the policy. Policy will be used for many objects. If we support denying, it may be complex. According to other products like Starburst and Databricks, they suggest that we shouldn't use many deny privileges. Are there any cases where we must support deny privilege in the policy?

2. This case won't happen. Because we stored the relation between role id and tag id in our storage. So we can avoid this issue.

[bharos]

Here's the flow as I understand it - want to check we're aligned.

An access rule is a system_access_control policy with four parts:

• scope — the policy is attached to a catalog/schema
• condition — the policy is bound to a tag, with a selector
• action — the privilege, in content
• principal — the policy is associated with a role

POST /policies                                { "name": "read_certified",
                                                "policyType": "system_access_control",
                                                "content": { "action": "SELECT_TABLE" } }
POST /objects/schema/lakehouse.sales/policies { "policiesToAdd": ["read_certified"] }   # scope
PUT  /tags/certified/policies/read_certified  { "selector": { "type": "ALL_VALUES" } }  # condition
POST /objects/role/analyst/policies           { "policiesToAdd": ["read_certified"] }   # principal

POST /objects/table/lakehouse.sales.orders/tags { "tagsToAdd": [{ "name": "certified" }] }

analyst can then read that table, and any future table in the schema tagged certified,
with no grant written. ALLOW only, and USE_CATALOG / USE_SCHEMA stay ordinary RBAC grants
since table access is conjunctive on the parent chain.

For the principal I'm proposing a policy-to-role relation rather than the role-to-tag one

• same cascade on role deletion, but tags keep applying only to data objects.
  It needs no new table: policy_relation_meta is already (policy_id, metadata_object_id, metadata_object_type), and ROLE is a MetadataObject.Type — just not in
  SUPPORTED_METADATA_OBJECT_TYPES_FOR_POLICIES.

Does that match what you had in mind?

[roryqi]

We will remove the support of policy on metadata object. You can read the design document about policy on tag.
I have added storage of policy on tag. The dispatcher part will be summited today.

[nevzheng]

@bharos @roryqi

Not quite the model we're thinking about. Here's ours, then how it differs from yours.

In the policy-on-tag model we rely on scope and hierarchy to propagate policies. Your example (hang the policy on the schema and on the role) is the policy-on-object model. That's different. Doing both is a misconfiguration.

The policy (who / where / what in content, hang on the tag only):

POST /api/metalakes/prod/policies
{
  "name": "analyst_read_certified_finance",
  "policyType": "system_access_control",
  "enabled": true,
  "content": {
    "action": "SELECT_TABLE",
    "role": { "name": "analyst" },
    "scope": { "catalog": "lakehouse", "schema": "sales" }
  }
}

PUT /api/metalakes/prod/tags/certified/policies/analyst_read_certified_finance
{
  "selector": { "type": "ALL_VALUES" }
}

What we're thinking:

sequenceDiagram
  actor Pat
  actor Alice
  participant Policy as POST /policies
  participant Tag as tag certified
  participant Engine
  participant Authz as Authorizer

  Pat->>Policy: create the policy above
  Pat->>Tag: PUT bind to certified
  Note over Tag: only hang

  Alice->>Engine: SELECT lakehouse.sales.orders
  Engine->>Authz: authorize alice
  Authz->>Authz: bearer alice → role analyst
  Authz->>Authz: effective tags on orders → certified
  Authz->>Authz: policy bound to certified
  Authz->>Authz: role, scope, action match
  Authz-->>Engine: ALLOW

Loading

Yours (three hangs: schema, tag, role):

POST /policies
{ "name": "read_certified", "policyType": "system_access_control",
  "content": { "action": "SELECT_TABLE" } }

POST /objects/schema/lakehouse.sales/policies
{ "policiesToAdd": ["read_certified"] }

PUT /tags/certified/policies/read_certified
{ "selector": { "type": "ALL_VALUES" } }

POST /objects/role/analyst/policies
{ "policiesToAdd": ["read_certified"] }

sequenceDiagram
  actor Pat
  actor Alice
  participant Policy as POST /policies
  participant Schema as schema sales
  participant Tag as tag certified
  participant Role as role analyst
  participant Engine
  participant Authz as Authorizer

  Pat->>Policy: create read_certified (action only)
  Pat->>Schema: POST hang on schema
  Pat->>Tag: PUT bind to certified
  Pat->>Role: POST hang on role

  Alice->>Engine: SELECT lakehouse.sales.orders
  Engine->>Authz: authorize alice
  Authz->>Authz: policy hung on analyst?
  Authz->>Authz: policy hung on sales?
  Authz->>Authz: table tagged certified?
  Authz-->>Engine: ALLOW

Loading

Different in three places:

1. Who. You hang the policy on the role object. We put analyst in content. At eval we take the bearer (Alice), expand roles, and match.
2. Where. You hang the policy on the schema. We put scope in content and check the table path. Children under that path pick it up through hierarchy, not through a second hang.
3. How many write paths. You use POST /objects/schema/.../policies and POST /objects/role/.../policies. That's policy-on-object. Rory is removing it. We only bind to the tag.

Same use case. Same four facts. One hang, not three.

Scope

In policy-on-tag, scope + hierarchy is how a rule propagates. content.scope is a path check. Same policy type, same tag bind. Tighter or wider as you need.

Schema-wide (the use case):

"scope": { "catalog": "lakehouse", "schema": "sales" }

orders and payments in sales can match. A certified table in hr does not.

One table (policy-on-object, emulated):

"scope": { "catalog": "lakehouse", "schema": "sales", "table": "orders" }

Only orders matches. payments does not, even if it is also certified.

Omit scope and every table with the tag can match. That's how you emulate "this policy is on this table" without POST /objects/table/orders/policies. The hang is still the tag. Scope is just how tight the path is.

Hanging the same policy on the schema or the role and on the tag is the other model. We shouldn't mix them. Walkthrough is in the top-level comment.

[nevzheng]

Hi @bharos, thanks for opening the discussion.

As @roryqi mentioned, we strongly prefer a policy-on-tag model (option A). I'll walk through your use case, then quote and address each concern you raised. @roryqi correct me if i'm wrong anywhere.

Use case

role analyst may read tables tagged certified in schema lakehouse.finance

Today a grant names a table. Tags are labels. Authorization never looks at them. You want the grant to follow the label, inside one schema. Tag orders certified, analyst can read it. Tag payments later, same rule. A certified table in hr is out. A finance table with no certified tag is out.

Here's how we would implement that with policy-on-tag. The tag bind is the condition. Role, scope, and action live in content. The authorizer checks them after object-policy lookup.

POST /api/metalakes/prod/policies
{
  "name": "analyst_read_certified_finance",
  "policyType": "system_access_control",
  "enabled": true,
  "content": {
    "action": "SELECT_TABLE",
    "role": { "name": "analyst" },
    "scope": { "catalog": "lakehouse", "schema": "finance" }
  }
}

PUT /api/metalakes/prod/tags/certified/policies/analyst_read_certified_finance
{
  "selector": { "type": "ALL_VALUES" }
}

POST /api/metalakes/prod/objects/TABLE/lakehouse.finance.orders/tags
{
  "tagsToAdd": [{ "name": "certified" }]
}

When Alice (an analyst) queries, her role is used in resolution. Access is granted or denied:

sequenceDiagram
  actor Alice
  participant Engine
  participant Authz as Authorizer
  participant Tags as Effective tags
  participant Rel as Policy-tag relations
  participant Policy as access_control policy
  participant FGAC as Read restrictions

  Alice->>Engine: SELECT lakehouse.finance.orders
  Engine->>Authz: authorize(user=alice, action=SELECT, table=orders)
  Authz->>Authz: resolve alice to role analyst
  Authz->>Tags: effective tags on orders
  Tags-->>Authz: certified
  Authz->>Rel: policies bound to certified
  Rel-->>Authz: analyst_read_certified_finance
  Authz->>Policy: check role, scope, action

  alt role analyst AND scope finance AND SELECT
    Policy-->>Authz: match
    Authz-->>Engine: ALLOW
    Engine->>FGAC: resolve read_restriction policies
    alt compose cleanly
      FGAC-->>Engine: row filter and/or mask
      Engine-->>Alice: granted, restricted
    else conflict
      FGAC-->>Engine: fail closed
      Engine-->>Alice: denied
    end
  else role, scope, or action miss
    Policy-->>Authz: no match
    Authz-->>Engine: DENY
    Engine-->>Alice: denied
  end

Loading

Eval: table has effective tag certified → this policy is a candidate → Alice's role, scope, and action match → ALLOW. A certified table in another schema does not pass scope. A finance table with no certified tag never sees this policy. Someone not in analyst is denied even if the table is certified and in finance. USE_CATALOG / USE_SCHEMA stay ordinary RBAC.

This is option A on the policy-on-tag write path. Coarse allow. When FGAC is factored in, we compose: Alice is allowed the table, then read-restriction policies on the same list apply (row filter, column mask). If two distinct filters or masks conflict on one resource, we fail closed.

Addressing your concerns

Where the rule lives

With policy-on-tag merged, there are two plausible homes for it. Option A — an access_control policy type, reusing Policy and the policy-tag relation. Option B — a separate entity with principal, action, effect and scope as columns.

We prefer A. Same policy object, same policy-tag relation, same selector. A new system_access_control type. Not a second table or REST resource.

Create authority

Is APPLY_POLICY + APPLY_TAG the intended authority for creating access rules, or should they require grant authority on the data they affect?

Create authority is separate. We consider that an auth-level question. You're right that it should be resolved. It is orthogonal to policy-on-tag. Policy-on-tag is how the rule attaches and evaluates. It is not how we decide who may create the rule.

On POST /policies we read the typed body, compute the required permissions from role / scope / action, and check them against the caller. Same on update, and on bind-to-tag if the bind widens who can read. I wouldn't hang the policy on the schema to get that check.

We understand this is a bit more involved than a typical HTTP access check. We accept that as part of the design tradeoff.

MANAGE_GRANTS isn't necessarily the right way to think about this. Policy-on-tag is a policy-authoring model, not a grant-row model. The check is who may manipulate this policy (and bind this tag), not who may GRANT SELECT on lakehouse.finance. We can still solve that with the right combination of policy and tag grants.

Role condition

Under A both would live inside the policy's opaque content, so rules cannot be looked up by role, and deleting a role leaves rules behind that still name it - a later role created with the same name silently inherits them.

analyst is not the principal. The principal is the bearer (Alice). analyst is a role condition on the policy, same as the tag condition and the scope. On resolve we take the bearer identity, expand roles, and match. That check is straightforward.

Lookup-by-role and role-delete are not blockers. If we want "show every rule for analyst," we index the role condition. For delete we use that index, or we mark the role as deleted so nothing matches it anymore. Either way the name-reuse leak goes away. Not a reason to hang the policy on the role object.

DENY

Tag values are nearest-wins and ancestor values are discarded rather than merged. So: lakehouse.finance is tagged sensitivity=restricted; a DENY policy is bound to sensitivity with TAG_VALUE("restricted"); the owner of lakehouse.finance.orders tags their own table sensitivity=public; the effective value at the table is now public, the selector stops matching, and the deny silently no longer applies.

That's a real situation. A deny on an ancestor and an allow (or a looser tag) on a child can silently change who can read. Nearest-wins makes a normal tag edit into an escape hatch. You're right to call that out.

Policy-on-tag doesn't preclude denies, and we can define resolution for them. We advocate allow lists instead. We see this as an end-user policy management issue. Ultimately it's up to the end user to write policies for their use case.

That ancestor-deny / child-allow case can be treated as a policy conflict and a failure condition. That's a resolution choice. It doesn't require hanging the policy on the object or a second attachment model.

Policy on schema and role

An access rule is a system_access_control policy with four parts: scope — the policy is attached to a catalog/schema; condition — the policy is bound to a tag; action — the privilege, in content; principal — the policy is associated with a role.

POST /objects/schema/lakehouse.sales/policies … POST /objects/role/analyst/policies

Does that match what you had in mind?

No. Condition on the tag is right. Scope and role do not need object attachment. That write path is policy-on-object. Rory is removing it. The use case above does the same four facts without those two POSTs: scope and role live in content, condition is the tag bind.

[nevzheng]

Please let me or Rory know if you have further questions or concerns, or if a part of the use case still doesn't fit. If possible, keep it concrete and grounded in CUJs / use cases (who is querying, which table, which tag, what should happen). That makes it much easier to discuss / design for.

[laserninja]

Coming in as an outside reader. A looks settled, so I'll skip that and raise two things about the evaluation path that I think need an answer in the design doc either way, plus a few smaller notes.

1. A tag rule can't be a jCasbin policy row

jcasbin_model.conf is p = sub, metadataType, metadataId, act, eft, loaded per-role into allowEnforcer / denyEnforcer. An access rule has no metadataId - it matches a set that changes when someone edits a tag. So either expand the tagged object set into p rows at load time (enforcer untouched, deny composition stays free, but every tag write invalidates the affected roles), or evaluate as a second stage after the enforcer (cheap to load, but policy_effect no longer describes the outcome).

The second matches the sequence diagram above. If it's that one, the doc should state explicitly: does an RBAC DENY beat a tag ALLOW? If the tag stage runs after the enforcer returns false and can flip it to true, a tag rule silently defeats every existing deny. I'd suggest unioning the tag ALLOW with the RBAC allow set and applying RBAC deny last - but the point is that it should be written down rather than fall out of call ordering.

Related, and currently unhandled: JcasbinAuthorizer caches are version-validated on user_meta / group_meta / role_meta.updated_at, and JcasbinChangeListener polls entity_change_log and owner_meta. Tag assignments and policy-tag binds are in neither path, so "tag it and access follows" has no propagation delay or HA story yet.

2. Dropping DENY reverses the escalation rather than removing it

I prefer not allowing deny privilege in the policy.

Understood, and the Starburst/Databricks comparison is fair. But allow-only means anyone who can apply a tag that has an access policy bound to it can pull objects into that policy. A table owner in lakehouse.finance tags their own table certified, and every analyst can read it. No policy was edited and no grant was written - that's an ordinary action by a different user holding APPLY_TAG, not a governance team authoring a bad policy.

Which is why I'd push back gently on this:

Create authority is separate. We consider that an auth-level question.

Separable from the attachment model, agreed. Not separable from shipping: without it, APPLY_POLICY + APPLY_TAG becomes strictly stronger than MANAGE_GRANTS, the privilege that governs granting today. The scope asymmetry from the original post is real too - ApplyTag.canBindTo accepts only METALAKE and TAG, while ManageGrants accepts all of MANAGE_GRANTS_SUPPORTED_TYPES - so "let this team administer access rules for their own catalog" isn't expressible through the policy and tag privileges alone. Deriving the required authority from content.scope is what makes per-catalog delegation possible, which I read as an argument for that approach.

If the check is derived from content, it covers four write paths, not one: creating the policy, updating content.action / content.scope or enabled, binding it to a tag, and applying a tag that already carries access policies. The last is the case above and the easiest to miss. One shared "can the caller grant action at scope" helper called from all four keeps them consistent. It can't ride on @AuthorizationExpression - expression() is a compile-time constant per endpoint and can't see the request body - so it has to be an explicit check after deserialization.

Smaller notes

• Listing. MetadataAuthzHelper.LIST_SHORT_CIRCUITS skips per-object checks when a parent grant exists and no deny does. A tag ALLOW breaks the mirror case: without a parent grant you can't return an empty list, because children may be allowed by tag. Committing to the inverse lookup - roles to policies to bound tags to tagged objects, intersected with the listed namespace - constrains the resolver to a batch/inverse API rather than just tagsFor(object), so it's worth settling before it's written.
• Typed content. Policy.BuiltInType already maps a type to a content class (ICEBERG_COMPACTION to IcebergDataCompactionContent). A real AccessControlContent - action checked against Privilege.Name, scope parsed as a path, role checked to exist - removes most of the opaque-JSON objection on its own.
• Role cascade. The objection was to the write path (POST /objects/role/analyst/policies), which is policy-on-object and is going away. Whether the relation is stored is separate: PolicyManager could maintain the role link as a derived record, same row shape, never user-writable, not in SUPPORTED_METADATA_OBJECT_TYPES_FOR_POLICIES. That gives cascade on role deletion and lookup-by-role without the write path anyone objects to. RoleChange has no rename, so delete-then-recreate under the same name is the only hazard, and this closes exactly that.

[nevzheng]

(1a)

A tag rule can't be a jCasbin policy row

jcasbin_model.conf is p = sub, metadataType, metadataId, act, eft, loaded per-role into allowEnforcer / denyEnforcer. An access rule has no metadataId - it matches a set that changes when someone edits a tag. So either expand the tagged object set into p rows at load time (enforcer untouched, deny composition stays free, but every tag write invalidates the affected roles), or evaluate as a second stage after the enforcer (cheap to load, but policy_effect no longer describes the outcome).

I think you’re conflating API with implementation. What I’m hearing is: (A) can’t be a native p row on today’s Casbin engine.

Agreed. In this codebase Casbin is not the store. We look up stored rules (today: role + securable objects; for (A): policy + tag bind), then Casbin interprets the loaded set. Today’s interpreter only matches p = sub, metadataType, metadataId, act, eft. A tag sentence is not that row. That does not make (A) the wrong API — it means a lookup/compile step sits in front of Casbin, same as today when a schema grant is stored by name/id and Casbin only sees the id.

For this conversation we’re designing the stored rules for (A). You already named the two ways to interpret them (expand vs second stage). I’m not picking that here.

(A) — the API (stored rules):

POST /api/metalakes/prod/policies
{
  "name": "analyst_read_certified",
  "policyType": "system_access_control",
  "content": {
    "action": "SELECT_TABLE",
    "role": { "name": "analyst" },
    "scope": { "catalog": "lakehouse", "schema": "sales" }
  }
}

PUT /api/metalakes/prod/tags/certified/policies/analyst_read_certified
{ "selector": { "type": "ALL_VALUES" } }

policy(analyst_read_certified) =
  type: system_access_control
  role: analyst
  action: SELECT_TABLE
  scope: lakehouse.sales

bind(tag=certified, policy=analyst_read_certified)
tag(object=orders, tag=certified)

Lookup, then today’s interpreter — your expand path:

policies = lookup_policies_bound_to(certified)
for obj in objects_tagged(certified) where obj in scope:
    p, roleId, TABLE, obj.id, SELECT_TABLE, allow
# Casbin then enforce(user, TABLE, obj.id, SELECT_TABLE) as today

no i
That still is metadataId. Tag writes invalidate/reload, as you said. g2 / a stage after enforce is a different interpreter (model change), not today’s engine. The design doc should pick the compiler; that’s not a reason to reject (A).

(1b)

The second matches the sequence diagram above. If it's that one, the doc should state explicitly: does an RBAC DENY beat a tag ALLOW? If the tag stage runs after the enforcer returns false and can flip it to true, a tag rule silently defeats every existing deny. I'd suggest unioning the tag ALLOW with the RBAC allow set and applying RBAC deny last - but the point is that it should be written down rather than fall out of call ordering.

I’m hearing two things in this:

1) How many policy systems do we have?
Today, one: RBAC grants, looked up then interpreted by Casbin. Tags and Policy objects are not a second access engine. Ideally we keep one unified engine that implements both policy-on-tag (A) and current RBAC — lookup both, compile into the same interpreter (the expand path in 1a: tagged objects become ordinary p rows next to role grants; policy_effect is still allow and not deny).

2) If that unified system produces this scenario
(DENY SELECT on secrets, and a tag ALLOW that also matches secrets): general principle — exact rules and rule sets are controlled by the end user. If their system produces this exception state, they should resolve it. It is not up to the engine to silently pick a winner via call order (e.g. flipping a deny to allow).

A perfectly valid — and conservative — action for the system is to highlight the conflict and deny access until the user fixes the rule set. How they fix it (edit the grant, untag, drop a policy) is theirs. Non-blocking for agreeing on (A) now; we still need to design so the engine doesn’t invent an allow.

(1c)

Related, and currently unhandled: JcasbinAuthorizer caches are version-validated on user_meta / group_meta / role_meta.updated_at, and JcasbinChangeListener polls entity_change_log and owner_meta. Tag assignments and policy-tag binds are in neither path, so "tag it and access follows" has no propagation delay or HA story yet.

Valid point on freshness — a complete design should address it. I think this is an implementation detail we can take up when we have a more concrete design or implementation. In general my principle is that cache is a performance optimization and should be considered carefully (correctness first; TTL/poller/invalidation after the store and interpreter are clear).

[nevzheng]

(2a)

Understood, and the Starburst/Databricks comparison is fair. But allow-only means anyone who can apply a tag that has an access policy bound to it can pull objects into that policy. A table owner in lakehouse.finance tags their own table certified, and every analyst can read it. No policy was edited and no grant was written - that's an ordinary action by a different user holding APPLY_TAG, not a governance team authoring a bad policy.

It sounds like we’re aligned on the generalities of tag-based policy governance: the policy is the rule, tagging is attachment, access is what we get when we evaluate.

(2b)

Create authority is separate. We consider that an auth-level question.

Separable from the attachment model, agreed. Not separable from shipping: without it, APPLY_POLICY + APPLY_TAG becomes strictly stronger than MANAGE_GRANTS. ApplyTag.canBindTo accepts only METALAKE and TAG, while ManageGrants accepts all of MANAGE_GRANTS_SUPPORTED_TYPES — so "let this team administer access rules for their own catalog" isn't expressible through the policy and tag privileges alone. Deriving the required authority from content.scope is what makes per-catalog delegation possible.

Agreed — the specifics of create authority for policy tags is something we should work out.

It sounds like we agree that tag-management permissions imply something stronger than today’s MANAGE_GRANTS. I agree with that — that’s a feature of the design. We can discuss it further.

What comes to mind right now: scope — which metadata objects a user is allowed to tag — seems easy; and which policies they are allowed to apply; and whether they have authority to CRUD policies. I would advocate for something simple to start with and adjust from there. We can examine permission models on tags in existing systems as a starting point.

(2c)

If the check is derived from content, it covers four write paths, not one: creating the policy, updating content.action / content.scope or enabled, binding it to a tag, and applying a tag that already carries access policies. The last is the case above and the easiest to miss. One shared "can the caller grant action at scope" helper called from all four keeps them consistent. It can't ride on @AuthorizationExpression — expression() is a compile-time constant per endpoint and can't see the request body — so it has to be an explicit check after deserialization.

I’m hearing you’re considering the permission models for policy tagging. Let’s defer this and look at permission models of other systems to compare.

We’re being specific here — four write paths, a shared helper, @AuthorizationExpression — when we should step back and think about the high-level design of permission. Agreed this is important.

[nevzheng]

(3a) Listing

MetadataAuthzHelper.LIST_SHORT_CIRCUITS skips per-object checks when a parent grant exists and no deny does. A tag ALLOW breaks the mirror case: without a parent grant you can't return an empty list, because children may be allowed by tag. Committing to the inverse lookup - roles to policies to bound tags to tagged objects, intersected with the listed namespace - constrains the resolver to a batch/inverse API rather than just tagsFor(object), so it's worth settling before it's written.

I’m hearing you’re concerned about listing as a CUJ.

User: Alice, role analyst
Goal: see which tables in lakehouse.sales she can query

Given

• USE_CATALOG on lakehouse, USE_SCHEMA on sales
• no schema-level SELECT_TABLE
• policy analyst_read_certified: SELECT_TABLE for analyst in lakehouse.sales, bound to tag certified
• orders is tagged certified; secrets is not

When Alice lists tables in lakehouse.sales

Then orders is in the list and secrets is not

That should be supported. I’m not sure on the specifics and would defer discussion. At this level of discussion, I’m confident there is a correct and performant way to implement this in the application and storage layers. For example a first pass would be to list all tables then evaluate access per table. We can evolve the storage and indexing model from there. In general this is a graph search — principal to roles to policies to tags to objects, intersected with the listed namespace.

(3b) Typed content

Policy.BuiltInType already maps a type to a content class (ICEBERG_COMPACTION to IcebergDataCompactionContent). A real AccessControlContent - action checked against Privilege.Name, scope parsed as a path, role checked to exist - removes most of the opaque-JSON objection on its own.

I’m hearing that we want typed policies like AccessControlContent. I agree this is a fine direction. We can compare against other systems for their policy types. However in general I think a good north star could be custom policies with arbitrary expressions. I don’t see why that couldn’t be supported if we do it well. That’s a thought for later — I don’t want to spend time on it right now.

(3c) Role cascade

The objection was to the write path (POST /objects/role/analyst/policies), which is policy-on-object and is going away. Whether the relation is stored is separate: PolicyManager could maintain the role link as a derived record, same row shape, never user-writable, not in SUPPORTED_METADATA_OBJECT_TYPES_FOR_POLICIES. That gives cascade on role deletion and lookup-by-role without the write path anyone objects to. RoleChange has no rename, so delete-then-recreate under the same name is the only hazard, and this closes exactly that.

Ack — that sounds like a reasonable implementation. We can defer to technical implementation and discussion for a more detailed design.

[laserninja]

Agreed on (A), and agreed the compiler choice is the doc's job rather than a reason to reject the API. Three of my points get smaller in light of that, one gets sharper, and one I'd narrow myself.

1b answers itself if the compiler is expand. JcasbinAuthorizer keeps allowEnforcer and denyEnforcer as separate enforcers over the same model, and authorizeInternal ends with ... && !hasDenyOnAny(...); denyInternalAuthorizer is built with narrowByActiveRoles = false so denies always apply. Compile tag rules into allowEnforcer as ordinary p rows and an RBAC DENY on secrets already beats a tag ALLOW, with no new precedence rule and nothing resolved by call order. My question only exists in the second-stage design. That's a real argument for expand and worth stating as one.

On "highlight the conflict and deny": same answer, different cost. Deny-wins gets it for free from the two enforcers; conflict surfacing additionally requires the engine to detect the allow/deny pair. Both are worth having, but as semantics-plus-lint rather than one instead of the other.

Expansion is over inherited tags, not directly assigned ones. Tag.inherited() is in the API and tag listing returns ancestors' tags, so objects_tagged(certified) has to mean every object whose effective tags include certified - tagging lakehouse expands to every table under it. That is the cost driver for expand, and I think it decides whether expand is viable, so it belongs in the comparison rather than in the implementation.

Which is the one reason I'd keep 1c in the doc. Agreed cache is normally an optimization, and under the second-stage design your principle holds exactly. Under expand the compiled rows are the authorization state, so a tag write that doesn't invalidate is wrong rather than stale-but-conservative - an untag leaves access granted. So the compiler choice decides whether freshness is a perf ticket or a correctness constraint. That's all I'd want written down.

Listing - narrower than I put it. Re-reading MetadataAuthzHelper, the short-circuit is allow-only: on a hit it returns the full list, on a miss it falls through to the per-object loop. Tag allows don't break it, so scratch that framing. The real point is smaller and matches your first pass: the fallback loop now needs effective tags plus bound policies per object, and there's already a batching seam for exactly that - preloadToCache and preloadOwner run immediately before it. A third preload over the listed identifiers is the whole fix. Worth writing only because it means the resolver needs a batch entry point; a per-object tagsFor(object) is what would make it N+1 and be awkward to walk back later.

2b/2c - happy to defer the model, with one gate. ApplyTag.canBindTo accepts only METALAKE and TAG today, so "stronger than MANAGE_GRANTS" currently means metalake-wide grant authority for anyone who can tag anything. Object-scoped APPLY_TAG before access policies may bind to tags, and the rest of the permission model can follow at its own pace.

On 3b, no objection to arbitrary expressions as a north star - typed AccessControlContent first is forward-compatible with it, and it makes the create-authority check derivable in the meantime.

[nevzheng]

(4b)

1b answers itself if the compiler is expand. JcasbinAuthorizer keeps allowEnforcer and denyEnforcer as separate enforcers over the same model, and authorizeInternal ends with ... && !hasDenyOnAny(...); denyInternalAuthorizer is built with narrowByActiveRoles = false so denies always apply. Compile tag rules into allowEnforcer as ordinary p rows and an RBAC DENY on secrets already beats a tag ALLOW, with no new precedence rule and nothing resolved by call order. My question only exists in the second-stage design. That's a real argument for expand and worth stating as one.

On "highlight the conflict and deny": same answer, different cost. Deny-wins gets it for free from the two enforcers; conflict surfacing additionally requires the engine to detect the allow/deny pair. Both are worth having, but as semantics-plus-lint rather than one instead of the other.

I’m advocating for failing closed if there’s an ambiguity. If two rules conflict, don’t pick a winner via precedence or call order — surface an ambiguous policy error and deny. “Which policy should have precedence” gets confusing. That’s safe, effective, conservative.

I agree that precedence rules can also be correct, but they need to be specified and tested well. Even then, I would still expect surprising behavior. We can try to compare to other systems.

(4c)

Expansion is over inherited tags, not directly assigned ones. Tag.inherited() is in the API and tag listing returns ancestors' tags, so objects_tagged(certified) has to mean every object whose effective tags include certified - tagging lakehouse expands to every table under it. That is the cost driver for expand, and I think it decides whether expand is viable, so it belongs in the comparison rather than in the implementation.

Reacting to this CUJ — what I’m hearing: Pat tags catalog lakehouse certified, and child tables inherit that tag, so they can match the access policy (still subject to scope). This sounds correct and reasonable if the end user configures it like this. I’ll leave it to implementation to decide.

(4d)

Which is the one reason I'd keep 1c in the doc. Agreed cache is normally an optimization, and under the second-stage design your principle holds exactly. Under expand the compiled rows are the authorization state, so a tag write that doesn't invalidate is wrong rather than stale-but-conservative - an untag leaves access granted. So the compiler choice decides whether freshness is a perf ticket or a correctness constraint. That's all I'd want written down.

What I’m hearing is you’re thinking that auth state could be part of caching, and that could affect correctness. In my opinion, we should just proceed without caching for now. And in general, invalidation vs TTL could just be system properties that we trade off. I don’t see this as super important at this stage. We need to define higher-level requirements to constrain this choice. My immediate thought is TTL is simpler.

(4e)

Listing - narrower than I put it. Re-reading MetadataAuthzHelper, the short-circuit is allow-only: on a hit it returns the full list, on a miss it falls through to the per-object loop. Tag allows don't break it, so scratch that framing. The real point is smaller and matches your first pass: the fallback loop now needs effective tags plus bound policies per object, and there's already a batching seam for exactly that - preloadToCache and preloadOwner run immediately before it. A third preload over the listed identifiers is the whole fix. Worth writing only because it means the resolver needs a batch entry point; a per-object tagsFor(object) is what would make it N+1 and be awkward to walk back later.

It sounds like you’re weighing the listing design more. Let’s table it and defer to the doc or implementation. Come back if there’s a concrete design split or choice to be made here.

(4f)

2b/2c - happy to defer the model, with one gate. ApplyTag.canBindTo accepts only METALAKE and TAG today, so "stronger than MANAGE_GRANTS" currently means metalake-wide grant authority for anyone who can tag anything. Object-scoped APPLY_TAG before access policies may bind to tags, and the rest of the permission model can follow at its own pace.

Let’s defer this to the design doc until we think about it more. I think the comparative analysis will be clarifying.

(4g)

On 3b, no objection to arbitrary expressions as a north star - typed AccessControlContent first is forward-compatible with it, and it makes the create-authority check derivable in the meantime.

Ack, agreed.

Nevin
Sent from my 🤖 (Cursor)

[nevzheng]

@bharos @roryqi @laserninja

We’ve gone fairly deep on evaluation, create authority, and listing. I’d like to zoom out to process for a moment.

Who will be writing the design proposal? What’s in scope for that doc vs later? What are the milestones, expected timeline, and priority?

Thanks.

Nevin
Sent from my 🤖 (Cursor)