Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Programmatic yanking #16912

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

warsaw
Copy link
Contributor

@warsaw warsaw commented Oct 17, 2024

Closes #12708

This adds "programmatic yanking", essentially an API for yanking and unyanking releases, along with setting the yank reason for yanked releases. It lays the groundwork for a REST-ish API for other programmatic access to warehouse data.

Highlights

  • Add a new permission for modifying releases called Permissions.APIModify. This permission is given to project owners, but not maintainers.
  • Adds a new route with name api.rest.release. The thinking here is that api. is the relevant root route name, and using api.rest.* gives us room to expand the REST API in the future. api.rest.release in that releases can be a root resource in this theoretical REST resource hierarchy. Eventually we may want to provide access to releases through project resources, but for now we only need to manipulate releases so keep the scope narrow. But this also keeps our options open for the future.
  • Routes /api/projects/{name}/{version} to this new route. User-facing API calls should go through /api/ and because we're essentially accessing project releases through the /api/projects root resource, that's the route pattern I've chosen. I don't know that it makes sense to expose an /api/releases root resource since how do you reference a release that doesn't go through its project?
  • Add two new API-centric views. The first, called json_release_get() responds only to GET on the release resource. API calls to this endpoint method do not need to be authenticated. Obviously, it returns the JSON representation of the release resource.
  • The second, called json_release_modify() responds only to PATCH methods, and only authenticated project owners can call this method. Why did I choose PATCH? Again, it's to keep the scope narrow for now. Because I'm trying to implement only yanking, this PR doesn't provide full resource replacement, i.e. PUT. I also do not like POST-centric "action" methods; they don't document or scale well, and aren't RESTful. So why not do things right, right from the start? The functionality here is a "partial" resource change, where we only support modifying the yanked and yanked_reason properties on the release resource, and even those properties are optional in the request. This is exactly what PATCH is designed for! In the future, we could support partial updates for other release properties using the same route, method, and view.

Semantics

  • You can set the yanked property to True or False. It must be a boolean.
  • If yanked is being set to False, then yanked_reason is ignored, and in fact any previous yanked_reason is deleted (i.e. set to the empty string).
  • If yanked is set to True, then you can optionally also set the yanked_reason to any string. JSON null is equivalent to setting it to the empty string.
  • Setting yanked is optional; you can set only the yanked_reason. If the release is in the unyanked state, i.e. yanked is already set to False, then yanked_reason is ignored.
  • However, if the release is in the yanked state, i.e. yanked is already set to True, then you can overwrite the yanked_reason.

Other changes

  • I found a comment in the code about an issue which is now closed. It's unrelated to this PR so I just added a note that the code in question should be reconsidered in the future.
  • Fixed a typo.

Open questions

  • I am not setting the openapi or api_version attributes in the @view_config(); should we do this, and if so, should we do this now or in a follow up PR? My thinking is yeah, maybe! I think we should also explicitly define a RESTful API version, but whether api-v1 is the right value for that or not is uncertain. I'm not touching the openapi.yaml file either for now.
  • I think this is the first general purpose user-centric API we're adding. I definitely intend to build PEP 694 support on similar concepts, so please keep in mind future RESTful APIs when thinking about the foundations laid down here. I think we're in good shape for that effort.
  • I should probably add documentation! We will want to document any RESTful API to warehouse/PyPI, but I'm not currently certain about the organization and tooling for that. Do we need that in this PR or in a follow up?
  • There's some opportunity to refactor a few things (see the TODO) comment, but should we do that now or in a follow up PR?

POST and GET to API endpoints works, but POST is unauthenticated.
* HTTP PATCH to provide a REST-ish API for modifying the release resource.
  Since we're currently only allowing modification of `yank` and
  `yank_reason`, and not the entire resource representation, I'm not using
  PUT.  I don't particularly like POST actions, so that's why I'm not using
  that method either.
* HTTP GET on the release returns the full release representation.
* Create a new route: api.rest.release; `api.` because these are
  client-accessible API methods.  `api.rest.` because I have a dream of a much
  fuller RESTful API for warehouse.
* Added a new APIModify permission.  This is given to user principles who are
  administrators of a project, not to uploaders.

Semantics:
* The PATCH request body can provide `yanked` and `yanked_reason` attributes.
* `yanked` must be a boolean
* `yanked_release` must be a string or null
* Both attributes are optional
* If neither are given, nothing changes
* If `yanked` is given then the release's `yanked` status is changed to the
  given boolean value
* If the release ends up unyanked, then the request's `yanked_reason` attribute
  is ignored and the `yanked_reason` is unconditionally reset to the empty
  string.
* If the release ends up yanked, then the request's `yanked_reason` is assigned
  to the JSON request attribute, if given.  If not given, the `yanked_reason`
  is not changed.

Note that this means if the release is *already* yanked, and either the
request provides `"yanked": true` or no `yanked` attribute, *and*
`yanked_reason` is given, the request will update the `yanked_reason`.

Missing:
* Tests
* Documentation
* Probably some refactoring
* openapi/api_version view attributes and integration
@warsaw warsaw requested a review from a team as a code owner October 17, 2024 20:11
@warsaw
Copy link
Contributor Author

warsaw commented Oct 17, 2024

I'm going to let CI complete before I click the "Update branch" button.

release.yanked_reason = "" if yanked_reason is None else yanked_reason

# Return the new JSON body of the release object.
return json_release(release, request)

Check warning

Code scanning / CodeQL

Reflected server-side cross-site scripting Medium

Cross-site scripting vulnerability due to a
user-provided value
.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a real issue? We use json_release(release, request) already above, and other views use it too, so it seems like it's a false positive.

@warsaw
Copy link
Contributor Author

warsaw commented Oct 17, 2024

I should also mention that a fun way to test this locally at the command line is to make serve on the branch, log in with the dstufft user using the instructions in the warehouse docs (you may need to make totp), and create an API token. I used the token name testapi. You'll want to copy the generated token somewhere handy.

Now, you can issue curl commands that access the new APIs. E.g. to retrieve the entire release resource representation, do this (you don't need to be authenticated).

$ curl -X GET http://localhost/api/projects/dstufft-testpkg/21.0 -H "Content-Type: application/json"

I like to pipe the output through jq to get a pretty printed JSON representation.

Keep an eye on the yanked and yanked_reason properties.

Now, to yank that release and set a reason do this:

$ curl -X PATCH http://localhost/api/projects/dstufft-testpkg/21.0 -H "Content-Type: application/json" -u __token__:pypi-<fill me in> -d '{"yanked": true, "yanked_reason": "because"}'

Now that the release has been yanked, you can change the reason:

$ curl -X PATCH http://localhost/api/projects/dstufft-testpkg/21.0 -H "Content-Type: application/json" -u __token__:pypi-<fill me in> -d '{"yanked_reason": "no reason"}'

You can unyank the release by doing:

curl -X PATCH http://localhost/api/projects/dstufft-testpkg/21.0 -H "Content-Type: application/json" -u __token__:pypi-<fill me in> -d '{"yanked": false}'

Feel free to throw bogus data at the -d option to see if you can break it 😄 - the implementation should be robust enough to return 400 Bad Request errors in that case. It'll also throw 403 Forbidden errors if you try to PATCH with invalid or missing credentials.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make it possible to programmatically yank a release
1 participant