jupyternotebooks/calm-requirements-api.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SAP Cloud ALM Project and Task API Demo for Requirements\n",
    "\n",
    "This notebook contains examples of SAP Business Hub API calls for SAP Cloud ALM for Implementation. The code cells below show how to use the Project and Task APIs to read and update Requirements.\n",
    "\n",
    "The API information and specification is available here:\n",
    "\n",
    "* https://api.sap.com/package/SAPCloudALM/rest - SAP Cloud ALM\n",
    "* https://api.sap.com/api/CALM_PJM/overview - SAP Cloud ALM Projects\n",
    "* https://api.sap.com/api/CALM_TKM/overview - SAP Cloud ALM Tasks\n",
    "\n",
    "Please note the license and other terms and conditions contained in this notebook's repository: https://github.com/SAP-samples/cloud-alm-api-examples\n",
    "\n",
    "## Python Dependencies Required\n",
    "\n",
    "In order to run the samples in this notebook, install the following dependencies:\n",
    "\n",
    "* Jupyter integration in Visual Studio Code: https://code.visualstudio.com/docs/python/jupyter-support \n",
    "* Python 3, a recent version, is Required. Python 3.8 was used here.\n",
    "* Requests - for handling HTTP GET/POST/PATCH/DELETE Requests - https://docs.python-requests.org/en/latest/user/install/#install\n",
    "* Requests-OAuthlib - for authentication with requests - https://requests-oauthlib.readthedocs.io/en/latest/index.html#installation\n",
    "* Pandas - Python data analysis - https://pandas.pydata.org/docs/getting_started/install.html\n",
    "* Plotly - for plotting interactive charts - https://plotly.com/python/getting-started/\n",
    "\n",
    "## APIs called\n",
    "\n",
    "API to get projects: https://<tenant url\\>/api/imp-pjm-srv/v1/projects\n",
    "\n",
    "API to get tasks, including Requirements: https://<tenant url\\>/api/imp-tkm-srv/v1/tasks?projectId=<project ID\\>\n",
    "\n",
    "## Requirement Specifics\n",
    "\n",
    "Requirements are treated exactly like tasks, and are accessed through the task API. There are several additional attributes relevant for requirements.\n",
    "\n",
    "* Requirement task type = `CALMREQU`\n",
    "* `subStatus` attribute for the Requirement Status\n",
    "* `approvalState` = `APPROVED` if the requirement has been approved\n",
    "\n",
    "Requirements have both a status, and corresponding to each status value, a subStatus.\n",
    "\n",
    "* `CREATED` = In Specification, main status = CIPREQUOPEN\n",
    "* `IN_REALIZATION` = In Realization, main status =  CIPREQUINP\n",
    "* `APPROVED_FOR_DEPLOYMENT` = In Deployment, main status = CIPREQUINP\n",
    "* `CONFIRMED` = Deployed, main status = CIPREQUCLOSE\n",
    "* `NOT_PLANNED` = Not Planned, main status = CIPREQUNO\n",
    "\n",
    "These values will be extended"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "---\n",
    "\n",
    "## Authentication information\n",
    "\n",
    "You must create a python module file called `apidata.py` and put the information specific to your tenant there. This inclduds\n",
    "\n",
    "* OAuth2 client ID and secret\n",
    "* Token url\n",
    "* Base URL for API calls\n",
    "\n",
    "Get client ID and secret variables from an external module: this information is senstive.\n",
    "\n",
    "These items can be retrieved from the BTP Cockpit \n",
    "\n",
    "### Format of module apidata.py for import\n",
    "\n",
    "```python\n",
    "ptm_all_client_id = r'get your client ID from BTP Cockpit'\n",
    "ptm_all_client_secret = r'get your client secret from BTP Cockpit'\n",
    "token_url = 'your token url'\n",
    "base_url = 'your base url'\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import apidata as ad\n",
    "\n",
    "client_id = ad.ptm_all_client_id\n",
    "client_secret = ad.ptm_all_client_secret\n",
    "token_url = ad.token_url\n",
    "base_url = ad.base_url"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get token for authentication\n",
    "\n",
    "Call OAuth token API with credential information. Add the resulting header to all requests.\n",
    "\n",
    "See Requests-OAuthlib documentation for Backend Application Flow:\n",
    "\n",
    "* https://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#backend-application-flow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "from requests_oauthlib import OAuth2Session\n",
    "from oauthlib.oauth2 import BackendApplicationClient\n",
    "\n",
    "client = BackendApplicationClient(client_id=client_id)\n",
    "oauth = OAuth2Session(client=client)\n",
    "token = oauth.fetch_token(token_url=token_url, client_id=client_id,\n",
    "        client_secret=client_secret)\n",
    "\n",
    "hed = {'Authorization': 'Bearer ' + token['access_token']}        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "---\n",
    "\n",
    "## Perform GET request to retrieve list of all projects\n",
    "\n",
    "Expected response: \"200 OK\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(base_url + '/api/calm-projects/v1/projects', headers=hed)\n",
    "\n",
    "print(response.status_code, response.reason)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parse JSON into Pandas Dataframe\n",
    "\n",
    "This takes the data returned from the projects API, which is in JSON format, and places it into a dataframe for further processing and analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.json_normalize(response.json())\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get a Project Name \n",
    "\n",
    "You may wish to adjust the code below to set a project name of your choosing, for example 'My API Test Project'. In this sample, we'll just take the first project in the list.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample_project_name = df.iloc[0]['name'] # First project in the list\n",
    "sample_project_name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Override Sample Project Name for Demo\n",
    "\n",
    "I'm setting my own project name here, beause I have one ready for my demos. You can do the same if you don't want to use the first project"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment and change to the name of a project you would like to use:\n",
    "\n",
    "#sample_project_name = 'API Demo Project'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get ID of a Selected Project\n",
    "\n",
    "Read ID of project 'Show & Tell 2021-Apr-30' from dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "stPrjID = df.loc[df['name'] == sample_project_name]\n",
    "show_tell_prj = stPrjID['id'].values[0]\n",
    "print('Project:', stPrjID['name'].values[0], 'ID:', show_tell_prj)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "---\n",
    "\n",
    "## Read Task list from selected project\n",
    "\n",
    "Use GET request from tasks API with selected project ID as input parameter.\n",
    "\n",
    "Expected response \"200 OK\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "taskURL = base_url + '/api/calm-tasks/v1/tasks?projectId=' + show_tell_prj\n",
    "\n",
    "response = requests.get(taskURL, headers=hed)\n",
    "\n",
    "print(response.status_code, response.reason)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parse project task list JSON into dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_stTasks = pd.json_normalize(response.json())\n",
    "\n",
    "df_stTasks.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_stTasks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Barchart plot of tasks by type\n",
    "\n",
    "Use built-in matplotlib of Pandas for quick data exploration\n",
    "\n",
    "Requirements have type `CALMREQU`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_stTasks.type.value_counts().plot.pie(title=\"Task Type\", figsize=(18,8))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze Requirements by substatus, approvalState\n",
    "\n",
    "Count Requirements in each substatus (called \"Requirement Status\")\n",
    "\n",
    "Count approved Requirements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_stTasks.groupby(by=['subStatus'])['id'].count().reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_stTasks.subStatus.value_counts().plot.pie(title=\"Requirement Status\", figsize=(18,8))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_stTasks.groupby(by=['approvalState'])['id'].count().reset_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "---\n",
    "\n",
    "## Create Requirement\n",
    "\n",
    "* Endpoint: `https://<tenant url>/api/imp-tkm-srv/v1/tasks`\n",
    "\n",
    "* Type: Post\n",
    "\n",
    "Append a timestamp to the end of the new task title for easy identification in UI.\n",
    "\n",
    "Expected response `201 Created`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime\n",
    "\n",
    "isoNow = datetime.datetime.now().isoformat()\n",
    "\n",
    "taskCreateURL = base_url + '/api/calm-tasks/v1/tasks'\n",
    "\n",
    "taskData = {\n",
    "    \"projectId\": show_tell_prj,\n",
    "    \"title\": \"Notebook API test Requirement \" + isoNow,\n",
    "    \"type\": \"CALMREQU\",\n",
    "    \"description\": \"test description. hello world\",\n",
    "    \"assigneeId\": \"test.user@example.com\"\n",
    "}\n",
    "\n",
    "response = requests.post(taskURL, headers=hed, json=taskData)\n",
    "\n",
    "print(response.status_code, response.reason)\n",
    "\n",
    "newTaskID = response.json()['id']\n",
    "print(\"New Requirement ID:\", newTaskID)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "---\n",
    "\n",
    "## Get Requirement Details\n",
    "\n",
    "Use GET request to read details of requirement just created\n",
    "\n",
    "* Endpoint: `https://<tenant url>/rest/tkm/v1/tasks/{taskGuid}`\n",
    "\n",
    "* Type: Get\n",
    "\n",
    "Expected response: `200 OK`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "taskURL = base_url + '/api/calm-tasks/v1/tasks/' + newTaskID\n",
    "\n",
    "response = requests.get(taskURL, headers=hed)\n",
    "\n",
    "print(response.status_code, response.reason)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Save and print response JSON"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "readTask = response.json()\n",
    "readTask"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "---\n",
    "\n",
    "## Modify Requirement\n",
    "\n",
    "Use PATCH method to update Requirement attributes\n",
    "\n",
    "Set the main status to `CIPREQUINP`\n",
    "\n",
    "Expected response: `200 OK`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "taskToModify = {}\n",
    "\n",
    "taskToModify['title'] = readTask['title'] + ' mod'\n",
    "taskToModify['dueDate'] = '2022-04-09'\n",
    "taskToModify['status'] = 'CIPREQUINP'\n",
    "taskToModify['description'] = 'This is a new <b>Description</b>. It replaces the old description'"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set subStatus, approvalState\n",
    "\n",
    "Set the `subStatus` and `approvalState` attributes, which are the Requirement specific attributes\n",
    "\n",
    "Set a corresponding subStatus for `CIPREQUINP` to `TO_BE_APPROVED` - \"In Approval\"\n",
    "\n",
    "Set the approvalState to `READY_4_APPR` - \"Ready for Approval\"\n",
    "\n",
    "See the schema definition in the SAP API Business Hub for allowed status, sub status, and approval state values. \n",
    "\n",
    "* https://api.sap.com/api/CALM_TKM/schema"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "taskToModify['subStatus'] = 'TO_BE_APPROVED'\n",
    "taskToModify['approvalState'] = 'READY_4_APPR'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "taskURL = base_url + '/api/calm-tasks/v1/tasks/' + newTaskID\n",
    "\n",
    "response = requests.patch(taskURL, headers=hed, json=taskToModify)\n",
    "\n",
    "print(response.status_code, response.reason)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "---\n",
    "\n",
    "## Delete Requirement\n",
    "\n",
    "DELETE method - the system **forbids** deletion of **approved** Requirements\n",
    "\n",
    "Expected response: `204 No Content` if the Requirement is not approved, `400 Bad Request` if the Requirement is approved.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "taskURL = base_url + '/api/calm-tasks/v1/tasks/' + newTaskID\n",
    "\n",
    "# Uncomment code below to test deletion\n",
    "\n",
    "#response = requests.delete(taskURL, headers=hed)\n",
    "\n",
    "#print(response.status_code, response.reason)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.7"
  },
  "metadata": {
   "interpreter": {
    "hash": "2db524e06e9f5f4ffedc911c917cb75e12dbc923643829bf417064a77eb14d37"
   }
  },
  "orig_nbformat": 2,
  "vscode": {
   "interpreter": {
    "hash": "be48207b29b5254ca9bf80bb7ffab8a957eac95ff018db1d599cb6629665e664"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}