Tools transition prepare release (#91)

* Add prepare release scripts and its tests
* Add pip tools for requirement management
* Address review notes for update_package_properties script

.github/workflows/tool-tests.yml

@@ -2,6 +2,9 @@ name: Tool Tests
 
 env:
   GH_TOKEN: ${{ secrets.GH_TOKEN }}
+  MICROSOFT_EMAIL: [email protected]
+  USER_NAME: Gurkan Indibay
+  MAIN_BRANCH: all-citus
 
 on:
   push:
@@ -25,10 +28,14 @@ jobs:
     steps:
       - name: Checkout repository
         uses: actions/checkout@v2
+      - name: Define git credentials
+        run: git config --global user.email "${MICROSOFT_EMAIL}"&& git config --global user.name "${USER_NAME}"
       - name: Install package dependencies
         run: sudo apt install libcurl4-openssl-dev libssl-dev
       - name: Install python requirements
-        run: python -m pip install -r python/requirements.txt
-      - name: Execute unit tests
-        run:  python -m pytest -q python/tests/test_update_package_properties.py
+        run: python -m pip install -r packaging_automation/requirements.txt
+      - name: Execute unit tests for "Update Package Properties"
+        run:  python -m pytest -q packaging_automation/tests/test_update_package_properties.py
+      - name: Execute unit tests for "Prepare Release"
+        run: python -m pytest -q packaging_automation/tests/test_prepare_release.py
 

bash/update_package_properties.sh

@@ -26,13 +26,21 @@ commit_message="Bump to ${PRJ_NAME} ${PRJ_VER}"
 
 git checkout -b "${pr_branch_name}"
 
-python tools/python/update_package_properties.py --gh_token "${GH_TOKEN}" --prj_name "${PRJ_NAME}" --prj_ver "${PRJ_VER}" --tag_name "${TAG_NAME}" --fancy "${FANCY}" \
-  --fancy_ver_no "${FANCY_VERSION_NO}" --email "${MICROSOFT_EMAIL}" --name "${NAME}" --date "$(date '+%Y.%m.%d %H:%M:%S %z')" --exec_path "$(pwd)"
+python tools/python/update_package_properties.py  --gh_token "${GH_TOKEN}" \
+                                                  --prj_name "${PRJ_NAME}" \
+                                                  --tag_name "${TAG_NAME}" \
+                                                  --fancy "${FANCY}" \
+                                                  --fancy_ver_no "${FANCY_VERSION_NO}" \
+                                                  --email "${MICROSOFT_EMAIL}" \
+                                                  --name "${NAME}" \
+                                                  --date "$(date '+%Y.%m.%d %H:%M:%S %z')" \
+                                                  --exec_path "$(pwd)"
 
 git commit -a -m "${commit_message}"
 
 echo "{\"title\":\"${commit_message}\", \"head\":\"${pr_branch_name}\", \"base\":\"${main_branch_name}\"}"
 
 git push origin "${pr_branch_name}"
 
-curl -g -H "Accept: application/vnd.github.v3.full+json" -X POST --user "${GH_TOKEN}:x-oauth-basic" -d "{\"title\":\"${commit_message}\", \"head\":\"${pr_branch_name}\", \"base\":\"${main_branch_name}\"}" https://api.github.com/repos/citusdata/packaging/pulls
+curl -g -H "Accept: application/vnd.github.v3.full+json" -X POST --user "${GH_TOKEN}:x-oauth-basic" -d \
+  "{\"title\":\"${commit_message}\", \"head\":\"${pr_branch_name}\", \"base\":\"${main_branch_name}\"}" https://api.github.com/repos/citusdata/packaging/pulls

packaging_automation/README.md

@@ -0,0 +1,79 @@
+# **Prepare Release Usage**
+
+prepare-release.py script performs the  pre-packaging configurations in citus/citus-enterprise projects.
+
+## Installation
+
+Before using script, you need to make sure that Python > 3.8 is installed in your system.
+
+### Clone Tools Repository
+
+git clone https://github.com/citusdata/tools.git
+
+Enter 'tools' directory
+
+``` console
+cd tools
+```
+
+### Install Required Python Libraries
+
+Verify pip installation 
+
+``` console
+python -m pip --version
+```
+Output should be like following
+
+``` console
+pip 21.1.2 from /home/vagrant/.local/lib/python3.8/site-packages/pip (python 3.8)
+```
+
+If you get error, you should first install pip
+``` console
+sudo apt install python3-pip
+```
+Install the required libraries to execute the script
+``` console
+python -m pip install -r packaging_automation/requirements.txt
+```
+If all the steps above completed successfully , you are ready for script execution
+
+## Script Usage
+
+Script can be used for either major release (i.e. third digit of release is '0' e.g. 10.1.0)  or 
+patch release (i.e. third digit of release is other than '0' e.g. 10.0.4).
+
+### Available flags
+
+**--gh_token:** Personal access token that is authorized to commit citus/citus-enterprise projects. (Required)
+
+**--prj_name:** Project to be released. Allowed values 'citus' and 'citus-enterprise (Required)
+
+**--prj_ver:** Upcoming version to be used for release. should include three level of digits separated by dots, e.g: 10.0.1
+(Required)
+
+**--main_branch:** Branch to be used as base to be used for configuration changes. There is no need for base scenario. 
+This flag can be used for testing purposes. If not used, default branch value is used; i.e. for 'citus' 'master, for 'citus-enterprise' 'enterprise-master'
+
+**--is_test:** If used, branches would not be pushed remote repository and created release branches would be prefixed with 'test'. Default value is False
+
+**--cherry_pick_enabled:** Available only for patch release. If used, --earliest_pr_date flag also should be used.Gets all PR's with 'backport' label created after earliest_pr_date
+
+**--earliest_pr_date:** Used with --cherry-pick-enabled flag. Date format is 'Y.m.d' e.g 2012.01.21. PR's merged after this date would be listed and cherry-picked.
+
+**--schema_version:** Available only for patch release. If used, schema version in citus.control file would be updated.
+
+###Example Usage
+
+####Major
+``` console
+python -m  packaging_automation.prepare_release --gh_token <your-personal-token> --prj_name citus --prj_ver 10.1.0 
+```
+#### Patch
+``` console
+python -m  packaging_automation.prepare_release --gh_token <your-personal-token> --prj_name citus-enterprise --prj_ver 10.0.4 --schema_version 10.0-5
+```
+
+
+