Skip to content

Commit 84720e4

Browse files
oshovaloshoval
committed
create_ticket, Initial version
Script now supports listing of sig/network open issues Signed-off-by: Or Shoval <[email protected]>
1 parent cb03a1a commit 84720e4

9 files changed

+334
-130
lines changed

‎.gitignore

Lines changed: 3 additions & 129 deletions
Original file line numberDiff line numberDiff line change
@@ -1,129 +1,3 @@
1-
# Byte-compiled / optimized / DLL files
2-
__pycache__/
3-
*.py[cod]
4-
*$py.class
5-
6-
# C extensions
7-
*.so
8-
9-
# Distribution / packaging
10-
.Python
11-
build/
12-
develop-eggs/
13-
dist/
14-
downloads/
15-
eggs/
16-
.eggs/
17-
lib/
18-
lib64/
19-
parts/
20-
sdist/
21-
var/
22-
wheels/
23-
pip-wheel-metadata/
24-
share/python-wheels/
25-
*.egg-info/
26-
.installed.cfg
27-
*.egg
28-
MANIFEST
29-
30-
# PyInstaller
31-
# Usually these files are written by a python script from a template
32-
# before PyInstaller builds the exe, so as to inject date/other infos into it.
33-
*.manifest
34-
*.spec
35-
36-
# Installer logs
37-
pip-log.txt
38-
pip-delete-this-directory.txt
39-
40-
# Unit test / coverage reports
41-
htmlcov/
42-
.tox/
43-
.nox/
44-
.coverage
45-
.coverage.*
46-
.cache
47-
nosetests.xml
48-
coverage.xml
49-
*.cover
50-
*.py,cover
51-
.hypothesis/
52-
.pytest_cache/
53-
54-
# Translations
55-
*.mo
56-
*.pot
57-
58-
# Django stuff:
59-
*.log
60-
local_settings.py
61-
db.sqlite3
62-
db.sqlite3-journal
63-
64-
# Flask stuff:
65-
instance/
66-
.webassets-cache
67-
68-
# Scrapy stuff:
69-
.scrapy
70-
71-
# Sphinx documentation
72-
docs/_build/
73-
74-
# PyBuilder
75-
target/
76-
77-
# Jupyter Notebook
78-
.ipynb_checkpoints
79-
80-
# IPython
81-
profile_default/
82-
ipython_config.py
83-
84-
# pyenv
85-
.python-version
86-
87-
# pipenv
88-
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
89-
# However, in case of collaboration, if having platform-specific dependencies or dependencies
90-
# having no cross-platform support, pipenv may install dependencies that don't work, or not
91-
# install all needed dependencies.
92-
#Pipfile.lock
93-
94-
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
95-
__pypackages__/
96-
97-
# Celery stuff
98-
celerybeat-schedule
99-
celerybeat.pid
100-
101-
# SageMath parsed files
102-
*.sage.py
103-
104-
# Environments
105-
.env
106-
.venv
107-
env/
108-
venv/
109-
ENV/
110-
env.bak/
111-
venv.bak/
112-
113-
# Spyder project settings
114-
.spyderproject
115-
.spyproject
116-
117-
# Rope project settings
118-
.ropeproject
119-
120-
# mkdocs documentation
121-
/site
122-
123-
# mypy
124-
.mypy_cache/
125-
.dmypy.json
126-
dmypy.json
127-
128-
# Pyre type checker
129-
.pyre/
1+
status.cfg
2+
__pycache__
3+
secret.txt

‎Dockerfile

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
FROM fedora
2+
3+
# Install packages
4+
RUN dnf install -y python3 \
5+
git \
6+
pip
7+
8+
RUN pip install requests
9+
RUN pip install jira
10+
11+
CMD [“echo”, “Hello World!”]
12+

‎LICENSE

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -186,7 +186,7 @@
186186
same "printed page" as the copyright notice for easier
187187
identification within third-party archives.
188188

189-
Copyright [yyyy] [name of copyright owner]
189+
Copyright [2021] [The KubeVirt Authors]
190190

191191
Licensed under the Apache License, Version 2.0 (the "License");
192192
you may not use this file except in compliance with the License.

‎README.md

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,77 @@
11
# github2jira
22
Scrap github issues and create Jira tickets
3+
4+
1. Create github token https://github.com/settings/tokens
5+
2. `export GITHUB_TOKEN="your_token"`
6+
3. Run `./create_ticket.py` in order to list sig/network issues
7+
8+
# Build docker image for the script
9+
10+
1. Run `docker build -f Dockerfile -t quay.io/oshoval/github:latest .`
11+
once its done, push it to quay, or rename and push a local registry.
12+
13+
# Run as k8s payload
14+
15+
1. Create secret.txt with the following contents
16+
`export GITHUB_TOKEN=<GIT_TOKEN>`
17+
18+
2. Create a configmap for the txt file
19+
`kubectl create configmap git-token --from-file=secret.txt`
20+
21+
3. Create a pod (good for testing)
22+
```
23+
apiVersion: v1
24+
kind: Pod
25+
metadata:
26+
name: github
27+
namespace: default
28+
spec:
29+
containers:
30+
- image: quay.io/oshoval/github:latest
31+
name: github
32+
command:
33+
- /bin/bash
34+
- -c
35+
- sleep infinity
36+
volumeMounts:
37+
- name: configs
38+
mountPath: /app/secret.txt
39+
subPath: secret.txt
40+
volumes:
41+
- name: configs
42+
configMap:
43+
name: git-token
44+
```
45+
46+
or a CronJob
47+
```
48+
apiVersion: batch/v1
49+
kind: CronJob
50+
metadata:
51+
name: cron-github
52+
spec:
53+
schedule: "0 */1 * * *"
54+
successfulJobsHistoryLimit: 1
55+
failedJobsHistoryLimit: 1
56+
jobTemplate:
57+
spec:
58+
template:
59+
spec:
60+
containers:
61+
- name: github
62+
image: quay.io/oshoval/github:latest
63+
imagePullPolicy: IfNotPresent
64+
command:
65+
- /bin/sh
66+
- -c
67+
- date; source /app/secret.txt; ./github2jira/create_ticket.py
68+
volumeMounts:
69+
- name: configs
70+
mountPath: /app/secret.txt
71+
subPath: secret.txt
72+
restartPolicy: Never
73+
volumes:
74+
- name: configs
75+
configMap:
76+
name: git-token
77+
```

‎create_ticket.py

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
#!/usr/bin/env python3
2+
3+
import os
4+
import json
5+
import sys
6+
from datetime import datetime
7+
import time
8+
import argparse
9+
10+
from jiralib import Jira
11+
from githublib import Github
12+
13+
# process upto this x weeks back
14+
max_delta_weeks = 2
15+
SECONDS_PER_WEEK = 604800
16+
# how many tickets can be opened on each cycle
17+
flood_protection_limit = 3
18+
19+
debug = 0
20+
dry_run = False
21+
22+
def check_time(epoch_time_now, created_at, max_delta):
23+
epoch = int(datetime.strptime(created_at, "%Y-%m-%dT%H:%M:%SZ").timestamp())
24+
return epoch_time_now - epoch < max_delta
25+
26+
def process_issue(jira, github, issue):
27+
html_url = issue["html_url"]
28+
issue_id = issue["number"]
29+
title = issue["title"]
30+
31+
if "pull" in html_url:
32+
return False
33+
34+
is_network = False
35+
for label in issue["labels"]:
36+
if label["name"] == "sig/network":
37+
is_network = True
38+
break
39+
40+
if is_network == False:
41+
return False
42+
43+
epoch_time_now = int(time.time())
44+
if jira.issue_exists(github.repo, issue_id) == False:
45+
if check_time(epoch_time_now, issue["created_at"], max_delta_weeks*SECONDS_PER_WEEK) == False:
46+
return False
47+
48+
if dry_run == False:
49+
created_issue = jira.create_issue(github.repo, issue_id, title, issue["html_url"])
50+
print(f'Created issue {jira.server}/browse/{created_issue} for {issue["html_url"]}')
51+
else:
52+
print(f'Dry Run Created issue for {issue["html_url"]}')
53+
return True
54+
else:
55+
print("Issue for", issue["html_url"], "already exists")
56+
57+
return False
58+
59+
def loop(jira, github):
60+
issues_created = 0
61+
62+
for page in range(1, 20):
63+
issues = github.get_issues(page)
64+
if len(issues) == 0:
65+
break
66+
67+
if debug:
68+
print(json.dumps(issues, sort_keys=True, indent=4))
69+
70+
for issue in issues:
71+
res = process_issue(jira, github, issue)
72+
if res == True:
73+
issues_created += 1
74+
if issues_created == flood_protection_limit:
75+
print("Flood protection limit reached, exiting")
76+
sys.exit(0)
77+
78+
def main():
79+
parser = argparse.ArgumentParser()
80+
parser.add_argument('server', help='jira server')
81+
parser.add_argument('username', help='jira username')
82+
parser.add_argument('project', help='jira project')
83+
parser.add_argument('project_id', help='jira project id')
84+
parser.add_argument('owner', help='github owner')
85+
parser.add_argument('repo', help='github repo')
86+
args = parser.parse_args()
87+
88+
if dry_run == True:
89+
print("Dry run enabled")
90+
91+
jira = Jira(args.server, args.username, args.project, args.project_id)
92+
github = Github(args.owner, args.repo)
93+
94+
loop(jira, github)
95+
96+
if __name__ == '__main__':
97+
main()

‎cronjob.yaml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
apiVersion: batch/v1
2+
kind: CronJob
3+
metadata:
4+
name: cron-github
5+
spec:
6+
schedule: "*/1 * * * *"
7+
successfulJobsHistoryLimit: 1
8+
failedJobsHistoryLimit: 1
9+
jobTemplate:
10+
spec:
11+
template:
12+
spec:
13+
containers:
14+
- name: github
15+
image: quay.io/oshoval/github:latest
16+
command:
17+
- /bin/sh
18+
- -ce
19+
- |
20+
source /app/secret.txt
21+
ls -l
22+
git clone https://github.com/oshoval/github2jira.git
23+
./github2jira/create_ticket.py
24+
volumeMounts:
25+
- name: configs
26+
mountPath: /app/secret.txt
27+
subPath: secret.txt
28+
restartPolicy: Never
29+
volumes:
30+
- name: configs
31+
configMap:
32+
name: git-token

0 commit comments

Comments
 (0)