Skip to content

Commit a416113

Browse files
vrdmrvrdmr
authored
Setup GitHub Actions for Unittests (Azure#759)
* Setup GitHub Action for Unittests. * Adding super-linter. * Adding `CodeCov` action and removing unit tests in AzureDevops.
1 parent 548d807 commit a416113

File tree

10 files changed

+205
-54
lines changed

10 files changed

+205
-54
lines changed

Diff for: ‎.github/linters/tox.ini

+39
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
[flake8]
2+
ignore =
3+
# module level import not at top of file
4+
E402,
5+
# line break after binary operator
6+
W504,
7+
# line break before binary operator
8+
W503,
9+
# E731: Do not assign a lambda expression, use a def
10+
E731
11+
12+
exclude =
13+
# No need to traverse our git directory
14+
.git,
15+
build,
16+
dist,
17+
.eggs,
18+
*.egg-info,
19+
examples,
20+
docker_files,
21+
.github,
22+
.local,
23+
docs,
24+
Samples,
25+
__pycache__,
26+
azure_functions_worker/protos/,
27+
azure_functions_worker/_thirdparty/typing_inspect.py,
28+
tests/unittests/test_typing_inspect.py,
29+
.venv*,
30+
.env*,
31+
.vscode,
32+
venv*,
33+
scripts
34+
35+
max-line-length = 80
36+
37+
per-file-ignores =
38+
# import not used
39+
__init__.py:F401

Diff for: ‎.github/workflows/linter.yml

+54
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
---
2+
###########################
3+
###########################
4+
## Linter GitHub Actions ##
5+
###########################
6+
###########################
7+
8+
name: Lint Code Base
9+
10+
#
11+
# Documentation:
12+
# https://help.github.com/en/articles/workflow-syntax-for-github-actions
13+
#
14+
15+
#############################
16+
# Start the job on all push #
17+
#############################
18+
on: [ push, pull_request, workflow_dispatch ]
19+
20+
###############
21+
# Set the Job #
22+
###############
23+
jobs:
24+
build:
25+
# Name the Job
26+
name: Lint Code Base
27+
# Set the agent to run on
28+
runs-on: ubuntu-latest
29+
30+
##################
31+
# Load all steps #
32+
##################
33+
steps:
34+
##########################
35+
# Checkout the code base #
36+
##########################
37+
- name: Checkout Code
38+
uses: actions/checkout@v2
39+
40+
################################
41+
# Run Linter against code base #
42+
################################
43+
- name: Lint Code Base
44+
uses: github/super-linter@v3
45+
env:
46+
VALIDATE_PYTHON: true
47+
VALIDATE_PYTHON_FLAKE8: true
48+
# VALIDATE_PYTHON_PYLINT: false # disable pylint, as we have not configure it
49+
# VALIDATE_PYTHON_BLACK: false # same as above
50+
PYTHON_FLAKE8_CONFIG_FILE: tox.ini
51+
FILTER_REGEX_INCLUDE: azure_functions_worker/.*
52+
FILTER_REGEX_EXCLUDE: tests/.*
53+
DEFAULT_BRANCH: dev
54+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Diff for: ‎.github/workflows/unittesting.yml

+45
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# This workflow will install Python dependencies, run tests and lint with a single version of Python
2+
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
3+
4+
name: CI Unit tests
5+
6+
on:
7+
push:
8+
branches: [ dev, master, main ]
9+
pull_request:
10+
branches: [ dev, master, main ]
11+
12+
jobs:
13+
build:
14+
name: "Python UT CI Run"
15+
runs-on: ubuntu-latest
16+
strategy:
17+
matrix:
18+
python-version: [ 3.6, 3.7, 3.8, 3.9 ]
19+
steps:
20+
- uses: actions/checkout@v2
21+
- name: Set up Python ${{ matrix.python-version }}
22+
uses: actions/setup-python@v2
23+
with:
24+
python-version: ${{ matrix.python-version }}
25+
- name: Set up Dotnet 2.2.207
26+
uses: actions/setup-dotnet@v1
27+
with:
28+
dotnet-version: '2.2.207'
29+
- name: Install dependencies and the worker
30+
run: |
31+
python -m pip install --upgrade pip
32+
python -m pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple -U -e .[dev]
33+
python -m pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple -U azure-functions --pre
34+
python setup.py build
35+
python setup.py webhost
36+
- name: Test with pytest
37+
run: |
38+
pytest --instafail --cov=./azure_functions_worker --cov-report xml --cov-branch tests/unittests
39+
- name: Codecov
40+
uses: codecov/[email protected]
41+
with:
42+
file: ./coverage.xml # optional
43+
flags: unittests # optional
44+
name: codecov # optional
45+
fail_ci_if_error: false # optional (default = false)

Diff for: ‎azure-pipelines.yml

-4
Original file line numberDiff line numberDiff line change
@@ -64,10 +64,6 @@ jobs:
6464
disableAutoCwd: true
6565
scriptPath: .ci/linux_devops_build.sh
6666
displayName: 'Build'
67-
- bash: |
68-
chmod +x .ci/linux_devops_unit_tests.sh
69-
.ci/linux_devops_unit_tests.sh
70-
displayName: 'Unit Tests'
7167
- bash: |
7268
chmod +x .ci/linux_devops_e2e_tests.sh
7369
.ci/linux_devops_e2e_tests.sh

Diff for: ‎azure_functions_worker/_thirdparty/typing_inspect.py

+20-14
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Imported from https://github.com/ilevkivskyi/typing_inspect/blob/168fa6f7c5c55f720ce6282727211cf4cf6368f6/typing_inspect.py
1+
# Imported from https://github.com/ilevkivskyi/typing_inspect/blob/168fa6f7c5c55f720ce6282727211cf4cf6368f6/typing_inspect.py # NoQA E501
22
# Author: Ivan Levkivskyi
33
# License: MIT
44

@@ -16,7 +16,6 @@
1616
if NEW_TYPING:
1717
import collections.abc
1818

19-
2019
if NEW_TYPING:
2120
from typing import (
2221
Generic, Callable, Union, TypeVar, ClassVar, Tuple, _GenericAlias
@@ -27,6 +26,10 @@
2726
_ClassVar, GenericMeta,
2827
)
2928

29+
NEW_39_TYPING = sys.version_info[:3] >= (3, 9, 0) # PEP 560
30+
if NEW_39_TYPING:
31+
from typing import _SpecialGenericAlias
32+
3033

3134
# from mypy_extensions import _TypedDictMeta
3235

@@ -51,18 +54,22 @@ def is_generic_type(tp):
5154
is_generic_type(Union[int, T]) == False
5255
is_generic_type(ClassVar[List[int]]) == False
5356
is_generic_type(Callable[..., T]) == False
54-
5557
is_generic_type(Generic) == True
5658
is_generic_type(Generic[T]) == True
5759
is_generic_type(Iterable[int]) == True
5860
is_generic_type(Mapping) == True
5961
is_generic_type(MutableMapping[T, List[int]]) == True
6062
is_generic_type(Sequence[Union[str, bytes]]) == True
6163
"""
64+
if NEW_39_TYPING:
65+
return (isinstance(tp, type) and issubclass(tp, Generic)
66+
or isinstance(tp, _SpecialGenericAlias)
67+
and tp.__origin__ not in (Union, tuple, ClassVar, collections.abc.Callable)) # NoQA E501
6268
if NEW_TYPING:
63-
return (isinstance(tp, type) and issubclass(tp, Generic) or
64-
isinstance(tp, _GenericAlias) and
65-
tp.__origin__ not in (Union, tuple, ClassVar, collections.abc.Callable))
69+
return (isinstance(tp, type)
70+
and issubclass(tp, Generic)
71+
or isinstance(tp, _GenericAlias)
72+
and tp.__origin__ not in (Union, tuple, ClassVar, collections.abc.Callable)) # NoQA E501
6673
return (isinstance(tp, GenericMeta) and not
6774
isinstance(tp, (CallableMeta, TupleMeta)))
6875

@@ -84,7 +91,7 @@ class MyClass(Callable[[int], int]):
8491
For more general tests use callable(), for more precise test
8592
(excluding subclasses) use::
8693
87-
get_origin(tp) is collections.abc.Callable # Callable prior to Python 3.7
94+
get_origin(tp) is collections.abc.Callable # Callable prior to Python 3.7 # NoQA E501
8895
"""
8996
if NEW_TYPING:
9097
return (tp is Callable or isinstance(tp, _GenericAlias) and
@@ -224,9 +231,8 @@ def get_parameters(tp):
224231
get_parameters(Mapping[T, Tuple[S_co, T]]) == (T, S_co)
225232
"""
226233
if NEW_TYPING:
227-
if (isinstance(tp, _GenericAlias) or
228-
isinstance(tp, type) and issubclass(tp, Generic) and
229-
tp is not Generic):
234+
if (isinstance(tp, _GenericAlias) or isinstance(tp, type) and
235+
issubclass(tp, Generic) and tp is not Generic): # NoQA E129
230236
return tp.__parameters__
231237
return ()
232238
if (
@@ -305,7 +311,7 @@ def get_args(tp, evaluate=None):
305311
raise ValueError('evaluate can only be True in Python 3.7')
306312
if isinstance(tp, _GenericAlias):
307313
res = tp.__args__
308-
if get_origin(tp) is collections.abc.Callable and res[0] is not Ellipsis:
314+
if get_origin(tp) is collections.abc.Callable and res[0] is not Ellipsis: # NoQA E501
309315
res = (list(res[:-1]), res[-1])
310316
return res
311317
return ()
@@ -356,8 +362,8 @@ class MyClass(List[int], Mapping[str, List[int]]):
356362

357363

358364
def typed_dict_keys(td):
359-
"""If td is a TypedDict class, return a dictionary mapping the typed keys to types.
360-
Otherwise, return None. Examples::
365+
"""If td is a TypedDict class, return a dictionary mapping the typed keys
366+
to types. Otherwise, return None. Examples::
361367
362368
class TD(TypedDict):
363369
x: int
@@ -370,6 +376,6 @@ class Other(dict):
370376
typed_dict_keys(dict) == None
371377
typed_dict_keys(Other) == None
372378
"""
373-
if isinstance(td, _TypedDictMeta):
379+
if isinstance(td, _TypedDictMeta): # NoQA F821
374380
return td.__annotations__.copy()
375381
return None

Diff for: ‎azure_functions_worker/functions.py

+10-10
Original file line numberDiff line numberDiff line change
@@ -73,13 +73,13 @@ def add_function(self, function_id: str,
7373
if desc.direction == protos.BindingInfo.inout:
7474
raise FunctionLoadError(
7575
func_name,
76-
f'"inout" bindings are not supported')
76+
'"inout" bindings are not supported')
7777

7878
if name == '$return':
7979
if desc.direction != protos.BindingInfo.out:
8080
raise FunctionLoadError(
8181
func_name,
82-
f'"$return" binding must have direction set to "out"')
82+
'"$return" binding must have direction set to "out"')
8383

8484
has_explicit_return = True
8585
return_binding_name = desc.type
@@ -104,14 +104,14 @@ def add_function(self, function_id: str,
104104
or ctx_anno.__name__ != 'Context'):
105105
raise FunctionLoadError(
106106
func_name,
107-
f'the "context" parameter is expected to be of '
108-
f'type azure.functions.Context, got '
107+
'the "context" parameter is expected to be of '
108+
'type azure.functions.Context, got '
109109
f'{ctx_anno!r}')
110110

111111
if set(params) - set(bound_params):
112112
raise FunctionLoadError(
113113
func_name,
114-
f'the following parameters are declared in Python but '
114+
'the following parameters are declared in Python but '
115115
f'not in function.json: {set(params) - set(bound_params)!r}')
116116

117117
if set(bound_params) - set(params):
@@ -180,15 +180,15 @@ def add_function(self, function_id: str,
180180
raise FunctionLoadError(
181181
func_name,
182182
f'binding {param.name} is declared to have the "out" '
183-
f'direction, but its annotation in Python is not '
184-
f'a subclass of azure.functions.Out')
183+
'direction, but its annotation in Python is not '
184+
'a subclass of azure.functions.Out')
185185

186186
if not is_binding_out and is_param_out:
187187
raise FunctionLoadError(
188188
func_name,
189189
f'binding {param.name} is declared to have the "in" '
190-
f'direction in function.json, but its annotation '
191-
f'is azure.functions.Out in Python')
190+
'direction in function.json, but its annotation '
191+
'is azure.functions.Out in Python')
192192

193193
if param_has_anno and param_py_type in (str, bytes) and (
194194
not bindings.has_implicit_output(desc.type)):
@@ -233,7 +233,7 @@ def add_function(self, function_id: str,
233233
and typing_inspect.get_origin(return_anno).__name__ == 'Out'):
234234
raise FunctionLoadError(
235235
func_name,
236-
f'return annotation should not be azure.functions.Out')
236+
'return annotation should not be azure.functions.Out')
237237

238238
return_pytype = return_anno
239239
if not isinstance(return_pytype, type):

Diff for: ‎azure_functions_worker/testutils.py

+22-21
Original file line numberDiff line numberDiff line change
@@ -252,7 +252,7 @@ def EventStream(self, client_response_iterator, context):
252252
f'unexpected {rtype!r} initial message from the worker')
253253

254254
if client_response.start_stream.worker_id != self._host.worker_id:
255-
raise AssertionError(f'worker_id mismatch')
255+
raise AssertionError('worker_id mismatch')
256256

257257
except Exception as ex:
258258
self._host._loop.call_soon_threadsafe(
@@ -590,25 +590,25 @@ def popen_webhost(*, stdout, stderr, script_root=FUNCS_PATH, port=None):
590590

591591
if not hostexe_args:
592592
raise RuntimeError('\n'.join([
593-
f'Unable to locate Azure Functions Host binary.',
594-
f'Please do one of the following:',
595-
f' * run the following command from the root folder of',
596-
f' the project:',
597-
f'',
593+
'Unable to locate Azure Functions Host binary.',
594+
'Please do one of the following:',
595+
' * run the following command from the root folder of',
596+
' the project:',
597+
'',
598598
f' $ {sys.executable} setup.py webhost',
599-
f'',
600-
f' * or download or build the Azure Functions Host and'
601-
f' then write the full path to WebHost.dll'
602-
f' into the `PYAZURE_WEBHOST_DLL` environment variable.',
603-
f' Alternatively, you can create the',
599+
'',
600+
' * or download or build the Azure Functions Host and'
601+
' then write the full path to WebHost.dll'
602+
' into the `PYAZURE_WEBHOST_DLL` environment variable.',
603+
' Alternatively, you can create the',
604604
f' {WORKER_CONFIG.name} file in the root folder',
605-
f' of the project with the following structure:',
606-
f'',
607-
f' [webhost]',
608-
f' dll = /path/Microsoft.Azure.WebJobs.Script.WebHost.dll',
609-
f' * or download Azure Functions Core Tools binaries and',
610-
f' then write the full path to func.exe into the ',
611-
f' `CORE_TOOLS_PATH` envrionment variable.'
605+
' of the project with the following structure:',
606+
'',
607+
' [webhost]',
608+
' dll = /path/Microsoft.Azure.WebJobs.Script.WebHost.dll',
609+
' * or download Azure Functions Core Tools binaries and',
610+
' then write the full path to func.exe into the ',
611+
' `CORE_TOOLS_PATH` envrionment variable.'
612612
]))
613613

614614
worker_path = os.environ.get('PYAZURE_WORKER_DIR')
@@ -677,22 +677,23 @@ def start_webhost(*, script_dir=None, stdout=None):
677677
port = _find_open_port()
678678
proc = popen_webhost(stdout=stdout, stderr=subprocess.STDOUT,
679679
script_root=script_root, port=port)
680-
time.sleep(3) # Giving host some time to start fully.
680+
time.sleep(10) # Giving host some time to start fully.
681681

682682
addr = f'http://{LOCALHOST}:{port}'
683683
for _ in range(10):
684684
try:
685685
r = requests.get(f'{addr}/api/ping',
686686
params={'code': 'testFunctionKey'})
687+
# Give the host a bit more time to settle
688+
time.sleep(2)
689+
687690
if 200 <= r.status_code < 300:
688691
# Give the host a bit more time to settle
689692
time.sleep(2)
690693
break
691694
except requests.exceptions.ConnectionError:
692695
pass
693-
694696
time.sleep(2)
695-
696697
else:
697698
proc.terminate()
698699
try:

0 commit comments

Comments
 (0)