Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
256 changes: 256 additions & 0 deletions ci/Jenkinsfile.test-e2e.android
Original file line number Diff line number Diff line change
@@ -0,0 +1,256 @@
#!/usr/bin/env groovy
library '[email protected]'

pipeline {
agent {
dockerfile {
label 'linuxcontainer'
filename 'tests.Dockerfile'
dir 'ci'
args '--user jenkins'
}
}

parameters {
gitParameter(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid using gitParameter, see this PR for problem description:

name: 'GIT_REF',
description: 'Git branch to checkout.',
branchFilter: 'origin/(.*)',
branch: '',
defaultValue: 'master',
quickFilterEnabled: false,
selectedValue: 'DEFAULT',
sortMode: 'ASCENDING_SMART',
tagFilter: '*',
type: 'PT_BRANCH'
)
string(
name: 'BUILD_SOURCE',
description: 'URL to APK or Jenkins build (pkg/*.apk). Required when BROWSERSTACK_APP_ID is empty.',
defaultValue: ''
)
string(
name: 'BROWSERSTACK_APP_ID',
description: 'Existing BrowserStack app identifier (bs://...). Leave empty to upload from BUILD_SOURCE.',
defaultValue: ''
)
choice(
name: 'TEST_DEVICE_ID',
description: 'BrowserStack device id to run on (leave empty to use the environment default).',
choices: ['', 'galaxy_tab_s10p_android_15', 'pixel_8_android_14', 'pixel_7_android_13', 'samsung_s23_android_13']
)
string(
name: 'PYTEST_ARGS',
description: 'Pytest flags (e.g. "-m smoke" or "-m critical") and other args.',
defaultValue: '-n=5 -m smoke tests'
)
}

options {
timestamps()
timeout(time: 120, unit: 'MINUTES')
buildDiscarder(logRotator(
daysToKeepStr: '30',
numToKeepStr: '30',
artifactNumToKeepStr: '30',
))
disableRestartFromStage()
}

environment {
VIRTUAL_ENV = "${env.WORKSPACE_TMP}/venv-appium"
PYTHONUNBUFFERED = "1"
TEST_DEVICE_ID = "${params.TEST_DEVICE_ID}"
BROWSERSTACK_PROJECT_NAME = "Mobile E2E ${utils.getBuildType()}"
}

stages {
stage('Prep') {
steps {
script {
setNewBuildName()
updateGitHubStatus()
}
}
}

stage('Setup Python environment') {
steps {
dir('test/e2e_appium') {
sh """
set -euo pipefail
python3 -m venv ${VIRTUAL_ENV}
source ${VIRTUAL_ENV}/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
"""
}
}
}

stage('Use provided BrowserStack app') {
when {
expression { params.BROWSERSTACK_APP_ID?.trim() }
}
steps {
script {
env.BROWSERSTACK_APP_ID = params.BROWSERSTACK_APP_ID.trim()
echo "Using provided BrowserStack app id: ${env.BROWSERSTACK_APP_ID}"
}
}
}
Comment on lines +91 to +101
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pointless stage that achieves nothing.


stage('Download') {
when {
allOf {
expression { !params.BROWSERSTACK_APP_ID?.trim() }
expression { params.BUILD_SOURCE?.startsWith('http') }
}
}
steps { timeout(5) { script { dir('test/e2e_appium') {
if (!params.BUILD_SOURCE?.trim()) {
error('Specify BUILD_SOURCE when BROWSERSTACK_APP_ID is empty.')
}
Comment on lines +111 to +113
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has no chance of happening.

sh 'mkdir -p ./pkg/'
fileOperations([
fileDownloadOperation(
url: params.BUILD_SOURCE,
targetFileName: 'downloaded.apk',
targetLocation: './pkg/',
userName: '',
password: '',
)
])
def apkPath = utils.findFile('test/e2e_appium/pkg/*.apk')
if (!apkPath) {
error("Unable to locate APK under pkg/. Ensure BUILD_SOURCE produces pkg/*.apk artifacts.")
}
env.APK_PATH = apkPath
echo "APK ready at ${env.APK_PATH}"
Comment on lines +124 to +129
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Entirely pointless.

} } } }
}

stage('Copy') {
when {
allOf {
expression { !params.BROWSERSTACK_APP_ID?.trim() }
expression { !params.BUILD_SOURCE?.startsWith('http') }
}
}
steps { timeout(5) { script { dir('test/e2e_appium') {
if (!params.BUILD_SOURCE?.trim()) {
error('Specify BUILD_SOURCE when BROWSERSTACK_APP_ID is empty.')
}
Comment on lines +141 to +143
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pointless.

copyArtifacts(
projectName: params.BUILD_SOURCE,
filter: 'pkg/*.apk',
selector: lastWithArtifacts(),
target: './'
)
def apkPath = utils.findFile('test/e2e_appium/pkg/*.apk')
if (!apkPath) {
error("Unable to locate APK under pkg/. Ensure BUILD_SOURCE produces pkg/*.apk artifacts.")
}
env.APK_PATH = apkPath
echo "APK ready at ${env.APK_PATH}"
Comment on lines +150 to +155
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pointless.

} } } }
}

stage('Upload APK to BrowserStack') {
when {
expression { !env.BROWSERSTACK_APP_ID }
}
steps {
script {
withCredentials([
usernamePassword(
credentialsId: 'browserstack-status-desktop',
usernameVariable: 'BROWSERSTACK_USERNAME',
passwordVariable: 'BROWSERSTACK_ACCESS_KEY'
)
]) {
def response = sh(
script: "./scripts/upload_browserstack_apk.sh",
returnStdout: true
).trim()
def result = readJSON text: response
def appUrl = result?.app_url
if (!appUrl) {
error("BrowserStack upload failed: ${response}")
}
env.BROWSERSTACK_APP_ID = appUrl
env.BROWSERSTACK_BUILD_NAME = result?.custom_id
env.BROWSERSTACK_BUILD_IDENTIFIER = env.BUILD_NUMBER
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is BUILD_NUMBER coming from?

echo "BrowserStack app uploaded: ${env.BROWSERSTACK_APP_ID}"
}
}
}
}

stage('Run pytest suite') {
steps {
script {
dir('test/e2e_appium') {
println("Using TEST_DEVICE_ID: ${env.TEST_DEVICE_ID}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what the point of this is since user can just look up job parameters.


withCredentials([
usernamePassword(
credentialsId: 'browserstack-status-desktop',
usernameVariable: 'BROWSERSTACK_USERNAME',
passwordVariable: 'BROWSERSTACK_ACCESS_KEY'
)
]) {
sh "${VIRTUAL_ENV}/bin/python -m pytest --env browserstack ${params.PYTEST_ARGS?.trim() ?: ''}"
}
}
}
}
}

stage('Publish test results') {
steps {
script {
def runId = env.E2E_RUN_ID?.trim()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this supposed to come from?

def reportsPattern = runId ? "test/e2e_appium/reports/${runId}/**/*.xml" : "test/e2e_appium/reports/**/*.xml"
junit allowEmptyResults: true, testResults: reportsPattern
def archivePattern = runId ? "test/e2e_appium/reports/${runId}/**/*" : "test/e2e_appium/reports/**/*"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this pattern wider than the one above? How are those artifacts useful?

archiveArtifacts artifacts: archivePattern, allowEmptyArchive: true
}
}
}
}

post {
success {
script {
github.notifyPR(true)
}
}
failure {
script {
github.notifyPR(false)
}
}
Comment on lines +224 to +233
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
success {
script {
github.notifyPR(true)
}
}
failure {
script {
github.notifyPR(false)
}
}
success { script { github.notifyPR(true) } }
failure { script { github.notifyPR(false) } }

cleanup {
cleanWs(disableDeferredWipeout: true)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
cleanWs(disableDeferredWipeout: true)
cleanWs(disableDeferredWipeout: true)
dir(env.WORKSPACE_TMP) { deleteDir() }

}
}
}

def setNewBuildName() {
if (currentBuild.upstreamBuilds) {
def parent = utils.parentOrCurrentBuild()
currentBuild.displayName = parent.getFullDisplayName().minus('status-desktop » ')
}
}

def updateGitHubStatus() {
if (params.BUILD_SOURCE ==~ /.*\/PR-[0-9]+\/?$/) {
github.statusUpdate(
context: 'jenkins/prs/tests/e2e-android',
commit: jenkins.getJobCommitByPath(params.BUILD_SOURCE),
repo_url: 'https://github.com/status-im/status-desktop'
)
}
}

45 changes: 45 additions & 0 deletions scripts/upload_browserstack_apk.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#!/usr/bin/env bash
set -euo pipefail

# Upload APK to BrowserStack and output JSON response to stdout
# Requires environment variables:
# APK_PATH - Path to the APK file
# BROWSERSTACK_USERNAME - BrowserStack username
# BROWSERSTACK_ACCESS_KEY - BrowserStack access key
# BUILD_NUMBER - Build number (optional, defaults to current timestamp)

if [[ -z "${APK_PATH:-}" ]]; then
echo "Error: APK_PATH environment variable is required" >&2
exit 1
fi
Comment on lines +11 to +14
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this an env variables rather than a positional argument?


if [[ ! -f "${APK_PATH}" ]]; then
echo "Error: APK_PATH does not exist or is not a file: ${APK_PATH}" >&2
exit 1
fi

if [[ ! -r "${APK_PATH}" ]]; then
echo "Error: APK_PATH is not readable: ${APK_PATH}" >&2
exit 1
fi
Comment on lines +16 to +24
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pointless double test, the -r will fail if -f would fail.


if [[ -z "${BROWSERSTACK_USERNAME:-}" ]]; then
echo "Error: BROWSERSTACK_USERNAME environment variable is required" >&2
exit 1
fi

if [[ -z "${BROWSERSTACK_ACCESS_KEY:-}" ]]; then
echo "Error: BROWSERSTACK_ACCESS_KEY environment variable is required" >&2
exit 1
fi

APK_NAME=$(basename "${APK_PATH}")
SANITIZED_NAME=$(printf '%s' "${APK_NAME}" | tr -cs '[:alnum:]._-' '-' | cut -c1-80)
BUILD_ID="${BUILD_NUMBER:-$(date +%s)}"
CUSTOM_ID="${SANITIZED_NAME}-${BUILD_ID}"

curl -s -u "${BROWSERSTACK_USERNAME}:${BROWSERSTACK_ACCESS_KEY}" \
-X POST "https://api-cloud.browserstack.com/app-automate/upload" \
-F "file=@${APK_PATH}" \
-F "custom_id=${CUSTOM_ID}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use --show-error and --fail-with-body to make debugging less painful:

Suggested change
-F "custom_id=${CUSTOM_ID}"
curl --request POST "https://api-cloud.browserstack.com/app-automate/upload"
--silent --show-error --fail-with-body \
--user "${BROWSERSTACK_USERNAME}:${BROWSERSTACK_ACCESS_KEY}" \
--form "file=@${APK_PATH}" \
--form "custom_id=${CUSTOM_ID}"

And use full flag names instead of cryptic one letter flags. Those are for CLI use, not for scripts.


1 change: 1 addition & 0 deletions test/e2e_appium/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ report/
reports/
screenshots/
logs/
log/
*.log
result.xml
junit.xml
Expand Down
8 changes: 3 additions & 5 deletions test/e2e_appium/config/environments/base.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,11 @@ timeouts:

logging:
level: "INFO"
enable_screenshots: true
enable_video_recording: true
enable_xml_report: true
enable_html_report: true
enable_junit_report: true

directories:
logs: "logs"
reports: "reports"
screenshots: "screenshots"
reports: "reports/${E2E_RUN_ID:-local}"
logs: "reports/${E2E_RUN_ID:-local}/logs"
screenshots: "reports/${E2E_RUN_ID:-local}/screenshots"
16 changes: 10 additions & 6 deletions test/e2e_appium/config/environments/browserstack.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ provider:
auth:
username: "${BROWSERSTACK_USERNAME:-}"
access_key: "${BROWSERSTACK_ACCESS_KEY:-}"
build_name_template: "Status_E2E_Mobile"
build_name_template: "${BROWSERSTACK_BUILD_NAME:-${BROWSERSTACK_BUILD_IDENTIFIER:-${GIT_COMMIT:-Status Mobile}}}"
build_identifier_template: "${BROWSERSTACK_BUILD_IDENTIFIER:-${GIT_COMMIT:-local}}"
session_name_template: "${TEST_NAME:-Status Test}"
project_name: "Status E2E Appium"
session_name_template: "${TEST_NAME:-${PYTEST_CURRENT_TEST:-Status Test}}"
project_name: "${BROWSERSTACK_PROJECT_NAME:-Status E2E Appium}"
hub_url: "https://hub-cloud.browserstack.com/wd/hub"
max_parallel_sessions: 5
sdk:
Expand All @@ -29,7 +29,13 @@ device_defaults:
newCommandTimeout: 300
bstack:options:
deviceOrientation: "landscape"
"appiumVersion" : "2.19.0"
"appiumVersion": "2.19.0"
video: true
debug: false
networkLogs: false
appiumLogs: true
deviceLogs: true
appProfiling: false
orientation: "LANDSCAPE"
appium:unicodeKeyboard: true
appium:resetKeyboard: true
Expand Down Expand Up @@ -76,7 +82,5 @@ execution:

logging:
level: "INFO"
enable_video_recording: true
enable_screenshots: true
enable_xml_report: true
enable_html_report: true
5 changes: 0 additions & 5 deletions test/e2e_appium/config/environments/local.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,4 @@ timeouts:

logging:
level: "DEBUG"
enable_video_recording: false

directories:
logs: "logs/local"
reports: "reports/local"
screenshots: "screenshots/local"
4 changes: 2 additions & 2 deletions test/e2e_appium/config/logging_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -247,9 +247,9 @@ def setup_logging(config: Optional[LoggingConfig] = None) -> Dict[str, Any]:
if config is None:
config = LoggingConfig()

# Create logs directory
# Create logs directory (create parents for nested per-run paths)
logs_dir = Path(config.logs_dir)
logs_dir.mkdir(exist_ok=True)
logs_dir.mkdir(parents=True, exist_ok=True)

# Clear any existing handlers
root_logger = logging.getLogger()
Expand Down
Loading