|
| 1 | +<!-- TOC --> |
1 | 2 | # Summary
|
2 | 3 |
|
3 | 4 | The Oracle Accelerated Data Science (ADS) SDK used by data scientists and analysts for
|
4 | 5 | data exploration and experimental machine learning to democratize machine learning and
|
5 |
| -analytics by providing easy-to-use, performant, and user friendly tools that |
| 6 | +analytics by providing easy-to-use, |
| 7 | +performant, and user friendly tools that |
6 | 8 | brings together the best of data science practices.
|
7 | 9 |
|
8 | 10 | The ADS SDK helps you connect to different data sources, perform exploratory data analysis,
|
9 | 11 | data visualization, feature engineering, model training, model evaluation, and
|
10 | 12 | model interpretation. ADS also allows you to connect to the model catalog to save and load
|
11 | 13 | models to and from the catalog.
|
12 | 14 |
|
| 15 | +- [Summary](#summary) |
| 16 | + - [Documentation](#documentation) |
| 17 | + - [Get Support](#get-support) |
| 18 | + - [Getting started](#getting-started) |
| 19 | + - [Step 1: Create a conda environment](#step-1-create-a-conda-environment) |
| 20 | + - [Step 2: Activate your environment](#step-2-activate-your-environment) |
| 21 | + - [Step 3: Clone ADS and install dependencies](#step-3-clone-ads-and-install-dependencies) |
| 22 | + - [Step 4: Setup configuration files](#step-4-setup-configuration-files) |
| 23 | + - [Step 5: Versioning and generation the wheel](#step-5-versioning-and-generation-the-wheel) |
| 24 | + - [Running tests](#running-tests) |
| 25 | + - [Running default setup tests](#running-default-setup-tests) |
| 26 | + - [Running all unit tests](#running-all-unit-tests) |
| 27 | + - [Running integration tests](#running-integration-tests) |
| 28 | + - [Running opctl integration tests](#running-opctl-integration-tests) |
| 29 | + - [Local Setup of AQUA API JupyterLab Server](#local-setup-of-aqua-api-jupyterlab-server) |
| 30 | + - [Step 1: Requirements](#step-1-requirements) |
| 31 | + - [Step 2: Create local .env files](#step-2-create-local-env-files) |
| 32 | + - [Step 3: Add the run\_ads.sh script in the ADS Repository](#step-3-add-the-run_adssh-script-in-the-ads-repository) |
| 33 | + - [Step 4: Run the JupyterLab Server](#step-4-run-the-jupyterlab-server) |
| 34 | + - [Step 5: Run the unit tests for the AQUA API](#step-5-run-the-unit-tests-for-the-aqua-api) |
| 35 | + - [Security](#security) |
| 36 | + - [License](#license) |
| 37 | + |
| 38 | + |
13 | 39 | ## Documentation
|
14 | 40 |
|
15 | 41 | - [ads-documentation](https://docs.oracle.com/en-us/iaas/tools/ads-sdk/latest/index.html)
|
@@ -137,6 +163,100 @@ To build development container, see the [Build Development Container Image](http
|
137 | 163 | python3 -m pytest tests/integration/opctl
|
138 | 164 | ```
|
139 | 165 |
|
| 166 | +## Local Setup of AQUA API JupyterLab Server |
| 167 | +These are the steps to run the AQUA (AI Quick Actions) API Server for development and testing purposes. The source code for the AQUA API Server is [here](https://github.com/oracle/accelerated-data-science/tree/21ba00b95aef8581991fee6c7d558e2f2b1680ac/ads/aqua) within this repository. |
| 168 | + |
| 169 | +### Step 1: Requirements |
| 170 | ++ Complete the [Getting Started](#getting-started) Section above, create a conda environment with python >3.9 or 3.10 |
| 171 | ++ install any Rest API Client in your IDE (Thunder Client on [vscode](https://marketplace.visualstudio.com/items?itemName=rangav.vscode-thunder-client) or Postman) |
| 172 | ++ Activate the conda environment from the Getting Started Section and run |
| 173 | + |
| 174 | +``` |
| 175 | +pip install -r test-requirements.txt |
| 176 | +``` |
| 177 | + |
| 178 | +### Step 2: Create local .env files |
| 179 | +Running the local JuypterLab server requires setting OCI authentication, proxy, and OCI namespace parameters. Adapt this .env file with your specific OCI profile and OCIDs to set these variables. |
| 180 | + |
| 181 | +``` |
| 182 | +CONDA_BUCKET_NS="your_conda_bucket" |
| 183 | +http_proxy="" |
| 184 | +https_proxy="" |
| 185 | +HTTP_PROXY="" |
| 186 | +HTTPS_PROXY="" |
| 187 | +OCI_ODSC_SERVICE_ENDPOINT="your_service_endpoint" |
| 188 | +AQUA_SERVICE_MODELS_BUCKET="service-managed-models" |
| 189 | +AQUA_TELEMETRY_BUCKET_NS="" |
| 190 | +PROJECT_COMPARTMENT_OCID="ocid1.compartment.oc1.<your_ocid>" |
| 191 | +OCI_CONFIG_PROFILE="your_oci_profile_name" |
| 192 | +OCI_IAM_TYPE="security_token" # no modification needed if using token-based auth |
| 193 | +TENANCY_OCID="ocid1.tenancy.oc1.<your_ocid>" |
| 194 | +AQUA_JOB_SUBNET_ID="ocid1.subnet.oc1.<your_ocid>" |
| 195 | +ODSC_MODEL_COMPARTMENT_OCID="ocid1.compartment.oc1.<your_ocid>" |
| 196 | +PROJECT_OCID="ocid1.datascienceproject.oc1.<your_ocid>" |
| 197 | +``` |
| 198 | + |
| 199 | +### Step 3: Add the run_ads.sh script in the ADS Repository |
| 200 | ++ add the shell script below and .env file from step 2 to your local directory of the cloned ADS Repository |
| 201 | ++ Run ```chmox +x run_ads.sh``` after you create this script. |
| 202 | +``` |
| 203 | +#!/bin/bash |
| 204 | +
|
| 205 | +#### Check if a CLI command is provided |
| 206 | +if [ "$#" -lt 1 ]; then |
| 207 | + echo "Usage: $0 <cli command>" |
| 208 | + exit 1 |
| 209 | +fi |
| 210 | +
|
| 211 | +#### Load environment variables from .env file |
| 212 | +if [ -f .env ]; then |
| 213 | + export $(grep -v '^#' .env.int | xargs) |
| 214 | +else |
| 215 | + echo "Error: .env.int file not found!" |
| 216 | + exit 1 |
| 217 | +fi |
| 218 | +
|
| 219 | +# Execute the CLI command |
| 220 | +"$@" |
| 221 | +``` |
| 222 | + |
| 223 | +### Step 4: Run the JupyterLab Server |
| 224 | +We can start the JupyterLab server using the following command |
| 225 | + |
| 226 | +``` |
| 227 | +./run_ads.sh jupyter lab --no-browser --ServerApp.disable_check_xsrf=True |
| 228 | +``` |
| 229 | ++ run ```pkill jupyter-lab``` to kill the JupyterLab server and re-run server to reflect changes made locally to the AQUA API |
| 230 | ++ to test if server is running via CLI, run this in terminal |
| 231 | + |
| 232 | +``` |
| 233 | +./run_ads.sh ads aqua model list |
| 234 | +``` |
| 235 | + |
| 236 | +To make calls to the API, use the link http://localhost:8888/aqua/insert_handler_here with a REST API Client like Thunder Client/ Postman. |
| 237 | + |
| 238 | +Examples of handlers |
| 239 | +``` |
| 240 | +GET http://localhost:8888/aqua/model # calling the model_handler.py |
| 241 | +
|
| 242 | +GET http://localhost:8888/aqua/deployments # calling the deployment_handler.py |
| 243 | +``` |
| 244 | +Handlers can be found [here](https://github.com/oracle/accelerated-data-science/tree/21ba00b95aef8581991fee6c7d558e2f2b1680ac/ads/aqua/extension). |
| 245 | + |
| 246 | +### Step 5: Run the unit tests for the AQUA API |
| 247 | +All the unit tests can be found [here](https://github.com/oracle/accelerated-data-science/tree/main/tests/unitary/with_extras/aqua). |
| 248 | +The following commands detail how the unit tests can be run. |
| 249 | +``` |
| 250 | +# Run all tests in AQUA project |
| 251 | +python -m pytest -q tests/unitary/with_extras/aqua/test_deployment.py |
| 252 | +
|
| 253 | +# Run all tests specific to a module within in AQUA project (ex. test_deployment.py, test_model.py, etc.) |
| 254 | +python -m pytest -q tests/unitary/with_extras/aqua/test_deployment.py |
| 255 | +
|
| 256 | +# Run specific test method within the module (replace test_get_deployment_default_params with targeted test method) |
| 257 | +python -m pytest tests/unitary/with_extras/aqua/test_deployment.py -k "test_get_deployment_default_params" |
| 258 | +``` |
| 259 | + |
140 | 260 | ## Security
|
141 | 261 |
|
142 | 262 | Consult the [security guide](./SECURITY.md) for our responsible security
|
|
0 commit comments