Skip to content

AZURE_SETUP.md

Latest commit

 

History

History
272 lines (190 loc) · 9.07 KB

AZURE_SETUP.md

File metadata and controls

272 lines (190 loc) · 9.07 KB
Raw

Azure Setup Guide

This guide walks through provisioning the Azure resources required to run Vigil against your existing Synapse workspace. All steps use the Azure Portal. Estimated time: 15–20 minutes.

Architecture overview

Your existing Synapse Workspace
        │
        │  Synapse Monitoring Operator (read-only)
        ▼
  Function App  ──── Storage Blob Data Contributor ────▶  Reports Storage Account
  (Vigil)                                                  └── reports/dailysnapshots/
        │
        │  Azure Communication Service Email Sender
        ▼
  Azure Communication Services
        │
        └──▶  Email recipients

Resources to create

Resource Purpose Estimated cost
Azure Communication Services Email delivery ~$0 (100 free emails/day)
Storage Account — Function runtime Azure Functions internal state ~$0–1/month
Storage Account — Reports CSV pipeline run archives ~$0–1/month
Log Analytics Workspace Telemetry backend for App Insights ~$0–2/month
Application Insights Structured logging and observability ~$0 (5 GB free/month)
Function App (Consumption, Linux) Hosts the Vigil monitoring function ~$0 (1M free executions/month)
Total ~$1–5/month

Prerequisites

  • An Azure subscription with an existing Synapse workspace
  • Contributor access to the subscription or resource group
  • Azure Functions Core Tools (for local development)

Step 1 — Resource group

Use an existing resource group or create a new one:

Portal: Search → Resource groups → Create

  • Choose your subscription, name, and region
  • Use the same region as your Synapse workspace where possible

Step 2 — Azure Communication Services

2a — Create the ACS resource

Portal: Search → Communication Services → Create

  • Select your resource group
  • Name: anything descriptive (e.g. vigil-acs)
  • Data location: United States (recommended)

2b — Verify an email domain

ACS requires a verified sender domain before it can deliver email.

  1. Open your ACS resource → Email → Domains → Add domain
  2. Choose Azure managed domain for the quickest setup (no DNS changes required)
  3. Once verified, copy the sender address — it will look like: DoNotReply@<guid>.azurecomm.net

You will need this address for the Function App settings in Step 6.

Step 3 — Storage accounts

You need two separate storage accounts: one for the Functions runtime, one for report archives.

3a — Function runtime storage

Portal: Search → Storage accounts → Create

  • Resource group: same as above
  • Name: e.g. vigilfnstore (globally unique, lowercase, no hyphens)
  • Region: same as your resource group
  • Performance: Standard | Redundancy: LRS

3b — Reports archive storage

Portal: Search → Storage accounts → Create (repeat with a different name, e.g. vigilrptstore)

After creating the reports storage account, add a container:

  1. Open the storage account → Containers → + Container
  2. Name: reports
  3. Public access level: Private

Step 4 — Application Insights

4a — Log Analytics Workspace

Portal: Search → Log Analytics workspaces → Create

  • Resource group and region: same as above
  • Name: e.g. vigil-logs
  • Pricing tier: Pay-as-you-go

4b — Application Insights

Portal: Search → Application Insights → Create

  • Resource group and region: same as above
  • Name: e.g. vigil-appinsights
  • Resource mode: Workspace-based
  • Log Analytics workspace: select the one you just created

Step 5 — Function App

Portal: Search → Function App → Create

Setting Value
Resource group Same as above
Function App name Globally unique (e.g. vigil-func-abc123)
Runtime stack Python
Version 3.12
Region Same as above
OS Linux
Hosting plan Consumption (Serverless)
Storage account Select the Function runtime storage account from Step 3a
Application Insights Select the instance from Step 4b

Enable system-assigned managed identity

After the Function App is created:

  1. Open the Function App → Identity (left menu)
  2. Under System assigned → set Status to On → Save
  3. Note the Object (principal) ID — you will need it for RBAC in Step 6

Step 6 — RBAC assignments

All three assignments use the Function App's system-assigned managed identity as the principal.

Go to each resource below, open Access control (IAM) → Add → Add role assignment:

Synapse Workspace → Function App

Field Value
Resource Your existing Synapse workspace
Role Synapse Monitoring Operator
Assign access to Managed identity
Select Your Function App

Note: Synapse roles are assigned via Synapse Studio, not the Azure Portal. Open Synapse Studio → Manage → Access control → Add → select the role and your Function App's managed identity.

Reports Storage Account → Function App

Field Value
Resource Reports storage account (Step 3b)
Role Storage Blob Data Contributor
Assign access to Managed identity
Select Your Function App

Azure Communication Services → Function App

Field Value
Resource ACS resource (Step 2a)
Role Azure Communication Service Email Sender
Assign access to Managed identity
Select Your Function App

Step 7 — Function App configuration

Portal: Function App → Configuration → Application settings → + New application setting

Add each of the following:

Required

Setting Value
SYNAPSE_ENDPOINT https://<your-workspace-name>.dev.azuresynapse.net
ACS_ENDPOINT https://<your-acs-resource>.communication.azure.com
SENDER_ADDRESS The verified sender address from Step 2b
RECIPIENT_ADDRESSES Comma-separated list of report recipients
BLOB_STORAGE_ACCOUNT_URL https://<reports-storage-account>.blob.core.windows.net
BLOB_CONTAINER_NAME reports (or your container)
BLOB_FOLDER_PATH dailysnapshots (or your folder path)

Optional (defaults shown)

Setting Default Description
MANAGED_IDENTITY_CLIENT_ID (empty) Only needed if using a user-assigned identity
HOURS_BACK 24 Hours of pipeline history to include in each report
MONITOR_SCHEDULE 0 0 6,8 * * * NCRONTAB schedule — default is 06:00 and 08:00 UTC daily

Archiving is mandatory in production. If any blob archive setting is missing, Vigil fails fast instead of silently skipping archive output.

Click Save after adding all settings.

Step 8 — Deploy the code

One-time GitHub Actions setup

The repository includes a deploy workflow template at .github/workflows/deploy.example.yml. To activate it:

  1. Copy it to .github/workflows/deploy.yml in your fork
  2. Add a repository variable: Settings → Variables → Actions → New repository variable
    • Name: AZURE_FUNCTIONAPP_NAME
    • Value: your Function App name from Step 5
  3. Add a repository secret: Settings → Secrets → Actions → New repository secret
    • Name: AZURE_PUBLISH_PROFILE
    • Value: paste the publish profile XML from Function App → Overview → Get publish profile
  4. (Optional) Create a production environment under Settings → Environments — the deploy job targets it by default, which lets you add approval gates or environment-level secrets later

Deploy

Once the workflow file is in place, push to main:

git push origin main

GitHub Actions will run tests then deploy automatically. You can also trigger a deploy manually from the Actions tab using the workflow_dispatch trigger.

The default CI/CD path uses a publish profile secret to keep adoption simple for forks. OIDC/federated identity is optional for teams that want stricter enterprise deployment controls.

Verifying the deployment

Trigger the function manually to confirm everything is wired up:

# Linux/macOS
curl -X POST "https://<func-name>.azurewebsites.net/admin/functions/vigil_monitor" \
  -H "x-functions-key: <host-key>" \
  -H "Content-Type: application/json" \
  -d "{}"

# Windows PowerShell
curl.exe -X POST "https://<func-name>.azurewebsites.net/admin/functions/vigil_monitor" `
  -H "x-functions-key: <host-key>" `
  -H "Content-Type: application/json" `
  -d "{}"

Get <host-key> from: Function App → App keys → copy the default key.

What to check:

Check Where
Email arrived Your recipient inbox
CSV uploaded Reports storage account → Containers → reports/dailysnapshots/
No errors Function App → Application Insights → Logs → query traces