Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): Support for multiple outgoing webhooks added #7578

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Sweta-Kumari-Sharma
Copy link
Contributor

@Sweta-Kumari-Sharma Sweta-Kumari-Sharma commented Mar 19, 2025

Type of Change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring
  • Dependency updates
  • Documentation
  • CI/CD

Description

In the current system, merchants can configure only one webhook endpoint per profile. This PR introduces support for configuring multiple webhook endpoints based on the event_type. These changes include modifications to the database schema and alterations in the webhook triggering logic.

Database Schema Changes

  1. Business Profile Schema Update:
    The webhook_details column in the business_profile table is being updated from a JSON object to an array of JSON objects. This will allow each business profile to store multiple webhook endpoints that can be associated with different event types.

Old Schema (Single webhook configuration):
{
"webhook_url": "https://example.com/webhook",
"events": ["payment_succeeded"]
}

New Schema (Multiple webhook configurations):
[
{
"webhook_url": "https://example1.com/webhook",
"events": ["payment_succeeded", "order_created"]
},
{
"webhook_url": "https://example2.com/webhook",
"events": ["payment_failed"]
}
]
2. Merchant Account Schema Update:
The merchant_account table might also include the webhook_details field to store webhook configurations for the merchant, allowing the use of multiple endpoints for different events.

The database migrations will need to reflect these changes to handle an array of webhook configurations instead of a single configuration.

Event Handling Logic
Triggering the First Webhook: When an event (e.g., a payment) occurs, the first webhook endpoint is triggered. This is determined by matching the event_type against the webhook configurations stored in the business_profile. The first matching webhook endpoint is used to trigger the webhook request.

Process Tracker for Remaining Webhooks: For the remaining webhook endpoints, even though they won't be triggered immediately, the system will log these events in the process tracker for future attempts. This is useful for retrying failed webhook calls or ensuring that the webhooks are eventually delivered.

Webhook Retry Handling: Each webhook delivery attempt, whether initial or retry, will be recorded in the process_tracker. This ensures that failed webhook deliveries are retried.

Additional Changes

  • This PR modifies the API contract
  • This PR modifies the database schema
  • This PR modifies application configuration/environment variables

Motivation and Context

How did you test it?

  1. Merchant account create
curl --location 'http://localhost:8080/accounts' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'api-key: test_admin' \
--data-raw '{
  "merchant_id": "merchant_1742455206",
  "locker_id": "m0010",
  "merchant_name": "NewAge Retailer",
  "merchant_details": {
    "primary_contact_person": "John Test",
    "primary_email": "[email protected]",
    "primary_phone": "sunt laborum",
    "secondary_contact_person": "John Test2",
    "secondary_email": "[email protected]",
    "secondary_phone": "cillum do dolor id",
    "website": "https://www.example.com",
    "about_business": "Online Retail with a wide selection of organic products for North America",
    "address": {
      "line1": "1467",
      "line2": "Harrison Street",
      "line3": "Harrison Street",
      "city": "San Fransico",
      "state": "California",
      "zip": "94122",
      "country": "US",
      "first_name":"john",
      "last_name":"Doe"
    }
  },
  "return_url": "https://google.com/success",
  "webhook_details": [{
    "webhook_url":"https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
    "events": ["payment_succeeded"]
  },
  {
    "webhook_url":"https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
    "events": ["payment_succeeded","refund_failed"]
  },
  {
    "webhook_url":"https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
    "events": ["refund_failed","payment_succeeded"]
  },
  {
    "webhook_url":"https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
    "events": ["payment_succeeded"]
  }
  ],
  
  "sub_merchants_enabled": false,
  "parent_merchant_id":"merchant_123",
  "metadata": {
    "city": "NY",
    "unit": "245"
  },
  "primary_business_details": [
    {
      "country": "US",
      "business": "default"
    }
  ]
}'

Response

{
    "merchant_id": "merchant_1742455203",
    "merchant_name": "NewAge Retailer",
    "return_url": "https://google.com/success",
    "enable_payment_response_hash": true,
    "payment_response_hash_key": "7aRicO9KmdjjlJDuOkBAopiLwDSEd3gGlI9axVuLWJuyrzXVFvEAwwqNieVnpItV",
    "redirect_to_merchant_with_http_post": false,
    "merchant_details": {
        "primary_contact_person": "John Test",
        "primary_phone": "sunt laborum",
        "primary_email": "[email protected]",
        "secondary_contact_person": "John Test2",
        "secondary_phone": "cillum do dolor id",
        "secondary_email": "[email protected]",
        "website": "https://www.example.com",
        "about_business": "Online Retail with a wide selection of organic products for North America",
        "address": {
            "city": "San Fransico",
            "country": "US",
            "line1": "1467",
            "line2": "Harrison Street",
            "line3": "Harrison Street",
            "zip": "94122",
            "state": "California",
            "first_name": "john",
            "last_name": "Doe"
        }
    },
    "webhook_details": [
        {
            "webhook_endpoint_id": "web_tPI8EiOff9DiQa5MNm5c",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "payment_succeeded"
            ],
            "status": "active"
        },
        {
            "webhook_endpoint_id": "web_H4agvxgq9tuRKweKl312",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "payment_succeeded",
                "refund_failed"
            ],
            "status": "active"
        },
        {
            "webhook_endpoint_id": "web_3E1nGa7ARCOQp3Xj05AU",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "refund_failed",
                "payment_succeeded"
            ],
            "status": "active"
        },
        {
            "webhook_endpoint_id": "web_PunZUsKuClhZRAhIn6aj",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "payment_succeeded"
            ],
            "status": "active"
        }
    ],
    "payout_routing_algorithm": null,
    "sub_merchants_enabled": false,
    "parent_merchant_id": null,
    "publishable_key": "pk_dev_4b3306f3862a4a22ab6538aa86c7cbf6",
    "metadata": {
        "city": "NY",
        "unit": "245",
        "compatible_connector": null
    },
    "locker_id": "m0010",
    "primary_business_details": [
        {
            "country": "US",
            "business": "default"
        }
    ],
    "frm_routing_algorithm": null,
    "organization_id": "org_n7iGzdWRqtzDwBExlREm",
    "is_recon_enabled": false,
    "default_profile": "pro_P4LiydzsnrV6Sw6tKjie",
    "recon_status": "not_requested",
    "pm_collect_link_config": null
}
  1. Updating business profile
curl --location 'http://localhost:8080/account/merchant_1742455203/business_profile/pro_P4LiydzsnrV6Sw6tKjie' \
--header 'Content-Type: application/json' \
--header 'api-key: test_admin' \
--data '{
    "profile_name": "Update",
    "return_url": "https://google.com/success",
    "enable_payment_response_hash": true,
    "redirect_to_merchant_with_http_post": false,
    "webhook_details": [{
            "webhook_endpoint_id": "web_tPI8EiOff9DiQa5MNm5c",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "payment_succeeded"
            ],
            "status": "active"
        },
        {
            "webhook_endpoint_id": "web_H4agvxgq9tuRKweKl312",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "payment_succeeded",
                "refund_failed"
            ],
            "status": "active"
        },
        {
            "webhook_endpoint_id": "web_3E1nGa7ARCOQp3Xj05AU",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "refund_failed",
                "payment_succeeded"
            ],
            "status": "deprecated"
        },
        {
            "webhook_endpoint_id": "web_PunZUsKuClhZRAhIn6aj",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "payment_succeeded"
            ],
            "status": "inactive"
        }
    ],
    "metadata": null,
    
    
    
    
    "intent_fulfillment_time": 900,
    "frm_routing_algorithm": null,
    "payout_routing_algorithm": null,
    "applepay_verified_domains": null,
    "session_expiry": 900,
    "payment_link_config": null,
    "authentication_connector_details": null,
    "use_billing_as_payment_method_billing": true,
    "collect_shipping_details_from_wallet_connector": false,
    "collect_billing_details_from_wallet_connector": false,
    "is_connector_agnostic_mit_enabled": false,
    "payout_link_config": null,
    "outgoing_webhook_custom_http_headers": null
}'

Response

{
    "merchant_id": "merchant_1742455203",
    "profile_id": "pro_P4LiydzsnrV6Sw6tKjie",
    "profile_name": "Update",
    "return_url": "https://google.com/success",
    "enable_payment_response_hash": true,
    "payment_response_hash_key": "7aRicO9KmdjjlJDuOkBAopiLwDSEd3gGlI9axVuLWJuyrzXVFvEAwwqNieVnpItV",
    "redirect_to_merchant_with_http_post": false,
    "webhook_details": [
        {
            "webhook_endpoint_id": "web_tPI8EiOff9DiQa5MNm5c",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "payment_succeeded"
            ],
            "status": "active"
        },
        {
            "webhook_endpoint_id": "web_H4agvxgq9tuRKweKl312",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "payment_succeeded",
                "refund_failed"
            ],
            "status": "active"
        },
        {
            "webhook_endpoint_id": "web_3E1nGa7ARCOQp3Xj05AU",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "refund_failed",
                "payment_succeeded"
            ],
            "status": "deprecated"
        },
        {
            "webhook_endpoint_id": "web_PunZUsKuClhZRAhIn6aj",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "payment_succeeded"
            ],
            "status": "inactive"
        }
    ],
    "metadata": null,
    "routing_algorithm": null,
    "intent_fulfillment_time": 900,
    "frm_routing_algorithm": null,
    "payout_routing_algorithm": null,
    "applepay_verified_domains": null,
    "session_expiry": 900,
    "payment_link_config": null,
    "authentication_connector_details": null,
    "use_billing_as_payment_method_billing": true,
    "extended_card_info_config": null,
    "collect_shipping_details_from_wallet_connector": false,
    "collect_billing_details_from_wallet_connector": false,
    "always_collect_shipping_details_from_wallet_connector": false,
    "always_collect_billing_details_from_wallet_connector": false,
    "is_connector_agnostic_mit_enabled": false,
    "payout_link_config": null,
    "outgoing_webhook_custom_http_headers": null,
    "tax_connector_id": null,
    "is_tax_connector_enabled": false,
    "is_network_tokenization_enabled": false,
    "is_auto_retries_enabled": false,
    "max_auto_retries_enabled": null,
    "always_request_extended_authorization": null,
    "is_click_to_pay_enabled": false,
    "authentication_product_ids": null,
    "card_testing_guard_config": {
        "card_ip_blocking_status": "disabled",
        "card_ip_blocking_threshold": 3,
        "guest_user_card_blocking_status": "disabled",
        "guest_user_card_blocking_threshold": 10,
        "customer_id_blocking_status": "disabled",
        "customer_id_blocking_threshold": 5,
        "card_testing_guard_expiry": 3600
    },
    "is_clear_pan_retries_enabled": false
}
  1. Fetching profile
{
    "merchant_id": "merchant_1742455203",
    "profile_id": "pro_P4LiydzsnrV6Sw6tKjie",
    "profile_name": "Update",
    "return_url": "https://google.com/success",
    "enable_payment_response_hash": true,
    "payment_response_hash_key": "7aRicO9KmdjjlJDuOkBAopiLwDSEd3gGlI9axVuLWJuyrzXVFvEAwwqNieVnpItV",
    "redirect_to_merchant_with_http_post": false,
    "webhook_details": [
        {
            "webhook_endpoint_id": "web_tPI8EiOff9DiQa5MNm5c",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "payment_succeeded"
            ],
            "status": "active"
        },
        {
            "webhook_endpoint_id": "web_H4agvxgq9tuRKweKl312",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "payment_succeeded",
                "refund_failed"
            ],
            "status": "active"
        },
        {
            "webhook_endpoint_id": "web_PunZUsKuClhZRAhIn6aj",
            "webhook_url": "https://webhook.site/80c342e2-5447-40f8-b470-7f80c7464050",
            "events": [
                "payment_succeeded"
            ],
            "status": "inactive"
        }
    ],
    "metadata": null,
    "routing_algorithm": null,
    "intent_fulfillment_time": 900,
    "frm_routing_algorithm": null,
    "payout_routing_algorithm": null,
    "applepay_verified_domains": null,
    "session_expiry": 900,
    "payment_link_config": null,
    "authentication_connector_details": null,
    "use_billing_as_payment_method_billing": true,
    "extended_card_info_config": null,
    "collect_shipping_details_from_wallet_connector": false,
    "collect_billing_details_from_wallet_connector": false,
    "always_collect_shipping_details_from_wallet_connector": false,
    "always_collect_billing_details_from_wallet_connector": false,
    "is_connector_agnostic_mit_enabled": false,
    "payout_link_config": null,
    "outgoing_webhook_custom_http_headers": null,
    "tax_connector_id": null,
    "is_tax_connector_enabled": false,
    "is_network_tokenization_enabled": false,
    "is_auto_retries_enabled": false,
    "max_auto_retries_enabled": null,
    "always_request_extended_authorization": null,
    "is_click_to_pay_enabled": false,
    "authentication_product_ids": null,
    "card_testing_guard_config": {
        "card_ip_blocking_status": "disabled",
        "card_ip_blocking_threshold": 3,
        "guest_user_card_blocking_status": "disabled",
        "guest_user_card_blocking_threshold": 10,
        "customer_id_blocking_status": "disabled",
        "customer_id_blocking_threshold": 5,
        "card_testing_guard_expiry": 3600
    },
    "is_clear_pan_retries_enabled": false
}

Checklist

  • I formatted the code cargo +nightly fmt --all
  • I addressed lints thrown by cargo clippy
  • I reviewed the submitted code
  • I added unit tests for my changes where possible

@Sweta-Kumari-Sharma Sweta-Kumari-Sharma added A-core Area: Core flows C-feature Category: Feature request or enhancement labels Mar 19, 2025
@Sweta-Kumari-Sharma Sweta-Kumari-Sharma self-assigned this Mar 19, 2025
@Sweta-Kumari-Sharma Sweta-Kumari-Sharma requested review from a team as code owners March 19, 2025 19:18
Copy link

semanticdiff-com bot commented Mar 19, 2025

Review changes with  SemanticDiff

Changed Files
File Status
  crates/hyperswitch_domain_models/src/business_profile.rs  73% smaller
  crates/diesel_models/src/business_profile.rs  34% smaller
  crates/api_models/src/admin.rs  31% smaller
  crates/diesel_models/src/schema.rs  21% smaller
  crates/router/src/core/webhooks/utils.rs  19% smaller
  crates/router/src/core/webhooks/outgoing.rs  15% smaller
  crates/router/src/types/transformers.rs  10% smaller
  crates/router/src/core/admin.rs  5% smaller
  crates/api_models/src/webhook_events.rs  0% smaller
  crates/common_enums/src/enums.rs  0% smaller
  crates/common_utils/src/id_type.rs  0% smaller
  crates/common_utils/src/id_type/webhook_endpoint.rs  0% smaller
  crates/common_utils/src/lib.rs  0% smaller
  crates/diesel_models/src/events.rs  0% smaller
  crates/diesel_models/src/merchant_account.rs  0% smaller
  crates/hyperswitch_domain_models/src/merchant_account.rs  0% smaller
  crates/router/src/core/webhooks/webhook_events.rs  0% smaller
  crates/router/src/db/events.rs  0% smaller
  crates/router/src/routes/profiles.rs  0% smaller
  crates/router/src/types/api/admin.rs  0% smaller
  crates/router/src/types/domain/event.rs  0% smaller
  crates/router/src/workflows/outgoing_webhook_retry.rs  0% smaller
  migrations/2025-03-06-104240_alter_business_profile_table/down.sql Unsupported file format
  migrations/2025-03-06-104240_alter_business_profile_table/up.sql Unsupported file format
  migrations/2025-03-11-090407_alter_merchant_account_table/down.sql Unsupported file format
  migrations/2025-03-11-090407_alter_merchant_account_table/up.sql Unsupported file format
  migrations/2025-03-17-080827_alter_events_table/down.sql Unsupported file format
  migrations/2025-03-17-080827_alter_events_table/up.sql Unsupported file format

@hyperswitch-bot hyperswitch-bot bot added the M-database-changes Metadata: This PR involves database schema changes label Mar 19, 2025
@@ -52,7 +52,7 @@ pub struct MerchantAccountCreate {
pub return_url: Option<url::Url>,

/// Webhook related details
pub webhook_details: Option<WebhookDetails>,
pub webhook_details: Option<Vec<Option<WebhookDetails>>>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two options here, which seem redundant. Is there any specific reason to do so?

///The password for Webhook login
#[schema(value_type = Option<String>, max_length = 255, example = "ekart@123")]
pub webhook_password: Option<Secret<String>>,
pub webhook_endpoint_id: Option<id_type::WebhookEndpointId>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this change backwards compatible?

@@ -239,6 +239,13 @@ pub fn generate_profile_id_of_default_length() -> id_type::ProfileId {
id_type::ProfileId::generate()
}

/// Generate a profile id with default length, with prefix as `pro`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change the doc comment

#[serde(rename_all = "snake_case")]
#[strum(serialize_all = "snake_case")]
pub enum OutgoingWebhookEndpointStatus {
Active,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please provide description to these enums. What is the difference between inactive and deprecated? if once made as deprecated, it cannot be made active?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-core Area: Core flows C-feature Category: Feature request or enhancement M-database-changes Metadata: This PR involves database schema changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants