Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 0 additions & 57 deletions docs/concepts/data-management/data-forwarding.mdx

This file was deleted.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
title: Amazon SQS
sidebar_order: 3
description: >-
Learn about Sentry's Amazon SQS data forwarder, which enables creating custom data pipelines with Sentry errors through AWS.
---

<Alert>
Data Forwarding is currently in a limited preview, and may not be accessible to all organizations.
</Alert>

Amazon SQS is a generic queueing service provided by Amazon Web Services that Sentry supports sending error events to. The SQS forwarder may be useful for building your own data pipelines that might require Sentry errors as a source to eventually compare with some other data living elsewhere.

Sentry supports Amazon SQS as a first-party **Data Forwarder**, which currently only supports forwarding error events. For more details on Data Forwarding in general (including basic setup and overrides), check out the [Data Forwarding](/organization/integrations/data-forwarding/) documentation.

## Installation

Navigate to **Settings » Data Forwarding** and add a new forwarder.

Ensure you have selected _Amazon SQS_ as the provider. For standard queues, the required fields are the **Queue URL**, **Region**, **Access Key**, and **Secret Key**.
If you are using a FIFO queue (`.fifo` suffix on your Queue URL), you will also need to provide a **Message Group ID**.

Optionally, you can also provide an **S3 Bucket** for storing events. This will allow forwarding of event payloads greater than 256KB, and the SQS message will contain the bucket URL for the event instead of the event payload itself.

![Amazon SQS data forwarder configuration form](./img/sentry-config-form.png)

For an AWS Access Key and Secret Key, you can get these from the AWS IAM console. Navigate to **IAM > Users > [Your User] > Security credentials** and click on **Create access key**.

![Amazon IAM access key settings](./img/iam-config.png)

For the Queue URL, Navigate to **Amazon SQS > Queues > [Your Queue]** and click on the name of the queue you'd like to use, or create a new one. Afterward, the details should be visible. This also apples to the S3 Bucket (if you are using one, **Amazon S3 > Buckets > [Your Bucket]**).

![Amazon SQS queue details](./img/sqs-config.png)

If you are using an S3 Bucket, ensure it is in the same region as the SQS queue.

The payload for the SQS message (or S3 bucket object) is identical to our standard API event payload, and will evolve over time. For more details on the format of this data, see our [API documentation](/api/events/retrieve-an-event-for-a-project/).

Once you’ve filled in the required fields, and added any projects you'd like events forwarded for, click **Complete Setup**. These projects will now begin forwarding events to your SQS queue!

If you modify any of the above details from the AWS console, you'll have to update the forwarder you've configured in Sentry as well, manually.

If you wish to modify your forwarder configuration per project, you can do so by following the [Project Overrides documentation](/organization/integrations/data-forwarding/#project-overrides).

## Troubleshooting

- Unlike the other data forwarders, Sentry does not rate limit its sending of events to Amazon SQS. This is separate from the rate limit Amazon SQS enforces. Keep that in mind if you are noticing inconsistencies in your data, or missing events.
- Double check that your specified region is correct, both when monitoring the queue/bucket in AWS, and configuring the forwarder in Sentry. Sentry expects the region in the format of `us-east-1`, `us-west-1`, etc.
- If your Sentry event payloads are greater than 256KB and you have not provided an S3 Bucket, the event will _not_ be forwarded to your SQS queue. You'll need to setup an S3 Bucket for these events to be forwarded.
- If you are using an S3 Bucket, ensure it is in the same AWS region as the SQS queue otherwise you may experience issues with forwarding events.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
---
title: Grafana
sidebar_order: 1
sidebar_order: 4
description: "Learn about Sentry's Grafana integration, which allows you to query and visualize Sentry data within Grafana."
---

<Alert level="warning">

If you make changes to your organization slug, you'll need to update your configuration for this integration. Learn more in our [troubleshooting guide](/organization/integrations/troubleshooting).

</Alert>

Visualize your Sentry [Issues](/product/issues/) and [Stats](/product/stats/) data in Grafana with the Sentry [data source plugin](https://grafana.com/grafana/plugins/grafana-sentry-datasource/). Once installed, you can combine Sentry data with that of other sources, like your infrastructure data through Prometheus or your business data from your data warehouse, in a single dashboard.

Unlike [Segment](../segment/), [Amazon SQS](../amazon-sqs/), and [Splunk](../splunk/), Grafana is not supported as a first-party data forwarder. Instead, you can find the installation instructions in the integration directory page.

![Grafana integration directory listing](./img/grafana-install.png)

Note: These instructions and the Grafana integration are maintained by Grafana Labs.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file not shown.
Binary file not shown.
94 changes: 58 additions & 36 deletions docs/organization/integrations/data-forwarding/index.mdx
Original file line number Diff line number Diff line change
@@ -1,58 +1,80 @@
---
title: "Data Forwarding"
sidebar_order: 3
sidebar_order: 60
description: "Learn about forwarding processed events to third-party providers."
---

<Alert>
Data Forwarding is currently in a limited preview, and may not be accessible to all organizations.
</Alert>

<Include name="feature-available-for-plan-trial-business.mdx" />

<Alert>
Sentry provides the ability to forward processed error events to certain third-party providers, namely [Segment](https://segment.com), [Amazon SQS](https://aws.amazon.com/sqs/), and [Splunk](https://www.splunk.com/).

This is often useful when you may want to analyze errors in relation to other datasets, empower teams outside of engineering, and other business intelligence use cases.
Currently, only error events will be forwarded. Forwarding of other event types (such as spans or logs) is not supported.

The plugins listed on this page have been deprecated and will be removed from Sentry in a future date.
For more details on setup and troubleshooting with specific data forwarders, refer to the provider documentation below:
- [Splunk](/organization/integrations/data-forwarding/splunk/)
- [Amazon SQS](/organization/integrations/data-forwarding/amazon-sqs/)
- [Segment](/organization/integrations/data-forwarding/segment/)

Note: Grafana is not a first-party data forwarder, and must be configured separately. For more details, refer to the [these docs](/organization/integrations/data-forwarding/grafana/).

## Global Configuration

<Alert>
Sentry owner or manager permissions are required to setup/modify a global data forwarder.
</Alert>

Sentry provides the ability to forward processed error events to certain third-party providers, namely [Segment](https://segment.com), [Amazon SQS](https://aws.amazon.com/sqs/), and [Splunk](https://www.splunk.com/).
With a global data forwarding configuration, you can setup a single forwarder for your organization and apply it to how ever many projects you'd like, with the additional ability to automatically begin forwarding events from new projects as soon as they're created.

To setup a global forwarder, navigate to **Settings » Data Forwarding**, and start the setup for a new provider. Note that we only allow one forwarder per provider per organization.

![Data forwarding landing page](./img/df-landing-page.png)

When creating a new forwarder, select the provider you'd like to forward events to and fill in the required fields. Each provider requires different configuration and permissions on that service, so please refer to the documentation below for more details on your specific provider. After a forwarder is created, you will not be able to switch providers, but you can modify the forwarding details at any time.

![Data forwarding global configuration form](./splunk/img/sentry-config-form.png)

You cannot create project-level overrides until you've setup the forwarder for the first time. Any projects you connect to the forwarder will begin forwarding (with the global configuration) as soon you complete the setup. You can also specify if new projects should automatically adopt the global configuration now.

![Data forwarding project configuration form](./img/df-project-form.png)

After completing the setup, you will be redirected to the forwarder list page, where you can see the newly-enabled forwarder and it's current status. Now you can edit the forwarder to add project-level overrides, if necessary.

![Data forwarding edit list page](./img/df-edit-page.png)

## Project Overrides

<Alert>
Owner, manager, or admin permissions are required to create/modify project-level overrides.

Users with Team Admin permissions can create/modify overrides as well, but only for their respective teams' projects.
</Alert>

This is often useful when you may want to analyze exceptions more deeply, or empower other teams, such as a Business Intelligence function.
You can manage your project overrides by navigating to **Settings » Data Forwarding » [Provider] » Edit** and scrolling to **Manage your overrides**.

Only error events will be forwarded. Forwarding of transaction events is not supported.
A project override will allow you to replace specified fields in the global configuration with custom values for a particular project. All the previously required fields on the provider are marked as optional, and only the specified fields will be overridden. If no value is provided for a field, the global configuration will be used.

Configure Data Forwarding for your organization by navigating to **Settings » Data Forwarding**, then providing the required information for the given integration. Note that only one org-level forwarder can be created for each integration. However,
configs can be overridden with [project overrides](./project-overrides).
![Data forwarding project override form](./img/df-override-form.png)

## Amazon SQS
Empty values (`""`) are permitted in the override form, so manually deleting an override will _not_ revert to the global configuration. If you wish to remove an override, be sure to click the **Clear Override** button at the bottom of the form. After saving, be sure to check the status of the override in the corner of it's form.

Integration with Amazon SQS makes it quick and easy to pipe exceptions back into your own systems.
### Migrating from Legacy Integrations

The payload for Amazon is identical to our standard API event payload, and will evolve over time. For more details on the format of this data, see our [API documentation](/api/events/retrieve-an-event-for-a-project/).
If you've been a Sentry customer for a while, you may have seen a legacy version of data forwarding which was applied at the project level through legacy integrations (plugins). This is no longer supported, and organizations should see they've been automatically migrated to the new global (organization-level) configuration.

## Segment
The migration process will retain all the project-level configuration using the project-overrides, which you can manage/modify as needed.

The Segment integration will generate _Error Captured_ events within your data pipeline. These events will **only** be captured for `error` events, and only when an ID is present the user context. The general shape of the event will look roughly similar to:

```json
{
"userId": "1",
"event": "Error Captured",
"properties": {
"environment": "production",
"eventId": "002c8bbde8324dae9f12a0b96f5b1e51",
"exceptionType": "ValueError",
"release": "a2def1",
"transaction": "/api/0/users/{user}/",
"userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0",
"page": {
"url": "https://sentry.io/api/0/users/{user}/",
"method": "GET",
"search": "",
"referer": "https://sentry.io/"
}
},
"timestamp": "2017-05-20T15:29:06Z"
}
```
## Troubleshooting

## Splunk
For provider-specific troubleshooting, refer to the provider documentation below:
- [Splunk](/organization/integrations/data-forwarding/splunk/#troubleshooting)
- [Amazon SQS](/organization/integrations/data-forwarding/amazon-sqs/#troubleshooting)
- [Segment](/organization/integrations/data-forwarding/segment/#troubleshooting)
- [Grafana (Third-Party)](/organization/integrations/data-forwarding/grafana/)


For details on the Splunk integration, visit the dedicated [Splunk integration documentation](/organization/integrations/data-visualization/splunk/).

This file was deleted.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
64 changes: 64 additions & 0 deletions docs/organization/integrations/data-forwarding/segment/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
title: Segment
sidebar_order: 2
description: >-
Learn about Sentry's Segment data forwarder, which enables tracking Sentry errors as Segment events.
---

<Alert>
Data Forwarding is currently in a limited preview, and may not be accessible to all organizations.
</Alert>

Segment is useful for collecting, responsibly managing, and integrating your customer data with many other applications, including Sentry.

Sentry supports Segment as a first-party **Data Forwarder**, which currently only supports forwarding error events. For more details on Data Forwarding in general (including basic setup and overrides), check out the [Data Forwarding](/organization/integrations/data-forwarding/) documentation.

## Installation

Navigate to **Settings » Data Forwarding** and add a new forwarder.

Ensure you have selected _Segment_ as the provider. The only required field for Segment is the **Write Key**.

![Segment data forwarder configuration form](./img/sentry-config-form.png)

You can get this value in your Segment workspace by navigating to **Connections > Catalog** and searching for **HTTP API**. This is a generic server forwarder that has the write key we'll need.

![Segment HTTP API source in catalog](./img/segment-source-catalog.png)

Click on **Add Source**, and it should provide you with a write key you can copy into Sentry's configuration form.

![Segment source configuration view](./img/segment-source-config.png)

Once you’ve filled in the required fields, and added any projects you'd like events forwarded for, click **Complete Setup**. These projects will now begin forwarding events to your Segment workspace!

If you modify the Segment write key later on, you'll have to update the forwarder you've configured in Sentry as well, manually.

If you wish to modify your forwarder configuration per project, you can do so by following the [Project Overrides documentation](/organization/integrations/data-forwarding/#project-overrides).

## Troubleshooting

- Sentry rate limits its sending of events to Segment at approximately 200 requests per second per organization. This is separate from the rate limit Segment enforces. Keep that in mind if you are noticing inconsistencies in your data, or missing events.
- Sentry will only forward error events to Segment when a [User, with an ID](/platforms/python/enriching-events/identify-user/) is present in the event context.
- Sentry uses a fixed event name of "Error Captured" for the event that is forwarded to Segment. At the moment, this cannot be modified. The general shape of the event will look roughly similar to:

```json
{
"userId": "1",
"event": "Error Captured",
"properties": {
"environment": "production",
"eventId": "002c8bbde8324dae9f12a0b96f5b1e51",
"exceptionType": "ValueError",
"release": "a2def1",
"transaction": "/api/0/users/{user}/",
"userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0",
"page": {
"url": "https://sentry.io/api/0/users/{user}/",
"method": "GET",
"search": "",
"referer": "https://sentry.io/"
}
},
"timestamp": "2017-05-20T15:29:06Z"
}
```
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading