Added description of failure scenarios when code uses databases and brokers together #3609

udidahan · 2022-09-04T11:25:45Z

When developers are building mission-critical systems, they need to be aware of the various failure scenarios and how to handle them.

This PR gives them one detailed example and points them at resources that explain the topic further.

…rokers together

prmerger-automator · 2022-09-04T11:25:58Z

@udidahan : Thanks for your contribution! The author(s) have been notified to review your proposed change.

udidahan · 2022-09-04T11:28:08Z

@EdPrice-MSFT is this more the kind of thing you had in mind? Thanks!

Jak-MS · 2022-09-06T15:37:33Z

@msimecek

Can you review this PR?

IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.

cc: @EdPrice-MSFT

#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

msimecek

Thank you for the contribution! A couple of comments from me.

msimecek · 2022-09-06T16:06:33Z

docs/reference-architectures/containers/aks-mission-critical/mission-critical-data-platform.md

+  - One common technique of making message handling idempotent is to check some persistent store, like a database, if the message has already been processed, and if so, not running the logic to process it again. 
+  - However, if the processing of the message includes database operations, specifically the insertion of new records with database-generated identifiers, as well as the emitting of new messages to the broker containing those identifiers, because there aren't distributed transactions which encompass both the database and the message broker, there can be a number of complications that can occur if the process running the code happens to fail, for example:
+    - If the code emitting the messages runs before the database transaction is committed, which is how many developers work using the Unit of Work pattern, those messages can "escape" if the failure occurs between calling the broker and asking that the database transaction be committed. As the transaction rolls back, those database-generated IDs are also undone, leaving them available to other code which may be running at the same time. This can cause recipients of the "escaped" messages to processes the wrong database entries, hurting the overall consistency and correctness of your system.
+    - For a deeper discussion on these failure scenarios and how to handle them, see [this webinar on the topic](https://particular.net/webinars/implementing-an-outbox).


Suggested change

- For a deeper discussion on these failure scenarios and how to handle them, see [this webinar on the topic](https://particular.net/webinars/implementing-an-outbox).

Please remove this. A link to your company's webinar, promoting your product, is not part of Azure Mission-critical guidance.

docs/reference-architectures/containers/aks-mission-critical/mission-critical-data-platform.md

Ed-Price · 2022-09-08T00:24:01Z

@EdPrice-MSFT is this more the kind of thing you had in mind? Thanks!

I was thinking more like having you create a new architecture, like a Solution Idea, which shows how Azure services work with your company's solutions.

Regarding this PR...

See feedback from @msimecek. Thanks, Martin!

ckittel · 2022-09-13T13:36:14Z

@udidahan do you plan on continuing this PR and following up with the suggestions from the current doc co-owner, @msimecek?

prmerger-automator · 2022-09-16T16:16:40Z

@udidahan : Thanks for your contribution! The author(s) have been notified to review your proposed change.

opbld31 · 2022-09-16T16:17:49Z

Docs Build status updates of commit 6c9acd3:

⚠️ Validation status: warnings

File	Status	Preview URL	Details
docs/reference-architectures/containers/aks-mission-critical/mission-critical-data-platform.md	⚠️Warning		Details

docs/reference-architectures/containers/aks-mission-critical/mission-critical-data-platform.md

Line 165, Column 59: [Warning: hard-coded-locale - See documentation] Link 'https://docs.microsoft.com/en-us/azure/architecture/best-practices/transactional-outbox-cosmos' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.
Line 165, Column 59: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://docs.microsoft.com/en-us/azure/architecture/best-practices/transactional-outbox-cosmos' will be broken in isolated environments. Replace with a relative link.

For more details, please refer to the build report.

If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.

Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.

Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.

For any questions, please:

Try searching the docs.microsoft.com contributor guides
Post your question in the Docs support channel

prmerger-automator · 2022-09-16T16:19:16Z

@udidahan : Thanks for your contribution! The author(s) have been notified to review your proposed change.

opbld30 · 2022-09-16T16:21:27Z

Docs Build status updates of commit 17f0044:

✅ Validation status: passed

File	Status	Preview URL	Details
docs/reference-architectures/containers/aks-mission-critical/mission-critical-data-platform.md	✅Succeeded

For more details, please refer to the build report.

Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.

For any questions, please:

Try searching the docs.microsoft.com contributor guides
Post your question in the Docs support channel

prmerger-automator · 2022-09-16T16:26:17Z

@udidahan : Thanks for your contribution! The author(s) have been notified to review your proposed change.

opbld32 · 2022-09-16T16:29:08Z

Docs Build status updates of commit 0349a10:

✅ Validation status: passed

File	Status	Preview URL	Details
docs/reference-architectures/containers/aks-mission-critical/mission-critical-data-platform.md	✅Succeeded

For more details, please refer to the build report.

Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.

For any questions, please:

Try searching the docs.microsoft.com contributor guides
Post your question in the Docs support channel

udidahan · 2022-09-16T16:30:30Z

I hope these changes address all the concerns raised. Thanks.

prmerger-automator · 2022-09-19T13:33:41Z

@udidahan : Thanks for your contribution! The author(s) have been notified to review your proposed change.

opbld30 · 2022-09-19T13:33:53Z

Docs Build status updates of commit 5152ded:

❌ Validation status: errors

Please follow instructions here which may help to resolve issue.

File	Status	Preview URL	Details
	❌Error		Details

[Error: RuningBuildFailed] Some unexpected errors happened when running build, please open a ticket in https://aka.ms/SiteHelp and include the error report for our team to troubleshoot

For more details, please refer to the build report.

If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.

Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.

Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.

For any questions, please:

Try searching the docs.microsoft.com contributor guides
Post your question in the Docs support channel

…ission-critical-data-platform.md Co-authored-by: Martin Simecek <[email protected]>

prmerger-automator · 2022-09-19T13:34:16Z

@udidahan : Thanks for your contribution! The author(s) have been notified to review your proposed change.

opbld31 · 2022-09-19T13:34:27Z

Docs Build status updates of commit 1c9e4fb:

❌ Validation status: errors

Please follow instructions here which may help to resolve issue.

File	Status	Preview URL	Details
	❌Error		Details

[Error: RuningBuildFailed] Some unexpected errors happened when running build, please open a ticket in https://aka.ms/SiteHelp and include the error report for our team to troubleshoot

For more details, please refer to the build report.

If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.

Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.

Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.

For any questions, please:

Try searching the docs.microsoft.com contributor guides
Post your question in the Docs support channel

udidahan · 2022-09-19T13:35:48Z

Changes incorporated 👍🏼

docs/reference-architectures/containers/aks-mission-critical/mission-critical-data-platform.md

…ission-critical-data-platform.md

prmerger-automator · 2022-09-19T13:53:56Z

@ckittel : Thanks for your contribution! The author(s) have been notified to review your proposed change.

opbld31 · 2022-09-19T13:54:06Z

Docs Build status updates of commit bcb5a07:

❌ Validation status: errors

Please follow instructions here which may help to resolve issue.

File	Status	Preview URL	Details
	❌Error		Details

[Error: RuningBuildFailed] Some unexpected errors happened when running build, please open a ticket in https://aka.ms/SiteHelp and include the error report for our team to troubleshoot

For more details, please refer to the build report.

If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.

Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.

Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.

For any questions, please:

Try searching the docs.microsoft.com contributor guides
Post your question in the Docs support channel

ckittel · 2022-09-19T13:54:14Z

Added a commit just to see if I can get the build to kick off again (might have an upstream problem here)

ckittel · 2022-09-19T13:55:31Z

Some unexpected errors happened when running build, please open a ticket in https://aka.ms/SiteHelp and include the error report for our team to troubleshoot

@udidahan -- looks like our build process is currently being temperamental. Nothing for you to address here, we'll keep trying.

udidahan · 2022-09-19T17:23:46Z

Thanks for letting me know. I was a bit worried 😅

Ed-Price

Finished edit. Looks great!

Let's lead the RFC quote to set it up better. Finished edit pass.

prmerger-automator · 2022-09-19T18:24:00Z

@EdPrice-MSFT : Thanks for your contribution! The author(s) have been notified to review your proposed change.

opbld33 · 2022-09-19T18:24:07Z

Docs Build status updates of commit edb5dee:

❌ Validation status: errors

Please follow instructions here which may help to resolve issue.

File	Status	Preview URL	Details
	❌Error		Details

[Error: RuningBuildFailed] Some unexpected errors happened when running build, please open a ticket in https://aka.ms/SiteHelp and include the error report for our team to troubleshoot

For more details, please refer to the build report.

If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.

Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.

Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.

For any questions, please:

Try searching the docs.microsoft.com contributor guides
Post your question in the Docs support channel

Ed-Price · 2022-09-19T18:30:51Z

I created an incident to resolve the build error: https://portal.microsofticm.com/imp/v3/incidents/details/336037212/home

Ed-Price

Made an edit pass. Please review.

Looks great!

Ed-Price · 2022-09-19T18:31:54Z

#Sign-Off

prmerger-automator · 2022-09-19T18:31:57Z

Invalid command: '#sign-off'. Only the assigned author of one or more file in this PR can sign off. @msimecek

opbld33 · 2022-09-19T20:24:19Z

Docs Build status updates of commit edb5dee:

❌ Validation status: errors

Please follow instructions here which may help to resolve issue.

File	Status	Preview URL	Details
	❌Error		Details

[Error: RuningBuildFailed] Some unexpected errors happened when running build, please open a ticket in https://aka.ms/SiteHelp and include the error report for our team to troubleshoot

For more details, please refer to the build report.

If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.

Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.

Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.

For any questions, please:

Try searching the docs.microsoft.com contributor guides
Post your question in the Docs support channel

udidahan · 2022-09-20T05:59:19Z

Thanks everyone for helping get this across the finish line.

BTW, how long does it usually take for these merges to make their way to the public site? The reason I ask is that I don't see the changes on the public site:

https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/containers/aks-mission-critical/mission-critical-data-platform#every-message-must-be-processed

udidahan · 2022-09-21T14:35:05Z

OK. I see that it's live now. Thank you all.

Added description of failure scenarios when code uses databases and b…

fb1574b

…rokers together

prmerger-automator bot added the do-not-merge label Sep 4, 2022

prmerger-automator bot requested a review from msimecek September 4, 2022 11:25

prmerger-automator bot assigned msimecek Sep 4, 2022

prmerger-automator bot added Change sent to author architecture-center/svc reference-architecture/subsvc labels Sep 4, 2022

prmerger-automator bot added the aq-pr-triaged Tracking label for the PR review team label Sep 6, 2022

msimecek suggested changes Sep 6, 2022

View reviewed changes

ckittel marked this pull request as draft September 9, 2022 13:56

Update mission-critical-data-platform.md

6c9acd3

prmerger-automator bot requested a review from msimecek September 16, 2022 16:16

Fix link

17f0044

prmerger-automator bot added the qualifies-for-auto-merge label Sep 16, 2022

Update mission-critical-data-platform.md

0349a10

prmerger-automator bot removed the qualifies-for-auto-merge label Sep 16, 2022

prmerger-automator bot added the qualifies-for-auto-merge label Sep 16, 2022

udidahan marked this pull request as ready for review September 16, 2022 16:31

prmerger-automator bot requested a review from msimecek September 19, 2022 13:33

Update docs/reference-architectures/containers/aks-mission-critical/m…

1c9e4fb

…ission-critical-data-platform.md Co-authored-by: Martin Simecek <[email protected]>

ckittel reviewed Sep 19, 2022

View reviewed changes

docs/reference-architectures/containers/aks-mission-critical/mission-critical-data-platform.md Outdated Show resolved Hide resolved

Update docs/reference-architectures/containers/aks-mission-critical/m…

bcb5a07

…ission-critical-data-platform.md

Ed-Price previously approved these changes Sep 19, 2022

View reviewed changes

Update mission-critical-data-platform.md

edb5dee

Let's lead the RFC quote to set it up better. Finished edit pass.

Ed-Price dismissed their stale review via edb5dee September 19, 2022 18:23

Ed-Price approved these changes Sep 19, 2022

View reviewed changes

Jak-MS closed this Sep 19, 2022

Jak-MS reopened this Sep 19, 2022

Jak-MS merged commit 25f1b9a into MicrosoftDocs:main Sep 19, 2022

udidahan mentioned this pull request Dec 27, 2022

Linking to libraries that support the pattern #3834

Merged

Added description of failure scenarios when code uses databases and brokers together #3609

Added description of failure scenarios when code uses databases and brokers together #3609

Conversation

udidahan commented Sep 4, 2022

prmerger-automator bot commented Sep 4, 2022

udidahan commented Sep 4, 2022

Jak-MS commented Sep 6, 2022

msimecek left a comment

Choose a reason for hiding this comment

msimecek Sep 6, 2022

Choose a reason for hiding this comment

msimecek Sep 6, 2022

Choose a reason for hiding this comment

udidahan Sep 16, 2022

Choose a reason for hiding this comment

Ed-Price commented Sep 8, 2022

ckittel commented Sep 13, 2022

prmerger-automator bot commented Sep 16, 2022

opbld31 commented Sep 16, 2022

⚠️ Validation status: warnings

docs/reference-architectures/containers/aks-mission-critical/mission-critical-data-platform.md

prmerger-automator bot commented Sep 16, 2022

opbld30 commented Sep 16, 2022

✅ Validation status: passed

prmerger-automator bot commented Sep 16, 2022

opbld32 commented Sep 16, 2022

✅ Validation status: passed

udidahan commented Sep 16, 2022

prmerger-automator bot commented Sep 19, 2022

opbld30 commented Sep 19, 2022

❌ Validation status: errors

prmerger-automator bot commented Sep 19, 2022

opbld31 commented Sep 19, 2022

❌ Validation status: errors

udidahan commented Sep 19, 2022

prmerger-automator bot commented Sep 19, 2022

opbld31 commented Sep 19, 2022

❌ Validation status: errors

ckittel commented Sep 19, 2022

ckittel commented Sep 19, 2022

udidahan commented Sep 19, 2022

Ed-Price left a comment

Choose a reason for hiding this comment

prmerger-automator bot commented Sep 19, 2022

opbld33 commented Sep 19, 2022

❌ Validation status: errors

Ed-Price commented Sep 19, 2022

Ed-Price left a comment

Choose a reason for hiding this comment

Ed-Price commented Sep 19, 2022

prmerger-automator bot commented Sep 19, 2022

opbld33 commented Sep 19, 2022

❌ Validation status: errors

udidahan commented Sep 20, 2022

udidahan commented Sep 21, 2022