docs(docs): Add Document Sync protocol specification to the architecture docs (#602)

* First commit of expanded architecture documentation

* docs(docs): remove stuff we dont want in git

* fix(docs): glossary in a table

* fix(docs): spelling

* fix(docs): fix athena references

* fix(docs): .pages

* fix(docs): markdown and spelling

* fix(docs): spelling

* fix(docs): Doc build fix

* fix(docs): update the ADRs

* fix(docs): Cross document references

* fix(docs): Add link anchors

* fix(docs): mdlint

* fix(docs): incorporate suggestions from previous PR

* docs(docs): Working on preliminary protocol spec for document exchange

* docs(docs): wip wire level protocol specification

* docs(docs): use blake3 instead of SHA256

* docs(docs): WIP message formats

* docs(docs): cleaning up message formats

* fix(docs): WIP updating message formats

* fix(docs): wip

* docs(docs): wip

* fix(docs): wip msg format cleanup

* docs(docs): cleaned up common msg

* fix(docs): further common msg cleanups

* fix(docs): wip

* fix(docs): add dynamic depth selection

* docs(docs): WIP add some diagrams

* fix(docs): Add icons to all pages for better consistency when viewing.  Restrict CID to SHA2-256

* fix(docs): Cleanup subscribe/unsubscribe sequence diagram

* fix(docs): Add publishing pre-condition to assist with doc dissemination

* docs(docs): wip fixing syn

* fix(docs): .syn wip

* fix(docs): WIP on .prv

* fix(docs): wip

* fix(docs): wip

* fix(docs): nested lists

* docs(docs): more protocl spec refinement

* fix(docs): cleanup pass

* fix(docs): spec poc finalized

* fix(docs): spelling and markdown formats

* fix(docs): Fix indentation issues caused when we moved to 4 indents for sub lists

* docs(docs): first draft runtime extension design spec

* docs(docs): Add runtime extension specification for doc sync protocol

* docs(docs): add steady state sync correction

* fix(docs): Clean up markdown format

* fix(docs): markdown issues

* fix(hermes): clarify bit ordering in traversal and left/right semantics

* fix(docs): markdown issues

* fix(docs): Try and fix wit markdown link

Author: stevenj

.config/dictionaries/project.dic

@@ -4,6 +4,7 @@ asat
 asyncio
 auditability
 autocheckpoint
+autonumber
 backpressure
 bech
 bkioshn
@@ -12,6 +13,8 @@ blosc
 bmac
 bootstrapper
 brotli
+bucketization
+bursty
 cabi
 canvaskit
 cardano
@@ -20,9 +23,12 @@ cbor
 cdylib
 chaincode
 childs
+chksum
 chrono
 cids
+cidv
 ciphertext
+ciphertexts
 codegen
 codepoints
 COEP
@@ -38,11 +44,14 @@ dashmap
 datelike
 dcbor
 Devnet
+DHKEM
+Diffie
 dotenv
 dotenvy
 dotstar
 earthfile
 Embedder
+enumerants
 EROFS
 errno
 fadvise
@@ -67,31 +76,43 @@ gossipsub
 happ
 hmod
 hostnames
+hpke
 httpmock
+IBLT
 idents
 ifmt
 instanciate
 ipfs
 jsonschema
 justfile
+keepalives
 libp
 libsqlite
 libtest
 linkat
 lintfix
+ltbase
 mapref
+mathrm
 mdlint
+Merkle
 metno
 mgrybyk
 minicbor
 miniprotocol
 mithril
+Mitzenmacher
 mkcron
 mkdelay
 mkdirat
+multicodec
+multihash
+multihashes
+Namespacing
 nanos
 newtype
 nextest
+nodesep
 nolfs
 nomutex
 nostack
@@ -107,6 +128,8 @@ pollables
 pollout
 pozhylenkov
 pread
+precheck
+prechecks
 preopened
 preopens
 preprod
@@ -116,11 +139,16 @@ pubkey
 pwrite
 qmark
 rafal
+rankdir
+ranksep
 readlinkat
+reentrancy
 removedir
 renameat
 reqwest
 rfind
+Rmax
+Rmin
 rngs
 rollouts
 rusqlite
@@ -142,6 +170,7 @@ subsec
 symlinkat
 syscall
 tailscale
+Tdiff
 testcov
 testdocs
 testunit
@@ -150,19 +179,23 @@ thollander
 threadsafe
 timelike
 tinygo
+Tmax
 toolsets
+Tquiet
 traceback
 traitreg
 tweakable
 txns
 txos
 unfinalized
 unlinkat
+unsub
 usvg
 utimensat
 utoipa
 utxo
 uuidv
+varint
 wasi
 wasip
 wasmparser
@@ -171,6 +204,7 @@ wasmtime
 webasm
 webpki
 witx
+Wpin
 xprivate
 xprv
 xpub

.markdownlint.jsonc

@@ -27,11 +27,11 @@
     // MD007/ul-indent - Unordered list indentation
     "MD007": {
         // Spaces for indent
-        "indent": 2,
+        "indent": 4,
         // Whether to indent the first level of the list
         "start_indented": false,
         // Spaces for first level indent (when start_indented is set)
-        "start_indent": 2
+        "start_indent": 4
     },
     // MD009/no-trailing-spaces - Trailing spaces
     "MD009": {

CONTRIBUTING.md

@@ -3,22 +3,22 @@
 First off, thanks for taking the time to contribute! ❤️
 
 * [Contributing to Hermes](#contributing-to-hermes)
-  * [Code of Conduct](#code-of-conduct)
-  * [I Have a Question](#i-have-a-question)
-  * [I Want To Contribute](#i-want-to-contribute)
-    * [Reporting Bugs](#reporting-bugs)
-      * [Before Submitting a Bug Report](#before-submitting-a-bug-report)
-      * [How Do I Submit a Good Bug Report?](#how-do-i-submit-a-good-bug-report)
-    * [Suggesting Enhancements](#suggesting-enhancements)
-      * [Before Submitting an Enhancement](#before-submitting-an-enhancement)
-      * [How Do I Submit a Good Enhancement Suggestion?](#how-do-i-submit-a-good-enhancement-suggestion)
-    * [Your First Code Contribution](#your-first-code-contribution)
-    * [Improving The Documentation](#improving-the-documentation)
-  * [Style guides](#style-guides)
-    * [Rust](#rust)
-    * [Dart](#dart)
-    * [Flutter](#flutter)
-    * [Commit Messages](#commit-messages)
+    * [Code of Conduct](#code-of-conduct)
+    * [I Have a Question](#i-have-a-question)
+    * [I Want To Contribute](#i-want-to-contribute)
+        * [Reporting Bugs](#reporting-bugs)
+            * [Before Submitting a Bug Report](#before-submitting-a-bug-report)
+            * [How Do I Submit a Good Bug Report?](#how-do-i-submit-a-good-bug-report)
+        * [Suggesting Enhancements](#suggesting-enhancements)
+            * [Before Submitting an Enhancement](#before-submitting-an-enhancement)
+            * [How Do I Submit a Good Enhancement Suggestion?](#how-do-i-submit-a-good-enhancement-suggestion)
+        * [Your First Code Contribution](#your-first-code-contribution)
+        * [Improving The Documentation](#improving-the-documentation)
+    * [Style guides](#style-guides)
+        * [Rust](#rust)
+        * [Dart](#dart)
+        * [Flutter](#flutter)
+        * [Commit Messages](#commit-messages)
 
 All types of contributions are encouraged and valued.
 Please make sure to read the relevant section before making your contribution.
@@ -42,7 +42,7 @@ Before you ask a question, it is best to search for existing
 [Issues](https://github.com/input-output-hk/hermes/issues)
 that might help you.
 In case you have found a suitable issue and still need clarification, you can write your question
-[here](https://github.com/input-output-hk/hermes/discussions).
+[in our discussions](https://github.com/input-output-hk/hermes/discussions).
 It is also advisable to search the internet for answers first.
 
 If you then still feel the need to ask a question and need clarification, we recommend the following:
@@ -75,11 +75,11 @@ Please complete the following steps in advance to help us fix any potential bug
 * Also make sure to search the internet (including Stack Overflow)
   to see if users outside the GitHub community have discussed the issue.
 * Collect information about the bug:
-  * Stack trace (Traceback)
-  * OS, Platform and Version (Windows, Linux, macOS, x86, ARM)
-  * Version of the interpreter, compiler, SDK, runtime environment, package manager, depending on what seems relevant.
-  * Possibly your input and the output
-  * Can you reliably reproduce the issue?
+    * Stack trace (Traceback)
+    * OS, Platform and Version (Windows, Linux, macOS, x86, ARM)
+    * Version of the interpreter, compiler, SDK, runtime environment, package manager, depending on what seems relevant.
+    * Possibly your input and the output
+    * Can you reliably reproduce the issue?
     And can you also reproduce it with older versions?
 
 #### How Do I Submit a Good Bug Report?

dashboard/README.md

@@ -3,14 +3,14 @@
 <!-- markdownlint-disable MD029 -->
 
 * [Dashboard](#dashboard)
-  * [Requirements](#requirements)
-  * [Getting Started](#getting-started)
-    * [Bootstrapping](#bootstrapping)
-  * [Running Tests](#running-tests)
-  * [Working with Translations](#working-with-translations)
-    * [Adding Strings](#adding-strings)
-    * [Adding Supported Locales](#adding-supported-locales)
-    * [Adding Translations](#adding-translations)
+    * [Requirements](#requirements)
+    * [Getting Started](#getting-started)
+        * [Bootstrapping](#bootstrapping)
+    * [Running Tests](#running-tests)
+    * [Working with Translations](#working-with-translations)
+        * [Adding Strings](#adding-strings)
+        * [Adding Supported Locales](#adding-supported-locales)
+        * [Adding Translations](#adding-translations)
 
 ## Requirements
 
@@ -25,9 +25,9 @@
 * [Fvm](https://fvm.app/)
 * [Fastlane](https://fastlane.tools)
 * Flutter & Dart plugins:
-  * [Visual Studio Code](https://flutter.dev/docs/get-started/editor?tab=vscode)
-  * [Android Studio / IntelliJ](https://flutter.dev/docs/get-started/editor?tab=androidstudio)
-  * [Emacs](https://docs.flutter.dev/get-started/editor?tab=emacs)
+    * [Visual Studio Code](https://flutter.dev/docs/get-started/editor?tab=vscode)
+    * [Android Studio / IntelliJ](https://flutter.dev/docs/get-started/editor?tab=androidstudio)
+    * [Emacs](https://docs.flutter.dev/get-started/editor?tab=emacs)
 
 ❗️We recommend using **Visual Studio Code** as the **default editor** for this project.
 

docs/FLUTTER_INTEGRATION_TEST.md

@@ -3,21 +3,21 @@
 <!-- cspell: words gcloud chromedriver gradlew ptarget pdart -->
 
 * [Flutter Integration Tests](#flutter-integration-tests)
-  * [Requirements](#requirements)
-  * [Run Integration Tests](#run-integration-tests)
-    * [CI](#ci)
-      * [View Integration Test Results](#view-integration-test-results)
-    * [Local](#local)
-      * [Web](#web)
-      * [iOS](#ios)
-        * [Run integration test in Xcode](#run-integration-test-in-xcode)
-        * [Run integration test from command line](#run-integration-test-from-command-line)
-      * [Android](#android)
-        * [Run integration test in Android Studio](#run-integration-test-in-android-studio)
-        * [Run integration test from command line](#run-integration-test-from-command-line-1)
-    * [Run integration test in Firebase Test Lab](#run-integration-test-in-firebase-test-lab)
-  * [Gcloud CLI](#gcloud-cli)
-  * [Links](#links)
+    * [Requirements](#requirements)
+    * [Run Integration Tests](#run-integration-tests)
+        * [CI](#ci)
+            * [View Integration Test Results](#view-integration-test-results)
+        * [Local](#local)
+            * [Web](#web)
+            * [iOS](#ios)
+                * [Run integration test in Xcode](#run-integration-test-in-xcode)
+                * [Run integration test from command line](#run-integration-test-from-command-line)
+            * [Android](#android)
+                * [Run integration test in Android Studio](#run-integration-test-in-android-studio)
+                * [Run integration test from command line](#run-integration-test-from-command-line-1)
+        * [Run integration test in Firebase Test Lab](#run-integration-test-in-firebase-test-lab)
+    * [Gcloud CLI](#gcloud-cli)
+    * [Links](#links)
 
 ## Requirements
 

docs/src/architecture/08_concepts/.pages

@@ -12,3 +12,4 @@ arrange:
   - hermes_packaging_requirements
   - cardano_chain_follower
   - hermes_signing_procedure
+  - document_sync

docs/src/architecture/08_concepts/cardano_chain_follower/index.md

@@ -0,0 +1,12 @@
+---
+icon: material/source-branch
+---
+
+# Cardano Chain Follower
+
+Concepts and structure of the Cardano chain follower used by Hermes to read blocks and follow chain updates.
+
+See also:
+
+* Overview: `overview.md`
+* Follower instances: `follower_instances.md`

docs/src/architecture/08_concepts/document_sync/.pages

@@ -0,0 +1,6 @@
+title: Document Synchronization
+arrange:
+  - index.md
+  - protocol_spec.md
+  - runtime_extension_spec.md
+

docs/src/architecture/08_concepts/document_sync/index.md

@@ -0,0 +1,12 @@
+---
+icon: material/sync
+---
+
+# Document Synchronization
+
+This section hosts the draft specifications for decentralized document synchronization in Hermes.
+
+* Protocol: see `protocol_spec.md`.
+* Runtime extension (WASM API): see `runtime_extension_spec.md`.
+
+Both documents are scaffolds containing section placeholders; details will be filled in progressively.