ledger alignment

Author: fivetran-avinash

CHANGELOG.md

@@ -2,14 +2,19 @@
 # dbt_xero v1.5.0
 
 ## Schema/Data Change
-**1 total change • 0 possible breaking changes**
+**6 total changes • 1 possible breaking change**
 
 | Data Model(s) | Change type | Old | New | Notes |
 | ---------- | ----------- | -------- | -------- | ----- |
 | `xero__cash_general_ledger` | New model | N/A | New end model | Cash-basis general ledger; one row per cash journal line. Enabled by default; disable by setting `xero__using_journal_cash: false`. |
+| `xero__general_ledger` | Bug fix | `payment_id` excluded `ARPREPAYMENT`, `AROVERPAYMENT` | Now includes all prepayment and overpayment source types | Aligns AR prepayment/overpayment handling with AP counterparts already present.<br><br>**Possible breaking change**: Rows with `ARPREPAYMENT` or `AROVERPAYMENT` source types previously had a `NULL` `payment_id` and will now be populated. |
+| `stg_xero__journal_cash`<br>`stg_xero__journal_cash_tmp` | New staging models | N/A | | Cash-basis journal header data from the `journal_cash` source table. |
+| `stg_xero__journal_cash_line`<br>`stg_xero__journal_cash_line_tmp` | New staging models | N/A | | Cash-basis journal line data from the `journal_cash_line` source table. |
+| `stg_xero__journal_cash_line_has_tracking_category`<br>`stg_xero__journal_cash_line_has_tracking_category_tmp` | New staging models | N/A | | Tracking category associations for cash journal lines. Enabled when `xero__using_journal_cash_line_tracking_category` is `true`. |
+| `int_xero__journal_cash_line_pivoted_tracking_categories` | New intermediate model | N/A | | Pivots tracking categories across cash journal lines. Enabled when both `xero__using_journal_cash_line_tracking_category` and `xero__using_tracking_categories` are `true`. |
 
 ## Feature Update
-- Adds cash-basis journal support. Staging models for three new Xero source tables (`journal_cash`, `journal_cash_line`, `journal_cash_line_has_tracking_category`) and a new `xero__cash_general_ledger` end model are now available, enabled by default. Disable by setting `xero__using_journal_cash: false`. Cash-basis tracking category pivoting is also available via the `xero__using_journal_cash_line_tracking_category` variable. ([#73](https://github.com/fivetran/dbt_xero/pull/73))
+- Adds cash-basis journal support via three new staging models (`stg_xero__journal_cash`, `stg_xero__journal_cash_line`, `stg_xero__journal_cash_line_has_tracking_category`) and the new `xero__cash_general_ledger` end model. The end model includes `contact_id`, `contact_name`, and `payment_id` resolution, mirrors the structure of `xero__general_ledger`, and adds an `accounting_basis` column set to `'cash'`. All models are enabled by default; disable by setting `xero__using_journal_cash: false`. Cash-basis tracking category pivoting is also available via `xero__using_journal_cash_line_tracking_category`.
 
 # dbt_xero v1.4.0
 

README.md

@@ -138,7 +138,7 @@ vars:
     xero__using_invoice_line_item_tracking_category: false          # default is true
     xero__using_journal_line_tracking_category: false               # default is true
     xero__using_tracking_categories: false                          # default is true
-    xero__using_journal_cash: false                                 # default is true
+    xero__using_journal_cash: false                                 # will disable both journal_cash and journal_cash_line sources; default is true
     xero__using_journal_cash_line_tracking_category: false          # default is true
 ```
 

models/staging/stg_xero.yml

@@ -20,7 +20,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
 
@@ -36,7 +36,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
 
@@ -74,7 +74,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
   - name: stg_xero__invoice
@@ -119,7 +119,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
   - name: stg_xero__journal_line
@@ -154,7 +154,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
   - name: stg_xero__journal
@@ -179,7 +179,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
 
@@ -197,7 +197,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
   - name: stg_xero__invoice_line_item_has_tracking_category
@@ -214,7 +214,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
   - name: stg_xero__journal_line_has_tracking_category 
@@ -233,7 +233,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
   - name: stg_xero__tracking_category 
@@ -250,7 +250,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
   - name: stg_xero__tracking_category_option
@@ -275,7 +275,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
   - name: stg_xero__tracking_category_has_option
@@ -290,7 +290,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
   - name: stg_xero__journal_cash
@@ -315,7 +315,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
   - name: stg_xero__journal_cash_line
@@ -350,7 +350,7 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.
 
   - name: stg_xero__journal_cash_line_has_tracking_category
@@ -375,5 +375,5 @@ models:
       - name: source_relation
         description: >
           The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema.
-          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioining together multiple
+          If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple
           sources, this will be an empty string.

models/staging/tmp/stg_xero__journal_cash_line_has_tracking_category_tmp.sql

@@ -1,4 +1,4 @@
-{{ config(enabled=var('xero__using_journal_cash_line_tracking_category', true)) }}
+{{ config(enabled=(var('xero__using_journal_cash', true) and var('xero__using_journal_cash_line_tracking_category', true))) }}
 
 {% if var('union_schemas', []) | length > 0 or var('union_databases', []) | length > 0 %}
 

models/xero.yml

@@ -224,6 +224,8 @@ models:
         description: The identifier for the related bank transaction, if the source transaction is a cash receipt or payment.
       - name: credit_note_id
         description: The identifier for the related credit note, if the source transaction is a credit note.
+      - name: payment_id
+        description: The identifier for the related payment, if the source transaction is a payment (e.g. ACCRECPAYMENT, ACCPAYPAYMENT, ARCREDITPAYMENT, APCREDITPAYMENT, APPREPAYMENT, APOVERPAYMENT, ARPREPAYMENT, AROVERPAYMENT).
       - name: contact_id
         description: The identifier for the contact associated with the source transaction.
       - name: contact_name

models/xero__cash_general_ledger.sql

@@ -85,13 +85,14 @@ with journals as (
 
         case when journals.source_type in ('ACCPAY', 'ACCREC') then journals.source_id end as invoice_id,
         case when journals.source_type in ('CASHREC','CASHPAID') then journals.source_id end as bank_transaction_id,
-        case when journals.source_type in ('ACCPAYCREDIT','ACCRECCREDIT') then journals.source_id end as credit_note_id
+        case when journals.source_type in ('ACCPAYCREDIT','ACCRECCREDIT') then journals.source_id end as credit_note_id,
+        case when journals.source_type in ('ACCRECPAYMENT','ACCPAYPAYMENT','ARCREDITPAYMENT','APCREDITPAYMENT','APPREPAYMENT','APOVERPAYMENT','ARPREPAYMENT','AROVERPAYMENT') then journals.source_id end as payment_id
 
         {% if using_tracking_categories and pivoted_columns|length > 0 %}
             {%- set accounts_columns = ['account_class', 'account_code', 'account_id', 'account_name', 'account_type'] %}
             {%- set journals_columns = ['accounting_basis', 'created_date_utc', 'journal_date', 'journal_id', 'journal_number', 'reference', 'source_id', 'source_relation', 'source_type'] %}
             {%- set journal_lines_columns = ['description', 'gross_amount', 'journal_line_id', 'net_amount', 'tax_amount', 'tax_name', 'tax_type'] %}
-            {%- set new_columns = ['invoice_id', 'bank_transaction_id', 'credit_note_id'] %}
+            {%- set new_columns = ['invoice_id', 'bank_transaction_id', 'credit_note_id', 'payment_id'] %}
             {%- set joined_columns = accounts_columns + journals_columns + journal_lines_columns + new_columns %}
 
             {% for col in pivoted_columns %}

models/xero__general_ledger.sql

@@ -84,7 +84,7 @@ with journals as (
         case when journals.source_type in ('CASHREC','CASHPAID') then journals.source_id end as bank_transaction_id,
         case when journals.source_type in ('TRANSFER') then journals.source_id end as bank_transfer_id,
         case when journals.source_type in ('MANJOURNAL') then journals.source_id end as manual_journal_id,
-        case when journals.source_type in ('APPREPAYMENT', 'APOVERPAYMENT', 'ACCPAYPAYMENT', 'ACCRECPAYMENT', 'ARCREDITPAYMENT', 'APCREDITPAYMENT') then journals.source_id end as payment_id,
+        case when journals.source_type in ('APPREPAYMENT', 'APOVERPAYMENT', 'ARPREPAYMENT', 'AROVERPAYMENT', 'ACCPAYPAYMENT', 'ACCRECPAYMENT', 'ARCREDITPAYMENT', 'APCREDITPAYMENT') then journals.source_id end as payment_id,
         case when journals.source_type in ('ACCPAYCREDIT','ACCRECCREDIT') then journals.source_id end as credit_note_id
 
         {% if using_tracking_categories and pivoted_columns|length > 0 %}