MattT/APPEALS-64534: Create PL/pgSQL Function to Return a Table of f_vacols_brieff Records Given a String Containing vacols_ids

[ThorntonMatthew]

Resolves APPEALS-64534

Description

Creates a PL/pgSQL function that within itself invokes gather_vacols_ids_of_hearing_schedulable_legacy_appeals() in order to identify which vacols_ids should be utilized to pull BRIEFF records. This is done so that the IDs will be constantized and the query pushed down to the remote database whenever querying the foreign table.

Acceptance Criteria

1. A PL/pgSQL function is created that
   1. SELECTs from f*vacols_brieff records whose bf_key values are in the result of calling the gather_vacols_ids_of_hearing_schedulable_legacy*appeals function (see APPEALS-64451)
      1. The query must be pushed down to the Oracle DB for execution - opposed to Postgres pulling back all 3mil records from the remote table to be locally processed.
   2. Returns a TABLE that returns all columns from the BRIEFF/fvacolsbrieff table
      1. Each column should be appropriately typed. If it's easier, feel free to create a custom type to represent all columns in a single entity.

Testing Plan

1. Go to Jira Issue/Test Plan Link or list them below

• For feature branches merging into main: Was this deployed to UAT?

Backend

Database Changes

Only for Schema Changes

• Add typical timestamps (created_at, updated_at) for new tables
• Update column comments; include a "PII" prefix to indicate definite or potential PII data content
• Have your migration classes inherit from Caseflow::Migration, especially when adding indexes (use add_safe_index) (see Writing DB migrations)
• Verify that migrate:rollback works as desired (change supported functions)
• Perform query profiling (eyeball Rails log, check bullet and fasterer output)
• For queries using raw sql was an explain plan run by System Team
• Add appropriate indexes (especially for foreign keys, polymorphic columns, unique constraints, and Rails scopes)
• Run make check-fks; add any missing foreign keys or add to config/initializers/immigrant.rb (see Record associations and Foreign Keys)
• Add belongs_to for associations to enable the schema diagrams to be automatically updated
• Document any non-obvious semantics or logic useful for interpreting database data at Caseflow Data Model and Dictionary

Best practices

Code Documentation Updates

• Add or update code comments at the top of the class, module, and/or component.

Tests

Test Coverage

Did you include any test coverage for your code? Check below:

• RSpec
• Jest
• Other

Code Climate

Your code does not add any new code climate offenses? If so why?

• No new code climate issues added

Monitoring, Logging, Auditing, Error, and Exception Handling Checklist

Monitoring

• Are performance metrics (e.g., response time, throughput) being tracked?
• Are key application components monitored (e.g., database, cache, queues)?
• Is there a system in place for setting up alerts based on performance thresholds?

Logging

• Are logs being produced at appropriate log levels (debug, info, warn, error, fatal)?
• Are logs structured (e.g., using log tags) for easier querying and analysis?
• Are sensitive data (e.g., passwords, tokens) redacted or omitted from logs?
• Is log retention and rotation configured correctly?
• Are logs being forwarded to a centralized logging system if needed?

Auditing

• Are user actions being logged for audit purposes?
• Are changes to critical data being tracked ?
• Are logs being securely stored and protected from tampering or exposing protected data?

Error Handling

• Are errors being caught and handled gracefully?
• Are appropriate error messages being displayed to users?
• Are critical errors being reported to an error tracking system (e.g., Sentry, ELK)?
• Are unhandled exceptions being caught at the application level ?

Exception Handling

• Are custom exceptions defined and used where appropriate?
• Is exception handling consistent throughout the codebase?
• Are exceptions logged with relevant context and stack trace information?
• Are exceptions being grouped and categorized for easier analysis and resolution?

[codeclimate]

Code Climate has analyzed commit 069432c and detected 1 issue on this pull request.

Here's the issue category breakdown:

Category	Count
Security	1

View more on Code Climate.

[ThorntonMatthew]

I broke this out to allow CI to run properly. Unfortunately, custom types aren't registered in the schema.rb file and aren't automatically loaded by our build commands.

[ThorntonMatthew]

By passing in IDs as constants to this query it avoids performing a full foreign table scan and the query is pushed down to the remote database.

[ThorntonMatthew]

This function has been tested in Prodtest and has been confirmed to complete execution within a few seconds. This is down from a couple of hours. 🤕

[noahhansen-gov]

Does this table not need to exist within the schema?

[ThorntonMatthew]

This is a custom type, and those don't appear to be recorded in the schema.rb file. Kind of a bummer...

[noahhansen-gov]

What is the purpose of this spec file? I don't see any expectations or assertions in it so confused as to what it's testing?

[ThorntonMatthew]

It's just a shared_context that can be "imported" into other specs. I found myself needing to stage all of the same data a second time and decided to extract it out.

caseflow/spec/db/functions/brieffs_awaiting_hearing_scheduling_spec.rb

Line 4 in e6e7d2f

include_context "Legacy appeals that may or may not appear in the NHQ"

caseflow/spec/db/functions/gather_vacols_ids_of_hearing_schedulable_legacy_appeals_spec.rb

Line 4 in e6e7d2f

include_context "Legacy appeals that may or may not appear in the NHQ"

[ThorntonMatthew]

This context will likely be used a few more times.