Development by cubap · Pull Request #378 · CenterForDigitalHumanities/TPEN-services

cubap · 2025-11-10T22:26:29Z

This pull request introduces several improvements to developer and AI assistant documentation, project environment configuration, and local tooling for the TPEN Services repository. The main changes include adding detailed setup and workflow instructions for both developers and AI assistants, updating environment configuration templates for development and production, and enhancing the local Claude AI integration with custom status line and settings. These updates aim to make onboarding, development, and validation more robust and user-friendly.

Documentation and Developer Guidance Enhancements:

Added a comprehensive .claude/CLAUDE.md file with detailed instructions for AI assistants, including project overview, setup, testing, validation, environment requirements, workflow, debugging, CI/CD, and developer preferences.
Improved .github/copilot-instructions.md to clarify environment setup, configuration layering, port usage, and validation steps. Now recommends copying .env.development for local development and documents the new configuration file structure. [1] [2]

Environment Configuration Updates:

Added .env.development and .env.production templates with clear separation of development and production settings, secrets, and documentation for each environment. These files guide users on how to configure their local or production environments securely and effectively. [1] [2]

Claude AI Integration Improvements:

Introduced .claude/settings.json to configure Claude's environment, including status line customization, output token limits, and assistant preferences for code handling.
Added .claude/statusline-command.sh, a script that generates a color-coded status line in the terminal, displaying branch, model, cost, API time, and code changes for enhanced developer feedback when using Claude.

Changed Collections Parameter for Save()

* getting started with Vault * newer Vault * tests * removing redundant code * proper tests passing * sample Vault * Bryan's refusal to .jsonld makes this not work as expected * loading resources This is incomplete by design. We need to ask Vault to add any resources we want resolved * expanding the Manifest a bit * touch up for merge * touch up for merge --------- Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>

@id

* Current Project IIIF manifest Creation * Refactored the exportManifest() and endpoint name * Changed the exportManifest() function * TPEN ID error handled * endpoint to move the manifest.json to TPEN-Static-Dev * Changed @id to id * Getting all Ids * Console Clean up * Added env variables * Adding Imports * Adding Imports * Deleting Ids * Updating Logic Added * User not a member cannot change the manifest * Removing project2 * Adding comments to functions

* add hotkeys service * hotkey endpoints * aggregate hotkeys during project retrieval * specify hotkey fields to include * cleanup * Update Hotkeys.js * Update ProjectFactory.mjs * Return hotkeys as an Array of Strings * aligning with Class changes * remove create, since .save is not acting correctly * cleanup and drop .post * tests restored no Jest here, just checking exists. * tests and sinon upgrade * no db tests directly * Update exists_unit.test.mjs * putting post back in... * adding create back with safety * adding upsert to accomodate bad errors * Update Hotkeys.js * uncatch to let errors through * expect the errors to come back * switch to jest tests --------- Co-authored-by: cubap <cubap@slu.edu>

* Adding endpoint to save changes from the layer * Adding New Layer * No Empty Label and no Annotations * Updated new Layer * Adding Items to partOf * Changing id convention * Removing updating layer * Annotation Change * Adding Delete Endpoint * not sure... This type of thing? * Label Change * Added Layer Class * Adding rerum ids to delete and add * Changing tests * Update exists.test.mjs * Changing LayerLabel * Changes AnnotationCollection Structure * Update Pages API * Updating partOf Ids in case of any change * Adding Layer Metadata Label Change * example results as a base for comments, delete these files before merge. * Adding AddLayer Commenting Rest of the APIs * Added DeleteLayer Back * Deleting json files * Updates to the comments * Making Project to Layer Changes * Renaming layerAnnotationCollection * Removing UpdateOne() changes * Removing UpdateOne() changes * Removing UpdateOne() changes * Error Handled for no ProjectID or LayerID * Clear Code * send() instead of json() --------- Co-authored-by: cubap <cubap@slu.edu> Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>

@mepripri

GitHub per @mepripri

…pleenv-file 91 create sampleenv file

* bring in vault improve Project building from Manifest import * fixes #206 * add singleton vault as utility fix #208 * cleanup logs

* determine data type by content * Removing type dependencies - Took controller and type out of controller - Added a function to detect the data types and assign the correct collection * matching tests to code move

Copilot

Pull Request Overview

This pull request introduces a comprehensive environment configuration refactoring and documentation improvements for TPEN Services. The main goal is to separate safe defaults from secrets, improve developer onboarding, and add better tooling support for AI assistants.

Key Changes:

Introduces layered environment configuration with config.env, .env.development, and .env.production templates
Adds env-loader.js for consistent environment variable loading across all entry points
Removes the Hotkeys class and migrates functionality to project interfaces namespace
Updates default port from 3001 to 3011
Adds smoke tests for post-deployment validation
Improves documentation (README.md, CONTRIBUTING.md, new CONFIG.md)

Reviewed Changes

Copilot reviewed 32 out of 38 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
env-loader.js	New module for layered environment variable loading with dotenv
config.env	Safe default configuration that can be committed to repository
.env.development	Development environment template with placeholder secrets
.env.production	Production environment template for deployment guidance
sample.env	Removed - replaced by layered configuration approach
bin/tpen3_services.js	Imports env-loader.js and updates default port to 3011
app.js	Removes dotenv/dotenv-expand imports, adds CORS configuration
package.json	Moves @jest/globals to devDependencies, removes dotenv-expand, updates test scripts
classes/HotKeys/	Removed - functionality migrated to project interfaces
classes/Project/ProjectFactory.js	Updates hotkeys to use interfaces namespace, improves image import logic
project/customMetadataRouter.js	New router for namespaced custom metadata with POST/PUT/GET endpoints
project/projectReadRouter.js	Adds interface filtering based on origin/query parameters
project/projectCreateRouter.js	Updates import-image endpoint to accept multiple image URLs
tests/smoke.test.js	New smoke tests for post-deployment validation
tests/mount.test.js	New route existence tests using Express routing table inspection
README.md	Comprehensive rewrite with quick start guide and configuration documentation
CONTRIBUTING.md	Updated with new environment configuration instructions
CONFIG.md	New comprehensive configuration guide
.github/workflows/	Updated workflows to use new environment configuration
ecosystem.config.json	New PM2 configuration for cluster deployment

Copilot · 2025-11-11T21:55:46Z

+    const { imageUrls, projectLabel } = req.body
+    if (!imageUrls || !projectLabel) {
+      return respondWithError(res, 400, "Image URL/URLs and project label are required")
+    }
+    if (!Array.isArray(imageUrls) || imageUrls.length === 0) {
+      return respondWithError(res, 400, "Image URLs must be a non-empty array")
    }


The API change from accepting imageUrl (singular) to imageUrls (array) is a breaking change. Consider supporting both parameter names for backward compatibility, or ensure this breaking change is documented and coordinated with API consumers.

Copilot · 2025-11-11T21:55:47Z

+          { id: `${process.env.TPENSTATIC}/${_id}/contentPage.json`, type: "AnnotationPage", items: [ { id: `${process.env.TPENSTATIC}/${_id}/content.json`, type: "Annotation", motivation: "painting", body: { id: imageURL, type: "Image", format: mime.lookup(imageURL) || "image/jpeg", width: dimensions?.width, height: dimensions?.height, ...(isIIFImage && { service: [{ id: IIIFServiceURL, type: IIIFServiceJson?.type, profile: IIIFServiceJson?.profile }] }) }, target: `${process.env.TPENSTATIC}/${_id}/canvas-${index + 1}.json` } ] }
+        ],


This line is extremely long (over 450 characters) and deeply nested, making it very difficult to read and maintain. Consider breaking this into multiple lines with proper indentation or extracting the nested objects into separate variables.

Suggested change

{ id: `${process.env.TPENSTATIC}/${_id}/contentPage.json`, type: "AnnotationPage", items: [ { id: `${process.env.TPENSTATIC}/${_id}/content.json`, type: "Annotation", motivation: "painting", body: { id: imageURL, type: "Image", format: mime.lookup(imageURL) || "image/jpeg", width: dimensions?.width, height: dimensions?.height, ...(isIIFImage && { service: [{ id: IIIFServiceURL, type: IIIFServiceJson?.type, profile: IIIFServiceJson?.profile }] }) }, target: `${process.env.TPENSTATIC}/${_id}/canvas-${index + 1}.json` } ] }

],

(() => {

// Build the body object for the Annotation

const body = {

id: imageURL,

type: "Image",

format: mime.lookup(imageURL) || "image/jpeg",

width: dimensions?.width,

height: dimensions?.height,

};

if (isIIFImage) {

body.service = [

{

id: IIIFServiceURL,

type: IIIFServiceJson?.type,

profile: IIIFServiceJson?.profile,

}

];

}

return {

id: `${process.env.TPENSTATIC}/${_id}/contentPage.json`,

type: "AnnotationPage",

items: [

{

id: `${process.env.TPENSTATIC}/${_id}/content.json`,

type: "Annotation",

motivation: "painting",

body,

target: `${process.env.TPENSTATIC}/${_id}/canvas-${index + 1}.json`

}

]

};

})()

Copilot · 2025-11-11T21:55:47Z

+            if (isSuspiciousJSON(payload[key], Object.keys(payload[key])) || isSuspiciousValueString(payload[key], true)) {
+                payload[key] = { $$unsafe: payload[key] }
+            }


Potential type error: Object.keys(payload[key]) will throw if payload[key] is not an object. The code should check if payload[key] is an object before calling Object.keys() on it.

Copilot · 2025-11-11T21:55:48Z

-      "userClassTests": "node --experimental-vm-modules node_modules/jest/bin/jest.js -t user_class ",
-      "importTests": "node --experimental-vm-modules node_modules/jest/bin/jest.js -t importTests ",
-      "inviteMemberTests": "node --experimental-vm-modules node_modules/jest/bin/jest.js -t inviteMemberTests "
+      "allTests": "node --import ./env-loader.js --experimental-vm-modules node_modules/jest/bin/jest.js",


All test scripts now use --import ./env-loader.js. This is good for consistency, but ensure that test-specific environment files (.env.test) are properly handled by the env-loader, especially since NODE_ENV may not be set to 'test' in all test invocation scenarios.

Copilot · 2025-11-11T21:55:49Z

+ */
+
+import { jest } from "@jest/globals"
+import request from "supertest"


Unused import request.

Copilot · 2025-11-11T21:55:50Z

+
+import { jest } from "@jest/globals"
+import request from "supertest"
+import app from "../../app.js"


Unused import app.

Copilot · 2025-11-11T21:55:50Z

+import Project from "../classes/Project/Project.js"
+import dbDriver from "../database/driver.js"
+import { ACTIONS, ENTITIES, SCOPES } from "./groups/permissions_parameters.js"
+import screenContentMiddleware, { isSuspiciousJSON, isSuspiciousValueString } from "../utilities/checkIfSuspicious.js"


Unused import screenContentMiddleware.

Copilot · 2025-11-11T21:55:50Z

+    if (!validateID(id)) return respondWithError(res, 400, "The TPEN3 project ID provided is invalid")
+
+    try {
+        const project = new Project(id)


Unused variable project.

Copilot · 2025-11-11T21:55:50Z

+  }
+
+  // If no namespaces specified, remove all interfaces
+  if (!namespaces || namespaces.length === 0) {


This negation always evaluates to false.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…nities/TPEN-services into development

* cleanup * Another pass

mepripri and others added 30 commits February 20, 2025 16:53

Removing the Limit from express.json()

e01c1e2

Removing the Limit from express.json()

e4bafbc

Merge pull request #178 from CenterForDigitalHumanities/save-metadata

e5325d1

Changed Collections Parameter for Save()

hotifx

f6e11dc

hotfix for symbols.

40f43aa

delete enabled

ec7a6ba

Create API.md

67a7dd4

collaborators and users

bd148f3

add markdown reader

ae78166

package for markdown

bbcfabb

Update API.md

ba27191

Update API.md

89925b2

Merge branch 'development' into 198-api-documentation

ebeb805

touch

9ced56a

ah codes

5abff41

proxy for internal use (#201)

d363d25

keep in line with main

22e9a2d

Create sample.env

747b7e0

Update sample.env

bb9598f

GitHub per @mepripri

Update CODEOWNERS

c14dcf8

Merge pull request #212 from CenterForDigitalHumanities/91-create-sam…

6348b32

…pleenv-file 91 create sampleenv file

dev it and hotkeys

09dbb0c

Merge branch '91-create-sampleenv-file' into development

de8f371

188 epic middleware to upgrade imported manifests (#209)

7baff4b

* bring in vault improve Project building from Manifest import * fixes #206 * add singleton vault as utility fix #208 * cleanup logs

139 factor out type type dependencies (#211)

bc2ff72

* determine data type by content * Removing type dependencies - Took controller and type out of controller - Added a function to detect the data types and assign the correct collection * matching tests to code move

align with main

726a2dc

stick a fork in it

9767730

thehabes had a problem deploying to development November 11, 2025 21:28 — with GitHub Actions Failure

I think it works!

59e62a0

thehabes had a problem deploying to development November 11, 2025 21:30 — with GitHub Actions Failure

Closer

b160c8d

thehabes had a problem deploying to development November 11, 2025 21:38 — with GitHub Actions Failure

stop smoking

cdfdc18

thehabes temporarily deployed to development November 11, 2025 21:42 — with GitHub Actions Inactive

thehabes had a problem deploying to development November 11, 2025 21:42 — with GitHub Actions Failure

yaml supports if tpen3_services isn't already registered in pm2

8563249

thehabes temporarily deployed to development November 11, 2025 21:46 — with GitHub Actions Inactive

thehabes requested a review from Copilot November 11, 2025 21:52

Copilot started reviewing on behalf of thehabes November 11, 2025 21:53 View session

Copilot finished reviewing on behalf of thehabes November 11, 2025 21:53

Copilot AI reviewed Nov 11, 2025

View reviewed changes

Update env-loader.js

da30f02

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

thehabes temporarily deployed to development November 11, 2025 21:58 — with GitHub Actions Inactive

thehabes temporarily deployed to development November 11, 2025 21:59 — with GitHub Actions Inactive

Update __tests__/mount.test.js

c9d2e1a

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

thehabes temporarily deployed to development November 11, 2025 21:59 — with GitHub Actions Inactive

thehabes temporarily deployed to development November 11, 2025 22:00 — with GitHub Actions Inactive

cubap added 2 commits November 12, 2025 09:06

Update Group.js

a559eb7

Merge branch 'development' of https://github.com/CenterForDigitalHuma…

d0f1d02

…nities/TPEN-services into development

cubap temporarily deployed to development November 12, 2025 15:09 — with GitHub Actions Inactive

Cleanup (#381)

2bd64a0

* cleanup * Another pass

thehabes temporarily deployed to development November 12, 2025 16:03 — with GitHub Actions Inactive

thehabes temporarily deployed to development November 12, 2025 16:04 — with GitHub Actions Inactive

thehabes merged commit d3fec27 into main Nov 12, 2025
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Development#378

Development#378
thehabes merged 261 commits into
mainfrom
development

cubap commented Nov 10, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Nov 11, 2025

Uh oh!

Copilot AI Nov 11, 2025

Uh oh!

Copilot AI Nov 11, 2025

Uh oh!

Uh oh!

Copilot AI Nov 11, 2025

Uh oh!

Copilot AI Nov 11, 2025

Uh oh!

Copilot AI Nov 11, 2025

Uh oh!

Copilot AI Nov 11, 2025

Uh oh!

Copilot AI Nov 11, 2025

Uh oh!

Copilot AI Nov 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

		{ id: `${process.env.TPENSTATIC}/${_id}/contentPage.json`, type: "AnnotationPage", items: [ { id: `${process.env.TPENSTATIC}/${_id}/content.json`, type: "Annotation", motivation: "painting", body: { id: imageURL, type: "Image", format: mime.lookup(imageURL) \|\| "image/jpeg", width: dimensions?.width, height: dimensions?.height, ...(isIIFImage && { service: [{ id: IIIFServiceURL, type: IIIFServiceJson?.type, profile: IIIFServiceJson?.profile }] }) }, target: `${process.env.TPENSTATIC}/${_id}/canvas-${index + 1}.json` } ] }
		],

Conversation

cubap commented Nov 10, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants