Conversation
* Updates and refactor. Dang, we cannot see the registered app routes like we used to. * packages * cleanup * old logs * cleanup for AI * undiff for AI * undiff for AI * Reformat exists tests * remove unused package * Add in check for /overwrite exists * cleanup * cleanup * Update __tests__/routes_mounted.test.js Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * cleanup * cleanup * cleanup --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Bring in search logic * It searches! * idNegotiation on search results * All the search logic * Lint, and add support for passing search options into the endpoint * polish * Update API documentation * polish * polish * polish * polish * polish * polish * polish * polish * exists test for new routes * Update public/API.html Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update public/API.html Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update public/API.html Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update controllers/search.js Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update controllers/search.js Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * get rid of utils. prefix from createExpressError * Update public/API.html Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update public/API.html Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * slop formatting * Touch ups to API.html as discussed at standup. * bump version because of new search feature --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Updated guidance to clarify that the file provides instructions for AI assistants rather than Claude Code.
Removed specific Bash command permissions from settings.
…anities#232) * Initial plan * Change slug default from empty string to null Co-authored-by: cubap <1119165+cubap@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: cubap <1119165+cubap@users.noreply.github.com>
…igitalHumanities#234) * Initial plan * Optimize queries to use _id instead of @id for root object lookups Co-authored-by: cubap <1119165+cubap@users.noreply.github.com> * Add null check validation for primeID before parsing Co-authored-by: cubap <1119165+cubap@users.noreply.github.com> * Add null check for rootObj after database query Co-authored-by: cubap <1119165+cubap@users.noreply.github.com> * Remove duplicate getAllVersions function and improve error handling Co-authored-by: cubap <1119165+cubap@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: cubap <1119165+cubap@users.noreply.github.com> Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>
…italHumanities#235) Updated the create and release controllers to assign the __rerum.slug property only when a slug value is present. This prevents undefined slugs from being set in the object metadata.
* Update packages, npm versions, and node versions * the force * the force --------- Co-authored-by: Claude Code <claude@anthropic.com>
| import controller from '../db-controller.js' | ||
|
|
||
| router.route('/') | ||
| .post(controller.searchAsWords) |
Check failure
Code scanning / CodeQL
Missing rate limiting High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 13 days ago
In general, this should be fixed by adding a rate-limiting middleware to the search routes so that requests to database-intensive search handlers are throttled. In an Express-style router, the recommended approach is to use a well-known library such as express-rate-limit, configure an appropriate window and maximum request count, and apply the limiter to the relevant routes.
Concretely for routes/search.js, we can import express-rate-limit, define a searchLimiter with parameters suitable for search operations (for example, a certain number of requests per 15-minute window), and then apply this middleware to both the '/' and '/phrase' routes. This can be done with router.use(searchLimiter) before the route definitions to cover all search routes in this router without changing existing handler logic, or by attaching the limiter directly in the route definitions (e.g., .post(searchLimiter, controller.searchAsWords)). Using router.use(searchLimiter) is simpler and keeps the behavior of controller.searchAsWords and controller.searchAsPhrase unchanged while ensuring that both existing and future search endpoints defined in this router are protected.
Implementation steps in this file:
- Add an import for
express-rate-limit. - Define a
searchLimiterconstant usingrateLimit({ ...config }). - Apply the limiter to this router using
router.use(searchLimiter)before therouter.route('/')androuter.route('/phrase')definitions.
No changes are needed inside the controller functions or elsewhere.
| @@ -1,7 +1,15 @@ | ||
| import express from 'express' | ||
| const router = express.Router() | ||
| import controller from '../db-controller.js' | ||
| import rateLimit from 'express-rate-limit' | ||
|
|
||
| const searchLimiter = rateLimit({ | ||
| windowMs: 15 * 60 * 1000, // 15 minutes | ||
| max: 100, // limit each IP to 100 search requests per windowMs | ||
| }) | ||
|
|
||
| router.use(searchLimiter) | ||
|
|
||
| router.route('/') | ||
| .post(controller.searchAsWords) | ||
| .all((req, res, next) => { |
| @@ -37,7 +37,8 @@ | ||
| "express-oauth2-jwt-bearer": "~1.7.1", | ||
| "express-urlrewrite": "~2.0.3", | ||
| "mongodb": "^7.0.0", | ||
| "morgan": "~1.10.1" | ||
| "morgan": "~1.10.1", | ||
| "express-rate-limit": "^8.2.1" | ||
| }, | ||
| "devDependencies": { | ||
| "@jest/globals": "^30.2.0", |
| Package | Version | Security advisories |
| express-rate-limit (npm) | 8.2.1 | None |
| }) | ||
|
|
||
| router.route('/phrase') | ||
| .post(controller.searchAsPhrase) |
Check failure
Code scanning / CodeQL
Missing rate limiting High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 13 days ago
In general, the problem is fixed by adding a rate-limiting middleware to the affected routes (or to the router as a whole) so that calls to controller.searchAsPhrase (and other search handlers) cannot be made arbitrarily often by a single client. In an Express app, this is commonly done using a well-known library such as express-rate-limit.
The best fix here, without changing existing functionality, is to import express-rate-limit in routes/search.js, configure a limiter that is reasonable for search operations, and apply it to these search routes. To be minimally invasive and avoid affecting unrelated parts of the app, we can attach the limiter to this router only (e.g., using router.use(limiter) before defining routes, or applying it per-route). Since both / and /phrase are search endpoints that likely hit the database, protecting the entire router is appropriate and addresses all alert variants. The change is limited to routes/search.js: add a RateLimit import, define a searchLimiter instance near the top, and apply it to the router via router.use(searchLimiter) before the route definitions.
Concretely:
-
Add
import rateLimit from 'express-rate-limit'(ES module style, consistent with existing imports). -
Define a limiter configuration, e.g.:
const searchLimiter = rateLimit({ windowMs: 15 * 60 * 1000, max: 100, })
-
Apply it before defining routes:
router.use(searchLimiter)so that all routes in this router, includingcontroller.searchAsPhraseandcontroller.searchAsWords, are rate-limited.
| @@ -1,7 +1,15 @@ | ||
| import express from 'express' | ||
| const router = express.Router() | ||
| import controller from '../db-controller.js' | ||
| import rateLimit from 'express-rate-limit' | ||
|
|
||
| const searchLimiter = rateLimit({ | ||
| windowMs: 15 * 60 * 1000, // 15 minutes | ||
| max: 100, // limit each IP to 100 search requests per windowMs | ||
| }) | ||
|
|
||
| router.use(searchLimiter) | ||
|
|
||
| router.route('/') | ||
| .post(controller.searchAsWords) | ||
| .all((req, res, next) => { |
| @@ -37,7 +37,8 @@ | ||
| "express-oauth2-jwt-bearer": "~1.7.1", | ||
| "express-urlrewrite": "~2.0.3", | ||
| "mongodb": "^7.0.0", | ||
| "morgan": "~1.10.1" | ||
| "morgan": "~1.10.1", | ||
| "express-rate-limit": "^8.2.1" | ||
| }, | ||
| "devDependencies": { | ||
| "@jest/globals": "^30.2.0", |
| Package | Version | Security advisories |
| express-rate-limit (npm) | 8.2.1 | None |
|
Recommending that you remove the .github/workflows folder. You should also check in on your Actions settings. The goal is to make sure you are not using the upstream repo's Actions and Workflows as they do not make sense for your context. The RERUM Server Community Edition is a separate product. One of the action items is to set up your own Actions and Workflows for CI, CD, and security. If you determine you still want to using something upstream then we can work to make that happen when we cross that bridge. It will be a good idea to review npm packages and use |
|
we have checked the APIs, and they are working fine in local. |
thehabes
left a comment
There was a problem hiding this comment.
I was able to get the code locally. All tests pass, the app runs, and the endpoints work in my Talend API as expected. AI reviews noted that this is a clean sync. This should be a good base to work from and can go to your main when you are ready.
Noting that we really want to find a way to stop the CodeQL scans. Those ❌'s are going to get very annoying until you actually want to use them. Please look for a way to do this in your repository/organization settings, and we will look for a way to do it in our repository/organization settings.
Worked on issue #24 to sync the upstream repo with the oss-slu repository