Mle 29889 param binding accept cts query#1089
Closed
RitaChen609 wants to merge 20 commits into
Closed
Conversation
MLE-26608 - merge release changes into develop
This reverts commit 5518e9c.
This reverts commit 23d8bbd.
This reverts commit 70f361b.
Bumped minimatch, and added a force on serialize-javascript (used by mocha) and underscore (used by jsdoc).
Adds explicit top-level permissions: contents: read to limit the default GITHUB_TOKEN scope for all jobs. Individual jobs that need write access (copyright-validation) already declare their own permissions block which overrides the default. This follows the principle of least privilege recommended in GitHub's PwnRequest security guidance.
…sions PDP-1182 SECCMP-1797: Add top-level permissions to restrict default token
…rom-develop PDP-1182: Remove per-repo pr-workflow.yaml
* MLE-28498 Permission fixes This contains permission fixes for testing as a result of the 11.3.5 changes. * MLE-28498 transform test fixes Updated transforms tests to expect rest-transform-user as a result of 11.3.5 changes from ML-28684. * MLE-28498 dmsdk fixes Copilot fix for flakey dmsdk tests * MLE-28498 Skip optic-fromDocs tests for < 12.1 After confirming with the MarkLogic Server team, op:from-docs is only supported in MLS 12.1 since it is a new feature. I have added a skip for anything lower than MLS 12.1
* MLE-27883 adapt cts.param in the Optic API for MLS 12.1 * MLE-27883 update Copyright * MLE-27883 fix to traverse the exported plan to verify the cts namespace
* MLE-28335 added fragment option in fromSearch - Add 'fragment' option support to fromSearch() for MLS 12.1+ - Valid values: 'document' (default), 'properties', 'locks', 'any' - Client-side validation in PlanSearchOption (plan-builder-base.js) - Updated JSDoc for fromSearch() in plan-builder-generated.js - Added xdmp-lock-acquire/release privileges to rest-evaluator role in both test-setup-users.js and rest-evaluator.json (Gradle config) - Added fragment option integration tests to test-basic/plan-search.js (TC0-TC5, gated on serverVersion >= 12.1) * MLE-28336 updated Copyright * MLE-28335 update fromSearchDocs with fragment option * MLE-28335 resolve npm audit vulnerabilities (0 remaining) Security fixes: - sanitize-html: 2.17.0 → ^2.17.4 (critical XSS via xmp passthrough, GHSA-rpr9-rxv7-x643) - brace-expansion override: 2.0.2 → 5.0.6 (DoS via zero-step sequence, GHSA-f886-m6hf-6m8v) - serialize-javascript override: 7.0.4 → 7.0.5 (CPU exhaustion DoS, GHSA-qj8w-gfj5-8c6v) - diff override: added 9.0.0 (DoS in parsePatch/applyPatch for mocha 11.4+, GHSA-73rr-hh4g-fpgx) - fast-uri, flatted, lodash, picomatch, postcss updated via npm audit fix * MLE-28335 update scheduled trigger for regressions * MLE-28335 skip cts.param integration tests for server < 12.1 * MLE-28335 fix to use the correct optic test database testconfig.restWriterConnection
* MLE-28583 xdmp.uriContentType and xdmp.uriFormat test fix xdmp.uriContentType and xdmp.uriFormat are no longer callable and these tests should only run on ML11 or lower * MLE-28583 transform user test fix The previous change was a regression fixed by the ML server team and the test has been updated to reflect the correct behavior
* MLE-29694 update qs to 6.15.2 * MLE-29694 fixed flaky fragment option tests for fromSearch * MLE-29694 fixed flaky fragment option tests * MLE-29694 fixed typo in comment
* MLE-30684 Switch to Harness Artifact Registry * MLE-30684 npm audit non blocking in Jenkins
* MLE-30686 Bump form-data Bump form-data from version 4.0.4 -> 4.0.6 to fix vulnerability * MLE-30964 Bump markdown-it override Bump markdown-it override from 14.1.1 -> 14.2.0
RitaChen609
requested review from
BillFarber,
rjrudin and
stevebio
as code owners
There was a problem hiding this comment.
Pull request overview
Adds MarkLogic Server 12.1-era Optic/CTS parameterization support to the Node client by introducing op.cts.param() placeholders inside CTS query trees and enabling CTS-query-shaped “bindings” to be injected into Optic plans at request time, along with a broad set of related Optic feature/tests updates.
Changes:
- Add
op.cts.param()(new server type + builder API) and widen CTS query-builder argument typing to acceptcts:paramplaceholders. - Enhance
rows.query()to substituteop:param/cts:paramnodes in exported plan JSON when the provided binding value is a plan-builder AST object (e.g., a CTS query). - Add/update runtime + integration tests covering cts.param/bindParam/fromSearch/fromSearchDocs/where scenarios and other MLS 12.1 Optic features (fragment option, fromDocs, transitiveClosure, vector updates), plus CI/config adjustments.
Reviewed changes
Copilot reviewed 34 out of 35 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test-typescript/optic-cts-param-runtime.test.ts | Runtime smoke tests for op.cts.param() construction and serialization. |
| test-typescript/optic-bindparam-ctsquery-runtime.test.ts | Runtime smoke tests for binding CTS queries into plans and export shapes. |
| test-complete/nodejs-transform-javascript.js | Promise error-handling cleanup in transform tests. |
| test-complete/nodejs-dmsdk-readall-1.js | Fix callback done() flow to avoid double-callback and early-return on error. |
| test-basic/transitive-closure.js | New integration tests for server-side transitiveClosure. |
| test-basic/ssl-min-allow-tls-test.js | Increase suite timeout for TLS test reliability. |
| test-basic/service-caller.js | Skip a flaky service-caller test (now marked skipped). |
| test-basic/plan-search.js | Add fragment option coverage for fromSearch / fromSearchDocs (MLS 12.1+). |
| test-basic/plan-builder-generated.js | Update/extend generated plan-builder integration expectations (incl. version skips). |
| test-basic/optic-vector.js | Update vector tests and expectations for vec/math behavior. |
| test-basic/optic-fromDocs.js | New integration tests for op.fromDocs() + op.columnBuilder() (MLS 12.1+). |
| test-basic/optic-cts-param-test.js | New integration tests for cts.param + CTS-query binding scenarios. |
| test-app/src/main/ml-data/optic/transitive-closure/transClosureTripleSet.xml | New test data for transitive-closure integration tests. |
| test-app/src/main/ml-data/optic/transitive-closure/permissions.properties | Permissions for newly added transitive-closure test data. |
| test-app/src/main/ml-data/optic/transitive-closure/collections.properties | Collections/graph mappings for transitive-closure test data. |
| test-app/src/main/ml-config/security/users/rest-writer.json | Add role(s) needed for new REST extension/feature tests. |
| test-app/src/main/ml-config/security/users/rest-transform-user.json | New user for transform invocation scenarios. |
| test-app/src/main/ml-config/security/users/rest-temporal-writer.json | Add role(s) needed for new REST extension/feature tests. |
| test-app/src/main/ml-config/security/users/rest-reader.json | Add role(s) needed for new REST extension/feature tests. |
| test-app/src/main/ml-config/security/users/rest-admin.json | Add additional roles needed for new test capabilities. |
| test-app/src/main/ml-config/security/roles/rest-login.json | New role granting xdmp:login privilege for transform isolation scenarios. |
| test-app/src/main/ml-config/security/roles/rest-evaluator.json | Extend evaluator role privileges for new tests (locks, login, etc.). |
| test-app/build.gradle | Use mlHost variable in curl tasks instead of hardcoded localhost. |
| package.json | Dependency/version bumps and override updates. |
| lib/server-types-generated.js | Introduce CtsParam server type and export it. |
| lib/rows.js | Add plan-param substitution logic for plan-builder bindings. |
| lib/requester.js | Minor formatting change to multipart binding append. |
| lib/plan-builder.js | Add CtsExpr.prototype.param() to construct cts:param. |
| lib/plan-builder-generated.js | Update CTS function arg typing + allow PlanParam in fromSearch/fromSearchDocs/where; add new Optic APIs (fromDocs/columnBuilder/context/xpath typing, etc.). |
| lib/plan-builder-base.js | Allow cts:param nodes (and vec types) through argument casting; validate fragment option; add transitive-closure options validation. |
| Jenkinsfile | Make npm audit non-blocking; adjust scheduled trigger time. |
| etc/test-setup-users.js | Add required privileges for new lock-based fragment tests. |
| .npmrc | Configure private registry, ignore scripts, and minimum release age. |
| .github/workflows/pr-workflow.yaml | Remove the PR workflow definition. |
Comment on lines
+82
to
+85
| testPlan([""],op.math.floor(op.vec.cosine(vec1, vec2))) | ||
| .then(function(response) { | ||
| assert(response.rows[0].t.value != null); | ||
| assert(response.rows[0].t.value == 0, 'Cosine similarity between orthogonal vectors should be 0'); | ||
| }).catch(error => done(error)); |
Comment on lines
+73
to
+74
| // errors all the time now, should fix. | ||
| it.skip('postOfUrlencodedForDocumentArray1 endpoint', function(done) { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MarkLogic Server 12.1.0 introduced support for
cts.param()placeholders within CTS query trees, enabling a CTS query (or an entire query plan input likefromSearch/fromSearchDocs/where) to be fully parameterized at plan definition time and then bound at execution time.