Repository navigation
chore: dev environment on df-dev, pilots' tooling and test standard - #19
Merged
Merged
Conversation
- dev/dev.json for df-dev: port variables, panes and their readiness urls, page to open - zellij layout on df-dev: status pane starts the containers, api waits for them (wait-deps) and runs nodemon directly, ui runs vite under df-mem-cap, no npm wrappers, no tee or log files, no nvm use - maildev behind its own compose profile, simple-directory sends the notification mails to it - mongo 8.0.17 with a 0.25 GB wiredTiger cache and a mongosh healthcheck; mongo 7 volumes must be reset (docker compose down -v) - simple-directory on the canonical block of the data-fair-service skill (healthcheck, auth rate limit, shared dev secrets and cipher password), NHI settings kept - shared dev secrets (secret-identities, secret-events, secret-sendmails) in the api development config, the tests and the dev scripts - nginx template on the shared head of the skill; /built routes and dev/data mount removed - init-env.sh, status.sh, worktree.sh and delete-worktree.sh replaced by df-dev, df-status and df-worktree; dev-api, dev-deps and dev-ui scripts removed - state-setup no longer tails logs, points to df-dev logs; state-teardown removed - CI: df-dev init-env, every compose profile started, api log shown when it does not start - @data-fair/lib-types-builder 1.14.5 (atomic writes: build-types no longer restarts the api) - AGENTS.md and CONTRIBUTING.md on df-dev
npm update across the workspaces, no range changed: - body-parser 2.4 (express 5.3), follow-redirects 1.16.1: the 3 high findings trivy reported in the 1.5.1 image - vite 8.3.5, source-map-js, fast-uri, qs, postcss-selector-parser, serialize-javascript npm audit: 8 vulnerabilities (4 high) before, 0 after, 7 (4 high) before in the shipped tree. trivy on the built image: 0 vulnerabilities (alpine and Node.js), 3 high before.
- dotenv-cli: dev-zellij was its only user, df-dev zellij reads .env itself - @data-fair/lib-common-types (scopes unit test), @data-fair/lib-utils (eslint config) and axios (types of the test helpers) were imported from the root and only resolved through the copies hoisted from the api workspace and lib-node
d-frame hardening: d-frame-content refuses non http(s) URLs from the parent (src, parent URL results) and validates the messages it receives; vue-router state changes no longer leave the parent and the child on different URLs after a cancelled or aborted navigation. Same API for d-frame-content, already used by data-fair, portals and customers.
- actions/checkout v7, actions/setup-node v7, docker/setup-buildx-action v4, docker/login-action v4, docker/metadata-action v6, docker/build-push-action v7, the majors portals already runs with the same inputs - setup-node reads the node version from .nvmrc, the major of the Dockerfile - releases: the major tag is parsed in a shell step instead of battila7/get-version-action
BatLeDev
marked this pull request as draft
October 28, 2026 16:51
- Dockerfile base on node:24.22.1-alpine3.24, pinned by digest as in catalogs and portals - .nvmrc 24.22.1, read by CI setup-node: 24.22.1 fixes CVE-2026-111497 (brotli decompression) - CONTRIBUTING.md asks for node 24.22.1+
- eslint 10, neostandard 0.14, eslint-plugin-vue 10, eslint-plugin-vuetify 2.7 with its flat/recommended-v4 config; vue/require-default-prop off as in catalogs - lint fixes: indentation of tests/fixtures/login.ts, Vuetify 4 typography classes, optional outputs prop (it has a default) - @commitlint/cli and config-conventional 21 replace commitlint 19, config in commitlint.config.ts, commit-msg lints "$1" (MERGE_MSG on merges) - TypeScript 6 (7 not taken: the fleet is on 6), tsconfig erasableSyntaxOnly and types ["node"] inside compilerOptions (the top-level "types" was ignored), @types/node 24 - generated .d.ts under ui/dts/ and gitignored, no longer committed; ui build is vite build only, vue-tsc runs in check-types after it - vite.config.ts: import.meta.dirname and an explicit .ts extension (native config loader) - jose 6 and dotenv 18 (tests and dev config only) - engines.node >=24.22.1
- the api pane and the ui pane commands run in CI as in catalogs: nginx proxies /events/ to vite, scopes.api.spec.ts got a 502 on /events/embed/subscriptions without it - readiness waits for both, tails dev/logs/*.log when they do not answer - test-results/ and dev/logs/ uploaded on failure, docker compose logs printed on failure - CI order of tooling.md: the UI build (generated .d.ts) before check-types
Its first compile on a cold vite went over the 5 s timeout of webhooks.e2e.spec.ts, which then passed on a re-run against the already compiled route.
lib-express warns on every request that passes it as a ?key= query parameter: the api tests filled the api pane with 'passing internal secret key through query parameter is not recommended'. The dev send-notifications script moves to the header too.
BatLeDev
changed the title
- the script posted to localhost:8082, a port df-dev no longer assigns: it reads DEV_API_PORT from .env and posts to the api directly (an internal secret request through nginx gets a 421) - no --experimental-strip-types flag, node 24 strips types by default - the development config error points to npx --no df-dev setup, dotenv-cli was removed
- dev page: the 'Built version' link pointed to /built/, a route the nginx template dropped - Dockerfile CMD without --experimental-strip-types, node 24 strips types by default - check skill builds the UI before check-types: the ui/dts files are no longer committed
… greater id The 'before' cursor filtered date <= last date AND _id < last id. Ids are random nanoids, so an older event whose id sorted after the last one of the page was left out of every next page (events.api.spec.ts failed now and then on an empty second page). The id now only breaks ties between events of the same date: date < d OR (date = d AND _id < id).
Deterministic ids and dates for the case the random nanoids only hit now and then; it fails without the cursor fix.
- startHookServer listens on port 0, the test reads its url: worktrees running their tests at the same time no longer receive each other's deliveries on 19876-19898 - targets where nothing should listen use closedPortUrl (port 1, privileged: no test binds it), a delivery there fails at once
- .husky/pre-push runs dev/pre-push.sh (catalogs' script, on origin/main): type check, unit tests, the @smoke api/e2e tests and the api/e2e spec files changed since origin/main, instead of the whole npm run quality; the full suite runs on pull requests and release tags - @smoke on 6 fast tests over the core flows - AGENTS.md: quality gates, @smoke rule, hook-server helper
- webhooks.api: the four 8 s sleeps waiting for the worker poll the webhook status instead - e2e: the 2 s sleeps before reading notifications are gone, POST /api/events creates them before answering - subscriptions.api WS: poll the message count, a fixed 200 ms stays only where nothing must come
…9 attempts - failedAttemptPatch (operations.ts, pure): error patch of a failed delivery, next attempt at the backoff of its rank, no next attempt after the 10th; the worker applies it - unit tests for both branches; the api test that seeded a webhook with 9 attempts to watch the 10th fail is removed, the backoff api test still covers the worker end to end
- trace retain-on-failure, screenshot only-on-failure, json report of the per-test durations in dev/logs/test-timings.json (sizes the smoke set) - e2e sessions from simple-directory's API (POST /api/auth/password + callback, one login per user and worker, renewed after 10 min) instead of the login form and its cookie cache; a future sd-keepalive timestamp stops lib-vue from rotating the shared exchange token - on a failure the api pane logs are attached to the report and the trace - test scripts without --max-failures, test-api-fast keeps it to iterate; unit fully parallel - AGENTS.md: traces, durations, test.slow()
A dependency discovered while browsing made vite re-optimize and reload the page, which broke a cold e2e run in the middle of an assertion. optimizeDeps.noDiscovery serves the ESM dependencies as is and only pre-bundles the listed commonjs ones (debug, lib-vue owner.js). Validated on an empty ui/node_modules/.vite: the 7 UI routes load once with no console error, e2e 16/16.
events, notifications, push, subscriptions, webhooks, scopes, identities and shared; setups, fixtures, support helpers, resources and the generator seed stay at the root of tests/. Relative imports and the AGENTS.md references follow.
- chromium installed by playwright, its binaries cached on the playwright version, the system packages installed on every run (as in catalogs) - npm test runs unit, api and e2e in the one quality job: the suite takes ~1.2 min locally, far from the ~10 min that would call for split jobs; the e2e-warmup project already covers a cold vite
- quality builds the UI before check-types: the ui/dts files are no longer committed, a fresh checkout failed on vue-tsc - api and ui workspace dev scripts removed: the panes and CI run their commands directly - tsconfig.json back to its compact arrays (a rewrite had expanded them) - pre-push.sh: no root spec pattern, every spec is under tests/features/
The v4 copied from catalogs were behind the current majors the other actions use: both newer majors move to the node 24 runtime, the inputs used here (path, key, name, retention-days) are unchanged, upload-artifact v7's archive option defaults to true.
BatLeDev
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Align events on the shared dev environment, tooling and test standard of the df-dev pilots (catalogs, portals, processings), and refresh its dependencies.
Why: agents operate every service the same way (
npx --no df-dev status | logs | restart) with less RAM per stack, and the quality gates, CI and tests follow the data-fair-service skill instead of events' own variants.Dev environment:
@data-fair/lib-dev:dev/dev.json, zellij layout with a status pane that starts the containers, nodemon and vite (df-mem-cap) run directly, no npm wrappers nor log files;init-env.sh,status.sh,worktree.sh,delete-worktree.shand thedev-*scripts replaced bydf-dev,df-worktree/builtroutes removedx-secret-key;send-notifications.tsreads the df-dev portsTooling:
flat/recommended-v4(lint fixes: Vuetify 4 typography classes, an optional prop); commitlint 21; TypeScript 6 witherasableSyntaxOnly.d.tsunderui/dts/, no longer committed; the UI build runs beforecheck-types.nvmrcandengines(CVE-2026-111497)Tests and quality gates:
dev/pre-push.sh(types, unit,@smoke, changed specs) instead of the wholequality; CI runs unit, api and e2e (chromium cached), uploadstest-results/anddev/logs/and prints the compose logs on failuredev/logs/test-timings.json; viteoptimizeDeps.noDiscovery(no reload on a cold run)tests/features/<feature>/; polls instead of fixed sleeps, the webhook retry decision unit-tested (failedAttemptPatch), webhook targets on OS-picked ports: 180 tests in 1.2 min (was 1.6)Fixes:
beforecursor skipped older events whose random id sorted after the last one of the page (empty next pages now and then); the id now only breaks ties of the same date@data-fair/frame0.19, jose 6, dotenv 18; CI actions on their current majorsHeads-up:
api/src/events/router.ts: pagination ofGET /api/eventschanges for any client followingnext(it now returns the events the old cursor skipped)npm ci,npx --no df-dev setup(new.envports, regeneratesui/dts), thennpm run dev-zellij; the mongo volume moves from 7 to 8 (docker compose down -vif it refuses to start); the simple-directorymasterimage must be recent (scoped NHI exchange)releases.yml(Keel major tag step) only runs on the next tag, untested; the node 24.22.1 image was built and boot-tested, not re-scanned with trivy@data-fair/lib-node2.17.1 peers 6 and 5); TypeScript 7 and@types/node26 on purpose (the fleet is on TS 6, node 24)