- Delete cli/.env.e2e.example (root-level duplicate).
The canonical template is cli/test/e2e/.env.e2e.example.
- Add missing @ts-expect-error before the early-return project.provide()
call in global-setup.ts (line 167). The second call at line 290 already
had the suppressor; the first one was accidentally left without it,
causing TS2345 in the CLI Tests CI.
When CI runs parallel suite jobs, each job's vitest globalSetup calls
provisionApps() independently. Simultaneous findAppByName() calls all
return 'not found' (race condition), causing each job to importFromDsl
and create duplicate apps — especially in auto_test1 (ws2-workflow.yml).
Fix:
1. Check if all DIFY_E2E_*_APP_ID env vars are pre-set (from CI provision
job outputs). If so, skip provisionApps entirely and reuse those IDs.
2. Fall back to env-sourced app IDs in capabilities assignment so that
provision job outputs are always honoured even if provisionApps runs.
Workflow (cli-e2e.yml):
- Plan B: split into parallel jobs after a shared provision step
- provision job: mints token + provisions DSL apps, outputs IDs to downstream jobs
- suite-run: matrix of 5 parallel jobs (basic/streaming/conversation/file/hitl)
- suite-last: serial, waits for all parallel jobs; runs use/devices/logout/agent
- Add DIFY_E2E_NO_KEYRING=1 globally (Linux CI has no keychain)
- timeout-minutes: 30 → 60; smoke filter: --testNamePattern → -t
New script (cli/scripts/e2e-provision.ts):
- Standalone Bun script: login + mint token + discover workspaces + provision apps
- Outputs to GITHUB_OUTPUT + .provision-output.json
Test fixes:
- run-app-streaming: DIFY_E2E_NO_KEYRING=1 in spawn-based tests
- run-app-basic: fix cache test (DIFY_CACHE_DIR + app-info.yml path)
- run-app-conversation/file: SSO injectAuth uses new hosts.yml format
- get-app-list: tag filter auto-provisions e2e-test tag
- vitest.e2e.config.ts: DIFY_E2E_SINGLE_FILE override + agent suite
Both augmentation approaches fail under tsgo in the Main CI pipeline:
- 'vitest' augmentation → TS2300 (re-exported ProvidedContext in @0.1.22)
- '@voidzero-dev/vite-plus-test' augmentation → TS2664 (module not found;
tsgo scans cli/ tree but cannot resolve pnpm virtual-store symlinks)
Use @ts-expect-error at the three call sites (global-setup, devices, logout)
as the only option that satisfies both tsgo and the ESLint ban-ts-comment
rule (which requires @ts-expect-error over @ts-ignore).
Root cause: keyof ProvidedContext = never in @0.1.22 because the interface
is empty. inject() / project.provide() have T extends keyof ProvidedContext,
so any string literal — including 'as any' casts — is TS2345 under tsgo
which scans the whole cli/ tree (not just the src/ include in tsconfig.json).
Fix: augment '@voidzero-dev/vite-plus-test' directly (where ProvidedContext
is defined as an empty interface meant for user extension via declaration
merging) instead of 'vitest'. This avoids TS2300 duplicate identifier in
@0.1.22 and restores full type safety for inject/provide without any cast.
Revert all previous workaround casts ('as any', 'as unknown as') now that
the augmentation path is correct.
In @0.1.22 ProvidedContext is an empty interface with no augmentation path
that avoids TS2345 (keyof ProvidedContext = never). Cast the string keys
with 'as any' (suppressed via eslint-disable) at each call site so both
project.provide() and inject() compile cleanly without module augmentation.
- Override pnpm to v11 (packageManager field) before CLI install step to
resolve ERR_PNPM_BAD_PM_VERSION conflict with setup-web's pnpm@9
- Add per-suite dedicated tokens (logoutToken / devicesToken) in global-setup
via device flow to prevent token cross-contamination between suites
- Translate all Chinese comments and test names to English across e2e suites
- Fix injectAuth: add tokenId field so devices revoke correctly detects selfHit
- Fix HITL test: read action id dynamically from pause response instead of
hardcoding 'submit'
Covers auth, config, run, and CLI framework scenarios against a live
staging server using vitest + real difyctl binary.
Suite layout:
test/e2e/
├── helpers/
│ ├── cli.ts — run(), withAuthFixture(), mintFreshToken()
│ ├── assert.ts — assertExitCode, assertJson, assertErrorEnvelope
│ ├── cleanup-registry.ts — staging data teardown
│ └── retry.ts — withRetry() for flaky network assertions
├── setup/
│ ├── global-setup.ts — health-check, disposable token mint
│ └── global-teardown.ts — conversation cleanup
└── suites/
├── auth/ — status, use, whoami, devices, logout
├── config/ — path, get/set/unset/view, env override
└── run/ — basic, streaming, conversation, file, HITL
Key design decisions:
- Each test uses an isolated temp configDir via withAuthFixture()
- Logout and devices-revoke tests run last to avoid invalidating
the shared E.token used by all other suites
- mintFreshToken() mints a disposable dfoa_ token on demand via the
device flow API so revoke tests never touch the primary session
- Global retry is 0; flaky network calls use withRetry() locally
- test:e2e:smoke script filters to [P0] cases via testNamePattern
package.json: add test:e2e / test:e2e:smoke / test:e2e:local scripts
.gitignore: exclude .env.e2e, oclif.manifest.json, tmp/
.env.e2e.example: credential template for local setup
gigglewang0417