Compare commits

..
Author SHA1 Message Date
JzoNg 5c8cdb4ca1 Merge branch 'main' into jzh 2026-07-27 17:56:29 +08:00
JzoNg dfa2859ea3 feat(contacts): confirm platform contact upgrades 2026-07-27 17:53:22 +08:00
JzoNg a68fca6f68 feat(members): confirm external contact upgrades 2026-07-27 17:39:44 +08:00
JzoNg f7a14cb99f chore: merge main into jzh 2026-07-27 16:41:26 +08:00
JzoNg 02f9e1b7ad merge main 2026-07-27 14:55:37 +08:00
JzoNg 5350700e47 feat(workflow): refresh Human Input migration UI 2026-07-27 14:22:59 +08:00
JzoNg 67a31db615 Merge branch 'main' into jzh 2026-07-27 13:39:32 +08:00
JzoNg ec3ff5cc01 chore(merge): merge main 2026-07-27 10:31:26 +08:00
QuantumGhost 7c2c319538 Add IM contact sync API change proposal 2026-07-27 02:44:47 +08:00
QuantumGhost 3815aac382 fix(api): keep IM delivery independent from email feature checks 2026-07-27 02:26:37 +08:00
QuantumGhost 4136705d97 fix(api): isolate human input delivery provider failures 2026-07-27 02:26:37 +08:00
QuantumGhost 14b740fc60 feat(api): dispatch human input forms through delivery providers 2026-07-27 02:26:37 +08:00
QuantumGhost 763d1695f2 fix(api): harden human input delivery payload handling 2026-07-27 02:25:57 +08:00
QuantumGhost bcefa8491d feat(api): add IM-ready human input delivery contract 2026-07-27 02:25:57 +08:00
QuantumGhost f89bcd0496 test(api): cover human input form delivery providers 2026-07-27 02:25:07 +08:00
QuantumGhost 08322dc8c2 feat(api): introduce human input form delivery providers 2026-07-27 02:24:17 +08:00
QuantumGhost fab7a0af7b Merge remote-tracking branch 'upstream/feat/hitl-im-dev' into feat/hitl-im-dev 2026-07-27 02:14:51 +08:00
QuantumGhost ce6a3787de test(api): strengthen contact directory branch coverage 2026-07-25 13:04:03 +08:00
QuantumGhost a277a0dc4c docs(openspec): archive human input v2 submission runtime 2026-07-25 12:41:51 +08:00
QuantumGhost f87b1f4693 fix(api): retry serialized submissions safely 2026-07-25 12:35:46 +08:00
QuantumGhost b3475ed624 fix(api): harden human input v2 submission invariants 2026-07-25 11:57:48 +08:00
QuantumGhost 92c3600fc4 feat: implement human input v2 submission runtime 2026-07-25 11:14:08 +08:00
QuantumGhost 363a7834aa docs(openspec): archive human input v2 OTP proof session 2026-07-25 10:08:56 +08:00
QuantumGhost be54c54a5d fix(api): tighten OTP proof session invariants 2026-07-25 10:05:00 +08:00
QuantumGhost 6f3d09009d feat(api): implement human input v2 OTP proof sessions 2026-07-25 09:36:52 +08:00
QuantumGhost d6583084ec docs(openspec): archive human input v2 form core 2026-07-25 08:35:00 +08:00
QuantumGhost 993ccfc7d8 fix(api): tighten human input v2 form invariants 2026-07-25 08:30:58 +08:00
QuantumGhost 8abbdec4fd feat(api): implement human input v2 form core 2026-07-25 08:07:08 +08:00
QuantumGhost 64dba42bf3 docs(openspec): archive human input v2 recipient resolution 2026-07-25 07:16:24 +08:00
QuantumGhost a3ded70600 fix(api): harden recipient resolution fallbacks 2026-07-25 07:13:43 +08:00
QuantumGhost fdde47bbc3 feat(api): implement human input v2 recipient resolution 2026-07-25 06:56:33 +08:00
QuantumGhost c16bbe786f docs(openspec): archive human input v2 IM control plane 2026-07-25 06:24:41 +08:00
QuantumGhost ab816a54aa fix(api): align IM sync contact semantics 2026-07-25 06:22:03 +08:00
QuantumGhost 2d3ba7fa5d fix(api): harden IM control plane invariants 2026-07-25 06:10:34 +08:00
QuantumGhost 7c5fc2a133 feat(api): implement human input v2 im control plane 2026-07-25 05:39:30 +08:00
QuantumGhost cb9ae0c0a6 docs(openspec): normalize contact directory spec 2026-07-25 04:48:35 +08:00
QuantumGhost ad31e11320 docs(openspec): archive human input v2 contact directory 2026-07-25 04:47:17 +08:00
QuantumGhost eedb4d29ea fix(api): scope contact directory deployment locking 2026-07-25 04:43:41 +08:00
QuantumGhost 4639e4b818 fix(api): serialize contact email claims 2026-07-25 04:24:04 +08:00
QuantumGhost 95e73dd4b1 fix(api): tighten contact directory invariants 2026-07-25 04:01:34 +08:00
QuantumGhost 05e2f1ddef fix(api): keep legacy human input version typing compatible 2026-07-25 03:28:29 +08:00
QuantumGhost 4ef0c50a97 feat(api): implement human input v2 contact directory 2026-07-25 03:26:43 +08:00
QuantumGhost 6c2ac9cd91 docs(openspec): split human input v2 domain core 2026-07-25 02:54:52 +08:00
QuantumGhost 318fbcef27 chore: update docs 2026-07-25 02:31:14 +08:00
QuantumGhost 4db897abc1 feat(api): add independent HITL v2 form models
Assisted-by: Codex
2026-07-25 02:03:08 +08:00
QuantumGhost 5f78012b7d docs(openspec): define human input v2 domain core
Assisted-by: Codex
2026-07-24 14:48:50 +08:00
JzoNg eed57c9abd Merge branch 'main' into jzh 2026-07-24 14:30:24 +08:00
QuantumGhost 179351d1fe Merge remote-tracking branch 'upstream/feat/hitl-im-dev' into feat/hitl-im-dev 2026-07-24 14:27:25 +08:00
JzoNg f1f20384eb Merge branch 'main' into jzh 2026-07-24 14:03:19 +08:00
QuantumGhost 4b2ceb1bf6 Merge remote-tracking branch 'upstream/feat/hitl-im-dev' into feat/hitl-im-dev 2026-07-24 11:56:21 +08:00
QuantumGhost eda9315a7b Merge remote-tracking branch 'upstream/feat/hitl-im-dev' into feat/hitl-im-dev 2026-07-24 11:55:46 +08:00
JzoNg 3a848a7a9c chore(merge): merge main 2026-07-24 11:25:45 +08:00
QuantumGhost 11b0723a92 chore: remove domain model stub 2026-07-24 11:03:11 +08:00
QuantumGhost f6440f6af4 update api stub, design spec and db models 2026-07-24 10:43:51 +08:00
QuantumGhost 462af76989 test(models): cover strict StrEnum JSON restoration
Assisted-by: Codex:gpt-5.6-sol
2026-07-24 05:03:22 +08:00
QuantumGhost 4b17446ed6 docs(models): clarify JSONModelColumn migration risks
Explain why legacy callers and stored data require compatibility review before adopting FrozenPydanticModelColumn.

Assisted-by: Codex
2026-07-23 15:33:48 +08:00
QuantumGhost 252054f9b6 fix(models): persist frozen Pydantic columns as text
Rename the strict immutable column type, deprecate JSONModelColumn, and cover single-encoded JSON storage.

Assisted-by: Codex
2026-07-23 15:29:21 +08:00
JzoNg 4193deb220 chore(merge): merge main 2026-07-23 15:26:49 +08:00
QuantumGhost 075d44a7fa refactor(models): split Pydantic JSON constructors
Assisted-by: Codex
2026-07-23 15:07:43 +08:00
QuantumGhost 5e696c9131 feat(models): add Pydantic model JSON type
Assisted-by: Codex
2026-07-23 14:40:43 +08:00
JzoNg 5f1646c14b feat(human-input): add batch migration and contact lookup boundaries 2026-07-23 11:09:29 +08:00
JzoNg 30b04d4d14 Merge branch 'main' into jzh 2026-07-23 09:59:09 +08:00
QuantumGhost e592c9f3ac docs: remove python stub from api summary docs 2026-07-23 09:20:59 +08:00
QuantumGhost fa5a2ff899 chore(api): move shared enum definitions to core/human_input_v2 directory 2026-07-23 09:20:25 +08:00
JzoNg 45d6efd8ab Merge branch 'main' into jzh 2026-07-23 07:29:26 +08:00
JzoNg d6b8466bcd refactor(contacts): simplify repository boundary 2026-07-22 18:04:12 +08:00
JzoNg 7fc5e60007 test(contacts): cover backend-aligned list contract 2026-07-22 17:42:36 +08:00
JzoNg 4e4f34be1f refactor(contacts): align management with list contract 2026-07-22 17:42:21 +08:00
QuantumGhost 09076f6555 docs: update spec and api stub 2026-07-22 17:02:23 +08:00
JzoNg dfcf690d9a feat(contacts): refine platform contact management 2026-07-22 16:37:24 +08:00
JzoNg ad1fe871f4 docs(openspec): clarify platform contact management 2026-07-22 16:37:08 +08:00
JzoNg f587807123 feat(contacts): add configured channel actions 2026-07-22 15:59:04 +08:00
JzoNg 6154e58da7 feat(contacts): add channels and email configuration 2026-07-22 15:52:45 +08:00
JzoNg 85d3226175 docs(openspec): update channels management requirements 2026-07-22 15:37:00 +08:00
QuantumGhost 47f20c8f3d chore: update openspec docs 2026-07-22 15:20:57 +08:00
JzoNg 074c172f52 chore(merge): resolve main branch conflicts 2026-07-22 14:54:53 +08:00
JzoNg 9986ff92b6 Merge branch 'main' into jzh 2026-07-21 17:02:03 +08:00
QuantumGhost ce694a20b4 Merge remote-tracking branch 'upstream/feat/hitl-im-dev' into feat/hitl-im-dev 2026-07-21 16:44:04 +08:00
QuantumGhost ef5933a90e chore(api): refine HITL v2 docs and API stubs 2026-07-21 16:12:35 +08:00
JzoNg ce66f7505c Merge branch 'main' into jzh 2026-07-20 17:35:46 +08:00
JzoNg 41024954b7 fix(human-input): rename recipients spec field 2026-07-20 17:29:38 +08:00
JzoNg 1af6a1803a Merge branch 'main' into jzh 2026-07-20 16:56:11 +08:00
JzoNg 446f0f993a chore(merge): merge main into jzh 2026-07-20 15:40:12 +08:00
JzoNg 3c88536e2a chore(openspec): complete Human Input v2 form tasks 2026-07-20 15:32:17 +08:00
JzoNg 34b71467e7 feat(file-uploader): add Human Input v2 upload routing 2026-07-20 15:32:03 +08:00
JzoNg 7f2e40225c feat(human-input-v2): add OTP-guarded public form route 2026-07-20 15:31:39 +08:00
JzoNg 33b0bc578f feat(human-input-v2): add mock-first transport contracts 2026-07-20 15:31:14 +08:00
JzoNg aa128184f8 refactor(human-input): extract shared public form presentation 2026-07-20 15:30:44 +08:00
JzoNg 47f6cac432 chore(merge): merge main into jzh 2026-07-20 14:43:52 +08:00
JzoNg 1c9d15116c chore(openspec): complete Human Input v2 migration tasks 2026-07-20 14:31:02 +08:00
JzoNg 679d011d17 feat(workflow): add Human Input v2 migration flow 2026-07-20 14:28:27 +08:00
JzoNg 1d4b5eba35 feat(workflow): gate Human Input v2 creation 2026-07-20 14:05:27 +08:00
JzoNg ad328d156a feat(workflow): add Human Input v2 migration planner 2026-07-20 13:57:15 +08:00
JzoNg 1e5c206d31 docs(openspec): propose Human Input v2 form page 2026-07-20 13:43:39 +08:00
JzoNg 8b66eedaba docs(openspec): propose Human Input v2 migration UI 2026-07-20 11:52:40 +08:00
JzoNg ea7465b853 merge main 2026-07-20 10:16:45 +08:00
QuantumGhost c5d19039c4 Merge remote-tracking branch 'upstream/feat/hitl-im-dev' into feat/hitl-im-dev 2026-07-19 11:33:39 +08:00
JzoNg 9cc072ed5d docs(openspec): complete Human Input v2 UI tasks 2026-07-18 14:07:15 +08:00
JzoNg 0c3e3f6f78 feat(workflow): integrate Human Input v2 workflow utilities 2026-07-18 14:06:53 +08:00
JzoNg 922565f5e3 feat(workflow): add Human Input v2 configuration UI 2026-07-18 14:06:38 +08:00
JzoNg 90487f57fc refactor(workflow): share Human Input editor infrastructure 2026-07-18 14:06:25 +08:00
JzoNg cee50acbc4 docs(openspec): document Human Input v2 UI acceptance 2026-07-18 14:04:49 +08:00
JzoNg 84fd4011ab Merge branch 'main' into jzh 2026-07-18 12:55:28 +08:00
JzoNg ef5f698746 Merge branch 'main' into jzh 2026-07-17 16:41:08 +08:00
JzoNg 2603ca1bfb feat(contacts): add mock-backed contacts management UI 2026-07-17 16:33:32 +08:00
QuantumGhost 9791c0891d chore: update frontend api stubs 2026-07-17 15:24:51 +08:00
JzoNg 93a745c9ca chore(contacts): complete IM platform frontend quality checks 2026-07-17 15:17:30 +08:00
JzoNg 78980e9f1f feat(contacts): implement IM contact sync and details 2026-07-17 15:17:29 +08:00
JzoNg b30c3578a7 feat(contacts): implement IM platform binding management UI 2026-07-17 15:17:29 +08:00
JzoNg ef5c3df88b feat(contacts): implement IM platform mock data boundary 2026-07-17 15:17:29 +08:00
JzoNg e35752ddbc fix(contacts): restrict IM platform entry to non-enterprise editions 2026-07-17 15:17:29 +08:00
JzoNg 5b5d62a2af feat(contacts): establish IM platform frontend baseline 2026-07-17 15:17:29 +08:00
JzoNg cdd111869b docs: human input node v2 2026-07-17 15:09:50 +08:00
JzoNg 4fd6c028af docs: contacts management ui 2026-07-17 15:09:50 +08:00
JzoNg abcc77204b docs: add IM platform binding 2026-07-17 15:09:50 +08:00
JzoNg dce8e971be Merge branch 'main' into tp 2026-07-17 15:09:02 +08:00
QuantumGhost 2249974daa feat: add API stub 2026-07-17 14:32:03 +08:00
QuantumGhost f49e3dd5b1 chore: clarify organization contact and platform contact 2026-07-17 14:32:03 +08:00
QuantumGhost 51c471a359 chore: remove outdated docs 2026-07-17 14:32:03 +08:00
QuantumGhost 1f05bdd543 chore: remove outdated docs 2026-07-17 14:32:03 +08:00
QuantumGhost 2250f83d0f chore: remove outdated docs 2026-07-17 14:32:03 +08:00
QuantumGhost 30fdbed7f2 chore: separate lark and feishu provider 2026-07-17 14:32:03 +08:00
QuantumGhost 3ff1c91d84 chore: remove outdated docs 2026-07-17 14:32:03 +08:00
QuantumGhost b9561cef58 chore: update specs 2026-07-17 14:32:02 +08:00
QuantumGhost ee1b9cb95f chore: update spec 2026-07-17 14:32:02 +08:00
QuantumGhost c1154686d8 chore: update openspec docs 2026-07-17 14:32:02 +08:00
JzoNg 3ead32a4e8 Merge branch 'main' into tp 2026-07-17 13:06:50 +08:00
JzoNg 6f1ecc47e8 Merge branch 'main' into tp 2026-07-17 11:24:24 +08:00
JzoNg 3d80e13f4f Merge branch 'main' into tp 2026-07-16 20:30:35 +08:00
JzoNg 75e2bcd460 Merge branch 'main' into tp 2026-07-16 17:47:25 +08:00
JzoNg 691eca212b Merge branch 'main' into tp 2026-07-16 11:25:16 +08:00
JzoNg e21a0883c1 Merge branch 'main' into tp 2026-07-16 08:13:27 +08:00
JzoNg ba1c13e37c Merge branch 'main' into tp 2026-07-14 14:16:26 +08:00
QuantumGhost 1e2f6f234e feat: add HITL IM DSL schema 2026-07-14 13:59:24 +08:00
QuantumGhost 6813e8d34f chore: introduce openspec 2026-07-14 10:36:52 +08:00
QuantumGhost e348d8accf Merge remote-tracking branch 'upstream/feat/hitl-im-dev' into feat/hitl-im-dev 2026-07-14 10:36:12 +08:00
QuantumGhost b71bbc2d3b Merge remote-tracking branch 'upstream/main' into feat/hitl-im-dev 2026-07-14 01:59:51 +08:00
JzoNg 69e18efab6 Merge branch 'main' into tp 2026-07-13 16:40:13 +08:00
QuantumGhost 196287fc8a docs: add HITL IM docs 2026-07-11 17:02:34 +08:00
2677 changed files with 82657 additions and 600397 deletions
-15
View File
@@ -1,15 +0,0 @@
{
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "npx -y block-no-verify@1.1.1"
}
]
}
]
}
}
-1
View File
@@ -8,7 +8,6 @@
**/*.pyc
**/.mypy_cache
**/.ruff_cache
knowledge-fs/
.git
.github
*.md
-9
View File
@@ -1,15 +1,6 @@
version: 2
updates:
- package-ecosystem: "npm"
directory: "/knowledge-fs"
open-pull-requests-limit: 10
schedule:
interval: "weekly"
groups:
knowledge-fs-dependencies:
patterns:
- "*"
- package-ecosystem: "uv"
directory: "/api"
open-pull-requests-limit: 10
+4 -51
View File
@@ -2,70 +2,23 @@ name: Deploy Dev
on:
workflow_run:
workflows: ["KnowledgeFS CI"]
workflows: ["Build and Push API & Web"]
branches:
- "deploy/konwledge"
- "deploy/dev"
types:
- completed
permissions: {}
jobs:
deploy:
runs-on: depot-ubuntu-24.04
permissions:
actions: read
contents: read
if: |
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'deploy/konwledge'
github.event.workflow_run.head_branch == 'deploy/dev'
steps:
- name: Wait for API and Web image build
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
timeout-minutes: 35
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const workflowId = "build-push.yml";
const headBranch = context.payload.workflow_run.head_branch;
const headSha = context.payload.workflow_run.head_sha;
const deadline = Date.now() + 30 * 60 * 1000;
const pollIntervalMs = 15 * 1000;
while (Date.now() < deadline) {
const { data } = await github.rest.actions.listWorkflowRuns({
owner: context.repo.owner,
repo: context.repo.repo,
workflow_id: workflowId,
branch: headBranch,
event: "push",
head_sha: headSha,
per_page: 10,
});
const run = data.workflow_runs[0];
if (!run) {
core.info(`Waiting for ${workflowId} to start for ${headSha}.`);
} else if (run.status !== "completed") {
core.info(`Waiting for ${run.html_url}; current status is ${run.status}.`);
} else if (run.conclusion !== "success") {
throw new Error(
`${workflowId} did not succeed for ${headSha}: ${run.conclusion} (${run.html_url})`,
);
} else {
core.info(`Both image workflows succeeded for ${headSha}: ${run.html_url}`);
return;
}
await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
}
throw new Error(`Timed out waiting for ${workflowId} to succeed for ${headSha}.`);
- name: Deploy to server
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2 # v1.2.5
with:
host: ${{ secrets.SSH_NEW_RAG_HOST }}
host: ${{ secrets.SSH_HOST }}
username: ${{ secrets.SSH_USER }}
key: ${{ secrets.SSH_PRIVATE_KEY }}
script: |
-43
View File
@@ -1,7 +1,6 @@
name: Deploy Knowledge
permissions:
actions: read
contents: read
on:
@@ -19,48 +18,6 @@ jobs:
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'deploy/konwledge'
steps:
- name: Wait for KnowledgeFS CI
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
timeout-minutes: 35
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const workflowId = "knowledge-fs-ci.yml";
const headBranch = context.payload.workflow_run.head_branch;
const headSha = context.payload.workflow_run.head_sha;
const deadline = Date.now() + 30 * 60 * 1000;
const pollIntervalMs = 15 * 1000;
while (Date.now() < deadline) {
const { data } = await github.rest.actions.listWorkflowRuns({
owner: context.repo.owner,
repo: context.repo.repo,
workflow_id: workflowId,
branch: headBranch,
event: "push",
head_sha: headSha,
per_page: 10,
});
const run = data.workflow_runs[0];
if (!run) {
core.info(`Waiting for ${workflowId} to start for ${headSha}.`);
} else if (run.status !== "completed") {
core.info(`Waiting for ${run.html_url}; current status is ${run.status}.`);
} else if (run.conclusion !== "success") {
throw new Error(
`${workflowId} did not succeed for ${headSha}: ${run.conclusion} (${run.html_url})`,
);
} else {
core.info(`KnowledgeFS CI succeeded for ${headSha}: ${run.html_url}`);
return;
}
await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
}
throw new Error(`Timed out waiting for ${workflowId} to succeed for ${headSha}.`);
- name: Deploy to server
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2 # v1.2.5
with:
-514
View File
@@ -1,514 +0,0 @@
name: KnowledgeFS CI
on:
pull_request:
branches: ["main"]
merge_group:
branches: ["main"]
types: [checks_requested]
push:
branches: ["main", "deploy/konwledge"]
workflow_dispatch:
permissions:
contents: read
pull-requests: read
concurrency:
group: knowledge-fs-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
env:
CI: true
DIFY_KNOWLEDGE_FS_API_IMAGE_NAME: >-
${{ vars.DIFY_KNOWLEDGE_FS_API_IMAGE_NAME || 'langgenius/dify-knowledge-fs-api' }}
jobs:
check-changes:
name: Check KnowledgeFS changes
runs-on: depot-ubuntu-24.04-4
outputs:
knowledge-fs: ${{ steps.changes.outputs.knowledge-fs }}
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 0
persist-credentials: false
- name: Detect KnowledgeFS changes
id: changes
uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2
with:
filters: |
knowledge-fs:
- 'knowledge-fs/**'
- 'knowledge-fs/packages/api/src/dify-capability-v2.ts'
- 'knowledge-fs/packages/api/src/knowledge-space-routes.ts'
- 'knowledge-fs/packages/api/src/upload-session-routes.ts'
- 'knowledge-fs/scripts/export-capability-v2-operations.mjs'
- 'knowledge-fs/scripts/export-openapi.mjs'
- 'api/dev/generate_knowledge_fs_contract.py'
- 'api/dev/knowledge_fs_product_contract.py'
- 'api/knowledge-fs-contract.lock.json'
- 'api/knowledge-fs-product-operation-gaps.json'
- 'api/knowledge-fs-product-operations.json'
- 'api/**/knowledge_fs/**'
- 'api/**/*knowledge_fs*'
- 'api/**/*knowledge-fs*'
- 'api/.env.example'
- 'api/app_factory.py'
- 'api/commands/__init__.py'
- 'api/controllers/console/__init__.py'
- 'api/controllers/console/workspace/rbac.py'
- 'api/controllers/service_api/__init__.py'
- 'api/core/agent/base_agent_runner.py'
- 'api/core/app/apps/agent_app/runtime_request_builder.py'
- 'api/core/rbac/entities.py'
- 'api/core/tools/__base/tool_runtime.py'
- 'api/core/tools/builtin_tool/_position.yaml'
- 'api/core/workflow/node_runtime.py'
- 'api/core/workflow/nodes/agent_v2/runtime_request_builder.py'
- 'api/extensions/ext_celery.py'
- 'api/extensions/ext_commands.py'
- 'api/models/__init__.py'
- 'api/services/account_service.py'
- 'api/services/agent_tool_inner_service.py'
- 'api/services/enterprise/rbac_service.py'
- 'api/services/entities/agent_tool_inner.py'
- 'api/services/knowledge_fs/**'
- 'api/services/knowledge_fs_capability.py'
- 'api/tests/unit_tests/dev/test_generate_knowledge_fs_contract.py'
- 'api/tests/unit_tests/controllers/console/workspace/test_rbac.py'
- 'api/tests/unit_tests/core/agent/test_base_agent_runner.py'
- 'api/tests/unit_tests/core/app/apps/agent_app/test_runtime_request_builder.py'
- 'api/tests/unit_tests/core/workflow/nodes/agent_v2/test_runtime_request_builder.py'
- 'api/tests/unit_tests/core/workflow/nodes/tool/test_tool_node_runtime.py'
- 'api/tests/unit_tests/core/workflow/test_node_runtime.py'
- 'api/tests/unit_tests/services/enterprise/test_rbac_service.py'
- 'api/tests/unit_tests/services/test_account_service.py'
- 'api/tests/unit_tests/services/test_agent_tool_inner_service.py'
- 'api/tests/unit_tests/services/test_knowledge_fs_capability.py'
- 'api/tests/unit_tests/services/test_knowledge_fs_product_operations.py'
- 'api/pyproject.toml'
- 'api/uv.lock'
- 'dify-agent/src/dify_agent/layers/dify_core_tools/client.py'
- 'dify-agent/tests/local/dify_agent/layers/dify_core_tools/test_client.py'
- 'packages/contracts/generated/api/console/**'
- 'packages/contracts/generated/api/service/**'
- 'docker/.env.example'
- 'docker/README.md'
- 'docker/dify-env-sync.py'
- 'docker/dify-env-sync.sh'
- 'docker/docker-compose-template.yaml'
- 'docker/docker-compose.yaml'
- 'docker/envs/core-services/api.env.example'
- 'docker/envs/core-services/knowledge-fs.env.example'
- 'docker/generate_docker_compose'
- 'docs/design/knowledge-fs*'
- '.github/dependabot.yml'
- '.github/workflows/knowledge-fs-ci.yml'
build:
name: Build KnowledgeFS API production image
needs:
- check-changes
- quality
if: >-
(needs.check-changes.outputs.knowledge-fs == 'true' || github.event_name == 'workflow_dispatch') &&
needs.quality.result == 'success'
runs-on: depot-ubuntu-24.04-4
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
- name: Login to Docker Hub
if: >-
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'push' &&
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/deploy/konwledge'))
uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4.4.0
with:
username: ${{ secrets.DOCKERHUB_USER }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Extract KnowledgeFS image metadata
id: meta
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
with:
images: ${{ env.DIFY_KNOWLEDGE_FS_API_IMAGE_NAME }}
tags: |
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
type=ref,event=branch
type=sha,format=long
- name: Build KnowledgeFS API image
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with:
context: ./knowledge-fs
file: ./knowledge-fs/apps/api/Dockerfile
labels: ${{ steps.meta.outputs.labels }}
platforms: linux/amd64
push: >-
${{ github.event_name == 'workflow_dispatch' ||
(github.event_name == 'push' &&
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/deploy/konwledge')) }}
tags: ${{ steps.meta.outputs.tags }}
quality:
name: Run KnowledgeFS quality and contract gates
needs: check-changes
if: needs.check-changes.outputs.knowledge-fs == 'true' || github.event_name == 'workflow_dispatch'
runs-on: depot-ubuntu-24.04-4
defaults:
run:
shell: bash
working-directory: ./knowledge-fs
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 0
persist-credentials: false
- name: Setup pnpm
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
with:
package_json_file: knowledge-fs/package.json
run_install: false
- name: Setup Node
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: 22
cache: pnpm
cache-dependency-path: knowledge-fs/pnpm-lock.yaml
- name: Install KnowledgeFS dependencies
run: pnpm install --frozen-lockfile
- name: Scan KnowledgeFS secrets
run: pnpm security:secrets
- name: Audit KnowledgeFS production dependencies
run: pnpm security:dependencies
- name: Run KnowledgeFS checks
run: pnpm check
- name: Build KnowledgeFS
run: pnpm build
- name: Lint KnowledgeFS
run: pnpm lint:backend
- name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
with:
enable-cache: true
python-version: "3.12"
cache-dependency-glob: |
api/uv.lock
dify-agent/uv.lock
- name: Verify Dify dependency lock
working-directory: .
run: uv lock --project api --check
- name: Install Dify contract dependencies
working-directory: .
run: uv sync --project api --locked --dev
- name: Collect Dify KnowledgeFS gate targets
working-directory: .
run: |
set -euo pipefail
target_dir="${RUNNER_TEMP:?}/knowledge-fs-ci-targets"
mkdir -p "$target_dir"
production_targets=()
add_production_target() {
local path="$1"
if [[ ! -f "$path" ]]; then
echo "required Dify KnowledgeFS production target is missing: $path" >&2
exit 1
fi
production_targets+=("$path")
}
while IFS= read -r -d '' path; do
if [[ "$path" == *knowledge_fs* ]]; then
add_production_target "$path"
fi
done < <(
find api \
\( -path 'api/.venv' -o -path 'api/tests' -o -path 'api/storage' \) -prune \
-o -type f -name '*.py' -print0
)
production_touchpoints=(
api/app_factory.py
api/commands/__init__.py
api/controllers/console/__init__.py
api/controllers/console/workspace/rbac.py
api/controllers/service_api/__init__.py
api/core/agent/base_agent_runner.py
api/core/app/apps/agent_app/runtime_request_builder.py
api/core/rbac/entities.py
api/core/tools/__base/tool_runtime.py
api/core/workflow/node_runtime.py
api/core/workflow/nodes/agent_v2/runtime_request_builder.py
api/extensions/ext_celery.py
api/extensions/ext_commands.py
api/models/__init__.py
api/services/account_service.py
api/services/agent_tool_inner_service.py
api/services/enterprise/rbac_service.py
api/services/entities/agent_tool_inner.py
)
for path in "${production_touchpoints[@]}"; do
add_production_target "$path"
done
printf '%s\0' "${production_touchpoints[@]}" > "$target_dir/glue-files"
if ((${#production_targets[@]} == 0)); then
echo "Dify KnowledgeFS production target set is empty" >&2
exit 1
fi
printf '%s\0' "${production_targets[@]}" > "$target_dir/production-files"
test_targets=()
add_test_target() {
local path="$1"
if [[ ! -f "$path" ]]; then
echo "required Dify KnowledgeFS unit test is missing: $path" >&2
exit 1
fi
test_targets+=("$path")
}
while IFS= read -r -d '' path; do
if [[ "$path" == *knowledge_fs* ]]; then
add_test_target "$path"
fi
done < <(find api/tests/unit_tests -type f -name '*.py' -print0)
test_touchpoints=(
api/tests/unit_tests/controllers/console/workspace/test_rbac.py
api/tests/unit_tests/core/agent/test_base_agent_runner.py
api/tests/unit_tests/core/app/apps/agent_app/test_runtime_request_builder.py
api/tests/unit_tests/core/workflow/nodes/agent_v2/test_runtime_request_builder.py
api/tests/unit_tests/core/workflow/nodes/tool/test_tool_node_runtime.py
api/tests/unit_tests/core/workflow/test_node_runtime.py
api/tests/unit_tests/services/enterprise/test_rbac_service.py
api/tests/unit_tests/services/test_account_service.py
api/tests/unit_tests/services/test_agent_tool_inner_service.py
)
for path in "${test_touchpoints[@]}"; do
add_test_target "$path"
done
required_test_scopes=(
/commands/
/configs/
/controllers/
/core/agent/
/core/app/apps/agent_app/
/core/tools/builtin_tool/providers/knowledge_fs/
/core/workflow/
/dev/
/extensions/
/migrations/
/models/
/repositories/
/services/
/tasks/
)
for required_scope in "${required_test_scopes[@]}"; do
scope_found=false
for path in "${test_targets[@]}"; do
if [[ "$path" == *"$required_scope"* ]]; then
scope_found=true
break
fi
done
if [[ "$scope_found" != true ]]; then
echo "required Dify KnowledgeFS test scope is empty: $required_scope" >&2
exit 1
fi
done
if ((${#test_targets[@]} == 0)); then
echo "Dify KnowledgeFS unit test target set is empty" >&2
exit 1
fi
printf '%s\0' "${test_targets[@]}" > "$target_dir/unit-test-files"
- name: Lint Dify KnowledgeFS integration
working-directory: .
run: |
set -euo pipefail
targets=()
while IFS= read -r -d '' path; do
targets+=("$path")
done < "${RUNNER_TEMP:?}/knowledge-fs-ci-targets/production-files"
if ((${#targets[@]} == 0)); then
echo "Dify KnowledgeFS production target manifest is empty" >&2
exit 1
fi
uv run --project api --dev ruff format --check "${targets[@]}"
uv run --project api --dev ruff check "${targets[@]}"
- name: Type-check Dify KnowledgeFS integration
working-directory: .
run: |
set -euo pipefail
targets=()
while IFS= read -r -d '' path; do
targets+=("$path")
done < "${RUNNER_TEMP:?}/knowledge-fs-ci-targets/production-files"
if ((${#targets[@]} == 0)); then
echo "Dify KnowledgeFS production target manifest is empty" >&2
exit 1
fi
PYREFLY_OUTPUT_FORMAT=github ./dev/pyrefly-check-local "${targets[@]}"
mypy_targets=()
for path in "${targets[@]}"; do
if [[ "$path" != api/migrations/* ]]; then
mypy_targets+=("${path#api/}")
fi
done
if ((${#mypy_targets[@]} == 0)); then
echo "Dify KnowledgeFS Mypy target set is empty" >&2
exit 1
fi
uv run --directory api --dev mypy \
--explicit-package-bases \
--exclude-gitignore \
--exclude '(^|/)conftest\.py$' \
--exclude 'tests/' \
--exclude 'migrations/' \
--check-untyped-defs \
--disable-error-code=import-untyped \
"${mypy_targets[@]}"
- name: Test Dify KnowledgeFS unit surface
working-directory: .
env:
COVERAGE_FILE: ${{ runner.temp }}/dify-knowledge-fs.coverage
run: |
set -euo pipefail
targets=()
while IFS= read -r -d '' path; do
targets+=("$path")
done < "${RUNNER_TEMP:?}/knowledge-fs-ci-targets/unit-test-files"
if ((${#targets[@]} == 0)); then
echo "Dify KnowledgeFS unit test target manifest is empty" >&2
exit 1
fi
uv run --project api --dev coverage run --branch --source=api -m pytest "${targets[@]}" --no-cov -q
- name: Enforce Dify KnowledgeFS focused coverage
working-directory: .
env:
COVERAGE_FILE: ${{ runner.temp }}/dify-knowledge-fs.coverage
KNOWLEDGE_FS_COVERAGE_BASE: >-
${{ github.event.pull_request.base.sha || github.event.merge_group.base_sha || github.event.before || '' }}
run: |
set -euo pipefail
report="${RUNNER_TEMP:?}/dify-knowledge-fs-coverage.json"
uv run --project api --dev coverage json --show-contexts -o "$report"
uv run --project api --dev python api/dev/check_knowledge_fs_coverage.py \
--coverage-json "$report" \
--glue-manifest "${RUNNER_TEMP:?}/knowledge-fs-ci-targets/glue-files" \
--base "$KNOWLEDGE_FS_COVERAGE_BASE" \
--minimum 90 \
--glue-minimum 90
- name: Verify Dify KnowledgeFS contract
working-directory: .
run: uv run --project api python api/dev/generate_knowledge_fs_contract.py --check
- name: Verify Dify Agent dependency lock
working-directory: .
run: uv lock --project dify-agent --check
- name: Install Dify Agent gate dependencies
working-directory: .
run: uv sync --project dify-agent --locked --dev
- name: Lint Dify Agent KnowledgeFS integration
working-directory: ./dify-agent
run: |
uv run --project . --dev ruff format --check \
src/dify_agent/layers/dify_core_tools/client.py \
tests/local/dify_agent/layers/dify_core_tools/test_client.py
uv run --project . --dev ruff check \
src/dify_agent/layers/dify_core_tools/client.py \
tests/local/dify_agent/layers/dify_core_tools/test_client.py
- name: Type-check Dify Agent KnowledgeFS integration
working-directory: ./dify-agent
run: >-
uv run --project . --dev basedpyright --level error
src/dify_agent/layers/dify_core_tools/client.py
tests/local/dify_agent/layers/dify_core_tools/test_client.py
- name: Test Dify Agent KnowledgeFS integration
working-directory: ./dify-agent
run: >-
uv run --project . --dev python -m pytest
tests/local/dify_agent/layers/dify_core_tools/test_client.py
-q
skip:
name: Skip KnowledgeFS quality and contract gates
needs: check-changes
if: needs.check-changes.outputs.knowledge-fs != 'true' && github.event_name != 'workflow_dispatch'
runs-on: depot-ubuntu-24.04-4
steps:
- name: Report skipped KnowledgeFS checks
run: echo "No KnowledgeFS-related changes detected; skipping KnowledgeFS checks."
final:
name: KnowledgeFS CI
if: ${{ always() }}
needs:
- check-changes
- build
- quality
- skip
runs-on: depot-ubuntu-24.04-4
steps:
- name: Finalize KnowledgeFS CI status
env:
EVENT_NAME: ${{ github.event_name }}
BUILD_RESULT: ${{ needs.build.result }}
KNOWLEDGE_FS_CHANGED: ${{ needs.check-changes.outputs.knowledge-fs }}
QUALITY_RESULT: ${{ needs.quality.result }}
SKIP_RESULT: ${{ needs.skip.result }}
run: |
if [[ "$EVENT_NAME" == 'workflow_dispatch' || "$KNOWLEDGE_FS_CHANGED" == 'true' ]]; then
if [[ "$BUILD_RESULT" == 'success' && "$QUALITY_RESULT" == 'success' ]]; then
echo "KnowledgeFS build and checks ran successfully."
exit 0
fi
echo "KnowledgeFS build or checks failed: build=$BUILD_RESULT quality=$QUALITY_RESULT" >&2
exit 1
fi
if [[ "$SKIP_RESULT" == 'success' ]]; then
echo "KnowledgeFS checks were skipped because no related files changed."
exit 0
fi
echo "KnowledgeFS change detection or skip reporting failed with result: $SKIP_RESULT" >&2
exit 1
-5
View File
@@ -30,11 +30,6 @@ share/python-wheels/
*.egg
MANIFEST
# KnowledgeFS is an independently rooted TypeScript workspace. Its admin `lib`
# directory contains source files rather than Python build output.
!/knowledge-fs/apps/admin/lib/
!/knowledge-fs/apps/admin/lib/**
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
+3 -18
View File
@@ -666,7 +666,6 @@ PLUGIN_REMOTE_INSTALL_PORT=5003
PLUGIN_REMOTE_INSTALL_HOST=localhost
PLUGIN_MAX_PACKAGE_SIZE=15728640
PLUGIN_MODEL_SCHEMA_CACHE_TTL=3600
PLUGIN_MODEL_PROVIDERS_CACHE_ENABLED=true
PLUGIN_MODEL_PROVIDERS_CACHE_TTL=86400
# Comma-separated marketplace plugin IDs whose latest versions are installed for newly registered users.
# Example: langgenius/openai,langgenius/gemini
@@ -686,25 +685,11 @@ AGENT_BACKEND_RUN_TIMEOUT_SECONDS=1200
# KnowledgeFS (Dataset 2.0)
KNOWLEDGE_FS_ENABLED=false
# Production deployments require HTTPS; plain HTTP is limited to non-production or loopback.
KNOWLEDGE_FS_BASE_URL=
KNOWLEDGE_FS_DIRECT_ORIGIN=
KNOWLEDGE_FS_LIFECYCLE_WORKER_ENABLED=false
KNOWLEDGE_FS_INTEGRATED_PROVISION_READY=false
KNOWLEDGE_FS_LEGACY_ACL_FREEZE_READY=false
KNOWLEDGE_FS_LIFECYCLE_POLL_INTERVAL_SECONDS=15
KNOWLEDGE_FS_LIFECYCLE_LEASE_SECONDS=60
KNOWLEDGE_FS_LIFECYCLE_BATCH_SIZE=25
KNOWLEDGE_FS_CAPABILITY_V2_ENABLED=false
KNOWLEDGE_FS_CAPABILITY_V2_SIGNING_KID=
KNOWLEDGE_FS_CAPABILITY_V2_PRIVATE_KEY_PEM=
KNOWLEDGE_FS_CAPABILITY_V2_PREVIOUS_PUBLIC_JWKS=
KNOWLEDGE_FS_CAPABILITY_V2_ISSUER=dify-control-plane
KNOWLEDGE_FS_CAPABILITY_V2_AUDIENCE=knowledge-fs
KNOWLEDGE_FS_CAPABILITY_V2_MAX_TTL_SECONDS=60
# Shared with KnowledgeFS; use at least 32 random characters.
KNOWLEDGE_FS_JWT_SECRET=
KNOWLEDGE_FS_SSE_READ_TIMEOUT_SECONDS=300
KNOWLEDGE_FS_TIMEOUT_SECONDS=10
KNOWLEDGE_FS_JWKS_CACHE_MAX_AGE_SECONDS=300
KNOWLEDGE_FS_PRODUCT_MAX_RESPONSE_BYTES=4194304
# Marketplace configuration
MARKETPLACE_ENABLED=true
+1 -3
View File
@@ -50,9 +50,7 @@ The scripts resolve paths relative to their location, so you can run them from a
./dev/start-worker
```
1. Start Celery Beat when scheduled tasks are needed. This is required when
`KNOWLEDGE_FS_LIFECYCLE_WORKER_ENABLED=true` so provisioning and deletion
outbox commands are dispatched to the lifecycle worker.
1. Optional: start Celery Beat (scheduled tasks).
```bash
./dev/start-beat
-2
View File
@@ -183,7 +183,6 @@ def initialize_extensions(app: DifyApp):
ext_forward_refs,
ext_hosting_provider,
ext_import_modules,
ext_knowledge_fs_observability,
ext_logging,
ext_login,
ext_logstore,
@@ -234,7 +233,6 @@ def initialize_extensions(app: DifyApp):
ext_enterprise_telemetry,
ext_request_logging,
ext_session_factory,
ext_knowledge_fs_observability,
ext_oauth_bearer,
]
for ext in extensions:
-2
View File
@@ -10,7 +10,6 @@ from .data_migration import (
import_migration_data,
migration_data_wizard,
)
from .knowledge_fs import knowledge_fs_control_space
from .plugin import (
backfill_plugin_auto_upgrade,
extract_plugins,
@@ -76,7 +75,6 @@ __all__ = [
"import_migration_data",
"install_plugins",
"install_rag_pipeline_plugins",
"knowledge_fs_control_space",
"legacy_model_types",
"migrate_annotation_vector_database",
"migrate_data_for_plugin",
-621
View File
@@ -1,621 +0,0 @@
"""Operator commands for the independent KnowledgeFS control-plane."""
from __future__ import annotations
import json
from collections.abc import Callable
from datetime import datetime
from functools import partial
from pathlib import Path
import click
from pydantic import BaseModel, ValidationError
from core.db.session_factory import session_factory
from services.knowledge_fs.cleanup import (
CleanupApprovalInput,
CleanupCompletionEvidenceInput,
CleanupReadinessEvidenceInput,
CleanupStartInput,
KnowledgeFSCleanupError,
KnowledgeFSCleanupService,
)
from services.knowledge_fs.control_space_commands import KnowledgeFSControlSpaceCommandService
from services.knowledge_fs.control_space_lifecycle import KnowledgeFSControlSpaceLifecycleError
from services.knowledge_fs.control_space_management import (
KnowledgeFSControlSpaceManagementService,
KnowledgeFSControlSpaceRegistration,
)
from services.knowledge_fs.cutover import (
CutoverSmokeResultsInput,
FinalDeltaInput,
KnowledgeFSCutoverError,
KnowledgeFSWorkspaceCutoverService,
LegacyDependencyInput,
QuarantineResolutionInput,
ShadowAuthorizationObservationInput,
ShadowCompletionInput,
WorkspaceInventoryInput,
)
from services.knowledge_fs.greenfield_initializer import KnowledgeFSWorkspaceGreenfieldInitializer
from services.knowledge_fs.orphan_reconciler import KnowledgeFSOrphanReconciler
from services.knowledge_fs.remote_registry import get_knowledge_fs_lifecycle_remote
@click.group("knowledge-fs-control-space")
def knowledge_fs_control_space() -> None:
"""Inspect and repair Dify-owned KnowledgeFS control-space state."""
@knowledge_fs_control_space.command("dry-run")
@click.option("--tenant-id", default=None)
def dry_run(tenant_id: str | None) -> None:
report = _management_service().dry_run(tenant_id=tenant_id)
click.echo(json.dumps(report._asdict(), sort_keys=True))
@knowledge_fs_control_space.command("inventory")
@click.option("--input", "input_path", type=click.Path(path_type=Path, exists=True, dir_okay=False), required=True)
@click.option("--apply", is_flag=True, default=False, help="Create ledgers; omitted means read-only inventory.")
def inventory(input_path: Path, apply: bool) -> None:
"""Validate strict Workspace inventory JSONL and optionally create ledgers."""
service = _cutover_service()
for payload in _read_jsonl(input_path, WorkspaceInventoryInput):
report = _operator_call(partial(service.inventory, payload, apply=apply))
_echo_json(report._asdict())
@knowledge_fs_control_space.command("register")
@click.option("--tenant-id", required=True)
@click.option("--owner-account-id", required=True)
@click.option("--provisioning-key", required=True)
@click.option("--knowledge-space-id", required=True)
@click.option("--knowledge-space-revision", type=click.IntRange(min=0), required=True)
def register(
tenant_id: str,
owner_account_id: str,
provisioning_key: str,
knowledge_space_id: str,
knowledge_space_revision: int,
) -> None:
control_space, replayed = _management_service().register(
KnowledgeFSControlSpaceRegistration(
tenant_id,
owner_account_id,
provisioning_key,
knowledge_space_id,
knowledge_space_revision,
)
)
click.echo(json.dumps({"control_space_id": control_space.id, "replayed": replayed}, sort_keys=True))
@knowledge_fs_control_space.command("backfill")
@click.option("--input", "input_path", type=click.Path(path_type=Path, exists=True, dir_okay=False), required=True)
@click.option("--apply", is_flag=True, default=False, help="Persist registrations; omitted means dry-run.")
def backfill(input_path: Path, apply: bool) -> None:
"""Backfill strict Workspace inventory JSONL; dry-run unless --apply is explicit."""
service = _cutover_service()
for payload in _read_jsonl(input_path, WorkspaceInventoryInput):
report = _operator_call(partial(service.backfill, payload, apply=apply))
_echo_json(report._asdict())
@knowledge_fs_control_space.command("quarantine-resolve")
@click.option("--input", "input_path", type=click.Path(path_type=Path, exists=True, dir_okay=False), required=True)
@click.option("--apply", is_flag=True, default=False, help="Persist resolutions; omitted means dry-run.")
def quarantine_resolve(input_path: Path, apply: bool) -> None:
"""Resolve strict tenant-scoped quarantine JSONL with immutable operator evidence."""
service = _cutover_service()
for payload in _read_jsonl(input_path, QuarantineResolutionInput):
report = _operator_call(partial(service.resolve_quarantine, payload, apply=apply))
_echo_json(report._asdict())
@knowledge_fs_control_space.command("shadow-start")
@click.option("--tenant-id", required=True)
@click.option("--expected-cas-version", type=click.IntRange(min=0), required=True)
@click.option("--at", "started_at", default=None, help="Optional explicit timezone-aware shadow start.")
def shadow_start(tenant_id: str, expected_cas_version: int, started_at: str | None) -> None:
service = _cutover_service()
_operator_call(
lambda: service.begin_shadow(
tenant_id=tenant_id,
expected_cas_version=expected_cas_version,
started_at=_parse_timestamp(started_at) if started_at is not None else None,
)
)
_echo_json(service.status(tenant_id=tenant_id))
@knowledge_fs_control_space.command("shadow-report")
@click.option("--input", "input_path", type=click.Path(path_type=Path, exists=True, dir_okay=False), required=True)
@click.option("--apply", is_flag=True, default=False, help="Persist observations; omitted means dry-run.")
def shadow_report(input_path: Path, apply: bool) -> None:
observations = _read_jsonl(input_path, ShadowAuthorizationObservationInput)
report = _operator_call(lambda: _cutover_service().record_shadow_report(observations, apply=apply))
_echo_json(report._asdict())
@knowledge_fs_control_space.command("shadow-complete")
@click.option("--input", "input_path", type=click.Path(path_type=Path, exists=True, dir_okay=False), required=True)
@click.option("--apply", is_flag=True, default=False, help="Persist completion; omitted means dry-run.")
def shadow_complete(input_path: Path, apply: bool) -> None:
payload = _read_one_jsonl(input_path, ShadowCompletionInput)
report = _operator_call(lambda: _cutover_service().complete_shadow(payload, apply=apply))
_echo_json(report._asdict())
@knowledge_fs_control_space.command("issue-approve")
@click.option("--tenant-id", required=True)
@click.option("--issue-key", required=True)
@click.option("--account-id", required=True)
@click.option("--at", "approved_at", required=True)
def issue_approve(tenant_id: str, issue_key: str, account_id: str, approved_at: str) -> None:
_operator_call(
lambda: _cutover_service().approve_issue_fail_closed(
tenant_id=tenant_id,
issue_key=issue_key,
account_id=account_id,
approved_at=_parse_timestamp(approved_at),
)
)
_echo_json(_cutover_service().status(tenant_id=tenant_id))
@knowledge_fs_control_space.command("issue-resolve")
@click.option("--tenant-id", required=True)
@click.option("--issue-key", required=True)
@click.option("--account-id", required=True)
@click.option("--at", "resolved_at", required=True)
def issue_resolve(tenant_id: str, issue_key: str, account_id: str, resolved_at: str) -> None:
_operator_call(
lambda: _cutover_service().resolve_issue(
tenant_id=tenant_id,
issue_key=issue_key,
account_id=account_id,
resolved_at=_parse_timestamp(resolved_at),
)
)
_echo_json(_cutover_service().status(tenant_id=tenant_id))
@knowledge_fs_control_space.command("shadow-approve")
@click.option("--tenant-id", required=True)
@click.option("--diff-key", required=True)
@click.option("--account-id", required=True)
@click.option("--at", "approved_at", required=True)
def shadow_approve(tenant_id: str, diff_key: str, account_id: str, approved_at: str) -> None:
_operator_call(
lambda: _cutover_service().approve_shadow_diff(
tenant_id=tenant_id,
diff_key=diff_key,
account_id=account_id,
approved_at=_parse_timestamp(approved_at),
)
)
_echo_json(_cutover_service().status(tenant_id=tenant_id))
@knowledge_fs_control_space.command("shadow-resolve")
@click.option("--tenant-id", required=True)
@click.option("--diff-key", required=True)
@click.option("--account-id", required=True)
@click.option("--at", "resolved_at", required=True)
def shadow_resolve(tenant_id: str, diff_key: str, account_id: str, resolved_at: str) -> None:
_operator_call(
lambda: _cutover_service().resolve_shadow_diff(
tenant_id=tenant_id,
diff_key=diff_key,
account_id=account_id,
resolved_at=_parse_timestamp(resolved_at),
)
)
_echo_json(_cutover_service().status(tenant_id=tenant_id))
@knowledge_fs_control_space.command("legacy-dashboard")
@click.option("--tenant-id", required=True)
@click.option("--input", "input_path", type=click.Path(path_type=Path, exists=True, dir_okay=False), required=True)
@click.option("--checked-at", required=True)
@click.option("--expected-cas-version", type=click.IntRange(min=0), default=None)
@click.option("--apply", is_flag=True, default=False, help="Persist gate evidence; omitted means read-only report.")
def legacy_dashboard(
tenant_id: str,
input_path: Path,
checked_at: str,
expected_cas_version: int | None,
apply: bool,
) -> None:
dependencies = _read_jsonl(input_path, LegacyDependencyInput, allow_empty=True)
report = _operator_call(
lambda: _cutover_service().legacy_dependency_dashboard(
tenant_id=tenant_id,
dependencies=dependencies,
expected_cas_version=expected_cas_version,
checked_at=_parse_timestamp(checked_at),
apply=apply,
)
)
_echo_json(report._asdict())
@knowledge_fs_control_space.command("legacy-check")
@click.option("--tenant-id", required=True)
def legacy_check(tenant_id: str) -> None:
status_report = _operator_call(lambda: _cutover_service().status(tenant_id=tenant_id))
passed = (
bool(status_report["legacy_dependency_ready"])
and status_report["open_issues"] == 0
and status_report["unresolved_cutover_quarantine"] == 0
)
_echo_json({"tenant_id": tenant_id, "passed": passed, "status": status_report})
if not passed:
raise click.exceptions.Exit(1)
@knowledge_fs_control_space.command("freeze")
@click.option("--tenant-id", required=True)
@click.option("--expected-cas-version", type=click.IntRange(min=0), required=True)
@click.option("--at", "freeze_at", required=True)
def freeze(tenant_id: str, expected_cas_version: int, freeze_at: str) -> None:
service = _cutover_service()
_operator_call(
lambda: service.freeze(
tenant_id=tenant_id,
expected_cas_version=expected_cas_version,
freeze_at=_parse_timestamp(freeze_at),
)
)
_echo_json(service.status(tenant_id=tenant_id))
@knowledge_fs_control_space.command("final-delta")
@click.option("--input", "input_path", type=click.Path(path_type=Path, exists=True, dir_okay=False), required=True)
def final_delta(input_path: Path) -> None:
payload = _read_one_jsonl(input_path, FinalDeltaInput)
service = _cutover_service()
_operator_call(lambda: service.apply_final_delta(payload))
_echo_json(service.status(tenant_id=str(payload.tenant_id)))
@knowledge_fs_control_space.command("cutover")
@click.option("--tenant-id", required=True)
@click.option("--expected-cas-version", type=click.IntRange(min=0), required=True)
@click.option("--at", "cutover_at", required=True)
@click.option("--rollback-cutoff-at", required=True)
def cutover(tenant_id: str, expected_cas_version: int, cutover_at: str, rollback_cutoff_at: str) -> None:
service = _cutover_service()
_operator_call(
lambda: service.cutover(
tenant_id=tenant_id,
expected_cas_version=expected_cas_version,
cutover_at=_parse_timestamp(cutover_at),
rollback_cutoff_at=_parse_timestamp(rollback_cutoff_at),
)
)
_echo_json(service.status(tenant_id=tenant_id))
@knowledge_fs_control_space.command("smoke")
@click.option("--tenant-id", required=True)
@click.option("--expected-cas-version", type=click.IntRange(min=0), required=True)
@click.option("--input", "input_path", type=click.Path(path_type=Path, exists=True, dir_okay=False), required=True)
def smoke(tenant_id: str, expected_cas_version: int, input_path: Path) -> None:
results = _read_one_jsonl(input_path, CutoverSmokeResultsInput)
service = _cutover_service()
_operator_call(
lambda: service.record_smoke_results(
tenant_id=tenant_id,
expected_cas_version=expected_cas_version,
results=results,
)
)
_echo_json(service.status(tenant_id=tenant_id))
@knowledge_fs_control_space.command("observe")
@click.option("--tenant-id", required=True)
@click.option("--expected-cas-version", type=click.IntRange(min=0), required=True)
@click.option("--started-at", default=None)
@click.option("--window-ends-at", default=None)
@click.option("--maximum-task-expires-at", default=None)
@click.option("--observed-at", default=None)
def observe(
tenant_id: str,
expected_cas_version: int,
started_at: str | None,
window_ends_at: str | None,
maximum_task_expires_at: str | None,
observed_at: str | None,
) -> None:
service = _cutover_service()
if observed_at is not None:
if any(value is not None for value in (started_at, window_ends_at, maximum_task_expires_at)):
raise click.UsageError("--observed-at cannot be combined with observation start options")
_operator_call(
lambda: service.complete_observation(
tenant_id=tenant_id,
expected_cas_version=expected_cas_version,
observed_at=_parse_timestamp(observed_at),
)
)
else:
if started_at is None or window_ends_at is None or maximum_task_expires_at is None:
raise click.UsageError(
"observation start requires --started-at, --window-ends-at, and --maximum-task-expires-at"
)
_operator_call(
lambda: service.begin_observation(
tenant_id=tenant_id,
expected_cas_version=expected_cas_version,
started_at=_parse_timestamp(started_at),
window_ends_at=_parse_timestamp(window_ends_at),
maximum_task_expires_at=_parse_timestamp(maximum_task_expires_at),
)
)
_echo_json(service.status(tenant_id=tenant_id))
@knowledge_fs_control_space.command("rollback")
@click.option("--tenant-id", required=True)
@click.option("--expected-cas-version", type=click.IntRange(min=0), required=True)
@click.option("--at", "rolled_back_at", required=True)
def rollback(tenant_id: str, expected_cas_version: int, rolled_back_at: str) -> None:
service = _cutover_service()
_operator_call(
lambda: service.rollback(
tenant_id=tenant_id,
expected_cas_version=expected_cas_version,
rolled_back_at=_parse_timestamp(rolled_back_at),
)
)
_echo_json(service.status(tenant_id=tenant_id))
@knowledge_fs_control_space.command("status")
@click.option("--tenant-id", required=True)
def status(tenant_id: str) -> None:
_echo_json(_operator_call(lambda: _cutover_service().status(tenant_id=tenant_id)))
@knowledge_fs_control_space.command("greenfield-initialize")
@click.option("--tenant-id", required=True)
def greenfield_initialize(tenant_id: str) -> None:
"""Idempotently initialize one Workspace that has no KnowledgeFS state."""
_operator_call(lambda: _greenfield_initializer().ensure_initialized(tenant_id=tenant_id))
_echo_json(_operator_call(lambda: _cutover_service().status(tenant_id=tenant_id)))
@knowledge_fs_control_space.command("cleanup-request")
@click.option("--input", "input_path", type=click.Path(path_type=Path, exists=True, dir_okay=False), required=True)
@click.option("--apply", is_flag=True, default=False, help="Persist readiness evidence; omitted means dry-run.")
def cleanup_request(input_path: Path, apply: bool) -> None:
payload = _read_one_jsonl(input_path, CleanupReadinessEvidenceInput)
report = _cleanup_call(lambda: _cleanup_service().request_cleanup(payload, apply=apply))
_echo_json(report._asdict())
@knowledge_fs_control_space.command("cleanup-approve")
@click.option("--input", "input_path", type=click.Path(path_type=Path, exists=True, dir_okay=False), required=True)
@click.option("--apply", is_flag=True, default=False, help="Persist four-eyes approval; omitted means dry-run.")
def cleanup_approve(input_path: Path, apply: bool) -> None:
payload = _read_one_jsonl(input_path, CleanupApprovalInput)
report = _cleanup_call(lambda: _cleanup_service().approve_cleanup(payload, apply=apply))
_echo_json(report._asdict())
@knowledge_fs_control_space.command("cleanup-start")
@click.option("--input", "input_path", type=click.Path(path_type=Path, exists=True, dir_okay=False), required=True)
@click.option("--apply", is_flag=True, default=False, help="Persist the irreversible fence; never runs deletion.")
@click.option(
"--acknowledge-irreversible",
is_flag=True,
default=False,
help="Required with --apply; confirms rollback will be permanently closed.",
)
def cleanup_start(input_path: Path, apply: bool, acknowledge_irreversible: bool) -> None:
payload = _read_one_jsonl(input_path, CleanupStartInput)
if apply and not acknowledge_irreversible:
raise click.UsageError("--apply requires --acknowledge-irreversible")
report = _cleanup_call(lambda: _cleanup_service().start_cleanup(payload, apply=apply))
_echo_json(report._asdict())
@knowledge_fs_control_space.command("cleanup-complete")
@click.option("--input", "input_path", type=click.Path(path_type=Path, exists=True, dir_okay=False), required=True)
@click.option("--apply", is_flag=True, default=False, help="Persist externally verified cleanup completion.")
@click.option(
"--acknowledge-executed",
is_flag=True,
default=False,
help="Required with --apply; confirms the reviewed destructive bundle already executed.",
)
def cleanup_complete(input_path: Path, apply: bool, acknowledge_executed: bool) -> None:
payload = _read_one_jsonl(input_path, CleanupCompletionEvidenceInput)
if apply and not acknowledge_executed:
raise click.UsageError("--apply requires --acknowledge-executed")
report = _cleanup_call(lambda: _cleanup_service().complete_cleanup(payload, apply=apply))
_echo_json(report._asdict())
@knowledge_fs_control_space.command("cleanup-status")
@click.option("--tenant-id", required=True)
@click.option("--request-id", required=True)
def cleanup_status(tenant_id: str, request_id: str) -> None:
_echo_json(_cleanup_call(lambda: _cleanup_service().status(tenant_id=tenant_id, request_id=request_id)))
@knowledge_fs_control_space.command("repair")
@click.option("--tenant-id", required=True)
@click.option("--control-space-id", required=True)
@click.option("--expected-resource-version", type=click.IntRange(min=0), required=True)
@click.option("--knowledge-space-id", required=True)
@click.option("--knowledge-space-revision", type=click.IntRange(min=0), required=True)
def repair(
tenant_id: str,
control_space_id: str,
expected_resource_version: int,
knowledge_space_id: str,
knowledge_space_revision: int,
) -> None:
control_space = _management_service().repair_registration(
tenant_id=tenant_id,
control_space_id=control_space_id,
expected_resource_version=expected_resource_version,
knowledge_space_id=knowledge_space_id,
knowledge_space_revision=knowledge_space_revision,
)
click.echo(json.dumps({"control_space_id": control_space.id, "state": control_space.state.value}, sort_keys=True))
@knowledge_fs_control_space.command("orphan-report")
@click.option("--limit", type=click.IntRange(min=1, max=10_000), default=500, show_default=True)
def orphan_report(limit: int) -> None:
report = KnowledgeFSOrphanReconciler(
session_factory.get_session_maker(),
get_knowledge_fs_lifecycle_remote(),
).reconcile(limit=limit, apply_repairs=False)
click.echo(json.dumps(report._asdict(), sort_keys=True))
@knowledge_fs_control_space.command("workspace-delete-request")
@click.option("--tenant-id", required=True)
@click.option("--apply", is_flag=True, default=False, help="Persist durable deletion intents; omitted means dry-run.")
def workspace_delete_request(tenant_id: str, apply: bool) -> None:
"""Route every KnowledgeFS Space through the canonical lifecycle deletion path."""
if not apply:
report = _management_service().dry_run(tenant_id=tenant_id)
_echo_json(
{
"apply": False,
"by_state": report.by_state,
"tenant_id": tenant_id,
"total": report.total,
}
)
return
results = _lifecycle_call(lambda: _lifecycle_service().request_workspace_cleanup(tenant_id=tenant_id))
_echo_json(
{
"apply": True,
"control_space_ids": [result.control_space.id for result in results],
"operation_ids": [result.outbox.operation_id for result in results if result.outbox is not None],
"tenant_id": tenant_id,
}
)
@knowledge_fs_control_space.command("workspace-delete-finalize")
@click.option("--tenant-id", required=True)
@click.option("--apply", is_flag=True, default=False, help="Purge terminal local control-plane rows.")
@click.option(
"--acknowledge-control-plane-purge",
is_flag=True,
default=False,
help="Required with --apply after every remote Space has reached deleted.",
)
def workspace_delete_finalize(tenant_id: str, apply: bool, acknowledge_control_plane_purge: bool) -> None:
"""Release the Workspace FK only after all remote deletions are terminal."""
service = _lifecycle_service()
if not apply:
_lifecycle_call(lambda: service.assert_workspace_deletion_allowed(tenant_id=tenant_id))
_echo_json({"apply": False, "ready": True, "tenant_id": tenant_id})
return
if not acknowledge_control_plane_purge:
raise click.UsageError("--apply requires --acknowledge-control-plane-purge")
deleted = _lifecycle_call(lambda: service.finalize_workspace_deletion(tenant_id=tenant_id))
_echo_json({"apply": True, "purged_control_spaces": deleted, "tenant_id": tenant_id})
def _read_jsonl[InputT: BaseModel](
input_path: Path, input_type: type[InputT], *, allow_empty: bool = False
) -> tuple[InputT, ...]:
records: list[InputT] = []
for line_number, line in enumerate(input_path.read_text(encoding="utf-8").splitlines(), start=1):
if not line.strip():
continue
try:
records.append(input_type.model_validate_json(line))
except ValidationError as exc:
raise click.ClickException(f"invalid strict JSONL at line {line_number}: {exc}") from exc
if not records and not allow_empty:
raise click.ClickException("strict JSONL input must contain at least one record")
return tuple(records)
def _read_one_jsonl[InputT: BaseModel](input_path: Path, input_type: type[InputT]) -> InputT:
records = _read_jsonl(input_path, input_type)
if len(records) != 1:
raise click.ClickException("this command requires exactly one JSONL record")
return records[0]
def _parse_timestamp(value: str) -> datetime:
try:
parsed = datetime.fromisoformat(value)
except ValueError as exc:
raise click.ClickException(f"invalid ISO-8601 timestamp: {value}") from exc
if parsed.tzinfo is None:
raise click.ClickException("operator timestamps must include an explicit timezone")
return parsed
def _operator_call[ResultT](operation: Callable[[], ResultT]) -> ResultT:
try:
return operation()
except KnowledgeFSCutoverError as exc:
raise click.ClickException(str(exc)) from exc
def _cleanup_call[ResultT](operation: Callable[[], ResultT]) -> ResultT:
try:
return operation()
except KnowledgeFSCleanupError as exc:
raise click.ClickException(str(exc)) from exc
def _lifecycle_call[ResultT](operation: Callable[[], ResultT]) -> ResultT:
try:
return operation()
except KnowledgeFSControlSpaceLifecycleError as exc:
raise click.ClickException(str(exc)) from exc
def _echo_json(payload: object) -> None:
click.echo(json.dumps(payload, default=str, sort_keys=True))
def _management_service() -> KnowledgeFSControlSpaceManagementService:
return KnowledgeFSControlSpaceManagementService(session_factory.get_session_maker())
def _cutover_service() -> KnowledgeFSWorkspaceCutoverService:
return KnowledgeFSWorkspaceCutoverService(
session_factory.get_session_maker(),
remote_factory=get_knowledge_fs_lifecycle_remote,
)
def _greenfield_initializer() -> KnowledgeFSWorkspaceGreenfieldInitializer:
return KnowledgeFSWorkspaceGreenfieldInitializer(
session_factory.get_session_maker(),
cutover=_cutover_service(),
)
def _cleanup_service() -> KnowledgeFSCleanupService:
return KnowledgeFSCleanupService(session_factory.get_session_maker())
def _lifecycle_service() -> KnowledgeFSControlSpaceCommandService:
return KnowledgeFSControlSpaceCommandService(session_factory.get_session_maker())
__all__ = ["knowledge_fs_control_space"]
+2 -1
View File
@@ -6,6 +6,7 @@ from sqlalchemy import delete, select, update
from sqlalchemy.orm import sessionmaker
from configs import dify_config
from enums.deployment_edition import DeploymentEdition
from events.app_event import app_was_created
from extensions.ext_database import db
from extensions.ext_redis import redis_client
@@ -42,7 +43,7 @@ def reset_encrypt_key_pair():
After the reset, all LLM credentials will become invalid, requiring re-entry.
Only support SELF_HOSTED mode.
"""
if dify_config.EDITION != "SELF_HOSTED":
if dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD:
click.echo(click.style("This command is only for SELF_HOSTED installations.", fg="red"))
return
with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
+19 -93
View File
@@ -1,72 +1,30 @@
"""Configuration for the optional KnowledgeFS control-plane integration."""
"""Configuration for the optional KnowledgeFS Console bridge."""
from ipaddress import ip_address
from urllib.parse import urlsplit
from pydantic import Field, PositiveFloat, PositiveInt, SecretStr, field_validator, model_validator
from pydantic import Field, PositiveFloat, SecretStr, field_validator, model_validator
from pydantic_settings import BaseSettings
class KnowledgeFSConfig(BaseSettings):
"""Server-only KnowledgeFS connection and rollout settings."""
"""Server-only settings for the KnowledgeFS production connection."""
KNOWLEDGE_FS_ENABLED: bool = Field(
default=False,
description="Enable the KnowledgeFS control-plane product routes.",
description="Enable the private KnowledgeFS Console bridge.",
)
KNOWLEDGE_FS_LIFECYCLE_WORKER_ENABLED: bool = Field(
default=False,
description="Enable delivery of durable KnowledgeFS lifecycle commands after every rollout gate is ready.",
)
KNOWLEDGE_FS_INTEGRATED_PROVISION_READY: bool = Field(
default=False,
description="Confirm that the Capability-v2 integrated provision route is deployed and verified.",
)
KNOWLEDGE_FS_LEGACY_ACL_FREEZE_READY: bool = Field(
default=False,
description="Confirm that legacy KFS ACL mutation is frozen for integrated mode.",
)
KNOWLEDGE_FS_LIFECYCLE_POLL_INTERVAL_SECONDS: PositiveInt = Field(default=15, le=300)
KNOWLEDGE_FS_LIFECYCLE_LEASE_SECONDS: PositiveInt = Field(default=60, le=600)
KNOWLEDGE_FS_LIFECYCLE_BATCH_SIZE: PositiveInt = Field(default=25, le=1_000)
KNOWLEDGE_FS_BASE_URL: str | None = Field(default=None, description="KnowledgeFS gateway base URL.")
KNOWLEDGE_FS_DIRECT_ORIGIN: str | None = Field(
KNOWLEDGE_FS_JWT_SECRET: SecretStr | None = Field(
default=None,
description="Public KnowledgeFS origin returned with direct upload capabilities.",
min_length=32,
description="Shared secret used to sign short-lived KnowledgeFS service JWTs.",
)
KNOWLEDGE_FS_DIRECT_UPLOAD_READY: bool = Field(
default=False,
description="Confirm that KnowledgeFS direct upload and its browser origin policy are deployed and verified.",
)
KNOWLEDGE_FS_CAPABILITY_V2_ENABLED: bool = Field(
default=False,
description="Prepare resource-scoped Capability v2 issuance; disabled until rollout approval.",
)
KNOWLEDGE_FS_CAPABILITY_V2_SIGNING_KID: str | None = Field(
default=None,
description="Identifier for the current asymmetric Capability v2 signing key.",
)
KNOWLEDGE_FS_CAPABILITY_V2_PRIVATE_KEY_PEM: SecretStr | None = Field(
default=None,
description="Server-only PEM for the current Capability v2 RSA signing key.",
)
KNOWLEDGE_FS_CAPABILITY_V2_PREVIOUS_PUBLIC_JWKS: str | None = Field(
default=None,
description="Optional public-only JWKS JSON retained during key rotation overlap.",
)
KNOWLEDGE_FS_CAPABILITY_V2_ISSUER: str = Field(default="dify-control-plane", min_length=1)
KNOWLEDGE_FS_CAPABILITY_V2_AUDIENCE: str = Field(default="knowledge-fs", min_length=1)
KNOWLEDGE_FS_CAPABILITY_V2_MAX_TTL_SECONDS: PositiveInt = Field(default=60, le=60)
KNOWLEDGE_FS_JWKS_CACHE_MAX_AGE_SECONDS: PositiveInt = Field(default=300, le=86_400)
KNOWLEDGE_FS_PRODUCT_MAX_RESPONSE_BYTES: PositiveInt = Field(default=4 * 1024 * 1024, le=16 * 1024 * 1024)
KNOWLEDGE_FS_SSE_READ_TIMEOUT_SECONDS: PositiveFloat = Field(default=300.0, le=3600.0, allow_inf_nan=False)
KNOWLEDGE_FS_TIMEOUT_SECONDS: PositiveFloat = Field(default=10.0, le=60.0, allow_inf_nan=False)
@field_validator(
"KNOWLEDGE_FS_BASE_URL",
"KNOWLEDGE_FS_DIRECT_ORIGIN",
"KNOWLEDGE_FS_CAPABILITY_V2_SIGNING_KID",
"KNOWLEDGE_FS_CAPABILITY_V2_PRIVATE_KEY_PEM",
"KNOWLEDGE_FS_CAPABILITY_V2_PREVIOUS_PUBLIC_JWKS",
"KNOWLEDGE_FS_JWT_SECRET",
mode="before",
)
@classmethod
@@ -82,57 +40,25 @@ class KnowledgeFSConfig(BaseSettings):
@field_validator("KNOWLEDGE_FS_BASE_URL")
@classmethod
def validate_base_url(cls, value: str | None) -> str | None:
return cls._validate_origin(value, name="KNOWLEDGE_FS_BASE_URL")
@field_validator("KNOWLEDGE_FS_DIRECT_ORIGIN")
@classmethod
def validate_direct_origin(cls, value: str | None) -> str | None:
return cls._validate_origin(value, name="KNOWLEDGE_FS_DIRECT_ORIGIN")
@classmethod
def _validate_origin(cls, value: str | None, *, name: str) -> str | None:
if value is None:
return None
parsed = urlsplit(value)
if parsed.scheme not in {"http", "https"} or not parsed.netloc:
raise ValueError(f"{name} must be an absolute HTTP(S) URL")
raise ValueError("KNOWLEDGE_FS_BASE_URL must be an absolute HTTP(S) URL")
try:
_ = parsed.port
except ValueError as exc:
raise ValueError(f"{name} must include a valid port") from exc
if parsed.username or parsed.password or parsed.query or parsed.fragment or parsed.path not in {"", "/"}:
raise ValueError(f"{name} must be an origin without credentials, path, query, or fragment")
raise ValueError("KNOWLEDGE_FS_BASE_URL must include a valid port") from exc
if parsed.username or parsed.password or parsed.query or parsed.fragment:
raise ValueError("KNOWLEDGE_FS_BASE_URL must not include credentials, query, or fragment")
return value.rstrip("/")
@model_validator(mode="after")
def validate_enabled_connection(self) -> "KnowledgeFSConfig":
if str(getattr(self, "DEPLOY_ENV", "")).strip().upper() == "PRODUCTION":
for name, value in (
("KNOWLEDGE_FS_BASE_URL", self.KNOWLEDGE_FS_BASE_URL),
("KNOWLEDGE_FS_DIRECT_ORIGIN", self.KNOWLEDGE_FS_DIRECT_ORIGIN),
):
if value and not self._is_secure_or_loopback_origin(value):
raise ValueError(f"{name} must use HTTPS in production unless it targets loopback")
if self.KNOWLEDGE_FS_ENABLED:
if not self.KNOWLEDGE_FS_BASE_URL:
raise ValueError("KnowledgeFS base URL is required when the integration is enabled")
if not self.KNOWLEDGE_FS_CAPABILITY_V2_ENABLED:
raise ValueError("KnowledgeFS product routes require Capability v2 when enabled")
if self.KNOWLEDGE_FS_CAPABILITY_V2_ENABLED and not (
self.KNOWLEDGE_FS_CAPABILITY_V2_SIGNING_KID and self.KNOWLEDGE_FS_CAPABILITY_V2_PRIVATE_KEY_PEM
):
raise ValueError("Capability v2 signing kid and private key are required when issuance is enabled")
if not self.KNOWLEDGE_FS_ENABLED:
return self
if bool(self.KNOWLEDGE_FS_BASE_URL) != bool(self.KNOWLEDGE_FS_JWT_SECRET):
raise ValueError("KNOWLEDGE_FS_BASE_URL and KNOWLEDGE_FS_JWT_SECRET must be configured together")
if not self.KNOWLEDGE_FS_BASE_URL:
raise ValueError("KnowledgeFS connection settings are required when the integration is enabled")
return self
@staticmethod
def _is_secure_or_loopback_origin(value: str) -> bool:
parsed = urlsplit(value)
if parsed.scheme == "https":
return True
hostname = (parsed.hostname or "").rstrip(".").lower()
if hostname == "localhost":
return True
try:
return ip_address(hostname).is_loopback
except ValueError:
return False
-6
View File
@@ -266,12 +266,6 @@ class PluginConfig(BaseSettings):
default=60 * 60,
)
PLUGIN_MODEL_PROVIDERS_CACHE_ENABLED: bool = Field(
description="Whether tenant plugin model providers are cached in Redis. Disable when plugins are installed "
"by a system other than this one, which cannot invalidate the cache when a tenant's plugins change.",
default=True,
)
PLUGIN_MODEL_PROVIDERS_CACHE_TTL: PositiveInt = Field(
description="TTL in seconds for caching tenant plugin model providers in Redis",
default=60 * 60 * 24,
-2
View File
@@ -77,5 +77,3 @@ COOKIE_NAME_PASSPORT = "passport"
HEADER_NAME_CSRF_TOKEN = "X-CSRF-Token"
HEADER_NAME_APP_CODE = "X-App-Code"
HEADER_NAME_PASSPORT = "X-App-Passport"
HEADER_NAME_IDEMPOTENCY_KEY = "Idempotency-Key"
HEADER_NAME_REQUEST_ID = "X-Request-ID"
+2
View File
@@ -25,6 +25,8 @@ HUMAN_INPUT_FORM_INPUT_EXAMPLE = {
class HumanInputFormSubmitPayload(BaseModel):
"""Legacy Human Input v1 submit payload shared by existing runtime surfaces."""
inputs: dict[str, JsonValue] = Field(
description=(
"Submitted human input values keyed by output variable name. "
@@ -0,0 +1,984 @@
"""Shared Human Input v2 transport contracts.
Request DTOs use normal Pydantic coercion and forbid unknown fields. Migration
input is the sole compatibility exception: it ignores unknown legacy fields,
defaults a missing version to ``"1"``, rejects any other explicit version, and
rejects duplicate node IDs. Its transport shape mirrors the frontend migration
adapter so the generated client can replace the temporary mock without changing
frontend orchestration.
Public v2, trusted Service API v2, and legacy v1 submit DTOs stay independent.
"""
from __future__ import annotations
from http import HTTPStatus
from typing import Annotated, Literal, Self, Union
from pydantic import BaseModel, ConfigDict, Discriminator, Field, JsonValue, field_validator, model_validator
from core.human_input_v2.entities import (
ContactId,
EmailProviderType,
HumanInputContactType,
IMBindingId,
IMBindingScope,
IMIdentityBindingStatus,
IMIdentityId,
IMIntegrationStatus,
IMProvider,
IMSyncRemovalReason,
IMSyncResultType,
IMSyncRunId,
IMSyncRunStatus,
OrganizationCandidateId,
)
from core.workflow.nodes.human_input.entities import FormInputConfig, UserActionConfig
from core.workflow.nodes.human_input.entities import HumanInputNodeDataFull as HITLv1NodeData
from core.workflow.nodes.human_input_v2.entities import Channel
from core.workflow.nodes.human_input_v2.entities import HumanInputNodeData as HITLv2NodeData
from fields.base import ResponseModel
from fields.pagination import PaginationParamsMixin, PaginationResultMixin
from fields.timestamp import Timestamp
from libs.helper import EmailStr
class _NoExtraModel(BaseModel):
model_config = ConfigDict(extra="forbid")
class _RequestModel(BaseModel):
"""Base request model that forbids unknown fields while accepting JSON-native values."""
model_config = ConfigDict(extra="forbid")
class _MigrationInputModel(BaseModel):
"""Forward-compatible migration input that ignores fields unknown to this backend version."""
model_config = ConfigDict(extra="ignore")
class ContactListQuery(PaginationParamsMixin, _NoExtraModel):
"""Query params for listing contacts in the workspace directory."""
group: HumanInputContactType | None = Field(
default=None,
description="Optional contact type filter. None means all contacts.",
)
keyword: str | None = Field(default=None, description="Free-text search against contact name or email.")
class ContactOptionsQuery(PaginationParamsMixin, _NoExtraModel):
"""Query params for selecting contacts in workflow editors."""
keyword: str | None = Field(default=None, description="Free-text search against selectable contact names.")
class OrganizationCandidatesQuery(PaginationParamsMixin, _NoExtraModel):
"""Query params for searching organization member candidates."""
keyword: str | None = Field(default=None, description="Free-text search against candidate name or email.")
ExternalContactName = Annotated[
str,
Field(
min_length=1,
max_length=255,
description="Display name shown in the contact directory.",
),
]
ExternalContactEmail = Annotated[
EmailStr,
Field(
description="Primary email used for delivery and identity verification.",
),
]
ExternalContactAvatar = Annotated[
str,
Field(
description=(
"Optional avatar file ID. Upload the avatar image first via "
"`POST /console/api/files/upload`, then use the returned file id here."
" Set to empty string for resetting to default avatar."
),
),
]
class ExternalContactCreateRequest(_RequestModel):
"""Request body for creating or updating one external contact."""
name: ExternalContactName
email: ExternalContactEmail
avatar: ExternalContactAvatar | None = None
class ExternalContactUpdateRequest(_RequestModel):
"""Request body for creating or updating one external contact."""
name: ExternalContactName | None = None
email: ExternalContactEmail | None = None
avatar: ExternalContactAvatar | None = None
class IMBinding(BaseModel):
id: IMBindingId = Field(description="Unique IM binding identifier.")
provider: IMProvider = Field(description="Provider of the IM binding.")
scope: IMBindingScope = Field(description="Scope of the IM binding.")
class HumanInputContactSummary(BaseModel):
"""A trimmed version of `HumanInputContact` that only includes the fields needed for workflow orchestration."""
id: ContactId = Field(description="Unique contact identifier.")
name: str = Field(description="Display name shown in the contact directory.")
avatar_url: str = Field(default="", description="URL of the contact's avatar.")
created_at: Timestamp = Field(description="Timestamp when the contact was created.")
class HumanInputContact(BaseModel):
"""One contact entity returned by contact-related APIs."""
id: ContactId = Field(description="Unique contact identifier.")
type: HumanInputContactType = Field(description="Resolved contact type in the current workspace scope.")
name: str = Field(description="Display name shown in the contact directory.")
email: str | None = Field(default=None, description="Primary contact email if one exists.")
avatar_url: str = Field(default="", description="URL of the contact's avatar.")
# the `im_bindings` field is always empty for EXTERNAL contacts
im_bindings: list[IMBinding] = Field(
default_factory=list[IMBinding],
description=(
"IM bindings that are bound to this contact. "
"Currently, only one IM binding is supported. "
"There is at most one IM binding per IM provider."
),
)
created_at: Timestamp = Field(description="Timestamp when the contact was created.")
class ContactOption(ResponseModel):
"""Least-privilege contact projection returned to workflow editors."""
id: ContactId = Field(description="Unique contact identifier persisted in workflow recipient configuration.")
type: HumanInputContactType = Field(description="Resolved contact type in the current workspace scope.")
name: str = Field(description="Display name shown in the contact picker.")
avatar_url: str | None = Field(default=None, description="Signed avatar URL if one is available.")
class ExternalContactCreateResponse(ResponseModel):
contact: HumanInputContact = Field(description="The created external contact.")
class ExternalContactUpdateResponse(ResponseModel):
contact: HumanInputContact = Field(description="The updated external contact. Fields are values after updating.")
class OrganizationCandidate(ResponseModel):
"""One organization member candidate that may become a platform contact."""
id: OrganizationCandidateId = Field(description="Organization candidate identifier.")
name: str = Field(description="Display name shown in the candidate list.")
email: str = Field(description="Primary organization email used for matching.")
avatar_url: str | None = Field(default=None, description="Signed avatar URL if one is available.")
class ListContactsResponse(PaginationResultMixin, ResponseModel):
"""Paginated response body for contact list APIs."""
data: list[HumanInputContact] = Field(description="Contacts returned for the current page.")
class ListContactOptionsResponse(PaginationResultMixin, ResponseModel):
"""Paginated editor-safe contact picker response."""
data: list[ContactOption] = Field(description="Selectable contacts returned for the current page.")
class GetContactResponse(ResponseModel):
"""Response body for one contact resolved in the current workspace scope."""
contact: HumanInputContact = Field(description="Contact resolved as workspace, platform, or external.")
class ListOrganizationCandidatesResponse(PaginationResultMixin, ResponseModel):
"""Paginated response body for organization candidate search."""
data: list[OrganizationCandidate] = Field(
description="Organization member candidates returned for the current page."
)
class AddPlatformContactsRequest(_RequestModel):
"""Request body for adding one or more organization members as platform contacts."""
candidate_ids: list[OrganizationCandidateId] = Field(
...,
min_length=1,
description="Organization candidate identifiers to project into the current workspace as platform contacts.",
)
class AddPlatformContactsResponse(ResponseModel):
"""Response body for adding platform contacts."""
data: list[HumanInputContact] = Field(description="Contacts created by the current add operation.")
class RemoveContactsRequest(_RequestModel):
"""Request body for batch-removing platform or external contacts."""
contact_ids: list[ContactId] = Field(
...,
min_length=1,
description="Contact identifiers selected for removal from the contact directory surface.",
)
class RemoveContactsResponse(ResponseModel):
"""Response body returned after batch-removing contacts."""
removed_contact_ids: list[ContactId] = Field(description="Contact identifiers removed by the current operation.")
class _FeishuLarkIMIntegrationCredentialsBase(_RequestModel):
"""Shared credential fields for Feishu and Lark integrations."""
app_id: str = Field(description="Feishu or Lark application identifier.")
app_secret: str | PreserveOriginalValue = Field(description="Feishu or Lark application secret.")
verification_token: str | PreserveOriginalValue | None = Field(
default=None, description="Optional callback verification token."
)
encrypt_key: str | PreserveOriginalValue | None = Field(default=None, description="Optional callback encrypt key.")
class FeishuIMIntegrationCredentials(_FeishuLarkIMIntegrationCredentialsBase):
"""Feishu integration credentials used by organization-level IM setup."""
provider: Literal[IMProvider.FEISHU] = Field(description="Discriminator for Feishu integration credentials.")
class LarkIMIntegrationCredentials(_FeishuLarkIMIntegrationCredentialsBase):
"""Lark integration credentials used by organization-level IM setup."""
provider: Literal[IMProvider.LARK] = Field(description="Discriminator for Lark integration credentials.")
class SlackIMIntegrationCredentials(_RequestModel):
"""Slack integration credentials used by organization-level IM setup."""
provider: Literal[IMProvider.SLACK] = Field(description="Discriminator for Slack integration credentials.")
client_id: str = Field(description="Slack OAuth client identifier.")
client_secret: str | PreserveOriginalValue = Field(description="Slack OAuth client secret.")
signing_secret: str | PreserveOriginalValue = Field(description="Slack signing secret used to verify callbacks.")
bot_token: str | PreserveOriginalValue = Field(
description="Slack bot token used for API calls and message delivery."
)
class DingTalkIMIntegrationCredentials(_RequestModel):
"""DingTalk integration credentials used by organization-level IM setup."""
provider: Literal[IMProvider.DING_TALK] = Field(description="Discriminator for DingTalk integration credentials.")
client_id: str = Field(description="DingTalk application client identifier.")
client_secret: str | PreserveOriginalValue = Field(
description="DingTalk application client secret. This field will be masked in response."
)
class MSTeamsIMIntegrationCredentials(_RequestModel):
"""Microsoft Teams integration credentials used by organization-level IM setup."""
provider: Literal[IMProvider.MS_TEAMS] = Field(
description="Discriminator for Microsoft Teams integration credentials."
)
tenant_id: str = Field(description="Microsoft Entra tenant identifier.")
client_id: str = Field(description="Microsoft Teams application client identifier.")
client_secret: str | PreserveOriginalValue = Field(
description="Microsoft Teams application client secret. This field will be masked in response"
)
class WeComIMIntegrationCredentials(_RequestModel):
"""WeCom integration credentials used by organization-level IM setup."""
provider: Literal[IMProvider.WE_COM] = Field(description="Discriminator for WeCom integration credentials.")
corp_id: str = Field(description="WeCom corporation identifier.")
agent_id: str = Field(description="WeCom agent identifier.")
secret: str | PreserveOriginalValue = Field(
description="WeCom application secret. This field will be masked in response"
)
IMIntegrationCredentials = Annotated[
FeishuIMIntegrationCredentials
| LarkIMIntegrationCredentials
| SlackIMIntegrationCredentials
| DingTalkIMIntegrationCredentials
| MSTeamsIMIntegrationCredentials
| WeComIMIntegrationCredentials,
Field(discriminator="provider"),
]
class _IMIntegrationRequest(_RequestModel):
"""Internal shared body for IM integration write/test operations."""
credentials: IMIntegrationCredentials = Field(description="Provider-specific IM integration credentials.")
class UpdateIMIntegrationRequest(_IMIntegrationRequest):
"""Request body for creating or updating one IM integration."""
expected_integration_id: str | None = Field(
default=None,
min_length=1,
description="Current integration identifier used with expected_config_version for compare-and-swap.",
)
expected_config_version: int | None = Field(
default=None,
ge=1,
description="Current integration revision used with expected_integration_id for compare-and-swap.",
)
@model_validator(mode="after")
def validate_complete_cas_token(self) -> Self:
has_integration_id = self.expected_integration_id is not None
has_config_version = self.expected_config_version is not None
if has_integration_id != has_config_version:
raise ValueError("expected_integration_id and expected_config_version must be provided together")
return self
class DeleteIMIntegrationQuery(_NoExtraModel):
"""CAS token required when deleting the current IM integration."""
expected_integration_id: str = Field(min_length=1, description="Current integration identifier.")
expected_config_version: int = Field(ge=1, description="Current integration revision.")
class TestIMIntegrationRequest(_IMIntegrationRequest):
"""Request body for testing one IM integration."""
class IMIntegration(ResponseModel):
"""One organization-level IM integration snapshot."""
provider: IMProvider | None = Field(
default=None,
description="Configured IM provider. None is allowed when the integration is not configured.",
)
status: IMIntegrationStatus = Field(description="Current integration connectivity state.")
callback_url: str | None = Field(
default=None,
description=(
"Callback URL expected by the provider. "
"None if the current deployment uses persistence connections for receive events."
),
)
permission_hint: str | None = Field(default=None, description="Operator-facing hint about permission issues.")
configured_at: Timestamp | None = Field(
default=None, description="Unix timestamp in milliseconds when the integration was created."
)
updated_at: Timestamp | None = Field(
default=None, description="Unix timestamp in milliseconds when the integration was last updated."
)
integration_id: str | None = Field(
default=None,
description="Stable integration identifier. None when no integration is configured.",
)
config_version: int | None = Field(
default=None,
ge=1,
description="Monotonic configuration revision. None when no integration is configured.",
)
class GetIMIntegrationResponse(ResponseModel):
"""Response body carrying one IM integration snapshot."""
integration: IMIntegration = Field(description="Current organization-level IM integration snapshot.")
class UpdateIMIntegrationResponse(ResponseModel):
"""Response body returned after updating one IM integration."""
integration: IMIntegration = Field(description="Saved organization-level IM integration snapshot.")
class TestIMIntegrationResponse(ResponseModel):
"""Response body returned by IM integration test APIs."""
status: IMIntegrationStatus = Field(description="Integration status mapped from the test result.")
message: str = Field(description="Human-readable explanation of the test result.")
class IMSyncRunResultCounts(ResponseModel):
"""Aggregate result counts for one IM sync run."""
added: int = Field(description="Number of entries newly matched and bound.")
not_matched: int = Field(description="Number of entries that could not be matched.")
failed: int = Field(description="Number of entries that failed to reconcile.")
removed: int = Field(description="Number of entries whose prior binding was removed.")
skipped: int = Field(description="Number of entries intentionally skipped.")
class IMSyncRun(ResponseModel):
"""One IM sync run snapshot.
The latest-only UI displays ``finished_at`` as the explicit sync time. The
transport contract intentionally does not expose a ``started_by`` actor.
"""
id: IMSyncRunId = Field(description="Unique sync run identifier.")
status: IMSyncRunStatus = Field(description="Current lifecycle state of the sync run.")
started_at: Timestamp | None = Field(
default=None, description="Unix timestamp in milliseconds when the sync run started."
)
finished_at: Timestamp | None = Field(
default=None,
description=(
"Unix timestamp in milliseconds when the sync run finished. "
"This is the sync time displayed by the latest-only UI and is None while the run is unfinished."
),
)
error_message: str | None = Field(
default=None,
description="Terminal error message. Present only when the sync run status is `failed`.",
)
result_counts: IMSyncRunResultCounts = Field(
description="Aggregate reconciliation counts for the current run snapshot.",
)
provider: IMProvider = Field(description="IM provider associated with the sync run.")
integration_id: str = Field(description="Integration identifier captured when the sync run was created.")
integration_config_version: int = Field(
ge=1,
description="Integration configuration revision captured when the sync run was created.",
)
class CreateIMSyncRunResponse(ResponseModel):
"""Response body returned after creating one sync run."""
run: IMSyncRun = Field(description="Newly created sync run snapshot.")
class IMDirectoryEntry(_RequestModel):
"""Normalized provider-side account observed during an IM sync run.
The entry is run-scoped input to identity and binding reconciliation. It does
not represent a stable Dify identity and must not be referenced by bindings
or runtime authorization. Sync results may retain a snapshot for display,
diagnostics, and audit; durable references use IMIdentity or IMBinding IDs.
"""
provider_user_id: str
display_name: str | None = None
email: str | None = None
class IMIdentitySnapshot(_RequestModel):
"""Last known persistent IM identity state retained by a sync result."""
identity_id: IMIdentityId
provider_user_id: str
display_name: str | None = None
email: str | None = None
class IMSyncResultAdded(BaseModel):
type: Literal[IMSyncResultType.ADDED] = IMSyncResultType.ADDED
contact: HumanInputContactSummary = Field(description="The contact that associated with this sync result.")
entry: IMDirectoryEntry = Field(description="Provider directory entry observed during the current sync run.")
class IMSyncResultRemoved(BaseModel):
type: Literal[IMSyncResultType.REMOVED] = IMSyncResultType.REMOVED
contact: HumanInputContactSummary = Field(description="The contact that associated with this sync result.")
last_known_identity: IMIdentitySnapshot = Field(
description="Last known persistent IM identity state before its binding was removed."
)
reason: IMSyncRemovalReason = Field(description="Reason the existing IM binding was removed.")
class IMSyncResultFailed(BaseModel):
type: Literal[IMSyncResultType.FAILED] = IMSyncResultType.FAILED
entry: IMDirectoryEntry | None = Field(
None, description="Provider directory entry observed before this reconciliation failure, if available."
)
reason: str = Field(description="Reason the binding failed to sync.")
class IMSyncResultSkipped(BaseModel):
type: Literal[IMSyncResultType.SKIPPED] = IMSyncResultType.SKIPPED
entry: IMDirectoryEntry | None = Field(
None, description="Provider directory entry observed before reconciliation was skipped, if available."
)
contact: HumanInputContactSummary = Field(description="The contact that associated with this sync result.")
class IMSyncResultNotMatched(BaseModel):
type: Literal[IMSyncResultType.NOT_MATCHED] = IMSyncResultType.NOT_MATCHED
entry: IMDirectoryEntry | None = Field(
None, description="Provider directory entry that could not be matched, if available."
)
IMSyncResult = Annotated[
Union[
IMSyncResultAdded,
IMSyncResultRemoved,
IMSyncResultFailed,
IMSyncResultNotMatched,
IMSyncResultSkipped,
],
Discriminator("type"),
]
class IMSyncResultItem(ResponseModel):
"""One paginated reconciliation result entry for the latest sync run."""
# The current implementation does not return IM binding status for other IM providers.
# According to the design, we should return IM binding status for all configured IM providers.
# However, this version allows only one configured IM provider, so this model excludes
# the IM binding status for other IM providers.
id: str = Field(description="Unique synchronization result identifier.")
result: IMSyncResult = Field(description="Result bucket this entry belongs to.")
class GetLatestIMSyncRunResponse(ResponseModel):
"""Response body for reading the latest IM sync run summary."""
run: IMSyncRun = Field(description="Latest sync run summary.")
class ListLatestIMSyncRunResultsQuery(PaginationParamsMixin, _NoExtraModel):
"""Query params for reading paginated latest-run results."""
result: IMSyncResultType = Field(
...,
description=(
"Required result bucket to paginate from the latest sync run. "
"There is no `all` bucket or unfiltered results mode."
),
)
class ListLatestIMSyncRunResultsResponse(PaginationResultMixin, ResponseModel):
"""Page-based latest-run results without cursor state or a repeated run summary."""
data: list[IMSyncResultItem] = Field(
description="Result entries returned with page, limit, and total metadata for the selected bucket."
)
class ListIMIdentitiesQuery(PaginationParamsMixin, _NoExtraModel):
"""Query params for searching synced IM identities."""
keyword: str | None = Field(
default=None,
description="Free-text search against identity display name, email, or provider user ID.",
)
class IMIdentity(ResponseModel):
"""One synced IM identity that may be bound or overridden."""
id: IMIdentityId = Field(description="Internal IM identity record identifier.")
provider: IMProvider = Field(description="IM provider that owns this identity.")
provider_user_id: str = Field(description="Provider-side user identifier.")
display_name: str | None = Field(default=None, description="Display name returned by the provider.")
email: str | None = Field(default=None, description="Email returned by the provider, if any.")
binding_status: IMIdentityBindingStatus = Field(
description="Whether this IM identity is currently bound to a contact."
)
class ListIMIdentitiesResponse(PaginationResultMixin, ResponseModel):
"""Paginated response body for synced IM identity search."""
data: list[IMIdentity] = Field(description="IM identities returned for the current page.")
class SetContactIMOverrideRequest(_RequestModel):
"""Request body for setting one workspace-scoped IM override."""
identity_id: IMIdentityId = Field(description="Synced IM identity identifier selected as the workspace override.")
class SetContactIMOverrideResponse(ResponseModel):
"""Response body returned after setting one contact IM override."""
contact: HumanInputContact = Field(description="Contact snapshot after the override is applied.")
class ResetContactIMOverrideResponse(ResponseModel):
"""Response body returned after resetting one contact IM override."""
contact: HumanInputContact = Field(description="Contact snapshot after the override is cleared.")
class CreateIMBindingRequest(_RequestModel):
"""Request body for setting one workspace-scoped IM override."""
identity_id: IMIdentityId = Field(description="Synced IM identity identifier selected as the workspace override.")
class CreateIMBindingResponse(ResponseModel):
"""Response body returned after binding one IM identity to the workspace."""
contact: HumanInputContact = Field(description="Contact snapshot after the IM identity is bound.")
class DeleteIMBindingQuery(_RequestModel):
binding_id: IMBindingId = Field(description="IM binding to unbind.")
class DeleteIMBindingResponse(ResponseModel):
pass
class MessageTemplateTestRequest(_RequestModel):
"""Request body for sending one message-template test notification."""
channel: Channel = Field(description="Target debug delivery channel used for the test send.")
inputs: dict[str, JsonValue] = Field(
default_factory=dict,
description="Variable values used when rendering the message template preview.",
)
class MessageTemplateTestResponse(ResponseModel):
"""Response body returned after one message-template test send."""
class FormAccessRequestResponse(ResponseModel):
"""Response body returned after creating one OTP challenge."""
expires_in_seconds: int = Field(description="Seconds until the current OTP challenge expires.")
resend_after_seconds: int = Field(description="Seconds until another OTP challenge may be requested.")
challenge_token: str = Field(description="The token used to complete the OTP challenge.")
class FormDefinitionResponse(ResponseModel):
"""Response body containing a resolved human-input form definition."""
form_content: str | None = Field(default=None, description="Rendered form body shown to the approver.")
inputs: list[FormInputConfig] = Field(default_factory=list, description="Resolved form input definitions.")
resolved_default_values: dict[str, str] = Field(
default_factory=dict,
description="Default values after variable resolution and stringification.",
)
user_actions: list[UserActionConfig] = Field(
default_factory=list,
description="Action buttons that can complete the form.",
)
expiration_time: int = Field(description="Unix timestamp when the current form expires.")
class ServiceFormQuery(_NoExtraModel):
"""Query params for reading one service-api human-input form."""
user: str = Field(min_length=1, description="End-user identifier used to scope the service API request.")
class BatchGetContactsQuery(_NoExtraModel):
contact_ids: list[ContactId] = Field(..., description="List of contact IDs to retrieve.")
class BatchGetContactsResponse(ResponseModel):
data: list[HumanInputContactSummary] = Field(..., description="List of retrieved human input contacts.")
class BatchGetContactOptionsQuery(_NoExtraModel):
contact_ids: list[ContactId] = Field(..., description="Contact IDs persisted in workflow recipient configuration.")
class BatchGetContactOptionsResponse(ResponseModel):
data: list[ContactOption] = Field(..., description="Selectable contacts resolved in request order.")
class HumanInputV2FormSubmitRequest(_RequestModel):
"""Public Human Input v2 submit payload, independent from the v1 form contract."""
inputs: dict[str, JsonValue] = Field(description="Submitted form values keyed by output variable name.")
action: str = Field(description="Identifier of the selected Human Input v2 action.")
challenge_token: str | None = Field(
default=None,
description="OTP challenge token returned by the Human Input v2 access-request endpoint.",
)
otp_code: str | None = Field(
default=None,
description="OTP code required when the current Human Input v2 approver uses email proof.",
)
@model_validator(mode="after")
def validate_complete_email_proof(self) -> Self:
has_challenge_token = self.challenge_token is not None
has_otp_code = self.otp_code is not None
if has_challenge_token != has_otp_code:
raise ValueError("challenge_token and otp_code must be provided together")
return self
class HumanInputV2ServiceFormSubmitRequest(_RequestModel):
"""Trusted Service API submit payload without public-web OTP proof fields."""
inputs: dict[str, JsonValue] = Field(description="Submitted form values keyed by output variable name.")
action: str = Field(description="Identifier of the selected Human Input v2 action.")
user: str = Field(min_length=1, description="End-user identifier scoped to the current app token.")
class FormUploadTokenResponse(ResponseModel):
"""Response body returned when issuing a Human Input v2 upload token."""
upload_token: str
expires_at: int
class FormSubmitResponse(ResponseModel):
"""Empty response body returned after a Human Input v2 form submission."""
# =================== Node migration related entities ===================
class LegacyHITLv1NodeData(HITLv1NodeData):
"""Legacy Human Input node data accepted by the v1-to-v2 migration helper.
Missing versions use the historical v1 default. Any explicit value other
than the string ``"1"`` is rejected before migration.
"""
model_config = ConfigDict(extra="ignore")
# Keep the mutable parent field type for static substitutability while
# preserving the literal transport schema and runtime validation.
version: str = Field(
default="1",
description=(
'Legacy Human Input node version. Missing values default to "1"; '
'any explicit value other than the string "1" is rejected.'
),
json_schema_extra={"const": "1"},
)
@field_validator("version")
@classmethod
def validate_version(cls, value: str) -> str:
if value != "1":
raise ValueError('version must be "1"')
return value
class NodeDataMigrationInput(_MigrationInputModel):
"""One legacy node submitted through the frontend migration adapter boundary."""
node_id: str = Field(
..., description="The identifier of node to migrate. Used to associate between request and response"
)
node_data: LegacyHITLv1NodeData = Field(..., description="The legacy Human Input node data to migrate.")
class NodeDataMigrationPayload(_MigrationInputModel):
"""Complete legacy-node batch submitted for one migration attempt."""
nodes: list[NodeDataMigrationInput] = Field(min_length=1)
@model_validator(mode="after")
def validate_unique_node_ids(self) -> Self:
node_ids = [node.node_id for node in self.nodes]
if len(node_ids) != len(set(node_ids)):
raise ValueError("node_id must be unique within one migration request")
return self
class NodeDataMigrationResult(ResponseModel):
"""One converted node returned with its frontend correlation identifier."""
node_id: str = Field(description="The identifier of the migrated node.")
node_data: HITLv2NodeData = Field(description="The complete converted Human Input v2 node data.")
class NodeDataMigrationResponse(ResponseModel):
"""Successful all-node conversion response."""
data: list[NodeDataMigrationResult]
NodeDataMigrationBlockerCode = Literal[
"unsupported-version",
"configured-disabled-method",
"unsupported-delivery-method",
"invalid-email-configuration",
"invalid-email",
"unresolved-member",
"conflicting-email-templates",
"missing-recipients",
]
class NodeDataMigrationBlocker(ResponseModel):
"""Stable node-scoped reason why the backend cannot produce lossless v2 data."""
node_id: str = Field(description="The identifier of the node that failed migration.")
node_title: str = Field(description="The node title used for actionable frontend feedback.")
code: NodeDataMigrationBlockerCode = Field(description="Machine-readable migration blocker code.")
method_id: str | None = Field(default=None, description="Legacy delivery method related to the blocker.")
value: str | None = Field(default=None, description="Safe legacy value related to the blocker.")
class NodeDataMigrationFailureResponse(ResponseModel):
"""Whole-batch failure response without partial converted node data."""
code: Literal["hitl_node_data_migration_failure"] = "hitl_node_data_migration_failure"
message: str = Field(..., description="overall error messages")
status: Literal[HTTPStatus.BAD_REQUEST] = HTTPStatus.BAD_REQUEST
blockers: list[NodeDataMigrationBlocker] = Field(
..., description="Node-scoped blockers that caused the whole batch to fail."
)
# =================== EmailProvider related entities ===================
class PreserveOriginalValue(_RequestModel):
tag: Literal["preserve_original_value"] = "preserve_original_value"
class ResendProviderUpdateConfig(_RequestModel):
type: Literal[EmailProviderType.RESEND] = EmailProviderType.RESEND
api_key: str | PreserveOriginalValue = Field(
...,
description=(
"Resend API key. "
"Setting this to `PreserveOriginalValue` while updating will preserve the previously set credential."
),
)
sender_email: str = Field(
..., description="The email address shown as the sender. Its domain must be verified in Resend."
)
sender_name: str = Field("", description="The sender's name displayed in the recipient's inbox.")
class ResendProviderConfigResponse(ResponseModel):
type: Literal[EmailProviderType.RESEND] = EmailProviderType.RESEND
api_key_configured: bool = Field(description="Whether a Resend API key has been configured.")
sender_email: str = Field(description="The email address shown as the sender.")
sender_name: str = Field("", description="The sender's name displayed in the recipient's inbox.")
EmailProviderUpdateConfig = ResendProviderUpdateConfig
EmailProviderConfigResponse = ResendProviderConfigResponse
class GetEmailProviderResponse(ResponseModel):
provider_config: EmailProviderConfigResponse | None = Field(
...,
description="The current email provider configuration. `None` if not set.",
)
class SetEmailProviderRequest(_RequestModel):
provider_config: EmailProviderUpdateConfig = Field(..., description="Email provider configuration update.")
class SetEmailProviderResponse(ResponseModel):
pass
class TestEmailProviderConfigRequest(_RequestModel):
pass
class TestEmailProviderConfigResponse(ResponseModel):
pass
__all__ = [
"AddPlatformContactsRequest",
"AddPlatformContactsResponse",
"BatchGetContactOptionsQuery",
"BatchGetContactOptionsResponse",
"ContactListQuery",
"ContactOption",
"ContactOptionsQuery",
"CreateIMSyncRunResponse",
"DeleteIMIntegrationQuery",
"DingTalkIMIntegrationCredentials",
"EmailProviderConfigResponse",
"EmailProviderType",
"EmailProviderUpdateConfig",
"ExternalContactCreateRequest",
"ExternalContactUpdateRequest",
"FeishuIMIntegrationCredentials",
"FormAccessRequestResponse",
"FormDefinitionResponse",
"FormSubmitResponse",
"FormUploadTokenResponse",
"GetContactResponse",
"GetEmailProviderResponse",
"GetIMIntegrationResponse",
"GetLatestIMSyncRunResponse",
"HumanInputContact",
"HumanInputContactType",
"HumanInputV2FormSubmitRequest",
"HumanInputV2ServiceFormSubmitRequest",
"IMIdentity",
"IMIdentityBindingStatus",
"IMIntegration",
"IMIntegrationCredentials",
"IMIntegrationStatus",
"IMProvider",
"IMSyncRemovalReason",
"IMSyncResultItem",
"IMSyncResultType",
"IMSyncRun",
"IMSyncRunResultCounts",
"IMSyncRunStatus",
"LarkIMIntegrationCredentials",
"ListContactOptionsResponse",
"ListContactsResponse",
"ListIMIdentitiesQuery",
"ListIMIdentitiesResponse",
"ListLatestIMSyncRunResultsQuery",
"ListLatestIMSyncRunResultsResponse",
"ListOrganizationCandidatesResponse",
"MSTeamsIMIntegrationCredentials",
"MessageTemplateTestRequest",
"MessageTemplateTestResponse",
"NodeDataMigrationFailureResponse",
"NodeDataMigrationPayload",
"NodeDataMigrationResponse",
"OrganizationCandidate",
"OrganizationCandidatesQuery",
"PreserveOriginalValue",
"RemoveContactsRequest",
"RemoveContactsResponse",
"ResendProviderConfigResponse",
"ResendProviderUpdateConfig",
"ResetContactIMOverrideResponse",
"ServiceFormQuery",
"SetContactIMOverrideRequest",
"SetContactIMOverrideResponse",
"SetEmailProviderRequest",
"SetEmailProviderResponse",
"SlackIMIntegrationCredentials",
"TestIMIntegrationRequest",
"TestIMIntegrationResponse",
"UpdateIMIntegrationRequest",
"UpdateIMIntegrationResponse",
"WeComIMIntegrationCredentials",
]
+20 -7
View File
@@ -10,6 +10,7 @@ from extensions.ext_database import db
from libs.login import current_account_with_tenant
from models.dataset import Dataset
from models.model import App
from services.agent.roster_service import AgentRosterService
from services.enterprise.rbac_service import RBACService
__all__ = ["RBACPermission", "RBACResourceScope", "enforce_rbac_access", "rbac_permission_required"]
@@ -51,7 +52,7 @@ def enforce_rbac_access(
check_resource_type = None if resource_type == RBACResourceScope.WORKSPACE else resource_type
resource_id = None
if resource_required and check_resource_type:
resource_id = _extract_resource_id(resource_type, path_args)
resource_id = _extract_resource_id(resource_type, tenant_id, path_args)
if _is_resource_owned_by_current_user(tenant_id, account_id, resource_type, resource_id):
return
allowed = RBACService.CheckAccess.check(
@@ -131,11 +132,14 @@ def _is_resource_owned_by_current_user(
return False
def _extract_resource_id(resource_type: RBACResourceScope, path_args: dict[str, object] | None = None) -> str:
def _extract_resource_id(
resource_type: RBACResourceScope, tenant_id: str, path_args: dict[str, object] | None = None
) -> str:
"""Extract the resource ID from matched path arguments.
Some legacy route classes use neutral names such as ``resource_id`` for
app/dataset resources, and Agent App routes use ``agent_id`` as the app id.
app/dataset resources, and Agent routes carry ``agent_id``, which is
resolved to the App backing that Agent.
Dataset endpoints behind a rag-pipeline route contain ``pipeline_id``
instead of ``dataset_id``. In that case we look up the associated
``Dataset`` row via ``Dataset.pipeline_id``.
@@ -146,10 +150,19 @@ def _extract_resource_id(resource_type: RBACResourceScope, path_args: dict[str,
matched_args = {**view_args, **(path_args or {})}
if resource_type == RBACResourceScope.APP:
app_id = matched_args.get("app_id") or matched_args.get("agent_id") or matched_args.get("resource_id")
if not app_id:
raise ValueError("Missing app_id in request path")
return str(app_id)
app_id = matched_args.get("app_id")
if app_id:
return str(app_id)
agent_id = matched_args.get("agent_id")
if agent_id:
authz_app_id = AgentRosterService(db.session).peek_authz_app_id(tenant_id=tenant_id, agent_id=str(agent_id))
return authz_app_id or str(agent_id)
resource_id = matched_args.get("resource_id")
if resource_id:
return str(resource_id)
raise ValueError("Missing app_id in request path")
if resource_type == RBACResourceScope.DATASET:
dataset_id = matched_args.get("dataset_id") or matched_args.get("resource_id")
+6 -2
View File
@@ -38,6 +38,7 @@ from . import (
feature,
human_input_form,
init_validate,
knowledge_fs_proxy,
notification,
onboarding,
ping,
@@ -75,6 +76,7 @@ from .app import (
workflow_app_log,
workflow_comment,
workflow_draft_variable,
workflow_human_input_v2,
workflow_node_output_inspector,
workflow_run,
workflow_statistic,
@@ -126,7 +128,6 @@ from .explore import (
saved_message,
trial,
)
from .knowledge_fs import resources as knowledge_fs_resources
from .snippets import snippet_workflow, snippet_workflow_draft_variable
from .socketio import workflow as socketio_workflow
@@ -138,6 +139,7 @@ from .workspace import (
account,
agent_providers,
endpoint,
human_input,
load_balancing_config,
members,
model_providers,
@@ -194,10 +196,11 @@ __all__ = [
"forgot_password",
"generator",
"hit_testing",
"human_input",
"human_input_form",
"init_validate",
"installed_app",
"knowledge_fs_resources",
"knowledge_fs_proxy",
"load_balancing_config",
"login",
"mcp_server",
@@ -241,6 +244,7 @@ __all__ = [
"workflow_app_log",
"workflow_comment",
"workflow_draft_variable",
"workflow_human_input_v2",
"workflow_node_output_inspector",
"workflow_run",
"workflow_run_archive",
@@ -230,6 +230,7 @@ class WorkflowAgentComposerSaveToRosterApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user_id
@with_current_tenant_id
@with_session
@@ -439,6 +440,7 @@ class SnippetAgentComposerSaveToRosterApi(Resource):
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user_id
@with_current_tenant_id
@with_session
@@ -478,6 +480,7 @@ class AgentComposerApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user_id
@with_current_tenant_id
@with_session
+36 -2
View File
@@ -534,9 +534,23 @@ def _parse_observability_time_range(start: str | None, end: str | None, account:
def _query_values(name: str, alias_name: str | None = None) -> list[str]:
values = request.args.getlist(name)
def _get_values(field_name: str) -> list[str]:
values = request.args.getlist(field_name)
indexed_values: list[tuple[int, list[str]]] = []
prefix = f"{field_name}["
for key in request.args:
if not key.startswith(prefix) or not key.endswith("]"):
continue
index = key[len(prefix) : -1]
if index.isdigit():
indexed_values.append((int(index), request.args.getlist(key)))
for _, items in sorted(indexed_values):
values.extend(items)
return values
values = _get_values(name)
if alias_name:
values.extend(request.args.getlist(alias_name))
values.extend(_get_values(alias_name))
return [value.strip() for value in values if value.strip()]
@@ -547,6 +561,7 @@ class AgentAppListApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -584,6 +599,7 @@ class AgentAppListApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -625,6 +641,7 @@ class AgentAppApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -650,6 +667,7 @@ class AgentAppApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session
def delete(self, session: Session, tenant_id: str, agent_id: UUID):
@@ -707,6 +725,7 @@ class AgentPublishApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -729,6 +748,7 @@ class AgentBuildDraftCheckoutApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -805,6 +825,7 @@ class AgentBuildDraftApplyApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -827,6 +848,7 @@ class AgentAppCopyApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -852,6 +874,7 @@ class AgentApiAccessApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
@@ -868,6 +891,7 @@ class AgentApiStatusApi(Resource):
@login_required
@is_admin_or_owner_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@with_current_tenant_id
@with_session
@@ -886,6 +910,7 @@ class AgentApiKeyListApi(BaseApiKeyListResource):
token_prefix = "app-"
@console_ns.response(200, "Agent service API keys", console_ns.models[ApiKeyList.__name__])
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID) -> dict[str, object]:
@@ -896,6 +921,7 @@ class AgentApiKeyListApi(BaseApiKeyListResource):
@console_ns.response(400, "Maximum keys exceeded")
@with_current_tenant_id
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@with_session
def post(self, session: Session, tenant_id: str, agent_id: UUID) -> tuple[dict[str, object], int]:
@@ -915,6 +941,7 @@ class AgentApiKeyApi(BaseApiKeyResource):
@console_ns.response(204, "Agent service API key deleted")
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@with_session
def delete(
@@ -960,6 +987,7 @@ class AgentLogsApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
@@ -998,6 +1026,7 @@ class AgentLogMessagesApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
@@ -1036,6 +1065,7 @@ class AgentLogSourcesApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
@@ -1056,6 +1086,7 @@ class AgentStatisticsSummaryApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
@@ -1081,6 +1112,7 @@ class AgentRosterVersionsApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
@@ -1096,6 +1128,7 @@ class AgentRosterVersionDetailApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID, version_id: UUID):
@@ -1116,6 +1149,7 @@ class AgentRosterVersionRestoreApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
+4
View File
@@ -12,6 +12,7 @@ from werkzeug.exceptions import Forbidden
from configs import dify_config
from controllers.common.schema import register_response_schema_models
from controllers.common.session import with_session
from controllers.console.app.wraps import agent_manage_required_for_agent_app
from fields.base import ResponseModel
from libs.helper import dump_response, to_timestamp
from libs.login import login_required
@@ -194,6 +195,7 @@ class AppApiKeyListResource(BaseApiKeyListResource):
@console_ns.doc(params={"resource_id": "App ID"})
@console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
@with_current_tenant_id
@agent_manage_required_for_agent_app
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
"""Get all API keys for an app"""
@@ -210,6 +212,7 @@ class AppApiKeyListResource(BaseApiKeyListResource):
@with_current_tenant_id
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
def post(self, session: Session, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
"""Create a new API key for an app"""
@@ -233,6 +236,7 @@ class AppApiKeyResource(BaseApiKeyResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
def delete(
self,
+25 -12
View File
@@ -9,7 +9,7 @@ from flask_restx import Resource
from pydantic import AliasChoices, BaseModel, Field, ValidationInfo, computed_field, field_validator, model_validator
from sqlalchemy import select
from sqlalchemy.orm import Session
from werkzeug.exceptions import BadRequest, NotFound
from werkzeug.exceptions import BadRequest, Forbidden, NotFound
from configs import dify_config
from controllers.common.app_access import resolve_app_access_filter
@@ -23,7 +23,7 @@ from controllers.common.schema import (
register_schema_models,
)
from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model, with_session
from controllers.console.app.wraps import agent_manage_required_for_agent_app, get_app_model, with_session
from controllers.console.workspace.models import LoadBalancingPayload
from controllers.console.wraps import (
RBACPermission,
@@ -76,6 +76,7 @@ from services.entities.knowledge_entities.knowledge_entities import (
WeightModel,
WeightVectorSetting,
)
from services.errors.account import NoPermissionError
from services.feature_service import FeatureService
from tasks.initialize_created_app_rbac_access_task import initialize_created_app_rbac_access_task
@@ -904,6 +905,7 @@ class AppApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def put(self, session: Session, app_model: App):
@@ -938,6 +940,7 @@ class AppApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_DELETE)
@agent_manage_required_for_agent_app
@with_session
@get_app_model
def delete(self, session: Session, app_model: App):
@@ -962,6 +965,7 @@ class AppCopyApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_CREATE_AND_MANAGEMENT)
@agent_manage_required_for_agent_app
@with_current_user
@with_current_tenant_id
@get_app_model(mode=None)
@@ -973,16 +977,19 @@ class AppCopyApi(Resource):
with Session(db.engine, expire_on_commit=False) as session:
import_service = AppDslService(session)
yaml_content = import_service.export_dsl(app_model=app_model, session=session, include_secret=True)
result = import_service.import_app(
account=current_user,
import_mode=ImportMode.YAML_CONTENT,
yaml_content=yaml_content,
name=args.name,
description=args.description,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
)
try:
result = import_service.import_app(
account=current_user,
import_mode=ImportMode.YAML_CONTENT,
yaml_content=yaml_content,
name=args.name,
description=args.description,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
)
except NoPermissionError as e:
raise Forbidden(str(e))
if result.status == ImportStatus.FAILED:
session.rollback()
return dump_response(AppImportResponse, result), 400
@@ -1036,6 +1043,7 @@ class AppExportApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_IMPORT_EXPORT_DSL)
@agent_manage_required_for_agent_app
@get_app_model
def get(self, app_model: App):
"""Export app"""
@@ -1060,6 +1068,7 @@ class AppPublishToCreatorsPlatformApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_IMPORT_EXPORT_DSL)
@agent_manage_required_for_agent_app
@with_current_user_id
@get_app_model(mode=None)
def post(self, current_user_id: str, app_model: App):
@@ -1090,6 +1099,7 @@ class AppNameApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def post(self, session: Session, app_model: App):
@@ -1117,6 +1127,7 @@ class AppIconApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def post(self, session: Session, app_model: App):
@@ -1150,6 +1161,7 @@ class AppSiteStatus(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def post(self, session: Session, app_model: App):
@@ -1177,6 +1189,7 @@ class AppApiStatus(Resource):
@is_admin_or_owner_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def post(self, session: Session, app_model: App):
+21 -13
View File
@@ -1,6 +1,7 @@
from flask_restx import Resource
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden
from configs import dify_config
from controllers.common.schema import register_enum_models, register_schema_models
@@ -28,6 +29,7 @@ from services.app_dsl_service import (
)
from services.enterprise.enterprise_service import EnterpriseService
from services.entities.dsl_entities import CheckDependenciesResult, ImportStatus
from services.errors.account import NoPermissionError
from services.feature_service import FeatureService
from .. import console_ns
@@ -91,18 +93,21 @@ class AppImportApi(Resource):
import_service = AppDslService(session)
# Import app
account = current_user
result = import_service.import_app(
account=account,
import_mode=args.mode,
yaml_content=args.yaml_content,
yaml_url=args.yaml_url,
name=args.name,
description=args.description,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
app_id=args.app_id,
)
try:
result = import_service.import_app(
account=account,
import_mode=args.mode,
yaml_content=args.yaml_content,
yaml_url=args.yaml_url,
name=args.name,
description=args.description,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
app_id=args.app_id,
)
except NoPermissionError as e:
raise Forbidden(str(e))
if result.status == ImportStatus.FAILED:
session.rollback()
else:
@@ -157,7 +162,10 @@ class AppImportConfirmApi(Resource):
import_service = AppDslService(session)
# Confirm import
account = current_user
result = import_service.confirm_import(import_id=import_id, account=account)
try:
result = import_service.confirm_import(import_id=import_id, account=account)
except NoPermissionError as e:
raise Forbidden(str(e))
if result.status == ImportStatus.FAILED:
session.rollback()
else:
+25 -14
View File
@@ -351,24 +351,31 @@ def _resolve_current_user_agent_debug_conversation_id(
app_model: App,
agent_id: str | None,
draft_type: AgentConfigDraftType,
start_new: bool = False,
) -> str:
"""Resolve the current editor's conversation without crossing draft surfaces."""
"""Resolve or rotate the current editor's conversation within one draft surface.
``start_new`` rotates the scoped mapping through ``AgentRosterService`` so
the old runtime session is retired before the new conversation is used.
Continuations and Build chat keep resolving the existing mapping.
"""
roster_service = AgentRosterService(session)
if agent_id:
return roster_service.get_or_create_agent_app_debug_conversation_id(
tenant_id=current_tenant_id,
agent_id=agent_id,
account_id=current_user.id,
draft_type=draft_type,
)
resolved_agent_id = agent_id
if not resolved_agent_id:
agent = roster_service.get_app_backing_agent(tenant_id=current_tenant_id, app_id=str(app_model.id))
if agent is None:
raise AgentNotFoundError()
resolved_agent_id = agent.id
agent = roster_service.get_app_backing_agent(tenant_id=current_tenant_id, app_id=str(app_model.id))
if agent is None:
raise AgentNotFoundError()
return roster_service.get_or_create_agent_app_debug_conversation_id(
resolve_conversation = (
roster_service.refresh_agent_app_debug_conversation_id
if start_new
else roster_service.get_or_create_agent_app_debug_conversation_id
)
return resolve_conversation(
tenant_id=current_tenant_id,
agent_id=agent.id,
agent_id=resolved_agent_id,
account_id=current_user.id,
draft_type=draft_type,
)
@@ -387,13 +394,17 @@ def _create_chat_message(
args = args_model.model_dump(exclude_none=True, by_alias=True)
if AppMode.value_of(app_model.mode) == AppMode.AGENT:
draft_type = AgentConfigDraftType(args_model.draft_type)
# Preview follows the normal chat contract: an omitted/empty conversation ID starts a new
# conversation. Build chat keeps its stable mapping so build drafts and finalization stay continuous.
debug_conversation_id = _resolve_current_user_agent_debug_conversation_id(
session=session,
current_tenant_id=current_tenant_id or app_model.tenant_id,
current_user=current_user,
app_model=app_model,
agent_id=agent_id,
draft_type=AgentConfigDraftType(args_model.draft_type),
draft_type=draft_type,
start_new=draft_type == AgentConfigDraftType.DRAFT and not args_model.conversation_id,
)
if args_model.conversation_id and args_model.conversation_id != debug_conversation_id:
raise NotFound("Conversation Not Exists.")
+3 -1
View File
@@ -10,7 +10,7 @@ from constants.languages import supported_language
from controllers.common.schema import register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model
from controllers.console.app.wraps import agent_manage_required_for_agent_app, get_app_model
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
@@ -93,6 +93,7 @@ class AppSite(Resource):
@login_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@account_initialization_required
@with_current_user
@with_session
@@ -145,6 +146,7 @@ class AppSiteAccessTokenReset(Resource):
@login_required
@is_admin_or_owner_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@account_initialization_required
@with_current_user
@with_session
@@ -0,0 +1,70 @@
"""Draft Human Input v2 workflow controller stubs."""
from __future__ import annotations
from http import HTTPStatus
from flask import abort
from flask_restx import Resource
from controllers.common.human_input_v2_contracts import (
MessageTemplateTestRequest,
MessageTemplateTestResponse,
)
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
account_initialization_required,
edit_permission_required,
rbac_permission_required,
setup_required,
with_current_user,
)
from libs.login import login_required
from models import Account
from models.model import AppMode
from .wraps import get_app_model
register_schema_models(console_ns, MessageTemplateTestRequest)
register_response_schema_models(console_ns, MessageTemplateTestResponse)
def _raise_stub_not_implemented() -> None:
abort(HTTPStatus.NOT_IMPLEMENTED, "Human Input v2 draft stub endpoint is not implemented yet.")
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/human-input/nodes/<string:node_id>/message-template/test")
class WorkflowDraftMessageTemplateTestApi(Resource):
@console_ns.expect(console_ns.models[MessageTemplateTestRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[MessageTemplateTestResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
@get_app_model(mode=[AppMode.WORKFLOW])
@with_current_user
@edit_permission_required
def post(self, current_user: Account, app_model, node_id: str):
MessageTemplateTestRequest.model_validate(console_ns.payload or {})
_raise_stub_not_implemented()
@console_ns.route(
"/apps/<uuid:app_id>/advanced-chat/workflows/draft/human-input/nodes/<string:node_id>/message-template/test"
)
class AdvancedChatDraftMessageTemplateTestApi(Resource):
@console_ns.expect(console_ns.models[MessageTemplateTestRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[MessageTemplateTestResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
@get_app_model(mode=[AppMode.ADVANCED_CHAT])
@with_current_user
@edit_permission_required
def post(self, current_user: Account, app_model, node_id: str):
MessageTemplateTestRequest.model_validate(console_ns.payload or {})
_raise_stub_not_implemented()
+48 -1
View File
@@ -12,14 +12,22 @@ from typing import cast, overload
from sqlalchemy import select
from sqlalchemy.orm import Session
from configs import dify_config
from controllers.common.session import with_session
from controllers.common.wraps import RBACPermission, RBACResourceScope, enforce_rbac_access
from controllers.console.app.error import AppNotFoundError
from extensions.ext_database import db
from libs.login import current_account_with_tenant
from models import App, AppMode, TrialApp
from models.agent import AgentScope
from services.recommended_app_service import RecommendedAppService
__all__ = ["get_app_model", "get_app_model_with_trial", "with_session"]
__all__ = [
"agent_manage_required_for_agent_app",
"get_app_model",
"get_app_model_with_trial",
"with_session",
]
def _load_app_model(session: Session, app_id: str) -> App | None:
@@ -48,6 +56,45 @@ def _load_app_model_with_trial(session: Session, app_id: str) -> App | None:
return app_model
def agent_manage_required_for_agent_app[**P, R](view: Callable[P, R]) -> Callable[P, R]:
"""Gate generic app management routes that target an Agent App.
A hidden workflow-only backing App only reuses the App runtime and is not
part of the general app management plane, so generic routes reject it
outright. Managing a roster Agent App mutates the roster Agent behind it
(rename/icon sync, archive, API enablement), so it additionally requires
workspace ``agent.manage`` on top of the route's existing App permission
checks when RBAC is enabled. A no-op for non-agent Apps. Must be placed
above ``get_app_model`` so the ``app_id`` path parameter is still present.
"""
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs) -> R:
raw_app_id = kwargs.get("app_id") or kwargs.get("resource_id")
if raw_app_id is not None:
app_model = _load_app_model_from_scoped_session(str(raw_app_id))
binding = (
app_model.agent_app_binding_with_session(session=db.session(), include_archived=True)
if app_model is not None
else None
)
if binding is not None:
if binding.scope == AgentScope.WORKFLOW_ONLY:
raise AppNotFoundError()
if dify_config.RBAC_ENABLED:
current_user, current_tenant_id = current_account_with_tenant()
enforce_rbac_access(
tenant_id=current_tenant_id,
account_id=current_user.id,
resource_type=RBACResourceScope.WORKSPACE,
scene=RBACPermission.AGENT_MANAGE,
resource_required=False,
)
return view(*args, **kwargs)
return decorated
def _get_injected_session(args: tuple[object, ...]) -> Session | None:
"""Return the request session inserted by `with_session`, if this handler has been migrated."""
if len(args) < 2:
+14
View File
@@ -7,10 +7,13 @@ from configs import dify_config
from constants.languages import supported_language
from controllers.common.schema import query_params_from_model, register_schema_models
from controllers.console import console_ns
from controllers.console.auth.error import InvitationAccountMismatchError
from controllers.console.error import AccountInFreezeError, AlreadyActivateError
from extensions.ext_database import db
from libs.datetime_utils import naive_utc_now
from libs.helper import EmailStr, timezone
from libs.login import current_account_with_tenant
from libs.token import extract_access_token
from models import AccountStatus
from models.account import TenantAccountJoin, TenantAccountRole
from services.account_service import RegisterService, TenantService
@@ -136,6 +139,12 @@ class ActivateApi(Resource):
)
@console_ns.response(400, "Already activated or invalid token")
def post(self):
"""Accept an invitation without letting an existing session act for another account.
Token-only activation remains available for legacy clients. When the request already
carries a console session, that session must belong to the account encoded in the
invitation before the token is consumed or tenant membership is changed.
"""
args = ActivatePayload.model_validate(console_ns.payload)
normalized_request_email = args.email.lower() if args.email else None
@@ -146,6 +155,11 @@ class ActivateApi(Resource):
raise AlreadyActivateError()
account = invitation["account"]
if extract_access_token(request):
current_account, _ = current_account_with_tenant()
if current_account.id != account.id:
raise InvitationAccountMismatchError()
if dify_config.BILLING_ENABLED and BillingService.is_email_in_freeze(account.email):
raise AccountInFreezeError()
+6
View File
@@ -13,6 +13,12 @@ class InvalidEmailError(BaseHTTPException):
code = 400
class InvitationAccountMismatchError(BaseHTTPException):
error_code = "invitation_account_mismatch"
description = "This invitation was sent to another account. Please sign in with the invited account."
code = 403
class PasswordMismatchError(BaseHTTPException):
error_code = "password_mismatch"
description = "The passwords do not match."
@@ -56,6 +56,7 @@ class OAuthProviderTokenResponse(BaseModel):
class OAuthProviderAccountResponse(BaseModel):
id: str
name: str
email: str
avatar: str | None = None
@@ -251,6 +252,7 @@ class OAuthServerUserAccountApi(Resource):
def post(self, oauth_provider_app: OAuthProviderApp, account: Account):
return jsonable_encoder(
{
"id": account.id,
"name": account.name,
"email": account.email,
"avatar": account.avatar,
+2 -1
View File
@@ -8,6 +8,7 @@ from sqlalchemy.orm import Session
from configs import dify_config
from controllers.fastopenapi import console_router
from enums.deployment_edition import DeploymentEdition
from extensions.ext_database import db
from models.model import DifySetup
from services.account_service import TenantService
@@ -63,7 +64,7 @@ def validate_init_password(payload: InitValidatePayload) -> InitValidateResponse
def get_init_validate_status() -> bool:
if dify_config.EDITION == "SELF_HOSTED":
if dify_config.DEPLOYMENT_EDITION != DeploymentEdition.CLOUD:
if os.environ.get("INIT_PASSWORD"):
if session.get("is_init_validated"):
return True
@@ -1,5 +0,0 @@
"""Typed Dify-owned KnowledgeFS Console product API."""
from . import resources
__all__ = ["resources"]
@@ -1,56 +0,0 @@
"""Stable, non-enumerating Console error contract for KnowledgeFS."""
from libs.exception import BaseHTTPException
class KnowledgeFSSpaceNotFoundHTTPError(BaseHTTPException):
error_code = "knowledge_fs_space_not_found"
description = "KnowledgeFS space was not found."
code = 404
class KnowledgeFSOperationUnavailableHTTPError(BaseHTTPException):
error_code = "knowledge_fs_operation_unavailable"
description = "KnowledgeFS operation is not available."
code = 503
class KnowledgeFSUpstreamUnavailableHTTPError(BaseHTTPException):
error_code = "knowledge_fs_upstream_unavailable"
description = "KnowledgeFS is unavailable."
code = 502
class KnowledgeFSInvalidRequestHTTPError(BaseHTTPException):
error_code = "knowledge_fs_invalid_request"
description = "KnowledgeFS request is invalid."
code = 400
class KnowledgeFSAccessDeniedHTTPError(BaseHTTPException):
error_code = "knowledge_fs_access_denied"
description = "KnowledgeFS operation is not allowed."
code = 403
class KnowledgeFSRateLimitHTTPError(BaseHTTPException):
error_code = "knowledge_fs_rate_limit_exceeded"
description = "KnowledgeFS operation rate limit exceeded."
code = 429
class KnowledgeFSQuotaExceededHTTPError(BaseHTTPException):
error_code = "knowledge_fs_quota_exceeded"
description = "KnowledgeFS operation quota exceeded."
code = 403
__all__ = [
"KnowledgeFSAccessDeniedHTTPError",
"KnowledgeFSInvalidRequestHTTPError",
"KnowledgeFSOperationUnavailableHTTPError",
"KnowledgeFSQuotaExceededHTTPError",
"KnowledgeFSRateLimitHTTPError",
"KnowledgeFSSpaceNotFoundHTTPError",
"KnowledgeFSUpstreamUnavailableHTTPError",
]
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,419 @@
"""Authenticated transport adapter for the Console-to-KnowledgeFS proxy.
These raw Blueprint routes deliberately stay outside Dify's OpenAPI surface:
KnowledgeFS owns the wire contract consumed by the frontend. The catch-all path
avoids resource-specific Dify controllers, while the forwarding module consumes
only the operations explicitly enabled by Dify's product registry. The registry
can be validated explicitly against the pinned KnowledgeFS contract during development.
Console auth and contract-specific dataset RBAC run before forwarding. Request
bodies are capped at 64 MiB, JSON and binary responses have separate bounds,
SSE responses remain streaming with a bounded idle read timeout, and only safe
response headers are exposed. Operation-specific upstream error mappings are
applied before Console JSON error handling; the default maps 401 to 502 so it
cannot trigger browser-session recovery and preserves resource-level 403.
"""
from __future__ import annotations
import logging
from collections.abc import Callable, Iterator
from functools import wraps
from http import HTTPStatus
from typing import NoReturn, cast
import httpx
from flask import Response, request, stream_with_context
from flask.typing import ResponseReturnValue
from werkzeug.exceptions import (
BadGateway,
Forbidden,
GatewayTimeout,
HTTPException,
NotFound,
RequestEntityTooLarge,
ServiceUnavailable,
default_exceptions,
)
from configs import dify_config
from controllers.console import api, bp
from controllers.console.wraps import (
account_initialization_required,
cloud_edition_billing_rate_limit_check,
setup_required,
)
from core.helper import ssrf_proxy
from libs.login import current_account_with_tenant, login_required
from services.knowledge_fs_operations import KnowledgeFSMethod
from services.knowledge_fs_proxy import (
KnowledgeFSAccessDeniedError,
KnowledgeFSAuthorization,
KnowledgeFSConfigurationError,
KnowledgeFSRouteNotAllowedError,
KnowledgeFSTimeoutError,
KnowledgeFSTransportError,
KnowledgeFSUpstreamResponse,
authorize_knowledge_fs_request,
get_knowledge_fs_operation,
proxy_authorized_knowledge_fs_request,
proxy_knowledge_fs_request,
)
logger = logging.getLogger(__name__)
type _KnowledgeFSRequestForwarder = Callable[
[str | None, str | None, bytes | None, bytes | None],
KnowledgeFSUpstreamResponse,
]
_MAX_PROXY_BODY_BYTES = 64 * 1024 * 1024
_RESPONSE_HEADER_ALLOWLIST = (
"Cache-Control",
"Content-Disposition",
"Content-Type",
"Retry-After",
"X-Trace-Id",
)
_RESPONSE_HEADER_DENYLIST = frozenset(
{
"authorization",
"connection",
"cookie",
"keep-alive",
"proxy-authenticate",
"proxy-authorization",
"set-cookie",
"te",
"trailer",
"transfer-encoding",
"upgrade",
}
)
def _console_api_errors[**P](
view: Callable[P, ResponseReturnValue],
) -> Callable[P, ResponseReturnValue]:
"""Route raw Blueprint exceptions through the Console API JSON handlers."""
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs) -> ResponseReturnValue:
try:
return view(*args, **kwargs)
except Exception as exc:
return api.handle_error(exc)
return decorated
def _knowledge_fs_enabled[**P](
view: Callable[P, ResponseReturnValue],
) -> Callable[P, ResponseReturnValue]:
"""Hide the complete KnowledgeFS route surface while the bridge is disabled."""
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs) -> ResponseReturnValue:
if not dify_config.KNOWLEDGE_FS_ENABLED:
raise NotFound()
return view(*args, **kwargs)
return decorated
def _translate_proxy_error(exc: Exception, *, tenant_id: str) -> NoReturn:
"""Map forwarding failures to the stable Console HTTP error surface."""
if isinstance(exc, KnowledgeFSRouteNotAllowedError):
raise NotFound() from exc
if isinstance(exc, KnowledgeFSAccessDeniedError):
raise Forbidden() from exc
if isinstance(exc, KnowledgeFSConfigurationError):
logger.error("KnowledgeFS request was blocked by invalid configuration for tenant_id=%s", tenant_id)
raise ServiceUnavailable("KnowledgeFS integration is misconfigured") from exc
if isinstance(exc, KnowledgeFSTimeoutError):
raise GatewayTimeout("KnowledgeFS request timed out") from exc
if isinstance(exc, KnowledgeFSTransportError):
logger.warning("KnowledgeFS transport request failed for tenant_id=%s", tenant_id)
raise BadGateway("KnowledgeFS is unavailable") from exc
raise exc
def _knowledge_fs_operation_access_required(
view: Callable[[KnowledgeFSAuthorization], ResponseReturnValue],
) -> Callable[[KnowledgeFSMethod, str], ResponseReturnValue]:
"""Authorize one declared operation before billing and request-body work."""
@wraps(view)
def decorated(method: KnowledgeFSMethod, upstream_path: str) -> ResponseReturnValue:
current_user, tenant_id = current_account_with_tenant()
try:
authorization = authorize_knowledge_fs_request(
account=current_user,
tenant_id=tenant_id,
method=method,
path=upstream_path,
)
except KnowledgeFSRouteNotAllowedError as exc:
raise NotFound() from exc
except KnowledgeFSAccessDeniedError as exc:
_translate_proxy_error(exc, tenant_id=tenant_id)
return view(authorization)
return decorated
def _request_body() -> bytes:
"""Read the raw body up to the proxy limit or raise RequestEntityTooLarge."""
body = request.stream.read(_MAX_PROXY_BODY_BYTES + 1)
if len(body) > _MAX_PROXY_BODY_BYTES:
raise RequestEntityTooLarge("KnowledgeFS proxy request body is too large")
return body
def _stream_response_body(
upstream: httpx.Response,
*,
tenant_id: str,
max_response_bytes: int,
) -> Iterator[bytes]:
"""Yield one bounded SSE response and always release its pooled connection."""
total_bytes = 0
try:
for chunk in upstream.iter_bytes():
total_bytes += len(chunk)
if total_bytes > max_response_bytes:
logger.warning("KnowledgeFS stream exceeded the proxy limit for tenant_id=%s", tenant_id)
raise ssrf_proxy.ResponseTooLargeError(f"response exceeded {max_response_bytes} bytes")
yield chunk
finally:
upstream.close()
def _proxy_response(
upstream_result: KnowledgeFSUpstreamResponse,
*,
tenant_id: str,
contract_response_headers: tuple[str, ...],
max_response_bytes: int,
) -> Response:
"""Expose raw content, status, and allowlisted headers from KnowledgeFS.
Raises:
HTTPException: KnowledgeFS returns a status normalized by the operation contract.
"""
upstream = upstream_result.response
mapped_status = dict(upstream_result.operation.error_status_map).get(upstream.status_code)
if mapped_status is not None:
upstream.close()
description = "KnowledgeFS upstream request failed"
if upstream.status_code == HTTPStatus.UNAUTHORIZED:
description = "KnowledgeFS authentication failed"
logger.error(
"KnowledgeFS rejected the Dify server credential with HTTP %s for tenant_id=%s",
upstream.status_code,
tenant_id,
)
exception_type = default_exceptions.get(mapped_status)
if exception_type is None:
exception = HTTPException(description)
exception.code = mapped_status
raise exception
raise exception_type(description)
allowed_header_names = dict.fromkeys(
name.lower() for name in (*_RESPONSE_HEADER_ALLOWLIST, *contract_response_headers)
)
headers = {
name: value
for name in allowed_header_names
if name not in _RESPONSE_HEADER_DENYLIST
if (value := upstream.headers.get(name)) is not None
}
if upstream_result.response_kind == "stream":
response = Response(
stream_with_context( # pyrefly: ignore[no-matching-overload]
_stream_response_body(
upstream,
tenant_id=tenant_id,
max_response_bytes=max_response_bytes,
)
),
status=upstream.status_code,
headers=headers,
)
response.call_on_close(upstream.close)
return response
try:
content = upstream.content
finally:
upstream.close()
return Response(content, status=upstream.status_code, headers=headers)
def _proxy_current_request(
*,
method: KnowledgeFSMethod,
tenant_id: str,
forward: _KnowledgeFSRequestForwarder,
) -> Response:
"""Forward the current raw request through one preconfigured service entry."""
if not dify_config.KNOWLEDGE_FS_ENABLED:
raise NotFound()
try:
proxy_result = forward(
request.headers.get("Accept"),
request.content_type,
request.query_string or None,
_request_body() if method != "GET" else None,
)
except (
KnowledgeFSConfigurationError,
KnowledgeFSAccessDeniedError,
KnowledgeFSRouteNotAllowedError,
KnowledgeFSTimeoutError,
KnowledgeFSTransportError,
) as exc:
_translate_proxy_error(exc, tenant_id=tenant_id)
return _proxy_response(
proxy_result,
tenant_id=tenant_id,
contract_response_headers=proxy_result.operation.response_headers,
max_response_bytes=proxy_result.operation.max_response_bytes,
)
def _proxy_request(
method: KnowledgeFSMethod,
upstream_path: str,
) -> Response:
"""Authorize and forward the current request through the combined service use case."""
if not dify_config.KNOWLEDGE_FS_ENABLED:
raise NotFound()
current_user, tenant_id = current_account_with_tenant()
def forward(
accept: str | None,
content_type: str | None,
query: bytes | None,
body: bytes | None,
) -> KnowledgeFSUpstreamResponse:
return proxy_knowledge_fs_request(
account=current_user,
method=method,
path=upstream_path,
tenant_id=tenant_id,
accept=accept,
content_type=content_type,
query=query,
body=body,
request_headers=request.headers,
)
return _proxy_current_request(method=method, tenant_id=tenant_id, forward=forward)
def _proxy_authorized_request(authorization: KnowledgeFSAuthorization) -> Response:
"""Forward the current request using one previously authorized operation capability.
Args:
authorization: Request-scoped capability produced before billing and body parsing.
Returns:
The filtered response returned by KnowledgeFS.
Raises:
HTTPException: The integration is disabled or forwarding fails.
"""
operation = authorization.operation
tenant_id = authorization.tenant_id
def forward(
accept: str | None,
content_type: str | None,
query: bytes | None,
body: bytes | None,
) -> KnowledgeFSUpstreamResponse:
return proxy_authorized_knowledge_fs_request(
authorization=authorization,
accept=accept,
content_type=content_type,
query=query,
body=body,
request_headers=request.headers,
)
return _proxy_current_request(method=operation.method, tenant_id=tenant_id, forward=forward)
@_knowledge_fs_enabled
@_knowledge_fs_operation_access_required
@cloud_edition_billing_rate_limit_check("knowledge")
def _proxy_knowledge_fs_non_get(
authorization: KnowledgeFSAuthorization,
) -> ResponseReturnValue:
"""Apply knowledge billing checks to one allowlisted non-GET operation."""
return _proxy_authorized_request(authorization)
@bp.route(
"/knowledge-fs/<path:upstream_path>",
methods=["OPTIONS"],
provide_automatic_options=False,
)
@_console_api_errors
@_knowledge_fs_enabled
def proxy_knowledge_fs_options(upstream_path: str) -> ResponseReturnValue:
"""Complete a CORS preflight only for an enabled Console operation."""
requested_method = cast(KnowledgeFSMethod, request.headers.get("Access-Control-Request-Method", "").upper())
try:
get_knowledge_fs_operation(requested_method, upstream_path)
except KnowledgeFSRouteNotAllowedError as exc:
raise NotFound() from exc
return Response(status=HTTPStatus.NO_CONTENT)
@bp.route(
"/knowledge-fs/<path:upstream_path>",
methods=["GET"],
provide_automatic_options=False,
)
@_console_api_errors
@_knowledge_fs_enabled
@setup_required
@login_required
@account_initialization_required
def proxy_knowledge_fs_get(upstream_path: str) -> ResponseReturnValue:
"""Forward one authenticated, dataset-readable GET request.
Args:
upstream_path: Relative KFS path captured after the Console proxy prefix.
Returns:
The filtered raw KnowledgeFS response or a Console JSON error response.
"""
if request.method != "GET":
raise NotFound()
return _proxy_request("GET", upstream_path)
@bp.route(
"/knowledge-fs/<path:upstream_path>",
methods=["DELETE", "PATCH", "POST", "PUT"],
provide_automatic_options=False,
)
@_console_api_errors
@_knowledge_fs_enabled
@setup_required
@login_required
@account_initialization_required
def proxy_knowledge_fs_write(upstream_path: str) -> ResponseReturnValue:
"""Forward one authenticated non-GET request under its contract access policy.
Args:
upstream_path: Relative KFS path captured after the Console proxy prefix.
Returns:
The filtered raw KnowledgeFS response or a Console JSON error response.
"""
method = cast(KnowledgeFSMethod, request.method)
return _proxy_knowledge_fs_non_get(method, upstream_path)
+3 -2
View File
@@ -6,6 +6,7 @@ from sqlalchemy import select
from configs import dify_config
from controllers.fastopenapi import console_router
from enums.deployment_edition import DeploymentEdition
from libs.helper import EmailStr, extract_remote_ip
from libs.password import valid_password
from models.model import DifySetup, db
@@ -52,7 +53,7 @@ def get_setup_status_api() -> SetupStatusResponse:
Only bootstrap-safe status information should be returned by this endpoint.
"""
if dify_config.EDITION == "SELF_HOSTED":
if dify_config.DEPLOYMENT_EDITION != DeploymentEdition.CLOUD:
setup_status = get_setup_status()
if setup_status and not isinstance(setup_status, bool):
return SetupStatusResponse(step="finished", setup_at=setup_status.setup_at.isoformat())
@@ -102,7 +103,7 @@ def setup_system(payload: SetupRequestPayload) -> SetupResponse:
def get_setup_status() -> DifySetup | bool | None:
if dify_config.EDITION == "SELF_HOSTED":
if dify_config.DEPLOYMENT_EDITION != DeploymentEdition.CLOUD:
return db.session.scalar(select(DifySetup).limit(1))
return True
+2 -1
View File
@@ -46,6 +46,7 @@ from controllers.console.wraps import (
with_current_tenant_id,
with_current_user,
)
from enums.deployment_edition import DeploymentEdition
from extensions.ext_database import db
from fields.base import ResponseModel
from fields.member_fields import AccountResponse
@@ -262,7 +263,7 @@ class AccountInitApi(Resource):
payload = console_ns.payload or {}
args = AccountInitPayload.model_validate(payload)
if dify_config.EDITION == "CLOUD":
if dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD:
if not args.invitation_code:
raise ValueError("invitation_code is required")
@@ -0,0 +1,541 @@
"""Workspace-level Human Input v2 controller stubs."""
from __future__ import annotations
from http import HTTPStatus
from flask import abort, request
from flask_restx import Resource
from controllers.common.human_input_v2_contracts import (
AddPlatformContactsRequest,
AddPlatformContactsResponse,
BatchGetContactOptionsQuery,
BatchGetContactOptionsResponse,
BatchGetContactsQuery,
BatchGetContactsResponse,
ContactListQuery,
ContactOption,
ContactOptionsQuery,
CreateIMBindingRequest,
CreateIMBindingResponse,
CreateIMSyncRunResponse,
DeleteIMBindingQuery,
DeleteIMBindingResponse,
DeleteIMIntegrationQuery,
ExternalContactCreateRequest,
ExternalContactCreateResponse,
ExternalContactUpdateRequest,
ExternalContactUpdateResponse,
GetContactResponse,
GetEmailProviderResponse,
GetIMIntegrationResponse,
GetLatestIMSyncRunResponse,
HumanInputContact,
HumanInputContactType,
IMIntegrationStatus,
IMProvider,
IMSyncResultType,
IMSyncRunStatus,
ListContactOptionsResponse,
ListContactsResponse,
ListIMIdentitiesQuery,
ListIMIdentitiesResponse,
ListLatestIMSyncRunResultsQuery,
ListLatestIMSyncRunResultsResponse,
ListOrganizationCandidatesResponse,
NodeDataMigrationFailureResponse,
NodeDataMigrationPayload,
NodeDataMigrationResponse,
OrganizationCandidatesQuery,
RemoveContactsRequest,
RemoveContactsResponse,
ResetContactIMOverrideResponse,
SetContactIMOverrideRequest,
SetContactIMOverrideResponse,
SetEmailProviderRequest,
SetEmailProviderResponse,
TestIMIntegrationRequest,
TestIMIntegrationResponse,
UpdateIMIntegrationRequest,
UpdateIMIntegrationResponse,
)
from controllers.common.schema import (
query_params_from_model,
query_params_from_request,
register_enum_models,
register_response_schema_models,
register_schema_models,
)
from controllers.console import console_ns
from controllers.console.wraps import (
account_initialization_required,
edit_permission_required,
is_admin_or_owner_required,
setup_required,
with_current_tenant_id,
)
from libs.login import login_required
register_enum_models(
console_ns,
HumanInputContactType,
IMIntegrationStatus,
IMSyncRunStatus,
IMSyncResultType,
IMProvider,
)
register_schema_models(
console_ns,
ContactListQuery,
ContactOptionsQuery,
BatchGetContactOptionsQuery,
OrganizationCandidatesQuery,
AddPlatformContactsRequest,
ExternalContactCreateRequest,
ExternalContactUpdateRequest,
RemoveContactsRequest,
UpdateIMIntegrationRequest,
DeleteIMIntegrationQuery,
TestIMIntegrationRequest,
ListIMIdentitiesQuery,
ListLatestIMSyncRunResultsQuery,
SetContactIMOverrideRequest,
CreateIMBindingRequest,
NodeDataMigrationPayload,
SetEmailProviderRequest,
)
register_response_schema_models(
console_ns,
HumanInputContact,
ContactOption,
GetContactResponse,
ExternalContactCreateResponse,
ExternalContactUpdateResponse,
AddPlatformContactsResponse,
ListContactsResponse,
ListContactOptionsResponse,
BatchGetContactOptionsResponse,
RemoveContactsResponse,
ListIMIdentitiesResponse,
GetIMIntegrationResponse,
UpdateIMIntegrationResponse,
TestIMIntegrationResponse,
CreateIMSyncRunResponse,
GetLatestIMSyncRunResponse,
ListLatestIMSyncRunResultsResponse,
ListOrganizationCandidatesResponse,
ResetContactIMOverrideResponse,
SetContactIMOverrideResponse,
CreateIMBindingResponse,
DeleteIMBindingResponse,
BatchGetContactsResponse,
NodeDataMigrationResponse,
NodeDataMigrationFailureResponse,
GetEmailProviderResponse,
SetEmailProviderResponse,
)
def _raise_stub_not_implemented() -> None:
abort(HTTPStatus.NOT_IMPLEMENTED, "Human Input v2 stub endpoint is not implemented yet.")
@console_ns.route("/workspaces/current/human-input/contacts")
class WorkspaceContactsApi(Resource):
@console_ns.doc(params=query_params_from_model(ContactListQuery))
@console_ns.response(200, "Success", console_ns.models[ListContactsResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def get(self, tenant_id: str):
ContactListQuery.model_validate(request.args.to_dict(flat=True))
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/contacts/<uuid:contact_id>")
class WorkspaceContactApi(Resource):
"""Read one contact only when it resolves in the current workspace scope."""
@console_ns.response(200, "Success", console_ns.models[GetContactResponse.__name__])
@console_ns.response(404, "Contact not found or absent in the current workspace")
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def get(self, tenant_id: str, contact_id: str):
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/contact-options")
class WorkspaceContactOptionsApi(Resource):
"""Search the current workspace's selectable Contact projection for workflow editors."""
@console_ns.doc(
params=query_params_from_model(ContactOptionsQuery),
description=(
"List editor-safe Contact options for static recipient selection. "
"The projection omits email, IM bindings, and management metadata; contacts that resolve as ABSENT "
"or are otherwise unavailable in the current workspace are omitted."
),
)
@console_ns.response(200, "Success", console_ns.models[ListContactOptionsResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_tenant_id
def get(self, tenant_id: str):
ContactOptionsQuery.model_validate(request.args.to_dict(flat=True))
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/organization-candidates")
class WorkspaceOrganizationCandidatesApi(Resource):
@console_ns.doc(params=query_params_from_model(OrganizationCandidatesQuery))
@console_ns.response(200, "Success", console_ns.models[ListOrganizationCandidatesResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def get(self, tenant_id: str):
OrganizationCandidatesQuery.model_validate(request.args.to_dict(flat=True))
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/contacts/platform")
class WorkspacePlatformContactsApi(Resource):
@console_ns.expect(console_ns.models[AddPlatformContactsRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[AddPlatformContactsResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def post(self, tenant_id: str):
AddPlatformContactsRequest.model_validate(console_ns.payload or {})
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/contacts/external")
class WorkspaceExternalContactsApi(Resource):
@console_ns.expect(console_ns.models[ExternalContactCreateRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[ExternalContactCreateResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def post(self, tenant_id: str):
ExternalContactCreateRequest.model_validate(console_ns.payload or {})
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/contacts/external/<uuid:contact_id>")
class WorkspaceExternalContactApi(Resource):
@console_ns.expect(console_ns.models[ExternalContactUpdateRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[ExternalContactUpdateResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def patch(self, tenant_id: str, contact_id: str):
ExternalContactUpdateRequest.model_validate(console_ns.payload or {})
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/contacts/remove")
class WorkspaceContactsRemoveApi(Resource):
@console_ns.expect(console_ns.models[RemoveContactsRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[RemoveContactsResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def post(self, tenant_id: str):
RemoveContactsRequest.model_validate(console_ns.payload or {})
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/im-integration")
class WorkspaceIMIntegrationApi(Resource):
@console_ns.response(200, "Success", console_ns.models[GetIMIntegrationResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def get(self, tenant_id: str):
_raise_stub_not_implemented()
@console_ns.expect(console_ns.models[UpdateIMIntegrationRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[UpdateIMIntegrationResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def put(self, tenant_id: str):
UpdateIMIntegrationRequest.model_validate(console_ns.payload or {})
_raise_stub_not_implemented()
@console_ns.doc(params=query_params_from_model(DeleteIMIntegrationQuery))
@console_ns.response(204, "IM integration deleted successfully")
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def delete(self, tenant_id: str):
query_params_from_request(DeleteIMIntegrationQuery)
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/im-integration/test")
class WorkspaceIMIntegrationTestApi(Resource):
@console_ns.expect(console_ns.models[TestIMIntegrationRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[TestIMIntegrationResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def post(self, tenant_id: str):
TestIMIntegrationRequest.model_validate(console_ns.payload or {})
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/im-sync-runs")
class WorkspaceIMSyncRunsApi(Resource):
@console_ns.response(200, "Success", console_ns.models[CreateIMSyncRunResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def post(self, tenant_id: str):
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/im-sync-runs/latest")
class WorkspaceLatestIMSyncRunApi(Resource):
@console_ns.doc(
description=(
"Return the latest IM sync run summary. The UI uses finished_at as the explicit sync time; "
"the response does not include started_by."
)
)
@console_ns.response(200, "Success", console_ns.models[GetLatestIMSyncRunResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def get(self, tenant_id: str):
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/im-sync-runs/latest/results")
class WorkspaceLatestIMSyncRunResultsApi(Resource):
@console_ns.doc(
params=query_params_from_model(ListLatestIMSyncRunResultsQuery),
description=(
"Return one required result bucket from the latest IM sync run using page and limit pagination. "
"There is no all filter; the response contains page, limit, and total metadata without a run summary."
),
)
@console_ns.response(200, "Success", console_ns.models[ListLatestIMSyncRunResultsResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def get(self, tenant_id: str):
ListLatestIMSyncRunResultsQuery.model_validate(request.args.to_dict(flat=True))
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/im-identities")
class WorkspaceIMIdentitiesApi(Resource):
@console_ns.doc(params=query_params_from_model(ListIMIdentitiesQuery))
@console_ns.response(200, "Success", console_ns.models[ListIMIdentitiesResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def get(self, tenant_id: str):
ListIMIdentitiesQuery.model_validate(request.args.to_dict(flat=True))
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/contacts/<uuid:contact_id>/im-override")
class WorkspaceContactIMOverrideApi(Resource):
@console_ns.doc(
description=(
"Set or reset the IM override for a contact. "
"This endpoint is used to override the IM identity for a contact in the workspace."
),
)
@console_ns.expect(console_ns.models[SetContactIMOverrideRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[SetContactIMOverrideResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def put(self, tenant_id: str, contact_id: str):
# This API only works in EE.
SetContactIMOverrideRequest.model_validate(console_ns.payload or {})
_raise_stub_not_implemented()
@console_ns.doc(
description=(
"Reset the IM override for a contact. "
"This endpoint is used to clear the IM identity override for a contact in the workspace."
),
)
@console_ns.response(200, "Success", console_ns.models[ResetContactIMOverrideResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def delete(self, tenant_id: str, contact_id: str):
# This API only works in EE.
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/contacts/<uuid:contact_id>/im-bindings")
class WorkspaceContactIMBindingsApi(Resource):
@console_ns.doc(
description=(
"Set an IM binding for a contact. Used for binding an IM identity to a contact. "
"This endpoint is not used for creating workspace IM override. "
"For that purpose, use WorkspaceContactIMOverrideApi.put instead."
),
)
@console_ns.expect(console_ns.models[CreateIMBindingRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[CreateIMBindingResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def put(self, tenant_id: str, contact_id: str):
CreateIMBindingRequest.model_validate(console_ns.payload or {})
_raise_stub_not_implemented()
@console_ns.response(200, "Success", console_ns.models[DeleteIMBindingResponse.__name__])
@console_ns.doc(
params=query_params_from_model(DeleteIMBindingQuery),
description=(
"Delete an IM binding for a contact. Used for removing contact IM binding information. "
"This endpoint is not used for resetting workspace IM override. For that purpose, use "
"WorkspaceContactIMOverrideApi.delete instead."
),
)
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def delete(self, tenant_id: str, contact_id: str):
query_params_from_request(DeleteIMBindingQuery)
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/contacts/batch")
class BatchGetContactsAPI(Resource):
@console_ns.doc(
params=query_params_from_model(BatchGetContactsQuery),
description=(
"Admin-only batch lookup for Contact management clients. "
"Workflow editors must use the editor-safe contact-options/batch projection."
),
)
@console_ns.response(200, "Success", console_ns.models[BatchGetContactsResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def get(self, tenant_id: str):
query_params_from_request(BatchGetContactsQuery, list_fields=("contact_ids",))
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/contact-options/batch")
class BatchGetContactOptionsAPI(Resource):
"""Resolve persisted Contact IDs through the same editor-safe selection projection."""
@console_ns.doc(
params=query_params_from_model(BatchGetContactOptionsQuery),
description=(
"Resolve Contact IDs persisted in workflow recipient configuration. "
"Contacts that resolve as ABSENT or are otherwise unavailable in the current workspace are omitted."
),
)
@console_ns.response(200, "Success", console_ns.models[BatchGetContactOptionsResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_tenant_id
def get(self, tenant_id: str):
query_params_from_request(BatchGetContactOptionsQuery, list_fields=("contact_ids",))
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/node-data-migration")
class NodeDataMigrationAPI(Resource):
@console_ns.doc(
description=(
"Migrate node data from HITLv1 to HITLv2. "
'A missing legacy version defaults to "1"; any other explicit version is rejected. '
"This endpoint only returns the migrated Human Input v2 node data to the client. "
"It does not update the workflow DSL."
),
)
@console_ns.expect(console_ns.models[NodeDataMigrationPayload.__name__])
@console_ns.response(200, "Success", console_ns.models[NodeDataMigrationResponse.__name__])
@console_ns.response(400, "Migration failed", console_ns.models[NodeDataMigrationFailureResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_tenant_id
def post(self, tenant_id: str):
NodeDataMigrationPayload.model_validate(console_ns.payload or {})
_raise_stub_not_implemented()
@console_ns.route("/workspaces/current/human-input/email-provider")
class HumanInputEmailProviderAPI(Resource):
@console_ns.doc(description="Retrieve the current email provider settings for human input")
@console_ns.response(200, "Success", console_ns.models[GetEmailProviderResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def get(self, tenant_id: str):
_raise_stub_not_implemented()
@console_ns.doc(description="update the current email provider settings for human input")
@console_ns.expect(console_ns.models[SetEmailProviderRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[SetEmailProviderResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@is_admin_or_owner_required
@with_current_tenant_id
def put(self, tenant_id: str):
SetEmailProviderRequest.model_validate(console_ns.payload or {})
_raise_stub_not_implemented()
+4 -8
View File
@@ -442,12 +442,10 @@ register_enum_models(
)
def _default_auto_upgrade_settings(
tenant_id: str,
category: TenantPluginAutoUpgradeCategory,
) -> AutoUpgradeSettingsResponse:
def _missing_auto_upgrade_settings(tenant_id: str) -> AutoUpgradeSettingsResponse:
"""Represent a missing persisted strategy as effectively disabled."""
return {
"strategy_setting": PluginAutoUpgradeService.default_strategy_setting_for_category(category),
"strategy_setting": TenantPluginAutoUpgradeStrategySetting.DISABLED,
"upgrade_time_of_day": PluginAutoUpgradeService.default_upgrade_time_of_day(tenant_id),
"upgrade_mode": TenantPluginAutoUpgradeMode.EXCLUDE,
"exclude_plugins": [],
@@ -1135,9 +1133,7 @@ class PluginFetchAutoUpgradeApi(Resource):
args = ParserAutoUpgradeFetch.model_validate(request.args.to_dict(flat=True))
auto_upgrade = PluginAutoUpgradeService.get_strategy(tenant_id, args.category, session=db.session())
auto_upgrade_dict = (
_auto_upgrade_settings_to_dict(auto_upgrade)
if auto_upgrade
else _default_auto_upgrade_settings(tenant_id, args.category)
_auto_upgrade_settings_to_dict(auto_upgrade) if auto_upgrade else _missing_auto_upgrade_settings(tenant_id)
)
return jsonable_encoder(
+3 -14
View File
@@ -346,13 +346,7 @@ class RBACRoleItemApi(Resource):
def put(self, role_id):
tenant_id, account_id = _current_ids()
request = _payload(_RoleUpsertRequest)
role = svc.RBACService.KnowledgeFSRoleMutations.update_role(
tenant_id,
account_id,
str(role_id),
request.to_mutation(),
session=db.session(),
)
role = svc.RBACService.Roles.update(tenant_id, account_id, str(role_id), request.to_mutation())
return _dump(role)
@login_required
@@ -362,12 +356,7 @@ class RBACRoleItemApi(Resource):
@console_ns.response(200, "Success", console_ns.models[svc.RBACRole.__name__])
def delete(self, role_id):
tenant_id, account_id = _current_ids()
svc.RBACService.KnowledgeFSRoleMutations.delete_role(
tenant_id,
account_id,
str(role_id),
session=db.session(),
)
svc.RBACService.Roles.delete(tenant_id, account_id, str(role_id))
return {"result": "success"}
@@ -926,7 +915,7 @@ class RBACMemberRolesApi(Resource):
tenant_id, account_id = _current_ids()
request = _payload(_ReplaceMemberRolesRequest)
return _dump(
svc.RBACService.KnowledgeFSRoleMutations.replace_member_roles(
svc.RBACService.MemberRoles.replace(
tenant_id,
account_id,
str(member_id),
@@ -37,6 +37,7 @@ from controllers.console.wraps import (
with_current_user,
)
from enums.cloud_plan import CloudPlan
from enums.deployment_edition import DeploymentEdition
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.helper import dump_response, to_timestamp
@@ -233,7 +234,7 @@ class TenantListApi(Resource):
tenants = [tenant for tenant, _ in tenant_rows]
tenant_dicts = []
is_enterprise_only = dify_config.ENTERPRISE_ENABLED and not dify_config.BILLING_ENABLED
is_saas = dify_config.EDITION == "CLOUD" and dify_config.BILLING_ENABLED
is_saas = dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD and dify_config.BILLING_ENABLED
tenant_plans: dict[str, SubscriptionPlan] = {}
if is_saas:
+4 -3
View File
@@ -20,6 +20,7 @@ from controllers.common.wraps import (
from controllers.console.auth.error import AuthenticationFailedError, EmailCodeError
from controllers.console.workspace.error import AccountNotInitializedError
from enums.cloud_plan import CloudPlan
from enums.deployment_edition import DeploymentEdition
from extensions.ext_database import db
from extensions.ext_redis import redis_client
from libs.encryption import FieldEncryption
@@ -129,7 +130,7 @@ def account_initialization_required[R](view: Callable[..., R]) -> Callable[...,
def only_edition_cloud[**P, R](view: Callable[P, R]) -> Callable[P, R]:
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs):
if dify_config.EDITION != "CLOUD":
if dify_config.DEPLOYMENT_EDITION != DeploymentEdition.CLOUD:
abort(404)
return view(*args, **kwargs)
@@ -151,7 +152,7 @@ def only_edition_enterprise[**P, R](view: Callable[P, R]) -> Callable[P, R]:
def only_edition_self_hosted[**P, R](view: Callable[P, R]) -> Callable[P, R]:
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs):
if dify_config.EDITION != "SELF_HOSTED":
if dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD:
abort(404)
return view(*args, **kwargs)
@@ -327,7 +328,7 @@ def setup_required[R](view: Callable[..., R]) -> Callable[..., R]:
# The overloads keep Resource methods method-aware for pyrefly while
# preserving support for plain functions used in tests and utilities.
# check setup
if dify_config.EDITION == "SELF_HOSTED" and not _is_setup_completed():
if dify_config.DEPLOYMENT_EDITION != DeploymentEdition.CLOUD and not _is_setup_completed():
if os.environ.get("INIT_PASSWORD"):
raise NotInitValidateError()
raise NotSetupError()
+2 -2
View File
@@ -20,10 +20,10 @@ from . import runtime_credentials as _runtime_credentials
from .agent import tools as _agent_tools
from .app import dsl as _app_dsl
from .knowledge import retrieval as _knowledge_retrieval
from .knowledge_fs import storage as _knowledge_fs_storage
from .plugin import agent_config as _agent_config
from .plugin import agent_drive as _agent_drive
from .plugin import plugin as _plugin
from .workspace import plugin_model_providers as _plugin_model_providers
from .workspace import workspace as _workspace
api.add_namespace(inner_api_ns)
@@ -33,10 +33,10 @@ __all__ = [
"_agent_drive",
"_agent_tools",
"_app_dsl",
"_knowledge_fs_storage",
"_knowledge_retrieval",
"_mail",
"_plugin",
"_plugin_model_providers",
"_runtime_credentials",
"_workspace",
"api",
@@ -1 +0,0 @@
"""Trusted KnowledgeFS inner API endpoints."""
@@ -1,303 +0,0 @@
"""Trusted KnowledgeFS gateway to Dify's configured object-storage backend."""
import json
from base64 import b64decode
from binascii import Error as BinasciiError
from http import HTTPStatus
from typing import NoReturn
from flask import Response, request
from flask_restx import Resource
from pydantic import BaseModel, ConfigDict, Field, TypeAdapter, ValidationError
from pydantic.alias_generators import to_camel
from controllers.common.schema import query_params_from_model, register_response_schema_models
from controllers.inner_api import inner_api_ns
from controllers.inner_api.wraps import knowledge_fs_inner_api_only
from fields.base import ResponseModel
from libs.exception import BaseHTTPException
from libs.helper import dump_response
from services.knowledge_fs.object_storage import (
KNOWLEDGE_FS_OBJECT_MAX_BYTES,
KnowledgeFSObjectList,
KnowledgeFSObjectMetadata,
KnowledgeFSObjectStorageChecksumError,
KnowledgeFSObjectStorageCorruptError,
KnowledgeFSObjectStorageError,
KnowledgeFSObjectStorageInvalidInputError,
KnowledgeFSObjectStorageService,
KnowledgeFSObjectStorageTooLargeError,
KnowledgeFSObjectStorageUnavailableError,
)
_METADATA_HEADER = "X-Knowledge-FS-Metadata"
_CHECKSUM_HEADER = "X-Knowledge-FS-Checksum-Sha256"
_CONTENT_TYPE_HEADER = "X-Knowledge-FS-Content-Type"
_MAX_ENCODED_METADATA_BYTES = 128 * 1024
_metadata_adapter = TypeAdapter(dict[str, str])
class KnowledgeFSObjectStorageHttpError(BaseHTTPException):
"""Safe HTTP representation of a KnowledgeFS storage boundary error."""
error_code = "knowledge_fs_object_storage_failed"
description = "KnowledgeFS object storage request failed."
code = HTTPStatus.INTERNAL_SERVER_ERROR
def __init__(self, *, error_code: str, description: str, status_code: HTTPStatus) -> None:
self.error_code = error_code
self.description = description
self.code = status_code
super().__init__(description)
class _CamelCaseResponse(ResponseModel):
model_config = ConfigDict(alias_generator=to_camel)
class KnowledgeFSObjectQuery(BaseModel):
model_config = ConfigDict(extra="forbid")
key: str = Field(description="Logical KnowledgeFS object key")
class KnowledgeFSObjectListQuery(BaseModel):
model_config = ConfigDict(extra="forbid")
prefix: str = Field(default="", description="Logical object-key prefix")
cursor: str | None = Field(default=None, description="Exclusive lexical key cursor")
limit: int = Field(default=100, ge=1, le=100, description="Maximum objects to return")
class KnowledgeFSObjectMetadataResponse(_CamelCaseResponse):
checksum_sha256_base64: str
content_type: str | None = None
key: str
metadata: dict[str, str]
size_bytes: int
class KnowledgeFSObjectListResponse(_CamelCaseResponse):
objects: list[KnowledgeFSObjectMetadataResponse]
next_cursor: str | None = None
class KnowledgeFSObjectHealthResponse(ResponseModel):
ok: bool
register_response_schema_models(
inner_api_ns,
KnowledgeFSObjectMetadataResponse,
KnowledgeFSObjectListResponse,
KnowledgeFSObjectHealthResponse,
)
@inner_api_ns.route("/knowledge-fs/storage/object")
class KnowledgeFSObjectApi(Resource):
"""Read, write, or delete one logical KnowledgeFS object."""
@knowledge_fs_inner_api_only
@inner_api_ns.doc(params=query_params_from_model(KnowledgeFSObjectQuery))
@inner_api_ns.response(
HTTPStatus.OK,
"Object stored",
inner_api_ns.models[KnowledgeFSObjectMetadataResponse.__name__],
)
def put(self) -> dict[str, object]:
try:
query = KnowledgeFSObjectQuery.model_validate(request.args.to_dict(flat=True))
metadata = _decode_metadata_header(request.headers.get(_METADATA_HEADER))
body = request.stream.read(KNOWLEDGE_FS_OBJECT_MAX_BYTES + 1)
if len(body) > KNOWLEDGE_FS_OBJECT_MAX_BYTES:
raise KnowledgeFSObjectStorageTooLargeError(f"object exceeds max bytes {KNOWLEDGE_FS_OBJECT_MAX_BYTES}")
result = KnowledgeFSObjectStorageService().put_object(
body=body,
checksum_sha256_base64=request.headers.get(_CHECKSUM_HEADER),
content_type=request.headers.get(_CONTENT_TYPE_HEADER),
key=query.key,
metadata=metadata,
)
except ValidationError as exc:
raise _invalid_request_error() from exc
except KnowledgeFSObjectStorageError as exc:
_raise_http_error(exc)
return _metadata_response(result)
@knowledge_fs_inner_api_only
@inner_api_ns.doc(params=query_params_from_model(KnowledgeFSObjectQuery))
@inner_api_ns.produces(["application/octet-stream"])
def get(self) -> Response:
try:
query = KnowledgeFSObjectQuery.model_validate(request.args.to_dict(flat=True))
service = KnowledgeFSObjectStorageService()
metadata = service.head_object(key=query.key)
if metadata is None:
raise _not_found_error()
body = service.load_stream(key=query.key)
if body is None:
raise _not_found_error()
except ValidationError as exc:
raise _invalid_request_error() from exc
except KnowledgeFSObjectStorageError as exc:
_raise_http_error(exc)
response = Response(
body,
content_type=metadata.content_type or "application/octet-stream",
)
response.content_length = metadata.size_bytes
response.headers[_CHECKSUM_HEADER] = metadata.checksum_sha256_base64
return response
@knowledge_fs_inner_api_only
@inner_api_ns.doc(params=query_params_from_model(KnowledgeFSObjectQuery))
@inner_api_ns.response(HTTPStatus.NO_CONTENT, "Object deleted")
def delete(self) -> tuple[str, int]:
try:
query = KnowledgeFSObjectQuery.model_validate(request.args.to_dict(flat=True))
KnowledgeFSObjectStorageService().delete_object(key=query.key)
except ValidationError as exc:
raise _invalid_request_error() from exc
except KnowledgeFSObjectStorageError as exc:
_raise_http_error(exc)
return "", HTTPStatus.NO_CONTENT
@inner_api_ns.route("/knowledge-fs/storage/object/metadata")
class KnowledgeFSObjectMetadataApi(Resource):
"""Read portable metadata for one logical KnowledgeFS object."""
@knowledge_fs_inner_api_only
@inner_api_ns.doc(params=query_params_from_model(KnowledgeFSObjectQuery))
@inner_api_ns.response(
HTTPStatus.OK,
"Object metadata",
inner_api_ns.models[KnowledgeFSObjectMetadataResponse.__name__],
)
def get(self) -> dict[str, object]:
try:
query = KnowledgeFSObjectQuery.model_validate(request.args.to_dict(flat=True))
result = KnowledgeFSObjectStorageService().head_object(key=query.key)
if result is None:
raise _not_found_error()
except ValidationError as exc:
raise _invalid_request_error() from exc
except KnowledgeFSObjectStorageError as exc:
_raise_http_error(exc)
return _metadata_response(result)
@inner_api_ns.route("/knowledge-fs/storage/objects")
class KnowledgeFSObjectListApi(Resource):
"""List logical KnowledgeFS objects with bounded keyset pagination."""
@knowledge_fs_inner_api_only
@inner_api_ns.doc(params=query_params_from_model(KnowledgeFSObjectListQuery))
@inner_api_ns.response(
HTTPStatus.OK,
"Object page",
inner_api_ns.models[KnowledgeFSObjectListResponse.__name__],
)
def get(self) -> dict[str, object]:
try:
query = KnowledgeFSObjectListQuery.model_validate(request.args.to_dict(flat=True))
result = KnowledgeFSObjectStorageService().list_objects(
cursor=query.cursor,
limit=query.limit,
prefix=query.prefix,
)
except ValidationError as exc:
raise _invalid_request_error() from exc
except KnowledgeFSObjectStorageError as exc:
_raise_http_error(exc)
return _list_response(result)
@inner_api_ns.route("/knowledge-fs/storage/health")
class KnowledgeFSObjectHealthApi(Resource):
"""Report whether Dify storage satisfies KnowledgeFS portable requirements."""
@knowledge_fs_inner_api_only
@inner_api_ns.response(
HTTPStatus.OK,
"Storage available",
inner_api_ns.models[KnowledgeFSObjectHealthResponse.__name__],
)
@inner_api_ns.response(HTTPStatus.SERVICE_UNAVAILABLE, "Storage unavailable")
def get(self) -> dict[str, bool] | tuple[dict[str, bool], int]:
if KnowledgeFSObjectStorageService().health():
return {"ok": True}
return {"ok": False}, HTTPStatus.SERVICE_UNAVAILABLE
def _decode_metadata_header(value: str | None) -> dict[str, str]:
if value is None:
return {}
if len(value.encode()) > _MAX_ENCODED_METADATA_BYTES:
raise KnowledgeFSObjectStorageInvalidInputError("object metadata header is too large")
try:
padding = "=" * (-len(value) % 4)
decoded = b64decode(value + padding, altchars=b"-_", validate=True)
return _metadata_adapter.validate_json(decoded)
except (BinasciiError, UnicodeEncodeError, ValidationError, json.JSONDecodeError) as exc:
raise KnowledgeFSObjectStorageInvalidInputError("object metadata header is invalid") from exc
def _metadata_response(metadata: KnowledgeFSObjectMetadata) -> dict[str, object]:
return dump_response(KnowledgeFSObjectMetadataResponse, metadata)
def _list_response(result: KnowledgeFSObjectList) -> dict[str, object]:
return dump_response(KnowledgeFSObjectListResponse, result)
def _raise_http_error(error: KnowledgeFSObjectStorageError) -> NoReturn:
if isinstance(error, KnowledgeFSObjectStorageTooLargeError):
raise KnowledgeFSObjectStorageHttpError(
error_code="knowledge_fs_object_too_large",
description="KnowledgeFS object exceeds the configured size limit.",
status_code=HTTPStatus.REQUEST_ENTITY_TOO_LARGE,
) from error
if isinstance(error, KnowledgeFSObjectStorageChecksumError):
raise KnowledgeFSObjectStorageHttpError(
error_code="knowledge_fs_object_checksum_mismatch",
description="KnowledgeFS object checksum does not match the request body.",
status_code=HTTPStatus.UNPROCESSABLE_ENTITY,
) from error
if isinstance(error, KnowledgeFSObjectStorageInvalidInputError):
raise _invalid_request_error() from error
if isinstance(error, KnowledgeFSObjectStorageCorruptError):
raise KnowledgeFSObjectStorageHttpError(
error_code="knowledge_fs_object_corrupt",
description="KnowledgeFS object metadata is inconsistent.",
status_code=HTTPStatus.BAD_GATEWAY,
) from error
if isinstance(error, KnowledgeFSObjectStorageUnavailableError):
raise KnowledgeFSObjectStorageHttpError(
error_code="knowledge_fs_object_storage_unavailable",
description="Dify object storage is unavailable for KnowledgeFS.",
status_code=HTTPStatus.SERVICE_UNAVAILABLE,
) from error
raise KnowledgeFSObjectStorageHttpError(
error_code="knowledge_fs_object_storage_failed",
description="KnowledgeFS object storage request failed.",
status_code=HTTPStatus.INTERNAL_SERVER_ERROR,
) from error
def _invalid_request_error() -> KnowledgeFSObjectStorageHttpError:
return KnowledgeFSObjectStorageHttpError(
error_code="knowledge_fs_object_storage_invalid_request",
description="KnowledgeFS object storage request is invalid.",
status_code=HTTPStatus.BAD_REQUEST,
)
def _not_found_error() -> KnowledgeFSObjectStorageHttpError:
return KnowledgeFSObjectStorageHttpError(
error_code="knowledge_fs_object_not_found",
description="KnowledgeFS object was not found.",
status_code=HTTPStatus.NOT_FOUND,
)
@@ -8,7 +8,6 @@ from controllers.inner_api.plugin.wraps import get_user_tenant, plugin_data
from controllers.inner_api.wraps import plugin_inner_api_only
from core.plugin.backwards_invocation.app import PluginAppBackwardsInvocation
from core.plugin.backwards_invocation.base import BaseBackwardsInvocationResponse
from core.plugin.backwards_invocation.datasource import PluginDatasourceBackwardsInvocation
from core.plugin.backwards_invocation.encrypt import PluginEncrypter
from core.plugin.backwards_invocation.model import PluginModelBackwardsInvocation
from core.plugin.backwards_invocation.node import PluginNodeBackwardsInvocation
@@ -16,12 +15,10 @@ from core.plugin.backwards_invocation.tool import PluginToolBackwardsInvocation
from core.plugin.entities.request import (
RequestFetchAppInfo,
RequestInvokeApp,
RequestInvokeDatasource,
RequestInvokeEncrypt,
RequestInvokeLLM,
RequestInvokeLLMWithStructuredOutput,
RequestInvokeModeration,
RequestInvokeMultimodalEmbedding,
RequestInvokeParameterExtractorNode,
RequestInvokeQuestionClassifierNode,
RequestInvokeRerank,
@@ -30,7 +27,6 @@ from core.plugin.entities.request import (
RequestInvokeTextEmbedding,
RequestInvokeTool,
RequestInvokeTTS,
RequestListModels,
RequestRequestDownloadFile,
RequestRequestUploadFile,
)
@@ -122,36 +118,6 @@ class PluginInvokeTextEmbeddingApi(Resource):
return jsonable_encoder(BaseBackwardsInvocationResponse(error=str(e)))
@inner_api_ns.route("/invoke/multimodal-embedding")
class PluginInvokeMultimodalEmbeddingApi(Resource):
@get_user_tenant
@setup_required
@plugin_inner_api_only
@plugin_data(payload_type=RequestInvokeMultimodalEmbedding)
@inner_api_ns.doc("plugin_invoke_multimodal_embedding")
@inner_api_ns.doc(description="Invoke multimodal embedding models through Dify model management")
@inner_api_ns.doc(
responses={
200: "Multimodal embedding successful",
401: "Unauthorized - invalid API key",
404: "Service not available",
}
)
def post(self, user_model: Account | EndUser, tenant_model: Tenant, payload: RequestInvokeMultimodalEmbedding):
try:
return jsonable_encoder(
BaseBackwardsInvocationResponse(
data=PluginModelBackwardsInvocation.invoke_multimodal_embedding(
user_id=user_model.id,
tenant=tenant_model,
payload=payload,
)
)
)
except Exception as e:
return jsonable_encoder(BaseBackwardsInvocationResponse(error=str(e)))
@inner_api_ns.route("/invoke/rerank")
class PluginInvokeRerankApi(Resource):
@get_user_tenant
@@ -178,70 +144,6 @@ class PluginInvokeRerankApi(Resource):
return jsonable_encoder(BaseBackwardsInvocationResponse(error=str(e)))
@inner_api_ns.route("/invoke/model-catalog")
class PluginModelCatalogApi(Resource):
@get_user_tenant
@setup_required
@plugin_inner_api_only
@plugin_data(payload_type=RequestListModels)
@inner_api_ns.doc("plugin_model_catalog")
@inner_api_ns.doc(description="List tenant-active models managed by Dify")
@inner_api_ns.doc(
responses={
200: "Model catalog lookup successful",
401: "Unauthorized - invalid API key",
404: "Service not available",
}
)
def post(self, user_model: Account | EndUser, tenant_model: Tenant, payload: RequestListModels):
try:
return jsonable_encoder(
BaseBackwardsInvocationResponse(
data=PluginModelBackwardsInvocation.list_models(
tenant_id=tenant_model.id,
user_id=user_model.id,
payload=payload,
)
)
)
except Exception as e:
return jsonable_encoder(BaseBackwardsInvocationResponse(error=str(e)))
@inner_api_ns.route("/invoke/datasource")
class PluginInvokeDatasourceApi(Resource):
"""Invoke an installed datasource with credentials resolved inside Dify."""
@get_user_tenant
@setup_required
@plugin_inner_api_only
@plugin_data(payload_type=RequestInvokeDatasource)
@inner_api_ns.doc("plugin_invoke_datasource")
@inner_api_ns.doc(description="Invoke datasource plugins through Dify credential management")
@inner_api_ns.doc(
responses={
200: "Datasource invocation successful (streaming response)",
401: "Unauthorized - invalid API key",
404: "Datasource provider, datasource, or credential not found",
}
)
def post(
self,
user_model: Account | EndUser,
tenant_model: Tenant,
payload: RequestInvokeDatasource,
):
response = PluginDatasourceBackwardsInvocation.invoke(
user_id=user_model.id,
tenant=tenant_model,
payload=payload,
)
return length_prefixed_response(
0xF,
PluginDatasourceBackwardsInvocation.convert_to_event_stream(response),
)
@inner_api_ns.route("/invoke/tts")
class PluginInvokeTTSApi(Resource):
@get_user_tenant
+7 -17
View File
@@ -1,6 +1,5 @@
from collections.abc import Callable
from functools import wraps
from uuid import UUID
from flask import current_app, request
from flask_login import user_logged_in
@@ -56,15 +55,14 @@ def get_user(tenant_id: str, user_id: str | None) -> EndUser:
# session_id, id is auto-generated) and a fresh EndUser
# was created per call, breaking multi-turn chat
# continuation (see #36736).
if _is_uuid(user_id):
user_model = session.scalar(
select(EndUser)
.where(
EndUser.id == user_id,
EndUser.tenant_id == tenant_id,
)
.limit(1)
user_model = session.scalar(
select(EndUser)
.where(
EndUser.id == user_id,
EndUser.tenant_id == tenant_id,
)
.limit(1)
)
if user_model is None:
user_model = session.scalar(
select(EndUser)
@@ -92,14 +90,6 @@ def get_user(tenant_id: str, user_id: str | None) -> EndUser:
return user_model
def _is_uuid(value: str) -> bool:
try:
UUID(value)
except ValueError:
return False
return True
def get_user_tenant[**P, R](view_func: Callable[P, R]) -> Callable[P, R]:
@wraps(view_func)
def decorated_view(*args: P.args, **kwargs: P.kwargs) -> R:
@@ -0,0 +1,39 @@
from flask_restx import Resource
from pydantic import BaseModel, ConfigDict, Field
from controllers.common.schema import register_schema_model
from controllers.console.wraps import setup_required
from controllers.inner_api import inner_api_ns
from controllers.inner_api.wraps import enterprise_inner_api_only
from core.plugin.plugin_service import PluginService
class InvalidatePluginModelProvidersCachePayload(BaseModel):
model_config = ConfigDict(extra="forbid")
tenant_ids: list[str] = Field(default_factory=list, description="Workspace ids whose cache should be invalidated")
register_schema_model(inner_api_ns, InvalidatePluginModelProvidersCachePayload)
@inner_api_ns.route("/enterprise/workspace/plugin-model-providers/invalidate")
class EnterprisePluginModelProvidersCacheInvalidate(Resource):
@setup_required
@enterprise_inner_api_only
@inner_api_ns.doc(
"enterprise_invalidate_plugin_model_providers_cache",
responses={
200: "Cache invalidated",
400: "Invalid request",
401: "Unauthorized - invalid API key",
},
)
@inner_api_ns.expect(inner_api_ns.models[InvalidatePluginModelProvidersCachePayload.__name__])
def post(self):
args = InvalidatePluginModelProvidersCachePayload.model_validate(inner_api_ns.payload or {})
for tenant_id in args.tenant_ids:
PluginService.invalidate_plugin_model_providers_cache(tenant_id)
return {"result": "success"}, 200
-6
View File
@@ -107,9 +107,3 @@ def agent_inner_api_only[**P, R](view: Callable[P, R]) -> Callable[P, R]:
"""
return plugin_inner_api_only(view)
def knowledge_fs_inner_api_only[**P, R](view: Callable[P, R]) -> Callable[P, R]:
"""Authenticate the trusted KnowledgeFS process on the shared inner bridge."""
return plugin_inner_api_only(view)
+2 -1
View File
@@ -8,6 +8,7 @@ from werkzeug.exceptions import InternalServerError
from configs import dify_config
from core.rbac import RBACPermission, RBACResourceScope
from enums.deployment_edition import DeploymentEdition
from libs.oauth_bearer import Scope, TokenType
from models.account import Account, Tenant, TenantAccountRole
from models.model import App, EndUser
@@ -26,7 +27,7 @@ class CallerKind(StrEnum):
def current_edition() -> Edition:
if dify_config.EDITION == "CLOUD":
if dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD:
return Edition.SAAS
if dify_config.ENTERPRISE_ENABLED:
return Edition.EE
-2
View File
@@ -38,7 +38,6 @@ from .dataset import (
)
from .dataset.rag_pipeline import rag_pipeline_workflow
from .end_user import end_user
from .knowledge_fs import resources as knowledge_fs_resources
from .workspace import models
__all__ = [
@@ -55,7 +54,6 @@ __all__ = [
"hit_testing",
"human_input_form",
"index",
"knowledge_fs_resources",
"message",
"metadata",
"models",
@@ -8,14 +8,18 @@ paused human input forms in workflow/chatflow runs.
import json
import logging
from collections.abc import Sequence
from http import HTTPStatus
from typing import Any
from flask import Response
from flask import Response, abort, request
from flask_restx import Resource
from pydantic import ConfigDict, Field
from werkzeug.exceptions import BadRequest, NotFound
from controllers.common.human_input import HumanInputFormSubmitPayload, stringify_form_default_values
from controllers.common.human_input_v2_contracts import FormDefinitionResponse as HumanInputV2FormDefinitionResponse
from controllers.common.human_input_v2_contracts import FormSubmitResponse as HumanInputV2FormSubmitResponse
from controllers.common.human_input_v2_contracts import HumanInputV2ServiceFormSubmitRequest, ServiceFormQuery
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.service_api import service_api_ns
from controllers.service_api.schema import expect_with_user
@@ -43,8 +47,19 @@ class HumanInputFormSubmitResponse(ResponseModel):
model_config = ConfigDict(extra="forbid")
register_schema_models(service_api_ns, HumanInputFormSubmitPayload)
register_response_schema_models(service_api_ns, HumanInputFormDefinitionResponse, HumanInputFormSubmitResponse)
register_schema_models(
service_api_ns,
HumanInputFormSubmitPayload,
HumanInputV2ServiceFormSubmitRequest,
ServiceFormQuery,
)
register_response_schema_models(
service_api_ns,
HumanInputFormDefinitionResponse,
HumanInputFormSubmitResponse,
HumanInputV2FormDefinitionResponse,
HumanInputV2FormSubmitResponse,
)
def _jsonify_form_definition(form: Form, *, inputs: Sequence[FormInputConfig] = ()) -> Response:
@@ -188,3 +203,33 @@ class WorkflowHumanInputFormApi(Resource):
raise NotFound("Form not found")
return {}, 200
@service_api_ns.route("/form/human-input/<string:form_token>")
class WorkflowHumanInputV2FormApi(Resource):
"""Trusted Service API stub for Human Input v2 forms.
This route does not alias the v1 handler. The implementation must resolve a
v2 form and reject v1 form tokens before reading or submitting it.
"""
@service_api_ns.response(
200,
"Form retrieved successfully",
service_api_ns.models[HumanInputV2FormDefinitionResponse.__name__],
)
@validate_app_token(fetch_user_arg=FetchUserArg(fetch_from=WhereisUserArg.QUERY, required=True))
def get(self, app_model: App, end_user: EndUser, form_token: str):
ServiceFormQuery.model_validate(request.args.to_dict(flat=True))
abort(HTTPStatus.NOT_IMPLEMENTED, "Human Input v2 Service API stub endpoint is not implemented yet.")
@service_api_ns.expect(service_api_ns.models[HumanInputV2ServiceFormSubmitRequest.__name__])
@service_api_ns.response(
200,
"Form submitted successfully",
service_api_ns.models[HumanInputV2FormSubmitResponse.__name__],
)
@validate_app_token(fetch_user_arg=FetchUserArg(fetch_from=WhereisUserArg.JSON, required=True))
def post(self, app_model: App, end_user: EndUser, form_token: str):
HumanInputV2ServiceFormSubmitRequest.model_validate(service_api_ns.payload or {})
abort(HTTPStatus.NOT_IMPLEMENTED, "Human Input v2 Service API stub endpoint is not implemented yet.")
@@ -1,5 +0,0 @@
"""KnowledgeFS-specific Service API authenticated by resource credentials."""
from . import resources
__all__ = ["resources"]
@@ -1,49 +0,0 @@
"""Stable KnowledgeFS Service API error contract."""
from libs.exception import BaseHTTPException
class KnowledgeFSInvalidCredentialHTTPError(BaseHTTPException):
error_code = "knowledge_fs_invalid_credential"
description = "Invalid KnowledgeFS service credential."
code = 401
class KnowledgeFSServiceOperationUnavailableHTTPError(BaseHTTPException):
error_code = "knowledge_fs_operation_unavailable"
description = "KnowledgeFS operation is not available."
code = 503
class KnowledgeFSServiceUpstreamUnavailableHTTPError(BaseHTTPException):
error_code = "knowledge_fs_upstream_unavailable"
description = "KnowledgeFS is unavailable."
code = 502
class KnowledgeFSServiceInvalidRequestHTTPError(BaseHTTPException):
error_code = "knowledge_fs_invalid_request"
description = "KnowledgeFS request is invalid."
code = 400
class KnowledgeFSServiceRateLimitHTTPError(BaseHTTPException):
error_code = "knowledge_fs_rate_limit_exceeded"
description = "KnowledgeFS operation rate limit exceeded."
code = 429
class KnowledgeFSServiceQuotaExceededHTTPError(BaseHTTPException):
error_code = "knowledge_fs_quota_exceeded"
description = "KnowledgeFS operation quota exceeded."
code = 403
__all__ = [
"KnowledgeFSInvalidCredentialHTTPError",
"KnowledgeFSServiceInvalidRequestHTTPError",
"KnowledgeFSServiceOperationUnavailableHTTPError",
"KnowledgeFSServiceQuotaExceededHTTPError",
"KnowledgeFSServiceRateLimitHTTPError",
"KnowledgeFSServiceUpstreamUnavailableHTTPError",
]
File diff suppressed because it is too large Load Diff
+2
View File
@@ -25,6 +25,7 @@ from . import (
forgot_password,
human_input_file_upload,
human_input_form,
human_input_form_access_request,
login,
message,
passport,
@@ -49,6 +50,7 @@ __all__ = [
"forgot_password",
"human_input_file_upload",
"human_input_form",
"human_input_form_access_request",
"login",
"message",
"passport",
@@ -0,0 +1,67 @@
"""Public web Human Input v2 form stubs.
The hyphenated v2 routes are intentionally separate from the legacy underscored
Human Input form routes. Each runtime path must reject tokens owned by the other
version when the service implementation is added.
"""
from __future__ import annotations
from http import HTTPStatus
from flask import abort
from flask_restx import Resource
from controllers.common.human_input_v2_contracts import (
FormAccessRequestResponse,
FormDefinitionResponse,
FormSubmitResponse,
FormUploadTokenResponse,
HumanInputV2FormSubmitRequest,
)
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.web import web_ns
register_schema_models(web_ns, HumanInputV2FormSubmitRequest)
register_response_schema_models(
web_ns,
FormAccessRequestResponse,
FormDefinitionResponse,
FormSubmitResponse,
FormUploadTokenResponse,
)
def _raise_stub_not_implemented() -> None:
abort(HTTPStatus.NOT_IMPLEMENTED, "Human Input v2 form stub endpoint is not implemented yet.")
@web_ns.route("/form/human-input/<string:form_token>")
class HumanInputV2FormApi(Resource):
"""Read or submit a Human Input v2 form without sharing v1 submission logic."""
@web_ns.response(200, "Success", web_ns.models[FormDefinitionResponse.__name__])
def get(self, form_token: str):
_raise_stub_not_implemented()
@web_ns.expect(web_ns.models[HumanInputV2FormSubmitRequest.__name__])
@web_ns.response(200, "Success", web_ns.models[FormSubmitResponse.__name__])
def post(self, form_token: str):
HumanInputV2FormSubmitRequest.model_validate(web_ns.payload or {})
_raise_stub_not_implemented()
@web_ns.route("/form/human-input/<string:form_token>/upload-token")
class HumanInputV2FormUploadTokenApi(Resource):
"""Issue an upload token for an active Human Input v2 form."""
@web_ns.response(200, "Success", web_ns.models[FormUploadTokenResponse.__name__])
def post(self, form_token: str):
_raise_stub_not_implemented()
@web_ns.route("/form/human-input/<string:form_token>/access-request")
class FormAccessRequestApi(Resource):
@web_ns.response(200, "Success", web_ns.models[FormAccessRequestResponse.__name__])
def post(self, form_token: str):
_raise_stub_not_implemented()
+6
View File
@@ -11,6 +11,7 @@ from werkzeug.exceptions import BadRequest, NotFound, Unauthorized
from constants import HEADER_NAME_APP_CODE
from controllers.web.error import WebAppAuthAccessDeniedError, WebAppAuthRequiredError
from core.logging.context import set_identity_context
from extensions.ext_database import db
from libs.passport import PassportService
from libs.token import extract_webapp_passport
@@ -28,6 +29,11 @@ def validate_jwt_token[**P, R](
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs) -> R:
app_model, end_user = decode_jwt_token()
set_identity_context(
tenant_id=end_user.tenant_id,
user_id=end_user.id,
user_type=end_user.type or "end_user",
)
return view(app_model, end_user, *args, **kwargs)
return decorated
-12
View File
@@ -14,10 +14,7 @@ from core.app.apps.base_app_queue_manager import AppQueueManager
from core.app.apps.base_app_runner import AppRunner
from core.app.entities.app_invoke_entities import (
AgentChatAppGenerateEntity,
DifyRunContext,
InvokeFrom,
ModelConfigWithCredentialsEntity,
UserFrom,
)
from core.app.file_access import DatabaseFileAccessController
from core.callback_handler.agent_tool_callback_handler import DifyAgentCallbackHandler
@@ -151,15 +148,6 @@ class BaseAgentRunner(AppRunner):
user_id=self.user_id,
invoke_from=self.application_generate_entity.invoke_from,
)
invoke_from = self.application_generate_entity.invoke_from
if isinstance(invoke_from, InvokeFrom):
tool_entity.runtime.dify_run_context = DifyRunContext(
tenant_id=self.tenant_id,
app_id=self.app_config.app_id,
user_id=self.user_id,
user_from=(UserFrom.ACCOUNT if invoke_from.runs_as_account() else UserFrom.END_USER),
invoke_from=invoke_from,
)
assert tool_entity.entity.description
message_tool = PromptMessageTool(
name=tool.tool_name,
+93
View File
@@ -0,0 +1,93 @@
"""Publication visibility rules for calling roster Agents from Workflows.
``Agent.active_config_is_published`` describes whether the editable shared
draft still matches the active snapshot. It is false both before the first
publish and after a published Agent receives new draft edits, so it must not be
used as a runtime availability flag. App-backed Agents are callable from a
Workflow only when the active snapshot has a revision created by a
publish-visible operation. Direct roster Agents are publish-visible by
construction and only need an active snapshot.
"""
from sqlalchemy import and_, or_, select
from sqlalchemy.orm import Session
from sqlalchemy.sql.elements import ColumnElement
from models.agent import Agent, AgentConfigRevision, AgentConfigRevisionOperation, AgentScope, AgentSource
PUBLISH_VISIBLE_APP_BACKED_REVISION_OPERATIONS = frozenset(
{
AgentConfigRevisionOperation.PUBLISH_DRAFT,
AgentConfigRevisionOperation.SAVE_CURRENT_VERSION,
AgentConfigRevisionOperation.SAVE_NEW_VERSION,
AgentConfigRevisionOperation.SAVE_NEW_AGENT,
AgentConfigRevisionOperation.SAVE_TO_ROSTER,
AgentConfigRevisionOperation.RESTORE_VERSION,
}
)
def workflow_callable_active_snapshot_filter() -> ColumnElement[bool]:
"""Return the SQL predicate for an Agent with a Workflow-callable active snapshot.
The caller remains responsible for tenant, roster scope, lifecycle status,
and model configuration filters. The correlated revision lookup makes the
predicate safe to compose into roster pagination queries.
"""
app_backed_agent = or_(
Agent.source == AgentSource.AGENT_APP,
and_(
Agent.source == AgentSource.IMPORTED,
Agent.scope == AgentScope.ROSTER,
Agent.app_id.is_not(None),
),
)
publish_visible_revision_exists = (
select(AgentConfigRevision.id)
.where(
AgentConfigRevision.tenant_id == Agent.tenant_id,
AgentConfigRevision.agent_id == Agent.id,
AgentConfigRevision.current_snapshot_id == Agent.active_config_snapshot_id,
AgentConfigRevision.operation.in_(PUBLISH_VISIBLE_APP_BACKED_REVISION_OPERATIONS),
)
.correlate(Agent)
.exists()
)
return and_(
Agent.active_config_snapshot_id.is_not(None),
or_(
~app_backed_agent,
publish_visible_revision_exists,
),
)
def agent_has_workflow_callable_active_snapshot(*, session: Session, agent: Agent) -> bool:
"""Return whether ``agent`` has an active snapshot visible to Workflow.
This object-level form is useful after ownership and lifecycle checks have
already loaded an Agent. It intentionally ignores dirty draft state so a
previously published snapshot keeps serving while later edits remain
unpublished.
"""
if not agent.active_config_snapshot_id:
return False
is_app_backed = agent.source == AgentSource.AGENT_APP or (
agent.source == AgentSource.IMPORTED and agent.scope == AgentScope.ROSTER and agent.app_id is not None
)
if not is_app_backed:
return True
return bool(
session.scalar(
select(AgentConfigRevision.id)
.where(
AgentConfigRevision.tenant_id == agent.tenant_id,
AgentConfigRevision.agent_id == agent.id,
AgentConfigRevision.current_snapshot_id == agent.active_config_snapshot_id,
AgentConfigRevision.operation.in_(PUBLISH_VISIBLE_APP_BACKED_REVISION_OPERATIONS),
)
.limit(1)
)
)
+14 -29
View File
@@ -683,42 +683,27 @@ class AgentAppGenerator(MessageBasedAppGenerator):
if draft_type == AgentConfigDraftType.DEBUG_BUILD.value
else AgentConfigDraftType.DRAFT
)
if effective_draft_type == AgentConfigDraftType.DRAFT:
from services.agent.composer_service import AgentComposerService
return AgentComposerService.get_or_create_normal_agent_draft(
session=session,
tenant_id=tenant_id,
agent=agent,
created_by=agent.updated_by or agent.created_by,
)
if not account_id:
raise AgentAppGeneratorError("Build draft requires an account user")
stmt = select(AgentConfigDraft).where(
AgentConfigDraft.tenant_id == tenant_id,
AgentConfigDraft.agent_id == agent.id,
AgentConfigDraft.draft_type == effective_draft_type,
AgentConfigDraft.draft_type == AgentConfigDraftType.DEBUG_BUILD,
AgentConfigDraft.account_id == account_id,
)
if effective_draft_type == AgentConfigDraftType.DEBUG_BUILD:
if not account_id:
raise AgentAppGeneratorError("Build draft requires an account user")
stmt = stmt.where(AgentConfigDraft.account_id == account_id)
else:
stmt = stmt.where(AgentConfigDraft.account_id.is_(None))
draft = session.scalar(stmt.order_by(AgentConfigDraft.updated_at.desc()).limit(1))
if draft is not None:
return draft
if effective_draft_type == AgentConfigDraftType.DEBUG_BUILD:
raise AgentAppGeneratorError("Agent build draft not found")
_, snapshot, agent_soul = AgentAppGenerator._resolve_agent_by_id(
tenant_id=tenant_id,
agent_id=agent.id,
snapshot_id=agent.active_config_snapshot_id,
session=session,
)
draft = AgentConfigDraft(
tenant_id=tenant_id,
agent_id=agent.id,
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
draft_owner_key="",
base_snapshot_id=snapshot.id,
config_snapshot=agent_soul,
created_by=agent.created_by,
updated_by=agent.updated_by,
)
session.add(draft)
session.flush()
return draft
raise AgentAppGeneratorError("Agent build draft not found")
@staticmethod
def _resolve_agent_by_id(
@@ -159,7 +159,6 @@ class AgentAppRuntimeRequestBuilder:
user_from=cast(DifyExecutionContextUserFrom, context.dify_context.user_from.value),
invoke_from=cast(DifyExecutionContextInvokeFrom, context.dify_context.invoke_from.value),
agent_mode="agent_app",
trace_id=context.dify_context.trace_session_id,
),
# ENG-616: expand slash-menu mention tokens to canonical names so
# no frontend-internal {{#…#}} marker ever reaches the model.
+3 -3
View File
@@ -88,7 +88,7 @@ from graphon.graph_events import (
from graphon.runtime import GraphRuntimeState, VariablePool
from graphon.variable_loader import DUMMY_VARIABLE_LOADER, VariableLoader, load_into_variable_pool
from models.workflow import Workflow
from tasks.mail_human_input_delivery_task import dispatch_human_input_email_task
from tasks.mail_human_input_delivery_task import dispatch_human_input_form_delivery_task
logger = logging.getLogger(__name__)
@@ -715,12 +715,12 @@ class WorkflowBasedAppRunner:
if not reason.form_id:
continue
try:
dispatch_human_input_email_task.apply_async(
dispatch_human_input_form_delivery_task.apply_async(
kwargs={"form_id": reason.form_id, "node_title": reason.node_title},
queue="mail",
)
except Exception: # pragma: no cover - defensive logging
logger.exception("Failed to enqueue human input email task for form %s", reason.form_id)
logger.exception("Failed to enqueue human input form delivery task for form %s", reason.form_id)
def _publish_event(self, event: AppQueueEvent):
self._queue_manager.publish(event, PublishFrom.APPLICATION_MANAGER)
@@ -38,11 +38,6 @@ class DatasourcePluginProviderController(ABC):
):
raise ToolProviderCredentialValidationError("Invalid credentials")
def validate_credentials(self, user_id: str, credentials: dict[str, Any]) -> None:
"""Validate credential shape and value against this installed provider declaration."""
self.validate_credentials_format(credentials)
self._validate_credentials(user_id, credentials)
@property
def provider_type(self) -> DatasourceProviderType:
"""
+2 -1
View File
@@ -6,6 +6,7 @@ from pydantic import BaseModel
from configs import dify_config
from core.entities import DEFAULT_PLUGIN_ID
from core.entities.provider_entities import ProviderQuotaType, QuotaUnit, RestrictModel
from enums.deployment_edition import DeploymentEdition
from graphon.model_runtime.entities.model_entities import ModelType
@@ -49,7 +50,7 @@ class HostingConfiguration:
self.moderation_config = None
def init_app(self, app: Flask):
if dify_config.EDITION != "CLOUD":
if dify_config.DEPLOYMENT_EDITION != DeploymentEdition.CLOUD:
return
self.provider_map[f"{DEFAULT_PLUGIN_ID}/azure_openai/azure_openai"] = self.init_azure_openai()
View File
@@ -0,0 +1,236 @@
"""Human Input v2 recipient planning, Form, and OTP aggregate boundaries.
The package owns separate Form and OTP aggregate boundaries plus grant,
delivery, and upload facts. It exposes domain and persistence ports without
importing transport, provider, database-session, or ORM concerns.
"""
from .delivery import (
ConsoleEndpointConfiguration,
DeliveryAttempt,
DeliveryEndpoint,
DeliveryEndpointConfiguration,
EmailEndpointConfiguration,
EmailProviderConfiguration,
EndpointAccessCapability,
IMEndpointConfiguration,
UploadCapability,
UploadCapabilityRef,
UploadFileAssociation,
WebEndpointConfiguration,
)
from .form import (
FormCreation,
FormInactiveReason,
FormSnapshotIdentifierFactory,
FormState,
FrozenFormAction,
FrozenFormDefinition,
HumanInputForm,
InactiveFormState,
InvalidApproverGrantError,
InvalidSelectedActionError,
SubmissionTransitionDecision,
WaitingFormState,
)
from .frozen_values import FrozenJSONArray, FrozenJSONObject, JSONPrimitive
from .grants import ApproverGrant, ApproverGrantRef, DeliveryEndpointRef, FormRef, OTPChallengeRef
from .otp import (
Clock,
ContactOTPSubject,
CurrentEmailOTPIdentity,
EmailAddressOTPSubject,
EmailOTPProofAuthorizationDecision,
EmailOTPSubject,
OTPChallenge,
OTPChallengePublicPrimitive,
OTPChallengeRejectionReason,
OTPChallengeRepository,
OTPChallengeState,
OTPCodeHash,
OTPCodeHasher,
OTPReplacementDecision,
OTPVerificationDecision,
VerifiedEmailOTPProof,
VerifiedEmailOTPProofPrimitive,
authorize_email_otp_proof,
)
from .ports import FormDefinitionProjection, FormDeliveryProjection, FormRepository
from .recipient_resolution import (
ApprovalSubject,
CanonicalSubjectKey,
ConsoleEndpointPlan,
ContactApprovalSubject,
ContactInitiatorSnapshot,
DebugRecipientReplacement,
DeliveryCapabilitySnapshot,
DeliveryEndpointPlan,
EmailAddressApprovalSubject,
EmailEndpointPlan,
EndUserApprovalSubject,
EndUserInitiatorSnapshot,
IMEndpointPlan,
MatchedRecipientSource,
RecipientRejectionReason,
RecipientResolutionFailureReason,
RecipientResolver,
RecipientSourceKind,
RejectedRecipient,
ResolvedApprovalPlan,
ResolvedApprover,
SubjectSnapshot,
WebEndpointPlan,
)
from .recipient_specifications import (
ContactRecipientSpecification,
CurrentInitiatorRecipientSpecification,
DynamicEmailRecipientSpecification,
DynamicRecipientValue,
OneTimeEmailRecipientSpecification,
RecipientSpecification,
RecipientSpecificationKind,
UnsupportedDynamicRecipientValue,
WorkflowRecipientSpecificationAdapter,
)
from .submission_authorization import (
AccountSubmissionActor,
AuthorizationContext,
AuthorizedSubmission,
CurrentContactAuthorizationFacts,
CurrentEndUserAuthorizationFacts,
CurrentIMAuthorizationFacts,
EmailAddressSubmissionActor,
EndUserSubmissionActor,
SubmissionActor,
SubmissionAuthorizationDecision,
SubmissionAuthorizationRejection,
SubmissionAuthorizer,
VerifiedAccountSessionProof,
VerifiedIMIdentityProof,
VerifiedSubmissionProof,
VerifiedTrustedEndUserProof,
)
from .submission_ports import (
AuthorizedSubmissionCommit,
RetryableSubmissionPersistenceError,
SubmissionAttemptScope,
SubmissionCommitResult,
SubmissionCommitStatus,
SubmissionRepository,
SubmissionTransaction,
)
from .submission_records import FormAuthorizationAuditEvent, FormAuthorizationAuditEventType, FormSubmission
__all__ = [
"AccountSubmissionActor",
"ApprovalSubject",
"ApproverGrant",
"ApproverGrantRef",
"AuthorizationContext",
"AuthorizedSubmission",
"AuthorizedSubmissionCommit",
"CanonicalSubjectKey",
"Clock",
"ConsoleEndpointConfiguration",
"ConsoleEndpointPlan",
"ContactApprovalSubject",
"ContactInitiatorSnapshot",
"ContactOTPSubject",
"ContactRecipientSpecification",
"CurrentContactAuthorizationFacts",
"CurrentEmailOTPIdentity",
"CurrentEndUserAuthorizationFacts",
"CurrentIMAuthorizationFacts",
"CurrentInitiatorRecipientSpecification",
"DebugRecipientReplacement",
"DeliveryAttempt",
"DeliveryCapabilitySnapshot",
"DeliveryEndpoint",
"DeliveryEndpointConfiguration",
"DeliveryEndpointPlan",
"DeliveryEndpointRef",
"DynamicEmailRecipientSpecification",
"DynamicRecipientValue",
"EmailAddressApprovalSubject",
"EmailAddressOTPSubject",
"EmailAddressSubmissionActor",
"EmailEndpointConfiguration",
"EmailEndpointPlan",
"EmailOTPProofAuthorizationDecision",
"EmailOTPSubject",
"EmailProviderConfiguration",
"EndUserApprovalSubject",
"EndUserInitiatorSnapshot",
"EndUserSubmissionActor",
"EndpointAccessCapability",
"FormAuthorizationAuditEvent",
"FormAuthorizationAuditEventType",
"FormCreation",
"FormDefinitionProjection",
"FormDeliveryProjection",
"FormInactiveReason",
"FormRef",
"FormRepository",
"FormSnapshotIdentifierFactory",
"FormState",
"FormSubmission",
"FrozenFormAction",
"FrozenFormDefinition",
"FrozenJSONArray",
"FrozenJSONObject",
"HumanInputForm",
"IMEndpointConfiguration",
"IMEndpointPlan",
"InactiveFormState",
"InvalidApproverGrantError",
"InvalidSelectedActionError",
"JSONPrimitive",
"MatchedRecipientSource",
"OTPChallenge",
"OTPChallengePublicPrimitive",
"OTPChallengeRef",
"OTPChallengeRejectionReason",
"OTPChallengeRepository",
"OTPChallengeState",
"OTPCodeHash",
"OTPCodeHasher",
"OTPReplacementDecision",
"OTPVerificationDecision",
"OneTimeEmailRecipientSpecification",
"RecipientRejectionReason",
"RecipientResolutionFailureReason",
"RecipientResolver",
"RecipientSourceKind",
"RecipientSpecification",
"RecipientSpecificationKind",
"RejectedRecipient",
"ResolvedApprovalPlan",
"ResolvedApprover",
"RetryableSubmissionPersistenceError",
"SubjectSnapshot",
"SubmissionActor",
"SubmissionAttemptScope",
"SubmissionAuthorizationDecision",
"SubmissionAuthorizationRejection",
"SubmissionAuthorizer",
"SubmissionCommitResult",
"SubmissionCommitStatus",
"SubmissionRepository",
"SubmissionTransaction",
"SubmissionTransitionDecision",
"UnsupportedDynamicRecipientValue",
"UploadCapability",
"UploadCapabilityRef",
"UploadFileAssociation",
"VerifiedAccountSessionProof",
"VerifiedEmailOTPProof",
"VerifiedEmailOTPProofPrimitive",
"VerifiedIMIdentityProof",
"VerifiedSubmissionProof",
"VerifiedTrustedEndUserProof",
"WaitingFormState",
"WebEndpointConfiguration",
"WebEndpointPlan",
"WorkflowRecipientSpecificationAdapter",
"authorize_email_otp_proof",
]
@@ -0,0 +1,287 @@
"""Frozen delivery, provider, endpoint-token, and upload facts.
Endpoints describe where a form can be delivered or interacted with. Their
tokens are scoped capabilities only; this module deliberately exposes no actor
or verified-proof conversion. Delivery attempts are append-only diagnostics and
cannot mutate :class:`HumanInputForm` lifecycle state.
"""
from __future__ import annotations
from dataclasses import dataclass
from hashlib import sha256
from typing import assert_never
from core.human_input_v2.entities import (
EmailProviderType,
HumanInputDeliveryAttemptStatus,
HumanInputDeliveryChannel,
IMProvider,
)
from core.human_input_v2.shared import (
AccountId,
AppId,
DeliveryAttemptId,
DeliveryEndpointId,
EmailProviderId,
IMBindingId,
IMIdentityId,
IntegrationId,
NormalizedEmail,
UploadCapabilityId,
UploadFileAssociationId,
UtcTimestamp,
WorkspaceId,
)
from .frozen_values import FrozenJSONObject
from .grants import ApproverGrantRef, DeliveryEndpointRef
from .recipient_resolution import (
ConsoleEndpointPlan,
DeliveryEndpointPlan,
EmailEndpointPlan,
IMEndpointPlan,
WebEndpointPlan,
)
def _validate_sha256(value: str, *, label: str) -> None:
if len(value) != 64 or any(character not in "0123456789abcdef" for character in value):
raise ValueError(f"{label} must be a lower-case SHA-256 digest")
@dataclass(frozen=True, slots=True)
class EndpointAccessCapability:
"""Hashed endpoint interaction capability that carries no identity proof."""
endpoint_ref: DeliveryEndpointRef
token_hash: str
def __post_init__(self) -> None:
_validate_sha256(self.token_hash, label="endpoint access token hash")
@dataclass(frozen=True, slots=True)
class EmailEndpointConfiguration:
"""Frozen Email delivery address."""
email_address: NormalizedEmail
@property
def channel(self) -> HumanInputDeliveryChannel:
return HumanInputDeliveryChannel.EMAIL
@dataclass(frozen=True, slots=True)
class IMEndpointConfiguration:
"""Credential-free IM interaction snapshot owned by one integration."""
integration_id: IntegrationId
provider: IMProvider
provider_tenant_id: str
identity_id: IMIdentityId
binding_id: IMBindingId | None
provider_user_id: str
def __post_init__(self) -> None:
if not self.provider_tenant_id or not self.provider_user_id:
raise ValueError("IM endpoint provider identities must not be blank")
@property
def channel(self) -> HumanInputDeliveryChannel:
return HumanInputDeliveryChannel.IM
@dataclass(frozen=True, slots=True)
class WebEndpointConfiguration:
"""Public or trusted-app web interaction surface."""
@property
def channel(self) -> HumanInputDeliveryChannel:
return HumanInputDeliveryChannel.WEB
@dataclass(frozen=True, slots=True)
class ConsoleEndpointConfiguration:
"""Authenticated console interaction surface."""
@property
def channel(self) -> HumanInputDeliveryChannel:
return HumanInputDeliveryChannel.CONSOLE
type DeliveryEndpointConfiguration = (
EmailEndpointConfiguration | IMEndpointConfiguration | WebEndpointConfiguration | ConsoleEndpointConfiguration
)
@dataclass(frozen=True, slots=True)
class DeliveryEndpoint:
"""Historical endpoint snapshot distinct from its approver grant."""
ref: DeliveryEndpointRef
configuration: DeliveryEndpointConfiguration
address_hash: str
access_capability: EndpointAccessCapability | None
created_at: UtcTimestamp
updated_at: UtcTimestamp
def __post_init__(self) -> None:
_validate_sha256(self.address_hash, label="endpoint address hash")
if self.access_capability is not None and self.access_capability.endpoint_ref != self.ref:
raise ValueError("endpoint access capability owner does not match the endpoint")
@property
def id(self) -> DeliveryEndpointId:
return self.ref.endpoint_id
@property
def grant_ref(self) -> ApproverGrantRef:
return self.ref.grant_ref
@property
def channel(self) -> HumanInputDeliveryChannel:
return self.configuration.channel
@classmethod
def from_plan(
cls,
*,
endpoint_id: DeliveryEndpointId,
grant_ref: ApproverGrantRef,
endpoint_plan: DeliveryEndpointPlan,
access_capability: EndpointAccessCapability | None,
now: UtcTimestamp,
) -> DeliveryEndpoint:
endpoint_ref = grant_ref.endpoint(endpoint_id)
configuration: DeliveryEndpointConfiguration
canonical_address: str
match endpoint_plan:
case EmailEndpointPlan(email_address=email_address):
configuration = EmailEndpointConfiguration(email_address)
canonical_address = f"email:{email_address}"
case IMEndpointPlan(
integration_id=integration_id,
provider=provider,
provider_tenant_id=provider_tenant_id,
identity_id=identity_id,
binding_id=binding_id,
provider_user_id=provider_user_id,
):
configuration = IMEndpointConfiguration(
integration_id=integration_id,
provider=provider,
provider_tenant_id=provider_tenant_id,
identity_id=identity_id,
binding_id=binding_id,
provider_user_id=provider_user_id,
)
canonical_address = f"im:{integration_id}:{provider.value}:{provider_user_id}"
case WebEndpointPlan():
configuration = WebEndpointConfiguration()
canonical_address = "web"
case ConsoleEndpointPlan():
configuration = ConsoleEndpointConfiguration()
canonical_address = "console"
case _:
assert_never(endpoint_plan)
return cls(
ref=endpoint_ref,
configuration=configuration,
address_hash=sha256(canonical_address.encode()).hexdigest(),
access_capability=access_capability,
created_at=now,
updated_at=now,
)
@dataclass(frozen=True, slots=True)
class DeliveryAttempt:
"""Append-only provider delivery outcome scoped to one endpoint."""
id: DeliveryAttemptId
endpoint_ref: DeliveryEndpointRef
attempt_number: int
status: HumanInputDeliveryAttemptStatus
scheduled_at: UtcTimestamp
started_at: UtcTimestamp | None
finished_at: UtcTimestamp | None
provider_message_id: str | None
failure_code: str | None
failure_reason: str | None
provider_response: FrozenJSONObject | None
created_at: UtcTimestamp
updated_at: UtcTimestamp
def __post_init__(self) -> None:
if self.attempt_number < 1:
raise ValueError("delivery attempt number must be positive")
if self.status is HumanInputDeliveryAttemptStatus.FAILED:
if self.finished_at is None:
raise ValueError("failed delivery attempt requires finished_at")
has_failure_code = self.failure_code is not None and bool(self.failure_code.strip())
has_failure_reason = self.failure_reason is not None and bool(self.failure_reason.strip())
if not has_failure_code and not has_failure_reason and self.provider_response is None:
raise ValueError("failed delivery attempt requires a failure diagnostic")
if self.status is not HumanInputDeliveryAttemptStatus.FAILED and (
self.failure_code is not None or self.failure_reason is not None
):
raise ValueError("only failed delivery attempts may contain failure diagnostics")
@dataclass(frozen=True, slots=True)
class EmailProviderConfiguration:
"""Workspace provider configuration kept outside the form domain lifecycle."""
id: EmailProviderId
workspace_id: WorkspaceId
provider: EmailProviderType
sender_email: NormalizedEmail
sender_name: str
encrypted_credentials: FrozenJSONObject
configured_by_account_id: AccountId | None
created_at: UtcTimestamp
updated_at: UtcTimestamp
@dataclass(frozen=True, slots=True)
class UploadCapabilityRef:
"""Upload capability reference carrying the complete endpoint owner chain."""
endpoint_ref: DeliveryEndpointRef
capability_id: UploadCapabilityId
app_id: AppId
@dataclass(frozen=True, slots=True)
class UploadCapability:
"""Hashed upload capability scoped to exactly one form endpoint."""
id: UploadCapabilityId
endpoint_ref: DeliveryEndpointRef
app_id: AppId
token_hash: str
created_at: UtcTimestamp
updated_at: UtcTimestamp
def __post_init__(self) -> None:
_validate_sha256(self.token_hash, label="upload token hash")
@property
def ref(self) -> UploadCapabilityRef:
return UploadCapabilityRef(self.endpoint_ref, self.id, self.app_id)
@dataclass(frozen=True, slots=True)
class UploadFileAssociation:
"""Durable file fact whose scope is inherited from its upload capability."""
id: UploadFileAssociationId
capability_ref: UploadCapabilityRef
upload_file_id: str
created_at: UtcTimestamp
updated_at: UtcTimestamp
def __post_init__(self) -> None:
if not self.upload_file_id.strip():
raise ValueError("upload file id must not be blank")
+261
View File
@@ -0,0 +1,261 @@
"""Rich Human Input v2 form aggregate and plan-to-snapshot creation.
``HumanInputForm`` directly owns every local lifecycle decision: persisted
status, node/global expiry, grant membership, and selected actions. It returns a
transition decision only; committing the first successful submission belongs to
the later submission transaction boundary.
"""
from __future__ import annotations
from dataclasses import dataclass
from enum import StrEnum
from typing import Protocol
from core.human_input_v2.entities import HumanInputV2FormKind, HumanInputV2FormStatus
from core.human_input_v2.shared import (
AppId,
ApproverGrantId,
DeliveryEndpointId,
UtcTimestamp,
)
from .delivery import DeliveryEndpoint
from .frozen_values import FrozenJSONObject
from .grants import ApproverGrant, FormRef
from .recipient_resolution import ResolvedApprovalPlan
class InvalidApproverGrantError(ValueError):
"""The selected grant does not belong to this form snapshot."""
class InvalidSelectedActionError(ValueError):
"""The selected action is absent from the frozen definition."""
@dataclass(frozen=True, slots=True)
class FrozenFormAction:
"""Immutable action values required for display and transition validation."""
id: str
title: str
button_style: str
def __post_init__(self) -> None:
if not self.id or not self.title or not self.button_style:
raise ValueError("frozen form action values must not be blank")
@dataclass(frozen=True, slots=True)
class FrozenFormDefinition:
"""Immutable render and validation definition captured at form creation."""
form_content: str
inputs: tuple[FrozenJSONObject, ...]
actions: tuple[FrozenFormAction, ...]
default_values: FrozenJSONObject
node_title: str | None
display_in_ui: bool | None
def __post_init__(self) -> None:
if not isinstance(self.inputs, tuple) or not isinstance(self.actions, tuple):
raise TypeError("frozen form definition collections must be immutable tuples")
action_ids = [action.id for action in self.actions]
if len(action_ids) != len(set(action_ids)):
raise ValueError("frozen form action identifiers must be unique")
def accepts_action(self, selected_action_id: str) -> bool:
return any(action.id == selected_action_id for action in self.actions)
class FormInactiveReason(StrEnum):
"""Transport-neutral reason why a form cannot accept a transition."""
SUBMITTED = "submitted"
TIMED_OUT = "timed_out"
STATUS_EXPIRED = "status_expired"
GLOBALLY_EXPIRED = "globally_expired"
@dataclass(frozen=True, slots=True)
class WaitingFormState:
"""Stable active-state result for a waiting form."""
is_waiting: bool = True
@dataclass(frozen=True, slots=True)
class InactiveFormState:
"""Stable inactive-state result independent from HTTP status codes."""
reason: FormInactiveReason
is_waiting: bool = False
type FormState = WaitingFormState | InactiveFormState
@dataclass(frozen=True, slots=True)
class SubmissionTransitionDecision:
"""Validated intent that does not claim persistence has committed."""
form_ref: FormRef
grant_id: ApproverGrantId
selected_action_id: str
decided_at: UtcTimestamp
class FormSnapshotIdentifierFactory(Protocol):
"""Provide child identifiers without coupling the domain to persistence."""
def new_grant_id(self) -> ApproverGrantId: ...
def new_endpoint_id(self) -> DeliveryEndpointId: ...
@dataclass(frozen=True, slots=True)
class FormCreation:
"""Complete form/grant/endpoint snapshot persisted by one transaction."""
form: HumanInputForm
endpoints: tuple[DeliveryEndpoint, ...]
def __post_init__(self) -> None:
if not isinstance(self.endpoints, tuple):
raise TypeError("form creation endpoints must be an immutable tuple")
grant_refs = {grant.ref for grant in self.form.grants}
if any(endpoint.grant_ref not in grant_refs for endpoint in self.endpoints):
raise ValueError("form creation contains an endpoint outside its grants")
@dataclass(frozen=True, slots=True)
class HumanInputForm:
"""Form root owning local lifecycle and submission transition invariants."""
ref: FormRef
app_id: AppId
definition: FrozenFormDefinition
rendered_content: str
node_timeout_at: UtcTimestamp
global_expires_at: UtcTimestamp
kind: HumanInputV2FormKind
status: HumanInputV2FormStatus
workflow_pause_id: str | None
node_execution_id: str | None
grants: tuple[ApproverGrant, ...]
created_at: UtcTimestamp
updated_at: UtcTimestamp
def __post_init__(self) -> None:
if not isinstance(self.grants, tuple):
raise TypeError("form grants must be an immutable tuple")
if self.kind is HumanInputV2FormKind.RUNTIME and (
self.workflow_pause_id is None or self.node_execution_id is None
):
raise ValueError("runtime form requires workflow pause and node execution owners")
if any(grant.ref.form_ref != self.ref for grant in self.grants):
raise ValueError("form contains a grant from another owner")
grant_ids = [grant.id for grant in self.grants]
subject_keys = [grant.subject_key for grant in self.grants]
if len(grant_ids) != len(set(grant_ids)) or len(subject_keys) != len(set(subject_keys)):
raise ValueError("form grants must have unique identifiers and canonical subjects")
def state_at(self, now: UtcTimestamp) -> FormState:
"""Return a stable status/expiry decision without changing persisted state."""
match self.status:
case HumanInputV2FormStatus.SUBMITTED:
return InactiveFormState(FormInactiveReason.SUBMITTED)
case HumanInputV2FormStatus.TIMEOUT:
return InactiveFormState(FormInactiveReason.TIMED_OUT)
case HumanInputV2FormStatus.EXPIRED:
return InactiveFormState(FormInactiveReason.STATUS_EXPIRED)
case HumanInputV2FormStatus.WAITING:
if now.value >= self.global_expires_at.value:
return InactiveFormState(FormInactiveReason.GLOBALLY_EXPIRED)
if now.value >= self.node_timeout_at.value:
return InactiveFormState(FormInactiveReason.TIMED_OUT)
return WaitingFormState()
raise AssertionError(f"unsupported Human Input form status: {self.status}")
def decide_submission(
self,
*,
grant_id: ApproverGrantId,
selected_action_id: str,
now: UtcTimestamp,
) -> SubmissionTransitionDecision | InactiveFormState:
"""Validate one local transition without mutating or persisting the form."""
state = self.state_at(now)
if isinstance(state, InactiveFormState):
return state
if not any(grant.id == grant_id for grant in self.grants):
raise InvalidApproverGrantError(str(grant_id))
if not self.definition.accepts_action(selected_action_id):
raise InvalidSelectedActionError(selected_action_id)
return SubmissionTransitionDecision(
form_ref=self.ref,
grant_id=grant_id,
selected_action_id=selected_action_id,
decided_at=now,
)
@classmethod
def create_from_plan(
cls,
*,
ref: FormRef,
app_id: AppId,
definition: FrozenFormDefinition,
rendered_content: str,
node_timeout_at: UtcTimestamp,
global_expires_at: UtcTimestamp,
kind: HumanInputV2FormKind,
workflow_pause_id: str | None,
node_execution_id: str | None,
plan: ResolvedApprovalPlan,
identifier_factory: FormSnapshotIdentifierFactory,
now: UtcTimestamp,
) -> FormCreation:
"""Map one deterministic resolved plan into a complete frozen snapshot."""
if not plan.approvers:
raise ValueError("form creation requires resolved approvers")
grants: list[ApproverGrant] = []
endpoints: list[DeliveryEndpoint] = []
for approver in plan.approvers:
grant = ApproverGrant.from_resolved_approver(
grant_id=identifier_factory.new_grant_id(),
form_ref=ref,
approver=approver,
now=now,
)
grants.append(grant)
for endpoint_plan in approver.endpoints:
endpoints.append(
DeliveryEndpoint.from_plan(
endpoint_id=identifier_factory.new_endpoint_id(),
grant_ref=grant.ref,
endpoint_plan=endpoint_plan,
access_capability=None,
now=now,
)
)
form = cls(
ref=ref,
app_id=app_id,
definition=definition,
rendered_content=rendered_content,
node_timeout_at=node_timeout_at,
global_expires_at=global_expires_at,
kind=kind,
status=HumanInputV2FormStatus.WAITING,
workflow_pause_id=workflow_pause_id,
node_execution_id=node_execution_id,
grants=tuple(grants),
created_at=now,
updated_at=now,
)
return FormCreation(form=form, endpoints=tuple(endpoints))
@@ -0,0 +1,100 @@
"""Recursively immutable JSON values used by frozen form snapshots.
Domain code never receives mutable dictionaries from persistence or provider
boundaries. Explicit conversion methods create fresh primitive containers only
when a caller crosses such a boundary.
"""
from __future__ import annotations
from collections.abc import Mapping, Sequence
from dataclasses import dataclass
from math import isfinite
type JSONScalar = str | int | float | bool | None
type JSONPrimitive = JSONScalar | list[JSONPrimitive] | dict[str, JSONPrimitive]
type FrozenJSONValue = JSONScalar | FrozenJSONArray | FrozenJSONObject
def _validate_frozen_json(value: object) -> None:
if value is None or isinstance(value, (str, bool, int)):
return
if isinstance(value, float):
if not isfinite(value):
raise ValueError("JSON numbers must be finite")
return
if isinstance(value, (FrozenJSONArray, FrozenJSONObject)):
return
raise TypeError(f"frozen JSON values cannot contain {type(value).__name__}")
def _freeze_json(value: object) -> FrozenJSONValue:
if value is None or isinstance(value, (str, bool, int)):
return value
if isinstance(value, float):
if not isfinite(value):
raise ValueError("JSON numbers must be finite")
return value
if isinstance(value, Mapping):
return FrozenJSONObject.from_mapping(value)
if isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)):
return FrozenJSONArray(tuple(_freeze_json(item) for item in value))
raise TypeError(f"unsupported JSON value type: {type(value).__name__}")
def _thaw_json(value: FrozenJSONValue) -> JSONPrimitive:
if isinstance(value, FrozenJSONObject):
return value.to_mapping()
if isinstance(value, FrozenJSONArray):
return value.to_list()
return value
@dataclass(frozen=True, slots=True)
class FrozenJSONArray:
"""Immutable JSON array with an explicit primitive conversion boundary."""
values: tuple[FrozenJSONValue, ...]
def __post_init__(self) -> None:
if not isinstance(self.values, tuple):
raise TypeError("frozen JSON array values must be an immutable tuple")
for value in self.values:
_validate_frozen_json(value)
def to_list(self) -> list[JSONPrimitive]:
return [_thaw_json(value) for value in self.values]
@dataclass(frozen=True, slots=True)
class FrozenJSONObject:
"""Immutable ordered JSON object independent from Pydantic and ORM types."""
entries: tuple[tuple[str, FrozenJSONValue], ...]
def __post_init__(self) -> None:
if not isinstance(self.entries, tuple):
raise TypeError("frozen JSON object entries must be an immutable tuple")
seen_keys: set[str] = set()
for entry in self.entries:
if not isinstance(entry, tuple) or len(entry) != 2:
raise TypeError("frozen JSON object entries must be key-value tuples")
key, value = entry
if not isinstance(key, str):
raise TypeError("JSON objects require string keys")
if key in seen_keys:
raise ValueError(f"duplicate JSON object key: {key}")
seen_keys.add(key)
_validate_frozen_json(value)
@classmethod
def from_mapping(cls, values: Mapping[str, object]) -> FrozenJSONObject:
entries: list[tuple[str, FrozenJSONValue]] = []
for key in sorted(values, key=lambda candidate: str(candidate)):
if not isinstance(key, str):
raise TypeError("JSON objects require string keys")
entries.append((key, _freeze_json(values[key])))
return cls(tuple(entries))
def to_mapping(self) -> dict[str, JSONPrimitive]:
return {key: _thaw_json(value) for key, value in self.entries}
+134
View File
@@ -0,0 +1,134 @@
"""Historical approver grants and form-scoped logical references.
Grants capture candidate authority at form creation. They are intentionally not
current authorization proofs: later submission code must revalidate the current
identity behind the frozen subject.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import assert_never
from core.human_input_v2.shared import (
ApproverGrantId,
DeliveryEndpointId,
FormId,
OTPChallengeId,
UtcTimestamp,
WorkspaceId,
)
from .recipient_resolution import (
ApprovalSubject,
CanonicalSubjectKey,
ContactApprovalSubject,
EmailAddressApprovalSubject,
EndUserApprovalSubject,
MatchedRecipientSource,
ResolvedApprover,
SubjectSnapshot,
)
def _subject_key(subject: ApprovalSubject) -> CanonicalSubjectKey:
match subject:
case ContactApprovalSubject(contact_id=contact_id):
return CanonicalSubjectKey.for_contact(contact_id)
case EndUserApprovalSubject(end_user_id=end_user_id):
return CanonicalSubjectKey.for_end_user(end_user_id)
case EmailAddressApprovalSubject(normalized_email=normalized_email):
return CanonicalSubjectKey.for_email(normalized_email)
case _:
assert_never(subject)
@dataclass(frozen=True, slots=True)
class FormRef:
"""Workspace-owned root reference; authorization still requires scoped queries."""
workspace_id: WorkspaceId
form_id: FormId
def grant(self, grant_id: ApproverGrantId) -> ApproverGrantRef:
return ApproverGrantRef(self, grant_id)
@dataclass(frozen=True, slots=True)
class ApproverGrantRef:
"""Grant reference carrying its complete form owner chain."""
form_ref: FormRef
grant_id: ApproverGrantId
def endpoint(self, endpoint_id: DeliveryEndpointId) -> DeliveryEndpointRef:
return DeliveryEndpointRef(self, endpoint_id)
def challenge(self, challenge_id: OTPChallengeId) -> OTPChallengeRef:
return OTPChallengeRef(self, challenge_id)
@dataclass(frozen=True, slots=True)
class OTPChallengeRef:
"""OTP proof-session reference carrying its complete grant owner chain."""
grant_ref: ApproverGrantRef
challenge_id: OTPChallengeId
@property
def form_ref(self) -> FormRef:
return self.grant_ref.form_ref
@dataclass(frozen=True, slots=True)
class DeliveryEndpointRef:
"""Endpoint reference carrying grant, form, and workspace ownership."""
grant_ref: ApproverGrantRef
endpoint_id: DeliveryEndpointId
@property
def form_ref(self) -> FormRef:
return self.grant_ref.form_ref
@dataclass(frozen=True, slots=True)
class ApproverGrant:
"""Frozen candidate approver with historical matched-source facts."""
ref: ApproverGrantRef
subject: ApprovalSubject
subject_key: CanonicalSubjectKey
matched_sources: tuple[MatchedRecipientSource, ...]
subject_snapshot: SubjectSnapshot
created_at: UtcTimestamp
updated_at: UtcTimestamp
def __post_init__(self) -> None:
if not isinstance(self.matched_sources, tuple):
raise TypeError("matched sources must be an immutable tuple")
if self.subject_key != _subject_key(self.subject):
raise ValueError("approver grant subject key does not match its subject")
@property
def id(self) -> ApproverGrantId:
return self.ref.grant_id
@classmethod
def from_resolved_approver(
cls,
*,
grant_id: ApproverGrantId,
form_ref: FormRef,
approver: ResolvedApprover,
now: UtcTimestamp,
) -> ApproverGrant:
return cls(
ref=form_ref.grant(grant_id),
subject=approver.subject,
subject_key=approver.subject_key,
matched_sources=approver.matched_sources,
subject_snapshot=approver.subject_snapshot,
created_at=now,
updated_at=now,
)
+460
View File
@@ -0,0 +1,460 @@
"""Independent Email OTP proof-session lifecycle and limited proof boundary.
The aggregate owns only challenge expiry, cooldown, counters, verification, and
invalidation. It never reads or mutates :class:`HumanInputForm`; submission code
must separately compare a verified proof with coherent current identity facts.
Plaintext codes are transient method inputs and are never retained or serialized.
"""
from __future__ import annotations
from dataclasses import dataclass, field, replace
from datetime import timedelta
from enum import StrEnum
from typing import Protocol, TypedDict
from core.human_input_v2.entities import HumanInputAuthorizationProofType, HumanInputOTPChallengeStatus
from core.human_input_v2.shared import ContactId, NormalizedEmail, OTPChallengeId, UtcTimestamp
from .grants import ApproverGrantRef, OTPChallengeRef
OTP_EXPIRY = timedelta(minutes=10)
OTP_RESEND_COOLDOWN = timedelta(seconds=60)
OTP_MAX_SEND_COUNT = 5
OTP_MAX_ATTEMPT_COUNT = 5
def _validate_sha256(value: str, *, label: str) -> None:
if len(value) != 64 or any(character not in "0123456789abcdef" for character in value):
raise ValueError(f"{label} must be a lower-case SHA-256 digest")
class Clock(Protocol):
"""Narrow clock port used to make all lifecycle boundaries deterministic."""
def now(self) -> UtcTimestamp: ...
@dataclass(frozen=True, slots=True)
class OTPCodeHash:
"""Opaque encoded code digest plus the verifier algorithm discriminator."""
encoded_value: str
algorithm: str
def __post_init__(self) -> None:
if not self.encoded_value or not self.algorithm.strip():
raise ValueError("OTP code hash values must not be blank")
class OTPCodeHasher(Protocol):
"""Hash and verify transient plaintext without exposing implementation policy."""
def hash_code(self, plaintext_code: str) -> OTPCodeHash: ...
def verify_code(self, plaintext_code: str, code_hash: OTPCodeHash) -> bool: ...
@dataclass(frozen=True, slots=True)
class ContactOTPSubject:
"""Contact incarnation captured when an OTP challenge is issued."""
contact_id: ContactId
@dataclass(frozen=True, slots=True)
class EmailAddressOTPSubject:
"""Standalone normalized Email subject captured from a one-time grant."""
normalized_email: NormalizedEmail
type EmailOTPSubject = ContactOTPSubject | EmailAddressOTPSubject
class OTPChallengeRejectionReason(StrEnum):
"""Transport-neutral reason for a rejected OTP or proof operation."""
EXPIRED = "expired"
RESEND_COOLDOWN = "resend_cooldown"
SEND_LIMIT_REACHED = "send_limit_reached"
ATTEMPT_LIMIT_REACHED = "attempt_limit_reached"
ALREADY_VERIFIED = "already_verified"
INVALIDATED = "invalidated"
INVALID_CODE = "invalid_code"
RAW_CODE_NOT_VERIFIED = "raw_code_not_verified"
GRANT_MISMATCH = "grant_mismatch"
STALE_IDENTITY = "stale_identity"
class OTPChallengePublicPrimitive(TypedDict):
"""Secret-free diagnostic form of challenge state."""
otp_challenge_id: str
form_id: str
approver_grant_id: str
status: str
email: str
send_count: int
attempt_count: int
expires_at: str
resend_after: str
verified_at: str | None
invalidated_at: str | None
class VerifiedEmailOTPProofPrimitive(TypedDict):
"""Primitive proof shape safe for authorization and audit boundaries."""
type: str
otp_challenge_id: str
form_id: str
approver_grant_id: str
subject_type: str
contact_id: str | None
verified_email: str
verified_at: str
@dataclass(frozen=True, slots=True)
class VerifiedEmailOTPProof:
"""Immutable Email verification fact that carries no submission authority."""
challenge_ref: OTPChallengeRef
subject: EmailOTPSubject
normalized_email: NormalizedEmail
verified_at: UtcTimestamp
def __post_init__(self) -> None:
if isinstance(self.subject, EmailAddressOTPSubject) and self.subject.normalized_email != self.normalized_email:
raise ValueError("OTP proof subject email must match the verified email")
def to_primitive(self) -> VerifiedEmailOTPProofPrimitive:
contact_id: str | None = None
subject_type = "email_address"
if isinstance(self.subject, ContactOTPSubject):
subject_type = "contact"
contact_id = str(self.subject.contact_id)
return {
"type": HumanInputAuthorizationProofType.EMAIL_OTP.value,
"otp_challenge_id": str(self.challenge_ref.challenge_id),
"form_id": str(self.challenge_ref.form_ref.form_id),
"approver_grant_id": str(self.challenge_ref.grant_ref.grant_id),
"subject_type": subject_type,
"contact_id": contact_id,
"verified_email": str(self.normalized_email),
"verified_at": self.verified_at.to_primitive(),
}
@dataclass(frozen=True, slots=True)
class OTPChallengeState:
"""Stable current usability result independent from transport status codes."""
status: HumanInputOTPChallengeStatus
rejection: OTPChallengeRejectionReason | None
@property
def is_usable(self) -> bool:
return self.rejection is None
@dataclass(frozen=True, slots=True)
class OTPReplacementDecision:
"""Immutable replacement result; persistence commits both states atomically."""
previous: OTPChallenge
replacement: OTPChallenge | None
rejection: OTPChallengeRejectionReason | None
@dataclass(frozen=True, slots=True)
class OTPVerificationDecision:
"""Verification result containing either one limited proof or one rejection."""
challenge: OTPChallenge
proof: VerifiedEmailOTPProof | None
rejection: OTPChallengeRejectionReason | None
@dataclass(frozen=True, slots=True)
class OTPChallenge:
"""Grant-scoped proof session whose counters never touch Form lifecycle state.
Persisted sessions reconstruct only when cooldown and expiry are the exact
durations derived from ``created_at``; stored timestamps are not policy input.
"""
ref: OTPChallengeRef
subject: EmailOTPSubject
normalized_email: NormalizedEmail
challenge_token_hash: str = field(repr=False)
code_hash: OTPCodeHash = field(repr=False)
status: HumanInputOTPChallengeStatus
expires_at: UtcTimestamp
resend_after: UtcTimestamp
send_count: int
attempt_count: int
verified_at: UtcTimestamp | None
invalidated_at: UtcTimestamp | None
created_at: UtcTimestamp
updated_at: UtcTimestamp
def __post_init__(self) -> None:
_validate_sha256(self.challenge_token_hash, label="challenge token hash")
if isinstance(self.subject, EmailAddressOTPSubject) and self.subject.normalized_email != self.normalized_email:
raise ValueError("OTP challenge subject email must match the destination email")
if not 1 <= self.send_count <= OTP_MAX_SEND_COUNT:
raise ValueError("OTP send count is outside the supported range")
if not 0 <= self.attempt_count <= OTP_MAX_ATTEMPT_COUNT:
raise ValueError("OTP attempt count is outside the supported range")
if self.resend_after.value != self.created_at.value + OTP_RESEND_COOLDOWN:
raise ValueError("OTP resend_after must equal created_at plus the resend cooldown")
if self.expires_at.value != self.created_at.value + OTP_EXPIRY:
raise ValueError("OTP expires_at must equal created_at plus the expiry duration")
if self.updated_at.value < self.created_at.value:
raise ValueError("OTP updated_at must not precede created_at")
if self.status is HumanInputOTPChallengeStatus.VERIFIED:
if self.verified_at is None or self.invalidated_at is not None:
raise ValueError("verified OTP challenge requires only verified_at")
elif self.status is HumanInputOTPChallengeStatus.INVALIDATED:
if self.invalidated_at is None or self.verified_at is not None:
raise ValueError("invalidated OTP challenge requires only invalidated_at")
elif self.verified_at is not None or self.invalidated_at is not None:
raise ValueError("pending and expired OTP challenges cannot have terminal timestamps")
@classmethod
def issue(
cls,
*,
challenge_ref: OTPChallengeRef,
subject: EmailOTPSubject,
normalized_email: NormalizedEmail,
challenge_token_hash: str,
plaintext_code: str,
send_count: int,
clock: Clock,
code_hasher: OTPCodeHasher,
) -> OTPChallenge:
"""Hash a transient code and create one pending proof session."""
now = clock.now()
return cls(
ref=challenge_ref,
subject=subject,
normalized_email=normalized_email,
challenge_token_hash=challenge_token_hash,
code_hash=code_hasher.hash_code(plaintext_code),
status=HumanInputOTPChallengeStatus.PENDING,
expires_at=UtcTimestamp(now.value + OTP_EXPIRY),
resend_after=UtcTimestamp(now.value + OTP_RESEND_COOLDOWN),
send_count=send_count,
attempt_count=0,
verified_at=None,
invalidated_at=None,
created_at=now,
updated_at=now,
)
def state_at(self, now: UtcTimestamp) -> OTPChallengeState:
match self.status:
case HumanInputOTPChallengeStatus.VERIFIED:
return OTPChallengeState(self.status, OTPChallengeRejectionReason.ALREADY_VERIFIED)
case HumanInputOTPChallengeStatus.INVALIDATED:
return OTPChallengeState(self.status, OTPChallengeRejectionReason.INVALIDATED)
case HumanInputOTPChallengeStatus.EXPIRED:
return OTPChallengeState(self.status, OTPChallengeRejectionReason.EXPIRED)
case HumanInputOTPChallengeStatus.PENDING:
if now.value >= self.expires_at.value:
return OTPChallengeState(HumanInputOTPChallengeStatus.EXPIRED, OTPChallengeRejectionReason.EXPIRED)
return OTPChallengeState(self.status, None)
raise AssertionError(f"unsupported OTP challenge status: {self.status}")
def replace(
self,
*,
challenge_ref: OTPChallengeRef,
challenge_token_hash: str,
plaintext_code: str,
clock: Clock,
code_hasher: OTPCodeHasher,
) -> OTPReplacementDecision:
"""Prepare an eligible replacement without persisting either state."""
now = clock.now()
state = self.state_at(now)
if (
state.rejection is OTPChallengeRejectionReason.EXPIRED
and self.status is HumanInputOTPChallengeStatus.PENDING
):
expired = replace(
self,
status=HumanInputOTPChallengeStatus.EXPIRED,
updated_at=now,
)
return OTPReplacementDecision(expired, None, OTPChallengeRejectionReason.EXPIRED)
if state.rejection is not None:
return OTPReplacementDecision(self, None, state.rejection)
if self.send_count >= OTP_MAX_SEND_COUNT:
return OTPReplacementDecision(self, None, OTPChallengeRejectionReason.SEND_LIMIT_REACHED)
if now.value < self.resend_after.value:
return OTPReplacementDecision(self, None, OTPChallengeRejectionReason.RESEND_COOLDOWN)
replacement_challenge = self.issue(
challenge_ref=challenge_ref,
subject=self.subject,
normalized_email=self.normalized_email,
challenge_token_hash=challenge_token_hash,
plaintext_code=plaintext_code,
send_count=self.send_count + 1,
clock=clock,
code_hasher=code_hasher,
)
invalidated = replace(
self,
status=HumanInputOTPChallengeStatus.INVALIDATED,
invalidated_at=now,
updated_at=now,
)
return OTPReplacementDecision(invalidated, replacement_challenge, None)
def verify(
self,
*,
plaintext_code: str,
clock: Clock,
code_hasher: OTPCodeHasher,
) -> OTPVerificationDecision:
"""Verify one transient code while preserving exact attempt boundaries."""
now = clock.now()
state = self.state_at(now)
if (
state.rejection is OTPChallengeRejectionReason.EXPIRED
and self.status is HumanInputOTPChallengeStatus.PENDING
):
expired = replace(
self,
status=HumanInputOTPChallengeStatus.EXPIRED,
updated_at=now,
)
return OTPVerificationDecision(expired, None, OTPChallengeRejectionReason.EXPIRED)
if state.rejection is not None:
return OTPVerificationDecision(self, None, state.rejection)
if self.attempt_count >= OTP_MAX_ATTEMPT_COUNT:
return OTPVerificationDecision(self, None, OTPChallengeRejectionReason.ATTEMPT_LIMIT_REACHED)
attempt_count = self.attempt_count + 1
if not code_hasher.verify_code(plaintext_code, self.code_hash):
attempted = replace(self, attempt_count=attempt_count, updated_at=now)
return OTPVerificationDecision(attempted, None, OTPChallengeRejectionReason.INVALID_CODE)
verified = replace(
self,
status=HumanInputOTPChallengeStatus.VERIFIED,
attempt_count=attempt_count,
verified_at=now,
updated_at=now,
)
proof = VerifiedEmailOTPProof(
challenge_ref=self.ref,
subject=self.subject,
normalized_email=self.normalized_email,
verified_at=now,
)
return OTPVerificationDecision(verified, proof, None)
def invalidate(self, *, clock: Clock) -> OTPChallenge:
"""Make a pending proof session unusable without changing its counters."""
if self.status is not HumanInputOTPChallengeStatus.PENDING:
return self
now = clock.now()
return replace(
self,
status=HumanInputOTPChallengeStatus.INVALIDATED,
invalidated_at=now,
updated_at=now,
)
def to_public_primitive(self) -> OTPChallengePublicPrimitive:
"""Return state diagnostics while deliberately excluding all hashes."""
return {
"otp_challenge_id": str(self.ref.challenge_id),
"form_id": str(self.ref.form_ref.form_id),
"approver_grant_id": str(self.ref.grant_ref.grant_id),
"status": self.status.value,
"email": str(self.normalized_email),
"send_count": self.send_count,
"attempt_count": self.attempt_count,
"expires_at": self.expires_at.to_primitive(),
"resend_after": self.resend_after.to_primitive(),
"verified_at": self.verified_at.to_primitive() if self.verified_at is not None else None,
"invalidated_at": self.invalidated_at.to_primitive() if self.invalidated_at is not None else None,
}
@dataclass(frozen=True, slots=True)
class CurrentEmailOTPIdentity:
"""Coherent current grant subject and Email facts loaded by submission persistence."""
grant_ref: ApproverGrantRef
subject: EmailOTPSubject | None
normalized_email: NormalizedEmail | None
@dataclass(frozen=True, slots=True)
class EmailOTPProofAuthorizationDecision:
"""OTP-specific proof decision consumed by the later Submission authorizer."""
proof: VerifiedEmailOTPProof | None
rejection: OTPChallengeRejectionReason | None
class OTPChallengeRepository(Protocol):
"""Grant-scoped atomic persistence operations for OTP proof sessions."""
def issue_initial(
self,
grant_ref: ApproverGrantRef,
*,
challenge_id: OTPChallengeId,
audit_event_id: str,
challenge_token_hash: str,
plaintext_code: str,
) -> OTPChallenge: ...
def replace_current(
self,
grant_ref: ApproverGrantRef,
*,
challenge_id: OTPChallengeId,
audit_event_id: str,
challenge_token_hash: str,
plaintext_code: str,
) -> OTPReplacementDecision: ...
def verify(self, challenge_ref: OTPChallengeRef, *, plaintext_code: str) -> OTPVerificationDecision: ...
def invalidate_current(self, grant_ref: ApproverGrantRef) -> OTPChallenge | None: ...
def load(self, challenge_ref: OTPChallengeRef) -> OTPChallenge | None: ...
def authorize_email_otp_proof(
candidate: object,
*,
current_identity: CurrentEmailOTPIdentity,
) -> EmailOTPProofAuthorizationDecision:
"""Reject raw codes and stale identity incarnations without authorizing submission."""
if not isinstance(candidate, VerifiedEmailOTPProof):
return EmailOTPProofAuthorizationDecision(None, OTPChallengeRejectionReason.RAW_CODE_NOT_VERIFIED)
if candidate.challenge_ref.grant_ref != current_identity.grant_ref:
return EmailOTPProofAuthorizationDecision(None, OTPChallengeRejectionReason.GRANT_MISMATCH)
if (
current_identity.subject is None
or current_identity.normalized_email is None
or candidate.subject != current_identity.subject
or candidate.normalized_email != current_identity.normalized_email
):
return EmailOTPProofAuthorizationDecision(None, OTPChallengeRejectionReason.STALE_IDENTITY)
return EmailOTPProofAuthorizationDecision(candidate, None)
+81
View File
@@ -0,0 +1,81 @@
"""Operation-oriented persistence ports for the Human Input v2 form boundary."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Protocol
from core.human_input_v2.entities import HumanInputV2FormStatus
from core.human_input_v2.shared import UtcTimestamp, WorkspaceId
from .delivery import DeliveryAttempt, DeliveryEndpoint, UploadCapability, UploadFileAssociation
from .form import FormCreation, FrozenFormDefinition, HumanInputForm
from .grants import ApproverGrant, DeliveryEndpointRef, FormRef
@dataclass(frozen=True, slots=True)
class FormDefinitionProjection:
"""Read model for rendering a form through one endpoint capability."""
form_ref: FormRef
endpoint_ref: DeliveryEndpointRef
definition: FrozenFormDefinition
rendered_content: str
status: HumanInputV2FormStatus
node_timeout_at: UtcTimestamp
global_expires_at: UtcTimestamp
@dataclass(frozen=True, slots=True)
class FormDeliveryProjection:
"""Read model containing only data needed to deliver one endpoint."""
form_ref: FormRef
grant: ApproverGrant
endpoint: DeliveryEndpoint
definition: FrozenFormDefinition
rendered_content: str
class FormRepository(Protocol):
"""Deep adapter contract whose operations own their query and transaction shape."""
def create_form(self, creation: FormCreation) -> HumanInputForm:
"""Persist form, grants, and endpoints atomically."""
...
def load_for_lifecycle(self, form_ref: FormRef) -> HumanInputForm | None:
"""Load the form and grants required for local transition decisions."""
...
def load_delivery_projection(self, endpoint_ref: DeliveryEndpointRef) -> FormDeliveryProjection | None:
"""Load exactly one endpoint with its grant and form delivery values."""
...
def load_definition_by_endpoint_token(
self,
*,
workspace_id: WorkspaceId,
token_hash: str,
) -> FormDefinitionProjection | None:
"""Resolve a scoped interaction capability without creating authority."""
...
def append_delivery_attempt(self, attempt: DeliveryAttempt) -> DeliveryAttempt:
"""Append one delivery fact without changing form lifecycle status."""
...
def create_upload_capability(self, capability: UploadCapability) -> UploadCapability:
"""Persist one endpoint-scoped upload capability."""
...
def associate_upload_file(self, association: UploadFileAssociation) -> UploadFileAssociation:
"""Associate a file only after validating the full capability owner chain."""
...
@@ -0,0 +1,775 @@
"""Canonical approval plan values and single-entry recipient resolution.
``RecipientResolver.resolve`` is the only public operation that converts saved
recipient specifications into approvers. Validation, Contact upgrade, subject
deduplication, matched-source aggregation, debug replacement, and endpoint
planning stay behind that interface so callers cannot apply them inconsistently.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import StrEnum
from hashlib import sha256
from typing import assert_never
from core.human_input_v2.contact_directory import (
Contact,
ContactDirectoryError,
ContactDirectoryPolicy,
ContactDirectorySnapshot,
ContactResolution,
)
from core.human_input_v2.entities import HumanInputApproverGrantSubjectType, HumanInputDeliveryChannel, IMProvider
from core.human_input_v2.im_integration import EffectiveIMBindingSnapshot
from core.human_input_v2.shared import (
ContactId,
EndUserId,
IMBindingId,
IMIdentityId,
IntegrationId,
NormalizedEmail,
)
from .recipient_specifications import (
ContactRecipientSpecification,
CurrentInitiatorRecipientSpecification,
DynamicEmailRecipientSpecification,
DynamicRecipientValue,
OneTimeEmailRecipientSpecification,
RecipientSpecification,
UnsupportedDynamicRecipientValue,
)
@dataclass(frozen=True, slots=True)
class CanonicalSubjectKey:
"""Portable form-scoped deduplication key, not an authorization identity."""
value: str
def __post_init__(self) -> None:
namespace, separator, identity = self.value.partition(":")
valid_named_identity = namespace in {"contact", "end_user"} and bool(identity)
valid_email_digest = (
namespace == "email_address"
and len(identity) == 64
and all(character in "0123456789abcdef" for character in identity)
)
if not separator or not (valid_named_identity or valid_email_digest):
raise ValueError("canonical subject key has an invalid portable format")
@classmethod
def for_contact(cls, contact_id: ContactId) -> CanonicalSubjectKey:
return cls(f"contact:{contact_id}")
@classmethod
def for_end_user(cls, end_user_id: EndUserId) -> CanonicalSubjectKey:
return cls(f"end_user:{end_user_id}")
@classmethod
def for_email(cls, normalized_email: NormalizedEmail) -> CanonicalSubjectKey:
digest = sha256(normalized_email.value.encode()).hexdigest()
return cls(f"email_address:{digest}")
@dataclass(frozen=True, slots=True)
class ContactApprovalSubject:
"""Approval authority backed by one canonical Contact."""
contact_id: ContactId
@property
def subject_type(self) -> HumanInputApproverGrantSubjectType:
return HumanInputApproverGrantSubjectType.CONTACT
def to_primitive(self) -> dict[str, object]:
return {"type": self.subject_type.value, "contact_id": self.contact_id.to_primitive()}
@dataclass(frozen=True, slots=True)
class EndUserApprovalSubject:
"""Approval authority backed by one app-scoped EndUser."""
end_user_id: EndUserId
@property
def subject_type(self) -> HumanInputApproverGrantSubjectType:
return HumanInputApproverGrantSubjectType.END_USER
def to_primitive(self) -> dict[str, object]:
return {"type": self.subject_type.value, "end_user_id": self.end_user_id.to_primitive()}
@dataclass(frozen=True, slots=True)
class EmailAddressApprovalSubject:
"""Task-scoped approval authority backed by one normalized Email address."""
normalized_email: NormalizedEmail
@property
def subject_type(self) -> HumanInputApproverGrantSubjectType:
return HumanInputApproverGrantSubjectType.EMAIL_ADDRESS
def to_primitive(self) -> dict[str, object]:
return {"type": self.subject_type.value, "normalized_email": self.normalized_email.to_primitive()}
type ApprovalSubject = ContactApprovalSubject | EndUserApprovalSubject | EmailAddressApprovalSubject
class RecipientSourceKind(StrEnum):
"""Stable source discriminator retained after canonicalization."""
STATIC_CONTACT = "static_contact"
ONE_TIME_EMAIL = "one_time_email"
DYNAMIC_EMAIL = "dynamic_email"
CURRENT_INITIATOR = "current_initiator"
DEBUG_REPLACEMENT = "debug_replacement"
@dataclass(frozen=True, slots=True)
class MatchedRecipientSource:
"""One ordered configured or request-scoped source of an approver."""
kind: RecipientSourceKind
position: int
reference: str | None
def __post_init__(self) -> None:
if self.position < 0:
raise ValueError("recipient source position must not be negative")
def to_primitive(self) -> dict[str, object]:
return {"kind": self.kind.value, "position": self.position, "reference": self.reference}
@dataclass(frozen=True, slots=True)
class SubjectSnapshot:
"""Display-only identity facts captured by resolution."""
display_name: str | None
email: str | None
def to_primitive(self) -> dict[str, object]:
return {"display_name": self.display_name, "email": self.email}
@dataclass(frozen=True, slots=True)
class EmailEndpointPlan:
"""Email delivery destination for one canonical approver."""
email_address: NormalizedEmail
@property
def channel(self) -> HumanInputDeliveryChannel:
return HumanInputDeliveryChannel.EMAIL
def to_primitive(self) -> dict[str, object]:
return {"channel": self.channel.value, "email_address": self.email_address.to_primitive()}
@dataclass(frozen=True, slots=True)
class IMEndpointPlan:
"""Credential-free IM delivery destination frozen from an effective binding."""
integration_id: IntegrationId
provider: IMProvider
provider_tenant_id: str
identity_id: IMIdentityId
binding_id: IMBindingId | None
provider_user_id: str
@property
def channel(self) -> HumanInputDeliveryChannel:
return HumanInputDeliveryChannel.IM
def to_primitive(self) -> dict[str, object]:
return {
"channel": self.channel.value,
"integration_id": self.integration_id.to_primitive(),
"provider": self.provider.value,
"provider_tenant_id": self.provider_tenant_id,
"identity_id": self.identity_id.to_primitive(),
"binding_id": self.binding_id.to_primitive() if self.binding_id is not None else None,
"provider_user_id": self.provider_user_id,
}
@dataclass(frozen=True, slots=True)
class WebEndpointPlan:
"""Public or trusted-app web interaction surface without a saved token."""
@property
def channel(self) -> HumanInputDeliveryChannel:
return HumanInputDeliveryChannel.WEB
def to_primitive(self) -> dict[str, object]:
return {"channel": self.channel.value}
@dataclass(frozen=True, slots=True)
class ConsoleEndpointPlan:
"""Authenticated console interaction surface without a notification address."""
@property
def channel(self) -> HumanInputDeliveryChannel:
return HumanInputDeliveryChannel.CONSOLE
def to_primitive(self) -> dict[str, object]:
return {"channel": self.channel.value}
type DeliveryEndpointPlan = EmailEndpointPlan | IMEndpointPlan | WebEndpointPlan | ConsoleEndpointPlan
class RecipientRejectionReason(StrEnum):
"""Transport-neutral reason for rejecting one recipient source."""
INVALID_CONTACT_ID = "invalid_contact_id"
CONTACT_UNAVAILABLE = "contact_unavailable"
INVALID_DYNAMIC_SELECTOR = "invalid_dynamic_selector"
DYNAMIC_VALUE_UNAVAILABLE = "dynamic_value_unavailable"
UNSUPPORTED_DYNAMIC_TYPE = "unsupported_dynamic_type"
INVALID_EMAIL = "invalid_email"
INITIATOR_UNAVAILABLE = "initiator_unavailable"
NO_USABLE_ENDPOINT = "no_usable_endpoint"
@dataclass(frozen=True, slots=True)
class RejectedRecipient:
"""Machine-readable source failure retained alongside valid approvers."""
source: MatchedRecipientSource
reason: RecipientRejectionReason
rejected_value: str | None
def to_primitive(self) -> dict[str, object]:
return {
"source": self.source.to_primitive(),
"reason": self.reason.value,
"rejected_value": self.rejected_value,
}
@dataclass(frozen=True, slots=True)
class ResolvedApprover:
"""One canonical subject with all matched sources and usable endpoints."""
subject: ApprovalSubject
subject_key: CanonicalSubjectKey
matched_sources: tuple[MatchedRecipientSource, ...]
subject_snapshot: SubjectSnapshot
endpoints: tuple[DeliveryEndpointPlan, ...]
def __post_init__(self) -> None:
if not isinstance(self.matched_sources, tuple) or not isinstance(self.endpoints, tuple):
raise TypeError("resolved approver collections must be immutable tuples")
if self.subject_key != _canonical_key_for_subject(self.subject):
raise ValueError("resolved approver subject key does not match its subject")
def to_primitive(self) -> dict[str, object]:
return {
"subject": self.subject.to_primitive(),
"subject_key": self.subject_key.value,
"matched_sources": [source.to_primitive() for source in self.matched_sources],
"subject_snapshot": self.subject_snapshot.to_primitive(),
"endpoints": [endpoint.to_primitive() for endpoint in self.endpoints],
}
class RecipientResolutionFailureReason(StrEnum):
"""Stable whole-plan failure independent from HTTP or provider semantics."""
NO_VALID_RECIPIENTS = "no_valid_recipients"
@dataclass(frozen=True, slots=True)
class ResolvedApprovalPlan:
"""Immutable complete output of one recipient resolution request."""
approvers: tuple[ResolvedApprover, ...]
rejected_recipients: tuple[RejectedRecipient, ...]
failure_reason: RecipientResolutionFailureReason | None
def __post_init__(self) -> None:
if not isinstance(self.approvers, tuple) or not isinstance(self.rejected_recipients, tuple):
raise TypeError("approval plan collections must be immutable tuples")
if self.approvers and self.failure_reason is not None:
raise ValueError("a plan with approvers cannot have a failure reason")
if not self.approvers and self.failure_reason is None:
raise ValueError("a plan without approvers must have a failure reason")
def to_primitive(self) -> dict[str, object]:
return {
"approvers": [approver.to_primitive() for approver in self.approvers],
"rejected_recipients": [rejection.to_primitive() for rejection in self.rejected_recipients],
"failure_reason": self.failure_reason.value if self.failure_reason is not None else None,
}
@dataclass(frozen=True, slots=True)
class ContactInitiatorSnapshot:
"""Current request initiator resolved to a canonical Contact reference."""
contact_id: ContactId
@dataclass(frozen=True, slots=True)
class EndUserInitiatorSnapshot:
"""Current request initiator resolved to one app-scoped EndUser."""
end_user_id: EndUserId
display_name: str | None
email: str | None
type InitiatorSnapshot = ContactInitiatorSnapshot | EndUserInitiatorSnapshot
@dataclass(frozen=True, slots=True)
class DebugRecipientReplacement:
"""Valid debug actor that replaces saved recipients for one request only."""
subject: InitiatorSnapshot
@dataclass(frozen=True, slots=True)
class DeliveryCapabilitySnapshot:
"""Request-scoped effective delivery and interaction capabilities.
IM values are already resolved by the IM control plane; this domain never
sees credentials, raw provider clients, or invalid binding candidates.
Explicit Web/Console sets prevent recipient resolution from inventing
interaction surfaces that the current runtime cannot actually expose.
"""
im_bindings: tuple[EffectiveIMBindingSnapshot, ...] = ()
contact_web_ids: frozenset[ContactId] = frozenset()
contact_console_ids: frozenset[ContactId] = frozenset()
end_user_web_ids: frozenset[EndUserId] = frozenset()
email_address_web_available: bool = False
def __post_init__(self) -> None:
if not isinstance(self.im_bindings, tuple):
raise TypeError("effective IM bindings must be an immutable tuple")
if not all(
isinstance(values, frozenset)
for values in (self.contact_web_ids, self.contact_console_ids, self.end_user_web_ids)
):
raise TypeError("interaction capability identifiers must be immutable frozensets")
@dataclass(slots=True)
class _PendingApprover:
"""Private mutable accumulator hidden behind the immutable resolver result."""
subject: ApprovalSubject
subject_key: CanonicalSubjectKey
subject_snapshot: SubjectSnapshot
first_source_position: int
matched_sources: list[MatchedRecipientSource] = field(default_factory=list)
endpoints: list[DeliveryEndpointPlan] = field(default_factory=list)
class RecipientResolver:
"""Resolve all recipient semantics through one deterministic domain entry."""
@staticmethod
def resolve(
*,
specifications: tuple[RecipientSpecification, ...],
directory: ContactDirectorySnapshot,
dynamic_values: tuple[DynamicRecipientValue, ...],
initiator: InitiatorSnapshot | None,
capabilities: DeliveryCapabilitySnapshot,
debug_replacement: DebugRecipientReplacement | None = None,
) -> ResolvedApprovalPlan:
"""Resolve immutable request inputs into one complete approval plan.
Invalid or unavailable sources are returned as typed rejection facts;
the method raises only when a caller violates an immutable input shape.
No database, provider, transport, or mutation side effects occur.
"""
if not isinstance(specifications, tuple) or not isinstance(dynamic_values, tuple):
raise TypeError("recipient resolution inputs must be immutable tuples")
pending_approvers: dict[CanonicalSubjectKey, _PendingApprover] = {}
rejected_recipients: list[RejectedRecipient] = []
dynamic_values_by_selector: dict[tuple[str, ...], DynamicRecipientValue] = {}
for dynamic_value in dynamic_values:
dynamic_values_by_selector.setdefault(dynamic_value.selector, dynamic_value)
if debug_replacement is not None:
source = MatchedRecipientSource(RecipientSourceKind.DEBUG_REPLACEMENT, 0, None)
RecipientResolver._resolve_initiator(
debug_replacement.subject,
source,
directory,
capabilities,
pending_approvers,
rejected_recipients,
)
else:
for position, specification in enumerate(specifications):
RecipientResolver._resolve_specification(
specification,
position,
directory,
dynamic_values_by_selector,
initiator,
capabilities,
pending_approvers,
rejected_recipients,
)
approvers: list[ResolvedApprover] = []
for pending in sorted(
pending_approvers.values(),
key=lambda candidate: (candidate.first_source_position, candidate.subject_key.value),
):
matched_sources = tuple(sorted(pending.matched_sources, key=_source_sort_key))
endpoints = tuple(sorted(pending.endpoints, key=_endpoint_sort_key))
if not endpoints:
rejected_recipients.extend(
RejectedRecipient(
source=source,
reason=RecipientRejectionReason.NO_USABLE_ENDPOINT,
rejected_value=pending.subject_key.value,
)
for source in matched_sources
)
continue
approvers.append(
ResolvedApprover(
subject=pending.subject,
subject_key=pending.subject_key,
matched_sources=matched_sources,
subject_snapshot=pending.subject_snapshot,
endpoints=endpoints,
)
)
ordered_rejections = tuple(sorted(rejected_recipients, key=_rejection_sort_key))
failure_reason = None if approvers else RecipientResolutionFailureReason.NO_VALID_RECIPIENTS
return ResolvedApprovalPlan(tuple(approvers), ordered_rejections, failure_reason)
@staticmethod
def _resolve_specification(
specification: RecipientSpecification,
position: int,
directory: ContactDirectorySnapshot,
dynamic_values_by_selector: dict[tuple[str, ...], DynamicRecipientValue],
initiator: InitiatorSnapshot | None,
capabilities: DeliveryCapabilitySnapshot,
pending_approvers: dict[CanonicalSubjectKey, _PendingApprover],
rejected_recipients: list[RejectedRecipient],
) -> None:
if isinstance(specification, ContactRecipientSpecification):
source = MatchedRecipientSource(
RecipientSourceKind.STATIC_CONTACT,
position,
specification.contact_id,
)
try:
contact_id = ContactId(specification.contact_id)
except ValueError:
rejected_recipients.append(
RejectedRecipient(source, RecipientRejectionReason.INVALID_CONTACT_ID, specification.contact_id)
)
return
RecipientResolver._resolve_contact(
contact_id,
source,
directory,
capabilities,
pending_approvers,
rejected_recipients,
)
return
if isinstance(specification, OneTimeEmailRecipientSpecification):
source = MatchedRecipientSource(
RecipientSourceKind.ONE_TIME_EMAIL,
position,
specification.email,
)
RecipientResolver._resolve_email(
specification.email,
source,
directory,
capabilities,
pending_approvers,
rejected_recipients,
)
return
if isinstance(specification, DynamicEmailRecipientSpecification):
selector_reference = ".".join(specification.selector)
source = MatchedRecipientSource(
RecipientSourceKind.DYNAMIC_EMAIL,
position,
selector_reference,
)
if not specification.selector or any(not component.strip() for component in specification.selector):
rejected_recipients.append(
RejectedRecipient(source, RecipientRejectionReason.INVALID_DYNAMIC_SELECTOR, selector_reference)
)
return
dynamic_value = dynamic_values_by_selector.get(specification.selector)
if dynamic_value is None:
rejected_recipients.append(
RejectedRecipient(source, RecipientRejectionReason.DYNAMIC_VALUE_UNAVAILABLE, selector_reference)
)
return
if isinstance(dynamic_value.value, UnsupportedDynamicRecipientValue):
rejected_recipients.append(
RejectedRecipient(
source,
RecipientRejectionReason.UNSUPPORTED_DYNAMIC_TYPE,
dynamic_value.value.value_type,
)
)
return
RecipientResolver._resolve_email(
dynamic_value.value,
source,
directory,
capabilities,
pending_approvers,
rejected_recipients,
)
return
if isinstance(specification, CurrentInitiatorRecipientSpecification):
source = MatchedRecipientSource(RecipientSourceKind.CURRENT_INITIATOR, position, None)
if initiator is None:
rejected_recipients.append(
RejectedRecipient(source, RecipientRejectionReason.INITIATOR_UNAVAILABLE, None)
)
return
RecipientResolver._resolve_initiator(
initiator,
source,
directory,
capabilities,
pending_approvers,
rejected_recipients,
)
return
assert_never(specification)
@staticmethod
def _resolve_initiator(
initiator: InitiatorSnapshot,
source: MatchedRecipientSource,
directory: ContactDirectorySnapshot,
capabilities: DeliveryCapabilitySnapshot,
pending_approvers: dict[CanonicalSubjectKey, _PendingApprover],
rejected_recipients: list[RejectedRecipient],
) -> None:
if isinstance(initiator, ContactInitiatorSnapshot):
RecipientResolver._resolve_contact(
initiator.contact_id,
source,
directory,
capabilities,
pending_approvers,
rejected_recipients,
)
return
normalized_email: NormalizedEmail | None = None
if initiator.email is not None:
try:
normalized_email = NormalizedEmail(initiator.email)
except ValueError:
normalized_email = None
subject = EndUserApprovalSubject(initiator.end_user_id)
endpoints: list[DeliveryEndpointPlan] = []
if normalized_email is not None:
endpoints.append(EmailEndpointPlan(normalized_email))
if initiator.end_user_id in capabilities.end_user_web_ids:
endpoints.append(WebEndpointPlan())
RecipientResolver._add_approver(
subject,
source,
SubjectSnapshot(initiator.display_name, normalized_email.value if normalized_email is not None else None),
endpoints,
pending_approvers,
)
@staticmethod
def _resolve_contact(
contact_id: ContactId,
source: MatchedRecipientSource,
directory: ContactDirectorySnapshot,
capabilities: DeliveryCapabilitySnapshot,
pending_approvers: dict[CanonicalSubjectKey, _PendingApprover],
rejected_recipients: list[RejectedRecipient],
) -> None:
try:
resolution = ContactDirectoryPolicy.resolve_for_workspace(directory, contact_id)
except ContactDirectoryError:
resolution = ContactResolution.ABSENT
contact = directory.find(contact_id)
if resolution is ContactResolution.ABSENT or contact is None:
rejected_recipients.append(
RejectedRecipient(source, RecipientRejectionReason.CONTACT_UNAVAILABLE, contact_id.value)
)
return
RecipientResolver._add_contact_approver(contact, source, capabilities, pending_approvers)
@staticmethod
def _resolve_email(
email: str,
source: MatchedRecipientSource,
directory: ContactDirectorySnapshot,
capabilities: DeliveryCapabilitySnapshot,
pending_approvers: dict[CanonicalSubjectKey, _PendingApprover],
rejected_recipients: list[RejectedRecipient],
) -> None:
"""Resolve Email input without bypassing an existing Contact's policy.
EmailAddress authority is valid only when the directory contains no
matching Contact. A matching but unavailable Contact fails closed so
callers cannot bypass Account availability or workspace visibility.
"""
try:
normalized_email = NormalizedEmail(email)
except ValueError:
rejected_recipients.append(RejectedRecipient(source, RecipientRejectionReason.INVALID_EMAIL, email))
return
matching_contacts = sorted(
(contact for contact in directory.contacts if contact.normalized_email == normalized_email),
key=lambda contact: contact.id.value,
)
for contact in matching_contacts:
try:
resolution = ContactDirectoryPolicy.resolve_for_workspace(directory, contact.id)
except ContactDirectoryError:
continue
if resolution is not ContactResolution.ABSENT:
RecipientResolver._add_contact_approver(contact, source, capabilities, pending_approvers)
return
if matching_contacts:
rejected_recipients.append(
RejectedRecipient(
source,
RecipientRejectionReason.CONTACT_UNAVAILABLE,
normalized_email.value,
)
)
return
subject = EmailAddressApprovalSubject(normalized_email)
endpoints: list[DeliveryEndpointPlan] = [EmailEndpointPlan(normalized_email)]
if capabilities.email_address_web_available:
endpoints.append(WebEndpointPlan())
RecipientResolver._add_approver(
subject,
source,
SubjectSnapshot(None, normalized_email.value),
endpoints,
pending_approvers,
)
@staticmethod
def _add_contact_approver(
contact: Contact,
source: MatchedRecipientSource,
capabilities: DeliveryCapabilitySnapshot,
pending_approvers: dict[CanonicalSubjectKey, _PendingApprover],
) -> None:
endpoints: list[DeliveryEndpointPlan] = []
if contact.normalized_email is not None:
endpoints.append(EmailEndpointPlan(contact.normalized_email))
endpoints.extend(
IMEndpointPlan(
integration_id=binding.integration_id,
provider=binding.provider,
provider_tenant_id=binding.provider_tenant_id,
identity_id=binding.identity_id,
binding_id=binding.binding_id,
provider_user_id=binding.provider_user_id,
)
for binding in capabilities.im_bindings
if binding.contact_id == contact.id
)
if contact.id in capabilities.contact_web_ids:
endpoints.append(WebEndpointPlan())
if contact.id in capabilities.contact_console_ids:
endpoints.append(ConsoleEndpointPlan())
RecipientResolver._add_approver(
ContactApprovalSubject(contact.id),
source,
SubjectSnapshot(contact.name, contact.email),
endpoints,
pending_approvers,
)
@staticmethod
def _add_approver(
subject: ApprovalSubject,
source: MatchedRecipientSource,
subject_snapshot: SubjectSnapshot,
endpoints: list[DeliveryEndpointPlan],
pending_approvers: dict[CanonicalSubjectKey, _PendingApprover],
) -> None:
subject_key = _canonical_key_for_subject(subject)
pending = pending_approvers.get(subject_key)
if pending is None:
pending = _PendingApprover(
subject=subject,
subject_key=subject_key,
subject_snapshot=subject_snapshot,
first_source_position=source.position,
)
pending_approvers[subject_key] = pending
if source not in pending.matched_sources:
pending.matched_sources.append(source)
for endpoint in endpoints:
if endpoint not in pending.endpoints:
pending.endpoints.append(endpoint)
def _canonical_key_for_subject(subject: ApprovalSubject) -> CanonicalSubjectKey:
if isinstance(subject, ContactApprovalSubject):
return CanonicalSubjectKey.for_contact(subject.contact_id)
if isinstance(subject, EndUserApprovalSubject):
return CanonicalSubjectKey.for_end_user(subject.end_user_id)
return CanonicalSubjectKey.for_email(subject.normalized_email)
_CHANNEL_ORDER = {
HumanInputDeliveryChannel.EMAIL: 0,
HumanInputDeliveryChannel.IM: 1,
HumanInputDeliveryChannel.WEB: 2,
HumanInputDeliveryChannel.CONSOLE: 3,
}
def _source_sort_key(source: MatchedRecipientSource) -> tuple[int, str, str]:
return source.position, source.kind.value, source.reference or ""
def _endpoint_sort_key(endpoint: DeliveryEndpointPlan) -> tuple[int, str, str, str]:
channel_order = _CHANNEL_ORDER[endpoint.channel]
if isinstance(endpoint, EmailEndpointPlan):
return channel_order, endpoint.email_address.value, "", ""
if isinstance(endpoint, IMEndpointPlan):
return (
channel_order,
endpoint.integration_id.value,
endpoint.provider.value,
endpoint.identity_id.value,
)
return channel_order, "", "", ""
def _rejection_sort_key(rejection: RejectedRecipient) -> tuple[int, str, str, str]:
source_key = _source_sort_key(rejection.source)
return source_key[0], source_key[1], rejection.reason.value, rejection.rejected_value or ""
@@ -0,0 +1,145 @@
"""Immutable recipient specifications at the workflow-to-approval boundary.
Saved node configuration intentionally retains unvalidated Email text. Runtime
validation belongs to :class:`RecipientResolver`, which can retain a typed
rejection without making workflow configuration parsing fail early.
"""
from __future__ import annotations
from dataclasses import dataclass
from enum import StrEnum
from typing import assert_never
from core.workflow.nodes.human_input_v2.entities import (
Contact as WorkflowContactRecipient,
)
from core.workflow.nodes.human_input_v2.entities import (
DynamicEmail as WorkflowDynamicEmailRecipient,
)
from core.workflow.nodes.human_input_v2.entities import (
HumanInputNodeData,
)
from core.workflow.nodes.human_input_v2.entities import (
Initiator as WorkflowInitiatorRecipient,
)
from core.workflow.nodes.human_input_v2.entities import (
OnetimeEmail as WorkflowOneTimeEmailRecipient,
)
class RecipientSpecificationKind(StrEnum):
"""Stable workflow recipient discriminator."""
CONTACT = "contact"
DYNAMIC_EMAIL = "dynamic_email"
ONETIME_EMAIL = "onetime_email"
INITIATOR = "initiator"
@dataclass(frozen=True, slots=True)
class ContactRecipientSpecification:
"""Saved reference to one Contact; current availability is resolved later."""
contact_id: str
def to_primitive(self) -> dict[str, object]:
return {"type": RecipientSpecificationKind.CONTACT.value, "contact_id": self.contact_id}
@dataclass(frozen=True, slots=True)
class OneTimeEmailRecipientSpecification:
"""Saved one-time Email text whose validity is decided per resolution."""
email: str
def to_primitive(self) -> dict[str, object]:
return {"type": RecipientSpecificationKind.ONETIME_EMAIL.value, "email": self.email}
@dataclass(frozen=True, slots=True)
class DynamicEmailRecipientSpecification:
"""Saved workflow selector whose current value is supplied separately."""
selector: tuple[str, ...]
def __post_init__(self) -> None:
if not isinstance(self.selector, tuple):
raise TypeError("dynamic email selector must be an immutable tuple")
def to_primitive(self) -> dict[str, object]:
return {"type": RecipientSpecificationKind.DYNAMIC_EMAIL.value, "selector": list(self.selector)}
@dataclass(frozen=True, slots=True)
class CurrentInitiatorRecipientSpecification:
"""Request-scoped current initiator recipient marker."""
def to_primitive(self) -> dict[str, object]:
return {"type": RecipientSpecificationKind.INITIATOR.value}
type RecipientSpecification = (
ContactRecipientSpecification
| OneTimeEmailRecipientSpecification
| DynamicEmailRecipientSpecification
| CurrentInitiatorRecipientSpecification
)
@dataclass(frozen=True, slots=True)
class UnsupportedDynamicRecipientValue:
"""Safe snapshot of a non-string workflow value without retaining its graph."""
value_type: str
def __post_init__(self) -> None:
if not self.value_type:
raise ValueError("unsupported dynamic recipient value type must not be blank")
@dataclass(frozen=True, slots=True)
class DynamicRecipientValue:
"""One evaluated selector value captured for a single resolution request."""
selector: tuple[str, ...]
value: str | UnsupportedDynamicRecipientValue
def __post_init__(self) -> None:
if not isinstance(self.selector, tuple):
raise TypeError("dynamic recipient value selector must be an immutable tuple")
@classmethod
def from_runtime(cls, selector: tuple[str, ...], value: object) -> DynamicRecipientValue:
"""Capture a runtime value without retaining mutable unsupported data."""
captured_value: str | UnsupportedDynamicRecipientValue
if isinstance(value, str):
captured_value = value
else:
captured_value = UnsupportedDynamicRecipientValue(type(value).__name__)
return cls(selector=selector, value=captured_value)
class WorkflowRecipientSpecificationAdapter:
"""Convert versioned workflow node values into approval-domain inputs."""
@staticmethod
def from_node_data(node_data: HumanInputNodeData) -> tuple[RecipientSpecification, ...]:
"""Copy ordered v2 node recipients into immutable domain values."""
specifications: list[RecipientSpecification] = []
for configured_recipient in node_data.recipients_spec:
specification: RecipientSpecification
if isinstance(configured_recipient, WorkflowContactRecipient):
specification = ContactRecipientSpecification(contact_id=configured_recipient.contact_id)
elif isinstance(configured_recipient, WorkflowOneTimeEmailRecipient):
specification = OneTimeEmailRecipientSpecification(email=configured_recipient.email)
elif isinstance(configured_recipient, WorkflowDynamicEmailRecipient):
specification = DynamicEmailRecipientSpecification(selector=tuple(configured_recipient.selector))
elif isinstance(configured_recipient, WorkflowInitiatorRecipient):
specification = CurrentInitiatorRecipientSpecification()
else:
assert_never(configured_recipient)
specifications.append(specification)
return tuple(specifications)
@@ -0,0 +1,405 @@
"""Pure current-state authorization for Human Input v2 submissions.
The module owns the cross-snapshot decision only. Callers must verify transport
credentials before constructing a proof and must load one coherent
``AuthorizationContext`` through persistence. The authorizer performs no I/O,
does not retain raw credentials, and never reloads Contact or IM binding facts.
"""
from __future__ import annotations
from dataclasses import dataclass
from enum import StrEnum
from typing import assert_never
from core.human_input_v2.entities import IMProvider
from core.human_input_v2.shared import (
AccountId,
AppId,
ContactId,
EndUserId,
IMBindingId,
IMIdentityId,
IntegrationId,
NormalizedEmail,
UtcTimestamp,
)
from .delivery import DeliveryEndpoint
from .form import (
FormInactiveReason,
HumanInputForm,
InactiveFormState,
InvalidApproverGrantError,
InvalidSelectedActionError,
SubmissionTransitionDecision,
)
from .grants import ApproverGrant, DeliveryEndpointRef
from .otp import ContactOTPSubject, EmailAddressOTPSubject, VerifiedEmailOTPProof
from .recipient_resolution import ContactApprovalSubject, EmailAddressApprovalSubject, EndUserApprovalSubject
@dataclass(frozen=True, slots=True)
class VerifiedAccountSessionProof:
"""Current Account identity produced by a trusted session verifier."""
account_id: AccountId
@dataclass(frozen=True, slots=True)
class VerifiedTrustedEndUserProof:
"""Current EndUser identity produced by a trusted app-token boundary."""
end_user_id: EndUserId
app_id: AppId
@dataclass(frozen=True, slots=True)
class VerifiedIMIdentityProof:
"""Current provider identity evidence without callback credentials or payloads."""
integration_id: IntegrationId
identity_id: IMIdentityId
binding_id: IMBindingId | None
provider: IMProvider
provider_tenant_id: str
provider_user_id: str
def __post_init__(self) -> None:
if not self.provider_tenant_id.strip() or not self.provider_user_id.strip():
raise ValueError("verified IM provider identities must not be blank")
type VerifiedSubmissionProof = (
VerifiedAccountSessionProof | VerifiedTrustedEndUserProof | VerifiedEmailOTPProof | VerifiedIMIdentityProof
)
@dataclass(frozen=True, slots=True)
class AccountSubmissionActor:
"""Current Dify Account that completed a submission."""
account_id: AccountId
@dataclass(frozen=True, slots=True)
class EndUserSubmissionActor:
"""Current app-scoped EndUser that completed a submission."""
end_user_id: EndUserId
@dataclass(frozen=True, slots=True)
class EmailAddressSubmissionActor:
"""Verified normalized Email identity that completed a submission."""
normalized_email: NormalizedEmail
type SubmissionActor = AccountSubmissionActor | EndUserSubmissionActor | EmailAddressSubmissionActor
@dataclass(frozen=True, slots=True)
class CurrentContactAuthorizationFacts:
"""Current Contact incarnation, Email, Account, and workspace availability."""
contact_id: ContactId
account_id: AccountId | None
normalized_email: NormalizedEmail | None
account_active: bool
workspace_available: bool
@dataclass(frozen=True, slots=True)
class CurrentEndUserAuthorizationFacts:
"""Current tenant/app ownership facts for one EndUser identity."""
end_user_id: EndUserId
app_id: AppId
workspace_available: bool
@dataclass(frozen=True, slots=True)
class CurrentIMAuthorizationFacts:
"""Credential-free effective IM binding observed in the authorization snapshot."""
integration_id: IntegrationId
provider: IMProvider
provider_tenant_id: str
contact_id: ContactId
account_id: AccountId | None
identity_id: IMIdentityId
binding_id: IMBindingId | None
provider_user_id: str
@dataclass(frozen=True, slots=True)
class AuthorizationContext:
"""One immutable tenant-scoped view used without later identity reloads."""
form: HumanInputForm
grant: ApproverGrant
endpoint: DeliveryEndpoint | None
current_contact: CurrentContactAuthorizationFacts | None
current_end_user: CurrentEndUserAuthorizationFacts | None
current_im_binding: CurrentIMAuthorizationFacts | None
def __post_init__(self) -> None:
if self.grant.ref.form_ref != self.form.ref or self.grant not in self.form.grants:
raise ValueError("authorization grant does not belong to the form snapshot")
if self.endpoint is not None and self.endpoint.grant_ref != self.grant.ref:
raise ValueError("authorization endpoint does not belong to the target grant")
class SubmissionAuthorizationRejection(StrEnum):
"""Stable transport-neutral reasons for denied submission authority."""
RAW_CREDENTIAL_NOT_VERIFIED = "raw_credential_not_verified"
FORM_ALREADY_SUBMITTED = "form_already_submitted"
FORM_TIMED_OUT = "form_timed_out"
FORM_STATUS_EXPIRED = "form_status_expired"
FORM_GLOBALLY_EXPIRED = "form_globally_expired"
GRANT_NOT_MATCHED = "grant_not_matched"
INVALID_SELECTED_ACTION = "invalid_selected_action"
STALE_IDENTITY = "stale_identity"
ACCOUNT_DISABLED = "account_disabled"
WORKSPACE_UNAVAILABLE = "workspace_unavailable"
END_USER_UNAVAILABLE = "end_user_unavailable"
IM_BINDING_CHANGED = "im_binding_changed"
@dataclass(frozen=True, slots=True)
class AuthorizedSubmission:
"""Current authority and local form transition prepared for atomic persistence."""
transition: SubmissionTransitionDecision
proof: VerifiedSubmissionProof
actor: SubmissionActor
endpoint_ref: DeliveryEndpointRef | None
@dataclass(frozen=True, slots=True)
class SubmissionAuthorizationDecision:
"""Exactly one authorized value or stable rejection."""
authorized: AuthorizedSubmission | None
rejection: SubmissionAuthorizationRejection | None
def __post_init__(self) -> None:
if (self.authorized is None) == (self.rejection is None):
raise ValueError("authorization decision requires exactly one outcome")
@classmethod
def accept(cls, authorized: AuthorizedSubmission) -> SubmissionAuthorizationDecision:
return cls(authorized, None)
@classmethod
def reject(cls, reason: SubmissionAuthorizationRejection) -> SubmissionAuthorizationDecision:
return cls(None, reason)
class SubmissionAuthorizer:
"""Stateless cross-snapshot policy that resolves one current business actor."""
@classmethod
def authorize(
cls,
*,
context: AuthorizationContext,
proof: object,
selected_action_id: str,
now: UtcTimestamp,
) -> SubmissionAuthorizationDecision:
"""Authorize verified proof against one already-loaded coherent context."""
if not isinstance(
proof,
VerifiedAccountSessionProof | VerifiedTrustedEndUserProof | VerifiedEmailOTPProof | VerifiedIMIdentityProof,
):
return SubmissionAuthorizationDecision.reject(SubmissionAuthorizationRejection.RAW_CREDENTIAL_NOT_VERIFIED)
state = context.form.state_at(now)
if isinstance(state, InactiveFormState):
return SubmissionAuthorizationDecision.reject(cls._inactive_rejection(state.reason))
try:
transition = context.form.decide_submission(
grant_id=context.grant.id,
selected_action_id=selected_action_id,
now=now,
)
except InvalidApproverGrantError:
return SubmissionAuthorizationDecision.reject(SubmissionAuthorizationRejection.GRANT_NOT_MATCHED)
except InvalidSelectedActionError:
return SubmissionAuthorizationDecision.reject(SubmissionAuthorizationRejection.INVALID_SELECTED_ACTION)
if isinstance(transition, InactiveFormState):
return SubmissionAuthorizationDecision.reject(cls._inactive_rejection(transition.reason))
actor_or_rejection = cls._resolve_actor(context, proof)
if isinstance(actor_or_rejection, SubmissionAuthorizationRejection):
return SubmissionAuthorizationDecision.reject(actor_or_rejection)
return SubmissionAuthorizationDecision.accept(
AuthorizedSubmission(
transition=transition,
proof=proof,
actor=actor_or_rejection,
endpoint_ref=context.endpoint.ref if context.endpoint is not None else None,
)
)
@staticmethod
def _inactive_rejection(reason: FormInactiveReason) -> SubmissionAuthorizationRejection:
match reason:
case FormInactiveReason.SUBMITTED:
return SubmissionAuthorizationRejection.FORM_ALREADY_SUBMITTED
case FormInactiveReason.TIMED_OUT:
return SubmissionAuthorizationRejection.FORM_TIMED_OUT
case FormInactiveReason.STATUS_EXPIRED:
return SubmissionAuthorizationRejection.FORM_STATUS_EXPIRED
case FormInactiveReason.GLOBALLY_EXPIRED:
return SubmissionAuthorizationRejection.FORM_GLOBALLY_EXPIRED
assert_never(reason)
@classmethod
def _resolve_actor(
cls,
context: AuthorizationContext,
proof: VerifiedSubmissionProof,
) -> SubmissionActor | SubmissionAuthorizationRejection:
match proof:
case VerifiedAccountSessionProof():
return cls._authorize_account(context, proof)
case VerifiedTrustedEndUserProof():
return cls._authorize_end_user(context, proof)
case VerifiedEmailOTPProof():
return cls._authorize_email(context, proof)
case VerifiedIMIdentityProof():
return cls._authorize_im(context, proof)
assert_never(proof)
@classmethod
def _authorize_account(
cls,
context: AuthorizationContext,
proof: VerifiedAccountSessionProof,
) -> SubmissionActor | SubmissionAuthorizationRejection:
if not isinstance(context.grant.subject, ContactApprovalSubject):
return SubmissionAuthorizationRejection.GRANT_NOT_MATCHED
current = cls._validate_current_contact(context, context.grant.subject.contact_id)
if isinstance(current, SubmissionAuthorizationRejection):
return current
if current.account_id != proof.account_id:
return SubmissionAuthorizationRejection.GRANT_NOT_MATCHED
return AccountSubmissionActor(proof.account_id)
@staticmethod
def _authorize_end_user(
context: AuthorizationContext,
proof: VerifiedTrustedEndUserProof,
) -> SubmissionActor | SubmissionAuthorizationRejection:
if not isinstance(context.grant.subject, EndUserApprovalSubject):
return SubmissionAuthorizationRejection.GRANT_NOT_MATCHED
current = context.current_end_user
if (
current is None
or current.end_user_id != context.grant.subject.end_user_id
or current.end_user_id != proof.end_user_id
or current.app_id != context.form.app_id
or current.app_id != proof.app_id
):
return SubmissionAuthorizationRejection.END_USER_UNAVAILABLE
if not current.workspace_available:
return SubmissionAuthorizationRejection.WORKSPACE_UNAVAILABLE
return EndUserSubmissionActor(current.end_user_id)
@classmethod
def _authorize_email(
cls,
context: AuthorizationContext,
proof: VerifiedEmailOTPProof,
) -> SubmissionActor | SubmissionAuthorizationRejection:
if proof.challenge_ref.grant_ref != context.grant.ref:
return SubmissionAuthorizationRejection.GRANT_NOT_MATCHED
subject = context.grant.subject
if isinstance(subject, EmailAddressApprovalSubject):
if (
not isinstance(proof.subject, EmailAddressOTPSubject)
or proof.subject.normalized_email != subject.normalized_email
or proof.normalized_email != subject.normalized_email
):
return SubmissionAuthorizationRejection.GRANT_NOT_MATCHED
return EmailAddressSubmissionActor(subject.normalized_email)
if not isinstance(subject, ContactApprovalSubject) or not isinstance(proof.subject, ContactOTPSubject):
return SubmissionAuthorizationRejection.GRANT_NOT_MATCHED
if proof.subject.contact_id != subject.contact_id:
return SubmissionAuthorizationRejection.GRANT_NOT_MATCHED
current = cls._validate_current_contact(context, subject.contact_id)
if isinstance(current, SubmissionAuthorizationRejection):
return current
if current.normalized_email is None or current.normalized_email != proof.normalized_email:
return SubmissionAuthorizationRejection.STALE_IDENTITY
if current.account_id is not None:
return AccountSubmissionActor(current.account_id)
return EmailAddressSubmissionActor(proof.normalized_email)
@classmethod
def _authorize_im(
cls,
context: AuthorizationContext,
proof: VerifiedIMIdentityProof,
) -> SubmissionActor | SubmissionAuthorizationRejection:
if not isinstance(context.grant.subject, ContactApprovalSubject):
return SubmissionAuthorizationRejection.GRANT_NOT_MATCHED
current_contact = cls._validate_current_contact(context, context.grant.subject.contact_id)
if isinstance(current_contact, SubmissionAuthorizationRejection):
return current_contact
if current_contact.account_id is None:
return SubmissionAuthorizationRejection.STALE_IDENTITY
current_im = context.current_im_binding
if current_im is None:
return SubmissionAuthorizationRejection.IM_BINDING_CHANGED
if (
current_im.contact_id != context.grant.subject.contact_id
or current_im.account_id != current_contact.account_id
or current_im.integration_id != proof.integration_id
or current_im.identity_id != proof.identity_id
or current_im.binding_id != proof.binding_id
or current_im.provider is not proof.provider
or current_im.provider_tenant_id != proof.provider_tenant_id
or current_im.provider_user_id != proof.provider_user_id
):
return SubmissionAuthorizationRejection.IM_BINDING_CHANGED
return AccountSubmissionActor(current_contact.account_id)
@staticmethod
def _validate_current_contact(
context: AuthorizationContext,
contact_id: ContactId,
) -> CurrentContactAuthorizationFacts | SubmissionAuthorizationRejection:
current = context.current_contact
if current is None or current.contact_id != contact_id:
return SubmissionAuthorizationRejection.STALE_IDENTITY
if current.account_id is not None and not current.account_active:
return SubmissionAuthorizationRejection.ACCOUNT_DISABLED
if not current.workspace_available:
return SubmissionAuthorizationRejection.WORKSPACE_UNAVAILABLE
return current
__all__ = [
"AccountSubmissionActor",
"AuthorizationContext",
"AuthorizedSubmission",
"CurrentContactAuthorizationFacts",
"CurrentEndUserAuthorizationFacts",
"CurrentIMAuthorizationFacts",
"EmailAddressSubmissionActor",
"EndUserSubmissionActor",
"SubmissionActor",
"SubmissionAuthorizationDecision",
"SubmissionAuthorizationRejection",
"SubmissionAuthorizer",
"VerifiedAccountSessionProof",
"VerifiedIMIdentityProof",
"VerifiedSubmissionProof",
"VerifiedTrustedEndUserProof",
]
@@ -0,0 +1,117 @@
"""Transaction-oriented ports for current authorization and first-success commit.
The transaction object keeps one coherent authorization context alive through
the winning write set. Generic CRUD is intentionally absent; persistence owns
the Form lock, rejection audit append, and atomic authorized commit shape.
"""
from __future__ import annotations
from contextlib import AbstractContextManager
from dataclasses import dataclass
from enum import StrEnum
from typing import Protocol
from core.human_input_v2.shared import (
ApproverGrantId,
AuditEventId,
DeliveryEndpointId,
SubmissionId,
UtcTimestamp,
)
from .frozen_values import FrozenJSONObject
from .grants import FormRef
from .submission_authorization import AuthorizationContext, AuthorizedSubmission
from .submission_records import FormAuthorizationAuditEvent, FormSubmission
class RetryableSubmissionPersistenceError(RuntimeError):
"""A complete submission transaction must be retried with a fresh snapshot."""
@dataclass(frozen=True, slots=True)
class SubmissionAttemptScope:
"""Complete logical owner chain selected before the transaction begins."""
form_ref: FormRef
approver_grant_id: ApproverGrantId
endpoint_id: DeliveryEndpointId | None
@dataclass(frozen=True, slots=True)
class AuthorizedSubmissionCommit:
"""Caller-owned identities and structured values for one authorized write set."""
submission_id: SubmissionId
authorization_audit_event_id: AuditEventId
authorized: AuthorizedSubmission
input_snapshot: FrozenJSONObject
canonical_values: FrozenJSONObject
def to_submission(
self,
*,
form_ref: FormRef,
approver_grant_id: ApproverGrantId,
endpoint_id: DeliveryEndpointId | None,
submitted_at: UtcTimestamp,
) -> FormSubmission:
"""Build the immutable record value after owner-scope validation."""
return FormSubmission(
id=self.submission_id,
form_ref=form_ref,
approver_grant_id=approver_grant_id,
endpoint_id=endpoint_id,
authorization_audit_event_id=self.authorization_audit_event_id,
actor=self.authorized.actor,
selected_action_id=self.authorized.transition.selected_action_id,
input_snapshot=self.input_snapshot,
canonical_values=self.canonical_values,
submitted_at=submitted_at,
created_at=submitted_at,
updated_at=submitted_at,
)
class SubmissionCommitStatus(StrEnum):
"""Stable first-success persistence outcome."""
COMMITTED = "committed"
ALREADY_COMPLETED = "already_completed"
@dataclass(frozen=True, slots=True)
class SubmissionCommitResult:
"""Committed winning submission or stable loser result."""
status: SubmissionCommitStatus
submission: FormSubmission | None
class SubmissionTransaction(Protocol):
"""One session-bound authorization and commit transaction."""
def load_authorization_context(self, *, proof: object) -> AuthorizationContext: ...
def append_rejection_audit(self, event: FormAuthorizationAuditEvent) -> None: ...
def commit_authorized_submission_once(self, commit: AuthorizedSubmissionCommit) -> SubmissionCommitResult: ...
class SubmissionRepository(Protocol):
"""Factory for one short transaction owning the complete submission use case."""
def transaction(self, scope: SubmissionAttemptScope) -> AbstractContextManager[SubmissionTransaction]: ...
__all__ = [
"AuthorizedSubmissionCommit",
"RetryableSubmissionPersistenceError",
"SubmissionAttemptScope",
"SubmissionCommitResult",
"SubmissionCommitStatus",
"SubmissionRepository",
"SubmissionTransaction",
]
@@ -0,0 +1,102 @@
"""Immutable submission and shared authorization-audit persistence values.
These values preserve business identity and structured snapshots without
exposing ORM records. Persistence mappers alone translate them to storage
columns and Pydantic JSON values.
"""
from __future__ import annotations
from dataclasses import dataclass
from enum import StrEnum
from core.human_input_v2.entities import HumanInputDeliveryChannel
from core.human_input_v2.shared import (
ApproverGrantId,
AuditEventId,
DeliveryEndpointId,
SubmissionId,
UtcTimestamp,
)
from .frozen_values import FrozenJSONObject
from .grants import FormRef
from .submission_authorization import SubmissionActor, VerifiedEmailOTPProof, VerifiedSubmissionProof
class FormAuthorizationAuditEventType(StrEnum):
"""Stable append-only event names owned by the shared audit table."""
OTP_CHALLENGE_ISSUED = "otp_challenge_issued"
SUBMISSION_AUTHORIZED = "submission_authorized"
SUBMISSION_REJECTED = "submission_rejected"
@dataclass(frozen=True, slots=True)
class FormAuthorizationAuditEvent:
"""Secret-free authorized, rejected, or OTP issuance audit fact."""
id: AuditEventId
event_type: FormAuthorizationAuditEventType
form_ref: FormRef
approver_grant_id: ApproverGrantId | None
endpoint_id: DeliveryEndpointId | None
channel: HumanInputDeliveryChannel | None
reason_code: str | None
reason_message: str | None
authorization_proof: VerifiedSubmissionProof | None
payload: FrozenJSONObject | None
occurred_at: UtcTimestamp
created_at: UtcTimestamp
updated_at: UtcTimestamp
def __post_init__(self) -> None:
if self.event_type is FormAuthorizationAuditEventType.SUBMISSION_AUTHORIZED:
if self.approver_grant_id is None or self.authorization_proof is None:
raise ValueError("authorized audit event requires a grant and verified proof")
if self.reason_code is not None:
raise ValueError("authorized audit event cannot contain a rejection reason")
self.validate_authorization_proof_owner()
if self.event_type is FormAuthorizationAuditEventType.SUBMISSION_REJECTED and not self.reason_code:
raise ValueError("rejected audit event requires a stable reason code")
def validate_authorization_proof_owner(self) -> None:
"""Reject authorized Email evidence captured for another form or grant."""
proof = self.authorization_proof
if not isinstance(proof, VerifiedEmailOTPProof):
return
if (
proof.challenge_ref.form_ref != self.form_ref
or proof.challenge_ref.grant_ref.grant_id != self.approver_grant_id
):
raise ValueError("authorized Email proof owner does not match the audit event")
@dataclass(frozen=True, slots=True)
class FormSubmission:
"""Immutable winning submission mapped independently from ORM lifetime."""
id: SubmissionId
form_ref: FormRef
approver_grant_id: ApproverGrantId
endpoint_id: DeliveryEndpointId | None
authorization_audit_event_id: AuditEventId
actor: SubmissionActor
selected_action_id: str
input_snapshot: FrozenJSONObject
canonical_values: FrozenJSONObject
submitted_at: UtcTimestamp
created_at: UtcTimestamp
updated_at: UtcTimestamp
def __post_init__(self) -> None:
if not self.selected_action_id.strip():
raise ValueError("submission selected action must not be blank")
__all__ = [
"FormAuthorizationAuditEvent",
"FormAuthorizationAuditEventType",
"FormSubmission",
]
@@ -0,0 +1,37 @@
"""Infrastructure-free Contact Directory domain boundary.
Transport and persistence layers may depend on this package. This package must
not import controllers, Flask, SQLAlchemy, sessions, or ORM records.
"""
from .entities import (
Contact,
ContactIdentitySource,
ContactOwner,
ContactSnapshot,
ExternalContactOwner,
OrganizationAccountOwner,
PlatformWorkspaceEntry,
WorkspaceMemberOwner,
)
from .errors import ContactDirectoryError, ContactRejection, ContactRejectionCode
from .policy import ContactDirectoryPolicy, ContactDirectorySnapshot, ContactResolution
from .ports import ContactDirectoryRepository
__all__ = [
"Contact",
"ContactDirectoryError",
"ContactDirectoryPolicy",
"ContactDirectoryRepository",
"ContactDirectorySnapshot",
"ContactIdentitySource",
"ContactOwner",
"ContactRejection",
"ContactRejectionCode",
"ContactResolution",
"ContactSnapshot",
"ExternalContactOwner",
"OrganizationAccountOwner",
"PlatformWorkspaceEntry",
"WorkspaceMemberOwner",
]
@@ -0,0 +1,229 @@
"""Canonical Contact identity independent of workspace-relative resolution.
Contacts own identity invariants and current profile facts. Membership,
allow-list state, database I/O, and transport serialization remain outside the
entity so its immutable lifecycle source cannot be confused with a query result.
"""
from __future__ import annotations
from dataclasses import dataclass
from enum import StrEnum
from core.human_input_v2.shared import (
AccountId,
ContactId,
DeploymentScope,
DirectoryScope,
NormalizedEmail,
PlatformEntryId,
UtcTimestamp,
WorkspaceId,
WorkspaceScope,
)
from .errors import ContactRejectionCode, reject
class ContactIdentitySource(StrEnum):
"""Immutable lifecycle source of a canonical Contact."""
ORGANIZATION_ACCOUNT = "organization_account"
WORKSPACE_MEMBER = "workspace_member"
EXTERNAL = "external"
@dataclass(frozen=True, slots=True)
class OrganizationAccountOwner:
"""Deployment-wide EE owner reference backed by one Account."""
account_id: AccountId
@dataclass(frozen=True, slots=True)
class WorkspaceMemberOwner:
"""Workspace owner reference backed by one current or historical Account."""
workspace_id: WorkspaceId
account_id: AccountId
@dataclass(frozen=True, slots=True)
class ExternalContactOwner:
"""Workspace owner reference for an address managed by administrators."""
workspace_id: WorkspaceId
type ContactOwner = OrganizationAccountOwner | WorkspaceMemberOwner | ExternalContactOwner
@dataclass(frozen=True, slots=True)
class Contact:
"""Canonical identity whose source and owner remain immutable.
Use the named factories for normal construction. ``create`` exists for
persistence mapping and validates the same source/owner invariant.
"""
id: ContactId
identity_source: ContactIdentitySource
owner: ContactOwner
name: str
normalized_name: str
email: str | None
normalized_email: NormalizedEmail | None
avatar_file_id: str | None
created_at: UtcTimestamp
updated_at: UtcTimestamp
def __post_init__(self) -> None:
expected_owner_type = {
ContactIdentitySource.ORGANIZATION_ACCOUNT: OrganizationAccountOwner,
ContactIdentitySource.WORKSPACE_MEMBER: WorkspaceMemberOwner,
ContactIdentitySource.EXTERNAL: ExternalContactOwner,
}[self.identity_source]
if not isinstance(self.owner, expected_owner_type):
raise reject(ContactRejectionCode.INVALID_OWNER)
if not self.name.strip():
raise reject(ContactRejectionCode.INVALID_NAME)
if self.identity_source is ContactIdentitySource.EXTERNAL and self.normalized_email is None:
raise reject(ContactRejectionCode.INVALID_EMAIL)
if self.email is None and self.normalized_email is not None:
raise reject(ContactRejectionCode.INVALID_EMAIL)
if self.email is not None:
try:
normalized_email = NormalizedEmail(self.email)
except ValueError as error:
raise reject(ContactRejectionCode.INVALID_EMAIL) from error
if self.normalized_email != normalized_email:
raise reject(ContactRejectionCode.INVALID_EMAIL)
object.__setattr__(self, "email", self.email.strip())
normalized_name = self.name.strip().casefold()
if self.normalized_name != normalized_name:
object.__setattr__(self, "normalized_name", normalized_name)
object.__setattr__(self, "name", self.name.strip())
@classmethod
def create(
cls,
*,
contact_id: ContactId,
identity_source: ContactIdentitySource,
owner: ContactOwner,
name: str,
email: str | None,
now: UtcTimestamp,
avatar_file_id: str | None = None,
created_at: UtcTimestamp | None = None,
) -> Contact:
normalized_email: NormalizedEmail | None = None
if email is not None:
try:
normalized_email = NormalizedEmail(email)
except ValueError as error:
raise reject(ContactRejectionCode.INVALID_EMAIL) from error
return cls(
id=contact_id,
identity_source=identity_source,
owner=owner,
name=name,
normalized_name=name.strip().casefold(),
email=email,
normalized_email=normalized_email,
avatar_file_id=avatar_file_id,
created_at=created_at or now,
updated_at=now,
)
@classmethod
def organization_account(
cls,
*,
contact_id: ContactId,
account_id: AccountId,
name: str,
email: str | None,
now: UtcTimestamp,
) -> Contact:
return cls.create(
contact_id=contact_id,
identity_source=ContactIdentitySource.ORGANIZATION_ACCOUNT,
owner=OrganizationAccountOwner(account_id),
name=name,
email=email,
now=now,
)
@classmethod
def workspace_member(
cls,
*,
contact_id: ContactId,
workspace_id: WorkspaceId,
account_id: AccountId,
name: str,
email: str | None,
now: UtcTimestamp,
) -> Contact:
return cls.create(
contact_id=contact_id,
identity_source=ContactIdentitySource.WORKSPACE_MEMBER,
owner=WorkspaceMemberOwner(workspace_id, account_id),
name=name,
email=email,
now=now,
)
@classmethod
def external(
cls,
*,
contact_id: ContactId,
workspace_id: WorkspaceId,
name: str,
email: str,
now: UtcTimestamp,
avatar_file_id: str | None = None,
) -> Contact:
return cls.create(
contact_id=contact_id,
identity_source=ContactIdentitySource.EXTERNAL,
owner=ExternalContactOwner(workspace_id),
name=name,
email=email,
now=now,
avatar_file_id=avatar_file_id,
)
@property
def account_id(self) -> AccountId | None:
if isinstance(self.owner, OrganizationAccountOwner | WorkspaceMemberOwner):
return self.owner.account_id
return None
@property
def directory_scope(self) -> DirectoryScope:
if isinstance(self.owner, OrganizationAccountOwner):
return DeploymentScope()
return WorkspaceScope(self.owner.workspace_id)
@dataclass(frozen=True, slots=True)
class ContactSnapshot:
"""Immutable Contact plus current Account availability for one operation."""
contact: Contact
account_available: bool
@dataclass(frozen=True, slots=True)
class PlatformWorkspaceEntry:
"""One workspace allow-list fact for an Organization Account Contact."""
id: PlatformEntryId
workspace_id: WorkspaceId
contact_id: ContactId
added_by_account_id: AccountId
created_at: UtcTimestamp
updated_at: UtcTimestamp
@@ -0,0 +1,48 @@
"""Transport-neutral Contact Directory rejection contracts."""
from dataclasses import dataclass
from enum import StrEnum
class ContactRejectionCode(StrEnum):
"""Stable machine-readable reasons returned by Contact Directory operations."""
INVALID_OWNER = "invalid_owner"
INVALID_EMAIL = "invalid_email"
INVALID_NAME = "invalid_name"
CONFLICTING_IDENTITY = "conflicting_identity"
CROSS_ORGANIZATION = "cross_organization"
ACCOUNT_UNAVAILABLE = "account_unavailable"
CONTACT_NOT_FOUND = "contact_not_found"
SETUP_ROW_MISSING = "setup_row_missing"
PERSISTENCE_FAILURE = "persistence_failure"
@dataclass(frozen=True, slots=True)
class ContactRejection:
"""Serializable domain rejection without HTTP or RPC semantics."""
reason: ContactRejectionCode
def to_primitive(self) -> dict[str, str]:
return {"reason": self.reason.value}
class ContactDirectoryError(Exception):
"""Exception carrier for one transport-neutral Contact rejection."""
rejection: ContactRejection
def __init__(self, rejection: ContactRejection) -> None:
self.rejection = rejection
super().__init__(rejection.reason.value)
@property
def code(self) -> ContactRejectionCode:
return self.rejection.reason
def reject(reason: ContactRejectionCode) -> ContactDirectoryError:
"""Build a domain exception while keeping reason construction consistent."""
return ContactDirectoryError(ContactRejection(reason))
@@ -0,0 +1,101 @@
"""Pure Contact Directory resolution and lifecycle policies.
The immutable snapshot supplies all operation-scoped facts. Policies never load
membership, Account, allow-list, or Contact records themselves.
"""
from __future__ import annotations
from dataclasses import dataclass
from enum import StrEnum
from core.human_input_v2.shared import AccountId, ContactId, NormalizedEmail, UtcTimestamp, WorkspaceId
from .entities import Contact, ExternalContactOwner, OrganizationAccountOwner, WorkspaceMemberOwner
from .errors import ContactRejectionCode, reject
class ContactResolution(StrEnum):
"""Workspace-relative availability of one canonical Contact."""
WORKSPACE = "workspace"
PLATFORM = "platform"
EXTERNAL = "external"
ABSENT = "absent"
@dataclass(frozen=True, slots=True)
class ContactDirectorySnapshot:
"""Coherent, request-scoped Contact facts for one workspace.
The snapshot is deliberately not a cache. Authorization callers that need
current facts must load a new snapshot in their own operation.
"""
workspace_id: WorkspaceId
contacts: tuple[Contact, ...] = ()
member_account_ids: frozenset[AccountId] = frozenset()
platform_contact_ids: frozenset[ContactId] = frozenset()
unavailable_account_ids: frozenset[AccountId] = frozenset()
def find(self, contact_id: ContactId) -> Contact | None:
return next((contact for contact in self.contacts if contact.id == contact_id), None)
class ContactDirectoryPolicy:
"""Stateless policy for workspace resolution and External lifecycle rules."""
@staticmethod
def resolve_for_workspace(snapshot: ContactDirectorySnapshot, contact_id: ContactId) -> ContactResolution:
contact = snapshot.find(contact_id)
if contact is None:
return ContactResolution.ABSENT
owner = contact.owner
if isinstance(owner, ExternalContactOwner):
if owner.workspace_id != snapshot.workspace_id:
raise reject(ContactRejectionCode.CROSS_ORGANIZATION)
return ContactResolution.EXTERNAL
if isinstance(owner, WorkspaceMemberOwner) and owner.workspace_id != snapshot.workspace_id:
raise reject(ContactRejectionCode.CROSS_ORGANIZATION)
account_id = contact.account_id
if account_id is None or account_id in snapshot.unavailable_account_ids:
return ContactResolution.ABSENT
if account_id in snapshot.member_account_ids:
return ContactResolution.WORKSPACE
if isinstance(owner, OrganizationAccountOwner) and contact.id in snapshot.platform_contact_ids:
return ContactResolution.PLATFORM
return ContactResolution.ABSENT
@staticmethod
def admit_external(
snapshot: ContactDirectorySnapshot,
*,
contact_id: ContactId,
name: str,
email: str,
now: UtcTimestamp,
avatar_file_id: str | None = None,
) -> Contact:
try:
normalized_email = NormalizedEmail(email)
except ValueError as error:
raise reject(ContactRejectionCode.INVALID_EMAIL) from error
if any(contact.normalized_email == normalized_email for contact in snapshot.contacts):
raise reject(ContactRejectionCode.CONFLICTING_IDENTITY)
return Contact.external(
contact_id=contact_id,
workspace_id=snapshot.workspace_id,
name=name,
email=email,
now=now,
avatar_file_id=avatar_file_id,
)
@staticmethod
def ensure_external_deletable(contact: Contact, workspace_id: WorkspaceId) -> None:
if not isinstance(contact.owner, ExternalContactOwner):
raise reject(ContactRejectionCode.INVALID_OWNER)
if contact.owner.workspace_id != workspace_id:
raise reject(ContactRejectionCode.CROSS_ORGANIZATION)
@@ -0,0 +1,47 @@
"""Aggregate-oriented persistence ports for Contact Directory operations.
Implementations own transactions, owner predicates, locking, mapping, and
rollback. Callers receive domain values and never persistence records.
"""
from typing import Protocol
from core.human_input_v2.shared import AccountId, ContactId, WorkspaceId
from .entities import Contact
from .policy import ContactDirectorySnapshot
class ContactDirectoryRepository(Protocol):
"""Persistence contract centered on coherent directory invariants."""
def load_snapshot(self, workspace_id: WorkspaceId) -> ContactDirectorySnapshot:
"""Load one immutable workspace-scoped directory view."""
...
def save_organization_contact(self, contact: Contact) -> Contact:
"""Create or update one deployment-owned Organization Contact with a serialized Email claim."""
...
def save_workspace_member_contact(self, contact: Contact) -> Contact:
"""Create or update one Contact backed by current workspace membership."""
...
def admit_external(self, workspace_id: WorkspaceId, *, name: str, email: str) -> Contact:
"""Atomically admit one External Contact against tenant and configured Organization identities."""
...
def set_platform_availability(
self,
workspace_id: WorkspaceId,
contact_id: ContactId,
*,
added_by_account_id: AccountId,
enabled: bool,
) -> None:
"""Atomically add or remove one EE Platform allow-list fact."""
...
def hard_delete_external(self, workspace_id: WorkspaceId, contact_id: ContactId) -> None:
"""Delete an External Contact without retaining an identity tombstone."""
...
+197
View File
@@ -0,0 +1,197 @@
from enum import StrEnum
from typing import NewType
from core.human_input_v2.shared import AccountId, NormalizedEmail, UtcTimestamp, WorkspaceId
class HumanInputContactType(StrEnum):
"""Concrete contact classification resolved in one workspace."""
WORKSPACE = "workspace"
PLATFORM = "platform"
EXTERNAL = "external"
class HumanInputApproverGrantSubjectType(StrEnum):
"""Business subject receiving approval authority for one Human Input form."""
CONTACT = "contact"
END_USER = "end_user"
EMAIL_ADDRESS = "email_address"
class HumanInputSubmissionActorType(StrEnum):
"""Business identity that completed one Human Input form submission."""
ACCOUNT = "account"
END_USER = "end_user"
EMAIL_ADDRESS = "email_address"
class HumanInputV2FormKind(StrEnum):
"""Persistence kind for an independently stored Human Input v2 form."""
RUNTIME = "runtime"
DELIVERY_TEST = "delivery_test"
class HumanInputV2FormStatus(StrEnum):
"""Lifecycle state of an independently stored Human Input v2 form."""
WAITING = "waiting"
EXPIRED = "expired"
SUBMITTED = "submitted"
TIMEOUT = "timeout"
class HumanInputAuthorizationProofType(StrEnum):
"""Verified evidence type retained for a Human Input authorization audit event."""
ACCOUNT_SESSION = "account_session"
EMAIL_OTP = "email_otp"
IM_IDENTITY = "im_identity"
TRUSTED_END_USER = "trusted_end_user"
class HumanInputDeliveryChannel(StrEnum):
"""Notification or interaction channel frozen for one form endpoint."""
EMAIL = "email"
IM = "im"
WEB = "web"
CONSOLE = "console"
class HumanInputDeliveryAttemptStatus(StrEnum):
"""Delivery lifecycle kept separate from the form state machine."""
QUEUED = "queued"
SENDING = "sending"
SENT = "sent"
FAILED = "failed"
class HumanInputOTPChallengeStatus(StrEnum):
"""Current usability of an email proof challenge."""
PENDING = "pending"
VERIFIED = "verified"
INVALIDATED = "invalidated"
EXPIRED = "expired"
class IMProvider(StrEnum):
"""IM provider supported by Human Input contact and delivery flows."""
FEISHU = "feishu"
SLACK = "slack"
DING_TALK = "ding_talk"
MS_TEAMS = "ms_teams"
WE_COM = "we_com"
LARK = "lark"
class IMBindingScope(StrEnum):
"""Resolution scope of a contact-to-IM-identity binding."""
WORKSPACE = "workspace"
ORGANIZATION = "organization"
class IMIntegrationStatus(StrEnum):
"""Connectivity state of an organization-level IM integration."""
NOT_CONFIGURED = "not_configured"
CONFIGURED = "configured"
CONNECTED = "connected"
PERMISSION_ISSUE = "permission_issue"
CALLBACK_ERROR = "callback_error"
CONNECTION_ERROR = "connection_error"
class IMIdentityBindingStatus(StrEnum):
"""Whether a synchronized IM identity is currently bound."""
UNBOUND = "unbound"
BOUND = "bound"
class IMSyncRunStatus(StrEnum):
"""Lifecycle state of a manual IM directory synchronization."""
QUEUED = "queued"
RUNNING = "running"
SUCCEEDED = "succeeded"
FAILED = "failed"
class IMSyncResultType(StrEnum):
"""Stable reconciliation bucket for one synchronized directory entry."""
ADDED = "added"
NOT_MATCHED = "not_matched"
FAILED = "failed"
REMOVED = "removed"
SKIPPED = "skipped"
class IMSyncRemovalReason(StrEnum):
"""Stable reason for removing or replacing a current IM binding."""
NOT_PRESENT_IN_DIRECTORY = "not_present_in_directory"
BINDING_INVALIDATED = "binding_invalidated"
BINDING_REPLACED = "binding_replaced"
class EmailProviderType(StrEnum):
"""Email provider supported by organization-level Human Input delivery."""
RESEND = "resend"
# Identifiers for organization candidates and contacts.
OrganizationCandidateId = NewType("OrganizationCandidateId", str)
# Legacy transport identifier. Contact Directory code uses the richer value
# object from ``core.human_input_v2.shared`` at its domain boundary.
ContactId = NewType("ContactId", str)
# Identifiers for synced IM identiies. This is not the same as user_id or account_id
# on the IM provier side. It is the identifier for the synced IM user record in Dify.
IMIdentityId = NewType("IMIdentityId", str)
# Identifiers for a full IM user synchorization.
IMSyncRunId = NewType("IMSyncRunId", str)
# Identifier for an IM binding, an association between an IM identity and a Dify contact.
IMBindingId = NewType("IMBindingId", str)
__all__ = [
"AccountId",
"ContactId",
"EmailProviderType",
"HumanInputApproverGrantSubjectType",
"HumanInputAuthorizationProofType",
"HumanInputContactType",
"HumanInputDeliveryAttemptStatus",
"HumanInputDeliveryChannel",
"HumanInputOTPChallengeStatus",
"HumanInputSubmissionActorType",
"HumanInputV2FormKind",
"HumanInputV2FormStatus",
"IMBindingId",
"IMBindingScope",
"IMIdentityBindingStatus",
"IMIdentityId",
"IMIntegrationStatus",
"IMProvider",
"IMSyncRemovalReason",
"IMSyncResultType",
"IMSyncRunId",
"IMSyncRunStatus",
"NormalizedEmail",
"OrganizationCandidateId",
"UtcTimestamp",
"WorkspaceId",
]
@@ -0,0 +1,82 @@
"""Infrastructure-free IM configuration, synchronization, and binding boundary.
This package depends on canonical Contact Directory facts and shared primitive
values. Provider clients, controllers, SQLAlchemy sessions, ORM records, and
provider transport payload types belong outside this boundary. The public API
hides configuration CAS, sync matching, and effective-binding priority behind
domain decisions and transaction-oriented ports.
"""
from .binding_resolution import (
BindingResolutionKind,
BindingResolutionResult,
EffectiveBindingResolver,
EffectiveIMBindingSnapshot,
)
from .integration import (
ConfigurationTransition,
ConfigurationTransitionKind,
CurrentStateInvalidation,
EncryptedCredentials,
IMIntegration,
IntegrationDeletion,
IntegrationRevisionToken,
ProviderTenantIdentity,
StaleRevision,
)
from .ports import (
ActiveRunDecision,
ActiveRunDecisionKind,
ApplyReconciliationResult,
ApplyReconciliationStatus,
IMControlPlaneRepository,
)
from .records import IMBinding, IMIdentity, OpaqueProviderPayload
from .state import IMIntegrationState
from .sync_reconciliation import (
IMSyncRun,
MatchKind,
ProviderDirectoryEntry,
ReconciliationAction,
ReconciliationPlan,
ReconciliationSnapshot,
SyncContactSnapshot,
SyncIdentitySnapshot,
SyncReconciler,
SyncResultFact,
)
__all__ = [
"ActiveRunDecision",
"ActiveRunDecisionKind",
"ApplyReconciliationResult",
"ApplyReconciliationStatus",
"BindingResolutionKind",
"BindingResolutionResult",
"ConfigurationTransition",
"ConfigurationTransitionKind",
"CurrentStateInvalidation",
"EffectiveBindingResolver",
"EffectiveIMBindingSnapshot",
"EncryptedCredentials",
"IMBinding",
"IMControlPlaneRepository",
"IMIdentity",
"IMIntegration",
"IMIntegrationState",
"IMSyncRun",
"IntegrationDeletion",
"IntegrationRevisionToken",
"MatchKind",
"OpaqueProviderPayload",
"ProviderDirectoryEntry",
"ProviderTenantIdentity",
"ReconciliationAction",
"ReconciliationPlan",
"ReconciliationSnapshot",
"StaleRevision",
"SyncContactSnapshot",
"SyncIdentitySnapshot",
"SyncReconciler",
"SyncResultFact",
]
@@ -0,0 +1,178 @@
"""Effective binding priority and credential-free consumer snapshots."""
from __future__ import annotations
from dataclasses import dataclass
from enum import StrEnum
from core.human_input_v2.contact_directory import ContactSnapshot
from core.human_input_v2.entities import IMBindingScope, IMProvider
from core.human_input_v2.shared import (
AccountId,
ContactId,
IMBindingId,
IMIdentityId,
IntegrationId,
WorkspaceId,
)
from .integration import IntegrationRevisionToken, ProviderTenantIdentity
from .records import IMBinding, IMIdentity
class BindingResolutionKind(StrEnum):
"""Stable priority result returned to control-plane consumers."""
WORKSPACE_OVERRIDE = "workspace_override"
ORGANIZATION_BINDING = "organization_binding"
EMAIL_FALLBACK = "email_fallback"
NOT_AVAILABLE = "not_available"
INVALID_BINDING = "invalid_binding"
@dataclass(frozen=True, slots=True)
class EffectiveIMBindingSnapshot:
"""Consumer-safe effective channel facts without credentials or raw payloads."""
integration_id: IntegrationId
integration_config_version: int
provider: IMProvider
provider_tenant_id: str
contact_id: ContactId
account_id: AccountId | None
identity_id: IMIdentityId
binding_id: IMBindingId | None
provider_user_id: str
display_name: str | None
email: str | None
@dataclass(frozen=True, slots=True)
class BindingResolutionResult:
"""Effective binding or stable rejection without leaking invalid records."""
kind: BindingResolutionKind
binding: EffectiveIMBindingSnapshot | None
class EffectiveBindingResolver:
"""Resolve workspace override, organization binding, then Email fallback."""
@staticmethod
def resolve(
*,
integration_revision: IntegrationRevisionToken,
provider_tenant: ProviderTenantIdentity,
workspace_id: WorkspaceId,
contact: ContactSnapshot,
identities: tuple[IMIdentity, ...],
bindings: tuple[IMBinding, ...],
) -> BindingResolutionResult:
identities_by_id = {identity.id: identity for identity in identities}
candidates = [binding for binding in bindings if binding.contact_id == contact.contact.id]
workspace_binding = next(
(
binding
for binding in candidates
if binding.scope is IMBindingScope.WORKSPACE and binding.scope_id == str(workspace_id)
),
None,
)
organization_binding = next(
(
binding
for binding in candidates
if binding.scope is IMBindingScope.ORGANIZATION
and binding.scope_id == str(integration_revision.integration_id)
),
None,
)
selected = workspace_binding or organization_binding
if selected is not None:
identity = identities_by_id.get(selected.identity_id)
if not EffectiveBindingResolver._matches_integration(
selected,
identity,
integration_revision,
provider_tenant,
):
return BindingResolutionResult(BindingResolutionKind.INVALID_BINDING, None)
assert identity is not None
kind = (
BindingResolutionKind.WORKSPACE_OVERRIDE
if selected is workspace_binding
else BindingResolutionKind.ORGANIZATION_BINDING
)
return BindingResolutionResult(
kind,
EffectiveBindingResolver._snapshot(
integration_revision,
provider_tenant,
contact,
identity,
selected.id,
),
)
normalized_email = contact.contact.normalized_email
if normalized_email is not None:
identity = next(
(
candidate
for candidate in identities
if candidate.integration_id == integration_revision.integration_id
and candidate.provider is provider_tenant.provider
and candidate.normalized_email == normalized_email
),
None,
)
if identity is not None:
return BindingResolutionResult(
BindingResolutionKind.EMAIL_FALLBACK,
EffectiveBindingResolver._snapshot(
integration_revision,
provider_tenant,
contact,
identity,
None,
),
)
return BindingResolutionResult(BindingResolutionKind.NOT_AVAILABLE, None)
@staticmethod
def _matches_integration(
binding: IMBinding,
identity: IMIdentity | None,
integration_revision: IntegrationRevisionToken,
provider_tenant: ProviderTenantIdentity,
) -> bool:
return (
identity is not None
and binding.integration_id == integration_revision.integration_id
and identity.integration_id == integration_revision.integration_id
and binding.provider is provider_tenant.provider
and identity.provider is provider_tenant.provider
)
@staticmethod
def _snapshot(
revision: IntegrationRevisionToken,
provider_tenant: ProviderTenantIdentity,
contact: ContactSnapshot,
identity: IMIdentity,
binding_id: IMBindingId | None,
) -> EffectiveIMBindingSnapshot:
return EffectiveIMBindingSnapshot(
integration_id=revision.integration_id,
integration_config_version=revision.config_version,
provider=provider_tenant.provider,
provider_tenant_id=provider_tenant.provider_tenant_id,
contact_id=contact.contact.id,
account_id=contact.contact.account_id,
identity_id=identity.id,
binding_id=binding_id,
provider_user_id=identity.provider_user_id,
display_name=identity.display_name,
email=identity.email,
)
@@ -0,0 +1,236 @@
"""IM Integration aggregate and complete compare-and-swap revision values."""
from __future__ import annotations
import json
from collections.abc import Mapping
from dataclasses import dataclass, replace
from enum import StrEnum
from pydantic import JsonValue
from core.human_input_v2.entities import IMIntegrationStatus, IMProvider
from core.human_input_v2.shared import AccountId, IntegrationId, UtcTimestamp, WorkspaceId
@dataclass(frozen=True, slots=True)
class EncryptedCredentials:
"""Immutable opaque encrypted configuration passed through the domain boundary."""
_serialized: str
@classmethod
def from_mapping(cls, values: Mapping[str, JsonValue]) -> EncryptedCredentials:
if not values:
raise ValueError("encrypted credentials must not be empty")
return cls(json.dumps(dict(values), sort_keys=True, separators=(",", ":")))
def to_mapping(self) -> dict[str, JsonValue]:
value = json.loads(self._serialized)
if not isinstance(value, dict):
raise ValueError("encrypted credentials must be a JSON object")
return value
@dataclass(frozen=True, slots=True)
class ProviderTenantIdentity:
"""Provider plus its confirmed organization or workspace identity."""
provider: IMProvider
provider_tenant_id: str
def __post_init__(self) -> None:
if not self.provider_tenant_id.strip():
raise ValueError("provider tenant id must not be blank")
object.__setattr__(self, "provider_tenant_id", self.provider_tenant_id.strip())
@dataclass(frozen=True, slots=True)
class IntegrationRevisionToken:
"""Complete CAS token that prevents identity-replacement ABA."""
integration_id: IntegrationId
config_version: int
def __post_init__(self) -> None:
if self.config_version < 1:
raise ValueError("config version must be positive")
@dataclass(frozen=True, slots=True)
class StaleRevision:
"""Stable rejection for a token that no longer names current configuration."""
expected: IntegrationRevisionToken
actual: IntegrationRevisionToken | None
class ConfigurationTransitionKind(StrEnum):
"""Current-state effect selected by one confirmed configuration write."""
CREDENTIAL_ROTATION = "credential_rotation"
PROVIDER_REPLACEMENT = "provider_replacement"
@dataclass(frozen=True, slots=True)
class CurrentStateInvalidation:
"""Current identity and binding cleanup owned by the configuration transaction."""
invalidate_identities: bool
invalidate_bindings: bool
@dataclass(frozen=True, slots=True)
class ConfigurationTransition:
"""Atomic configuration write plus its current-state cleanup decision."""
expected_revision: IntegrationRevisionToken
kind: ConfigurationTransitionKind
integration: IMIntegration
invalidation: CurrentStateInvalidation
@dataclass(frozen=True, slots=True)
class IntegrationDeletion:
"""CAS-authorized deletion and current-state invalidation decision."""
expected_revision: IntegrationRevisionToken
invalidation: CurrentStateInvalidation = CurrentStateInvalidation(True, True)
@dataclass(frozen=True, slots=True)
class IMIntegration:
"""Organization IM configuration aggregate.
Provider reads and credential encryption happen before construction.
Configuration transitions return decisions; persistence adapters alone make
them atomic and decide whether the expected revision is still current.
Connectivity diagnostics are non-configuration state and retain the token.
"""
id: IntegrationId
workspace_id: WorkspaceId | None
provider_tenant: ProviderTenantIdentity
encrypted_credentials: EncryptedCredentials
configured_by_account_id: AccountId | None
callback_url: str | None
config_version: int
status: IMIntegrationStatus
safe_status_reason: str | None
last_checked_at: UtcTimestamp | None
created_at: UtcTimestamp
updated_at: UtcTimestamp
def __post_init__(self) -> None:
if self.config_version < 1:
raise ValueError("config version must be positive")
@classmethod
def create(
cls,
*,
integration_id: IntegrationId,
workspace_id: WorkspaceId | None,
provider_tenant: ProviderTenantIdentity,
encrypted_credentials: EncryptedCredentials,
configured_by_account_id: AccountId | None,
callback_url: str | None,
now: UtcTimestamp,
) -> IMIntegration:
return cls(
id=integration_id,
workspace_id=workspace_id,
provider_tenant=provider_tenant,
encrypted_credentials=encrypted_credentials,
configured_by_account_id=configured_by_account_id,
callback_url=callback_url,
config_version=1,
status=IMIntegrationStatus.CONFIGURED,
safe_status_reason=None,
last_checked_at=None,
created_at=now,
updated_at=now,
)
@property
def revision(self) -> IntegrationRevisionToken:
return IntegrationRevisionToken(self.id, self.config_version)
def reconfigure(
self,
*,
expected_revision: IntegrationRevisionToken,
provider_tenant: ProviderTenantIdentity,
encrypted_credentials: EncryptedCredentials,
configured_by_account_id: AccountId | None,
callback_url: str | None,
now: UtcTimestamp,
replacement_integration_id: IntegrationId | None = None,
) -> ConfigurationTransition | StaleRevision:
"""Plan a confirmed rotation or replacement without performing I/O."""
if expected_revision != self.revision:
return StaleRevision(expected_revision, self.revision)
if provider_tenant == self.provider_tenant:
if replacement_integration_id not in (None, self.id):
raise ValueError("credential rotation must preserve integration identity")
updated = replace(
self,
encrypted_credentials=encrypted_credentials,
configured_by_account_id=configured_by_account_id,
callback_url=callback_url,
config_version=self.config_version + 1,
status=IMIntegrationStatus.CONFIGURED,
safe_status_reason=None,
last_checked_at=None,
updated_at=now,
)
return ConfigurationTransition(
expected_revision=expected_revision,
kind=ConfigurationTransitionKind.CREDENTIAL_ROTATION,
integration=updated,
invalidation=CurrentStateInvalidation(False, False),
)
if replacement_integration_id is None or replacement_integration_id == self.id:
raise ValueError("provider replacement requires a new integration identity")
replacement = IMIntegration.create(
integration_id=replacement_integration_id,
workspace_id=self.workspace_id,
provider_tenant=provider_tenant,
encrypted_credentials=encrypted_credentials,
configured_by_account_id=configured_by_account_id,
callback_url=callback_url,
now=now,
)
return ConfigurationTransition(
expected_revision=expected_revision,
kind=ConfigurationTransitionKind.PROVIDER_REPLACEMENT,
integration=replacement,
invalidation=CurrentStateInvalidation(True, True),
)
def plan_deletion(self, expected_revision: IntegrationRevisionToken) -> IntegrationDeletion | StaleRevision:
"""Return deletion cleanup only when the complete token is current."""
if expected_revision != self.revision:
return StaleRevision(expected_revision, self.revision)
return IntegrationDeletion(expected_revision)
def record_diagnostics(
self,
*,
status: IMIntegrationStatus,
safe_status_reason: str | None,
checked_at: UtcTimestamp,
) -> IMIntegration:
"""Update connection diagnostics without advancing configuration."""
return replace(
self,
status=status,
safe_status_reason=safe_status_reason,
last_checked_at=checked_at,
updated_at=checked_at,
)
@@ -0,0 +1,123 @@
"""Transaction-oriented persistence ports for IM Control Plane invariants.
Implementations own CAS predicates, Integration row locks, eager loading,
revision-guarded apply, rollback, and append-only result persistence. Generic
table CRUD is intentionally absent.
"""
from dataclasses import dataclass
from enum import StrEnum
from typing import Protocol
from core.human_input_v2.entities import IMProvider
from core.human_input_v2.shared import (
AccountId,
ContactId,
IMSyncRunId,
IntegrationId,
UtcTimestamp,
WorkspaceId,
)
from .binding_resolution import BindingResolutionResult
from .integration import (
ConfigurationTransition,
IMIntegration,
IntegrationDeletion,
IntegrationRevisionToken,
StaleRevision,
)
from .sync_reconciliation import IMSyncRun, ReconciliationPlan, ReconciliationSnapshot, SyncResultFact
class ActiveRunDecisionKind(StrEnum):
"""Outcome of Integration-locked sync run creation."""
CREATED = "created"
EXISTING_ACTIVE = "existing_active"
STALE_REVISION = "stale_revision"
@dataclass(frozen=True, slots=True)
class ActiveRunDecision:
"""New or existing active run, or a stable stale-revision rejection."""
kind: ActiveRunDecisionKind
run: IMSyncRun | None
stale_revision: StaleRevision | None = None
class ApplyReconciliationStatus(StrEnum):
"""Stable outcome of one idempotent revision-guarded apply."""
APPLIED = "applied"
ALREADY_APPLIED = "already_applied"
STALE_REVISION = "stale_revision"
@dataclass(frozen=True, slots=True)
class ApplyReconciliationResult:
"""Run and append-only facts returned after reconciliation apply."""
status: ApplyReconciliationStatus
run: IMSyncRun
results: tuple[SyncResultFact, ...]
class IMControlPlaneRepository(Protocol):
"""Atomic persistence capabilities required by the IM domain."""
def create_integration(self, integration: IMIntegration) -> IMIntegration:
"""Create the first integration configuration for its owner scope."""
...
def compare_and_swap_configuration(self, transition: ConfigurationTransition) -> IMIntegration | StaleRevision:
"""Atomically apply rotation or replacement and its invalidation plan."""
...
def compare_and_swap_delete(self, deletion: IntegrationDeletion) -> None | StaleRevision:
"""Delete current configuration and current children under complete CAS."""
...
def create_or_get_active_run(
self,
integration_revision: IntegrationRevisionToken,
*,
sync_run_id: IMSyncRunId,
started_by_account_id: AccountId | None,
now: UtcTimestamp,
) -> ActiveRunDecision:
"""Lock Integration and return at most one active run."""
...
def load_reconciliation_snapshot(self, sync_run_id: IMSyncRunId) -> ReconciliationSnapshot:
"""Load current identities, bindings, and eligible Contact facts."""
...
def apply_reconciliation(self, plan: ReconciliationPlan, *, now: UtcTimestamp) -> ApplyReconciliationResult:
"""Apply one plan using its persisted sync run capture as CAS authority."""
...
def resolve_effective_binding(
self,
*,
integration_id: IntegrationId,
provider: IMProvider,
workspace_id: WorkspaceId,
contact_id: ContactId,
) -> BindingResolutionResult:
"""Load and resolve one credential-free effective binding snapshot."""
...
def append_sync_results(self, results: tuple[SyncResultFact, ...]) -> None:
"""Append diagnostic result facts without changing current state."""
...
__all__ = [
"ActiveRunDecision",
"ActiveRunDecisionKind",
"ApplyReconciliationResult",
"ApplyReconciliationStatus",
"IMControlPlaneRepository",
]
@@ -0,0 +1,145 @@
"""Current IM identity and binding values shared by sync and resolution."""
from __future__ import annotations
import json
from collections.abc import Mapping
from dataclasses import dataclass
from pydantic import JsonValue
from core.human_input_v2.entities import IMBindingScope, IMProvider
from core.human_input_v2.shared import (
AccountId,
ContactId,
IMBindingId,
IMIdentityId,
IMSyncRunId,
IntegrationId,
NormalizedEmail,
UtcTimestamp,
)
@dataclass(frozen=True, slots=True)
class OpaqueProviderPayload:
"""Immutable provider JSON retained only for persistence diagnostics."""
_serialized: str
@classmethod
def from_mapping(cls, values: Mapping[str, JsonValue]) -> OpaqueProviderPayload:
return cls(json.dumps(dict(values), sort_keys=True, separators=(",", ":")))
def to_mapping(self) -> dict[str, JsonValue]:
value = json.loads(self._serialized)
if not isinstance(value, dict):
raise ValueError("provider payload must be a JSON object")
return value
@dataclass(frozen=True, slots=True)
class IMIdentity:
"""Current provider identity independent from ORM lifetime and raw clients."""
id: IMIdentityId
integration_id: IntegrationId
provider: IMProvider
provider_user_id: str
display_name: str | None
normalized_name: str | None
email: str | None
normalized_email: NormalizedEmail | None
raw_payload: OpaqueProviderPayload
last_seen_sync_run_id: IMSyncRunId | None
last_seen_at: UtcTimestamp | None
created_at: UtcTimestamp
updated_at: UtcTimestamp
def __post_init__(self) -> None:
if not self.provider_user_id.strip():
raise ValueError("provider user id must not be blank")
@classmethod
def create(
cls,
*,
identity_id: IMIdentityId,
integration_id: IntegrationId,
provider: IMProvider,
provider_user_id: str,
display_name: str | None,
email: str | None,
raw_payload: Mapping[str, JsonValue],
last_seen_sync_run_id: IMSyncRunId | None,
last_seen_at: UtcTimestamp | None,
now: UtcTimestamp,
created_at: UtcTimestamp | None = None,
) -> IMIdentity:
clean_name = display_name.strip() if display_name is not None else None
clean_email = email.strip() if email is not None else None
return cls(
id=identity_id,
integration_id=integration_id,
provider=provider,
provider_user_id=provider_user_id.strip(),
display_name=clean_name,
normalized_name=clean_name.casefold() if clean_name else None,
email=clean_email,
normalized_email=NormalizedEmail(clean_email) if clean_email else None,
raw_payload=OpaqueProviderPayload.from_mapping(raw_payload),
last_seen_sync_run_id=last_seen_sync_run_id,
last_seen_at=last_seen_at,
created_at=created_at or now,
updated_at=now,
)
@dataclass(frozen=True, slots=True)
class IMBinding:
"""Current Contact-to-provider identity association in one resolution scope."""
id: IMBindingId
integration_id: IntegrationId
scope: IMBindingScope
scope_id: str
contact_id: ContactId
identity_id: IMIdentityId
provider: IMProvider
bound_by_account_id: AccountId | None
created_at: UtcTimestamp
updated_at: UtcTimestamp
def __post_init__(self) -> None:
if not self.scope_id.strip():
raise ValueError("binding scope id must not be blank")
if self.scope is IMBindingScope.ORGANIZATION and self.scope_id != str(self.integration_id):
raise ValueError("organization binding scope must be its integration")
@classmethod
def create(
cls,
*,
binding_id: IMBindingId,
integration_id: IntegrationId,
scope: IMBindingScope,
scope_id: str,
contact_id: ContactId,
identity_id: IMIdentityId,
provider: IMProvider,
bound_by_account_id: AccountId | None,
now: UtcTimestamp,
created_at: UtcTimestamp | None = None,
) -> IMBinding:
return cls(
id=binding_id,
integration_id=integration_id,
scope=scope,
scope_id=scope_id,
contact_id=contact_id,
identity_id=identity_id,
provider=provider,
bound_by_account_id=bound_by_account_id,
created_at=created_at or now,
updated_at=now,
)
@@ -0,0 +1,18 @@
"""Explicit aggregate-load snapshot spanning IM persistence records."""
from dataclasses import dataclass
from .integration import IMIntegration
from .records import IMBinding, IMIdentity
from .sync_reconciliation import IMSyncRun, SyncResultFact
@dataclass(frozen=True, slots=True)
class IMIntegrationState:
"""Eagerly loaded Integration with mapped current and historical children."""
integration: IMIntegration
identities: tuple[IMIdentity, ...]
bindings: tuple[IMBinding, ...]
sync_runs: tuple[IMSyncRun, ...]
sync_results: tuple[SyncResultFact, ...]
@@ -0,0 +1,276 @@
"""Pure provider directory matching and immutable reconciliation plans."""
from __future__ import annotations
from dataclasses import dataclass, replace
from enum import StrEnum
from pydantic import JsonValue
from core.human_input_v2.contact_directory import ContactIdentitySource, ContactSnapshot
from core.human_input_v2.entities import IMProvider, IMSyncRemovalReason, IMSyncResultType, IMSyncRunStatus
from core.human_input_v2.shared import (
AccountId,
ContactId,
IMBindingId,
IMIdentityId,
IMSyncResultId,
IMSyncRunId,
IntegrationId,
NormalizedEmail,
UtcTimestamp,
)
from .integration import IntegrationRevisionToken
from .records import IMBinding, IMIdentity, OpaqueProviderPayload
class MatchKind(StrEnum):
"""Stable explanation for how one provider entry was classified."""
PROVIDER_USER_ID = "provider_user_id"
NORMALIZED_EMAIL = "normalized_email"
UNMATCHED = "unmatched"
@dataclass(frozen=True, slots=True)
class ProviderDirectoryEntry:
"""Provider-neutral directory values consumed by the pure reconciler."""
provider_user_id: str
display_name: str | None
email: str | None
normalized_email: NormalizedEmail | None
raw_payload: OpaqueProviderPayload
@classmethod
def create(
cls,
*,
provider_user_id: str,
display_name: str | None,
email: str | None,
raw_payload: dict[str, JsonValue],
) -> ProviderDirectoryEntry:
clean_email = email.strip() if email is not None else None
return cls(
provider_user_id=provider_user_id.strip(),
display_name=display_name.strip() if display_name is not None else None,
email=clean_email,
normalized_email=NormalizedEmail(clean_email) if clean_email else None,
raw_payload=OpaqueProviderPayload.from_mapping(raw_payload),
)
@dataclass(frozen=True, slots=True)
class ReconciliationSnapshot:
"""Coherent current facts loaded before provider entries are matched."""
identities: tuple[IMIdentity, ...] = ()
bindings: tuple[IMBinding, ...] = ()
contacts: tuple[ContactSnapshot, ...] = ()
@dataclass(frozen=True, slots=True)
class ReconciliationAction:
"""One provider entry match without persistence side effects."""
entry: ProviderDirectoryEntry
match_kind: MatchKind
identity_id: IMIdentityId | None
binding_id: IMBindingId | None
contact_id: ContactId | None
@dataclass(frozen=True, slots=True)
class ReconciliationPlan:
"""Immutable plan whose captured revision must be checked again at apply."""
sync_run_id: IMSyncRunId
integration_revision: IntegrationRevisionToken
provider: IMProvider
actions: tuple[ReconciliationAction, ...]
removed_identity_ids: tuple[IMIdentityId, ...]
@dataclass(frozen=True, slots=True)
class SyncResultFact:
"""Append-only outcome for one action, removed binding, or diagnostic.
Removing an identity emits one fact per removed binding so every scope
override remains auditable. An identity without bindings emits one fact
whose binding and Contact fields are absent.
"""
id: IMSyncResultId
integration_id: IntegrationId
sync_run_id: IMSyncRunId
result_type: IMSyncResultType
provider_user_id: str | None
display_name: str | None
email: str | None
normalized_email: NormalizedEmail | None
contact_id: ContactId | None
identity_id: IMIdentityId | None
binding_id: IMBindingId | None
removal_reason: IMSyncRemovalReason | None
reason_code: str | None
reason_message: str | None
directory_entry_payload: OpaqueProviderPayload | None
contact_snapshot: SyncContactSnapshot | None
identity_snapshot: SyncIdentitySnapshot | None
created_at: UtcTimestamp
updated_at: UtcTimestamp
@dataclass(frozen=True, slots=True)
class SyncContactSnapshot:
"""Immutable Contact display values retained by a historical result."""
contact_id: ContactId
name: str
email: str | None
avatar_file_id: str | None
@dataclass(frozen=True, slots=True)
class SyncIdentitySnapshot:
"""Immutable last-known provider identity retained after current deletion."""
identity_id: IMIdentityId
provider: IMProvider
provider_user_id: str
display_name: str | None
email: str | None
@dataclass(frozen=True, slots=True)
class IMSyncRun:
"""Independent sync aggregate that captures one complete Integration token."""
id: IMSyncRunId
integration_revision: IntegrationRevisionToken
provider: IMProvider
status: IMSyncRunStatus
added_count: int
not_matched_count: int
failed_count: int
removed_count: int
skipped_count: int
started_by_account_id: AccountId | None
started_at: UtcTimestamp | None
finished_at: UtcTimestamp | None
error_code: str | None
error_message: str | None
created_at: UtcTimestamp
updated_at: UtcTimestamp
@classmethod
def create(
cls,
*,
sync_run_id: IMSyncRunId,
integration_revision: IntegrationRevisionToken,
provider: IMProvider,
started_by_account_id: AccountId | None,
now: UtcTimestamp,
) -> IMSyncRun:
return cls(
id=sync_run_id,
integration_revision=integration_revision,
provider=provider,
status=IMSyncRunStatus.QUEUED,
added_count=0,
not_matched_count=0,
failed_count=0,
removed_count=0,
skipped_count=0,
started_by_account_id=started_by_account_id,
started_at=None,
finished_at=None,
error_code=None,
error_message=None,
created_at=now,
updated_at=now,
)
@property
def is_active(self) -> bool:
return self.status in (IMSyncRunStatus.QUEUED, IMSyncRunStatus.RUNNING)
def start(self, now: UtcTimestamp) -> IMSyncRun:
if self.status is not IMSyncRunStatus.QUEUED:
return self
return replace(self, status=IMSyncRunStatus.RUNNING, started_at=now, updated_at=now)
class SyncReconciler:
"""Stateless matching policy with no provider or persistence dependencies.
Email fallback accepts available account-backed Contacts: EE Organization
Accounts and CE/SaaS workspace members. External Contacts never participate.
"""
@staticmethod
def reconcile(
*,
sync_run_id: IMSyncRunId,
integration_revision: IntegrationRevisionToken,
provider: IMProvider,
entries: tuple[ProviderDirectoryEntry, ...],
snapshot: ReconciliationSnapshot,
) -> ReconciliationPlan:
identities = {
identity.provider_user_id: identity
for identity in snapshot.identities
if identity.integration_id == integration_revision.integration_id and identity.provider is provider
}
bindings_by_identity: dict[IMIdentityId, IMBinding] = {}
for binding in sorted(snapshot.bindings, key=lambda item: item.scope.value, reverse=True):
bindings_by_identity.setdefault(binding.identity_id, binding)
contacts_by_email = {
item.contact.normalized_email: item.contact
for item in snapshot.contacts
if item.account_available
and item.contact.identity_source
in (ContactIdentitySource.ORGANIZATION_ACCOUNT, ContactIdentitySource.WORKSPACE_MEMBER)
and item.contact.normalized_email is not None
}
actions: list[ReconciliationAction] = []
seen_provider_user_ids: set[str] = set()
for entry in entries:
seen_provider_user_ids.add(entry.provider_user_id)
identity = identities.get(entry.provider_user_id)
if identity is not None:
matched_binding = bindings_by_identity.get(identity.id)
actions.append(
ReconciliationAction(
entry=entry,
match_kind=MatchKind.PROVIDER_USER_ID,
identity_id=identity.id,
binding_id=matched_binding.id if matched_binding is not None else None,
contact_id=matched_binding.contact_id if matched_binding is not None else None,
)
)
continue
contact = contacts_by_email.get(entry.normalized_email) if entry.normalized_email is not None else None
actions.append(
ReconciliationAction(
entry=entry,
match_kind=MatchKind.NORMALIZED_EMAIL if contact is not None else MatchKind.UNMATCHED,
identity_id=None,
binding_id=None,
contact_id=contact.id if contact is not None else None,
)
)
removed = tuple(
identity.id
for identity in snapshot.identities
if identity.integration_id == integration_revision.integration_id
and identity.provider is provider
and identity.provider_user_id not in seen_provider_user_ids
)
return ReconciliationPlan(sync_run_id, integration_revision, provider, tuple(actions), removed)
@@ -0,0 +1,63 @@
"""Stable Human Input v2 values shared across domain contexts.
This package contains infrastructure-free values only. Feature-specific ownership
and lifecycle rules belong to their bounded context rather than this package.
"""
from .values import (
AccountId,
AppId,
ApproverGrantId,
AuditEventId,
ContactId,
DeliveryAttemptId,
DeliveryEndpointId,
DeploymentScope,
DirectoryScope,
EmailProviderId,
EndUserId,
FormId,
IMBindingId,
IMIdentityId,
IMSyncResultId,
IMSyncRunId,
IntegrationId,
NormalizedEmail,
OTPChallengeId,
PlatformEntryId,
SubmissionId,
UploadCapabilityId,
UploadFileAssociationId,
UtcTimestamp,
WorkspaceId,
WorkspaceScope,
)
__all__ = [
"AccountId",
"AppId",
"ApproverGrantId",
"AuditEventId",
"ContactId",
"DeliveryAttemptId",
"DeliveryEndpointId",
"DeploymentScope",
"DirectoryScope",
"EmailProviderId",
"EndUserId",
"FormId",
"IMBindingId",
"IMIdentityId",
"IMSyncResultId",
"IMSyncRunId",
"IntegrationId",
"NormalizedEmail",
"OTPChallengeId",
"PlatformEntryId",
"SubmissionId",
"UploadCapabilityId",
"UploadFileAssociationId",
"UtcTimestamp",
"WorkspaceId",
"WorkspaceScope",
]
+173
View File
@@ -0,0 +1,173 @@
"""Primitive-independent identifiers, scopes, email, and time values."""
from __future__ import annotations
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import override
@dataclass(frozen=True, slots=True)
class _Identifier:
"""Non-empty string identifier with explicit primitive serialization."""
value: str
def __post_init__(self) -> None:
if not isinstance(self.value, str) or not self.value.strip():
raise ValueError(f"{type(self).__name__} must not be blank")
object.__setattr__(self, "value", self.value.strip())
@override
def __str__(self) -> str:
return self.value
def to_primitive(self) -> str:
return self.value
class AccountId(_Identifier):
"""Identifier of an Account record."""
class ContactId(_Identifier):
"""Identifier of a canonical Contact."""
class EndUserId(_Identifier):
"""Identifier of one app-scoped EndUser identity."""
class PlatformEntryId(_Identifier):
"""Identifier of one Platform allow-list entry."""
class WorkspaceId(_Identifier):
"""Identifier of the workspace that owns or resolves a Contact."""
class AppId(_Identifier):
"""Identifier of the application that owns a Human Input form."""
class FormId(_Identifier):
"""Identifier of one Human Input v2 form root."""
class ApproverGrantId(_Identifier):
"""Identifier of one form-scoped approver grant."""
class OTPChallengeId(_Identifier):
"""Identifier of one grant-scoped OTP proof session."""
class DeliveryEndpointId(_Identifier):
"""Identifier of one frozen form delivery endpoint."""
class DeliveryAttemptId(_Identifier):
"""Identifier of one append-only delivery attempt."""
class SubmissionId(_Identifier):
"""Identifier of one immutable winning form submission."""
class AuditEventId(_Identifier):
"""Identifier of one append-only Human Input audit event."""
class EmailProviderId(_Identifier):
"""Identifier of one workspace email provider configuration."""
class UploadCapabilityId(_Identifier):
"""Identifier of one endpoint-scoped upload capability."""
class UploadFileAssociationId(_Identifier):
"""Identifier of one durable uploaded-file association."""
class IntegrationId(_Identifier):
"""Identifier of one IM Integration configuration identity."""
class IMIdentityId(_Identifier):
"""Identifier of one current synchronized provider identity."""
class IMBindingId(_Identifier):
"""Identifier of one current Contact-to-IM-identity binding."""
class IMSyncRunId(_Identifier):
"""Identifier of one IM directory synchronization run."""
class IMSyncResultId(_Identifier):
"""Identifier of one append-only synchronization result fact."""
@dataclass(frozen=True, slots=True)
class NormalizedEmail:
"""Case-insensitive canonical email used for identity comparisons."""
value: str
def __post_init__(self) -> None:
if not isinstance(self.value, str):
raise ValueError("value must be a valid email")
normalized = self.value.strip().casefold()
local, separator, domain = normalized.partition("@")
if not separator or not local or not domain or " " in normalized or "@" in domain:
raise ValueError("value must be a valid email")
object.__setattr__(self, "value", normalized)
@override
def __str__(self) -> str:
return self.value
def to_primitive(self) -> str:
return self.value
@dataclass(frozen=True, slots=True)
class DeploymentScope:
"""Deployment-wide owner scope used by EE Organization contacts."""
def to_primitive(self) -> dict[str, str]:
return {"kind": "deployment"}
@dataclass(frozen=True, slots=True)
class WorkspaceScope:
"""Owner scope for workspace-owned contacts and directory operations."""
workspace_id: WorkspaceId
def to_primitive(self) -> dict[str, str]:
return {"kind": "workspace", "workspace_id": self.workspace_id.to_primitive()}
type DirectoryScope = DeploymentScope | WorkspaceScope
@dataclass(frozen=True, slots=True)
class UtcTimestamp:
"""Timezone-aware timestamp normalized to UTC at construction."""
value: datetime
def __post_init__(self) -> None:
if not isinstance(self.value, datetime) or self.value.tzinfo is None or self.value.utcoffset() is None:
raise ValueError("value must be a timezone-aware datetime")
object.__setattr__(self, "value", self.value.astimezone(UTC))
@classmethod
def now(cls) -> UtcTimestamp:
return cls(datetime.now(UTC))
def to_primitive(self) -> str:
return self.value.isoformat().replace("+00:00", "Z")
+46 -44
View File
@@ -21,6 +21,7 @@ from core.model_manager import ModelInstance, ModelManager
from core.rag.cleaner.clean_processor import CleanProcessor
from core.rag.datasource.keyword.keyword_factory import Keyword
from core.rag.docstore.dataset_docstore import DatasetDocumentStore
from core.rag.embedding.token_counter import calculate_segment_token_counts
from core.rag.extractor.entity.datasource_type import DatasourceType
from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo
from core.rag.index_processor.constant.index_type import IndexStructureType, IndexTechniqueType
@@ -113,17 +114,25 @@ class IndexingRunner:
current_user=current_user,
session=session,
)
token_counts = calculate_segment_token_counts(dataset=dataset, documents=documents)
total_tokens = sum(token_counts)
# save segment
self._load_segments(dataset, requeried_document, documents, session)
self._load_segments(
session=session,
dataset=dataset,
dataset_document=requeried_document,
documents=documents,
token_counts=token_counts,
)
session.commit()
# load
self._load(
index_processor=index_processor,
session=session,
dataset=dataset,
dataset_document=requeried_document,
documents=documents,
session=session,
total_tokens=total_tokens,
)
except DocumentIsPausedError:
raise DocumentIsPausedError(f"Document paused, document id: {document_id}")
@@ -190,17 +199,25 @@ class IndexingRunner:
current_user=current_user,
session=session,
)
token_counts = calculate_segment_token_counts(dataset=dataset, documents=documents)
total_tokens = sum(token_counts)
# save segment
self._load_segments(dataset, requeried_document, documents, session)
self._load_segments(
session=session,
dataset=dataset,
dataset_document=requeried_document,
documents=documents,
token_counts=token_counts,
)
session.commit()
# load
self._load(
index_processor=index_processor,
session=session,
dataset=dataset,
dataset_document=requeried_document,
documents=documents,
session=session,
total_tokens=total_tokens,
)
except DocumentIsPausedError:
raise DocumentIsPausedError(f"Document paused, document id: {document_id}")
@@ -225,7 +242,7 @@ class IndexingRunner:
if not dataset:
raise ValueError("no dataset found")
# get exist document_segment list and delete
# get existing document segments
document_segments = session.scalars(
select(DocumentSegment).where(
DocumentSegment.dataset_id == dataset.id,
@@ -264,15 +281,15 @@ class IndexingRunner:
child_documents.append(child_document)
document.children = child_documents
documents.append(document)
# Preserve the full document total even when only incomplete segments are re-indexed.
total_tokens = sum(document_segment.tokens for document_segment in document_segments)
# build index
index_type = requeried_document.doc_form
index_processor = IndexProcessorFactory(index_type).init_index_processor()
self._load(
index_processor=index_processor,
session=session,
dataset=dataset,
dataset_document=requeried_document,
documents=documents,
session=session,
total_tokens=total_tokens,
)
except DocumentIsPausedError:
raise DocumentIsPausedError(f"Document paused, document id: {document_id}")
@@ -601,28 +618,16 @@ class IndexingRunner:
def _load(
self,
index_processor: BaseIndexProcessor,
session: Session,
dataset: Dataset,
dataset_document: DatasetDocument,
documents: list[Document],
session: Session,
):
"""
insert index and update document/segment status to completed
"""
total_tokens: int,
) -> None:
"""Build indexes and mark the document complete using the token total computed before hash sharding."""
embedding_model_instance = None
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
embedding_model_instance = self._get_model_manager(dataset.tenant_id).get_model_instance(
tenant_id=dataset.tenant_id,
provider=dataset.embedding_model_provider,
model_type=ModelType.TEXT_EMBEDDING,
model=dataset.embedding_model,
)
# chunk nodes by chunk size
# Build indexes using the existing hash-based worker groups.
indexing_start_at = time.perf_counter()
tokens = 0
create_keyword_thread = None
if (
dataset_document.doc_form != IndexStructureType.PARENT_CHILD_INDEX
@@ -659,12 +664,11 @@ class IndexingRunner:
chunk_documents,
dataset.id,
dataset_document.id,
embedding_model_instance,
)
)
for future in futures:
tokens += future.result()
future.result()
if (
dataset_document.doc_form != IndexStructureType.PARENT_CHILD_INDEX
and dataset.indexing_technique == IndexTechniqueType.ECONOMY
@@ -679,7 +683,7 @@ class IndexingRunner:
document_id=dataset_document.id,
after_indexing_status=IndexingStatus.COMPLETED,
extra_update_params={
DatasetDocument.tokens: tokens,
DatasetDocument.tokens: total_tokens,
DatasetDocument.completed_at: naive_utc_now(),
DatasetDocument.indexing_latency: indexing_end_at - indexing_start_at,
DatasetDocument.error: None,
@@ -720,8 +724,7 @@ class IndexingRunner:
chunk_documents: list[Document],
dataset_id: str,
dataset_document_id: str,
embedding_model_instance: ModelInstance | None,
):
) -> None:
with flask_app.app_context():
with session_factory.create_session() as session:
dataset = session.get(Dataset, dataset_id)
@@ -735,11 +738,6 @@ class IndexingRunner:
# check document is paused
self._check_document_paused_status(dataset_document.id)
tokens = 0
if embedding_model_instance:
page_content_list = [document.page_content for document in chunk_documents]
tokens += sum(embedding_model_instance.get_text_embedding_num_tokens(page_content_list))
multimodal_documents = []
for document in chunk_documents:
if document.attachments and dataset.is_multimodal:
@@ -773,8 +771,6 @@ class IndexingRunner:
session.commit()
return tokens
@staticmethod
def _check_document_paused_status(document_id: str):
indexing_cache_key = f"document_{document_id}_is_paused"
@@ -864,8 +860,14 @@ class IndexingRunner:
return documents
def _load_segments(
self, dataset: Dataset, dataset_document: DatasetDocument, documents: list[Document], session: Session
):
self,
session: Session,
dataset: Dataset,
dataset_document: DatasetDocument,
documents: list[Document],
token_counts: list[int],
) -> None:
"""Persist transformed documents and their precomputed token counts before indexing starts."""
# save node to document segment
doc_store = DatasetDocumentStore(
dataset=dataset, user_id=dataset_document.created_by, document_id=dataset_document.id
@@ -873,9 +875,10 @@ class IndexingRunner:
# add document segments
doc_store.add_documents(
session=session,
docs=documents,
save_child=dataset_document.doc_form == IndexStructureType.PARENT_CHILD_INDEX,
session=session,
token_counts=token_counts,
)
# update document status to indexing
@@ -900,7 +903,6 @@ class IndexingRunner:
DocumentSegment.indexing_at: naive_utc_now(),
},
)
pass
class DocumentIsPausedError(Exception):
+34 -2
View File
@@ -6,9 +6,21 @@ using Python's contextvars for thread-safe and async-safe storage.
import uuid
from contextvars import ContextVar
from typing import NamedTuple
class IdentityContext(NamedTuple):
"""Immutable identity values captured for logging."""
tenant_id: str
user_id: str
user_type: str
_request_id: ContextVar[str] = ContextVar("log_request_id", default="")
_trace_id: ContextVar[str] = ContextVar("log_trace_id", default="")
_EMPTY_IDENTITY_CONTEXT = IdentityContext(tenant_id="", user_id="", user_type="")
_identity: ContextVar[IdentityContext] = ContextVar("log_identity", default=_EMPTY_IDENTITY_CONTEXT)
def get_request_id() -> str:
@@ -21,15 +33,35 @@ def get_trace_id() -> str:
return _trace_id.get()
def get_identity_context() -> IdentityContext:
"""Get the immutable tenant, user, and user-type snapshot for logging."""
return _identity.get()
def set_identity_context(
*, tenant_id: str | None = None, user_id: str | None = None, user_type: str | None = None
) -> None:
"""Set primitive identity values already resolved by an authentication boundary."""
_identity.set(
IdentityContext(
tenant_id=tenant_id or "",
user_id=user_id or "",
user_type=user_type or "",
)
)
def init_request_context() -> None:
"""Initialize request context. Call at start of each request."""
"""Initialize request context and discard identity left by earlier work."""
req_id = uuid.uuid4().hex[:10]
trace_id = uuid.uuid5(uuid.NAMESPACE_DNS, req_id).hex
_request_id.set(req_id)
_trace_id.set(trace_id)
_identity.set(_EMPTY_IDENTITY_CONTEXT)
def clear_request_context() -> None:
"""Clear request context. Call at end of request (optional)."""
"""Clear request context at a request or task lifecycle boundary."""
_request_id.set("")
_trace_id.set("")
_identity.set(_EMPTY_IDENTITY_CONTEXT)
+9 -45
View File
@@ -4,10 +4,7 @@ import contextlib
import logging
from typing import override
import flask
from core.logging.context import get_request_id, get_trace_id
from core.logging.structured_formatter import IdentityDict
from core.logging.context import get_identity_context, get_request_id, get_trace_id
class TraceContextFilter(logging.Filter):
@@ -51,49 +48,16 @@ class TraceContextFilter(logging.Filter):
class IdentityContextFilter(logging.Filter):
"""
Filter that adds user identity context to log records.
Extracts tenant_id, user_id, and user_type from Flask-Login current_user.
"""Add an identity snapshot without invoking authentication or database work.
Logging can run while other libraries hold internal locks, so this filter must
only read primitive ContextVar values populated by authentication boundaries.
"""
@override
def filter(self, record: logging.LogRecord) -> bool:
identity = self._extract_identity()
record.tenant_id = identity.get("tenant_id", "")
record.user_id = identity.get("user_id", "")
record.user_type = identity.get("user_type", "")
identity = get_identity_context()
record.tenant_id = identity.tenant_id
record.user_id = identity.user_id
record.user_type = identity.user_type
return True
def _extract_identity(self) -> IdentityDict:
"""Extract identity from current_user if in request context."""
try:
if not flask.has_request_context():
return {}
from flask_login import current_user
# Check if user is authenticated using the proxy
if not current_user.is_authenticated:
return {}
# Access the underlying user object
user = current_user
from models import Account
from models.model import EndUser
identity: IdentityDict = {}
match user:
case Account():
if user.current_tenant_id:
identity["tenant_id"] = user.current_tenant_id
identity["user_id"] = user.id
identity["user_type"] = "account"
case EndUser():
identity["tenant_id"] = user.tenant_id
identity["user_id"] = user.id
identity["user_type"] = user.type or "end_user"
return identity
except Exception:
return {}
@@ -1,110 +0,0 @@
"""Datasource backward invocation through Dify's tenant-bound runtime.
This module is the internal service boundary used by trusted callers such as
KnowledgeFS. It resolves installed provider declarations and Dify-owned
credential references before reaching ``PluginDatasourceManager``; callers
must never provide raw datasource credentials or a plugin-daemon API key.
"""
from collections.abc import Generator
from typing import Any, cast
from pydantic import BaseModel
from core.datasource.datasource_manager import DatasourceManager
from core.datasource.entities.datasource_entities import (
OnlineDriveBrowseFilesRequest,
OnlineDriveDownloadFileRequest,
)
from core.datasource.online_document.online_document_plugin import OnlineDocumentDatasourcePlugin
from core.datasource.online_drive.online_drive_plugin import OnlineDriveDatasourcePlugin
from core.datasource.website_crawl.website_crawl_plugin import WebsiteCrawlDatasourcePlugin
from core.plugin.backwards_invocation.base import BaseBackwardsInvocation
from core.plugin.entities.request import RequestInvokeDatasource
from models.account import Tenant
from models.provider_ids import DatasourceProviderID
from services.datasource_provider_service import DatasourceProviderService
class PluginDatasourceBackwardsInvocation(BaseBackwardsInvocation):
"""Resolve and invoke a datasource without exposing credential material to the caller."""
@classmethod
def invoke(
cls,
*,
user_id: str,
tenant: Tenant,
payload: RequestInvokeDatasource,
) -> Generator[BaseModel | dict[str, Any], None, None]:
"""Yield datasource messages for one validated inner-runtime request."""
provider_id = DatasourceProviderID(payload.provider)
canonical_provider_id = str(provider_id)
controller = DatasourceManager.get_datasource_plugin_provider(
provider_id=canonical_provider_id,
tenant_id=tenant.id,
datasource_type=payload.datasource_type,
)
if controller.entity.provider_type != payload.datasource_type:
raise ValueError("Datasource provider type mismatch")
# Resolving the datasource from the installed declaration prevents a caller
# from dispatching an arbitrary datasource name under a valid plugin ID.
runtime = controller.get_datasource(payload.datasource)
credentials = DatasourceProviderService().get_datasource_credentials(
tenant_id=tenant.id,
provider=provider_id.provider_name,
plugin_id=provider_id.plugin_id,
credential_id=payload.credential_id,
)
if controller.need_credentials and not credentials:
raise ValueError("Datasource credential not found")
if payload.operation == "validate_credentials":
controller.validate_credentials(user_id=user_id, credentials=credentials)
yield {"result": True}
return
runtime.runtime.credentials = credentials
provider_type = runtime.datasource_provider_type()
match payload.operation:
case "get_website_crawl":
website = cast(WebsiteCrawlDatasourcePlugin, runtime)
yield from website.get_website_crawl(
user_id=user_id,
datasource_parameters=payload.datasource_parameters,
provider_type=provider_type,
)
case "get_online_document_pages":
document = cast(OnlineDocumentDatasourcePlugin, runtime)
yield from document.get_online_document_pages(
user_id=user_id,
datasource_parameters=payload.datasource_parameters,
provider_type=provider_type,
)
case "get_online_document_page_content":
if payload.page is None:
raise ValueError("Online-document page input is required")
document = cast(OnlineDocumentDatasourcePlugin, runtime)
yield from document.get_online_document_page_content(
user_id=user_id,
datasource_parameters=payload.page,
provider_type=provider_type,
)
case "online_drive_browse_files":
drive = cast(OnlineDriveDatasourcePlugin, runtime)
yield from drive.online_drive_browse_files(
user_id=user_id,
request=OnlineDriveBrowseFilesRequest.model_validate(payload.request),
provider_type=provider_type,
)
case "online_drive_download_file":
drive = cast(OnlineDriveDatasourcePlugin, runtime)
yield from drive.online_drive_download_file(
user_id=user_id,
request=OnlineDriveDownloadFileRequest.model_validate(payload.request),
provider_type=provider_type,
)
case _:
raise ValueError(f"Unsupported datasource operation: {payload.operation}")
+2 -109
View File
@@ -1,31 +1,22 @@
import tempfile
from binascii import hexlify, unhexlify
from collections.abc import Generator, Mapping
from enum import Enum
from collections.abc import Generator
from typing import Any
from pydantic import BaseModel
from core.app.llm import deduct_llm_quota
from core.llm_generator.output_parser.structured_output import invoke_llm_with_structured_output
from core.model_manager import ModelManager
from core.plugin.backwards_invocation.base import BaseBackwardsInvocation
from core.plugin.entities.request import (
InvokableModelCatalogItem,
InvokableModelCatalogPage,
RequestInvokeLLM,
RequestInvokeLLMWithStructuredOutput,
RequestInvokeModeration,
RequestInvokeMultimodalEmbedding,
RequestInvokeRerank,
RequestInvokeSpeech2Text,
RequestInvokeSummary,
RequestInvokeTextEmbedding,
RequestInvokeTTS,
RequestListModels,
)
from core.plugin.impl.model_runtime_factory import create_plugin_provider_manager
from core.plugin.plugin_service import PluginService
from core.tools.entities.tool_entities import ToolProviderType
from core.tools.utils.model_invocation_utils import ModelInvocationUtils
from graphon.model_runtime.entities.llm_entities import (
@@ -42,20 +33,6 @@ from graphon.model_runtime.entities.message_entities import (
)
from graphon.model_runtime.entities.model_entities import ModelType
from models.account import Tenant
from models.provider_ids import ModelProviderID
def _json_compatible(value: Any) -> Any:
"""Convert model-runtime metadata into stable JSON-compatible values."""
if isinstance(value, BaseModel):
return value.model_dump(mode="json")
if isinstance(value, Enum):
return value.value
if isinstance(value, Mapping):
return {str(_json_compatible(key)): _json_compatible(child) for key, child in value.items()}
if isinstance(value, list | tuple | set):
return [_json_compatible(child) for child in value]
return value
class PluginModelBackwardsInvocation(BaseBackwardsInvocation):
@@ -206,30 +183,7 @@ class PluginModelBackwardsInvocation(BaseBackwardsInvocation):
)
# invoke model
response = model_instance.invoke_text_embedding(texts=payload.texts, input_type=payload.input_type)
return response
@classmethod
def invoke_multimodal_embedding(
cls,
user_id: str,
tenant: Tenant,
payload: RequestInvokeMultimodalEmbedding,
):
"""Invoke multimodal embedding through the tenant-bound model instance."""
model_instance = cls._get_bound_model_instance(
tenant_id=tenant.id,
user_id=user_id,
provider=payload.provider,
model_type=payload.model_type,
model=payload.model,
)
response = model_instance.invoke_multimodal_embedding(
multimodel_documents=[document.model_dump(exclude_none=True) for document in payload.documents],
input_type=payload.input_type,
)
response = model_instance.invoke_text_embedding(texts=payload.texts)
return response
@@ -256,67 +210,6 @@ class PluginModelBackwardsInvocation(BaseBackwardsInvocation):
return response
@classmethod
def list_models(
cls,
tenant_id: str,
user_id: str,
payload: RequestListModels,
) -> InvokableModelCatalogPage:
"""List only models that are active for the tenant's Dify configuration."""
provider_manager = create_plugin_provider_manager(tenant_id=tenant_id, user_id=user_id)
active_models = provider_manager.get_configurations(tenant_id).get_models(
model_type=payload.model_type,
only_active=True,
)
installed_identities: dict[str, str] = {}
for plugin in PluginService.list(tenant_id):
existing = installed_identities.get(plugin.plugin_id)
if existing is not None and existing != plugin.plugin_unique_identifier:
raise ValueError(f"Ambiguous installed identity for model plugin {plugin.plugin_id}")
installed_identities[plugin.plugin_id] = plugin.plugin_unique_identifier
requested_provider = str(ModelProviderID(payload.provider)) if payload.provider else None
matched_models = [
model
for model in active_models
if (requested_provider is None or model.provider.provider == requested_provider)
and (payload.model is None or model.model == payload.model)
]
matched_models.sort(key=lambda model: (model.provider.provider, model.model))
page_models = matched_models[payload.offset : payload.offset + payload.limit]
items: list[InvokableModelCatalogItem] = []
for model in page_models:
provider_id = ModelProviderID(model.provider.provider)
unique_identifier = installed_identities.get(provider_id.plugin_id)
if unique_identifier is None:
raise ValueError(f"Installed identity not found for active model plugin {provider_id.plugin_id}")
items.append(
InvokableModelCatalogItem(
plugin_id=provider_id.plugin_id,
plugin_unique_identifier=unique_identifier,
provider=provider_id.provider_name,
model=model.model,
model_type=model.model_type,
capabilities={
"deprecated": model.deprecated,
"features": _json_compatible(model.features or []),
"fetchFrom": _json_compatible(model.fetch_from),
"modelProperties": _json_compatible(model.model_properties),
"modelType": model.model_type.value,
"status": _json_compatible(model.status),
},
)
)
next_offset = payload.offset + len(page_models)
return InvokableModelCatalogPage(
items=items,
next_offset=next_offset if next_offset < len(matched_models) else None,
)
@classmethod
def invoke_tts(cls, user_id: str, tenant: Tenant, payload: RequestInvokeTTS):
"""

Some files were not shown because too many files have changed in this diff Show More