bump to 0.6.15 (#6592 )

fix(variables): NoneVariable should inherit from NoneSegment. (#6584 )
feat: update prompt generate (#6516 )
2024-07-23 22:46:42 +08:00 · 2024-07-23 21:46:08 +08:00 · 2024-07-23 19:52:14 +08:00 · 2024-07-23 19:51:38 +08:00 · 2024-07-23 17:59:32 +08:00 · 2024-07-23 17:24:29 +08:00
284 changed files with 5956 additions and 1281 deletions
@@ -0,0 +1,24 @@
+title: "General Discussion"
+body:
+  - type: checkboxes
+    attributes:
+      label: Self Checks
+      description: "To make sure we get to you in time, please check the following :)"
+      options:
+        - label: I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
+          required: true
+        - label: I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)).
+          required: true
+        - label: "请务必使用英文提交 Issue，否则会被关闭。谢谢！:）"
+          required: true
+        - label: "Please do not modify this template :) and fill in all the required fields."
+          required: true
+  - type: textarea
+    attributes:
+      label: Content
+      placeholder: Please describe the content you would like to discuss.
+    validations:
+      required: true
+  - type: markdown
+    attributes:
+      value: Please limit one request per issue.
@@ -0,0 +1,30 @@
+title: "Help"
+body:
+  - type: checkboxes
+    attributes:
+      label: Self Checks
+      description: "To make sure we get to you in time, please check the following :)"
+      options:
+        - label: I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
+          required: true
+        - label: I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)).
+          required: true
+        - label: "请务必使用英文提交 Issue，否则会被关闭。谢谢！:）"
+          required: true
+        - label: "Please do not modify this template :) and fill in all the required fields."
+          required: true
+  - type: textarea
+    attributes:
+      label: 1. Is this request related to a challenge you're experiencing? Tell me about your story.
+      placeholder: Please describe the specific scenario or problem you're facing as clearly as possible. For instance "I was trying to use [feature] for [specific task], and [what happened]... It was frustrating because...."
+    validations:
+      required: true
+  - type: textarea
+    attributes:
+      label: 2. Additional context or comments
+      placeholder: (Any other information, comments, documentations, links, or screenshots that would provide more clarity. This is the place to add anything else not covered above.)
+    validations:
+      required: false
+  - type: markdown
+    attributes:
+      value: Please limit one request per issue.
@@ -0,0 +1,37 @@
+title: Suggestions for New Features
+body:
+  - type: checkboxes
+    attributes:
+      label: Self Checks
+      description: "To make sure we get to you in time, please check the following :)"
+      options:
+        - label: I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
+          required: true
+        - label: I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)).
+          required: true
+        - label: "请务必使用英文提交 Issue，否则会被关闭。谢谢！:）"
+          required: true
+        - label: "Please do not modify this template :) and fill in all the required fields."
+          required: true
+  - type: textarea
+    attributes:
+      label: 1. Is this request related to a challenge you're experiencing? Tell me about your story.
+      placeholder: Please describe the specific scenario or problem you're facing as clearly as possible. For instance "I was trying to use [feature] for [specific task], and [what happened]... It was frustrating because...."
+    validations:
+      required: true
+  - type: textarea
+    attributes:
+      label: 2. Additional context or comments
+      placeholder: (Any other information, comments, documentations, links, or screenshots that would provide more clarity. This is the place to add anything else not covered above.)
+    validations:
+      required: false
+  - type: checkboxes
+    attributes:
+      label: 3. Can you help us with this feature?
+      description: Let us know! This is not a commitment, but a starting point for collaboration.
+      options:
+        - label: I am interested in contributing to this feature.
+          required: false
+  - type: markdown
+    attributes:
+      value: Please limit one request per issue.
@@ -174,5 +174,6 @@ sdks/python-client/dify_client.egg-info
 .vscode/*
 !.vscode/launch.json
 pyrightconfig.json
+api/.vscode

 .idea/
@@ -216,7 +216,6 @@ At the same time, please consider supporting Dify by sharing it on social media

 * [Github Discussion](https://github.com/langgenius/dify/discussions). Best for: sharing feedback and asking questions.
 * [GitHub Issues](https://github.com/langgenius/dify/issues). Best for: bugs you encounter using Dify.AI, and feature proposals. See our [Contribution Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md).
-* [Email](mailto:[email protected]?subject=[GitHub]Questions%20About%20Dify). Best for: questions you have about using Dify.AI.
 * [Discord](https://discord.gg/FngNHpbcY7). Best for: sharing your applications and hanging out with the community.
 * [Twitter](https://twitter.com/dify_ai). Best for: sharing your applications and hanging out with the community.

@@ -199,7 +199,6 @@ docker compose up -d
 ## المجتمع والاتصال
 * [مناقشة Github](https://github.com/langgenius/dify/discussions). الأفضل لـ: مشاركة التعليقات وطرح الأسئلة.
 * [المشكلات على GitHub](https://github.com/langgenius/dify/issues). الأفضل لـ: الأخطاء التي تواجهها في استخدام Dify.AI، واقتراحات الميزات. انظر [دليل المساهمة](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md).
-* [البريد الإلكتروني](mailto:[email protected]?subject=[GitHub]Questions%20About%20Dify). الأفضل لـ: الأسئلة التي تتعلق باستخدام Dify.AI.
 * [Discord](https://discord.gg/FngNHpbcY7). الأفضل لـ: مشاركة تطبيقاتك والترفيه مع المجتمع.
 * [تويتر](https://twitter.com/dify_ai). الأفضل لـ: مشاركة تطبيقاتك والترفيه مع المجتمع.

@@ -224,7 +224,6 @@ Al mismo tiempo, considera apoyar a Dify compartiéndolo en redes sociales y en

 * [Discusión en GitHub](https://github.com/langgenius/dify/discussions). Lo mejor para: compartir comentarios y hacer preguntas.
 * [Reporte de problemas en GitHub](https://github.com/langgenius/dify/issues). Lo mejor para: errores que encuentres usando Dify.AI y propuestas de características. Consulta nuestra [Guía de contribución](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md).
-* [Correo electrónico](mailto:[email protected]?subject=[GitHub]Questions%20About%20Dify). Lo mejor para: preguntas que tengas sobre el uso de Dify.AI.
 * [Discord](https://discord.gg/FngNHpbcY7). Lo mejor para: compartir tus aplicaciones y pasar el rato con la comunidad.
 * [Twitter](https://twitter.com/dify_ai). Lo mejor para: compartir tus aplicaciones y pasar el rato con la comunidad.

@@ -222,7 +222,6 @@ Dans le même temps, veuillez envisager de soutenir Dify en le partageant sur le

 * [Discussion GitHub](https://github.com/langgenius/dify/discussions). Meilleur pour: partager des commentaires et poser des questions.
 * [Problèmes GitHub](https://github.com/langgenius/dify/issues). Meilleur pour: les bogues que vous rencontrez en utilisant Dify.AI et les propositions de fonctionnalités. Consultez notre [Guide de contribution](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md).
-* [E-mail](mailto:[email protected]?subject=[GitHub]Questions%20About%20Dify). Meilleur pour: les questions que vous avez sur l'utilisation de Dify.AI.
 * [Discord](https://discord.gg/FngNHpbcY7). Meilleur pour: partager vos applications et passer du temps avec la communauté.
 * [Twitter](https://twitter.com/dify_ai). Meilleur pour: partager vos applications et passer du temps avec la communauté.

@@ -221,7 +221,6 @@ docker compose up -d

 * [Github Discussion](https://github.com/langgenius/dify/discussions). 主に: フィードバックの共有や質問。
 * [GitHub Issues](https://github.com/langgenius/dify/issues). 主に: Dify.AIを使用する際に発生するエラーや問題については、[貢献ガイド](CONTRIBUTING_JA.md)を参照してください
-* [Email](mailto:[email protected]?subject=[GitHub]Questions%20About%20Dify). 主に: Dify.AIの使用に関する質問。
 * [Discord](https://discord.gg/FngNHpbcY7). 主に: アプリケーションの共有やコミュニティとの交流。
 * [Twitter](https://twitter.com/dify_ai). 主に: アプリケーションの共有やコミュニティとの交流。

@@ -239,7 +238,7 @@ docker compose up -d
    <td>無料の30分間のミーティングをスケジュール</td>
  </tr>
  <tr>
-    <td><a href='mailto:support@dify.ai?subject=[GitHub]Technical%20Support'>技術サポート</a></td>
+    <td><a href='https://github.com/langgenius/dify/issues'>技術サポート</a></td>
    <td>技術的な問題やサポートに関する質問</td>
  </tr>
  <tr>
@@ -224,7 +224,6 @@ At the same time, please consider supporting Dify by sharing it on social media

 ). Best for: sharing feedback and asking questions.
 * [GitHub Issues](https://github.com/langgenius/dify/issues). Best for: bugs you encounter using Dify.AI, and feature proposals. See our [Contribution Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md).
-* [Email](mailto:[email protected]?subject=[GitHub]Questions%20About%20Dify). Best for: questions you have about using Dify.AI.
 * [Discord](https://discord.gg/FngNHpbcY7). Best for: sharing your applications and hanging out with the community.
 * [Twitter](https://twitter.com/dify_ai). Best for: sharing your applications and hanging out with the community.

@@ -214,7 +214,6 @@ Dify를 Kubernetes에 배포하고 프리미엄 스케일링 설정을 구성했

 * [Github 토론](https://github.com/langgenius/dify/discussions). 피드백 공유 및 질문하기에 적합합니다.
 * [GitHub 이슈](https://github.com/langgenius/dify/issues). Dify.AI 사용 중 발견한 버그와 기능 제안에 적합합니다. [기여 가이드](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md)를 참조하세요. 
-* [이메일](mailto:[email protected]?subject=[GitHub]Questions%20About%20Dify). Dify.AI 사용에 대한 질문하기에 적합합니다.
 * [디스코드](https://discord.gg/FngNHpbcY7). 애플리케이션 공유 및 커뮤니티와 소통하기에 적합합니다.
 * [트위터](https://twitter.com/dify_ai). 애플리케이션 공유 및 커뮤니티와 소통하기에 적합합니다.

@@ -1,7 +1,5 @@
 import os

-from configs import dify_config
-
 if os.environ.get("DEBUG", "false").lower() != 'true':
    from gevent import monkey

@@ -23,7 +21,9 @@ from flask import Flask, Response, request
 from flask_cors import CORS
 from werkzeug.exceptions import Unauthorized

+import contexts
 from commands import register_commands
+from configs import dify_config

 # DO NOT REMOVE BELOW
 from events import event_handlers
@@ -181,7 +181,10 @@ def load_user_from_request(request_from_flask_login):
    decoded = PassportService().verify(auth_token)
    user_id = decoded.get('user_id')

-    return AccountService.load_logged_in_account(account_id=user_id, token=auth_token)
+    account = AccountService.load_logged_in_account(account_id=user_id, token=auth_token)
+    if account:
+        contexts.tenant_id.set(account.current_tenant_id)
+    return account


@login_manager.unauthorized_handler
@@ -406,7 +406,6 @@ class DataSetConfig(BaseSettings):
        default=False,
    )

-
 class WorkspaceConfig(BaseSettings):
    """
    Workspace configs
@@ -9,7 +9,7 @@ class PackagingInfo(BaseSettings):

    CURRENT_VERSION: str = Field(
        description='Dify version',
-        default='0.6.14',
+        default='0.6.15',
    )

    COMMIT_SHA: str = Field(
@@ -0,0 +1,2 @@
+# TODO: Update all string in code to use this constant
+HIDDEN_VALUE = '[__HIDDEN__]'
@@ -0,0 +1,3 @@
+from contextvars import ContextVar
+
+tenant_id: ContextVar[str] = ContextVar('tenant_id')
@@ -212,7 +212,7 @@ class AppCopyApi(Resource):
        parser.add_argument('icon_background', type=str, location='json')
        args = parser.parse_args()

-        data = AppDslService.export_dsl(app_model=app_model)
+        data = AppDslService.export_dsl(app_model=app_model, include_secret=True)
        app = AppDslService.import_and_create_new_app(
            tenant_id=current_user.current_tenant_id,
            data=data,
@@ -234,8 +234,13 @@ class AppExportApi(Resource):
        if not current_user.is_editor:
            raise Forbidden()

+        # Add include_secret params
+        parser = reqparse.RequestParser()
+        parser.add_argument('include_secret', type=inputs.boolean, default=False, location='args')
+        args = parser.parse_args()
+
        return {
-            "data": AppDslService.export_dsl(app_model=app_model)
+            "data": AppDslService.export_dsl(app_model=app_model, include_secret=args['include_secret'])
        }


@@ -22,17 +22,19 @@ class RuleGenerateApi(Resource):
    @account_initialization_required
    def post(self):
        parser = reqparse.RequestParser()
-        parser.add_argument('audiences', type=str, required=True, nullable=False, location='json')
-        parser.add_argument('hoping_to_solve', type=str, required=True, nullable=False, location='json')
+        parser.add_argument('instruction', type=str, required=True, nullable=False, location='json')
+        parser.add_argument('model_config', type=dict, required=True, nullable=False, location='json')
+        parser.add_argument('no_variable', type=bool, required=True, default=False, location='json')
        args = parser.parse_args()

        account = current_user

        try:
            rules = LLMGenerator.generate_rule_config(
-                account.current_tenant_id,
-                args['audiences'],
-                args['hoping_to_solve']
+                tenant_id=account.current_tenant_id,
+                instruction=args['instruction'],
+                model_config=args['model_config'],
+                no_variable=args['no_variable']
            )
        except ProviderTokenNotInitError as ex:
            raise ProviderNotInitializeError(ex.description)
@@ -281,7 +281,7 @@ class UserSatisfactionRateStatistic(Resource):
                        SELECT date(DATE_TRUNC('day', m.created_at AT TIME ZONE 'UTC' AT TIME ZONE :tz )) AS date, 
                            COUNT(m.id) as message_count, COUNT(mf.id) as feedback_count 
                            FROM messages m
-                            LEFT JOIN message_feedbacks mf on mf.message_id=m.id
+                            LEFT JOIN message_feedbacks mf on mf.message_id=m.id and mf.rating='like'
                            WHERE m.app_id = :app_id 
                        '''
        arg_dict = {'tz': account.timezone, 'app_id': app_model.id}
@@ -13,6 +13,7 @@ from controllers.console.setup import setup_required
 from controllers.console.wraps import account_initialization_required
 from core.app.apps.base_app_queue_manager import AppQueueManager
 from core.app.entities.app_invoke_entities import InvokeFrom
+from core.app.segments import factory
 from core.errors.error import AppInvokeQuotaExceededError
 from fields.workflow_fields import workflow_fields
 from fields.workflow_run_fields import workflow_run_node_execution_fields
@@ -41,7 +42,7 @@ class DraftWorkflowApi(Resource):
        # The role of the current user in the ta table must be admin, owner, or editor
        if not current_user.is_editor:
            raise Forbidden()
-        
+
        # fetch draft workflow by app_model
        workflow_service = WorkflowService()
        workflow = workflow_service.get_draft_workflow(app_model=app_model)
@@ -64,13 +65,15 @@ class DraftWorkflowApi(Resource):
        if not current_user.is_editor:
            raise Forbidden()
        
-        content_type = request.headers.get('Content-Type')
+        content_type = request.headers.get('Content-Type', '')

        if 'application/json' in content_type:
            parser = reqparse.RequestParser()
            parser.add_argument('graph', type=dict, required=True, nullable=False, location='json')
            parser.add_argument('features', type=dict, required=True, nullable=False, location='json')
            parser.add_argument('hash', type=str, required=False, location='json')
+            # TODO: set this to required=True after frontend is updated
+            parser.add_argument('environment_variables', type=list, required=False, location='json')
            args = parser.parse_args()
        elif 'text/plain' in content_type:
            try:
@@ -84,7 +87,8 @@ class DraftWorkflowApi(Resource):
                args = {
                    'graph': data.get('graph'),
                    'features': data.get('features'),
-                    'hash': data.get('hash')
+                    'hash': data.get('hash'),
+                    'environment_variables': data.get('environment_variables')
                }
            except json.JSONDecodeError:
                return {'message': 'Invalid JSON data'}, 400
@@ -94,12 +98,15 @@ class DraftWorkflowApi(Resource):
        workflow_service = WorkflowService()

        try:
+            environment_variables_list = args.get('environment_variables') or []
+            environment_variables = [factory.build_variable_from_mapping(obj) for obj in environment_variables_list]
            workflow = workflow_service.sync_draft_workflow(
                app_model=app_model,
-                graph=args.get('graph'),
-                features=args.get('features'),
+                graph=args['graph'],
+                features=args['features'],
                unique_hash=args.get('hash'),
-                account=current_user
+                account=current_user,
+                environment_variables=environment_variables,
            )
        except WorkflowHashNotEqualError:
            raise DraftWorkflowNotSync()
@@ -1,10 +1,11 @@
 import flask_restful
-from flask import current_app, request
+from flask import request
 from flask_login import current_user
 from flask_restful import Resource, marshal, marshal_with, reqparse
 from werkzeug.exceptions import Forbidden, NotFound

 import services
+from configs import dify_config
 from controllers.console import api
 from controllers.console.apikey import api_key_fields, api_key_list
 from controllers.console.app.error import ProviderNotInitializeError
@@ -530,7 +531,7 @@ class DatasetApiBaseUrlApi(Resource):
    @account_initialization_required
    def get(self):
        return {
-            'api_base_url': (current_app.config['SERVICE_API_URL'] if current_app.config['SERVICE_API_URL']
+            'api_base_url': (dify_config.SERVICE_API_URL if dify_config.SERVICE_API_URL
                             else request.host_url.rstrip('/')) + '/v1'
        }

@@ -540,15 +541,15 @@ class DatasetRetrievalSettingApi(Resource):
    @login_required
    @account_initialization_required
    def get(self):
-        vector_type = current_app.config['VECTOR_STORE']
+        vector_type = dify_config.VECTOR_STORE
        match vector_type:
-            case VectorType.MILVUS | VectorType.RELYT | VectorType.PGVECTOR | VectorType.TIDB_VECTOR | VectorType.CHROMA | VectorType.TENCENT | VectorType.ORACLE:
+            case VectorType.MILVUS | VectorType.RELYT | VectorType.PGVECTOR | VectorType.TIDB_VECTOR | VectorType.CHROMA | VectorType.TENCENT:
                return {
                    'retrieval_method': [
                        RetrievalMethod.SEMANTIC_SEARCH.value
                    ]
                }
-            case VectorType.QDRANT | VectorType.WEAVIATE | VectorType.OPENSEARCH | VectorType.ANALYTICDB | VectorType.MYSCALE:
+            case VectorType.QDRANT | VectorType.WEAVIATE | VectorType.OPENSEARCH | VectorType.ANALYTICDB | VectorType.MYSCALE | VectorType.ORACLE:
                return {
                    'retrieval_method': [
                        RetrievalMethod.SEMANTIC_SEARCH.value,
@@ -566,13 +567,13 @@ class DatasetRetrievalSettingMockApi(Resource):
    @account_initialization_required
    def get(self, vector_type):
        match vector_type:
-            case VectorType.MILVUS | VectorType.RELYT | VectorType.PGVECTOR | VectorType.TIDB_VECTOR | VectorType.CHROMA | VectorType.TENCENT | VectorType.ORACLE:
+            case VectorType.MILVUS | VectorType.RELYT | VectorType.PGVECTOR | VectorType.TIDB_VECTOR | VectorType.CHROMA | VectorType.TENCENT:
                return {
                    'retrieval_method': [
                        RetrievalMethod.SEMANTIC_SEARCH.value
                    ]
                }
-            case VectorType.QDRANT | VectorType.WEAVIATE | VectorType.OPENSEARCH| VectorType.ANALYTICDB | VectorType.MYSCALE:
+            case VectorType.QDRANT | VectorType.WEAVIATE | VectorType.OPENSEARCH| VectorType.ANALYTICDB | VectorType.MYSCALE | VectorType.ORACLE:
                return {
                    'retrieval_method': [
                        RetrievalMethod.SEMANTIC_SEARCH.value,
@@ -1,8 +1,9 @@
-from flask import current_app, request
+from flask import request
 from flask_login import current_user
 from flask_restful import Resource, marshal_with

 import services
+from configs import dify_config
 from controllers.console import api
 from controllers.console.datasets.error import (
    FileTooLargeError,
@@ -26,9 +27,9 @@ class FileApi(Resource):
    @account_initialization_required
    @marshal_with(upload_config_fields)
    def get(self):
-        file_size_limit = current_app.config.get("UPLOAD_FILE_SIZE_LIMIT")
-        batch_count_limit = current_app.config.get("UPLOAD_FILE_BATCH_LIMIT")
-        image_file_size_limit = current_app.config.get("UPLOAD_IMAGE_FILE_SIZE_LIMIT")
+        file_size_limit = dify_config.UPLOAD_FILE_SIZE_LIMIT
+        batch_count_limit = dify_config.UPLOAD_FILE_BATCH_LIMIT
+        image_file_size_limit = dify_config.UPLOAD_IMAGE_FILE_SIZE_LIMIT
        return {
            'file_size_limit': file_size_limit,
            'batch_count_limit': batch_count_limit,
@@ -76,7 +77,7 @@ class FileSupportTypeApi(Resource):
    @login_required
    @account_initialization_required
    def get(self):
-        etl_type = current_app.config['ETL_TYPE']
+        etl_type = dify_config.ETL_TYPE
        allowed_extensions = UNSTRUCTURED_ALLOWED_EXTENSIONS if etl_type == 'Unstructured' else ALLOWED_EXTENSIONS
        return {'allowed_extensions': allowed_extensions}

@@ -1,7 +1,7 @@

-from flask import current_app
 from flask_restful import fields, marshal_with

+from configs import dify_config
 from controllers.console import api
 from controllers.console.app.error import AppUnavailableError
 from controllers.console.explore.wraps import InstalledAppResource
@@ -78,7 +78,7 @@ class AppParameterApi(InstalledAppResource):
                                                     "transfer_methods": ["remote_url", "local_file"]
                                                 }}),
            'system_parameters': {
-                'image_file_size_limit': current_app.config.get('UPLOAD_IMAGE_FILE_SIZE_LIMIT')
+                'image_file_size_limit': dify_config.UPLOAD_IMAGE_FILE_SIZE_LIMIT
            }
        }

@@ -1,8 +1,9 @@
 import os

-from flask import current_app, session
+from flask import session
 from flask_restful import Resource, reqparse

+from configs import dify_config
 from libs.helper import str_len
 from models.model import DifySetup
 from services.account_service import TenantService
@@ -40,7 +41,7 @@ class InitValidateAPI(Resource):
        return {'result': 'success'}, 201

 def get_init_validate_status():
-    if current_app.config['EDITION'] == 'SELF_HOSTED':
+    if dify_config.EDITION == 'SELF_HOSTED':
        if os.environ.get('INIT_PASSWORD'):
            return session.get('is_init_validated') or DifySetup.query.first()
    
@@ -1,8 +1,9 @@
 from functools import wraps

-from flask import current_app, request
+from flask import request
 from flask_restful import Resource, reqparse

+from configs import dify_config
 from libs.helper import email, get_remote_ip, str_len
 from libs.password import valid_password
 from models.model import DifySetup
@@ -17,7 +18,7 @@ from .wraps import only_edition_self_hosted
 class SetupApi(Resource):

    def get(self):
-        if current_app.config['EDITION'] == 'SELF_HOSTED':
+        if dify_config.EDITION == 'SELF_HOSTED':
            setup_status = get_setup_status()
            if setup_status:
                return {
@@ -77,7 +78,7 @@ def setup_required(view):


 def get_setup_status():
-    if current_app.config['EDITION'] == 'SELF_HOSTED':
+    if dify_config.EDITION == 'SELF_HOSTED':
        return DifySetup.query.first()
    else:
        return True
@@ -3,9 +3,10 @@ import json
 import logging

 import requests
-from flask import current_app
 from flask_restful import Resource, reqparse

+from configs import dify_config
+
 from . import api


@@ -15,16 +16,16 @@ class VersionApi(Resource):
        parser = reqparse.RequestParser()
        parser.add_argument('current_version', type=str, required=True, location='args')
        args = parser.parse_args()
-        check_update_url = current_app.config['CHECK_UPDATE_URL']
+        check_update_url = dify_config.CHECK_UPDATE_URL

        result = {
-            'version': current_app.config['CURRENT_VERSION'],
+            'version': dify_config.CURRENT_VERSION,
            'release_date': '',
            'release_notes': '',
            'can_auto_update': False,
            'features': {
-                'can_replace_logo': current_app.config['CAN_REPLACE_LOGO'],
-                'model_load_balancing_enabled': current_app.config['MODEL_LB_ENABLED']
+                'can_replace_logo': dify_config.CAN_REPLACE_LOGO,
+                'model_load_balancing_enabled': dify_config.MODEL_LB_ENABLED
            }
        }

@@ -1,10 +1,11 @@
 import datetime

 import pytz
-from flask import current_app, request
+from flask import request
 from flask_login import current_user
 from flask_restful import Resource, fields, marshal_with, reqparse

+from configs import dify_config
 from constants.languages import supported_language
 from controllers.console import api
 from controllers.console.setup import setup_required
@@ -36,7 +37,7 @@ class AccountInitApi(Resource):

        parser = reqparse.RequestParser()

-        if current_app.config['EDITION'] == 'CLOUD':
+        if dify_config.EDITION == 'CLOUD':
            parser.add_argument('invitation_code', type=str, location='json')

        parser.add_argument(
@@ -45,7 +46,7 @@ class AccountInitApi(Resource):
                            required=True, location='json')
        args = parser.parse_args()

-        if current_app.config['EDITION'] == 'CLOUD':
+        if dify_config.EDITION == 'CLOUD':
            if not args['invitation_code']:
                raise ValueError('invitation_code is required')

@@ -1,8 +1,8 @@
-from flask import current_app
 from flask_login import current_user
 from flask_restful import Resource, abort, marshal_with, reqparse

 import services
+from configs import dify_config
 from controllers.console import api
 from controllers.console.setup import setup_required
 from controllers.console.wraps import account_initialization_required, cloud_edition_billing_resource_check
@@ -48,7 +48,7 @@ class MemberInviteEmailApi(Resource):

        inviter = current_user
        invitation_results = []
-        console_web_url = current_app.config.get("CONSOLE_WEB_URL")
+        console_web_url = dify_config.CONSOLE_WEB_URL
        for invitee_email in invitee_emails:
            try:
                token = RegisterService.invite_new_member(inviter.current_tenant, invitee_email, interface_language, role=invitee_role, inviter=inviter)
@@ -1,10 +1,11 @@
 import io

-from flask import current_app, send_file
+from flask import send_file
 from flask_login import current_user
 from flask_restful import Resource, reqparse
 from werkzeug.exceptions import Forbidden

+from configs import dify_config
 from controllers.console import api
 from controllers.console.setup import setup_required
 from controllers.console.wraps import account_initialization_required
@@ -104,7 +105,7 @@ class ToolBuiltinProviderIconApi(Resource):
    @setup_required
    def get(self, provider):
        icon_bytes, mimetype = BuiltinToolManageService.get_builtin_tool_provider_icon(provider)
-        icon_cache_max_age = current_app.config.get('TOOL_ICON_CACHE_MAX_AGE')
+        icon_cache_max_age = dify_config.TOOL_ICON_CACHE_MAX_AGE
        return send_file(io.BytesIO(icon_bytes), mimetype=mimetype, max_age=icon_cache_max_age)

 class ToolApiProviderAddApi(Resource):
@@ -1,9 +1,10 @@
 import json
 from functools import wraps

-from flask import abort, current_app, request
+from flask import abort, request
 from flask_login import current_user

+from configs import dify_config
 from controllers.console.workspace.error import AccountNotInitializedError
 from services.feature_service import FeatureService
 from services.operation_service import OperationService
@@ -26,7 +27,7 @@ def account_initialization_required(view):
 def only_edition_cloud(view):
    @wraps(view)
    def decorated(*args, **kwargs):
-        if current_app.config['EDITION'] != 'CLOUD':
+        if dify_config.EDITION != 'CLOUD':
            abort(404)

        return view(*args, **kwargs)
@@ -37,7 +38,7 @@ def only_edition_cloud(view):
 def only_edition_self_hosted(view):
    @wraps(view)
    def decorated(*args, **kwargs):
-        if current_app.config['EDITION'] != 'SELF_HOSTED':
+        if dify_config.EDITION != 'SELF_HOSTED':
            abort(404)

        return view(*args, **kwargs)
@@ -342,10 +342,14 @@ class FunctionCallAgentRunner(BaseAgentRunner):
        """
        tool_calls = []
        for prompt_message in llm_result_chunk.delta.message.tool_calls:
+            args = {}
+            if prompt_message.function.arguments != '':
+                args = json.loads(prompt_message.function.arguments)
+
            tool_calls.append((
                prompt_message.id,
                prompt_message.function.name,
-                json.loads(prompt_message.function.arguments),
+                args,
            ))

        return tool_calls
@@ -359,10 +363,14 @@ class FunctionCallAgentRunner(BaseAgentRunner):
        """
        tool_calls = []
        for prompt_message in llm_result.message.tool_calls:
+            args = {}
+            if prompt_message.function.arguments != '':
+                args = json.loads(prompt_message.function.arguments)
+
            tool_calls.append((
                prompt_message.id,
                prompt_message.function.name,
-                json.loads(prompt_message.function.arguments),
+                args,
            ))

        return tool_calls
@@ -1,6 +1,7 @@
-from typing import Optional, Union
+from collections.abc import Mapping
+from typing import Any

-from core.app.app_config.entities import AppAdditionalFeatures, EasyUIBasedAppModelConfigFrom
+from core.app.app_config.entities import AppAdditionalFeatures
 from core.app.app_config.features.file_upload.manager import FileUploadConfigManager
 from core.app.app_config.features.more_like_this.manager import MoreLikeThisConfigManager
 from core.app.app_config.features.opening_statement.manager import OpeningStatementConfigManager
@@ -10,37 +11,19 @@ from core.app.app_config.features.suggested_questions_after_answer.manager impor
    SuggestedQuestionsAfterAnswerConfigManager,
 )
 from core.app.app_config.features.text_to_speech.manager import TextToSpeechConfigManager
-from models.model import AppMode, AppModelConfig
+from models.model import AppMode


 class BaseAppConfigManager:
-
    @classmethod
-    def convert_to_config_dict(cls, config_from: EasyUIBasedAppModelConfigFrom,
-                               app_model_config: Union[AppModelConfig, dict],
-                               config_dict: Optional[dict] = None) -> dict:
-        """
-        Convert app model config to config dict
-        :param config_from: app model config from
-        :param app_model_config: app model config
-        :param config_dict: app model config dict
-        :return:
-        """
-        if config_from != EasyUIBasedAppModelConfigFrom.ARGS:
-            app_model_config_dict = app_model_config.to_dict()
-            config_dict = app_model_config_dict.copy()
-
-        return config_dict
-
-    @classmethod
-    def convert_features(cls, config_dict: dict, app_mode: AppMode) -> AppAdditionalFeatures:
+    def convert_features(cls, config_dict: Mapping[str, Any], app_mode: AppMode) -> AppAdditionalFeatures:
        """
        Convert app config to app model config

        :param config_dict: app config
        :param app_mode: app mode
        """
-        config_dict = config_dict.copy()
+        config_dict = dict(config_dict.items())

        additional_features = AppAdditionalFeatures()
        additional_features.show_retrieve_source = RetrievalResourceConfigManager.convert(
@@ -1,11 +1,12 @@
-from typing import Optional
+from collections.abc import Mapping
+from typing import Any, Optional

 from core.app.app_config.entities import FileExtraConfig


 class FileUploadConfigManager:
    @classmethod
-    def convert(cls, config: dict, is_vision: bool = True) -> Optional[FileExtraConfig]:
+    def convert(cls, config: Mapping[str, Any], is_vision: bool = True) -> Optional[FileExtraConfig]:
        """
        Convert model config to model config

@@ -3,13 +3,13 @@ from core.app.app_config.entities import TextToSpeechEntity

 class TextToSpeechConfigManager:
    @classmethod
-    def convert(cls, config: dict) -> bool:
+    def convert(cls, config: dict):
        """
        Convert model config to model config

        :param config: model config args
        """
-        text_to_speech = False
+        text_to_speech = None
        text_to_speech_dict = config.get('text_to_speech')
        if text_to_speech_dict:
            if text_to_speech_dict.get('enabled'):
@@ -1,3 +1,4 @@
+import contextvars
 import logging
 import os
 import threading
@@ -8,6 +9,7 @@ from typing import Union
 from flask import Flask, current_app
 from pydantic import ValidationError

+import contexts
 from core.app.app_config.features.file_upload.manager import FileUploadConfigManager
 from core.app.apps.advanced_chat.app_config_manager import AdvancedChatAppConfigManager
 from core.app.apps.advanced_chat.app_runner import AdvancedChatAppRunner
@@ -107,6 +109,7 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
            extras=extras,
            trace_manager=trace_manager
        )
+        contexts.tenant_id.set(application_generate_entity.app_config.tenant_id)

        return self._generate(
            app_model=app_model,
@@ -173,6 +176,7 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
                inputs=args['inputs']
            )
        )
+        contexts.tenant_id.set(application_generate_entity.app_config.tenant_id)

        return self._generate(
            app_model=app_model,
@@ -225,6 +229,8 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
            'queue_manager': queue_manager,
            'conversation_id': conversation.id,
            'message_id': message.id,
+            'user': user,
+            'context': contextvars.copy_context()
        })

        worker_thread.start()
@@ -249,7 +255,9 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
                         application_generate_entity: AdvancedChatAppGenerateEntity,
                         queue_manager: AppQueueManager,
                         conversation_id: str,
-                         message_id: str) -> None:
+                         message_id: str,
+                         user: Account,
+                         context: contextvars.Context) -> None:
        """
        Generate worker in a new thread.
        :param flask_app: Flask app
@@ -259,6 +267,8 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
        :param message_id: message ID
        :return:
        """
+        for var, val in context.items():
+            var.set(val)
        with flask_app.app_context():
            try:
                runner = AdvancedChatAppRunner()
@@ -1,7 +1,8 @@
 import logging
 import os
 import time
-from typing import Optional, cast
+from collections.abc import Mapping
+from typing import Any, Optional, cast

 from core.app.apps.advanced_chat.app_config_manager import AdvancedChatAppConfig
 from core.app.apps.advanced_chat.workflow_event_trigger_callback import WorkflowEventTriggerCallback
@@ -14,6 +15,7 @@ from core.app.entities.app_invoke_entities import (
 )
 from core.app.entities.queue_entities import QueueAnnotationReplyEvent, QueueStopEvent, QueueTextChunkEvent
 from core.moderation.base import ModerationException
+from core.workflow.callbacks.base_workflow_callback import WorkflowCallback
 from core.workflow.entities.node_entities import SystemVariable
 from core.workflow.nodes.base_node import UserFrom
 from core.workflow.workflow_engine_manager import WorkflowEngineManager
@@ -87,7 +89,7 @@ class AdvancedChatAppRunner(AppRunner):

        db.session.close()

-        workflow_callbacks = [WorkflowEventTriggerCallback(
+        workflow_callbacks: list[WorkflowCallback] = [WorkflowEventTriggerCallback(
            queue_manager=queue_manager,
            workflow=workflow
        )]
@@ -161,7 +163,7 @@ class AdvancedChatAppRunner(AppRunner):
            self, queue_manager: AppQueueManager,
            app_record: App,
            app_generate_entity: AdvancedChatAppGenerateEntity,
-            inputs: dict,
+            inputs: Mapping[str, Any],
            query: str,
            message_id: str
    ) -> bool:
@@ -1,9 +1,11 @@
 import json
 from collections.abc import Generator
-from typing import cast
+from typing import Any, cast

 from core.app.apps.base_app_generate_response_converter import AppGenerateResponseConverter
 from core.app.entities.task_entities import (
+    AppBlockingResponse,
+    AppStreamResponse,
    ChatbotAppBlockingResponse,
    ChatbotAppStreamResponse,
    ErrorStreamResponse,
@@ -18,12 +20,13 @@ class AdvancedChatAppGenerateResponseConverter(AppGenerateResponseConverter):
    _blocking_response_type = ChatbotAppBlockingResponse

    @classmethod
-    def convert_blocking_full_response(cls, blocking_response: ChatbotAppBlockingResponse) -> dict:
+    def convert_blocking_full_response(cls, blocking_response: AppBlockingResponse) -> dict[str, Any]:
        """
        Convert blocking full response.
        :param blocking_response: blocking response
        :return:
        """
+        blocking_response = cast(ChatbotAppBlockingResponse, blocking_response)
        response = {
            'event': 'message',
            'task_id': blocking_response.task_id,
@@ -39,7 +42,7 @@ class AdvancedChatAppGenerateResponseConverter(AppGenerateResponseConverter):
        return response

    @classmethod
-    def convert_blocking_simple_response(cls, blocking_response: ChatbotAppBlockingResponse) -> dict:
+    def convert_blocking_simple_response(cls, blocking_response: AppBlockingResponse) -> dict[str, Any]:
        """
        Convert blocking simple response.
        :param blocking_response: blocking response
@@ -53,8 +56,7 @@ class AdvancedChatAppGenerateResponseConverter(AppGenerateResponseConverter):
        return response

    @classmethod
-    def convert_stream_full_response(cls, stream_response: Generator[ChatbotAppStreamResponse, None, None]) \
-            -> Generator[str, None, None]:
+    def convert_stream_full_response(cls, stream_response: Generator[AppStreamResponse, None, None]) -> Generator[str, Any, None]:
        """
        Convert stream full response.
        :param stream_response: stream response
@@ -83,8 +85,7 @@ class AdvancedChatAppGenerateResponseConverter(AppGenerateResponseConverter):
            yield json.dumps(response_chunk)

    @classmethod
-    def convert_stream_simple_response(cls, stream_response: Generator[ChatbotAppStreamResponse, None, None]) \
-            -> Generator[str, None, None]:
+    def convert_stream_simple_response(cls, stream_response: Generator[AppStreamResponse, None, None]) -> Generator[str, Any, None]:
        """
        Convert stream simple response.
        :param stream_response: stream response
@@ -118,7 +118,7 @@ class AdvancedChatAppGenerateTaskPipeline(BasedGenerateTaskPipeline, WorkflowCyc
        self._stream_generate_routes = self._get_stream_generate_routes()
        self._conversation_name_generate_thread = None

-    def process(self) -> Union[ChatbotAppBlockingResponse, Generator[ChatbotAppStreamResponse, None, None]]:
+    def process(self):
        """
        Process generate task pipeline.
        :return:
@@ -141,8 +141,7 @@ class AdvancedChatAppGenerateTaskPipeline(BasedGenerateTaskPipeline, WorkflowCyc
        else:
            return self._to_blocking_response(generator)

-    def _to_blocking_response(self, generator: Generator[StreamResponse, None, None]) \
-            -> ChatbotAppBlockingResponse:
+    def _to_blocking_response(self, generator: Generator[StreamResponse, None, None]) -> ChatbotAppBlockingResponse:
        """
        Process blocking response.
        :return:
@@ -172,8 +171,7 @@ class AdvancedChatAppGenerateTaskPipeline(BasedGenerateTaskPipeline, WorkflowCyc

        raise Exception('Queue listening stopped unexpectedly.')

-    def _to_stream_response(self, generator: Generator[StreamResponse, None, None]) \
-            -> Generator[ChatbotAppStreamResponse, None, None]:
+    def _to_stream_response(self, generator: Generator[StreamResponse, None, None]) -> Generator[ChatbotAppStreamResponse, Any, None]:
        """
        To stream response.
        :return:
@@ -14,13 +14,13 @@ from core.app.entities.queue_entities import (
    QueueWorkflowStartedEvent,
    QueueWorkflowSucceededEvent,
 )
-from core.workflow.callbacks.base_workflow_callback import BaseWorkflowCallback
+from core.workflow.callbacks.base_workflow_callback import WorkflowCallback
 from core.workflow.entities.base_node_data_entities import BaseNodeData
 from core.workflow.entities.node_entities import NodeType
 from models.workflow import Workflow


-class WorkflowEventTriggerCallback(BaseWorkflowCallback):
+class WorkflowEventTriggerCallback(WorkflowCallback):

    def __init__(self, queue_manager: AppQueueManager, workflow: Workflow):
        self._queue_manager = queue_manager
@@ -1,7 +1,7 @@
 import logging
 from abc import ABC, abstractmethod
 from collections.abc import Generator
-from typing import Union
+from typing import Any, Union

 from core.app.entities.app_invoke_entities import InvokeFrom
 from core.app.entities.task_entities import AppBlockingResponse, AppStreamResponse
@@ -15,44 +15,41 @@ class AppGenerateResponseConverter(ABC):
    @classmethod
    def convert(cls, response: Union[
        AppBlockingResponse,
-        Generator[AppStreamResponse, None, None]
-    ], invoke_from: InvokeFrom) -> Union[
-        dict,
-        Generator[str, None, None]
-    ]:
+        Generator[AppStreamResponse, Any, None]
+    ], invoke_from: InvokeFrom):
        if invoke_from in [InvokeFrom.DEBUGGER, InvokeFrom.SERVICE_API]:
-            if isinstance(response, cls._blocking_response_type):
+            if isinstance(response, AppBlockingResponse):
                return cls.convert_blocking_full_response(response)
            else:
-                def _generate():
+                def _generate_full_response() -> Generator[str, Any, None]:
                    for chunk in cls.convert_stream_full_response(response):
                        if chunk == 'ping':
                            yield f'event: {chunk}\n\n'
                        else:
                            yield f'data: {chunk}\n\n'

-                return _generate()
+                return _generate_full_response()
        else:
-            if isinstance(response, cls._blocking_response_type):
+            if isinstance(response, AppBlockingResponse):
                return cls.convert_blocking_simple_response(response)
            else:
-                def _generate():
+                def _generate_simple_response() -> Generator[str, Any, None]:
                    for chunk in cls.convert_stream_simple_response(response):
                        if chunk == 'ping':
                            yield f'event: {chunk}\n\n'
                        else:
                            yield f'data: {chunk}\n\n'

-                return _generate()
+                return _generate_simple_response()

    @classmethod
    @abstractmethod
-    def convert_blocking_full_response(cls, blocking_response: AppBlockingResponse) -> dict:
+    def convert_blocking_full_response(cls, blocking_response: AppBlockingResponse) -> dict[str, Any]:
        raise NotImplementedError

    @classmethod
    @abstractmethod
-    def convert_blocking_simple_response(cls, blocking_response: AppBlockingResponse) -> dict:
+    def convert_blocking_simple_response(cls, blocking_response: AppBlockingResponse) -> dict[str, Any]:
        raise NotImplementedError

    @classmethod
@@ -68,7 +65,7 @@ class AppGenerateResponseConverter(ABC):
        raise NotImplementedError

    @classmethod
-    def _get_simple_metadata(cls, metadata: dict) -> dict:
+    def _get_simple_metadata(cls, metadata: dict[str, Any]):
        """
        Get simple metadata.
        :param metadata: metadata
@@ -1,3 +1,4 @@
+import contextvars
 import logging
 import os
 import threading
@@ -8,6 +9,7 @@ from typing import Union
 from flask import Flask, current_app
 from pydantic import ValidationError

+import contexts
 from core.app.app_config.features.file_upload.manager import FileUploadConfigManager
 from core.app.apps.base_app_generator import BaseAppGenerator
 from core.app.apps.base_app_queue_manager import AppQueueManager, GenerateTaskStoppedException, PublishFrom
@@ -38,7 +40,7 @@ class WorkflowAppGenerator(BaseAppGenerator):
        invoke_from: InvokeFrom,
        stream: bool = True,
        call_depth: int = 0,
-    ) -> Union[dict, Generator[dict, None, None]]:
+    ):
        """
        Generate App response.

@@ -86,6 +88,7 @@ class WorkflowAppGenerator(BaseAppGenerator):
            call_depth=call_depth,
            trace_manager=trace_manager
        )
+        contexts.tenant_id.set(application_generate_entity.app_config.tenant_id)

        return self._generate(
            app_model=app_model,
@@ -126,7 +129,8 @@ class WorkflowAppGenerator(BaseAppGenerator):
        worker_thread = threading.Thread(target=self._generate_worker, kwargs={
            'flask_app': current_app._get_current_object(),
            'application_generate_entity': application_generate_entity,
-            'queue_manager': queue_manager
+            'queue_manager': queue_manager,
+            'context': contextvars.copy_context()
        })

        worker_thread.start()
@@ -150,8 +154,7 @@ class WorkflowAppGenerator(BaseAppGenerator):
                                  node_id: str,
                                  user: Account,
                                  args: dict,
-                                  stream: bool = True) \
-            -> Union[dict, Generator[dict, None, None]]:
+                                  stream: bool = True):
        """
        Generate App response.

@@ -193,6 +196,7 @@ class WorkflowAppGenerator(BaseAppGenerator):
                inputs=args['inputs']
            )
        )
+        contexts.tenant_id.set(application_generate_entity.app_config.tenant_id)

        return self._generate(
            app_model=app_model,
@@ -205,7 +209,8 @@ class WorkflowAppGenerator(BaseAppGenerator):

    def _generate_worker(self, flask_app: Flask,
                         application_generate_entity: WorkflowAppGenerateEntity,
-                         queue_manager: AppQueueManager) -> None:
+                         queue_manager: AppQueueManager,
+                         context: contextvars.Context) -> None:
        """
        Generate worker in a new thread.
        :param flask_app: Flask app
@@ -213,6 +218,8 @@ class WorkflowAppGenerator(BaseAppGenerator):
        :param queue_manager: queue manager
        :return:
        """
+        for var, val in context.items():
+            var.set(val)
        with flask_app.app_context():
            try:
                # workflow app
@@ -10,6 +10,7 @@ from core.app.entities.app_invoke_entities import (
    InvokeFrom,
    WorkflowAppGenerateEntity,
 )
+from core.workflow.callbacks.base_workflow_callback import WorkflowCallback
 from core.workflow.entities.node_entities import SystemVariable
 from core.workflow.nodes.base_node import UserFrom
 from core.workflow.workflow_engine_manager import WorkflowEngineManager
@@ -57,7 +58,7 @@ class WorkflowAppRunner:

        db.session.close()

-        workflow_callbacks = [WorkflowEventTriggerCallback(
+        workflow_callbacks: list[WorkflowCallback] = [WorkflowEventTriggerCallback(
            queue_manager=queue_manager,
            workflow=workflow
        )]
@@ -14,13 +14,13 @@ from core.app.entities.queue_entities import (
    QueueWorkflowStartedEvent,
    QueueWorkflowSucceededEvent,
 )
-from core.workflow.callbacks.base_workflow_callback import BaseWorkflowCallback
+from core.workflow.callbacks.base_workflow_callback import WorkflowCallback
 from core.workflow.entities.base_node_data_entities import BaseNodeData
 from core.workflow.entities.node_entities import NodeType
 from models.workflow import Workflow


-class WorkflowEventTriggerCallback(BaseWorkflowCallback):
+class WorkflowEventTriggerCallback(WorkflowCallback):

    def __init__(self, queue_manager: AppQueueManager, workflow: Workflow):
        self._queue_manager = queue_manager
@@ -2,7 +2,7 @@ from typing import Optional

 from core.app.entities.queue_entities import AppQueueEvent
 from core.model_runtime.utils.encoders import jsonable_encoder
-from core.workflow.callbacks.base_workflow_callback import BaseWorkflowCallback
+from core.workflow.callbacks.base_workflow_callback import WorkflowCallback
 from core.workflow.entities.base_node_data_entities import BaseNodeData
 from core.workflow.entities.node_entities import NodeType

@@ -15,7 +15,7 @@ _TEXT_COLOR_MAPPING = {
 }


-class WorkflowLoggingCallback(BaseWorkflowCallback):
+class WorkflowLoggingCallback(WorkflowCallback):

    def __init__(self) -> None:
        self.current_node_id = None
@@ -1,3 +1,4 @@
+from collections.abc import Mapping
 from enum import Enum
 from typing import Any, Optional

@@ -76,7 +77,7 @@ class AppGenerateEntity(BaseModel):
    # app config
    app_config: AppConfig

-    inputs: dict[str, Any]
+    inputs: Mapping[str, Any]
    files: list[FileVar] = []
    user_id: str

@@ -140,7 +141,7 @@ class AdvancedChatAppGenerateEntity(AppGenerateEntity):
    app_config: WorkflowUIBasedAppConfig

    conversation_id: Optional[str] = None
-    query: Optional[str] = None
+    query: str

    class SingleIterationRunEntity(BaseModel):
        """
@@ -0,0 +1,30 @@
+from .segment_group import SegmentGroup
+from .segments import NoneSegment, Segment
+from .types import SegmentType
+from .variables import (
+    ArrayVariable,
+    FileVariable,
+    FloatVariable,
+    IntegerVariable,
+    NoneVariable,
+    ObjectVariable,
+    SecretVariable,
+    StringVariable,
+    Variable,
+)
+
+__all__ = [
+    'IntegerVariable',
+    'FloatVariable',
+    'ObjectVariable',
+    'SecretVariable',
+    'FileVariable',
+    'StringVariable',
+    'ArrayVariable',
+    'Variable',
+    'SegmentType',
+    'SegmentGroup',
+    'Segment',
+    'NoneSegment',
+    'NoneVariable',
+]
@@ -0,0 +1,67 @@
+from collections.abc import Mapping
+from typing import Any
+
+from core.file.file_obj import FileVar
+
+from .segments import Segment, StringSegment
+from .types import SegmentType
+from .variables import (
+    ArrayVariable,
+    FileVariable,
+    FloatVariable,
+    IntegerVariable,
+    NoneVariable,
+    ObjectVariable,
+    SecretVariable,
+    StringVariable,
+    Variable,
+)
+
+
+def build_variable_from_mapping(m: Mapping[str, Any], /) -> Variable:
+    if (value_type := m.get('value_type')) is None:
+        raise ValueError('missing value type')
+    if not m.get('name'):
+        raise ValueError('missing name')
+    if (value := m.get('value')) is None:
+        raise ValueError('missing value')
+    match value_type:
+        case SegmentType.STRING:
+            return StringVariable.model_validate(m)
+        case SegmentType.NUMBER if isinstance(value, int):
+            return IntegerVariable.model_validate(m)
+        case SegmentType.NUMBER if isinstance(value, float):
+            return FloatVariable.model_validate(m)
+        case SegmentType.SECRET:
+            return SecretVariable.model_validate(m)
+        case SegmentType.NUMBER if not isinstance(value, float | int):
+            raise ValueError(f'invalid number value {value}')
+    raise ValueError(f'not supported value type {value_type}')
+
+
+def build_anonymous_variable(value: Any, /) -> Variable:
+    if value is None:
+        return NoneVariable(name='anonymous')
+    if isinstance(value, str):
+        return StringVariable(name='anonymous', value=value)
+    if isinstance(value, int):
+        return IntegerVariable(name='anonymous', value=value)
+    if isinstance(value, float):
+        return FloatVariable(name='anonymous', value=value)
+    if isinstance(value, dict):
+        # TODO: Limit the depth of the object
+        obj = {k: build_anonymous_variable(v) for k, v in value.items()}
+        return ObjectVariable(name='anonymous', value=obj)
+    if isinstance(value, list):
+        # TODO: Limit the depth of the array
+        elements = [build_anonymous_variable(v) for v in value]
+        return ArrayVariable(name='anonymous', value=elements)
+    if isinstance(value, FileVar):
+        return FileVariable(name='anonymous', value=value)
+    raise ValueError(f'not supported value {value}')
+
+
+def build_segment(value: Any, /) -> Segment:
+    if isinstance(value, str):
+        return StringSegment(value=value)
+    raise ValueError(f'not supported value {value}')
@@ -0,0 +1,17 @@
+import re
+
+from core.app.segments import SegmentGroup, factory
+from core.workflow.entities.variable_pool import VariablePool
+
+VARIABLE_PATTERN = re.compile(r'\{\{#([a-zA-Z0-9_]{1,50}(?:\.[a-zA-Z_][a-zA-Z0-9_]{0,29}){1,10})#\}\}')
+
+
+def convert_template(*, template: str, variable_pool: VariablePool):
+    parts = re.split(VARIABLE_PATTERN, template)
+    segments = []
+    for part in parts:
+        if '.' in part and (value := variable_pool.get(part.split('.'))):
+            segments.append(value)
+        else:
+            segments.append(factory.build_segment(part))
+    return SegmentGroup(segments=segments)
@@ -0,0 +1,19 @@
+from pydantic import BaseModel
+
+from .segments import Segment
+
+
+class SegmentGroup(BaseModel):
+    segments: list[Segment]
+
+    @property
+    def text(self):
+        return ''.join([segment.text for segment in self.segments])
+
+    @property
+    def log(self):
+        return ''.join([segment.log for segment in self.segments])
+
+    @property
+    def markdown(self):
+        return ''.join([segment.markdown for segment in self.segments])
@@ -0,0 +1,65 @@
+from typing import Any
+
+from pydantic import BaseModel, ConfigDict, field_validator
+
+from .types import SegmentType
+
+
+class Segment(BaseModel):
+    model_config = ConfigDict(frozen=True)
+
+    value_type: SegmentType
+    value: Any
+
+    @field_validator('value_type')
+    def validate_value_type(cls, value):
+        """
+        This validator checks if the provided value is equal to the default value of the 'value_type' field.
+        If the value is different, a ValueError is raised.
+        """
+        if value != cls.model_fields['value_type'].default:
+            raise ValueError("Cannot modify 'value_type'")
+        return value
+
+    @property
+    def text(self) -> str:
+        return str(self.value)
+
+    @property
+    def log(self) -> str:
+        return str(self.value)
+
+    @property
+    def markdown(self) -> str:
+        return str(self.value)
+
+    def to_object(self) -> Any:
+        if isinstance(self.value, Segment):
+            return self.value.to_object()
+        if isinstance(self.value, list):
+            return [v.to_object() for v in self.value]
+        if isinstance(self.value, dict):
+            return {k: v.to_object() for k, v in self.value.items()}
+        return self.value
+
+
+class NoneSegment(Segment):
+    value_type: SegmentType = SegmentType.NONE
+    value: None = None
+
+    @property
+    def text(self) -> str:
+        return 'null'
+
+    @property
+    def log(self) -> str:
+        return 'null'
+
+    @property
+    def markdown(self) -> str:
+        return 'null'
+
+
+class StringSegment(Segment):
+    value_type: SegmentType = SegmentType.STRING
+    value: str
@@ -0,0 +1,11 @@
+from enum import Enum
+
+
+class SegmentType(str, Enum):
+    NONE = 'none'
+    NUMBER = 'number'
+    STRING = 'string'
+    SECRET = 'secret'
+    ARRAY = 'array'
+    OBJECT = 'object'
+    FILE = 'file'
@@ -0,0 +1,88 @@
+import json
+from collections.abc import Mapping, Sequence
+
+from pydantic import Field
+
+from core.file.file_obj import FileVar
+from core.helper import encrypter
+
+from .segments import NoneSegment, Segment, StringSegment
+from .types import SegmentType
+
+
+class Variable(Segment):
+    """
+    A variable is a segment that has a name.
+    """
+
+    id: str = Field(
+        default='',
+        description="Unique identity for variable. It's only used by environment variables now.",
+    )
+    name: str
+
+
+class StringVariable(StringSegment, Variable):
+    pass
+
+
+class FloatVariable(Variable):
+    value_type: SegmentType = SegmentType.NUMBER
+    value: float
+
+
+class IntegerVariable(Variable):
+    value_type: SegmentType = SegmentType.NUMBER
+    value: int
+
+
+class ObjectVariable(Variable):
+    value_type: SegmentType = SegmentType.OBJECT
+    value: Mapping[str, Variable]
+
+    @property
+    def text(self) -> str:
+        # TODO: Process variables.
+        return json.dumps(self.model_dump()['value'], ensure_ascii=False)
+
+    @property
+    def log(self) -> str:
+        # TODO: Process variables.
+        return json.dumps(self.model_dump()['value'], ensure_ascii=False, indent=2)
+
+    @property
+    def markdown(self) -> str:
+        # TODO: Use markdown code block
+        return json.dumps(self.model_dump()['value'], ensure_ascii=False, indent=2)
+
+
+class ArrayVariable(Variable):
+    value_type: SegmentType = SegmentType.ARRAY
+    value: Sequence[Variable]
+
+    @property
+    def markdown(self) -> str:
+        return '\n'.join(['- ' + item.markdown for item in self.value])
+
+
+class FileVariable(Variable):
+    value_type: SegmentType = SegmentType.FILE
+    # TODO: embed FileVar in this model.
+    value: FileVar
+
+    @property
+    def markdown(self) -> str:
+        return self.value.to_markdown()
+
+
+class SecretVariable(StringVariable):
+    value_type: SegmentType = SegmentType.SECRET
+
+    @property
+    def log(self) -> str:
+        return encrypter.obfuscated_token(self.value)
+
+
+class NoneVariable(NoneSegment, Variable):
+    value_type: SegmentType = SegmentType.NONE
+    value: None = None
@@ -1,9 +1,11 @@
 import os
+from collections.abc import Mapping, Sequence
 from typing import Any, Optional, TextIO, Union

 from pydantic import BaseModel

 from core.ops.ops_trace_manager import TraceQueueManager, TraceTask, TraceTaskName
+from core.tools.entities.tool_entities import ToolInvokeMessage

 _TEXT_COLOR_MAPPING = {
    "blue": "36;1",
@@ -43,7 +45,7 @@ class DifyAgentCallbackHandler(BaseModel):
    def on_tool_start(
        self,
        tool_name: str,
-        tool_inputs: dict[str, Any],
+        tool_inputs: Mapping[str, Any],
    ) -> None:
        """Do nothing."""
        print_text("\n[on_tool_start] ToolCall:" + tool_name + "\n" + str(tool_inputs) + "\n", color=self.color)
@@ -51,8 +53,8 @@ class DifyAgentCallbackHandler(BaseModel):
    def on_tool_end(
        self,
        tool_name: str,
-        tool_inputs: dict[str, Any],
-        tool_outputs: str,
+        tool_inputs: Mapping[str, Any],
+        tool_outputs: Sequence[ToolInvokeMessage],
        message_id: Optional[str] = None,
        timer: Optional[Any] = None,
        trace_manager: Optional[TraceQueueManager] = None
@@ -1,4 +1,5 @@
-from typing import Union
+from collections.abc import Mapping, Sequence
+from typing import Any, Union

 import requests

@@ -16,7 +17,7 @@ class MessageFileParser:
        self.tenant_id = tenant_id
        self.app_id = app_id

-    def validate_and_transform_files_arg(self, files: list[dict], file_extra_config: FileExtraConfig,
+    def validate_and_transform_files_arg(self, files: Sequence[Mapping[str, Any]], file_extra_config: FileExtraConfig,
                                         user: Union[Account, EndUser]) -> list[FileVar]:
        """
        validate and transform files arg
@@ -21,7 +21,7 @@ logger = logging.getLogger(__name__)
 CODE_EXECUTION_ENDPOINT = dify_config.CODE_EXECUTION_ENDPOINT
 CODE_EXECUTION_API_KEY = dify_config.CODE_EXECUTION_API_KEY

-CODE_EXECUTION_TIMEOUT= (10, 60)
+CODE_EXECUTION_TIMEOUT = (10, 60)

 class CodeExecutionException(Exception):
    pass
@@ -64,7 +64,7 @@ class CodeExecutor:

    @classmethod
    def execute_code(cls, 
-                     language: Literal['python3', 'javascript', 'jinja2'], 
+                     language: CodeLanguage, 
                     preload: str, 
                     code: str, 
                     dependencies: Optional[list[CodeDependency]] = None) -> str:
@@ -119,7 +119,7 @@ class CodeExecutor:
        return response.data.stdout

    @classmethod
-    def execute_workflow_code_template(cls, language: Literal['python3', 'javascript', 'jinja2'], code: str, inputs: dict, dependencies: Optional[list[CodeDependency]] = None) -> dict:
+    def execute_workflow_code_template(cls, language: CodeLanguage, code: str, inputs: dict, dependencies: Optional[list[CodeDependency]] = None) -> dict:
        """
        Execute code
        :param language: code language
@@ -6,11 +6,16 @@ from models.account import Tenant


 def obfuscated_token(token: str):
-    return token[:6] + '*' * (len(token) - 8) + token[-2:]
+    if not token:
+        return token
+    if len(token) <= 8:
+        return '*' * 20
+    return token[:6] + '*' * 12 + token[-2:]


 def encrypt_token(tenant_id: str, token: str):
-    tenant = db.session.query(Tenant).filter(Tenant.id == tenant_id).first()
+    if not (tenant := db.session.query(Tenant).filter(Tenant.id == tenant_id).first()):
+        raise ValueError(f'Tenant with id {tenant_id} not found')
    encrypted_token = rsa.encrypt(token, tenant.encrypt_public_key)
    return base64.b64encode(encrypted_token).decode()

@@ -14,6 +14,9 @@ def get_position_map(folder_path: str, *, file_name: str = "_position.yaml") ->
    :return: a dict with name as key and index as value
    """
    position_file_name = os.path.join(folder_path, file_name)
+    if not position_file_name or not os.path.exists(position_file_name):
+        return {}
+    
    positions = load_yaml_file(position_file_name, ignore_error=True)
    position_map = {}
    index = 0
@@ -3,10 +3,13 @@ import logging
 import re
 from typing import Optional

-from core.llm_generator.output_parser.errors import OutputParserException
 from core.llm_generator.output_parser.rule_config_generator import RuleConfigGeneratorOutputParser
 from core.llm_generator.output_parser.suggested_questions_after_answer import SuggestedQuestionsAfterAnswerOutputParser
-from core.llm_generator.prompts import CONVERSATION_TITLE_PROMPT, GENERATOR_QA_PROMPT
+from core.llm_generator.prompts import (
+    CONVERSATION_TITLE_PROMPT,
+    GENERATOR_QA_PROMPT,
+    WORKFLOW_RULE_CONFIG_PROMPT_GENERATE_TEMPLATE,
+)
 from core.model_manager import ModelManager
 from core.model_runtime.entities.message_entities import SystemPromptMessage, UserPromptMessage
 from core.model_runtime.entities.model_entities import ModelType
@@ -115,55 +118,158 @@ class LLMGenerator:
        return questions

    @classmethod
-    def generate_rule_config(cls, tenant_id: str, audiences: str, hoping_to_solve: str) -> dict:
+    def generate_rule_config(cls, tenant_id: str, instruction: str, model_config: dict, no_variable: bool) -> dict:
        output_parser = RuleConfigGeneratorOutputParser()

+        error = ""
+        error_step = ""
+        rule_config = {
+            "prompt": "",
+            "variables": [],
+            "opening_statement": "",
+            "error": ""
+        }
+        model_parameters = {
+            "max_tokens": 512,
+            "temperature": 0.01
+        }
+
+        if no_variable:
+            prompt_template = PromptTemplateParser(
+                WORKFLOW_RULE_CONFIG_PROMPT_GENERATE_TEMPLATE
+            )
+
+            prompt_generate = prompt_template.format(
+                inputs={
+                    "TASK_DESCRIPTION": instruction,
+                },
+                remove_template_variables=False
+            )
+
+            prompt_messages = [UserPromptMessage(content=prompt_generate)]
+
+            model_manager = ModelManager()
+
+            model_instance = model_manager.get_default_model_instance(
+                tenant_id=tenant_id,
+                model_type=ModelType.LLM,
+            )
+
+            try:
+                response = model_instance.invoke_llm(
+                    prompt_messages=prompt_messages,
+                    model_parameters=model_parameters,
+                    stream=False
+                )
+
+                rule_config["prompt"] = response.message.content
+                
+            except InvokeError as e:
+                error = str(e)
+                error_step = "generate rule config"
+            except Exception as e:
+                logging.exception(e)
+                rule_config["error"] = str(e)
+
+            rule_config["error"] = f"Failed to {error_step}. Error: {error}" if error else ""
+
+            return rule_config
+
+        # get rule config prompt, parameter and statement
+        prompt_generate, parameter_generate, statement_generate = output_parser.get_format_instructions()
+
        prompt_template = PromptTemplateParser(
-            template=output_parser.get_format_instructions()
+            prompt_generate
        )

-        prompt = prompt_template.format(
+        parameter_template = PromptTemplateParser(
+            parameter_generate
+        )
+
+        statement_template = PromptTemplateParser(
+            statement_generate
+        )
+
+        # format the prompt_generate_prompt
+        prompt_generate_prompt = prompt_template.format(
            inputs={
-                "audiences": audiences,
-                "hoping_to_solve": hoping_to_solve,
-                "variable": "{{variable}}",
-                "lanA": "{{lanA}}",
-                "lanB": "{{lanB}}",
-                "topic": "{{topic}}"
+                "TASK_DESCRIPTION": instruction,
            },
            remove_template_variables=False
        )
+        prompt_messages = [UserPromptMessage(content=prompt_generate_prompt)]

+        # get model instance
        model_manager = ModelManager()
-        model_instance = model_manager.get_default_model_instance(
+        model_instance = model_manager.get_model_instance(
            tenant_id=tenant_id,
            model_type=ModelType.LLM,
+            provider=model_config.get("provider") if model_config else None,
+            model=model_config.get("name") if model_config else None,
        )

-        prompt_messages = [UserPromptMessage(content=prompt)]
-
        try:
-            response = model_instance.invoke_llm(
-                prompt_messages=prompt_messages,
-                model_parameters={
-                    "max_tokens": 512,
-                    "temperature": 0
+            try:
+                # the first step to generate the task prompt
+                prompt_content = model_instance.invoke_llm(
+                    prompt_messages=prompt_messages,
+                    model_parameters=model_parameters,
+                    stream=False
+                )
+            except InvokeError as e:
+                error = str(e)
+                error_step = "generate prefix prompt"
+                rule_config["error"] = f"Failed to {error_step}. Error: {error}" if error else ""
+
+                return rule_config
+
+            rule_config["prompt"] = prompt_content.message.content
+
+            parameter_generate_prompt = parameter_template.format(
+                inputs={
+                    "INPUT_TEXT": prompt_content.message.content,
                },
-                stream=False
+                remove_template_variables=False
            )
+            parameter_messages = [UserPromptMessage(content=parameter_generate_prompt)]
+
+            # the second step to generate the task_parameter and task_statement
+            statement_generate_prompt = statement_template.format(
+                inputs={
+                    "TASK_DESCRIPTION": instruction,
+                    "INPUT_TEXT": prompt_content.message.content,
+                },
+                remove_template_variables=False
+            )
+            statement_messages = [UserPromptMessage(content=statement_generate_prompt)]
+
+            try:
+                parameter_content = model_instance.invoke_llm(
+                    prompt_messages=parameter_messages,
+                    model_parameters=model_parameters,
+                    stream=False
+                )
+                rule_config["variables"] = re.findall(r'"\s*([^"]+)\s*"', parameter_content.message.content)
+            except InvokeError as e:
+                error = str(e)
+                error_step = "generate variables"
+
+            try:
+                statement_content = model_instance.invoke_llm(
+                    prompt_messages=statement_messages,
+                    model_parameters=model_parameters,
+                    stream=False
+                )
+                rule_config["opening_statement"] = statement_content.message.content
+            except InvokeError as e:
+                error = str(e)
+                error_step = "generate conversation opener"

-            rule_config = output_parser.parse(response.message.content)
-        except InvokeError as e:
-            raise e
-        except OutputParserException:
-            raise ValueError('Please give a valid input for intended audience or hoping to solve problems.')
        except Exception as e:
            logging.exception(e)
-            rule_config = {
-                "prompt": "",
-                "variables": [],
-                "opening_statement": ""
-            }
+            rule_config["error"] = str(e)
+
+        rule_config["error"] = f"Failed to {error_step}. Error: {error}" if error else ""

        return rule_config

@@ -1,14 +1,18 @@
 from typing import Any

 from core.llm_generator.output_parser.errors import OutputParserException
-from core.llm_generator.prompts import RULE_CONFIG_GENERATE_TEMPLATE
+from core.llm_generator.prompts import (
+    RULE_CONFIG_PARAMETER_GENERATE_TEMPLATE,
+    RULE_CONFIG_PROMPT_GENERATE_TEMPLATE,
+    RULE_CONFIG_STATEMENT_GENERATE_TEMPLATE,
+)
 from libs.json_in_md_parser import parse_and_check_json_markdown


 class RuleConfigGeneratorOutputParser:

-    def get_format_instructions(self) -> str:
-        return RULE_CONFIG_GENERATE_TEMPLATE
+    def get_format_instructions(self) -> tuple[str, str, str]:
+        return RULE_CONFIG_PROMPT_GENERATE_TEMPLATE, RULE_CONFIG_PARAMETER_GENERATE_TEMPLATE, RULE_CONFIG_STATEMENT_GENERATE_TEMPLATE

    def parse(self, text: str) -> Any:
        try:
@@ -81,65 +81,73 @@ GENERATOR_QA_PROMPT = (
    '<QA Pairs>'
 )

-RULE_CONFIG_GENERATE_TEMPLATE = """Given MY INTENDED AUDIENCES and HOPING TO SOLVE using a language model, please select \
-the model prompt that best suits the input. 
-You will be provided with the prompt, variables, and an opening statement. 
-Only the content enclosed in double curly braces, such as {{variable}}, in the prompt can be considered as a variable; \
-otherwise, it cannot exist as a variable in the variables.
-If you believe revising the original input will result in a better response from the language model, you may \
-suggest revisions.
+WORKFLOW_RULE_CONFIG_PROMPT_GENERATE_TEMPLATE = """
+Here is a task description for which I would like you to create a high-quality prompt template for:
+<task_description>
+{{TASK_DESCRIPTION}}
+</task_description>
+Based on task description, please create a well-structured prompt template that another AI could use to consistently complete the task. The prompt template should include:
+- Do not inlcude <input> or <output> section and variables in the prompt, assume user will add them at their own will. 
+- Clear instructions for the AI that will be using this prompt, demarcated with <instructions> tags. The instructions should provide step-by-step directions on how to complete the task using the input variables. Also Specifies in the instructions that the output should not contain any xml tag. 
+- Relevant examples if needed to clarify the task further, demarcated with <example> tags. Do not include variables in the prompt. Give three pairs of input and output examples.   
+- Include other relevant sections demarcated with appropriate XML tags like <examples>, <instructions>.
+- Use the same language as task description. 
+- Output in ``` xml ``` and start with <instruction>
+Please generate the full prompt template with at least 300 words and output only the prompt template.
+"""

-<<PRINCIPLES OF GOOD PROMPT>>
-Integrate the intended audience in the prompt e.g. the audience is an expert in the field.
-Break down complex tasks into a sequence of simpler prompts in an interactive conversation.
-Implement example-driven prompting (Use few-shot prompting). 
-When formatting your prompt start with Instruction followed by either Example if relevant. \
-Subsequently present your content. Use one or more line breaks to separate instructions examples questions context and input data.
-Incorporate the following phrases: “Your task is” and “You MUST”.
-Incorporate the following phrases: “You will be penalized”.
-Use leading words like writing “think step by step”.
-Add to your prompt the following phrase “Ensure that your answer is unbiased and does not rely on stereotypes”.
-Assign a role to the large language models.
-Use Delimiters.
-To write an essay /text /paragraph /article or any type of text that should be detailed: “Write a detailed [essay/text/paragraph] for me on [topic] in detail by adding all the information necessary”.
-Clearly state the requirements that the model must follow in order to produce content in the form of the keywords regulations hint or instructions
+RULE_CONFIG_PROMPT_GENERATE_TEMPLATE = """
+Here is a task description for which I would like you to create a high-quality prompt template for:
+<task_description>
+{{TASK_DESCRIPTION}}
+</task_description>
+Based on task description, please create a well-structured prompt template that another AI could use to consistently complete the task. The prompt template should include:
+- Descriptive variable names surrounded by {{ }} (two curly brackets) to indicate where the actual values will be substituted in. Choose variable names that clearly indicate the type of value expected. Variable names have to be composed of number, english alphabets and underline and nothing else. 
+- Clear instructions for the AI that will be using this prompt, demarcated with <instructions> tags. The instructions should provide step-by-step directions on how to complete the task using the input variables. Also Specifies in the instructions that the output should not contain any xml tag. 
+- Relevant examples if needed to clarify the task further, demarcated with <example> tags. Do not use curly brackets any other than in <instruction> section. 
+- Any other relevant sections demarcated with appropriate XML tags like <input>, <output>, etc.
+- Use the same language as task description. 
+- Output in ``` xml ``` and start with <instruction>
+Please generate the full prompt template and output only the prompt template.
+"""

-<< FORMATTING >>
-Return a markdown code snippet with a JSON object formatted to look like, \
-no any other string out of markdown code snippet:
-```json
-{{{{
-    "prompt": string \\ generated prompt
-    "variables": list of string \\ variables
-    "opening_statement": string \\ an opening statement to guide users on how to ask questions with generated prompt \
-and fill in variables, with a welcome sentence, and keep TLDR.
-}}}}
-```
+RULE_CONFIG_PARAMETER_GENERATE_TEMPLATE = """
+I need to extract the following information from the input text. The <information to be extracted> tag specifies the 'type', 'description' and 'required' of the information to be extracted. 
+<information to be extracted>
+variables name bounded two double curly brackets. Variable name has to be composed of number, english alphabets and underline and nothing else. 
+</information to be extracted>

-<< EXAMPLES >>
-[EXAMPLE A]
-```json
-{
-  "prompt": "I need your help to translate the following {{Input_language}}paper paragraph into {{Target_language}}, in a style similar to a popular science magazine in {{Target_language}}. #### Rules Ensure accurate conveyance of the original text's facts and context during translation. Maintain the original paragraph format and retain technical terms and company abbreviations ",
-  "variables": ["Input_language", "Target_language"],
-  "opening_statement": " Hi. I am your translation assistant. I can help you with any translation and ensure accurate conveyance of information. "
-}
-```
+Step 1: Carefully read the input and understand the structure of the expected output.
+Step 2: Extract relevant parameters from the provided text based on the name and description of object. 
+Step 3: Structure the extracted parameters to JSON object as specified in <structure>.
+Step 4: Ensure that the list of variable_names is properly formatted and valid. The output should not contain any XML tags. Output an empty list if there is no valid variable name in input text. 

-[EXAMPLE B]
-```json
-{
-  "prompt": "Your task is to review the provided meeting notes and create a concise summary that captures the essential information, focusing on key takeaways and action items assigned to specific individuals or departments during the meeting. Use clear and professional language, and organize the summary in a logical manner using appropriate formatting such as headings, subheadings, and bullet points. Ensure that the summary is easy to understand and provides a comprehensive but succinct overview of the meeting's content, with a particular focus on clearly indicating who is responsible for each action item.",
-  "variables": ["meeting_notes"],
-  "opening_statement": "Hi! I'm your meeting notes summarizer AI. I can help you with any meeting notes and ensure accurate conveyance of information."
-}
-```
+### Structure
+Here is the structure of the expected output, I should always follow the output structure. 
+["variable_name_1", "variable_name_2"]

-<< MY INTENDED AUDIENCES >>
-{{audiences}}
+### Input Text
+Inside <text></text> XML tags, there is a text that I should extract parameters and convert to a JSON object.
+<text>
+{{INPUT_TEXT}}
+</text>

-<< HOPING TO SOLVE >>
-{{hoping_to_solve}}
+### Answer
+I should always output a valid list. Output nothing other than the list of variable_name. Output an empty list if there is no variable name in input text.
+"""

-<< OUTPUT >>
-"""
+RULE_CONFIG_STATEMENT_GENERATE_TEMPLATE = """
+<instruction>
+Step 1: Identify the purpose of the chatbot from the variable {{TASK_DESCRIPTION}} and infer chatbot's tone  (e.g., friendly, professional, etc.) to add personality traits. 
+Step 2: Create a coherent and engaging opening statement.
+Step 3: Ensure the output is welcoming and clearly explains what the chatbot is designed to do. Do not include any XML tags in the output.
+Please use the same language as the user's input language. If user uses chinese then generate opening statement in chinese,  if user uses english then generate opening statement in english. 
+Example Input: 
+Provide customer support for an e-commerce website
+Example Output: 
+Welcome! I'm here to assist you with any questions or issues you might have with your shopping experience. Whether you're looking for product information, need help with your order, or have any other inquiries, feel free to ask. I'm friendly, helpful, and ready to support you in any way I can.
+<Task>
+Here is the task description: {{INPUT_TEXT}}
+
+You just need to generate the output
+"""
@@ -410,7 +410,7 @@ class LBModelManager:
        self._model = model
        self._load_balancing_configs = load_balancing_configs

-        for load_balancing_config in self._load_balancing_configs:
+        for load_balancing_config in self._load_balancing_configs[:]:  # Iterate over a shallow copy of the list
            if load_balancing_config.name == "__inherit__":
                if not managed_credentials:
                    # remove __inherit__ if managed credentials is not provided
@@ -23,6 +23,7 @@
 - tongyi
 - wenxin
 - moonshot
+- tencent
 - jina
 - chatglm
 - yi
@@ -501,7 +501,7 @@ class AzureOpenAILargeLanguageModel(_CommonAzureOpenAI, LargeLanguageModel):
                        sub_messages.append(sub_message_dict)
                message_dict = {"role": "user", "content": sub_messages}
        elif isinstance(message, AssistantPromptMessage):
-            message = cast(AssistantPromptMessage, message)
+            # message = cast(AssistantPromptMessage, message)
            message_dict = {"role": "assistant", "content": message.content}
            if message.tool_calls:
                message_dict["tool_calls"] = [helper.dump_model(tool_call) for tool_call in message.tool_calls]
@@ -0,0 +1,13 @@
+<svg viewBox="0 83.15545000000002 402.45098039215685 76.44" data-name="图层 1" id="图层_1"
+  xmlns="http://www.w3.org/2000/svg" style="max-height: 500px" width="402.45098039215685"
+  height="76.44">
+  <defs>
+    <style>.cls-1{fill:#4999d4}</style>
+  </defs>
+  <title>tencent-cloud</title>
+  <path
+    d="M27.569 113.353a17.56 17.56 0 0 1 33.148-3.743.158.158 0 0 0 .194.105 21.267 21.267 0 0 1 7.008-.729c.235.018.327-.116.25-.33a24.828 24.828 0 0 0-47.933 4.444.082.082 0 0 0 .016 0 18.537 18.537 0 0 0-9.85 31.533 18.007 18.007 0 0 0 10.325 5h-.001a43.066 43.066 0 0 0 5.266.282c1.68.011 33.725.008 35.067.008 2.7 0 4.457-.002 6.345-.14a18.245 18.245 0 0 0 11.723-5.15 18.532 18.532 0 0 0-12.901-31.789 18.06 18.06 0 0 0-11.704 4.285c-1.467 1.196-3.006 2.626-4.944 4.508-.642.625-13.336 12.94-21.67 21.028-1.16-.005-2.828-.021-4.306-.07a11.704 11.704 0 0 1-8.125-3.148A11.275 11.275 0 0 1 23.33 120.1a11.706 11.706 0 0 1 7.646 3.062c1.44 1.192 4.633 4 6.035 5.263a.17.17 0 0 0 .24.002l4.945-4.825a.176.176 0 0 0-.004-.27c-2.378-2.15-5.749-5.158-7.778-6.669a18.874 18.874 0 0 0-6.844-3.31zm46.482 26.094a11.704 11.704 0 0 1-8.125 3.147 168.92 168.92 0 0 1-5.204.073h-22.38c8.142-7.91 15.245-14.808 16.051-15.59.738-.717 2.398-2.306 3.83-3.595 3.145-2.831 5.974-3.4 7.976-3.382a11.275 11.275 0 0 1 7.852 19.347z"
+    class="cls-1" />
+  <path
+    d="M302.794 129.138a.24.24 0 0 0-.419-.163 16.062 16.062 0 0 1-11.961 5.469c-7.7 0-12.674-5.32-12.674-13.552 0-8.388 4.74-13.599 12.37-13.599a17.274 17.274 0 0 1 11.828 4.996.24.24 0 0 0 .414-.168v-4.245a18.595 18.595 0 0 0-12.243-4.502 15.358 15.358 0 0 0-11.733 4.845c-2.857 3.138-4.366 7.52-4.366 12.674 0 10.478 6.592 17.518 16.404 17.518a18.517 18.517 0 0 0 12.38-4.624zM93.47 104.248v3.638h11.506v29.657h3.982v-29.657h11.506v-3.638H93.47zM390.972 115.625c-2.059-2.232-5.501-3.043-7.978-3.043a11.24 11.24 0 0 0-8.363 3.475 13.26 13.26 0 0 0-3.277 9.243c0 8.883 5.894 12.86 11.735 12.86 3.412 0 6.228-1.22 7.883-3.38v2.812h3.971V104.3h-3.97zm-.105 9.769c0 4.545-2.639 9.147-7.683 9.147-3.788 0-7.865-2.803-7.865-8.958 0-4.522 2.388-9.389 7.63-9.389 5.844 0 7.918 4.956 7.918 9.2zM308.064 104.3h4.031v33.292h-4.031zM192.033 131.427a.24.24 0 0 0-.403-.18 13.772 13.772 0 0 1-8.006 3.255c-5.344 0-8.796-3.583-8.796-9.128 0-5.546 3.452-9.129 8.796-9.129a12.973 12.973 0 0 1 8.01 2.8.24.24 0 0 0 .399-.183v-3.668a17.567 17.567 0 0 0-8.621-2.615c-7.38 0-12.34 5.142-12.34 12.795 0 7.652 4.96 12.794 12.34 12.794a14.903 14.903 0 0 0 8.62-2.624zM139.984 130.642a.241.241 0 0 0-.436-.143 12.896 12.896 0 0 1-8.605 4.214 8.725 8.725 0 0 1-6.104-2.141 8.634 8.634 0 0 1-2.372-6.07h18.629v-.285c0-5.254-.896-8.233-3.194-10.625a10.42 10.42 0 0 0-7.849-3.01c-6.969 0-11.472 4.987-11.471 12.715.254 8.04 4.822 12.84 12.218 12.84a13.447 13.447 0 0 0 9.184-3.52zm-2.653-7.564h-14.872l.006-.075a7.37 7.37 0 0 1 7.481-6.93c4.341 0 7.17 2.657 7.38 6.933zM362.877 125.584c0 6.632-3.825 8.985-7.1 8.985a6.287 6.287 0 0 1-6.549-6.263V113.15h-3.79v15.088c0 5.842 4.172 9.922 10.145 9.922 3.308 0 5.89-1.114 7.53-3.235v2.66h3.614V113.15h-3.85zM156.747 112.583a9.79 9.79 0 0 0-7.364 2.887v-2.334h-3.762v24.456h3.846v-13.935a7.592 7.592 0 0 1 1.982-5.51 6.75 6.75 0 0 1 5.197-1.91 5.994 5.994 0 0 1 6.055 6.339v15.016h3.847v-15.197a9.756 9.756 0 0 0-2.767-7.26 9.907 9.907 0 0 0-7.034-2.552zM217.156 130.642a.241.241 0 0 0-.436-.143 12.896 12.896 0 0 1-8.605 4.214 8.725 8.725 0 0 1-6.104-2.141 8.634 8.634 0 0 1-2.372-6.07h18.629v-.285c0-5.254-.895-8.233-3.193-10.625a10.42 10.42 0 0 0-7.85-3.01c-6.968 0-11.471 4.987-11.471 12.715.254 8.04 4.822 12.84 12.218 12.84a13.447 13.447 0 0 0 9.184-3.52zm-2.653-7.564h-14.871l.005-.075a7.37 7.37 0 0 1 7.481-6.93c4.342 0 7.17 2.657 7.381 6.933zM233.857 112.583a9.79 9.79 0 0 0-7.365 2.887v-2.334h-3.762v24.456h3.847v-13.935a7.592 7.592 0 0 1 1.982-5.51 6.75 6.75 0 0 1 5.196-1.91 5.994 5.994 0 0 1 6.056 6.339v15.016h3.846v-15.197a9.756 9.756 0 0 0-2.767-7.26 9.907 9.907 0 0 0-7.033-2.552zM256.236 137.917a19.963 19.963 0 0 0 5.009-1.138v-3.683a.241.241 0 0 0-.321-.229A29.455 29.455 0 0 1 256.8 134c-.402.064-.756.12-1.02-.104a.897.897 0 0 1-.263-.777V116.69h7.04v-3.559h-7.04v-6.516h-3.997v6.516h-4.012v3.558h4.012v16.815a4.207 4.207 0 0 0 1.309 3.327 5.088 5.088 0 0 0 3.406 1.085zM329.224 112.63c-7.093 0-11.859 5.13-11.859 12.764s4.766 12.764 11.859 12.764 11.858-5.13 11.858-12.764-4.765-12.764-11.858-12.764zm-8.18 12.739l-.001-.01c.014-5.41 3.299-9.043 8.18-9.043 4.893 0 8.18 3.648 8.182 9.078-.002 5.429-3.29 9.078-8.181 9.078-4.89 0-8.175-3.678-8.18-9.103z" />
+</svg>
@@ -0,0 +1,13 @@
+<svg viewBox="0 83.15545000000002 402.45098039215685 76.44" data-name="图层 1" id="图层_1"
+  xmlns="http://www.w3.org/2000/svg" style="max-height: 500px" width="402.45098039215685"
+  height="76.44">
+  <defs>
+    <style>.cls-1{fill:#4999d4}</style>
+  </defs>
+  <title>tencent-cloud</title>
+  <path
+    d="M27.569 113.353a17.56 17.56 0 0 1 33.148-3.743.158.158 0 0 0 .194.105 21.267 21.267 0 0 1 7.008-.729c.235.018.327-.116.25-.33a24.828 24.828 0 0 0-47.933 4.444.082.082 0 0 0 .016 0 18.537 18.537 0 0 0-9.85 31.533 18.007 18.007 0 0 0 10.325 5h-.001a43.066 43.066 0 0 0 5.266.282c1.68.011 33.725.008 35.067.008 2.7 0 4.457-.002 6.345-.14a18.245 18.245 0 0 0 11.723-5.15 18.532 18.532 0 0 0-12.901-31.789 18.06 18.06 0 0 0-11.704 4.285c-1.467 1.196-3.006 2.626-4.944 4.508-.642.625-13.336 12.94-21.67 21.028-1.16-.005-2.828-.021-4.306-.07a11.704 11.704 0 0 1-8.125-3.148A11.275 11.275 0 0 1 23.33 120.1a11.706 11.706 0 0 1 7.646 3.062c1.44 1.192 4.633 4 6.035 5.263a.17.17 0 0 0 .24.002l4.945-4.825a.176.176 0 0 0-.004-.27c-2.378-2.15-5.749-5.158-7.778-6.669a18.874 18.874 0 0 0-6.844-3.31zm46.482 26.094a11.704 11.704 0 0 1-8.125 3.147 168.92 168.92 0 0 1-5.204.073h-22.38c8.142-7.91 15.245-14.808 16.051-15.59.738-.717 2.398-2.306 3.83-3.595 3.145-2.831 5.974-3.4 7.976-3.382a11.275 11.275 0 0 1 7.852 19.347z"
+    class="cls-1" />
+  <path
+    d="M302.794 129.138a.24.24 0 0 0-.419-.163 16.062 16.062 0 0 1-11.961 5.469c-7.7 0-12.674-5.32-12.674-13.552 0-8.388 4.74-13.599 12.37-13.599a17.274 17.274 0 0 1 11.828 4.996.24.24 0 0 0 .414-.168v-4.245a18.595 18.595 0 0 0-12.243-4.502 15.358 15.358 0 0 0-11.733 4.845c-2.857 3.138-4.366 7.52-4.366 12.674 0 10.478 6.592 17.518 16.404 17.518a18.517 18.517 0 0 0 12.38-4.624zM93.47 104.248v3.638h11.506v29.657h3.982v-29.657h11.506v-3.638H93.47zM390.972 115.625c-2.059-2.232-5.501-3.043-7.978-3.043a11.24 11.24 0 0 0-8.363 3.475 13.26 13.26 0 0 0-3.277 9.243c0 8.883 5.894 12.86 11.735 12.86 3.412 0 6.228-1.22 7.883-3.38v2.812h3.971V104.3h-3.97zm-.105 9.769c0 4.545-2.639 9.147-7.683 9.147-3.788 0-7.865-2.803-7.865-8.958 0-4.522 2.388-9.389 7.63-9.389 5.844 0 7.918 4.956 7.918 9.2zM308.064 104.3h4.031v33.292h-4.031zM192.033 131.427a.24.24 0 0 0-.403-.18 13.772 13.772 0 0 1-8.006 3.255c-5.344 0-8.796-3.583-8.796-9.128 0-5.546 3.452-9.129 8.796-9.129a12.973 12.973 0 0 1 8.01 2.8.24.24 0 0 0 .399-.183v-3.668a17.567 17.567 0 0 0-8.621-2.615c-7.38 0-12.34 5.142-12.34 12.795 0 7.652 4.96 12.794 12.34 12.794a14.903 14.903 0 0 0 8.62-2.624zM139.984 130.642a.241.241 0 0 0-.436-.143 12.896 12.896 0 0 1-8.605 4.214 8.725 8.725 0 0 1-6.104-2.141 8.634 8.634 0 0 1-2.372-6.07h18.629v-.285c0-5.254-.896-8.233-3.194-10.625a10.42 10.42 0 0 0-7.849-3.01c-6.969 0-11.472 4.987-11.471 12.715.254 8.04 4.822 12.84 12.218 12.84a13.447 13.447 0 0 0 9.184-3.52zm-2.653-7.564h-14.872l.006-.075a7.37 7.37 0 0 1 7.481-6.93c4.341 0 7.17 2.657 7.38 6.933zM362.877 125.584c0 6.632-3.825 8.985-7.1 8.985a6.287 6.287 0 0 1-6.549-6.263V113.15h-3.79v15.088c0 5.842 4.172 9.922 10.145 9.922 3.308 0 5.89-1.114 7.53-3.235v2.66h3.614V113.15h-3.85zM156.747 112.583a9.79 9.79 0 0 0-7.364 2.887v-2.334h-3.762v24.456h3.846v-13.935a7.592 7.592 0 0 1 1.982-5.51 6.75 6.75 0 0 1 5.197-1.91 5.994 5.994 0 0 1 6.055 6.339v15.016h3.847v-15.197a9.756 9.756 0 0 0-2.767-7.26 9.907 9.907 0 0 0-7.034-2.552zM217.156 130.642a.241.241 0 0 0-.436-.143 12.896 12.896 0 0 1-8.605 4.214 8.725 8.725 0 0 1-6.104-2.141 8.634 8.634 0 0 1-2.372-6.07h18.629v-.285c0-5.254-.895-8.233-3.193-10.625a10.42 10.42 0 0 0-7.85-3.01c-6.968 0-11.471 4.987-11.471 12.715.254 8.04 4.822 12.84 12.218 12.84a13.447 13.447 0 0 0 9.184-3.52zm-2.653-7.564h-14.871l.005-.075a7.37 7.37 0 0 1 7.481-6.93c4.342 0 7.17 2.657 7.381 6.933zM233.857 112.583a9.79 9.79 0 0 0-7.365 2.887v-2.334h-3.762v24.456h3.847v-13.935a7.592 7.592 0 0 1 1.982-5.51 6.75 6.75 0 0 1 5.196-1.91 5.994 5.994 0 0 1 6.056 6.339v15.016h3.846v-15.197a9.756 9.756 0 0 0-2.767-7.26 9.907 9.907 0 0 0-7.033-2.552zM256.236 137.917a19.963 19.963 0 0 0 5.009-1.138v-3.683a.241.241 0 0 0-.321-.229A29.455 29.455 0 0 1 256.8 134c-.402.064-.756.12-1.02-.104a.897.897 0 0 1-.263-.777V116.69h7.04v-3.559h-7.04v-6.516h-3.997v6.516h-4.012v3.558h4.012v16.815a4.207 4.207 0 0 0 1.309 3.327 5.088 5.088 0 0 0 3.406 1.085zM329.224 112.63c-7.093 0-11.859 5.13-11.859 12.764s4.766 12.764 11.859 12.764 11.858-5.13 11.858-12.764-4.765-12.764-11.858-12.764zm-8.18 12.739l-.001-.01c.014-5.41 3.299-9.043 8.18-9.043 4.893 0 8.18 3.648 8.182 9.078-.002 5.429-3.29 9.078-8.181 9.078-4.89 0-8.175-3.678-8.18-9.103z" />
+</svg>
@@ -0,0 +1,11 @@
+<svg viewBox="0 83.15545000000002 85 76.44" data-name="图层 1" id="图层_1"
+  xmlns="http://www.w3.org/2000/svg" style="max-height: 500px" width="85"
+  height="76.44">
+  <defs>
+    <style>.cls-1{fill:#4999d4}</style>
+  </defs>
+  <title>tencent-cloud</title>
+  <path
+    d="M27.569 113.353a17.56 17.56 0 0 1 33.148-3.743.158.158 0 0 0 .194.105 21.267 21.267 0 0 1 7.008-.729c.235.018.327-.116.25-.33a24.828 24.828 0 0 0-47.933 4.444.082.082 0 0 0 .016 0 18.537 18.537 0 0 0-9.85 31.533 18.007 18.007 0 0 0 10.325 5h-.001a43.066 43.066 0 0 0 5.266.282c1.68.011 33.725.008 35.067.008 2.7 0 4.457-.002 6.345-.14a18.245 18.245 0 0 0 11.723-5.15 18.532 18.532 0 0 0-12.901-31.789 18.06 18.06 0 0 0-11.704 4.285c-1.467 1.196-3.006 2.626-4.944 4.508-.642.625-13.336 12.94-21.67 21.028-1.16-.005-2.828-.021-4.306-.07a11.704 11.704 0 0 1-8.125-3.148A11.275 11.275 0 0 1 23.33 120.1a11.706 11.706 0 0 1 7.646 3.062c1.44 1.192 4.633 4 6.035 5.263a.17.17 0 0 0 .24.002l4.945-4.825a.176.176 0 0 0-.004-.27c-2.378-2.15-5.749-5.158-7.778-6.669a18.874 18.874 0 0 0-6.844-3.31zm46.482 26.094a11.704 11.704 0 0 1-8.125 3.147 168.92 168.92 0 0 1-5.204.073h-22.38c8.142-7.91 15.245-14.808 16.051-15.59.738-.717 2.398-2.306 3.83-3.595 3.145-2.831 5.974-3.4 7.976-3.382a11.275 11.275 0 0 1 7.852 19.347z"
+    class="cls-1" />
+</svg>
@@ -0,0 +1,156 @@
+import base64
+import hashlib
+import hmac
+import time
+
+import requests
+
+
+class Credential:
+    def __init__(self, secret_id, secret_key):
+        self.secret_id = secret_id
+        self.secret_key = secret_key
+
+
+class FlashRecognitionRequest:
+    def __init__(self, voice_format="mp3", engine_type="16k_zh"):
+        self.engine_type = engine_type
+        self.speaker_diarization = 0
+        self.hotword_id = ""
+        self.customization_id = ""
+        self.filter_dirty = 0
+        self.filter_modal = 0
+        self.filter_punc = 0
+        self.convert_num_mode = 1
+        self.word_info = 0
+        self.voice_format = voice_format
+        self.first_channel_only = 1
+        self.reinforce_hotword = 0
+        self.sentence_max_length = 0
+
+    def set_first_channel_only(self, first_channel_only):
+        self.first_channel_only = first_channel_only
+
+    def set_speaker_diarization(self, speaker_diarization):
+        self.speaker_diarization = speaker_diarization
+
+    def set_filter_dirty(self, filter_dirty):
+        self.filter_dirty = filter_dirty
+
+    def set_filter_modal(self, filter_modal):
+        self.filter_modal = filter_modal
+
+    def set_filter_punc(self, filter_punc):
+        self.filter_punc = filter_punc
+
+    def set_convert_num_mode(self, convert_num_mode):
+        self.convert_num_mode = convert_num_mode
+
+    def set_word_info(self, word_info):
+        self.word_info = word_info
+
+    def set_hotword_id(self, hotword_id):
+        self.hotword_id = hotword_id
+
+    def set_customization_id(self, customization_id):
+        self.customization_id = customization_id
+
+    def set_voice_format(self, voice_format):
+        self.voice_format = voice_format
+
+    def set_sentence_max_length(self, sentence_max_length):
+        self.sentence_max_length = sentence_max_length
+
+    def set_reinforce_hotword(self, reinforce_hotword):
+        self.reinforce_hotword = reinforce_hotword
+
+
+class FlashRecognizer:
+    """
+    reponse:
+    request_id        string
+    status            Integer    
+    message           String    
+    audio_duration    Integer
+    flash_result      Result Array
+
+    Result:
+    text              String
+    channel_id        Integer
+    sentence_list     Sentence Array
+
+    Sentence:
+    text              String
+    start_time        Integer    
+    end_time          Integer    
+    speaker_id        Integer    
+    word_list         Word Array
+
+    Word:
+    word              String 
+    start_time        Integer 
+    end_time          Integer 
+    stable_flag：     Integer 
+    """
+
+    def __init__(self, appid, credential):
+        self.credential = credential
+        self.appid = appid
+
+    def _format_sign_string(self, param):
+        signstr = "POSTasr.cloud.tencent.com/asr/flash/v1/"
+        for t in param:
+            if 'appid' in t:
+                signstr += str(t[1])
+                break
+        signstr += "?"
+        for x in param:
+            tmp = x
+            if 'appid' in x:
+                continue
+            for t in tmp:
+                signstr += str(t)
+                signstr += "="
+            signstr = signstr[:-1]
+            signstr += "&"
+        signstr = signstr[:-1]
+        return signstr
+
+    def _build_header(self):
+        header = {"Host": "asr.cloud.tencent.com"}
+        return header
+
+    def _sign(self, signstr, secret_key):
+        hmacstr = hmac.new(secret_key.encode('utf-8'),
+                           signstr.encode('utf-8'), hashlib.sha1).digest()
+        s = base64.b64encode(hmacstr)
+        s = s.decode('utf-8')
+        return s
+
+    def _build_req_with_signature(self, secret_key, params, header):
+        query = sorted(params.items(), key=lambda d: d[0])
+        signstr = self._format_sign_string(query)
+        signature = self._sign(signstr, secret_key)
+        header["Authorization"] = signature
+        requrl = "https://"
+        requrl += signstr[4::]
+        return requrl
+
+    def _create_query_arr(self, req):
+        return {
+            'appid': self.appid, 'secretid': self.credential.secret_id, 'timestamp': str(int(time.time())),
+             'engine_type': req.engine_type, 'voice_format': req.voice_format,
+             'speaker_diarization': req.speaker_diarization, 'hotword_id': req.hotword_id,
+             'customization_id': req.customization_id, 'filter_dirty': req.filter_dirty,
+             'filter_modal': req.filter_modal, 'filter_punc': req.filter_punc,
+             'convert_num_mode': req.convert_num_mode, 'word_info': req.word_info,
+             'first_channel_only': req.first_channel_only, 'reinforce_hotword': req.reinforce_hotword,
+             'sentence_max_length': req.sentence_max_length
+        }
+
+    def recognize(self, req, data):
+        header = self._build_header()
+        query_arr = self._create_query_arr(req)
+        req_url = self._build_req_with_signature(self.credential.secret_key, query_arr, header)
+        r = requests.post(req_url, headers=header, data=data)
+        return r.text
@@ -0,0 +1,92 @@
+import json
+from typing import IO, Optional
+
+import requests
+
+from core.model_runtime.errors.invoke import (
+    InvokeAuthorizationError,
+    InvokeConnectionError,
+    InvokeError,
+)
+from core.model_runtime.errors.validate import CredentialsValidateFailedError
+from core.model_runtime.model_providers.__base.speech2text_model import Speech2TextModel
+from core.model_runtime.model_providers.tencent.speech2text.flash_recognizer import (
+    Credential,
+    FlashRecognitionRequest,
+    FlashRecognizer,
+)
+
+
+class TencentSpeech2TextModel(Speech2TextModel):
+    def _invoke(self, model: str, credentials: dict,
+                file: IO[bytes], user: Optional[str] = None) \
+            -> str:
+        """
+        Invoke speech2text model
+
+        :param model: model name
+        :param credentials: model credentials
+        :param file: audio file
+        :param user: unique user id
+        :return: text for given audio file
+        """
+        return self._speech2text_invoke(model, credentials, file)
+
+    def validate_credentials(self, model: str, credentials: dict) -> None:
+        """
+        Validate model credentials
+
+        :param model: model name
+        :param credentials: model credentials
+        :return:
+        """
+        try:
+            audio_file_path = self._get_demo_file_path()
+
+            with open(audio_file_path, 'rb') as audio_file:
+                self._speech2text_invoke(model, credentials, audio_file)
+        except Exception as ex:
+            raise CredentialsValidateFailedError(str(ex))
+
+    def _speech2text_invoke(self, model: str, credentials: dict, file: IO[bytes]) -> str:
+        """
+        Invoke speech2text model
+
+        :param model: model name
+        :param credentials: model credentials
+        :param file: audio file
+        :return: text for given audio file
+        """
+        app_id = credentials["app_id"]
+        secret_id = credentials["secret_id"]
+        secret_key = credentials["secret_key"]
+        voice_format = file.voice_format if hasattr(file, "voice_format") else "mp3"
+        tencent_voice_recognizer = FlashRecognizer(app_id, Credential(secret_id, secret_key))
+        resp = tencent_voice_recognizer.recognize(FlashRecognitionRequest(voice_format), file)
+        resp = json.loads(resp)
+        code = resp["code"]
+        message = resp["message"]
+        if code == 4002:
+            raise CredentialsValidateFailedError(str(message))
+        elif code != 0:
+            return f"Tencent ASR Recognition failed with code {code} and message {message}"
+        return "\n".join(item["text"] for item in resp["flash_result"])
+
+    @property
+    def _invoke_error_mapping(self) -> dict[type[InvokeError], list[type[Exception]]]:
+        """
+        Map model invoke error to unified error
+        The key is the error type thrown to the caller
+        The value is the error type thrown by the model,
+        which needs to be converted into a unified error type for the caller.
+
+        :return: Invoke error mapping
+        """
+        return {
+            InvokeConnectionError: [
+                requests.exceptions.ConnectionError
+            ],
+            InvokeAuthorizationError: [
+                CredentialsValidateFailedError
+            ]
+        }
@@ -0,0 +1,5 @@
+model: tencent
+model_type: speech2text
+model_properties:
+  file_upload_limit: 25
+  supported_file_extensions: flac,mp3,mp4,mpeg,mpga,m4a,ogg,wav,webm
@@ -0,0 +1,29 @@
+import logging
+
+from core.model_runtime.entities.model_entities import ModelType
+from core.model_runtime.errors.validate import CredentialsValidateFailedError
+from core.model_runtime.model_providers.__base.model_provider import ModelProvider
+
+logger = logging.getLogger(__name__)
+
+
+class TencentProvider(ModelProvider):
+    def validate_provider_credentials(self, credentials: dict) -> None:
+        """
+        Validate provider credentials
+
+        if validate failed, raise exception
+
+        :param credentials: provider credentials, credentials form defined in `provider_credential_schema`.
+        """
+        try:
+            model_instance = self.get_model_instance(ModelType.SPEECH2TEXT)
+            model_instance.validate_credentials(
+                model='tencent',
+                credentials=credentials
+            )
+        except CredentialsValidateFailedError as ex:
+            raise ex
+        except Exception as ex:
+            logger.exception(f'{self.get_provider_schema().provider} credentials validate failed')
+            raise ex
@@ -0,0 +1,49 @@
+provider: tencent
+label:
+  zh_Hans: 腾讯云
+  en_US: Tencent
+icon_small:
+  en_US: icon_s_en.svg
+icon_large:
+  zh_Hans: icon_l_zh.svg
+  en_US: icon_l_en.svg
+background: "#E5E7EB"
+help:
+  title:
+    en_US: Get your API key from Tencent AI
+    zh_Hans: 从腾讯云获取 API Key
+  url:
+    en_US: https://cloud.tencent.com/product/asr
+supported_model_types:
+  - speech2text
+configurate_methods:
+  - predefined-model
+provider_credential_schema:
+  credential_form_schemas:
+    - variable: app_id
+      label:
+        zh_Hans: APPID
+        en_US: APPID
+      type: text-input
+      required: true
+      placeholder:
+        zh_Hans: 在此输入您的腾讯语音识别服务的 APPID
+        en_US: Enter the APPID of your Tencent Cloud ASR service
+    - variable: secret_id
+      label:
+        zh_Hans: SecretId
+        en_US: SecretId
+      type: secret-input
+      required: true
+      placeholder:
+        zh_Hans: 在此输入您的腾讯语音识别服务的 SecretId
+        en_US: Enter the SecretId of your Tencent Cloud ASR service
+    - variable: secret_key
+      label:
+        zh_Hans: SecretKey
+        en_US: SecretKey
+      type: secret-input
+      required: true
+      placeholder:
+        zh_Hans: 在此输入您的腾讯语音识别服务的 SecretKey
+        en_US: Enter the SecretKey of your Tencent Cloud ASR service
@@ -262,6 +262,10 @@ You should also complete the text started with ``` but not tell ``` directly.
        :param prompt_messages: prompt messages
        :return: llm response
        """
+        if response.status_code != 200 and response.status_code != HTTPStatus.OK:
+            raise ServiceUnavailableError(
+                response.message
+            )
        # transform assistant message to prompt message
        assistant_prompt_message = AssistantPromptMessage(
            content=response.output.choices[0].message.content,
@@ -38,7 +38,7 @@ class BaseKeyword(ABC):
        raise NotImplementedError

    def _filter_duplicate_texts(self, texts: list[Document]) -> list[Document]:
-        for text in texts:
+        for text in texts[:]:
            doc_id = text.metadata['doc_id']
            exists_duplicate_node = self.text_exists(doc_id)
            if exists_duplicate_node:
@@ -110,7 +110,7 @@ class RetrievalService:
                )

                documents = keyword.search(
-                    query,
+                    cls.escape_query_for_search(query),
                    top_k=top_k
                )
                all_documents.extend(documents)
@@ -132,7 +132,7 @@ class RetrievalService:
                )

                documents = vector.search_by_vector(
-                    query,
+                    cls.escape_query_for_search(query),
                    search_type='similarity_score_threshold',
                    top_k=top_k,
                    score_threshold=score_threshold,
@@ -170,7 +170,7 @@ class RetrievalService:
                )

                documents = vector_processor.search_by_full_text(
-                    query,
+                    cls.escape_query_for_search(query),
                    top_k=top_k
                )
                if documents:
@@ -186,3 +186,7 @@ class RetrievalService:
                        all_documents.extend(documents)
            except Exception as e:
                exceptions.append(str(e))
+
+    @staticmethod
+    def escape_query_for_search(query: str) -> str:
+        return query.replace('"', '\\"')
@@ -293,15 +293,18 @@ class AnalyticdbVector(BaseVector):
        return documents

    def delete(self) -> None:
-        from alibabacloud_gpdb20160503 import models as gpdb_20160503_models
-        request = gpdb_20160503_models.DeleteCollectionRequest(
-            collection=self._collection_name,
-            dbinstance_id=self.config.instance_id,
-            namespace=self.config.namespace,
-            namespace_password=self.config.namespace_password,
-            region_id=self.config.region_id,
-        )
-        self._client.delete_collection(request)
+        try:
+            from alibabacloud_gpdb20160503 import models as gpdb_20160503_models
+            request = gpdb_20160503_models.DeleteCollectionRequest(
+                collection=self._collection_name,
+                dbinstance_id=self.config.instance_id,
+                namespace=self.config.namespace,
+                namespace_password=self.config.namespace_password,
+                region_id=self.config.region_id,
+            )
+            self._client.delete_collection(request)
+        except Exception as e:
+            raise e

 class AnalyticdbVectorFactory(AbstractVectorFactory):
    def init_vector(self, dataset: Dataset, attributes: list, embeddings: Embeddings):
@@ -115,7 +115,8 @@ class MilvusVector(BaseVector):
            uri = "https://" + str(self._client_config.host) + ":" + str(self._client_config.port)
        else:
            uri = "http://" + str(self._client_config.host) + ":" + str(self._client_config.port)
-        connections.connect(alias=alias, uri=uri, user=self._client_config.user, password=self._client_config.password)
+        connections.connect(alias=alias, uri=uri, user=self._client_config.user, password=self._client_config.password,
+                            db_name=self._client_config.database)

        from pymilvus import utility
        if utility.has_collection(self._collection_name, using=alias):
@@ -130,7 +131,8 @@ class MilvusVector(BaseVector):
            uri = "https://" + str(self._client_config.host) + ":" + str(self._client_config.port)
        else:
            uri = "http://" + str(self._client_config.host) + ":" + str(self._client_config.port)
-        connections.connect(alias=alias, uri=uri, user=self._client_config.user, password=self._client_config.password)
+        connections.connect(alias=alias, uri=uri, user=self._client_config.user, password=self._client_config.password,
+                            db_name=self._client_config.database)

        from pymilvus import utility
        if utility.has_collection(self._collection_name, using=alias):
@@ -1,11 +1,15 @@
 import array
 import json
+import re
 import uuid
 from contextlib import contextmanager
 from typing import Any

+import jieba.posseg as pseg
+import nltk
 import numpy
 import oracledb
+from nltk.corpus import stopwords
 from pydantic import BaseModel, model_validator

 from configs import dify_config
@@ -50,6 +54,11 @@ CREATE TABLE IF NOT EXISTS {table_name} (
    ,embedding vector NOT NULL
 )
 """
+SQL_CREATE_INDEX = """
+CREATE INDEX idx_docs_{table_name} ON {table_name}(text) 
+INDEXTYPE IS CTXSYS.CONTEXT PARAMETERS 
+('FILTER CTXSYS.NULL_FILTER SECTION GROUP CTXSYS.HTML_SECTION_GROUP LEXER sys.my_chinese_vgram_lexer')
+"""


 class OracleVector(BaseVector):
@@ -188,7 +197,53 @@ class OracleVector(BaseVector):
        return docs

    def search_by_full_text(self, query: str, **kwargs: Any) -> list[Document]:
-        # do not support bm25 search
+        top_k = kwargs.get("top_k", 5)
+        # just not implement fetch by score_threshold now, may be later
+        score_threshold = kwargs.get("score_threshold") if kwargs.get("score_threshold") else 0.0
+        if len(query) > 0:
+            # Check which language the query is in
+            zh_pattern = re.compile('[\u4e00-\u9fa5]+')
+            match = zh_pattern.search(query)
+            entities = []
+            #  match: query condition maybe is a chinese sentence, so using Jieba split,else using nltk split
+            if match:
+                words = pseg.cut(query)
+                current_entity = ""
+                for word, pos in words:
+                    if pos == 'nr' or pos == 'Ng' or pos == 'eng' or pos == 'nz' or pos == 'n' or pos == 'ORG' or pos == 'v':  # nr: 人名, ns: 地名, nt: 机构名
+                        current_entity += word
+                    else:
+                        if current_entity:
+                            entities.append(current_entity)
+                            current_entity = ""
+                if current_entity:
+                    entities.append(current_entity)
+            else:
+                try:
+                    nltk.data.find('tokenizers/punkt')
+                    nltk.data.find('corpora/stopwords')
+                except LookupError:
+                    nltk.download('punkt')
+                    nltk.download('stopwords')
+                    print("run download")
+                e_str = re.sub(r'[^\w ]', '', query)
+                all_tokens = nltk.word_tokenize(e_str)
+                stop_words = stopwords.words('english')
+                for token in all_tokens:
+                    if token not in stop_words:
+                        entities.append(token)
+            with self._get_cursor() as cur:
+                cur.execute(
+                    f"select meta, text FROM {self.table_name} WHERE CONTAINS(text, :1, 1) > 0 order by score(1) desc fetch first {top_k} rows only",
+                    [" ACCUM ".join(entities)]
+                )
+                docs = []
+                for record in cur:
+                    metadata, text = record
+                    docs.append(Document(page_content=text, metadata=metadata))
+            return docs
+        else:
+            return [Document(page_content="", metadata="")]
        return []

    def delete(self) -> None:
@@ -206,6 +261,8 @@ class OracleVector(BaseVector):
            with self._get_cursor() as cur:
                cur.execute(SQL_CREATE_TABLE.format(table_name=self.table_name))
            redis_client.set(collection_exist_cache_key, 1, ex=3600)
+            with self._get_cursor() as cur:
+                cur.execute(SQL_CREATE_INDEX.format(table_name=self.table_name))


 class OracleVectorFactory(AbstractVectorFactory):
@@ -198,8 +198,6 @@ class TencentVector(BaseVector):
        self._db.drop_collection(name=self._collection_name)


-
-
 class TencentVectorFactory(AbstractVectorFactory):
    def init_vector(self, dataset: Dataset, attributes: list, embeddings: Embeddings) -> TencentVector:

@@ -57,7 +57,7 @@ class BaseVector(ABC):
        raise NotImplementedError

    def _filter_duplicate_texts(self, texts: list[Document]) -> list[Document]:
-        for text in texts:
+        for text in texts[:]:
            doc_id = text.metadata['doc_id']
            exists_duplicate_node = self.text_exists(doc_id)
            if exists_duplicate_node:
@@ -67,3 +67,7 @@ class BaseVector(ABC):

    def _get_uuids(self, texts: list[Document]) -> list[str]:
        return [text.metadata['doc_id'] for text in texts]
+
+    @property
+    def collection_name(self):
+        return self._collection_name
@@ -9,6 +9,7 @@ from core.rag.datasource.entity.embedding import Embeddings
 from core.rag.datasource.vdb.vector_base import BaseVector
 from core.rag.datasource.vdb.vector_type import VectorType
 from core.rag.models.document import Document
+from extensions.ext_redis import redis_client
 from models.dataset import Dataset


@@ -134,6 +135,10 @@ class Vector:

    def delete(self) -> None:
        self._vector_processor.delete()
+        # delete collection redis cache
+        if self._vector_processor.collection_name:
+            collection_exist_cache_key = 'vector_indexing_{}'.format(self._vector_processor.collection_name)
+            redis_client.delete(collection_exist_cache_key)

    def _get_embeddings(self) -> Embeddings:
        model_manager = ModelManager()
@@ -148,7 +153,7 @@ class Vector:
        return CacheEmbedding(embedding_model)

    def _filter_duplicate_texts(self, texts: list[Document]) -> list[Document]:
-        for text in texts:
+        for text in texts[:]:
            doc_id = text.metadata['doc_id']
            exists_duplicate_node = self.text_exists(doc_id)
            if exists_duplicate_node:
@@ -0,0 +1,16 @@
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="80" height="80" viewBox="0 0 80 80">
+  <defs>
+    <clipPath id="clip-画板_16">
+      <rect width="80" height="80"/>
+    </clipPath>
+  </defs>
+  <g id="画板_16" data-name="画板 – 16" clip-path="url(#clip-画板_16)">
+    <g id="组_27" data-name="组 27">
+      <circle id="椭圆_13" data-name="椭圆 13" cx="40" cy="40" r="40" fill="#3e96f3"/>
+      <g id="组_26" data-name="组 26" transform="translate(16.361 16.674)">
+        <path id="路径_63230" data-name="路径 63230" d="M92.52,179.361c5.974,0,8.592-2.307,8.592-8.2,0-6.428.067-32.5.067-32.5h-.738a5.221,5.221,0,0,0-4.005,1.344H86.076A5.3,5.3,0,0,0,82,138.6h-.582v32.855c0,5.487,2.461,7.906,8.413,7.906Zm-5.348-33.93h8.189c-.022,4.457-.022,8.04-.022,10.952H87.15C87.15,153.471,87.173,149.888,87.173,145.431Zm-.067,25.823c0-2.016,0-3.315.022-9.451h8.189c-.022,6.316-.022,7.547-.022,9.429,0,1.792-1.052,2.71-2.842,2.71H89.7A2.447,2.447,0,0,1,87.106,171.253Z" transform="translate(-81.4 -137.973)" fill="#fff"/>
+        <path id="路径_63231" data-name="路径 63231" d="M172.508,168.341h12.373v6.495c0,2.508-.873,3.27-3.468,3.27H178.84v4.972h4.027c5.437,0,7.988-2.419,7.988-7.637V143.549c0-5.33-2.17-7.749-8.323-7.749h-7.719c-6.22,0-8.323,2.441-8.323,7.7v27.413c0,3.942-1.611,6.293-4.99,7.525v4.614c7.652-.851,10.986-4.838,10.986-12.027C172.508,170.178,172.508,169.26,172.508,168.341Zm0-24.255c0-2.038.85-2.844,3.132-2.844h6.332c1.991,0,2.909.9,2.909,2.934v4.994H172.508Zm0,10.5h12.373v8.331H172.508Z" transform="translate(-143.578 -135.8)" fill="#fff"/>
+      </g>
+    </g>
+  </g>
+</svg>
@@ -0,0 +1,8 @@
+from typing import Any
+
+from core.tools.provider.builtin_tool_provider import BuiltinToolProviderController
+
+
+class HapProvider(BuiltinToolProviderController):
+    def _validate_credentials(self, credentials: dict[str, Any]) -> None:
+        pass
@@ -0,0 +1,15 @@
+identity:
+  author: Mingdao
+  name: hap
+  label:
+    en_US: HAP
+    zh_Hans: HAP
+    pt_BR: HAP
+  description:
+    en_US: "Hyper application platform that is particularly friendly to AI"
+    zh_Hans: "对 AI 特别友好的超级应用平台"
+    pt_BR: "Plataforma de aplicação hiper que é particularmente amigável à IA"
+  icon: icon.svg
+  tags:
+    - productivity
+credentials_for_provider:
@@ -0,0 +1,53 @@
+import json
+from typing import Any, Union
+
+import httpx
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class AddWorksheetRecordTool(BuiltinTool):
+
+    def _invoke(self, user_id: str, tool_parameters: dict[str, Any]
+                ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+        
+        appkey = tool_parameters.get('appkey', '')
+        if not appkey:
+            return self.create_text_message('Invalid parameter App Key')
+        sign = tool_parameters.get('sign', '')
+        if not sign:
+            return self.create_text_message('Invalid parameter Sign')
+        worksheet_id = tool_parameters.get('worksheet_id', '')
+        if not worksheet_id:
+            return self.create_text_message('Invalid parameter Worksheet ID')
+        record_data = tool_parameters.get('record_data', '')
+        if not record_data:
+            return self.create_text_message('Invalid parameter Record Row Data')
+        
+        host = tool_parameters.get('host', '')
+        if not host:
+            host = 'https://api.mingdao.com'
+        elif not host.startswith(("http://", "https://")):
+            return self.create_text_message('Invalid parameter Host Address')
+        else:
+            host = f"{host[:-1] if host.endswith('/') else host}/api"
+
+        url = f"{host}/v2/open/worksheet/addRow"
+        headers = {'Content-Type': 'application/json'}
+        payload = {"appKey": appkey, "sign": sign, "worksheetId": worksheet_id}
+
+        try:
+            payload['controls'] = json.loads(record_data)
+            res = httpx.post(url, headers=headers, json=payload, timeout=60)
+            res.raise_for_status()
+            res_json = res.json()
+            if res_json.get('error_code') != 1:
+                return self.create_text_message(f"Failed to add the new record. {res_json['error_msg']}")
+            return self.create_text_message(f"New record added successfully. The record ID is {res_json['data']}.")
+        except httpx.RequestError as e:
+            return self.create_text_message(f"Failed to add the new record, request error: {e}")
+        except json.JSONDecodeError as e:
+            return self.create_text_message(f"Failed to parse JSON response: {e}")
+        except Exception as e:
+            return self.create_text_message(f"Failed to add the new record, unexpected error: {e}")
@@ -0,0 +1,78 @@
+identity:
+  name: add_worksheet_record
+  author: Ryan Tian
+  label:
+    en_US: Add Worksheet Record
+    zh_Hans: 新增一条工作表记录
+description:
+  human:
+    en_US: Adds a new record to the specified worksheet
+    zh_Hans: 向指定的工作表新增一条记录数据
+  llm: A tool to append a new data entry into a specified worksheet.
+parameters:
+  - name: appkey
+    type: secret-input
+    required: true
+    label:
+      en_US: App Key
+      zh_Hans: App Key
+    human_description:
+      en_US: The AppKey parameter for the HAP application, typically found in the application's API documentation.
+      zh_Hans: HAP 应用的 AppKey 参数，可以从应用 API 文档中查找到
+    llm_description: the AppKey parameter for the HAP application
+    form: form
+
+  - name: sign
+    type: secret-input
+    required: true
+    label:
+      en_US: Sign
+      zh_Hans: Sign
+    human_description:
+      en_US: The Sign parameter for the HAP application
+      zh_Hans: HAP 应用的 Sign 参数
+    llm_description: the Sign parameter for the HAP application
+    form: form
+
+  - name: worksheet_id
+    type: string
+    required: true
+    label:
+      en_US: Worksheet ID
+      zh_Hans: 工作表 ID
+    human_description:
+      en_US: The ID of the specified worksheet
+      zh_Hans: 要获取字段信息的工作表 ID
+    llm_description: The ID of the specified worksheet which to get the fields information.
+    form: llm
+
+  - name: record_data
+    type: string
+    required: true
+    label:
+      en_US: Record Row Data
+      zh_Hans: 记录数据
+    human_description:
+      en_US: The fields with data of the specified record
+      zh_Hans: 要新增的记录数据，JSON 对象数组格式。数组元素属性：controlId-字段ID，value-字段值
+    llm_description: |
+      The fields with data of the specified record which to be created. It is in the format of an array of JSON objects, and the structure is defined as follows:
+      ```
+      type RowData = {
+        controlId: string; // Field ID to be updated
+        value: string; // Field value to be updated
+      }[];
+      ```
+    form: llm
+
+  - name: host
+    type: string
+    required: false
+    label:
+      en_US: Host Address
+      zh_Hans: 服务器地址
+    human_description:
+      en_US: The address for the privately deployed HAP server.
+      zh_Hans: 私有部署 HAP 服务器地址，公有云无需填写
+    llm_description: the address for the privately deployed HAP server.
+    form: form
@@ -0,0 +1,49 @@
+from typing import Any, Union
+
+import httpx
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class DeleteWorksheetRecordTool(BuiltinTool):
+
+    def _invoke(self, user_id: str, tool_parameters: dict[str, Any]
+                ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+        
+        appkey = tool_parameters.get('appkey', '')
+        if not appkey:
+            return self.create_text_message('Invalid parameter App Key')
+        sign = tool_parameters.get('sign', '')
+        if not sign:
+            return self.create_text_message('Invalid parameter Sign')
+        worksheet_id = tool_parameters.get('worksheet_id', '')
+        if not worksheet_id:
+            return self.create_text_message('Invalid parameter Worksheet ID')
+        row_id = tool_parameters.get('row_id', '')
+        if not row_id:
+            return self.create_text_message('Invalid parameter Record Row ID')
+        
+        host = tool_parameters.get('host', '')
+        if not host:
+            host = 'https://api.mingdao.com'
+        elif not host.startswith(("http://", "https://")):
+            return self.create_text_message('Invalid parameter Host Address')
+        else:
+            host = f"{host[:-1] if host.endswith('/') else host}/api"
+
+        url = f"{host}/v2/open/worksheet/deleteRow"
+        headers = {'Content-Type': 'application/json'}
+        payload = {"appKey": appkey, "sign": sign, "worksheetId": worksheet_id, "rowId": row_id}
+
+        try:
+            res = httpx.post(url, headers=headers, json=payload, timeout=30)
+            res.raise_for_status()
+            res_json = res.json()
+            if res_json.get('error_code') != 1:
+                return self.create_text_message(f"Failed to delete the record. {res_json['error_msg']}")
+            return self.create_text_message("Successfully deleted the record.")
+        except httpx.RequestError as e:
+            return self.create_text_message(f"Failed to delete the record, request error: {e}")
+        except Exception as e:
+            return self.create_text_message(f"Failed to delete the record, unexpected error: {e}")
@@ -0,0 +1,71 @@
+identity:
+  name: delete_worksheet_record
+  author: Ryan Tian
+  label:
+    en_US: Delete Worksheet Record
+    zh_Hans: 删除指定的一条工作表记录
+description:
+  human:
+    en_US: Deletes a single record from a worksheet based on the specified record row ID
+    zh_Hans: 根据指定的记录ID删除一条工作表记录数据
+  llm: A tool to remove a particular record from a worksheet by specifying its unique record identifier.
+parameters:
+  - name: appkey
+    type: secret-input
+    required: true
+    label:
+      en_US: App Key
+      zh_Hans: App Key
+    human_description:
+      en_US: The AppKey parameter for the HAP application, typically found in the application's API documentation.
+      zh_Hans: HAP 应用的 AppKey 参数，可以从应用 API 文档中查找到
+    llm_description: the AppKey parameter for the HAP application
+    form: form
+
+  - name: sign
+    type: secret-input
+    required: true
+    label:
+      en_US: Sign
+      zh_Hans: Sign
+    human_description:
+      en_US: The Sign parameter for the HAP application
+      zh_Hans: HAP 应用的 Sign 参数
+    llm_description: the Sign parameter for the HAP application
+    form: form
+
+  - name: worksheet_id
+    type: string
+    required: true
+    label:
+      en_US: Worksheet ID
+      zh_Hans: 工作表 ID
+    human_description:
+      en_US: The ID of the specified worksheet
+      zh_Hans: 要获取字段信息的工作表 ID
+    llm_description: The ID of the specified worksheet which to get the fields information.
+    form: llm
+
+  - name: row_id
+    type: string
+    required: true
+    label:
+      en_US: Record Row ID
+      zh_Hans: 记录 ID
+    human_description:
+      en_US: The row ID of the specified record
+      zh_Hans: 要删除的记录 ID
+    llm_description: The row ID of the specified record which to be deleted.
+    form: llm
+
+  - name: host
+    type: string
+    required: false
+    label:
+      en_US: Host Address
+      zh_Hans: 服务器地址
+    human_description:
+      en_US: The address for the privately deployed HAP server.
+      zh_Hans: 私有部署 HAP 服务器地址，公有云无需填写
+    llm_description: the address for the privately deployed HAP server.
+    form: form
@@ -0,0 +1,148 @@
+import json
+from typing import Any, Union
+
+import httpx
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class GetWorksheetFieldsTool(BuiltinTool):
+
+    def _invoke(self, user_id: str, tool_parameters: dict[str, Any]
+                ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+        
+        appkey = tool_parameters.get('appkey', '')
+        if not appkey:
+            return self.create_text_message('Invalid parameter App Key')
+        sign = tool_parameters.get('sign', '')
+        if not sign:
+            return self.create_text_message('Invalid parameter Sign')
+        worksheet_id = tool_parameters.get('worksheet_id', '')
+        if not worksheet_id:
+            return self.create_text_message('Invalid parameter Worksheet ID')
+        
+        host = tool_parameters.get('host', '')
+        if not host:
+            host = 'https://api.mingdao.com'
+        elif not host.startswith(("http://", "https://")):
+            return self.create_text_message('Invalid parameter Host Address')
+        else:
+            host = f"{host[:-1] if host.endswith('/') else host}/api"
+
+        url = f"{host}/v2/open/worksheet/getWorksheetInfo"
+        headers = {'Content-Type': 'application/json'}
+        payload = {"appKey": appkey, "sign": sign, "worksheetId": worksheet_id}
+
+        try:
+            res = httpx.post(url, headers=headers, json=payload, timeout=60)
+            res.raise_for_status()
+            res_json = res.json()
+            if res_json.get('error_code') != 1:
+                return self.create_text_message(f"Failed to get the worksheet information. {res_json['error_msg']}")
+            
+            fields_json, fields_table = self.get_controls(res_json['data']['controls'])
+            result_type = tool_parameters.get('result_type', 'table')
+            return self.create_text_message(
+                text=json.dumps(fields_json, ensure_ascii=False) if result_type == 'json' else fields_table
+            )
+        except httpx.RequestError as e:
+            return self.create_text_message(f"Failed to get the worksheet information, request error: {e}")
+        except json.JSONDecodeError as e:
+            return self.create_text_message(f"Failed to parse JSON response: {e}")
+        except Exception as e:
+            return self.create_text_message(f"Failed to get the worksheet information, unexpected error: {e}")
+
+    def get_field_type_by_id(self, field_type_id: int) -> str:
+        field_type_map = {
+            2: "Text",
+            3: "Text-Phone",
+            4: "Text-Phone",
+            5: "Text-Email",
+            6: "Number",
+            7: "Text",
+            8: "Number",
+            9: "Option-Single Choice",
+            10: "Option-Multiple Choices",
+            11: "Option-Single Choice",
+            15: "Date",
+            16: "Date",
+            24: "Option-Region",
+            25: "Text",
+            26: "Option-Member",
+            27: "Option-Department",
+            28: "Number",
+            29: "Option-Linked Record",
+            30: "Unknown Type",
+            31: "Number",
+            32: "Text",
+            33: "Text",
+            35: "Option-Linked Record",
+            36: "Number-Yes1/No0",
+            37: "Number",
+            38: "Date",
+            40: "Location",
+            41: "Text",
+            46: "Time",
+            48: "Option-Organizational Role",
+            50: "Text",
+            51: "Query Record",
+        }
+        return field_type_map.get(field_type_id, '')
+
+    def get_controls(self, controls: list) -> dict:
+        fields = []
+        fields_list = ['|fieldId|fieldName|fieldType|fieldTypeId|description|options|','|'+'---|'*6]
+        for control in controls:
+            if control['type'] in self._get_ignore_types():
+                continue
+            field_type_id = control['type']
+            field_type = self.get_field_type_by_id(control['type'])
+            if field_type_id == 30:
+                source_type = control['sourceControl']['type']
+                if source_type in self._get_ignore_types():
+                    continue
+                else:
+                    field_type_id = source_type
+                    field_type = self.get_field_type_by_id(source_type)
+            field = {
+                'id': control['controlId'],
+                'name': control['controlName'],
+                'type': field_type,
+                'typeId': field_type_id,
+                'description': control['remark'].replace('\n', ' ').replace('\t', '  '),
+                'options': self._extract_options(control),
+            }
+            fields.append(field)
+            fields_list.append(f"|{field['id']}|{field['name']}|{field['type']}|{field['typeId']}|{field['description']}|{field['options'] if field['options'] else ''}|")
+
+        fields.append({
+            'id': 'ctime',
+            'name': 'Created Time',
+            'type': self.get_field_type_by_id(16),
+            'typeId': 16,
+            'description': '',
+            'options': []
+        })
+        fields_list.append("|ctime|Created Time|Date|16|||")
+        return fields, '\n'.join(fields_list)
+
+    def _extract_options(self, control: dict) -> list:
+        options = []
+        if control['type'] in [9, 10, 11]:
+            options.extend([{"key": opt['key'], "value": opt['value']} for opt in control.get('options', [])])
+        elif control['type'] in [28, 36]:
+            itemnames = control['advancedSetting'].get('itemnames')
+            if itemnames and itemnames.startswith('[{'):
+                try:
+                    options = json.loads(itemnames)
+                except json.JSONDecodeError:
+                    pass
+        elif control['type'] == 30:
+            source_type = control['sourceControl']['type']
+            if source_type not in self._get_ignore_types():
+                options.extend([{"key": opt['key'], "value": opt['value']} for opt in control.get('options', [])])
+        return options
+    
+    def _get_ignore_types(self):
+        return {14, 21, 22, 34, 42, 43, 45, 47, 49, 10010}
@@ -0,0 +1,80 @@
+identity:
+  name: get_worksheet_fields
+  author: Ryan Tian
+  label:
+    en_US: Get Worksheet Fields
+    zh_Hans: 获取工作表字段结构
+description:
+  human:
+    en_US: Get fields information of the worksheet
+    zh_Hans: 获取指定工作表的所有字段结构信息
+  llm: A tool to get fields information of the specific worksheet.
+parameters:
+  - name: appkey
+    type: secret-input
+    required: true
+    label:
+      en_US: App Key
+      zh_Hans: App Key
+    human_description:
+      en_US: The AppKey parameter for the HAP application, typically found in the application's API documentation.
+      zh_Hans: HAP 应用的 AppKey 参数，可以从应用 API 文档中查找到
+    llm_description: the AppKey parameter for the HAP application
+    form: form
+
+  - name: sign
+    type: secret-input
+    required: true
+    label:
+      en_US: Sign
+      zh_Hans: Sign
+    human_description:
+      en_US: The Sign parameter for the HAP application
+      zh_Hans: HAP 应用的 Sign 参数
+    llm_description: the Sign parameter for the HAP application
+    form: form
+
+  - name: worksheet_id
+    type: string
+    required: true
+    label:
+      en_US: Worksheet ID
+      zh_Hans: 工作表 ID
+    human_description:
+      en_US: The ID of the specified worksheet
+      zh_Hans: 要获取字段信息的工作表 ID
+    llm_description: The ID of the specified worksheet which to get the fields information.
+    form: llm
+
+  - name: host
+    type: string
+    required: false
+    label:
+      en_US: Host Address
+      zh_Hans: 服务器地址
+    human_description:
+      en_US: The address for the privately deployed HAP server.
+      zh_Hans: 私有部署 HAP 服务器地址，公有云无需填写
+    llm_description: the address for the privately deployed HAP server.
+    form: form
+
+  - name: result_type
+    type: select
+    required: true
+    options:
+      - value: table
+        label:
+          en_US: table text
+          zh_Hans: 表格文本
+      - value: json
+        label:
+          en_US: json text
+          zh_Hans: JSON文本
+    default: table
+    label:
+      en_US: Result type
+      zh_Hans: 结果类型
+    human_description:
+      en_US: used for selecting the result type, table styled text or json text
+      zh_Hans: 用于选择结果类型，使用表格格式文本还是JSON格式文本
+    form: form
@@ -0,0 +1,130 @@
+import json
+from typing import Any, Union
+
+import httpx
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class GetWorksheetPivotDataTool(BuiltinTool):
+
+    def _invoke(self, user_id: str, tool_parameters: dict[str, Any]
+                ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+        
+        appkey = tool_parameters.get('appkey', '')
+        if not appkey:
+            return self.create_text_message('Invalid parameter App Key')
+        sign = tool_parameters.get('sign', '')
+        if not sign:
+            return self.create_text_message('Invalid parameter Sign')
+        worksheet_id = tool_parameters.get('worksheet_id', '')
+        if not worksheet_id:
+            return self.create_text_message('Invalid parameter Worksheet ID')
+        x_column_fields = tool_parameters.get('x_column_fields', '')
+        if not x_column_fields or not x_column_fields.startswith('['):
+            return self.create_text_message('Invalid parameter Column Fields')
+        y_row_fields = tool_parameters.get('y_row_fields', '')
+        if y_row_fields and not y_row_fields.strip().startswith('['):
+            return self.create_text_message('Invalid parameter Row Fields')
+        elif not y_row_fields:
+            y_row_fields = '[]'
+        value_fields = tool_parameters.get('value_fields', '')
+        if not value_fields or not value_fields.strip().startswith('['):
+            return self.create_text_message('Invalid parameter Value Fields')
+        
+        host = tool_parameters.get('host', '')
+        if not host:
+            host = 'https://api.mingdao.com'
+        elif not host.startswith(("http://", "https://")):
+            return self.create_text_message('Invalid parameter Host Address')
+        else:
+            host = f"{host[:-1] if host.endswith('/') else host}/api"
+
+        url = f"{host}/report/getPivotData"
+        headers = {'Content-Type': 'application/json'}
+        payload = {"appKey": appkey, "sign": sign, "worksheetId": worksheet_id, "options": {"showTotal": True}}
+
+        try:
+            x_column_fields = json.loads(x_column_fields)
+            payload['columns'] = x_column_fields
+            y_row_fields = json.loads(y_row_fields)
+            if y_row_fields: payload['rows'] = y_row_fields
+            value_fields = json.loads(value_fields)
+            payload['values'] = value_fields
+            sort_fields = tool_parameters.get('sort_fields', '')
+            if not sort_fields: sort_fields = '[]'
+            sort_fields = json.loads(sort_fields)
+            if sort_fields: payload['options']['sort'] = sort_fields
+            res = httpx.post(url, headers=headers, json=payload, timeout=60)
+            res.raise_for_status()
+            res_json = res.json()
+            if res_json.get('status') != 1:
+                return self.create_text_message(f"Failed to get the worksheet pivot data. {res_json['msg']}")
+            
+            pivot_json = self.generate_pivot_json(res_json['data'])
+            pivot_table = self.generate_pivot_table(res_json['data'])
+            result_type = tool_parameters.get('result_type', '')
+            text = pivot_table if result_type == 'table' else json.dumps(pivot_json, ensure_ascii=False)
+            return self.create_text_message(text)
+        except httpx.RequestError as e:
+            return self.create_text_message(f"Failed to get the worksheet pivot data, request error: {e}")
+        except json.JSONDecodeError as e:
+            return self.create_text_message(f"Failed to parse JSON response: {e}")
+        except Exception as e:
+            return self.create_text_message(f"Failed to get the worksheet pivot data, unexpected error: {e}")
+
+    def generate_pivot_table(self, data: dict[str, Any]) -> str:
+        columns = data['metadata']['columns']
+        rows = data['metadata']['rows']
+        values = data['metadata']['values']
+
+        rows_data = data['data']
+
+        header = ([row['displayName'] for row in rows] if rows else []) + [column['displayName'] for column in columns] + [value['displayName'] for value in values]
+        line = (['---'] * len(rows) if rows else []) + ['---'] * len(columns) + ['--:'] * len(values)
+
+        table = [header, line]
+        for row in rows_data:
+            row_data = [self.replace_pipe(row['rows'][r['controlId']]) for r in rows] if rows else []
+            row_data.extend([self.replace_pipe(row['columns'][column['controlId']]) for column in columns])
+            row_data.extend([self.replace_pipe(str(row['values'][value['controlId']])) for value in values])
+            table.append(row_data)
+
+        return '\n'.join([('|'+'|'.join(row) +'|') for row in table])
+    
+    def replace_pipe(self, text: str) -> str:
+        return text.replace('|', '▏').replace('\n', ' ')
+    
+    def generate_pivot_json(self, data: dict[str, Any]) -> dict:
+        fields = {
+            "x-axis": [
+                {"fieldId": column["controlId"], "fieldName": column["displayName"]}
+                for column in data["metadata"]["columns"]
+            ],
+            "y-axis": [
+                {"fieldId": row["controlId"], "fieldName": row["displayName"]}
+                for row in data["metadata"]["rows"]
+            ] if data["metadata"]["rows"] else [],
+            "values": [
+                {"fieldId": value["controlId"], "fieldName": value["displayName"]}
+                for value in data["metadata"]["values"]
+            ]
+        }
+        # fields = ([
+        #     {"fieldId": row["controlId"], "fieldName": row["displayName"]}
+        #     for row in data["metadata"]["rows"]
+        # ] if data["metadata"]["rows"] else []) + [
+        #     {"fieldId": column["controlId"], "fieldName": column["displayName"]}
+        #     for column in data["metadata"]["columns"]
+        # ] + [
+        #     {"fieldId": value["controlId"], "fieldName": value["displayName"]}
+        #     for value in data["metadata"]["values"]
+        # ]
+        rows = []
+        for row in data["data"]:
+            row_data = row["rows"] if row["rows"] else {}
+            row_data.update(row["columns"])
+            row_data.update(row["values"])
+            rows.append(row_data)
+        return {"fields": fields, "rows": rows, "summary": data["metadata"]["totalRow"]}
@@ -0,0 +1,248 @@
+identity:
+  name: get_worksheet_pivot_data
+  author: Ryan Tian
+  label:
+    en_US: Get Worksheet Pivot Data
+    zh_Hans: 获取工作表统计透视数据
+description:
+  human:
+    en_US: Retrieve statistical pivot table data from a specified worksheet
+    zh_Hans: 从指定的工作表中检索统计透视表数据
+  llm: A tool for extracting statistical pivot table data from a specific worksheet, providing summarized information for analysis and reporting purposes.
+parameters:
+  - name: appkey
+    type: secret-input
+    required: true
+    label:
+      en_US: App Key
+      zh_Hans: App Key
+    human_description:
+      en_US: The AppKey parameter for the HAP application, typically found in the application's API documentation.
+      zh_Hans: HAP 应用的 AppKey 参数，可以从应用 API 文档中查找到
+    llm_description: the AppKey parameter for the HAP application
+    form: form
+
+  - name: sign
+    type: secret-input
+    required: true
+    label:
+      en_US: Sign
+      zh_Hans: Sign
+    human_description:
+      en_US: The Sign parameter for the HAP application
+      zh_Hans: HAP 应用的 Sign 参数
+    llm_description: the Sign parameter for the HAP application
+    form: form
+
+  - name: worksheet_id
+    type: string
+    required: true
+    label:
+      en_US: Worksheet ID
+      zh_Hans: 工作表 ID
+    human_description:
+      en_US: The ID of the specified worksheet
+      zh_Hans: 要获取字段信息的工作表 ID
+    llm_description: The ID of the specified worksheet which to get the fields information.
+    form: llm
+
+  - name: x_column_fields
+    type: string
+    required: true
+    label:
+      en_US: Columns (X-axis)
+      zh_Hans: 统计列字段（X轴）
+    human_description:
+      en_US: The column fields that make up the pivot table's X-axis groups or other dimensions for the X-axis in pivot charts
+      zh_Hans: 组成透视表的统计列或者统计图表的X轴分组及X轴其它维度。JSON 对象数组格式，数组元素属性：controlId-列ID，displayName-显示名称，particleSize(可选)-字段类型是日期或者地区时，通过此参数设置统计维度（日期时间：1-日，2-周，3-月；地区：1-全国，2-省，3-市）
+    llm_description: |
+      This parameter allows you to specify the columns that make up the pivot table's X-axis groups or other dimensions for the X-axis in pivot charts. It is formatted as a JSON array, with its structure defined as follows:
+      ```
+      type XColumnFields = { // X-axis or column object array
+        controlId: string; // fieldId
+        displayName: string; // displayName
+        particleSize?: number; // field type is date or area, set the statistical dimension (date time: 1-day, 2-week, 3-month; area: 1-nation, 2-province, 3-city)
+      }[];
+      ```
+    form: llm
+
+  - name: y_row_fields
+    type: string
+    required: false
+    label:
+      en_US: Rows (Y-axis)
+      zh_Hans: 统计行字段（Y轴）
+    human_description:
+      en_US: The row fields that make up the pivot table's Y-axis groups or other dimensions for the Y-axis in pivot charts
+      zh_Hans: 组成透视表的统计行或者统计图表的Y轴分组及Y轴其它维度。JSON 对象数组格式，数组元素属性：controlId-列ID，displayName-显示名称，particleSize(可选)-字段类型是日期或者地区时，通过此参数设置统计维度（日期时间：1-日，2-周，3-月；地区：1-全国，2-省，3-市）
+    llm_description: |
+      This parameter allows you to specify the rows that make up the pivot table's Y-axis groups or other dimensions for the Y-axis in pivot charts. It is formatted as a JSON array, with its structure defined as follows:
+      ```
+      type YRowFields = { // Y-axis or row object array
+        controlId: string; // fieldId
+        displayName: string; // displayName
+        particleSize?: number; // field type is date or area, set the statistical dimension (date time: 1-day, 2-week, 3-month; area: 1-nation, 2-province, 3-city)
+      }[];
+      ```
+    form: llm
+
+  - name: value_fields
+    type: string
+    required: true
+    label:
+      en_US: Aggregated Values
+      zh_Hans: 统计值字段
+    human_description:
+      en_US: The aggregated value fields in the pivot table
+      zh_Hans: 透视表中经过聚合计算后的统计值字段。JSON 对象数组格式，数组元素属性：controlId-列ID，displayName-显示名称，aggregation-聚合方式（SUM,AVG,MIN,MAX,COUNT）
+    llm_description: |
+      This parameter allows you to specify the aggregated value fields in the pivot table. It is formatted as a JSON array, with its structure defined as follows:
+      ```
+      type ValueFields = { // aggregated value object array
+        controlId: string; // fieldId
+        displayName: string; // displayName
+        aggregation: string; // aggregation method, e.g.: SUM, AVG, MIN, MAX, COUNT
+      }[];
+      ```
+    form: llm
+
+  - name: filters
+    type: string
+    required: false
+    label:
+      en_US: Filter Set
+      zh_Hans: 筛选器组合
+    human_description:
+      en_US: A combination of filters applied to query records, formatted as a JSON array. See the application's API documentation for details on its structure and usage.
+      zh_Hans: 查询记录的筛选条件组合，格式为 JSON 数组，可以从应用 API 文档中了解参数结构详情
+    llm_description: |
+      This parameter allows you to specify a set of conditions that records must meet to be included in the result set. It is formatted as a JSON array, with its structure defined as follows:
+      ```
+      type Filters = { // filter object array
+        controlId: string; // fieldId
+        dataType: number; // fieldTypeId
+        spliceType: number; // condition concatenation method, 1: And, 2: Or
+        filterType: number; // expression type, refer to the <FilterTypeEnum Reference> for enumerable values
+        values?: string[]; // values in the condition, for option-type fields, multiple values can be passed
+        value?: string; // value in the condition, a single value can be passed according to the field type
+        dateRange?: number; // date range, mandatory when filterType is 17 or 18, refer to the <DateRangeEnum Reference> for enumerable values
+        minValue?: string; // minimum value for custom range
+        maxValue?: string; // maximum value for custom range
+        isAsc?: boolean; // ascending order, false: descending, true: ascending
+      }[];
+      ```
+      For option-type fields, if this option field has `options`, then you need to get the corresponding `key` value from the `options` in the current field information via `value`, and pass it into `values` in array format. Do not use the `options` value of other fields as input conditions.
+
+      ### FilterTypeEnum Reference
+      ```
+      Enum Value, Enum Character, Description
+      1, Like, Contains
+      2, Eq, Is (Equal)
+      3, Start, Starts With
+      4, End, Ends With
+      5, NotLike, Does Not Contain
+      6, Ne, Is Not (Not Equal)
+      7, IsEmpty, Empty
+      8, HasValue, Not Empty
+      11, Between, Within Range
+      12, NotBetween, Outside Range
+      13, Gt, Greater Than
+      14, Gte, Greater Than or Equal To
+      15, Lt, Less Than
+      16, Lte, Less Than or Equal To
+      17, DateEnum, Date Is
+      18, NotDateEnum, Date Is Not
+      21, MySelf, Owned by Me
+      22, UnRead, Unread
+      23, Sub, Owned by Subordinate
+      24, RCEq, Associated Field Is
+      25, RCNe, Associated Field Is Not
+      26, ArrEq, Array Equals
+      27, ArrNe, Array Does Not Equal
+      31, DateBetween, Date Within Range (can only be used with minValue and maxValue)
+      32, DateNotBetween, Date Not Within Range (can only be used with minValue and maxValue)
+      33, DateGt, Date Later Than
+      34, DateGte, Date Later Than or Equal To
+      35, DateLt, Date Earlier Than
+      36, DateLte, Date Earlier Than or Equal To
+      ```
+
+      ### DateRangeEnum Reference
+      ```
+      Enum Value, Enum Character, Description
+      1, Today, Today
+      2, Yesterday, Yesterday
+      3, Tomorrow, Tomorrow
+      4, ThisWeek, This Week
+      5, LastWeek, Last Week
+      6, NextWeek, Next Week
+      7, ThisMonth, This Month
+      8, LastMonth, Last Month
+      9, NextMonth, Next Month
+      12, ThisQuarter, This Quarter
+      13, LastQuarter, Last Quarter
+      14, NextQuarter, Next Quarter
+      15, ThisYear, This Year
+      16, LastYear, Last Year
+      17, NextYear, Next Year
+      18, Customize, Custom
+      21, Last7Day, Past 7 Days
+      22, Last14Day, Past 14 Days
+      23, Last30Day, Past 30 Days
+      31, Next7Day, Next 7 Days
+      32, Next14Day, Next 14 Days
+      33, Next33Day, Next 33 Days
+      ```
+    form: llm
+
+  - name: sort_fields
+    type: string
+    required: false
+    label:
+      en_US: Sort Fields
+      zh_Hans: 排序字段
+    human_description:
+      en_US: The fields to used for sorting
+      zh_Hans: 用于确定排序的字段，不超过3个
+    llm_description: |
+      This optional parameter specifies the unique identifier of the fields that will be used to sort the results. It is in the format of an array of JSON objects, and its structure is defined as follows:
+      ```
+      type SortByFields = {
+        controlId: string; // Field ID used for sorting
+        isAsc: boolean; // Sorting direction, true indicates ascending order, false indicates descending order
+      }[];
+      ```
+    form: llm
+
+  - name: host
+    type: string
+    required: false
+    label:
+      en_US: Host Address
+      zh_Hans: 服务器地址
+    human_description:
+      en_US: The address for the privately deployed HAP server.
+      zh_Hans: 私有部署 HAP 服务器地址，公有云无需填写
+    llm_description: the address for the privately deployed HAP server.
+    form: form
+
+  - name: result_type
+    type: select
+    required: true
+    options:
+      - value: table
+        label:
+          en_US: table text
+          zh_Hans: 表格文本
+      - value: json
+        label:
+          en_US: json text
+          zh_Hans: JSON文本
+    default: table
+    label:
+      en_US: Result type
+      zh_Hans: 结果类型
+    human_description:
+      en_US: used for selecting the result type, table styled text or json text
+      zh_Hans: 用于选择结果类型，使用表格格式文本还是JSON格式文本
+    form: form
@@ -0,0 +1,209 @@
+import json
+import re
+from typing import Any, Union
+
+import httpx
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class ListWorksheetRecordsTool(BuiltinTool):
+    def _invoke(self, user_id: str, tool_parameters: dict[str, Any]
+                ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+
+        appkey = tool_parameters.get('appkey', '')
+        if not appkey:
+            return self.create_text_message('Invalid parameter App Key')
+
+        sign = tool_parameters.get('sign', '')
+        if not sign:
+            return self.create_text_message('Invalid parameter Sign')
+
+        worksheet_id = tool_parameters.get('worksheet_id', '')
+        if not worksheet_id:
+            return self.create_text_message('Invalid parameter Worksheet ID')
+
+        host = tool_parameters.get('host', '')
+        if not host:
+            host = 'https://api.mingdao.com'
+        elif not (host.startswith("http://") or host.startswith("https://")):
+            return self.create_text_message('Invalid parameter Host Address')
+        else:
+            host = f"{host[:-1] if host.endswith('/') else host}/api"
+        
+        url_fields = f"{host}/v2/open/worksheet/getWorksheetInfo"
+        headers = {'Content-Type': 'application/json'}
+        payload = {"appKey": appkey, "sign": sign, "worksheetId": worksheet_id}
+
+        field_ids = tool_parameters.get('field_ids', '')
+
+        try:
+            res = httpx.post(url_fields, headers=headers, json=payload, timeout=30)
+            res_json = res.json()
+            if res.is_success:
+                if res_json['error_code'] != 1:
+                    return self.create_text_message("Failed to get the worksheet information. {}".format(res_json['error_msg']))
+                else:
+                    worksheet_name = res_json['data']['name']
+                    fields, schema, table_header = self.get_schema(res_json['data']['controls'], field_ids)
+            else:
+                return self.create_text_message(
+                    f"Failed to get the worksheet information, status code: {res.status_code}, response: {res.text}")
+        except Exception as e:
+            return self.create_text_message("Failed to get the worksheet information, something went wrong: {}".format(e))
+
+        if field_ids:
+            payload['controls'] = [v.strip() for v in field_ids.split(',')] if field_ids else []
+        filters = tool_parameters.get('filters', '')
+        if filters:
+            payload['filters'] = json.loads(filters)
+        sort_id = tool_parameters.get('sort_id', '')
+        sort_is_asc = tool_parameters.get('sort_is_asc', False)
+        if sort_id:
+            payload['sortId'] = sort_id
+            payload['isAsc'] = sort_is_asc
+        limit = tool_parameters.get('limit', 50)
+        payload['pageSize'] = limit
+        page_index = tool_parameters.get('page_index', 1)
+        payload['pageIndex'] = page_index
+        payload['useControlId'] = True
+        payload['listType'] = 1
+
+        url = f"{host}/v2/open/worksheet/getFilterRows"
+        try:
+            res = httpx.post(url, headers=headers, json=payload, timeout=90)
+            res_json = res.json()
+            if res.is_success:
+                if res_json['error_code'] != 1:
+                    return self.create_text_message("Failed to get the records. {}".format(res_json['error_msg']))
+                else:
+                    result = {
+                        "fields": fields,
+                        "rows": [],
+                        "total": res_json.get("data", {}).get("total"),
+                        "payload": {key: payload[key] for key in ['worksheetId', 'controls', 'filters', 'sortId', 'isAsc', 'pageSize', 'pageIndex'] if key in payload}
+                    }
+                    rows = res_json.get("data", {}).get("rows", [])
+                    result_type = tool_parameters.get('result_type', '')
+                    if not result_type: result_type = 'table'
+                    if result_type == 'json':
+                        for row in rows:
+                            result['rows'].append(self.get_row_field_value(row, schema))
+                        return self.create_text_message(json.dumps(result, ensure_ascii=False))
+                    else:
+                        result_text = f"Found {result['total']} rows in worksheet \"{worksheet_name}\"."
+                        if result['total'] > 0:
+                            result_text += f" The following are {result['total'] if result['total'] < limit else limit} pieces of data presented in a table format:\n\n{table_header}"
+                            for row in rows:
+                                result_values = []
+                                for f in fields:
+                                    result_values.append(self.handle_value_type(row[f['fieldId']], schema[f['fieldId']]))
+                                result_text += '\n|'+'|'.join(result_values)+'|'
+                        return self.create_text_message(result_text)
+            else:
+                return self.create_text_message(
+                    f"Failed to get the records, status code: {res.status_code}, response: {res.text}")
+        except Exception as e:
+            return self.create_text_message("Failed to get the records, something went wrong: {}".format(e))
+
+
+    def get_row_field_value(self, row: dict, schema: dict):
+        row_value = {"rowid": row["rowid"]}
+        for field in schema:
+            row_value[field] = self.handle_value_type(row[field], schema[field])
+        return row_value
+
+
+    def get_schema(self, controls: list, fieldids: str): 
+        allow_fields = {v.strip() for v in fieldids.split(',')} if fieldids else set()
+        fields = []
+        schema = {}
+        field_names = []
+        for control in controls:
+            control_type_id = self.get_real_type_id(control)
+            if (control_type_id in self._get_ignore_types()) or (allow_fields and not control['controlId'] in allow_fields):
+                continue
+            else:
+                fields.append({'fieldId': control['controlId'], 'fieldName': control['controlName']})
+                schema[control['controlId']] = {'typeId': control_type_id, 'options': self.set_option(control)}
+                field_names.append(control['controlName'])
+        if (not allow_fields or ('ctime' in allow_fields)):
+            fields.append({'fieldId': 'ctime', 'fieldName': 'Created Time'})
+            schema['ctime'] = {'typeId': 16, 'options': {}}
+            field_names.append("Created Time")
+        fields.append({'fieldId':'rowid', 'fieldName': 'Record Row ID'})
+        schema['rowid'] = {'typeId': 2, 'options': {}}
+        field_names.append("Record Row ID")
+        return fields, schema, '|'+'|'.join(field_names)+'|\n|'+'---|'*len(field_names)
+    
+    def get_real_type_id(self, control: dict) -> int:
+        return control['sourceControlType'] if control['type'] == 30 else control['type']
+    
+    def set_option(self, control: dict) -> dict:
+        options = {}
+        if control.get('options'):
+            options = {option['key']: option['value'] for option in control['options']}
+        elif control.get('advancedSetting', {}).get('itemnames'):
+            try:
+                itemnames = json.loads(control['advancedSetting']['itemnames'])
+                options = {item['key']: item['value'] for item in itemnames}
+            except json.JSONDecodeError:
+                pass
+        return options
+
+    def _get_ignore_types(self):
+        return {14, 21, 22, 34, 42, 43, 45, 47, 49, 10010}
+    
+    def handle_value_type(self, value, field):
+        type_id = field.get("typeId")
+        if type_id == 10:
+            value = value if isinstance(value, str) else "、".join(value)
+        elif type_id in [28, 36]:
+            value = field.get("options", {}).get(value, value)
+        elif type_id in [26, 27, 48, 14]:
+            value = self.process_value(value)
+        elif type_id in [35, 29]:
+            value = self.parse_cascade_or_associated(field, value)
+        elif type_id == 40:
+            value = self.parse_location(value)
+        return self.rich_text_to_plain_text(value) if value else ''
+
+    def process_value(self, value):
+        if isinstance(value, str):
+            if value.startswith("[{\"accountId\""):
+                value = json.loads(value)
+                value = ', '.join([item['fullname'] for item in value])
+            elif value.startswith("[{\"departmentId\""):
+                value = json.loads(value)
+                value = '、'.join([item['departmentName'] for item in value])
+            elif value.startswith("[{\"organizeId\""):
+                value = json.loads(value)
+                value = '、'.join([item['organizeName'] for item in value])
+            elif value.startswith("[{\"file_id\""):
+                value = ''
+            elif value == '[]':
+                value = ''
+        elif hasattr(value, 'accountId'):
+            value = value['fullname']
+        return value
+
+    def parse_cascade_or_associated(self, field, value):
+        if (field['typeId'] == 35 and value.startswith('[')) or (field['typeId'] == 29 and value.startswith('[{')):
+            value = json.loads(value)
+            value = value[0]['name'] if len(value) > 0 else ''
+        else:
+            value = ''
+        return value
+
+    def parse_location(self, value):
+        if len(value) > 10:
+            parsed_value = json.loads(value)
+            value = parsed_value.get("address", "")
+        else:
+            value = ""
+        return value
+
+    def rich_text_to_plain_text(self, rich_text):
+        text = re.sub(r'<[^>]+>', '', rich_text) if '<' in rich_text else rich_text
+        return text.replace("|", "▏").replace("\n", " ")
@@ -0,0 +1,226 @@
+identity:
+  name: list_worksheet_records
+  author: Ryan Tian
+  label:
+    en_US: List Worksheet Records
+    zh_Hans: 查询工作表记录数据
+description:
+  human:
+    en_US: List records from the worksheet
+    zh_Hans: 查询工作表的记录列表数据，一次最多1000行，可分页获取
+  llm: A tool to retrieve record data from the specific worksheet.
+parameters:
+  - name: appkey
+    type: secret-input
+    required: true
+    label:
+      en_US: App Key
+      zh_Hans: App Key
+    human_description:
+      en_US: The AppKey parameter for the HAP application, typically found in the application's API documentation.
+      zh_Hans: HAP 应用的 AppKey 参数，可以从应用 API 文档中查找到
+    llm_description: the AppKey parameter for the HAP application
+    form: form
+
+  - name: sign
+    type: secret-input
+    required: true
+    label:
+      en_US: Sign
+      zh_Hans: Sign
+    human_description:
+      en_US: The Sign parameter for the HAP application
+      zh_Hans: HAP 应用的 Sign 参数
+    llm_description: the Sign parameter for the HAP application
+    form: form
+
+  - name: worksheet_id
+    type: string
+    required: true
+    label:
+      en_US: Worksheet ID
+      zh_Hans: 工作表 ID
+    human_description:
+      en_US: The ID of the worksheet from which to retrieve record data
+      zh_Hans: 要获取记录数据的工作表 ID
+    llm_description: This parameter specifies the ID of the worksheet where the records are stored.
+    form: llm
+
+  - name: field_ids
+    type: string
+    required: false
+    label:
+      en_US: Field IDs
+      zh_Hans: 字段 ID 列表
+    human_description:
+      en_US: A comma-separated list of field IDs whose data to retrieve. If not provided, all fields' data will be fetched
+      zh_Hans: 要获取记录数据的字段 ID，多个 ID 间用英文逗号隔开，不传此参数则将获取所有字段的数据
+    llm_description: This optional parameter lets you specify a comma-separated list of field IDs. Unless the user explicitly requests to output the specified field in the question, this parameter should usually be omitted. If this parameter is omitted, the API will return data for all fields by default. When provided, only the data associated with these fields will be included in the response.
+    form: llm
+
+  - name: filters
+    type: string
+    required: false
+    label:
+      en_US: Filter Set
+      zh_Hans: 筛选器组合
+    human_description:
+      en_US: A combination of filters applied to query records, formatted as a JSON array. See the application's API documentation for details on its structure and usage.
+      zh_Hans: 查询记录的筛选条件组合，格式为 JSON 数组，可以从应用 API 文档中了解参数结构详情
+    llm_description: |
+      This parameter allows you to specify a set of conditions that records must meet to be included in the result set. It is formatted as a JSON array, with its structure defined as follows:
+      ```
+      type Filters = { // filter object array
+        controlId: string; // fieldId
+        dataType: number; // fieldTypeId
+        spliceType: number; // condition concatenation method, 1: And, 2: Or
+        filterType: number; // expression type, refer to the <FilterTypeEnum Reference> for enumerable values
+        values?: string[]; // values in the condition, for option-type fields, multiple values can be passed
+        value?: string; // value in the condition, a single value can be passed according to the field type
+        dateRange?: number; // date range, mandatory when filterType is 17 or 18, refer to the <DateRangeEnum Reference> for enumerable values
+        minValue?: string; // minimum value for custom range
+        maxValue?: string; // maximum value for custom range
+        isAsc?: boolean; // ascending order, false: descending, true: ascending
+      }[];
+      ```
+      For option-type fields, if this option field has `options`, then you need to get the corresponding `key` value from the `options` in the current field information via `value`, and pass it into `values` in array format. Do not use the `options` value of other fields as input conditions.
+
+      ### FilterTypeEnum Reference
+      ```
+      Enum Value, Enum Character, Description
+      1, Like, Contains(Include)
+      2, Eq, Is (Equal)
+      3, Start, Starts With
+      4, End, Ends With
+      5, NotLike, Does Not Contain(Not Include)
+      6, Ne, Is Not (Not Equal)
+      7, IsEmpty, Empty
+      8, HasValue, Not Empty
+      11, Between, Within Range(Belong to)
+      12, NotBetween, Outside Range(Not belong to)
+      13, Gt, Greater Than
+      14, Gte, Greater Than or Equal To
+      15, Lt, Less Than
+      16, Lte, Less Than or Equal To
+      17, DateEnum, Date Is
+      18, NotDateEnum, Date Is Not
+      24, RCEq, Associated Field Is
+      25, RCNe, Associated Field Is Not
+      26, ArrEq, Array Equals
+      27, ArrNe, Array Does Not Equal
+      31, DateBetween, Date Within Range (can only be used with minValue and maxValue)
+      32, DateNotBetween, Date Not Within Range (can only be used with minValue and maxValue)
+      33, DateGt, Date Later Than
+      34, DateGte, Date Later Than or Equal To
+      35, DateLt, Date Earlier Than
+      36, DateLte, Date Earlier Than or Equal To
+      ```
+
+      ### DateRangeEnum Reference
+      ```
+      Enum Value, Enum Character, Description
+      1, Today, Today
+      2, Yesterday, Yesterday
+      3, Tomorrow, Tomorrow
+      4, ThisWeek, This Week
+      5, LastWeek, Last Week
+      6, NextWeek, Next Week
+      7, ThisMonth, This Month
+      8, LastMonth, Last Month
+      9, NextMonth, Next Month
+      12, ThisQuarter, This Quarter
+      13, LastQuarter, Last Quarter
+      14, NextQuarter, Next Quarter
+      15, ThisYear, This Year
+      16, LastYear, Last Year
+      17, NextYear, Next Year
+      18, Customize, Custom
+      21, Last7Day, Past 7 Days
+      22, Last14Day, Past 14 Days
+      23, Last30Day, Past 30 Days
+      31, Next7Day, Next 7 Days
+      32, Next14Day, Next 14 Days
+      33, Next33Day, Next 33 Days
+      ```
+    form: llm
+
+  - name: sort_id
+    type: string
+    required: false
+    label:
+      en_US: Sort Field ID
+      zh_Hans: 排序字段 ID
+    human_description:
+      en_US: The ID of the field used for sorting
+      zh_Hans: 用以排序的字段 ID
+    llm_description: This optional parameter specifies the unique identifier of the field that will be used to sort the results. It should be set to the ID of an existing field within your data structure.
+    form: llm
+
+  - name: sort_is_asc
+    type: boolean
+    required: false
+    label:
+      en_US: Ascending Order
+      zh_Hans: 是否升序排列
+    human_description:
+      en_US: Determines whether the sorting is in ascending (true) or descending (false) order
+      zh_Hans: 排序字段的排序方式：true-升序，false-降序
+    llm_description: This optional parameter controls the direction of the sort. If set to true, the results will be sorted in ascending order; if false, they will be sorted in descending order.
+    form: llm
+
+  - name: limit
+    type: number
+    required: false
+    label:
+      en_US: Record Limit
+      zh_Hans: 记录数量限制
+    human_description:
+      en_US: The maximum number of records to retrieve
+      zh_Hans: 要获取的记录数量限制条数
+    llm_description: This optional parameter allows you to specify the maximum number of records that should be returned in the result set. When retrieving paginated record data, this parameter indicates the number of rows to fetch per page, and must be used in conjunction with the `page_index` parameter.
+    form: llm
+
+  - name: page_index
+    type: number
+    required: false
+    label:
+      en_US: Page Index
+      zh_Hans: 页码
+    human_description:
+      en_US: The page number when paginating through a list of records
+      zh_Hans: 分页读取记录列表时的页码
+    llm_description: This parameter is used when you need to paginate through a large set of records. The default value is 1, which refers to the first page. When it is used, the meaning of the `limit` parameter becomes the number of records per page.
+    form: llm
+
+  - name: host
+    type: string
+    required: false
+    label:
+      en_US: Host Address
+      zh_Hans: 服务器地址
+    human_description:
+      en_US: The address for the privately deployed HAP server.
+      zh_Hans: 私有部署 HAP 服务器地址，公有云无需填写
+    llm_description: the address for the privately deployed HAP server.
+    form: form
+
+  - name: result_type
+    type: select
+    required: true
+    options:
+      - value: table
+        label:
+          en_US: table text
+          zh_Hans: 表格文本
+      - value: json
+        label:
+          en_US: json text
+          zh_Hans: JSON文本
+    default: table
+    label:
+      en_US: Result type
+      zh_Hans: 结果类型
+    human_description:
+      en_US: used for selecting the result type, table styled text or json text
+      zh_Hans: 用于选择结果类型，使用表格格式文本还是JSON格式文本
+    form: form
@@ -0,0 +1,82 @@
+import json
+from typing import Any, Union
+
+import httpx
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class ListWorksheetsTool(BuiltinTool):
+
+    def _invoke(self, user_id: str, tool_parameters: dict[str, Any]
+                ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+
+        appkey = tool_parameters.get('appkey', '')
+        if not appkey:
+            return self.create_text_message('Invalid parameter App Key')
+        sign = tool_parameters.get('sign', '')
+        if not sign:
+            return self.create_text_message('Invalid parameter Sign')
+        
+        host = tool_parameters.get('host', '')
+        if not host:
+            host = 'https://api.mingdao.com'
+        elif not (host.startswith("http://") or host.startswith("https://")):
+            return self.create_text_message('Invalid parameter Host Address')
+        else:
+            host = f"{host[:-1] if host.endswith('/') else host}/api"
+        url = f"{host}/v1/open/app/get"
+
+        result_type = tool_parameters.get('result_type', '')
+        if not result_type:
+            result_type = 'table'
+
+        headers = { 'Content-Type': 'application/json' }
+        params = { "appKey": appkey, "sign": sign, }
+        try:
+            res = httpx.get(url, headers=headers, params=params, timeout=30)
+            res_json = res.json()
+            if res.is_success:
+                if res_json['error_code'] != 1:
+                    return self.create_text_message("Failed to access the application. {}".format(res_json['error_msg']))
+                else:
+                    if result_type == 'json':
+                        worksheets = []
+                        for section in res_json['data']['sections']:
+                            worksheets.extend(self._extract_worksheets(section, result_type))
+                        return self.create_text_message(text=json.dumps(worksheets, ensure_ascii=False))
+                    else:
+                        worksheets = '|worksheetId|worksheetName|description|\n|---|---|---|'
+                        for section in res_json['data']['sections']:
+                            worksheets += self._extract_worksheets(section, result_type)
+                        return self.create_text_message(worksheets)
+
+            else:
+                return self.create_text_message(
+                    f"Failed to list worksheets, status code: {res.status_code}, response: {res.text}")
+        except Exception as e:
+            return self.create_text_message("Failed to list worksheets, something went wrong: {}".format(e))
+
+    def _extract_worksheets(self, section, type):
+        items = []
+        tables = ''
+        for item in section.get('items', []):
+            if item.get('type') == 0 and (not 'notes' in item or item.get('notes') != 'NO'):
+                if type == 'json':
+                    filtered_item = {
+                        'id': item['id'],
+                        'name': item['name'],
+                        'notes': item.get('notes', '')
+                    }
+                    items.append(filtered_item)
+                else:
+                    tables += f"\n|{item['id']}|{item['name']}|{item.get('notes', '')}|"
+
+        for child_section in section.get('childSections', []):
+            if type == 'json':
+                items.extend(self._extract_worksheets(child_section, 'json'))
+            else:
+                tables += self._extract_worksheets(child_section, 'table')
+        
+        return items if type == 'json' else tables
@@ -0,0 +1,68 @@
+identity:
+  name: list_worksheets
+  author: Ryan Tian
+  label:
+    en_US: List Worksheets
+    zh_Hans: 获取应用下所有工作表
+description:
+  human:
+    en_US: List worksheets within an application
+    zh_Hans: 获取应用下的所有工作表和说明信息
+  llm: A tool to list worksheets info within an application, imported parameter is AppKey and Sign of the application.
+parameters:
+  - name: appkey
+    type: secret-input
+    required: true
+    label:
+      en_US: App Key
+      zh_Hans: App Key
+    human_description:
+      en_US: The AppKey parameter for the HAP application, typically found in the application's API documentation.
+      zh_Hans: HAP 应用的 AppKey 参数，可以从应用 API 文档中查找到
+    llm_description: the AppKey parameter for the HAP application
+    form: form
+
+  - name: sign
+    type: secret-input
+    required: true
+    label:
+      en_US: Sign
+      zh_Hans: Sign
+    human_description:
+      en_US: The Sign parameter for the HAP application
+      zh_Hans: HAP 应用的 Sign 参数
+    llm_description: the Sign parameter for the HAP application
+    form: form
+
+  - name: host
+    type: string
+    required: false
+    label:
+      en_US: Host Address
+      zh_Hans: 服务器地址
+    human_description:
+      en_US: The address for the privately deployed HAP server.
+      zh_Hans: 私有部署 HAP 服务器地址，公有云无需填写
+    llm_description: the address for the privately deployed HAP server.
+    form: form
+
+  - name: result_type
+    type: select
+    required: true
+    options:
+      - value: table
+        label:
+          en_US: table text
+          zh_Hans: 表格文本
+      - value: json
+        label:
+          en_US: json text
+          zh_Hans: JSON文本
+    default: table
+    label:
+      en_US: Result type
+      zh_Hans: 结果类型
+    human_description:
+      en_US: used for selecting the result type, table styled text or json text
+      zh_Hans: 用于选择结果类型，使用表格格式文本还是JSON格式文本
+    form: form
@@ -0,0 +1,56 @@
+import json
+from typing import Any, Union
+
+import httpx
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class UpdateWorksheetRecordTool(BuiltinTool):
+
+    def _invoke(self, user_id: str, tool_parameters: dict[str, Any]
+                ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+        
+        appkey = tool_parameters.get('appkey', '')
+        if not appkey:
+            return self.create_text_message('Invalid parameter App Key')
+        sign = tool_parameters.get('sign', '')
+        if not sign:
+            return self.create_text_message('Invalid parameter Sign')
+        worksheet_id = tool_parameters.get('worksheet_id', '')
+        if not worksheet_id:
+            return self.create_text_message('Invalid parameter Worksheet ID')
+        row_id = tool_parameters.get('row_id', '')
+        if not row_id:
+            return self.create_text_message('Invalid parameter Record Row ID')
+        record_data = tool_parameters.get('record_data', '')
+        if not record_data:
+            return self.create_text_message('Invalid parameter Record Row Data')
+        
+        host = tool_parameters.get('host', '')
+        if not host:
+            host = 'https://api.mingdao.com'
+        elif not host.startswith(("http://", "https://")):
+            return self.create_text_message('Invalid parameter Host Address')
+        else:
+            host = f"{host[:-1] if host.endswith('/') else host}/api"
+
+        url = f"{host}/v2/open/worksheet/editRow"
+        headers = {'Content-Type': 'application/json'}
+        payload = {"appKey": appkey, "sign": sign, "worksheetId": worksheet_id, "rowId": row_id}
+
+        try:
+            payload['controls'] = json.loads(record_data)
+            res = httpx.post(url, headers=headers, json=payload, timeout=60)
+            res.raise_for_status()
+            res_json = res.json()
+            if res_json.get('error_code') != 1:
+                return self.create_text_message(f"Failed to update the record. {res_json['error_msg']}")
+            return self.create_text_message("Record updated successfully.")
+        except httpx.RequestError as e:
+            return self.create_text_message(f"Failed to update the record, request error: {e}")
+        except json.JSONDecodeError as e:
+            return self.create_text_message(f"Failed to parse JSON response: {e}")
+        except Exception as e:
+            return self.create_text_message(f"Failed to update the record, unexpected error: {e}")
@@ -0,0 +1,90 @@
+identity:
+  name: update_worksheet_record
+  author: Ryan Tian
+  label:
+    en_US: Update Worksheet Record
+    zh_Hans: 更新指定的一条工作表记录
+description:
+  human:
+    en_US: Updates a single record in a worksheet based on the specified record row ID
+    zh_Hans: 根据指定的记录ID更新一条工作表记录数据
+  llm: A tool to modify existing information within a particular record of a worksheet by referencing its unique identifier.
+parameters:
+  - name: appkey
+    type: secret-input
+    required: true
+    label:
+      en_US: App Key
+      zh_Hans: App Key
+    human_description:
+      en_US: The AppKey parameter for the HAP application, typically found in the application's API documentation.
+      zh_Hans: HAP 应用的 AppKey 参数，可以从应用 API 文档中查找到
+    llm_description: the AppKey parameter for the HAP application
+    form: form
+
+  - name: sign
+    type: secret-input
+    required: true
+    label:
+      en_US: Sign
+      zh_Hans: Sign
+    human_description:
+      en_US: The Sign parameter for the HAP application
+      zh_Hans: HAP 应用的 Sign 参数
+    llm_description: the Sign parameter for the HAP application
+    form: form
+
+  - name: worksheet_id
+    type: string
+    required: true
+    label:
+      en_US: Worksheet ID
+      zh_Hans: 工作表 ID
+    human_description:
+      en_US: The ID of the specified worksheet
+      zh_Hans: 要获取字段信息的工作表 ID
+    llm_description: The ID of the specified worksheet which to get the fields information.
+    form: llm
+
+  - name: row_id
+    type: string
+    required: true
+    label:
+      en_US: Record Row ID
+      zh_Hans: 记录 ID
+    human_description:
+      en_US: The row ID of the specified record
+      zh_Hans: 要更新的记录 ID
+    llm_description: The row ID of the specified record which to be updated.
+    form: llm
+
+  - name: record_data
+    type: string
+    required: true
+    label:
+      en_US: Record Row Data
+      zh_Hans: 记录数据
+    human_description:
+      en_US: The fields with data of the specified record
+      zh_Hans: 要更新的记录数据，JSON 对象数组格式。数组元素属性：controlId-字段ID，value-字段值
+    llm_description: |
+      The fields with data of the specified record which to be updated. It is in the format of an array of JSON objects, and the structure is defined as follows:
+      ```
+      type RowData = {
+        controlId: string; // Field ID to be updated
+        value: string; // Field value to be updated
+      }[];
+      ```
+    form: llm
+
+  - name: host
+    type: string
+    required: false
+    label:
+      en_US: Host Address
+      zh_Hans: 服务器地址
+    human_description:
+      en_US: The address for the privately deployed HAP server.
+      zh_Hans: 私有部署 HAP 服务器地址，公有云无需填写
+    llm_description: the address for the privately deployed HAP server.
+    form: form
@@ -1,4 +1,5 @@
 from abc import ABC, abstractmethod
+from collections.abc import Mapping
 from copy import deepcopy
 from enum import Enum
 from typing import Any, Optional, Union
@@ -190,8 +191,9 @@ class Tool(BaseModel, ABC):

        return result

-    def invoke(self, user_id: str, tool_parameters: dict[str, Any]) -> list[ToolInvokeMessage]:
+    def invoke(self, user_id: str, tool_parameters: Mapping[str, Any]) -> list[ToolInvokeMessage]:
        # update tool_parameters
+        # TODO: Fix type error.
        if self.runtime.runtime_parameters:
            tool_parameters.update(self.runtime.runtime_parameters)

@@ -208,7 +210,7 @@ class Tool(BaseModel, ABC):

        return result

-    def _transform_tool_parameters_type(self, tool_parameters: dict[str, Any]) -> dict[str, Any]:
+    def _transform_tool_parameters_type(self, tool_parameters: Mapping[str, Any]) -> dict[str, Any]:
        """
        Transform tool parameters type
        """
@@ -241,7 +243,7 @@ class Tool(BaseModel, ABC):

            :return: the runtime parameters
        """
-        return self.parameters
+        return self.parameters or []
    
    def get_all_runtime_parameters(self) -> list[ToolParameter]:
        """
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
takatostandGitHub	49729647ea	bump to 0.6.15 (#6592 )	2024-07-23 22:46:42 +08:00
-LAN-andGitHub	85a883e281	fix(variables): NoneVariable should inherit from NoneSegment. (#6584 )	2024-07-23 21:46:08 +08:00
JoeandGitHub	8123a00e97	feat: update prompt generate (#6516 )	2024-07-23 19:52:14 +08:00
JoelandGitHub	0f6a064c08	chore: enchance auto generate prompt (#6564 )	2024-07-23 19:51:38 +08:00
-LAN-andGitHub	2bc0632d0d	fix(segments): Support NoneType. (#6581 )	2024-07-23 17:59:32 +08:00
Lance MaoandGitHub	75445a0c66	fix audio not working during development due to react's useEffect wil be triggered twice (#6126 )	2024-07-23 17:24:29 +08:00
JoelandGitHub	6a9d202414	chore: layout UI upgrade (#6577 )	2024-07-23 17:11:02 +08:00
-LAN-andGitHub	ad7552ea8d	fix(api/core/workflow/nodes/llm/llm_node.py): Fix LLM Node error. (#6576 )	2024-07-23 17:09:16 +08:00
非法操作andGitHub	c0ada940bd	fix: tool params not work as expected when develop a tool (#6550 )	2024-07-23 17:00:39 +08:00
takatostandGitHub	1690788827	fix: name 'current_app' is not defined in recommended_app_service (#6574 )	2024-07-23 16:48:21 +08:00
Lance MaoandGitHub	7c55c39085	feat: add tencent asr (#6091 )	2024-07-23 16:38:39 +08:00
非法操作andGitHub	f17d4fe412	fix: extract only `like` feedback to caculate User Satisfaction (#6553 )	2024-07-23 16:32:36 +08:00
-LAN-andGitHub	f019bc4bd7	feat(variables): Support `to_object`. (#6572 )	2024-07-23 16:22:06 +08:00
-LAN-andGitHub	cfc408095c	fix(api/nodes): Fallback to `get_any` in some nodes that use object or array. (#6566 )	2024-07-23 15:51:07 +08:00
takatostandGitHub	6b5fac3004	fix: fetch context error in llm node (#6562 )	2024-07-23 15:04:51 +08:00
崔亮andGitHub	0569c547ee	fix the issue of MILVUS_DATABASE has no effect. (#6424 )	2024-07-23 15:03:55 +08:00
tmuifeandGitHub	06fc1bce9e	Add search by full text when using Oracle23ai as vector DB (#6559 )	2024-07-23 15:03:21 +08:00
Sangmin AhnandGitHub	093b8ca475	fix: escape double quotation marks in the vector DB search query (#6506 )	2024-07-23 15:02:25 +08:00
Ryan Tian GitHub takatost	5fcc2caeed	feat: add Mingdao HAP tool, implemented read and maintain HAP application worksheet data. (#6257 ) Co-authored-by: takatost <[email protected]>	2024-07-23 14:34:19 +08:00
guogeer GitHub jinqi.guo	f30a51e673	fix: chat flow chat with annotation or moderation but answer empty (#6202 ) Co-authored-by: jinqi.guo <[email protected]>	2024-07-23 14:13:58 +08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	642723d09e	chore(deps): bump sentry-sdk from 1.39.2 to 2.8.0 in /api (#6517 ) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-07-23 13:48:23 +08:00
JoelandGitHub	155e708540	Revert "chore: improve prompt auto generator" (#6556 )	2024-07-23 13:35:35 +08:00
JoelandGitHub	d726473c6d	Revert "chore: use node specify llm to auto generate prompt" (#6555 )	2024-07-23 13:31:32 +08:00
crazywoolaandGitHub	e80412df23	feat: renanme template (#6547 )	2024-07-23 10:07:54 +08:00
crazywoolaandGitHub	66765acf00	Update help.yml (#6546 )	2024-07-23 10:01:19 +08:00
crazywoolaandGitHub	7208ea1da9	fix: template (#6545 )	2024-07-23 09:58:47 +08:00
crazywoolaandGitHub	5e2f3ec6f0	update discussion template (#6544 )	2024-07-23 09:44:27 +08:00
-LAN-andGitHub	cd7fa8027a	fix(api/core/model_manager.py): Avoid mutation during iteration. (#6536 )	2024-07-22 22:58:22 +08:00
-LAN-andGitHub	617847e3c0	fix(api/services/app_generate_service.py): Remove wrong type hints. (#6535 )	2024-07-22 22:58:07 +08:00
crazywoolaandGitHub	71a7211411	Feat/add email support for pro and team (#6533 )	2024-07-22 19:56:46 +08:00
JoelandGitHub	dc7335cdf8	chore: use node specify llm to auto generate prompt (#6525 )	2024-07-22 18:16:33 +08:00
crazywoolaandGitHub	a7c1e4c7ae	chore: remove support email from readme (#6530 )	2024-07-22 17:23:19 +08:00
zxhlyhandGitHub	87594008f8	fix: iteration node bg color (#6523 )	2024-07-22 15:43:24 +08:00
-LAN-GitHub JzoNg	5e6fc58db3	Feat/environment variables in workflow (#6515 ) Co-authored-by: JzoNg <[email protected]>	2024-07-22 15:29:39 +08:00
Jason TanandGitHub	87d583f454	fix: privilege for editor role (#6521 )	2024-07-22 15:01:25 +08:00
BenjaminandGitHub	a67831773f	refactor: handle missing position file gracefully (#6513 )	2024-07-22 13:24:32 +08:00
Jian YuandGitHub	5b89b6fe2d	allow custom base_url of dify api server (#6510 )	2024-07-22 13:24:24 +08:00
JoelandGitHub	a6350daa02	chore: improve prompt auto generator (#6514 )	2024-07-22 11:44:12 +08:00
sinoandGitHub	dfb6f4fec6	fix: extract tool calls correctly while arguments is empty (#6503 )	2024-07-22 07:43:18 +08:00
JyongandGitHub	f38034e455	clean vector collection redis cache (#6494 )	2024-07-21 15:09:09 +08:00
Shoya SHIRAKIandGitHub	c57b3931d5	refactor(api): switch to dify_config in controllers/console (#6485 )	2024-07-21 01:11:40 +08:00
JyongandGitHub	f73a3a58ae	update delete embeddings by id (#6489 )	2024-07-20 09:04:21 +08:00
JyongandGitHub	1e0e573165	update clean embedding cache query logic (#6483 )	2024-07-20 01:29:25 +08:00
JoeandGitHub	27e08a8e2e	Fix/extra table tracing app config (#6487 )	2024-07-20 00:53:31 +08:00