From a4dcd20d4cd89f84842bf605716afc10f22a0902 Mon Sep 17 00:00:00 2001 From: Liam Sommer Date: Mon, 20 Jul 2026 13:37:28 +0200 Subject: [PATCH 1/9] SAS-1901: updated docs for streaming --- content/en/docs/genai/v2/reference-guide/agent-commons.md | 6 +++++- .../en/docs/genai/v2/reference-guide/conversational-ui.md | 8 ++++++++ content/en/docs/genai/v2/reference-guide/genai-commons.md | 5 ++++- 3 files changed, 17 insertions(+), 2 deletions(-) diff --git a/content/en/docs/genai/v2/reference-guide/agent-commons.md b/content/en/docs/genai/v2/reference-guide/agent-commons.md index f9b48aab861..b2bcbc434a5 100644 --- a/content/en/docs/genai/v2/reference-guide/agent-commons.md +++ b/content/en/docs/genai/v2/reference-guide/agent-commons.md @@ -115,6 +115,10 @@ Users can create two types of agents: {{< figure src="/attachments/genai/agentcommons/agent_builder_task.png" alt="" >}} +#### Control model behavior {#model-settings} + +By clicking the icon next to the selected model, you can control model behavior by configuring the temperature, top P, and maximum number of tokens on the version. If the selected model and connector support it (currently only Mendix Cloud GenAI), you can also enable backend streaming to use the model's streaming API, and frontend streaming to send generated chunks to the user interface as they become available (see [ConversationalUI](/agents/agents-kit-2/reference-guide/conversational-ui/#streaming) documentation for more). UI streaming is supported only for chat agents and for a chat interface from the Conversational UI module. + #### Defining Context Entity {#define-context-entity} If your agent's prompt includes variables, your app must define an entity with attributes that match the variable names. An object of this entity serves as the context object, which holds the context data that will be passed when the **call agent** operation is triggered. For more details, see the [Use the agent in the app logic](#app-logic) section below. @@ -161,7 +165,7 @@ Note that [user access approval](/agents/agents-kit-2/reference-guide/commons/#e #### Testing and Refining the Agent -While writing the [system prompt](/agents/glossary/#system-prompt) (for both chat and task types) or the [user prompt](/agents/glossary/#user-prompt) (only for the task type), the prompt engineer can include variables by enclosing them in double braces, for example, `{{variable}}`. The actual values of these placeholders are typically known at runtime based on the user's page context. +While writing the [system prompt](/agents/glossary/#system-prompt) (for both and task types) or the [user prompt](/agents/glossary/#user-prompt) (only for the task type), the prompt engineer can include variables by enclosing them in double braces, for example, `{{variable}}`. The actual values of these placeholders are typically known at runtime based on the user's page context. To test the behavior of the prompts, a test can be executed. The prompt engineer must provide test values for all variables defined in the prompts. Additionally, multiple sets of test values for the variables can be defined and run in bulk. Based on the test results, the prompt engineer can add, remove, or rephrase certain parts of the prompt. ### Using the Agent in the App Logic {#app-logic} diff --git a/content/en/docs/genai/v2/reference-guide/conversational-ui.md b/content/en/docs/genai/v2/reference-guide/conversational-ui.md index 4f824f176b5..17708a7da0b 100644 --- a/content/en/docs/genai/v2/reference-guide/conversational-ui.md +++ b/content/en/docs/genai/v2/reference-guide/conversational-ui.md @@ -220,6 +220,14 @@ This changes how action microflows are used, because they are called each time a If no [user-visibility](/agents/agents-kit-2/reference-guide/commons/#enum-useraccessapproval) is configured for tools and you want to avoid storing tool messages, you can change the Boolean `SaveToolCallHistory` to *false* on the [Request](/agents/agents-kit-2/reference-guide/commons/#request). Note that [knowledge base retrievals](/agents/agents-kit-2/reference-guide/commons/#add-knowledge-base-to-request) are set to `HiddenForUser` by default. +### Streaming {#streaming} + +Conversational UI chats can also be streamed to the UI, which expands the assistant response as the LLM generates new content. To use streaming, use [AgentCommons](/agents/agents-kit-2/reference-guide/agent-commons/) to configure a chat agent in the [model settings](/agents/agents-kit-2/reference-guide/agent-commons/#model-settings), enable the applicable streaming options there, and use that agent when calling the `New Chat for Agent` action and in your action microflow. Not all models and connectors support streaming. Currently, streaming is supported only for Mendix Cloud GenAI. + +Note that the following limitations apply for streaming because the logic is called in isolated transactions: +* Tool microflows can only read and change objects in the database, and user interaction is limited, for example, opening a page. +* Similarly, action microflows can only read and change objects in the database. + ### Human in the Loop {#human-in-the-loop} When using the [Function Calling](/agents/function-calling/) pattern by adding tools to the request, you can control when those tools get executed and if they are visible to the user by setting [user access approval](/agents/agents-kit-2/reference-guide/commons/#enum-useraccessapproval) per tool. [Human in the loop](/agents/glossary/#human-in-the-loop) describes a pattern where the AI can perform powerful tasks, but still requires humans to take certain decisions and oversee the agent's behavior. When using the ConversationalUI module, its basic action microflow pattern to execute requests with history and UI snippets to display the chat, human in the loop works out of the box. Note that action microflows are called until there is a final assistant's response as described in [Using Tool or Knowledge Base Calling](#action-microflow-tool-calling) section above, even if all tools are executed without user interaction. diff --git a/content/en/docs/genai/v2/reference-guide/genai-commons.md b/content/en/docs/genai/v2/reference-guide/genai-commons.md index f583a97a8c7..dd668adf3d8 100644 --- a/content/en/docs/genai/v2/reference-guide/genai-commons.md +++ b/content/en/docs/genai/v2/reference-guide/genai-commons.md @@ -80,9 +80,10 @@ The `DeployedModel` entity replaces the capabilities that the `Connection` entit | `Model` | The model identifier of the LLM provider. | | `OutputModality` | The type of information the model returns. | | `Microflow` | The microflow to execute for the specified model and modality. | -| `SupportsSystemPrompt` | Enum to specify if the model supports system prompts. | | `SupportsConversationsWithHistory` | Enum to specify if the model supports conversation with history. | | `SupportsFunctionCalling` | Enum to specify if the model supports function calling. | +| `SupportsStreaming` | An enum to specify if the model supports calling the streaming API. | +| `SupportsSystemPrompt` | Enum to specify if the model supports system prompts. | | `IsActive` | Boolean to specify if the model is active/usable with the current authentication settings and user preference. | #### `ConsumedKnowledgeBase` {#consumed-knowledge-base} @@ -233,6 +234,8 @@ The `Request` is an input object for the chat completions operations defined in | `ToolChoice` | Controls which (if any) tool is called by the model. For more information, see the [ENUM_ToolChoice](#enum-toolchoice) section containing a description of the possible values. | | `_AgentVersionId` | The `AgentVersionId`, set if the execution of the request was called from an Agent. | | `SaveToolCallHistory` | Indicates if the tool calls are stored for later continuation (must be implemented). | +| `UseStreamingAPI` | Indicates if the LLM call should be called with streaming. | +| `StreamingResponseWriterId` | Indicates if streaming data should be pushed to the UI. Must not be set manually! This is only set in backend if a ConversationalUI chat interface is being used with an agent that has streaming enabled. | #### `Message` {#message} From 401a179546c7d2c87d5486f08614ba4b5fd8e8e4 Mon Sep 17 00:00:00 2001 From: Liam Sommer Date: Tue, 21 Jul 2026 15:41:34 +0200 Subject: [PATCH 2/9] added more context --- content/en/docs/genai/v2/reference-guide/agent-commons.md | 2 +- .../en/docs/genai/v2/reference-guide/conversational-ui.md | 7 ++++--- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/content/en/docs/genai/v2/reference-guide/agent-commons.md b/content/en/docs/genai/v2/reference-guide/agent-commons.md index b2bcbc434a5..aa59877ee5d 100644 --- a/content/en/docs/genai/v2/reference-guide/agent-commons.md +++ b/content/en/docs/genai/v2/reference-guide/agent-commons.md @@ -117,7 +117,7 @@ Users can create two types of agents: #### Control model behavior {#model-settings} -By clicking the icon next to the selected model, you can control model behavior by configuring the temperature, top P, and maximum number of tokens on the version. If the selected model and connector support it (currently only Mendix Cloud GenAI), you can also enable backend streaming to use the model's streaming API, and frontend streaming to send generated chunks to the user interface as they become available (see [ConversationalUI](/agents/agents-kit-2/reference-guide/conversational-ui/#streaming) documentation for more). UI streaming is supported only for chat agents and for a chat interface from the Conversational UI module. +By clicking the icon next to the selected model, you can control model behavior by configuring the temperature, top P, and maximum number of tokens on the version. If the selected model and connector support it (currently only Mendix Cloud GenAI), you can also enable backend streaming to use the model's streaming API, and frontend streaming to send generated chunks to the user interface as they become available (see [ConversationalUI](/agents/agents-kit-2/reference-guide/conversational-ui/#streaming) documentation for more). UI streaming is supported only for chat agents and for a chat interface from the Conversational UI module. For task agents only backend streaming is possible because the UI is part of the implementation.s #### Defining Context Entity {#define-context-entity} diff --git a/content/en/docs/genai/v2/reference-guide/conversational-ui.md b/content/en/docs/genai/v2/reference-guide/conversational-ui.md index 17708a7da0b..b144ec2b68a 100644 --- a/content/en/docs/genai/v2/reference-guide/conversational-ui.md +++ b/content/en/docs/genai/v2/reference-guide/conversational-ui.md @@ -224,9 +224,10 @@ If no [user-visibility](/agents/agents-kit-2/reference-guide/commons/#enum-usera Conversational UI chats can also be streamed to the UI, which expands the assistant response as the LLM generates new content. To use streaming, use [AgentCommons](/agents/agents-kit-2/reference-guide/agent-commons/) to configure a chat agent in the [model settings](/agents/agents-kit-2/reference-guide/agent-commons/#model-settings), enable the applicable streaming options there, and use that agent when calling the `New Chat for Agent` action and in your action microflow. Not all models and connectors support streaming. Currently, streaming is supported only for Mendix Cloud GenAI. -Note that the following limitations apply for streaming because the logic is called in isolated transactions: -* Tool microflows can only read and change objects in the database, and user interaction is limited, for example, opening a page. -* Similarly, action microflows can only read and change objects in the database. +Note that the following limitations apply for streaming because the microflows are called in isolated transactions: +* Tool microflows can only read objects from the database and changes to objects are only stored within the microflow if not committed. Additionally, user interaction is limited, for example, opening a page or showing a message to the user. +* This similarly applies for action microflows. +A workaround to these limitations would be to make changes to persistent objects, for example the ChatContext which is available in both the action microflow and on the page. ### Human in the Loop {#human-in-the-loop} From b702c965e57a0e3db9b70da5db56558e583d07f0 Mon Sep 17 00:00:00 2001 From: oenerayca <123177148+oenerayca@users.noreply.github.com> Date: Tue, 21 Jul 2026 15:45:22 +0200 Subject: [PATCH 3/9] Update agent-commons.md - small typo fix --- content/en/docs/genai/v2/reference-guide/agent-commons.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/genai/v2/reference-guide/agent-commons.md b/content/en/docs/genai/v2/reference-guide/agent-commons.md index b2bcbc434a5..2a90c7c7c63 100644 --- a/content/en/docs/genai/v2/reference-guide/agent-commons.md +++ b/content/en/docs/genai/v2/reference-guide/agent-commons.md @@ -165,7 +165,7 @@ Note that [user access approval](/agents/agents-kit-2/reference-guide/commons/#e #### Testing and Refining the Agent -While writing the [system prompt](/agents/glossary/#system-prompt) (for both and task types) or the [user prompt](/agents/glossary/#user-prompt) (only for the task type), the prompt engineer can include variables by enclosing them in double braces, for example, `{{variable}}`. The actual values of these placeholders are typically known at runtime based on the user's page context. +While writing the [system prompt](/agents/glossary/#system-prompt) (for both chat and task types) or the [user prompt](/agents/glossary/#user-prompt) (only for the task type), the prompt engineer can include variables by enclosing them in double braces, for example, `{{variable}}`. The actual values of these placeholders are typically known at runtime based on the user's page context. To test the behavior of the prompts, a test can be executed. The prompt engineer must provide test values for all variables defined in the prompts. Additionally, multiple sets of test values for the variables can be defined and run in bulk. Based on the test results, the prompt engineer can add, remove, or rephrase certain parts of the prompt. ### Using the Agent in the App Logic {#app-logic} From 04e15c4ddc4a1d0fbb4e291c9db65d33528dff38 Mon Sep 17 00:00:00 2001 From: oenerayca <123177148+oenerayca@users.noreply.github.com> Date: Tue, 21 Jul 2026 15:46:51 +0200 Subject: [PATCH 4/9] Update conversational-ui.md - small typo fix --- content/en/docs/genai/v2/reference-guide/conversational-ui.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/genai/v2/reference-guide/conversational-ui.md b/content/en/docs/genai/v2/reference-guide/conversational-ui.md index 17708a7da0b..2260474f44d 100644 --- a/content/en/docs/genai/v2/reference-guide/conversational-ui.md +++ b/content/en/docs/genai/v2/reference-guide/conversational-ui.md @@ -222,7 +222,7 @@ If no [user-visibility](/agents/agents-kit-2/reference-guide/commons/#enum-usera ### Streaming {#streaming} -Conversational UI chats can also be streamed to the UI, which expands the assistant response as the LLM generates new content. To use streaming, use [AgentCommons](/agents/agents-kit-2/reference-guide/agent-commons/) to configure a chat agent in the [model settings](/agents/agents-kit-2/reference-guide/agent-commons/#model-settings), enable the applicable streaming options there, and use that agent when calling the `New Chat for Agent` action and in your action microflow. Not all models and connectors support streaming. Currently, streaming is supported only for Mendix Cloud GenAI. +Conversational UI chats can also be streamed to the UI, which expands the assistant response as the LLM generates new content. To use streaming, use [AgentCommons](/agents/agents-kit-2/reference-guide/agent-commons/) to configure a chat agent. In the [model settings](/agents/agents-kit-2/reference-guide/agent-commons/#model-settings), enable the applicable streaming options, and use that agent when calling the `New Chat for Agent` action and in your action microflow. Not all models and connectors support streaming. Currently, streaming is supported only for Mendix Cloud GenAI. Note that the following limitations apply for streaming because the logic is called in isolated transactions: * Tool microflows can only read and change objects in the database, and user interaction is limited, for example, opening a page. From d8eff3a4a361371c587f247a56f704d2deef6f1d Mon Sep 17 00:00:00 2001 From: oenerayca <123177148+oenerayca@users.noreply.github.com> Date: Tue, 21 Jul 2026 15:48:14 +0200 Subject: [PATCH 5/9] Update genai-commons.md - small wording change --- content/en/docs/genai/v2/reference-guide/genai-commons.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/genai/v2/reference-guide/genai-commons.md b/content/en/docs/genai/v2/reference-guide/genai-commons.md index dd668adf3d8..8af1778ae1c 100644 --- a/content/en/docs/genai/v2/reference-guide/genai-commons.md +++ b/content/en/docs/genai/v2/reference-guide/genai-commons.md @@ -82,7 +82,7 @@ The `DeployedModel` entity replaces the capabilities that the `Connection` entit | `Microflow` | The microflow to execute for the specified model and modality. | | `SupportsConversationsWithHistory` | Enum to specify if the model supports conversation with history. | | `SupportsFunctionCalling` | Enum to specify if the model supports function calling. | -| `SupportsStreaming` | An enum to specify if the model supports calling the streaming API. | +| `SupportsStreaming` | Enum to specify if the model supports calling the streaming API. | | `SupportsSystemPrompt` | Enum to specify if the model supports system prompts. | | `IsActive` | Boolean to specify if the model is active/usable with the current authentication settings and user preference. | From 1e4b721db93936bcffd25a5e0df365a735a86c15 Mon Sep 17 00:00:00 2001 From: oenerayca <123177148+oenerayca@users.noreply.github.com> Date: Wed, 22 Jul 2026 09:27:04 +0200 Subject: [PATCH 6/9] Update agent-commons.md - small typo fix --- content/en/docs/genai/v2/reference-guide/agent-commons.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/genai/v2/reference-guide/agent-commons.md b/content/en/docs/genai/v2/reference-guide/agent-commons.md index 585d2cef2aa..deede535b57 100644 --- a/content/en/docs/genai/v2/reference-guide/agent-commons.md +++ b/content/en/docs/genai/v2/reference-guide/agent-commons.md @@ -117,7 +117,7 @@ Users can create two types of agents: #### Control model behavior {#model-settings} -By clicking the icon next to the selected model, you can control model behavior by configuring the temperature, top P, and maximum number of tokens on the version. If the selected model and connector support it (currently only Mendix Cloud GenAI), you can also enable backend streaming to use the model's streaming API, and frontend streaming to send generated chunks to the user interface as they become available (see [ConversationalUI](/agents/agents-kit-2/reference-guide/conversational-ui/#streaming) documentation for more). UI streaming is supported only for chat agents and for a chat interface from the Conversational UI module. For task agents only backend streaming is possible because the UI is part of the implementation.s +By clicking the icon next to the selected model, you can control model behavior by configuring the temperature, top P, and maximum number of tokens on the version. If the selected model and connector support it (currently only Mendix Cloud GenAI), you can also enable backend streaming to use the model's streaming API, and frontend streaming to send generated chunks to the user interface as they become available (see [ConversationalUI](/agents/agents-kit-2/reference-guide/conversational-ui/#streaming) documentation for more). UI streaming is supported only for chat agents and for a chat interface from the Conversational UI module. For task agents only backend streaming is possible because the UI is part of the custom implementation. #### Defining Context Entity {#define-context-entity} From dbd2591b05399174991bbc8908b7c824b4baedc9 Mon Sep 17 00:00:00 2001 From: Liam Sommer <161324156+liamsommer-mx@users.noreply.github.com> Date: Fri, 31 Jul 2026 15:50:03 +0200 Subject: [PATCH 7/9] Update agent-editor.md with streaming limitation in playground --- content/en/docs/genai/v2/reference-guide/agent-editor.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/en/docs/genai/v2/reference-guide/agent-editor.md b/content/en/docs/genai/v2/reference-guide/agent-editor.md index ba42b4be9bf..1a2a9b365e6 100644 --- a/content/en/docs/genai/v2/reference-guide/agent-editor.md +++ b/content/en/docs/genai/v2/reference-guide/agent-editor.md @@ -265,6 +265,7 @@ Use version control to view and restore previous agent versions. This lets you i * MCP tool support is limited to whole-server integration. Selecting individual tools from a consumed MCP service to be added to an agent is not yet supported. That also means that the tool choice option `Tool` can only refer to a microflow tool currently. * If a document referenced by an Agent document is excluded, Studio Pro shows a consistency error. These consistency errors may not be resolved automatically when you include the excluded document again. Resolve this by synchronizing the app directory (F4) or by making a small change in any agent-related document (for example, add a character to a system prompt and remove it again). * The extension creates a `/agenteditor` log folder in the app directory. This folder is not excluded from version control automatically when you include the module from Marketplace. Add this folder to `.gitignore` manually, as described in the [First-time setup](#setup) section. +* While it is possible to configure streaming as well as user permissions for tools and knowledge bases, these settings are currently not reflected in the playground in Studio Pro, but will be considered at runtime when calling an agent defined in Studio Pro. ## Troubleshooting {#troubleshooting} From 8d1edf699a1dfc713d1ed26af0ddd47be71632e2 Mon Sep 17 00:00:00 2001 From: Liam Sommer <161324156+liamsommer-mx@users.noreply.github.com> Date: Fri, 31 Jul 2026 16:01:20 +0200 Subject: [PATCH 8/9] Update test agent with streaming limitation --- content/en/docs/genai/v2/reference-guide/agent-editor.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/content/en/docs/genai/v2/reference-guide/agent-editor.md b/content/en/docs/genai/v2/reference-guide/agent-editor.md index 1a2a9b365e6..76db1a7a186 100644 --- a/content/en/docs/genai/v2/reference-guide/agent-editor.md +++ b/content/en/docs/genai/v2/reference-guide/agent-editor.md @@ -225,6 +225,10 @@ When these conditions are met, you can use the test functionality to validate pr If a call fails during testing, a generic error message is shown in the Agent Editor UI. Detailed error information is available in the running app console in Studio Pro (the **Console** pane), similar to errors you would inspect while testing the app itself. +{{% alert color="info" %}} +While it is possible to configure streaming as well as user permissions for tools and knowledge bases, these settings are currently not reflected in the playground in Studio Pro, but will be considered at runtime when calling an agent defined in Studio Pro. +{{% /alert %}} + ### Including the Agent in the App Logic {#call-agent} Include an agent in the app logic by calling it from a microflow. Agent Editor provides **Call Agent** toolbox actions in the **Agent Editor** category: From c882bceacd01fe77e68c949ada2fc1068a0948f2 Mon Sep 17 00:00:00 2001 From: Karuna Vengurlekar Date: Mon, 3 Aug 2026 14:09:41 +0530 Subject: [PATCH 9/9] Add review --- .../docs/genai/v2/reference-guide/agent-commons.md | 6 +++--- .../en/docs/genai/v2/reference-guide/agent-editor.md | 4 ++-- .../genai/v2/reference-guide/conversational-ui.md | 12 +++++++----- .../docs/genai/v2/reference-guide/genai-commons.md | 6 +++--- 4 files changed, 15 insertions(+), 13 deletions(-) diff --git a/content/en/docs/genai/v2/reference-guide/agent-commons.md b/content/en/docs/genai/v2/reference-guide/agent-commons.md index deede535b57..592cfbdab45 100644 --- a/content/en/docs/genai/v2/reference-guide/agent-commons.md +++ b/content/en/docs/genai/v2/reference-guide/agent-commons.md @@ -111,13 +111,13 @@ Users can create two types of agents: * **Chat Agent**: Intended for scenarios where the end-user interacts through a chat interface, or where the agent is called conversationally by another agent. -* **Task Agent**: Designed for isolated agentic patterns such as background processes, subagents in an Agent-as-Tool setup, or any use case that doesn't require a conversational interface with historical context. +* **Task Agent**: Designed for isolated agentic patterns such as background processes, subagents in an Agent-as-Tool setup, or any use case that does not require a conversational interface with historical context. {{< figure src="/attachments/genai/agentcommons/agent_builder_task.png" alt="" >}} -#### Control model behavior {#model-settings} +#### Control Model Behavior {#model-settings} -By clicking the icon next to the selected model, you can control model behavior by configuring the temperature, top P, and maximum number of tokens on the version. If the selected model and connector support it (currently only Mendix Cloud GenAI), you can also enable backend streaming to use the model's streaming API, and frontend streaming to send generated chunks to the user interface as they become available (see [ConversationalUI](/agents/agents-kit-2/reference-guide/conversational-ui/#streaming) documentation for more). UI streaming is supported only for chat agents and for a chat interface from the Conversational UI module. For task agents only backend streaming is possible because the UI is part of the custom implementation. +Click the icon next to the selected model to control model behavior by configuring the temperature, top P, and maximum number of tokens. If the selected model and connector support it (currently only Mendix Cloud GenAI), you can also enable backend streaming to use the model's streaming API, and frontend streaming to send generated chunks to the user interface as they become available. For details, see [Streaming](/agents/agents-kit-2/reference-guide/conversational-ui/#streaming) in the *Conversational UI* documentation. UI streaming is supported only for chat agents using a chat interface from the Conversational UI module. For task agents, only backend streaming is available because the UI is part of the custom implementation. #### Defining Context Entity {#define-context-entity} diff --git a/content/en/docs/genai/v2/reference-guide/agent-editor.md b/content/en/docs/genai/v2/reference-guide/agent-editor.md index 76db1a7a186..f10397d4fd5 100644 --- a/content/en/docs/genai/v2/reference-guide/agent-editor.md +++ b/content/en/docs/genai/v2/reference-guide/agent-editor.md @@ -226,7 +226,7 @@ When these conditions are met, you can use the test functionality to validate pr If a call fails during testing, a generic error message is shown in the Agent Editor UI. Detailed error information is available in the running app console in Studio Pro (the **Console** pane), similar to errors you would inspect while testing the app itself. {{% alert color="info" %}} -While it is possible to configure streaming as well as user permissions for tools and knowledge bases, these settings are currently not reflected in the playground in Studio Pro, but will be considered at runtime when calling an agent defined in Studio Pro. +Streaming and user permissions for tools and knowledge bases can be configured, but these settings are not reflected in the playground in Studio Pro. They are applied at runtime when calling an agent defined in Studio Pro. {{% /alert %}} ### Including the Agent in the App Logic {#call-agent} @@ -269,7 +269,7 @@ Use version control to view and restore previous agent versions. This lets you i * MCP tool support is limited to whole-server integration. Selecting individual tools from a consumed MCP service to be added to an agent is not yet supported. That also means that the tool choice option `Tool` can only refer to a microflow tool currently. * If a document referenced by an Agent document is excluded, Studio Pro shows a consistency error. These consistency errors may not be resolved automatically when you include the excluded document again. Resolve this by synchronizing the app directory (F4) or by making a small change in any agent-related document (for example, add a character to a system prompt and remove it again). * The extension creates a `/agenteditor` log folder in the app directory. This folder is not excluded from version control automatically when you include the module from Marketplace. Add this folder to `.gitignore` manually, as described in the [First-time setup](#setup) section. -* While it is possible to configure streaming as well as user permissions for tools and knowledge bases, these settings are currently not reflected in the playground in Studio Pro, but will be considered at runtime when calling an agent defined in Studio Pro. +* Streaming and user permissions for tools and knowledge bases can be configured, but these settings are not reflected in the playground in Studio Pro. They are applied at runtime when calling an agent defined in Studio Pro. ## Troubleshooting {#troubleshooting} diff --git a/content/en/docs/genai/v2/reference-guide/conversational-ui.md b/content/en/docs/genai/v2/reference-guide/conversational-ui.md index 85fa25327af..1bbb6c6a274 100644 --- a/content/en/docs/genai/v2/reference-guide/conversational-ui.md +++ b/content/en/docs/genai/v2/reference-guide/conversational-ui.md @@ -222,12 +222,14 @@ If no [user-visibility](/agents/agents-kit-2/reference-guide/commons/#enum-usera ### Streaming {#streaming} -Conversational UI chats can also be streamed to the UI, which expands the assistant response as the LLM generates new content. To use streaming, use [AgentCommons](/agents/agents-kit-2/reference-guide/agent-commons/) to configure a chat agent. In the [model settings](/agents/agents-kit-2/reference-guide/agent-commons/#model-settings), enable the applicable streaming options, and use that agent when calling the `New Chat for Agent` action and in your action microflow. Not all models and connectors support streaming. Currently, streaming is supported only for Mendix Cloud GenAI. +Conversational UI chats can also stream responses to the UI, expanding the assistant response as the LLM generates new content. To use streaming, use [Agent Commons](/agents/agents-kit-2/reference-guide/agent-commons/) to configure a chat agent. In the [model settings](/agents/agents-kit-2/reference-guide/agent-commons/#model-settings), enable the applicable streaming options, and use that agent when calling the `New Chat for Agent` action and in your action microflow. Not all models and connectors support streaming. Currently, streaming is supported only for Mendix Cloud GenAI. -Note that the following limitations apply for streaming because the microflows are called in isolated transactions: -* Tool microflows can only read objects from the database and changes to objects are only stored within the microflow if not committed. Additionally, user interaction is limited, for example, opening a page or showing a message to the user. -* This similarly applies for action microflows. -A workaround to these limitations would be to make changes to persistent objects, for example the ChatContext which is available in both the action microflow and on the page. +The following limitations apply to streaming because microflows run in isolated transactions: + +* Tool microflows can only read objects from the database. Changes to objects are stored only within the microflow if not committed. Additionally, user interaction is limited—for example, opening a page or showing a message to the user. +* The same limitations apply to action microflows. + +A workaround is to make changes to persistent objects—for example, `ChatContext`, which is available in both the action microflow and on the page. ### Human in the Loop {#human-in-the-loop} diff --git a/content/en/docs/genai/v2/reference-guide/genai-commons.md b/content/en/docs/genai/v2/reference-guide/genai-commons.md index 8af1778ae1c..835f920d14d 100644 --- a/content/en/docs/genai/v2/reference-guide/genai-commons.md +++ b/content/en/docs/genai/v2/reference-guide/genai-commons.md @@ -233,9 +233,9 @@ The `Request` is an input object for the chat completions operations defined in | `TopP` | An alternative to temperature for controlling the randomness of the model response. `TopP` defines a probability threshold so that only words with probabilities greater than or equal to the threshold will be included in the response. Mendix recommends steering either the temperature or `TopP`, but not both. | | `ToolChoice` | Controls which (if any) tool is called by the model. For more information, see the [ENUM_ToolChoice](#enum-toolchoice) section containing a description of the possible values. | | `_AgentVersionId` | The `AgentVersionId`, set if the execution of the request was called from an Agent. | -| `SaveToolCallHistory` | Indicates if the tool calls are stored for later continuation (must be implemented). | -| `UseStreamingAPI` | Indicates if the LLM call should be called with streaming. | -| `StreamingResponseWriterId` | Indicates if streaming data should be pushed to the UI. Must not be set manually! This is only set in backend if a ConversationalUI chat interface is being used with an agent that has streaming enabled. | +| `SaveToolCallHistory` | Indicates if the tool calls are stored for later continuation (must be implemented). | +| `UseStreamingAPI` | Indicates whether the LLM call uses streaming. | +| `StreamingResponseWriterId` | Indicates whether streaming data is pushed to the UI. Do not set this manually. It is set in the backend only when a Conversational UI chat interface is used with an agent that has streaming enabled. | #### `Message` {#message}