diff --git a/content/en/docs/genai/v2/reference-guide/agent-commons.md b/content/en/docs/genai/v2/reference-guide/agent-commons.md index f9b48aab861..592cfbdab45 100644 --- a/content/en/docs/genai/v2/reference-guide/agent-commons.md +++ b/content/en/docs/genai/v2/reference-guide/agent-commons.md @@ -111,10 +111,14 @@ Users can create two types of agents: * **Chat Agent**: Intended for scenarios where the end-user interacts through a chat interface, or where the agent is called conversationally by another agent. -* **Task Agent**: Designed for isolated agentic patterns such as background processes, subagents in an Agent-as-Tool setup, or any use case that doesn't require a conversational interface with historical context. +* **Task Agent**: Designed for isolated agentic patterns such as background processes, subagents in an Agent-as-Tool setup, or any use case that does not require a conversational interface with historical context. {{< figure src="/attachments/genai/agentcommons/agent_builder_task.png" alt="" >}} +#### Control Model Behavior {#model-settings} + +Click the icon next to the selected model to control model behavior by configuring the temperature, top P, and maximum number of tokens. If the selected model and connector support it (currently only Mendix Cloud GenAI), you can also enable backend streaming to use the model's streaming API, and frontend streaming to send generated chunks to the user interface as they become available. For details, see [Streaming](/agents/agents-kit-2/reference-guide/conversational-ui/#streaming) in the *Conversational UI* documentation. UI streaming is supported only for chat agents using a chat interface from the Conversational UI module. For task agents, only backend streaming is available because the UI is part of the custom implementation. + #### Defining Context Entity {#define-context-entity} If your agent's prompt includes variables, your app must define an entity with attributes that match the variable names. An object of this entity serves as the context object, which holds the context data that will be passed when the **call agent** operation is triggered. For more details, see the [Use the agent in the app logic](#app-logic) section below. diff --git a/content/en/docs/genai/v2/reference-guide/agent-editor.md b/content/en/docs/genai/v2/reference-guide/agent-editor.md index dfc110c9c70..60eaff72bbd 100644 --- a/content/en/docs/genai/v2/reference-guide/agent-editor.md +++ b/content/en/docs/genai/v2/reference-guide/agent-editor.md @@ -239,6 +239,10 @@ When these conditions are met, you can use the test functionality to validate pr If a call fails during testing, a generic error message is shown in the Agent Editor UI. Detailed error information is available in the running app console in Studio Pro (the **Console** pane), similar to errors you would inspect while testing the app itself. +{{% alert color="info" %}} +Streaming and user permissions for tools and knowledge bases can be configured, but these settings are not reflected in the playground in Studio Pro. They are applied at runtime when calling an agent defined in Studio Pro. +{{% /alert %}} + ### Including the Agent in the App Logic {#call-agent} Include an agent in the app logic by calling it from a microflow. Agent Editor provides **Call Agent** toolbox actions in the **Agent Editor** category: @@ -278,7 +282,8 @@ Use version control to view and restore previous agent versions. This lets you i * Support for Mac users is limited. Some functionalities might not work, such as doing a test call for Model documents. Mendix recommends using Studio Pro on Windows to use all features of Agent Editor. * MCP tool support is limited to whole-server integration. Selecting individual tools from a consumed MCP service to be added to an agent is not yet supported. That also means that the tool choice option `Tool` can only refer to a microflow tool currently. * If a document referenced by an Agent document is excluded, Studio Pro shows a consistency error. These consistency errors may not be resolved automatically when you include the excluded document again. Resolve this by synchronizing the app directory (F4) or by making a small change in any agent-related document (for example, add a character to a system prompt and remove it again). -* The extension creates a `/agenteditor` log folder in the app directory. This folder is not excluded from version control automatically when you include the module from Marketplace. Add this folder to `.gitignore` manually, as described in the [First-Time Setup](#setup) section. +* The extension creates a `/agenteditor` log folder in the app directory. This folder is not excluded from version control automatically when you include the module from Marketplace. Add this folder to `.gitignore` manually, as described in the [First-time setup](#setup) section. +* Streaming and user permissions for tools and knowledge bases can be configured, but these settings are not reflected in the playground in Studio Pro. They are applied at runtime when calling an agent defined in Studio Pro. ## Troubleshooting {#troubleshooting} diff --git a/content/en/docs/genai/v2/reference-guide/conversational-ui.md b/content/en/docs/genai/v2/reference-guide/conversational-ui.md index 4f824f176b5..1bbb6c6a274 100644 --- a/content/en/docs/genai/v2/reference-guide/conversational-ui.md +++ b/content/en/docs/genai/v2/reference-guide/conversational-ui.md @@ -220,6 +220,17 @@ This changes how action microflows are used, because they are called each time a If no [user-visibility](/agents/agents-kit-2/reference-guide/commons/#enum-useraccessapproval) is configured for tools and you want to avoid storing tool messages, you can change the Boolean `SaveToolCallHistory` to *false* on the [Request](/agents/agents-kit-2/reference-guide/commons/#request). Note that [knowledge base retrievals](/agents/agents-kit-2/reference-guide/commons/#add-knowledge-base-to-request) are set to `HiddenForUser` by default. +### Streaming {#streaming} + +Conversational UI chats can also stream responses to the UI, expanding the assistant response as the LLM generates new content. To use streaming, use [Agent Commons](/agents/agents-kit-2/reference-guide/agent-commons/) to configure a chat agent. In the [model settings](/agents/agents-kit-2/reference-guide/agent-commons/#model-settings), enable the applicable streaming options, and use that agent when calling the `New Chat for Agent` action and in your action microflow. Not all models and connectors support streaming. Currently, streaming is supported only for Mendix Cloud GenAI. + +The following limitations apply to streaming because microflows run in isolated transactions: + +* Tool microflows can only read objects from the database. Changes to objects are stored only within the microflow if not committed. Additionally, user interaction is limited—for example, opening a page or showing a message to the user. +* The same limitations apply to action microflows. + +A workaround is to make changes to persistent objects—for example, `ChatContext`, which is available in both the action microflow and on the page. + ### Human in the Loop {#human-in-the-loop} When using the [Function Calling](/agents/function-calling/) pattern by adding tools to the request, you can control when those tools get executed and if they are visible to the user by setting [user access approval](/agents/agents-kit-2/reference-guide/commons/#enum-useraccessapproval) per tool. [Human in the loop](/agents/glossary/#human-in-the-loop) describes a pattern where the AI can perform powerful tasks, but still requires humans to take certain decisions and oversee the agent's behavior. When using the ConversationalUI module, its basic action microflow pattern to execute requests with history and UI snippets to display the chat, human in the loop works out of the box. Note that action microflows are called until there is a final assistant's response as described in [Using Tool or Knowledge Base Calling](#action-microflow-tool-calling) section above, even if all tools are executed without user interaction. diff --git a/content/en/docs/genai/v2/reference-guide/genai-commons.md b/content/en/docs/genai/v2/reference-guide/genai-commons.md index f583a97a8c7..835f920d14d 100644 --- a/content/en/docs/genai/v2/reference-guide/genai-commons.md +++ b/content/en/docs/genai/v2/reference-guide/genai-commons.md @@ -80,9 +80,10 @@ The `DeployedModel` entity replaces the capabilities that the `Connection` entit | `Model` | The model identifier of the LLM provider. | | `OutputModality` | The type of information the model returns. | | `Microflow` | The microflow to execute for the specified model and modality. | -| `SupportsSystemPrompt` | Enum to specify if the model supports system prompts. | | `SupportsConversationsWithHistory` | Enum to specify if the model supports conversation with history. | | `SupportsFunctionCalling` | Enum to specify if the model supports function calling. | +| `SupportsStreaming` | Enum to specify if the model supports calling the streaming API. | +| `SupportsSystemPrompt` | Enum to specify if the model supports system prompts. | | `IsActive` | Boolean to specify if the model is active/usable with the current authentication settings and user preference. | #### `ConsumedKnowledgeBase` {#consumed-knowledge-base} @@ -232,7 +233,9 @@ The `Request` is an input object for the chat completions operations defined in | `TopP` | An alternative to temperature for controlling the randomness of the model response. `TopP` defines a probability threshold so that only words with probabilities greater than or equal to the threshold will be included in the response. Mendix recommends steering either the temperature or `TopP`, but not both. | | `ToolChoice` | Controls which (if any) tool is called by the model. For more information, see the [ENUM_ToolChoice](#enum-toolchoice) section containing a description of the possible values. | | `_AgentVersionId` | The `AgentVersionId`, set if the execution of the request was called from an Agent. | -| `SaveToolCallHistory` | Indicates if the tool calls are stored for later continuation (must be implemented). | +| `SaveToolCallHistory` | Indicates if the tool calls are stored for later continuation (must be implemented). | +| `UseStreamingAPI` | Indicates whether the LLM call uses streaming. | +| `StreamingResponseWriterId` | Indicates whether streaming data is pushed to the UI. Do not set this manually. It is set in the backend only when a Conversational UI chat interface is used with an agent that has streaming enabled. | #### `Message` {#message}