AI settings
Project Settings > AI controls the project context that Katalon AI uses before it answers or acts. The settings include the Knowledge Base, Agent Rules, Project Memory, and MCP & Skills.
- An account administrator must enable AI in System > Configuration > AI Service.
- Your role must have the project AI permission to view and use these settings.
- A project administrator is required to enable Agent Rules and Project Memory.
How the settings workβ
- Knowledge Base provides project data and uploaded documents as reference material.
- Agent Rules define project-wide standards that Katalon AI must follow. A known issue allows a conflicting personal memory to override a rule.
- Project Memory stores your personal, durable preferences for the project. Other project members cannot see your memories.
- MCP & Skills provides the MCP endpoint and setup instructions for connecting external AI clients and Katalon True skills.
Account administrators control which AI features are available to your account, and can view a history of changes to AI settings and keys. See Choose which AI features are available. Changes to project-level settings are recorded in a backend audit stream. There is no project-level audit view.
Knowledge Baseβ
The Knowledge Base combines project data with documents uploaded to the project. After indexing, Katalon AI Assistant and connected agents can use the content within your existing permissions.
A project's knowledge base gets its information from two sources:
| Source | What it holds | How it gets there |
|---|---|---|
| Project data | Test cases, test suites, test folders, and requirements already in the project | Synced automatically when ingestion is on |
| Your documents | Files you upload, such as PRDs and specs | Uploaded by you |
Both sources are available through the same knowledge base. Katalon AI Assistant and agents connected through the MCP server can access only what you already have permission to access. Turning on the knowledge base does not grant anyone additional access.
Set up the Knowledge Baseβ
Knowledge Base ingestion is Off by default for new projects. Turn on Enable Knowledge Base before uploading documents.
- Go to Project Settings > AI > Knowledge Base.
- Switch on Enable Knowledge Base. The page displays the current status, last update time, and counts for synced test cases, test suites, test folders, and requirements.
- In Your documents, select Upload files, or drag files onto the upload area.
- Wait until each document shows Completed. It is then available to Katalon AI Assistant and connected agents.
After the initial sync, changes to the project are synced automatically about every 15 minutes.
Use this interactive walkthrough to see the complete setup:
Turning Enable Knowledge Base off in project settings pauses syncing. Previously indexed content remains searchable.
You can also turn Knowledge Base off for a single conversation from the Sources button in the Katalon AI composer. See Choose context and knowledge sources for a conversation.
Manage documentsβ
- Supported file types and size:
.pdf,.docx,.xlsx,.pptx,.txt,.csv,.md,.png,.jpg,.jpeg. Each file can be up to 100.0 MB. - Each filename must be unique within the project. Rename a duplicate file, or use Replace on the existing document's row.
- Use the Search filename field to find a document. The dropdowns next to it let you filter by file type and status.
- Ingest status: Documents go through several processing stages. Only documents marked Completed are available to Katalon AI Assistant. If processing fails, hover over the status to see why.
- Select Clear all to remove all uploaded documents and synced project data from the knowledge base.
Deleting cannot be undone. You must upload your documents again, and project data must go through a new full sync.
Conflict checksβ
When Katalon AI Assistant generates test cases or analyzes a requirement, it checks the requirement against the documents and project data in the knowledge base and ends its response with a Potential conflicts section. If the knowledge base is empty, rebuilding, failed, or out of date, the response says it could not check. To learn more, see Review potential conflicts.
Known limitationsβ
- If you permanently delete a project item, it may remain in the knowledge base until the next full sync.
- When generating test cases, the knowledge base does not yet help prevent duplicate test cases.
Agent Rulesβ
Agent Rules set project-wide standards for Katalon AI. Turn on Enable Agent Rules to apply them. You can create Agent Rules only on this page. Asking Katalon AI to remember something in chat creates a personal memory in Project Memory.
- To add a rule, select Add the first rule or Add rule. Enter a required Rule and optional Details. Tag is optional. Use Details to explain when the rule applies; vague notes such as "Added for testing" can make it unclear to Katalon AI.
- To manage rules, use the search field, Edit, Delete, or Export. The search matches whole words only.
- To import rules, select Import and upload a CSV file with a
Rulecolumn.TagandDetailsare optional. - If you leave Tag empty, Katalon AI assigns a tag based on the rule text when you save. See Automatic tags.
- After you import a CSV file, the Result step lists rules as added or not added. For each rule that was not added, it shows the reason and the CSV line number.
Katalon AI identifies the rules it applied in its response.
Note: A conflicting personal memory can override an Agent Rule due to a known issue.
Rule tagsβ
The tag tells Katalon AI what kind of statement the rule is. It does not change how strictly Katalon AI follows the rule.
| Tag | Use it for | Example rule |
|---|---|---|
convention | How your team does the work: naming, structure, format, where things go. If Katalon AI ignores it, the result works but does not match your standard. | Test case titles use the format <Feature> - <Scenario>. |
constraint | A hard limit. If Katalon AI ignores it, the result is wrong, unsafe, or unusable. | Never run tests against the production environment. |
environment | A fact about the application or system under test. It describes the system and gives no instruction. | The staging database resets every day at 02:00 UTC. |
correction | A mistake Katalon AI made before, with the right answer. | The checkout page in this app is called "Basket". Do not call it "Cart". |
verdict | How to judge a type of result, so Katalon AI decides the same way each time. | Classify a timeout on the payment step in staging as an environment issue. |
preference | A choice with no team standard behind it. For a personal choice, use Project Memory instead. | Keep generated test cases under 10 steps. |
Automatic tagsβ
When you add or import a rule without a valid tag, Katalon AI assigns one based on the rule text.
- When you add a rule, Katalon AI sets the tag when you save. Saving can take several seconds while it detects the tag.
- When you import a CSV file, Katalon AI keeps valid tags and assigns tags to rules with missing or unknown tags. The Preview step shows an unknown tag as No Tag; the table shows the assigned tag.
- When no tag fits, the rule gets the
conventiontag. - You can edit an automatically assigned tag at any time.
- Katalon AI does not change the tags of rules that already exist.
Rules of thumbβ
- If the rule states a fact about the system under test, select
environment. - If ignoring the rule makes the result wrong or unsafe, select
constraint. - If the rule fixes a mistake Katalon AI made before, select
correction. - If the rule tells Katalon AI how to judge a result, select
verdict. - If the rule describes how your team names or structures work, select
convention. - If you are not sure, select
convention. The tag does not change how the rule applies.
A rule about AI test runs can mix a fact and an instruction. Split it into separate rules. For example, "The QA environment uses Okta single sign-on" is an environment rule, and "Do not run AI tests on the shared QA environment during business hours" is a constraint rule.
Sample promptsβ
Use these prompts in Katalon AI Assistant to see how a rule of each tag affects the result. Each prompt assumes the example rule from the table above is active, with details that explain it. Katalon AI applies rules in chat, including when it sets up a Run with AI for you.
Show sample prompts
| Tag | Prompt | Expected result |
|---|---|---|
convention | Write test cases for the login page. | Each test case title uses the format <Feature> - <Scenario>, and Katalon AI lists the rule under Rules applied. |
constraint | Run the checkout suite with AI on production. | Katalon AI does not start the run and tells you which rule stops it. |
environment | When should we schedule the nightly regression run on staging? | Katalon AI suggests a time after the 02:00 UTC database reset. |
correction | Write test cases for the cart page. | The test case titles and steps use "Basket". |
verdict | A staging run failed with a timeout on the payment step. Should I file a defect? | Katalon AI classifies the failure as an environment issue and says that no defect is needed. |
preference | Generate test cases for the password reset flow. | Each test case has fewer than 10 steps. |
When Katalon AI generates test cases, its reply lists the rules it followed under Rules applied. To see all rules in force, ask "Which agent rules apply to this project?"
Review suggested rulesβ
Katalon AI can suggest rules from your conversations. Katalon True also creates a suggested rule for each custom instruction that moves to Agent Rules (see Custom instructions moved to Agent Rules). Suggested rules appear in the Needs your review section above the rule table. A suggested rule has no effect until a project admin accepts it.
- Each row shows the rule and its tag, newest first.
- The section shows the first two suggestions. Select Show N more to see the rest, or Show less to collapse the list. You can also collapse the whole section.
- Accept all and Dismiss all sit together in the section header. They apply to every suggested rule in the current project. The search field and the tag filter of the rule table do not limit them.
- Select Dismiss all in the confirmation dialog to dismiss every suggested rule in the project, or Cancel to keep them.
- Dismissing suggested rules does not change accepted rules or personal memories. The Needs your review section disappears when no suggested rules remain.
Custom instructions moved to Agent Rulesβ
Katalon True proposes your active personal custom instructions as Agent Rules so a project admin can review that guidance.
- For each active personal custom instruction, Katalon True creates one suggested rule in every project you can open. The rule has the
conventiontag. The rule name is the instruction name, and the Details field holds the instruction text without changes. - A project admin reviews each suggested rule in Project Settings > AI > Agent Rules > Needs your review, then accepts or dismisses it. A suggested rule has no effect on Katalon AI until a project admin accepts it.
- If Enable Agent Rules is off in a project, the suggested rule still appears. It applies only after an admin turns the setting on. Katalon True does not turn it on for you.
- Inactive custom instructions are not moved.
- If an instruction contains a credential such as a password or token, Katalon True does not move it.
- The migration does not create the same suggestion twice. If an admin dismisses a migrated suggestion, the migration does not recreate it.
- After the move, open Custom Instructions in Katalon AI to see a notice and a Review Agent rules button. The list is read-only. You can turn an instruction on or off and delete it, but you cannot edit it or add a new one. If you are not a project admin, ask your project admin to review the suggested rules.
Creating a suggested rule does not stop your active custom instruction from applying. After a project admin accepts the matching rule, delete the custom instruction to avoid applying the same guidance twice.
Duplicate rulesβ
Katalon True treats a rule as a duplicate only when both the Rule text and the Details text match an existing rule in the project. Rules that share the same Details but have different Rule text are saved as separate rules.
- When you add a duplicate rule by hand, the dialog stays open and shows This rule already exists in this project, so it was not added again.
- The Preview step does not flag duplicates. After import, the Result step lists duplicate rules under Not added with the reason Same rule already exists and their CSV line numbers.
- Rules with almost the same text, such as "Run tests with AI using Chrome" and "Run tests with AI using Chrome browser", are not duplicates. Katalon True saves both. Review and delete the extra rule yourself.
Project Memoryβ
Project Memory stores your own durable preferences for the project. Turn on Enable Project Memory, then ask Katalon AI to remember a preference, such as βRemember that our checkout tests use the staging environment.β You can also manage memories from Personalization in the Katalon AI header.
Project memories are personal and are not visible to other project members. A conflicting personal memory can override an Agent Rule due to a known issue.
Review suggested memoriesβ
Katalon AI can suggest a memory after a conversation. Open Personalization in the Katalon AI header to review suggestions. A suggestion has no effect until you accept it.
- Suggestions appear newest first. The list shows the first two suggestions. Select Show N more to see the rest. You can collapse the section.
- Select Dismiss all to dismiss suggestions awaiting review. In the inline confirmation, select Dismiss all again to confirm, or Cancel to keep them.
- Dismiss all includes suggestions beyond the first two preview rows. Your saved memories stay unchanged.
- During a large dismissal, the panel shows progress, for example "Dismissing 25 of 40". If some suggestions cannot be dismissed, an error notification reports how many were dismissed and how many were not.
- When no suggestions remain, the review section and Dismiss all disappear.
Turn a source off for one conversationβ
You can turn Knowledge Base and Project memory on or off for one Katalon AI conversation. Your project settings stay the same. See Choose context and knowledge sources for a conversation.
MCP & Skillsβ
Your MCP serverβ
The Your MCP server section provides one endpoint for the whole account. The same endpoint works for every project in the account. Select Copy to copy the endpoint. On first use, your agent signs you in through the browser and asks you to choose a workspace.
Set up your agentβ
Setup varies by client. Select your client to view its setup command or configuration, then select Copy and run it in the client. The available clients are Claude Code, GitHub Copilot, Codex, Cursor, Claude Desktop, and ChatGPT. For Claude Code and Codex, the skills and MCP server are installed together.
After setup, the client signs you in through the browser on first use and asks which workspace to use.
Katalon True skillsβ
The page links to the Katalon True skills repository. It provides 18 reusable skills covering the testing lifecycle, including planning, design, management, review, execution, analysis, and maintenance. The skills help agents use your test assets, requirements, and runs according to your project conventions.