Question about AI user responsiveness towards users who are not authoritative over a project #2893
Zak Siddiqui opened 2 weeks ago

If there is an issue that is being watched by an AI user, then will the AI user be influenced by comments from human users (i.e., non-AI) who are not the project owner or are not the Assignee?

My motivation for asking this question is that if I have a public repository and I want the AI to obey my instructions but I don't necessarily want the AI to consider comments by other users who are not authoritative over this project.

I hope the question makes sense.

Don't assume. Ask high-level clarifying questions if needed.

  • Numbered questions (1, 2, 3, etc)
  • Lettered answer options (A, B, C, etc)
  • Pros and Cons? Your recommendation?
  • Service Desk commented 2 weeks ago

    Yes — but control is by AI entitlement, not by project ownership or issue Assignee.

    Who can trigger the AI

    1. Mention — Any entitled user can @mention the AI in an issue/PR comment and give instructions. Entitlement is configured on the AI user (entitle to all, or specific users/groups). Site administrators are always entitled.
    2. Proactive watching — If the AI has proactive behavior enabled and is watching the issue, it may respond to relevant comments from entitled users even without being mentioned.
    3. Assignment — Entitled projects can assign the AI as Assignee/reviewer (project entitlement; separate from user entitlement for mentions/chat).

    If a user who is not entitled mentions the AI, the AI replies that they are not entitled to interact with it, and does not run the task.

    Project owner / Assignee status alone does not decide this.

    What the AI sees once it runs

    When an entitled user successfully triggers a run (or the AI is assigned work), the AI loads the full issue/PR context, including comments from other participants. So content from non-authoritative users can still appear in context after a run starts — the main gate is who can start a run.

    Practical setup for a public repository

    To prefer your instructions and keep untrusted users from directing the AI:

    1. Disable Entitle to All Users and Projects on the AI user.
    2. Entitle only yourself (and trusted groups) so only those users can chat with / mention the AI.
    3. Entitle only the projects where assignment should be allowed.
    4. Leave proactive off unless you want auto-responses to entitled users without mentions.
    5. Optionally add a system prompt that prefers maintainer instructions when context is mixed.

    Docs: Working with AI User and Give AI User Instructions.

    @zaakiy Does this answer your question, or do you want more detail on a specific scenario (for example proactive vs mention-only, or system-prompt wording)?

  • Zak Siddiqui commented 2 weeks ago

    @servicedesk

    Assignment — Entitled projects can assign the AI as Assignee/reviewer (project entitlement; separate from user entitlement for mentions/chat).

    @servicedesk Since creating this question, I encountered a problem.

    Background

    I have a master issue in a parent repository that is controlling and managing child issues into other child repositories

    AI users is entitled to all users and all projects.

    Problem

    multiple workspaces are being launched for the same work. The parent issue in the parent project is @ing the AI users in order to begin work or begin a review for a pull request. but at the same time, the child issues in the child projects, since they are also entitled with the AI, they are doing things like performing automatic reviews., since I have a rule where any pull request is automatically assigned to an AI reviewer.

    Question

    Could you recommend me a more suitable method to control these duplicate task work spaces that are being created for the same tasks?

    My current AI System prompts

    Sharing them in case it helps with context.

    leadplanner
    You are an issue planning agent for this code repository.
    
    Do not use subagents. Do not make any commits. Do not create any pull requests. You must not modify code. Your role is to either submit comments to issues and potentially also to pull requests. This comment must be in the nature of a high-level plan that another autonomous AI agent can execute. 
    
    It is desirable to ask high-level clarifying questions if needed.
    - Numbered questions (1, 2, 3, etc)
    - Lettered answer options (A, B, C, etc)
    - Pros and Cons? Your recommendation?
    
    leadmechanic
    Don't assume. Ask high-level clarifying questions if needed.
    - Numbered questions (1, 2, 3, etc)
    - Lettered answer options (A, B, C, etc)
    - Pros and Cons? Your recommendation?
    
    Do not use subagents.
    
    if any pull requests are created, set the reviewer to `@leadreviewer` after you have added any relevant context in the comments. 
    
    juniormechanic
    When you are assigned with an issue of type "Question", try to answer user the question without creating branch or change code. If user is fine with the answer, close the issue then; otherwise, escalate to @zak. 
    
    You are free to consider sibling projects in order to help provide the answer. 
    
    Consider if AWS Infrastructure changes are required in https://code.mydomain.io/DevOps/terraform
    
    Do not use subagents.
    
    Don't assume. Ask high-level clarifying questions if needed.
    - Numbered questions (1, 2, 3, etc)
    - Lettered answer options (A, B, C, etc)
    - Pros and Cons? Your recommendation?
    
    leadreviewer
    Perform a review of the PR. If an issue is referenced, obatin the text of the issue and its comments to ensure that the changes align with intentions.
    
    Don't assume. Ask high-level clarifying questions if needed.
    - Numbered questions (1, 2, 3, etc)
    - Lettered answer options (A, B, C, etc)
    - Pros and Cons? Your recommendation?
    
    Do not use subagents.
    
    
    
  • Service Desk commented 2 weeks ago

    @zaakiy Duplicate workspaces usually come from two independent triggers for the same AI on the same work, not from parent/child linking itself.

    Why this happens with your setup

    With Entitle to All Users and Projects, every project (parent and children) can:

    1. Start a workspace when someone @mentions the AI (your parent-issue orchestration), and
    2. Start another workspace when a child auto-assigns that AI as PR reviewer (your PR rule).

    Each trigger creates its own workspace. Mentions and assignment are separate entry points.

    Also: if you later turn off “entitle to all” and entitle a parent project, assignment entitlement still cascades to all sub-projects. So child auto-assign rules keep working for that AI unless you remove those rules or stop entitling that tree for that AI.

    Pick one orchestration model and stick to it:

    Option A — Parent-orchestrated (best fit for your “master issue controls children” flow)

    Pros: One place decides when work/review starts; fewer duplicates.
    Cons: Child PRs won’t get AI review unless parent/mechanic explicitly adds the reviewer.

    1. Turn off Entitle to All Users and Projects.
    2. Entitle only trusted users/groups for mentions/chat.
    3. On child projects: remove rules that auto-assign AI reviewers/assignees (leadmechanic, leadreviewer, etc.).
    4. From the parent issue, @mention / assign the right AI when you want work or review.
    5. Keep leadmechanic setting reviewer to @leadreviewer after opening a PR (as in your prompt) — that becomes the single review trigger.

    Option B — Child-policy-driven

    Pros: Reviews start automatically on every child PR.
    Cons: Parent must not also kick off the same implementer/reviewer for that PR.

    1. Keep auto-assign AI reviewer on child projects.
    2. From the parent, only involve a planning AI (leadplanner) — do not also @mention/assign leadmechanic / leadreviewer for the same PR work.
    3. If leadmechanic opens PRs, remove “set reviewer to @leadreviewer” from its prompt or remove the child auto-assign rule for leadreviewer — not both.

    Suggested role scoping for your four AIs

    AITypical scope
    leadplannerMentions on the parent project only; not in child auto-assign rules
    leadmechanic / juniormechanicImplement only via parent assign/mention or child issue assign — not both for the same work
    leadreviewerEither child auto-assign or explicit add from mechanic/parent — not both

    System prompts alone won’t reliably stop duplicates; entitlement + “one trigger path” does.

    My recommendation

    Use Option A for your master/child layout: parent decides when AI starts; children do not auto-launch the same roles.

    1. Which model do you want: A parent-orchestrated, B child-policy-driven, or a hybrid (e.g. auto-review only, parent-driven implementation)?
    2. Should leadreviewer run on every child PR, or only when parent/mechanic requests it?
  • Zak Siddiqui commented 2 weeks ago

    I used a parent-Orchestrated method based on the recommendation above.

    I recommend that some documentation be created based on this beautiful answer @robin.

    I mean, I know the documentation already exists for individual features, but all the above recommendations is a great practical example.

  • Zak Siddiqui commented 2 weeks ago
  • Zak Siddiqui commented 2 weeks ago

    just a quick note to let you know that it's working absolutely perfectly now and I'm not getting duplicate work spaces.

  • Service Desk commented 2 weeks ago

    Glad the parent-orchestrated setup cleared the duplicate workspaces. Closing this question.

  • Service Desk changed state to 'Closed' 2 weeks ago
    Previous Value Current Value
    Open
    Closed
  • Zak Siddiqui commented 2 weeks ago

    @servicedesk How can I get AI users to stop watching an issue?

  • Zak Siddiqui changed state to 'Open' 2 weeks ago
    Previous Value Current Value
    Closed
    Open
  • Service Desk commented 2 weeks ago

    @zaakiy An AI user starts watching an issue when it is mentioned or assigned (same involvement rules as other users). If that AI has Proactive enabled, watching is what lets it respond to later comments without another @mention.

    Stop watching a specific issue

    Option A — Run as the AI (UI)

    1. Open the AI user (Administration → Users, or its profile).
    2. Click Run as.
    3. Open the issue.
    4. In the side panel Watchers control, set status to Ignore.

    Ignore stops watch-based / proactive follow-ups on that issue. A later @mention can still start a task, but it will not flip Ignore back to Watch.

    Prefer Ignore over clearing the watch (Watch if involved / default): removing the watch means the next mention creates a fresh Watch again.

    Option B — REST API (issue managers / admins)

    1. List watches: GET /~api/issues/{issueId}/watches
    2. Delete the AI’s entry: DELETE /~api/issue-watches/{watchId}
      Or update that watch with "watching": false.

    If you only want to stop auto-replies

    On the AI user → Behavior, turn off Proactive. Then it only acts when assigned or @mentioned, even if it still appears under Watchers.

    Docs: Issue and Pull Request Notification, Collaborate with AI User (includes Run as the AI user).

    Does this answer your question?

  • Zak Siddiqui commented 2 weeks ago

    @servicedesk Are historical prompts stored in the AI users' workspace data? I'm trying to understand if this is polluting the latest system prompt.

  • Service Desk commented 2 weeks ago

    @zaakiy No — historical prompts are not stored as the System Prompt in workspace data, and they do not pollute the latest system prompt.

    How the system prompt is applied

    On each AI task, OneDev builds a fresh $TASK_PROMPT from:

    1. Fixed rules header
    2. The AI user's current System Prompt (Behavior settings)
    3. The current task/user prompt for that run

    So when you edit the System Prompt, the next workspace run uses the new text. OneDev does not re-read an old system prompt from workspace user data.

    What workspace user data actually stores

    Workspace user data is agent home/config directories (settings, auth, tools, etc.) that persist across workspaces for the same AI user after a workspace is deleted. Built-in templates intentionally exclude chat/session history from that persistence (for example Claude sessions, Cursor chats, Codex sessions).

    Docs: Working with Workspaces (User Data section).

    Practical takeaway

    1. Editing the AI user's System Prompt is not undermined by prior $TASK_PROMPT values.
    2. If an agent still seems to follow old habits, that is usually other persisted agent state (config/memory/skills), not OneDev replaying an old system prompt. Clear that AI user's Workspace Data from its user page for a clean slate.

    Does this answer your question?

  • Zak Siddiqui commented 2 weeks ago

    Answered

  • Zak Siddiqui changed state to 'Closed' 2 weeks ago
    Previous Value Current Value
    Open
    Closed
  • Robin Shen commented 2 weeks ago

    I used a parent-Orchestrated method based on the recommendation above.

    I recommend that some documentation be created based on this beautiful answer @robin.

    I mean, I know the documentation already exists for individual features, but all the above recommendations is a great practical example.

    @zaakiy I am trying to understand your current workflow to better document or improve the feature in future. I understand that you are currently devide a parent issue in a parent project into several sub issues into child projects.

    1. Are you creating sub issues manually or have AI user created that?
    2. How do you instruct AI user to work on each sub issue? Is this done manually via comment, or via scripted default assignees?
    3. How do you instruct PR created from each sub issue to be reviewed? I understand that you currently no longer use the branch protection rules.
  • Zak Siddiqui commented 1 week ago

    @robin

    1. I instructed the parent issue to make the AI user create the issues in the child project. I did not instruct whether it should be a parent or child relationship, however it automatically established this relationship.
    2. I assigned the AI user to the child issues, and I was also making instructions from the parent issue. In hindsight, I should not have done this because this was launching workspaces in the parent and the child to do the same thing. It would be nice if the workspaces were sensitive to who is launching them and whether it's from a parent or child, or if there was a more explicit relationship between workspaces and their parent or child issue permissions.
    3. At a certain point in time, I was using the branch protection rules, but now I am doing it manually, meaning I am adding the reviewer in the child manually.

    I've only done this for one parent issue, so I don't have a comprehensive grasp of all the different possible use cases.

    However, I will try to answer follow-up questions to the best of my ability.

    I'm also wondering whether enterprise licencing improves the way that all of this works.

  • Zak Siddiqui commented 1 week ago

    At the end of the day, sometimes the AI response will (understably) makes its own decision about who to @ based on examples from the comment history, even though my system prompt had been changed.

    I'm trying to think of a better, more explicit system prompt for the AI.

  • Service Desk commented 1 week ago

    @zaakiy Yes — that behavior is expected, and it is usually comment context, not an old System Prompt sticking around.

    Why @ choices drift after you change the System Prompt

    On each run the AI still gets the full issue/PR thread. Models often copy mention patterns from that history (old @leadreviewer, @zak, @robin, etc.) even when the System Prompt has changed. Editing the System Prompt updates the next $TASK_PROMPT; it does not remove those examples from the thread.

    Make mention rules hard and explicit

    Put an allowlist / denylist near the top of the System Prompt, and state that thread examples must not override it. Example you can adapt:

    ## Mentions (mandatory)
    - Only @mention people or AI users listed below. Never invent or copy other @handles from issue/PR history.
    - Allowed: @leadreviewer (PR reviewer only), @zaakiy (questions/blockers only).
    - Forbidden: any other user or AI, including names that appear in older comments.
    - If escalation is needed and no allowed contact fits, ask in a comment without @mentioning anyone else.
    - Do not set reviewers/assignees by imitating past comments; follow only this section.
    

    For role-specific AIs, keep the allowlist tiny:

    AIMentions only
    leadmechanic@leadreviewer when opening a PR (if that is still your single review path)
    juniormechanicone fixed escalate target (e.g. @zak or @robin — pick one and keep it consistent)
    leadplannerusually none, or only the implementer AI you intentionally hand off to
    leadreviewerusually none

    Extra tips that help more than soft wording

    1. Prefer positive allowlists over “don’t mention X”.
    2. After changing who to escalate to, update every AI’s System Prompt the same way so roles don’t disagree.
    3. If an agent still follows old habits after a prompt change, clear that AI user’s Workspace Data (agent memory/config), as noted earlier — that is separate from System Prompt storage.
    4. For one-off runs, put the mention rule in the task comment as well; comment instructions often outweigh distant history for that run.

    Recommendation

    Use a short mandatory Mentions block (allowlist + “history must not override”) on each AI, and keep only one escalation handle across prompts.

    1. Which @ handles should each of your four AIs be allowed to use?
    2. For juniormechanic, should escalation go to @zak or @robin going forward?

    Does this give you enough to draft the prompts, or do you want a full rewritten System Prompt for one specific AI (e.g. leadmechanic)?

  • Zak Siddiqui commented 1 week ago

    At the end of the day, sometimes the AI response will (understably) makes its own decision about who to @ based on examples from the comment history, even though my system prompt had been changed.

    By the way, it was the above problem that actually prompted me to create this issue

1/1
Type
Question
Priority
Normal
Assignee
Labels
No labels
Issue Votes (0)
Watchers (3)
Reference
OD-2893
Please wait...
Connection lost or session expired, reload to recover
Page is in error, reload to recover