Enforce AI reviewer to either approve a PR or request for changes #2720
Glenn D. opened 3 months ago

I am testing an AI user to help me review my pull requests. It reviews the PR just fine, but when all is OK, does not approve them. When I specifically ask it to approve, he says it's approved, but the PR status does not change.

Is approving the PR a tool call that might not be happening? How would an AI user do an approve?

image.png

  • Robin Shen commented 3 months ago

    If you ask the AI user to review code in comment, it will never approve or request for changes for the PR. You will need to add the AI user as a reviewer instead.

  • Glenn D. commented 3 months ago

    I did. The first AI response (scout) is after I created the PR with the AI agent as reviewer

  • Glenn D. commented 3 months ago

    Maybe this screenshot provides more info:

    image_2.png

  • Robin Shen commented 3 months ago

    The system prompt already tells the AI model to call the PR approve tool upon accepted review. It works well for models with good tool call capabilities, such as Claude Haiku etc.

  • Robin Shen commented 3 months ago

    Maybe another place to force the LLM to call a taskComplete with PR action before it completes its review.

  • Robin Shen changed fields 3 months ago
    Name Previous Value Current Value
    Type
    Question
    Improvement
  • Robin Shen changed title 3 months ago
    Previous Value Current Value
    AI User not approving pull requests - how does it work?
    AI reviewer does not approve the PR
  • Robin Shen changed title 3 months ago
    Previous Value Current Value
    AI reviewer does not approve the PR
    Enforce AI reviewer to either approve a PR or request for changes
  • Glenn D. commented 3 months ago

    I was wondering if there is any place in the folders, or if a configuration page could be added, that allows users to see/tweak the system prompts used by OneDev?

    Local models used by users are most likely not as powerful as the cloud models (I can run max 24B while cloud models go up to 256B+), and in my limited experience, some need some "extra coaching" to get something done.

    Another possibility would be to have an overview of available calls, so that I can tweak the AI User's system prompt.

  • Glenn D. commented 3 months ago

    Maybe another place to force the LLM to call a taskComplete

    In my previous post here I had an issue with chat text disappearing. My workaround was to forbid calling "taskComplete", could this be what is now causing the PR to not go through?

  • Robin Shen commented 3 months ago

    You may AI settings of the AI user, and there is a system prompt setting. "taskComplete" tool is not currently not being used for code review tasks.

  • OneDev commented 2 months ago

    State changed as build OD-7256 (15.0.0) is successful

  • OneDev changed state to 'Closed' 2 months ago
    Previous Value Current Value
    Open
    Closed
1/1
Type
Improvement
Priority
Normal
Assignee
Labels
No labels
Issue Votes (0)
Watchers (3)
Reference
OD-2720
Please wait...
Connection lost or session expired, reload to recover
Page is in error, reload to recover