[Discussion] Integrating OneDev with a LLM agent is a nightmare #8
JA opened 2 weeks ago

Hi,

I just created my account so I could post some modest feedback. I have my own custom LLM agent, built with Pydantic, but it could be any agent.

I have now spent about three hours trying to understand the OneDev integration. Here is a summary of the issues I ran into.

  1. At first, I thought there was an MCP integration, until I learned that it had been removed.

  2. I was then instructed to install tod and the companion skills.

My first reaction: this feels like the wrong approach to me. I do not want my agent to access shell commands, and I do not really understand this current trend of going full CLI when REST APIs are available almost everywhere.

  1. Still, I hesitated and decided to have a look at the companion skills.

  2. The main skill presents itself as an umbrella skill and immediately refers to several “sub-skills”. So, to interact with OneDev, I apparently need nine skills.

My feedback: this means I now need to edit the skill to remove what I do not need. Not ideal.

  1. At that stage, I thought: my agent needs to know the list of my projects.

My feedback: apparently, this cannot be done with tod. That was a no-go for me, so I immediately gave up on using tod.

  1. I then looked at the OneDev REST API.

My feedback: it seems excellent. This is the right direction. At least I can list my projects.

  1. I wanted to copy and paste the Swagger/OpenAPI specification into Cursor so it could implement a Pydantic toolset.

But after looking everywhere, I could not find any specification. I asked ChatGPT. No success either. No machine-readable API spec. That was very frustrating.

  1. Eventually, my LLM had to read the source code to understand the API.

It seems to work now, although I am not fully sure yet.

  1. My LLM also told me that there is no endpoint to retrieve the possible status transitions for an issue. So, apparently, we need to retrieve the project settings to get the issue statuses. But those can be null if there are no custom statuses, so we then need to retrieve the global settings.

My feedback: that is a lot of requests just to manage issue statuses.

  1. A request to ~/api/settings/issue showed something that I found really surprising:
{
  "name": "In Progress",
  "description": "The AI assistant should switch the working directory to the branch corresponding to the issue by running below command:\n\ngit fetch <onedev remote> main && git checkout -b <branch-name> <onedev remote>/main && git push -u <onedev remote> <branch-name>\n\nHere <branch name> should be derived from user instruction or issue title, and <onedev remote> should be retrieved by calling tool getCurrentRemote",
  "color": "#FFA700"
}

Am I misunderstanding something, or does the status description contain agent instructions exposed through the API? I do not understand this design choice.

I like OneDev. It has a unique value proposition. But the current AI-oriented integration feels messy to me.

What I would suggest, in the spirit of KISS:

  • Integrate an MCP server directly into OneDev, with selectable tools. That would be much simpler. I do not understand the CLI-first trend when everything could be done through a cleaner decentralized architecture.
  • Provide a one-click copy/paste or download option for the API specification.
  • Include possible issue statuses or transitions directly in either: the issue endpoint and the project endpoint
  • Avoid exposing AI-specific instructions through the regular API data model.

Obviously, I assume there may be good reasons for the current design that I do not understand, so please take this feedback with a grain of salt. I took the time to write it because I like the product and wanted to contribute a user perspective.

This is external feedback from a consumer, not expert advice.

Disclaimer: this post has been reviewed by AI, mainly for french to english translation

  • Robin Shen commented 2 weeks ago

    Thanks for the feedback. The most advantage of using CLI+skills approach in OneDev is that CLI can be used with or without LLM agent. Also it is pretty difficult to debug MCP server when something is not working. So I agree with the trend of CLI+skills, when the host can run CLI. However for situations when running CLI is not possible (for instance to interact with OneDev via Slack channel), a http based MCP server is still necessary, and we plan to add that in future versions.

    As to TOD skills, some skills do use other skills in order not to repeat samething over and over. If you want to interact with OneDev server inside LLM agent, it is suggested to install all of them. Making LLM agent working with OneDev RESTful api can be difficult, as it is not standard OpenAPI/Swagger endpoint, although the docs can be accessed from http://<onedev server>/~help/api. To work with issue status, you will need to go with issue settings http://<onedev server>/~help/api/io.onedev.server.rest.resource.SettingResource/getIssueSetting. Also the description of issue state is no longer used, and you may just delete it safely.

  • Robin Shen commented 2 weeks ago

    Skills now updated to be self-contained. Each skill can be installed/used independently.

  • Robin Shen commented 1 week ago

    Feel free to reopen if you have more questions

  • Robin Shen changed state to 'Closed' 1 week ago
    Previous Value Current Value
    Open
    Closed
1/1
Type
Improvement
Priority
Normal
Assignee
Not assigned
Labels
No labels
Issue Votes (0)
Watchers (3)
Reference
onedev/tod#8
Please wait...
Connection lost or session expired, reload to recover
Page is in error, reload to recover