failed to create fsnotify watcher: too many open files (OD-2007)
Daniel Kollmannsberger opened 1 year ago

Hey Robin,

had no issues for a long time now 😀

While using cicd (kubernetes pool for backend execution) I encountered following issue:

failed to create fsnotify watcher: too many open files (in the webpanel)

logs for the execution pod on the kubernetes cluster are normal.

Do you know anything about it? Or do I have to find out more?

Latest Version. Happened after upgrade to latest version. But I have been on 10.5.0 before.

  • Daniel Kollmannsberger commented 1 year ago

    And I have seen, that after the maven push is done, the cicd job keeps running.

  • Robin Shen commented 1 year ago

    Is this reported from k8s side? If so, this does not seem like a OneDev issue as it just launches jobs as standard pods.

  • Daniel Kollmannsberger commented 1 year ago

    Nope, it seems to be on OneDev site. Because the pod logs normal. How does onedev access the logs? Maybe I can debug it and find the solution

  • Robin Shen commented 1 year ago

    Please show me the log containing the error.

  • Robin Shen changed state to 'Closed' 1 year ago
    Previous Value Current Value
    Open
    Closed
  • Robin Shen commented 1 year ago

    Feel free to reopen if there is more info

  • Daniel Kollmannsberger commented 1 year ago
    
    
    21:38:42 Checking cluster access...
    21:38:43 Preparing job (executor: onedev-knet-jobs, namespace: onedev-knet-jobs-65-9-0)...
    21:38:45 Running job on node knet-wn-01...
    21:38:45 Starting job containers...
    21:38:47 failed to create fsnotify watcher: too many open files
    21:39:35 failed to create fsnotify watcher: too many open files
    21:39:43 2024-08-21T21:39:39.495799785+02:00 Running step "extract build version"...
    21:39:43 2024-08-21T21:39:39.498094011+02:00 Detecting project version (may require some time while downloading maven dependencies)...
    21:39:43 2024-08-21T21:39:39.498774218+02:00 /onedev-build/command/step-1.sh: line 2: mvn: not found
    21:39:43 2024-08-21T21:39:39.499868339+02:00 Step "extract build version" is successful
    21:39:43 failed to create fsnotify watcher: too many open files
    21:39:44 2024-08-21T21:39:39.615466957+02:00 Running step "set build version"...
    21:39:44 failed to create fsnotify watcher: too many open files
    21:39:45 2024-08-21T21:39:45.108524782+02:00 Running step "build lib with maven"...
    21:39:45 failed to create fsnotify watcher: too many open files
    

    example log from cicd job in onedev ui

  • Daniel Kollmannsberger commented 1 year ago

    That was the problem - thanks

issue 1/1
Type
Bug
Priority
Normal
Assignee
Affected Versions
11.0.0
Labels
No labels
Issue Votes (0)
Watchers (2)
Reference
OD-2007
Please wait...
Connection lost or session expired, reload to recover
Page is in error, reload to recover