|
wojtek opened 7 months ago
|
||||
|
When a project is created, OneDev will create file To find out which project is in problem, please edit conf/logback.xml to enable debug logging. |
||||
|
|
Looking at the ST and sources it looks like the clustering configuration type ( So the question is - is this information is stored in the filesystem or somesuch? |
||||
|
|
This is stored in the file system. Please just delete this file as the empty file is not valid. |
||||
|
|
For which packet I should enable the logging to get the information?
It occured to me - shouldn't 1dev handle this case and remove the file on it's own? |
||||
|
|
Yes enable this package is fine. Adding project information to every error message is quite verbose considering this happens rarely.
Handing every edge cases is very verbose, especially for memory and file system errors. Giving error message and provides ways to recover is enough I think. |
||||
|
|
Robin Shen changed state to 'Closed' 7 months ago
|
||||
|
|
Actually there is a slight problem - given that process crasshes immediatelly then the pod is terminated thus making it impossible to access it - to change configuration to figure out the filename/path or to actually delete it.
It's not very edge case, and it may prevent pod startup (as seen above). And I wouldn't consider giving relevant information as "verbose" in errors. |
||||
|
|
wojtek changed state to 'Open' 7 months ago
|
||||
|
|
OK, I got the logs: So it seems that the offending project is I tried browsing the filesystem but wihout luck - no file with such name: nor actuall file named as such: Widening the search with wildcards still doesn't yield anything: EDIT:
Would it be possible to add more logging to: EDIT2: My bad, upon looking again at the issue and comments the file is named: |
||||
|
|
But it doesn't work still, another exception(s): Looking at the exception the issue comes from the fact that the file in which version information is store is also corrupted? EDIT: I search for all the files named It seems that EDIT: yes, adding |
||||
|
|
I will add code to check if the type and version file is empty to tolerate such errors. However it is nearly impossible to guard everywhere for such error in case of node crash causing file content not being flushed to disk. To solve such error, periodical backup or HA should be considered. |
||||
|
|
Further investigation: Tolerating such errors can be harmful, for instance, reading the replica type file and assume it to be PRIMARY in case of reading errors can cause it sends outdated to other replicas. The correct approach would be to report errors as soon as possible to get human intervention. I will improve the error reporting to include file path in this case. |
||||
|
|
Also with 9.2.1 you may override command and args via helm values.yaml so that you can run |
||||
|
|
Thank you for the comments. You have a point regarding handling it automatically but I would argue that it's not edge case - errors and data corruption can happen. Having better error logs indicating what the problem was and pointing in the right direction (file path!) would go a long way! |
||||
|
|
Robin Shen changed state to 'Closed' 6 months ago
|
| Type |
Question
|
| Priority |
Normal
|
| Assignee | |
| Labels |
No labels
|
Issue Votes (0)
Watchers (3)
While working on importing from YT (https://code.onedev.io/onedev/server/~issues/1596) I tried to maximise memory assigned to 1dev pod. k8s allowed it but running the import resulted in k8s node crash - later on k8s recreated node and pods but 1dev can't start:
(version 9.2.0)