Hanging git submodule checkout (OD-1975)
Closed
wojtek opened 3 weeks ago

I'm facing issue similar to #OD-648 - we have a job that has git-submodule configured (git submodule init and then git submodule update --recursive --remote) and 1dev simply gets stuck at checking it out.

I launch shell at the current step and obviously any checkout of remote fails because it asks for credentials (the sources are on the same 1dev instance).

Linked issue mentions:

This tutorial has details on how to use custom clone credential:

https://code.onedev.io/projects/162/blob/main/pages/push-in-job.md

But that page doesn't exist…

I enabled "Retrieve Submodules" and 1dev gets stuck just the same… (running already for a couple of minutes)

18:32:32 Running step "checkout current"...
18:32:32 Checking out code...
18:32:33 Retrieving submodules...
Robin Shen commented 3 weeks ago

Please demonstrate the issue by creating sample projects at code.onedev.io.

Robin Shen commented 2 weeks ago

Please upgrade to build OD-5236 (10.9.4) to see if it fixes the issue.

wojtek commented 2 weeks ago

Would that apply even if we have 10.9.1 (and not .3 mentioned in the issue)?

Robin Shen commented 2 weeks ago

The issue fixed in 10.9.4 is introduced in 10.9.3. So this should not be the same as yours.

Robin Shen changed state to 'Closed' 2 weeks ago
Previous Value Current Value
Open
Closed
Robin Shen commented 2 weeks ago

Closing. Feel free to reopen if there is more info

wojtek commented 5 days ago

I upgraded to 10.9.4 and the issue exists (same build hangs). I was trying to reproduce the issue on our environment but was getting

20:37:35 fatal: could not read Username for 'https://tigase.dev': No such device or address
20:37:35 fatal: clone of 'https://tigase.dev/tigase/_server/tigase-xmltools' into submodule path '/onedev-build/workspace/xmltools' failed
20:37:35 Failed to clone 'xmltools'. Retry scheduled
20:37:35 fatal: could not read Username for 'https://tigase.dev': No such device or address
20:37:35 fatal: clone of 'https://tigase.dev/tigase/_server/tigase-xmltools' into submodule path '/onedev-build/workspace/xmltools' failed
20:37:35 Failed to clone 'xmltools' a second time, aborting

Same issue happens here: https://code.onedev.io/checkout-submodule/~builds/2

I tried to peruse example from https://code.onedev.io/code-checkout-OD-1980/~files/main/.onedev-buildspec.yml but it doesn't exists anymore…

I found https://docs.onedev.io/tutorials/cicd/clone-submodules-ssh - does it mean that it's required to use SSH for submodules to build them in 1dev?

EDIT: about the last one - it feels like security issue - posting MY private key to project configuration just so it could pull submodules (even if they are public) looks like asking for trouble with leaking keys...

wojtek commented 5 days ago

I set it with ssh but now I get:

21:16:19 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
21:16:19 @    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
21:16:19 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
21:16:19 IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
21:16:19 Someone could be eavesdropping on you right now (man-in-the-middle attack)!
21:16:19 It is also possible that a host key has just been changed.
21:16:19 The fingerprint for the RSA key sent by the remote host is
21:16:19 SHA256:vQj5FL/NSjy7z1t4/sHpt+NVC2QZMG0PugBYmLDTITc.
21:16:19 Please contact your system administrator.
21:16:19 Add correct host key in /agent/work/temp/onedev-build-1287-30/user/.ssh/known_hosts to get rid of this message.
21:16:19 Offending RSA key in /agent/work/temp/onedev-build-1287-30/user/.ssh/known_hosts:1
21:16:19   remove with:
21:16:19   ssh-keygen -f "/agent/work/temp/onedev-build-1287-30/user/.ssh/known_hosts" -R "tigase.dev"
21:16:19 Host key for tigase.dev has changed and you have requested strict checking.
21:16:19 Host key verification failed.
21:16:19 fatal: Could not read from remote repository.
21:16:19 
21:16:19 Please make sure you have the correct access rights
21:16:19 and the repository exists.

Yes, I do have same host for Server URL * and SSH Root URL (when it was empty the error was the same…)

wojtek commented 5 days ago

After using SSH here it keeps complainign about https://code.onedev.io/checkout-submodule/~builds/3

21:24:57 fatal: could not read Username for 'https://code.onedev.io': No such device or address
21:24:57 fatal: clone of 'https://code.onedev.io/onedev/server-ee' into submodule path '/home/onedev/onedev/temp/server/onedev-build-1150-3/workspace/server/server-ee' failed
21:24:57 Failed to clone 'server-ee'. Retry scheduled
21:24:57 fatal: could not read Username for 'https://code.onedev.io': No such device or address
21:24:57 fatal: clone of 'https://code.onedev.io/onedev/server-ee' into submodule path '/home/onedev/onedev/temp/server/onedev-build-1150-3/workspace/server/server-ee' failed
21:24:57 Failed to clone 'server-ee' a second time, aborting
21:24:57 fatal: Failed to recurse into submodule path 'server'

Funny thing is submodule points to plain server repo:

https://code.onedev.io/checkout-submodule/~files/master/.gitmodules

[submodule "server"]
	path = server
	url = https://code.onedev.io/onedev/server
wojtek changed state to 'Open' 5 days ago
Previous Value Current Value
Closed
Open
Robin Shen commented 5 days ago

Since your submodule needs to be cloned via http protocol, you will need to create a access token with permission to clone that submodule, and configure the checkout step to use that as http clone credential. I've modified your checkout step to be so.

Also I changed submodule from server to commons as server itself contains ee submodule which will fail to chone with your own access token.

Robin Shen commented 5 days ago

Forget to menion that if your submodule is defined as ssh protocol, you will need to set up the checkout step to use ssh clone, and provide private key. This is also the same for GitHub/GitLab.

wojtek commented 4 days ago

Thanks! It works with token. I think that the guide could be improved.

One suggestion thoug - if "checkout submodule" is selected then maybe force selecting checout type (https/ssh) and indicate that a peoperly configured access token is required?

(EDIT: I removed the sample project)

Robin Shen changed state to 'Closed' 4 days ago
Previous Value Current Value
Open
Closed
Robin Shen commented 4 days ago

Doc will be improved in future versions.

issue 1 of 1
Type
Question
Priority
Normal
Assignee
Labels
No labels
Issue Votes (0)
Watchers (2)
Reference
OD-1975
Please wait...
Page is in error, reload to recover